2023-09-13T04:10:33.759 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-13T04:10:33.760 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/blogbench}} --verbose --owner scheduled_rishabh@teuthology --name rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi/7394725 -- /home/teuthworker/archive/rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi/7394725/orig.config.yaml 2023-09-13T04:10:33.853 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T04:10:33.950 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi/7394725 branch: wip-rishabh-2023sep12-b2 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/blogbench}} email: null first_in_suite: false job_id: '7394725' kernel: client: branch: testing flavor: default kdb: true sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-2023sep12-b2 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 kclient: mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023sep12-b2 sha1: 4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2531 sha1: 4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 sleep_before_teardown: 0 subset: 165/256 suite: fs suite_branch: wip-rishabh-2023sep12-b2 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi002.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi149.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi156.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-09-12_12:12:15 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1394 2023-09-13T04:10:33.950 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/qa; will attempt to use it 2023-09-13T04:10:33.951 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/qa/tasks 2023-09-13T04:10:33.951 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-09-13T04:10:33.974 INFO:teuthology.task.internal:Checking packages... 2023-09-13T04:10:34.016 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5' 2023-09-13T04:10:34.016 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-13T04:10:34.017 INFO:teuthology.packaging:ref: None 2023-09-13T04:10:34.017 INFO:teuthology.packaging:tag: None 2023-09-13T04:10:34.018 INFO:teuthology.packaging:branch: wip-rishabh-2023sep12-b2 2023-09-13T04:10:34.018 INFO:teuthology.packaging:sha1: 4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 2023-09-13T04:10:34.019 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023sep12-b2 2023-09-13T04:10:34.239 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6083.g4a754ac9 2023-09-13T04:10:34.240 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-09-13T04:10:34.247 INFO:teuthology.task.internal:no buildpackages task found 2023-09-13T04:10:34.248 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-09-13T04:10:34.268 INFO:teuthology.task.internal:Saving configuration 2023-09-13T04:10:34.298 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-09-13T04:10:34.305 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-13T04:10:34.333 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi002.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi/7394725', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-13 04:05:19.552561', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-13T04:10:34.355 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi/7394725', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-13 04:05:19.550477', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-13T04:10:34.380 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi/7394725', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-13 04:05:19.554928', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-13T04:10:34.380 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-09-13T04:10:34.388 INFO:teuthology.task.internal:roles: ubuntu@smithi002.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-09-13T04:10:34.388 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-09-13T04:10:34.389 INFO:teuthology.task.internal:roles: ubuntu@smithi156.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-09-13T04:10:34.389 INFO:teuthology.run_tasks:Running task console_log... 2023-09-13T04:10:34.538 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f33aca87c10>, signals=[15]) 2023-09-13T04:10:34.540 INFO:teuthology.run_tasks:Running task internal.connect... 2023-09-13T04:10:34.547 INFO:teuthology.task.internal:Opening connections... 2023-09-13T04:10:34.547 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi002.front.sepia.ceph.com 2023-09-13T04:10:34.549 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T04:10:34.618 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi149.front.sepia.ceph.com 2023-09-13T04:10:34.620 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T04:10:34.686 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi156.front.sepia.ceph.com 2023-09-13T04:10:34.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T04:10:34.754 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-09-13T04:10:34.762 DEBUG:teuthology.orchestra.run.smithi002:> uname -m 2023-09-13T04:10:34.785 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2023-09-13T04:10:34.785 DEBUG:teuthology.orchestra.run.smithi002:> cat /etc/os-release 2023-09-13T04:10:34.845 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="Red Hat Enterprise Linux" 2023-09-13T04:10:34.846 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="8.6 (Ootpa)" 2023-09-13T04:10:34.846 INFO:teuthology.orchestra.run.smithi002.stdout:ID="rhel" 2023-09-13T04:10:34.846 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="fedora" 2023-09-13T04:10:34.847 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="8.6" 2023-09-13T04:10:34.847 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el8" 2023-09-13T04:10:34.847 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-13T04:10:34.848 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2023-09-13T04:10:34.848 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-13T04:10:34.848 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://www.redhat.com/" 2023-09-13T04:10:34.848 INFO:teuthology.orchestra.run.smithi002.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-13T04:10:34.849 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-13T04:10:34.849 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:10:34.849 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-13T04:10:34.850 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-13T04:10:34.850 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-13T04:10:34.850 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-13T04:10:34.851 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2023-09-13T04:10:34.877 DEBUG:teuthology.orchestra.run.smithi149:> uname -m 2023-09-13T04:10:34.899 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2023-09-13T04:10:34.900 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release 2023-09-13T04:10:34.959 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="Red Hat Enterprise Linux" 2023-09-13T04:10:34.959 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="8.6 (Ootpa)" 2023-09-13T04:10:34.960 INFO:teuthology.orchestra.run.smithi149.stdout:ID="rhel" 2023-09-13T04:10:34.960 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="fedora" 2023-09-13T04:10:34.960 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="8.6" 2023-09-13T04:10:34.960 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el8" 2023-09-13T04:10:34.960 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-13T04:10:34.961 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31" 2023-09-13T04:10:34.961 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-13T04:10:34.961 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://www.redhat.com/" 2023-09-13T04:10:34.961 INFO:teuthology.orchestra.run.smithi149.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-13T04:10:34.962 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-13T04:10:34.962 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-09-13T04:10:34.962 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-13T04:10:34.962 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-13T04:10:34.962 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-13T04:10:34.963 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-13T04:10:34.963 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2023-09-13T04:10:34.985 DEBUG:teuthology.orchestra.run.smithi156:> uname -m 2023-09-13T04:10:35.006 INFO:teuthology.orchestra.run.smithi156.stdout:x86_64 2023-09-13T04:10:35.006 DEBUG:teuthology.orchestra.run.smithi156:> cat /etc/os-release 2023-09-13T04:10:35.065 INFO:teuthology.orchestra.run.smithi156.stdout:NAME="Red Hat Enterprise Linux" 2023-09-13T04:10:35.066 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION="8.6 (Ootpa)" 2023-09-13T04:10:35.066 INFO:teuthology.orchestra.run.smithi156.stdout:ID="rhel" 2023-09-13T04:10:35.066 INFO:teuthology.orchestra.run.smithi156.stdout:ID_LIKE="fedora" 2023-09-13T04:10:35.066 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_ID="8.6" 2023-09-13T04:10:35.067 INFO:teuthology.orchestra.run.smithi156.stdout:PLATFORM_ID="platform:el8" 2023-09-13T04:10:35.067 INFO:teuthology.orchestra.run.smithi156.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-13T04:10:35.067 INFO:teuthology.orchestra.run.smithi156.stdout:ANSI_COLOR="0;31" 2023-09-13T04:10:35.067 INFO:teuthology.orchestra.run.smithi156.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-13T04:10:35.067 INFO:teuthology.orchestra.run.smithi156.stdout:HOME_URL="https://www.redhat.com/" 2023-09-13T04:10:35.068 INFO:teuthology.orchestra.run.smithi156.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-13T04:10:35.068 INFO:teuthology.orchestra.run.smithi156.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-13T04:10:35.068 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-13T04:10:35.068 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-13T04:10:35.068 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-13T04:10:35.069 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-13T04:10:35.069 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-13T04:10:35.069 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2023-09-13T04:10:35.093 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-09-13T04:10:35.104 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-09-13T04:10:35.109 INFO:teuthology.task.internal:Checking for old test directory... 2023-09-13T04:10:35.110 DEBUG:teuthology.orchestra.run.smithi002:> test '!' -e /home/ubuntu/cephtest 2023-09-13T04:10:35.113 DEBUG:teuthology.orchestra.run.smithi149:> test '!' -e /home/ubuntu/cephtest 2023-09-13T04:10:35.115 DEBUG:teuthology.orchestra.run.smithi156:> test '!' -e /home/ubuntu/cephtest 2023-09-13T04:10:35.133 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-09-13T04:10:35.139 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-09-13T04:10:35.139 DEBUG:teuthology.orchestra.run.smithi002:> test -z $(ls -A /var/lib/ceph) 2023-09-13T04:10:35.172 DEBUG:teuthology.orchestra.run.smithi149:> test -z $(ls -A /var/lib/ceph) 2023-09-13T04:10:35.175 DEBUG:teuthology.orchestra.run.smithi156:> test -z $(ls -A /var/lib/ceph) 2023-09-13T04:10:35.206 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-09-13T04:10:35.285 INFO:teuthology.run_tasks:Running task kernel... 2023-09-13T04:10:35.300 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'} 2023-09-13T04:10:35.300 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-09-13T04:10:35.301 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-09-13T04:10:35.301 DEBUG:teuthology.orchestra.run.smithi002:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-13T04:10:35.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T04:10:35.321 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-09-13T04:10:35.651 DEBUG:teuthology.task.kernel:sha1 for client.0 is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-09-13T04:10:35.651 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-09-13T04:10:35.652 DEBUG:teuthology.orchestra.run.smithi002:> uname -r 2023-09-13T04:10:35.672 INFO:teuthology.orchestra.run.smithi002.stdout:4.18.0-372.9.1.el8.x86_64 2023-09-13T04:10:35.672 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-09-13T04:10:35.673 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-09-13T04:10:35.674 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum upgrade -y linux-firmware 2023-09-13T04:10:36.950 INFO:teuthology.orchestra.run.smithi002.stdout:Updating Subscription Management repositories. 2023-09-13T04:10:36.950 INFO:teuthology.orchestra.run.smithi002.stdout:Unable to read consumer identity 2023-09-13T04:10:38.446 INFO:teuthology.orchestra.run.smithi002.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2023-09-13T04:10:40.163 INFO:teuthology.orchestra.run.smithi002.stdout:Extra Packages for Enterprise Linux 9.3 MB/s | 16 MB 00:01 2023-09-13T04:10:43.930 INFO:teuthology.orchestra.run.smithi002.stdout:lab-extras 443 kB/s | 24 kB 00:00 2023-09-13T04:10:44.656 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2023-09-13T04:10:44.657 INFO:teuthology.orchestra.run.smithi002.stdout:Nothing to do. 2023-09-13T04:10:44.657 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2023-09-13T04:10:44.804 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-09-13T04:10:44.805 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-09-13T04:10:44.944 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ 2023-09-13T04:10:44.944 DEBUG:teuthology.orchestra.run.smithi002:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ --input-file=- 2023-09-13T04:10:44.947 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi002.front.sepia.ceph.com to complete... 2023-09-13T04:10:58.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13 04:10:58 URL:https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [380759012/380759012] -> "/tmp/kernel.x86_64.rpm" [1] 2023-09-13T04:10:58.779 INFO:teuthology.orchestra.run.smithi002.stderr:FINISHED --2023-09-13 04:10:58-- 2023-09-13T04:10:58.779 INFO:teuthology.orchestra.run.smithi002.stderr:Total wall clock time: 14s 2023-09-13T04:10:58.779 INFO:teuthology.orchestra.run.smithi002.stderr:Downloaded: 1 files, 363M in 13s (26.9 MB/s) 2023-09-13T04:10:58.780 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-09-13T04:10:58.781 DEBUG:teuthology.orchestra.run.smithi002:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-09-13T04:11:00.437 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying... ######################################## 2023-09-13T04:11:00.491 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing... ######################################## 2023-09-13T04:11:00.582 INFO:teuthology.orchestra.run.smithi002.stdout:Updating / installing... 2023-09-13T04:11:41.255 INFO:teuthology.orchestra.run.smithi002.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-09-13T04:12:05.275 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi002.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-09-13T04:12:05.276 DEBUG:teuthology.orchestra.run.smithi002:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-09-13T04:12:05.343 INFO:teuthology.orchestra.run.smithi002.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-09-13T04:12:05.344 INFO:teuthology.orchestra.run.smithi002.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-09-13T04:12:05.344 INFO:teuthology.orchestra.run.smithi002.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-09-13T04:12:05.344 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-09-13T04:12:05.344 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-09-13T04:12:05.345 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-09-13T04:12:05.345 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-09-13T04:12:05.345 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-09-13T04:12:05.345 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-09-13T04:12:05.345 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-09-13T04:12:05.346 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-09-13T04:12:05.346 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-09-13T04:12:05.346 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-09-13T04:12:05.346 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-09-13T04:12:05.347 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-09-13T04:12:05.347 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-09-13T04:12:05.347 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-09-13T04:12:05.347 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-09-13T04:12:05.347 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-09-13T04:12:05.348 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-09-13T04:12:05.348 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-09-13T04:12:05.348 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-09-13T04:12:05.348 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-09-13T04:12:05.349 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-09-13T04:12:05.349 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-09-13T04:12:05.349 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-09-13T04:12:05.349 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-09-13T04:12:05.349 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-09-13T04:12:05.350 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-09-13T04:12:05.350 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-09-13T04:12:05.350 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-09-13T04:12:05.350 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-09-13T04:12:05.351 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-09-13T04:12:05.351 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-09-13T04:12:05.351 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-09-13T04:12:05.351 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-09-13T04:12:05.351 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-09-13T04:12:05.352 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-09-13T04:12:05.352 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-09-13T04:12:05.352 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-09-13T04:12:05.352 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-09-13T04:12:05.352 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-09-13T04:12:05.353 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-09-13T04:12:05.353 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-09-13T04:12:05.353 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-09-13T04:12:05.353 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-09-13T04:12:05.353 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-09-13T04:12:05.353 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-09-13T04:12:05.354 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-09-13T04:12:05.354 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-09-13T04:12:05.354 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-09-13T04:12:05.354 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-09-13T04:12:05.355 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-09-13T04:12:05.355 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-09-13T04:12:05.355 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-09-13T04:12:05.355 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-09-13T04:12:05.355 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-09-13T04:12:05.356 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-09-13T04:12:05.356 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-09-13T04:12:05.356 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-09-13T04:12:05.356 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-09-13T04:12:05.356 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-09-13T04:12:05.357 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-09-13T04:12:05.357 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-09-13T04:12:05.357 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-09-13T04:12:05.357 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-09-13T04:12:05.357 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-09-13T04:12:05.358 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-09-13T04:12:05.358 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-09-13T04:12:05.358 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-09-13T04:12:05.358 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-09-13T04:12:05.358 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-09-13T04:12:05.359 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-09-13T04:12:05.359 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-09-13T04:12:05.359 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-09-13T04:12:05.359 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-09-13T04:12:05.359 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-09-13T04:12:05.360 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-09-13T04:12:05.360 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-09-13T04:12:05.360 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-09-13T04:12:05.360 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-09-13T04:12:05.360 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-09-13T04:12:05.361 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-09-13T04:12:05.361 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-09-13T04:12:05.361 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-09-13T04:12:05.361 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-09-13T04:12:05.361 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-09-13T04:12:05.362 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-09-13T04:12:05.362 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-09-13T04:12:05.362 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-09-13T04:12:05.362 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-09-13T04:12:05.363 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-09-13T04:12:05.363 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-09-13T04:12:05.363 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-09-13T04:12:05.363 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-09-13T04:12:05.363 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-09-13T04:12:05.364 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-09-13T04:12:05.364 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-09-13T04:12:05.364 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-09-13T04:12:05.364 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-09-13T04:12:05.364 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-09-13T04:12:05.365 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-09-13T04:12:05.365 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-09-13T04:12:05.365 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-09-13T04:12:05.365 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-09-13T04:12:05.365 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-09-13T04:12:05.366 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-09-13T04:12:05.366 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-09-13T04:12:05.366 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-09-13T04:12:05.366 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-09-13T04:12:05.366 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-09-13T04:12:05.367 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-09-13T04:12:05.367 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-09-13T04:12:05.367 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-09-13T04:12:05.367 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-09-13T04:12:05.368 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-09-13T04:12:05.368 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-09-13T04:12:05.368 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-09-13T04:12:05.368 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-09-13T04:12:05.368 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-09-13T04:12:05.368 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-09-13T04:12:05.369 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-09-13T04:12:05.369 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-09-13T04:12:05.369 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-09-13T04:12:05.369 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-09-13T04:12:05.369 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-09-13T04:12:05.370 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-09-13T04:12:05.370 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-09-13T04:12:05.370 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-09-13T04:12:05.370 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-09-13T04:12:05.371 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-09-13T04:12:05.371 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-09-13T04:12:05.371 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-09-13T04:12:05.371 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-09-13T04:12:05.371 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-09-13T04:12:05.372 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-09-13T04:12:05.372 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-09-13T04:12:05.372 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-09-13T04:12:05.372 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-09-13T04:12:05.372 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-09-13T04:12:05.373 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-09-13T04:12:05.373 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-09-13T04:12:05.373 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-09-13T04:12:05.373 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-09-13T04:12:05.373 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-09-13T04:12:05.374 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-09-13T04:12:05.374 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-09-13T04:12:05.374 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-09-13T04:12:05.374 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-09-13T04:12:05.374 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-09-13T04:12:05.375 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-09-13T04:12:05.375 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-09-13T04:12:05.375 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-09-13T04:12:05.375 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-09-13T04:12:05.376 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-09-13T04:12:05.376 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-09-13T04:12:05.376 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-09-13T04:12:05.377 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-09-13T04:12:05.377 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-09-13T04:12:05.377 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-09-13T04:12:05.378 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-09-13T04:12:05.378 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-09-13T04:12:05.378 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-09-13T04:12:05.379 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-09-13T04:12:05.379 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-09-13T04:12:05.379 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-09-13T04:12:05.379 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-09-13T04:12:05.380 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-09-13T04:12:05.380 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-09-13T04:12:05.380 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-09-13T04:12:05.380 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-09-13T04:12:05.380 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-09-13T04:12:05.381 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-09-13T04:12:05.381 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-09-13T04:12:05.381 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-09-13T04:12:05.381 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-09-13T04:12:05.381 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-09-13T04:12:05.382 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-09-13T04:12:05.382 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-09-13T04:12:05.382 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-09-13T04:12:05.382 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-09-13T04:12:05.382 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-09-13T04:12:05.383 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-09-13T04:12:05.383 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-09-13T04:12:05.383 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-09-13T04:12:05.383 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-09-13T04:12:05.383 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-09-13T04:12:05.384 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-09-13T04:12:05.384 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-09-13T04:12:05.384 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-09-13T04:12:05.384 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-09-13T04:12:05.384 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-09-13T04:12:05.385 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-09-13T04:12:05.385 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-09-13T04:12:05.385 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-09-13T04:12:05.385 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-09-13T04:12:05.385 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-09-13T04:12:05.386 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-09-13T04:12:05.386 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-09-13T04:12:05.386 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-09-13T04:12:05.386 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-09-13T04:12:05.387 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-09-13T04:12:05.387 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-09-13T04:12:05.387 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-09-13T04:12:05.387 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-09-13T04:12:05.387 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-09-13T04:12:05.388 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-09-13T04:12:05.388 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-09-13T04:12:05.388 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-09-13T04:12:05.388 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-09-13T04:12:05.388 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-09-13T04:12:05.389 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-09-13T04:12:05.389 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-09-13T04:12:05.389 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-09-13T04:12:05.389 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-09-13T04:12:05.389 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-09-13T04:12:05.389 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-09-13T04:12:05.390 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-09-13T04:12:05.390 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-09-13T04:12:05.390 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-09-13T04:12:05.390 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-09-13T04:12:05.390 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-09-13T04:12:05.391 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-09-13T04:12:05.391 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-09-13T04:12:05.391 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-09-13T04:12:05.391 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-09-13T04:12:05.391 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-09-13T04:12:05.392 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-09-13T04:12:05.392 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-09-13T04:12:05.392 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-09-13T04:12:05.392 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-09-13T04:12:05.392 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-09-13T04:12:05.393 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-09-13T04:12:05.393 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-09-13T04:12:05.393 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-09-13T04:12:05.393 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-09-13T04:12:05.393 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-09-13T04:12:05.394 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-09-13T04:12:05.394 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-09-13T04:12:05.394 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-09-13T04:12:05.394 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-09-13T04:12:05.394 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-09-13T04:12:05.395 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-09-13T04:12:05.395 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-09-13T04:12:05.395 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-09-13T04:12:05.395 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-09-13T04:12:05.395 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-09-13T04:12:05.396 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-09-13T04:12:05.396 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-09-13T04:12:05.396 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-09-13T04:12:05.396 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-09-13T04:12:05.396 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-09-13T04:12:05.397 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-09-13T04:12:05.397 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-09-13T04:12:05.397 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-09-13T04:12:05.397 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-09-13T04:12:05.397 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-09-13T04:12:05.398 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-09-13T04:12:05.398 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-09-13T04:12:05.398 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-09-13T04:12:05.398 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-09-13T04:12:05.398 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-09-13T04:12:05.399 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-09-13T04:12:05.399 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-09-13T04:12:05.399 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-09-13T04:12:05.399 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-09-13T04:12:05.399 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-09-13T04:12:05.399 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-09-13T04:12:05.400 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-09-13T04:12:05.400 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-09-13T04:12:05.400 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-09-13T04:12:05.400 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-09-13T04:12:05.400 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-09-13T04:12:05.401 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-09-13T04:12:05.401 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-09-13T04:12:05.401 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-09-13T04:12:05.401 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-09-13T04:12:05.401 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-09-13T04:12:05.402 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-09-13T04:12:05.402 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-09-13T04:12:05.402 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-09-13T04:12:05.402 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-09-13T04:12:05.402 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-09-13T04:12:05.403 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-09-13T04:12:05.403 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-09-13T04:12:05.403 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-09-13T04:12:05.403 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-09-13T04:12:05.403 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-09-13T04:12:05.404 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-09-13T04:12:05.404 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-09-13T04:12:05.404 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-09-13T04:12:05.404 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-09-13T04:12:05.404 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-09-13T04:12:05.405 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-09-13T04:12:05.405 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-09-13T04:12:05.405 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-09-13T04:12:05.405 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-09-13T04:12:05.405 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-09-13T04:12:05.406 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-09-13T04:12:05.406 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-09-13T04:12:05.406 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-09-13T04:12:05.406 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-09-13T04:12:05.406 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-09-13T04:12:05.407 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-09-13T04:12:05.407 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-09-13T04:12:05.407 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-09-13T04:12:05.407 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-09-13T04:12:05.407 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-09-13T04:12:05.408 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-09-13T04:12:05.408 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-09-13T04:12:05.408 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-09-13T04:12:05.408 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-09-13T04:12:05.408 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-09-13T04:12:05.409 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-09-13T04:12:05.409 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-09-13T04:12:05.409 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-09-13T04:12:05.409 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-09-13T04:12:05.409 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-09-13T04:12:05.410 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-09-13T04:12:05.410 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-09-13T04:12:05.410 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-09-13T04:12:05.410 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-09-13T04:12:05.410 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-09-13T04:12:05.410 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-09-13T04:12:05.411 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-09-13T04:12:05.411 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-09-13T04:12:05.411 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-09-13T04:12:05.411 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-09-13T04:12:05.411 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-09-13T04:12:05.412 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-09-13T04:12:05.412 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-09-13T04:12:05.412 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-09-13T04:12:05.412 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-09-13T04:12:05.412 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-09-13T04:12:05.413 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-09-13T04:12:05.413 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-09-13T04:12:05.413 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-09-13T04:12:05.413 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-09-13T04:12:05.413 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-09-13T04:12:05.414 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-09-13T04:12:05.414 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-09-13T04:12:05.414 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-09-13T04:12:05.414 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-09-13T04:12:05.414 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-09-13T04:12:05.415 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-09-13T04:12:05.415 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-09-13T04:12:05.415 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-09-13T04:12:05.415 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-09-13T04:12:05.415 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-09-13T04:12:05.416 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-09-13T04:12:05.416 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-09-13T04:12:05.416 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-09-13T04:12:05.416 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-09-13T04:12:05.416 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-09-13T04:12:05.417 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-09-13T04:12:05.417 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-09-13T04:12:05.417 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-09-13T04:12:05.417 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-09-13T04:12:05.418 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-09-13T04:12:05.418 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-09-13T04:12:05.418 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-09-13T04:12:05.419 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-09-13T04:12:05.419 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-09-13T04:12:05.419 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-09-13T04:12:05.420 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-09-13T04:12:05.420 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-09-13T04:12:05.420 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-09-13T04:12:05.420 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-09-13T04:12:05.421 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-09-13T04:12:05.421 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-09-13T04:12:05.421 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-09-13T04:12:05.421 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-09-13T04:12:05.421 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-09-13T04:12:05.422 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-09-13T04:12:05.422 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-09-13T04:12:05.422 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-09-13T04:12:05.422 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-09-13T04:12:05.423 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-09-13T04:12:05.423 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-09-13T04:12:05.423 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-09-13T04:12:05.423 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-09-13T04:12:05.423 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-09-13T04:12:05.424 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-09-13T04:12:05.424 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-09-13T04:12:05.424 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-09-13T04:12:05.424 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-09-13T04:12:05.424 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-09-13T04:12:05.425 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-09-13T04:12:05.425 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-09-13T04:12:05.425 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-09-13T04:12:05.425 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-09-13T04:12:05.425 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-09-13T04:12:05.426 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-09-13T04:12:05.426 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-09-13T04:12:05.426 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-09-13T04:12:05.426 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-09-13T04:12:05.426 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-09-13T04:12:05.427 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-09-13T04:12:05.427 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-09-13T04:12:05.427 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-09-13T04:12:05.427 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-09-13T04:12:05.427 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-09-13T04:12:05.428 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-09-13T04:12:05.428 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-09-13T04:12:05.428 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-09-13T04:12:05.428 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-09-13T04:12:05.428 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-09-13T04:12:05.429 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-09-13T04:12:05.429 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-09-13T04:12:05.429 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-09-13T04:12:05.429 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-09-13T04:12:05.429 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-09-13T04:12:05.430 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-09-13T04:12:05.430 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-09-13T04:12:05.430 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-09-13T04:12:05.430 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-09-13T04:12:05.430 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-09-13T04:12:05.431 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-09-13T04:12:05.431 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-09-13T04:12:05.431 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-09-13T04:12:05.431 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-09-13T04:12:05.431 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-09-13T04:12:05.432 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-09-13T04:12:05.432 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-09-13T04:12:05.432 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-09-13T04:12:05.432 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-09-13T04:12:05.432 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-09-13T04:12:05.433 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-09-13T04:12:05.433 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-09-13T04:12:05.433 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-09-13T04:12:05.433 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-09-13T04:12:05.433 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-09-13T04:12:05.433 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-09-13T04:12:05.434 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-09-13T04:12:05.434 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-09-13T04:12:05.434 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-09-13T04:12:05.434 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-09-13T04:12:05.434 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-09-13T04:12:05.435 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-09-13T04:12:05.435 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-09-13T04:12:05.435 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-09-13T04:12:05.435 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-09-13T04:12:05.435 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-09-13T04:12:05.436 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-09-13T04:12:05.436 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-09-13T04:12:05.436 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-09-13T04:12:05.436 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-09-13T04:12:05.436 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-09-13T04:12:05.437 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-09-13T04:12:05.437 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-09-13T04:12:05.437 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-09-13T04:12:05.437 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-09-13T04:12:05.437 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-09-13T04:12:05.438 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-09-13T04:12:05.438 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-09-13T04:12:05.438 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-09-13T04:12:05.438 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-09-13T04:12:05.438 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-09-13T04:12:05.439 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-09-13T04:12:05.439 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-09-13T04:12:05.439 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-09-13T04:12:05.439 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-09-13T04:12:05.439 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-09-13T04:12:05.439 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-09-13T04:12:05.440 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-09-13T04:12:05.440 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-09-13T04:12:05.440 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-09-13T04:12:05.440 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-09-13T04:12:05.440 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-09-13T04:12:05.441 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-09-13T04:12:05.441 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-09-13T04:12:05.441 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-09-13T04:12:05.441 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-09-13T04:12:05.441 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-09-13T04:12:05.442 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-09-13T04:12:05.442 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-09-13T04:12:05.442 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-09-13T04:12:05.442 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-09-13T04:12:05.442 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-09-13T04:12:05.443 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-09-13T04:12:05.443 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-09-13T04:12:05.443 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-09-13T04:12:05.443 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-09-13T04:12:05.443 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-09-13T04:12:05.444 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-09-13T04:12:05.444 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-09-13T04:12:05.444 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-09-13T04:12:05.444 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-09-13T04:12:05.444 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-09-13T04:12:05.445 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-09-13T04:12:05.445 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-09-13T04:12:05.445 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-09-13T04:12:05.445 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-09-13T04:12:05.445 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-09-13T04:12:05.446 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-09-13T04:12:05.446 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-09-13T04:12:05.446 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-09-13T04:12:05.446 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-09-13T04:12:05.446 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-09-13T04:12:05.447 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-09-13T04:12:05.447 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-09-13T04:12:05.447 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-09-13T04:12:05.447 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-09-13T04:12:05.447 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-09-13T04:12:05.448 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-09-13T04:12:05.448 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-09-13T04:12:05.448 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-09-13T04:12:05.448 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-09-13T04:12:05.448 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-09-13T04:12:05.449 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-09-13T04:12:05.449 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-09-13T04:12:05.449 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-09-13T04:12:05.449 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-09-13T04:12:05.449 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-09-13T04:12:05.449 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-09-13T04:12:05.450 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-09-13T04:12:05.450 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-09-13T04:12:05.450 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-09-13T04:12:05.450 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-09-13T04:12:05.450 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-09-13T04:12:05.451 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-09-13T04:12:05.451 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-09-13T04:12:05.451 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-09-13T04:12:05.451 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-09-13T04:12:05.451 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-09-13T04:12:05.452 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-09-13T04:12:05.452 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-09-13T04:12:05.452 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-09-13T04:12:05.452 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-09-13T04:12:05.452 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-09-13T04:12:05.453 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-09-13T04:12:05.453 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-09-13T04:12:05.453 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-09-13T04:12:05.453 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-09-13T04:12:05.453 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-09-13T04:12:05.454 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-09-13T04:12:05.454 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-09-13T04:12:05.454 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-09-13T04:12:05.454 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-09-13T04:12:05.454 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-09-13T04:12:05.455 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-09-13T04:12:05.455 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-09-13T04:12:05.455 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-09-13T04:12:05.455 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-09-13T04:12:05.455 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-09-13T04:12:05.455 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-09-13T04:12:05.456 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-09-13T04:12:05.456 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-09-13T04:12:05.456 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-09-13T04:12:05.456 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-09-13T04:12:05.456 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-09-13T04:12:05.457 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-09-13T04:12:05.457 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-09-13T04:12:05.457 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-09-13T04:12:05.457 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-09-13T04:12:05.457 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-09-13T04:12:05.458 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-09-13T04:12:05.458 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-09-13T04:12:05.458 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-09-13T04:12:05.458 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-09-13T04:12:05.458 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-09-13T04:12:05.459 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-09-13T04:12:05.459 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-09-13T04:12:05.459 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-09-13T04:12:05.459 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-09-13T04:12:05.459 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-09-13T04:12:05.460 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-09-13T04:12:05.460 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-09-13T04:12:05.460 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-09-13T04:12:05.460 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-09-13T04:12:05.460 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-09-13T04:12:05.461 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-09-13T04:12:05.461 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-09-13T04:12:05.461 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-09-13T04:12:05.461 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-09-13T04:12:05.461 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-09-13T04:12:05.461 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-09-13T04:12:05.462 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-09-13T04:12:05.462 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-09-13T04:12:05.462 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-09-13T04:12:05.462 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-09-13T04:12:05.463 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-09-13T04:12:05.463 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-09-13T04:12:05.463 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-09-13T04:12:05.463 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-09-13T04:12:05.463 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-09-13T04:12:05.464 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-09-13T04:12:05.464 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-09-13T04:12:05.464 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-09-13T04:12:05.464 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-09-13T04:12:05.464 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-09-13T04:12:05.464 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-09-13T04:12:05.465 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-09-13T04:12:05.465 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-09-13T04:12:05.465 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-09-13T04:12:05.465 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-09-13T04:12:05.465 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-09-13T04:12:05.466 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-09-13T04:12:05.466 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-09-13T04:12:05.466 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-09-13T04:12:05.466 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-09-13T04:12:05.466 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-09-13T04:12:05.467 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-09-13T04:12:05.467 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-09-13T04:12:05.467 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-09-13T04:12:05.467 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-09-13T04:12:05.467 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-09-13T04:12:05.468 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-09-13T04:12:05.468 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-09-13T04:12:05.468 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-09-13T04:12:05.468 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-09-13T04:12:05.468 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-09-13T04:12:05.469 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-09-13T04:12:05.469 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-09-13T04:12:05.469 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-09-13T04:12:05.469 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-09-13T04:12:05.469 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-09-13T04:12:05.469 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-09-13T04:12:05.470 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-09-13T04:12:05.470 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-09-13T04:12:05.470 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-09-13T04:12:05.470 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-09-13T04:12:05.470 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-09-13T04:12:05.471 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-09-13T04:12:05.471 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-09-13T04:12:05.471 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-09-13T04:12:05.471 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-09-13T04:12:05.471 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-09-13T04:12:05.472 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-09-13T04:12:05.472 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-09-13T04:12:05.472 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-09-13T04:12:05.472 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-09-13T04:12:05.472 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-09-13T04:12:05.473 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-09-13T04:12:05.473 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-09-13T04:12:05.473 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-09-13T04:12:05.473 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-09-13T04:12:05.473 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-09-13T04:12:05.474 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-09-13T04:12:05.474 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-09-13T04:12:05.474 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-09-13T04:12:05.474 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-09-13T04:12:05.474 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-09-13T04:12:05.475 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-09-13T04:12:05.475 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-09-13T04:12:05.475 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-09-13T04:12:05.475 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-09-13T04:12:05.475 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-09-13T04:12:05.476 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-09-13T04:12:05.476 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-09-13T04:12:05.476 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-09-13T04:12:05.476 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-09-13T04:12:05.476 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-09-13T04:12:05.477 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-09-13T04:12:05.477 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-09-13T04:12:05.477 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-09-13T04:12:05.477 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-09-13T04:12:05.477 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-09-13T04:12:05.477 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-09-13T04:12:05.478 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-09-13T04:12:05.478 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-09-13T04:12:05.478 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-09-13T04:12:05.478 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-09-13T04:12:05.479 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-09-13T04:12:05.479 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-09-13T04:12:05.479 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-09-13T04:12:05.479 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-09-13T04:12:05.479 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-09-13T04:12:05.480 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-09-13T04:12:05.480 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-09-13T04:12:05.480 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-09-13T04:12:05.480 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-09-13T04:12:05.480 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-09-13T04:12:05.481 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-09-13T04:12:05.481 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-09-13T04:12:05.481 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-09-13T04:12:05.481 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-09-13T04:12:05.481 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-09-13T04:12:05.482 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-09-13T04:12:05.482 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-09-13T04:12:05.482 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-09-13T04:12:05.482 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-09-13T04:12:05.482 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-09-13T04:12:05.483 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-09-13T04:12:05.483 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-09-13T04:12:05.483 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-09-13T04:12:05.483 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-09-13T04:12:05.483 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-09-13T04:12:05.484 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-09-13T04:12:05.484 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-09-13T04:12:05.484 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-09-13T04:12:05.484 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-09-13T04:12:05.484 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-09-13T04:12:05.485 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-09-13T04:12:05.485 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-09-13T04:12:05.485 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-09-13T04:12:05.485 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-09-13T04:12:05.485 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-09-13T04:12:05.486 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-09-13T04:12:05.486 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-09-13T04:12:05.486 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-09-13T04:12:05.486 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-09-13T04:12:05.486 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-09-13T04:12:05.487 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-09-13T04:12:05.487 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-09-13T04:12:05.487 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-09-13T04:12:05.487 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-09-13T04:12:05.487 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-09-13T04:12:05.488 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-09-13T04:12:05.488 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-09-13T04:12:05.488 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-09-13T04:12:05.488 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-09-13T04:12:05.488 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-09-13T04:12:05.489 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-09-13T04:12:05.489 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-09-13T04:12:05.489 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-09-13T04:12:05.489 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-09-13T04:12:05.489 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-09-13T04:12:05.490 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-09-13T04:12:05.490 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-09-13T04:12:05.490 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-09-13T04:12:05.490 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-09-13T04:12:05.490 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-09-13T04:12:05.491 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-09-13T04:12:05.491 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-09-13T04:12:05.491 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-09-13T04:12:05.491 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-09-13T04:12:05.491 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-09-13T04:12:05.492 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-09-13T04:12:05.492 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-09-13T04:12:05.492 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-09-13T04:12:05.492 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-09-13T04:12:05.493 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-09-13T04:12:05.493 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-09-13T04:12:05.493 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-09-13T04:12:05.493 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-09-13T04:12:05.493 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-09-13T04:12:05.494 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-09-13T04:12:05.494 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-09-13T04:12:05.494 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-09-13T04:12:05.494 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-09-13T04:12:05.494 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-09-13T04:12:05.495 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-09-13T04:12:05.495 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-09-13T04:12:05.495 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-09-13T04:12:05.495 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-09-13T04:12:05.495 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-09-13T04:12:05.496 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-09-13T04:12:05.496 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-09-13T04:12:05.496 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-09-13T04:12:05.496 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-09-13T04:12:05.496 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-09-13T04:12:05.497 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-09-13T04:12:05.497 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-09-13T04:12:05.497 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-09-13T04:12:05.497 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-09-13T04:12:05.497 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-09-13T04:12:05.498 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-09-13T04:12:05.498 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-09-13T04:12:05.498 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-09-13T04:12:05.498 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-09-13T04:12:05.498 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-09-13T04:12:05.499 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-09-13T04:12:05.499 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-09-13T04:12:05.499 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-09-13T04:12:05.499 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-09-13T04:12:05.499 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-09-13T04:12:05.500 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-09-13T04:12:05.500 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-09-13T04:12:05.500 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-09-13T04:12:05.500 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-09-13T04:12:05.500 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-09-13T04:12:05.501 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-09-13T04:12:05.501 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-09-13T04:12:05.501 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-09-13T04:12:05.501 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-09-13T04:12:05.501 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-09-13T04:12:05.502 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-09-13T04:12:05.502 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-09-13T04:12:05.502 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-09-13T04:12:05.502 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-09-13T04:12:05.502 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-09-13T04:12:05.503 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-09-13T04:12:05.503 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-09-13T04:12:05.503 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-09-13T04:12:05.503 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-09-13T04:12:05.503 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-09-13T04:12:05.504 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-09-13T04:12:05.504 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-09-13T04:12:05.504 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-09-13T04:12:05.504 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-09-13T04:12:05.504 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-09-13T04:12:05.505 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-09-13T04:12:05.505 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-09-13T04:12:05.505 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-09-13T04:12:05.505 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-09-13T04:12:05.505 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-09-13T04:12:05.506 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-09-13T04:12:05.506 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-09-13T04:12:05.506 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-09-13T04:12:05.506 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-09-13T04:12:05.506 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-09-13T04:12:05.507 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-09-13T04:12:05.507 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-09-13T04:12:05.507 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-09-13T04:12:05.507 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-09-13T04:12:05.507 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-09-13T04:12:05.508 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-09-13T04:12:05.508 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-09-13T04:12:05.508 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-09-13T04:12:05.508 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-09-13T04:12:05.508 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-09-13T04:12:05.509 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-09-13T04:12:05.509 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-09-13T04:12:05.509 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-09-13T04:12:05.509 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-09-13T04:12:05.509 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-09-13T04:12:05.510 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-09-13T04:12:05.510 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-09-13T04:12:05.510 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-09-13T04:12:05.510 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-09-13T04:12:05.510 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-09-13T04:12:05.511 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-09-13T04:12:05.511 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-09-13T04:12:05.511 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-09-13T04:12:05.511 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-09-13T04:12:05.511 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-09-13T04:12:05.512 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-09-13T04:12:05.512 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-09-13T04:12:05.512 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-09-13T04:12:05.512 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-09-13T04:12:05.512 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-09-13T04:12:05.513 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-09-13T04:12:05.513 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-09-13T04:12:05.513 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-09-13T04:12:05.513 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-09-13T04:12:05.513 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-09-13T04:12:05.514 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-09-13T04:12:05.514 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-09-13T04:12:05.514 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-09-13T04:12:05.514 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-09-13T04:12:05.514 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-09-13T04:12:05.515 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-09-13T04:12:05.515 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-09-13T04:12:05.515 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-09-13T04:12:05.515 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-09-13T04:12:05.515 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-09-13T04:12:05.516 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-09-13T04:12:05.516 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-09-13T04:12:05.516 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-09-13T04:12:05.516 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-09-13T04:12:05.516 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-09-13T04:12:05.517 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-09-13T04:12:05.517 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-09-13T04:12:05.517 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-09-13T04:12:05.517 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-09-13T04:12:05.517 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-09-13T04:12:05.518 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-09-13T04:12:05.518 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-09-13T04:12:05.518 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-09-13T04:12:05.518 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-09-13T04:12:05.518 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-09-13T04:12:05.519 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-09-13T04:12:05.519 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-09-13T04:12:05.519 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-09-13T04:12:05.519 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-09-13T04:12:05.519 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-09-13T04:12:05.520 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-09-13T04:12:05.520 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-09-13T04:12:05.520 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-09-13T04:12:05.520 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-09-13T04:12:05.520 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-09-13T04:12:05.521 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-09-13T04:12:05.521 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-09-13T04:12:05.521 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-09-13T04:12:05.521 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-09-13T04:12:05.521 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-09-13T04:12:05.522 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-09-13T04:12:05.522 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-09-13T04:12:05.522 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-09-13T04:12:05.522 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-09-13T04:12:05.522 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-09-13T04:12:05.523 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-09-13T04:12:05.523 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-09-13T04:12:05.523 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-09-13T04:12:05.523 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-09-13T04:12:05.523 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-09-13T04:12:05.524 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-09-13T04:12:05.524 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-09-13T04:12:05.524 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-09-13T04:12:05.524 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-09-13T04:12:05.524 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-09-13T04:12:05.524 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-09-13T04:12:05.525 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-09-13T04:12:05.525 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-09-13T04:12:05.525 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-09-13T04:12:05.525 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-09-13T04:12:05.525 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-09-13T04:12:05.526 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-09-13T04:12:05.526 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-09-13T04:12:05.526 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-09-13T04:12:05.526 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-09-13T04:12:05.526 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-09-13T04:12:05.527 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-09-13T04:12:05.527 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-09-13T04:12:05.527 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-09-13T04:12:05.527 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-09-13T04:12:05.527 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-09-13T04:12:05.528 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-09-13T04:12:05.528 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-09-13T04:12:05.528 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-09-13T04:12:05.528 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-09-13T04:12:05.528 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-09-13T04:12:05.529 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-09-13T04:12:05.529 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-09-13T04:12:05.529 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-09-13T04:12:05.529 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-09-13T04:12:05.529 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-09-13T04:12:05.529 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-09-13T04:12:05.530 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-09-13T04:12:05.530 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-09-13T04:12:05.530 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-09-13T04:12:05.530 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-09-13T04:12:05.530 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-09-13T04:12:05.531 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-09-13T04:12:05.531 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-09-13T04:12:05.531 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-09-13T04:12:05.531 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-09-13T04:12:05.531 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-09-13T04:12:05.532 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-09-13T04:12:05.533 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-09-13T04:12:05.534 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-09-13T04:12:05.534 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-09-13T04:12:05.534 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-09-13T04:12:05.534 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-09-13T04:12:05.534 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-09-13T04:12:05.535 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-09-13T04:12:05.535 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-09-13T04:12:05.535 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-09-13T04:12:05.535 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-09-13T04:12:05.535 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-09-13T04:12:05.536 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-09-13T04:12:05.536 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-09-13T04:12:05.536 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-09-13T04:12:05.536 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-09-13T04:12:05.536 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-09-13T04:12:05.536 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-09-13T04:12:05.537 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-09-13T04:12:05.537 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-09-13T04:12:05.537 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-09-13T04:12:05.537 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-09-13T04:12:05.537 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-09-13T04:12:05.538 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-09-13T04:12:05.538 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-09-13T04:12:05.538 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-09-13T04:12:05.538 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-09-13T04:12:05.538 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-09-13T04:12:05.539 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-09-13T04:12:05.539 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-09-13T04:12:05.539 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-09-13T04:12:05.539 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-09-13T04:12:05.539 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-09-13T04:12:05.540 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-09-13T04:12:05.540 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-09-13T04:12:05.540 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-09-13T04:12:05.540 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-09-13T04:12:05.540 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-09-13T04:12:05.540 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-09-13T04:12:05.541 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-09-13T04:12:05.541 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-09-13T04:12:05.541 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-09-13T04:12:05.541 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-09-13T04:12:05.541 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-09-13T04:12:05.542 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-09-13T04:12:05.542 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-09-13T04:12:05.542 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-09-13T04:12:05.542 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-09-13T04:12:05.542 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-09-13T04:12:05.543 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-09-13T04:12:05.543 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-09-13T04:12:05.543 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-09-13T04:12:05.543 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-09-13T04:12:05.543 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-09-13T04:12:05.544 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-09-13T04:12:05.544 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-09-13T04:12:05.544 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-09-13T04:12:05.544 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-09-13T04:12:05.544 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-09-13T04:12:05.544 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-09-13T04:12:05.545 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-09-13T04:12:05.545 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-09-13T04:12:05.545 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-09-13T04:12:05.545 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-09-13T04:12:05.545 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-09-13T04:12:05.546 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-09-13T04:12:05.546 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-09-13T04:12:05.546 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-09-13T04:12:05.546 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-09-13T04:12:05.546 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-09-13T04:12:05.547 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-09-13T04:12:05.547 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-09-13T04:12:05.547 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-09-13T04:12:05.547 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-09-13T04:12:05.547 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-09-13T04:12:05.547 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-09-13T04:12:05.548 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-09-13T04:12:05.548 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-09-13T04:12:05.548 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-09-13T04:12:05.548 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-09-13T04:12:05.548 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-09-13T04:12:05.549 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-09-13T04:12:05.549 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-09-13T04:12:05.549 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-09-13T04:12:05.549 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-09-13T04:12:05.549 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-09-13T04:12:05.550 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-09-13T04:12:05.550 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-09-13T04:12:05.550 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-09-13T04:12:05.550 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-09-13T04:12:05.550 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-09-13T04:12:05.551 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-09-13T04:12:05.551 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-09-13T04:12:05.551 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-09-13T04:12:05.551 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-09-13T04:12:05.551 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-09-13T04:12:05.551 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-09-13T04:12:05.552 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-09-13T04:12:05.552 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-09-13T04:12:05.552 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-09-13T04:12:05.552 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-09-13T04:12:05.552 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-09-13T04:12:05.553 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-09-13T04:12:05.553 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-09-13T04:12:05.553 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-09-13T04:12:05.553 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-09-13T04:12:05.553 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-09-13T04:12:05.554 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-09-13T04:12:05.554 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-09-13T04:12:05.554 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-09-13T04:12:05.554 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-09-13T04:12:05.554 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-09-13T04:12:05.555 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-09-13T04:12:05.555 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-09-13T04:12:05.555 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-09-13T04:12:05.555 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-09-13T04:12:05.555 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-09-13T04:12:05.555 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-09-13T04:12:05.556 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-09-13T04:12:05.556 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-09-13T04:12:05.556 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-09-13T04:12:05.556 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-09-13T04:12:05.556 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-09-13T04:12:05.557 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-09-13T04:12:05.557 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-09-13T04:12:05.557 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-09-13T04:12:05.557 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-09-13T04:12:05.557 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-09-13T04:12:05.558 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-09-13T04:12:05.558 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-09-13T04:12:05.558 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-09-13T04:12:05.558 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-09-13T04:12:05.558 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-09-13T04:12:05.559 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-09-13T04:12:05.559 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-09-13T04:12:05.559 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-09-13T04:12:05.559 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-09-13T04:12:05.559 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-09-13T04:12:05.559 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-09-13T04:12:05.560 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-09-13T04:12:05.560 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-09-13T04:12:05.560 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-09-13T04:12:05.560 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-09-13T04:12:05.560 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-09-13T04:12:05.561 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-09-13T04:12:05.561 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-09-13T04:12:05.561 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-09-13T04:12:05.561 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-09-13T04:12:05.561 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-09-13T04:12:05.562 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-09-13T04:12:05.562 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-09-13T04:12:05.562 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-09-13T04:12:05.562 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-09-13T04:12:05.562 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-09-13T04:12:05.563 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-09-13T04:12:05.563 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-09-13T04:12:05.563 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-09-13T04:12:05.563 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-09-13T04:12:05.563 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-09-13T04:12:05.563 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-09-13T04:12:05.564 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-09-13T04:12:05.564 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-09-13T04:12:05.564 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-09-13T04:12:05.564 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-09-13T04:12:05.564 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-09-13T04:12:05.565 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-09-13T04:12:05.565 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-09-13T04:12:05.565 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-09-13T04:12:05.565 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-09-13T04:12:05.565 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-09-13T04:12:05.566 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-09-13T04:12:05.566 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-09-13T04:12:05.566 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-09-13T04:12:05.566 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-09-13T04:12:05.566 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-09-13T04:12:05.567 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-09-13T04:12:05.567 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-09-13T04:12:05.567 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-09-13T04:12:05.567 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-09-13T04:12:05.567 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-09-13T04:12:05.567 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-09-13T04:12:05.568 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-09-13T04:12:05.568 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-09-13T04:12:05.568 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-09-13T04:12:05.568 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-09-13T04:12:05.568 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-09-13T04:12:05.569 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-09-13T04:12:05.569 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-09-13T04:12:05.569 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-09-13T04:12:05.569 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-09-13T04:12:05.569 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-09-13T04:12:05.570 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-09-13T04:12:05.570 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-09-13T04:12:05.570 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-09-13T04:12:05.570 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-09-13T04:12:05.570 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-09-13T04:12:05.570 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-09-13T04:12:05.571 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-09-13T04:12:05.571 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-09-13T04:12:05.571 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-09-13T04:12:05.571 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-09-13T04:12:05.571 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-09-13T04:12:05.572 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-09-13T04:12:05.572 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-09-13T04:12:05.572 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-09-13T04:12:05.572 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-09-13T04:12:05.572 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-09-13T04:12:05.573 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-09-13T04:12:05.573 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-09-13T04:12:05.573 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-09-13T04:12:05.573 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-09-13T04:12:05.573 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-09-13T04:12:05.574 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-09-13T04:12:05.574 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-09-13T04:12:05.574 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-09-13T04:12:05.574 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-09-13T04:12:05.574 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-09-13T04:12:05.575 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-09-13T04:12:05.575 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-09-13T04:12:05.575 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-09-13T04:12:05.575 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-09-13T04:12:05.575 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-09-13T04:12:05.576 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-09-13T04:12:05.576 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-09-13T04:12:05.576 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-09-13T04:12:05.576 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-09-13T04:12:05.576 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-09-13T04:12:05.576 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-09-13T04:12:05.577 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-09-13T04:12:05.577 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-09-13T04:12:05.577 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-09-13T04:12:05.577 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-09-13T04:12:05.577 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-09-13T04:12:05.578 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-09-13T04:12:05.578 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-09-13T04:12:05.578 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-09-13T04:12:05.578 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-09-13T04:12:05.578 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-09-13T04:12:05.579 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-09-13T04:12:05.579 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-09-13T04:12:05.579 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-09-13T04:12:05.579 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-09-13T04:12:05.579 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-09-13T04:12:05.580 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-09-13T04:12:05.580 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-09-13T04:12:05.580 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-09-13T04:12:05.580 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-09-13T04:12:05.580 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-09-13T04:12:05.581 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-09-13T04:12:05.581 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-09-13T04:12:05.581 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-09-13T04:12:05.581 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-09-13T04:12:05.581 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-09-13T04:12:05.581 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-09-13T04:12:05.582 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-09-13T04:12:05.582 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-09-13T04:12:05.582 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-09-13T04:12:05.582 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-09-13T04:12:05.582 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-09-13T04:12:05.583 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-09-13T04:12:05.583 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-09-13T04:12:05.583 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-09-13T04:12:05.583 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-09-13T04:12:05.583 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-09-13T04:12:05.584 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-09-13T04:12:05.584 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-09-13T04:12:05.584 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-09-13T04:12:05.584 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-09-13T04:12:05.584 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-09-13T04:12:05.585 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-09-13T04:12:05.585 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-09-13T04:12:05.585 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-09-13T04:12:05.585 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-09-13T04:12:05.585 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-09-13T04:12:05.586 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-09-13T04:12:05.586 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-09-13T04:12:05.586 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-09-13T04:12:05.586 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-09-13T04:12:05.586 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-09-13T04:12:05.587 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-09-13T04:12:05.587 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-09-13T04:12:05.587 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-09-13T04:12:05.587 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-09-13T04:12:05.587 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-09-13T04:12:05.587 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-09-13T04:12:05.588 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-09-13T04:12:05.588 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-09-13T04:12:05.588 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-09-13T04:12:05.588 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-09-13T04:12:05.588 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-09-13T04:12:05.589 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-09-13T04:12:05.589 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-09-13T04:12:05.589 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-09-13T04:12:05.589 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-09-13T04:12:05.589 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-09-13T04:12:05.590 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-09-13T04:12:05.590 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-09-13T04:12:05.590 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-09-13T04:12:05.590 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-09-13T04:12:05.590 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-09-13T04:12:05.591 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-09-13T04:12:05.591 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-09-13T04:12:05.591 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-09-13T04:12:05.591 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-09-13T04:12:05.591 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-09-13T04:12:05.591 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-09-13T04:12:05.592 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-09-13T04:12:05.592 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-09-13T04:12:05.592 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-09-13T04:12:05.592 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-09-13T04:12:05.592 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-09-13T04:12:05.593 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-09-13T04:12:05.593 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-09-13T04:12:05.593 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-09-13T04:12:05.593 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-09-13T04:12:05.593 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-09-13T04:12:05.594 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-09-13T04:12:05.594 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-09-13T04:12:05.594 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-09-13T04:12:05.594 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-09-13T04:12:05.594 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-09-13T04:12:05.595 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-09-13T04:12:05.595 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-09-13T04:12:05.595 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-09-13T04:12:05.595 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-09-13T04:12:05.595 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-09-13T04:12:05.596 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-09-13T04:12:05.596 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-09-13T04:12:05.596 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-09-13T04:12:05.596 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-09-13T04:12:05.596 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-09-13T04:12:05.596 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-09-13T04:12:05.597 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-09-13T04:12:05.597 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-09-13T04:12:05.597 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-09-13T04:12:05.597 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-09-13T04:12:05.597 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-09-13T04:12:05.598 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-09-13T04:12:05.598 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-09-13T04:12:05.598 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-09-13T04:12:05.598 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-09-13T04:12:05.598 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-09-13T04:12:05.599 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-09-13T04:12:05.599 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-09-13T04:12:05.599 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-09-13T04:12:05.599 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-09-13T04:12:05.599 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-09-13T04:12:05.600 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-09-13T04:12:05.600 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-09-13T04:12:05.600 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-09-13T04:12:05.600 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-09-13T04:12:05.600 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-09-13T04:12:05.600 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-09-13T04:12:05.601 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-09-13T04:12:05.601 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-09-13T04:12:05.601 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-09-13T04:12:05.601 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-09-13T04:12:05.601 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-09-13T04:12:05.602 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-09-13T04:12:05.602 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-09-13T04:12:05.602 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-09-13T04:12:05.602 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-09-13T04:12:05.602 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-09-13T04:12:05.603 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-09-13T04:12:05.603 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-09-13T04:12:05.603 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-09-13T04:12:05.603 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-09-13T04:12:05.603 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-09-13T04:12:05.604 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-09-13T04:12:05.604 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-09-13T04:12:05.604 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-09-13T04:12:05.604 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-09-13T04:12:05.604 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-09-13T04:12:05.604 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-09-13T04:12:05.605 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-09-13T04:12:05.605 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-09-13T04:12:05.605 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-09-13T04:12:05.605 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-09-13T04:12:05.605 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-09-13T04:12:05.606 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-09-13T04:12:05.606 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-09-13T04:12:05.606 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-09-13T04:12:05.606 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-09-13T04:12:05.606 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-09-13T04:12:05.607 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-09-13T04:12:05.607 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-09-13T04:12:05.607 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-09-13T04:12:05.607 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-09-13T04:12:05.607 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-09-13T04:12:05.607 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-09-13T04:12:05.608 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-09-13T04:12:05.608 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-09-13T04:12:05.608 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-09-13T04:12:05.608 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-09-13T04:12:05.608 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-09-13T04:12:05.609 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-09-13T04:12:05.609 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-09-13T04:12:05.609 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-09-13T04:12:05.609 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-09-13T04:12:05.609 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-09-13T04:12:05.610 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-09-13T04:12:05.610 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-09-13T04:12:05.610 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-09-13T04:12:05.610 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-09-13T04:12:05.610 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-09-13T04:12:05.610 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-09-13T04:12:05.611 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-09-13T04:12:05.611 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-09-13T04:12:05.611 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-09-13T04:12:05.611 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-09-13T04:12:05.611 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-09-13T04:12:05.612 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-09-13T04:12:05.612 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-09-13T04:12:05.612 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-09-13T04:12:05.612 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-09-13T04:12:05.612 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-09-13T04:12:05.613 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-09-13T04:12:05.613 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-09-13T04:12:05.613 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-09-13T04:12:05.613 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-09-13T04:12:05.613 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-09-13T04:12:05.613 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-09-13T04:12:05.614 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-09-13T04:12:05.614 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-09-13T04:12:05.614 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-09-13T04:12:05.614 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-09-13T04:12:05.614 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-09-13T04:12:05.615 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-09-13T04:12:05.615 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-09-13T04:12:05.615 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-09-13T04:12:05.615 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-09-13T04:12:05.615 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-09-13T04:12:05.616 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-09-13T04:12:05.616 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-09-13T04:12:05.616 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-09-13T04:12:05.616 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-09-13T04:12:05.616 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-09-13T04:12:05.617 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-09-13T04:12:05.617 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-09-13T04:12:05.617 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-09-13T04:12:05.617 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-09-13T04:12:05.617 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-09-13T04:12:05.618 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-09-13T04:12:05.618 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-09-13T04:12:05.618 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-09-13T04:12:05.618 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-09-13T04:12:05.618 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-09-13T04:12:05.619 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-09-13T04:12:05.619 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-09-13T04:12:05.619 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-09-13T04:12:05.619 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-09-13T04:12:05.619 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-09-13T04:12:05.620 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-09-13T04:12:05.620 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-09-13T04:12:05.620 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-09-13T04:12:05.620 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-09-13T04:12:05.620 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-09-13T04:12:05.621 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-09-13T04:12:05.621 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-09-13T04:12:05.621 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-09-13T04:12:05.621 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-09-13T04:12:05.621 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-09-13T04:12:05.621 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-09-13T04:12:05.622 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-09-13T04:12:05.622 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-09-13T04:12:05.622 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-09-13T04:12:05.622 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-09-13T04:12:05.622 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-09-13T04:12:05.623 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-09-13T04:12:05.623 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-09-13T04:12:05.623 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-09-13T04:12:05.623 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-09-13T04:12:05.623 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-09-13T04:12:05.624 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-09-13T04:12:05.624 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-09-13T04:12:05.624 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-09-13T04:12:05.624 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-09-13T04:12:05.624 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-09-13T04:12:05.625 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-09-13T04:12:05.625 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-09-13T04:12:05.625 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-09-13T04:12:05.625 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-09-13T04:12:05.625 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-09-13T04:12:05.626 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-09-13T04:12:05.626 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-09-13T04:12:05.626 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-09-13T04:12:05.626 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-09-13T04:12:05.626 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-09-13T04:12:05.627 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-09-13T04:12:05.627 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-09-13T04:12:05.627 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-09-13T04:12:05.627 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-09-13T04:12:05.627 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-09-13T04:12:05.628 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-09-13T04:12:05.628 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-09-13T04:12:05.628 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-09-13T04:12:05.628 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-09-13T04:12:05.628 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-09-13T04:12:05.629 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-09-13T04:12:05.629 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-09-13T04:12:05.629 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-09-13T04:12:05.629 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-09-13T04:12:05.629 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-09-13T04:12:05.629 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-09-13T04:12:05.630 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-09-13T04:12:05.630 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-09-13T04:12:05.630 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-09-13T04:12:05.630 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-09-13T04:12:05.630 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-09-13T04:12:05.631 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-09-13T04:12:05.631 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-09-13T04:12:05.631 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-09-13T04:12:05.631 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-09-13T04:12:05.631 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-09-13T04:12:05.632 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-09-13T04:12:05.632 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-09-13T04:12:05.632 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-09-13T04:12:05.632 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-09-13T04:12:05.632 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-09-13T04:12:05.633 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-09-13T04:12:05.633 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-09-13T04:12:05.633 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-09-13T04:12:05.633 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-09-13T04:12:05.633 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-09-13T04:12:05.633 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-09-13T04:12:05.634 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-09-13T04:12:05.634 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-09-13T04:12:05.634 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-09-13T04:12:05.634 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-09-13T04:12:05.634 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-09-13T04:12:05.635 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-09-13T04:12:05.635 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-09-13T04:12:05.635 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-09-13T04:12:05.635 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-09-13T04:12:05.635 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-09-13T04:12:05.636 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-09-13T04:12:05.636 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-09-13T04:12:05.636 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-09-13T04:12:05.636 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-09-13T04:12:05.636 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-09-13T04:12:05.636 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-09-13T04:12:05.637 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-09-13T04:12:05.637 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-09-13T04:12:05.637 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-09-13T04:12:05.637 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-09-13T04:12:05.637 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-09-13T04:12:05.638 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-09-13T04:12:05.638 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-09-13T04:12:05.638 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-09-13T04:12:05.638 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-09-13T04:12:05.638 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-09-13T04:12:05.639 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-09-13T04:12:05.639 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-09-13T04:12:05.639 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-09-13T04:12:05.639 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-09-13T04:12:05.639 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-09-13T04:12:05.640 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-09-13T04:12:05.640 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-09-13T04:12:05.640 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-09-13T04:12:05.640 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-09-13T04:12:05.640 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-09-13T04:12:05.640 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-09-13T04:12:05.641 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-09-13T04:12:05.641 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-09-13T04:12:05.641 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-09-13T04:12:05.641 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-09-13T04:12:05.641 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-09-13T04:12:05.642 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-09-13T04:12:05.642 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-09-13T04:12:05.642 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-09-13T04:12:05.642 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-09-13T04:12:05.642 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-09-13T04:12:05.643 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-09-13T04:12:05.643 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-09-13T04:12:05.643 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-09-13T04:12:05.643 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-09-13T04:12:05.643 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-09-13T04:12:05.644 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-09-13T04:12:05.644 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-09-13T04:12:05.644 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-09-13T04:12:05.644 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-09-13T04:12:05.644 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-09-13T04:12:05.644 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-09-13T04:12:05.645 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-09-13T04:12:05.645 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-09-13T04:12:05.645 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-09-13T04:12:05.645 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-09-13T04:12:05.645 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-09-13T04:12:05.646 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-09-13T04:12:05.646 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-09-13T04:12:05.646 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-09-13T04:12:05.646 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-09-13T04:12:05.646 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-09-13T04:12:05.647 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-09-13T04:12:05.647 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-09-13T04:12:05.647 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-09-13T04:12:05.647 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-09-13T04:12:05.647 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-09-13T04:12:05.647 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-09-13T04:12:05.648 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-09-13T04:12:05.648 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-09-13T04:12:05.648 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-09-13T04:12:05.648 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-09-13T04:12:05.648 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-09-13T04:12:05.649 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-09-13T04:12:05.649 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-09-13T04:12:05.649 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-09-13T04:12:05.649 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-09-13T04:12:05.649 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-09-13T04:12:05.650 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-09-13T04:12:05.650 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-09-13T04:12:05.650 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-09-13T04:12:05.650 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-09-13T04:12:05.650 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-09-13T04:12:05.651 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-09-13T04:12:05.651 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-09-13T04:12:05.651 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-09-13T04:12:05.651 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-09-13T04:12:05.651 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-09-13T04:12:05.651 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-09-13T04:12:05.652 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-09-13T04:12:05.652 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-09-13T04:12:05.652 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-09-13T04:12:05.652 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-09-13T04:12:05.652 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-09-13T04:12:05.653 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-09-13T04:12:05.653 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-09-13T04:12:05.653 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-09-13T04:12:05.653 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-09-13T04:12:05.653 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-09-13T04:12:05.654 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-09-13T04:12:05.654 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-09-13T04:12:05.654 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-09-13T04:12:05.654 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-09-13T04:12:05.654 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-09-13T04:12:05.654 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-09-13T04:12:05.655 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-09-13T04:12:05.655 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-09-13T04:12:05.655 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-09-13T04:12:05.655 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-09-13T04:12:05.655 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-09-13T04:12:05.656 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-09-13T04:12:05.656 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-09-13T04:12:05.656 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-09-13T04:12:05.656 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-09-13T04:12:05.656 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-09-13T04:12:05.657 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-09-13T04:12:05.657 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-09-13T04:12:05.657 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-09-13T04:12:05.657 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-09-13T04:12:05.657 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-09-13T04:12:05.658 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-09-13T04:12:05.658 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-09-13T04:12:05.658 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-09-13T04:12:05.658 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-09-13T04:12:05.658 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-09-13T04:12:05.659 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-09-13T04:12:05.659 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-09-13T04:12:05.659 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-09-13T04:12:05.659 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-09-13T04:12:05.659 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-09-13T04:12:05.659 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-09-13T04:12:05.660 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-09-13T04:12:05.660 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-09-13T04:12:05.660 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-09-13T04:12:05.660 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-09-13T04:12:05.660 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-09-13T04:12:05.661 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-09-13T04:12:05.661 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-09-13T04:12:05.661 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-09-13T04:12:05.661 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-09-13T04:12:05.661 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-09-13T04:12:05.662 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-09-13T04:12:05.662 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-09-13T04:12:05.662 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-09-13T04:12:05.662 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-09-13T04:12:05.662 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-09-13T04:12:05.663 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-09-13T04:12:05.663 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-09-13T04:12:05.663 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-09-13T04:12:05.663 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-09-13T04:12:05.663 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-09-13T04:12:05.664 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-09-13T04:12:05.664 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-09-13T04:12:05.664 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-09-13T04:12:05.664 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-09-13T04:12:05.664 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-09-13T04:12:05.664 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-09-13T04:12:05.665 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-09-13T04:12:05.665 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-09-13T04:12:05.665 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-09-13T04:12:05.665 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-09-13T04:12:05.665 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-09-13T04:12:05.666 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-09-13T04:12:05.666 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-09-13T04:12:05.666 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-09-13T04:12:05.666 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-09-13T04:12:05.666 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-09-13T04:12:05.667 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-09-13T04:12:05.667 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-09-13T04:12:05.667 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-09-13T04:12:05.667 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-09-13T04:12:05.667 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-09-13T04:12:05.668 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-09-13T04:12:05.668 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-09-13T04:12:05.668 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-09-13T04:12:05.668 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-09-13T04:12:05.668 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-09-13T04:12:05.669 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-09-13T04:12:05.669 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-09-13T04:12:05.669 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-09-13T04:12:05.669 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-09-13T04:12:05.669 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-09-13T04:12:05.670 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-09-13T04:12:05.670 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-09-13T04:12:05.670 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-09-13T04:12:05.670 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-09-13T04:12:05.670 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-09-13T04:12:05.670 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-09-13T04:12:05.671 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-09-13T04:12:05.671 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-09-13T04:12:05.671 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-09-13T04:12:05.671 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-09-13T04:12:05.671 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-09-13T04:12:05.672 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-09-13T04:12:05.672 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-09-13T04:12:05.672 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-09-13T04:12:05.672 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-09-13T04:12:05.672 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-09-13T04:12:05.673 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-09-13T04:12:05.673 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-09-13T04:12:05.673 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-09-13T04:12:05.673 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-09-13T04:12:05.673 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-09-13T04:12:05.674 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-09-13T04:12:05.674 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-09-13T04:12:05.674 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-09-13T04:12:05.674 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-09-13T04:12:05.674 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-09-13T04:12:05.675 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-09-13T04:12:05.675 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-09-13T04:12:05.675 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-09-13T04:12:05.675 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-09-13T04:12:05.675 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-09-13T04:12:05.675 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-09-13T04:12:05.676 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-09-13T04:12:05.676 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-09-13T04:12:05.676 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-09-13T04:12:05.676 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-09-13T04:12:05.676 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-09-13T04:12:05.677 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-09-13T04:12:05.677 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-09-13T04:12:05.677 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-09-13T04:12:05.677 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-09-13T04:12:05.677 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-09-13T04:12:05.678 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-09-13T04:12:05.678 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-09-13T04:12:05.678 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-09-13T04:12:05.678 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-09-13T04:12:05.678 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-09-13T04:12:05.679 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-09-13T04:12:05.679 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-09-13T04:12:05.679 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-09-13T04:12:05.679 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-09-13T04:12:05.679 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-09-13T04:12:05.680 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-09-13T04:12:05.680 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-09-13T04:12:05.680 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-09-13T04:12:05.680 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-09-13T04:12:05.680 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-09-13T04:12:05.680 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-09-13T04:12:05.681 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-09-13T04:12:05.681 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-09-13T04:12:05.681 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-09-13T04:12:05.681 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-09-13T04:12:05.681 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-09-13T04:12:05.682 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-09-13T04:12:05.682 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-09-13T04:12:05.682 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-09-13T04:12:05.682 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-09-13T04:12:05.682 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-09-13T04:12:05.683 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-09-13T04:12:05.683 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-09-13T04:12:05.683 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-09-13T04:12:05.683 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-09-13T04:12:05.683 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-09-13T04:12:05.684 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-09-13T04:12:05.684 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-09-13T04:12:05.684 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-09-13T04:12:05.684 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-09-13T04:12:05.684 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-09-13T04:12:05.685 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-09-13T04:12:05.685 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-09-13T04:12:05.685 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-09-13T04:12:05.685 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-09-13T04:12:05.685 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-09-13T04:12:05.686 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-09-13T04:12:05.686 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-09-13T04:12:05.686 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-09-13T04:12:05.686 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-09-13T04:12:05.686 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-09-13T04:12:05.687 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-09-13T04:12:05.687 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-09-13T04:12:05.687 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-09-13T04:12:05.687 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-09-13T04:12:05.687 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-09-13T04:12:05.687 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-09-13T04:12:05.688 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-09-13T04:12:05.688 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-09-13T04:12:05.688 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-09-13T04:12:05.688 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-09-13T04:12:05.688 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-09-13T04:12:05.689 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-09-13T04:12:05.689 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-09-13T04:12:05.689 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-09-13T04:12:05.689 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-09-13T04:12:05.689 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-09-13T04:12:05.689 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-09-13T04:12:05.690 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-09-13T04:12:05.690 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-09-13T04:12:05.690 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-09-13T04:12:05.690 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-09-13T04:12:05.690 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-09-13T04:12:05.691 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-09-13T04:12:05.691 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-09-13T04:12:05.691 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-09-13T04:12:05.691 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-09-13T04:12:05.691 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-09-13T04:12:05.692 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-09-13T04:12:05.692 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-09-13T04:12:05.692 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-09-13T04:12:05.692 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-09-13T04:12:05.692 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-09-13T04:12:05.693 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-09-13T04:12:05.693 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-09-13T04:12:05.693 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-09-13T04:12:05.693 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-09-13T04:12:05.693 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-09-13T04:12:05.693 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-09-13T04:12:05.694 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-09-13T04:12:05.694 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-09-13T04:12:05.694 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-09-13T04:12:05.694 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-09-13T04:12:05.694 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-09-13T04:12:05.695 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-09-13T04:12:05.695 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-09-13T04:12:05.695 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-09-13T04:12:05.695 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-09-13T04:12:05.695 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-09-13T04:12:05.696 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-09-13T04:12:05.696 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-09-13T04:12:05.696 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-09-13T04:12:05.696 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-09-13T04:12:05.696 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-09-13T04:12:05.697 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-09-13T04:12:05.697 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-09-13T04:12:05.697 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-09-13T04:12:05.697 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-09-13T04:12:05.697 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-09-13T04:12:05.697 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-09-13T04:12:05.698 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-09-13T04:12:05.698 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-09-13T04:12:05.698 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-09-13T04:12:05.698 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-09-13T04:12:05.698 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-09-13T04:12:05.699 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-09-13T04:12:05.699 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-09-13T04:12:05.699 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-09-13T04:12:05.699 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-09-13T04:12:05.699 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-09-13T04:12:05.700 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-09-13T04:12:05.700 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-09-13T04:12:05.700 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-09-13T04:12:05.700 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-09-13T04:12:05.700 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-09-13T04:12:05.701 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-09-13T04:12:05.701 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-09-13T04:12:05.701 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-09-13T04:12:05.701 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-09-13T04:12:05.701 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-09-13T04:12:05.701 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-09-13T04:12:05.702 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-09-13T04:12:05.702 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-09-13T04:12:05.702 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-09-13T04:12:05.702 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-09-13T04:12:05.702 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-09-13T04:12:05.703 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-09-13T04:12:05.703 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-09-13T04:12:05.703 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-09-13T04:12:05.703 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-09-13T04:12:05.703 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-09-13T04:12:05.704 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-09-13T04:12:05.704 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-09-13T04:12:05.704 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-09-13T04:12:05.704 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-09-13T04:12:05.704 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-09-13T04:12:05.704 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-09-13T04:12:05.705 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-09-13T04:12:05.705 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-09-13T04:12:05.705 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-09-13T04:12:05.705 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-09-13T04:12:05.705 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-09-13T04:12:05.706 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-09-13T04:12:05.706 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-09-13T04:12:05.706 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-09-13T04:12:05.706 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-09-13T04:12:05.706 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-09-13T04:12:05.706 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-09-13T04:12:05.707 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-09-13T04:12:05.707 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-09-13T04:12:05.707 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-09-13T04:12:05.707 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-09-13T04:12:05.707 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-09-13T04:12:05.708 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-09-13T04:12:05.708 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-09-13T04:12:05.708 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-09-13T04:12:05.708 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-09-13T04:12:05.708 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-09-13T04:12:05.709 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-09-13T04:12:05.709 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-09-13T04:12:05.709 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-09-13T04:12:05.709 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-09-13T04:12:05.709 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-09-13T04:12:05.710 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-09-13T04:12:05.710 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-09-13T04:12:05.710 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-09-13T04:12:05.710 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-09-13T04:12:05.710 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-09-13T04:12:05.710 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-09-13T04:12:05.711 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-09-13T04:12:05.711 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-09-13T04:12:05.711 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-09-13T04:12:05.711 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-09-13T04:12:05.711 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-09-13T04:12:05.712 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-09-13T04:12:05.712 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-09-13T04:12:05.712 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-09-13T04:12:05.712 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-09-13T04:12:05.712 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-09-13T04:12:05.713 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-09-13T04:12:05.713 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-09-13T04:12:05.713 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-09-13T04:12:05.713 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-09-13T04:12:05.713 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-09-13T04:12:05.713 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-09-13T04:12:05.714 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-09-13T04:12:05.714 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-09-13T04:12:05.714 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-09-13T04:12:05.714 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-09-13T04:12:05.714 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-09-13T04:12:05.715 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-09-13T04:12:05.715 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-09-13T04:12:05.715 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-09-13T04:12:05.715 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-09-13T04:12:05.715 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-09-13T04:12:05.716 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-09-13T04:12:05.716 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-09-13T04:12:05.716 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-09-13T04:12:05.716 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-09-13T04:12:05.716 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-09-13T04:12:05.717 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-09-13T04:12:05.717 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-09-13T04:12:05.717 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-09-13T04:12:05.717 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-09-13T04:12:05.717 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-09-13T04:12:05.717 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-09-13T04:12:05.718 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-09-13T04:12:05.718 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-09-13T04:12:05.718 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-09-13T04:12:05.718 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-09-13T04:12:05.718 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-09-13T04:12:05.719 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-09-13T04:12:05.719 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-09-13T04:12:05.719 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-09-13T04:12:05.719 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-09-13T04:12:05.719 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-09-13T04:12:05.720 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-09-13T04:12:05.720 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-09-13T04:12:05.720 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-09-13T04:12:05.720 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-09-13T04:12:05.720 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-09-13T04:12:05.720 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-09-13T04:12:05.721 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-09-13T04:12:05.721 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-09-13T04:12:05.721 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-09-13T04:12:05.721 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-09-13T04:12:05.721 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-09-13T04:12:05.722 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-09-13T04:12:05.722 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-09-13T04:12:05.722 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-09-13T04:12:05.722 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-09-13T04:12:05.722 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-09-13T04:12:05.723 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-09-13T04:12:05.723 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-09-13T04:12:05.723 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-09-13T04:12:05.723 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-09-13T04:12:05.723 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-09-13T04:12:05.724 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-09-13T04:12:05.724 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-09-13T04:12:05.724 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-09-13T04:12:05.724 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-09-13T04:12:05.724 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-09-13T04:12:05.725 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-09-13T04:12:05.725 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-09-13T04:12:05.725 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-09-13T04:12:05.725 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-09-13T04:12:05.725 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-09-13T04:12:05.725 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-09-13T04:12:05.726 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-09-13T04:12:05.726 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-09-13T04:12:05.726 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-09-13T04:12:05.726 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-09-13T04:12:05.726 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-09-13T04:12:05.727 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-09-13T04:12:05.727 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-09-13T04:12:05.727 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-09-13T04:12:05.727 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-09-13T04:12:05.727 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-09-13T04:12:05.728 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-09-13T04:12:05.728 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-09-13T04:12:05.728 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-09-13T04:12:05.728 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-09-13T04:12:05.728 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-09-13T04:12:05.729 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-09-13T04:12:05.729 DEBUG:teuthology.orchestra.run.smithi002:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-09-13T04:12:05.757 INFO:teuthology.orchestra.run.smithi002.stdout:postinstall scriptlet (using /bin/sh): 2023-09-13T04:12:05.758 INFO:teuthology.orchestra.run.smithi002.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-09-13T04:12:05.758 INFO:teuthology.orchestra.run.smithi002.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-09-13T04:12:05.758 INFO:teuthology.orchestra.run.smithi002.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-09-13T04:12:05.759 INFO:teuthology.orchestra.run.smithi002.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-09-13T04:12:05.759 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-09-13T04:12:05.759 INFO:teuthology.orchestra.run.smithi002.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-09-13T04:12:05.759 INFO:teuthology.orchestra.run.smithi002.stdout:fi 2023-09-13T04:12:05.759 INFO:teuthology.orchestra.run.smithi002.stdout:preuninstall scriptlet (using /bin/sh): 2023-09-13T04:12:05.760 INFO:teuthology.orchestra.run.smithi002.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-09-13T04:12:05.760 INFO:teuthology.orchestra.run.smithi002.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-09-13T04:12:05.760 INFO:teuthology.orchestra.run.smithi002.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-09-13T04:12:05.761 INFO:teuthology.orchestra.run.smithi002.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-09-13T04:12:05.761 INFO:teuthology.orchestra.run.smithi002.stdout:fi 2023-09-13T04:12:05.761 INFO:teuthology.orchestra.run.smithi002.stdout:postuninstall scriptlet (using /bin/sh): 2023-09-13T04:12:05.761 INFO:teuthology.orchestra.run.smithi002.stdout:if [ -x /sbin/update-bootloader ]; then 2023-09-13T04:12:05.761 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-09-13T04:12:05.762 INFO:teuthology.orchestra.run.smithi002.stdout:fi 2023-09-13T04:12:05.763 DEBUG:teuthology.orchestra.run.smithi002:> sudo rpm -qi grub2-tools 2023-09-13T04:12:05.842 INFO:teuthology.orchestra.run.smithi002.stdout:Name : grub2-tools 2023-09-13T04:12:05.842 INFO:teuthology.orchestra.run.smithi002.stdout:Epoch : 1 2023-09-13T04:12:05.843 INFO:teuthology.orchestra.run.smithi002.stdout:Version : 2.02 2023-09-13T04:12:05.843 INFO:teuthology.orchestra.run.smithi002.stdout:Release : 123.el8 2023-09-13T04:12:05.843 INFO:teuthology.orchestra.run.smithi002.stdout:Architecture: x86_64 2023-09-13T04:12:05.843 INFO:teuthology.orchestra.run.smithi002.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-09-13T04:12:05.843 INFO:teuthology.orchestra.run.smithi002.stdout:Group : System Environment/Base 2023-09-13T04:12:05.844 INFO:teuthology.orchestra.run.smithi002.stdout:Size : 9265450 2023-09-13T04:12:05.844 INFO:teuthology.orchestra.run.smithi002.stdout:License : GPLv3+ 2023-09-13T04:12:05.844 INFO:teuthology.orchestra.run.smithi002.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-09-13T04:12:05.844 INFO:teuthology.orchestra.run.smithi002.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-09-13T04:12:05.844 INFO:teuthology.orchestra.run.smithi002.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-09-13T04:12:05.845 INFO:teuthology.orchestra.run.smithi002.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-09-13T04:12:05.845 INFO:teuthology.orchestra.run.smithi002.stdout:Relocations : (not relocatable) 2023-09-13T04:12:05.845 INFO:teuthology.orchestra.run.smithi002.stdout:Packager : Red Hat, Inc. 2023-09-13T04:12:05.845 INFO:teuthology.orchestra.run.smithi002.stdout:Vendor : Red Hat, Inc. 2023-09-13T04:12:05.846 INFO:teuthology.orchestra.run.smithi002.stdout:URL : http://www.gnu.org/software/grub/ 2023-09-13T04:12:05.846 INFO:teuthology.orchestra.run.smithi002.stdout:Summary : Support tools for GRUB. 2023-09-13T04:12:05.846 INFO:teuthology.orchestra.run.smithi002.stdout:Description : 2023-09-13T04:12:05.846 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:12:05.846 INFO:teuthology.orchestra.run.smithi002.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-09-13T04:12:05.847 INFO:teuthology.orchestra.run.smithi002.stdout:customizable bootloader with modular architecture. It supports a rich 2023-09-13T04:12:05.847 INFO:teuthology.orchestra.run.smithi002.stdout:variety of kernel formats, file systems, computer architectures and 2023-09-13T04:12:05.847 INFO:teuthology.orchestra.run.smithi002.stdout:hardware devices. 2023-09-13T04:12:05.847 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:12:05.847 INFO:teuthology.orchestra.run.smithi002.stdout:This subpackage provides tools for support of all platforms. 2023-09-13T04:12:05.848 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-09-13T04:12:05.849 INFO:teuthology.task.kernel:Updating grub on smithi002 to boot 6.5.0-rc7-gfef64f93da2b 2023-09-13T04:12:05.849 DEBUG:teuthology.orchestra.run.smithi002:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-09-13T04:12:10.122 INFO:teuthology.orchestra.run.smithi002.stderr:Generating grub configuration file ... 2023-09-13T04:12:11.983 INFO:teuthology.orchestra.run.smithi002.stderr:done 2023-09-13T04:12:11.985 DEBUG:teuthology.orchestra.run.smithi002:> mktemp 2023-09-13T04:12:12.007 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/tmp.6h7cikGQ5o 2023-09-13T04:12:12.007 DEBUG:teuthology.orchestra.run.smithi002:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.6h7cikGQ5o 2023-09-13T04:12:12.079 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0666 /tmp/tmp.6h7cikGQ5o 2023-09-13T04:12:12.231 DEBUG:teuthology.orchestra.remote:smithi002:/tmp/tmp.6h7cikGQ5o is 6KB 2023-09-13T04:12:12.279 DEBUG:teuthology.orchestra.run.smithi002:> rm -fr /tmp/tmp.6h7cikGQ5o 2023-09-13T04:12:12.296 DEBUG:teuthology.orchestra.run.smithi002:> sudo /bin/ls /boot/loader/entries || true 2023-09-13T04:12:12.364 INFO:teuthology.orchestra.run.smithi002.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-09-13T04:12:12.365 INFO:teuthology.orchestra.run.smithi002.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-09-13T04:12:12.365 INFO:teuthology.orchestra.run.smithi002.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc7-gfef64f93da2b.conf 2023-09-13T04:12:12.366 DEBUG:teuthology.orchestra.run.smithi002:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc7-gfef64f93da2b 2023-09-13T04:12:12.520 DEBUG:teuthology.orchestra.run.smithi002:> sudo shutdown -r now 2023-09-13T04:12:42.522 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-09-13T04:12:42.523 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2023-09-13T04:12:42.524 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T04:13:42.527 DEBUG:teuthology.orchestra.remote:timed out 2023-09-13T04:13:51.529 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2023-09-13T04:13:51.531 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T04:13:53.041 DEBUG:teuthology.orchestra.run.smithi002:> true 2023-09-13T04:13:53.498 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com' 2023-09-13T04:13:53.498 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc7-gfef64f93da2b"... 2023-09-13T04:13:53.498 DEBUG:teuthology.orchestra.run.smithi002:> uname -r 2023-09-13T04:13:53.540 INFO:teuthology.orchestra.run.smithi002.stdout:6.5.0-rc7-gfef64f93da2b 2023-09-13T04:13:53.540 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-09-13T04:13:53.541 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-13T04:13:53.541 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-09-13T04:13:54.542 DEBUG:teuthology.parallel:result is None 2023-09-13T04:13:54.543 INFO:teuthology.run_tasks:Running task internal.base... 2023-09-13T04:13:54.551 INFO:teuthology.task.internal:Creating test directory... 2023-09-13T04:13:54.551 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-13T04:13:54.557 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-13T04:13:54.560 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-13T04:13:54.597 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-09-13T04:13:54.604 INFO:teuthology.run_tasks:Running task internal.archive... 2023-09-13T04:13:54.614 INFO:teuthology.task.internal:Creating archive directory... 2023-09-13T04:13:54.615 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-13T04:13:54.642 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-13T04:13:54.644 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-13T04:13:54.708 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-09-13T04:13:54.717 INFO:teuthology.task.internal:Enabling coredump saving... 2023-09-13T04:13:54.718 DEBUG:teuthology.orchestra.run.smithi149:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-13T04:13:54.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T04:13:54.738 DEBUG:teuthology.orchestra.run.smithi156:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-13T04:13:54.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T04:13:54.759 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-13T04:13:54.763 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-13T04:13:54.781 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-13T04:13:54.826 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-13T04:13:54.844 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-13T04:13:54.862 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-13T04:13:54.880 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-13T04:13:55.006 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-13T04:13:55.045 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-13T04:13:55.049 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-09-13T04:13:55.056 INFO:teuthology.task.internal:Configuring sudo... 2023-09-13T04:13:55.057 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-13T04:13:55.061 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-13T04:13:55.064 DEBUG:teuthology.orchestra.run.smithi156:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-13T04:13:55.127 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-09-13T04:13:55.136 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-09-13T04:13:55.137 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-13T04:13:55.141 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-13T04:13:55.143 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-13T04:13:55.180 DEBUG:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-13T04:13:55.293 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-13T04:13:55.368 DEBUG:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-13T04:13:55.446 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-13T04:13:55.524 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-09-13T04:13:55.524 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-13T04:13:55.600 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-13T04:13:55.647 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-13T04:13:55.713 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-13T04:13:55.758 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-13T04:13:55.802 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2023-09-13T04:13:55.803 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-13T04:13:55.890 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-13T04:13:55.934 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-13T04:13:55.999 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-13T04:13:56.045 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-13T04:13:56.091 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-13T04:13:56.091 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-13T04:13:56.160 DEBUG:teuthology.orchestra.run.smithi002:> sudo service rsyslog restart 2023-09-13T04:13:56.166 DEBUG:teuthology.orchestra.run.smithi149:> sudo service rsyslog restart 2023-09-13T04:13:56.168 DEBUG:teuthology.orchestra.run.smithi156:> sudo service rsyslog restart 2023-09-13T04:13:56.230 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-13T04:13:56.264 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-13T04:13:56.290 INFO:teuthology.orchestra.run.smithi002.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-13T04:13:56.751 INFO:teuthology.run_tasks:Running task internal.timer... 2023-09-13T04:13:56.759 INFO:teuthology.task.internal:Starting timer... 2023-09-13T04:13:56.759 INFO:teuthology.run_tasks:Running task pcp... 2023-09-13T04:13:56.769 INFO:teuthology.run_tasks:Running task selinux... 2023-09-13T04:13:56.778 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-09-13T04:13:56.778 DEBUG:teuthology.orchestra.run.smithi002:> sudo service auditd rotate 2023-09-13T04:13:56.914 INFO:teuthology.orchestra.run.smithi002.stdout:Rotating logs: 2023-09-13T04:13:56.918 DEBUG:teuthology.orchestra.run.smithi149:> sudo service auditd rotate 2023-09-13T04:13:56.991 INFO:teuthology.orchestra.run.smithi149.stdout:Rotating logs: 2023-09-13T04:13:56.993 DEBUG:teuthology.orchestra.run.smithi156:> sudo service auditd rotate 2023-09-13T04:13:57.051 INFO:teuthology.orchestra.run.smithi156.stdout:Rotating logs: 2023-09-13T04:13:57.053 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-09-13T04:13:57.053 DEBUG:teuthology.orchestra.run.smithi002:> /usr/sbin/getenforce 2023-09-13T04:13:57.103 INFO:teuthology.orchestra.run.smithi002.stdout:Permissive 2023-09-13T04:13:57.104 DEBUG:teuthology.orchestra.run.smithi149:> /usr/sbin/getenforce 2023-09-13T04:13:57.130 INFO:teuthology.orchestra.run.smithi149.stdout:Permissive 2023-09-13T04:13:57.131 DEBUG:teuthology.orchestra.run.smithi156:> /usr/sbin/getenforce 2023-09-13T04:13:57.151 INFO:teuthology.orchestra.run.smithi156.stdout:Permissive 2023-09-13T04:13:57.151 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi002.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi149.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi156.front.sepia.ceph.com': 'permissive'} 2023-09-13T04:13:57.151 DEBUG:teuthology.orchestra.run.smithi002:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-13T04:13:57.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T04:13:57.223 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-13T04:13:57.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T04:13:57.254 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-13T04:13:57.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T04:13:57.282 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-09-13T04:13:57.282 DEBUG:teuthology.orchestra.run.smithi002:> sudo /usr/sbin/setenforce permissive 2023-09-13T04:13:57.351 DEBUG:teuthology.orchestra.run.smithi149:> sudo /usr/sbin/setenforce permissive 2023-09-13T04:13:57.378 DEBUG:teuthology.orchestra.run.smithi156:> sudo /usr/sbin/setenforce permissive 2023-09-13T04:13:57.404 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-09-13T04:13:57.415 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-09-13T04:13:57.444 INFO:teuthology.repo_utils:Fetching main from origin 2023-09-13T04:13:57.585 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-09-13T04:13:57.628 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-09-13T04:13:57.629 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi002.front.sepia.ceph.com,smithi149.front.sepia.ceph.com,smithi156.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-09-13T04:26:29.184 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi002.front.sepia.ceph.com'), Remote(name='ubuntu@smithi149.front.sepia.ceph.com'), Remote(name='ubuntu@smithi156.front.sepia.ceph.com')] 2023-09-13T04:26:29.220 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2023-09-13T04:26:29.222 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T04:26:29.313 DEBUG:teuthology.orchestra.run.smithi002:> true 2023-09-13T04:26:29.385 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com' 2023-09-13T04:26:29.386 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2023-09-13T04:26:29.386 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T04:26:29.454 DEBUG:teuthology.orchestra.run.smithi149:> true 2023-09-13T04:26:29.480 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2023-09-13T04:26:29.481 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-09-13T04:26:29.481 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T04:26:29.548 DEBUG:teuthology.orchestra.run.smithi156:> true 2023-09-13T04:26:29.617 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-09-13T04:26:29.618 INFO:teuthology.run_tasks:Running task clock... 2023-09-13T04:26:29.629 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-09-13T04:26:29.629 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-13T04:26:29.630 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-13T04:26:29.635 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-13T04:26:29.635 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-13T04:26:29.637 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-13T04:26:29.638 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-13T04:26:29.668 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-13T04:26:29.683 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-13T04:26:29.693 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-13T04:26:29.705 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-13T04:26:29.708 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-13T04:26:29.740 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-13T04:26:29.755 INFO:teuthology.orchestra.run.smithi149.stderr:sudo: ntpd: command not found 2023-09-13T04:26:29.767 INFO:teuthology.orchestra.run.smithi149.stdout:506 Cannot talk to daemon 2023-09-13T04:26:29.777 INFO:teuthology.orchestra.run.smithi156.stderr:sudo: ntpd: command not found 2023-09-13T04:26:29.782 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-13T04:26:29.789 INFO:teuthology.orchestra.run.smithi156.stdout:506 Cannot talk to daemon 2023-09-13T04:26:29.796 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-13T04:26:29.804 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-13T04:26:29.818 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-13T04:26:29.839 INFO:teuthology.orchestra.run.smithi002.stderr:sudo: ntpd: command not found 2023-09-13T04:26:29.864 INFO:teuthology.orchestra.run.smithi149.stderr:bash: ntpq: command not found 2023-09-13T04:26:29.867 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-13T04:26:29.867 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2023-09-13T04:26:29.868 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T04:26:29.868 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T04:26:29.868 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T04:26:29.868 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T04:26:29.869 INFO:teuthology.orchestra.run.smithi002.stdout:506 Cannot talk to daemon 2023-09-13T04:26:29.893 INFO:teuthology.orchestra.run.smithi156.stderr:bash: ntpq: command not found 2023-09-13T04:26:29.897 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-13T04:26:29.897 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2023-09-13T04:26:29.897 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T04:26:29.898 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T04:26:29.898 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T04:26:29.898 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T04:26:29.904 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-13T04:26:29.939 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-13T04:26:30.040 INFO:teuthology.orchestra.run.smithi002.stderr:bash: ntpq: command not found 2023-09-13T04:26:30.047 INFO:teuthology.orchestra.run.smithi002.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-13T04:26:30.047 INFO:teuthology.orchestra.run.smithi002.stdout:=============================================================================== 2023-09-13T04:26:30.047 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T04:26:30.047 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T04:26:30.048 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T04:26:30.048 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T04:26:30.049 INFO:teuthology.run_tasks:Running task pexec... 2023-09-13T04:26:30.059 INFO:teuthology.task.pexec:Executing custom commands... 2023-09-13T04:26:30.059 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi002.front.sepia.ceph.com 2023-09-13T04:26:30.060 DEBUG:teuthology.orchestra.run.smithi002:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-13T04:26:30.060 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi149.front.sepia.ceph.com 2023-09-13T04:26:30.060 DEBUG:teuthology.orchestra.run.smithi149:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-13T04:26:30.061 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi156.front.sepia.ceph.com 2023-09-13T04:26:30.061 DEBUG:teuthology.orchestra.run.smithi156:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-13T04:26:30.435 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-09-13T04:26:30.436 INFO:teuthology.orchestra.run.smithi149.stdout:Updating Subscription Management repositories. 2023-09-13T04:26:30.579 INFO:teuthology.orchestra.run.smithi002.stdout:Updating Subscription Management repositories. 2023-09-13T04:26:31.249 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:04:52 ago on Wed 13 Sep 2023 04:21:38 AM UTC. 2023-09-13T04:26:31.279 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:04:51 ago on Wed 13 Sep 2023 04:21:39 AM UTC. 2023-09-13T04:26:31.560 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:51 ago on Wed 13 Sep 2023 04:24:40 AM UTC. 2023-09-13T04:26:32.997 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2023-09-13T04:26:32.998 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2023-09-13T04:26:32.999 INFO:teuthology.orchestra.run.smithi149.stdout: Package Architecture Version Repository Size 2023-09-13T04:26:32.999 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2023-09-13T04:26:32.999 INFO:teuthology.orchestra.run.smithi149.stdout:Resetting modules: 2023-09-13T04:26:32.999 INFO:teuthology.orchestra.run.smithi149.stdout: container-tools 2023-09-13T04:26:33.000 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-09-13T04:26:33.000 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2023-09-13T04:26:33.000 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2023-09-13T04:26:33.000 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-09-13T04:26:33.129 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2023-09-13T04:26:33.130 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2023-09-13T04:26:33.131 INFO:teuthology.orchestra.run.smithi156.stdout: Package Architecture Version Repository Size 2023-09-13T04:26:33.131 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2023-09-13T04:26:33.131 INFO:teuthology.orchestra.run.smithi156.stdout:Resetting modules: 2023-09-13T04:26:33.131 INFO:teuthology.orchestra.run.smithi156.stdout: container-tools 2023-09-13T04:26:33.131 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-13T04:26:33.132 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2023-09-13T04:26:33.132 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2023-09-13T04:26:33.132 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-13T04:26:33.644 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2023-09-13T04:26:33.645 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2023-09-13T04:26:33.645 INFO:teuthology.orchestra.run.smithi002.stdout: Package Architecture Version Repository Size 2023-09-13T04:26:33.646 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2023-09-13T04:26:33.646 INFO:teuthology.orchestra.run.smithi002.stdout:Resetting modules: 2023-09-13T04:26:33.646 INFO:teuthology.orchestra.run.smithi002.stdout: container-tools 2023-09-13T04:26:33.646 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:26:33.646 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2023-09-13T04:26:33.647 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2023-09-13T04:26:33.647 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:26:33.989 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:04:54 ago on Wed 13 Sep 2023 04:21:39 AM UTC. 2023-09-13T04:26:34.104 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:04:55 ago on Wed 13 Sep 2023 04:21:38 AM UTC. 2023-09-13T04:26:34.911 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:54 ago on Wed 13 Sep 2023 04:24:40 AM UTC. 2023-09-13T04:26:36.670 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2023-09-13T04:26:36.861 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2023-09-13T04:26:37.158 INFO:teuthology.orchestra.run.smithi149.stdout:Updating Subscription Management repositories. 2023-09-13T04:26:37.363 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-09-13T04:26:37.405 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2023-09-13T04:26:37.965 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:04:58 ago on Wed 13 Sep 2023 04:21:39 AM UTC. 2023-09-13T04:26:38.062 INFO:teuthology.orchestra.run.smithi002.stdout:Updating Subscription Management repositories. 2023-09-13T04:26:38.127 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:04:59 ago on Wed 13 Sep 2023 04:21:38 AM UTC. 2023-09-13T04:26:39.019 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:58 ago on Wed 13 Sep 2023 04:24:40 AM UTC. 2023-09-13T04:26:39.689 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2023-09-13T04:26:39.787 INFO:teuthology.orchestra.run.smithi149.stdout:============================================================================================================ 2023-09-13T04:26:39.787 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2023-09-13T04:26:39.787 INFO:teuthology.orchestra.run.smithi149.stdout:============================================================================================================ 2023-09-13T04:26:39.787 INFO:teuthology.orchestra.run.smithi149.stdout:Installing group/module packages: 2023-09-13T04:26:39.788 INFO:teuthology.orchestra.run.smithi149.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-13T04:26:39.788 INFO:teuthology.orchestra.run.smithi149.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-13T04:26:39.788 INFO:teuthology.orchestra.run.smithi149.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-13T04:26:39.788 INFO:teuthology.orchestra.run.smithi149.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-13T04:26:39.788 INFO:teuthology.orchestra.run.smithi149.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-13T04:26:39.789 INFO:teuthology.orchestra.run.smithi149.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-13T04:26:39.789 INFO:teuthology.orchestra.run.smithi149.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-13T04:26:39.789 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2023-09-13T04:26:39.789 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-13T04:26:39.789 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-13T04:26:39.790 INFO:teuthology.orchestra.run.smithi149.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-13T04:26:39.790 INFO:teuthology.orchestra.run.smithi149.stdout:Installing module profiles: 2023-09-13T04:26:39.790 INFO:teuthology.orchestra.run.smithi149.stdout: container-tools/common 2023-09-13T04:26:39.790 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-09-13T04:26:39.790 INFO:teuthology.orchestra.run.smithi149.stdout:Enabling module streams: 2023-09-13T04:26:39.791 INFO:teuthology.orchestra.run.smithi149.stdout: container-tools rhel8 2023-09-13T04:26:39.791 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-09-13T04:26:39.791 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2023-09-13T04:26:39.791 INFO:teuthology.orchestra.run.smithi149.stdout:============================================================================================================ 2023-09-13T04:26:39.791 INFO:teuthology.orchestra.run.smithi149.stdout:Install 10 Packages 2023-09-13T04:26:39.792 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-09-13T04:26:39.792 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 18 M 2023-09-13T04:26:39.793 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 65 M 2023-09-13T04:26:39.793 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2023-09-13T04:26:39.864 INFO:teuthology.orchestra.run.smithi149.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 332 kB/s | 25 kB 00:00 2023-09-13T04:26:39.874 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2023-09-13T04:26:39.889 INFO:teuthology.orchestra.run.smithi149.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 408 kB/s | 41 kB 00:00 2023-09-13T04:26:39.938 INFO:teuthology.orchestra.run.smithi149.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 632 kB/s | 94 kB 00:00 2023-09-13T04:26:39.965 INFO:teuthology.orchestra.run.smithi149.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 492 kB/s | 49 kB 00:00 2023-09-13T04:26:39.967 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================================ 2023-09-13T04:26:39.968 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2023-09-13T04:26:39.968 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================================ 2023-09-13T04:26:39.968 INFO:teuthology.orchestra.run.smithi156.stdout:Installing group/module packages: 2023-09-13T04:26:39.968 INFO:teuthology.orchestra.run.smithi156.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-13T04:26:39.968 INFO:teuthology.orchestra.run.smithi156.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-13T04:26:39.969 INFO:teuthology.orchestra.run.smithi156.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-13T04:26:39.969 INFO:teuthology.orchestra.run.smithi156.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-13T04:26:39.969 INFO:teuthology.orchestra.run.smithi156.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-13T04:26:39.969 INFO:teuthology.orchestra.run.smithi156.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-13T04:26:39.970 INFO:teuthology.orchestra.run.smithi156.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-13T04:26:39.970 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2023-09-13T04:26:39.970 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-13T04:26:39.970 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-13T04:26:39.970 INFO:teuthology.orchestra.run.smithi156.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-13T04:26:39.971 INFO:teuthology.orchestra.run.smithi156.stdout:Installing module profiles: 2023-09-13T04:26:39.971 INFO:teuthology.orchestra.run.smithi156.stdout: container-tools/common 2023-09-13T04:26:39.971 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-13T04:26:39.971 INFO:teuthology.orchestra.run.smithi156.stdout:Enabling module streams: 2023-09-13T04:26:39.971 INFO:teuthology.orchestra.run.smithi156.stdout: container-tools rhel8 2023-09-13T04:26:39.972 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-13T04:26:39.972 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2023-09-13T04:26:39.972 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================================ 2023-09-13T04:26:39.972 INFO:teuthology.orchestra.run.smithi156.stdout:Install 10 Packages 2023-09-13T04:26:39.972 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-13T04:26:39.973 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 18 M 2023-09-13T04:26:39.973 INFO:teuthology.orchestra.run.smithi156.stdout:Installed size: 65 M 2023-09-13T04:26:39.974 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2023-09-13T04:26:39.998 INFO:teuthology.orchestra.run.smithi149.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 4.7 MB/s | 524 kB 00:00 2023-09-13T04:26:40.024 INFO:teuthology.orchestra.run.smithi149.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-09-13T04:26:40.037 INFO:teuthology.orchestra.run.smithi156.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.4 MB/s | 94 kB 00:00 2023-09-13T04:26:40.040 INFO:teuthology.orchestra.run.smithi149.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.9 MB/s | 213 kB 00:00 2023-09-13T04:26:40.062 INFO:teuthology.orchestra.run.smithi156.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 269 kB/s | 25 kB 00:00 2023-09-13T04:26:40.088 INFO:teuthology.orchestra.run.smithi156.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 347 kB/s | 41 kB 00:00 2023-09-13T04:26:40.113 INFO:teuthology.orchestra.run.smithi156.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 653 kB/s | 49 kB 00:00 2023-09-13T04:26:40.147 INFO:teuthology.orchestra.run.smithi156.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-09-13T04:26:40.158 INFO:teuthology.orchestra.run.smithi149.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 19 MB/s | 2.2 MB 00:00 2023-09-13T04:26:40.172 INFO:teuthology.orchestra.run.smithi156.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-09-13T04:26:40.197 INFO:teuthology.orchestra.run.smithi156.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-09-13T04:26:40.275 INFO:teuthology.orchestra.run.smithi149.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 28 MB/s | 7.0 MB 00:00 2023-09-13T04:26:40.289 INFO:teuthology.orchestra.run.smithi156.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 24 MB/s | 2.2 MB 00:00 2023-09-13T04:26:40.375 INFO:teuthology.orchestra.run.smithi149.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 18 MB/s | 8.0 MB 00:00 2023-09-13T04:26:40.376 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2023-09-13T04:26:40.376 INFO:teuthology.orchestra.run.smithi149.stdout:Total 31 MB/s | 18 MB 00:00 2023-09-13T04:26:40.406 INFO:teuthology.orchestra.run.smithi156.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 30 MB/s | 7.0 MB 00:00 2023-09-13T04:26:40.498 INFO:teuthology.orchestra.run.smithi156.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 19 MB/s | 8.0 MB 00:00 2023-09-13T04:26:40.499 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2023-09-13T04:26:40.499 INFO:teuthology.orchestra.run.smithi156.stdout:Total 34 MB/s | 18 MB 00:00 2023-09-13T04:26:40.574 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2023-09-13T04:26:40.593 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2023-09-13T04:26:40.593 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2023-09-13T04:26:40.700 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2023-09-13T04:26:40.718 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2023-09-13T04:26:40.719 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2023-09-13T04:26:40.776 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2023-09-13T04:26:40.780 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2023-09-13T04:26:40.899 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2023-09-13T04:26:40.903 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2023-09-13T04:26:40.968 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2023-09-13T04:26:41.073 INFO:teuthology.orchestra.run.smithi002.stdout:============================================================================================================ 2023-09-13T04:26:41.073 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2023-09-13T04:26:41.073 INFO:teuthology.orchestra.run.smithi002.stdout:============================================================================================================ 2023-09-13T04:26:41.073 INFO:teuthology.orchestra.run.smithi002.stdout:Installing group/module packages: 2023-09-13T04:26:41.074 INFO:teuthology.orchestra.run.smithi002.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-13T04:26:41.074 INFO:teuthology.orchestra.run.smithi002.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-13T04:26:41.074 INFO:teuthology.orchestra.run.smithi002.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-13T04:26:41.074 INFO:teuthology.orchestra.run.smithi002.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-13T04:26:41.075 INFO:teuthology.orchestra.run.smithi002.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-13T04:26:41.075 INFO:teuthology.orchestra.run.smithi002.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-13T04:26:41.075 INFO:teuthology.orchestra.run.smithi002.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-13T04:26:41.075 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2023-09-13T04:26:41.075 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-13T04:26:41.076 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-13T04:26:41.076 INFO:teuthology.orchestra.run.smithi002.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-13T04:26:41.076 INFO:teuthology.orchestra.run.smithi002.stdout:Installing module profiles: 2023-09-13T04:26:41.076 INFO:teuthology.orchestra.run.smithi002.stdout: container-tools/common 2023-09-13T04:26:41.076 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:26:41.077 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling module streams: 2023-09-13T04:26:41.077 INFO:teuthology.orchestra.run.smithi002.stdout: container-tools rhel8 2023-09-13T04:26:41.077 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:26:41.077 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2023-09-13T04:26:41.078 INFO:teuthology.orchestra.run.smithi002.stdout:============================================================================================================ 2023-09-13T04:26:41.078 INFO:teuthology.orchestra.run.smithi002.stdout:Install 10 Packages 2023-09-13T04:26:41.078 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:26:41.079 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 18 M 2023-09-13T04:26:41.079 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 65 M 2023-09-13T04:26:41.079 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2023-09-13T04:26:41.146 INFO:teuthology.orchestra.run.smithi002.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 356 kB/s | 25 kB 00:00 2023-09-13T04:26:41.171 INFO:teuthology.orchestra.run.smithi002.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 428 kB/s | 41 kB 00:00 2023-09-13T04:26:41.188 INFO:teuthology.orchestra.run.smithi002.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 838 kB/s | 94 kB 00:00 2023-09-13T04:26:41.205 INFO:teuthology.orchestra.run.smithi002.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 839 kB/s | 49 kB 00:00 2023-09-13T04:26:41.247 INFO:teuthology.orchestra.run.smithi002.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.8 MB/s | 524 kB 00:00 2023-09-13T04:26:41.272 INFO:teuthology.orchestra.run.smithi002.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.2 MB/s | 149 kB 00:00 2023-09-13T04:26:41.306 INFO:teuthology.orchestra.run.smithi002.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.6 MB/s | 213 kB 00:00 2023-09-13T04:26:41.321 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2023-09-13T04:26:41.402 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2023-09-13T04:26:41.447 INFO:teuthology.orchestra.run.smithi002.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 16 MB/s | 2.2 MB 00:00 2023-09-13T04:26:41.497 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-13T04:26:41.608 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-13T04:26:41.623 INFO:teuthology.orchestra.run.smithi002.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 20 MB/s | 7.0 MB 00:00 2023-09-13T04:26:41.674 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-13T04:26:41.715 INFO:teuthology.orchestra.run.smithi002.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 15 MB/s | 8.0 MB 00:00 2023-09-13T04:26:41.716 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2023-09-13T04:26:41.716 INFO:teuthology.orchestra.run.smithi002.stdout:Total 28 MB/s | 18 MB 00:00 2023-09-13T04:26:41.788 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-13T04:26:41.885 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-13T04:26:41.974 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-13T04:26:42.056 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2023-09-13T04:26:42.080 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2023-09-13T04:26:42.081 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2023-09-13T04:26:42.111 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-13T04:26:42.185 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-13T04:26:42.494 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-13T04:26:42.534 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-13T04:26:42.577 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2023-09-13T04:26:42.581 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2023-09-13T04:26:43.188 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2023-09-13T04:26:43.298 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-13T04:26:43.320 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-13T04:26:43.383 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-13T04:26:43.587 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-13T04:26:43.826 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-13T04:26:44.068 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-13T04:26:44.163 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-13T04:26:44.228 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-13T04:26:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-13T04:26:44.442 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-13T04:26:44.449 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-13T04:26:44.589 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-13T04:26:44.654 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-13T04:26:44.767 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-13T04:26:44.831 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-13T04:26:45.035 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-13T04:26:45.035 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-13T04:26:45.035 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-13T04:26:45.035 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-13T04:26:45.036 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-13T04:26:45.036 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-13T04:26:45.036 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-13T04:26:45.036 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-13T04:26:45.036 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-13T04:26:45.037 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-13T04:26:45.099 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-13T04:26:45.099 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-13T04:26:45.099 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-13T04:26:45.100 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-13T04:26:45.100 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-13T04:26:45.100 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-13T04:26:45.100 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-13T04:26:45.100 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-13T04:26:45.101 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-13T04:26:45.101 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-13T04:26:45.300 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-13T04:26:46.274 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-13T04:26:46.275 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:05:07 ago on Wed 13 Sep 2023 04:21:38 AM UTC. 2023-09-13T04:26:46.279 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-13T04:26:46.279 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:05:06 ago on Wed 13 Sep 2023 04:21:39 AM UTC. 2023-09-13T04:26:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-13T04:26:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-13T04:26:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-13T04:26:47.024 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-13T04:26:47.453 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-13T04:26:47.453 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-13T04:26:47.454 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-13T04:26:47.454 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-13T04:26:47.454 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-13T04:26:47.454 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-13T04:26:47.455 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-13T04:26:47.455 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-13T04:26:47.455 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-13T04:26:47.455 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-13T04:26:48.946 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-13T04:26:48.947 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:02:08 ago on Wed 13 Sep 2023 04:24:40 AM UTC. 2023-09-13T04:26:49.668 INFO:teuthology.orchestra.run.smithi156.stdout:Installed products updated. 2023-09-13T04:26:49.704 INFO:teuthology.orchestra.run.smithi149.stdout:Installed products updated. 2023-09-13T04:26:49.892 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-13T04:26:49.892 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2023-09-13T04:26:49.892 INFO:teuthology.orchestra.run.smithi156.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-13T04:26:49.892 INFO:teuthology.orchestra.run.smithi156.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-13T04:26:49.893 INFO:teuthology.orchestra.run.smithi156.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-13T04:26:49.893 INFO:teuthology.orchestra.run.smithi156.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-13T04:26:49.893 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-13T04:26:49.893 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-13T04:26:49.893 INFO:teuthology.orchestra.run.smithi156.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-13T04:26:49.894 INFO:teuthology.orchestra.run.smithi156.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-13T04:26:49.894 INFO:teuthology.orchestra.run.smithi156.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-13T04:26:49.894 INFO:teuthology.orchestra.run.smithi156.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-13T04:26:49.894 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-13T04:26:49.894 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2023-09-13T04:26:49.926 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-09-13T04:26:49.926 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2023-09-13T04:26:49.927 INFO:teuthology.orchestra.run.smithi149.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-13T04:26:49.927 INFO:teuthology.orchestra.run.smithi149.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-13T04:26:49.927 INFO:teuthology.orchestra.run.smithi149.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-13T04:26:49.927 INFO:teuthology.orchestra.run.smithi149.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-13T04:26:49.927 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-13T04:26:49.928 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-13T04:26:49.928 INFO:teuthology.orchestra.run.smithi149.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-13T04:26:49.928 INFO:teuthology.orchestra.run.smithi149.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-13T04:26:49.928 INFO:teuthology.orchestra.run.smithi149.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-13T04:26:49.928 INFO:teuthology.orchestra.run.smithi149.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-13T04:26:49.929 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-09-13T04:26:49.929 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2023-09-13T04:26:50.072 DEBUG:teuthology.parallel:result is None 2023-09-13T04:26:50.120 DEBUG:teuthology.parallel:result is None 2023-09-13T04:26:51.783 INFO:teuthology.orchestra.run.smithi002.stdout:Installed products updated. 2023-09-13T04:26:52.012 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:26:52.012 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2023-09-13T04:26:52.013 INFO:teuthology.orchestra.run.smithi002.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-13T04:26:52.013 INFO:teuthology.orchestra.run.smithi002.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-13T04:26:52.013 INFO:teuthology.orchestra.run.smithi002.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-13T04:26:52.013 INFO:teuthology.orchestra.run.smithi002.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-13T04:26:52.013 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-13T04:26:52.014 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-13T04:26:52.014 INFO:teuthology.orchestra.run.smithi002.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-13T04:26:52.014 INFO:teuthology.orchestra.run.smithi002.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-13T04:26:52.014 INFO:teuthology.orchestra.run.smithi002.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-13T04:26:52.015 INFO:teuthology.orchestra.run.smithi002.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-13T04:26:52.015 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:26:52.015 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2023-09-13T04:26:52.229 DEBUG:teuthology.parallel:result is None 2023-09-13T04:26:52.230 INFO:teuthology.run_tasks:Running task install... 2023-09-13T04:26:52.242 DEBUG:teuthology.task.install:project ceph 2023-09-13T04:26:52.243 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5'}} 2023-09-13T04:26:52.243 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5'} 2023-09-13T04:26:52.243 INFO:teuthology.task.install:Using flavor: default 2023-09-13T04:26:52.250 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-09-13T04:26:52.250 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-09-13T04:26:52.251 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5', 'tag': None, 'wait_for_package': False} 2023-09-13T04:26:52.251 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 2023-09-13T04:26:52.254 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5', 'tag': None, 'wait_for_package': False} 2023-09-13T04:26:52.254 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 2023-09-13T04:26:52.255 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5', 'tag': None, 'wait_for_package': False} 2023-09-13T04:26:52.256 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 2023-09-13T04:26:52.393 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023sep12-b2/4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/centos/8/flavors/default/ 2023-09-13T04:26:52.394 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6083.g4a754ac9 2023-09-13T04:26:52.476 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023sep12-b2/4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/centos/8/flavors/default/ 2023-09-13T04:26:52.476 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6083.g4a754ac9 2023-09-13T04:26:52.478 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023sep12-b2/4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/centos/8/flavors/default/ 2023-09-13T04:26:52.478 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6083.g4a754ac9 2023-09-13T04:26:52.715 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023sep12-b2/4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023sep12-b2/4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023sep12-b2/4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-13T04:26:52.715 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-09-13T04:26:52.715 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-13T04:26:52.741 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023sep12-b2/4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023sep12-b2/4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023sep12-b2/4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-13T04:26:52.742 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-13T04:26:52.742 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-13T04:26:52.748 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023sep12-b2/4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023sep12-b2/4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023sep12-b2/4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-13T04:26:52.748 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2023-09-13T04:26:52.748 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-13T04:26:52.769 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-13T04:26:52.769 DEBUG:teuthology.orchestra.run.smithi156:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-13T04:26:52.778 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-13T04:26:52.778 DEBUG:teuthology.orchestra.run.smithi149:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-13T04:26:52.786 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-13T04:26:52.787 DEBUG:teuthology.orchestra.run.smithi002:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-13T04:26:52.837 DEBUG:teuthology.orchestra.run.smithi156:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-13T04:26:52.844 DEBUG:teuthology.orchestra.run.smithi149:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-13T04:26:52.898 DEBUG:teuthology.orchestra.run.smithi002:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-13T04:26:52.915 DEBUG:teuthology.orchestra.run.smithi156:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-13T04:26:52.922 DEBUG:teuthology.orchestra.run.smithi149:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-13T04:26:52.983 INFO:teuthology.orchestra.run.smithi156.stdout:check_obsoletes = 1 2023-09-13T04:26:52.984 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum clean all 2023-09-13T04:26:52.988 INFO:teuthology.orchestra.run.smithi149.stdout:check_obsoletes = 1 2023-09-13T04:26:52.989 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum clean all 2023-09-13T04:26:52.995 DEBUG:teuthology.orchestra.run.smithi002:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-13T04:26:53.064 INFO:teuthology.orchestra.run.smithi002.stdout:check_obsoletes = 1 2023-09-13T04:26:53.068 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum clean all 2023-09-13T04:26:53.376 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-09-13T04:26:53.379 INFO:teuthology.orchestra.run.smithi149.stdout:Updating Subscription Management repositories. 2023-09-13T04:26:53.551 INFO:teuthology.orchestra.run.smithi002.stdout:Updating Subscription Management repositories. 2023-09-13T04:26:53.681 INFO:teuthology.orchestra.run.smithi156.stdout:44 files removed 2023-09-13T04:26:53.696 INFO:teuthology.orchestra.run.smithi149.stdout:44 files removed 2023-09-13T04:26:53.735 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-13T04:26:53.754 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-13T04:26:53.956 INFO:teuthology.orchestra.run.smithi002.stdout:44 files removed 2023-09-13T04:26:54.012 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-13T04:26:54.085 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-09-13T04:26:54.105 INFO:teuthology.orchestra.run.smithi149.stdout:Updating Subscription Management repositories. 2023-09-13T04:26:54.489 INFO:teuthology.orchestra.run.smithi002.stdout:Updating Subscription Management repositories. 2023-09-13T04:26:55.298 INFO:teuthology.orchestra.run.smithi156.stdout:ceph packages for x86_64 99 kB/s | 81 kB 00:00 2023-09-13T04:26:55.367 INFO:teuthology.orchestra.run.smithi149.stdout:ceph packages for x86_64 93 kB/s | 81 kB 00:00 2023-09-13T04:26:55.909 INFO:teuthology.orchestra.run.smithi002.stdout:ceph packages for x86_64 86 kB/s | 81 kB 00:00 2023-09-13T04:26:56.026 INFO:teuthology.orchestra.run.smithi156.stdout:ceph noarch packages 14 kB/s | 9.4 kB 00:00 2023-09-13T04:26:56.160 INFO:teuthology.orchestra.run.smithi149.stdout:ceph noarch packages 12 kB/s | 9.4 kB 00:00 2023-09-13T04:26:56.626 INFO:teuthology.orchestra.run.smithi002.stdout:ceph noarch packages 14 kB/s | 9.4 kB 00:00 2023-09-13T04:26:56.828 INFO:teuthology.orchestra.run.smithi156.stdout:ceph source packages 1.9 kB/s | 1.5 kB 00:00 2023-09-13T04:26:56.879 INFO:teuthology.orchestra.run.smithi149.stdout:ceph source packages 2.1 kB/s | 1.5 kB 00:00 2023-09-13T04:26:57.011 INFO:teuthology.orchestra.run.smithi156.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2023-09-13T04:26:57.012 INFO:teuthology.orchestra.run.smithi149.stdout:Copr repo for python3-asyncssh owned by ceph 31 kB/s | 3.5 kB 00:00 2023-09-13T04:26:57.419 INFO:teuthology.orchestra.run.smithi002.stdout:ceph source packages 1.9 kB/s | 1.5 kB 00:00 2023-09-13T04:26:57.586 INFO:teuthology.orchestra.run.smithi002.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2023-09-13T04:26:57.933 INFO:teuthology.orchestra.run.smithi156.stdout:Extra Packages for Enterprise Linux 18 MB/s | 16 MB 00:00 2023-09-13T04:26:58.175 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 14 MB/s | 16 MB 00:01 2023-09-13T04:27:00.370 INFO:teuthology.orchestra.run.smithi002.stdout:Extra Packages for Enterprise Linux 5.7 MB/s | 16 MB 00:02 2023-09-13T04:27:01.512 INFO:teuthology.orchestra.run.smithi156.stdout:lab-extras 612 kB/s | 24 kB 00:00 2023-09-13T04:27:01.788 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 408 kB/s | 24 kB 00:00 2023-09-13T04:27:02.380 INFO:teuthology.orchestra.run.smithi156.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 59 MB/s | 47 MB 00:00 2023-09-13T04:27:02.722 INFO:teuthology.orchestra.run.smithi149.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 61 MB/s | 53 MB 00:00 2023-09-13T04:27:04.636 INFO:teuthology.orchestra.run.smithi002.stdout:lab-extras 464 kB/s | 24 kB 00:00 2023-09-13T04:27:05.905 INFO:teuthology.orchestra.run.smithi002.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 45 MB/s | 53 MB 00:01 2023-09-13T04:27:10.273 INFO:teuthology.orchestra.run.smithi156.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 65 MB/s | 53 MB 00:00 2023-09-13T04:27:10.715 INFO:teuthology.orchestra.run.smithi149.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 61 MB/s | 47 MB 00:00 2023-09-13T04:27:15.140 INFO:teuthology.orchestra.run.smithi002.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 47 MB 00:01 2023-09-13T04:27:21.238 INFO:teuthology.orchestra.run.smithi156.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-13T04:27:21.239 INFO:teuthology.orchestra.run.smithi156.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-13T04:27:21.254 INFO:teuthology.orchestra.run.smithi156.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-13T04:27:21.255 INFO:teuthology.orchestra.run.smithi156.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-13T04:27:21.256 INFO:teuthology.orchestra.run.smithi156.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-13T04:27:21.257 INFO:teuthology.orchestra.run.smithi156.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-13T04:27:21.260 INFO:teuthology.orchestra.run.smithi156.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-13T04:27:21.263 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-13T04:27:21.264 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-13T04:27:21.395 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2023-09-13T04:27:21.407 INFO:teuthology.orchestra.run.smithi156.stdout:=========================================================================================================================== 2023-09-13T04:27:21.407 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2023-09-13T04:27:21.407 INFO:teuthology.orchestra.run.smithi156.stdout:=========================================================================================================================== 2023-09-13T04:27:21.408 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2023-09-13T04:27:21.408 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-13T04:27:21.408 INFO:teuthology.orchestra.run.smithi156.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-13T04:27:21.408 INFO:teuthology.orchestra.run.smithi156.stdout: ceph x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 6.4 k 2023-09-13T04:27:21.408 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 5.4 M 2023-09-13T04:27:21.409 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 919 k 2023-09-13T04:27:21.409 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 167 k 2023-09-13T04:27:21.409 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 1.6 M 2023-09-13T04:27:21.409 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 132 k 2023-09-13T04:27:21.409 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 1.7 M 2023-09-13T04:27:21.410 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 7.4 M 2023-09-13T04:27:21.410 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 47 k 2023-09-13T04:27:21.410 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 14 M 2023-09-13T04:27:21.410 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 49 M 2023-09-13T04:27:21.410 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 261 k 2023-09-13T04:27:21.411 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 179 k 2023-09-13T04:27:21.411 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 260 k 2023-09-13T04:27:21.411 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 18 k 2023-09-13T04:27:21.411 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-13T04:27:21.411 INFO:teuthology.orchestra.run.smithi156.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-13T04:27:21.412 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-13T04:27:21.412 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 31 k 2023-09-13T04:27:21.412 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2 x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 839 k 2023-09-13T04:27:21.412 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 139 k 2023-09-13T04:27:21.412 INFO:teuthology.orchestra.run.smithi156.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-13T04:27:21.413 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-13T04:27:21.413 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-13T04:27:21.413 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-13T04:27:21.413 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-13T04:27:21.413 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-13T04:27:21.414 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 197 k 2023-09-13T04:27:21.414 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 370 k 2023-09-13T04:27:21.414 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 365 k 2023-09-13T04:27:21.414 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 107 k 2023-09-13T04:27:21.414 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 93 k 2023-09-13T04:27:21.415 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 3.6 M 2023-09-13T04:27:21.415 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 179 k 2023-09-13T04:27:21.415 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-13T04:27:21.415 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrading: 2023-09-13T04:27:21.415 INFO:teuthology.orchestra.run.smithi156.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-13T04:27:21.416 INFO:teuthology.orchestra.run.smithi156.stdout: librados2 x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 3.7 M 2023-09-13T04:27:21.416 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1 x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 3.7 M 2023-09-13T04:27:21.416 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2023-09-13T04:27:21.416 INFO:teuthology.orchestra.run.smithi156.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-13T04:27:21.416 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 22 M 2023-09-13T04:27:21.417 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 23 k 2023-09-13T04:27:21.417 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 2.3 M 2023-09-13T04:27:21.417 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 242 k 2023-09-13T04:27:21.417 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 4.6 M 2023-09-13T04:27:21.417 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 18 M 2023-09-13T04:27:21.417 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 15 k 2023-09-13T04:27:21.418 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 24 k 2023-09-13T04:27:21.418 INFO:teuthology.orchestra.run.smithi156.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-13T04:27:21.418 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-13T04:27:21.418 INFO:teuthology.orchestra.run.smithi156.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-13T04:27:21.418 INFO:teuthology.orchestra.run.smithi156.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-13T04:27:21.419 INFO:teuthology.orchestra.run.smithi156.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-13T04:27:21.419 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 183 k 2023-09-13T04:27:21.419 INFO:teuthology.orchestra.run.smithi156.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-13T04:27:21.419 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-13T04:27:21.419 INFO:teuthology.orchestra.run.smithi156.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-13T04:27:21.420 INFO:teuthology.orchestra.run.smithi156.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-13T04:27:21.420 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-13T04:27:21.420 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-13T04:27:21.420 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1 x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 518 k 2023-09-13T04:27:21.420 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-13T04:27:21.420 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2 x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 6.8 M 2023-09-13T04:27:21.421 INFO:teuthology.orchestra.run.smithi156.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-13T04:27:21.421 INFO:teuthology.orchestra.run.smithi156.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-13T04:27:21.421 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-13T04:27:21.421 INFO:teuthology.orchestra.run.smithi156.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-13T04:27:21.421 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-13T04:27:21.422 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-13T04:27:21.422 INFO:teuthology.orchestra.run.smithi156.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-13T04:27:21.422 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-13T04:27:21.422 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-13T04:27:21.423 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-13T04:27:21.423 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-13T04:27:21.423 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-13T04:27:21.423 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-13T04:27:21.423 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-13T04:27:21.424 INFO:teuthology.orchestra.run.smithi156.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-13T04:27:21.424 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-13T04:27:21.424 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 45 k 2023-09-13T04:27:21.424 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 125 k 2023-09-13T04:27:21.424 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-13T04:27:21.424 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-13T04:27:21.425 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-13T04:27:21.425 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-13T04:27:21.425 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-13T04:27:21.425 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-13T04:27:21.425 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-13T04:27:21.426 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-13T04:27:21.426 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-13T04:27:21.426 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-13T04:27:21.426 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-13T04:27:21.426 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-13T04:27:21.427 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-13T04:27:21.427 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-13T04:27:21.427 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-13T04:27:21.427 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-13T04:27:21.427 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-13T04:27:21.427 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-13T04:27:21.428 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-13T04:27:21.428 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-13T04:27:21.428 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-13T04:27:21.428 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-13T04:27:21.428 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-13T04:27:21.429 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-13T04:27:21.429 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-13T04:27:21.429 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-13T04:27:21.429 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-13T04:27:21.429 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-13T04:27:21.430 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-13T04:27:21.430 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-13T04:27:21.430 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-13T04:27:21.430 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-13T04:27:21.430 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-13T04:27:21.431 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-13T04:27:21.431 INFO:teuthology.orchestra.run.smithi156.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-13T04:27:21.431 INFO:teuthology.orchestra.run.smithi156.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-13T04:27:21.431 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-13T04:27:21.431 INFO:teuthology.orchestra.run.smithi156.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-13T04:27:21.432 INFO:teuthology.orchestra.run.smithi156.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-13T04:27:21.432 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-13T04:27:21.432 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-13T04:27:21.432 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-13T04:27:21.432 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-13T04:27:21.433 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-13T04:27:21.433 INFO:teuthology.orchestra.run.smithi156.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-13T04:27:21.433 INFO:teuthology.orchestra.run.smithi156.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-13T04:27:21.433 INFO:teuthology.orchestra.run.smithi156.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-13T04:27:21.433 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-13T04:27:21.434 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-13T04:27:21.434 INFO:teuthology.orchestra.run.smithi156.stdout:Enabling module streams: 2023-09-13T04:27:21.434 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql 10 2023-09-13T04:27:21.434 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-13T04:27:21.434 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2023-09-13T04:27:21.435 INFO:teuthology.orchestra.run.smithi156.stdout:=========================================================================================================================== 2023-09-13T04:27:21.435 INFO:teuthology.orchestra.run.smithi156.stdout:Install 128 Packages 2023-09-13T04:27:21.435 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrade 3 Packages 2023-09-13T04:27:21.435 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-13T04:27:21.436 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 208 M 2023-09-13T04:27:21.436 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2023-09-13T04:27:21.646 INFO:teuthology.orchestra.run.smithi149.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-13T04:27:21.647 INFO:teuthology.orchestra.run.smithi149.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-13T04:27:21.661 INFO:teuthology.orchestra.run.smithi149.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-13T04:27:21.662 INFO:teuthology.orchestra.run.smithi149.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-13T04:27:21.663 INFO:teuthology.orchestra.run.smithi149.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-13T04:27:21.664 INFO:teuthology.orchestra.run.smithi149.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-13T04:27:21.666 INFO:teuthology.orchestra.run.smithi149.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-13T04:27:21.669 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-13T04:27:21.671 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-13T04:27:21.802 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2023-09-13T04:27:21.815 INFO:teuthology.orchestra.run.smithi149.stdout:=========================================================================================================================== 2023-09-13T04:27:21.815 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2023-09-13T04:27:21.815 INFO:teuthology.orchestra.run.smithi149.stdout:=========================================================================================================================== 2023-09-13T04:27:21.815 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2023-09-13T04:27:21.816 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-13T04:27:21.816 INFO:teuthology.orchestra.run.smithi149.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-13T04:27:21.816 INFO:teuthology.orchestra.run.smithi149.stdout: ceph x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 6.4 k 2023-09-13T04:27:21.816 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 5.4 M 2023-09-13T04:27:21.816 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 919 k 2023-09-13T04:27:21.817 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 167 k 2023-09-13T04:27:21.817 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 1.6 M 2023-09-13T04:27:21.817 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 132 k 2023-09-13T04:27:21.817 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 1.7 M 2023-09-13T04:27:21.817 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 7.4 M 2023-09-13T04:27:21.818 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 47 k 2023-09-13T04:27:21.818 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 14 M 2023-09-13T04:27:21.818 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 49 M 2023-09-13T04:27:21.818 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 261 k 2023-09-13T04:27:21.818 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 179 k 2023-09-13T04:27:21.819 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 260 k 2023-09-13T04:27:21.819 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 18 k 2023-09-13T04:27:21.819 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-13T04:27:21.819 INFO:teuthology.orchestra.run.smithi149.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-13T04:27:21.819 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-13T04:27:21.820 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 31 k 2023-09-13T04:27:21.820 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2 x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 839 k 2023-09-13T04:27:21.820 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 139 k 2023-09-13T04:27:21.820 INFO:teuthology.orchestra.run.smithi149.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-13T04:27:21.820 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-13T04:27:21.821 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-13T04:27:21.821 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-13T04:27:21.821 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-13T04:27:21.821 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-13T04:27:21.821 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 197 k 2023-09-13T04:27:21.822 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 370 k 2023-09-13T04:27:21.822 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 365 k 2023-09-13T04:27:21.822 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 107 k 2023-09-13T04:27:21.822 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 93 k 2023-09-13T04:27:21.822 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 3.6 M 2023-09-13T04:27:21.823 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 179 k 2023-09-13T04:27:21.823 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-13T04:27:21.823 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrading: 2023-09-13T04:27:21.823 INFO:teuthology.orchestra.run.smithi149.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-13T04:27:21.823 INFO:teuthology.orchestra.run.smithi149.stdout: librados2 x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 3.7 M 2023-09-13T04:27:21.824 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1 x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 3.7 M 2023-09-13T04:27:21.824 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2023-09-13T04:27:21.824 INFO:teuthology.orchestra.run.smithi149.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-13T04:27:21.824 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 22 M 2023-09-13T04:27:21.824 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 23 k 2023-09-13T04:27:21.825 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 2.3 M 2023-09-13T04:27:21.825 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 242 k 2023-09-13T04:27:21.825 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 4.6 M 2023-09-13T04:27:21.825 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 18 M 2023-09-13T04:27:21.825 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 15 k 2023-09-13T04:27:21.826 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 24 k 2023-09-13T04:27:21.826 INFO:teuthology.orchestra.run.smithi149.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-13T04:27:21.826 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-13T04:27:21.826 INFO:teuthology.orchestra.run.smithi149.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-13T04:27:21.827 INFO:teuthology.orchestra.run.smithi149.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-13T04:27:21.827 INFO:teuthology.orchestra.run.smithi149.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-13T04:27:21.827 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 183 k 2023-09-13T04:27:21.827 INFO:teuthology.orchestra.run.smithi149.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-13T04:27:21.827 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-13T04:27:21.828 INFO:teuthology.orchestra.run.smithi149.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-13T04:27:21.828 INFO:teuthology.orchestra.run.smithi149.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-13T04:27:21.828 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-13T04:27:21.828 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-13T04:27:21.828 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1 x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 518 k 2023-09-13T04:27:21.829 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-13T04:27:21.829 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2 x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 6.8 M 2023-09-13T04:27:21.829 INFO:teuthology.orchestra.run.smithi149.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-13T04:27:21.829 INFO:teuthology.orchestra.run.smithi149.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-13T04:27:21.829 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-13T04:27:21.829 INFO:teuthology.orchestra.run.smithi149.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-13T04:27:21.830 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-13T04:27:21.830 INFO:teuthology.orchestra.run.smithi149.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-13T04:27:21.830 INFO:teuthology.orchestra.run.smithi149.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-13T04:27:21.830 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-13T04:27:21.830 INFO:teuthology.orchestra.run.smithi149.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-13T04:27:21.831 INFO:teuthology.orchestra.run.smithi149.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-13T04:27:21.831 INFO:teuthology.orchestra.run.smithi149.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-13T04:27:21.831 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-13T04:27:21.831 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-13T04:27:21.832 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-13T04:27:21.832 INFO:teuthology.orchestra.run.smithi149.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-13T04:27:21.832 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-13T04:27:21.832 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 45 k 2023-09-13T04:27:21.832 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 125 k 2023-09-13T04:27:21.832 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-13T04:27:21.833 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-13T04:27:21.833 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-13T04:27:21.833 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-13T04:27:21.833 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-13T04:27:21.833 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-13T04:27:21.834 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-13T04:27:21.834 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-13T04:27:21.834 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-13T04:27:21.834 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-13T04:27:21.834 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-13T04:27:21.835 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-13T04:27:21.835 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-13T04:27:21.835 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-13T04:27:21.835 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-13T04:27:21.835 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-13T04:27:21.836 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-13T04:27:21.836 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-13T04:27:21.836 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-13T04:27:21.836 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-13T04:27:21.836 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-13T04:27:21.837 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-13T04:27:21.837 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-13T04:27:21.837 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-13T04:27:21.837 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-13T04:27:21.837 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-13T04:27:21.838 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-13T04:27:21.838 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-13T04:27:21.838 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-13T04:27:21.838 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-13T04:27:21.838 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-13T04:27:21.839 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-13T04:27:21.839 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-13T04:27:21.839 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-13T04:27:21.839 INFO:teuthology.orchestra.run.smithi149.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-13T04:27:21.839 INFO:teuthology.orchestra.run.smithi149.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-13T04:27:21.840 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-13T04:27:21.840 INFO:teuthology.orchestra.run.smithi149.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-13T04:27:21.840 INFO:teuthology.orchestra.run.smithi149.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-13T04:27:21.840 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-13T04:27:21.840 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-13T04:27:21.840 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-13T04:27:21.841 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-13T04:27:21.841 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-13T04:27:21.841 INFO:teuthology.orchestra.run.smithi149.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-13T04:27:21.841 INFO:teuthology.orchestra.run.smithi149.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-13T04:27:21.842 INFO:teuthology.orchestra.run.smithi149.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-13T04:27:21.842 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-13T04:27:21.842 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-13T04:27:21.842 INFO:teuthology.orchestra.run.smithi149.stdout:Enabling module streams: 2023-09-13T04:27:21.842 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql 10 2023-09-13T04:27:21.843 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-09-13T04:27:21.843 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2023-09-13T04:27:21.843 INFO:teuthology.orchestra.run.smithi149.stdout:=========================================================================================================================== 2023-09-13T04:27:21.843 INFO:teuthology.orchestra.run.smithi149.stdout:Install 128 Packages 2023-09-13T04:27:21.843 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrade 3 Packages 2023-09-13T04:27:21.844 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-09-13T04:27:21.844 INFO:teuthology.orchestra.run.smithi156.stdout:(1/131): ceph-18.0.0-6083.g4a754ac9.el8.x86_64. 17 kB/s | 6.4 kB 00:00 2023-09-13T04:27:21.845 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 208 M 2023-09-13T04:27:21.845 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2023-09-13T04:27:22.474 INFO:teuthology.orchestra.run.smithi149.stdout:(1/131): ceph-18.0.0-6083.g4a754ac9.el8.x86_64. 19 kB/s | 6.4 kB 00:00 2023-09-13T04:27:22.533 INFO:teuthology.orchestra.run.smithi156.stdout:(2/131): ceph-fuse-18.0.0-6083.g4a754ac9.el8.x8 1.3 MB/s | 919 kB 00:00 2023-09-13T04:27:22.641 INFO:teuthology.orchestra.run.smithi156.stdout:(3/131): ceph-immutable-object-cache-18.0.0-608 1.5 MB/s | 167 kB 00:00 2023-09-13T04:27:23.183 INFO:teuthology.orchestra.run.smithi149.stdout:(2/131): ceph-fuse-18.0.0-6083.g4a754ac9.el8.x8 1.3 MB/s | 919 kB 00:00 2023-09-13T04:27:23.341 INFO:teuthology.orchestra.run.smithi149.stdout:(3/131): ceph-immutable-object-cache-18.0.0-608 1.0 MB/s | 167 kB 00:00 2023-09-13T04:27:23.499 INFO:teuthology.orchestra.run.smithi156.stdout:(4/131): ceph-mds-18.0.0-6083.g4a754ac9.el8.x86 2.7 MB/s | 2.3 MB 00:00 2023-09-13T04:27:24.173 INFO:teuthology.orchestra.run.smithi156.stdout:(5/131): ceph-mgr-18.0.0-6083.g4a754ac9.el8.x86 2.4 MB/s | 1.6 MB 00:00 2023-09-13T04:27:24.233 INFO:teuthology.orchestra.run.smithi149.stdout:(4/131): ceph-mds-18.0.0-6083.g4a754ac9.el8.x86 2.6 MB/s | 2.3 MB 00:00 2023-09-13T04:27:24.850 INFO:teuthology.orchestra.run.smithi149.stdout:(5/131): ceph-base-18.0.0-6083.g4a754ac9.el8.x8 2.0 MB/s | 5.4 MB 00:02 2023-09-13T04:27:24.892 INFO:teuthology.orchestra.run.smithi149.stdout:(6/131): ceph-mgr-18.0.0-6083.g4a754ac9.el8.x86 2.5 MB/s | 1.6 MB 00:00 2023-09-13T04:27:25.667 INFO:teuthology.orchestra.run.smithi156.stdout:(6/131): ceph-mon-18.0.0-6083.g4a754ac9.el8.x86 3.1 MB/s | 4.6 MB 00:01 2023-09-13T04:27:26.642 INFO:teuthology.orchestra.run.smithi149.stdout:(7/131): ceph-mon-18.0.0-6083.g4a754ac9.el8.x86 2.6 MB/s | 4.6 MB 00:01 2023-09-13T04:27:26.717 INFO:teuthology.orchestra.run.smithi156.stdout:(7/131): ceph-base-18.0.0-6083.g4a754ac9.el8.x8 1.0 MB/s | 5.4 MB 00:05 2023-09-13T04:27:27.484 INFO:teuthology.orchestra.run.smithi002.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-13T04:27:27.485 INFO:teuthology.orchestra.run.smithi002.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-13T04:27:27.500 INFO:teuthology.orchestra.run.smithi002.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-13T04:27:27.502 INFO:teuthology.orchestra.run.smithi002.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-13T04:27:27.503 INFO:teuthology.orchestra.run.smithi002.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-13T04:27:27.504 INFO:teuthology.orchestra.run.smithi002.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-13T04:27:27.507 INFO:teuthology.orchestra.run.smithi002.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-13T04:27:27.510 INFO:teuthology.orchestra.run.smithi002.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-13T04:27:27.511 INFO:teuthology.orchestra.run.smithi002.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-13T04:27:27.660 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2023-09-13T04:27:27.673 INFO:teuthology.orchestra.run.smithi002.stdout:=========================================================================================================================== 2023-09-13T04:27:27.674 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2023-09-13T04:27:27.674 INFO:teuthology.orchestra.run.smithi002.stdout:=========================================================================================================================== 2023-09-13T04:27:27.674 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2023-09-13T04:27:27.674 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-13T04:27:27.675 INFO:teuthology.orchestra.run.smithi002.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-13T04:27:27.675 INFO:teuthology.orchestra.run.smithi002.stdout: ceph x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 6.4 k 2023-09-13T04:27:27.675 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-base x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 5.4 M 2023-09-13T04:27:27.675 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-fuse x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 919 k 2023-09-13T04:27:27.675 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 167 k 2023-09-13T04:27:27.676 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 1.6 M 2023-09-13T04:27:27.676 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 132 k 2023-09-13T04:27:27.676 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 1.7 M 2023-09-13T04:27:27.676 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 7.4 M 2023-09-13T04:27:27.676 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-rook noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 47 k 2023-09-13T04:27:27.676 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-radosgw x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 14 M 2023-09-13T04:27:27.677 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-test x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 49 M 2023-09-13T04:27:27.677 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-volume noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 261 k 2023-09-13T04:27:27.677 INFO:teuthology.orchestra.run.smithi002.stdout: cephadm noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 179 k 2023-09-13T04:27:27.677 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-mirror x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 260 k 2023-09-13T04:27:27.677 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-top noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 18 k 2023-09-13T04:27:27.678 INFO:teuthology.orchestra.run.smithi002.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-13T04:27:27.678 INFO:teuthology.orchestra.run.smithi002.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-13T04:27:27.678 INFO:teuthology.orchestra.run.smithi002.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-13T04:27:27.678 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs-devel x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 31 k 2023-09-13T04:27:27.678 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs2 x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 839 k 2023-09-13T04:27:27.679 INFO:teuthology.orchestra.run.smithi002.stdout: librados-devel x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 139 k 2023-09-13T04:27:27.679 INFO:teuthology.orchestra.run.smithi002.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-13T04:27:27.679 INFO:teuthology.orchestra.run.smithi002.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-13T04:27:27.679 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-13T04:27:27.679 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-13T04:27:27.680 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-13T04:27:27.680 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-13T04:27:27.680 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cephfs x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 197 k 2023-09-13T04:27:27.680 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rados x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 370 k 2023-09-13T04:27:27.680 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rbd x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 365 k 2023-09-13T04:27:27.680 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rgw x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 107 k 2023-09-13T04:27:27.681 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-fuse x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 93 k 2023-09-13T04:27:27.681 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-mirror x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 3.6 M 2023-09-13T04:27:27.681 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-nbd x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 179 k 2023-09-13T04:27:27.681 INFO:teuthology.orchestra.run.smithi002.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-13T04:27:27.681 INFO:teuthology.orchestra.run.smithi002.stdout:Upgrading: 2023-09-13T04:27:27.682 INFO:teuthology.orchestra.run.smithi002.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-13T04:27:27.682 INFO:teuthology.orchestra.run.smithi002.stdout: librados2 x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 3.7 M 2023-09-13T04:27:27.682 INFO:teuthology.orchestra.run.smithi002.stdout: librbd1 x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 3.7 M 2023-09-13T04:27:27.682 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2023-09-13T04:27:27.682 INFO:teuthology.orchestra.run.smithi002.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-13T04:27:27.683 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-common x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 22 M 2023-09-13T04:27:27.683 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 23 k 2023-09-13T04:27:27.683 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mds x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 2.3 M 2023-09-13T04:27:27.683 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 242 k 2023-09-13T04:27:27.683 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mon x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 4.6 M 2023-09-13T04:27:27.684 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-osd x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 18 M 2023-09-13T04:27:27.684 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6083.g4a754ac9.el8 ceph-noarch 15 k 2023-09-13T04:27:27.684 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-selinux x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 24 k 2023-09-13T04:27:27.684 INFO:teuthology.orchestra.run.smithi002.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-13T04:27:27.684 INFO:teuthology.orchestra.run.smithi002.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-13T04:27:27.684 INFO:teuthology.orchestra.run.smithi002.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-13T04:27:27.685 INFO:teuthology.orchestra.run.smithi002.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-13T04:27:27.685 INFO:teuthology.orchestra.run.smithi002.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-13T04:27:27.685 INFO:teuthology.orchestra.run.smithi002.stdout: libcephsqlite x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 183 k 2023-09-13T04:27:27.685 INFO:teuthology.orchestra.run.smithi002.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-13T04:27:27.685 INFO:teuthology.orchestra.run.smithi002.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-13T04:27:27.686 INFO:teuthology.orchestra.run.smithi002.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-13T04:27:27.686 INFO:teuthology.orchestra.run.smithi002.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-13T04:27:27.686 INFO:teuthology.orchestra.run.smithi002.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-13T04:27:27.686 INFO:teuthology.orchestra.run.smithi002.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-13T04:27:27.686 INFO:teuthology.orchestra.run.smithi002.stdout: libradosstriper1 x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 518 k 2023-09-13T04:27:27.687 INFO:teuthology.orchestra.run.smithi002.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-13T04:27:27.687 INFO:teuthology.orchestra.run.smithi002.stdout: librgw2 x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 6.8 M 2023-09-13T04:27:27.687 INFO:teuthology.orchestra.run.smithi002.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-13T04:27:27.687 INFO:teuthology.orchestra.run.smithi002.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-13T04:27:27.687 INFO:teuthology.orchestra.run.smithi002.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-13T04:27:27.687 INFO:teuthology.orchestra.run.smithi002.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-13T04:27:27.688 INFO:teuthology.orchestra.run.smithi002.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-13T04:27:27.688 INFO:teuthology.orchestra.run.smithi002.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-13T04:27:27.688 INFO:teuthology.orchestra.run.smithi002.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-13T04:27:27.688 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-13T04:27:27.688 INFO:teuthology.orchestra.run.smithi002.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-13T04:27:27.689 INFO:teuthology.orchestra.run.smithi002.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-13T04:27:27.689 INFO:teuthology.orchestra.run.smithi002.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-13T04:27:27.689 INFO:teuthology.orchestra.run.smithi002.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-13T04:27:27.689 INFO:teuthology.orchestra.run.smithi002.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-13T04:27:27.690 INFO:teuthology.orchestra.run.smithi002.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-13T04:27:27.690 INFO:teuthology.orchestra.run.smithi002.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-13T04:27:27.690 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-13T04:27:27.690 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-argparse x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 45 k 2023-09-13T04:27:27.691 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-common x86_64 2:18.0.0-6083.g4a754ac9.el8 ceph 125 k 2023-09-13T04:27:27.691 INFO:teuthology.orchestra.run.smithi002.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-13T04:27:27.691 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-13T04:27:27.691 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-13T04:27:27.691 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-13T04:27:27.691 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-13T04:27:27.692 INFO:teuthology.orchestra.run.smithi002.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-13T04:27:27.692 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-13T04:27:27.692 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-13T04:27:27.692 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-13T04:27:27.693 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-13T04:27:27.693 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-13T04:27:27.693 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-13T04:27:27.693 INFO:teuthology.orchestra.run.smithi002.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-13T04:27:27.693 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-13T04:27:27.694 INFO:teuthology.orchestra.run.smithi002.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-13T04:27:27.694 INFO:teuthology.orchestra.run.smithi002.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-13T04:27:27.694 INFO:teuthology.orchestra.run.smithi002.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-13T04:27:27.694 INFO:teuthology.orchestra.run.smithi002.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-13T04:27:27.695 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-13T04:27:27.695 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-13T04:27:27.695 INFO:teuthology.orchestra.run.smithi002.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-13T04:27:27.695 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-13T04:27:27.695 INFO:teuthology.orchestra.run.smithi002.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-13T04:27:27.696 INFO:teuthology.orchestra.run.smithi002.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-13T04:27:27.696 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-13T04:27:27.696 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-13T04:27:27.696 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-13T04:27:27.696 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-13T04:27:27.697 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-13T04:27:27.697 INFO:teuthology.orchestra.run.smithi002.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-13T04:27:27.697 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-13T04:27:27.697 INFO:teuthology.orchestra.run.smithi002.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-13T04:27:27.697 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-13T04:27:27.698 INFO:teuthology.orchestra.run.smithi002.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-13T04:27:27.698 INFO:teuthology.orchestra.run.smithi002.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-13T04:27:27.698 INFO:teuthology.orchestra.run.smithi002.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-13T04:27:27.698 INFO:teuthology.orchestra.run.smithi002.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-13T04:27:27.698 INFO:teuthology.orchestra.run.smithi002.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-13T04:27:27.699 INFO:teuthology.orchestra.run.smithi002.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-13T04:27:27.699 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-13T04:27:27.699 INFO:teuthology.orchestra.run.smithi002.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-13T04:27:27.699 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-13T04:27:27.699 INFO:teuthology.orchestra.run.smithi002.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-13T04:27:27.700 INFO:teuthology.orchestra.run.smithi002.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-13T04:27:27.700 INFO:teuthology.orchestra.run.smithi002.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-13T04:27:27.700 INFO:teuthology.orchestra.run.smithi002.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-13T04:27:27.700 INFO:teuthology.orchestra.run.smithi002.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-13T04:27:27.700 INFO:teuthology.orchestra.run.smithi002.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-13T04:27:27.700 INFO:teuthology.orchestra.run.smithi002.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-13T04:27:27.701 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling module streams: 2023-09-13T04:27:27.701 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql 10 2023-09-13T04:27:27.701 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:27:27.701 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2023-09-13T04:27:27.701 INFO:teuthology.orchestra.run.smithi002.stdout:=========================================================================================================================== 2023-09-13T04:27:27.702 INFO:teuthology.orchestra.run.smithi002.stdout:Install 128 Packages 2023-09-13T04:27:27.702 INFO:teuthology.orchestra.run.smithi002.stdout:Upgrade 3 Packages 2023-09-13T04:27:27.702 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:27:27.703 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 208 M 2023-09-13T04:27:27.703 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2023-09-13T04:27:28.007 INFO:teuthology.orchestra.run.smithi002.stdout:(1/131): ceph-18.0.0-6083.g4a754ac9.el8.x86_64. 23 kB/s | 6.4 kB 00:00 2023-09-13T04:27:28.590 INFO:teuthology.orchestra.run.smithi002.stdout:(2/131): ceph-fuse-18.0.0-6083.g4a754ac9.el8.x8 1.5 MB/s | 919 kB 00:00 2023-09-13T04:27:28.799 INFO:teuthology.orchestra.run.smithi002.stdout:(3/131): ceph-immutable-object-cache-18.0.0-608 804 kB/s | 167 kB 00:00 2023-09-13T04:27:29.149 INFO:teuthology.orchestra.run.smithi002.stdout:(4/131): ceph-base-18.0.0-6083.g4a754ac9.el8.x8 3.8 MB/s | 5.4 MB 00:01 2023-09-13T04:27:29.426 INFO:teuthology.orchestra.run.smithi149.stdout:(8/131): ceph-osd-18.0.0-6083.g4a754ac9.el8.x86 3.9 MB/s | 18 MB 00:04 2023-09-13T04:27:29.433 INFO:teuthology.orchestra.run.smithi002.stdout:(5/131): ceph-mgr-18.0.0-6083.g4a754ac9.el8.x86 5.7 MB/s | 1.6 MB 00:00 2023-09-13T04:27:29.467 INFO:teuthology.orchestra.run.smithi149.stdout:(9/131): ceph-selinux-18.0.0-6083.g4a754ac9.el8 588 kB/s | 24 kB 00:00 2023-09-13T04:27:29.483 INFO:teuthology.orchestra.run.smithi002.stdout:(6/131): ceph-mds-18.0.0-6083.g4a754ac9.el8.x86 3.4 MB/s | 2.3 MB 00:00 2023-09-13T04:27:29.609 INFO:teuthology.orchestra.run.smithi156.stdout:(8/131): ceph-common-18.0.0-6083.g4a754ac9.el8. 2.7 MB/s | 22 MB 00:08 2023-09-13T04:27:29.700 INFO:teuthology.orchestra.run.smithi156.stdout:(9/131): ceph-selinux-18.0.0-6083.g4a754ac9.el8 265 kB/s | 24 kB 00:00 2023-09-13T04:27:30.225 INFO:teuthology.orchestra.run.smithi002.stdout:(7/131): ceph-mon-18.0.0-6083.g4a754ac9.el8.x86 5.8 MB/s | 4.6 MB 00:00 2023-09-13T04:27:30.251 INFO:teuthology.orchestra.run.smithi149.stdout:(10/131): ceph-common-18.0.0-6083.g4a754ac9.el8 2.8 MB/s | 22 MB 00:08 2023-09-13T04:27:30.485 INFO:teuthology.orchestra.run.smithi149.stdout:(11/131): cephfs-mirror-18.0.0-6083.g4a754ac9.e 1.1 MB/s | 260 kB 00:00 2023-09-13T04:27:30.585 INFO:teuthology.orchestra.run.smithi149.stdout:(12/131): libcephfs-devel-18.0.0-6083.g4a754ac9 310 kB/s | 31 kB 00:00 2023-09-13T04:27:31.002 INFO:teuthology.orchestra.run.smithi149.stdout:(13/131): libcephfs2-18.0.0-6083.g4a754ac9.el8. 2.0 MB/s | 839 kB 00:00 2023-09-13T04:27:31.144 INFO:teuthology.orchestra.run.smithi149.stdout:(14/131): libcephsqlite-18.0.0-6083.g4a754ac9.e 1.3 MB/s | 183 kB 00:00 2023-09-13T04:27:31.236 INFO:teuthology.orchestra.run.smithi149.stdout:(15/131): librados-devel-18.0.0-6083.g4a754ac9. 1.5 MB/s | 139 kB 00:00 2023-09-13T04:27:31.292 INFO:teuthology.orchestra.run.smithi156.stdout:(10/131): ceph-osd-18.0.0-6083.g4a754ac9.el8.x8 3.2 MB/s | 18 MB 00:05 2023-09-13T04:27:31.468 INFO:teuthology.orchestra.run.smithi156.stdout:(11/131): cephfs-mirror-18.0.0-6083.g4a754ac9.e 1.5 MB/s | 260 kB 00:00 2023-09-13T04:27:31.495 INFO:teuthology.orchestra.run.smithi149.stdout:(16/131): libradosstriper1-18.0.0-6083.g4a754ac 2.0 MB/s | 518 kB 00:00 2023-09-13T04:27:31.559 INFO:teuthology.orchestra.run.smithi156.stdout:(12/131): libcephfs-devel-18.0.0-6083.g4a754ac9 339 kB/s | 31 kB 00:00 2023-09-13T04:27:31.919 INFO:teuthology.orchestra.run.smithi156.stdout:(13/131): libcephfs2-18.0.0-6083.g4a754ac9.el8. 2.3 MB/s | 839 kB 00:00 2023-09-13T04:27:32.044 INFO:teuthology.orchestra.run.smithi156.stdout:(14/131): libcephsqlite-18.0.0-6083.g4a754ac9.e 1.4 MB/s | 183 kB 00:00 2023-09-13T04:27:32.161 INFO:teuthology.orchestra.run.smithi156.stdout:(15/131): librados-devel-18.0.0-6083.g4a754ac9. 1.2 MB/s | 139 kB 00:00 2023-09-13T04:27:32.267 INFO:teuthology.orchestra.run.smithi002.stdout:(8/131): ceph-radosgw-18.0.0-6083.g4a754ac9.el8 6.7 MB/s | 14 MB 00:02 2023-09-13T04:27:32.319 INFO:teuthology.orchestra.run.smithi156.stdout:(16/131): libradosstriper1-18.0.0-6083.g4a754ac 3.2 MB/s | 518 kB 00:00 2023-09-13T04:27:32.342 INFO:teuthology.orchestra.run.smithi002.stdout:(9/131): ceph-selinux-18.0.0-6083.g4a754ac9.el8 326 kB/s | 24 kB 00:00 2023-09-13T04:27:32.878 INFO:teuthology.orchestra.run.smithi149.stdout:(17/131): ceph-radosgw-18.0.0-6083.g4a754ac9.el 2.2 MB/s | 14 MB 00:06 2023-09-13T04:27:32.970 INFO:teuthology.orchestra.run.smithi149.stdout:(18/131): python3-ceph-argparse-18.0.0-6083.g4a 490 kB/s | 45 kB 00:00 2023-09-13T04:27:33.070 INFO:teuthology.orchestra.run.smithi149.stdout:(19/131): python3-ceph-common-18.0.0-6083.g4a75 1.2 MB/s | 125 kB 00:00 2023-09-13T04:27:33.245 INFO:teuthology.orchestra.run.smithi149.stdout:(20/131): python3-cephfs-18.0.0-6083.g4a754ac9. 1.1 MB/s | 197 kB 00:00 2023-09-13T04:27:33.504 INFO:teuthology.orchestra.run.smithi149.stdout:(21/131): python3-rados-18.0.0-6083.g4a754ac9.e 1.4 MB/s | 370 kB 00:00 2023-09-13T04:27:33.746 INFO:teuthology.orchestra.run.smithi149.stdout:(22/131): python3-rbd-18.0.0-6083.g4a754ac9.el8 1.5 MB/s | 365 kB 00:00 2023-09-13T04:27:33.827 INFO:teuthology.orchestra.run.smithi156.stdout:(17/131): ceph-radosgw-18.0.0-6083.g4a754ac9.el 1.9 MB/s | 14 MB 00:07 2023-09-13T04:27:33.846 INFO:teuthology.orchestra.run.smithi149.stdout:(23/131): python3-rgw-18.0.0-6083.g4a754ac9.el8 1.0 MB/s | 107 kB 00:00 2023-09-13T04:27:33.902 INFO:teuthology.orchestra.run.smithi156.stdout:(18/131): python3-ceph-argparse-18.0.0-6083.g4a 599 kB/s | 45 kB 00:00 2023-09-13T04:27:33.921 INFO:teuthology.orchestra.run.smithi149.stdout:(24/131): rbd-fuse-18.0.0-6083.g4a754ac9.el8.x8 1.2 MB/s | 93 kB 00:00 2023-09-13T04:27:33.925 INFO:teuthology.orchestra.run.smithi002.stdout:(10/131): ceph-osd-18.0.0-6083.g4a754ac9.el8.x8 4.0 MB/s | 18 MB 00:04 2023-09-13T04:27:33.978 INFO:teuthology.orchestra.run.smithi156.stdout:(19/131): python3-ceph-common-18.0.0-6083.g4a75 1.6 MB/s | 125 kB 00:00 2023-09-13T04:27:34.059 INFO:teuthology.orchestra.run.smithi002.stdout:(11/131): cephfs-mirror-18.0.0-6083.g4a754ac9.e 1.9 MB/s | 260 kB 00:00 2023-09-13T04:27:34.078 INFO:teuthology.orchestra.run.smithi156.stdout:(20/131): python3-cephfs-18.0.0-6083.g4a754ac9. 1.9 MB/s | 197 kB 00:00 2023-09-13T04:27:34.142 INFO:teuthology.orchestra.run.smithi002.stdout:(12/131): libcephfs-devel-18.0.0-6083.g4a754ac9 373 kB/s | 31 kB 00:00 2023-09-13T04:27:34.262 INFO:teuthology.orchestra.run.smithi156.stdout:(21/131): python3-rados-18.0.0-6083.g4a754ac9.e 2.0 MB/s | 370 kB 00:00 2023-09-13T04:27:34.393 INFO:teuthology.orchestra.run.smithi002.stdout:(13/131): libcephfs2-18.0.0-6083.g4a754ac9.el8. 3.3 MB/s | 839 kB 00:00 2023-09-13T04:27:34.485 INFO:teuthology.orchestra.run.smithi002.stdout:(14/131): libcephsqlite-18.0.0-6083.g4a754ac9.e 2.0 MB/s | 183 kB 00:00 2023-09-13T04:27:34.530 INFO:teuthology.orchestra.run.smithi149.stdout:(25/131): librgw2-18.0.0-6083.g4a754ac9.el8.x86 2.2 MB/s | 6.8 MB 00:03 2023-09-13T04:27:34.537 INFO:teuthology.orchestra.run.smithi156.stdout:(22/131): python3-rbd-18.0.0-6083.g4a754ac9.el8 1.3 MB/s | 365 kB 00:00 2023-09-13T04:27:34.552 INFO:teuthology.orchestra.run.smithi002.stdout:(15/131): librados-devel-18.0.0-6083.g4a754ac9. 2.0 MB/s | 139 kB 00:00 2023-09-13T04:27:34.622 INFO:teuthology.orchestra.run.smithi149.stdout:(26/131): rbd-nbd-18.0.0-6083.g4a754ac9.el8.x86 1.9 MB/s | 179 kB 00:00 2023-09-13T04:27:34.704 INFO:teuthology.orchestra.run.smithi156.stdout:(23/131): python3-rgw-18.0.0-6083.g4a754ac9.el8 642 kB/s | 107 kB 00:00 2023-09-13T04:27:34.719 INFO:teuthology.orchestra.run.smithi002.stdout:(16/131): libradosstriper1-18.0.0-6083.g4a754ac 3.0 MB/s | 518 kB 00:00 2023-09-13T04:27:34.739 INFO:teuthology.orchestra.run.smithi149.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6083.g 197 kB/s | 23 kB 00:00 2023-09-13T04:27:34.839 INFO:teuthology.orchestra.run.smithi149.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6083.g4a754ac 1.3 MB/s | 132 kB 00:00 2023-09-13T04:27:34.871 INFO:teuthology.orchestra.run.smithi156.stdout:(24/131): librgw2-18.0.0-6083.g4a754ac9.el8.x86 2.7 MB/s | 6.8 MB 00:02 2023-09-13T04:27:34.896 INFO:teuthology.orchestra.run.smithi156.stdout:(25/131): rbd-fuse-18.0.0-6083.g4a754ac9.el8.x8 488 kB/s | 93 kB 00:00 2023-09-13T04:27:35.180 INFO:teuthology.orchestra.run.smithi156.stdout:(26/131): rbd-nbd-18.0.0-6083.g4a754ac9.el8.x86 632 kB/s | 179 kB 00:00 2023-09-13T04:27:35.397 INFO:teuthology.orchestra.run.smithi156.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6083.g 106 kB/s | 23 kB 00:00 2023-09-13T04:27:35.531 INFO:teuthology.orchestra.run.smithi149.stdout:(29/131): rbd-mirror-18.0.0-6083.g4a754ac9.el8. 2.2 MB/s | 3.6 MB 00:01 2023-09-13T04:27:35.597 INFO:teuthology.orchestra.run.smithi156.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6083.g4a754ac 660 kB/s | 132 kB 00:00 2023-09-13T04:27:35.657 INFO:teuthology.orchestra.run.smithi149.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6083.g4a754 2.1 MB/s | 1.7 MB 00:00 2023-09-13T04:27:35.857 INFO:teuthology.orchestra.run.smithi149.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6083.g4a 1.2 MB/s | 242 kB 00:00 2023-09-13T04:27:35.974 INFO:teuthology.orchestra.run.smithi149.stdout:(32/131): ceph-mgr-rook-18.0.0-6083.g4a754ac9.e 406 kB/s | 47 kB 00:00 2023-09-13T04:27:36.091 INFO:teuthology.orchestra.run.smithi149.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6083.g4 125 kB/s | 15 kB 00:00 2023-09-13T04:27:36.273 INFO:teuthology.orchestra.run.smithi156.stdout:(29/131): rbd-mirror-18.0.0-6083.g4a754ac9.el8. 2.6 MB/s | 3.6 MB 00:01 2023-09-13T04:27:36.282 INFO:teuthology.orchestra.run.smithi149.stdout:(34/131): ceph-volume-18.0.0-6083.g4a754ac9.el8 1.3 MB/s | 261 kB 00:00 2023-09-13T04:27:36.377 INFO:teuthology.orchestra.run.smithi002.stdout:(17/131): librgw2-18.0.0-6083.g4a754ac9.el8.x86 4.1 MB/s | 6.8 MB 00:01 2023-09-13T04:27:36.391 INFO:teuthology.orchestra.run.smithi149.stdout:(35/131): cephadm-18.0.0-6083.g4a754ac9.el8.noa 1.6 MB/s | 179 kB 00:00 2023-09-13T04:27:36.494 INFO:teuthology.orchestra.run.smithi002.stdout:(18/131): python3-ceph-argparse-18.0.0-6083.g4a 384 kB/s | 45 kB 00:00 2023-09-13T04:27:36.529 INFO:teuthology.orchestra.run.smithi149.stdout:(36/131): cephfs-top-18.0.0-6083.g4a754ac9.el8. 130 kB/s | 18 kB 00:00 2023-09-13T04:27:36.578 INFO:teuthology.orchestra.run.smithi002.stdout:(19/131): python3-ceph-common-18.0.0-6083.g4a75 1.5 MB/s | 125 kB 00:00 2023-09-13T04:27:36.678 INFO:teuthology.orchestra.run.smithi002.stdout:(20/131): python3-cephfs-18.0.0-6083.g4a754ac9. 1.9 MB/s | 197 kB 00:00 2023-09-13T04:27:36.739 INFO:teuthology.orchestra.run.smithi156.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6083.g4a754 1.5 MB/s | 1.7 MB 00:01 2023-09-13T04:27:36.837 INFO:teuthology.orchestra.run.smithi002.stdout:(21/131): python3-rados-18.0.0-6083.g4a754ac9.e 2.3 MB/s | 370 kB 00:00 2023-09-13T04:27:36.931 INFO:teuthology.orchestra.run.smithi156.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6083.g4a 1.2 MB/s | 242 kB 00:00 2023-09-13T04:27:37.004 INFO:teuthology.orchestra.run.smithi002.stdout:(22/131): python3-rbd-18.0.0-6083.g4a754ac9.el8 2.1 MB/s | 365 kB 00:00 2023-09-13T04:27:37.040 INFO:teuthology.orchestra.run.smithi156.stdout:(32/131): ceph-mgr-rook-18.0.0-6083.g4a754ac9.e 438 kB/s | 47 kB 00:00 2023-09-13T04:27:37.104 INFO:teuthology.orchestra.run.smithi002.stdout:(23/131): python3-rgw-18.0.0-6083.g4a754ac9.el8 1.0 MB/s | 107 kB 00:00 2023-09-13T04:27:37.149 INFO:teuthology.orchestra.run.smithi156.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6083.g4 134 kB/s | 15 kB 00:00 2023-09-13T04:27:37.188 INFO:teuthology.orchestra.run.smithi002.stdout:(24/131): rbd-fuse-18.0.0-6083.g4a754ac9.el8.x8 1.1 MB/s | 93 kB 00:00 2023-09-13T04:27:37.332 INFO:teuthology.orchestra.run.smithi156.stdout:(34/131): ceph-volume-18.0.0-6083.g4a754ac9.el8 1.4 MB/s | 261 kB 00:00 2023-09-13T04:27:37.375 INFO:teuthology.orchestra.run.smithi149.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 362 kB/s | 306 kB 00:00 2023-09-13T04:27:37.430 INFO:teuthology.orchestra.run.smithi002.stdout:(25/131): ceph-common-18.0.0-6083.g4a754ac9.el8 2.3 MB/s | 22 MB 00:09 2023-09-13T04:27:37.449 INFO:teuthology.orchestra.run.smithi156.stdout:(35/131): cephadm-18.0.0-6083.g4a754ac9.el8.noa 1.5 MB/s | 179 kB 00:00 2023-09-13T04:27:37.492 INFO:teuthology.orchestra.run.smithi149.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 506 kB/s | 59 kB 00:00 2023-09-13T04:27:37.558 INFO:teuthology.orchestra.run.smithi156.stdout:(36/131): cephfs-top-18.0.0-6083.g4a754ac9.el8. 166 kB/s | 18 kB 00:00 2023-09-13T04:27:37.609 INFO:teuthology.orchestra.run.smithi149.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 643 kB/s | 75 kB 00:00 2023-09-13T04:27:37.613 INFO:teuthology.orchestra.run.smithi002.stdout:(26/131): rbd-nbd-18.0.0-6083.g4a754ac9.el8.x86 979 kB/s | 179 kB 00:00 2023-09-13T04:27:37.691 INFO:teuthology.orchestra.run.smithi156.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 2.2 MB/s | 306 kB 00:00 2023-09-13T04:27:37.725 INFO:teuthology.orchestra.run.smithi156.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.7 MB/s | 59 kB 00:00 2023-09-13T04:27:37.767 INFO:teuthology.orchestra.run.smithi156.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.8 MB/s | 75 kB 00:00 2023-09-13T04:27:37.834 INFO:teuthology.orchestra.run.smithi156.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 7.0 MB/s | 476 kB 00:00 2023-09-13T04:27:37.868 INFO:teuthology.orchestra.run.smithi156.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.3 MB/s | 44 kB 00:00 2023-09-13T04:27:37.910 INFO:teuthology.orchestra.run.smithi156.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 4.3 MB/s | 185 kB 00:00 2023-09-13T04:27:37.943 INFO:teuthology.orchestra.run.smithi156.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 988 kB/s | 33 kB 00:00 2023-09-13T04:27:37.977 INFO:teuthology.orchestra.run.smithi156.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 491 kB/s | 16 kB 00:00 2023-09-13T04:27:38.019 INFO:teuthology.orchestra.run.smithi156.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 4.1 MB/s | 173 kB 00:00 2023-09-13T04:27:38.034 INFO:teuthology.orchestra.run.smithi149.stdout:(40/131): ceph-mgr-diskprediction-local-18.0.0- 3.0 MB/s | 7.4 MB 00:02 2023-09-13T04:27:38.069 INFO:teuthology.orchestra.run.smithi156.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 7.5 MB/s | 384 kB 00:00 2023-09-13T04:27:38.111 INFO:teuthology.orchestra.run.smithi156.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.9 MB/s | 81 kB 00:00 2023-09-13T04:27:38.153 INFO:teuthology.orchestra.run.smithi156.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 266 kB/s | 11 kB 00:00 2023-09-13T04:27:38.164 INFO:teuthology.orchestra.run.smithi002.stdout:(27/131): ceph-test-18.0.0-6083.g4a754ac9.el8.x 8.4 MB/s | 49 MB 00:05 2023-09-13T04:27:38.189 INFO:teuthology.orchestra.run.smithi002.stdout:(28/131): ceph-grafana-dashboards-18.0.0-6083.g 40 kB/s | 23 kB 00:00 2023-09-13T04:27:38.195 INFO:teuthology.orchestra.run.smithi156.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 422 kB/s | 18 kB 00:00 2023-09-13T04:27:38.214 INFO:teuthology.orchestra.run.smithi002.stdout:(29/131): ceph-mgr-cephadm-18.0.0-6083.g4a754ac 2.6 MB/s | 132 kB 00:00 2023-09-13T04:27:38.281 INFO:teuthology.orchestra.run.smithi002.stdout:(30/131): rbd-mirror-18.0.0-6083.g4a754ac9.el8. 3.3 MB/s | 3.6 MB 00:01 2023-09-13T04:27:38.287 INFO:teuthology.orchestra.run.smithi156.stdout:(50/131): ceph-mgr-diskprediction-local-18.0.0- 3.7 MB/s | 7.4 MB 00:02 2023-09-13T04:27:38.321 INFO:teuthology.orchestra.run.smithi156.stdout:(51/131): python3-kubernetes-11.0.0-6.el8.noarc 9.3 MB/s | 1.2 MB 00:00 2023-09-13T04:27:38.360 INFO:teuthology.orchestra.run.smithi149.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 634 kB/s | 476 kB 00:00 2023-09-13T04:27:38.363 INFO:teuthology.orchestra.run.smithi156.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.4 MB/s | 59 kB 00:00 2023-09-13T04:27:38.390 INFO:teuthology.orchestra.run.smithi002.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6083.g4a 2.2 MB/s | 242 kB 00:00 2023-09-13T04:27:38.396 INFO:teuthology.orchestra.run.smithi156.stdout:(53/131): python3-logutils-0.3.5-11.el8.noarch. 449 kB/s | 49 kB 00:00 2023-09-13T04:27:38.422 INFO:teuthology.orchestra.run.smithi156.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2023-09-13T04:27:38.457 INFO:teuthology.orchestra.run.smithi002.stdout:(32/131): ceph-mgr-rook-18.0.0-6083.g4a754ac9.e 713 kB/s | 47 kB 00:00 2023-09-13T04:27:38.472 INFO:teuthology.orchestra.run.smithi156.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 311 kB/s | 16 kB 00:00 2023-09-13T04:27:38.497 INFO:teuthology.orchestra.run.smithi156.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.7 MB/s | 283 kB 00:00 2023-09-13T04:27:38.523 INFO:teuthology.orchestra.run.smithi156.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 673 kB/s | 34 kB 00:00 2023-09-13T04:27:38.540 INFO:teuthology.orchestra.run.smithi156.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 4.6 MB/s | 196 kB 00:00 2023-09-13T04:27:38.548 INFO:teuthology.orchestra.run.smithi002.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6083.g4 159 kB/s | 15 kB 00:00 2023-09-13T04:27:38.568 INFO:teuthology.orchestra.run.smithi149.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 890 kB/s | 185 kB 00:00 2023-09-13T04:27:38.590 INFO:teuthology.orchestra.run.smithi156.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 960 kB/s | 64 kB 00:00 2023-09-13T04:27:38.602 INFO:teuthology.orchestra.run.smithi149.stdout:(43/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 77 kB/s | 44 kB 00:00 2023-09-13T04:27:38.615 INFO:teuthology.orchestra.run.smithi156.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 248 kB/s | 19 kB 00:00 2023-09-13T04:27:38.641 INFO:teuthology.orchestra.run.smithi156.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 475 kB/s | 24 kB 00:00 2023-09-13T04:27:38.666 INFO:teuthology.orchestra.run.smithi156.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 583 kB/s | 29 kB 00:00 2023-09-13T04:27:38.674 INFO:teuthology.orchestra.run.smithi002.stdout:(34/131): ceph-volume-18.0.0-6083.g4a754ac9.el8 2.0 MB/s | 261 kB 00:00 2023-09-13T04:27:38.685 INFO:teuthology.orchestra.run.smithi149.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 282 kB/s | 33 kB 00:00 2023-09-13T04:27:38.691 INFO:teuthology.orchestra.run.smithi156.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 550 kB/s | 27 kB 00:00 2023-09-13T04:27:38.711 INFO:teuthology.orchestra.run.smithi149.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 150 kB/s | 16 kB 00:00 2023-09-13T04:27:38.716 INFO:teuthology.orchestra.run.smithi002.stdout:(35/131): ceph-mgr-dashboard-18.0.0-6083.g4a754 3.3 MB/s | 1.7 MB 00:00 2023-09-13T04:27:38.716 INFO:teuthology.orchestra.run.smithi156.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2023-09-13T04:27:38.741 INFO:teuthology.orchestra.run.smithi002.stdout:(36/131): cephadm-18.0.0-6083.g4a754ac9.el8.noa 2.6 MB/s | 179 kB 00:00 2023-09-13T04:27:38.742 INFO:teuthology.orchestra.run.smithi156.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 4.9 MB/s | 251 kB 00:00 2023-09-13T04:27:38.767 INFO:teuthology.orchestra.run.smithi156.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-09-13T04:27:38.784 INFO:teuthology.orchestra.run.smithi156.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 2.0 MB/s | 85 kB 00:00 2023-09-13T04:27:38.809 INFO:teuthology.orchestra.run.smithi156.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 548 kB/s | 23 kB 00:00 2023-09-13T04:27:38.835 INFO:teuthology.orchestra.run.smithi156.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-09-13T04:27:38.842 INFO:teuthology.orchestra.run.smithi002.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 12 MB/s | 7.4 MB 00:00 2023-09-13T04:27:38.859 INFO:teuthology.orchestra.run.smithi002.stdout:(38/131): cephfs-top-18.0.0-6083.g4a754ac9.el8. 126 kB/s | 18 kB 00:00 2023-09-13T04:27:38.893 INFO:teuthology.orchestra.run.smithi156.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 16 MB/s | 1.7 MB 00:00 2023-09-13T04:27:38.960 INFO:teuthology.orchestra.run.smithi156.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.6 MB/s | 110 kB 00:00 2023-09-13T04:27:38.978 INFO:teuthology.orchestra.run.smithi149.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 591 kB/s | 173 kB 00:00 2023-09-13T04:27:38.994 INFO:teuthology.orchestra.run.smithi156.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-09-13T04:27:39.044 INFO:teuthology.orchestra.run.smithi156.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-09-13T04:27:39.095 INFO:teuthology.orchestra.run.smithi156.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.4 MB/s | 323 kB 00:00 2023-09-13T04:27:39.109 INFO:teuthology.orchestra.run.smithi002.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 224 kB/s | 59 kB 00:00 2023-09-13T04:27:39.128 INFO:teuthology.orchestra.run.smithi156.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 1.1 MB/s | 39 kB 00:00 2023-09-13T04:27:39.134 INFO:teuthology.orchestra.run.smithi002.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 273 kB/s | 75 kB 00:00 2023-09-13T04:27:39.179 INFO:teuthology.orchestra.run.smithi156.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-09-13T04:27:39.201 INFO:teuthology.orchestra.run.smithi002.stdout:(41/131): gperftools-libs-2.7-9.el8.x86_64.rpm 667 kB/s | 306 kB 00:00 2023-09-13T04:27:39.212 INFO:teuthology.orchestra.run.smithi149.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 347 kB/s | 81 kB 00:00 2023-09-13T04:27:39.226 INFO:teuthology.orchestra.run.smithi002.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 474 kB/s | 44 kB 00:00 2023-09-13T04:27:39.270 INFO:teuthology.orchestra.run.smithi149.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 688 kB/s | 384 kB 00:00 2023-09-13T04:27:39.302 INFO:teuthology.orchestra.run.smithi002.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2023-09-13T04:27:39.312 INFO:teuthology.orchestra.run.smithi156.stdout:(77/131): python3-scipy-1.0.0-21.module+el8.5.0 29 MB/s | 14 MB 00:00 2023-09-13T04:27:39.320 INFO:teuthology.orchestra.run.smithi149.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 103 kB/s | 11 kB 00:00 2023-09-13T04:27:39.346 INFO:teuthology.orchestra.run.smithi156.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 3.1 MB/s | 538 kB 00:00 2023-09-13T04:27:39.360 INFO:teuthology.orchestra.run.smithi002.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 247 kB/s | 33 kB 00:00 2023-09-13T04:27:39.370 INFO:teuthology.orchestra.run.smithi149.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 175 kB/s | 18 kB 00:00 2023-09-13T04:27:39.371 INFO:teuthology.orchestra.run.smithi156.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-09-13T04:27:39.386 INFO:teuthology.orchestra.run.smithi002.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 195 kB/s | 16 kB 00:00 2023-09-13T04:27:39.396 INFO:teuthology.orchestra.run.smithi156.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-09-13T04:27:39.411 INFO:teuthology.orchestra.run.smithi002.stdout:(46/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.5 MB/s | 476 kB 00:00 2023-09-13T04:27:39.438 INFO:teuthology.orchestra.run.smithi156.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-09-13T04:27:39.481 INFO:teuthology.orchestra.run.smithi156.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 632 kB/s | 27 kB 00:00 2023-09-13T04:27:39.486 INFO:teuthology.orchestra.run.smithi002.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.1 MB/s | 81 kB 00:00 2023-09-13T04:27:39.496 INFO:teuthology.orchestra.run.smithi149.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 390 kB/s | 49 kB 00:00 2023-09-13T04:27:39.520 INFO:teuthology.orchestra.run.smithi002.stdout:(48/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.1 MB/s | 173 kB 00:00 2023-09-13T04:27:39.545 INFO:teuthology.orchestra.run.smithi002.stdout:(49/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.4 MB/s | 384 kB 00:00 2023-09-13T04:27:39.564 INFO:teuthology.orchestra.run.smithi156.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 3.8 MB/s | 320 kB 00:00 2023-09-13T04:27:39.570 INFO:teuthology.orchestra.run.smithi002.stdout:(50/131): python3-jaraco-6.2-6.el8.noarch.rpm 133 kB/s | 11 kB 00:00 2023-09-13T04:27:39.596 INFO:teuthology.orchestra.run.smithi002.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 233 kB/s | 18 kB 00:00 2023-09-13T04:27:39.604 INFO:teuthology.orchestra.run.smithi149.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 543 kB/s | 59 kB 00:00 2023-09-13T04:27:39.623 INFO:teuthology.orchestra.run.smithi156.stdout:(84/131): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2023-09-13T04:27:39.646 INFO:teuthology.orchestra.run.smithi002.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 649 kB/s | 49 kB 00:00 2023-09-13T04:27:39.648 INFO:teuthology.orchestra.run.smithi156.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 525 kB/s | 44 kB 00:00 2023-09-13T04:27:39.671 INFO:teuthology.orchestra.run.smithi002.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 782 kB/s | 59 kB 00:00 2023-09-13T04:27:39.699 INFO:teuthology.orchestra.run.smithi156.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 8.9 MB/s | 457 kB 00:00 2023-09-13T04:27:39.713 INFO:teuthology.orchestra.run.smithi002.stdout:(54/131): python3-kubernetes-11.0.0-6.el8.noarc 7.0 MB/s | 1.2 MB 00:00 2023-09-13T04:27:39.721 INFO:teuthology.orchestra.run.smithi149.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 516 kB/s | 60 kB 00:00 2023-09-13T04:27:39.772 INFO:teuthology.orchestra.run.smithi002.stdout:(55/131): python3-natsort-7.1.1-2.el8.noarch.rp 481 kB/s | 60 kB 00:00 2023-09-13T04:27:39.773 INFO:teuthology.orchestra.run.smithi156.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 25 MB/s | 3.7 MB 00:00 2023-09-13T04:27:39.797 INFO:teuthology.orchestra.run.smithi002.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 187 kB/s | 16 kB 00:00 2023-09-13T04:27:39.799 INFO:teuthology.orchestra.run.smithi156.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.2 MB/s | 225 kB 00:00 2023-09-13T04:27:39.816 INFO:teuthology.orchestra.run.smithi156.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2023-09-13T04:27:39.822 INFO:teuthology.orchestra.run.smithi002.stdout:(57/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.8 MB/s | 283 kB 00:00 2023-09-13T04:27:39.833 INFO:teuthology.orchestra.run.smithi156.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 3.7 MB/s | 126 kB 00:00 2023-09-13T04:27:39.848 INFO:teuthology.orchestra.run.smithi002.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 447 kB/s | 34 kB 00:00 2023-09-13T04:27:39.849 INFO:teuthology.orchestra.run.smithi156.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 595 kB/s | 20 kB 00:00 2023-09-13T04:27:39.875 INFO:teuthology.orchestra.run.smithi156.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.4 MB/s | 103 kB 00:00 2023-09-13T04:27:39.881 INFO:teuthology.orchestra.run.smithi002.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 2.3 MB/s | 196 kB 00:00 2023-09-13T04:27:39.907 INFO:teuthology.orchestra.run.smithi002.stdout:(60/131): python3-rsa-4.9-2.el8.noarch.rpm 767 kB/s | 64 kB 00:00 2023-09-13T04:27:39.909 INFO:teuthology.orchestra.run.smithi156.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-09-13T04:27:39.932 INFO:teuthology.orchestra.run.smithi002.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 223 kB/s | 19 kB 00:00 2023-09-13T04:27:39.951 INFO:teuthology.orchestra.run.smithi156.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.7 MB/s | 198 kB 00:00 2023-09-13T04:27:39.957 INFO:teuthology.orchestra.run.smithi002.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 316 kB/s | 24 kB 00:00 2023-09-13T04:27:39.983 INFO:teuthology.orchestra.run.smithi002.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 387 kB/s | 29 kB 00:00 2023-09-13T04:27:40.008 INFO:teuthology.orchestra.run.smithi002.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 365 kB/s | 27 kB 00:00 2023-09-13T04:27:40.050 INFO:teuthology.orchestra.run.smithi002.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 2.7 MB/s | 255 kB 00:00 2023-09-13T04:27:40.051 INFO:teuthology.orchestra.run.smithi156.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 28 MB/s | 4.9 MB 00:00 2023-09-13T04:27:40.075 INFO:teuthology.orchestra.run.smithi002.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.7 MB/s | 251 kB 00:00 2023-09-13T04:27:40.092 INFO:teuthology.orchestra.run.smithi002.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 733 kB/s | 61 kB 00:00 2023-09-13T04:27:40.109 INFO:teuthology.orchestra.run.smithi156.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 3.4 MB/s | 202 kB 00:00 2023-09-13T04:27:40.126 INFO:teuthology.orchestra.run.smithi002.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-09-13T04:27:40.176 INFO:teuthology.orchestra.run.smithi002.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 229 kB/s | 23 kB 00:00 2023-09-13T04:27:40.185 INFO:teuthology.orchestra.run.smithi156.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2023-09-13T04:27:40.201 INFO:teuthology.orchestra.run.smithi002.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 930 kB/s | 70 kB 00:00 2023-09-13T04:27:40.210 INFO:teuthology.orchestra.run.smithi156.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.4 MB/s | 354 kB 00:00 2023-09-13T04:27:40.255 INFO:teuthology.orchestra.run.smithi149.stdout:(54/131): ceph-test-18.0.0-6083.g4a754ac9.el8.x 4.5 MB/s | 49 MB 00:10 2023-09-13T04:27:40.268 INFO:teuthology.orchestra.run.smithi002.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 9.9 MB/s | 1.7 MB 00:00 2023-09-13T04:27:40.280 INFO:teuthology.orchestra.run.smithi149.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 506 kB/s | 283 kB 00:00 2023-09-13T04:27:40.285 INFO:teuthology.orchestra.run.smithi156.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 11 MB/s | 808 kB 00:00 2023-09-13T04:27:40.310 INFO:teuthology.orchestra.run.smithi002.stdout:(72/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 349 kB/s | 47 kB 00:00 2023-09-13T04:27:40.344 INFO:teuthology.orchestra.run.smithi002.stdout:(73/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.5 MB/s | 200 kB 00:00 2023-09-13T04:27:40.369 INFO:teuthology.orchestra.run.smithi002.stdout:(74/131): python3-jwt-1.6.1-2.el8.noarch.rpm 461 kB/s | 43 kB 00:00 2023-09-13T04:27:40.389 INFO:teuthology.orchestra.run.smithi149.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 311 kB/s | 34 kB 00:00 2023-09-13T04:27:40.394 INFO:teuthology.orchestra.run.smithi002.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 960 kB/s | 48 kB 00:00 2023-09-13T04:27:40.420 INFO:teuthology.orchestra.run.smithi002.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-09-13T04:27:40.427 INFO:teuthology.orchestra.run.smithi156.stdout:(100/131): postgresql-server-10.21-2.module+el8 21 MB/s | 5.1 MB 00:00 2023-09-13T04:27:40.445 INFO:teuthology.orchestra.run.smithi002.stdout:(77/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.7 MB/s | 220 kB 00:00 2023-09-13T04:27:40.470 INFO:teuthology.orchestra.run.smithi002.stdout:(78/131): python3-requests-oauthlib-1.0.0-1.el8 565 kB/s | 43 kB 00:00 2023-09-13T04:27:40.496 INFO:teuthology.orchestra.run.smithi002.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 1.4 MB/s | 109 kB 00:00 2023-09-13T04:27:40.521 INFO:teuthology.orchestra.run.smithi002.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.0 MB/s | 229 kB 00:00 2023-09-13T04:27:40.579 INFO:teuthology.orchestra.run.smithi002.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 224 kB/s | 19 kB 00:00 2023-09-13T04:27:40.613 INFO:teuthology.orchestra.run.smithi002.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 1.6 MB/s | 155 kB 00:00 2023-09-13T04:27:40.647 INFO:teuthology.orchestra.run.smithi002.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.1 MB/s | 560 kB 00:00 2023-09-13T04:27:40.672 INFO:teuthology.orchestra.run.smithi002.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 306 kB/s | 18 kB 00:00 2023-09-13T04:27:40.694 INFO:teuthology.orchestra.run.smithi156.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 3.7 MB/s | 1.5 MB 00:00 2023-09-13T04:27:40.705 INFO:teuthology.orchestra.run.smithi002.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 4.3 MB/s | 559 kB 00:00 2023-09-13T04:27:40.720 INFO:teuthology.orchestra.run.smithi156.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 164 kB/s | 47 kB 00:00 2023-09-13T04:27:40.739 INFO:teuthology.orchestra.run.smithi002.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-09-13T04:27:40.745 INFO:teuthology.orchestra.run.smithi156.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-09-13T04:27:40.771 INFO:teuthology.orchestra.run.smithi156.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 847 kB/s | 43 kB 00:00 2023-09-13T04:27:40.773 INFO:teuthology.orchestra.run.smithi002.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.7 MB/s | 605 kB 00:00 2023-09-13T04:27:40.795 INFO:teuthology.orchestra.run.smithi156.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-09-13T04:27:40.798 INFO:teuthology.orchestra.run.smithi002.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.2 MB/s | 72 kB 00:00 2023-09-13T04:27:40.821 INFO:teuthology.orchestra.run.smithi156.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 960 kB/s | 48 kB 00:00 2023-09-13T04:27:40.823 INFO:teuthology.orchestra.run.smithi002.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-09-13T04:27:40.846 INFO:teuthology.orchestra.run.smithi156.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-09-13T04:27:40.849 INFO:teuthology.orchestra.run.smithi002.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-09-13T04:27:40.858 INFO:teuthology.orchestra.run.smithi149.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 418 kB/s | 196 kB 00:00 2023-09-13T04:27:40.872 INFO:teuthology.orchestra.run.smithi156.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 852 kB/s | 43 kB 00:00 2023-09-13T04:27:40.874 INFO:teuthology.orchestra.run.smithi002.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-09-13T04:27:40.897 INFO:teuthology.orchestra.run.smithi156.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-09-13T04:27:40.899 INFO:teuthology.orchestra.run.smithi002.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-09-13T04:27:40.922 INFO:teuthology.orchestra.run.smithi156.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-09-13T04:27:40.924 INFO:teuthology.orchestra.run.smithi002.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 769 kB/s | 39 kB 00:00 2023-09-13T04:27:40.956 INFO:teuthology.orchestra.run.smithi156.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-09-13T04:27:40.981 INFO:teuthology.orchestra.run.smithi156.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 320 kB/s | 19 kB 00:00 2023-09-13T04:27:41.000 INFO:teuthology.orchestra.run.smithi002.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 6.3 MB/s | 644 kB 00:00 2023-09-13T04:27:41.007 INFO:teuthology.orchestra.run.smithi156.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-09-13T04:27:41.033 INFO:teuthology.orchestra.run.smithi002.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 1.5 MB/s | 171 kB 00:00 2023-09-13T04:27:41.040 INFO:teuthology.orchestra.run.smithi156.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 9.3 MB/s | 559 kB 00:00 2023-09-13T04:27:41.059 INFO:teuthology.orchestra.run.smithi002.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.3 MB/s | 138 kB 00:00 2023-09-13T04:27:41.066 INFO:teuthology.orchestra.run.smithi156.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 306 kB/s | 18 kB 00:00 2023-09-13T04:27:41.099 INFO:teuthology.orchestra.run.smithi156.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-09-13T04:27:41.103 INFO:teuthology.orchestra.run.smithi149.stdout:(58/131): python3-portend-2.6-1.el8.noarch.rpm 18 kB/s | 16 kB 00:00 2023-09-13T04:27:41.124 INFO:teuthology.orchestra.run.smithi156.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-09-13T04:27:41.134 INFO:teuthology.orchestra.run.smithi002.stdout:(97/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 5.4 MB/s | 2.3 MB 00:00 2023-09-13T04:27:41.159 INFO:teuthology.orchestra.run.smithi002.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.1 MB/s | 110 kB 00:00 2023-09-13T04:27:41.166 INFO:teuthology.orchestra.run.smithi156.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.7 MB/s | 72 kB 00:00 2023-09-13T04:27:41.206 INFO:teuthology.orchestra.run.smithi149.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 185 kB/s | 64 kB 00:00 2023-09-13T04:27:41.209 INFO:teuthology.orchestra.run.smithi002.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-09-13T04:27:41.217 INFO:teuthology.orchestra.run.smithi156.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 20 MB/s | 2.3 MB 00:00 2023-09-13T04:27:41.234 INFO:teuthology.orchestra.run.smithi156.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.2 MB/s | 83 kB 00:00 2023-09-13T04:27:41.240 INFO:teuthology.orchestra.run.smithi149.stdout:(60/131): python3-kubernetes-11.0.0-6.el8.noarc 622 kB/s | 1.2 MB 00:01 2023-09-13T04:27:41.243 INFO:teuthology.orchestra.run.smithi002.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.0 MB/s | 259 kB 00:00 2023-09-13T04:27:41.259 INFO:teuthology.orchestra.run.smithi156.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.4 MB/s | 60 kB 00:00 2023-09-13T04:27:41.277 INFO:teuthology.orchestra.run.smithi002.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-09-13T04:27:41.284 INFO:teuthology.orchestra.run.smithi156.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-09-13T04:27:41.301 INFO:teuthology.orchestra.run.smithi156.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.4 MB/s | 58 kB 00:00 2023-09-13T04:27:41.302 INFO:teuthology.orchestra.run.smithi002.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 661 kB/s | 39 kB 00:00 2023-09-13T04:27:41.307 INFO:teuthology.orchestra.run.smithi149.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 92 kB/s | 19 kB 00:00 2023-09-13T04:27:41.318 INFO:teuthology.orchestra.run.smithi156.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 1.1 MB/s | 39 kB 00:00 2023-09-13T04:27:41.332 INFO:teuthology.orchestra.run.smithi149.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 190 kB/s | 24 kB 00:00 2023-09-13T04:27:41.357 INFO:teuthology.orchestra.run.smithi149.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 249 kB/s | 29 kB 00:00 2023-09-13T04:27:41.368 INFO:teuthology.orchestra.run.smithi156.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 9.4 MB/s | 644 kB 00:00 2023-09-13T04:27:41.374 INFO:teuthology.orchestra.run.smithi002.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 7.0 MB/s | 688 kB 00:00 2023-09-13T04:27:41.394 INFO:teuthology.orchestra.run.smithi156.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.2 MB/s | 171 kB 00:00 2023-09-13T04:27:41.416 INFO:teuthology.orchestra.run.smithi149.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 253 kB/s | 27 kB 00:00 2023-09-13T04:27:41.419 INFO:teuthology.orchestra.run.smithi156.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-09-13T04:27:41.420 INFO:teuthology.orchestra.run.smithi002.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 4.5 MB/s | 538 kB 00:00 2023-09-13T04:27:41.445 INFO:teuthology.orchestra.run.smithi002.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.2 MB/s | 157 kB 00:00 2023-09-13T04:27:41.479 INFO:teuthology.orchestra.run.smithi002.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2023-09-13T04:27:41.520 INFO:teuthology.orchestra.run.smithi002.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 9.3 MB/s | 713 kB 00:00 2023-09-13T04:27:41.554 INFO:teuthology.orchestra.run.smithi002.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 798 kB/s | 27 kB 00:00 2023-09-13T04:27:41.604 INFO:teuthology.orchestra.run.smithi002.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-09-13T04:27:41.616 INFO:teuthology.orchestra.run.smithi149.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 306 kB/s | 61 kB 00:00 2023-09-13T04:27:41.646 INFO:teuthology.orchestra.run.smithi002.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-09-13T04:27:41.708 INFO:teuthology.orchestra.run.smithi149.stdout:(66/131): python3-waitress-1.4.3-1.el8.noarch.r 680 kB/s | 255 kB 00:00 2023-09-13T04:27:41.733 INFO:teuthology.orchestra.run.smithi149.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 668 kB/s | 251 kB 00:00 2023-09-13T04:27:41.796 INFO:teuthology.orchestra.run.smithi002.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 15 MB/s | 4.8 MB 00:00 2023-09-13T04:27:41.817 INFO:teuthology.orchestra.run.smithi149.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 426 kB/s | 85 kB 00:00 2023-09-13T04:27:41.851 INFO:teuthology.orchestra.run.smithi149.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 161 kB/s | 23 kB 00:00 2023-09-13T04:27:41.863 INFO:teuthology.orchestra.run.smithi002.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 6.7 MB/s | 457 kB 00:00 2023-09-13T04:27:41.876 INFO:teuthology.orchestra.run.smithi149.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 70 kB 00:00 2023-09-13T04:27:41.913 INFO:teuthology.orchestra.run.smithi002.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 4.4 MB/s | 225 kB 00:00 2023-09-13T04:27:41.928 INFO:teuthology.orchestra.run.smithi156.stdout:(128/131): ceph-test-18.0.0-6083.g4a754ac9.el8. 4.0 MB/s | 49 MB 00:12 2023-09-13T04:27:41.943 INFO:teuthology.orchestra.run.smithi149.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 510 kB/s | 47 kB 00:00 2023-09-13T04:27:41.972 INFO:teuthology.orchestra.run.smithi002.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 11 MB/s | 3.7 MB 00:00 2023-09-13T04:27:41.976 INFO:teuthology.orchestra.run.smithi149.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.1 MB/s | 200 kB 00:00 2023-09-13T04:27:41.986 INFO:teuthology.orchestra.run.smithi156.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-09-13T04:27:41.998 INFO:teuthology.orchestra.run.smithi002.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 765 kB/s | 64 kB 00:00 2023-09-13T04:27:42.002 INFO:teuthology.orchestra.run.smithi149.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 730 kB/s | 43 kB 00:00 2023-09-13T04:27:42.027 INFO:teuthology.orchestra.run.smithi149.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-09-13T04:27:42.048 INFO:teuthology.orchestra.run.smithi002.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.6 MB/s | 126 kB 00:00 2023-09-13T04:27:42.052 INFO:teuthology.orchestra.run.smithi149.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 958 kB/s | 48 kB 00:00 2023-09-13T04:27:42.073 INFO:teuthology.orchestra.run.smithi002.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 266 kB/s | 20 kB 00:00 2023-09-13T04:27:42.099 INFO:teuthology.orchestra.run.smithi002.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-09-13T04:27:42.103 INFO:teuthology.orchestra.run.smithi149.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.1 MB/s | 87 kB 00:00 2023-09-13T04:27:42.145 INFO:teuthology.orchestra.run.smithi149.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 465 kB/s | 43 kB 00:00 2023-09-13T04:27:42.170 INFO:teuthology.orchestra.run.smithi149.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 1.6 MB/s | 109 kB 00:00 2023-09-13T04:27:42.187 INFO:teuthology.orchestra.run.smithi149.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 5.3 MB/s | 229 kB 00:00 2023-09-13T04:27:42.221 INFO:teuthology.orchestra.run.smithi149.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 11 MB/s | 560 kB 00:00 2023-09-13T04:27:42.246 INFO:teuthology.orchestra.run.smithi149.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 319 kB/s | 19 kB 00:00 2023-09-13T04:27:42.271 INFO:teuthology.orchestra.run.smithi149.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-09-13T04:27:42.274 INFO:teuthology.orchestra.run.smithi002.stdout:(119/131): python3-scipy-1.0.0-21.module+el8.5. 11 MB/s | 14 MB 00:01 2023-09-13T04:27:42.305 INFO:teuthology.orchestra.run.smithi149.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 9.4 MB/s | 559 kB 00:00 2023-09-13T04:27:42.316 INFO:teuthology.orchestra.run.smithi002.stdout:(120/131): libtool-2.4.6-25.el8.x86_64.rpm 2.9 MB/s | 709 kB 00:00 2023-09-13T04:27:42.330 INFO:teuthology.orchestra.run.smithi149.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 306 kB/s | 18 kB 00:00 2023-09-13T04:27:42.337 INFO:teuthology.orchestra.run.smithi156.stdout:(130/131): librados2-18.0.0-6083.g4a754ac9.el8. 4.0 MB/s | 3.7 MB 00:00 2023-09-13T04:27:42.349 INFO:teuthology.orchestra.run.smithi002.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 2.6 MB/s | 198 kB 00:00 2023-09-13T04:27:42.363 INFO:teuthology.orchestra.run.smithi149.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-09-13T04:27:42.389 INFO:teuthology.orchestra.run.smithi149.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-09-13T04:27:42.393 INFO:teuthology.orchestra.run.smithi002.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-09-13T04:27:42.431 INFO:teuthology.orchestra.run.smithi149.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-09-13T04:27:42.442 INFO:teuthology.orchestra.run.smithi002.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-09-13T04:27:42.470 INFO:teuthology.orchestra.run.smithi156.stdout:(131/131): librbd1-18.0.0-6083.g4a754ac9.el8.x8 3.5 MB/s | 3.7 MB 00:01 2023-09-13T04:27:42.473 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2023-09-13T04:27:42.473 INFO:teuthology.orchestra.run.smithi156.stdout:Total 9.9 MB/s | 208 MB 00:21 2023-09-13T04:27:42.481 INFO:teuthology.orchestra.run.smithi149.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 20 MB/s | 2.3 MB 00:00 2023-09-13T04:27:42.506 INFO:teuthology.orchestra.run.smithi149.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-09-13T04:27:42.523 INFO:teuthology.orchestra.run.smithi149.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.4 MB/s | 60 kB 00:00 2023-09-13T04:27:42.548 INFO:teuthology.orchestra.run.smithi149.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 2.3 MB/s | 98 kB 00:00 2023-09-13T04:27:42.574 INFO:teuthology.orchestra.run.smithi149.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-09-13T04:27:42.584 INFO:teuthology.orchestra.run.smithi002.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 10 MB/s | 4.9 MB 00:00 2023-09-13T04:27:42.599 INFO:teuthology.orchestra.run.smithi149.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 767 kB/s | 39 kB 00:00 2023-09-13T04:27:42.641 INFO:teuthology.orchestra.run.smithi149.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 9.4 MB/s | 644 kB 00:00 2023-09-13T04:27:42.666 INFO:teuthology.orchestra.run.smithi149.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-09-13T04:27:42.675 INFO:teuthology.orchestra.run.smithi002.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 13 MB/s | 4.7 MB 00:00 2023-09-13T04:27:42.691 INFO:teuthology.orchestra.run.smithi149.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-09-13T04:27:42.717 INFO:teuthology.orchestra.run.smithi002.stdout:(126/131): postgresql-contrib-10.21-2.module+el 6.0 MB/s | 808 kB 00:00 2023-09-13T04:27:42.742 INFO:teuthology.orchestra.run.smithi149.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.2 MB/s | 110 kB 00:00 2023-09-13T04:27:42.792 INFO:teuthology.orchestra.run.smithi149.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-09-13T04:27:42.834 INFO:teuthology.orchestra.run.smithi002.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 9.5 MB/s | 1.5 MB 00:00 2023-09-13T04:27:42.842 INFO:teuthology.orchestra.run.smithi149.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.0 MB/s | 259 kB 00:00 2023-09-13T04:27:42.893 INFO:teuthology.orchestra.run.smithi149.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-09-13T04:27:42.927 INFO:teuthology.orchestra.run.smithi002.stdout:(128/131): postgresql-server-10.21-2.module+el8 10 MB/s | 5.1 MB 00:00 2023-09-13T04:27:42.935 INFO:teuthology.orchestra.run.smithi149.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 931 kB/s | 39 kB 00:00 2023-09-13T04:27:43.018 INFO:teuthology.orchestra.run.smithi002.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 8.5 MB/s | 798 kB 00:00 2023-09-13T04:27:43.085 INFO:teuthology.orchestra.run.smithi149.stdout:(102/131): python3-scipy-1.0.0-21.module+el8.5. 33 MB/s | 14 MB 00:00 2023-09-13T04:27:43.119 INFO:teuthology.orchestra.run.smithi149.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 3.7 MB/s | 688 kB 00:00 2023-09-13T04:27:43.161 INFO:teuthology.orchestra.run.smithi149.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.0 MB/s | 538 kB 00:00 2023-09-13T04:27:43.185 INFO:teuthology.orchestra.run.smithi002.stdout:(130/131): librbd1-18.0.0-6083.g4a754ac9.el8.x8 11 MB/s | 3.7 MB 00:00 2023-09-13T04:27:43.186 INFO:teuthology.orchestra.run.smithi149.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-09-13T04:27:43.203 INFO:teuthology.orchestra.run.smithi149.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-09-13T04:27:43.236 INFO:teuthology.orchestra.run.smithi149.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2023-09-13T04:27:43.285 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2023-09-13T04:27:43.295 INFO:teuthology.orchestra.run.smithi149.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 455 kB/s | 27 kB 00:00 2023-09-13T04:27:43.311 INFO:teuthology.orchestra.run.smithi002.stdout:(131/131): librados2-18.0.0-6083.g4a754ac9.el8. 6.3 MB/s | 3.7 MB 00:00 2023-09-13T04:27:43.313 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2023-09-13T04:27:43.313 INFO:teuthology.orchestra.run.smithi002.stdout:Total 13 MB/s | 208 MB 00:15 2023-09-13T04:27:43.370 INFO:teuthology.orchestra.run.smithi149.stdout:(109/131): python3-babel-2.5.1-7.el8.noarch.rpm 29 MB/s | 4.8 MB 00:00 2023-09-13T04:27:43.395 INFO:teuthology.orchestra.run.smithi149.stdout:(110/131): flex-2.6.1-9.el8.x86_64.rpm 3.1 MB/s | 320 kB 00:00 2023-09-13T04:27:43.405 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2023-09-13T04:27:43.405 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2023-09-13T04:27:43.420 INFO:teuthology.orchestra.run.smithi149.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 876 kB/s | 44 kB 00:00 2023-09-13T04:27:43.479 INFO:teuthology.orchestra.run.smithi149.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 7.7 MB/s | 457 kB 00:00 2023-09-13T04:27:43.530 INFO:teuthology.orchestra.run.smithi149.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 4.4 MB/s | 225 kB 00:00 2023-09-13T04:27:43.580 INFO:teuthology.orchestra.run.smithi149.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 20 MB/s | 3.7 MB 00:00 2023-09-13T04:27:43.605 INFO:teuthology.orchestra.run.smithi149.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 850 kB/s | 64 kB 00:00 2023-09-13T04:27:43.631 INFO:teuthology.orchestra.run.smithi149.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-09-13T04:27:43.656 INFO:teuthology.orchestra.run.smithi149.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 398 kB/s | 20 kB 00:00 2023-09-13T04:27:43.672 INFO:teuthology.orchestra.run.smithi149.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2023-09-13T04:27:43.706 INFO:teuthology.orchestra.run.smithi149.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 14 MB/s | 709 kB 00:00 2023-09-13T04:27:43.748 INFO:teuthology.orchestra.run.smithi149.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-09-13T04:27:43.840 INFO:teuthology.orchestra.run.smithi149.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 29 MB/s | 4.9 MB 00:00 2023-09-13T04:27:43.899 INFO:teuthology.orchestra.run.smithi149.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 3.4 MB/s | 202 kB 00:00 2023-09-13T04:27:43.982 INFO:teuthology.orchestra.run.smithi149.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2023-09-13T04:27:44.007 INFO:teuthology.orchestra.run.smithi149.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.2 MB/s | 354 kB 00:00 2023-09-13T04:27:44.058 INFO:teuthology.orchestra.run.smithi149.stdout:(125/131): postgresql-contrib-10.21-2.module+el 16 MB/s | 808 kB 00:00 2023-09-13T04:27:44.100 INFO:teuthology.orchestra.run.smithi149.stdout:(126/131): thrift-0.13.0-2.el8.x86_64.rpm 751 kB/s | 1.7 MB 00:02 2023-09-13T04:27:44.175 INFO:teuthology.orchestra.run.smithi149.stdout:(127/131): postgresql-server-10.21-2.module+el8 26 MB/s | 5.1 MB 00:00 2023-09-13T04:27:44.217 INFO:teuthology.orchestra.run.smithi149.stdout:(128/131): postgresql-10.21-2.module+el8.6.0+15 9.5 MB/s | 1.5 MB 00:00 2023-09-13T04:27:44.276 INFO:teuthology.orchestra.run.smithi149.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-09-13T04:27:44.533 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2023-09-13T04:27:44.537 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2023-09-13T04:27:44.559 INFO:teuthology.orchestra.run.smithi149.stdout:(130/131): librados2-18.0.0-6083.g4a754ac9.el8. 8.1 MB/s | 3.7 MB 00:00 2023-09-13T04:27:44.726 INFO:teuthology.orchestra.run.smithi149.stdout:(131/131): librbd1-18.0.0-6083.g4a754ac9.el8.x8 6.7 MB/s | 3.7 MB 00:00 2023-09-13T04:27:44.730 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2023-09-13T04:27:44.731 INFO:teuthology.orchestra.run.smithi149.stdout:Total 9.1 MB/s | 208 MB 00:22 2023-09-13T04:27:44.989 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2023-09-13T04:27:45.137 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2023-09-13T04:27:45.137 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2023-09-13T04:27:45.543 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2023-09-13T04:27:45.663 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2023-09-13T04:27:45.663 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2023-09-13T04:27:46.805 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2023-09-13T04:27:46.809 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2023-09-13T04:27:46.812 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2023-09-13T04:27:46.818 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2023-09-13T04:27:49.439 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2023-09-13T04:27:49.621 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-13T04:27:49.786 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-13T04:27:50.021 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-09-13T04:27:50.223 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-13T04:27:50.327 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-13T04:27:50.556 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-13T04:27:50.965 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-13T04:27:51.398 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librados2-2:18.0.0-6083.g4a754ac9.el8.x86_64 6/134 2023-09-13T04:27:51.635 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2023-09-13T04:27:51.799 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-13T04:27:51.812 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-2:18.0.0-6083.g4a754ac9.el8.x86_64 6/134 2023-09-13T04:27:52.031 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librbd1-2:18.0.0-6083.g4a754ac9.el8.x86_64 7/134 2023-09-13T04:27:52.065 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-13T04:27:52.150 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-2:18.0.0-6083.g4a754ac9.el8.x86_64 7/134 2023-09-13T04:27:52.217 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-13T04:27:52.314 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-13T04:27:52.323 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs2-2:18.0.0-6083.g4a754ac9.el8.x86_64 8/134 2023-09-13T04:27:52.396 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2023-09-13T04:27:52.403 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephfs2-2:18.0.0-6083.g4a754ac9.el8.x86_64 8/134 2023-09-13T04:27:52.542 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-13T04:27:52.591 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-13T04:27:52.607 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rados-2:18.0.0-6083.g4a754ac9.el8.x86_64 9/134 2023-09-13T04:27:52.810 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-13T04:27:52.845 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-13T04:27:52.937 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-13T04:27:53.067 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-13T04:27:53.150 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librados2-2:18.0.0-6083.g4a754ac9.el8.x86_64 5/134 2023-09-13T04:27:53.349 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-13T04:27:53.463 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-13T04:27:53.495 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-13T04:27:53.557 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:18.0.0-6083.g4a754ac9.el8.x86_64 5/134 2023-09-13T04:27:53.691 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-13T04:27:53.766 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-13T04:27:53.800 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librbd1-2:18.0.0-6083.g4a754ac9.el8.x86_64 6/134 2023-09-13T04:27:53.830 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephsqlite-2:18.0.0-6083.g4a754ac9.el8.x86_64 14/134 2023-09-13T04:27:53.930 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:18.0.0-6083.g4a754ac9.el8.x86_64 6/134 2023-09-13T04:27:53.936 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6083.g4a754ac9.el8.x86_64 14/134 2023-09-13T04:27:54.081 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libradosstriper1-2:18.0.0-6083.g4a754ac9.el8.x86 15/134 2023-09-13T04:27:54.100 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs2-2:18.0.0-6083.g4a754ac9.el8.x86_64 7/134 2023-09-13T04:27:54.172 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6083.g4a754ac9.el8.x86 15/134 2023-09-13T04:27:54.197 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephfs2-2:18.0.0-6083.g4a754ac9.el8.x86_64 7/134 2023-09-13T04:27:54.260 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-13T04:27:54.297 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-13T04:27:54.347 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rados-2:18.0.0-6083.g4a754ac9.el8.x86_64 8/134 2023-09-13T04:27:54.432 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-13T04:27:54.482 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : librados2-2:18.0.0-6083.g4a754ac9.el8.x86_64 5/134 2023-09-13T04:27:54.517 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-09-13T04:27:54.564 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-13T04:27:54.788 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-13T04:27:54.995 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librados2-2:18.0.0-6083.g4a754ac9.el8.x86_64 5/134 2023-09-13T04:27:55.003 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-13T04:27:55.165 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-13T04:27:55.174 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : librbd1-2:18.0.0-6083.g4a754ac9.el8.x86_64 6/134 2023-09-13T04:27:55.344 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librbd1-2:18.0.0-6083.g4a754ac9.el8.x86_64 6/134 2023-09-13T04:27:55.355 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-13T04:27:55.388 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-13T04:27:55.524 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephfs2-2:18.0.0-6083.g4a754ac9.el8.x86_64 7/134 2023-09-13T04:27:55.647 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libcephfs2-2:18.0.0-6083.g4a754ac9.el8.x86_64 7/134 2023-09-13T04:27:55.682 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-13T04:27:55.825 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephsqlite-2:18.0.0-6083.g4a754ac9.el8.x86_64 14/134 2023-09-13T04:27:55.847 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rados-2:18.0.0-6083.g4a754ac9.el8.x86_64 8/134 2023-09-13T04:27:55.931 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6083.g4a754ac9.el8.x86_64 14/134 2023-09-13T04:27:55.939 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-13T04:27:56.046 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-09-13T04:27:56.075 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libradosstriper1-2:18.0.0-6083.g4a754ac9.el8.x86 15/134 2023-09-13T04:27:56.155 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-09-13T04:27:56.231 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6083.g4a754ac9.el8.x86 15/134 2023-09-13T04:27:56.361 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-13T04:27:56.462 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-09-13T04:27:56.614 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-13T04:27:56.622 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-09-13T04:27:56.800 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-09-13T04:27:56.845 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-09-13T04:27:56.958 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-13T04:27:57.067 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-13T04:27:57.089 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-13T04:27:57.103 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-09-13T04:27:57.250 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-13T04:27:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-13T04:27:57.324 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-09-13T04:27:57.482 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephsqlite-2:18.0.0-6083.g4a754ac9.el8.x86_64 14/134 2023-09-13T04:27:57.576 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-09-13T04:27:57.617 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6083.g4a754ac9.el8.x86_64 14/134 2023-09-13T04:27:57.803 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-09-13T04:27:57.823 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libradosstriper1-2:18.0.0-6083.g4a754ac9.el8.x86 15/134 2023-09-13T04:27:57.894 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-13T04:27:58.014 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6083.g4a754ac9.el8.x86 15/134 2023-09-13T04:27:58.014 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-09-13T04:27:58.075 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-13T04:27:58.268 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-13T04:27:58.340 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-09-13T04:27:58.429 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-common-2:18.0.0-6083.g4a754ac9.el8. 26/134 2023-09-13T04:27:58.550 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-argparse-2:18.0.0-6083.g4a754ac9.el 27/134 2023-09-13T04:27:58.551 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-09-13T04:27:58.652 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-13T04:27:58.781 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-09-13T04:27:58.841 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-09-13T04:27:58.940 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-13T04:27:59.123 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-13T04:27:59.290 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-13T04:27:59.442 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cephfs-2:18.0.0-6083.g4a754ac9.el8.x86_6 28/134 2023-09-13T04:27:59.526 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-09-13T04:27:59.755 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-09-13T04:27:59.761 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-09-13T04:27:59.956 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-09-13T04:28:00.041 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-09-13T04:28:00.054 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-13T04:28:00.226 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-13T04:28:00.265 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-13T04:28:00.307 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-09-13T04:28:00.397 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-common-2:18.0.0-6083.g4a754ac9.el8. 26/134 2023-09-13T04:28:00.504 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-13T04:28:00.519 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-argparse-2:18.0.0-6083.g4a754ac9.el 27/134 2023-09-13T04:28:00.737 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cephfs-2:18.0.0-6083.g4a754ac9.el8.x86_6 28/134 2023-09-13T04:28:00.970 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-13T04:28:00.991 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-13T04:28:01.148 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-09-13T04:28:02.035 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-09-13T04:28:02.268 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-13T04:28:02.339 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-09-13T04:28:02.580 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-09-13T04:28:02.614 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-09-13T04:28:02.759 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-13T04:28:02.833 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-13T04:28:02.906 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-13T04:28:03.025 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-13T04:28:03.123 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ceph-common-2:18.0.0-6083.g4a754ac9.el8. 26/134 2023-09-13T04:28:03.210 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rbd-2:18.0.0-6083.g4a754ac9.el8.x86_64 36/134 2023-09-13T04:28:03.288 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ceph-argparse-2:18.0.0-6083.g4a754ac9.el 27/134 2023-09-13T04:28:03.375 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librados-devel-2:18.0.0-6083.g4a754ac9.el8.x86_6 37/134 2023-09-13T04:28:03.573 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cephfs-2:18.0.0-6083.g4a754ac9.el8.x86_6 28/134 2023-09-13T04:28:03.578 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-09-13T04:28:03.748 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-09-13T04:28:03.781 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-13T04:28:03.785 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-09-13T04:28:03.932 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-09-13T04:28:04.079 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-09-13T04:28:04.083 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-09-13T04:28:04.337 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-09-13T04:28:04.359 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-09-13T04:28:04.569 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-09-13T04:28:04.614 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-09-13T04:28:04.818 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-13T04:28:04.946 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-09-13T04:28:05.009 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-13T04:28:05.179 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rbd-2:18.0.0-6083.g4a754ac9.el8.x86_64 36/134 2023-09-13T04:28:05.232 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-09-13T04:28:05.338 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librados-devel-2:18.0.0-6083.g4a754ac9.el8.x86_6 37/134 2023-09-13T04:28:05.439 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-09-13T04:28:05.497 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-09-13T04:28:05.624 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-09-13T04:28:05.650 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-09-13T04:28:05.806 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-09-13T04:28:05.863 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-09-13T04:28:05.883 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-09-13T04:28:05.993 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-13T04:28:06.086 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-09-13T04:28:06.196 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-09-13T04:28:06.319 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-09-13T04:28:06.453 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-09-13T04:28:06.473 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-13T04:28:06.549 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-13T04:28:06.654 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-09-13T04:28:06.753 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-09-13T04:28:06.875 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-09-13T04:28:06.927 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-09-13T04:28:07.033 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-09-13T04:28:07.097 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-09-13T04:28:07.190 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-09-13T04:28:07.220 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-09-13T04:28:07.283 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-09-13T04:28:07.451 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-09-13T04:28:07.536 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-09-13T04:28:07.602 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-09-13T04:28:07.623 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-09-13T04:28:07.814 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-09-13T04:28:07.821 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-09-13T04:28:07.893 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-09-13T04:28:08.021 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-09-13T04:28:08.067 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-09-13T04:28:08.146 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-09-13T04:28:08.293 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-09-13T04:28:08.404 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-13T04:28:08.413 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-09-13T04:28:08.485 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-09-13T04:28:08.606 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-13T04:28:08.654 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-09-13T04:28:08.724 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-09-13T04:28:08.828 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-09-13T04:28:08.839 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rbd-2:18.0.0-6083.g4a754ac9.el8.x86_64 36/134 2023-09-13T04:28:08.906 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-09-13T04:28:09.009 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-09-13T04:28:09.030 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librados-devel-2:18.0.0-6083.g4a754ac9.el8.x86_6 37/134 2023-09-13T04:28:09.161 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-09-13T04:28:09.229 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-09-13T04:28:09.240 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-09-13T04:28:09.374 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-09-13T04:28:09.418 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-09-13T04:28:09.444 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-09-13T04:28:09.528 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-13T04:28:09.593 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-13T04:28:09.646 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-09-13T04:28:09.743 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-09-13T04:28:09.772 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-09-13T04:28:09.825 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-09-13T04:28:10.014 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-09-13T04:28:10.038 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-09-13T04:28:10.052 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-09-13T04:28:10.178 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-13T04:28:10.206 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-09-13T04:28:10.305 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-09-13T04:28:10.401 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-13T04:28:10.443 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-09-13T04:28:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-13T04:28:10.579 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-13T04:28:10.609 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-09-13T04:28:10.652 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-13T04:28:10.817 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-09-13T04:28:10.845 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-09-13T04:28:10.894 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-13T04:28:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-09-13T04:28:11.033 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-13T04:28:11.120 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-09-13T04:28:11.181 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-13T04:28:11.232 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-09-13T04:28:11.410 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-09-13T04:28:11.414 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-13T04:28:11.418 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-09-13T04:28:11.642 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-13T04:28:11.651 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-09-13T04:28:11.652 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-09-13T04:28:11.813 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-09-13T04:28:11.843 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-13T04:28:11.932 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-09-13T04:28:12.031 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-13T04:28:12.082 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-09-13T04:28:12.097 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-13T04:28:12.209 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-13T04:28:12.258 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-13T04:28:12.315 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-09-13T04:28:12.476 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-13T04:28:12.732 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-13T04:28:12.765 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-09-13T04:28:12.796 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-13T04:28:12.900 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-13T04:28:13.013 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-13T04:28:13.071 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-09-13T04:28:13.159 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-13T04:28:13.160 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-13T04:28:13.317 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-13T04:28:13.330 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-09-13T04:28:13.377 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-13T04:28:13.549 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-13T04:28:13.561 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-13T04:28:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-09-13T04:28:13.760 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-13T04:28:13.770 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-13T04:28:13.925 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-09-13T04:28:13.933 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-13T04:28:14.092 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-13T04:28:14.236 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-13T04:28:14.308 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-09-13T04:28:14.485 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-13T04:28:14.581 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-09-13T04:28:14.751 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-13T04:28:14.816 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-09-13T04:28:14.862 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-13T04:28:14.944 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-13T04:28:15.058 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-09-13T04:28:15.194 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-13T04:28:15.206 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-13T04:28:15.288 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-09-13T04:28:15.452 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-13T04:28:15.455 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-13T04:28:15.461 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-09-13T04:28:15.630 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-13T04:28:15.770 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-13T04:28:15.830 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-13T04:28:15.853 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-09-13T04:28:16.071 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-13T04:28:16.201 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-09-13T04:28:16.274 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-13T04:28:16.508 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-09-13T04:28:16.741 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-09-13T04:28:16.917 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-13T04:28:16.948 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-13T04:28:17.090 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-13T04:28:17.139 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-13T04:28:17.217 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-13T04:28:17.310 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librgw2-2:18.0.0-6083.g4a754ac9.el8.x86_64 92/134 2023-09-13T04:28:17.379 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librgw2-2:18.0.0-6083.g4a754ac9.el8.x86_64 92/134 2023-09-13T04:28:17.454 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-13T04:28:17.464 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-13T04:28:17.569 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rgw-2:18.0.0-6083.g4a754ac9.el8.x86_64 93/134 2023-09-13T04:28:17.713 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-13T04:28:17.762 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-13T04:28:17.884 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-13T04:28:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-13T04:28:18.110 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-13T04:28:18.299 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-13T04:28:18.309 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-13T04:28:18.548 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-13T04:28:19.104 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-13T04:28:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-13T04:28:19.271 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librgw2-2:18.0.0-6083.g4a754ac9.el8.x86_64 92/134 2023-09-13T04:28:19.317 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-13T04:28:19.355 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librgw2-2:18.0.0-6083.g4a754ac9.el8.x86_64 92/134 2023-09-13T04:28:19.495 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-13T04:28:19.539 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rgw-2:18.0.0-6083.g4a754ac9.el8.x86_64 93/134 2023-09-13T04:28:19.743 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-13T04:28:20.002 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-13T04:28:20.219 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-13T04:28:20.503 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-13T04:28:20.747 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-13T04:28:20.925 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-13T04:28:21.194 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-13T04:28:21.294 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:18.0.0-6083.g4a754ac9.el8.x86_64 94/134 2023-09-13T04:28:21.560 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-common-2:18.0.0-6083.g4a754ac9.el8.x86_64 94/134 2023-09-13T04:28:22.156 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:18.0.0-6083.g4a754ac9.el8.x86_64 94/134 2023-09-13T04:28:22.658 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-13T04:28:22.959 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-13T04:28:22.975 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-base-2:18.0.0-6083.g4a754ac9.el8.x86_64 95/134 2023-09-13T04:28:23.069 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-base-2:18.0.0-6083.g4a754ac9.el8.x86_64 95/134 2023-09-13T04:28:23.209 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-selinux-2:18.0.0-6083.g4a754ac9.el8.x86_64 96/134 2023-09-13T04:28:23.269 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-13T04:28:23.300 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:18.0.0-6083.g4a754ac9.el8.x86_64 94/134 2023-09-13T04:28:23.612 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-common-2:18.0.0-6083.g4a754ac9.el8.x86_64 94/134 2023-09-13T04:28:23.670 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-13T04:28:24.016 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-13T04:28:24.212 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:18.0.0-6083.g4a754ac9.el8.x86_64 94/134 2023-09-13T04:28:24.241 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-13T04:28:24.494 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-base-2:18.0.0-6083.g4a754ac9.el8.x86_64 95/134 2023-09-13T04:28:24.573 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-base-2:18.0.0-6083.g4a754ac9.el8.x86_64 95/134 2023-09-13T04:28:24.737 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-selinux-2:18.0.0-6083.g4a754ac9.el8.x86_64 96/134 2023-09-13T04:28:25.329 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-13T04:28:25.536 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librgw2-2:18.0.0-6083.g4a754ac9.el8.x86_64 92/134 2023-09-13T04:28:25.628 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librgw2-2:18.0.0-6083.g4a754ac9.el8.x86_64 92/134 2023-09-13T04:28:25.803 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rgw-2:18.0.0-6083.g4a754ac9.el8.x86_64 93/134 2023-09-13T04:28:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-common-2:18.0.0-6083.g4a754ac9.el8.x86_64 94/134 2023-09-13T04:28:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-common-2:18.0.0-6083.g4a754ac9.el8.x86_64 94/134 2023-09-13T04:28:32.034 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-common-2:18.0.0-6083.g4a754ac9.el8.x86_64 94/134 2023-09-13T04:28:32.341 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-base-2:18.0.0-6083.g4a754ac9.el8.x86_64 95/134 2023-09-13T04:28:32.502 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-base-2:18.0.0-6083.g4a754ac9.el8.x86_64 95/134 2023-09-13T04:28:32.751 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-selinux-2:18.0.0-6083.g4a754ac9.el8.x86_64 96/134 2023-09-13T04:28:42.047 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6083.g4a754ac9.el8.x86_64 96/134 2023-09-13T04:28:42.048 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /sys 2023-09-13T04:28:42.048 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /proc 2023-09-13T04:28:42.048 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /dev 2023-09-13T04:28:42.048 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /run 2023-09-13T04:28:42.049 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /mnt 2023-09-13T04:28:42.049 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /var/tmp 2023-09-13T04:28:42.049 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /home 2023-09-13T04:28:42.049 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /tmp 2023-09-13T04:28:42.049 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /dev 2023-09-13T04:28:42.050 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-13T04:28:42.398 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-volume-2:18.0.0-6083.g4a754ac9.el8.noarch 97/134 2023-09-13T04:28:43.719 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6083.g4a754ac9.el8.x86_64 96/134 2023-09-13T04:28:43.719 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /sys 2023-09-13T04:28:43.720 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /proc 2023-09-13T04:28:43.720 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /dev 2023-09-13T04:28:43.720 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /run 2023-09-13T04:28:43.720 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /mnt 2023-09-13T04:28:43.720 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /var/tmp 2023-09-13T04:28:43.721 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /home 2023-09-13T04:28:43.721 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /tmp 2023-09-13T04:28:43.721 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /dev 2023-09-13T04:28:43.721 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-09-13T04:28:43.999 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-volume-2:18.0.0-6083.g4a754ac9.el8.noarch 97/134 2023-09-13T04:28:44.059 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-volume-2:18.0.0-6083.g4a754ac9.el8.noarch 97/134 2023-09-13T04:28:44.163 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-osd-2:18.0.0-6083.g4a754ac9.el8.x86_64 98/134 2023-09-13T04:28:44.383 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-osd-2:18.0.0-6083.g4a754ac9.el8.x86_64 98/134 2023-09-13T04:28:44.556 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mds-2:18.0.0-6083.g4a754ac9.el8.x86_64 99/134 2023-09-13T04:28:45.011 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mds-2:18.0.0-6083.g4a754ac9.el8.x86_64 99/134 2023-09-13T04:28:45.156 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mon-2:18.0.0-6083.g4a754ac9.el8.x86_64 100/134 2023-09-13T04:28:45.173 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mon-2:18.0.0-6083.g4a754ac9.el8.x86_64 100/134 2023-09-13T04:28:45.646 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-volume-2:18.0.0-6083.g4a754ac9.el8.noarch 97/134 2023-09-13T04:28:45.791 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-osd-2:18.0.0-6083.g4a754ac9.el8.x86_64 98/134 2023-09-13T04:28:46.011 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-osd-2:18.0.0-6083.g4a754ac9.el8.x86_64 98/134 2023-09-13T04:28:46.225 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mds-2:18.0.0-6083.g4a754ac9.el8.x86_64 99/134 2023-09-13T04:28:46.680 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mds-2:18.0.0-6083.g4a754ac9.el8.x86_64 99/134 2023-09-13T04:28:46.797 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephadm-2:18.0.0-6083.g4a754ac9.el8.noarch 101/134 2023-09-13T04:28:46.843 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mon-2:18.0.0-6083.g4a754ac9.el8.x86_64 100/134 2023-09-13T04:28:46.860 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mon-2:18.0.0-6083.g4a754ac9.el8.x86_64 100/134 2023-09-13T04:28:47.007 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephadm-2:18.0.0-6083.g4a754ac9.el8.noarch 101/134 2023-09-13T04:28:47.162 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6083.g4a754ac9.e 102/134 2023-09-13T04:28:47.381 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6083.g4a754ac9. 103/134 2023-09-13T04:28:47.563 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6083.g4a754ac9.el8.noa 104/134 2023-09-13T04:28:47.782 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6083.g4a754ac9.el8.noa 104/134 2023-09-13T04:28:48.029 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6083.g4a754ac9.el8.n 105/134 2023-09-13T04:28:48.739 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephadm-2:18.0.0-6083.g4a754ac9.el8.noarch 101/134 2023-09-13T04:28:48.968 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephadm-2:18.0.0-6083.g4a754ac9.el8.noarch 101/134 2023-09-13T04:28:48.989 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6083.g4a754ac9.el8.n 105/134 2023-09-13T04:28:49.106 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6083.g4a754ac9.e 102/134 2023-09-13T04:28:49.188 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6083.g4a7 106/134 2023-09-13T04:28:49.279 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6083.g4a7 106/134 2023-09-13T04:28:49.300 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6083.g4a754ac9. 103/134 2023-09-13T04:28:49.475 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6083.g4a754ac9.el8.noa 104/134 2023-09-13T04:28:49.620 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6083.g4a754ac9.el 107/134 2023-09-13T04:28:49.707 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6083.g4a754ac9.el8.noa 104/134 2023-09-13T04:28:49.833 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-2:18.0.0-6083.g4a754ac9.el8.x86_64 108/134 2023-09-13T04:28:49.869 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6083.g4a754ac9.el8.x86_64 108/134 2023-09-13T04:28:50.024 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6083.g4a754ac9.el8.n 105/134 2023-09-13T04:28:50.030 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-rook-2:18.0.0-6083.g4a754ac9.el8.noarch 109/134 2023-09-13T04:28:50.041 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6083.g4a754ac9.el8.noarch 109/134 2023-09-13T04:28:50.985 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6083.g4a754ac9.el8.n 105/134 2023-09-13T04:28:51.175 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6083.g4a7 106/134 2023-09-13T04:28:51.265 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6083.g4a7 106/134 2023-09-13T04:28:51.373 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-2:18.0.0-6083.g4a754ac9.el8.x86_64 110/134 2023-09-13T04:28:51.612 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-radosgw-2:18.0.0-6083.g4a754ac9.el8.x86_64 111/134 2023-09-13T04:28:51.624 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6083.g4a754ac9.el 107/134 2023-09-13T04:28:51.646 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6083.g4a754ac9.el8.x86_64 111/134 2023-09-13T04:28:51.805 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6083.g4a754 112/134 2023-09-13T04:28:51.832 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-2:18.0.0-6083.g4a754ac9.el8.x86_64 108/134 2023-09-13T04:28:51.847 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6083.g4a754 112/134 2023-09-13T04:28:51.866 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6083.g4a754ac9.el8.x86_64 108/134 2023-09-13T04:28:52.022 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-mirror-2:18.0.0-6083.g4a754ac9.el8.x86_64 113/134 2023-09-13T04:28:52.041 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-rook-2:18.0.0-6083.g4a754ac9.el8.noarch 109/134 2023-09-13T04:28:52.052 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6083.g4a754ac9.el8.noarch 109/134 2023-09-13T04:28:52.381 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6083.g4a754ac9.el8.x86_64 113/134 2023-09-13T04:28:52.571 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-mirror-2:18.0.0-6083.g4a754ac9.el8.x86_64 114/134 2023-09-13T04:28:53.378 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-2:18.0.0-6083.g4a754ac9.el8.x86_64 110/134 2023-09-13T04:28:53.579 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-radosgw-2:18.0.0-6083.g4a754ac9.el8.x86_64 111/134 2023-09-13T04:28:53.614 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6083.g4a754ac9.el8.x86_64 111/134 2023-09-13T04:28:53.766 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6083.g4a754 112/134 2023-09-13T04:28:53.807 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6083.g4a754 112/134 2023-09-13T04:28:53.974 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-mirror-2:18.0.0-6083.g4a754ac9.el8.x86_64 113/134 2023-09-13T04:28:54.262 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6083.g4a754ac9.el8.x86_64 96/134 2023-09-13T04:28:54.262 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /sys 2023-09-13T04:28:54.263 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /proc 2023-09-13T04:28:54.263 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /dev 2023-09-13T04:28:54.263 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /run 2023-09-13T04:28:54.263 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /mnt 2023-09-13T04:28:54.263 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /var/tmp 2023-09-13T04:28:54.263 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /home 2023-09-13T04:28:54.264 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /tmp 2023-09-13T04:28:54.264 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /dev 2023-09-13T04:28:54.264 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:28:54.333 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6083.g4a754ac9.el8.x86_64 113/134 2023-09-13T04:28:54.474 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-mirror-2:18.0.0-6083.g4a754ac9.el8.x86_64 114/134 2023-09-13T04:28:54.707 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-volume-2:18.0.0-6083.g4a754ac9.el8.noarch 97/134 2023-09-13T04:28:56.576 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-volume-2:18.0.0-6083.g4a754ac9.el8.noarch 97/134 2023-09-13T04:28:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-osd-2:18.0.0-6083.g4a754ac9.el8.x86_64 98/134 2023-09-13T04:28:57.057 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-osd-2:18.0.0-6083.g4a754ac9.el8.x86_64 98/134 2023-09-13T04:28:57.187 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6083.g4a754ac9.el8.x86_64 114/134 2023-09-13T04:28:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mds-2:18.0.0-6083.g4a754ac9.el8.x86_64 99/134 2023-09-13T04:28:57.516 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-test-2:18.0.0-6083.g4a754ac9.el8.x86_64 115/134 2023-09-13T04:28:57.645 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-09-13T04:28:57.792 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mds-2:18.0.0-6083.g4a754ac9.el8.x86_64 99/134 2023-09-13T04:28:57.836 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-09-13T04:28:57.998 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mon-2:18.0.0-6083.g4a754ac9.el8.x86_64 100/134 2023-09-13T04:28:58.040 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mon-2:18.0.0-6083.g4a754ac9.el8.x86_64 100/134 2023-09-13T04:28:58.715 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-09-13T04:28:59.040 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-09-13T04:28:59.088 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6083.g4a754ac9.el8.x86_64 114/134 2023-09-13T04:28:59.215 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-fuse-2:18.0.0-6083.g4a754ac9.el8.x86_64 119/134 2023-09-13T04:28:59.408 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-13T04:28:59.574 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs-devel-2:18.0.0-6083.g4a754ac9.el8.x86_ 121/134 2023-09-13T04:28:59.730 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-13T04:28:59.966 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-test-2:18.0.0-6083.g4a754ac9.el8.x86_64 115/134 2023-09-13T04:29:00.276 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-09-13T04:29:00.451 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-fuse-2:18.0.0-6083.g4a754ac9.el8.x86_64 117/134 2023-09-13T04:29:00.729 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-09-13T04:29:00.881 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-09-13T04:29:00.974 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: cephadm-2:18.0.0-6083.g4a754ac9.el8.noarch 101/134 2023-09-13T04:29:01.071 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-09-13T04:29:01.201 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephadm-2:18.0.0-6083.g4a754ac9.el8.noarch 101/134 2023-09-13T04:29:01.294 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-09-13T04:29:01.345 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6083.g4a754ac9.e 102/134 2023-09-13T04:29:01.502 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs-devel-2:18.0.0-6083.g4a754ac9.el8.x86_ 121/134 2023-09-13T04:29:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6083.g4a754ac9. 103/134 2023-09-13T04:29:01.649 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-13T04:29:01.866 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6083.g4a754ac9.el8.noa 104/134 2023-09-13T04:29:01.932 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-13T04:29:02.159 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6083.g4a754ac9.el8.noa 104/134 2023-09-13T04:29:02.203 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-13T04:29:02.222 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-13T04:29:02.387 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-top-2:18.0.0-6083.g4a754ac9.el8.noarch 124/134 2023-09-13T04:29:02.419 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6083.g4a754ac9.el8.n 105/134 2023-09-13T04:29:02.678 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-fuse-2:18.0.0-6083.g4a754ac9.el8.x86_64 125/134 2023-09-13T04:29:02.992 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-nbd-2:18.0.0-6083.g4a754ac9.el8.x86_64 126/134 2023-09-13T04:29:03.313 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-13T04:29:03.436 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-13T04:29:03.445 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-13T04:29:03.605 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6083.g4a754ac9.el8.n 105/134 2023-09-13T04:29:03.630 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-09-13T04:29:03.819 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-09-13T04:29:03.821 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6083.g4a7 106/134 2023-09-13T04:29:03.891 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-13T04:29:03.898 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-09-13T04:29:03.956 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6083.g4a7 106/134 2023-09-13T04:29:04.103 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-09-13T04:29:04.115 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-09-13T04:29:04.115 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-13T04:29:04.198 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-13T04:29:04.215 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-13T04:29:04.289 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-13T04:29:04.289 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-13T04:29:04.353 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6083.g4a754ac9.el 107/134 2023-09-13T04:29:04.450 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-top-2:18.0.0-6083.g4a754ac9.el8.noarch 124/134 2023-09-13T04:29:04.525 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-13T04:29:04.580 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-2:18.0.0-6083.g4a754ac9.el8.x86_64 108/134 2023-09-13T04:29:04.647 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-fuse-2:18.0.0-6083.g4a754ac9.el8.x86_64 125/134 2023-09-13T04:29:04.647 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6083.g4a754ac9.el8.x86_64 108/134 2023-09-13T04:29:04.735 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-13T04:29:04.880 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-rook-2:18.0.0-6083.g4a754ac9.el8.noarch 109/134 2023-09-13T04:29:04.909 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6083.g4a754ac9.el8.noarch 109/134 2023-09-13T04:29:04.948 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-nbd-2:18.0.0-6083.g4a754ac9.el8.x86_64 126/134 2023-09-13T04:29:05.228 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-13T04:29:05.229 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-2:18.0.0-6083.g4a754ac9.el8.x86_64 1/134 2023-09-13T04:29:05.229 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-base-2:18.0.0-6083.g4a754ac9.el8.x86_64 2/134 2023-09-13T04:29:05.229 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-common-2:18.0.0-6083.g4a754ac9.el8.x86_64 3/134 2023-09-13T04:29:05.229 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-fuse-2:18.0.0-6083.g4a754ac9.el8.x86_64 4/134 2023-09-13T04:29:05.230 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6083.g4a754 5/134 2023-09-13T04:29:05.230 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mds-2:18.0.0-6083.g4a754ac9.el8.x86_64 6/134 2023-09-13T04:29:05.230 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-2:18.0.0-6083.g4a754ac9.el8.x86_64 7/134 2023-09-13T04:29:05.230 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mon-2:18.0.0-6083.g4a754ac9.el8.x86_64 8/134 2023-09-13T04:29:05.230 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-osd-2:18.0.0-6083.g4a754ac9.el8.x86_64 9/134 2023-09-13T04:29:05.231 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-radosgw-2:18.0.0-6083.g4a754ac9.el8.x86_64 10/134 2023-09-13T04:29:05.231 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-selinux-2:18.0.0-6083.g4a754ac9.el8.x86_64 11/134 2023-09-13T04:29:05.231 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-test-2:18.0.0-6083.g4a754ac9.el8.x86_64 12/134 2023-09-13T04:29:05.231 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-mirror-2:18.0.0-6083.g4a754ac9.el8.x86_64 13/134 2023-09-13T04:29:05.231 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs-devel-2:18.0.0-6083.g4a754ac9.el8.x86_ 14/134 2023-09-13T04:29:05.232 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs2-2:18.0.0-6083.g4a754ac9.el8.x86_64 15/134 2023-09-13T04:29:05.232 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephsqlite-2:18.0.0-6083.g4a754ac9.el8.x86_64 16/134 2023-09-13T04:29:05.232 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados-devel-2:18.0.0-6083.g4a754ac9.el8.x86_6 17/134 2023-09-13T04:29:05.232 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libradosstriper1-2:18.0.0-6083.g4a754ac9.el8.x86 18/134 2023-09-13T04:29:05.232 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librgw2-2:18.0.0-6083.g4a754ac9.el8.x86_64 19/134 2023-09-13T04:29:05.233 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6083.g4a754ac9.el 20/134 2023-09-13T04:29:05.233 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-common-2:18.0.0-6083.g4a754ac9.el8. 21/134 2023-09-13T04:29:05.233 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cephfs-2:18.0.0-6083.g4a754ac9.el8.x86_6 22/134 2023-09-13T04:29:05.233 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rados-2:18.0.0-6083.g4a754ac9.el8.x86_64 23/134 2023-09-13T04:29:05.233 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rbd-2:18.0.0-6083.g4a754ac9.el8.x86_64 24/134 2023-09-13T04:29:05.234 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rgw-2:18.0.0-6083.g4a754ac9.el8.x86_64 25/134 2023-09-13T04:29:05.243 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-fuse-2:18.0.0-6083.g4a754ac9.el8.x86_64 26/134 2023-09-13T04:29:05.243 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-mirror-2:18.0.0-6083.g4a754ac9.el8.x86_64 27/134 2023-09-13T04:29:05.244 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-nbd-2:18.0.0-6083.g4a754ac9.el8.x86_64 28/134 2023-09-13T04:29:05.244 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6083.g4a754ac9. 29/134 2023-09-13T04:29:05.244 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6083.g4a754ac9.el8.noa 30/134 2023-09-13T04:29:05.244 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6083.g4a754ac9.el8.n 31/134 2023-09-13T04:29:05.244 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6083.g4a7 32/134 2023-09-13T04:29:05.245 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6083.g4a754ac9.el 33/134 2023-09-13T04:29:05.245 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6083.g4a754ac9.el8.noarch 34/134 2023-09-13T04:29:05.245 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6083.g4a754ac9.e 35/134 2023-09-13T04:29:05.245 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-volume-2:18.0.0-6083.g4a754ac9.el8.noarch 36/134 2023-09-13T04:29:05.245 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephadm-2:18.0.0-6083.g4a754ac9.el8.noarch 37/134 2023-09-13T04:29:05.246 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-top-2:18.0.0-6083.g4a754ac9.el8.noarch 38/134 2023-09-13T04:29:05.246 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-13T04:29:05.246 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-13T04:29:05.246 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-13T04:29:05.246 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-13T04:29:05.247 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-13T04:29:05.247 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-13T04:29:05.247 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-13T04:29:05.247 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-13T04:29:05.247 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-13T04:29:05.248 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-13T04:29:05.248 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-13T04:29:05.248 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-13T04:29:05.248 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-13T04:29:05.248 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-13T04:29:05.249 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-13T04:29:05.249 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-13T04:29:05.249 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-13T04:29:05.249 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-13T04:29:05.249 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-13T04:29:05.250 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-13T04:29:05.250 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-13T04:29:05.250 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-13T04:29:05.250 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-13T04:29:05.250 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-13T04:29:05.251 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-13T04:29:05.251 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-13T04:29:05.251 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-13T04:29:05.251 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-13T04:29:05.251 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-13T04:29:05.252 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-13T04:29:05.252 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-13T04:29:05.252 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-13T04:29:05.252 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-13T04:29:05.252 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-09-13T04:29:05.252 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-09-13T04:29:05.253 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-09-13T04:29:05.253 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-09-13T04:29:05.253 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-09-13T04:29:05.253 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-09-13T04:29:05.253 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-09-13T04:29:05.254 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-09-13T04:29:05.254 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-09-13T04:29:05.254 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-09-13T04:29:05.254 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-09-13T04:29:05.254 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-09-13T04:29:05.255 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-09-13T04:29:05.255 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-09-13T04:29:05.255 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-09-13T04:29:05.255 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-09-13T04:29:05.255 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-09-13T04:29:05.256 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-09-13T04:29:05.256 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-09-13T04:29:05.256 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-09-13T04:29:05.256 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-09-13T04:29:05.256 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-09-13T04:29:05.256 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-09-13T04:29:05.257 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-09-13T04:29:05.257 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-09-13T04:29:05.257 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-09-13T04:29:05.257 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-09-13T04:29:05.257 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-09-13T04:29:05.258 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-09-13T04:29:05.258 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-09-13T04:29:05.258 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-09-13T04:29:05.258 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-09-13T04:29:05.258 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-09-13T04:29:05.258 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-09-13T04:29:05.259 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-09-13T04:29:05.259 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-09-13T04:29:05.259 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-09-13T04:29:05.259 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-09-13T04:29:05.259 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-09-13T04:29:05.260 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-09-13T04:29:05.260 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-09-13T04:29:05.260 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-09-13T04:29:05.260 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-09-13T04:29:05.260 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-09-13T04:29:05.261 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-09-13T04:29:05.261 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-09-13T04:29:05.261 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-09-13T04:29:05.261 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-09-13T04:29:05.261 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-13T04:29:05.262 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-09-13T04:29:05.262 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-13T04:29:05.262 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-09-13T04:29:05.262 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-09-13T04:29:05.262 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-09-13T04:29:05.263 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-09-13T04:29:05.263 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-09-13T04:29:05.263 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-13T04:29:05.263 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-2:18.0.0-6083.g4a754ac9.el8.x86_64 129/134 2023-09-13T04:29:05.263 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-13T04:29:05.264 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-2:18.0.0-6083.g4a754ac9.el8.x86_64 131/134 2023-09-13T04:29:05.264 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-13T04:29:05.264 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-13T04:29:05.315 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-13T04:29:05.489 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-09-13T04:29:05.570 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-09-13T04:29:05.772 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-09-13T04:29:05.784 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-09-13T04:29:05.981 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-13T04:29:05.988 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-13T04:29:06.124 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-09-13T04:29:06.125 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-13T04:29:06.325 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-13T04:29:06.326 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-13T04:29:06.419 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-2:18.0.0-6083.g4a754ac9.el8.x86_64 110/134 2023-09-13T04:29:06.569 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-13T04:29:06.624 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-radosgw-2:18.0.0-6083.g4a754ac9.el8.x86_64 111/134 2023-09-13T04:29:06.688 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6083.g4a754ac9.el8.x86_64 111/134 2023-09-13T04:29:06.788 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-13T04:29:06.864 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6083.g4a754 112/134 2023-09-13T04:29:06.937 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6083.g4a754 112/134 2023-09-13T04:29:07.155 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephfs-mirror-2:18.0.0-6083.g4a754ac9.el8.x86_64 113/134 2023-09-13T04:29:07.301 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-13T04:29:07.301 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-2:18.0.0-6083.g4a754ac9.el8.x86_64 1/134 2023-09-13T04:29:07.302 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-base-2:18.0.0-6083.g4a754ac9.el8.x86_64 2/134 2023-09-13T04:29:07.302 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-common-2:18.0.0-6083.g4a754ac9.el8.x86_64 3/134 2023-09-13T04:29:07.302 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-fuse-2:18.0.0-6083.g4a754ac9.el8.x86_64 4/134 2023-09-13T04:29:07.302 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6083.g4a754 5/134 2023-09-13T04:29:07.302 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mds-2:18.0.0-6083.g4a754ac9.el8.x86_64 6/134 2023-09-13T04:29:07.303 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-2:18.0.0-6083.g4a754ac9.el8.x86_64 7/134 2023-09-13T04:29:07.303 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mon-2:18.0.0-6083.g4a754ac9.el8.x86_64 8/134 2023-09-13T04:29:07.303 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-osd-2:18.0.0-6083.g4a754ac9.el8.x86_64 9/134 2023-09-13T04:29:07.303 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-radosgw-2:18.0.0-6083.g4a754ac9.el8.x86_64 10/134 2023-09-13T04:29:07.303 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-selinux-2:18.0.0-6083.g4a754ac9.el8.x86_64 11/134 2023-09-13T04:29:07.304 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-test-2:18.0.0-6083.g4a754ac9.el8.x86_64 12/134 2023-09-13T04:29:07.304 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-mirror-2:18.0.0-6083.g4a754ac9.el8.x86_64 13/134 2023-09-13T04:29:07.304 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs-devel-2:18.0.0-6083.g4a754ac9.el8.x86_ 14/134 2023-09-13T04:29:07.304 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs2-2:18.0.0-6083.g4a754ac9.el8.x86_64 15/134 2023-09-13T04:29:07.304 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephsqlite-2:18.0.0-6083.g4a754ac9.el8.x86_64 16/134 2023-09-13T04:29:07.305 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados-devel-2:18.0.0-6083.g4a754ac9.el8.x86_6 17/134 2023-09-13T04:29:07.305 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libradosstriper1-2:18.0.0-6083.g4a754ac9.el8.x86 18/134 2023-09-13T04:29:07.305 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librgw2-2:18.0.0-6083.g4a754ac9.el8.x86_64 19/134 2023-09-13T04:29:07.305 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6083.g4a754ac9.el 20/134 2023-09-13T04:29:07.305 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-common-2:18.0.0-6083.g4a754ac9.el8. 21/134 2023-09-13T04:29:07.305 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cephfs-2:18.0.0-6083.g4a754ac9.el8.x86_6 22/134 2023-09-13T04:29:07.306 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rados-2:18.0.0-6083.g4a754ac9.el8.x86_64 23/134 2023-09-13T04:29:07.306 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rbd-2:18.0.0-6083.g4a754ac9.el8.x86_64 24/134 2023-09-13T04:29:07.306 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rgw-2:18.0.0-6083.g4a754ac9.el8.x86_64 25/134 2023-09-13T04:29:07.306 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-fuse-2:18.0.0-6083.g4a754ac9.el8.x86_64 26/134 2023-09-13T04:29:07.306 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-mirror-2:18.0.0-6083.g4a754ac9.el8.x86_64 27/134 2023-09-13T04:29:07.307 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-nbd-2:18.0.0-6083.g4a754ac9.el8.x86_64 28/134 2023-09-13T04:29:07.315 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6083.g4a754ac9. 29/134 2023-09-13T04:29:07.316 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6083.g4a754ac9.el8.noa 30/134 2023-09-13T04:29:07.316 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6083.g4a754ac9.el8.n 31/134 2023-09-13T04:29:07.316 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6083.g4a7 32/134 2023-09-13T04:29:07.316 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6083.g4a754ac9.el 33/134 2023-09-13T04:29:07.317 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6083.g4a754ac9.el8.noarch 34/134 2023-09-13T04:29:07.317 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6083.g4a754ac9.e 35/134 2023-09-13T04:29:07.317 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-volume-2:18.0.0-6083.g4a754ac9.el8.noarch 36/134 2023-09-13T04:29:07.317 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephadm-2:18.0.0-6083.g4a754ac9.el8.noarch 37/134 2023-09-13T04:29:07.317 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-top-2:18.0.0-6083.g4a754ac9.el8.noarch 38/134 2023-09-13T04:29:07.318 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-13T04:29:07.318 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-13T04:29:07.318 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-13T04:29:07.318 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-13T04:29:07.318 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-13T04:29:07.319 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-13T04:29:07.319 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-13T04:29:07.319 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-13T04:29:07.319 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-13T04:29:07.319 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-13T04:29:07.320 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-13T04:29:07.320 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-13T04:29:07.320 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-13T04:29:07.320 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-13T04:29:07.320 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-13T04:29:07.321 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-13T04:29:07.321 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-13T04:29:07.321 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-13T04:29:07.321 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-13T04:29:07.321 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-13T04:29:07.322 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-13T04:29:07.322 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-13T04:29:07.322 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-13T04:29:07.322 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-13T04:29:07.322 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-13T04:29:07.323 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-13T04:29:07.323 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-13T04:29:07.323 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-13T04:29:07.323 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-13T04:29:07.323 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-13T04:29:07.324 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-13T04:29:07.324 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-13T04:29:07.324 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-13T04:29:07.324 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-09-13T04:29:07.324 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-09-13T04:29:07.324 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-09-13T04:29:07.325 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-09-13T04:29:07.325 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-09-13T04:29:07.325 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-09-13T04:29:07.325 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-09-13T04:29:07.325 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-09-13T04:29:07.326 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-09-13T04:29:07.326 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-09-13T04:29:07.326 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-09-13T04:29:07.326 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-09-13T04:29:07.326 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-09-13T04:29:07.327 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-09-13T04:29:07.327 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-09-13T04:29:07.327 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-09-13T04:29:07.327 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-09-13T04:29:07.327 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-09-13T04:29:07.328 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-09-13T04:29:07.328 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-09-13T04:29:07.328 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-09-13T04:29:07.328 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-09-13T04:29:07.328 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-09-13T04:29:07.329 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-09-13T04:29:07.329 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-09-13T04:29:07.329 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-09-13T04:29:07.329 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-09-13T04:29:07.329 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-09-13T04:29:07.329 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-09-13T04:29:07.330 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-09-13T04:29:07.330 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-09-13T04:29:07.330 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-09-13T04:29:07.330 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-09-13T04:29:07.330 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-09-13T04:29:07.331 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-09-13T04:29:07.331 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-09-13T04:29:07.331 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-09-13T04:29:07.331 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-09-13T04:29:07.331 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-09-13T04:29:07.332 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-09-13T04:29:07.332 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-09-13T04:29:07.332 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-09-13T04:29:07.332 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-09-13T04:29:07.332 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-09-13T04:29:07.333 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-09-13T04:29:07.333 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-09-13T04:29:07.333 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-09-13T04:29:07.333 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-09-13T04:29:07.333 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-09-13T04:29:07.333 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-09-13T04:29:07.334 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-09-13T04:29:07.334 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-09-13T04:29:07.334 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-09-13T04:29:07.334 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-09-13T04:29:07.335 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-09-13T04:29:07.335 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-09-13T04:29:07.335 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-09-13T04:29:07.335 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:18.0.0-6083.g4a754ac9.el8.x86_64 129/134 2023-09-13T04:29:07.335 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-13T04:29:07.335 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:18.0.0-6083.g4a754ac9.el8.x86_64 131/134 2023-09-13T04:29:07.336 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-13T04:29:07.336 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-13T04:29:07.606 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6083.g4a754ac9.el8.x86_64 113/134 2023-09-13T04:29:07.645 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-13T04:29:07.646 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:01:57 ago on Wed 13 Sep 2023 04:27:10 AM UTC. 2023-09-13T04:29:07.780 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-mirror-2:18.0.0-6083.g4a754ac9.el8.x86_64 114/134 2023-09-13T04:29:09.765 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-13T04:29:09.765 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:59 ago on Wed 13 Sep 2023 04:27:10 AM UTC. 2023-09-13T04:29:10.353 INFO:teuthology.orchestra.run.smithi156.stdout:Installed products updated. 2023-09-13T04:29:10.564 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-13T04:29:10.564 INFO:teuthology.orchestra.run.smithi156.stdout:Upgraded: 2023-09-13T04:29:10.564 INFO:teuthology.orchestra.run.smithi156.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-13T04:29:10.565 INFO:teuthology.orchestra.run.smithi156.stdout: librados2-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.565 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.565 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2023-09-13T04:29:10.566 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-13T04:29:10.566 INFO:teuthology.orchestra.run.smithi156.stdout: automake-1.16.1-7.el8.noarch 2023-09-13T04:29:10.567 INFO:teuthology.orchestra.run.smithi156.stdout: bison-3.0.4-10.el8.x86_64 2023-09-13T04:29:10.567 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.567 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.568 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.568 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.568 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:10.569 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.569 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.569 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.570 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:10.570 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:10.570 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:10.571 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:10.571 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:10.571 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.572 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.572 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:10.572 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.573 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.573 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.573 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:10.574 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:10.574 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.574 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:10.575 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-13T04:29:10.575 INFO:teuthology.orchestra.run.smithi156.stdout: flex-2.6.1-9.el8.x86_64 2023-09-13T04:29:10.575 INFO:teuthology.orchestra.run.smithi156.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-13T04:29:10.576 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-13T04:29:10.576 INFO:teuthology.orchestra.run.smithi156.stdout: jq-1.6-3.el8.x86_64 2023-09-13T04:29:10.576 INFO:teuthology.orchestra.run.smithi156.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-13T04:29:10.577 INFO:teuthology.orchestra.run.smithi156.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-13T04:29:10.577 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-13T04:29:10.577 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.578 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.578 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.578 INFO:teuthology.orchestra.run.smithi156.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-13T04:29:10.579 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-13T04:29:10.579 INFO:teuthology.orchestra.run.smithi156.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-13T04:29:10.579 INFO:teuthology.orchestra.run.smithi156.stdout: libpq-13.5-1.el8.x86_64 2023-09-13T04:29:10.580 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-13T04:29:10.580 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-13T04:29:10.581 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.581 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.581 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-13T04:29:10.582 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.582 INFO:teuthology.orchestra.run.smithi156.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-13T04:29:10.582 INFO:teuthology.orchestra.run.smithi156.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-13T04:29:10.583 INFO:teuthology.orchestra.run.smithi156.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-13T04:29:10.583 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-13T04:29:10.583 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-13T04:29:10.584 INFO:teuthology.orchestra.run.smithi156.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-13T04:29:10.584 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-13T04:29:10.584 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-13T04:29:10.585 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-13T04:29:10.585 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-13T04:29:10.585 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-13T04:29:10.585 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-13T04:29:10.586 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-13T04:29:10.586 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-13T04:29:10.587 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-13T04:29:10.587 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-13T04:29:10.587 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-13T04:29:10.588 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-13T04:29:10.588 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-13T04:29:10.588 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-13T04:29:10.589 INFO:teuthology.orchestra.run.smithi156.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-13T04:29:10.589 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-13T04:29:10.589 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.589 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.590 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.590 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-13T04:29:10.590 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-13T04:29:10.590 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-13T04:29:10.590 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-13T04:29:10.591 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-13T04:29:10.591 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-13T04:29:10.591 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-13T04:29:10.591 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-13T04:29:10.591 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-13T04:29:10.592 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-13T04:29:10.592 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-13T04:29:10.592 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-13T04:29:10.592 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-13T04:29:10.592 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-13T04:29:10.592 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-13T04:29:10.593 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-13T04:29:10.593 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-13T04:29:10.593 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-13T04:29:10.593 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-13T04:29:10.593 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-13T04:29:10.594 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-13T04:29:10.594 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-13T04:29:10.594 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend-2.6-1.el8.noarch 2023-09-13T04:29:10.594 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-13T04:29:10.594 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-13T04:29:10.595 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-13T04:29:10.595 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-13T04:29:10.595 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-13T04:29:10.595 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-13T04:29:10.595 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.595 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.596 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-13T04:29:10.596 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-13T04:29:10.596 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.596 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-13T04:29:10.596 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-13T04:29:10.597 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-13T04:29:10.597 INFO:teuthology.orchestra.run.smithi156.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-13T04:29:10.597 INFO:teuthology.orchestra.run.smithi156.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-13T04:29:10.597 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-13T04:29:10.597 INFO:teuthology.orchestra.run.smithi156.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-13T04:29:10.598 INFO:teuthology.orchestra.run.smithi156.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-13T04:29:10.598 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-13T04:29:10.598 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-13T04:29:10.598 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-13T04:29:10.598 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-13T04:29:10.599 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-13T04:29:10.599 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.599 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.599 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:10.600 INFO:teuthology.orchestra.run.smithi156.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-13T04:29:10.600 INFO:teuthology.orchestra.run.smithi156.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-13T04:29:10.600 INFO:teuthology.orchestra.run.smithi156.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-13T04:29:10.600 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-13T04:29:10.600 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-13T04:29:10.601 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-13T04:29:10.601 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-13T04:29:10.602 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2023-09-13T04:29:10.754 DEBUG:teuthology.parallel:result is None 2023-09-13T04:29:12.365 INFO:teuthology.orchestra.run.smithi149.stdout:Installed products updated. 2023-09-13T04:29:12.580 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-09-13T04:29:12.580 INFO:teuthology.orchestra.run.smithi149.stdout:Upgraded: 2023-09-13T04:29:12.580 INFO:teuthology.orchestra.run.smithi149.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-13T04:29:12.580 INFO:teuthology.orchestra.run.smithi149.stdout: librados2-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.581 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.581 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2023-09-13T04:29:12.581 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-13T04:29:12.581 INFO:teuthology.orchestra.run.smithi149.stdout: automake-1.16.1-7.el8.noarch 2023-09-13T04:29:12.581 INFO:teuthology.orchestra.run.smithi149.stdout: bison-3.0.4-10.el8.x86_64 2023-09-13T04:29:12.582 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.582 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.582 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.582 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.582 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:12.583 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.583 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.583 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.583 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:12.583 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:12.584 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:12.584 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:12.584 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:12.584 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.584 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.585 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:12.585 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.585 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.585 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.585 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:12.586 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:12.586 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.586 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:12.586 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-13T04:29:12.587 INFO:teuthology.orchestra.run.smithi149.stdout: flex-2.6.1-9.el8.x86_64 2023-09-13T04:29:12.587 INFO:teuthology.orchestra.run.smithi149.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-13T04:29:12.587 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-13T04:29:12.588 INFO:teuthology.orchestra.run.smithi149.stdout: jq-1.6-3.el8.x86_64 2023-09-13T04:29:12.589 INFO:teuthology.orchestra.run.smithi149.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-13T04:29:12.589 INFO:teuthology.orchestra.run.smithi149.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-13T04:29:12.589 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-13T04:29:12.589 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.589 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.590 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.590 INFO:teuthology.orchestra.run.smithi149.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-13T04:29:12.590 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-13T04:29:12.590 INFO:teuthology.orchestra.run.smithi149.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-13T04:29:12.591 INFO:teuthology.orchestra.run.smithi149.stdout: libpq-13.5-1.el8.x86_64 2023-09-13T04:29:12.591 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-13T04:29:12.591 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-13T04:29:12.591 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.591 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.592 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-13T04:29:12.592 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.592 INFO:teuthology.orchestra.run.smithi149.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-13T04:29:12.592 INFO:teuthology.orchestra.run.smithi149.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-13T04:29:12.592 INFO:teuthology.orchestra.run.smithi149.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-13T04:29:12.593 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-13T04:29:12.593 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-13T04:29:12.593 INFO:teuthology.orchestra.run.smithi149.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-13T04:29:12.593 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-13T04:29:12.594 INFO:teuthology.orchestra.run.smithi149.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-13T04:29:12.594 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-13T04:29:12.594 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-13T04:29:12.594 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-13T04:29:12.594 INFO:teuthology.orchestra.run.smithi149.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-13T04:29:12.595 INFO:teuthology.orchestra.run.smithi149.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-13T04:29:12.595 INFO:teuthology.orchestra.run.smithi149.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-13T04:29:12.595 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-13T04:29:12.595 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-13T04:29:12.596 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-13T04:29:12.596 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-13T04:29:12.596 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-13T04:29:12.596 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-13T04:29:12.596 INFO:teuthology.orchestra.run.smithi149.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-13T04:29:12.597 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-13T04:29:12.597 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.597 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.597 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.597 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-13T04:29:12.597 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-13T04:29:12.598 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-13T04:29:12.598 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-13T04:29:12.598 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-13T04:29:12.598 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-13T04:29:12.599 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-13T04:29:12.599 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-13T04:29:12.599 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-13T04:29:12.599 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-13T04:29:12.599 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-13T04:29:12.600 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-13T04:29:12.600 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-13T04:29:12.600 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-13T04:29:12.600 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-13T04:29:12.600 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-13T04:29:12.601 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-13T04:29:12.601 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-13T04:29:12.601 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-13T04:29:12.601 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-13T04:29:12.601 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-13T04:29:12.601 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-13T04:29:12.602 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend-2.6-1.el8.noarch 2023-09-13T04:29:12.602 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-13T04:29:12.602 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-13T04:29:12.602 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-13T04:29:12.603 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-13T04:29:12.603 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-13T04:29:12.603 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-13T04:29:12.603 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.603 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.604 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-13T04:29:12.604 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-13T04:29:12.604 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.604 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-13T04:29:12.604 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-13T04:29:12.604 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-13T04:29:12.605 INFO:teuthology.orchestra.run.smithi149.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-13T04:29:12.605 INFO:teuthology.orchestra.run.smithi149.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-13T04:29:12.605 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-13T04:29:12.605 INFO:teuthology.orchestra.run.smithi149.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-13T04:29:12.605 INFO:teuthology.orchestra.run.smithi149.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-13T04:29:12.606 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-13T04:29:12.606 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-13T04:29:12.606 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-13T04:29:12.606 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-13T04:29:12.606 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-13T04:29:12.607 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.607 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.607 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:12.607 INFO:teuthology.orchestra.run.smithi149.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-13T04:29:12.607 INFO:teuthology.orchestra.run.smithi149.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-13T04:29:12.608 INFO:teuthology.orchestra.run.smithi149.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-13T04:29:12.608 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-13T04:29:12.608 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-13T04:29:12.609 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-13T04:29:12.609 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-09-13T04:29:12.609 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2023-09-13T04:29:12.797 DEBUG:teuthology.parallel:result is None 2023-09-13T04:29:13.202 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6083.g4a754ac9.el8.x86_64 114/134 2023-09-13T04:29:14.500 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-test-2:18.0.0-6083.g4a754ac9.el8.x86_64 115/134 2023-09-13T04:29:14.886 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-09-13T04:29:15.077 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-fuse-2:18.0.0-6083.g4a754ac9.el8.x86_64 117/134 2023-09-13T04:29:15.342 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-09-13T04:29:15.545 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-09-13T04:29:15.810 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-09-13T04:29:16.083 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-09-13T04:29:16.299 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephfs-devel-2:18.0.0-6083.g4a754ac9.el8.x86_ 121/134 2023-09-13T04:29:16.463 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-13T04:29:20.061 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-13T04:29:20.361 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-13T04:29:20.399 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-13T04:29:20.629 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephfs-top-2:18.0.0-6083.g4a754ac9.el8.noarch 124/134 2023-09-13T04:29:20.880 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-fuse-2:18.0.0-6083.g4a754ac9.el8.x86_64 125/134 2023-09-13T04:29:21.199 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-nbd-2:18.0.0-6083.g4a754ac9.el8.x86_64 126/134 2023-09-13T04:29:21.603 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-13T04:29:21.794 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-09-13T04:29:21.932 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-09-13T04:29:22.136 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-09-13T04:29:22.164 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-09-13T04:29:22.394 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-13T04:29:22.417 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-13T04:29:22.573 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-09-13T04:29:22.573 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-13T04:29:22.917 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-13T04:29:22.918 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-13T04:29:23.243 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-13T04:29:23.428 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-13T04:29:23.723 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-13T04:29:23.723 INFO:teuthology.orchestra.run.smithi002.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-09-13T04:29:23.724 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:29:24.387 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-2:18.0.0-6083.g4a754ac9.el8.x86_64 1/134 2023-09-13T04:29:24.387 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-base-2:18.0.0-6083.g4a754ac9.el8.x86_64 2/134 2023-09-13T04:29:24.388 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-common-2:18.0.0-6083.g4a754ac9.el8.x86_64 3/134 2023-09-13T04:29:24.388 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-fuse-2:18.0.0-6083.g4a754ac9.el8.x86_64 4/134 2023-09-13T04:29:24.388 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6083.g4a754 5/134 2023-09-13T04:29:24.388 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mds-2:18.0.0-6083.g4a754ac9.el8.x86_64 6/134 2023-09-13T04:29:24.388 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-2:18.0.0-6083.g4a754ac9.el8.x86_64 7/134 2023-09-13T04:29:24.389 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mon-2:18.0.0-6083.g4a754ac9.el8.x86_64 8/134 2023-09-13T04:29:24.389 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-osd-2:18.0.0-6083.g4a754ac9.el8.x86_64 9/134 2023-09-13T04:29:24.389 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-radosgw-2:18.0.0-6083.g4a754ac9.el8.x86_64 10/134 2023-09-13T04:29:24.389 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-selinux-2:18.0.0-6083.g4a754ac9.el8.x86_64 11/134 2023-09-13T04:29:24.389 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-test-2:18.0.0-6083.g4a754ac9.el8.x86_64 12/134 2023-09-13T04:29:24.390 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephfs-mirror-2:18.0.0-6083.g4a754ac9.el8.x86_64 13/134 2023-09-13T04:29:24.390 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephfs-devel-2:18.0.0-6083.g4a754ac9.el8.x86_ 14/134 2023-09-13T04:29:24.390 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephfs2-2:18.0.0-6083.g4a754ac9.el8.x86_64 15/134 2023-09-13T04:29:24.390 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephsqlite-2:18.0.0-6083.g4a754ac9.el8.x86_64 16/134 2023-09-13T04:29:24.390 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados-devel-2:18.0.0-6083.g4a754ac9.el8.x86_6 17/134 2023-09-13T04:29:24.391 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libradosstriper1-2:18.0.0-6083.g4a754ac9.el8.x86 18/134 2023-09-13T04:29:24.391 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librgw2-2:18.0.0-6083.g4a754ac9.el8.x86_64 19/134 2023-09-13T04:29:24.391 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6083.g4a754ac9.el 20/134 2023-09-13T04:29:24.391 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ceph-common-2:18.0.0-6083.g4a754ac9.el8. 21/134 2023-09-13T04:29:24.391 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cephfs-2:18.0.0-6083.g4a754ac9.el8.x86_6 22/134 2023-09-13T04:29:24.392 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rados-2:18.0.0-6083.g4a754ac9.el8.x86_64 23/134 2023-09-13T04:29:24.392 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rbd-2:18.0.0-6083.g4a754ac9.el8.x86_64 24/134 2023-09-13T04:29:24.392 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rgw-2:18.0.0-6083.g4a754ac9.el8.x86_64 25/134 2023-09-13T04:29:24.392 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-fuse-2:18.0.0-6083.g4a754ac9.el8.x86_64 26/134 2023-09-13T04:29:24.392 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-mirror-2:18.0.0-6083.g4a754ac9.el8.x86_64 27/134 2023-09-13T04:29:24.393 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-nbd-2:18.0.0-6083.g4a754ac9.el8.x86_64 28/134 2023-09-13T04:29:24.393 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6083.g4a754ac9. 29/134 2023-09-13T04:29:24.393 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6083.g4a754ac9.el8.noa 30/134 2023-09-13T04:29:24.393 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6083.g4a754ac9.el8.n 31/134 2023-09-13T04:29:24.393 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6083.g4a7 32/134 2023-09-13T04:29:24.394 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6083.g4a754ac9.el 33/134 2023-09-13T04:29:24.394 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6083.g4a754ac9.el8.noarch 34/134 2023-09-13T04:29:24.394 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6083.g4a754ac9.e 35/134 2023-09-13T04:29:24.394 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-volume-2:18.0.0-6083.g4a754ac9.el8.noarch 36/134 2023-09-13T04:29:24.395 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephadm-2:18.0.0-6083.g4a754ac9.el8.noarch 37/134 2023-09-13T04:29:24.395 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephfs-top-2:18.0.0-6083.g4a754ac9.el8.noarch 38/134 2023-09-13T04:29:24.395 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-13T04:29:24.395 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-13T04:29:24.395 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-13T04:29:24.396 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-13T04:29:24.396 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-13T04:29:24.397 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-13T04:29:24.397 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-13T04:29:24.397 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-13T04:29:24.397 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-13T04:29:24.398 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-13T04:29:24.398 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-13T04:29:24.398 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-13T04:29:24.398 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-13T04:29:24.398 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-13T04:29:24.399 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-13T04:29:24.399 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-13T04:29:24.399 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-13T04:29:24.399 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-13T04:29:24.399 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-13T04:29:24.400 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-13T04:29:24.400 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-13T04:29:24.400 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-13T04:29:24.400 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-13T04:29:24.400 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-13T04:29:24.401 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-13T04:29:24.401 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-13T04:29:24.401 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-13T04:29:24.401 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-13T04:29:24.402 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-13T04:29:24.402 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-13T04:29:24.402 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-13T04:29:24.402 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-13T04:29:24.402 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-13T04:29:24.403 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-09-13T04:29:24.403 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-09-13T04:29:24.403 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-09-13T04:29:24.403 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-09-13T04:29:24.403 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-09-13T04:29:24.404 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-09-13T04:29:24.404 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-09-13T04:29:24.404 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-09-13T04:29:24.404 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-09-13T04:29:24.404 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-09-13T04:29:24.405 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-09-13T04:29:24.405 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-09-13T04:29:24.405 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-09-13T04:29:24.405 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-09-13T04:29:24.405 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-09-13T04:29:24.406 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-09-13T04:29:24.406 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-09-13T04:29:24.406 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-09-13T04:29:24.406 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-09-13T04:29:24.407 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-09-13T04:29:24.407 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-09-13T04:29:24.407 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-09-13T04:29:24.407 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-09-13T04:29:24.407 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-09-13T04:29:24.407 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-09-13T04:29:24.408 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-09-13T04:29:24.408 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-09-13T04:29:24.408 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-09-13T04:29:24.408 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-09-13T04:29:24.408 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-09-13T04:29:24.409 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-09-13T04:29:24.409 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-09-13T04:29:24.409 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-09-13T04:29:24.409 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-09-13T04:29:24.409 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-09-13T04:29:24.410 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-09-13T04:29:24.410 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-09-13T04:29:24.410 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-09-13T04:29:24.411 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-09-13T04:29:24.411 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-09-13T04:29:24.411 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-09-13T04:29:24.412 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-09-13T04:29:24.412 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-09-13T04:29:24.413 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-09-13T04:29:24.413 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-09-13T04:29:24.414 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-09-13T04:29:24.414 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-09-13T04:29:24.415 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-09-13T04:29:24.415 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-09-13T04:29:24.415 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-09-13T04:29:24.416 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-09-13T04:29:24.416 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-09-13T04:29:24.416 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-09-13T04:29:24.417 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-09-13T04:29:24.417 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-09-13T04:29:24.417 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-09-13T04:29:24.417 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-09-13T04:29:24.417 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados2-2:18.0.0-6083.g4a754ac9.el8.x86_64 129/134 2023-09-13T04:29:24.418 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-13T04:29:24.418 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librbd1-2:18.0.0-6083.g4a754ac9.el8.x86_64 131/134 2023-09-13T04:29:24.418 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-13T04:29:24.418 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-13T04:29:27.104 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-13T04:29:27.104 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:02:13 ago on Wed 13 Sep 2023 04:27:14 AM UTC. 2023-09-13T04:29:29.860 INFO:teuthology.orchestra.run.smithi002.stdout:Installed products updated. 2023-09-13T04:29:30.102 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:29:30.102 INFO:teuthology.orchestra.run.smithi002.stdout:Upgraded: 2023-09-13T04:29:30.103 INFO:teuthology.orchestra.run.smithi002.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-13T04:29:30.103 INFO:teuthology.orchestra.run.smithi002.stdout: librados2-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.103 INFO:teuthology.orchestra.run.smithi002.stdout: librbd1-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.103 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2023-09-13T04:29:30.103 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-13T04:29:30.104 INFO:teuthology.orchestra.run.smithi002.stdout: automake-1.16.1-7.el8.noarch 2023-09-13T04:29:30.104 INFO:teuthology.orchestra.run.smithi002.stdout: bison-3.0.4-10.el8.x86_64 2023-09-13T04:29:30.104 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.104 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-base-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.104 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-common-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.105 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-fuse-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.105 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-grafana-dashboards-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:30.105 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-immutable-object-cache-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.105 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mds-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.105 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.106 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-cephadm-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:30.106 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-dashboard-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:30.106 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:30.106 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-modules-core-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:30.106 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-rook-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mon-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-osd-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-prometheus-alerts-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-radosgw-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-selinux-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-test-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.108 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-volume-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:30.108 INFO:teuthology.orchestra.run.smithi002.stdout: cephadm-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:30.108 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-mirror-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.108 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-top-2:18.0.0-6083.g4a754ac9.el8.noarch 2023-09-13T04:29:30.108 INFO:teuthology.orchestra.run.smithi002.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-13T04:29:30.109 INFO:teuthology.orchestra.run.smithi002.stdout: flex-2.6.1-9.el8.x86_64 2023-09-13T04:29:30.109 INFO:teuthology.orchestra.run.smithi002.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-13T04:29:30.109 INFO:teuthology.orchestra.run.smithi002.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-13T04:29:30.109 INFO:teuthology.orchestra.run.smithi002.stdout: jq-1.6-3.el8.x86_64 2023-09-13T04:29:30.109 INFO:teuthology.orchestra.run.smithi002.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-13T04:29:30.110 INFO:teuthology.orchestra.run.smithi002.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-13T04:29:30.110 INFO:teuthology.orchestra.run.smithi002.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-13T04:29:30.110 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs-devel-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.110 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs2-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.110 INFO:teuthology.orchestra.run.smithi002.stdout: libcephsqlite-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.111 INFO:teuthology.orchestra.run.smithi002.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-13T04:29:30.111 INFO:teuthology.orchestra.run.smithi002.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-13T04:29:30.111 INFO:teuthology.orchestra.run.smithi002.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-13T04:29:30.111 INFO:teuthology.orchestra.run.smithi002.stdout: libpq-13.5-1.el8.x86_64 2023-09-13T04:29:30.111 INFO:teuthology.orchestra.run.smithi002.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-13T04:29:30.112 INFO:teuthology.orchestra.run.smithi002.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-13T04:29:30.112 INFO:teuthology.orchestra.run.smithi002.stdout: librados-devel-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.112 INFO:teuthology.orchestra.run.smithi002.stdout: libradosstriper1-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.112 INFO:teuthology.orchestra.run.smithi002.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-13T04:29:30.112 INFO:teuthology.orchestra.run.smithi002.stdout: librgw2-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.112 INFO:teuthology.orchestra.run.smithi002.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-13T04:29:30.113 INFO:teuthology.orchestra.run.smithi002.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-13T04:29:30.113 INFO:teuthology.orchestra.run.smithi002.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-13T04:29:30.113 INFO:teuthology.orchestra.run.smithi002.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-13T04:29:30.113 INFO:teuthology.orchestra.run.smithi002.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-13T04:29:30.114 INFO:teuthology.orchestra.run.smithi002.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-13T04:29:30.114 INFO:teuthology.orchestra.run.smithi002.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-13T04:29:30.114 INFO:teuthology.orchestra.run.smithi002.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-13T04:29:30.114 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-13T04:29:30.115 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-13T04:29:30.115 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-13T04:29:30.115 INFO:teuthology.orchestra.run.smithi002.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-13T04:29:30.115 INFO:teuthology.orchestra.run.smithi002.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-13T04:29:30.115 INFO:teuthology.orchestra.run.smithi002.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-13T04:29:30.115 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-13T04:29:30.116 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-13T04:29:30.116 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-13T04:29:30.116 INFO:teuthology.orchestra.run.smithi002.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-13T04:29:30.116 INFO:teuthology.orchestra.run.smithi002.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-13T04:29:30.116 INFO:teuthology.orchestra.run.smithi002.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-13T04:29:30.117 INFO:teuthology.orchestra.run.smithi002.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-13T04:29:30.117 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-13T04:29:30.117 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-argparse-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.117 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-common-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.117 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cephfs-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.118 INFO:teuthology.orchestra.run.smithi002.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-13T04:29:30.118 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-13T04:29:30.118 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-13T04:29:30.118 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-13T04:29:30.118 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-13T04:29:30.119 INFO:teuthology.orchestra.run.smithi002.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-13T04:29:30.119 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-13T04:29:30.119 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-13T04:29:30.119 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-13T04:29:30.119 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-13T04:29:30.120 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-13T04:29:30.120 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-13T04:29:30.120 INFO:teuthology.orchestra.run.smithi002.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-13T04:29:30.120 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-13T04:29:30.120 INFO:teuthology.orchestra.run.smithi002.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-13T04:29:30.120 INFO:teuthology.orchestra.run.smithi002.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-13T04:29:30.121 INFO:teuthology.orchestra.run.smithi002.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-13T04:29:30.121 INFO:teuthology.orchestra.run.smithi002.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-13T04:29:30.121 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-13T04:29:30.121 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-13T04:29:30.121 INFO:teuthology.orchestra.run.smithi002.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-13T04:29:30.121 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-13T04:29:30.122 INFO:teuthology.orchestra.run.smithi002.stdout: python3-portend-2.6-1.el8.noarch 2023-09-13T04:29:30.122 INFO:teuthology.orchestra.run.smithi002.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-13T04:29:30.122 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-13T04:29:30.122 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-13T04:29:30.123 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-13T04:29:30.123 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-13T04:29:30.123 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-13T04:29:30.123 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rados-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.123 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rbd-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.124 INFO:teuthology.orchestra.run.smithi002.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-13T04:29:30.124 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-13T04:29:30.124 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rgw-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.124 INFO:teuthology.orchestra.run.smithi002.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-13T04:29:30.125 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-13T04:29:30.125 INFO:teuthology.orchestra.run.smithi002.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-13T04:29:30.125 INFO:teuthology.orchestra.run.smithi002.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-13T04:29:30.125 INFO:teuthology.orchestra.run.smithi002.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-13T04:29:30.125 INFO:teuthology.orchestra.run.smithi002.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-13T04:29:30.126 INFO:teuthology.orchestra.run.smithi002.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-13T04:29:30.126 INFO:teuthology.orchestra.run.smithi002.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-13T04:29:30.126 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-13T04:29:30.126 INFO:teuthology.orchestra.run.smithi002.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-13T04:29:30.126 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-13T04:29:30.127 INFO:teuthology.orchestra.run.smithi002.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-13T04:29:30.127 INFO:teuthology.orchestra.run.smithi002.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-13T04:29:30.127 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-fuse-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.127 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-mirror-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.127 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-nbd-2:18.0.0-6083.g4a754ac9.el8.x86_64 2023-09-13T04:29:30.127 INFO:teuthology.orchestra.run.smithi002.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-13T04:29:30.128 INFO:teuthology.orchestra.run.smithi002.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-13T04:29:30.128 INFO:teuthology.orchestra.run.smithi002.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-13T04:29:30.128 INFO:teuthology.orchestra.run.smithi002.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-13T04:29:30.128 INFO:teuthology.orchestra.run.smithi002.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-13T04:29:30.129 INFO:teuthology.orchestra.run.smithi002.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-13T04:29:30.129 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:29:30.129 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2023-09-13T04:29:30.461 DEBUG:teuthology.parallel:result is None 2023-09-13T04:29:30.462 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 2023-09-13T04:29:30.680 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-13T04:29:30.736 INFO:teuthology.orchestra.run.smithi002.stdout:18.0.0-6083.g4a754ac9.el8 2023-09-13T04:29:30.737 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6083.g4a754ac9.el8 2023-09-13T04:29:30.737 INFO:teuthology.task.install:The correct ceph version 18.0.0-6083.g4a754ac9 is installed. 2023-09-13T04:29:30.738 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 2023-09-13T04:29:30.880 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-13T04:29:30.907 INFO:teuthology.orchestra.run.smithi149.stdout:18.0.0-6083.g4a754ac9.el8 2023-09-13T04:29:30.908 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6083.g4a754ac9.el8 2023-09-13T04:29:30.908 INFO:teuthology.task.install:The correct ceph version 18.0.0-6083.g4a754ac9 is installed. 2023-09-13T04:29:30.909 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 2023-09-13T04:29:31.117 DEBUG:teuthology.orchestra.run.smithi156:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-13T04:29:31.146 INFO:teuthology.orchestra.run.smithi156.stdout:18.0.0-6083.g4a754ac9.el8 2023-09-13T04:29:31.146 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6083.g4a754ac9.el8 2023-09-13T04:29:31.146 INFO:teuthology.task.install:The correct ceph version 18.0.0-6083.g4a754ac9 is installed. 2023-09-13T04:29:31.147 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-09-13T04:29:31.148 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-09-13T04:29:31.148 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-13T04:29:31.242 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2023-09-13T04:29:31.242 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-13T04:29:31.289 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-13T04:29:31.290 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-13T04:29:31.338 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-09-13T04:29:31.338 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-09-13T04:29:31.339 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/daemon-helper 2023-09-13T04:29:31.412 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-13T04:29:31.523 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2023-09-13T04:29:31.524 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/daemon-helper 2023-09-13T04:29:31.558 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-13T04:29:31.628 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-13T04:29:31.629 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/daemon-helper 2023-09-13T04:29:31.661 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-13T04:29:31.731 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-09-13T04:29:31.731 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-09-13T04:29:31.732 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-13T04:29:31.801 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-13T04:29:31.909 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2023-09-13T04:29:31.909 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-13T04:29:31.941 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-13T04:29:32.011 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-13T04:29:32.011 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-13T04:29:32.045 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-13T04:29:32.115 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-09-13T04:29:32.115 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-09-13T04:29:32.116 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/stdin-killer 2023-09-13T04:29:32.185 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-13T04:29:32.293 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2023-09-13T04:29:32.294 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/stdin-killer 2023-09-13T04:29:32.326 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-13T04:29:32.396 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-13T04:29:32.397 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/stdin-killer 2023-09-13T04:29:32.428 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-13T04:29:32.499 INFO:teuthology.run_tasks:Running task cephadm... 2023-09-13T04:29:32.577 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5'} 2023-09-13T04:29:32.578 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 2023-09-13T04:29:32.579 INFO:tasks.cephadm:Cluster fsid is 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:29:32.579 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-09-13T04:29:32.580 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.2', 'mon.b': '172.21.15.149', 'mon.c': '172.21.15.156'} 2023-09-13T04:29:32.580 INFO:tasks.cephadm:First mon is mon.a on smithi002 2023-09-13T04:29:32.580 INFO:tasks.cephadm:First mgr is x 2023-09-13T04:29:32.580 INFO:tasks.cephadm:Normalizing hostnames... 2023-09-13T04:29:32.581 DEBUG:teuthology.orchestra.run.smithi002:> sudo hostname $(hostname -s) 2023-09-13T04:29:32.654 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname $(hostname -s) 2023-09-13T04:29:32.687 DEBUG:teuthology.orchestra.run.smithi156:> sudo hostname $(hostname -s) 2023-09-13T04:29:32.719 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-09-13T04:29:32.720 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 2023-09-13T04:29:32.932 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/72555/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-6083-g4a754ac9', 'node_name': '172.21.2.5+braggi05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-6083.g4a754ac9'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023sep12-b2/4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-09-12 09:50:44.139622', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-2023sep12-b2', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-rishabh-2023sep12-b2/4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-09-13T04:29:33.071 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-rishabh-2023sep12-b2, sha1 4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 2023-09-13T04:29:33.072 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-2023sep12-b2/4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/centos/8/x86_64/flavors/default/cephadm 2023-09-13T04:29:33.072 DEBUG:teuthology.orchestra.run.smithi002:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-2023sep12-b2/4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-13T04:29:33.510 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-rw-r--. 1 ubuntu ubuntu 728366 Sep 13 04:29 /home/ubuntu/cephtest/cephadm 2023-09-13T04:29:33.510 DEBUG:teuthology.orchestra.run.smithi149:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-2023sep12-b2/4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-13T04:29:34.111 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-rw-r--. 1 ubuntu ubuntu 728366 Sep 13 04:29 /home/ubuntu/cephtest/cephadm 2023-09-13T04:29:34.111 DEBUG:teuthology.orchestra.run.smithi156:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-2023sep12-b2/4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-13T04:29:34.516 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-rw-r--. 1 ubuntu ubuntu 728366 Sep 13 04:29 /home/ubuntu/cephtest/cephadm 2023-09-13T04:29:34.517 DEBUG:teuthology.orchestra.run.smithi002:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-13T04:29:34.569 DEBUG:teuthology.orchestra.run.smithi149:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-13T04:29:34.590 DEBUG:teuthology.orchestra.run.smithi156:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-13T04:29:34.626 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 on all hosts... 2023-09-13T04:29:34.627 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 pull 2023-09-13T04:29:34.632 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 pull 2023-09-13T04:29:34.635 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 pull 2023-09-13T04:29:36.379 INFO:teuthology.orchestra.run.smithi149.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5... 2023-09-13T04:29:36.652 INFO:teuthology.orchestra.run.smithi156.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5... 2023-09-13T04:29:39.041 INFO:teuthology.orchestra.run.smithi002.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5... 2023-09-13T04:30:02.576 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2023-09-13T04:30:02.577 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_version": "ceph version 18.0.0-6083-g4a754ac9 (4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5) reef (dev)", 2023-09-13T04:30:02.577 INFO:teuthology.orchestra.run.smithi149.stdout: "image_id": "ac82d04f119cd3d453966b1e12977b81535381fb4e22ee1754e33c107a26ea70", 2023-09-13T04:30:02.577 INFO:teuthology.orchestra.run.smithi149.stdout: "repo_digests": [ 2023-09-13T04:30:02.577 INFO:teuthology.orchestra.run.smithi149.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:eb90da591c5598a9ef0b91a8a434a296d6fd7e7f7757865710c99a2f150ff6a7" 2023-09-13T04:30:02.578 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2023-09-13T04:30:02.578 INFO:teuthology.orchestra.run.smithi149.stdout:} 2023-09-13T04:30:06.250 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-09-13T04:30:06.250 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_version": "ceph version 18.0.0-6083-g4a754ac9 (4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5) reef (dev)", 2023-09-13T04:30:06.250 INFO:teuthology.orchestra.run.smithi156.stdout: "image_id": "ac82d04f119cd3d453966b1e12977b81535381fb4e22ee1754e33c107a26ea70", 2023-09-13T04:30:06.251 INFO:teuthology.orchestra.run.smithi156.stdout: "repo_digests": [ 2023-09-13T04:30:06.251 INFO:teuthology.orchestra.run.smithi156.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:eb90da591c5598a9ef0b91a8a434a296d6fd7e7f7757865710c99a2f150ff6a7" 2023-09-13T04:30:06.251 INFO:teuthology.orchestra.run.smithi156.stdout: ] 2023-09-13T04:30:06.251 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-09-13T04:30:11.708 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:30:11.709 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_version": "ceph version 18.0.0-6083-g4a754ac9 (4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5) reef (dev)", 2023-09-13T04:30:11.709 INFO:teuthology.orchestra.run.smithi002.stdout: "image_id": "ac82d04f119cd3d453966b1e12977b81535381fb4e22ee1754e33c107a26ea70", 2023-09-13T04:30:11.709 INFO:teuthology.orchestra.run.smithi002.stdout: "repo_digests": [ 2023-09-13T04:30:11.709 INFO:teuthology.orchestra.run.smithi002.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:eb90da591c5598a9ef0b91a8a434a296d6fd7e7f7757865710c99a2f150ff6a7" 2023-09-13T04:30:11.709 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2023-09-13T04:30:11.710 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:30:11.734 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /etc/ceph 2023-09-13T04:30:11.808 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /etc/ceph 2023-09-13T04:30:11.843 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /etc/ceph 2023-09-13T04:30:11.881 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 777 /etc/ceph 2023-09-13T04:30:11.951 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 777 /etc/ceph 2023-09-13T04:30:11.981 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 777 /etc/ceph 2023-09-13T04:30:12.012 INFO:tasks.cephadm:Writing seed config... 2023-09-13T04:30:12.014 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-09-13T04:30:12.014 INFO:tasks.cephadm: override: [client] debug client = 20 2023-09-13T04:30:12.014 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-09-13T04:30:12.015 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-09-13T04:30:12.015 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-09-13T04:30:12.015 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-09-13T04:30:12.015 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-09-13T04:30:12.016 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-09-13T04:30:12.016 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-09-13T04:30:12.016 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-09-13T04:30:12.017 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-09-13T04:30:12.017 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-09-13T04:30:12.017 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-09-13T04:30:12.017 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-09-13T04:30:12.018 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-09-13T04:30:12.018 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-09-13T04:30:12.018 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-09-13T04:30:12.019 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-09-13T04:30:12.019 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-09-13T04:30:12.019 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-09-13T04:30:12.019 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-09-13T04:30:12.020 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-09-13T04:30:12.020 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-09-13T04:30:12.020 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-09-13T04:30:12.020 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-09-13T04:30:12.021 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-09-13T04:30:12.021 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-09-13T04:30:12.021 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-09-13T04:30:12.021 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-09-13T04:30:12.022 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-09-13T04:30:12.022 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-09-13T04:30:12.022 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-09-13T04:30:12.023 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-09-13T04:30:12.023 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-09-13T04:30:12.023 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-09-13T04:30:12.024 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-09-13T04:30:12.024 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-09-13T04:30:12.024 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-09-13T04:30:12.024 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-09-13T04:30:12.025 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-09-13T04:30:12.025 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-09-13T04:30:12.025 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-09-13T04:30:12.025 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-09-13T04:30:12.026 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-09-13T04:30:12.027 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-09-13T04:30:12.027 DEBUG:teuthology.orchestra.run.smithi002:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-09-13T04:30:12.070 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-09-13T04:30:12.071 DEBUG:teuthology.orchestra.run.smithi002:mon.a> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mon.a.service 2023-09-13T04:30:12.117 DEBUG:teuthology.orchestra.run.smithi002:mgr.x> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mgr.x.service 2023-09-13T04:30:12.162 INFO:tasks.cephadm:Bootstrapping... 2023-09-13T04:30:12.162 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 -v bootstrap --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.2 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-09-13T04:30:12.186 INFO:journalctl@ceph.mon.a.smithi002.stdout:-- Logs begin at Wed 2023-09-13 04:13:33 UTC. -- 2023-09-13T04:30:12.232 INFO:journalctl@ceph.mgr.x.smithi002.stdout:-- Logs begin at Wed 2023-09-13 04:13:33 UTC. -- 2023-09-13T04:30:12.398 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2023-09-13T04:30:12.398 INFO:teuthology.orchestra.run.smithi002.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5', '-v', 'bootstrap', '--fsid', '22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.2', '--skip-admin-label'] 2023-09-13T04:30:12.572 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout 4.1.1 2023-09-13T04:30:12.573 INFO:teuthology.orchestra.run.smithi002.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-09-13T04:30:12.573 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying podman|docker is present... 2023-09-13T04:30:12.743 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout 4.1.1 2023-09-13T04:30:12.743 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying lvm2 is present... 2023-09-13T04:30:12.744 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying time synchronization is in place... 2023-09-13T04:30:12.758 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-09-13T04:30:12.758 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-09-13T04:30:12.771 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-09-13T04:30:12.771 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2023-09-13T04:30:12.784 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout enabled 2023-09-13T04:30:12.797 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout active 2023-09-13T04:30:12.798 INFO:teuthology.orchestra.run.smithi002.stdout:Unit chronyd.service is enabled and running 2023-09-13T04:30:12.798 INFO:teuthology.orchestra.run.smithi002.stdout:Repeating the final host check... 2023-09-13T04:30:12.967 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout 4.1.1 2023-09-13T04:30:12.967 INFO:teuthology.orchestra.run.smithi002.stdout:podman (/bin/podman) version 4.1.1 is present 2023-09-13T04:30:12.967 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl is present 2023-09-13T04:30:12.968 INFO:teuthology.orchestra.run.smithi002.stdout:lvcreate is present 2023-09-13T04:30:12.980 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-09-13T04:30:12.980 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-09-13T04:30:12.993 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-09-13T04:30:12.993 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2023-09-13T04:30:13.007 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout enabled 2023-09-13T04:30:13.020 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout active 2023-09-13T04:30:13.021 INFO:teuthology.orchestra.run.smithi002.stdout:Unit chronyd.service is enabled and running 2023-09-13T04:30:13.021 INFO:teuthology.orchestra.run.smithi002.stdout:Host looks OK 2023-09-13T04:30:13.021 INFO:teuthology.orchestra.run.smithi002.stdout:Cluster fsid: 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:30:13.021 INFO:teuthology.orchestra.run.smithi002.stdout:Acquiring lock 140622733837312 on /run/cephadm/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da.lock 2023-09-13T04:30:13.022 INFO:teuthology.orchestra.run.smithi002.stdout:Lock 140622733837312 acquired on /run/cephadm/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da.lock 2023-09-13T04:30:13.022 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying IP 172.21.15.2 port 3300 ... 2023-09-13T04:30:13.022 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying IP 172.21.15.2 port 6789 ... 2023-09-13T04:30:13.023 INFO:teuthology.orchestra.run.smithi002.stdout:Base mon IP(s) is [172.21.15.2:3300, 172.21.15.2:6789], mon addrv is [v2:172.21.15.2:3300,v1:172.21.15.2:6789] 2023-09-13T04:30:13.029 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-09-13T04:30:13.030 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.2 metric 100 2023-09-13T04:30:13.036 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-09-13T04:30:13.037 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-09-13T04:30:13.037 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1799sec hoplimit 64 pref medium 2023-09-13T04:30:13.043 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-09-13T04:30:13.043 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-09-13T04:30:13.043 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-09-13T04:30:13.043 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-09-13T04:30:13.044 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1920/64 scope link 2023-09-13T04:30:13.044 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-09-13T04:30:13.044 INFO:teuthology.orchestra.run.smithi002.stdout:Mon IP `172.21.15.2` is in CIDR network `172.21.0.0/20` 2023-09-13T04:30:13.045 INFO:teuthology.orchestra.run.smithi002.stdout:Mon IP `172.21.15.2` is in CIDR network `172.21.0.0/20` 2023-09-13T04:30:13.045 INFO:teuthology.orchestra.run.smithi002.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-09-13T04:30:13.045 INFO:teuthology.orchestra.run.smithi002.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-09-13T04:30:13.046 INFO:teuthology.orchestra.run.smithi002.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5... 2023-09-13T04:30:13.620 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout ac82d04f119cd3d453966b1e12977b81535381fb4e22ee1754e33c107a26ea70 2023-09-13T04:30:13.621 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5... 2023-09-13T04:30:13.621 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Getting image source signatures 2023-09-13T04:30:13.621 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Copying blob sha256:0b53ffc1534070466eed446ff3128d72b2b3e0d7ba8de1a754a39e0b329d99bb 2023-09-13T04:30:13.621 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Copying blob sha256:aba40bcce7ff9e06f5f6d056b94d79b3c5f9fa5aba1a4d6d95b68b2cadf32b62 2023-09-13T04:30:13.622 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Copying config sha256:ac82d04f119cd3d453966b1e12977b81535381fb4e22ee1754e33c107a26ea70 2023-09-13T04:30:13.622 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Writing manifest to image destination 2023-09-13T04:30:13.622 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Storing signatures 2023-09-13T04:30:14.596 INFO:teuthology.orchestra.run.smithi002.stdout:ceph: stdout ceph version 18.0.0-6083-g4a754ac9 (4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5) reef (dev) 2023-09-13T04:30:14.596 INFO:teuthology.orchestra.run.smithi002.stdout:Ceph version: ceph version 18.0.0-6083-g4a754ac9 (4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5) reef (dev) 2023-09-13T04:30:14.597 INFO:teuthology.orchestra.run.smithi002.stdout:Extracting ceph user uid/gid from container image... 2023-09-13T04:30:15.474 INFO:teuthology.orchestra.run.smithi002.stdout:stat: stdout 167 167 2023-09-13T04:30:15.474 INFO:teuthology.orchestra.run.smithi002.stdout:Creating initial keys... 2023-09-13T04:30:16.248 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph-authtool: stdout AQDXOgFlObtiMxAAI//mXoHD4L9eqWyOgG+6vQ== 2023-09-13T04:30:17.123 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph-authtool: stdout AQDYOgFlzUTkJxAAP4UIPbRTcNx9kK2/oZ5cog== 2023-09-13T04:30:17.997 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph-authtool: stdout AQDZOgFljcJmHxAAsC08d5hcn0Bi8Jhh+WOFFA== 2023-09-13T04:30:17.998 INFO:teuthology.orchestra.run.smithi002.stdout:Creating initial monmap... 2023-09-13T04:30:19.174 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-09-13T04:30:19.175 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-09-13T04:30:19.175 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:30:19.175 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-09-13T04:30:19.175 INFO:teuthology.orchestra.run.smithi002.stdout:monmaptool for a [v2:172.21.15.2:3300,v1:172.21.15.2:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-09-13T04:30:19.176 INFO:teuthology.orchestra.run.smithi002.stdout:setting min_mon_release = pacific 2023-09-13T04:30:19.176 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: set fsid to 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:30:19.176 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-09-13T04:30:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:30:19.176 INFO:teuthology.orchestra.run.smithi002.stdout:Creating mon... 2023-09-13T04:30:20.052 INFO:teuthology.orchestra.run.smithi002.stdout:create mon.a on 2023-09-13T04:30:20.301 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-09-13T04:30:20.510 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-09-13T04:30:20.760 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da.target → /etc/systemd/system/ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da.target. 2023-09-13T04:30:20.760 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da.target → /etc/systemd/system/ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da.target. 2023-09-13T04:30:21.144 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mon.a 2023-09-13T04:30:21.144 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to reset failed state of unit ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mon.a.service: Unit ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mon.a.service not loaded. 2023-09-13T04:30:21.355 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da.target.wants/ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mon.a.service → /etc/systemd/system/ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@.service. 2023-09-13T04:30:21.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:21 smithi002 systemd[1]: Starting Ceph mon.a for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:30:22.114 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:21 smithi002 podman[97113]: 2023-09-13T04:30:22.162 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-13T04:30:22.162 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2023-09-13T04:30:22.181 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-13T04:30:22.182 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2023-09-13T04:30:22.182 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2023-09-13T04:30:22.182 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to enable service . firewalld.service is not available 2023-09-13T04:30:22.183 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mon to start... 2023-09-13T04:30:22.183 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mon... 2023-09-13T04:30:22.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:22 smithi002 bash[97113]: 37b4343ae65987cd0f65c31390df937126a50d2e270bb58af180d26b3afa3ff2 2023-09-13T04:30:22.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:22 smithi002 systemd[1]: Started Ceph mon.a for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:30:22.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:22 smithi002 ceph-mon[97153]: mkfs 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:30:22.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:22 smithi002 ceph-mon[97153]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-13T04:30:23.372 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout cluster: 2023-09-13T04:30:23.372 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout id: 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:30:23.372 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-09-13T04:30:23.372 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2023-09-13T04:30:23.373 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout services: 2023-09-13T04:30:23.373 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.719949s) 2023-09-13T04:30:23.373 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-09-13T04:30:23.373 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-09-13T04:30:23.373 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2023-09-13T04:30:23.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout data: 2023-09-13T04:30:23.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-09-13T04:30:23.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-09-13T04:30:23.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-09-13T04:30:23.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout pgs: 2023-09-13T04:30:23.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2023-09-13T04:30:23.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.796+0000 7fc4f9505700 1 Processor -- start 2023-09-13T04:30:23.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.796+0000 7fc4f9505700 1 -- start start 2023-09-13T04:30:23.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.796+0000 7fc4f9505700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4f407be90 0x7fc4f407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:23.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.796+0000 7fc4f9505700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f407a8c0 con 0x7fc4f407be90 2023-09-13T04:30:23.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.796+0000 7fc4f2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4f407be90 0x7fc4f407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:23.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.796+0000 7fc4f2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4f407be90 0x7fc4f407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35474/0 (socket says 172.21.15.2:35474) 2023-09-13T04:30:23.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.796+0000 7fc4f2ffd700 1 -- 172.21.15.2:0/2697835332 learned_addr learned my addr 172.21.15.2:0/2697835332 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:30:23.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.796+0000 7fc4f2ffd700 1 -- 172.21.15.2:0/2697835332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f407aa00 con 0x7fc4f407be90 2023-09-13T04:30:23.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.797+0000 7fc4f2ffd700 1 --2- 172.21.15.2:0/2697835332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4f407be90 0x7fc4f407a2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40098a0 tx=0x7fc4e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d226b8808f9e7eb2 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:23.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.797+0000 7fc4f1ffb700 1 -- 172.21.15.2:0/2697835332 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4e4004160 con 0x7fc4f407be90 2023-09-13T04:30:23.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.797+0000 7fc4f1ffb700 1 -- 172.21.15.2:0/2697835332 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e40042c0 con 0x7fc4f407be90 2023-09-13T04:30:23.378 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.797+0000 7fc4f1ffb700 1 -- 172.21.15.2:0/2697835332 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4e40045a0 con 0x7fc4f407be90 2023-09-13T04:30:23.378 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.797+0000 7fc4f9505700 1 -- 172.21.15.2:0/2697835332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4f407be90 msgr2=0x7fc4f407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:23.378 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.797+0000 7fc4f9505700 1 --2- 172.21.15.2:0/2697835332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4f407be90 0x7fc4f407a2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40098a0 tx=0x7fc4e4009bb0 comp rx=0 tx=0).stop 2023-09-13T04:30:23.378 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.798+0000 7fc4f9505700 1 -- 172.21.15.2:0/2697835332 shutdown_connections 2023-09-13T04:30:23.378 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.798+0000 7fc4f9505700 1 --2- 172.21.15.2:0/2697835332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4f407be90 0x7fc4f407a2f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:23.379 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.798+0000 7fc4f9505700 1 -- 172.21.15.2:0/2697835332 >> 172.21.15.2:0/2697835332 conn(0x7fc4f4101b00 msgr2=0x7fc4f4103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:30:23.379 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.798+0000 7fc4f9505700 1 -- 172.21.15.2:0/2697835332 shutdown_connections 2023-09-13T04:30:23.379 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.798+0000 7fc4f9505700 1 -- 172.21.15.2:0/2697835332 wait complete. 2023-09-13T04:30:23.379 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.799+0000 7fc4f9505700 1 Processor -- start 2023-09-13T04:30:23.379 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.799+0000 7fc4f9505700 1 -- start start 2023-09-13T04:30:23.380 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.799+0000 7fc4f9505700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4f407be90 0x7fc4f419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:23.380 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.799+0000 7fc4f9505700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f419c120 con 0x7fc4f407be90 2023-09-13T04:30:23.380 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.800+0000 7fc4f2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4f407be90 0x7fc4f419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:23.380 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.800+0000 7fc4f2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4f407be90 0x7fc4f419bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35486/0 (socket says 172.21.15.2:35486) 2023-09-13T04:30:23.380 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.800+0000 7fc4f2ffd700 1 -- 172.21.15.2:0/3932201076 learned_addr learned my addr 172.21.15.2:0/3932201076 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:30:23.381 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.800+0000 7fc4f2ffd700 1 -- 172.21.15.2:0/3932201076 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e4009550 con 0x7fc4f407be90 2023-09-13T04:30:23.381 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.800+0000 7fc4f2ffd700 1 --2- 172.21.15.2:0/3932201076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4f407be90 0x7fc4f419bbe0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc4f407b360 tx=0x7fc4e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:23.381 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.800+0000 7fc4dbfff700 1 -- 172.21.15.2:0/3932201076 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4e4011630 con 0x7fc4f407be90 2023-09-13T04:30:23.381 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.800+0000 7fc4dbfff700 1 -- 172.21.15.2:0/3932201076 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e4011790 con 0x7fc4f407be90 2023-09-13T04:30:23.381 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.800+0000 7fc4dbfff700 1 -- 172.21.15.2:0/3932201076 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4e4011a30 con 0x7fc4f407be90 2023-09-13T04:30:23.382 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.801+0000 7fc4f9505700 1 -- 172.21.15.2:0/3932201076 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f419c320 con 0x7fc4f407be90 2023-09-13T04:30:23.382 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.801+0000 7fc4f9505700 1 -- 172.21.15.2:0/3932201076 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f419c740 con 0x7fc4f407be90 2023-09-13T04:30:23.382 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.801+0000 7fc4dbfff700 1 -- 172.21.15.2:0/3932201076 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc4e4022070 con 0x7fc4f407be90 2023-09-13T04:30:23.382 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.801+0000 7fc4dbfff700 1 -- 172.21.15.2:0/3932201076 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc4e401c450 con 0x7fc4f407be90 2023-09-13T04:30:23.382 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.802+0000 7fc4f9505700 1 -- 172.21.15.2:0/3932201076 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f4195b00 con 0x7fc4f407be90 2023-09-13T04:30:23.383 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.806+0000 7fc4dbfff700 1 -- 172.21.15.2:0/3932201076 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fc4e401b450 con 0x7fc4f407be90 2023-09-13T04:30:23.383 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.881+0000 7fc4f9505700 1 -- 172.21.15.2:0/3932201076 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fc4f407ad70 con 0x7fc4f407be90 2023-09-13T04:30:23.383 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.881+0000 7fc4dbfff700 1 -- 172.21.15.2:0/3932201076 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fc4e401b5f0 con 0x7fc4f407be90 2023-09-13T04:30:23.383 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.882+0000 7fc4f9505700 1 -- 172.21.15.2:0/3932201076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4f407be90 msgr2=0x7fc4f419bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:23.384 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.882+0000 7fc4f9505700 1 --2- 172.21.15.2:0/3932201076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4f407be90 0x7fc4f419bbe0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc4f407b360 tx=0x7fc4e4004750 comp rx=0 tx=0).stop 2023-09-13T04:30:23.384 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.882+0000 7fc4f9505700 1 -- 172.21.15.2:0/3932201076 shutdown_connections 2023-09-13T04:30:23.384 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.882+0000 7fc4f9505700 1 --2- 172.21.15.2:0/3932201076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4f407be90 0x7fc4f419bbe0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:23.384 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.882+0000 7fc4f9505700 1 -- 172.21.15.2:0/3932201076 >> 172.21.15.2:0/3932201076 conn(0x7fc4f4101b00 msgr2=0x7fc4f4103440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:30:23.385 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.882+0000 7fc4f9505700 1 -- 172.21.15.2:0/3932201076 shutdown_connections 2023-09-13T04:30:23.385 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:22.882+0000 7fc4f9505700 1 -- 172.21.15.2:0/3932201076 wait complete. 2023-09-13T04:30:23.385 INFO:teuthology.orchestra.run.smithi002.stdout:mon is available 2023-09-13T04:30:23.385 INFO:teuthology.orchestra.run.smithi002.stdout:Assimilating anything we can from ceph.conf... 2023-09-13T04:30:23.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:23 smithi002 ceph-mon[97153]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-13T04:30:23.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:23 smithi002 ceph-mon[97153]: monmap e1: 1 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0]} removed_ranks: {} 2023-09-13T04:30:23.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:23 smithi002 ceph-mon[97153]: fsmap 2023-09-13T04:30:23.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:23 smithi002 ceph-mon[97153]: osdmap e1: 0 total, 0 up, 0 in 2023-09-13T04:30:23.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:23 smithi002 ceph-mon[97153]: mgrmap e1: no daemons active 2023-09-13T04:30:23.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:23 smithi002 ceph-mon[97153]: from='client.? 172.21.15.2:0/3932201076' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-09-13T04:30:25.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:25 smithi002 ceph-mon[97153]: from='client.? 172.21.15.2:0/3038767869' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-09-13T04:30:25.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:25 smithi002 ceph-mon[97153]: from='client.? 172.21.15.2:0/3038767869' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-09-13T04:30:25.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2023-09-13T04:30:25.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [global] 2023-09-13T04:30:25.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout fsid = 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:30:25.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.2:3300,v1:172.21.15.2:6789] 2023-09-13T04:30:25.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-09-13T04:30:25.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-09-13T04:30:25.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-09-13T04:30:25.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-09-13T04:30:25.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2023-09-13T04:30:25.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [mgr] 2023-09-13T04:30:25.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-09-13T04:30:25.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2023-09-13T04:30:25.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [osd] 2023-09-13T04:30:25.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-09-13T04:30:25.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-09-13T04:30:25.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-09-13T04:30:25.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-09-13T04:30:25.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-09-13T04:30:25.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.239+0000 7fb2018f7700 1 Processor -- start 2023-09-13T04:30:25.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.239+0000 7fb2018f7700 1 -- start start 2023-09-13T04:30:25.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.239+0000 7fb2018f7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1fc108730 0x7fb1fc108b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:25.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.239+0000 7fb2018f7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc109120 con 0x7fb1fc108730 2023-09-13T04:30:25.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.239+0000 7fb1faffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1fc108730 0x7fb1fc108b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:25.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.240+0000 7fb1faffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1fc108730 0x7fb1fc108b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58696/0 (socket says 172.21.15.2:58696) 2023-09-13T04:30:25.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.240+0000 7fb1faffd700 1 -- 172.21.15.2:0/2264471289 learned_addr learned my addr 172.21.15.2:0/2264471289 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:30:25.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.240+0000 7fb1faffd700 1 -- 172.21.15.2:0/2264471289 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1fc109930 con 0x7fb1fc108730 2023-09-13T04:30:25.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.240+0000 7fb1faffd700 1 --2- 172.21.15.2:0/2264471289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1fc108730 0x7fb1fc108b50 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec0098a0 tx=0x7fb1ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319fb98c941f3e50 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:25.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.240+0000 7fb1f9ffb700 1 -- 172.21.15.2:0/2264471289 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1ec004160 con 0x7fb1fc108730 2023-09-13T04:30:25.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.240+0000 7fb1f9ffb700 1 -- 172.21.15.2:0/2264471289 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ec0042c0 con 0x7fb1fc108730 2023-09-13T04:30:25.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.240+0000 7fb1f9ffb700 1 -- 172.21.15.2:0/2264471289 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1ec0045a0 con 0x7fb1fc108730 2023-09-13T04:30:25.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.241+0000 7fb2018f7700 1 -- 172.21.15.2:0/2264471289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1fc108730 msgr2=0x7fb1fc108b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:25.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.241+0000 7fb2018f7700 1 --2- 172.21.15.2:0/2264471289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1fc108730 0x7fb1fc108b50 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec0098a0 tx=0x7fb1ec009bb0 comp rx=0 tx=0).stop 2023-09-13T04:30:25.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.241+0000 7fb2018f7700 1 -- 172.21.15.2:0/2264471289 shutdown_connections 2023-09-13T04:30:25.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.241+0000 7fb2018f7700 1 --2- 172.21.15.2:0/2264471289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1fc108730 0x7fb1fc108b50 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:25.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.241+0000 7fb2018f7700 1 -- 172.21.15.2:0/2264471289 >> 172.21.15.2:0/2264471289 conn(0x7fb1fc07bb10 msgr2=0x7fb1fc1064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:30:25.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.241+0000 7fb2018f7700 1 -- 172.21.15.2:0/2264471289 shutdown_connections 2023-09-13T04:30:25.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.241+0000 7fb2018f7700 1 -- 172.21.15.2:0/2264471289 wait complete. 2023-09-13T04:30:25.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.242+0000 7fb2018f7700 1 Processor -- start 2023-09-13T04:30:25.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.242+0000 7fb2018f7700 1 -- start start 2023-09-13T04:30:25.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.242+0000 7fb2018f7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1fc108730 0x7fb1fc19bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:25.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.242+0000 7fb2018f7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc19c1a0 con 0x7fb1fc108730 2023-09-13T04:30:25.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.243+0000 7fb1faffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1fc108730 0x7fb1fc19bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:25.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.243+0000 7fb1faffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1fc108730 0x7fb1fc19bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58698/0 (socket says 172.21.15.2:58698) 2023-09-13T04:30:25.866 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.243+0000 7fb1faffd700 1 -- 172.21.15.2:0/3038767869 learned_addr learned my addr 172.21.15.2:0/3038767869 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:30:25.866 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.243+0000 7fb1faffd700 1 -- 172.21.15.2:0/3038767869 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ec009550 con 0x7fb1fc108730 2023-09-13T04:30:25.866 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.243+0000 7fb1faffd700 1 --2- 172.21.15.2:0/3038767869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1fc108730 0x7fb1fc19bc60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec004000 tx=0x7fb1ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:25.866 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.244+0000 7fb2008f5700 1 -- 172.21.15.2:0/3038767869 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1ec011630 con 0x7fb1fc108730 2023-09-13T04:30:25.866 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.244+0000 7fb2008f5700 1 -- 172.21.15.2:0/3038767869 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ec011790 con 0x7fb1fc108730 2023-09-13T04:30:25.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.244+0000 7fb2018f7700 1 -- 172.21.15.2:0/3038767869 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1fc19c3a0 con 0x7fb1fc108730 2023-09-13T04:30:25.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.244+0000 7fb2008f5700 1 -- 172.21.15.2:0/3038767869 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1ec011a30 con 0x7fb1fc108730 2023-09-13T04:30:25.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.244+0000 7fb2018f7700 1 -- 172.21.15.2:0/3038767869 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1fc19c7c0 con 0x7fb1fc108730 2023-09-13T04:30:25.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.244+0000 7fb2008f5700 1 -- 172.21.15.2:0/3038767869 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb1ec022070 con 0x7fb1fc108730 2023-09-13T04:30:25.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.244+0000 7fb2008f5700 1 -- 172.21.15.2:0/3038767869 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb1ec01c450 con 0x7fb1fc108730 2023-09-13T04:30:25.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.245+0000 7fb2018f7700 1 -- 172.21.15.2:0/3038767869 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1fc1954c0 con 0x7fb1fc108730 2023-09-13T04:30:25.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.248+0000 7fb2008f5700 1 -- 172.21.15.2:0/3038767869 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fb1ec01b450 con 0x7fb1fc108730 2023-09-13T04:30:25.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.321+0000 7fb2018f7700 1 -- 172.21.15.2:0/3038767869 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fb1fc0622e0 con 0x7fb1fc108730 2023-09-13T04:30:25.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.326+0000 7fb2008f5700 1 -- 172.21.15.2:0/3038767869 <== mon.0 v2:172.21.15.2:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb1ec01b5f0 con 0x7fb1fc108730 2023-09-13T04:30:25.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.326+0000 7fb2008f5700 1 -- 172.21.15.2:0/3038767869 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+427 (secure 0 0 0) 0x7fb1ec0375f0 con 0x7fb1fc108730 2023-09-13T04:30:25.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.327+0000 7fb2018f7700 1 -- 172.21.15.2:0/3038767869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1fc108730 msgr2=0x7fb1fc19bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:25.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.327+0000 7fb2018f7700 1 --2- 172.21.15.2:0/3038767869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1fc108730 0x7fb1fc19bc60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec004000 tx=0x7fb1ec004750 comp rx=0 tx=0).stop 2023-09-13T04:30:25.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.327+0000 7fb2018f7700 1 -- 172.21.15.2:0/3038767869 shutdown_connections 2023-09-13T04:30:25.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.327+0000 7fb2018f7700 1 --2- 172.21.15.2:0/3038767869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1fc108730 0x7fb1fc19bc60 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:25.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.327+0000 7fb2018f7700 1 -- 172.21.15.2:0/3038767869 >> 172.21.15.2:0/3038767869 conn(0x7fb1fc07bb10 msgr2=0x7fb1fc190110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:30:25.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.327+0000 7fb2018f7700 1 -- 172.21.15.2:0/3038767869 shutdown_connections 2023-09-13T04:30:25.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:25.327+0000 7fb2018f7700 1 -- 172.21.15.2:0/3038767869 wait complete. 2023-09-13T04:30:25.870 INFO:teuthology.orchestra.run.smithi002.stdout:Generating new minimal ceph.conf... 2023-09-13T04:30:26.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:26 smithi002 ceph-mon[97153]: from='client.? 172.21.15.2:0/619748232' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:30:26.937 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.393+0000 7f82e28c1700 1 Processor -- start 2023-09-13T04:30:26.937 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.393+0000 7f82e28c1700 1 -- start start 2023-09-13T04:30:26.937 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.393+0000 7f82e28c1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82dc07beb0 0x7f82dc07a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:26.938 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.393+0000 7f82e28c1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc07a8e0 con 0x7f82dc07beb0 2023-09-13T04:30:26.938 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.394+0000 7f82dbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82dc07beb0 0x7f82dc07a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:26.938 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.394+0000 7f82dbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82dc07beb0 0x7f82dc07a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58714/0 (socket says 172.21.15.2:58714) 2023-09-13T04:30:26.938 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.394+0000 7f82dbfff700 1 -- 172.21.15.2:0/2695610311 learned_addr learned my addr 172.21.15.2:0/2695610311 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:30:26.939 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.395+0000 7f82dbfff700 1 -- 172.21.15.2:0/2695610311 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc07aa20 con 0x7f82dc07beb0 2023-09-13T04:30:26.939 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.395+0000 7f82dbfff700 1 --2- 172.21.15.2:0/2695610311 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82dc07beb0 0x7f82dc07a310 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0098a0 tx=0x7f82cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53c54bb42cf37cb server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:26.939 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.395+0000 7f82daffd700 1 -- 172.21.15.2:0/2695610311 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82cc004160 con 0x7f82dc07beb0 2023-09-13T04:30:26.939 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.395+0000 7f82daffd700 1 -- 172.21.15.2:0/2695610311 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f82cc0042c0 con 0x7f82dc07beb0 2023-09-13T04:30:26.939 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.395+0000 7f82daffd700 1 -- 172.21.15.2:0/2695610311 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82cc011a00 con 0x7f82dc07beb0 2023-09-13T04:30:26.940 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.396+0000 7f82e28c1700 1 -- 172.21.15.2:0/2695610311 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82dc07beb0 msgr2=0x7f82dc07a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:26.940 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.396+0000 7f82e28c1700 1 --2- 172.21.15.2:0/2695610311 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82dc07beb0 0x7f82dc07a310 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0098a0 tx=0x7f82cc009bb0 comp rx=0 tx=0).stop 2023-09-13T04:30:26.940 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.396+0000 7f82e28c1700 1 -- 172.21.15.2:0/2695610311 shutdown_connections 2023-09-13T04:30:26.940 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.396+0000 7f82e28c1700 1 --2- 172.21.15.2:0/2695610311 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82dc07beb0 0x7f82dc07a310 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:26.940 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.396+0000 7f82e28c1700 1 -- 172.21.15.2:0/2695610311 >> 172.21.15.2:0/2695610311 conn(0x7f82dc101b00 msgr2=0x7f82dc103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:30:26.941 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.396+0000 7f82e28c1700 1 -- 172.21.15.2:0/2695610311 shutdown_connections 2023-09-13T04:30:26.941 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.396+0000 7f82e28c1700 1 -- 172.21.15.2:0/2695610311 wait complete. 2023-09-13T04:30:26.941 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.397+0000 7f82e28c1700 1 Processor -- start 2023-09-13T04:30:26.941 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.397+0000 7f82e28c1700 1 -- start start 2023-09-13T04:30:26.941 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.397+0000 7f82e28c1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82dc07beb0 0x7f82dc19c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:26.942 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.398+0000 7f82e28c1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc19c960 con 0x7f82dc07beb0 2023-09-13T04:30:26.942 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.398+0000 7f82dbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82dc07beb0 0x7f82dc19c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:26.942 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.398+0000 7f82dbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82dc07beb0 0x7f82dc19c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58730/0 (socket says 172.21.15.2:58730) 2023-09-13T04:30:26.942 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.398+0000 7f82dbfff700 1 -- 172.21.15.2:0/619748232 learned_addr learned my addr 172.21.15.2:0/619748232 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:30:26.942 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.398+0000 7f82dbfff700 1 -- 172.21.15.2:0/619748232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82cc009550 con 0x7f82dc07beb0 2023-09-13T04:30:26.943 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.398+0000 7f82dbfff700 1 --2- 172.21.15.2:0/619748232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82dc07beb0 0x7f82dc19c420 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009fb0 tx=0x7f82cc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:26.943 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.399+0000 7f82d97fa700 1 -- 172.21.15.2:0/619748232 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82cc003930 con 0x7f82dc07beb0 2023-09-13T04:30:26.943 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.399+0000 7f82d97fa700 1 -- 172.21.15.2:0/619748232 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f82cc003a90 con 0x7f82dc07beb0 2023-09-13T04:30:26.943 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.399+0000 7f82e28c1700 1 -- 172.21.15.2:0/619748232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82dc19cb60 con 0x7f82dc07beb0 2023-09-13T04:30:26.944 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.399+0000 7f82d97fa700 1 -- 172.21.15.2:0/619748232 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82cc0195e0 con 0x7f82dc07beb0 2023-09-13T04:30:26.944 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.399+0000 7f82e28c1700 1 -- 172.21.15.2:0/619748232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82dc19cf80 con 0x7f82dc07beb0 2023-09-13T04:30:26.944 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.399+0000 7f82d97fa700 1 -- 172.21.15.2:0/619748232 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f82cc019ba0 con 0x7f82dc07beb0 2023-09-13T04:30:26.945 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.399+0000 7f82d97fa700 1 -- 172.21.15.2:0/619748232 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f82cc0243f0 con 0x7f82dc07beb0 2023-09-13T04:30:26.945 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.400+0000 7f82e28c1700 1 -- 172.21.15.2:0/619748232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82dc0622e0 con 0x7f82dc07beb0 2023-09-13T04:30:26.945 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.403+0000 7f82d97fa700 1 -- 172.21.15.2:0/619748232 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f82cc003c00 con 0x7f82dc07beb0 2023-09-13T04:30:26.946 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.476+0000 7f82e28c1700 1 -- 172.21.15.2:0/619748232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f82dc10cb50 con 0x7f82dc07beb0 2023-09-13T04:30:26.946 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.476+0000 7f82d97fa700 1 -- 172.21.15.2:0/619748232 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+173 (secure 0 0 0) 0x7f82cc01a9d0 con 0x7f82dc07beb0 2023-09-13T04:30:26.946 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.477+0000 7f82e28c1700 1 -- 172.21.15.2:0/619748232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82dc07beb0 msgr2=0x7f82dc19c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:26.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.477+0000 7f82e28c1700 1 --2- 172.21.15.2:0/619748232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82dc07beb0 0x7f82dc19c420 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009fb0 tx=0x7f82cc011ad0 comp rx=0 tx=0).stop 2023-09-13T04:30:26.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.477+0000 7f82e28c1700 1 -- 172.21.15.2:0/619748232 shutdown_connections 2023-09-13T04:30:26.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.477+0000 7f82e28c1700 1 --2- 172.21.15.2:0/619748232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82dc07beb0 0x7f82dc19c420 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:26.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.477+0000 7f82e28c1700 1 -- 172.21.15.2:0/619748232 >> 172.21.15.2:0/619748232 conn(0x7f82dc101b00 msgr2=0x7f82dc1031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:30:26.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.477+0000 7f82e28c1700 1 -- 172.21.15.2:0/619748232 shutdown_connections 2023-09-13T04:30:26.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:26.477+0000 7f82e28c1700 1 -- 172.21.15.2:0/619748232 wait complete. 2023-09-13T04:30:26.948 INFO:teuthology.orchestra.run.smithi002.stdout:Restarting the monitor... 2023-09-13T04:30:27.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:26 smithi002 systemd[1]: Stopping Ceph mon.a for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:30:27.721 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:27 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mon-a[97129]: 2023-09-13T04:30:27.434+0000 7ff69e608700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:30:27.721 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:27 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mon-a[97129]: 2023-09-13T04:30:27.434+0000 7ff69e608700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-09-13T04:30:28.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:27 smithi002 bash[97489]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mon-a 2023-09-13T04:30:28.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:28 smithi002 systemd[1]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mon.a.service: Succeeded. 2023-09-13T04:30:28.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:28 smithi002 systemd[1]: Stopped Ceph mon.a for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:30:28.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:28 smithi002 systemd[1]: Starting Ceph mon.a for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:30:29.071 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:28 smithi002 podman[97611]: 2023-09-13T04:30:29.088 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 bash[97611]: 0b778730e2c8cae136863e2b11f23c206843e0c2766bea67f6c88a1d65edab5e 2023-09-13T04:30:29.089 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: set uid:gid to 167:167 (ceph:ceph) 2023-09-13T04:30:29.127 INFO:teuthology.orchestra.run.smithi002.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-09-13T04:30:29.391 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: ceph version 18.0.0-6083-g4a754ac9 (4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5) reef (dev), process ceph-mon, pid 7 2023-09-13T04:30:29.391 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: pidfile_write: ignore empty --pid-file 2023-09-13T04:30:29.391 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: load: jerasure load: lrc 2023-09-13T04:30:29.391 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: RocksDB version: 7.9.2 2023-09-13T04:30:29.392 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Git sha 0 2023-09-13T04:30:29.392 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Compile date 2023-09-12 08:33:53 2023-09-13T04:30:29.392 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: DB SUMMARY 2023-09-13T04:30:29.392 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: DB Session ID: JW13SI3WAJIFSK6STCLV 2023-09-13T04:30:29.392 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: CURRENT file: CURRENT 2023-09-13T04:30:29.393 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: IDENTITY file: IDENTITY 2023-09-13T04:30:29.393 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-09-13T04:30:29.393 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-09-13T04:30:29.393 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99242 ; 2023-09-13T04:30:29.393 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.error_if_exists: 0 2023-09-13T04:30:29.393 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.create_if_missing: 0 2023-09-13T04:30:29.394 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.paranoid_checks: 1 2023-09-13T04:30:29.394 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-13T04:30:29.394 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-13T04:30:29.394 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-13T04:30:29.394 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.env: 0x561bb81ecda0 2023-09-13T04:30:29.395 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.fs: PosixFileSystem 2023-09-13T04:30:29.395 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.info_log: 0x561bb8d314a0 2023-09-13T04:30:29.395 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_file_opening_threads: 16 2023-09-13T04:30:29.395 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.statistics: (nil) 2023-09-13T04:30:29.395 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.use_fsync: 0 2023-09-13T04:30:29.396 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_log_file_size: 0 2023-09-13T04:30:29.396 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-13T04:30:29.396 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-13T04:30:29.396 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.keep_log_file_num: 1000 2023-09-13T04:30:29.396 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.recycle_log_file_num: 0 2023-09-13T04:30:29.396 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.allow_fallocate: 1 2023-09-13T04:30:29.397 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.allow_mmap_reads: 0 2023-09-13T04:30:29.397 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.allow_mmap_writes: 0 2023-09-13T04:30:29.397 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.use_direct_reads: 0 2023-09-13T04:30:29.397 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-13T04:30:29.397 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.create_missing_column_families: 0 2023-09-13T04:30:29.398 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.db_log_dir: 2023-09-13T04:30:29.398 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.wal_dir: 2023-09-13T04:30:29.398 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-13T04:30:29.398 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-13T04:30:29.398 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-13T04:30:29.399 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-13T04:30:29.399 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-13T04:30:29.399 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-13T04:30:29.399 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.advise_random_on_open: 1 2023-09-13T04:30:29.399 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.db_write_buffer_size: 0 2023-09-13T04:30:29.400 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.write_buffer_manager: 0x561bb8ceb4a0 2023-09-13T04:30:29.400 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-13T04:30:29.400 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-13T04:30:29.400 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-13T04:30:29.400 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.rate_limiter: (nil) 2023-09-13T04:30:29.401 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-13T04:30:29.401 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.wal_recovery_mode: 2 2023-09-13T04:30:29.401 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.enable_thread_tracking: 0 2023-09-13T04:30:29.401 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.enable_pipelined_write: 0 2023-09-13T04:30:29.401 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.unordered_write: 0 2023-09-13T04:30:29.402 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-13T04:30:29.402 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-13T04:30:29.402 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-13T04:30:29.402 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-13T04:30:29.402 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.row_cache: None 2023-09-13T04:30:29.402 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.wal_filter: None 2023-09-13T04:30:29.403 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-13T04:30:29.403 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.allow_ingest_behind: 0 2023-09-13T04:30:29.403 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.two_write_queues: 0 2023-09-13T04:30:29.403 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.manual_wal_flush: 0 2023-09-13T04:30:29.403 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.wal_compression: 0 2023-09-13T04:30:29.404 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.atomic_flush: 0 2023-09-13T04:30:29.404 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-13T04:30:29.404 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-13T04:30:29.404 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-13T04:30:29.404 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.log_readahead_size: 0 2023-09-13T04:30:29.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-13T04:30:29.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.best_efforts_recovery: 0 2023-09-13T04:30:29.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-13T04:30:29.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-13T04:30:29.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.allow_data_in_errors: 0 2023-09-13T04:30:29.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.db_host_id: __hostname__ 2023-09-13T04:30:29.406 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-13T04:30:29.406 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_background_jobs: 2 2023-09-13T04:30:29.406 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_background_compactions: -1 2023-09-13T04:30:29.406 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_subcompactions: 1 2023-09-13T04:30:29.406 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-13T04:30:29.407 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-13T04:30:29.407 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-13T04:30:29.407 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_total_wal_size: 0 2023-09-13T04:30:29.407 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-13T04:30:29.407 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-13T04:30:29.407 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-13T04:30:29.408 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-13T04:30:29.408 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_open_files: -1 2023-09-13T04:30:29.408 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.bytes_per_sync: 0 2023-09-13T04:30:29.408 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-13T04:30:29.408 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-13T04:30:29.409 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.compaction_readahead_size: 0 2023-09-13T04:30:29.409 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_background_flushes: -1 2023-09-13T04:30:29.409 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Compression algorithms supported: 2023-09-13T04:30:29.409 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-13T04:30:29.409 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: kZSTD supported: 0 2023-09-13T04:30:29.409 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: kXpressCompression supported: 0 2023-09-13T04:30:29.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: kLZ4HCCompression supported: 1 2023-09-13T04:30:29.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: kZlibCompression supported: 1 2023-09-13T04:30:29.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: kSnappyCompression supported: 1 2023-09-13T04:30:29.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: kLZ4Compression supported: 1 2023-09-13T04:30:29.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: kBZip2Compression supported: 0 2023-09-13T04:30:29.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-13T04:30:29.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-13T04:30:29.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-09-13T04:30:29.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-13T04:30:29.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-13T04:30:29.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.merge_operator: 2023-09-13T04:30:29.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.compaction_filter: None 2023-09-13T04:30:29.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.compaction_filter_factory: None 2023-09-13T04:30:29.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.sst_partitioner_factory: None 2023-09-13T04:30:29.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-13T04:30:29.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-13T04:30:29.413 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561bb8d315a0) 2023-09-13T04:30:29.413 INFO:journalctl@ceph.mon.a.smithi002.stdout: cache_index_and_filter_blocks: 1 2023-09-13T04:30:29.413 INFO:journalctl@ceph.mon.a.smithi002.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-13T04:30:29.413 INFO:journalctl@ceph.mon.a.smithi002.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-13T04:30:29.413 INFO:journalctl@ceph.mon.a.smithi002.stdout: pin_top_level_index_and_filter: 1 2023-09-13T04:30:29.414 INFO:journalctl@ceph.mon.a.smithi002.stdout: index_type: 0 2023-09-13T04:30:29.414 INFO:journalctl@ceph.mon.a.smithi002.stdout: data_block_index_type: 0 2023-09-13T04:30:29.414 INFO:journalctl@ceph.mon.a.smithi002.stdout: index_shortening: 1 2023-09-13T04:30:29.414 INFO:journalctl@ceph.mon.a.smithi002.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-13T04:30:29.414 INFO:journalctl@ceph.mon.a.smithi002.stdout: checksum: 4 2023-09-13T04:30:29.415 INFO:journalctl@ceph.mon.a.smithi002.stdout: no_block_cache: 0 2023-09-13T04:30:29.415 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_cache: 0x561bb8d891f0 2023-09-13T04:30:29.415 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_cache_name: BinnedLRUCache 2023-09-13T04:30:29.415 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_cache_options: 2023-09-13T04:30:29.415 INFO:journalctl@ceph.mon.a.smithi002.stdout: capacity : 536870912 2023-09-13T04:30:29.415 INFO:journalctl@ceph.mon.a.smithi002.stdout: num_shard_bits : 4 2023-09-13T04:30:29.416 INFO:journalctl@ceph.mon.a.smithi002.stdout: strict_capacity_limit : 0 2023-09-13T04:30:29.416 INFO:journalctl@ceph.mon.a.smithi002.stdout: high_pri_pool_ratio: 0.000 2023-09-13T04:30:29.416 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_cache_compressed: (nil) 2023-09-13T04:30:29.416 INFO:journalctl@ceph.mon.a.smithi002.stdout: persistent_cache: (nil) 2023-09-13T04:30:29.416 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_size: 4096 2023-09-13T04:30:29.416 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_size_deviation: 10 2023-09-13T04:30:29.417 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_restart_interval: 16 2023-09-13T04:30:29.417 INFO:journalctl@ceph.mon.a.smithi002.stdout: index_block_restart_interval: 1 2023-09-13T04:30:29.417 INFO:journalctl@ceph.mon.a.smithi002.stdout: metadata_block_size: 4096 2023-09-13T04:30:29.417 INFO:journalctl@ceph.mon.a.smithi002.stdout: partition_filters: 0 2023-09-13T04:30:29.417 INFO:journalctl@ceph.mon.a.smithi002.stdout: use_delta_encoding: 1 2023-09-13T04:30:29.417 INFO:journalctl@ceph.mon.a.smithi002.stdout: filter_policy: bloomfilter 2023-09-13T04:30:29.418 INFO:journalctl@ceph.mon.a.smithi002.stdout: whole_key_filtering: 1 2023-09-13T04:30:29.418 INFO:journalctl@ceph.mon.a.smithi002.stdout: verify_compression: 0 2023-09-13T04:30:29.418 INFO:journalctl@ceph.mon.a.smithi002.stdout: read_amp_bytes_per_bit: 0 2023-09-13T04:30:29.418 INFO:journalctl@ceph.mon.a.smithi002.stdout: format_version: 5 2023-09-13T04:30:29.418 INFO:journalctl@ceph.mon.a.smithi002.stdout: enable_index_compression: 1 2023-09-13T04:30:29.419 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_align: 0 2023-09-13T04:30:29.419 INFO:journalctl@ceph.mon.a.smithi002.stdout: max_auto_readahead_size: 262144 2023-09-13T04:30:29.419 INFO:journalctl@ceph.mon.a.smithi002.stdout: prepopulate_block_cache: 0 2023-09-13T04:30:29.419 INFO:journalctl@ceph.mon.a.smithi002.stdout: initial_auto_readahead_size: 8192 2023-09-13T04:30:29.419 INFO:journalctl@ceph.mon.a.smithi002.stdout: num_file_reads_for_auto_readahead: 2 2023-09-13T04:30:29.419 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.write_buffer_size: 33554432 2023-09-13T04:30:29.420 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_write_buffer_number: 2 2023-09-13T04:30:29.420 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.compression: NoCompression 2023-09-13T04:30:29.420 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.bottommost_compression: Disabled 2023-09-13T04:30:29.420 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.prefix_extractor: nullptr 2023-09-13T04:30:29.420 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-13T04:30:29.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.num_levels: 7 2023-09-13T04:30:29.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-13T04:30:29.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-13T04:30:29.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-13T04:30:29.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-13T04:30:29.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-13T04:30:29.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-13T04:30:29.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-13T04:30:29.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-13T04:30:29.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-13T04:30:29.423 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-13T04:30:29.423 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-13T04:30:29.424 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-13T04:30:29.424 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-13T04:30:29.424 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.compression_opts.level: 32767 2023-09-13T04:30:29.424 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.compression_opts.strategy: 0 2023-09-13T04:30:29.424 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-13T04:30:29.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-13T04:30:29.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-13T04:30:29.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-13T04:30:29.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.compression_opts.enabled: false 2023-09-13T04:30:29.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-13T04:30:29.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-13T04:30:29.426 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-13T04:30:29.426 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-13T04:30:29.426 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.target_file_size_base: 67108864 2023-09-13T04:30:29.426 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-13T04:30:29.426 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-13T04:30:29.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-13T04:30:29.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-13T04:30:29.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-13T04:30:29.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-13T04:30:29.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-13T04:30:29.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-13T04:30:29.428 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-13T04:30:29.428 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-13T04:30:29.428 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-13T04:30:29.428 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-13T04:30:29.428 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-13T04:30:29.429 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-13T04:30:29.429 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.arena_block_size: 1048576 2023-09-13T04:30:29.429 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-13T04:30:29.429 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-13T04:30:29.429 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.disable_auto_compactions: 0 2023-09-13T04:30:29.429 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-13T04:30:29.430 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-13T04:30:29.430 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-13T04:30:29.430 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-13T04:30:29.430 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-13T04:30:29.430 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-13T04:30:29.431 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-13T04:30:29.431 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-13T04:30:29.431 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-13T04:30:29.431 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-13T04:30:29.431 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.table_properties_collectors: 2023-09-13T04:30:29.432 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.inplace_update_support: 0 2023-09-13T04:30:29.432 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-13T04:30:29.432 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-13T04:30:29.432 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-13T04:30:29.432 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-13T04:30:29.433 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.bloom_locality: 0 2023-09-13T04:30:29.433 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.max_successive_merges: 0 2023-09-13T04:30:29.433 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-13T04:30:29.433 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.paranoid_file_checks: 0 2023-09-13T04:30:29.433 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.force_consistency_checks: 1 2023-09-13T04:30:29.433 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.report_bg_io_stats: 0 2023-09-13T04:30:29.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.ttl: 2592000 2023-09-13T04:30:29.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-13T04:30:29.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-13T04:30:29.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-13T04:30:29.435 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.enable_blob_files: false 2023-09-13T04:30:29.435 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.min_blob_size: 0 2023-09-13T04:30:29.435 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.blob_file_size: 268435456 2023-09-13T04:30:29.435 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-13T04:30:29.435 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-13T04:30:29.435 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-13T04:30:29.436 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-13T04:30:29.436 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-13T04:30:29.436 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.blob_file_starting_level: 0 2023-09-13T04:30:29.436 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-13T04:30:29.436 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-09-13T04:30:29.436 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-09-13T04:30:29.437 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 39ccb145-0f13-4a60-9115-8b7d1ca681f6 2023-09-13T04:30:29.437 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694579429093642, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-09-13T04:30:29.437 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-09-13T04:30:29.437 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694579429095841, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94249, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 323, "table_properties": {"data_size": 92254, "index_size": 277, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13658, "raw_average_key_size": 49, "raw_value_size": 84794, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 274, "num_filter_entries": 274, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1694579429, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "39ccb145-0f13-4a60-9115-8b7d1ca681f6", "db_session_id": "JW13SI3WAJIFSK6STCLV", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-09-13T04:30:29.437 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694579429095992, "job": 1, "event": "recovery_finished"} 2023-09-13T04:30:29.438 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-09-13T04:30:29.438 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-13T04:30:29.438 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561bb8e26000 2023-09-13T04:30:29.438 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: DB pointer 0x561bb8e12000 2023-09-13T04:30:29.438 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-13T04:30:29.439 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-13T04:30:29.439 INFO:journalctl@ceph.mon.a.smithi002.stdout: ** DB Stats ** 2023-09-13T04:30:29.439 INFO:journalctl@ceph.mon.a.smithi002.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-13T04:30:29.439 INFO:journalctl@ceph.mon.a.smithi002.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-13T04:30:29.439 INFO:journalctl@ceph.mon.a.smithi002.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-13T04:30:29.439 INFO:journalctl@ceph.mon.a.smithi002.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-13T04:30:29.440 INFO:journalctl@ceph.mon.a.smithi002.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-13T04:30:29.440 INFO:journalctl@ceph.mon.a.smithi002.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-13T04:30:29.440 INFO:journalctl@ceph.mon.a.smithi002.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-13T04:30:29.440 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2023-09-13T04:30:29.440 INFO:journalctl@ceph.mon.a.smithi002.stdout: ** Compaction Stats [default] ** 2023-09-13T04:30:29.441 INFO:journalctl@ceph.mon.a.smithi002.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-13T04:30:29.441 INFO:journalctl@ceph.mon.a.smithi002.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-13T04:30:29.441 INFO:journalctl@ceph.mon.a.smithi002.stdout: L0 2/0 93.88 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 59.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2023-09-13T04:30:29.441 INFO:journalctl@ceph.mon.a.smithi002.stdout: Sum 2/0 93.88 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 59.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2023-09-13T04:30:29.441 INFO:journalctl@ceph.mon.a.smithi002.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 59.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2023-09-13T04:30:29.441 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2023-09-13T04:30:29.442 INFO:journalctl@ceph.mon.a.smithi002.stdout: ** Compaction Stats [default] ** 2023-09-13T04:30:29.442 INFO:journalctl@ceph.mon.a.smithi002.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-13T04:30:29.442 INFO:journalctl@ceph.mon.a.smithi002.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-13T04:30:29.442 INFO:journalctl@ceph.mon.a.smithi002.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 59.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2023-09-13T04:30:29.442 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2023-09-13T04:30:29.442 INFO:journalctl@ceph.mon.a.smithi002.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-13T04:30:29.443 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2023-09-13T04:30:29.443 INFO:journalctl@ceph.mon.a.smithi002.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-13T04:30:29.443 INFO:journalctl@ceph.mon.a.smithi002.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-13T04:30:29.443 INFO:journalctl@ceph.mon.a.smithi002.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-13T04:30:29.443 INFO:journalctl@ceph.mon.a.smithi002.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-13T04:30:29.444 INFO:journalctl@ceph.mon.a.smithi002.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-13T04:30:29.444 INFO:journalctl@ceph.mon.a.smithi002.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-13T04:30:29.444 INFO:journalctl@ceph.mon.a.smithi002.stdout: Cumulative compaction: 0.00 GB write, 11.35 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-13T04:30:29.444 INFO:journalctl@ceph.mon.a.smithi002.stdout: Interval compaction: 0.00 GB write, 11.35 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-13T04:30:29.444 INFO:journalctl@ceph.mon.a.smithi002.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-13T04:30:29.445 INFO:journalctl@ceph.mon.a.smithi002.stdout: Block cache BinnedLRUCache@0x561bb8d891f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-09-13T04:30:29.445 INFO:journalctl@ceph.mon.a.smithi002.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-09-13T04:30:29.445 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2023-09-13T04:30:29.445 INFO:journalctl@ceph.mon.a.smithi002.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-13T04:30:29.445 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: starting mon.a rank 0 at public addrs [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] at bind addrs [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:30:29.446 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: mon.a@-1(???) e1 preinit fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:30:29.446 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: mon.a@-1(???).mds e1 new map 2023-09-13T04:30:29.446 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: mon.a@-1(???).mds e1 print_map 2023-09-13T04:30:29.446 INFO:journalctl@ceph.mon.a.smithi002.stdout: e1 2023-09-13T04:30:29.446 INFO:journalctl@ceph.mon.a.smithi002.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-13T04:30:29.447 INFO:journalctl@ceph.mon.a.smithi002.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-13T04:30:29.447 INFO:journalctl@ceph.mon.a.smithi002.stdout: legacy client fscid: -1 2023-09-13T04:30:29.447 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2023-09-13T04:30:29.447 INFO:journalctl@ceph.mon.a.smithi002.stdout: No filesystems configured 2023-09-13T04:30:29.447 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-13T04:30:29.447 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-13T04:30:29.448 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-13T04:30:29.448 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-13T04:30:29.448 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-09-13T04:30:29.448 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: expand_channel_meta expand map: {default=false} 2023-09-13T04:30:29.448 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: expand_channel_meta from 'false' to 'false' 2023-09-13T04:30:29.449 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: expand_channel_meta expanded map: {default=false} 2023-09-13T04:30:29.449 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: expand_channel_meta expand map: {default=info} 2023-09-13T04:30:29.449 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: expand_channel_meta from 'info' to 'info' 2023-09-13T04:30:29.449 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: expand_channel_meta expanded map: {default=info} 2023-09-13T04:30:29.449 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: expand_channel_meta expand map: {default=daemon} 2023-09-13T04:30:29.449 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: expand_channel_meta from 'daemon' to 'daemon' 2023-09-13T04:30:29.450 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: expand_channel_meta expanded map: {default=daemon} 2023-09-13T04:30:29.450 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: expand_channel_meta expand map: {default=debug} 2023-09-13T04:30:29.450 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: expand_channel_meta from 'debug' to 'debug' 2023-09-13T04:30:29.450 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: expand_channel_meta expanded map: {default=debug} 2023-09-13T04:30:29.450 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 systemd[1]: Started Ceph mon.a for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:30:29.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-13T04:30:29.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: monmap e1: 1 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0]} removed_ranks: {} 2023-09-13T04:30:29.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: fsmap 2023-09-13T04:30:29.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: osdmap e1: 0 total, 0 up, 0 in 2023-09-13T04:30:29.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:29 smithi002 ceph-mon[97651]: mgrmap e1: no daemons active 2023-09-13T04:30:30.212 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.655+0000 7f3bc926c700 1 Processor -- start 2023-09-13T04:30:30.212 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.655+0000 7f3bc926c700 1 -- start start 2023-09-13T04:30:30.212 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.656+0000 7f3bc926c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bc4106520 0x7f3bc4106940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:30.213 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.656+0000 7f3bc926c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc4106f10 con 0x7f3bc4106520 2023-09-13T04:30:30.213 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.656+0000 7f3bc2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bc4106520 0x7f3bc4106940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:30.213 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.657+0000 7f3bc2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bc4106520 0x7f3bc4106940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58736/0 (socket says 172.21.15.2:58736) 2023-09-13T04:30:30.213 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.657+0000 7f3bc2ffd700 1 -- 172.21.15.2:0/2606583982 learned_addr learned my addr 172.21.15.2:0/2606583982 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:30:30.213 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.657+0000 7f3bc2ffd700 1 -- 172.21.15.2:0/2606583982 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc4107720 con 0x7f3bc4106520 2023-09-13T04:30:30.214 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.657+0000 7f3bc2ffd700 1 --2- 172.21.15.2:0/2606583982 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bc4106520 0x7f3bc4106940 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40098a0 tx=0x7f3bb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db807e4833cf828 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:30.214 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.657+0000 7f3bc1ffb700 1 -- 172.21.15.2:0/2606583982 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3bb4004160 con 0x7f3bc4106520 2023-09-13T04:30:30.214 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.657+0000 7f3bc1ffb700 1 -- 172.21.15.2:0/2606583982 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3bb40042c0 con 0x7f3bc4106520 2023-09-13T04:30:30.215 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.657+0000 7f3bc1ffb700 1 -- 172.21.15.2:0/2606583982 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3bb4011a00 con 0x7f3bc4106520 2023-09-13T04:30:30.215 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.658+0000 7f3bc926c700 1 -- 172.21.15.2:0/2606583982 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bc4106520 msgr2=0x7f3bc4106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:30.215 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.658+0000 7f3bc926c700 1 --2- 172.21.15.2:0/2606583982 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bc4106520 0x7f3bc4106940 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40098a0 tx=0x7f3bb4009bb0 comp rx=0 tx=0).stop 2023-09-13T04:30:30.215 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.658+0000 7f3bc926c700 1 -- 172.21.15.2:0/2606583982 shutdown_connections 2023-09-13T04:30:30.215 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.658+0000 7f3bc926c700 1 --2- 172.21.15.2:0/2606583982 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bc4106520 0x7f3bc4106940 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:30.216 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.658+0000 7f3bc926c700 1 -- 172.21.15.2:0/2606583982 >> 172.21.15.2:0/2606583982 conn(0x7f3bc4101b00 msgr2=0x7f3bc4103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:30:30.216 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.658+0000 7f3bc926c700 1 -- 172.21.15.2:0/2606583982 shutdown_connections 2023-09-13T04:30:30.216 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.658+0000 7f3bc926c700 1 -- 172.21.15.2:0/2606583982 wait complete. 2023-09-13T04:30:30.216 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.659+0000 7f3bc926c700 1 Processor -- start 2023-09-13T04:30:30.216 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.659+0000 7f3bc926c700 1 -- start start 2023-09-13T04:30:30.216 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.660+0000 7f3bc926c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bc4106520 0x7f3bc419c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:30.217 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.660+0000 7f3bc926c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc419c970 con 0x7f3bc4106520 2023-09-13T04:30:30.217 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.660+0000 7f3bc2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bc4106520 0x7f3bc419c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:30.217 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.660+0000 7f3bc2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bc4106520 0x7f3bc419c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58740/0 (socket says 172.21.15.2:58740) 2023-09-13T04:30:30.217 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.660+0000 7f3bc2ffd700 1 -- 172.21.15.2:0/4276648120 learned_addr learned my addr 172.21.15.2:0/4276648120 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:30:30.217 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.660+0000 7f3bc2ffd700 1 -- 172.21.15.2:0/4276648120 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb4009550 con 0x7f3bc4106520 2023-09-13T04:30:30.218 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.661+0000 7f3bc2ffd700 1 --2- 172.21.15.2:0/4276648120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bc4106520 0x7f3bc419c430 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4006af0 tx=0x7f3bb4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:30.218 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.661+0000 7f3babfff700 1 -- 172.21.15.2:0/4276648120 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3bb4003a40 con 0x7f3bc4106520 2023-09-13T04:30:30.218 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.661+0000 7f3babfff700 1 -- 172.21.15.2:0/4276648120 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3bb4003ba0 con 0x7f3bc4106520 2023-09-13T04:30:30.218 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.661+0000 7f3babfff700 1 -- 172.21.15.2:0/4276648120 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3bb401a700 con 0x7f3bc4106520 2023-09-13T04:30:30.219 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.661+0000 7f3bc926c700 1 -- 172.21.15.2:0/4276648120 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc419cb70 con 0x7f3bc4106520 2023-09-13T04:30:30.219 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.661+0000 7f3bc926c700 1 -- 172.21.15.2:0/4276648120 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc419cf90 con 0x7f3bc4106520 2023-09-13T04:30:30.219 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.661+0000 7f3babfff700 1 -- 172.21.15.2:0/4276648120 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3bb401ac30 con 0x7f3bc4106520 2023-09-13T04:30:30.219 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.661+0000 7f3babfff700 1 -- 172.21.15.2:0/4276648120 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3bb4019700 con 0x7f3bc4106520 2023-09-13T04:30:30.219 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.662+0000 7f3bc926c700 1 -- 172.21.15.2:0/4276648120 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc40622e0 con 0x7f3bc4106520 2023-09-13T04:30:30.219 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.665+0000 7f3babfff700 1 -- 172.21.15.2:0/4276648120 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f3bb402de40 con 0x7f3bc4106520 2023-09-13T04:30:30.220 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.738+0000 7f3bc926c700 1 -- 172.21.15.2:0/4276648120 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f3bc410a920 con 0x7f3bc4106520 2023-09-13T04:30:30.220 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.741+0000 7f3babfff700 1 -- 172.21.15.2:0/4276648120 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3bb4024ad0 con 0x7f3bc4106520 2023-09-13T04:30:30.220 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.742+0000 7f3bc926c700 1 -- 172.21.15.2:0/4276648120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bc4106520 msgr2=0x7f3bc419c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:30.220 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.742+0000 7f3bc926c700 1 --2- 172.21.15.2:0/4276648120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bc4106520 0x7f3bc419c430 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4006af0 tx=0x7f3bb4011ad0 comp rx=0 tx=0).stop 2023-09-13T04:30:30.220 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.742+0000 7f3bc926c700 1 -- 172.21.15.2:0/4276648120 shutdown_connections 2023-09-13T04:30:30.221 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.742+0000 7f3bc926c700 1 --2- 172.21.15.2:0/4276648120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bc4106520 0x7f3bc419c430 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:30.221 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.742+0000 7f3bc926c700 1 -- 172.21.15.2:0/4276648120 >> 172.21.15.2:0/4276648120 conn(0x7f3bc4101b00 msgr2=0x7f3bc41031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:30:30.221 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.742+0000 7f3bc926c700 1 -- 172.21.15.2:0/4276648120 shutdown_connections 2023-09-13T04:30:30.221 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:29.742+0000 7f3bc926c700 1 -- 172.21.15.2:0/4276648120 wait complete. 2023-09-13T04:30:30.221 INFO:teuthology.orchestra.run.smithi002.stdout:Wrote config to /etc/ceph/ceph.conf 2023-09-13T04:30:30.266 INFO:teuthology.orchestra.run.smithi002.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-09-13T04:30:30.266 INFO:teuthology.orchestra.run.smithi002.stdout:Creating mgr... 2023-09-13T04:30:30.267 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying port 0.0.0.0:9283 ... 2023-09-13T04:30:30.267 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying port 0.0.0.0:8765 ... 2023-09-13T04:30:30.639 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mgr.x 2023-09-13T04:30:30.639 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to reset failed state of unit ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mgr.x.service: Unit ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mgr.x.service not loaded. 2023-09-13T04:30:30.847 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da.target.wants/ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mgr.x.service → /etc/systemd/system/ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@.service. 2023-09-13T04:30:31.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:30 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/4276648120' entity='client.admin' 2023-09-13T04:30:31.007 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:30 smithi002 systemd[1]: Starting Ceph mgr.x for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:30:31.639 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:31 smithi002 podman[97878]: 2023-09-13T04:30:31.640 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:31 smithi002 bash[97878]: b25c57d8eb9f25dc76ae4e60cdf4746c613133773aeb52425a94c058bc4d473b 2023-09-13T04:30:31.663 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-13T04:30:31.663 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2023-09-13T04:30:31.680 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-13T04:30:31.680 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2023-09-13T04:30:31.680 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2023-09-13T04:30:31.680 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to enable service . firewalld.service is not available 2023-09-13T04:30:31.698 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-13T04:30:31.698 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2023-09-13T04:30:31.714 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-13T04:30:31.715 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2023-09-13T04:30:31.715 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2023-09-13T04:30:31.715 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-09-13T04:30:31.715 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr to start... 2023-09-13T04:30:31.716 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr... 2023-09-13T04:30:31.900 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:31 smithi002 systemd[1]: Started Ceph mgr.x for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:30:31.900 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:31 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:31.788+0000 7f1bbc019200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-13T04:30:31.900 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:31 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:31.899+0000 7f1bbc019200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-13T04:30:32.151 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:32 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:32.019+0000 7f1bbc019200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-13T04:30:32.423 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:32 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:32.150+0000 7f1bbc019200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-13T04:30:32.424 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:32 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:32.388+0000 7f1bbc019200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-13T04:30:32.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:32 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2335504835' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-13T04:30:32.914 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2023-09-13T04:30:32.923 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2023-09-13T04:30:32.923 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsid": "22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da", 2023-09-13T04:30:32.924 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "health": { 2023-09-13T04:30:32.924 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-13T04:30:32.924 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-13T04:30:32.924 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-13T04:30:32.924 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-09-13T04:30:32.925 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-13T04:30:32.925 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-13T04:30:32.925 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 0 2023-09-13T04:30:32.925 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2023-09-13T04:30:32.925 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-13T04:30:32.926 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "a" 2023-09-13T04:30:32.926 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2023-09-13T04:30:32.926 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-09-13T04:30:32.926 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-13T04:30:32.926 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-13T04:30:32.927 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-13T04:30:32.927 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-13T04:30:32.927 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-09-13T04:30:32.927 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-13T04:30:32.927 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-13T04:30:32.927 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-13T04:30:32.928 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-13T04:30:32.928 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-13T04:30:32.928 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-13T04:30:32.928 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-13T04:30:32.928 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-13T04:30:32.929 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-09-13T04:30:32.929 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-13T04:30:32.929 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-13T04:30:32.929 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-13T04:30:32.929 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-13T04:30:32.929 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-13T04:30:32.930 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-13T04:30:32.930 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-13T04:30:32.930 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-13T04:30:32.930 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-13T04:30:32.930 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-09-13T04:30:32.931 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-13T04:30:32.931 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-13T04:30:32.931 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-13T04:30:32.931 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-13T04:30:32.931 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-09-13T04:30:32.931 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-13T04:30:32.932 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": false, 2023-09-13T04:30:32.932 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-13T04:30:32.932 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-13T04:30:32.932 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-13T04:30:32.932 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-13T04:30:32.933 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "restful" 2023-09-13T04:30:32.933 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2023-09-13T04:30:32.933 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-13T04:30:32.933 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-09-13T04:30:32.933 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-13T04:30:32.934 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-13T04:30:32.934 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modified": "2023-09-13T04:30:22.165522+0000", 2023-09-13T04:30:32.934 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-13T04:30:32.934 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-09-13T04:30:32.934 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-13T04:30:32.935 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2023-09-13T04:30:32.935 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.312+0000 7fc0a7486700 1 Processor -- start 2023-09-13T04:30:32.935 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.312+0000 7fc0a7486700 1 -- start start 2023-09-13T04:30:32.935 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.313+0000 7fc0a7486700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0a007be90 0x7fc0a007a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:32.935 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.313+0000 7fc0a7486700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a007a8c0 con 0x7fc0a007be90 2023-09-13T04:30:32.936 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.313+0000 7fc0a5222700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0a007be90 0x7fc0a007a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:32.936 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.313+0000 7fc0a5222700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0a007be90 0x7fc0a007a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58760/0 (socket says 172.21.15.2:58760) 2023-09-13T04:30:32.936 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.313+0000 7fc0a5222700 1 -- 172.21.15.2:0/1167063362 learned_addr learned my addr 172.21.15.2:0/1167063362 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:30:32.936 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.314+0000 7fc0a5222700 1 -- 172.21.15.2:0/1167063362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a007aa00 con 0x7fc0a007be90 2023-09-13T04:30:32.937 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.314+0000 7fc0a5222700 1 --2- 172.21.15.2:0/1167063362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0a007be90 0x7fc0a007a2f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0098a0 tx=0x7fc09c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75331efe89a4f187 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:32.937 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.314+0000 7fc097fff700 1 -- 172.21.15.2:0/1167063362 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc09c004160 con 0x7fc0a007be90 2023-09-13T04:30:32.937 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.314+0000 7fc097fff700 1 -- 172.21.15.2:0/1167063362 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc09c0042c0 con 0x7fc0a007be90 2023-09-13T04:30:32.937 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.314+0000 7fc097fff700 1 -- 172.21.15.2:0/1167063362 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc09c011a00 con 0x7fc0a007be90 2023-09-13T04:30:32.937 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.315+0000 7fc0a7486700 1 -- 172.21.15.2:0/1167063362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0a007be90 msgr2=0x7fc0a007a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:32.938 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.315+0000 7fc0a7486700 1 --2- 172.21.15.2:0/1167063362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0a007be90 0x7fc0a007a2f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0098a0 tx=0x7fc09c009bb0 comp rx=0 tx=0).stop 2023-09-13T04:30:32.938 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.315+0000 7fc0a7486700 1 -- 172.21.15.2:0/1167063362 shutdown_connections 2023-09-13T04:30:32.938 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.315+0000 7fc0a7486700 1 --2- 172.21.15.2:0/1167063362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0a007be90 0x7fc0a007a2f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:32.938 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.315+0000 7fc0a7486700 1 -- 172.21.15.2:0/1167063362 >> 172.21.15.2:0/1167063362 conn(0x7fc0a0101b00 msgr2=0x7fc0a0103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:30:32.939 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.315+0000 7fc0a7486700 1 -- 172.21.15.2:0/1167063362 shutdown_connections 2023-09-13T04:30:32.939 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.315+0000 7fc0a7486700 1 -- 172.21.15.2:0/1167063362 wait complete. 2023-09-13T04:30:32.939 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.316+0000 7fc0a7486700 1 Processor -- start 2023-09-13T04:30:32.939 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.316+0000 7fc0a7486700 1 -- start start 2023-09-13T04:30:32.939 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.317+0000 7fc0a7486700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0a007be90 0x7fc0a019c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:32.940 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.317+0000 7fc0a7486700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a019c950 con 0x7fc0a007be90 2023-09-13T04:30:32.940 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.317+0000 7fc0a5222700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0a007be90 0x7fc0a019c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:32.940 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.317+0000 7fc0a5222700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0a007be90 0x7fc0a019c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58764/0 (socket says 172.21.15.2:58764) 2023-09-13T04:30:32.940 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.317+0000 7fc0a5222700 1 -- 172.21.15.2:0/2335504835 learned_addr learned my addr 172.21.15.2:0/2335504835 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:30:32.940 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.317+0000 7fc0a5222700 1 -- 172.21.15.2:0/2335504835 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09c009550 con 0x7fc0a007be90 2023-09-13T04:30:32.941 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.318+0000 7fc0a5222700 1 --2- 172.21.15.2:0/2335504835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0a007be90 0x7fc0a019c410 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009fb0 tx=0x7fc09c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:32.941 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.318+0000 7fc0967fc700 1 -- 172.21.15.2:0/2335504835 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc09c003930 con 0x7fc0a007be90 2023-09-13T04:30:32.941 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.318+0000 7fc0967fc700 1 -- 172.21.15.2:0/2335504835 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc09c003a90 con 0x7fc0a007be90 2023-09-13T04:30:32.941 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.318+0000 7fc0a7486700 1 -- 172.21.15.2:0/2335504835 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a019cb50 con 0x7fc0a007be90 2023-09-13T04:30:32.941 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.318+0000 7fc0967fc700 1 -- 172.21.15.2:0/2335504835 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc09c019610 con 0x7fc0a007be90 2023-09-13T04:30:32.942 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.318+0000 7fc0a7486700 1 -- 172.21.15.2:0/2335504835 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a019cf70 con 0x7fc0a007be90 2023-09-13T04:30:32.942 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.318+0000 7fc0967fc700 1 -- 172.21.15.2:0/2335504835 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc09c019b80 con 0x7fc0a007be90 2023-09-13T04:30:32.942 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.318+0000 7fc0967fc700 1 -- 172.21.15.2:0/2335504835 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc09c0243f0 con 0x7fc0a007be90 2023-09-13T04:30:32.942 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.319+0000 7fc0a7486700 1 -- 172.21.15.2:0/2335504835 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a00622e0 con 0x7fc0a007be90 2023-09-13T04:30:32.942 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.322+0000 7fc0967fc700 1 -- 172.21.15.2:0/2335504835 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fc09c02de40 con 0x7fc0a007be90 2023-09-13T04:30:32.943 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.400+0000 7fc0a7486700 1 -- 172.21.15.2:0/2335504835 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc0a010c910 con 0x7fc0a007be90 2023-09-13T04:30:32.943 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.400+0000 7fc0967fc700 1 -- 172.21.15.2:0/2335504835 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fc09c003c00 con 0x7fc0a007be90 2023-09-13T04:30:32.943 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.401+0000 7fc0a7486700 1 -- 172.21.15.2:0/2335504835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0a007be90 msgr2=0x7fc0a019c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:32.943 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.401+0000 7fc0a7486700 1 --2- 172.21.15.2:0/2335504835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0a007be90 0x7fc0a019c410 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009fb0 tx=0x7fc09c011ad0 comp rx=0 tx=0).stop 2023-09-13T04:30:32.943 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.401+0000 7fc0a7486700 1 -- 172.21.15.2:0/2335504835 shutdown_connections 2023-09-13T04:30:32.944 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.401+0000 7fc0a7486700 1 --2- 172.21.15.2:0/2335504835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0a007be90 0x7fc0a019c410 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:32.944 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.401+0000 7fc0a7486700 1 -- 172.21.15.2:0/2335504835 >> 172.21.15.2:0/2335504835 conn(0x7fc0a0101b00 msgr2=0x7fc0a01031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:30:32.944 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.401+0000 7fc0a7486700 1 -- 172.21.15.2:0/2335504835 shutdown_connections 2023-09-13T04:30:32.944 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:32.401+0000 7fc0a7486700 1 -- 172.21.15.2:0/2335504835 wait complete. 2023-09-13T04:30:32.944 INFO:teuthology.orchestra.run.smithi002.stdout:mgr not available, waiting (1/15)... 2023-09-13T04:30:33.176 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:32 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:32.887+0000 7f1bbc019200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-13T04:30:33.177 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:33 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:33.056+0000 7f1bbc019200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-13T04:30:33.505 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:33 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:33.176+0000 7f1bbc019200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-13T04:30:33.506 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:33 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:33.306+0000 7f1bbc019200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-13T04:30:34.255 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:33 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:33.872+0000 7f1bbc019200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-13T04:30:35.256 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:34 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:34.789+0000 7f1bbc019200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-13T04:30:35.256 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:34 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:34.903+0000 7f1bbc019200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-13T04:30:36.255 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:35 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:35.931+0000 7f1bbc019200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-13T04:30:36.255 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:36 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:36.075+0000 7f1bbc019200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-13T04:30:36.687 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:36 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:36.308+0000 7f1bbc019200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-13T04:30:36.969 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:36 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:36.686+0000 7f1bbc019200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-13T04:30:37.255 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:36 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:36.969+0000 7f1bbc019200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-13T04:30:37.255 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:37 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:37.170+0000 7f1bbc019200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-13T04:30:38.005 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:37 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:37.523+0000 7f1bbc019200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-13T04:30:38.924 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:38 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:38.488+0000 7f1bbc019200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-13T04:30:38.925 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:38 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:38.670+0000 7f1bbc019200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-13T04:30:39.255 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:38 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:38.924+0000 7f1bbc019200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-13T04:30:39.256 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:39 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:39.082+0000 7f1bbc019200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-13T04:30:39.755 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:39 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:39.372+0000 7f1bbc019200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-13T04:30:40.755 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:40 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:40.484+0000 7f1bbc019200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-13T04:30:40.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:40 smithi002 ceph-mon[97651]: Activating manager daemon x 2023-09-13T04:30:40.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:40 smithi002 ceph-mon[97651]: mgrmap e2: x(active, starting, since 0.00429241s) 2023-09-13T04:30:40.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:40 smithi002 ceph-mon[97651]: from='mgr.14100 172.21.15.2:0/1508766245' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-13T04:30:40.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:40 smithi002 ceph-mon[97651]: from='mgr.14100 172.21.15.2:0/1508766245' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-13T04:30:40.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:40 smithi002 ceph-mon[97651]: from='mgr.14100 172.21.15.2:0/1508766245' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-13T04:30:40.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:40 smithi002 ceph-mon[97651]: from='mgr.14100 172.21.15.2:0/1508766245' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T04:30:40.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:40 smithi002 ceph-mon[97651]: from='mgr.14100 172.21.15.2:0/1508766245' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-13T04:30:40.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:40 smithi002 ceph-mon[97651]: Manager daemon x is now available 2023-09-13T04:30:40.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:40 smithi002 ceph-mon[97651]: from='mgr.14100 172.21.15.2:0/1508766245' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-13T04:30:40.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:40 smithi002 ceph-mon[97651]: from='mgr.14100 172.21.15.2:0/1508766245' entity='mgr.x' 2023-09-13T04:30:40.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:40 smithi002 ceph-mon[97651]: from='mgr.14100 172.21.15.2:0/1508766245' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-13T04:30:40.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:40 smithi002 ceph-mon[97651]: from='mgr.14100 172.21.15.2:0/1508766245' entity='mgr.x' 2023-09-13T04:30:40.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:40 smithi002 ceph-mon[97651]: from='mgr.14100 172.21.15.2:0/1508766245' entity='mgr.x' 2023-09-13T04:30:42.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:42 smithi002 ceph-mon[97651]: mgrmap e3: x(active, since 1.00977s) 2023-09-13T04:30:43.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:43 smithi002 ceph-mon[97651]: mgrmap e4: x(active, since 2s) 2023-09-13T04:30:43.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:43 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2906144243' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-13T04:30:43.823 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2023-09-13T04:30:43.824 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2023-09-13T04:30:43.824 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsid": "22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da", 2023-09-13T04:30:43.824 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "health": { 2023-09-13T04:30:43.824 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-13T04:30:43.825 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-13T04:30:43.825 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-13T04:30:43.825 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-09-13T04:30:43.825 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-13T04:30:43.825 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-13T04:30:43.826 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 0 2023-09-13T04:30:43.826 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2023-09-13T04:30:43.826 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-13T04:30:43.826 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "a" 2023-09-13T04:30:43.826 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2023-09-13T04:30:43.827 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2023-09-13T04:30:43.827 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-13T04:30:43.827 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-13T04:30:43.827 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-13T04:30:43.827 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-13T04:30:43.828 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-09-13T04:30:43.828 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-13T04:30:43.828 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-13T04:30:43.828 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-13T04:30:43.828 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-13T04:30:43.836 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-13T04:30:43.836 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-13T04:30:43.837 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-13T04:30:43.837 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-13T04:30:43.837 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-09-13T04:30:43.837 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-13T04:30:43.837 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-13T04:30:43.838 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-13T04:30:43.838 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-13T04:30:43.838 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-13T04:30:43.838 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-13T04:30:43.839 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-13T04:30:43.839 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-13T04:30:43.839 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-13T04:30:43.839 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-09-13T04:30:43.839 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-13T04:30:43.839 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-13T04:30:43.840 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-13T04:30:43.840 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-13T04:30:43.840 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-09-13T04:30:43.840 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-13T04:30:43.840 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-13T04:30:43.841 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-13T04:30:43.841 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-13T04:30:43.841 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-13T04:30:43.841 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-13T04:30:43.841 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "restful" 2023-09-13T04:30:43.842 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2023-09-13T04:30:43.842 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-13T04:30:43.842 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-09-13T04:30:43.842 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-13T04:30:43.842 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-13T04:30:43.843 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modified": "2023-09-13T04:30:22.165522+0000", 2023-09-13T04:30:43.843 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-13T04:30:43.843 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-09-13T04:30:43.843 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-13T04:30:43.843 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2023-09-13T04:30:43.843 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.831+0000 7f0458886700 1 Processor -- start 2023-09-13T04:30:43.844 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.831+0000 7f0458886700 1 -- start start 2023-09-13T04:30:43.844 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.832+0000 7f0458886700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0454106520 0x7f0454106940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:43.844 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.832+0000 7f0458886700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0454106f10 con 0x7f0454106520 2023-09-13T04:30:43.844 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.832+0000 7f045259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0454106520 0x7f0454106940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:43.844 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.832+0000 7f045259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0454106520 0x7f0454106940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41652/0 (socket says 172.21.15.2:41652) 2023-09-13T04:30:43.845 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.832+0000 7f045259c700 1 -- 172.21.15.2:0/1644401844 learned_addr learned my addr 172.21.15.2:0/1644401844 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:30:43.845 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.833+0000 7f045259c700 1 -- 172.21.15.2:0/1644401844 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0454107720 con 0x7f0454106520 2023-09-13T04:30:43.845 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.833+0000 7f045259c700 1 --2- 172.21.15.2:0/1644401844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0454106520 0x7f0454106940 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f04440098a0 tx=0x7f0444009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e3ddd62c660069 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:43.845 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.833+0000 7f045159a700 1 -- 172.21.15.2:0/1644401844 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0444004160 con 0x7f0454106520 2023-09-13T04:30:43.845 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.833+0000 7f045159a700 1 -- 172.21.15.2:0/1644401844 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f04440042c0 con 0x7f0454106520 2023-09-13T04:30:43.846 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.833+0000 7f045159a700 1 -- 172.21.15.2:0/1644401844 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0444011a00 con 0x7f0454106520 2023-09-13T04:30:43.846 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.833+0000 7f0458886700 1 -- 172.21.15.2:0/1644401844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0454106520 msgr2=0x7f0454106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:43.846 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.833+0000 7f0458886700 1 --2- 172.21.15.2:0/1644401844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0454106520 0x7f0454106940 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f04440098a0 tx=0x7f0444009bb0 comp rx=0 tx=0).stop 2023-09-13T04:30:43.846 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.834+0000 7f0458886700 1 -- 172.21.15.2:0/1644401844 shutdown_connections 2023-09-13T04:30:43.846 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.834+0000 7f0458886700 1 --2- 172.21.15.2:0/1644401844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0454106520 0x7f0454106940 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:43.847 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.834+0000 7f0458886700 1 -- 172.21.15.2:0/1644401844 >> 172.21.15.2:0/1644401844 conn(0x7f0454101b00 msgr2=0x7f0454103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:30:43.847 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.834+0000 7f0458886700 1 -- 172.21.15.2:0/1644401844 shutdown_connections 2023-09-13T04:30:43.847 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.834+0000 7f0458886700 1 -- 172.21.15.2:0/1644401844 wait complete. 2023-09-13T04:30:43.847 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.835+0000 7f0458886700 1 Processor -- start 2023-09-13T04:30:43.847 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.835+0000 7f0458886700 1 -- start start 2023-09-13T04:30:43.848 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.835+0000 7f0458886700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0454106520 0x7f045419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:43.848 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.835+0000 7f0458886700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045419a760 con 0x7f0454106520 2023-09-13T04:30:43.848 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.835+0000 7f045259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0454106520 0x7f045419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:43.848 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.836+0000 7f045259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0454106520 0x7f045419a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41654/0 (socket says 172.21.15.2:41654) 2023-09-13T04:30:43.848 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.836+0000 7f045259c700 1 -- 172.21.15.2:0/2906144243 learned_addr learned my addr 172.21.15.2:0/2906144243 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:30:43.849 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.836+0000 7f045259c700 1 -- 172.21.15.2:0/2906144243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0444009550 con 0x7f0454106520 2023-09-13T04:30:43.849 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.836+0000 7f045259c700 1 --2- 172.21.15.2:0/2906144243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0454106520 0x7f045419a220 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0444006af0 tx=0x7f0444011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:43.849 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.836+0000 7f04437fe700 1 -- 172.21.15.2:0/2906144243 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0444003a40 con 0x7f0454106520 2023-09-13T04:30:43.849 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.836+0000 7f04437fe700 1 -- 172.21.15.2:0/2906144243 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0444003ba0 con 0x7f0454106520 2023-09-13T04:30:43.849 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.836+0000 7f04437fe700 1 -- 172.21.15.2:0/2906144243 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f044401a700 con 0x7f0454106520 2023-09-13T04:30:43.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.836+0000 7f0458886700 1 -- 172.21.15.2:0/2906144243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f045419a960 con 0x7f0454106520 2023-09-13T04:30:43.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.837+0000 7f0458886700 1 -- 172.21.15.2:0/2906144243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f045419ad80 con 0x7f0454106520 2023-09-13T04:30:43.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.838+0000 7f04437fe700 1 -- 172.21.15.2:0/2906144243 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 4) v1 ==== 45320+0+0 (secure 0 0 0) 0x7f0444024ad0 con 0x7f0454106520 2023-09-13T04:30:43.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.838+0000 7f0458886700 1 -- 172.21.15.2:0/2906144243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04540622e0 con 0x7f0454106520 2023-09-13T04:30:43.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.838+0000 7f04437fe700 1 --2- 172.21.15.2:0/2906144243 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7f043c038360 0x7f043c03a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:43.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.838+0000 7f04437fe700 1 -- 172.21.15.2:0/2906144243 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f044404c1b0 con 0x7f0454106520 2023-09-13T04:30:43.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.838+0000 7f0451d9b700 1 --2- 172.21.15.2:0/2906144243 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7f043c038360 0x7f043c03a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:43.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.839+0000 7f0451d9b700 1 --2- 172.21.15.2:0/2906144243 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7f043c038360 0x7f043c03a820 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0448006fd0 tx=0x7f0448006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:43.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:42.845+0000 7f04437fe700 1 -- 172.21.15.2:0/2906144243 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0444018bd0 con 0x7f0454106520 2023-09-13T04:30:43.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:43.133+0000 7f0458886700 1 -- 172.21.15.2:0/2906144243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f045410a9a0 con 0x7f0454106520 2023-09-13T04:30:43.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:43.134+0000 7f04437fe700 1 -- 172.21.15.2:0/2906144243 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f04440187c0 con 0x7f0454106520 2023-09-13T04:30:43.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:43.137+0000 7f0458886700 1 -- 172.21.15.2:0/2906144243 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7f043c038360 msgr2=0x7f043c03a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:43.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:43.137+0000 7f0458886700 1 --2- 172.21.15.2:0/2906144243 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7f043c038360 0x7f043c03a820 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0448006fd0 tx=0x7f0448006e40 comp rx=0 tx=0).stop 2023-09-13T04:30:43.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:43.137+0000 7f0458886700 1 -- 172.21.15.2:0/2906144243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0454106520 msgr2=0x7f045419a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:43.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:43.137+0000 7f0458886700 1 --2- 172.21.15.2:0/2906144243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0454106520 0x7f045419a220 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0444006af0 tx=0x7f0444011ad0 comp rx=0 tx=0).stop 2023-09-13T04:30:43.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:43.137+0000 7f0458886700 1 -- 172.21.15.2:0/2906144243 shutdown_connections 2023-09-13T04:30:43.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:43.137+0000 7f0458886700 1 --2- 172.21.15.2:0/2906144243 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7f043c038360 0x7f043c03a820 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:43.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:43.137+0000 7f0458886700 1 --2- 172.21.15.2:0/2906144243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0454106520 0x7f045419a220 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:43.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:43.137+0000 7f0458886700 1 -- 172.21.15.2:0/2906144243 >> 172.21.15.2:0/2906144243 conn(0x7f0454101b00 msgr2=0x7f04541031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:30:43.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:43.137+0000 7f0458886700 1 -- 172.21.15.2:0/2906144243 shutdown_connections 2023-09-13T04:30:43.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:43.137+0000 7f0458886700 1 -- 172.21.15.2:0/2906144243 wait complete. 2023-09-13T04:30:43.854 INFO:teuthology.orchestra.run.smithi002.stdout:mgr is available 2023-09-13T04:30:45.220 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2023-09-13T04:30:45.220 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [global] 2023-09-13T04:30:45.221 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout fsid = 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:30:45.221 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-09-13T04:30:45.221 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-09-13T04:30:45.221 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-09-13T04:30:45.221 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-09-13T04:30:45.222 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2023-09-13T04:30:45.222 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [mgr] 2023-09-13T04:30:45.222 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-09-13T04:30:45.223 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2023-09-13T04:30:45.223 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [osd] 2023-09-13T04:30:45.223 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-09-13T04:30:45.224 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-09-13T04:30:45.224 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-09-13T04:30:45.224 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-09-13T04:30:45.225 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-09-13T04:30:45.225 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.474+0000 7fc7389f9700 1 Processor -- start 2023-09-13T04:30:45.225 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.474+0000 7fc7389f9700 1 -- start start 2023-09-13T04:30:45.225 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.474+0000 7fc7389f9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc734106f90 0x7fc734109380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:45.225 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.474+0000 7fc7389f9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc734079ec0 con 0x7fc734106f90 2023-09-13T04:30:45.226 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.475+0000 7fc73259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc734106f90 0x7fc734109380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:45.226 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.475+0000 7fc73259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc734106f90 0x7fc734109380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43972/0 (socket says 172.21.15.2:43972) 2023-09-13T04:30:45.226 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.475+0000 7fc73259c700 1 -- 172.21.15.2:0/692140300 learned_addr learned my addr 172.21.15.2:0/692140300 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:30:45.226 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.475+0000 7fc73259c700 1 -- 172.21.15.2:0/692140300 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7341098c0 con 0x7fc734106f90 2023-09-13T04:30:45.226 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.475+0000 7fc73259c700 1 --2- 172.21.15.2:0/692140300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc734106f90 0x7fc734109380 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc7240098a0 tx=0x7fc724009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5ca60999c3264 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:45.227 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.475+0000 7fc73159a700 1 -- 172.21.15.2:0/692140300 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc724004160 con 0x7fc734106f90 2023-09-13T04:30:45.227 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.476+0000 7fc73159a700 1 -- 172.21.15.2:0/692140300 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc7240042c0 con 0x7fc734106f90 2023-09-13T04:30:45.227 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.476+0000 7fc73159a700 1 -- 172.21.15.2:0/692140300 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc724011a00 con 0x7fc734106f90 2023-09-13T04:30:45.227 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.476+0000 7fc7389f9700 1 -- 172.21.15.2:0/692140300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc734106f90 msgr2=0x7fc734109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:45.227 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.476+0000 7fc7389f9700 1 --2- 172.21.15.2:0/692140300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc734106f90 0x7fc734109380 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc7240098a0 tx=0x7fc724009bb0 comp rx=0 tx=0).stop 2023-09-13T04:30:45.228 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.476+0000 7fc7389f9700 1 -- 172.21.15.2:0/692140300 shutdown_connections 2023-09-13T04:30:45.228 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.476+0000 7fc7389f9700 1 --2- 172.21.15.2:0/692140300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc734106f90 0x7fc734109380 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:45.228 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.476+0000 7fc7389f9700 1 -- 172.21.15.2:0/692140300 >> 172.21.15.2:0/692140300 conn(0x7fc7341009e0 msgr2=0x7fc734102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:30:45.229 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.476+0000 7fc7389f9700 1 -- 172.21.15.2:0/692140300 shutdown_connections 2023-09-13T04:30:45.229 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.476+0000 7fc7389f9700 1 -- 172.21.15.2:0/692140300 wait complete. 2023-09-13T04:30:45.229 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.478+0000 7fc7389f9700 1 Processor -- start 2023-09-13T04:30:45.229 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.478+0000 7fc7389f9700 1 -- start start 2023-09-13T04:30:45.229 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.478+0000 7fc7389f9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc734106f90 0x7fc73419a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:45.230 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.478+0000 7fc7389f9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73419a6e0 con 0x7fc734106f90 2023-09-13T04:30:45.230 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.479+0000 7fc73259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc734106f90 0x7fc73419a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:45.230 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.479+0000 7fc73259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc734106f90 0x7fc73419a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43976/0 (socket says 172.21.15.2:43976) 2023-09-13T04:30:45.230 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.479+0000 7fc73259c700 1 -- 172.21.15.2:0/2812415740 learned_addr learned my addr 172.21.15.2:0/2812415740 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:30:45.231 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.479+0000 7fc73259c700 1 -- 172.21.15.2:0/2812415740 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc724009550 con 0x7fc734106f90 2023-09-13T04:30:45.231 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.479+0000 7fc73259c700 1 --2- 172.21.15.2:0/2812415740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc734106f90 0x7fc73419a1a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc724006af0 tx=0x7fc724011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:45.231 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.480+0000 7fc7237fe700 1 -- 172.21.15.2:0/2812415740 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc724003a40 con 0x7fc734106f90 2023-09-13T04:30:45.231 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.480+0000 7fc7237fe700 1 -- 172.21.15.2:0/2812415740 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc724003ba0 con 0x7fc734106f90 2023-09-13T04:30:45.231 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.480+0000 7fc7389f9700 1 -- 172.21.15.2:0/2812415740 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc73419a8e0 con 0x7fc734106f90 2023-09-13T04:30:45.232 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.480+0000 7fc7237fe700 1 -- 172.21.15.2:0/2812415740 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc72401a6e0 con 0x7fc734106f90 2023-09-13T04:30:45.232 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.480+0000 7fc7389f9700 1 -- 172.21.15.2:0/2812415740 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73419ad00 con 0x7fc734106f90 2023-09-13T04:30:45.232 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.481+0000 7fc7237fe700 1 -- 172.21.15.2:0/2812415740 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 4) v1 ==== 45320+0+0 (secure 0 0 0) 0x7fc724024a60 con 0x7fc734106f90 2023-09-13T04:30:45.232 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.481+0000 7fc7389f9700 1 -- 172.21.15.2:0/2812415740 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7340622e0 con 0x7fc734106f90 2023-09-13T04:30:45.232 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.481+0000 7fc7237fe700 1 --2- 172.21.15.2:0/2812415740 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7fc71c038360 0x7fc71c03a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:45.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.481+0000 7fc7237fe700 1 -- 172.21.15.2:0/2812415740 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc72404c1f0 con 0x7fc734106f90 2023-09-13T04:30:45.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.481+0000 7fc731d9b700 1 --2- 172.21.15.2:0/2812415740 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7fc71c038360 0x7fc71c03a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:45.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.482+0000 7fc731d9b700 1 --2- 172.21.15.2:0/2812415740 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7fc71c038360 0x7fc71c03a820 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc728006fd0 tx=0x7fc728006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:45.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.487+0000 7fc7237fe700 1 -- 172.21.15.2:0/2812415740 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc724024430 con 0x7fc734106f90 2023-09-13T04:30:45.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.696+0000 7fc7389f9700 1 -- 172.21.15.2:0/2812415740 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fc73410a6c0 con 0x7fc734106f90 2023-09-13T04:30:45.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.697+0000 7fc7237fe700 1 -- 172.21.15.2:0/2812415740 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fc724018350 con 0x7fc734106f90 2023-09-13T04:30:45.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.699+0000 7fc7389f9700 1 -- 172.21.15.2:0/2812415740 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7fc71c038360 msgr2=0x7fc71c03a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:45.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.699+0000 7fc7389f9700 1 --2- 172.21.15.2:0/2812415740 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7fc71c038360 0x7fc71c03a820 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc728006fd0 tx=0x7fc728006e40 comp rx=0 tx=0).stop 2023-09-13T04:30:45.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.699+0000 7fc7389f9700 1 -- 172.21.15.2:0/2812415740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc734106f90 msgr2=0x7fc73419a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:45.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.699+0000 7fc7389f9700 1 --2- 172.21.15.2:0/2812415740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc734106f90 0x7fc73419a1a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc724006af0 tx=0x7fc724011ad0 comp rx=0 tx=0).stop 2023-09-13T04:30:45.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.699+0000 7fc7389f9700 1 -- 172.21.15.2:0/2812415740 shutdown_connections 2023-09-13T04:30:45.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.699+0000 7fc7389f9700 1 --2- 172.21.15.2:0/2812415740 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7fc71c038360 0x7fc71c03a820 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:45.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.699+0000 7fc7389f9700 1 --2- 172.21.15.2:0/2812415740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc734106f90 0x7fc73419a1a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:45.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.699+0000 7fc7389f9700 1 -- 172.21.15.2:0/2812415740 >> 172.21.15.2:0/2812415740 conn(0x7fc7341009e0 msgr2=0x7fc7341017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:30:45.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.700+0000 7fc7389f9700 1 -- 172.21.15.2:0/2812415740 shutdown_connections 2023-09-13T04:30:45.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:44.700+0000 7fc7389f9700 1 -- 172.21.15.2:0/2812415740 wait complete. 2023-09-13T04:30:45.236 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling cephadm module... 2023-09-13T04:30:45.489 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:45 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2812415740' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-09-13T04:30:46.377 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:46 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2236681381' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-09-13T04:30:46.377 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:46 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: ignoring --setuser ceph since I am not root 2023-09-13T04:30:46.377 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:46 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: ignoring --setgroup ceph since I am not root 2023-09-13T04:30:46.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.747+0000 7fb122781700 1 Processor -- start 2023-09-13T04:30:46.601 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.748+0000 7fb122781700 1 -- start start 2023-09-13T04:30:46.601 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.748+0000 7fb122781700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb11c07be90 0x7fb11c07a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:46.601 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.748+0000 7fb122781700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c07a8c0 con 0x7fb11c07be90 2023-09-13T04:30:46.601 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.748+0000 7fb11bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb11c07be90 0x7fb11c07a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:46.602 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.748+0000 7fb11bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb11c07be90 0x7fb11c07a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43992/0 (socket says 172.21.15.2:43992) 2023-09-13T04:30:46.602 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.748+0000 7fb11bfff700 1 -- 172.21.15.2:0/2887549196 learned_addr learned my addr 172.21.15.2:0/2887549196 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:30:46.602 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.749+0000 7fb11bfff700 1 -- 172.21.15.2:0/2887549196 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11c07aa00 con 0x7fb11c07be90 2023-09-13T04:30:46.602 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.749+0000 7fb11bfff700 1 --2- 172.21.15.2:0/2887549196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb11c07be90 0x7fb11c07a2f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0098a0 tx=0x7fb10c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c40c5f4d00803a4 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:46.603 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.749+0000 7fb11affd700 1 -- 172.21.15.2:0/2887549196 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb10c004160 con 0x7fb11c07be90 2023-09-13T04:30:46.603 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.749+0000 7fb11affd700 1 -- 172.21.15.2:0/2887549196 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb10c0042c0 con 0x7fb11c07be90 2023-09-13T04:30:46.603 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.749+0000 7fb11affd700 1 -- 172.21.15.2:0/2887549196 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb10c011a00 con 0x7fb11c07be90 2023-09-13T04:30:46.603 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.750+0000 7fb122781700 1 -- 172.21.15.2:0/2887549196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb11c07be90 msgr2=0x7fb11c07a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:46.603 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.750+0000 7fb122781700 1 --2- 172.21.15.2:0/2887549196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb11c07be90 0x7fb11c07a2f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0098a0 tx=0x7fb10c009bb0 comp rx=0 tx=0).stop 2023-09-13T04:30:46.604 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.750+0000 7fb122781700 1 -- 172.21.15.2:0/2887549196 shutdown_connections 2023-09-13T04:30:46.604 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.750+0000 7fb122781700 1 --2- 172.21.15.2:0/2887549196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb11c07be90 0x7fb11c07a2f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:46.604 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.750+0000 7fb122781700 1 -- 172.21.15.2:0/2887549196 >> 172.21.15.2:0/2887549196 conn(0x7fb11c101b00 msgr2=0x7fb11c103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:30:46.604 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.750+0000 7fb122781700 1 -- 172.21.15.2:0/2887549196 shutdown_connections 2023-09-13T04:30:46.604 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.750+0000 7fb122781700 1 -- 172.21.15.2:0/2887549196 wait complete. 2023-09-13T04:30:46.604 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.751+0000 7fb122781700 1 Processor -- start 2023-09-13T04:30:46.605 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.751+0000 7fb122781700 1 -- start start 2023-09-13T04:30:46.605 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.752+0000 7fb122781700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb11c07be90 0x7fb11c19c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:46.605 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.752+0000 7fb122781700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c19c950 con 0x7fb11c07be90 2023-09-13T04:30:46.605 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.752+0000 7fb11bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb11c07be90 0x7fb11c19c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:46.605 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.752+0000 7fb11bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb11c07be90 0x7fb11c19c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44002/0 (socket says 172.21.15.2:44002) 2023-09-13T04:30:46.606 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.752+0000 7fb11bfff700 1 -- 172.21.15.2:0/2236681381 learned_addr learned my addr 172.21.15.2:0/2236681381 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:30:46.606 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.752+0000 7fb11bfff700 1 -- 172.21.15.2:0/2236681381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c009550 con 0x7fb11c07be90 2023-09-13T04:30:46.606 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.752+0000 7fb11bfff700 1 --2- 172.21.15.2:0/2236681381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb11c07be90 0x7fb11c19c410 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb10c006af0 tx=0x7fb10c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:46.606 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.753+0000 7fb1197fa700 1 -- 172.21.15.2:0/2236681381 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb10c003a40 con 0x7fb11c07be90 2023-09-13T04:30:46.606 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.753+0000 7fb1197fa700 1 -- 172.21.15.2:0/2236681381 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb10c003ba0 con 0x7fb11c07be90 2023-09-13T04:30:46.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.753+0000 7fb122781700 1 -- 172.21.15.2:0/2236681381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb11c19cb50 con 0x7fb11c07be90 2023-09-13T04:30:46.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.753+0000 7fb1197fa700 1 -- 172.21.15.2:0/2236681381 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb10c01a6e0 con 0x7fb11c07be90 2023-09-13T04:30:46.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.753+0000 7fb122781700 1 -- 172.21.15.2:0/2236681381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb11c19cf70 con 0x7fb11c07be90 2023-09-13T04:30:46.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.754+0000 7fb1197fa700 1 -- 172.21.15.2:0/2236681381 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 4) v1 ==== 45320+0+0 (secure 0 0 0) 0x7fb10c024430 con 0x7fb11c07be90 2023-09-13T04:30:46.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.754+0000 7fb122781700 1 -- 172.21.15.2:0/2236681381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb11c0622e0 con 0x7fb11c07be90 2023-09-13T04:30:46.608 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.754+0000 7fb1197fa700 1 --2- 172.21.15.2:0/2236681381 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7fb104038310 0x7fb10403a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:46.608 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.754+0000 7fb1197fa700 1 -- 172.21.15.2:0/2236681381 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb10c04c210 con 0x7fb11c07be90 2023-09-13T04:30:46.608 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.754+0000 7fb11b7fe700 1 --2- 172.21.15.2:0/2236681381 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7fb104038310 0x7fb10403a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:46.608 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.755+0000 7fb11b7fe700 1 --2- 172.21.15.2:0/2236681381 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7fb104038310 0x7fb10403a7d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb110006fd0 tx=0x7fb110006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:46.609 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:45.759+0000 7fb1197fa700 1 -- 172.21.15.2:0/2236681381 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb10c0246e0 con 0x7fb11c07be90 2023-09-13T04:30:46.609 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:46.028+0000 7fb122781700 1 -- 172.21.15.2:0/2236681381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fb11c10c910 con 0x7fb11c07be90 2023-09-13T04:30:46.609 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:46.111+0000 7fb1197fa700 1 -- 172.21.15.2:0/2236681381 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 5) v1 ==== 45331+0+0 (secure 0 0 0) 0x7fb10c02d8c0 con 0x7fb11c07be90 2023-09-13T04:30:46.609 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:46.112+0000 7fb1197fa700 1 -- 172.21.15.2:0/2236681381 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fb10c0176b0 con 0x7fb11c07be90 2023-09-13T04:30:46.610 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:46.115+0000 7fb122781700 1 -- 172.21.15.2:0/2236681381 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7fb104038310 msgr2=0x7fb10403a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:46.610 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:46.115+0000 7fb122781700 1 --2- 172.21.15.2:0/2236681381 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7fb104038310 0x7fb10403a7d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb110006fd0 tx=0x7fb110006e40 comp rx=0 tx=0).stop 2023-09-13T04:30:46.610 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:46.115+0000 7fb122781700 1 -- 172.21.15.2:0/2236681381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb11c07be90 msgr2=0x7fb11c19c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:46.610 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:46.115+0000 7fb122781700 1 --2- 172.21.15.2:0/2236681381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb11c07be90 0x7fb11c19c410 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb10c006af0 tx=0x7fb10c011ad0 comp rx=0 tx=0).stop 2023-09-13T04:30:46.610 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:46.115+0000 7fb122781700 1 -- 172.21.15.2:0/2236681381 shutdown_connections 2023-09-13T04:30:46.611 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:46.115+0000 7fb122781700 1 --2- 172.21.15.2:0/2236681381 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7fb104038310 0x7fb10403a7d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:46.611 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:46.115+0000 7fb122781700 1 --2- 172.21.15.2:0/2236681381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb11c07be90 0x7fb11c19c410 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:46.611 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:46.115+0000 7fb122781700 1 -- 172.21.15.2:0/2236681381 >> 172.21.15.2:0/2236681381 conn(0x7fb11c101b00 msgr2=0x7fb11c1031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:30:46.611 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:46.116+0000 7fb122781700 1 -- 172.21.15.2:0/2236681381 shutdown_connections 2023-09-13T04:30:46.612 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:46.116+0000 7fb122781700 1 -- 172.21.15.2:0/2236681381 wait complete. 2023-09-13T04:30:46.738 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:46 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:46.375+0000 7fd1ee180200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-13T04:30:46.738 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:46 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:46.484+0000 7fd1ee180200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-13T04:30:46.738 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:46 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:46.606+0000 7fd1ee180200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-13T04:30:47.005 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:46 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:46.736+0000 7fd1ee180200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-13T04:30:47.006 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:46 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:46.974+0000 7fd1ee180200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-13T04:30:47.413 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:47 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2236681381' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-09-13T04:30:47.414 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:47 smithi002 ceph-mon[97651]: mgrmap e5: x(active, since 5s) 2023-09-13T04:30:47.752 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:47 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:47.454+0000 7fd1ee180200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-13T04:30:47.752 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:47 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:47.629+0000 7fd1ee180200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-13T04:30:47.878 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2023-09-13T04:30:47.878 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-09-13T04:30:47.878 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-13T04:30:47.878 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-09-13T04:30:47.879 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-09-13T04:30:47.879 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2023-09-13T04:30:47.879 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.108+0000 7f0ab8972700 1 Processor -- start 2023-09-13T04:30:47.879 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.109+0000 7f0ab8972700 1 -- start start 2023-09-13T04:30:47.879 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.109+0000 7f0ab8972700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ab407be90 0x7f0ab407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:47.880 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.109+0000 7f0ab8972700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab407a8c0 con 0x7f0ab407be90 2023-09-13T04:30:47.880 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.109+0000 7f0ab259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ab407be90 0x7f0ab407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:47.880 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.110+0000 7f0ab259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ab407be90 0x7f0ab407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44024/0 (socket says 172.21.15.2:44024) 2023-09-13T04:30:47.880 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.110+0000 7f0ab259c700 1 -- 172.21.15.2:0/774455946 learned_addr learned my addr 172.21.15.2:0/774455946 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:30:47.880 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.110+0000 7f0ab259c700 1 -- 172.21.15.2:0/774455946 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab407aa00 con 0x7f0ab407be90 2023-09-13T04:30:47.885 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.110+0000 7f0ab259c700 1 --2- 172.21.15.2:0/774455946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ab407be90 0x7f0ab407a2f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0aa40098a0 tx=0x7f0aa4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a787210695bb16 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:47.885 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.113+0000 7f0ab159a700 1 -- 172.21.15.2:0/774455946 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0aa4004160 con 0x7f0ab407be90 2023-09-13T04:30:47.886 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.113+0000 7f0ab159a700 1 -- 172.21.15.2:0/774455946 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0aa40042c0 con 0x7f0ab407be90 2023-09-13T04:30:47.886 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.113+0000 7f0ab159a700 1 -- 172.21.15.2:0/774455946 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0aa4011a00 con 0x7f0ab407be90 2023-09-13T04:30:47.886 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.113+0000 7f0ab8972700 1 -- 172.21.15.2:0/774455946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ab407be90 msgr2=0x7f0ab407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:47.886 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.113+0000 7f0ab8972700 1 --2- 172.21.15.2:0/774455946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ab407be90 0x7f0ab407a2f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0aa40098a0 tx=0x7f0aa4009bb0 comp rx=0 tx=0).stop 2023-09-13T04:30:47.886 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.114+0000 7f0ab8972700 1 -- 172.21.15.2:0/774455946 shutdown_connections 2023-09-13T04:30:47.887 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.114+0000 7f0ab8972700 1 --2- 172.21.15.2:0/774455946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ab407be90 0x7f0ab407a2f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:47.887 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.114+0000 7f0ab8972700 1 -- 172.21.15.2:0/774455946 >> 172.21.15.2:0/774455946 conn(0x7f0ab4101b00 msgr2=0x7f0ab4103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:30:47.887 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.114+0000 7f0ab8972700 1 -- 172.21.15.2:0/774455946 shutdown_connections 2023-09-13T04:30:47.887 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.114+0000 7f0ab8972700 1 -- 172.21.15.2:0/774455946 wait complete. 2023-09-13T04:30:47.887 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.115+0000 7f0ab8972700 1 Processor -- start 2023-09-13T04:30:47.888 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.115+0000 7f0ab8972700 1 -- start start 2023-09-13T04:30:47.888 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.115+0000 7f0ab8972700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ab407be90 0x7f0ab419c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:47.888 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.115+0000 7f0ab8972700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab419c950 con 0x7f0ab407be90 2023-09-13T04:30:47.888 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.115+0000 7f0ab259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ab407be90 0x7f0ab419c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:47.888 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.116+0000 7f0ab259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ab407be90 0x7f0ab419c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44026/0 (socket says 172.21.15.2:44026) 2023-09-13T04:30:47.889 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.116+0000 7f0ab259c700 1 -- 172.21.15.2:0/577908096 learned_addr learned my addr 172.21.15.2:0/577908096 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:30:47.889 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.116+0000 7f0ab259c700 1 -- 172.21.15.2:0/577908096 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa4009550 con 0x7f0ab407be90 2023-09-13T04:30:47.889 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.116+0000 7f0ab259c700 1 --2- 172.21.15.2:0/577908096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ab407be90 0x7f0ab419c410 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4006af0 tx=0x7f0aa4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:47.889 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.116+0000 7f0aa37fe700 1 -- 172.21.15.2:0/577908096 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0aa4003a40 con 0x7f0ab407be90 2023-09-13T04:30:47.889 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.116+0000 7f0aa37fe700 1 -- 172.21.15.2:0/577908096 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0aa4003ba0 con 0x7f0ab407be90 2023-09-13T04:30:47.890 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.116+0000 7f0aa37fe700 1 -- 172.21.15.2:0/577908096 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0aa401a6e0 con 0x7f0ab407be90 2023-09-13T04:30:47.890 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.116+0000 7f0ab8972700 1 -- 172.21.15.2:0/577908096 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab419cb50 con 0x7f0ab407be90 2023-09-13T04:30:47.890 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.117+0000 7f0ab8972700 1 -- 172.21.15.2:0/577908096 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab419cf70 con 0x7f0ab407be90 2023-09-13T04:30:47.890 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.118+0000 7f0ab8972700 1 -- 172.21.15.2:0/577908096 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab40622e0 con 0x7f0ab407be90 2023-09-13T04:30:47.890 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.118+0000 7f0aa37fe700 1 -- 172.21.15.2:0/577908096 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 45331+0+0 (secure 0 0 0) 0x7f0aa4024430 con 0x7f0ab407be90 2023-09-13T04:30:47.891 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.118+0000 7f0aa37fe700 1 --2- 172.21.15.2:0/577908096 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7f0a9c0383b0 0x7f0a9c03a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:47.891 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.118+0000 7f0aa37fe700 1 -- 172.21.15.2:0/577908096 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0aa404c220 con 0x7f0ab407be90 2023-09-13T04:30:47.891 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.119+0000 7f0ab1d9b700 1 -- 172.21.15.2:0/577908096 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7f0a9c0383b0 msgr2=0x7f0a9c03a870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/2772434518 2023-09-13T04:30:47.891 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.119+0000 7f0ab1d9b700 1 --2- 172.21.15.2:0/577908096 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7f0a9c0383b0 0x7f0a9c03a870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-13T04:30:47.891 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.123+0000 7f0aa37fe700 1 -- 172.21.15.2:0/577908096 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0aa40246e0 con 0x7f0ab407be90 2023-09-13T04:30:47.892 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.320+0000 7f0ab1d9b700 1 -- 172.21.15.2:0/577908096 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7f0a9c0383b0 msgr2=0x7f0a9c03a870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/2772434518 2023-09-13T04:30:47.892 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.320+0000 7f0ab1d9b700 1 --2- 172.21.15.2:0/577908096 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7f0a9c0383b0 0x7f0a9c03a870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-13T04:30:47.892 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.388+0000 7f0ab8972700 1 -- 172.21.15.2:0/577908096 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f0ab410c910 con 0x7f0ab407be90 2023-09-13T04:30:47.892 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.388+0000 7f0aa37fe700 1 -- 172.21.15.2:0/577908096 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f0aa4018350 con 0x7f0ab407be90 2023-09-13T04:30:47.892 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.391+0000 7f0ab8972700 1 -- 172.21.15.2:0/577908096 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7f0a9c0383b0 msgr2=0x7f0a9c03a870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-13T04:30:47.893 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.391+0000 7f0ab8972700 1 --2- 172.21.15.2:0/577908096 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7f0a9c0383b0 0x7f0a9c03a870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:47.893 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.391+0000 7f0ab8972700 1 -- 172.21.15.2:0/577908096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ab407be90 msgr2=0x7f0ab419c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:47.893 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.391+0000 7f0ab8972700 1 --2- 172.21.15.2:0/577908096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ab407be90 0x7f0ab419c410 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4006af0 tx=0x7f0aa4011ad0 comp rx=0 tx=0).stop 2023-09-13T04:30:47.893 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.391+0000 7f0ab8972700 1 -- 172.21.15.2:0/577908096 shutdown_connections 2023-09-13T04:30:47.893 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.391+0000 7f0ab8972700 1 --2- 172.21.15.2:0/577908096 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7f0a9c0383b0 0x7f0a9c03a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:47.894 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.391+0000 7f0ab8972700 1 --2- 172.21.15.2:0/577908096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ab407be90 0x7f0ab419c410 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:47.894 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.391+0000 7f0ab8972700 1 -- 172.21.15.2:0/577908096 >> 172.21.15.2:0/577908096 conn(0x7f0ab4101b00 msgr2=0x7f0ab41031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:30:47.894 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.391+0000 7f0ab8972700 1 -- 172.21.15.2:0/577908096 shutdown_connections 2023-09-13T04:30:47.894 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:47.391+0000 7f0ab8972700 1 -- 172.21.15.2:0/577908096 wait complete. 2023-09-13T04:30:47.895 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for the mgr to restart... 2023-09-13T04:30:47.895 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr epoch 5... 2023-09-13T04:30:48.005 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:47 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:47.750+0000 7fd1ee180200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-13T04:30:48.006 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:47 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:47.885+0000 7fd1ee180200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-13T04:30:48.481 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:48 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/577908096' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-09-13T04:30:48.755 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:48 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:48.480+0000 7fd1ee180200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-13T04:30:49.755 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:49 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:49.358+0000 7fd1ee180200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-13T04:30:49.755 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:49 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:49.469+0000 7fd1ee180200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-13T04:30:50.755 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:50 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:50.455+0000 7fd1ee180200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-13T04:30:50.755 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:50 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:50.577+0000 7fd1ee180200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-13T04:30:51.181 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:50 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:50.806+0000 7fd1ee180200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-13T04:30:51.181 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:51 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:51.180+0000 7fd1ee180200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-13T04:30:51.755 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:51 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:51.462+0000 7fd1ee180200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-13T04:30:51.756 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:51 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:51.662+0000 7fd1ee180200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-13T04:30:52.505 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:52 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:52.012+0000 7fd1ee180200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-13T04:30:53.255 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:52 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:52.981+0000 7fd1ee180200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-13T04:30:53.256 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:53 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:53.163+0000 7fd1ee180200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-13T04:30:53.755 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:53 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:53.417+0000 7fd1ee180200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-13T04:30:53.755 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:53 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:53.569+0000 7fd1ee180200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-13T04:30:54.255 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:53 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:53.854+0000 7fd1ee180200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-13T04:30:55.255 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:30:54 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:30:54.960+0000 7fd1ee180200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-13T04:30:55.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:55 smithi002 ceph-mon[97651]: Active manager daemon x restarted 2023-09-13T04:30:55.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:55 smithi002 ceph-mon[97651]: Activating manager daemon x 2023-09-13T04:30:55.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:55 smithi002 ceph-mon[97651]: osdmap e2: 0 total, 0 up, 0 in 2023-09-13T04:30:55.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:55 smithi002 ceph-mon[97651]: mgrmap e6: x(active, starting, since 0.00386698s) 2023-09-13T04:30:55.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:55 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T04:30:55.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:55 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-13T04:30:55.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:55 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-13T04:30:55.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:55 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-13T04:30:55.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:55 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-13T04:30:55.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:55 smithi002 ceph-mon[97651]: Manager daemon x is now available 2023-09-13T04:30:55.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:55 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' 2023-09-13T04:30:55.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:55 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' 2023-09-13T04:30:55.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:55 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:30:55.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:55 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:30:55.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:55 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:30:55.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:55 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-13T04:30:55.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:55 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-13T04:30:56.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:56 smithi002 ceph-mon[97651]: Found migration_current of "None". Setting to last migration. 2023-09-13T04:30:56.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:56 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' 2023-09-13T04:30:56.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:56 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' 2023-09-13T04:30:56.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:56 smithi002 ceph-mon[97651]: mgrmap e7: x(active, since 1.00655s) 2023-09-13T04:30:56.495 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2023-09-13T04:30:56.496 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-09-13T04:30:56.496 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "initialized": true 2023-09-13T04:30:56.496 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2023-09-13T04:30:56.496 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.448+0000 7ff07a928700 1 Processor -- start 2023-09-13T04:30:56.496 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.448+0000 7ff07a928700 1 -- start start 2023-09-13T04:30:56.497 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.448+0000 7ff07a928700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff07407be90 0x7ff07407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:56.497 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.449+0000 7ff07a928700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07407a8c0 con 0x7ff07407be90 2023-09-13T04:30:56.497 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.449+0000 7ff073fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff07407be90 0x7ff07407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:56.497 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.449+0000 7ff073fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff07407be90 0x7ff07407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44036/0 (socket says 172.21.15.2:44036) 2023-09-13T04:30:56.498 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.449+0000 7ff073fff700 1 -- 172.21.15.2:0/3765967406 learned_addr learned my addr 172.21.15.2:0/3765967406 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:30:56.498 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.449+0000 7ff073fff700 1 -- 172.21.15.2:0/3765967406 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07407aa00 con 0x7ff07407be90 2023-09-13T04:30:56.498 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.449+0000 7ff073fff700 1 --2- 172.21.15.2:0/3765967406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff07407be90 0x7ff07407a2f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff0640098a0 tx=0x7ff064009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a313c39840d941c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:56.498 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.450+0000 7ff072ffd700 1 -- 172.21.15.2:0/3765967406 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff064004160 con 0x7ff07407be90 2023-09-13T04:30:56.498 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.450+0000 7ff072ffd700 1 -- 172.21.15.2:0/3765967406 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff0640042c0 con 0x7ff07407be90 2023-09-13T04:30:56.499 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.450+0000 7ff072ffd700 1 -- 172.21.15.2:0/3765967406 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff064011a00 con 0x7ff07407be90 2023-09-13T04:30:56.499 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.450+0000 7ff07a928700 1 -- 172.21.15.2:0/3765967406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff07407be90 msgr2=0x7ff07407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:56.499 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.450+0000 7ff07a928700 1 --2- 172.21.15.2:0/3765967406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff07407be90 0x7ff07407a2f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff0640098a0 tx=0x7ff064009bb0 comp rx=0 tx=0).stop 2023-09-13T04:30:56.499 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.450+0000 7ff07a928700 1 -- 172.21.15.2:0/3765967406 shutdown_connections 2023-09-13T04:30:56.499 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.450+0000 7ff07a928700 1 --2- 172.21.15.2:0/3765967406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff07407be90 0x7ff07407a2f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:56.500 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.450+0000 7ff07a928700 1 -- 172.21.15.2:0/3765967406 >> 172.21.15.2:0/3765967406 conn(0x7ff074101b00 msgr2=0x7ff074103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:30:56.500 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.451+0000 7ff07a928700 1 -- 172.21.15.2:0/3765967406 shutdown_connections 2023-09-13T04:30:56.500 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.451+0000 7ff07a928700 1 -- 172.21.15.2:0/3765967406 wait complete. 2023-09-13T04:30:56.500 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.452+0000 7ff07a928700 1 Processor -- start 2023-09-13T04:30:56.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.452+0000 7ff07a928700 1 -- start start 2023-09-13T04:30:56.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.453+0000 7ff07a928700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff07407be90 0x7ff07419c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:56.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.453+0000 7ff07a928700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07419c950 con 0x7ff07407be90 2023-09-13T04:30:56.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.453+0000 7ff073fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff07407be90 0x7ff07419c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:56.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.453+0000 7ff073fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff07407be90 0x7ff07419c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44046/0 (socket says 172.21.15.2:44046) 2023-09-13T04:30:56.502 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.453+0000 7ff073fff700 1 -- 172.21.15.2:0/2403806534 learned_addr learned my addr 172.21.15.2:0/2403806534 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:30:56.502 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.453+0000 7ff073fff700 1 -- 172.21.15.2:0/2403806534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff064009550 con 0x7ff07407be90 2023-09-13T04:30:56.502 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.454+0000 7ff073fff700 1 --2- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff07407be90 0x7ff07419c410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff064006af0 tx=0x7ff064011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:56.502 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.454+0000 7ff0717fa700 1 -- 172.21.15.2:0/2403806534 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff064003a40 con 0x7ff07407be90 2023-09-13T04:30:56.502 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.454+0000 7ff0717fa700 1 -- 172.21.15.2:0/2403806534 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff064003ba0 con 0x7ff07407be90 2023-09-13T04:30:56.503 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.454+0000 7ff0717fa700 1 -- 172.21.15.2:0/2403806534 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff06401a6e0 con 0x7ff07407be90 2023-09-13T04:30:56.503 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.454+0000 7ff07a928700 1 -- 172.21.15.2:0/2403806534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff07419cb50 con 0x7ff07407be90 2023-09-13T04:30:56.503 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.454+0000 7ff07a928700 1 -- 172.21.15.2:0/2403806534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff07419cf70 con 0x7ff07407be90 2023-09-13T04:30:56.503 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.456+0000 7ff0717fa700 1 -- 172.21.15.2:0/2403806534 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 45331+0+0 (secure 0 0 0) 0x7ff06401a860 con 0x7ff07407be90 2023-09-13T04:30:56.503 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.456+0000 7ff0717fa700 1 --2- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7ff05c0383b0 0x7ff05c03a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:56.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.456+0000 7ff0717fa700 1 -- 172.21.15.2:0/2403806534 --> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff05c03af80 con 0x7ff05c0383b0 2023-09-13T04:30:56.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.456+0000 7ff0717fa700 1 -- 172.21.15.2:0/2403806534 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff06404c1a0 con 0x7ff07407be90 2023-09-13T04:30:56.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.456+0000 7ff0737fe700 1 -- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7ff05c0383b0 msgr2=0x7ff05c03a870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/2772434518 2023-09-13T04:30:56.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.456+0000 7ff0737fe700 1 --2- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7ff05c0383b0 0x7ff05c03a870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-13T04:30:56.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.657+0000 7ff0737fe700 1 -- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7ff05c0383b0 msgr2=0x7ff05c03a870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/2772434518 2023-09-13T04:30:56.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:48.657+0000 7ff0737fe700 1 --2- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7ff05c0383b0 0x7ff05c03a870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-13T04:30:56.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:49.058+0000 7ff0737fe700 1 -- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7ff05c0383b0 msgr2=0x7ff05c03a870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/2772434518 2023-09-13T04:30:56.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:49.058+0000 7ff0737fe700 1 --2- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7ff05c0383b0 0x7ff05c03a870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-13T04:30:56.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:49.859+0000 7ff0737fe700 1 -- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7ff05c0383b0 msgr2=0x7ff05c03a870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/2772434518 2023-09-13T04:30:56.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:49.859+0000 7ff0737fe700 1 --2- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7ff05c0383b0 0x7ff05c03a870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-13T04:30:56.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:51.461+0000 7ff0737fe700 1 -- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7ff05c0383b0 msgr2=0x7ff05c03a870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/2772434518 2023-09-13T04:30:56.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:51.461+0000 7ff0737fe700 1 --2- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7ff05c0383b0 0x7ff05c03a870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-09-13T04:30:56.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:54.663+0000 7ff0737fe700 1 -- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7ff05c0383b0 msgr2=0x7ff05c03a870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/2772434518 2023-09-13T04:30:56.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:54.663+0000 7ff0737fe700 1 --2- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7ff05c0383b0 0x7ff05c03a870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-09-13T04:30:56.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:54.964+0000 7ff0717fa700 1 -- 172.21.15.2:0/2403806534 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mgrmap(e 6) v1 ==== 45098+0+0 (secure 0 0 0) 0x7ff06402d3f0 con 0x7ff07407be90 2023-09-13T04:30:56.507 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:54.964+0000 7ff0717fa700 1 -- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7ff05c0383b0 msgr2=0x7ff05c03a870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-13T04:30:56.507 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:54.964+0000 7ff0717fa700 1 --2- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7ff05c0383b0 0x7ff05c03a870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:56.507 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:55.966+0000 7ff0717fa700 1 -- 172.21.15.2:0/2403806534 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 7) v1 ==== 45225+0+0 (secure 0 0 0) 0x7ff06404dcf0 con 0x7ff07407be90 2023-09-13T04:30:56.507 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:55.966+0000 7ff0717fa700 1 --2- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7ff05c03bc90 0x7ff05c03e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:56.507 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:55.966+0000 7ff0717fa700 1 -- 172.21.15.2:0/2403806534 --> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff05c03af80 con 0x7ff05c03bc90 2023-09-13T04:30:56.508 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:55.967+0000 7ff0737fe700 1 --2- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7ff05c03bc90 0x7ff05c03e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:56.508 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:55.968+0000 7ff0737fe700 1 --2- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7ff05c03bc90 0x7ff05c03e080 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff068003a10 tx=0x7ff0680092b0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:56.508 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:55.969+0000 7ff0717fa700 1 -- 172.21.15.2:0/2403806534 <== mgr.14116 v2:172.21.15.2:6800/1393377368 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7ff05c03af80 con 0x7ff05c03bc90 2023-09-13T04:30:56.508 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:55.980+0000 7ff07a928700 1 -- 172.21.15.2:0/2403806534 --> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7ff07419d750 con 0x7ff05c03bc90 2023-09-13T04:30:56.508 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:55.980+0000 7ff0717fa700 1 -- 172.21.15.2:0/2403806534 <== mgr.14116 v2:172.21.15.2:6800/1393377368 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ff07419d750 con 0x7ff05c03bc90 2023-09-13T04:30:56.509 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:55.980+0000 7ff07a928700 1 -- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7ff05c03bc90 msgr2=0x7ff05c03e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:56.509 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:55.980+0000 7ff07a928700 1 --2- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7ff05c03bc90 0x7ff05c03e080 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff068003a10 tx=0x7ff0680092b0 comp rx=0 tx=0).stop 2023-09-13T04:30:56.509 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:55.980+0000 7ff07a928700 1 -- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff07407be90 msgr2=0x7ff07419c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:56.509 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:55.980+0000 7ff07a928700 1 --2- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff07407be90 0x7ff07419c410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff064006af0 tx=0x7ff064011ad0 comp rx=0 tx=0).stop 2023-09-13T04:30:56.509 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:55.980+0000 7ff07a928700 1 -- 172.21.15.2:0/2403806534 shutdown_connections 2023-09-13T04:30:56.510 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:55.980+0000 7ff07a928700 1 --2- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7ff05c03bc90 0x7ff05c03e080 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:56.510 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:55.980+0000 7ff07a928700 1 --2- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff07407be90 0x7ff07419c410 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:56.510 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:55.980+0000 7ff07a928700 1 --2- 172.21.15.2:0/2403806534 >> [v2:172.21.15.2:6800/2772434518,v1:172.21.15.2:6801/2772434518] conn(0x7ff05c0383b0 0x7ff05c03a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:56.510 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:55.980+0000 7ff07a928700 1 -- 172.21.15.2:0/2403806534 >> 172.21.15.2:0/2403806534 conn(0x7ff074101b00 msgr2=0x7ff0741031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:30:56.510 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:55.981+0000 7ff07a928700 1 -- 172.21.15.2:0/2403806534 shutdown_connections 2023-09-13T04:30:56.511 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:55.981+0000 7ff07a928700 1 -- 172.21.15.2:0/2403806534 wait complete. 2023-09-13T04:30:56.511 INFO:teuthology.orchestra.run.smithi002.stdout:mgr epoch 5 is available 2023-09-13T04:30:56.511 INFO:teuthology.orchestra.run.smithi002.stdout:Setting orchestrator backend to cephadm... 2023-09-13T04:30:57.444 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:57 smithi002 ceph-mon[97651]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-09-13T04:30:57.444 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:57 smithi002 ceph-mon[97651]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-09-13T04:30:57.444 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:57 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:30:57.776 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.055+0000 7f8fd9689700 1 Processor -- start 2023-09-13T04:30:57.777 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.056+0000 7f8fd9689700 1 -- start start 2023-09-13T04:30:57.777 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.056+0000 7f8fd9689700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fd4106f90 0x7f8fd4109380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:57.777 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.056+0000 7f8fd9689700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd4079ec0 con 0x7f8fd4106f90 2023-09-13T04:30:57.777 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.057+0000 7f8fd2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fd4106f90 0x7f8fd4109380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:57.777 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.057+0000 7f8fd2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fd4106f90 0x7f8fd4109380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45300/0 (socket says 172.21.15.2:45300) 2023-09-13T04:30:57.778 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.057+0000 7f8fd2ffd700 1 -- 172.21.15.2:0/1740731812 learned_addr learned my addr 172.21.15.2:0/1740731812 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:30:57.778 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.057+0000 7f8fd2ffd700 1 -- 172.21.15.2:0/1740731812 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd41098c0 con 0x7f8fd4106f90 2023-09-13T04:30:57.778 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.057+0000 7f8fd2ffd700 1 --2- 172.21.15.2:0/1740731812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fd4106f90 0x7f8fd4109380 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40098a0 tx=0x7f8fc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3008f6907c7e22 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:57.778 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.057+0000 7f8fd1ffb700 1 -- 172.21.15.2:0/1740731812 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fc4004160 con 0x7f8fd4106f90 2023-09-13T04:30:57.779 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.057+0000 7f8fd1ffb700 1 -- 172.21.15.2:0/1740731812 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8fc40042c0 con 0x7f8fd4106f90 2023-09-13T04:30:57.779 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.057+0000 7f8fd1ffb700 1 -- 172.21.15.2:0/1740731812 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fc4011a00 con 0x7f8fd4106f90 2023-09-13T04:30:57.779 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.058+0000 7f8fd9689700 1 -- 172.21.15.2:0/1740731812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fd4106f90 msgr2=0x7f8fd4109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:57.779 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.058+0000 7f8fd9689700 1 --2- 172.21.15.2:0/1740731812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fd4106f90 0x7f8fd4109380 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40098a0 tx=0x7f8fc4009bb0 comp rx=0 tx=0).stop 2023-09-13T04:30:57.779 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.058+0000 7f8fd9689700 1 -- 172.21.15.2:0/1740731812 shutdown_connections 2023-09-13T04:30:57.779 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.058+0000 7f8fd9689700 1 --2- 172.21.15.2:0/1740731812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fd4106f90 0x7f8fd4109380 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:57.780 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.058+0000 7f8fd9689700 1 -- 172.21.15.2:0/1740731812 >> 172.21.15.2:0/1740731812 conn(0x7f8fd41009e0 msgr2=0x7f8fd4102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:30:57.780 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.058+0000 7f8fd9689700 1 -- 172.21.15.2:0/1740731812 shutdown_connections 2023-09-13T04:30:57.780 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.058+0000 7f8fd9689700 1 -- 172.21.15.2:0/1740731812 wait complete. 2023-09-13T04:30:57.780 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.059+0000 7f8fd9689700 1 Processor -- start 2023-09-13T04:30:57.780 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.059+0000 7f8fd9689700 1 -- start start 2023-09-13T04:30:57.780 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.060+0000 7f8fd9689700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fd4106f90 0x7f8fd41068f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:57.781 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.060+0000 7f8fd9689700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd4106e30 con 0x7f8fd4106f90 2023-09-13T04:30:57.781 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.060+0000 7f8fd2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fd4106f90 0x7f8fd41068f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:57.781 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.060+0000 7f8fd2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fd4106f90 0x7f8fd41068f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45308/0 (socket says 172.21.15.2:45308) 2023-09-13T04:30:57.781 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.060+0000 7f8fd2ffd700 1 -- 172.21.15.2:0/3996372819 learned_addr learned my addr 172.21.15.2:0/3996372819 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:30:57.781 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.060+0000 7f8fd2ffd700 1 -- 172.21.15.2:0/3996372819 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc4009550 con 0x7f8fd4106f90 2023-09-13T04:30:57.782 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.060+0000 7f8fd2ffd700 1 --2- 172.21.15.2:0/3996372819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fd4106f90 0x7f8fd41068f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4006af0 tx=0x7f8fc4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:57.782 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.061+0000 7f8fbbfff700 1 -- 172.21.15.2:0/3996372819 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fc4003a40 con 0x7f8fd4106f90 2023-09-13T04:30:57.782 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.061+0000 7f8fbbfff700 1 -- 172.21.15.2:0/3996372819 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8fc4003ba0 con 0x7f8fd4106f90 2023-09-13T04:30:57.782 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.061+0000 7f8fd9689700 1 -- 172.21.15.2:0/3996372819 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd4104f40 con 0x7f8fd4106f90 2023-09-13T04:30:57.782 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.061+0000 7f8fbbfff700 1 -- 172.21.15.2:0/3996372819 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fc401a6e0 con 0x7f8fd4106f90 2023-09-13T04:30:57.783 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.061+0000 7f8fd9689700 1 -- 172.21.15.2:0/3996372819 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd4105360 con 0x7f8fd4106f90 2023-09-13T04:30:57.783 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.062+0000 7f8fbbfff700 1 -- 172.21.15.2:0/3996372819 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 7) v1 ==== 45225+0+0 (secure 0 0 0) 0x7f8fc401a860 con 0x7f8fd4106f90 2023-09-13T04:30:57.783 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.062+0000 7f8fd9689700 1 -- 172.21.15.2:0/3996372819 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd410a6c0 con 0x7f8fd4106f90 2023-09-13T04:30:57.783 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.062+0000 7f8fbbfff700 1 --2- 172.21.15.2:0/3996372819 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f8fbc038280 0x7f8fbc03a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:57.783 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.062+0000 7f8fbbfff700 1 -- 172.21.15.2:0/3996372819 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8fc404c120 con 0x7f8fd4106f90 2023-09-13T04:30:57.784 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.063+0000 7f8fd27fc700 1 --2- 172.21.15.2:0/3996372819 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f8fbc038280 0x7f8fbc03a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:57.784 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.063+0000 7f8fd27fc700 1 --2- 172.21.15.2:0/3996372819 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f8fbc038280 0x7f8fbc03a740 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8006fd0 tx=0x7f8fc8006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:57.784 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.068+0000 7f8fbbfff700 1 -- 172.21.15.2:0/3996372819 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8fc4024a60 con 0x7f8fd4106f90 2023-09-13T04:30:57.784 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.114+0000 7f8fbbfff700 1 -- 172.21.15.2:0/3996372819 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 8) v1 ==== 45331+0+0 (secure 0 0 0) 0x7f8fc402d3f0 con 0x7f8fd4106f90 2023-09-13T04:30:57.784 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.295+0000 7f8fd9689700 1 -- 172.21.15.2:0/3996372819 --> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fd4105ad0 con 0x7f8fbc038280 2023-09-13T04:30:57.785 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.302+0000 7f8fbbfff700 1 -- 172.21.15.2:0/3996372819 <== mgr.14116 v2:172.21.15.2:6800/1393377368 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8fd4105ad0 con 0x7f8fbc038280 2023-09-13T04:30:57.785 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.305+0000 7f8fd9689700 1 -- 172.21.15.2:0/3996372819 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f8fbc038280 msgr2=0x7f8fbc03a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:57.785 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.305+0000 7f8fd9689700 1 --2- 172.21.15.2:0/3996372819 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f8fbc038280 0x7f8fbc03a740 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8006fd0 tx=0x7f8fc8006e40 comp rx=0 tx=0).stop 2023-09-13T04:30:57.785 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.305+0000 7f8fd9689700 1 -- 172.21.15.2:0/3996372819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fd4106f90 msgr2=0x7f8fd41068f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:57.785 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.305+0000 7f8fd9689700 1 --2- 172.21.15.2:0/3996372819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fd4106f90 0x7f8fd41068f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4006af0 tx=0x7f8fc4011ad0 comp rx=0 tx=0).stop 2023-09-13T04:30:57.786 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.305+0000 7f8fd9689700 1 -- 172.21.15.2:0/3996372819 shutdown_connections 2023-09-13T04:30:57.786 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.305+0000 7f8fd9689700 1 --2- 172.21.15.2:0/3996372819 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f8fbc038280 0x7f8fbc03a740 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:57.786 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.305+0000 7f8fd9689700 1 --2- 172.21.15.2:0/3996372819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fd4106f90 0x7f8fd41068f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:57.786 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.305+0000 7f8fd9689700 1 -- 172.21.15.2:0/3996372819 >> 172.21.15.2:0/3996372819 conn(0x7f8fd41009e0 msgr2=0x7f8fd41017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:30:57.786 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.306+0000 7f8fd9689700 1 -- 172.21.15.2:0/3996372819 shutdown_connections 2023-09-13T04:30:57.787 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:57.306+0000 7f8fd9689700 1 -- 172.21.15.2:0/3996372819 wait complete. 2023-09-13T04:30:58.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:58 smithi002 ceph-mon[97651]: [13/Sep/2023:04:30:56] ENGINE Bus STARTING 2023-09-13T04:30:58.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:58 smithi002 ceph-mon[97651]: [13/Sep/2023:04:30:56] ENGINE Serving on https://172.21.15.2:7150 2023-09-13T04:30:58.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:58 smithi002 ceph-mon[97651]: [13/Sep/2023:04:30:56] ENGINE Serving on http://172.21.15.2:8765 2023-09-13T04:30:58.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:58 smithi002 ceph-mon[97651]: [13/Sep/2023:04:30:56] ENGINE Bus STARTED 2023-09-13T04:30:58.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:58 smithi002 ceph-mon[97651]: mgrmap e8: x(active, since 2s) 2023-09-13T04:30:58.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:58 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' 2023-09-13T04:30:58.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:58 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:30:58.956 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout value unchanged 2023-09-13T04:30:58.957 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.277+0000 7f651b63c700 1 Processor -- start 2023-09-13T04:30:58.957 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.277+0000 7f651b63c700 1 -- start start 2023-09-13T04:30:58.957 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.278+0000 7f651b63c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f651407bed0 0x7f651407a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:58.957 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.278+0000 7f651b63c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651407a900 con 0x7f651407bed0 2023-09-13T04:30:58.958 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.278+0000 7f65193d8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f651407bed0 0x7f651407a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:58.958 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.278+0000 7f65193d8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f651407bed0 0x7f651407a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45322/0 (socket says 172.21.15.2:45322) 2023-09-13T04:30:58.958 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.278+0000 7f65193d8700 1 -- 172.21.15.2:0/2063638017 learned_addr learned my addr 172.21.15.2:0/2063638017 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:30:58.958 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.278+0000 7f65193d8700 1 -- 172.21.15.2:0/2063638017 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651407aa40 con 0x7f651407bed0 2023-09-13T04:30:58.958 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.279+0000 7f65193d8700 1 --2- 172.21.15.2:0/2063638017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f651407bed0 0x7f651407a330 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f65100098a0 tx=0x7f6510009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4290c80f5743cf server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:58.959 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.279+0000 7f650bfff700 1 -- 172.21.15.2:0/2063638017 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6510004160 con 0x7f651407bed0 2023-09-13T04:30:58.959 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.279+0000 7f650bfff700 1 -- 172.21.15.2:0/2063638017 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f65100042c0 con 0x7f651407bed0 2023-09-13T04:30:58.959 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.279+0000 7f650bfff700 1 -- 172.21.15.2:0/2063638017 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6510011a00 con 0x7f651407bed0 2023-09-13T04:30:58.959 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.279+0000 7f651b63c700 1 -- 172.21.15.2:0/2063638017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f651407bed0 msgr2=0x7f651407a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:58.959 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.279+0000 7f651b63c700 1 --2- 172.21.15.2:0/2063638017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f651407bed0 0x7f651407a330 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f65100098a0 tx=0x7f6510009bb0 comp rx=0 tx=0).stop 2023-09-13T04:30:58.960 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.280+0000 7f651b63c700 1 -- 172.21.15.2:0/2063638017 shutdown_connections 2023-09-13T04:30:58.960 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.280+0000 7f651b63c700 1 --2- 172.21.15.2:0/2063638017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f651407bed0 0x7f651407a330 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:58.960 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.280+0000 7f651b63c700 1 -- 172.21.15.2:0/2063638017 >> 172.21.15.2:0/2063638017 conn(0x7f6514101b00 msgr2=0x7f6514103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:30:58.960 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.280+0000 7f651b63c700 1 -- 172.21.15.2:0/2063638017 shutdown_connections 2023-09-13T04:30:58.960 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.280+0000 7f651b63c700 1 -- 172.21.15.2:0/2063638017 wait complete. 2023-09-13T04:30:58.961 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.281+0000 7f651b63c700 1 Processor -- start 2023-09-13T04:30:58.961 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.282+0000 7f651b63c700 1 -- start start 2023-09-13T04:30:58.961 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.282+0000 7f651b63c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f651407bed0 0x7f651419c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:58.961 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.282+0000 7f651b63c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651419c940 con 0x7f651407bed0 2023-09-13T04:30:58.961 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.282+0000 7f65193d8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f651407bed0 0x7f651419c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:58.962 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.282+0000 7f65193d8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f651407bed0 0x7f651419c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45334/0 (socket says 172.21.15.2:45334) 2023-09-13T04:30:58.962 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.282+0000 7f65193d8700 1 -- 172.21.15.2:0/896673029 learned_addr learned my addr 172.21.15.2:0/896673029 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:30:58.962 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.283+0000 7f65193d8700 1 -- 172.21.15.2:0/896673029 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6510009550 con 0x7f651407bed0 2023-09-13T04:30:58.962 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.283+0000 7f65193d8700 1 --2- 172.21.15.2:0/896673029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f651407bed0 0x7f651419c400 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6510006af0 tx=0x7f6510011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:58.962 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.283+0000 7f650a7fc700 1 -- 172.21.15.2:0/896673029 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6510003a40 con 0x7f651407bed0 2023-09-13T04:30:58.963 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.283+0000 7f650a7fc700 1 -- 172.21.15.2:0/896673029 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6510003ba0 con 0x7f651407bed0 2023-09-13T04:30:58.963 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.283+0000 7f651b63c700 1 -- 172.21.15.2:0/896673029 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f651419cb40 con 0x7f651407bed0 2023-09-13T04:30:58.963 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.283+0000 7f650a7fc700 1 -- 172.21.15.2:0/896673029 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f651001a700 con 0x7f651407bed0 2023-09-13T04:30:58.963 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.283+0000 7f651b63c700 1 -- 172.21.15.2:0/896673029 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651419cf60 con 0x7f651407bed0 2023-09-13T04:30:58.963 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.284+0000 7f650a7fc700 1 -- 172.21.15.2:0/896673029 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45331+0+0 (secure 0 0 0) 0x7f6510024ad0 con 0x7f651407bed0 2023-09-13T04:30:58.964 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.284+0000 7f651b63c700 1 -- 172.21.15.2:0/896673029 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65140622e0 con 0x7f651407bed0 2023-09-13T04:30:58.964 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.285+0000 7f650a7fc700 1 --2- 172.21.15.2:0/896673029 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f65000383b0 0x7f650003a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:30:58.964 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.285+0000 7f650a7fc700 1 -- 172.21.15.2:0/896673029 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f651004c260 con 0x7f651407bed0 2023-09-13T04:30:58.964 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.285+0000 7f6518bd7700 1 --2- 172.21.15.2:0/896673029 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f65000383b0 0x7f650003a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:30:58.964 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.286+0000 7f6518bd7700 1 --2- 172.21.15.2:0/896673029 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f65000383b0 0x7f650003a870 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6504006fd0 tx=0x7f6504006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:30:58.965 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.290+0000 7f650a7fc700 1 -- 172.21.15.2:0/896673029 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6510018b80 con 0x7f651407bed0 2023-09-13T04:30:58.965 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.503+0000 7f651b63c700 1 -- 172.21.15.2:0/896673029 --> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f6514103de0 con 0x7f65000383b0 2023-09-13T04:30:58.965 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.504+0000 7f650a7fc700 1 -- 172.21.15.2:0/896673029 <== mgr.14116 v2:172.21.15.2:6800/1393377368 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f6514103de0 con 0x7f65000383b0 2023-09-13T04:30:58.965 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.506+0000 7f651b63c700 1 -- 172.21.15.2:0/896673029 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f65000383b0 msgr2=0x7f650003a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:58.965 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.506+0000 7f651b63c700 1 --2- 172.21.15.2:0/896673029 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f65000383b0 0x7f650003a870 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6504006fd0 tx=0x7f6504006e40 comp rx=0 tx=0).stop 2023-09-13T04:30:58.966 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.506+0000 7f651b63c700 1 -- 172.21.15.2:0/896673029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f651407bed0 msgr2=0x7f651419c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:30:58.966 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.506+0000 7f651b63c700 1 --2- 172.21.15.2:0/896673029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f651407bed0 0x7f651419c400 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6510006af0 tx=0x7f6510011ad0 comp rx=0 tx=0).stop 2023-09-13T04:30:58.966 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.506+0000 7f651b63c700 1 -- 172.21.15.2:0/896673029 shutdown_connections 2023-09-13T04:30:58.966 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.506+0000 7f651b63c700 1 --2- 172.21.15.2:0/896673029 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f65000383b0 0x7f650003a870 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:58.966 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.506+0000 7f651b63c700 1 --2- 172.21.15.2:0/896673029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f651407bed0 0x7f651419c400 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:30:58.967 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.506+0000 7f651b63c700 1 -- 172.21.15.2:0/896673029 >> 172.21.15.2:0/896673029 conn(0x7f6514101b00 msgr2=0x7f65141031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:30:58.967 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.507+0000 7f651b63c700 1 -- 172.21.15.2:0/896673029 shutdown_connections 2023-09-13T04:30:58.967 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:58.507+0000 7f651b63c700 1 -- 172.21.15.2:0/896673029 wait complete. 2023-09-13T04:30:58.967 INFO:teuthology.orchestra.run.smithi002.stdout:Generating ssh key... 2023-09-13T04:30:59.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:30:59 smithi002 ceph-mon[97651]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:31:00.491 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:00 smithi002 ceph-mon[97651]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:31:00.755 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: Generating public/private rsa key pair. 2023-09-13T04:31:00.756 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: Your identification has been saved in /tmp/tmp0pzwnddq/key. 2023-09-13T04:31:00.756 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: Your public key has been saved in /tmp/tmp0pzwnddq/key.pub. 2023-09-13T04:31:00.756 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: The key fingerprint is: 2023-09-13T04:31:00.756 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: SHA256:OMiwbjPG9yche0xsoOkb9utfbggVmjx98rw7Ri8Us54 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:31:00.757 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: The key's randomart image is: 2023-09-13T04:31:00.757 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: +---[RSA 3072]----+ 2023-09-13T04:31:00.757 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: | | 2023-09-13T04:31:00.757 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: | . | 2023-09-13T04:31:00.757 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: | .. + . | 2023-09-13T04:31:00.758 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: | +*.+.+ | 2023-09-13T04:31:00.758 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: | .oo=o=S+ | 2023-09-13T04:31:00.758 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: | oo o =.* | 2023-09-13T04:31:00.758 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: | .X .B *.+ | 2023-09-13T04:31:00.758 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: | +.*..=oE . | 2023-09-13T04:31:00.759 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: | .o++o=o+ | 2023-09-13T04:31:00.759 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: +----[SHA256]-----+ 2023-09-13T04:31:00.939 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.493+0000 7f9598bfd700 1 Processor -- start 2023-09-13T04:31:00.939 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.494+0000 7f9598bfd700 1 -- start start 2023-09-13T04:31:00.939 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.494+0000 7f9598bfd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f959407beb0 0x7f959407a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:00.940 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.494+0000 7f9598bfd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959407a8e0 con 0x7f959407beb0 2023-09-13T04:31:00.940 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.494+0000 7f959259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f959407beb0 0x7f959407a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:00.940 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.494+0000 7f959259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f959407beb0 0x7f959407a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45342/0 (socket says 172.21.15.2:45342) 2023-09-13T04:31:00.940 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.494+0000 7f959259c700 1 -- 172.21.15.2:0/2916064477 learned_addr learned my addr 172.21.15.2:0/2916064477 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:00.941 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.495+0000 7f959259c700 1 -- 172.21.15.2:0/2916064477 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959407aa20 con 0x7f959407beb0 2023-09-13T04:31:00.941 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.495+0000 7f959259c700 1 --2- 172.21.15.2:0/2916064477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f959407beb0 0x7f959407a310 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f95840098a0 tx=0x7f9584009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27a6131b71f72b8 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:00.941 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.495+0000 7f959159a700 1 -- 172.21.15.2:0/2916064477 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9584004160 con 0x7f959407beb0 2023-09-13T04:31:00.941 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.495+0000 7f959159a700 1 -- 172.21.15.2:0/2916064477 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f95840042c0 con 0x7f959407beb0 2023-09-13T04:31:00.942 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.495+0000 7f959159a700 1 -- 172.21.15.2:0/2916064477 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9584011a00 con 0x7f959407beb0 2023-09-13T04:31:00.942 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.496+0000 7f9598bfd700 1 -- 172.21.15.2:0/2916064477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f959407beb0 msgr2=0x7f959407a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:00.942 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.496+0000 7f9598bfd700 1 --2- 172.21.15.2:0/2916064477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f959407beb0 0x7f959407a310 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f95840098a0 tx=0x7f9584009bb0 comp rx=0 tx=0).stop 2023-09-13T04:31:00.942 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.496+0000 7f9598bfd700 1 -- 172.21.15.2:0/2916064477 shutdown_connections 2023-09-13T04:31:00.942 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.496+0000 7f9598bfd700 1 --2- 172.21.15.2:0/2916064477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f959407beb0 0x7f959407a310 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:00.943 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.496+0000 7f9598bfd700 1 -- 172.21.15.2:0/2916064477 >> 172.21.15.2:0/2916064477 conn(0x7f9594101b00 msgr2=0x7f9594103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:00.943 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.496+0000 7f9598bfd700 1 -- 172.21.15.2:0/2916064477 shutdown_connections 2023-09-13T04:31:00.943 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.496+0000 7f9598bfd700 1 -- 172.21.15.2:0/2916064477 wait complete. 2023-09-13T04:31:00.943 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.497+0000 7f9598bfd700 1 Processor -- start 2023-09-13T04:31:00.944 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.497+0000 7f9598bfd700 1 -- start start 2023-09-13T04:31:00.944 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.497+0000 7f9598bfd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f959407beb0 0x7f959419c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:00.944 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.497+0000 7f9598bfd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959419c960 con 0x7f959407beb0 2023-09-13T04:31:00.944 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.498+0000 7f959259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f959407beb0 0x7f959419c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:00.944 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.498+0000 7f959259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f959407beb0 0x7f959419c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45346/0 (socket says 172.21.15.2:45346) 2023-09-13T04:31:00.945 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.498+0000 7f959259c700 1 -- 172.21.15.2:0/3426075886 learned_addr learned my addr 172.21.15.2:0/3426075886 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:00.945 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.498+0000 7f959259c700 1 -- 172.21.15.2:0/3426075886 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9584009550 con 0x7f959407beb0 2023-09-13T04:31:00.945 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.498+0000 7f959259c700 1 --2- 172.21.15.2:0/3426075886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f959407beb0 0x7f959419c420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9584006af0 tx=0x7f9584011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:00.945 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.498+0000 7f95837fe700 1 -- 172.21.15.2:0/3426075886 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9584003a40 con 0x7f959407beb0 2023-09-13T04:31:00.945 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.498+0000 7f95837fe700 1 -- 172.21.15.2:0/3426075886 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9584003ba0 con 0x7f959407beb0 2023-09-13T04:31:00.946 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.498+0000 7f9598bfd700 1 -- 172.21.15.2:0/3426075886 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f959419cb60 con 0x7f959407beb0 2023-09-13T04:31:00.946 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.498+0000 7f95837fe700 1 -- 172.21.15.2:0/3426075886 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f958401a700 con 0x7f959407beb0 2023-09-13T04:31:00.946 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.499+0000 7f9598bfd700 1 -- 172.21.15.2:0/3426075886 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f959419cf80 con 0x7f959407beb0 2023-09-13T04:31:00.946 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.500+0000 7f95837fe700 1 -- 172.21.15.2:0/3426075886 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45331+0+0 (secure 0 0 0) 0x7f9584024ad0 con 0x7f959407beb0 2023-09-13T04:31:00.946 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.500+0000 7f9598bfd700 1 -- 172.21.15.2:0/3426075886 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f959410c8f0 con 0x7f959407beb0 2023-09-13T04:31:00.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.500+0000 7f95837fe700 1 --2- 172.21.15.2:0/3426075886 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f957c0383b0 0x7f957c03a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:00.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.500+0000 7f95837fe700 1 -- 172.21.15.2:0/3426075886 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f958404c2f0 con 0x7f959407beb0 2023-09-13T04:31:00.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.500+0000 7f9591d9b700 1 --2- 172.21.15.2:0/3426075886 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f957c0383b0 0x7f957c03a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:00.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.501+0000 7f9591d9b700 1 --2- 172.21.15.2:0/3426075886 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f957c0383b0 0x7f957c03a870 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9588006fd0 tx=0x7f9588006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:00.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.506+0000 7f95837fe700 1 -- 172.21.15.2:0/3426075886 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9584018b80 con 0x7f959407beb0 2023-09-13T04:31:00.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:30:59.716+0000 7f9598bfd700 1 -- 172.21.15.2:0/3426075886 --> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f959419f9c0 con 0x7f957c0383b0 2023-09-13T04:31:00.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:00.497+0000 7f95837fe700 1 -- 172.21.15.2:0/3426075886 <== mgr.14116 v2:172.21.15.2:6800/1393377368 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f959419f9c0 con 0x7f957c0383b0 2023-09-13T04:31:00.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:00.500+0000 7f9598bfd700 1 -- 172.21.15.2:0/3426075886 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f957c0383b0 msgr2=0x7f957c03a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:00.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:00.500+0000 7f9598bfd700 1 --2- 172.21.15.2:0/3426075886 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f957c0383b0 0x7f957c03a870 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9588006fd0 tx=0x7f9588006e40 comp rx=0 tx=0).stop 2023-09-13T04:31:00.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:00.500+0000 7f9598bfd700 1 -- 172.21.15.2:0/3426075886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f959407beb0 msgr2=0x7f959419c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:00.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:00.500+0000 7f9598bfd700 1 --2- 172.21.15.2:0/3426075886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f959407beb0 0x7f959419c420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9584006af0 tx=0x7f9584011ad0 comp rx=0 tx=0).stop 2023-09-13T04:31:00.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:00.500+0000 7f9598bfd700 1 -- 172.21.15.2:0/3426075886 shutdown_connections 2023-09-13T04:31:00.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:00.500+0000 7f9598bfd700 1 --2- 172.21.15.2:0/3426075886 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f957c0383b0 0x7f957c03a870 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:00.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:00.500+0000 7f9598bfd700 1 --2- 172.21.15.2:0/3426075886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f959407beb0 0x7f959419c420 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:00.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:00.500+0000 7f9598bfd700 1 -- 172.21.15.2:0/3426075886 >> 172.21.15.2:0/3426075886 conn(0x7f9594101b00 msgr2=0x7f95941031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:00.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:00.500+0000 7f9598bfd700 1 -- 172.21.15.2:0/3426075886 shutdown_connections 2023-09-13T04:31:00.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:00.500+0000 7f9598bfd700 1 -- 172.21.15.2:0/3426075886 wait complete. 2023-09-13T04:31:01.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:01 smithi002 ceph-mon[97651]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:31:01.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:01 smithi002 ceph-mon[97651]: Generating ssh key... 2023-09-13T04:31:01.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:01 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' 2023-09-13T04:31:01.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:01 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' 2023-09-13T04:31:02.218 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4/C7a9RXXyi7ovgVmcDjsCZ3q6B6mNpo/oRUdKobI5m6AJRvYNUaIB49L3JBgkgsI71Qvy2DT5b3PSaK6uHnk/DbY/xQ99ERuexjUtDbULn8inZupw9VYJpWuCahAm/9/y9eAj5n99d0hVAw4BqO9/jyhybiib7gMZtZRgllB39V3oWbZl0FDCwXUae7fqE8hSHrBLvhYfKw/sfSN0GbTHK53Xq1KgeHIjkfrHCHoihySn6rXPUa3Ppf5d7WY7zC3nyod1XuunrC/S3pHD7xXUxnOKGIXU0CnXeDujGYUkyJ9bbXg758ZdnfyhPInz/exIupLHajfj5vi6QQHFWiiOjvqsrloNCUC6+xJC/+EOiqaYkMxGwORrPbD6pOj9OrnctKyAd0wpOtRqG7JRWlwD6eEVU0m7yPjKPtIonsXQ/pP6jvZe8TxzxkPvCVFqlnrL2aI9zp46JDY9dLOfxVUhk3wt8VUbKpPMiWOkXTEaoE4eT04ZIL2iuZLiQ5hsIs= ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:31:02.219 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.476+0000 7fed48783700 1 Processor -- start 2023-09-13T04:31:02.219 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.477+0000 7fed48783700 1 -- start start 2023-09-13T04:31:02.219 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.477+0000 7fed48783700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed40106f90 0x7fed40109380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:02.219 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.477+0000 7fed48783700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed40079ec0 con 0x7fed40106f90 2023-09-13T04:31:02.219 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.477+0000 7fed4651f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed40106f90 0x7fed40109380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:02.220 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.478+0000 7fed4651f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed40106f90 0x7fed40109380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45362/0 (socket says 172.21.15.2:45362) 2023-09-13T04:31:02.220 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.478+0000 7fed4651f700 1 -- 172.21.15.2:0/2610488985 learned_addr learned my addr 172.21.15.2:0/2610488985 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:02.220 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.478+0000 7fed4651f700 1 -- 172.21.15.2:0/2610488985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed401098c0 con 0x7fed40106f90 2023-09-13T04:31:02.220 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.478+0000 7fed4651f700 1 --2- 172.21.15.2:0/2610488985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed40106f90 0x7fed40109380 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fed3c0098a0 tx=0x7fed3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c1c81a076a8621 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:02.220 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.478+0000 7fed4551d700 1 -- 172.21.15.2:0/2610488985 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed3c004160 con 0x7fed40106f90 2023-09-13T04:31:02.221 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.478+0000 7fed4551d700 1 -- 172.21.15.2:0/2610488985 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fed3c0042c0 con 0x7fed40106f90 2023-09-13T04:31:02.221 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.478+0000 7fed4551d700 1 -- 172.21.15.2:0/2610488985 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed3c011a00 con 0x7fed40106f90 2023-09-13T04:31:02.221 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.479+0000 7fed48783700 1 -- 172.21.15.2:0/2610488985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed40106f90 msgr2=0x7fed40109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:02.221 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.479+0000 7fed48783700 1 --2- 172.21.15.2:0/2610488985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed40106f90 0x7fed40109380 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fed3c0098a0 tx=0x7fed3c009bb0 comp rx=0 tx=0).stop 2023-09-13T04:31:02.221 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.479+0000 7fed48783700 1 -- 172.21.15.2:0/2610488985 shutdown_connections 2023-09-13T04:31:02.222 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.479+0000 7fed48783700 1 --2- 172.21.15.2:0/2610488985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed40106f90 0x7fed40109380 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:02.222 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.479+0000 7fed48783700 1 -- 172.21.15.2:0/2610488985 >> 172.21.15.2:0/2610488985 conn(0x7fed401009e0 msgr2=0x7fed40102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:02.222 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.479+0000 7fed48783700 1 -- 172.21.15.2:0/2610488985 shutdown_connections 2023-09-13T04:31:02.222 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.479+0000 7fed48783700 1 -- 172.21.15.2:0/2610488985 wait complete. 2023-09-13T04:31:02.222 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.480+0000 7fed48783700 1 Processor -- start 2023-09-13T04:31:02.223 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.480+0000 7fed48783700 1 -- start start 2023-09-13T04:31:02.223 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.481+0000 7fed48783700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed40106f90 0x7fed4019c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:02.223 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.481+0000 7fed48783700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4019c910 con 0x7fed40106f90 2023-09-13T04:31:02.223 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.481+0000 7fed4651f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed40106f90 0x7fed4019c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:02.223 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.481+0000 7fed4651f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed40106f90 0x7fed4019c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45378/0 (socket says 172.21.15.2:45378) 2023-09-13T04:31:02.224 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.481+0000 7fed4651f700 1 -- 172.21.15.2:0/3238029773 learned_addr learned my addr 172.21.15.2:0/3238029773 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:02.224 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.481+0000 7fed4651f700 1 -- 172.21.15.2:0/3238029773 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3c009550 con 0x7fed40106f90 2023-09-13T04:31:02.224 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.481+0000 7fed4651f700 1 --2- 172.21.15.2:0/3238029773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed40106f90 0x7fed4019c3d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009fb0 tx=0x7fed3c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:02.224 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.482+0000 7fed377fe700 1 -- 172.21.15.2:0/3238029773 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed3c003930 con 0x7fed40106f90 2023-09-13T04:31:02.224 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.482+0000 7fed377fe700 1 -- 172.21.15.2:0/3238029773 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fed3c003a90 con 0x7fed40106f90 2023-09-13T04:31:02.225 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.482+0000 7fed48783700 1 -- 172.21.15.2:0/3238029773 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed4019cb10 con 0x7fed40106f90 2023-09-13T04:31:02.225 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.482+0000 7fed377fe700 1 -- 172.21.15.2:0/3238029773 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed3c019610 con 0x7fed40106f90 2023-09-13T04:31:02.225 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.482+0000 7fed48783700 1 -- 172.21.15.2:0/3238029773 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed4019cf30 con 0x7fed40106f90 2023-09-13T04:31:02.225 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.483+0000 7fed377fe700 1 -- 172.21.15.2:0/3238029773 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45331+0+0 (secure 0 0 0) 0x7fed3c01a9d0 con 0x7fed40106f90 2023-09-13T04:31:02.225 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.483+0000 7fed48783700 1 -- 172.21.15.2:0/3238029773 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed400622e0 con 0x7fed40106f90 2023-09-13T04:31:02.225 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.483+0000 7fed377fe700 1 --2- 172.21.15.2:0/3238029773 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7fed2c038360 0x7fed2c03a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:02.226 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.483+0000 7fed377fe700 1 -- 172.21.15.2:0/3238029773 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fed3c04c450 con 0x7fed40106f90 2023-09-13T04:31:02.226 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.484+0000 7fed45d1e700 1 --2- 172.21.15.2:0/3238029773 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7fed2c038360 0x7fed2c03a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:02.226 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.484+0000 7fed45d1e700 1 --2- 172.21.15.2:0/3238029773 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7fed2c038360 0x7fed2c03a820 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fed30006fd0 tx=0x7fed30006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:02.226 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.489+0000 7fed377fe700 1 -- 172.21.15.2:0/3238029773 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fed3c018760 con 0x7fed40106f90 2023-09-13T04:31:02.227 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.702+0000 7fed48783700 1 -- 172.21.15.2:0/3238029773 --> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fed40101ee0 con 0x7fed2c038360 2023-09-13T04:31:02.227 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.703+0000 7fed377fe700 1 -- 172.21.15.2:0/3238029773 <== mgr.14116 v2:172.21.15.2:6800/1393377368 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fed40101ee0 con 0x7fed2c038360 2023-09-13T04:31:02.227 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.705+0000 7fed48783700 1 -- 172.21.15.2:0/3238029773 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7fed2c038360 msgr2=0x7fed2c03a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:02.227 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.705+0000 7fed48783700 1 --2- 172.21.15.2:0/3238029773 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7fed2c038360 0x7fed2c03a820 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fed30006fd0 tx=0x7fed30006e40 comp rx=0 tx=0).stop 2023-09-13T04:31:02.227 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.705+0000 7fed48783700 1 -- 172.21.15.2:0/3238029773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed40106f90 msgr2=0x7fed4019c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:02.227 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.705+0000 7fed48783700 1 --2- 172.21.15.2:0/3238029773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed40106f90 0x7fed4019c3d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009fb0 tx=0x7fed3c011ad0 comp rx=0 tx=0).stop 2023-09-13T04:31:02.228 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.705+0000 7fed48783700 1 -- 172.21.15.2:0/3238029773 shutdown_connections 2023-09-13T04:31:02.228 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.705+0000 7fed48783700 1 --2- 172.21.15.2:0/3238029773 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7fed2c038360 0x7fed2c03a820 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:02.228 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.705+0000 7fed48783700 1 --2- 172.21.15.2:0/3238029773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed40106f90 0x7fed4019c3d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:02.228 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.705+0000 7fed48783700 1 -- 172.21.15.2:0/3238029773 >> 172.21.15.2:0/3238029773 conn(0x7fed401009e0 msgr2=0x7fed401017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:02.228 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.705+0000 7fed48783700 1 -- 172.21.15.2:0/3238029773 shutdown_connections 2023-09-13T04:31:02.229 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:01.706+0000 7fed48783700 1 -- 172.21.15.2:0/3238029773 wait complete. 2023-09-13T04:31:02.229 INFO:teuthology.orchestra.run.smithi002.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-09-13T04:31:02.229 INFO:teuthology.orchestra.run.smithi002.stdout:Adding key to root@localhost authorized_keys... 2023-09-13T04:31:02.229 INFO:teuthology.orchestra.run.smithi002.stdout:Adding host smithi002... 2023-09-13T04:31:02.480 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:02 smithi002 ceph-mon[97651]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:31:03.635 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:03 smithi002 ceph-mon[97651]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi002", "addr": "172.21.15.2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:31:05.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:05 smithi002 ceph-mon[97651]: Deploying cephadm binary to smithi002 2023-09-13T04:31:06.809 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Added host 'smithi002' with addr '172.21.15.2' 2023-09-13T04:31:06.809 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.738+0000 7f59f20ba700 1 Processor -- start 2023-09-13T04:31:06.810 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.738+0000 7f59f20ba700 1 -- start start 2023-09-13T04:31:06.810 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.738+0000 7f59f20ba700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59ec07be70 0x7f59ec07a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:06.810 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.738+0000 7f59f20ba700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec07a8a0 con 0x7f59ec07be70 2023-09-13T04:31:06.810 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.739+0000 7f59eb7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59ec07be70 0x7f59ec07a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:06.811 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.739+0000 7f59eb7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59ec07be70 0x7f59ec07a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45390/0 (socket says 172.21.15.2:45390) 2023-09-13T04:31:06.811 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.739+0000 7f59eb7fe700 1 -- 172.21.15.2:0/1322259453 learned_addr learned my addr 172.21.15.2:0/1322259453 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:06.811 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.739+0000 7f59eb7fe700 1 -- 172.21.15.2:0/1322259453 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ec07a9e0 con 0x7f59ec07be70 2023-09-13T04:31:06.811 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.739+0000 7f59eb7fe700 1 --2- 172.21.15.2:0/1322259453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59ec07be70 0x7f59ec07a2d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0098a0 tx=0x7f59dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76136fee93e727cd server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:06.811 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.739+0000 7f59ea7fc700 1 -- 172.21.15.2:0/1322259453 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59dc004160 con 0x7f59ec07be70 2023-09-13T04:31:06.812 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.740+0000 7f59ea7fc700 1 -- 172.21.15.2:0/1322259453 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f59dc0042c0 con 0x7f59ec07be70 2023-09-13T04:31:06.812 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.740+0000 7f59ea7fc700 1 -- 172.21.15.2:0/1322259453 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59dc011a00 con 0x7f59ec07be70 2023-09-13T04:31:06.812 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.740+0000 7f59f20ba700 1 -- 172.21.15.2:0/1322259453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59ec07be70 msgr2=0x7f59ec07a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:06.812 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.740+0000 7f59f20ba700 1 --2- 172.21.15.2:0/1322259453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59ec07be70 0x7f59ec07a2d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0098a0 tx=0x7f59dc009bb0 comp rx=0 tx=0).stop 2023-09-13T04:31:06.812 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.740+0000 7f59f20ba700 1 -- 172.21.15.2:0/1322259453 shutdown_connections 2023-09-13T04:31:06.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.740+0000 7f59f20ba700 1 --2- 172.21.15.2:0/1322259453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59ec07be70 0x7f59ec07a2d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:06.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.740+0000 7f59f20ba700 1 -- 172.21.15.2:0/1322259453 >> 172.21.15.2:0/1322259453 conn(0x7f59ec101b00 msgr2=0x7f59ec103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:06.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.740+0000 7f59f20ba700 1 -- 172.21.15.2:0/1322259453 shutdown_connections 2023-09-13T04:31:06.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.740+0000 7f59f20ba700 1 -- 172.21.15.2:0/1322259453 wait complete. 2023-09-13T04:31:06.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.741+0000 7f59f20ba700 1 Processor -- start 2023-09-13T04:31:06.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.742+0000 7f59f20ba700 1 -- start start 2023-09-13T04:31:06.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.742+0000 7f59f20ba700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59ec07be70 0x7f59ec19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:06.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.742+0000 7f59f20ba700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec19c930 con 0x7f59ec07be70 2023-09-13T04:31:06.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.742+0000 7f59eb7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59ec07be70 0x7f59ec19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:06.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.742+0000 7f59eb7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59ec07be70 0x7f59ec19c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45396/0 (socket says 172.21.15.2:45396) 2023-09-13T04:31:06.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.742+0000 7f59eb7fe700 1 -- 172.21.15.2:0/1987340314 learned_addr learned my addr 172.21.15.2:0/1987340314 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:06.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.743+0000 7f59eb7fe700 1 -- 172.21.15.2:0/1987340314 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59dc009550 con 0x7f59ec07be70 2023-09-13T04:31:06.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.743+0000 7f59eb7fe700 1 --2- 172.21.15.2:0/1987340314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59ec07be70 0x7f59ec19c3f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f59dc006af0 tx=0x7f59dc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:06.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.743+0000 7f59e8ff9700 1 -- 172.21.15.2:0/1987340314 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59dc003a40 con 0x7f59ec07be70 2023-09-13T04:31:06.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.743+0000 7f59e8ff9700 1 -- 172.21.15.2:0/1987340314 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f59dc003ba0 con 0x7f59ec07be70 2023-09-13T04:31:06.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.743+0000 7f59e8ff9700 1 -- 172.21.15.2:0/1987340314 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59dc01a6e0 con 0x7f59ec07be70 2023-09-13T04:31:06.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.743+0000 7f59f20ba700 1 -- 172.21.15.2:0/1987340314 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ec19cb30 con 0x7f59ec07be70 2023-09-13T04:31:06.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.743+0000 7f59f20ba700 1 -- 172.21.15.2:0/1987340314 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ec19cf50 con 0x7f59ec07be70 2023-09-13T04:31:06.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.744+0000 7f59e8ff9700 1 -- 172.21.15.2:0/1987340314 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45331+0+0 (secure 0 0 0) 0x7f59dc024a60 con 0x7f59ec07be70 2023-09-13T04:31:06.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.744+0000 7f59f20ba700 1 -- 172.21.15.2:0/1987340314 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59ec0622e0 con 0x7f59ec07be70 2023-09-13T04:31:06.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.745+0000 7f59e8ff9700 1 --2- 172.21.15.2:0/1987340314 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f59d40383b0 0x7f59d403a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:06.817 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.745+0000 7f59e8ff9700 1 -- 172.21.15.2:0/1987340314 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f59dc04c320 con 0x7f59ec07be70 2023-09-13T04:31:06.817 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.745+0000 7f59eaffd700 1 --2- 172.21.15.2:0/1987340314 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f59d40383b0 0x7f59d403a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:06.817 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.746+0000 7f59eaffd700 1 --2- 172.21.15.2:0/1987340314 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f59d40383b0 0x7f59d403a870 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f59e0006fd0 tx=0x7f59e0006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:06.817 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.750+0000 7f59e8ff9700 1 -- 172.21.15.2:0/1987340314 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f59dc024430 con 0x7f59ec07be70 2023-09-13T04:31:06.817 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:02.961+0000 7f59f20ba700 1 -- 172.21.15.2:0/1987340314 --> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi002", "addr": "172.21.15.2", "target": ["mon-mgr", ""]}) v1 -- 0x7f59ec103de0 con 0x7f59d40383b0 2023-09-13T04:31:06.818 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:06.263+0000 7f59e8ff9700 1 -- 172.21.15.2:0/1987340314 <== mgr.14116 v2:172.21.15.2:6800/1393377368 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f59ec103de0 con 0x7f59d40383b0 2023-09-13T04:31:06.818 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:06.266+0000 7f59f20ba700 1 -- 172.21.15.2:0/1987340314 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f59d40383b0 msgr2=0x7f59d403a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:06.818 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:06.266+0000 7f59f20ba700 1 --2- 172.21.15.2:0/1987340314 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f59d40383b0 0x7f59d403a870 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f59e0006fd0 tx=0x7f59e0006e40 comp rx=0 tx=0).stop 2023-09-13T04:31:06.818 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:06.266+0000 7f59f20ba700 1 -- 172.21.15.2:0/1987340314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59ec07be70 msgr2=0x7f59ec19c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:06.818 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:06.266+0000 7f59f20ba700 1 --2- 172.21.15.2:0/1987340314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59ec07be70 0x7f59ec19c3f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f59dc006af0 tx=0x7f59dc011ad0 comp rx=0 tx=0).stop 2023-09-13T04:31:06.819 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:06.266+0000 7f59f20ba700 1 -- 172.21.15.2:0/1987340314 shutdown_connections 2023-09-13T04:31:06.819 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:06.266+0000 7f59f20ba700 1 --2- 172.21.15.2:0/1987340314 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f59d40383b0 0x7f59d403a870 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:06.819 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:06.266+0000 7f59f20ba700 1 --2- 172.21.15.2:0/1987340314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59ec07be70 0x7f59ec19c3f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:06.819 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:06.266+0000 7f59f20ba700 1 -- 172.21.15.2:0/1987340314 >> 172.21.15.2:0/1987340314 conn(0x7f59ec101b00 msgr2=0x7f59ec1031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:06.819 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:06.267+0000 7f59f20ba700 1 -- 172.21.15.2:0/1987340314 shutdown_connections 2023-09-13T04:31:06.819 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:06.267+0000 7f59f20ba700 1 -- 172.21.15.2:0/1987340314 wait complete. 2023-09-13T04:31:06.820 INFO:teuthology.orchestra.run.smithi002.stdout:Deploying unmanaged mon service... 2023-09-13T04:31:07.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:07 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' 2023-09-13T04:31:07.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:07 smithi002 ceph-mon[97651]: Added host smithi002 2023-09-13T04:31:07.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:07 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:31:08.495 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-09-13T04:31:08.496 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.412+0000 7f5367caa700 1 Processor -- start 2023-09-13T04:31:08.496 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.412+0000 7f5367caa700 1 -- start start 2023-09-13T04:31:08.496 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.412+0000 7f5367caa700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f536007beb0 0x7f536007a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:08.496 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.412+0000 7f5367caa700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536007a8e0 con 0x7f536007beb0 2023-09-13T04:31:08.497 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.413+0000 7f5365a46700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f536007beb0 0x7f536007a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:08.497 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.413+0000 7f5365a46700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f536007beb0 0x7f536007a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60652/0 (socket says 172.21.15.2:60652) 2023-09-13T04:31:08.497 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.413+0000 7f5365a46700 1 -- 172.21.15.2:0/67723191 learned_addr learned my addr 172.21.15.2:0/67723191 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:08.497 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.414+0000 7f5365a46700 1 -- 172.21.15.2:0/67723191 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536007aa20 con 0x7f536007beb0 2023-09-13T04:31:08.497 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.414+0000 7f5365a46700 1 --2- 172.21.15.2:0/67723191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f536007beb0 0x7f536007a310 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f535c0098a0 tx=0x7f535c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11cbc5c46b4390b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:08.498 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.414+0000 7f5364a44700 1 -- 172.21.15.2:0/67723191 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f535c004160 con 0x7f536007beb0 2023-09-13T04:31:08.499 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.414+0000 7f5364a44700 1 -- 172.21.15.2:0/67723191 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f535c0042c0 con 0x7f536007beb0 2023-09-13T04:31:08.499 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.414+0000 7f5364a44700 1 -- 172.21.15.2:0/67723191 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f535c011a00 con 0x7f536007beb0 2023-09-13T04:31:08.499 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.415+0000 7f5367caa700 1 -- 172.21.15.2:0/67723191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f536007beb0 msgr2=0x7f536007a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:08.499 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.415+0000 7f5367caa700 1 --2- 172.21.15.2:0/67723191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f536007beb0 0x7f536007a310 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f535c0098a0 tx=0x7f535c009bb0 comp rx=0 tx=0).stop 2023-09-13T04:31:08.499 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.415+0000 7f5367caa700 1 -- 172.21.15.2:0/67723191 shutdown_connections 2023-09-13T04:31:08.500 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.415+0000 7f5367caa700 1 --2- 172.21.15.2:0/67723191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f536007beb0 0x7f536007a310 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:08.500 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.415+0000 7f5367caa700 1 -- 172.21.15.2:0/67723191 >> 172.21.15.2:0/67723191 conn(0x7f5360101b00 msgr2=0x7f5360103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:08.500 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.415+0000 7f5367caa700 1 -- 172.21.15.2:0/67723191 shutdown_connections 2023-09-13T04:31:08.500 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.415+0000 7f5367caa700 1 -- 172.21.15.2:0/67723191 wait complete. 2023-09-13T04:31:08.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.416+0000 7f5367caa700 1 Processor -- start 2023-09-13T04:31:08.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.417+0000 7f5367caa700 1 -- start start 2023-09-13T04:31:08.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.417+0000 7f5367caa700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f536007beb0 0x7f536019c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:08.502 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.417+0000 7f5367caa700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536019c960 con 0x7f536007beb0 2023-09-13T04:31:08.502 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.417+0000 7f5365a46700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f536007beb0 0x7f536019c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:08.503 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.417+0000 7f5365a46700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f536007beb0 0x7f536019c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60668/0 (socket says 172.21.15.2:60668) 2023-09-13T04:31:08.503 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.417+0000 7f5365a46700 1 -- 172.21.15.2:0/1669892227 learned_addr learned my addr 172.21.15.2:0/1669892227 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:08.503 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.419+0000 7f5365a46700 1 -- 172.21.15.2:0/1669892227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c009550 con 0x7f536007beb0 2023-09-13T04:31:08.503 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.419+0000 7f5365a46700 1 --2- 172.21.15.2:0/1669892227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f536007beb0 0x7f536019c420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f535c006af0 tx=0x7f535c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:08.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.419+0000 7f5356ffd700 1 -- 172.21.15.2:0/1669892227 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f535c003a40 con 0x7f536007beb0 2023-09-13T04:31:08.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.419+0000 7f5356ffd700 1 -- 172.21.15.2:0/1669892227 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f535c003ba0 con 0x7f536007beb0 2023-09-13T04:31:08.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.419+0000 7f5367caa700 1 -- 172.21.15.2:0/1669892227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f536019cb60 con 0x7f536007beb0 2023-09-13T04:31:08.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.419+0000 7f5356ffd700 1 -- 172.21.15.2:0/1669892227 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f535c01a700 con 0x7f536007beb0 2023-09-13T04:31:08.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.419+0000 7f5367caa700 1 -- 172.21.15.2:0/1669892227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f536019cf80 con 0x7f536007beb0 2023-09-13T04:31:08.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.420+0000 7f5367caa700 1 -- 172.21.15.2:0/1669892227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53600622e0 con 0x7f536007beb0 2023-09-13T04:31:08.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.421+0000 7f5356ffd700 1 -- 172.21.15.2:0/1669892227 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45331+0+0 (secure 0 0 0) 0x7f535c01a980 con 0x7f536007beb0 2023-09-13T04:31:08.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.421+0000 7f5356ffd700 1 --2- 172.21.15.2:0/1669892227 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f534c038360 0x7f534c03a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:08.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.421+0000 7f5356ffd700 1 -- 172.21.15.2:0/1669892227 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f535c04c500 con 0x7f536007beb0 2023-09-13T04:31:08.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.421+0000 7f5365245700 1 --2- 172.21.15.2:0/1669892227 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f534c038360 0x7f534c03a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:08.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.422+0000 7f5365245700 1 --2- 172.21.15.2:0/1669892227 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f534c038360 0x7f534c03a820 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5350006fd0 tx=0x7f5350006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:08.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.427+0000 7f5356ffd700 1 -- 172.21.15.2:0/1669892227 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f535c018530 con 0x7f536007beb0 2023-09-13T04:31:08.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.657+0000 7f5367caa700 1 -- 172.21.15.2:0/1669892227 --> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f5360103e00 con 0x7f534c038360 2023-09-13T04:31:08.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.663+0000 7f5356ffd700 1 -- 172.21.15.2:0/1669892227 <== mgr.14116 v2:172.21.15.2:6800/1393377368 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f5360103e00 con 0x7f534c038360 2023-09-13T04:31:08.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.665+0000 7f5367caa700 1 -- 172.21.15.2:0/1669892227 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f534c038360 msgr2=0x7f534c03a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:08.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.665+0000 7f5367caa700 1 --2- 172.21.15.2:0/1669892227 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f534c038360 0x7f534c03a820 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5350006fd0 tx=0x7f5350006e40 comp rx=0 tx=0).stop 2023-09-13T04:31:08.507 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.665+0000 7f5367caa700 1 -- 172.21.15.2:0/1669892227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f536007beb0 msgr2=0x7f536019c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:08.507 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.665+0000 7f5367caa700 1 --2- 172.21.15.2:0/1669892227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f536007beb0 0x7f536019c420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f535c006af0 tx=0x7f535c011ad0 comp rx=0 tx=0).stop 2023-09-13T04:31:08.507 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.665+0000 7f5367caa700 1 -- 172.21.15.2:0/1669892227 shutdown_connections 2023-09-13T04:31:08.507 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.665+0000 7f5367caa700 1 --2- 172.21.15.2:0/1669892227 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f534c038360 0x7f534c03a820 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:08.507 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.665+0000 7f5367caa700 1 --2- 172.21.15.2:0/1669892227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f536007beb0 0x7f536019c420 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:08.508 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.665+0000 7f5367caa700 1 -- 172.21.15.2:0/1669892227 >> 172.21.15.2:0/1669892227 conn(0x7f5360101b00 msgr2=0x7f53601031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:08.508 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.666+0000 7f5367caa700 1 -- 172.21.15.2:0/1669892227 shutdown_connections 2023-09-13T04:31:08.508 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:07.666+0000 7f5367caa700 1 -- 172.21.15.2:0/1669892227 wait complete. 2023-09-13T04:31:08.508 INFO:teuthology.orchestra.run.smithi002.stdout:Deploying unmanaged mgr service... 2023-09-13T04:31:08.731 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:08 smithi002 ceph-mon[97651]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:31:08.732 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:08 smithi002 ceph-mon[97651]: Saving service mon spec with placement count:5 2023-09-13T04:31:08.732 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:08 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' 2023-09-13T04:31:09.981 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-09-13T04:31:09.981 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.251+0000 7f2c64ae3700 1 Processor -- start 2023-09-13T04:31:09.982 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.252+0000 7f2c64ae3700 1 -- start start 2023-09-13T04:31:09.982 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.252+0000 7f2c64ae3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c6007be70 0x7f2c6007a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:09.982 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.252+0000 7f2c64ae3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6007a8a0 con 0x7f2c6007be70 2023-09-13T04:31:09.982 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.252+0000 7f2c5e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c6007be70 0x7f2c6007a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:09.983 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:09 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' 2023-09-13T04:31:09.983 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:09 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' 2023-09-13T04:31:09.983 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:09 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' 2023-09-13T04:31:09.987 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.252+0000 7f2c5e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c6007be70 0x7f2c6007a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60682/0 (socket says 172.21.15.2:60682) 2023-09-13T04:31:09.988 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.252+0000 7f2c5e59c700 1 -- 172.21.15.2:0/4137348950 learned_addr learned my addr 172.21.15.2:0/4137348950 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:09.988 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.253+0000 7f2c5e59c700 1 -- 172.21.15.2:0/4137348950 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c6007a9e0 con 0x7f2c6007be70 2023-09-13T04:31:09.988 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.253+0000 7f2c5e59c700 1 --2- 172.21.15.2:0/4137348950 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c6007be70 0x7f2c6007a2d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2c500098a0 tx=0x7f2c50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd03bbb2cc244df0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:09.988 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.253+0000 7f2c5d59a700 1 -- 172.21.15.2:0/4137348950 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c50004160 con 0x7f2c6007be70 2023-09-13T04:31:09.989 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.253+0000 7f2c5d59a700 1 -- 172.21.15.2:0/4137348950 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c500042c0 con 0x7f2c6007be70 2023-09-13T04:31:09.989 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.253+0000 7f2c5d59a700 1 -- 172.21.15.2:0/4137348950 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c500119b0 con 0x7f2c6007be70 2023-09-13T04:31:09.989 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.254+0000 7f2c64ae3700 1 -- 172.21.15.2:0/4137348950 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c6007be70 msgr2=0x7f2c6007a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:09.989 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.254+0000 7f2c64ae3700 1 --2- 172.21.15.2:0/4137348950 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c6007be70 0x7f2c6007a2d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2c500098a0 tx=0x7f2c50009bb0 comp rx=0 tx=0).stop 2023-09-13T04:31:09.989 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.254+0000 7f2c64ae3700 1 -- 172.21.15.2:0/4137348950 shutdown_connections 2023-09-13T04:31:09.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.254+0000 7f2c64ae3700 1 --2- 172.21.15.2:0/4137348950 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c6007be70 0x7f2c6007a2d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:09.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.254+0000 7f2c64ae3700 1 -- 172.21.15.2:0/4137348950 >> 172.21.15.2:0/4137348950 conn(0x7f2c60101b00 msgr2=0x7f2c60103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:09.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.254+0000 7f2c64ae3700 1 -- 172.21.15.2:0/4137348950 shutdown_connections 2023-09-13T04:31:09.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.254+0000 7f2c64ae3700 1 -- 172.21.15.2:0/4137348950 wait complete. 2023-09-13T04:31:09.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.255+0000 7f2c64ae3700 1 Processor -- start 2023-09-13T04:31:09.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.255+0000 7f2c64ae3700 1 -- start start 2023-09-13T04:31:09.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.256+0000 7f2c64ae3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c6007be70 0x7f2c6019c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:09.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.256+0000 7f2c64ae3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6019c960 con 0x7f2c6007be70 2023-09-13T04:31:09.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.256+0000 7f2c5e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c6007be70 0x7f2c6019c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:09.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.256+0000 7f2c5e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c6007be70 0x7f2c6019c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60688/0 (socket says 172.21.15.2:60688) 2023-09-13T04:31:09.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.256+0000 7f2c5e59c700 1 -- 172.21.15.2:0/2905953057 learned_addr learned my addr 172.21.15.2:0/2905953057 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:09.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.256+0000 7f2c5e59c700 1 -- 172.21.15.2:0/2905953057 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c50009550 con 0x7f2c6007be70 2023-09-13T04:31:09.994 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.256+0000 7f2c5e59c700 1 --2- 172.21.15.2:0/2905953057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c6007be70 0x7f2c6019c420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2c50011980 tx=0x7f2c50011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:09.994 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.257+0000 7f2c4f7fe700 1 -- 172.21.15.2:0/2905953057 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c50003a60 con 0x7f2c6007be70 2023-09-13T04:31:09.994 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.257+0000 7f2c4f7fe700 1 -- 172.21.15.2:0/2905953057 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c50003bc0 con 0x7f2c6007be70 2023-09-13T04:31:09.994 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.257+0000 7f2c64ae3700 1 -- 172.21.15.2:0/2905953057 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c6019cb60 con 0x7f2c6007be70 2023-09-13T04:31:09.995 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.257+0000 7f2c4f7fe700 1 -- 172.21.15.2:0/2905953057 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c5001a670 con 0x7f2c6007be70 2023-09-13T04:31:09.995 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.257+0000 7f2c64ae3700 1 -- 172.21.15.2:0/2905953057 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c6019cf80 con 0x7f2c6007be70 2023-09-13T04:31:09.995 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.258+0000 7f2c64ae3700 1 -- 172.21.15.2:0/2905953057 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c600622e0 con 0x7f2c6007be70 2023-09-13T04:31:09.995 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.258+0000 7f2c4f7fe700 1 -- 172.21.15.2:0/2905953057 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45331+0+0 (secure 0 0 0) 0x7f2c50024a00 con 0x7f2c6007be70 2023-09-13T04:31:09.995 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.258+0000 7f2c4f7fe700 1 --2- 172.21.15.2:0/2905953057 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f2c480383b0 0x7f2c4803a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:09.995 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.258+0000 7f2c4f7fe700 1 -- 172.21.15.2:0/2905953057 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2c5004c430 con 0x7f2c6007be70 2023-09-13T04:31:09.996 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.258+0000 7f2c5dd9b700 1 --2- 172.21.15.2:0/2905953057 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f2c480383b0 0x7f2c4803a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:09.996 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.259+0000 7f2c5dd9b700 1 --2- 172.21.15.2:0/2905953057 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f2c480383b0 0x7f2c4803a870 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2c54006fd0 tx=0x7f2c54006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:09.996 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.264+0000 7f2c4f7fe700 1 -- 172.21.15.2:0/2905953057 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2c50024440 con 0x7f2c6007be70 2023-09-13T04:31:09.996 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.504+0000 7f2c64ae3700 1 -- 172.21.15.2:0/2905953057 --> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2c60103de0 con 0x7f2c480383b0 2023-09-13T04:31:09.996 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.509+0000 7f2c4f7fe700 1 -- 172.21.15.2:0/2905953057 <== mgr.14116 v2:172.21.15.2:6800/1393377368 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f2c60103de0 con 0x7f2c480383b0 2023-09-13T04:31:09.997 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.511+0000 7f2c64ae3700 1 -- 172.21.15.2:0/2905953057 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f2c480383b0 msgr2=0x7f2c4803a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:09.997 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.511+0000 7f2c64ae3700 1 --2- 172.21.15.2:0/2905953057 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f2c480383b0 0x7f2c4803a870 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2c54006fd0 tx=0x7f2c54006e40 comp rx=0 tx=0).stop 2023-09-13T04:31:09.997 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.511+0000 7f2c64ae3700 1 -- 172.21.15.2:0/2905953057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c6007be70 msgr2=0x7f2c6019c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:09.997 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.512+0000 7f2c64ae3700 1 --2- 172.21.15.2:0/2905953057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c6007be70 0x7f2c6019c420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2c50011980 tx=0x7f2c50011a60 comp rx=0 tx=0).stop 2023-09-13T04:31:09.997 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.512+0000 7f2c64ae3700 1 -- 172.21.15.2:0/2905953057 shutdown_connections 2023-09-13T04:31:09.998 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.512+0000 7f2c64ae3700 1 --2- 172.21.15.2:0/2905953057 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f2c480383b0 0x7f2c4803a870 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:09.998 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.512+0000 7f2c64ae3700 1 --2- 172.21.15.2:0/2905953057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c6007be70 0x7f2c6019c420 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:09.998 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.512+0000 7f2c64ae3700 1 -- 172.21.15.2:0/2905953057 >> 172.21.15.2:0/2905953057 conn(0x7f2c60101b00 msgr2=0x7f2c601031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:09.998 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.512+0000 7f2c64ae3700 1 -- 172.21.15.2:0/2905953057 shutdown_connections 2023-09-13T04:31:09.998 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:09.512+0000 7f2c64ae3700 1 -- 172.21.15.2:0/2905953057 wait complete. 2023-09-13T04:31:11.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:10 smithi002 ceph-mon[97651]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:31:11.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:10 smithi002 ceph-mon[97651]: Saving service mgr spec with placement count:2 2023-09-13T04:31:11.271 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.566+0000 7f65c7251700 1 Processor -- start 2023-09-13T04:31:11.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.566+0000 7f65c7251700 1 -- start start 2023-09-13T04:31:11.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.566+0000 7f65c7251700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0108730 0x7f65c0108b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:11.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.566+0000 7f65c7251700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c0109120 con 0x7f65c0108730 2023-09-13T04:31:11.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.567+0000 7f65c4fed700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0108730 0x7f65c0108b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:11.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.567+0000 7f65c4fed700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0108730 0x7f65c0108b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60692/0 (socket says 172.21.15.2:60692) 2023-09-13T04:31:11.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.567+0000 7f65c4fed700 1 -- 172.21.15.2:0/193359816 learned_addr learned my addr 172.21.15.2:0/193359816 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:11.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.567+0000 7f65c4fed700 1 -- 172.21.15.2:0/193359816 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c0109930 con 0x7f65c0108730 2023-09-13T04:31:11.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.567+0000 7f65c4fed700 1 --2- 172.21.15.2:0/193359816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0108730 0x7f65c0108b50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f65b40098a0 tx=0x7f65b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d5b8c641565d93 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:11.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.568+0000 7f65bf7fe700 1 -- 172.21.15.2:0/193359816 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65b4004160 con 0x7f65c0108730 2023-09-13T04:31:11.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.568+0000 7f65bf7fe700 1 -- 172.21.15.2:0/193359816 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65b40042c0 con 0x7f65c0108730 2023-09-13T04:31:11.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.568+0000 7f65bf7fe700 1 -- 172.21.15.2:0/193359816 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65b40119b0 con 0x7f65c0108730 2023-09-13T04:31:11.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.568+0000 7f65c7251700 1 -- 172.21.15.2:0/193359816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0108730 msgr2=0x7f65c0108b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:11.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.568+0000 7f65c7251700 1 --2- 172.21.15.2:0/193359816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0108730 0x7f65c0108b50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f65b40098a0 tx=0x7f65b4009bb0 comp rx=0 tx=0).stop 2023-09-13T04:31:11.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.568+0000 7f65c7251700 1 -- 172.21.15.2:0/193359816 shutdown_connections 2023-09-13T04:31:11.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.568+0000 7f65c7251700 1 --2- 172.21.15.2:0/193359816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0108730 0x7f65c0108b50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:11.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.568+0000 7f65c7251700 1 -- 172.21.15.2:0/193359816 >> 172.21.15.2:0/193359816 conn(0x7f65c007bb10 msgr2=0x7f65c01064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:11.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.569+0000 7f65c7251700 1 -- 172.21.15.2:0/193359816 shutdown_connections 2023-09-13T04:31:11.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.569+0000 7f65c7251700 1 -- 172.21.15.2:0/193359816 wait complete. 2023-09-13T04:31:11.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.570+0000 7f65c7251700 1 Processor -- start 2023-09-13T04:31:11.276 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.571+0000 7f65c7251700 1 -- start start 2023-09-13T04:31:11.276 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.571+0000 7f65c7251700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0108730 0x7f65c019c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:11.276 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.571+0000 7f65c7251700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c019c9c0 con 0x7f65c0108730 2023-09-13T04:31:11.276 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.571+0000 7f65c4fed700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0108730 0x7f65c019c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:11.277 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.571+0000 7f65c4fed700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0108730 0x7f65c019c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60700/0 (socket says 172.21.15.2:60700) 2023-09-13T04:31:11.277 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.571+0000 7f65c4fed700 1 -- 172.21.15.2:0/3343721589 learned_addr learned my addr 172.21.15.2:0/3343721589 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:11.277 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.571+0000 7f65c4fed700 1 -- 172.21.15.2:0/3343721589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b4009550 con 0x7f65c0108730 2023-09-13T04:31:11.277 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.572+0000 7f65c4fed700 1 --2- 172.21.15.2:0/3343721589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0108730 0x7f65c019c480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f65b4000c00 tx=0x7f65b4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:11.277 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.572+0000 7f65bdffb700 1 -- 172.21.15.2:0/3343721589 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65b40039f0 con 0x7f65c0108730 2023-09-13T04:31:11.278 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.572+0000 7f65bdffb700 1 -- 172.21.15.2:0/3343721589 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65b4003b50 con 0x7f65c0108730 2023-09-13T04:31:11.278 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.572+0000 7f65c7251700 1 -- 172.21.15.2:0/3343721589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c019cbc0 con 0x7f65c0108730 2023-09-13T04:31:11.278 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.572+0000 7f65bdffb700 1 -- 172.21.15.2:0/3343721589 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65b401a780 con 0x7f65c0108730 2023-09-13T04:31:11.278 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.572+0000 7f65c7251700 1 -- 172.21.15.2:0/3343721589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c019cfe0 con 0x7f65c0108730 2023-09-13T04:31:11.278 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.574+0000 7f65bdffb700 1 -- 172.21.15.2:0/3343721589 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45331+0+0 (secure 0 0 0) 0x7f65b4024440 con 0x7f65c0108730 2023-09-13T04:31:11.279 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.574+0000 7f65c7251700 1 -- 172.21.15.2:0/3343721589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c00622e0 con 0x7f65c0108730 2023-09-13T04:31:11.279 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.574+0000 7f65bdffb700 1 --2- 172.21.15.2:0/3343721589 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f65b0038360 0x7f65b003a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:11.279 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.574+0000 7f65bdffb700 1 -- 172.21.15.2:0/3343721589 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f65b404c410 con 0x7f65c0108730 2023-09-13T04:31:11.279 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.575+0000 7f65bffff700 1 --2- 172.21.15.2:0/3343721589 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f65b0038360 0x7f65b003a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:11.279 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.575+0000 7f65bffff700 1 --2- 172.21.15.2:0/3343721589 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f65b0038360 0x7f65b003a820 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f65ac006fd0 tx=0x7f65ac006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:11.280 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.580+0000 7f65bdffb700 1 -- 172.21.15.2:0/3343721589 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f65b40246f0 con 0x7f65c0108730 2023-09-13T04:31:11.280 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.793+0000 7f65c7251700 1 -- 172.21.15.2:0/3343721589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f65c010cb20 con 0x7f65c0108730 2023-09-13T04:31:11.280 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.797+0000 7f65bdffb700 1 -- 172.21.15.2:0/3343721589 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f65b4018350 con 0x7f65c0108730 2023-09-13T04:31:11.280 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.800+0000 7f65c7251700 1 -- 172.21.15.2:0/3343721589 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f65b0038360 msgr2=0x7f65b003a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:11.280 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.800+0000 7f65c7251700 1 --2- 172.21.15.2:0/3343721589 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f65b0038360 0x7f65b003a820 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f65ac006fd0 tx=0x7f65ac006e40 comp rx=0 tx=0).stop 2023-09-13T04:31:11.281 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.800+0000 7f65c7251700 1 -- 172.21.15.2:0/3343721589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0108730 msgr2=0x7f65c019c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:11.281 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.800+0000 7f65c7251700 1 --2- 172.21.15.2:0/3343721589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0108730 0x7f65c019c480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f65b4000c00 tx=0x7f65b4011a30 comp rx=0 tx=0).stop 2023-09-13T04:31:11.281 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.800+0000 7f65c7251700 1 -- 172.21.15.2:0/3343721589 shutdown_connections 2023-09-13T04:31:11.281 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.800+0000 7f65c7251700 1 --2- 172.21.15.2:0/3343721589 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f65b0038360 0x7f65b003a820 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:11.282 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.800+0000 7f65c7251700 1 --2- 172.21.15.2:0/3343721589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0108730 0x7f65c019c480 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:11.282 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.800+0000 7f65c7251700 1 -- 172.21.15.2:0/3343721589 >> 172.21.15.2:0/3343721589 conn(0x7f65c007bb10 msgr2=0x7f65c01067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:11.282 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.800+0000 7f65c7251700 1 -- 172.21.15.2:0/3343721589 shutdown_connections 2023-09-13T04:31:11.282 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:10.800+0000 7f65c7251700 1 -- 172.21.15.2:0/3343721589 wait complete. 2023-09-13T04:31:12.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:11 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3343721589' entity='client.admin' 2023-09-13T04:31:12.755 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.086+0000 7f509ea87700 1 Processor -- start 2023-09-13T04:31:12.756 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.086+0000 7f509ea87700 1 -- start start 2023-09-13T04:31:12.756 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.087+0000 7f509ea87700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5098108730 0x7f5098108b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:12.756 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.087+0000 7f509ea87700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5098109120 con 0x7f5098108730 2023-09-13T04:31:12.756 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.087+0000 7f509c823700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5098108730 0x7f5098108b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:12.756 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.087+0000 7f509c823700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5098108730 0x7f5098108b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60710/0 (socket says 172.21.15.2:60710) 2023-09-13T04:31:12.757 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.087+0000 7f509c823700 1 -- 172.21.15.2:0/1135903775 learned_addr learned my addr 172.21.15.2:0/1135903775 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:12.757 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.088+0000 7f509c823700 1 -- 172.21.15.2:0/1135903775 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5098109930 con 0x7f5098108730 2023-09-13T04:31:12.757 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.088+0000 7f509c823700 1 --2- 172.21.15.2:0/1135903775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5098108730 0x7f5098108b50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f508c0098a0 tx=0x7f508c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b5dda535002a87 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:12.757 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.088+0000 7f50977fe700 1 -- 172.21.15.2:0/1135903775 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f508c004160 con 0x7f5098108730 2023-09-13T04:31:12.757 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.088+0000 7f50977fe700 1 -- 172.21.15.2:0/1135903775 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f508c0042c0 con 0x7f5098108730 2023-09-13T04:31:12.758 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.088+0000 7f50977fe700 1 -- 172.21.15.2:0/1135903775 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f508c0119b0 con 0x7f5098108730 2023-09-13T04:31:12.758 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.089+0000 7f509ea87700 1 -- 172.21.15.2:0/1135903775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5098108730 msgr2=0x7f5098108b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:12.758 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.089+0000 7f509ea87700 1 --2- 172.21.15.2:0/1135903775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5098108730 0x7f5098108b50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f508c0098a0 tx=0x7f508c009bb0 comp rx=0 tx=0).stop 2023-09-13T04:31:12.758 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.089+0000 7f509ea87700 1 -- 172.21.15.2:0/1135903775 shutdown_connections 2023-09-13T04:31:12.758 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.089+0000 7f509ea87700 1 --2- 172.21.15.2:0/1135903775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5098108730 0x7f5098108b50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:12.759 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.089+0000 7f509ea87700 1 -- 172.21.15.2:0/1135903775 >> 172.21.15.2:0/1135903775 conn(0x7f509807bb10 msgr2=0x7f50981064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:12.759 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.089+0000 7f509ea87700 1 -- 172.21.15.2:0/1135903775 shutdown_connections 2023-09-13T04:31:12.759 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.089+0000 7f509ea87700 1 -- 172.21.15.2:0/1135903775 wait complete. 2023-09-13T04:31:12.759 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.090+0000 7f509ea87700 1 Processor -- start 2023-09-13T04:31:12.759 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.090+0000 7f509ea87700 1 -- start start 2023-09-13T04:31:12.760 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.091+0000 7f509ea87700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5098108730 0x7f509819c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:12.760 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.091+0000 7f509ea87700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509819c9a0 con 0x7f5098108730 2023-09-13T04:31:12.760 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.091+0000 7f509c823700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5098108730 0x7f509819c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:12.760 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.091+0000 7f509c823700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5098108730 0x7f509819c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60722/0 (socket says 172.21.15.2:60722) 2023-09-13T04:31:12.760 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.091+0000 7f509c823700 1 -- 172.21.15.2:0/2867698784 learned_addr learned my addr 172.21.15.2:0/2867698784 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:12.761 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.091+0000 7f509c823700 1 -- 172.21.15.2:0/2867698784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c009550 con 0x7f5098108730 2023-09-13T04:31:12.761 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.092+0000 7f509c823700 1 --2- 172.21.15.2:0/2867698784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5098108730 0x7f509819c460 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f508c003e60 tx=0x7f508c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:12.762 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.092+0000 7f5095ffb700 1 -- 172.21.15.2:0/2867698784 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f508c003970 con 0x7f5098108730 2023-09-13T04:31:12.762 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.092+0000 7f509ea87700 1 -- 172.21.15.2:0/2867698784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f509819cba0 con 0x7f5098108730 2023-09-13T04:31:12.762 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.092+0000 7f5095ffb700 1 -- 172.21.15.2:0/2867698784 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f508c003ad0 con 0x7f5098108730 2023-09-13T04:31:12.762 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.092+0000 7f5095ffb700 1 -- 172.21.15.2:0/2867698784 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f508c0195e0 con 0x7f5098108730 2023-09-13T04:31:12.762 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.092+0000 7f509ea87700 1 -- 172.21.15.2:0/2867698784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f509819cfc0 con 0x7f5098108730 2023-09-13T04:31:12.763 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.093+0000 7f5095ffb700 1 -- 172.21.15.2:0/2867698784 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45331+0+0 (secure 0 0 0) 0x7f508c019850 con 0x7f5098108730 2023-09-13T04:31:12.763 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.093+0000 7f509ea87700 1 -- 172.21.15.2:0/2867698784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50981965d0 con 0x7f5098108730 2023-09-13T04:31:12.763 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.094+0000 7f5095ffb700 1 --2- 172.21.15.2:0/2867698784 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f5088038360 0x7f508803a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:12.763 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.094+0000 7f5095ffb700 1 -- 172.21.15.2:0/2867698784 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f508c04c530 con 0x7f5098108730 2023-09-13T04:31:12.763 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.094+0000 7f5097fff700 1 --2- 172.21.15.2:0/2867698784 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f5088038360 0x7f508803a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:12.764 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.095+0000 7f5097fff700 1 --2- 172.21.15.2:0/2867698784 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f5088038360 0x7f508803a820 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5084006fd0 tx=0x7f5084006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:12.764 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.099+0000 7f5095ffb700 1 -- 172.21.15.2:0/2867698784 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f508c018530 con 0x7f5098108730 2023-09-13T04:31:12.764 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.313+0000 7f509ea87700 1 -- 172.21.15.2:0/2867698784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f50980622e0 con 0x7f5098108730 2023-09-13T04:31:12.764 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.317+0000 7f5095ffb700 1 -- 172.21.15.2:0/2867698784 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f508c018350 con 0x7f5098108730 2023-09-13T04:31:12.764 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.319+0000 7f509ea87700 1 -- 172.21.15.2:0/2867698784 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f5088038360 msgr2=0x7f508803a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:12.765 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.319+0000 7f509ea87700 1 --2- 172.21.15.2:0/2867698784 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f5088038360 0x7f508803a820 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5084006fd0 tx=0x7f5084006e40 comp rx=0 tx=0).stop 2023-09-13T04:31:12.765 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.319+0000 7f509ea87700 1 -- 172.21.15.2:0/2867698784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5098108730 msgr2=0x7f509819c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:12.765 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.319+0000 7f509ea87700 1 --2- 172.21.15.2:0/2867698784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5098108730 0x7f509819c460 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f508c003e60 tx=0x7f508c011aa0 comp rx=0 tx=0).stop 2023-09-13T04:31:12.765 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.319+0000 7f509ea87700 1 -- 172.21.15.2:0/2867698784 shutdown_connections 2023-09-13T04:31:12.765 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.319+0000 7f509ea87700 1 --2- 172.21.15.2:0/2867698784 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f5088038360 0x7f508803a820 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:12.766 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.319+0000 7f509ea87700 1 --2- 172.21.15.2:0/2867698784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5098108730 0x7f509819c460 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:12.766 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.319+0000 7f509ea87700 1 -- 172.21.15.2:0/2867698784 >> 172.21.15.2:0/2867698784 conn(0x7f509807bb10 msgr2=0x7f5098106840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:12.766 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.320+0000 7f509ea87700 1 -- 172.21.15.2:0/2867698784 shutdown_connections 2023-09-13T04:31:12.766 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:12.320+0000 7f509ea87700 1 -- 172.21.15.2:0/2867698784 wait complete. 2023-09-13T04:31:12.766 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling the dashboard module... 2023-09-13T04:31:13.072 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:12 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' 2023-09-13T04:31:13.073 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:12 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2867698784' entity='client.admin' 2023-09-13T04:31:14.751 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:14 smithi002 ceph-mon[97651]: from='mgr.14116 172.21.15.2:0/1003693067' entity='mgr.x' 2023-09-13T04:31:14.751 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:14 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/63869362' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-09-13T04:31:14.752 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:14 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: ignoring --setuser ceph since I am not root 2023-09-13T04:31:14.752 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:14 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: ignoring --setgroup ceph since I am not root 2023-09-13T04:31:15.005 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:14 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:14.797+0000 7f9fc18b1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-13T04:31:15.006 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:14 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:14.906+0000 7f9fc18b1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-13T04:31:15.155 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.357+0000 7f4c08835700 1 Processor -- start 2023-09-13T04:31:15.155 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.357+0000 7f4c08835700 1 -- start start 2023-09-13T04:31:15.155 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.358+0000 7f4c08835700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c0407bed0 0x7f4c0407a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:15.156 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.358+0000 7f4c08835700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0407a900 con 0x7f4c0407bed0 2023-09-13T04:31:15.156 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.358+0000 7f4c0259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c0407bed0 0x7f4c0407a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:15.156 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.358+0000 7f4c0259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c0407bed0 0x7f4c0407a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60730/0 (socket says 172.21.15.2:60730) 2023-09-13T04:31:15.156 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.358+0000 7f4c0259c700 1 -- 172.21.15.2:0/112419065 learned_addr learned my addr 172.21.15.2:0/112419065 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:15.156 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.358+0000 7f4c0259c700 1 -- 172.21.15.2:0/112419065 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0407aa40 con 0x7f4c0407bed0 2023-09-13T04:31:15.157 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.359+0000 7f4c0259c700 1 --2- 172.21.15.2:0/112419065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c0407bed0 0x7f4c0407a330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4bf40098a0 tx=0x7f4bf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e482da0b836aa65b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:15.157 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.359+0000 7f4c0159a700 1 -- 172.21.15.2:0/112419065 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bf4004160 con 0x7f4c0407bed0 2023-09-13T04:31:15.157 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.359+0000 7f4c0159a700 1 -- 172.21.15.2:0/112419065 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bf40042c0 con 0x7f4c0407bed0 2023-09-13T04:31:15.157 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.359+0000 7f4c0159a700 1 -- 172.21.15.2:0/112419065 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bf40119b0 con 0x7f4c0407bed0 2023-09-13T04:31:15.157 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.359+0000 7f4c08835700 1 -- 172.21.15.2:0/112419065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c0407bed0 msgr2=0x7f4c0407a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:15.158 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.359+0000 7f4c08835700 1 --2- 172.21.15.2:0/112419065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c0407bed0 0x7f4c0407a330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4bf40098a0 tx=0x7f4bf4009bb0 comp rx=0 tx=0).stop 2023-09-13T04:31:15.158 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.359+0000 7f4c08835700 1 -- 172.21.15.2:0/112419065 shutdown_connections 2023-09-13T04:31:15.158 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.359+0000 7f4c08835700 1 --2- 172.21.15.2:0/112419065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c0407bed0 0x7f4c0407a330 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:15.158 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.359+0000 7f4c08835700 1 -- 172.21.15.2:0/112419065 >> 172.21.15.2:0/112419065 conn(0x7f4c04101b00 msgr2=0x7f4c04103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:15.158 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.360+0000 7f4c08835700 1 -- 172.21.15.2:0/112419065 shutdown_connections 2023-09-13T04:31:15.159 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.360+0000 7f4c08835700 1 -- 172.21.15.2:0/112419065 wait complete. 2023-09-13T04:31:15.159 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.361+0000 7f4c08835700 1 Processor -- start 2023-09-13T04:31:15.159 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.361+0000 7f4c08835700 1 -- start start 2023-09-13T04:31:15.160 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.362+0000 7f4c08835700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c0407bed0 0x7f4c0419c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:15.160 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.362+0000 7f4c08835700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0419c960 con 0x7f4c0407bed0 2023-09-13T04:31:15.160 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.362+0000 7f4c0259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c0407bed0 0x7f4c0419c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:15.160 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.362+0000 7f4c0259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c0407bed0 0x7f4c0419c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60744/0 (socket says 172.21.15.2:60744) 2023-09-13T04:31:15.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.362+0000 7f4c0259c700 1 -- 172.21.15.2:0/63869362 learned_addr learned my addr 172.21.15.2:0/63869362 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:15.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.362+0000 7f4c0259c700 1 -- 172.21.15.2:0/63869362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf4009550 con 0x7f4c0407bed0 2023-09-13T04:31:15.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.363+0000 7f4c0259c700 1 --2- 172.21.15.2:0/63869362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c0407bed0 0x7f4c0419c420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4000c00 tx=0x7f4bf4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:15.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.363+0000 7f4bf37fe700 1 -- 172.21.15.2:0/63869362 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bf40039f0 con 0x7f4c0407bed0 2023-09-13T04:31:15.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.363+0000 7f4bf37fe700 1 -- 172.21.15.2:0/63869362 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bf4003b50 con 0x7f4c0407bed0 2023-09-13T04:31:15.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.363+0000 7f4c08835700 1 -- 172.21.15.2:0/63869362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0419cb60 con 0x7f4c0407bed0 2023-09-13T04:31:15.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.363+0000 7f4bf37fe700 1 -- 172.21.15.2:0/63869362 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bf401a780 con 0x7f4c0407bed0 2023-09-13T04:31:15.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.363+0000 7f4c08835700 1 -- 172.21.15.2:0/63869362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0419cf80 con 0x7f4c0407bed0 2023-09-13T04:31:15.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.364+0000 7f4c08835700 1 -- 172.21.15.2:0/63869362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c040622e0 con 0x7f4c0407bed0 2023-09-13T04:31:15.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.364+0000 7f4bf37fe700 1 -- 172.21.15.2:0/63869362 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45331+0+0 (secure 0 0 0) 0x7f4bf4024440 con 0x7f4c0407bed0 2023-09-13T04:31:15.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.365+0000 7f4bf37fe700 1 --2- 172.21.15.2:0/63869362 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f4bec038360 0x7f4bec03a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:15.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.365+0000 7f4bf37fe700 1 -- 172.21.15.2:0/63869362 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4bf404c410 con 0x7f4c0407bed0 2023-09-13T04:31:15.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.365+0000 7f4c01d9b700 1 --2- 172.21.15.2:0/63869362 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f4bec038360 0x7f4bec03a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:15.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.366+0000 7f4c01d9b700 1 --2- 172.21.15.2:0/63869362 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f4bec038360 0x7f4bec03a820 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8006fd0 tx=0x7f4bf8006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:15.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.370+0000 7f4bf37fe700 1 -- 172.21.15.2:0/63869362 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4bf40246f0 con 0x7f4c0407bed0 2023-09-13T04:31:15.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:13.657+0000 7f4c08835700 1 -- 172.21.15.2:0/63869362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f4c0410c920 con 0x7f4c0407bed0 2023-09-13T04:31:15.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:14.488+0000 7f4bf37fe700 1 -- 172.21.15.2:0/63869362 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 9) v1 ==== 45344+0+0 (secure 0 0 0) 0x7f4bf402d910 con 0x7f4c0407bed0 2023-09-13T04:31:15.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:14.490+0000 7f4bf37fe700 1 -- 172.21.15.2:0/63869362 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f4bf4010590 con 0x7f4c0407bed0 2023-09-13T04:31:15.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:14.493+0000 7f4c08835700 1 -- 172.21.15.2:0/63869362 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f4bec038360 msgr2=0x7f4bec03a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:15.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:14.493+0000 7f4c08835700 1 --2- 172.21.15.2:0/63869362 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f4bec038360 0x7f4bec03a820 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8006fd0 tx=0x7f4bf8006e40 comp rx=0 tx=0).stop 2023-09-13T04:31:15.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:14.493+0000 7f4c08835700 1 -- 172.21.15.2:0/63869362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c0407bed0 msgr2=0x7f4c0419c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:15.166 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:14.493+0000 7f4c08835700 1 --2- 172.21.15.2:0/63869362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c0407bed0 0x7f4c0419c420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4000c00 tx=0x7f4bf4011a30 comp rx=0 tx=0).stop 2023-09-13T04:31:15.166 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:14.493+0000 7f4c08835700 1 -- 172.21.15.2:0/63869362 shutdown_connections 2023-09-13T04:31:15.166 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:14.493+0000 7f4c08835700 1 --2- 172.21.15.2:0/63869362 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f4bec038360 0x7f4bec03a820 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:15.166 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:14.493+0000 7f4c08835700 1 --2- 172.21.15.2:0/63869362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c0407bed0 0x7f4c0419c420 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:15.166 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:14.493+0000 7f4c08835700 1 -- 172.21.15.2:0/63869362 >> 172.21.15.2:0/63869362 conn(0x7f4c04101b00 msgr2=0x7f4c041031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:15.167 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:14.493+0000 7f4c08835700 1 -- 172.21.15.2:0/63869362 shutdown_connections 2023-09-13T04:31:15.167 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:14.493+0000 7f4c08835700 1 -- 172.21.15.2:0/63869362 wait complete. 2023-09-13T04:31:15.411 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:15 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:15.025+0000 7f9fc18b1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-13T04:31:15.412 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:15 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:15.166+0000 7f9fc18b1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-13T04:31:15.412 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:15 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:15.411+0000 7f9fc18b1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-13T04:31:15.681 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:15 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/63869362' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-09-13T04:31:15.681 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:15 smithi002 ceph-mon[97651]: mgrmap e9: x(active, since 19s) 2023-09-13T04:31:16.005 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:15 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:15.915+0000 7f9fc18b1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-13T04:31:16.347 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:16 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:16.086+0000 7f9fc18b1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-13T04:31:16.347 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:16 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:16.206+0000 7f9fc18b1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-13T04:31:16.645 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2023-09-13T04:31:16.645 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-09-13T04:31:16.645 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-13T04:31:16.645 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-09-13T04:31:16.646 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-09-13T04:31:16.646 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2023-09-13T04:31:16.646 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.960+0000 7fc579cbb700 1 Processor -- start 2023-09-13T04:31:16.646 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.960+0000 7fc579cbb700 1 -- start start 2023-09-13T04:31:16.647 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.961+0000 7fc579cbb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc57407be70 0x7fc57407a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:16.647 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.961+0000 7fc579cbb700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57407a8a0 con 0x7fc57407be70 2023-09-13T04:31:16.647 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.961+0000 7fc5737fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc57407be70 0x7fc57407a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:16.647 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.961+0000 7fc5737fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc57407be70 0x7fc57407a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41938/0 (socket says 172.21.15.2:41938) 2023-09-13T04:31:16.647 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.961+0000 7fc5737fe700 1 -- 172.21.15.2:0/4127002236 learned_addr learned my addr 172.21.15.2:0/4127002236 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:16.648 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.961+0000 7fc5737fe700 1 -- 172.21.15.2:0/4127002236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc57407a9e0 con 0x7fc57407be70 2023-09-13T04:31:16.648 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.962+0000 7fc5737fe700 1 --2- 172.21.15.2:0/4127002236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc57407be70 0x7fc57407a2d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc5640098a0 tx=0x7fc564009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8deadcf6385c2b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:16.648 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.962+0000 7fc5727fc700 1 -- 172.21.15.2:0/4127002236 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc564004160 con 0x7fc57407be70 2023-09-13T04:31:16.648 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.962+0000 7fc5727fc700 1 -- 172.21.15.2:0/4127002236 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5640042c0 con 0x7fc57407be70 2023-09-13T04:31:16.648 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.962+0000 7fc5727fc700 1 -- 172.21.15.2:0/4127002236 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5640119b0 con 0x7fc57407be70 2023-09-13T04:31:16.649 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.963+0000 7fc579cbb700 1 -- 172.21.15.2:0/4127002236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc57407be70 msgr2=0x7fc57407a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:16.649 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.963+0000 7fc579cbb700 1 --2- 172.21.15.2:0/4127002236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc57407be70 0x7fc57407a2d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc5640098a0 tx=0x7fc564009bb0 comp rx=0 tx=0).stop 2023-09-13T04:31:16.649 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.963+0000 7fc579cbb700 1 -- 172.21.15.2:0/4127002236 shutdown_connections 2023-09-13T04:31:16.649 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.963+0000 7fc579cbb700 1 --2- 172.21.15.2:0/4127002236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc57407be70 0x7fc57407a2d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:16.650 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.963+0000 7fc579cbb700 1 -- 172.21.15.2:0/4127002236 >> 172.21.15.2:0/4127002236 conn(0x7fc574101b00 msgr2=0x7fc574103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:16.650 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.963+0000 7fc579cbb700 1 -- 172.21.15.2:0/4127002236 shutdown_connections 2023-09-13T04:31:16.650 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.963+0000 7fc579cbb700 1 -- 172.21.15.2:0/4127002236 wait complete. 2023-09-13T04:31:16.650 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.964+0000 7fc579cbb700 1 Processor -- start 2023-09-13T04:31:16.650 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.964+0000 7fc579cbb700 1 -- start start 2023-09-13T04:31:16.651 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.964+0000 7fc579cbb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc57407be70 0x7fc57419c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:16.651 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.964+0000 7fc579cbb700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57419c960 con 0x7fc57407be70 2023-09-13T04:31:16.651 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.965+0000 7fc5737fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc57407be70 0x7fc57419c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:16.651 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.965+0000 7fc5737fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc57407be70 0x7fc57419c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41940/0 (socket says 172.21.15.2:41940) 2023-09-13T04:31:16.651 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.965+0000 7fc5737fe700 1 -- 172.21.15.2:0/2025432063 learned_addr learned my addr 172.21.15.2:0/2025432063 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:16.652 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.965+0000 7fc5737fe700 1 -- 172.21.15.2:0/2025432063 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc564009550 con 0x7fc57407be70 2023-09-13T04:31:16.652 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.965+0000 7fc5737fe700 1 --2- 172.21.15.2:0/2025432063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc57407be70 0x7fc57419c420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc564011980 tx=0x7fc564011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:16.652 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.965+0000 7fc570ff9700 1 -- 172.21.15.2:0/2025432063 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc564003a60 con 0x7fc57407be70 2023-09-13T04:31:16.652 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.966+0000 7fc570ff9700 1 -- 172.21.15.2:0/2025432063 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc564003bc0 con 0x7fc57407be70 2023-09-13T04:31:16.652 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.966+0000 7fc570ff9700 1 -- 172.21.15.2:0/2025432063 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc56401a670 con 0x7fc57407be70 2023-09-13T04:31:16.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.966+0000 7fc579cbb700 1 -- 172.21.15.2:0/2025432063 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc57419cb60 con 0x7fc57407be70 2023-09-13T04:31:16.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.966+0000 7fc579cbb700 1 -- 172.21.15.2:0/2025432063 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc57419cf80 con 0x7fc57407be70 2023-09-13T04:31:16.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.967+0000 7fc570ff9700 1 -- 172.21.15.2:0/2025432063 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 9) v1 ==== 45344+0+0 (secure 0 0 0) 0x7fc564024a00 con 0x7fc57407be70 2023-09-13T04:31:16.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.967+0000 7fc55a7fc700 1 -- 172.21.15.2:0/2025432063 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc57410c920 con 0x7fc57407be70 2023-09-13T04:31:16.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.967+0000 7fc570ff9700 1 --2- 172.21.15.2:0/2025432063 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7fc55c038400 0x7fc55c03a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:16.654 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.967+0000 7fc570ff9700 1 -- 172.21.15.2:0/2025432063 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc56404c420 con 0x7fc57407be70 2023-09-13T04:31:16.654 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.967+0000 7fc572ffd700 1 -- 172.21.15.2:0/2025432063 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7fc55c038400 msgr2=0x7fc55c03a8c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1393377368 2023-09-13T04:31:16.654 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.967+0000 7fc572ffd700 1 --2- 172.21.15.2:0/2025432063 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7fc55c038400 0x7fc55c03a8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-13T04:31:16.654 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:15.973+0000 7fc570ff9700 1 -- 172.21.15.2:0/2025432063 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc564024440 con 0x7fc57407be70 2023-09-13T04:31:16.654 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:16.168+0000 7fc572ffd700 1 -- 172.21.15.2:0/2025432063 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7fc55c038400 msgr2=0x7fc55c03a8c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1393377368 2023-09-13T04:31:16.655 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:16.168+0000 7fc572ffd700 1 --2- 172.21.15.2:0/2025432063 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7fc55c038400 0x7fc55c03a8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-13T04:31:16.655 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:16.243+0000 7fc55a7fc700 1 -- 172.21.15.2:0/2025432063 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fc57410cb60 con 0x7fc57407be70 2023-09-13T04:31:16.655 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:16.243+0000 7fc570ff9700 1 -- 172.21.15.2:0/2025432063 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fc564018350 con 0x7fc57407be70 2023-09-13T04:31:16.655 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:16.245+0000 7fc55a7fc700 1 -- 172.21.15.2:0/2025432063 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7fc55c038400 msgr2=0x7fc55c03a8c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-13T04:31:16.656 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:16.245+0000 7fc55a7fc700 1 --2- 172.21.15.2:0/2025432063 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7fc55c038400 0x7fc55c03a8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:16.656 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:16.245+0000 7fc55a7fc700 1 -- 172.21.15.2:0/2025432063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc57407be70 msgr2=0x7fc57419c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:16.656 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:16.245+0000 7fc55a7fc700 1 --2- 172.21.15.2:0/2025432063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc57407be70 0x7fc57419c420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc564011980 tx=0x7fc564011a60 comp rx=0 tx=0).stop 2023-09-13T04:31:16.656 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:16.246+0000 7fc55a7fc700 1 -- 172.21.15.2:0/2025432063 shutdown_connections 2023-09-13T04:31:16.656 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:16.246+0000 7fc55a7fc700 1 --2- 172.21.15.2:0/2025432063 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7fc55c038400 0x7fc55c03a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:16.657 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:16.246+0000 7fc55a7fc700 1 --2- 172.21.15.2:0/2025432063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc57407be70 0x7fc57419c420 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:16.657 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:16.246+0000 7fc55a7fc700 1 -- 172.21.15.2:0/2025432063 >> 172.21.15.2:0/2025432063 conn(0x7fc574101b00 msgr2=0x7fc5741031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:16.657 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:16.246+0000 7fc55a7fc700 1 -- 172.21.15.2:0/2025432063 shutdown_connections 2023-09-13T04:31:16.657 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:16.246+0000 7fc55a7fc700 1 -- 172.21.15.2:0/2025432063 wait complete. 2023-09-13T04:31:16.657 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for the mgr to restart... 2023-09-13T04:31:16.657 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr epoch 9... 2023-09-13T04:31:16.683 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:16 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2025432063' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-09-13T04:31:16.683 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:16 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:16.346+0000 7f9fc18b1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-13T04:31:16.943 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:16 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:16.913+0000 7f9fc18b1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-13T04:31:18.255 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:17 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:17.804+0000 7f9fc18b1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-13T04:31:18.256 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:17 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:17.914+0000 7f9fc18b1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-13T04:31:19.255 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:18 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:18.900+0000 7f9fc18b1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-13T04:31:19.256 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:19 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:19.023+0000 7f9fc18b1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-13T04:31:19.633 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:19 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:19.255+0000 7f9fc18b1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-13T04:31:19.915 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:19 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:19.632+0000 7f9fc18b1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-13T04:31:20.255 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:19 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:19.915+0000 7f9fc18b1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-13T04:31:20.306 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:20 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:20.115+0000 7f9fc18b1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-13T04:31:20.755 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:20 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:20.470+0000 7f9fc18b1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-13T04:31:21.755 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:21 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:21.435+0000 7f9fc18b1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-13T04:31:21.755 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:21 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:21.617+0000 7f9fc18b1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-13T04:31:22.255 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:21 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:21.874+0000 7f9fc18b1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-13T04:31:22.256 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:22 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:22.026+0000 7f9fc18b1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-13T04:31:22.755 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:22 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:22.311+0000 7f9fc18b1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-13T04:31:23.755 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:31:23 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:31:23.415+0000 7f9fc18b1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-13T04:31:23.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:23 smithi002 ceph-mon[97651]: Active manager daemon x restarted 2023-09-13T04:31:23.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:23 smithi002 ceph-mon[97651]: Activating manager daemon x 2023-09-13T04:31:23.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:23 smithi002 ceph-mon[97651]: osdmap e3: 0 total, 0 up, 0 in 2023-09-13T04:31:23.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:23 smithi002 ceph-mon[97651]: mgrmap e10: x(active, starting, since 0.00399721s) 2023-09-13T04:31:23.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T04:31:23.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-13T04:31:23.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-13T04:31:23.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-13T04:31:23.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-13T04:31:23.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:23 smithi002 ceph-mon[97651]: Manager daemon x is now available 2023-09-13T04:31:23.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:31:23.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:31:24.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-13T04:31:24.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-13T04:31:24.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:24 smithi002 ceph-mon[97651]: mgrmap e11: x(active, since 1.0088s) 2023-09-13T04:31:24.849 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2023-09-13T04:31:24.849 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-09-13T04:31:24.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "initialized": true 2023-09-13T04:31:24.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2023-09-13T04:31:24.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.186+0000 7f2d5543e700 1 Processor -- start 2023-09-13T04:31:24.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.186+0000 7f2d5543e700 1 -- start start 2023-09-13T04:31:24.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.187+0000 7f2d5543e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d50106520 0x7f2d50106940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:24.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.187+0000 7f2d5543e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d50106f10 con 0x7f2d50106520 2023-09-13T04:31:24.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.187+0000 7f2d4effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d50106520 0x7f2d50106940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:24.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.187+0000 7f2d4effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d50106520 0x7f2d50106940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41944/0 (socket says 172.21.15.2:41944) 2023-09-13T04:31:24.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.187+0000 7f2d4effd700 1 -- 172.21.15.2:0/1650614271 learned_addr learned my addr 172.21.15.2:0/1650614271 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:24.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.187+0000 7f2d4effd700 1 -- 172.21.15.2:0/1650614271 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d50107720 con 0x7f2d50106520 2023-09-13T04:31:24.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.188+0000 7f2d4effd700 1 --2- 172.21.15.2:0/1650614271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d50106520 0x7f2d50106940 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2d400098a0 tx=0x7f2d40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5baca75bd98edfe server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:24.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.188+0000 7f2d4dffb700 1 -- 172.21.15.2:0/1650614271 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d40004160 con 0x7f2d50106520 2023-09-13T04:31:24.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.188+0000 7f2d4dffb700 1 -- 172.21.15.2:0/1650614271 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d400042c0 con 0x7f2d50106520 2023-09-13T04:31:24.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.188+0000 7f2d4dffb700 1 -- 172.21.15.2:0/1650614271 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d400119b0 con 0x7f2d50106520 2023-09-13T04:31:24.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.188+0000 7f2d5543e700 1 -- 172.21.15.2:0/1650614271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d50106520 msgr2=0x7f2d50106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:24.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.188+0000 7f2d5543e700 1 --2- 172.21.15.2:0/1650614271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d50106520 0x7f2d50106940 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2d400098a0 tx=0x7f2d40009bb0 comp rx=0 tx=0).stop 2023-09-13T04:31:24.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.189+0000 7f2d5543e700 1 -- 172.21.15.2:0/1650614271 shutdown_connections 2023-09-13T04:31:24.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.189+0000 7f2d5543e700 1 --2- 172.21.15.2:0/1650614271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d50106520 0x7f2d50106940 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:24.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.189+0000 7f2d5543e700 1 -- 172.21.15.2:0/1650614271 >> 172.21.15.2:0/1650614271 conn(0x7f2d50101b00 msgr2=0x7f2d50103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:24.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.189+0000 7f2d5543e700 1 -- 172.21.15.2:0/1650614271 shutdown_connections 2023-09-13T04:31:24.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.189+0000 7f2d5543e700 1 -- 172.21.15.2:0/1650614271 wait complete. 2023-09-13T04:31:24.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.190+0000 7f2d5543e700 1 Processor -- start 2023-09-13T04:31:24.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.190+0000 7f2d5543e700 1 -- start start 2023-09-13T04:31:24.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.190+0000 7f2d5543e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d50106520 0x7f2d5019c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:24.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.190+0000 7f2d5543e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5019c990 con 0x7f2d50106520 2023-09-13T04:31:24.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.190+0000 7f2d4effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d50106520 0x7f2d5019c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:24.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.190+0000 7f2d4effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d50106520 0x7f2d5019c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41954/0 (socket says 172.21.15.2:41954) 2023-09-13T04:31:24.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.190+0000 7f2d4effd700 1 -- 172.21.15.2:0/1079371819 learned_addr learned my addr 172.21.15.2:0/1079371819 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:24.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.191+0000 7f2d4effd700 1 -- 172.21.15.2:0/1079371819 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d40009550 con 0x7f2d50106520 2023-09-13T04:31:24.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.191+0000 7f2d4effd700 1 --2- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d50106520 0x7f2d5019c450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2d40000c00 tx=0x7f2d40011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:24.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.191+0000 7f2d37fff700 1 -- 172.21.15.2:0/1079371819 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d400039f0 con 0x7f2d50106520 2023-09-13T04:31:24.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.191+0000 7f2d37fff700 1 -- 172.21.15.2:0/1079371819 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d40003b50 con 0x7f2d50106520 2023-09-13T04:31:24.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.191+0000 7f2d5543e700 1 -- 172.21.15.2:0/1079371819 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d5019cb90 con 0x7f2d50106520 2023-09-13T04:31:24.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.191+0000 7f2d37fff700 1 -- 172.21.15.2:0/1079371819 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d4001a780 con 0x7f2d50106520 2023-09-13T04:31:24.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.192+0000 7f2d5543e700 1 -- 172.21.15.2:0/1079371819 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d5019cfb0 con 0x7f2d50106520 2023-09-13T04:31:24.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.192+0000 7f2d37fff700 1 -- 172.21.15.2:0/1079371819 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 9) v1 ==== 45344+0+0 (secure 0 0 0) 0x7f2d40024440 con 0x7f2d50106520 2023-09-13T04:31:24.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.193+0000 7f2d37fff700 1 --2- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f2d38038400 0x7f2d3803a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:24.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.193+0000 7f2d37fff700 1 -- 172.21.15.2:0/1079371819 --> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d3803afd0 con 0x7f2d38038400 2023-09-13T04:31:24.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.193+0000 7f2d37fff700 1 -- 172.21.15.2:0/1079371819 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2d4004c370 con 0x7f2d50106520 2023-09-13T04:31:24.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.193+0000 7f2d4e7fc700 1 -- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f2d38038400 msgr2=0x7f2d3803a8c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1393377368 2023-09-13T04:31:24.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.193+0000 7f2d4e7fc700 1 --2- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f2d38038400 0x7f2d3803a8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-13T04:31:24.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.394+0000 7f2d4e7fc700 1 -- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f2d38038400 msgr2=0x7f2d3803a8c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1393377368 2023-09-13T04:31:24.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.394+0000 7f2d4e7fc700 1 --2- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f2d38038400 0x7f2d3803a8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-13T04:31:24.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.795+0000 7f2d4e7fc700 1 -- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f2d38038400 msgr2=0x7f2d3803a8c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1393377368 2023-09-13T04:31:24.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:17.795+0000 7f2d4e7fc700 1 --2- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f2d38038400 0x7f2d3803a8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-13T04:31:24.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:18.596+0000 7f2d4e7fc700 1 -- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f2d38038400 msgr2=0x7f2d3803a8c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1393377368 2023-09-13T04:31:24.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:18.596+0000 7f2d4e7fc700 1 --2- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f2d38038400 0x7f2d3803a8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-13T04:31:24.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:20.197+0000 7f2d4e7fc700 1 -- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f2d38038400 msgr2=0x7f2d3803a8c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1393377368 2023-09-13T04:31:24.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:20.198+0000 7f2d4e7fc700 1 --2- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f2d38038400 0x7f2d3803a8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-09-13T04:31:24.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:23.401+0000 7f2d4e7fc700 1 -- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f2d38038400 msgr2=0x7f2d3803a8c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1393377368 2023-09-13T04:31:24.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:23.401+0000 7f2d4e7fc700 1 --2- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f2d38038400 0x7f2d3803a8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-09-13T04:31:24.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:23.420+0000 7f2d37fff700 1 -- 172.21.15.2:0/1079371819 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mgrmap(e 10) v1 ==== 45111+0+0 (secure 0 0 0) 0x7f2d4002d910 con 0x7f2d50106520 2023-09-13T04:31:24.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:23.420+0000 7f2d37fff700 1 -- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f2d38038400 msgr2=0x7f2d3803a8c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-13T04:31:24.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:23.420+0000 7f2d37fff700 1 --2- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f2d38038400 0x7f2d3803a8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:24.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:24.423+0000 7f2d37fff700 1 -- 172.21.15.2:0/1079371819 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 11) v1 ==== 45238+0+0 (secure 0 0 0) 0x7f2d4000fdc0 con 0x7f2d50106520 2023-09-13T04:31:24.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:24.423+0000 7f2d37fff700 1 --2- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2d3803be70 0x7f2d3803e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:24.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:24.423+0000 7f2d37fff700 1 -- 172.21.15.2:0/1079371819 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d3803afd0 con 0x7f2d3803be70 2023-09-13T04:31:24.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:24.424+0000 7f2d4e7fc700 1 --2- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2d3803be70 0x7f2d3803e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:24.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:24.426+0000 7f2d4e7fc700 1 --2- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2d3803be70 0x7f2d3803e260 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2d44003a10 tx=0x7f2d44009270 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:24.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:24.426+0000 7f2d37fff700 1 -- 172.21.15.2:0/1079371819 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f2d3803afd0 con 0x7f2d3803be70 2023-09-13T04:31:24.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:24.439+0000 7f2d5543e700 1 -- 172.21.15.2:0/1079371819 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f2d5019d790 con 0x7f2d3803be70 2023-09-13T04:31:24.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:24.439+0000 7f2d37fff700 1 -- 172.21.15.2:0/1079371819 <== mgr.14148 v2:172.21.15.2:6800/1461251021 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f2d5019d790 con 0x7f2d3803be70 2023-09-13T04:31:24.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:24.440+0000 7f2d5543e700 1 -- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2d3803be70 msgr2=0x7f2d3803e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:24.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:24.440+0000 7f2d5543e700 1 --2- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2d3803be70 0x7f2d3803e260 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2d44003a10 tx=0x7f2d44009270 comp rx=0 tx=0).stop 2023-09-13T04:31:24.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:24.440+0000 7f2d5543e700 1 -- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d50106520 msgr2=0x7f2d5019c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:24.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:24.440+0000 7f2d5543e700 1 --2- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d50106520 0x7f2d5019c450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2d40000c00 tx=0x7f2d40011a30 comp rx=0 tx=0).stop 2023-09-13T04:31:24.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:24.440+0000 7f2d5543e700 1 -- 172.21.15.2:0/1079371819 shutdown_connections 2023-09-13T04:31:24.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:24.440+0000 7f2d5543e700 1 --2- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2d3803be70 0x7f2d3803e260 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:24.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:24.440+0000 7f2d5543e700 1 --2- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d50106520 0x7f2d5019c450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:24.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:24.440+0000 7f2d5543e700 1 --2- 172.21.15.2:0/1079371819 >> [v2:172.21.15.2:6800/1393377368,v1:172.21.15.2:6801/1393377368] conn(0x7f2d38038400 0x7f2d3803a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:24.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:24.440+0000 7f2d5543e700 1 -- 172.21.15.2:0/1079371819 >> 172.21.15.2:0/1079371819 conn(0x7f2d50101b00 msgr2=0x7f2d501031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:24.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:24.440+0000 7f2d5543e700 1 -- 172.21.15.2:0/1079371819 shutdown_connections 2023-09-13T04:31:24.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:24.440+0000 7f2d5543e700 1 -- 172.21.15.2:0/1079371819 wait complete. 2023-09-13T04:31:24.865 INFO:teuthology.orchestra.run.smithi002.stdout:mgr epoch 9 is available 2023-09-13T04:31:24.866 INFO:teuthology.orchestra.run.smithi002.stdout:Generating a dashboard self-signed certificate... 2023-09-13T04:31:25.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:25 smithi002 ceph-mon[97651]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-09-13T04:31:25.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:25 smithi002 ceph-mon[97651]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-09-13T04:31:26.428 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-09-13T04:31:26.429 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.446+0000 7f46ed1aa700 1 Processor -- start 2023-09-13T04:31:26.429 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.446+0000 7f46ed1aa700 1 -- start start 2023-09-13T04:31:26.429 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.446+0000 7f46ed1aa700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e8106520 0x7f46e8106940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:26.429 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.446+0000 7f46ed1aa700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e8106f10 con 0x7f46e8106520 2023-09-13T04:31:26.429 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.446+0000 7f46e6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e8106520 0x7f46e8106940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:26.430 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.447+0000 7f46e6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e8106520 0x7f46e8106940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35124/0 (socket says 172.21.15.2:35124) 2023-09-13T04:31:26.430 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.447+0000 7f46e6d9d700 1 -- 172.21.15.2:0/1532623038 learned_addr learned my addr 172.21.15.2:0/1532623038 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:26.430 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.447+0000 7f46e6d9d700 1 -- 172.21.15.2:0/1532623038 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e8107720 con 0x7f46e8106520 2023-09-13T04:31:26.430 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.447+0000 7f46e6d9d700 1 --2- 172.21.15.2:0/1532623038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e8106520 0x7f46e8106940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f46d80098a0 tx=0x7f46d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22141877a254df4c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:26.430 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.447+0000 7f46e5d9b700 1 -- 172.21.15.2:0/1532623038 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46d8004160 con 0x7f46e8106520 2023-09-13T04:31:26.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.447+0000 7f46e5d9b700 1 -- 172.21.15.2:0/1532623038 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46d80042c0 con 0x7f46e8106520 2023-09-13T04:31:26.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.448+0000 7f46e5d9b700 1 -- 172.21.15.2:0/1532623038 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46d80119b0 con 0x7f46e8106520 2023-09-13T04:31:26.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.448+0000 7f46ed1aa700 1 -- 172.21.15.2:0/1532623038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e8106520 msgr2=0x7f46e8106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:26.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.448+0000 7f46ed1aa700 1 --2- 172.21.15.2:0/1532623038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e8106520 0x7f46e8106940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f46d80098a0 tx=0x7f46d8009bb0 comp rx=0 tx=0).stop 2023-09-13T04:31:26.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.448+0000 7f46ed1aa700 1 -- 172.21.15.2:0/1532623038 shutdown_connections 2023-09-13T04:31:26.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.448+0000 7f46ed1aa700 1 --2- 172.21.15.2:0/1532623038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e8106520 0x7f46e8106940 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:26.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.448+0000 7f46ed1aa700 1 -- 172.21.15.2:0/1532623038 >> 172.21.15.2:0/1532623038 conn(0x7f46e8101b00 msgr2=0x7f46e8103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:26.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.448+0000 7f46ed1aa700 1 -- 172.21.15.2:0/1532623038 shutdown_connections 2023-09-13T04:31:26.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.448+0000 7f46ed1aa700 1 -- 172.21.15.2:0/1532623038 wait complete. 2023-09-13T04:31:26.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.449+0000 7f46ed1aa700 1 Processor -- start 2023-09-13T04:31:26.433 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.449+0000 7f46ed1aa700 1 -- start start 2023-09-13T04:31:26.433 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.450+0000 7f46ed1aa700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e8106520 0x7f46e807bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:26.433 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.450+0000 7f46ed1aa700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e807c1a0 con 0x7f46e8106520 2023-09-13T04:31:26.433 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.450+0000 7f46e6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e8106520 0x7f46e807bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:26.433 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.450+0000 7f46e6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e8106520 0x7f46e807bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35136/0 (socket says 172.21.15.2:35136) 2023-09-13T04:31:26.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.450+0000 7f46e6d9d700 1 -- 172.21.15.2:0/4027535642 learned_addr learned my addr 172.21.15.2:0/4027535642 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:26.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.450+0000 7f46e6d9d700 1 -- 172.21.15.2:0/4027535642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d8009550 con 0x7f46e8106520 2023-09-13T04:31:26.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.451+0000 7f46e6d9d700 1 --2- 172.21.15.2:0/4027535642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e8106520 0x7f46e807bc60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f46d8000c00 tx=0x7f46d8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:26.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.451+0000 7f46cffff700 1 -- 172.21.15.2:0/4027535642 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46d80039f0 con 0x7f46e8106520 2023-09-13T04:31:26.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.451+0000 7f46cffff700 1 -- 172.21.15.2:0/4027535642 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46d8003b50 con 0x7f46e8106520 2023-09-13T04:31:26.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.451+0000 7f46cffff700 1 -- 172.21.15.2:0/4027535642 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46d801a780 con 0x7f46e8106520 2023-09-13T04:31:26.435 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.451+0000 7f46ed1aa700 1 -- 172.21.15.2:0/4027535642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e807a2b0 con 0x7f46e8106520 2023-09-13T04:31:26.435 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.451+0000 7f46ed1aa700 1 -- 172.21.15.2:0/4027535642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e807a6d0 con 0x7f46e8106520 2023-09-13T04:31:26.435 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.452+0000 7f46ed1aa700 1 -- 172.21.15.2:0/4027535642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e80622e0 con 0x7f46e8106520 2023-09-13T04:31:26.435 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.452+0000 7f46cffff700 1 -- 172.21.15.2:0/4027535642 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 11) v1 ==== 45238+0+0 (secure 0 0 0) 0x7f46d8024440 con 0x7f46e8106520 2023-09-13T04:31:26.435 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.453+0000 7f46cffff700 1 --2- 172.21.15.2:0/4027535642 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f46d00382d0 0x7f46d003a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:26.436 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.453+0000 7f46cffff700 1 -- 172.21.15.2:0/4027535642 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f46d804c240 con 0x7f46e8106520 2023-09-13T04:31:26.436 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.453+0000 7f46e659c700 1 --2- 172.21.15.2:0/4027535642 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f46d00382d0 0x7f46d003a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:26.436 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.454+0000 7f46e659c700 1 --2- 172.21.15.2:0/4027535642 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f46d00382d0 0x7f46d003a790 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f46dc006fd0 tx=0x7f46dc006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:26.436 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.458+0000 7f46cffff700 1 -- 172.21.15.2:0/4027535642 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f46d8018db0 con 0x7f46e8106520 2023-09-13T04:31:26.436 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.473+0000 7f46cffff700 1 -- 172.21.15.2:0/4027535642 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 12) v1 ==== 45344+0+0 (secure 0 0 0) 0x7f46d802d910 con 0x7f46e8106520 2023-09-13T04:31:26.437 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.687+0000 7f46ed1aa700 1 -- 172.21.15.2:0/4027535642 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f46e810a920 con 0x7f46d00382d0 2023-09-13T04:31:26.437 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.767+0000 7f46cffff700 1 -- 172.21.15.2:0/4027535642 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f46e810a920 con 0x7f46d00382d0 2023-09-13T04:31:26.437 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.770+0000 7f46ed1aa700 1 -- 172.21.15.2:0/4027535642 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f46d00382d0 msgr2=0x7f46d003a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:26.437 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.770+0000 7f46ed1aa700 1 --2- 172.21.15.2:0/4027535642 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f46d00382d0 0x7f46d003a790 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f46dc006fd0 tx=0x7f46dc006e40 comp rx=0 tx=0).stop 2023-09-13T04:31:26.437 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.770+0000 7f46ed1aa700 1 -- 172.21.15.2:0/4027535642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e8106520 msgr2=0x7f46e807bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:26.438 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.770+0000 7f46ed1aa700 1 --2- 172.21.15.2:0/4027535642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e8106520 0x7f46e807bc60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f46d8000c00 tx=0x7f46d8011a30 comp rx=0 tx=0).stop 2023-09-13T04:31:26.438 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.770+0000 7f46ed1aa700 1 -- 172.21.15.2:0/4027535642 shutdown_connections 2023-09-13T04:31:26.438 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.770+0000 7f46ed1aa700 1 --2- 172.21.15.2:0/4027535642 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f46d00382d0 0x7f46d003a790 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:26.438 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.770+0000 7f46ed1aa700 1 --2- 172.21.15.2:0/4027535642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e8106520 0x7f46e807bc60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:26.438 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.770+0000 7f46ed1aa700 1 -- 172.21.15.2:0/4027535642 >> 172.21.15.2:0/4027535642 conn(0x7f46e8101b00 msgr2=0x7f46e81031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:26.439 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.771+0000 7f46ed1aa700 1 -- 172.21.15.2:0/4027535642 shutdown_connections 2023-09-13T04:31:26.439 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:25.771+0000 7f46ed1aa700 1 -- 172.21.15.2:0/4027535642 wait complete. 2023-09-13T04:31:26.439 INFO:teuthology.orchestra.run.smithi002.stdout:Creating initial admin user... 2023-09-13T04:31:26.720 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:26 smithi002 ceph-mon[97651]: [13/Sep/2023:04:31:24] ENGINE Bus STARTING 2023-09-13T04:31:26.720 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:26 smithi002 ceph-mon[97651]: [13/Sep/2023:04:31:24] ENGINE Serving on http://172.21.15.2:8765 2023-09-13T04:31:26.720 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:26 smithi002 ceph-mon[97651]: [13/Sep/2023:04:31:24] ENGINE Serving on https://172.21.15.2:7150 2023-09-13T04:31:26.721 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:26 smithi002 ceph-mon[97651]: [13/Sep/2023:04:31:24] ENGINE Bus STARTED 2023-09-13T04:31:26.721 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:26 smithi002 ceph-mon[97651]: mgrmap e12: x(active, since 2s) 2023-09-13T04:31:26.721 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:26 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:26.721 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:26 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:26.722 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:26 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:27.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:27 smithi002 ceph-mon[97651]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:31:27.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:27 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:28.013 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$3zhXe9b3BMzQ8Wookb3YE.VoQaKvTg0QjURUbOAkAHEHIKgPGW7dC", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1694579487, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-09-13T04:31:28.013 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.024+0000 7fe8dee5c700 1 Processor -- start 2023-09-13T04:31:28.013 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.024+0000 7fe8dee5c700 1 -- start start 2023-09-13T04:31:28.013 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.024+0000 7fe8dee5c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8d807be90 0x7fe8d807a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:28.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.024+0000 7fe8dee5c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d807a8c0 con 0x7fe8d807be90 2023-09-13T04:31:28.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.025+0000 7fe8dcbf8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8d807be90 0x7fe8d807a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:28.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.025+0000 7fe8dcbf8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8d807be90 0x7fe8d807a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35146/0 (socket says 172.21.15.2:35146) 2023-09-13T04:31:28.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.025+0000 7fe8dcbf8700 1 -- 172.21.15.2:0/4096110610 learned_addr learned my addr 172.21.15.2:0/4096110610 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:28.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.025+0000 7fe8dcbf8700 1 -- 172.21.15.2:0/4096110610 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d807aa00 con 0x7fe8d807be90 2023-09-13T04:31:28.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.026+0000 7fe8dcbf8700 1 --2- 172.21.15.2:0/4096110610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8d807be90 0x7fe8d807a2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc0098a0 tx=0x7fe8cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6dbce9ad646d928 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:28.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.026+0000 7fe8d77fe700 1 -- 172.21.15.2:0/4096110610 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8cc004160 con 0x7fe8d807be90 2023-09-13T04:31:28.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.026+0000 7fe8d77fe700 1 -- 172.21.15.2:0/4096110610 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8cc0042c0 con 0x7fe8d807be90 2023-09-13T04:31:28.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.026+0000 7fe8d77fe700 1 -- 172.21.15.2:0/4096110610 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8cc0119b0 con 0x7fe8d807be90 2023-09-13T04:31:28.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.026+0000 7fe8dee5c700 1 -- 172.21.15.2:0/4096110610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8d807be90 msgr2=0x7fe8d807a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:28.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.026+0000 7fe8dee5c700 1 --2- 172.21.15.2:0/4096110610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8d807be90 0x7fe8d807a2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc0098a0 tx=0x7fe8cc009bb0 comp rx=0 tx=0).stop 2023-09-13T04:31:28.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.027+0000 7fe8dee5c700 1 -- 172.21.15.2:0/4096110610 shutdown_connections 2023-09-13T04:31:28.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.027+0000 7fe8dee5c700 1 --2- 172.21.15.2:0/4096110610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8d807be90 0x7fe8d807a2f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:28.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.027+0000 7fe8dee5c700 1 -- 172.21.15.2:0/4096110610 >> 172.21.15.2:0/4096110610 conn(0x7fe8d8101b00 msgr2=0x7fe8d8103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:28.017 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.027+0000 7fe8dee5c700 1 -- 172.21.15.2:0/4096110610 shutdown_connections 2023-09-13T04:31:28.017 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.027+0000 7fe8dee5c700 1 -- 172.21.15.2:0/4096110610 wait complete. 2023-09-13T04:31:28.017 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.028+0000 7fe8dee5c700 1 Processor -- start 2023-09-13T04:31:28.017 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.028+0000 7fe8dee5c700 1 -- start start 2023-09-13T04:31:28.017 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.028+0000 7fe8dee5c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8d807be90 0x7fe8d819c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:28.018 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.028+0000 7fe8dee5c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d819c980 con 0x7fe8d807be90 2023-09-13T04:31:28.018 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.029+0000 7fe8dcbf8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8d807be90 0x7fe8d819c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:28.018 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.029+0000 7fe8dcbf8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8d807be90 0x7fe8d819c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35160/0 (socket says 172.21.15.2:35160) 2023-09-13T04:31:28.018 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.029+0000 7fe8dcbf8700 1 -- 172.21.15.2:0/1207395599 learned_addr learned my addr 172.21.15.2:0/1207395599 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:28.018 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.029+0000 7fe8dcbf8700 1 -- 172.21.15.2:0/1207395599 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8cc009550 con 0x7fe8d807be90 2023-09-13T04:31:28.019 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.029+0000 7fe8dcbf8700 1 --2- 172.21.15.2:0/1207395599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8d807be90 0x7fe8d819c440 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc011980 tx=0x7fe8cc011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:28.019 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.030+0000 7fe8d5ffb700 1 -- 172.21.15.2:0/1207395599 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8cc003a60 con 0x7fe8d807be90 2023-09-13T04:31:28.019 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.030+0000 7fe8dee5c700 1 -- 172.21.15.2:0/1207395599 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d819cb80 con 0x7fe8d807be90 2023-09-13T04:31:28.019 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.030+0000 7fe8d5ffb700 1 -- 172.21.15.2:0/1207395599 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8cc003bc0 con 0x7fe8d807be90 2023-09-13T04:31:28.020 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.030+0000 7fe8d5ffb700 1 -- 172.21.15.2:0/1207395599 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8cc01a670 con 0x7fe8d807be90 2023-09-13T04:31:28.020 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.030+0000 7fe8dee5c700 1 -- 172.21.15.2:0/1207395599 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d819cfa0 con 0x7fe8d807be90 2023-09-13T04:31:28.020 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.031+0000 7fe8dee5c700 1 -- 172.21.15.2:0/1207395599 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d80622e0 con 0x7fe8d807be90 2023-09-13T04:31:28.020 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.031+0000 7fe8d5ffb700 1 -- 172.21.15.2:0/1207395599 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 12) v1 ==== 45344+0+0 (secure 0 0 0) 0x7fe8cc024a00 con 0x7fe8d807be90 2023-09-13T04:31:28.020 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.032+0000 7fe8d5ffb700 1 --2- 172.21.15.2:0/1207395599 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe8c8038400 0x7fe8c803a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:28.020 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.032+0000 7fe8d5ffb700 1 -- 172.21.15.2:0/1207395599 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe8cc04c450 con 0x7fe8d807be90 2023-09-13T04:31:28.021 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.032+0000 7fe8d7fff700 1 --2- 172.21.15.2:0/1207395599 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe8c8038400 0x7fe8c803a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:28.021 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.033+0000 7fe8d7fff700 1 --2- 172.21.15.2:0/1207395599 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe8c8038400 0x7fe8c803a8c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4006fd0 tx=0x7fe8c4006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:28.021 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.037+0000 7fe8d5ffb700 1 -- 172.21.15.2:0/1207395599 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe8cc024440 con 0x7fe8d807be90 2023-09-13T04:31:28.021 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.269+0000 7fe8dee5c700 1 -- 172.21.15.2:0/1207395599 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fe8d8062100 con 0x7fe8c8038400 2023-09-13T04:31:28.022 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.516+0000 7fe8d5ffb700 1 -- 172.21.15.2:0/1207395599 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fe8d8062100 con 0x7fe8c8038400 2023-09-13T04:31:28.022 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.518+0000 7fe8dee5c700 1 -- 172.21.15.2:0/1207395599 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe8c8038400 msgr2=0x7fe8c803a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:28.022 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.518+0000 7fe8dee5c700 1 --2- 172.21.15.2:0/1207395599 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe8c8038400 0x7fe8c803a8c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4006fd0 tx=0x7fe8c4006e40 comp rx=0 tx=0).stop 2023-09-13T04:31:28.022 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.518+0000 7fe8dee5c700 1 -- 172.21.15.2:0/1207395599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8d807be90 msgr2=0x7fe8d819c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:28.022 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.518+0000 7fe8dee5c700 1 --2- 172.21.15.2:0/1207395599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8d807be90 0x7fe8d819c440 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc011980 tx=0x7fe8cc011a60 comp rx=0 tx=0).stop 2023-09-13T04:31:28.023 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.519+0000 7fe8dee5c700 1 -- 172.21.15.2:0/1207395599 shutdown_connections 2023-09-13T04:31:28.023 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.519+0000 7fe8dee5c700 1 --2- 172.21.15.2:0/1207395599 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe8c8038400 0x7fe8c803a8c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:28.023 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.519+0000 7fe8dee5c700 1 --2- 172.21.15.2:0/1207395599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8d807be90 0x7fe8d819c440 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:28.023 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.519+0000 7fe8dee5c700 1 -- 172.21.15.2:0/1207395599 >> 172.21.15.2:0/1207395599 conn(0x7fe8d8101b00 msgr2=0x7fe8d81031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:28.023 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.519+0000 7fe8dee5c700 1 -- 172.21.15.2:0/1207395599 shutdown_connections 2023-09-13T04:31:28.024 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:27.519+0000 7fe8dee5c700 1 -- 172.21.15.2:0/1207395599 wait complete. 2023-09-13T04:31:28.024 INFO:teuthology.orchestra.run.smithi002.stdout:Fetching dashboard port number... 2023-09-13T04:31:28.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:28 smithi002 ceph-mon[97651]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:31:28.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:28 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:29.695 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 8443 2023-09-13T04:31:29.696 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.618+0000 7f52c0769700 1 Processor -- start 2023-09-13T04:31:29.696 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.618+0000 7f52c0769700 1 -- start start 2023-09-13T04:31:29.696 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.619+0000 7f52c0769700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b8106520 0x7f52b8106940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:29.696 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.619+0000 7f52c0769700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b8106f10 con 0x7f52b8106520 2023-09-13T04:31:29.696 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.619+0000 7f52be505700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b8106520 0x7f52b8106940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:29.696 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.619+0000 7f52be505700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b8106520 0x7f52b8106940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35166/0 (socket says 172.21.15.2:35166) 2023-09-13T04:31:29.697 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.619+0000 7f52be505700 1 -- 172.21.15.2:0/633557120 learned_addr learned my addr 172.21.15.2:0/633557120 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:29.697 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.620+0000 7f52be505700 1 -- 172.21.15.2:0/633557120 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b8107720 con 0x7f52b8106520 2023-09-13T04:31:29.697 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.620+0000 7f52be505700 1 --2- 172.21.15.2:0/633557120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b8106520 0x7f52b8106940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f52b40098a0 tx=0x7f52b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f747141459d61b35 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:29.697 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.620+0000 7f52bd503700 1 -- 172.21.15.2:0/633557120 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52b4004160 con 0x7f52b8106520 2023-09-13T04:31:29.697 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.620+0000 7f52bd503700 1 -- 172.21.15.2:0/633557120 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52b40042c0 con 0x7f52b8106520 2023-09-13T04:31:29.698 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.620+0000 7f52bd503700 1 -- 172.21.15.2:0/633557120 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52b40119b0 con 0x7f52b8106520 2023-09-13T04:31:29.698 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.621+0000 7f52c0769700 1 -- 172.21.15.2:0/633557120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b8106520 msgr2=0x7f52b8106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:29.698 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.621+0000 7f52c0769700 1 --2- 172.21.15.2:0/633557120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b8106520 0x7f52b8106940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f52b40098a0 tx=0x7f52b4009bb0 comp rx=0 tx=0).stop 2023-09-13T04:31:29.698 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.621+0000 7f52c0769700 1 -- 172.21.15.2:0/633557120 shutdown_connections 2023-09-13T04:31:29.698 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.621+0000 7f52c0769700 1 --2- 172.21.15.2:0/633557120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b8106520 0x7f52b8106940 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:29.699 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.621+0000 7f52c0769700 1 -- 172.21.15.2:0/633557120 >> 172.21.15.2:0/633557120 conn(0x7f52b8101b00 msgr2=0x7f52b8103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:29.699 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.621+0000 7f52c0769700 1 -- 172.21.15.2:0/633557120 shutdown_connections 2023-09-13T04:31:29.699 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.621+0000 7f52c0769700 1 -- 172.21.15.2:0/633557120 wait complete. 2023-09-13T04:31:29.699 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.622+0000 7f52c0769700 1 Processor -- start 2023-09-13T04:31:29.699 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.622+0000 7f52c0769700 1 -- start start 2023-09-13T04:31:29.700 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.623+0000 7f52c0769700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b8106520 0x7f52b819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:29.700 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.623+0000 7f52c0769700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b819a780 con 0x7f52b8106520 2023-09-13T04:31:29.700 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.623+0000 7f52be505700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b8106520 0x7f52b819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:29.700 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.623+0000 7f52be505700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b8106520 0x7f52b819a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35174/0 (socket says 172.21.15.2:35174) 2023-09-13T04:31:29.700 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.623+0000 7f52be505700 1 -- 172.21.15.2:0/591302513 learned_addr learned my addr 172.21.15.2:0/591302513 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:29.700 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.623+0000 7f52be505700 1 -- 172.21.15.2:0/591302513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b4009550 con 0x7f52b8106520 2023-09-13T04:31:29.701 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.624+0000 7f52be505700 1 --2- 172.21.15.2:0/591302513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b8106520 0x7f52b819a240 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f52b4000c00 tx=0x7f52b4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:29.701 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.624+0000 7f52af7fe700 1 -- 172.21.15.2:0/591302513 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52b40039f0 con 0x7f52b8106520 2023-09-13T04:31:29.701 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.624+0000 7f52af7fe700 1 -- 172.21.15.2:0/591302513 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52b4003b50 con 0x7f52b8106520 2023-09-13T04:31:29.701 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.624+0000 7f52af7fe700 1 -- 172.21.15.2:0/591302513 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52b401a780 con 0x7f52b8106520 2023-09-13T04:31:29.702 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.624+0000 7f52c0769700 1 -- 172.21.15.2:0/591302513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b819a980 con 0x7f52b8106520 2023-09-13T04:31:29.702 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.624+0000 7f52c0769700 1 -- 172.21.15.2:0/591302513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b819ada0 con 0x7f52b8106520 2023-09-13T04:31:29.702 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.625+0000 7f52c0769700 1 -- 172.21.15.2:0/591302513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b810a950 con 0x7f52b8106520 2023-09-13T04:31:29.702 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.625+0000 7f52af7fe700 1 -- 172.21.15.2:0/591302513 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 12) v1 ==== 45344+0+0 (secure 0 0 0) 0x7f52b4024440 con 0x7f52b8106520 2023-09-13T04:31:29.702 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.626+0000 7f52af7fe700 1 --2- 172.21.15.2:0/591302513 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f52a4038400 0x7f52a403a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:29.702 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.626+0000 7f52af7fe700 1 -- 172.21.15.2:0/591302513 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f52b404c460 con 0x7f52b8106520 2023-09-13T04:31:29.703 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.626+0000 7f52bdd04700 1 --2- 172.21.15.2:0/591302513 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f52a4038400 0x7f52a403a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:29.703 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.627+0000 7f52bdd04700 1 --2- 172.21.15.2:0/591302513 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f52a4038400 0x7f52a403a8c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f52a8006fd0 tx=0x7f52a8006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:29.703 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.631+0000 7f52af7fe700 1 -- 172.21.15.2:0/591302513 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f52b4018d90 con 0x7f52b8106520 2023-09-13T04:31:29.703 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.853+0000 7f52c0769700 1 -- 172.21.15.2:0/591302513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f52b819d7e0 con 0x7f52b8106520 2023-09-13T04:31:29.703 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.854+0000 7f52af7fe700 1 -- 172.21.15.2:0/591302513 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f52b819d7e0 con 0x7f52b8106520 2023-09-13T04:31:29.704 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.856+0000 7f52c0769700 1 -- 172.21.15.2:0/591302513 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f52a4038400 msgr2=0x7f52a403a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:29.704 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.856+0000 7f52c0769700 1 --2- 172.21.15.2:0/591302513 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f52a4038400 0x7f52a403a8c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f52a8006fd0 tx=0x7f52a8006e40 comp rx=0 tx=0).stop 2023-09-13T04:31:29.704 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.856+0000 7f52c0769700 1 -- 172.21.15.2:0/591302513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b8106520 msgr2=0x7f52b819a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:29.704 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.856+0000 7f52c0769700 1 --2- 172.21.15.2:0/591302513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b8106520 0x7f52b819a240 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f52b4000c00 tx=0x7f52b4011a30 comp rx=0 tx=0).stop 2023-09-13T04:31:29.704 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.856+0000 7f52c0769700 1 -- 172.21.15.2:0/591302513 shutdown_connections 2023-09-13T04:31:29.704 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.856+0000 7f52c0769700 1 --2- 172.21.15.2:0/591302513 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f52a4038400 0x7f52a403a8c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:29.705 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.857+0000 7f52c0769700 1 --2- 172.21.15.2:0/591302513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b8106520 0x7f52b819a240 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:29.705 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.857+0000 7f52c0769700 1 -- 172.21.15.2:0/591302513 >> 172.21.15.2:0/591302513 conn(0x7f52b8101b00 msgr2=0x7f52b81031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:29.705 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.857+0000 7f52c0769700 1 -- 172.21.15.2:0/591302513 shutdown_connections 2023-09-13T04:31:29.705 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:28.857+0000 7f52c0769700 1 -- 172.21.15.2:0/591302513 wait complete. 2023-09-13T04:31:29.717 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-13T04:31:29.718 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2023-09-13T04:31:29.734 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-13T04:31:29.734 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2023-09-13T04:31:29.735 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2023-09-13T04:31:29.735 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-09-13T04:31:29.738 INFO:teuthology.orchestra.run.smithi002.stdout:Ceph Dashboard is now available at: 2023-09-13T04:31:29.738 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:31:29.738 INFO:teuthology.orchestra.run.smithi002.stdout: URL: https://smithi002.front.sepia.ceph.com:8443/ 2023-09-13T04:31:29.738 INFO:teuthology.orchestra.run.smithi002.stdout: User: admin 2023-09-13T04:31:29.738 INFO:teuthology.orchestra.run.smithi002.stdout: Password: 8vm8bfmpkh 2023-09-13T04:31:29.739 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:31:29.739 INFO:teuthology.orchestra.run.smithi002.stdout:Saving cluster configuration to /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config directory 2023-09-13T04:31:29.739 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling autotune for osd_memory_target 2023-09-13T04:31:29.789 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:29 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/591302513' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-09-13T04:31:31.021 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.301+0000 7f7d7a19b700 1 Processor -- start 2023-09-13T04:31:31.025 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.301+0000 7f7d7a19b700 1 -- start start 2023-09-13T04:31:31.026 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.301+0000 7f7d7a19b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d74106520 0x7f7d74106940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:31.026 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.301+0000 7f7d7a19b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d74106f10 con 0x7f7d74106520 2023-09-13T04:31:31.026 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.302+0000 7f7d737fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d74106520 0x7f7d74106940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:31.026 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.302+0000 7f7d737fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d74106520 0x7f7d74106940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35178/0 (socket says 172.21.15.2:35178) 2023-09-13T04:31:31.027 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.302+0000 7f7d737fe700 1 -- 172.21.15.2:0/4131161359 learned_addr learned my addr 172.21.15.2:0/4131161359 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:31.027 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.302+0000 7f7d737fe700 1 -- 172.21.15.2:0/4131161359 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d74107720 con 0x7f7d74106520 2023-09-13T04:31:31.027 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.302+0000 7f7d737fe700 1 --2- 172.21.15.2:0/4131161359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d74106520 0x7f7d74106940 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7d640098a0 tx=0x7f7d64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4f86072a92f463 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:31.027 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.303+0000 7f7d727fc700 1 -- 172.21.15.2:0/4131161359 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d64004160 con 0x7f7d74106520 2023-09-13T04:31:31.027 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.303+0000 7f7d727fc700 1 -- 172.21.15.2:0/4131161359 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d640042c0 con 0x7f7d74106520 2023-09-13T04:31:31.028 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.303+0000 7f7d727fc700 1 -- 172.21.15.2:0/4131161359 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d640119b0 con 0x7f7d74106520 2023-09-13T04:31:31.028 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.303+0000 7f7d7a19b700 1 -- 172.21.15.2:0/4131161359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d74106520 msgr2=0x7f7d74106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:31.028 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.303+0000 7f7d7a19b700 1 --2- 172.21.15.2:0/4131161359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d74106520 0x7f7d74106940 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7d640098a0 tx=0x7f7d64009bb0 comp rx=0 tx=0).stop 2023-09-13T04:31:31.028 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.303+0000 7f7d7a19b700 1 -- 172.21.15.2:0/4131161359 shutdown_connections 2023-09-13T04:31:31.028 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.303+0000 7f7d7a19b700 1 --2- 172.21.15.2:0/4131161359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d74106520 0x7f7d74106940 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:31.028 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.303+0000 7f7d7a19b700 1 -- 172.21.15.2:0/4131161359 >> 172.21.15.2:0/4131161359 conn(0x7f7d74101b00 msgr2=0x7f7d74103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:31.029 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.303+0000 7f7d7a19b700 1 -- 172.21.15.2:0/4131161359 shutdown_connections 2023-09-13T04:31:31.029 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.303+0000 7f7d7a19b700 1 -- 172.21.15.2:0/4131161359 wait complete. 2023-09-13T04:31:31.029 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.304+0000 7f7d7a19b700 1 Processor -- start 2023-09-13T04:31:31.029 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.305+0000 7f7d7a19b700 1 -- start start 2023-09-13T04:31:31.029 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.305+0000 7f7d7a19b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d74106520 0x7f7d7419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:31.030 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.305+0000 7f7d7a19b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7419a780 con 0x7f7d74106520 2023-09-13T04:31:31.030 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.305+0000 7f7d737fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d74106520 0x7f7d7419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:31.030 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.305+0000 7f7d737fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d74106520 0x7f7d7419a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35184/0 (socket says 172.21.15.2:35184) 2023-09-13T04:31:31.030 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.305+0000 7f7d737fe700 1 -- 172.21.15.2:0/1174295771 learned_addr learned my addr 172.21.15.2:0/1174295771 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:31.030 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.305+0000 7f7d737fe700 1 -- 172.21.15.2:0/1174295771 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d64009550 con 0x7f7d74106520 2023-09-13T04:31:31.031 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.306+0000 7f7d737fe700 1 --2- 172.21.15.2:0/1174295771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d74106520 0x7f7d7419a240 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7d64000c00 tx=0x7f7d64011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:31.031 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.306+0000 7f7d70ff9700 1 -- 172.21.15.2:0/1174295771 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d640039f0 con 0x7f7d74106520 2023-09-13T04:31:31.031 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.306+0000 7f7d70ff9700 1 -- 172.21.15.2:0/1174295771 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d64003b50 con 0x7f7d74106520 2023-09-13T04:31:31.031 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.306+0000 7f7d70ff9700 1 -- 172.21.15.2:0/1174295771 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d6401a780 con 0x7f7d74106520 2023-09-13T04:31:31.031 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.306+0000 7f7d7a19b700 1 -- 172.21.15.2:0/1174295771 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7419a980 con 0x7f7d74106520 2023-09-13T04:31:31.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.306+0000 7f7d7a19b700 1 -- 172.21.15.2:0/1174295771 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7419ada0 con 0x7f7d74106520 2023-09-13T04:31:31.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.307+0000 7f7d70ff9700 1 -- 172.21.15.2:0/1174295771 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45386+0+0 (secure 0 0 0) 0x7f7d64024440 con 0x7f7d74106520 2023-09-13T04:31:31.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.307+0000 7f7d7a19b700 1 -- 172.21.15.2:0/1174295771 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d7410a950 con 0x7f7d74106520 2023-09-13T04:31:31.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.308+0000 7f7d70ff9700 1 --2- 172.21.15.2:0/1174295771 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f7d5c0384a0 0x7f7d5c03a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:31.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.308+0000 7f7d70ff9700 1 -- 172.21.15.2:0/1174295771 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7d6404c520 con 0x7f7d74106520 2023-09-13T04:31:31.033 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.308+0000 7f7d72ffd700 1 --2- 172.21.15.2:0/1174295771 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f7d5c0384a0 0x7f7d5c03a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:31.033 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.308+0000 7f7d72ffd700 1 --2- 172.21.15.2:0/1174295771 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f7d5c0384a0 0x7f7d5c03a960 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7d68006fd0 tx=0x7f7d68006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:31.033 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.313+0000 7f7d70ff9700 1 -- 172.21.15.2:0/1174295771 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7d64018db0 con 0x7f7d74106520 2023-09-13T04:31:31.033 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.522+0000 7f7d7a19b700 1 -- 172.21.15.2:0/1174295771 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f7d7419d7e0 con 0x7f7d74106520 2023-09-13T04:31:31.033 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.523+0000 7f7d70ff9700 1 -- 172.21.15.2:0/1174295771 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f7d7419d7e0 con 0x7f7d74106520 2023-09-13T04:31:31.034 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.526+0000 7f7d7a19b700 1 -- 172.21.15.2:0/1174295771 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f7d5c0384a0 msgr2=0x7f7d5c03a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:31.034 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.526+0000 7f7d7a19b700 1 --2- 172.21.15.2:0/1174295771 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f7d5c0384a0 0x7f7d5c03a960 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7d68006fd0 tx=0x7f7d68006e40 comp rx=0 tx=0).stop 2023-09-13T04:31:31.034 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.526+0000 7f7d7a19b700 1 -- 172.21.15.2:0/1174295771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d74106520 msgr2=0x7f7d7419a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:31.034 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.526+0000 7f7d7a19b700 1 --2- 172.21.15.2:0/1174295771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d74106520 0x7f7d7419a240 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7d64000c00 tx=0x7f7d64011a30 comp rx=0 tx=0).stop 2023-09-13T04:31:31.034 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.526+0000 7f7d7a19b700 1 -- 172.21.15.2:0/1174295771 shutdown_connections 2023-09-13T04:31:31.035 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.526+0000 7f7d7a19b700 1 --2- 172.21.15.2:0/1174295771 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f7d5c0384a0 0x7f7d5c03a960 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:31.035 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.526+0000 7f7d7a19b700 1 --2- 172.21.15.2:0/1174295771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d74106520 0x7f7d7419a240 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:31.035 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.526+0000 7f7d7a19b700 1 -- 172.21.15.2:0/1174295771 >> 172.21.15.2:0/1174295771 conn(0x7f7d74101b00 msgr2=0x7f7d741031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:31.035 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.526+0000 7f7d7a19b700 1 -- 172.21.15.2:0/1174295771 shutdown_connections 2023-09-13T04:31:31.035 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:30.526+0000 7f7d7a19b700 1 -- 172.21.15.2:0/1174295771 wait complete. 2023-09-13T04:31:31.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:31 smithi002 ceph-mon[97651]: mgrmap e13: x(active, since 6s) 2023-09-13T04:31:32.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.573+0000 7fdc06373700 1 Processor -- start 2023-09-13T04:31:32.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.574+0000 7fdc06373700 1 -- start start 2023-09-13T04:31:32.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.574+0000 7fdc06373700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc0007bed0 0x7fdc0007a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:32.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.574+0000 7fdc06373700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0007a900 con 0x7fdc0007bed0 2023-09-13T04:31:32.405 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.574+0000 7fdbfffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc0007bed0 0x7fdc0007a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:32.405 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.574+0000 7fdbfffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc0007bed0 0x7fdc0007a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35192/0 (socket says 172.21.15.2:35192) 2023-09-13T04:31:32.405 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.574+0000 7fdbfffff700 1 -- 172.21.15.2:0/3878651154 learned_addr learned my addr 172.21.15.2:0/3878651154 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:32.405 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.575+0000 7fdbfffff700 1 -- 172.21.15.2:0/3878651154 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0007aa40 con 0x7fdc0007bed0 2023-09-13T04:31:32.405 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.575+0000 7fdbfffff700 1 --2- 172.21.15.2:0/3878651154 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc0007bed0 0x7fdc0007a330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00098a0 tx=0x7fdbf0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed58137aad711680 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:32.405 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.575+0000 7fdbfeffd700 1 -- 172.21.15.2:0/3878651154 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbf0004160 con 0x7fdc0007bed0 2023-09-13T04:31:32.406 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.575+0000 7fdbfeffd700 1 -- 172.21.15.2:0/3878651154 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbf00042c0 con 0x7fdc0007bed0 2023-09-13T04:31:32.406 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.575+0000 7fdbfeffd700 1 -- 172.21.15.2:0/3878651154 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbf00119b0 con 0x7fdc0007bed0 2023-09-13T04:31:32.406 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.576+0000 7fdc06373700 1 -- 172.21.15.2:0/3878651154 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc0007bed0 msgr2=0x7fdc0007a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:32.406 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.576+0000 7fdc06373700 1 --2- 172.21.15.2:0/3878651154 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc0007bed0 0x7fdc0007a330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00098a0 tx=0x7fdbf0009bb0 comp rx=0 tx=0).stop 2023-09-13T04:31:32.406 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.576+0000 7fdc06373700 1 -- 172.21.15.2:0/3878651154 shutdown_connections 2023-09-13T04:31:32.407 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.576+0000 7fdc06373700 1 --2- 172.21.15.2:0/3878651154 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc0007bed0 0x7fdc0007a330 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:32.407 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.576+0000 7fdc06373700 1 -- 172.21.15.2:0/3878651154 >> 172.21.15.2:0/3878651154 conn(0x7fdc00101b00 msgr2=0x7fdc00103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:32.407 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.576+0000 7fdc06373700 1 -- 172.21.15.2:0/3878651154 shutdown_connections 2023-09-13T04:31:32.407 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.576+0000 7fdc06373700 1 -- 172.21.15.2:0/3878651154 wait complete. 2023-09-13T04:31:32.407 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.577+0000 7fdc06373700 1 Processor -- start 2023-09-13T04:31:32.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.577+0000 7fdc06373700 1 -- start start 2023-09-13T04:31:32.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.577+0000 7fdc06373700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc0007bed0 0x7fdc0019c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:32.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.577+0000 7fdc06373700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0019c960 con 0x7fdc0007bed0 2023-09-13T04:31:32.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.578+0000 7fdbfffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc0007bed0 0x7fdc0019c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:32.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.578+0000 7fdbfffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc0007bed0 0x7fdc0019c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35196/0 (socket says 172.21.15.2:35196) 2023-09-13T04:31:32.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.578+0000 7fdbfffff700 1 -- 172.21.15.2:0/1788947466 learned_addr learned my addr 172.21.15.2:0/1788947466 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:32.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.578+0000 7fdbfffff700 1 -- 172.21.15.2:0/1788947466 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf0009550 con 0x7fdc0007bed0 2023-09-13T04:31:32.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.578+0000 7fdbfffff700 1 --2- 172.21.15.2:0/1788947466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc0007bed0 0x7fdc0019c420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0000c00 tx=0x7fdbf0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:32.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.579+0000 7fdbfd7fa700 1 -- 172.21.15.2:0/1788947466 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbf00039f0 con 0x7fdc0007bed0 2023-09-13T04:31:32.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.579+0000 7fdbfd7fa700 1 -- 172.21.15.2:0/1788947466 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbf0003b50 con 0x7fdc0007bed0 2023-09-13T04:31:32.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.579+0000 7fdbfd7fa700 1 -- 172.21.15.2:0/1788947466 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbf001a780 con 0x7fdc0007bed0 2023-09-13T04:31:32.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.579+0000 7fdc06373700 1 -- 172.21.15.2:0/1788947466 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc0019cb60 con 0x7fdc0007bed0 2023-09-13T04:31:32.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.579+0000 7fdc06373700 1 -- 172.21.15.2:0/1788947466 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc0019cf80 con 0x7fdc0007bed0 2023-09-13T04:31:32.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.580+0000 7fdbfd7fa700 1 -- 172.21.15.2:0/1788947466 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45386+0+0 (secure 0 0 0) 0x7fdbf0024440 con 0x7fdc0007bed0 2023-09-13T04:31:32.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.580+0000 7fdc06373700 1 -- 172.21.15.2:0/1788947466 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc000622e0 con 0x7fdc0007bed0 2023-09-13T04:31:32.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.580+0000 7fdbfd7fa700 1 --2- 172.21.15.2:0/1788947466 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fdbe80384a0 0x7fdbe803a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:32.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.581+0000 7fdbfd7fa700 1 -- 172.21.15.2:0/1788947466 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdbf004c4f0 con 0x7fdc0007bed0 2023-09-13T04:31:32.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.581+0000 7fdbff7fe700 1 --2- 172.21.15.2:0/1788947466 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fdbe80384a0 0x7fdbe803a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:32.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.581+0000 7fdbff7fe700 1 --2- 172.21.15.2:0/1788947466 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fdbe80384a0 0x7fdbe803a960 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4006fd0 tx=0x7fdbf4006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:32.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.586+0000 7fdbfd7fa700 1 -- 172.21.15.2:0/1788947466 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fdbf0018d90 con 0x7fdc0007bed0 2023-09-13T04:31:32.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.906+0000 7fdc06373700 1 -- 172.21.15.2:0/1788947466 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fdc0010c920 con 0x7fdc0007bed0 2023-09-13T04:31:32.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.908+0000 7fdbfd7fa700 1 -- 172.21.15.2:0/1788947466 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7fdbf0018350 con 0x7fdc0007bed0 2023-09-13T04:31:32.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.910+0000 7fdc06373700 1 -- 172.21.15.2:0/1788947466 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fdbe80384a0 msgr2=0x7fdbe803a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:32.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.910+0000 7fdc06373700 1 --2- 172.21.15.2:0/1788947466 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fdbe80384a0 0x7fdbe803a960 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4006fd0 tx=0x7fdbf4006e40 comp rx=0 tx=0).stop 2023-09-13T04:31:32.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.911+0000 7fdc06373700 1 -- 172.21.15.2:0/1788947466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc0007bed0 msgr2=0x7fdc0019c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:32.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.911+0000 7fdc06373700 1 --2- 172.21.15.2:0/1788947466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc0007bed0 0x7fdc0019c420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0000c00 tx=0x7fdbf0011a30 comp rx=0 tx=0).stop 2023-09-13T04:31:32.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.911+0000 7fdc06373700 1 -- 172.21.15.2:0/1788947466 shutdown_connections 2023-09-13T04:31:32.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.911+0000 7fdc06373700 1 --2- 172.21.15.2:0/1788947466 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fdbe80384a0 0x7fdbe803a960 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:32.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.911+0000 7fdc06373700 1 --2- 172.21.15.2:0/1788947466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc0007bed0 0x7fdc0019c420 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:32.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.911+0000 7fdc06373700 1 -- 172.21.15.2:0/1788947466 >> 172.21.15.2:0/1788947466 conn(0x7fdc00101b00 msgr2=0x7fdc001031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:32.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.911+0000 7fdc06373700 1 -- 172.21.15.2:0/1788947466 shutdown_connections 2023-09-13T04:31:32.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-09-13T04:31:31.911+0000 7fdc06373700 1 -- 172.21.15.2:0/1788947466 wait complete. 2023-09-13T04:31:32.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-09-13T04:31:32.414 INFO:teuthology.orchestra.run.smithi002.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-09-13T04:31:32.414 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:31:32.414 INFO:teuthology.orchestra.run.smithi002.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-09-13T04:31:32.415 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:31:32.415 INFO:teuthology.orchestra.run.smithi002.stdout:Or, if you are only running a single cluster on this host: 2023-09-13T04:31:32.415 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:31:32.415 INFO:teuthology.orchestra.run.smithi002.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-09-13T04:31:32.415 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:31:32.415 INFO:teuthology.orchestra.run.smithi002.stdout:Please consider enabling telemetry to help improve Ceph: 2023-09-13T04:31:32.416 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:31:32.416 INFO:teuthology.orchestra.run.smithi002.stdout: ceph telemetry on 2023-09-13T04:31:32.416 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:31:32.416 INFO:teuthology.orchestra.run.smithi002.stdout:For more information see: 2023-09-13T04:31:32.416 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:31:32.417 INFO:teuthology.orchestra.run.smithi002.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-09-13T04:31:32.417 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:31:32.417 INFO:teuthology.orchestra.run.smithi002.stdout:Bootstrap complete. 2023-09-13T04:31:32.461 INFO:tasks.cephadm:Fetching config... 2023-09-13T04:31:32.461 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-09-13T04:31:32.461 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-09-13T04:31:32.506 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-09-13T04:31:32.506 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-09-13T04:31:32.507 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-09-13T04:31:32.591 INFO:tasks.cephadm:Fetching mon keyring... 2023-09-13T04:31:32.591 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-09-13T04:31:32.592 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/keyring of=/dev/stdout 2023-09-13T04:31:32.700 INFO:tasks.cephadm:Fetching pub ssh key... 2023-09-13T04:31:32.700 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-09-13T04:31:32.701 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-09-13T04:31:32.786 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-09-13T04:31:32.787 DEBUG:teuthology.orchestra.run.smithi002:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4/C7a9RXXyi7ovgVmcDjsCZ3q6B6mNpo/oRUdKobI5m6AJRvYNUaIB49L3JBgkgsI71Qvy2DT5b3PSaK6uHnk/DbY/xQ99ERuexjUtDbULn8inZupw9VYJpWuCahAm/9/y9eAj5n99d0hVAw4BqO9/jyhybiib7gMZtZRgllB39V3oWbZl0FDCwXUae7fqE8hSHrBLvhYfKw/sfSN0GbTHK53Xq1KgeHIjkfrHCHoihySn6rXPUa3Ppf5d7WY7zC3nyod1XuunrC/S3pHD7xXUxnOKGIXU0CnXeDujGYUkyJ9bbXg758ZdnfyhPInz/exIupLHajfj5vi6QQHFWiiOjvqsrloNCUC6+xJC/+EOiqaYkMxGwORrPbD6pOj9OrnctKyAd0wpOtRqG7JRWlwD6eEVU0m7yPjKPtIonsXQ/pP6jvZe8TxzxkPvCVFqlnrL2aI9zp46JDY9dLOfxVUhk3wt8VUbKpPMiWOkXTEaoE4eT04ZIL2iuZLiQ5hsIs= ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-13T04:31:32.931 INFO:teuthology.orchestra.run.smithi002.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4/C7a9RXXyi7ovgVmcDjsCZ3q6B6mNpo/oRUdKobI5m6AJRvYNUaIB49L3JBgkgsI71Qvy2DT5b3PSaK6uHnk/DbY/xQ99ERuexjUtDbULn8inZupw9VYJpWuCahAm/9/y9eAj5n99d0hVAw4BqO9/jyhybiib7gMZtZRgllB39V3oWbZl0FDCwXUae7fqE8hSHrBLvhYfKw/sfSN0GbTHK53Xq1KgeHIjkfrHCHoihySn6rXPUa3Ppf5d7WY7zC3nyod1XuunrC/S3pHD7xXUxnOKGIXU0CnXeDujGYUkyJ9bbXg758ZdnfyhPInz/exIupLHajfj5vi6QQHFWiiOjvqsrloNCUC6+xJC/+EOiqaYkMxGwORrPbD6pOj9OrnctKyAd0wpOtRqG7JRWlwD6eEVU0m7yPjKPtIonsXQ/pP6jvZe8TxzxkPvCVFqlnrL2aI9zp46JDY9dLOfxVUhk3wt8VUbKpPMiWOkXTEaoE4eT04ZIL2iuZLiQ5hsIs= ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:31:32.954 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:32 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1788947466' entity='client.admin' 2023-09-13T04:31:32.963 DEBUG:teuthology.orchestra.run.smithi149:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4/C7a9RXXyi7ovgVmcDjsCZ3q6B6mNpo/oRUdKobI5m6AJRvYNUaIB49L3JBgkgsI71Qvy2DT5b3PSaK6uHnk/DbY/xQ99ERuexjUtDbULn8inZupw9VYJpWuCahAm/9/y9eAj5n99d0hVAw4BqO9/jyhybiib7gMZtZRgllB39V3oWbZl0FDCwXUae7fqE8hSHrBLvhYfKw/sfSN0GbTHK53Xq1KgeHIjkfrHCHoihySn6rXPUa3Ppf5d7WY7zC3nyod1XuunrC/S3pHD7xXUxnOKGIXU0CnXeDujGYUkyJ9bbXg758ZdnfyhPInz/exIupLHajfj5vi6QQHFWiiOjvqsrloNCUC6+xJC/+EOiqaYkMxGwORrPbD6pOj9OrnctKyAd0wpOtRqG7JRWlwD6eEVU0m7yPjKPtIonsXQ/pP6jvZe8TxzxkPvCVFqlnrL2aI9zp46JDY9dLOfxVUhk3wt8VUbKpPMiWOkXTEaoE4eT04ZIL2iuZLiQ5hsIs= ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-13T04:31:33.009 INFO:teuthology.orchestra.run.smithi149.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4/C7a9RXXyi7ovgVmcDjsCZ3q6B6mNpo/oRUdKobI5m6AJRvYNUaIB49L3JBgkgsI71Qvy2DT5b3PSaK6uHnk/DbY/xQ99ERuexjUtDbULn8inZupw9VYJpWuCahAm/9/y9eAj5n99d0hVAw4BqO9/jyhybiib7gMZtZRgllB39V3oWbZl0FDCwXUae7fqE8hSHrBLvhYfKw/sfSN0GbTHK53Xq1KgeHIjkfrHCHoihySn6rXPUa3Ppf5d7WY7zC3nyod1XuunrC/S3pHD7xXUxnOKGIXU0CnXeDujGYUkyJ9bbXg758ZdnfyhPInz/exIupLHajfj5vi6QQHFWiiOjvqsrloNCUC6+xJC/+EOiqaYkMxGwORrPbD6pOj9OrnctKyAd0wpOtRqG7JRWlwD6eEVU0m7yPjKPtIonsXQ/pP6jvZe8TxzxkPvCVFqlnrL2aI9zp46JDY9dLOfxVUhk3wt8VUbKpPMiWOkXTEaoE4eT04ZIL2iuZLiQ5hsIs= ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:31:33.022 DEBUG:teuthology.orchestra.run.smithi156:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4/C7a9RXXyi7ovgVmcDjsCZ3q6B6mNpo/oRUdKobI5m6AJRvYNUaIB49L3JBgkgsI71Qvy2DT5b3PSaK6uHnk/DbY/xQ99ERuexjUtDbULn8inZupw9VYJpWuCahAm/9/y9eAj5n99d0hVAw4BqO9/jyhybiib7gMZtZRgllB39V3oWbZl0FDCwXUae7fqE8hSHrBLvhYfKw/sfSN0GbTHK53Xq1KgeHIjkfrHCHoihySn6rXPUa3Ppf5d7WY7zC3nyod1XuunrC/S3pHD7xXUxnOKGIXU0CnXeDujGYUkyJ9bbXg758ZdnfyhPInz/exIupLHajfj5vi6QQHFWiiOjvqsrloNCUC6+xJC/+EOiqaYkMxGwORrPbD6pOj9OrnctKyAd0wpOtRqG7JRWlwD6eEVU0m7yPjKPtIonsXQ/pP6jvZe8TxzxkPvCVFqlnrL2aI9zp46JDY9dLOfxVUhk3wt8VUbKpPMiWOkXTEaoE4eT04ZIL2iuZLiQ5hsIs= ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-13T04:31:33.067 INFO:teuthology.orchestra.run.smithi156.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4/C7a9RXXyi7ovgVmcDjsCZ3q6B6mNpo/oRUdKobI5m6AJRvYNUaIB49L3JBgkgsI71Qvy2DT5b3PSaK6uHnk/DbY/xQ99ERuexjUtDbULn8inZupw9VYJpWuCahAm/9/y9eAj5n99d0hVAw4BqO9/jyhybiib7gMZtZRgllB39V3oWbZl0FDCwXUae7fqE8hSHrBLvhYfKw/sfSN0GbTHK53Xq1KgeHIjkfrHCHoihySn6rXPUa3Ppf5d7WY7zC3nyod1XuunrC/S3pHD7xXUxnOKGIXU0CnXeDujGYUkyJ9bbXg758ZdnfyhPInz/exIupLHajfj5vi6QQHFWiiOjvqsrloNCUC6+xJC/+EOiqaYkMxGwORrPbD6pOj9OrnctKyAd0wpOtRqG7JRWlwD6eEVU0m7yPjKPtIonsXQ/pP6jvZe8TxzxkPvCVFqlnrL2aI9zp46JDY9dLOfxVUhk3wt8VUbKpPMiWOkXTEaoE4eT04ZIL2iuZLiQ5hsIs= ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:31:33.080 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-09-13T04:31:33.825 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:31:35.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.897+0000 7f55ca551700 1 -- 172.21.15.2:0/3115911932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c40ff0c0 msgr2=0x7f55c40ff4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:35.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.897+0000 7f55ca551700 1 --2- 172.21.15.2:0/3115911932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c40ff0c0 0x7f55c40ff4e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009910 tx=0x7f55b4009c20 comp rx=0 tx=0).stop 2023-09-13T04:31:35.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.898+0000 7f55ca551700 1 -- 172.21.15.2:0/3115911932 shutdown_connections 2023-09-13T04:31:35.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.898+0000 7f55ca551700 1 --2- 172.21.15.2:0/3115911932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c40ff0c0 0x7f55c40ff4e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:35.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.898+0000 7f55ca551700 1 -- 172.21.15.2:0/3115911932 >> 172.21.15.2:0/3115911932 conn(0x7f55c40fac80 msgr2=0x7f55c40fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:35.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.898+0000 7f55ca551700 1 -- 172.21.15.2:0/3115911932 shutdown_connections 2023-09-13T04:31:35.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.898+0000 7f55ca551700 1 -- 172.21.15.2:0/3115911932 wait complete. 2023-09-13T04:31:35.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.899+0000 7f55ca551700 1 Processor -- start 2023-09-13T04:31:35.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.899+0000 7f55ca551700 1 -- start start 2023-09-13T04:31:35.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.899+0000 7f55ca551700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c40ff0c0 0x7f55c4197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:35.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.899+0000 7f55ca551700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c4198120 con 0x7f55c40ff0c0 2023-09-13T04:31:35.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.899+0000 7f55c3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c40ff0c0 0x7f55c4197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:35.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.900+0000 7f55c3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c40ff0c0 0x7f55c4197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42920/0 (socket says 172.21.15.2:42920) 2023-09-13T04:31:35.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.900+0000 7f55c3fff700 1 -- 172.21.15.2:0/45035340 learned_addr learned my addr 172.21.15.2:0/45035340 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:35.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.900+0000 7f55c3fff700 1 -- 172.21.15.2:0/45035340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b40095f0 con 0x7f55c40ff0c0 2023-09-13T04:31:35.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.900+0000 7f55c3fff700 1 --2- 172.21.15.2:0/45035340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c40ff0c0 0x7f55c4197be0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f55b4005f50 tx=0x7f55b40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:35.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.900+0000 7f55c17fa700 1 -- 172.21.15.2:0/45035340 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55b401d070 con 0x7f55c40ff0c0 2023-09-13T04:31:35.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.900+0000 7f55c17fa700 1 -- 172.21.15.2:0/45035340 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55b4022470 con 0x7f55c40ff0c0 2023-09-13T04:31:35.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.900+0000 7f55ca551700 1 -- 172.21.15.2:0/45035340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c4198320 con 0x7f55c40ff0c0 2023-09-13T04:31:35.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.900+0000 7f55c17fa700 1 -- 172.21.15.2:0/45035340 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55b400f5f0 con 0x7f55c40ff0c0 2023-09-13T04:31:35.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.901+0000 7f55ca551700 1 -- 172.21.15.2:0/45035340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c41987c0 con 0x7f55c40ff0c0 2023-09-13T04:31:35.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.902+0000 7f55ca551700 1 -- 172.21.15.2:0/45035340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c41915e0 con 0x7f55c40ff0c0 2023-09-13T04:31:35.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.902+0000 7f55c17fa700 1 -- 172.21.15.2:0/45035340 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45386+0+0 (secure 0 0 0) 0x7f55b4022850 con 0x7f55c40ff0c0 2023-09-13T04:31:35.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.902+0000 7f55c17fa700 1 --2- 172.21.15.2:0/45035340 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f55ac038450 0x7f55ac03a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:35.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.903+0000 7f55c17fa700 1 -- 172.21.15.2:0/45035340 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f55b404c590 con 0x7f55c40ff0c0 2023-09-13T04:31:35.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.903+0000 7f55c37fe700 1 --2- 172.21.15.2:0/45035340 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f55ac038450 0x7f55ac03a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:35.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.903+0000 7f55c37fe700 1 --2- 172.21.15.2:0/45035340 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f55ac038450 0x7f55ac03a910 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f55b8006fd0 tx=0x7f55b8006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:35.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:35.908+0000 7f55c17fa700 1 -- 172.21.15.2:0/45035340 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f55b402a360 con 0x7f55c40ff0c0 2023-09-13T04:31:36.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:36.118+0000 7f55ca551700 1 -- 172.21.15.2:0/45035340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f55c4108820 con 0x7f55c40ff0c0 2023-09-13T04:31:36.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:36.122+0000 7f55c17fa700 1 -- 172.21.15.2:0/45035340 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f55b4027020 con 0x7f55c40ff0c0 2023-09-13T04:31:36.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:36.125+0000 7f55ca551700 1 -- 172.21.15.2:0/45035340 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f55ac038450 msgr2=0x7f55ac03a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:36.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:36.125+0000 7f55ca551700 1 --2- 172.21.15.2:0/45035340 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f55ac038450 0x7f55ac03a910 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f55b8006fd0 tx=0x7f55b8006e40 comp rx=0 tx=0).stop 2023-09-13T04:31:36.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:36.125+0000 7f55ca551700 1 -- 172.21.15.2:0/45035340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c40ff0c0 msgr2=0x7f55c4197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:36.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:36.125+0000 7f55ca551700 1 --2- 172.21.15.2:0/45035340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c40ff0c0 0x7f55c4197be0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f55b4005f50 tx=0x7f55b40057c0 comp rx=0 tx=0).stop 2023-09-13T04:31:36.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:36.125+0000 7f55ca551700 1 -- 172.21.15.2:0/45035340 shutdown_connections 2023-09-13T04:31:36.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:36.125+0000 7f55ca551700 1 --2- 172.21.15.2:0/45035340 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f55ac038450 0x7f55ac03a910 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:36.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:36.125+0000 7f55ca551700 1 --2- 172.21.15.2:0/45035340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c40ff0c0 0x7f55c4197be0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:36.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:36.125+0000 7f55ca551700 1 -- 172.21.15.2:0/45035340 >> 172.21.15.2:0/45035340 conn(0x7f55c40fac80 msgr2=0x7f55c40fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:36.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:36.125+0000 7f55ca551700 1 -- 172.21.15.2:0/45035340 shutdown_connections 2023-09-13T04:31:36.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:36.125+0000 7f55ca551700 1 -- 172.21.15.2:0/45035340 wait complete. 2023-09-13T04:31:36.600 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-09-13T04:31:36.600 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-09-13T04:31:37.488 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:31:37.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:37 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/45035340' entity='client.admin' 2023-09-13T04:31:37.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:37 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:38.335 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:38 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:38.336 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:38 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi002", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:31:38.336 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:38 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:38.336 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:38 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:31:38.336 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:38 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:38.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.988+0000 7ffad0d2e700 1 -- 172.21.15.2:0/2884336876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffacc074ce0 msgr2=0x7ffacc073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:38.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.988+0000 7ffad0d2e700 1 --2- 172.21.15.2:0/2884336876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffacc074ce0 0x7ffacc073140 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009910 tx=0x7ffabc009c20 comp rx=0 tx=0).stop 2023-09-13T04:31:38.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.989+0000 7ffad0d2e700 1 -- 172.21.15.2:0/2884336876 shutdown_connections 2023-09-13T04:31:38.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.989+0000 7ffad0d2e700 1 --2- 172.21.15.2:0/2884336876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffacc074ce0 0x7ffacc073140 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:38.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.989+0000 7ffad0d2e700 1 -- 172.21.15.2:0/2884336876 >> 172.21.15.2:0/2884336876 conn(0x7ffacc0fbd80 msgr2=0x7ffacc0fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:38.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.989+0000 7ffad0d2e700 1 -- 172.21.15.2:0/2884336876 shutdown_connections 2023-09-13T04:31:38.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.989+0000 7ffad0d2e700 1 -- 172.21.15.2:0/2884336876 wait complete. 2023-09-13T04:31:38.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.990+0000 7ffad0d2e700 1 Processor -- start 2023-09-13T04:31:38.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.990+0000 7ffad0d2e700 1 -- start start 2023-09-13T04:31:38.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.990+0000 7ffad0d2e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffacc074ce0 0x7ffacc197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:38.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.990+0000 7ffad0d2e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc198060 con 0x7ffacc074ce0 2023-09-13T04:31:38.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.991+0000 7ffaca59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffacc074ce0 0x7ffacc197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:38.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.991+0000 7ffaca59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffacc074ce0 0x7ffacc197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42938/0 (socket says 172.21.15.2:42938) 2023-09-13T04:31:38.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.991+0000 7ffaca59c700 1 -- 172.21.15.2:0/4193911839 learned_addr learned my addr 172.21.15.2:0/4193911839 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:38.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.991+0000 7ffaca59c700 1 -- 172.21.15.2:0/4193911839 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffabc0095f0 con 0x7ffacc074ce0 2023-09-13T04:31:38.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.991+0000 7ffaca59c700 1 --2- 172.21.15.2:0/4193911839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffacc074ce0 0x7ffacc197b20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffabc004f40 tx=0x7ffabc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:38.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.991+0000 7ffabb7fe700 1 -- 172.21.15.2:0/4193911839 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffabc01d070 con 0x7ffacc074ce0 2023-09-13T04:31:38.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.991+0000 7ffabb7fe700 1 -- 172.21.15.2:0/4193911839 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffabc00bc10 con 0x7ffacc074ce0 2023-09-13T04:31:38.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.991+0000 7ffabb7fe700 1 -- 172.21.15.2:0/4193911839 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffabc003a70 con 0x7ffacc074ce0 2023-09-13T04:31:38.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.991+0000 7ffad0d2e700 1 -- 172.21.15.2:0/4193911839 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffacc198260 con 0x7ffacc074ce0 2023-09-13T04:31:38.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.992+0000 7ffad0d2e700 1 -- 172.21.15.2:0/4193911839 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffacc198700 con 0x7ffacc074ce0 2023-09-13T04:31:38.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.993+0000 7ffabb7fe700 1 -- 172.21.15.2:0/4193911839 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45386+0+0 (secure 0 0 0) 0x7ffabc003ce0 con 0x7ffacc074ce0 2023-09-13T04:31:38.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.993+0000 7ffad0d2e700 1 -- 172.21.15.2:0/4193911839 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffacc191520 con 0x7ffacc074ce0 2023-09-13T04:31:38.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.993+0000 7ffabb7fe700 1 --2- 172.21.15.2:0/4193911839 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ffab4038450 0x7ffab403a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:38.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.993+0000 7ffabb7fe700 1 -- 172.21.15.2:0/4193911839 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ffabc04c020 con 0x7ffacc074ce0 2023-09-13T04:31:38.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.993+0000 7ffac9d9b700 1 --2- 172.21.15.2:0/4193911839 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ffab4038450 0x7ffab403a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:38.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.994+0000 7ffac9d9b700 1 --2- 172.21.15.2:0/4193911839 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ffab4038450 0x7ffab403a910 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffac0006fd0 tx=0x7ffac0006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:38.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:38.998+0000 7ffabb7fe700 1 -- 172.21.15.2:0/4193911839 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ffabc07ee10 con 0x7ffacc074ce0 2023-09-13T04:31:39.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:39.215+0000 7ffad0d2e700 1 -- 172.21.15.2:0/4193911839 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7ffacc02cf60 con 0x7ffab4038450 2023-09-13T04:31:39.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:39.219+0000 7ffabb7fe700 1 -- 172.21.15.2:0/4193911839 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ffacc02cf60 con 0x7ffab4038450 2023-09-13T04:31:39.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:39.222+0000 7ffad0d2e700 1 -- 172.21.15.2:0/4193911839 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ffab4038450 msgr2=0x7ffab403a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:39.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:39.222+0000 7ffad0d2e700 1 --2- 172.21.15.2:0/4193911839 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ffab4038450 0x7ffab403a910 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffac0006fd0 tx=0x7ffac0006e40 comp rx=0 tx=0).stop 2023-09-13T04:31:39.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:39.222+0000 7ffad0d2e700 1 -- 172.21.15.2:0/4193911839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffacc074ce0 msgr2=0x7ffacc197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:39.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:39.222+0000 7ffad0d2e700 1 --2- 172.21.15.2:0/4193911839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffacc074ce0 0x7ffacc197b20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffabc004f40 tx=0x7ffabc0057c0 comp rx=0 tx=0).stop 2023-09-13T04:31:39.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:39.222+0000 7ffad0d2e700 1 -- 172.21.15.2:0/4193911839 shutdown_connections 2023-09-13T04:31:39.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:39.222+0000 7ffad0d2e700 1 --2- 172.21.15.2:0/4193911839 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ffab4038450 0x7ffab403a910 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:39.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:39.222+0000 7ffad0d2e700 1 --2- 172.21.15.2:0/4193911839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffacc074ce0 0x7ffacc197b20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:39.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:39.222+0000 7ffad0d2e700 1 -- 172.21.15.2:0/4193911839 >> 172.21.15.2:0/4193911839 conn(0x7ffacc0fbd80 msgr2=0x7ffacc0fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:39.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:39.222+0000 7ffad0d2e700 1 -- 172.21.15.2:0/4193911839 shutdown_connections 2023-09-13T04:31:39.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:39.222+0000 7ffad0d2e700 1 -- 172.21.15.2:0/4193911839 wait complete. 2023-09-13T04:31:39.765 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi149 2023-09-13T04:31:39.766 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2023-09-13T04:31:39.766 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.conf 2023-09-13T04:31:39.787 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2023-09-13T04:31:39.787 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-13T04:31:39.846 INFO:tasks.cephadm:Adding host smithi149 to orchestrator... 2023-09-13T04:31:39.846 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph orch host add smithi149 2023-09-13T04:31:40.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:40 smithi002 ceph-mon[97651]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:31:40.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:40 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:40.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:40 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:31:40.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:40 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:31:40.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:40 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:31:40.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:40 smithi002 ceph-mon[97651]: Updating smithi002:/etc/ceph/ceph.conf 2023-09-13T04:31:40.605 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:31:41.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:41 smithi002 ceph-mon[97651]: Updating smithi002:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:31:41.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:41 smithi002 ceph-mon[97651]: Updating smithi002:/etc/ceph/ceph.client.admin.keyring 2023-09-13T04:31:42.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.244+0000 7fd59e560700 1 -- 172.21.15.2:0/761932614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5980730e0 msgr2=0x7fd5980734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:42.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.244+0000 7fd59e560700 1 --2- 172.21.15.2:0/761932614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5980730e0 0x7fd5980734c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd588009910 tx=0x7fd588009c20 comp rx=0 tx=0).stop 2023-09-13T04:31:42.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.244+0000 7fd59e560700 1 -- 172.21.15.2:0/761932614 shutdown_connections 2023-09-13T04:31:42.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.244+0000 7fd59e560700 1 --2- 172.21.15.2:0/761932614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5980730e0 0x7fd5980734c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:42.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.244+0000 7fd59e560700 1 -- 172.21.15.2:0/761932614 >> 172.21.15.2:0/761932614 conn(0x7fd5980fc2a0 msgr2=0x7fd5980fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:42.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.244+0000 7fd59e560700 1 -- 172.21.15.2:0/761932614 shutdown_connections 2023-09-13T04:31:42.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.244+0000 7fd59e560700 1 -- 172.21.15.2:0/761932614 wait complete. 2023-09-13T04:31:42.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.245+0000 7fd59e560700 1 Processor -- start 2023-09-13T04:31:42.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.245+0000 7fd59e560700 1 -- start start 2023-09-13T04:31:42.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.245+0000 7fd59e560700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5980730e0 0x7fd598197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:42.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.245+0000 7fd59e560700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5981983d0 con 0x7fd5980730e0 2023-09-13T04:31:42.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.246+0000 7fd597fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5980730e0 0x7fd598197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:42.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.246+0000 7fd597fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5980730e0 0x7fd598197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42952/0 (socket says 172.21.15.2:42952) 2023-09-13T04:31:42.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.246+0000 7fd597fff700 1 -- 172.21.15.2:0/3252036016 learned_addr learned my addr 172.21.15.2:0/3252036016 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:42.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.246+0000 7fd597fff700 1 -- 172.21.15.2:0/3252036016 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5880095f0 con 0x7fd5980730e0 2023-09-13T04:31:42.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.246+0000 7fd597fff700 1 --2- 172.21.15.2:0/3252036016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5980730e0 0x7fd598197e90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd588006010 tx=0x7fd588004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:42.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.246+0000 7fd5957fa700 1 -- 172.21.15.2:0/3252036016 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd58801d070 con 0x7fd5980730e0 2023-09-13T04:31:42.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.247+0000 7fd5957fa700 1 -- 172.21.15.2:0/3252036016 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5880044a0 con 0x7fd5980730e0 2023-09-13T04:31:42.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.247+0000 7fd5957fa700 1 -- 172.21.15.2:0/3252036016 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd588003a70 con 0x7fd5980730e0 2023-09-13T04:31:42.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.247+0000 7fd59e560700 1 -- 172.21.15.2:0/3252036016 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5981985d0 con 0x7fd5980730e0 2023-09-13T04:31:42.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.247+0000 7fd59e560700 1 -- 172.21.15.2:0/3252036016 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5981989f0 con 0x7fd5980730e0 2023-09-13T04:31:42.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.248+0000 7fd59e560700 1 -- 172.21.15.2:0/3252036016 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5980622e0 con 0x7fd5980730e0 2023-09-13T04:31:42.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.248+0000 7fd5957fa700 1 -- 172.21.15.2:0/3252036016 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45386+0+0 (secure 0 0 0) 0x7fd588003c60 con 0x7fd5980730e0 2023-09-13T04:31:42.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.248+0000 7fd5957fa700 1 --2- 172.21.15.2:0/3252036016 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd580038450 0x7fd58003a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:42.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.248+0000 7fd5957fa700 1 -- 172.21.15.2:0/3252036016 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd58804c2c0 con 0x7fd5980730e0 2023-09-13T04:31:42.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.249+0000 7fd5977fe700 1 --2- 172.21.15.2:0/3252036016 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd580038450 0x7fd58003a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:42.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.249+0000 7fd5977fe700 1 --2- 172.21.15.2:0/3252036016 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd580038450 0x7fd58003a910 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd58c006fd0 tx=0x7fd58c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:42.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.254+0000 7fd5957fa700 1 -- 172.21.15.2:0/3252036016 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd588029360 con 0x7fd5980730e0 2023-09-13T04:31:42.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:42.469+0000 7fd59e560700 1 -- 172.21.15.2:0/3252036016 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}) v1 -- 0x7fd59810ce50 con 0x7fd580038450 2023-09-13T04:31:42.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:42 smithi002 ceph-mon[97651]: Updating smithi002:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T04:31:42.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:42 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:42.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:42 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:42.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:42 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:43.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:43 smithi002 ceph-mon[97651]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:31:44.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:44 smithi002 ceph-mon[97651]: Deploying cephadm binary to smithi149 2023-09-13T04:31:44.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:44 smithi002 ceph-mon[97651]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:31:44.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:44.913+0000 7fd5957fa700 1 -- 172.21.15.2:0/3252036016 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fd59810ce50 con 0x7fd580038450 2023-09-13T04:31:44.915 INFO:teuthology.orchestra.run.smithi002.stdout:Added host 'smithi149' with addr '172.21.15.149' 2023-09-13T04:31:44.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:44.915+0000 7fd59e560700 1 -- 172.21.15.2:0/3252036016 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd580038450 msgr2=0x7fd58003a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:44.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:44.915+0000 7fd59e560700 1 --2- 172.21.15.2:0/3252036016 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd580038450 0x7fd58003a910 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd58c006fd0 tx=0x7fd58c006e40 comp rx=0 tx=0).stop 2023-09-13T04:31:44.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:44.915+0000 7fd59e560700 1 -- 172.21.15.2:0/3252036016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5980730e0 msgr2=0x7fd598197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:44.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:44.915+0000 7fd59e560700 1 --2- 172.21.15.2:0/3252036016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5980730e0 0x7fd598197e90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd588006010 tx=0x7fd588004ff0 comp rx=0 tx=0).stop 2023-09-13T04:31:44.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:44.915+0000 7fd59e560700 1 -- 172.21.15.2:0/3252036016 shutdown_connections 2023-09-13T04:31:44.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:44.915+0000 7fd59e560700 1 --2- 172.21.15.2:0/3252036016 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd580038450 0x7fd58003a910 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:44.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:44.915+0000 7fd59e560700 1 --2- 172.21.15.2:0/3252036016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5980730e0 0x7fd598197e90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:44.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:44.915+0000 7fd59e560700 1 -- 172.21.15.2:0/3252036016 >> 172.21.15.2:0/3252036016 conn(0x7fd5980fc2a0 msgr2=0x7fd5981072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:44.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:44.916+0000 7fd59e560700 1 -- 172.21.15.2:0/3252036016 shutdown_connections 2023-09-13T04:31:44.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:44.916+0000 7fd59e560700 1 -- 172.21.15.2:0/3252036016 wait complete. 2023-09-13T04:31:45.409 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph orch host ls --format=json 2023-09-13T04:31:46.140 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:31:46.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:45 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:46.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:45 smithi002 ceph-mon[97651]: Added host smithi149 2023-09-13T04:31:46.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:45 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:31:46.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:45 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:46.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:45 smithi002 ceph-mon[97651]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:31:47.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:47 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:47.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.759+0000 7f01ccc9e700 1 -- 172.21.15.2:0/3118847340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01c8106750 msgr2=0x7f01c8106b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:47.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.759+0000 7f01ccc9e700 1 --2- 172.21.15.2:0/3118847340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01c8106750 0x7f01c8106b30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f01b8009910 tx=0x7f01b8009c20 comp rx=0 tx=0).stop 2023-09-13T04:31:47.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.759+0000 7f01ccc9e700 1 -- 172.21.15.2:0/3118847340 shutdown_connections 2023-09-13T04:31:47.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.759+0000 7f01ccc9e700 1 --2- 172.21.15.2:0/3118847340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01c8106750 0x7f01c8106b30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:47.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.759+0000 7f01ccc9e700 1 -- 172.21.15.2:0/3118847340 >> 172.21.15.2:0/3118847340 conn(0x7f01c80fc240 msgr2=0x7f01c80fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:47.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.759+0000 7f01ccc9e700 1 -- 172.21.15.2:0/3118847340 shutdown_connections 2023-09-13T04:31:47.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.759+0000 7f01ccc9e700 1 -- 172.21.15.2:0/3118847340 wait complete. 2023-09-13T04:31:47.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.761+0000 7f01ccc9e700 1 Processor -- start 2023-09-13T04:31:47.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.761+0000 7f01ccc9e700 1 -- start start 2023-09-13T04:31:47.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.761+0000 7f01ccc9e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01c8106750 0x7f01c819bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:47.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.761+0000 7f01ccc9e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c819c060 con 0x7f01c8106750 2023-09-13T04:31:47.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.762+0000 7f01c659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01c8106750 0x7f01c819bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:47.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.762+0000 7f01c659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01c8106750 0x7f01c819bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47424/0 (socket says 172.21.15.2:47424) 2023-09-13T04:31:47.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.762+0000 7f01c659c700 1 -- 172.21.15.2:0/4093455595 learned_addr learned my addr 172.21.15.2:0/4093455595 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:47.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.762+0000 7f01c659c700 1 -- 172.21.15.2:0/4093455595 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b80095f0 con 0x7f01c8106750 2023-09-13T04:31:47.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.762+0000 7f01c659c700 1 --2- 172.21.15.2:0/4093455595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01c8106750 0x7f01c819bb20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f01b8006010 tx=0x7f01b8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:47.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.762+0000 7f01b77fe700 1 -- 172.21.15.2:0/4093455595 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01b801d070 con 0x7f01c8106750 2023-09-13T04:31:47.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.763+0000 7f01b77fe700 1 -- 172.21.15.2:0/4093455595 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01b800bbb0 con 0x7f01c8106750 2023-09-13T04:31:47.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.763+0000 7f01ccc9e700 1 -- 172.21.15.2:0/4093455595 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c819c2c0 con 0x7f01c8106750 2023-09-13T04:31:47.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.763+0000 7f01b77fe700 1 -- 172.21.15.2:0/4093455595 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01b8003930 con 0x7f01c8106750 2023-09-13T04:31:47.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.763+0000 7f01ccc9e700 1 -- 172.21.15.2:0/4093455595 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c8195b10 con 0x7f01c8106750 2023-09-13T04:31:47.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.764+0000 7f01ccc9e700 1 -- 172.21.15.2:0/4093455595 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01c80622e0 con 0x7f01c8106750 2023-09-13T04:31:47.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.764+0000 7f01b77fe700 1 -- 172.21.15.2:0/4093455595 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45386+0+0 (secure 0 0 0) 0x7f01b800bd20 con 0x7f01c8106750 2023-09-13T04:31:47.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.764+0000 7f01b77fe700 1 --2- 172.21.15.2:0/4093455595 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f01b0038400 0x7f01b003a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:47.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.764+0000 7f01b77fe700 1 -- 172.21.15.2:0/4093455595 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f01b804c170 con 0x7f01c8106750 2023-09-13T04:31:47.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.764+0000 7f01c5d9b700 1 --2- 172.21.15.2:0/4093455595 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f01b0038400 0x7f01b003a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:47.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.765+0000 7f01c5d9b700 1 --2- 172.21.15.2:0/4093455595 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f01b0038400 0x7f01b003a8c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f01bc006fd0 tx=0x7f01bc006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:47.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.769+0000 7f01b77fe700 1 -- 172.21.15.2:0/4093455595 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f01b8029360 con 0x7f01c8106750 2023-09-13T04:31:47.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.985+0000 7f01ccc9e700 1 -- 172.21.15.2:0/4093455595 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f01c810aac0 con 0x7f01b0038400 2023-09-13T04:31:47.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.986+0000 7f01b77fe700 1 -- 172.21.15.2:0/4093455595 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f01c810aac0 con 0x7f01b0038400 2023-09-13T04:31:47.988 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:31:47.988 INFO:teuthology.orchestra.run.smithi002.stdout:[{"addr": "172.21.15.2", "hostname": "smithi002", "labels": [], "status": ""}, {"addr": "172.21.15.149", "hostname": "smithi149", "labels": [], "status": ""}] 2023-09-13T04:31:47.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.988+0000 7f01ccc9e700 1 -- 172.21.15.2:0/4093455595 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f01b0038400 msgr2=0x7f01b003a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:47.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.988+0000 7f01ccc9e700 1 --2- 172.21.15.2:0/4093455595 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f01b0038400 0x7f01b003a8c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f01bc006fd0 tx=0x7f01bc006e40 comp rx=0 tx=0).stop 2023-09-13T04:31:47.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.988+0000 7f01ccc9e700 1 -- 172.21.15.2:0/4093455595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01c8106750 msgr2=0x7f01c819bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:47.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.988+0000 7f01ccc9e700 1 --2- 172.21.15.2:0/4093455595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01c8106750 0x7f01c819bb20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f01b8006010 tx=0x7f01b8004ff0 comp rx=0 tx=0).stop 2023-09-13T04:31:47.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.988+0000 7f01ccc9e700 1 -- 172.21.15.2:0/4093455595 shutdown_connections 2023-09-13T04:31:47.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.988+0000 7f01ccc9e700 1 --2- 172.21.15.2:0/4093455595 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f01b0038400 0x7f01b003a8c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:47.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.988+0000 7f01ccc9e700 1 --2- 172.21.15.2:0/4093455595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01c8106750 0x7f01c819bb20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:47.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.989+0000 7f01ccc9e700 1 -- 172.21.15.2:0/4093455595 >> 172.21.15.2:0/4093455595 conn(0x7f01c80fc240 msgr2=0x7f01c8108be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:47.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.989+0000 7f01ccc9e700 1 -- 172.21.15.2:0/4093455595 shutdown_connections 2023-09-13T04:31:47.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:47.989+0000 7f01ccc9e700 1 -- 172.21.15.2:0/4093455595 wait complete. 2023-09-13T04:31:48.517 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi156 2023-09-13T04:31:48.517 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-13T04:31:48.518 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/etc/ceph/ceph.conf 2023-09-13T04:31:48.539 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-13T04:31:48.539 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-13T04:31:48.599 INFO:tasks.cephadm:Adding host smithi156 to orchestrator... 2023-09-13T04:31:48.599 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph orch host add smithi156 2023-09-13T04:31:48.663 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:48 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:48.663 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:48 smithi002 ceph-mon[97651]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:31:49.329 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:31:49.645 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:49 smithi002 ceph-mon[97651]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:31:50.588 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:50 smithi002 ceph-mon[97651]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:31:50.588 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:50 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:50.588 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:50 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:50.589 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:50 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:50.589 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:50 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:50.589 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:50 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:31:50.589 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:50 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:31:50.589 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:50 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:31:50.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.843+0000 7f3e6c6d8700 1 -- 172.21.15.2:0/4089798814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e640730e0 msgr2=0x7f3e640734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.843+0000 7f3e6c6d8700 1 --2- 172.21.15.2:0/4089798814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e640730e0 0x7f3e640734c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3e60009910 tx=0x7f3e60009c20 comp rx=0 tx=0).stop 2023-09-13T04:31:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.843+0000 7f3e6c6d8700 1 -- 172.21.15.2:0/4089798814 shutdown_connections 2023-09-13T04:31:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.843+0000 7f3e6c6d8700 1 --2- 172.21.15.2:0/4089798814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e640730e0 0x7f3e640734c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.843+0000 7f3e6c6d8700 1 -- 172.21.15.2:0/4089798814 >> 172.21.15.2:0/4089798814 conn(0x7f3e640fc2a0 msgr2=0x7f3e640fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.843+0000 7f3e6c6d8700 1 -- 172.21.15.2:0/4089798814 shutdown_connections 2023-09-13T04:31:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.843+0000 7f3e6c6d8700 1 -- 172.21.15.2:0/4089798814 wait complete. 2023-09-13T04:31:50.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.844+0000 7f3e6c6d8700 1 Processor -- start 2023-09-13T04:31:50.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.844+0000 7f3e6c6d8700 1 -- start start 2023-09-13T04:31:50.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.845+0000 7f3e6c6d8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e640730e0 0x7f3e64197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:50.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.845+0000 7f3e6c6d8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e641983d0 con 0x7f3e640730e0 2023-09-13T04:31:50.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.845+0000 7f3e6a474700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e640730e0 0x7f3e64197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:50.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.845+0000 7f3e6a474700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e640730e0 0x7f3e64197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47448/0 (socket says 172.21.15.2:47448) 2023-09-13T04:31:50.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.845+0000 7f3e6a474700 1 -- 172.21.15.2:0/2788547905 learned_addr learned my addr 172.21.15.2:0/2788547905 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:50.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.845+0000 7f3e6a474700 1 -- 172.21.15.2:0/2788547905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e600095f0 con 0x7f3e640730e0 2023-09-13T04:31:50.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.846+0000 7f3e6a474700 1 --2- 172.21.15.2:0/2788547905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e640730e0 0x7f3e64197e90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3e60006010 tx=0x7f3e60004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:50.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.846+0000 7f3e5b7fe700 1 -- 172.21.15.2:0/2788547905 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e6001d070 con 0x7f3e640730e0 2023-09-13T04:31:50.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.846+0000 7f3e6c6d8700 1 -- 172.21.15.2:0/2788547905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e641985d0 con 0x7f3e640730e0 2023-09-13T04:31:50.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.846+0000 7f3e5b7fe700 1 -- 172.21.15.2:0/2788547905 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e600044a0 con 0x7f3e640730e0 2023-09-13T04:31:50.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.846+0000 7f3e5b7fe700 1 -- 172.21.15.2:0/2788547905 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e60003a70 con 0x7f3e640730e0 2023-09-13T04:31:50.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.846+0000 7f3e6c6d8700 1 -- 172.21.15.2:0/2788547905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e641989f0 con 0x7f3e640730e0 2023-09-13T04:31:50.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.847+0000 7f3e5b7fe700 1 -- 172.21.15.2:0/2788547905 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45386+0+0 (secure 0 0 0) 0x7f3e60003cb0 con 0x7f3e640730e0 2023-09-13T04:31:50.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.847+0000 7f3e6c6d8700 1 -- 172.21.15.2:0/2788547905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e640622e0 con 0x7f3e640730e0 2023-09-13T04:31:50.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.848+0000 7f3e5b7fe700 1 --2- 172.21.15.2:0/2788547905 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f3e50038400 0x7f3e5003a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:50.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.848+0000 7f3e5b7fe700 1 -- 172.21.15.2:0/2788547905 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3e6004c1e0 con 0x7f3e640730e0 2023-09-13T04:31:50.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.848+0000 7f3e69c73700 1 --2- 172.21.15.2:0/2788547905 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f3e50038400 0x7f3e5003a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:50.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.849+0000 7f3e69c73700 1 --2- 172.21.15.2:0/2788547905 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f3e50038400 0x7f3e5003a8c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3e54006fd0 tx=0x7f3e54006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:50.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:50.853+0000 7f3e5b7fe700 1 -- 172.21.15.2:0/2788547905 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3e60029540 con 0x7f3e640730e0 2023-09-13T04:31:51.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:51.068+0000 7f3e6c6d8700 1 -- 172.21.15.2:0/2788547905 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e6410ce50 con 0x7f3e50038400 2023-09-13T04:31:51.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:51 smithi002 ceph-mon[97651]: Updating smithi149:/etc/ceph/ceph.conf 2023-09-13T04:31:51.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:51 smithi002 ceph-mon[97651]: Updating smithi149:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:31:51.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:51 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:51.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:51 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:51.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:51 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:52.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:52 smithi002 ceph-mon[97651]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2023-09-13T04:31:52.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:52 smithi002 ceph-mon[97651]: Updating smithi149:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T04:31:52.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:52 smithi002 ceph-mon[97651]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:31:52.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:52 smithi002 ceph-mon[97651]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:31:52.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:52 smithi002 ceph-mon[97651]: Deploying cephadm binary to smithi156 2023-09-13T04:31:53.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:53.480+0000 7f3e5b7fe700 1 -- 172.21.15.2:0/2788547905 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f3e6410ce50 con 0x7f3e50038400 2023-09-13T04:31:53.482 INFO:teuthology.orchestra.run.smithi002.stdout:Added host 'smithi156' with addr '172.21.15.156' 2023-09-13T04:31:53.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:53.483+0000 7f3e6c6d8700 1 -- 172.21.15.2:0/2788547905 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f3e50038400 msgr2=0x7f3e5003a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:53.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:53.483+0000 7f3e6c6d8700 1 --2- 172.21.15.2:0/2788547905 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f3e50038400 0x7f3e5003a8c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3e54006fd0 tx=0x7f3e54006e40 comp rx=0 tx=0).stop 2023-09-13T04:31:53.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:53.483+0000 7f3e6c6d8700 1 -- 172.21.15.2:0/2788547905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e640730e0 msgr2=0x7f3e64197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:53.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:53.483+0000 7f3e6c6d8700 1 --2- 172.21.15.2:0/2788547905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e640730e0 0x7f3e64197e90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3e60006010 tx=0x7f3e60004ff0 comp rx=0 tx=0).stop 2023-09-13T04:31:53.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:53.483+0000 7f3e6c6d8700 1 -- 172.21.15.2:0/2788547905 shutdown_connections 2023-09-13T04:31:53.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:53.483+0000 7f3e6c6d8700 1 --2- 172.21.15.2:0/2788547905 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f3e50038400 0x7f3e5003a8c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:53.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:53.483+0000 7f3e6c6d8700 1 --2- 172.21.15.2:0/2788547905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e640730e0 0x7f3e64197e90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:53.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:53.483+0000 7f3e6c6d8700 1 -- 172.21.15.2:0/2788547905 >> 172.21.15.2:0/2788547905 conn(0x7f3e640fc2a0 msgr2=0x7f3e641072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:53.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:53.483+0000 7f3e6c6d8700 1 -- 172.21.15.2:0/2788547905 shutdown_connections 2023-09-13T04:31:53.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:53.483+0000 7f3e6c6d8700 1 -- 172.21.15.2:0/2788547905 wait complete. 2023-09-13T04:31:53.916 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph orch host ls --format=json 2023-09-13T04:31:54.651 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:31:54.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:54 smithi002 ceph-mon[97651]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:31:54.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:54 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:54.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:54 smithi002 ceph-mon[97651]: Added host smithi156 2023-09-13T04:31:54.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:54 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:31:54.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:54 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:56.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:55 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:56.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.391+0000 7f3b3540e700 1 -- 172.21.15.2:0/1068186296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b30106750 msgr2=0x7f3b30106b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:56.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.391+0000 7f3b3540e700 1 --2- 172.21.15.2:0/1068186296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b30106750 0x7f3b30106b30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3b20009910 tx=0x7f3b20009c20 comp rx=0 tx=0).stop 2023-09-13T04:31:56.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.391+0000 7f3b3540e700 1 -- 172.21.15.2:0/1068186296 shutdown_connections 2023-09-13T04:31:56.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.391+0000 7f3b3540e700 1 --2- 172.21.15.2:0/1068186296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b30106750 0x7f3b30106b30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:56.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.391+0000 7f3b3540e700 1 -- 172.21.15.2:0/1068186296 >> 172.21.15.2:0/1068186296 conn(0x7f3b300fc240 msgr2=0x7f3b300fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:56.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.391+0000 7f3b3540e700 1 -- 172.21.15.2:0/1068186296 shutdown_connections 2023-09-13T04:31:56.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.391+0000 7f3b3540e700 1 -- 172.21.15.2:0/1068186296 wait complete. 2023-09-13T04:31:56.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.392+0000 7f3b3540e700 1 Processor -- start 2023-09-13T04:31:56.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.392+0000 7f3b3540e700 1 -- start start 2023-09-13T04:31:56.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.393+0000 7f3b3540e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b30106750 0x7f3b30197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:56.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.393+0000 7f3b3540e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b30198330 con 0x7f3b30106750 2023-09-13T04:31:56.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.393+0000 7f3b2effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b30106750 0x7f3b30197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:56.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.393+0000 7f3b2effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b30106750 0x7f3b30197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59840/0 (socket says 172.21.15.2:59840) 2023-09-13T04:31:56.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.393+0000 7f3b2effd700 1 -- 172.21.15.2:0/3508527198 learned_addr learned my addr 172.21.15.2:0/3508527198 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:56.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.393+0000 7f3b2effd700 1 -- 172.21.15.2:0/3508527198 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b200095f0 con 0x7f3b30106750 2023-09-13T04:31:56.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.393+0000 7f3b2effd700 1 --2- 172.21.15.2:0/3508527198 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b30106750 0x7f3b30197df0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3b20004f40 tx=0x7f3b20005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:56.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.394+0000 7f3b17fff700 1 -- 172.21.15.2:0/3508527198 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b2001d070 con 0x7f3b30106750 2023-09-13T04:31:56.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.394+0000 7f3b17fff700 1 -- 172.21.15.2:0/3508527198 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b20004370 con 0x7f3b30106750 2023-09-13T04:31:56.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.394+0000 7f3b17fff700 1 -- 172.21.15.2:0/3508527198 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b20003970 con 0x7f3b30106750 2023-09-13T04:31:56.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.394+0000 7f3b3540e700 1 -- 172.21.15.2:0/3508527198 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b30198590 con 0x7f3b30106750 2023-09-13T04:31:56.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.394+0000 7f3b3540e700 1 -- 172.21.15.2:0/3508527198 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b30198a10 con 0x7f3b30106750 2023-09-13T04:31:56.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.395+0000 7f3b17fff700 1 -- 172.21.15.2:0/3508527198 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45386+0+0 (secure 0 0 0) 0x7f3b20003be0 con 0x7f3b30106750 2023-09-13T04:31:56.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.395+0000 7f3b3540e700 1 -- 172.21.15.2:0/3508527198 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b300622e0 con 0x7f3b30106750 2023-09-13T04:31:56.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.395+0000 7f3b17fff700 1 --2- 172.21.15.2:0/3508527198 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f3b18038450 0x7f3b1803a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:56.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.395+0000 7f3b17fff700 1 -- 172.21.15.2:0/3508527198 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3b2004c090 con 0x7f3b30106750 2023-09-13T04:31:56.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.396+0000 7f3b2e7fc700 1 --2- 172.21.15.2:0/3508527198 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f3b18038450 0x7f3b1803a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:56.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.396+0000 7f3b2e7fc700 1 --2- 172.21.15.2:0/3508527198 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f3b18038450 0x7f3b1803a910 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3b24006fd0 tx=0x7f3b24006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:56.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.401+0000 7f3b17fff700 1 -- 172.21.15.2:0/3508527198 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3b2007ee10 con 0x7f3b30106750 2023-09-13T04:31:56.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.613+0000 7f3b3540e700 1 -- 172.21.15.2:0/3508527198 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3b300008d0 con 0x7f3b18038450 2023-09-13T04:31:56.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.614+0000 7f3b17fff700 1 -- 172.21.15.2:0/3508527198 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7f3b300008d0 con 0x7f3b18038450 2023-09-13T04:31:56.615 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:31:56.615 INFO:teuthology.orchestra.run.smithi002.stdout:[{"addr": "172.21.15.2", "hostname": "smithi002", "labels": [], "status": ""}, {"addr": "172.21.15.149", "hostname": "smithi149", "labels": [], "status": ""}, {"addr": "172.21.15.156", "hostname": "smithi156", "labels": [], "status": ""}] 2023-09-13T04:31:56.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.617+0000 7f3b3540e700 1 -- 172.21.15.2:0/3508527198 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f3b18038450 msgr2=0x7f3b1803a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:56.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.617+0000 7f3b3540e700 1 --2- 172.21.15.2:0/3508527198 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f3b18038450 0x7f3b1803a910 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3b24006fd0 tx=0x7f3b24006e40 comp rx=0 tx=0).stop 2023-09-13T04:31:56.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.617+0000 7f3b3540e700 1 -- 172.21.15.2:0/3508527198 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b30106750 msgr2=0x7f3b30197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:56.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.617+0000 7f3b3540e700 1 --2- 172.21.15.2:0/3508527198 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b30106750 0x7f3b30197df0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3b20004f40 tx=0x7f3b20005020 comp rx=0 tx=0).stop 2023-09-13T04:31:56.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.617+0000 7f3b3540e700 1 -- 172.21.15.2:0/3508527198 shutdown_connections 2023-09-13T04:31:56.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.617+0000 7f3b3540e700 1 --2- 172.21.15.2:0/3508527198 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f3b18038450 0x7f3b1803a910 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:56.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.617+0000 7f3b3540e700 1 --2- 172.21.15.2:0/3508527198 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b30106750 0x7f3b30197df0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:56.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.617+0000 7f3b3540e700 1 -- 172.21.15.2:0/3508527198 >> 172.21.15.2:0/3508527198 conn(0x7f3b300fc240 msgr2=0x7f3b30108b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:56.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.617+0000 7f3b3540e700 1 -- 172.21.15.2:0/3508527198 shutdown_connections 2023-09-13T04:31:56.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:56.617+0000 7f3b3540e700 1 -- 172.21.15.2:0/3508527198 wait complete. 2023-09-13T04:31:57.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:56 smithi002 ceph-mon[97651]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:31:57.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:56 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:31:57.141 INFO:tasks.cephadm:Setting crush tunables to default 2023-09-13T04:31:57.141 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph osd crush tunables default 2023-09-13T04:31:57.869 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:31:58.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:57 smithi002 ceph-mon[97651]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:31:58.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:58 smithi002 ceph-mon[97651]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:31:59.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.354+0000 7fe2ac9b0700 1 -- 172.21.15.2:0/2118694673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2a8106770 msgr2=0x7fe2a8106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:31:59.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.354+0000 7fe2ac9b0700 1 --2- 172.21.15.2:0/2118694673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2a8106770 0x7fe2a8106b50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe298009910 tx=0x7fe298009c20 comp rx=0 tx=0).stop 2023-09-13T04:31:59.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.354+0000 7fe2ac9b0700 1 -- 172.21.15.2:0/2118694673 shutdown_connections 2023-09-13T04:31:59.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.354+0000 7fe2ac9b0700 1 --2- 172.21.15.2:0/2118694673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2a8106770 0x7fe2a8106b50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:31:59.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.354+0000 7fe2ac9b0700 1 -- 172.21.15.2:0/2118694673 >> 172.21.15.2:0/2118694673 conn(0x7fe2a80751b0 msgr2=0x7fe2a80755c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:31:59.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.354+0000 7fe2ac9b0700 1 -- 172.21.15.2:0/2118694673 shutdown_connections 2023-09-13T04:31:59.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.354+0000 7fe2ac9b0700 1 -- 172.21.15.2:0/2118694673 wait complete. 2023-09-13T04:31:59.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.355+0000 7fe2ac9b0700 1 Processor -- start 2023-09-13T04:31:59.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.356+0000 7fe2ac9b0700 1 -- start start 2023-09-13T04:31:59.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.356+0000 7fe2ac9b0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2a8106770 0x7fe2a80ff780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:59.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.356+0000 7fe2ac9b0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a8101700 con 0x7fe2a8106770 2023-09-13T04:31:59.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.356+0000 7fe2a659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2a8106770 0x7fe2a80ff780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:59.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.356+0000 7fe2a659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2a8106770 0x7fe2a80ff780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59864/0 (socket says 172.21.15.2:59864) 2023-09-13T04:31:59.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.356+0000 7fe2a659c700 1 -- 172.21.15.2:0/3106530642 learned_addr learned my addr 172.21.15.2:0/3106530642 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:31:59.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.357+0000 7fe2a659c700 1 -- 172.21.15.2:0/3106530642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2980095f0 con 0x7fe2a8106770 2023-09-13T04:31:59.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.357+0000 7fe2a659c700 1 --2- 172.21.15.2:0/3106530642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2a8106770 0x7fe2a80ff780 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe298006010 tx=0x7fe298004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:59.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.357+0000 7fe2977fe700 1 -- 172.21.15.2:0/3106530642 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe29801d070 con 0x7fe2a8106770 2023-09-13T04:31:59.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.357+0000 7fe2ac9b0700 1 -- 172.21.15.2:0/3106530642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a80ffcc0 con 0x7fe2a8106770 2023-09-13T04:31:59.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.357+0000 7fe2977fe700 1 -- 172.21.15.2:0/3106530642 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe29800bc10 con 0x7fe2a8106770 2023-09-13T04:31:59.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.357+0000 7fe2977fe700 1 -- 172.21.15.2:0/3106530642 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe298003910 con 0x7fe2a8106770 2023-09-13T04:31:59.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.357+0000 7fe2ac9b0700 1 -- 172.21.15.2:0/3106530642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a8100140 con 0x7fe2a8106770 2023-09-13T04:31:59.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.358+0000 7fe2977fe700 1 -- 172.21.15.2:0/3106530642 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45386+0+0 (secure 0 0 0) 0x7fe29800bd80 con 0x7fe2a8106770 2023-09-13T04:31:59.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.358+0000 7fe2ac9b0700 1 -- 172.21.15.2:0/3106530642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2a8105a20 con 0x7fe2a8106770 2023-09-13T04:31:59.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.359+0000 7fe2977fe700 1 --2- 172.21.15.2:0/3106530642 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe290038400 0x7fe29003a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:31:59.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.359+0000 7fe2977fe700 1 -- 172.21.15.2:0/3106530642 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe29804c0c0 con 0x7fe2a8106770 2023-09-13T04:31:59.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.359+0000 7fe2a5d9b700 1 --2- 172.21.15.2:0/3106530642 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe290038400 0x7fe29003a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:31:59.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.360+0000 7fe2a5d9b700 1 --2- 172.21.15.2:0/3106530642 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe290038400 0x7fe29003a8c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe29c006fd0 tx=0x7fe29c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:31:59.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.364+0000 7fe2977fe700 1 -- 172.21.15.2:0/3106530642 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe298029360 con 0x7fe2a8106770 2023-09-13T04:31:59.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:31:59.581+0000 7fe2ac9b0700 1 -- 172.21.15.2:0/3106530642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fe2a80622e0 con 0x7fe2a8106770 2023-09-13T04:32:00.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:59 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:00.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:59 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:00.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:59 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:00.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:59 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:00.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:59 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:32:00.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:59 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:00.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:59 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:32:00.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:59 smithi002 ceph-mon[97651]: Updating smithi156:/etc/ceph/ceph.conf 2023-09-13T04:32:00.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:59 smithi002 ceph-mon[97651]: Updating smithi156:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:32:00.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:59 smithi002 ceph-mon[97651]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2023-09-13T04:32:00.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:59 smithi002 ceph-mon[97651]: Updating smithi156:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T04:32:00.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:59 smithi002 ceph-mon[97651]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:00.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:59 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3106530642' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-13T04:32:00.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:59 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:00.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:59 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:00.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:31:59 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:00.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:00.129+0000 7fe2977fe700 1 -- 172.21.15.2:0/3106530642 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fe298026020 con 0x7fe2a8106770 2023-09-13T04:32:00.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:00.132+0000 7fe2ac9b0700 1 -- 172.21.15.2:0/3106530642 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe290038400 msgr2=0x7fe29003a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:00.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:00.132+0000 7fe2ac9b0700 1 --2- 172.21.15.2:0/3106530642 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe290038400 0x7fe29003a8c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe29c006fd0 tx=0x7fe29c006e40 comp rx=0 tx=0).stop 2023-09-13T04:32:00.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:00.132+0000 7fe2ac9b0700 1 -- 172.21.15.2:0/3106530642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2a8106770 msgr2=0x7fe2a80ff780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:00.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:00.132+0000 7fe2ac9b0700 1 --2- 172.21.15.2:0/3106530642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2a8106770 0x7fe2a80ff780 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe298006010 tx=0x7fe298004ff0 comp rx=0 tx=0).stop 2023-09-13T04:32:00.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:00.132+0000 7fe2ac9b0700 1 -- 172.21.15.2:0/3106530642 shutdown_connections 2023-09-13T04:32:00.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:00.132+0000 7fe2ac9b0700 1 --2- 172.21.15.2:0/3106530642 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe290038400 0x7fe29003a8c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:00.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:00.132+0000 7fe2ac9b0700 1 --2- 172.21.15.2:0/3106530642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2a8106770 0x7fe2a80ff780 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:00.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:00.132+0000 7fe2ac9b0700 1 -- 172.21.15.2:0/3106530642 >> 172.21.15.2:0/3106530642 conn(0x7fe2a80751b0 msgr2=0x7fe2a80fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:32:00.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:00.132+0000 7fe2ac9b0700 1 -- 172.21.15.2:0/3106530642 shutdown_connections 2023-09-13T04:32:00.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:00.132+0000 7fe2ac9b0700 1 -- 172.21.15.2:0/3106530642 wait complete. 2023-09-13T04:32:00.134 INFO:teuthology.orchestra.run.smithi002.stderr:adjusted tunables profile to default 2023-09-13T04:32:00.648 INFO:tasks.cephadm:Adding mon.a on smithi002 2023-09-13T04:32:00.649 INFO:tasks.cephadm:Adding mon.b on smithi149 2023-09-13T04:32:00.649 INFO:tasks.cephadm:Adding mon.c on smithi156 2023-09-13T04:32:00.649 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph orch apply mon '3;smithi002:172.21.15.2=a;smithi149:172.21.15.149=b;smithi156:172.21.15.156=c' 2023-09-13T04:32:01.235 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:32:01.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:01 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3106530642' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-13T04:32:01.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:01 smithi002 ceph-mon[97651]: osdmap e4: 0 total, 0 up, 0 in 2023-09-13T04:32:02.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.338+0000 7f23bf415700 1 -- 172.21.15.156:0/2881052711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b8105660 msgr2=0x7f23b8105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:02.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.338+0000 7f23bf415700 1 --2- 172.21.15.156:0/2881052711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b8105660 0x7f23b8105a40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009910 tx=0x7f23a8009c20 comp rx=0 tx=0).stop 2023-09-13T04:32:02.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.338+0000 7f23bf415700 1 -- 172.21.15.156:0/2881052711 shutdown_connections 2023-09-13T04:32:02.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.338+0000 7f23bf415700 1 --2- 172.21.15.156:0/2881052711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b8105660 0x7f23b8105a40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:02.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.338+0000 7f23bf415700 1 -- 172.21.15.156:0/2881052711 >> 172.21.15.156:0/2881052711 conn(0x7f23b80751b0 msgr2=0x7f23b80755c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:32:02.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.338+0000 7f23bf415700 1 -- 172.21.15.156:0/2881052711 shutdown_connections 2023-09-13T04:32:02.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.338+0000 7f23bf415700 1 -- 172.21.15.156:0/2881052711 wait complete. 2023-09-13T04:32:02.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.338+0000 7f23bf415700 1 Processor -- start 2023-09-13T04:32:02.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.338+0000 7f23bf415700 1 -- start start 2023-09-13T04:32:02.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.338+0000 7f23bf415700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b8105660 0x7f23b8196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:32:02.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.338+0000 7f23bf415700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b8197000 con 0x7f23b8105660 2023-09-13T04:32:02.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.340+0000 7f23bd1b1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b8105660 0x7f23b8196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:32:02.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.340+0000 7f23bd1b1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b8105660 0x7f23b8196ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.156:48618/0 (socket says 172.21.15.156:48618) 2023-09-13T04:32:02.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.340+0000 7f23bd1b1700 1 -- 172.21.15.156:0/1383774915 learned_addr learned my addr 172.21.15.156:0/1383774915 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-13T04:32:02.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.340+0000 7f23bd1b1700 1 -- 172.21.15.156:0/1383774915 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a80095f0 con 0x7f23b8105660 2023-09-13T04:32:02.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.341+0000 7f23bd1b1700 1 --2- 172.21.15.156:0/1383774915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b8105660 0x7f23b8196ac0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f23a8004f40 tx=0x7f23a8005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:32:02.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.341+0000 7f23ae7fc700 1 -- 172.21.15.156:0/1383774915 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f23a801d070 con 0x7f23b8105660 2023-09-13T04:32:02.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.341+0000 7f23ae7fc700 1 -- 172.21.15.156:0/1383774915 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23a8004370 con 0x7f23b8105660 2023-09-13T04:32:02.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.341+0000 7f23bf415700 1 -- 172.21.15.156:0/1383774915 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b8197260 con 0x7f23b8105660 2023-09-13T04:32:02.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.341+0000 7f23ae7fc700 1 -- 172.21.15.156:0/1383774915 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f23a8003970 con 0x7f23b8105660 2023-09-13T04:32:02.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.341+0000 7f23bf415700 1 -- 172.21.15.156:0/1383774915 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b81976e0 con 0x7f23b8105660 2023-09-13T04:32:02.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.342+0000 7f23bf415700 1 -- 172.21.15.156:0/1383774915 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b8197920 con 0x7f23b8105660 2023-09-13T04:32:02.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.342+0000 7f23ae7fc700 1 -- 172.21.15.156:0/1383774915 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45386+0+0 (secure 0 0 0) 0x7f23a8003c30 con 0x7f23b8105660 2023-09-13T04:32:02.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.343+0000 7f23ae7fc700 1 --2- 172.21.15.156:0/1383774915 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f23a4038400 0x7f23a403a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:32:02.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.343+0000 7f23ae7fc700 1 -- 172.21.15.156:0/1383774915 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f23a804c760 con 0x7f23b8105660 2023-09-13T04:32:02.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.343+0000 7f23bc9b0700 1 --2- 172.21.15.156:0/1383774915 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f23a4038400 0x7f23a403a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:32:02.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.343+0000 7f23bc9b0700 1 --2- 172.21.15.156:0/1383774915 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f23a4038400 0x7f23a403a8c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f23b4006fd0 tx=0x7f23b4006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:32:02.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.348+0000 7f23ae7fc700 1 -- 172.21.15.156:0/1383774915 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f23a800e5c0 con 0x7f23b8105660 2023-09-13T04:32:02.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:02 smithi002 ceph-mon[97651]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:02.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.543+0000 7f23bf415700 1 -- 172.21.15.156:0/1383774915 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi002:172.21.15.2=a;smithi149:172.21.15.149=b;smithi156:172.21.15.156=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f23b8109950 con 0x7f23a4038400 2023-09-13T04:32:02.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.551+0000 7f23ae7fc700 1 -- 172.21.15.156:0/1383774915 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f23b8109950 con 0x7f23a4038400 2023-09-13T04:32:02.551 INFO:teuthology.orchestra.run.smithi156.stdout:Scheduled mon update... 2023-09-13T04:32:02.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.552+0000 7f23bf415700 1 -- 172.21.15.156:0/1383774915 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f23a4038400 msgr2=0x7f23a403a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:02.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.552+0000 7f23bf415700 1 --2- 172.21.15.156:0/1383774915 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f23a4038400 0x7f23a403a8c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f23b4006fd0 tx=0x7f23b4006e40 comp rx=0 tx=0).stop 2023-09-13T04:32:02.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.552+0000 7f23bf415700 1 -- 172.21.15.156:0/1383774915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b8105660 msgr2=0x7f23b8196ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:02.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.552+0000 7f23bf415700 1 --2- 172.21.15.156:0/1383774915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b8105660 0x7f23b8196ac0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f23a8004f40 tx=0x7f23a8005020 comp rx=0 tx=0).stop 2023-09-13T04:32:02.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.552+0000 7f23bf415700 1 -- 172.21.15.156:0/1383774915 shutdown_connections 2023-09-13T04:32:02.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.552+0000 7f23bf415700 1 --2- 172.21.15.156:0/1383774915 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f23a4038400 0x7f23a403a8c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:02.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.552+0000 7f23bf415700 1 --2- 172.21.15.156:0/1383774915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b8105660 0x7f23b8196ac0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:02.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.552+0000 7f23bf415700 1 -- 172.21.15.156:0/1383774915 >> 172.21.15.156:0/1383774915 conn(0x7f23b80751b0 msgr2=0x7f23b80fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:32:02.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.553+0000 7f23bf415700 1 -- 172.21.15.156:0/1383774915 shutdown_connections 2023-09-13T04:32:02.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:02.553+0000 7f23bf415700 1 -- 172.21.15.156:0/1383774915 wait complete. 2023-09-13T04:32:03.018 DEBUG:teuthology.orchestra.run.smithi149:mon.b> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mon.b.service 2023-09-13T04:32:03.021 DEBUG:teuthology.orchestra.run.smithi156:mon.c> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mon.c.service 2023-09-13T04:32:03.023 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-13T04:32:03.023 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph mon dump -f json 2023-09-13T04:32:03.053 INFO:journalctl@ceph.mon.c.smithi156.stdout:-- Logs begin at Wed 2023-09-13 04:09:45 UTC. -- 2023-09-13T04:32:03.056 INFO:journalctl@ceph.mon.b.smithi149.stdout:-- Logs begin at Wed 2023-09-13 04:09:45 UTC. -- 2023-09-13T04:32:03.785 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:32:04.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:03 smithi002 ceph-mon[97651]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi002:172.21.15.2=a;smithi149:172.21.15.149=b;smithi156:172.21.15.156=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:32:04.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:03 smithi002 ceph-mon[97651]: Saving service mon spec with placement smithi002:172.21.15.2=a;smithi149:172.21.15.149=b;smithi156:172.21.15.156=c;count:3 2023-09-13T04:32:04.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:03 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:04.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:03 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:32:04.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:03 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:04.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:03 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:32:04.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:03 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:04.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:03 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T04:32:04.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:03 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:04.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:03 smithi002 ceph-mon[97651]: Deploying daemon mon.c on smithi156 2023-09-13T04:32:05.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:04 smithi002 ceph-mon[97651]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:05.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.311+0000 7fc191ea1700 1 -- 172.21.15.156:0/1574933767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc18c104610 msgr2=0x7fc18c1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:05.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.311+0000 7fc191ea1700 1 --2- 172.21.15.156:0/1574933767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc18c104610 0x7fc18c1049f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009910 tx=0x7fc17c009c20 comp rx=0 tx=0).stop 2023-09-13T04:32:05.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.311+0000 7fc191ea1700 1 -- 172.21.15.156:0/1574933767 shutdown_connections 2023-09-13T04:32:05.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.311+0000 7fc191ea1700 1 --2- 172.21.15.156:0/1574933767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc18c104610 0x7fc18c1049f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:05.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.311+0000 7fc191ea1700 1 -- 172.21.15.156:0/1574933767 >> 172.21.15.156:0/1574933767 conn(0x7fc18c075da0 msgr2=0x7fc18c0761b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:32:05.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.311+0000 7fc191ea1700 1 -- 172.21.15.156:0/1574933767 shutdown_connections 2023-09-13T04:32:05.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.311+0000 7fc191ea1700 1 -- 172.21.15.156:0/1574933767 wait complete. 2023-09-13T04:32:05.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.311+0000 7fc191ea1700 1 Processor -- start 2023-09-13T04:32:05.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.312+0000 7fc191ea1700 1 -- start start 2023-09-13T04:32:05.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.312+0000 7fc191ea1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc18c104610 0x7fc18c196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:32:05.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.312+0000 7fc191ea1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c196fb0 con 0x7fc18c104610 2023-09-13T04:32:05.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.312+0000 7fc18b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc18c104610 0x7fc18c196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:32:05.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.312+0000 7fc18b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc18c104610 0x7fc18c196a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.156:48624/0 (socket says 172.21.15.156:48624) 2023-09-13T04:32:05.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.312+0000 7fc18b7fe700 1 -- 172.21.15.156:0/907011935 learned_addr learned my addr 172.21.15.156:0/907011935 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-13T04:32:05.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.312+0000 7fc18b7fe700 1 -- 172.21.15.156:0/907011935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c0095f0 con 0x7fc18c104610 2023-09-13T04:32:05.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.313+0000 7fc18b7fe700 1 --2- 172.21.15.156:0/907011935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc18c104610 0x7fc18c196a70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc17c006010 tx=0x7fc17c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:32:05.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.313+0000 7fc188ff9700 1 -- 172.21.15.156:0/907011935 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc17c01d070 con 0x7fc18c104610 2023-09-13T04:32:05.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.313+0000 7fc191ea1700 1 -- 172.21.15.156:0/907011935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc18c197210 con 0x7fc18c104610 2023-09-13T04:32:05.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.313+0000 7fc188ff9700 1 -- 172.21.15.156:0/907011935 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc17c00bc10 con 0x7fc18c104610 2023-09-13T04:32:05.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.313+0000 7fc188ff9700 1 -- 172.21.15.156:0/907011935 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc17c003910 con 0x7fc18c104610 2023-09-13T04:32:05.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.313+0000 7fc191ea1700 1 -- 172.21.15.156:0/907011935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18c197690 con 0x7fc18c104610 2023-09-13T04:32:05.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.314+0000 7fc191ea1700 1 -- 172.21.15.156:0/907011935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc18c1978e0 con 0x7fc18c104610 2023-09-13T04:32:05.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.314+0000 7fc188ff9700 1 -- 172.21.15.156:0/907011935 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45386+0+0 (secure 0 0 0) 0x7fc17c00bd80 con 0x7fc18c104610 2023-09-13T04:32:05.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.314+0000 7fc188ff9700 1 --2- 172.21.15.156:0/907011935 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc178038400 0x7fc17803a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:32:05.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.314+0000 7fc188ff9700 1 -- 172.21.15.156:0/907011935 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc17c04bf70 con 0x7fc18c104610 2023-09-13T04:32:05.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.314+0000 7fc18affd700 1 --2- 172.21.15.156:0/907011935 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc178038400 0x7fc17803a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:32:05.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.315+0000 7fc18affd700 1 --2- 172.21.15.156:0/907011935 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc178038400 0x7fc17803a8c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc174006fd0 tx=0x7fc174006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:32:05.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.320+0000 7fc188ff9700 1 -- 172.21.15.156:0/907011935 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc18c1978e0 con 0x7fc18c104610 2023-09-13T04:32:05.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.619+0000 7fc191ea1700 1 -- 172.21.15.156:0/907011935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc18c1978e0 con 0x7fc18c104610 2023-09-13T04:32:05.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.620+0000 7fc188ff9700 1 -- 172.21.15.156:0/907011935 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+732 (secure 0 0 0) 0x7fc18c1978e0 con 0x7fc18c104610 2023-09-13T04:32:05.621 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-13T04:32:05.621 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":1,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","modified":"2023-09-13T04:30:18.425317Z","created":"2023-09-13T04:30:18.425317Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-09-13T04:32:05.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.623+0000 7fc191ea1700 1 -- 172.21.15.156:0/907011935 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc178038400 msgr2=0x7fc17803a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:05.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.623+0000 7fc191ea1700 1 --2- 172.21.15.156:0/907011935 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc178038400 0x7fc17803a8c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc174006fd0 tx=0x7fc174006e40 comp rx=0 tx=0).stop 2023-09-13T04:32:05.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.623+0000 7fc191ea1700 1 -- 172.21.15.156:0/907011935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc18c104610 msgr2=0x7fc18c196a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:05.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.623+0000 7fc191ea1700 1 --2- 172.21.15.156:0/907011935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc18c104610 0x7fc18c196a70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc17c006010 tx=0x7fc17c004ff0 comp rx=0 tx=0).stop 2023-09-13T04:32:05.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.623+0000 7fc191ea1700 1 -- 172.21.15.156:0/907011935 shutdown_connections 2023-09-13T04:32:05.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.623+0000 7fc191ea1700 1 --2- 172.21.15.156:0/907011935 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc178038400 0x7fc17803a8c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:05.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.623+0000 7fc191ea1700 1 --2- 172.21.15.156:0/907011935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc18c104610 0x7fc18c196a70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:05.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.623+0000 7fc191ea1700 1 -- 172.21.15.156:0/907011935 >> 172.21.15.156:0/907011935 conn(0x7fc18c075da0 msgr2=0x7fc18c0fbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:32:05.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.623+0000 7fc191ea1700 1 -- 172.21.15.156:0/907011935 shutdown_connections 2023-09-13T04:32:05.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:05.623+0000 7fc191ea1700 1 -- 172.21.15.156:0/907011935 wait complete. 2023-09-13T04:32:05.625 INFO:teuthology.orchestra.run.smithi156.stderr:dumped monmap epoch 1 2023-09-13T04:32:06.296 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: mon.c@-1(???) e0 preinit fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:32:06.296 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: mon.c@-1(synchronizing).mds e1 new map 2023-09-13T04:32:06.296 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: mon.c@-1(synchronizing).mds e1 print_map 2023-09-13T04:32:06.297 INFO:journalctl@ceph.mon.c.smithi156.stdout: e1 2023-09-13T04:32:06.297 INFO:journalctl@ceph.mon.c.smithi156.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-13T04:32:06.297 INFO:journalctl@ceph.mon.c.smithi156.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-13T04:32:06.297 INFO:journalctl@ceph.mon.c.smithi156.stdout: legacy client fscid: -1 2023-09-13T04:32:06.297 INFO:journalctl@ceph.mon.c.smithi156.stdout: 2023-09-13T04:32:06.298 INFO:journalctl@ceph.mon.c.smithi156.stdout: No filesystems configured 2023-09-13T04:32:06.298 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-09-13T04:32:06.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-09-13T04:32:06.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-09-13T04:32:06.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-09-13T04:32:06.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-09-13T04:32:06.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-09-13T04:32:06.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-13T04:32:06.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-13T04:32:06.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-13T04:32:06.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-13T04:32:06.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:06.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:06.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:06.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:32:06.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:06.690 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:06.690 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:06.690 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:06.690 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:06.690 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:32:06.691 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:06.691 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:32:06.691 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: Updating smithi149:/etc/ceph/ceph.conf 2023-09-13T04:32:06.691 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: Updating smithi149:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:32:06.691 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:06.692 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:06.692 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:06.692 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2023-09-13T04:32:06.692 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: Updating smithi149:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T04:32:06.692 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:32:06.693 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:06.693 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: Deploying cephadm binary to smithi156 2023-09-13T04:32:06.693 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:06.693 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:06.693 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: Added host smithi156 2023-09-13T04:32:06.694 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:32:06.694 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:06.694 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:06.694 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:06.694 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:06.695 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:32:06.695 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:06.695 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:06.695 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:06.695 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:06.696 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:06.696 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:32:06.696 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:06.696 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:32:06.696 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: Updating smithi156:/etc/ceph/ceph.conf 2023-09-13T04:32:06.697 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: Updating smithi156:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:32:06.697 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2023-09-13T04:32:06.697 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: Updating smithi156:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T04:32:06.697 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:06.697 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3106530642' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-13T04:32:06.697 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:06.698 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:06.698 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:06.698 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3106530642' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-13T04:32:06.698 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: osdmap e4: 0 total, 0 up, 0 in 2023-09-13T04:32:06.698 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:06.699 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi002:172.21.15.2=a;smithi149:172.21.15.149=b;smithi156:172.21.15.156=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:32:06.699 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: Saving service mon spec with placement smithi002:172.21.15.2=a;smithi149:172.21.15.149=b;smithi156:172.21.15.156=c;count:3 2023-09-13T04:32:06.699 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:06.699 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:32:06.699 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:06.700 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:32:06.700 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:06.700 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T04:32:06.700 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:06.700 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: Deploying daemon mon.c on smithi156 2023-09-13T04:32:06.701 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:06.701 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-09-13T04:32:06.701 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-09-13T04:32:06.701 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: expand_channel_meta expand map: {default=false} 2023-09-13T04:32:06.701 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: expand_channel_meta from 'false' to 'false' 2023-09-13T04:32:06.702 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: expand_channel_meta expanded map: {default=false} 2023-09-13T04:32:06.702 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: expand_channel_meta expand map: {default=info} 2023-09-13T04:32:06.702 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: expand_channel_meta from 'info' to 'info' 2023-09-13T04:32:06.702 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: expand_channel_meta expanded map: {default=info} 2023-09-13T04:32:06.702 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: expand_channel_meta expand map: {default=daemon} 2023-09-13T04:32:06.702 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: expand_channel_meta from 'daemon' to 'daemon' 2023-09-13T04:32:06.703 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: expand_channel_meta expanded map: {default=daemon} 2023-09-13T04:32:06.703 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: expand_channel_meta expand map: {default=debug} 2023-09-13T04:32:06.703 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: expand_channel_meta from 'debug' to 'debug' 2023-09-13T04:32:06.703 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 ceph-mon[133675]: expand_channel_meta expanded map: {default=debug} 2023-09-13T04:32:06.703 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:06 smithi156 systemd[1]: Started Ceph mon.c for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:32:07.078 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-13T04:32:07.079 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph mon dump -f json 2023-09-13T04:32:07.645 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-13T04:32:11.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:11 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T04:32:11.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:11 smithi156 ceph-mon[133675]: mon.a calling monitor election 2023-09-13T04:32:11.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:11 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:32:11.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:11 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:32:11.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:11 smithi156 ceph-mon[133675]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:11.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:11 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:32:11.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:11 smithi156 ceph-mon[133675]: mon.c calling monitor election 2023-09-13T04:32:11.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:11 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:32:11.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:11 smithi156 ceph-mon[133675]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:11.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:11 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:32:11.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:11 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:32:11.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:11 smithi156 ceph-mon[133675]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-13T04:32:11.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:11 smithi156 ceph-mon[133675]: monmap e2: 2 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-09-13T04:32:11.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:11 smithi156 ceph-mon[133675]: fsmap 2023-09-13T04:32:11.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:11 smithi156 ceph-mon[133675]: osdmap e4: 0 total, 0 up, 0 in 2023-09-13T04:32:11.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:11 smithi156 ceph-mon[133675]: mgrmap e13: x(active, since 47s) 2023-09-13T04:32:11.690 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:11 smithi156 ceph-mon[133675]: overall HEALTH_OK 2023-09-13T04:32:11.690 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:11 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:11.690 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:11 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:11.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.744+0000 7fd3b6dc9700 1 -- 172.21.15.156:0/335901980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3a00035c0 msgr2=0x7fd3a0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:11.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.744+0000 7fd3b6dc9700 1 --2- 172.21.15.156:0/335901980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3a00035c0 0x7fd3a0005a50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0068db0 tx=0x7fd3a400feb0 comp rx=0 tx=0).stop 2023-09-13T04:32:11.745 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.744+0000 7fd3b6dc9700 1 -- 172.21.15.156:0/335901980 shutdown_connections 2023-09-13T04:32:11.745 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.744+0000 7fd3b6dc9700 1 --2- 172.21.15.156:0/335901980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3a00035c0 0x7fd3a0005a50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:11.745 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.744+0000 7fd3b6dc9700 1 -- 172.21.15.156:0/335901980 >> 172.21.15.156:0/335901980 conn(0x7fd3b00fb2a0 msgr2=0x7fd3b00fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:32:11.745 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.744+0000 7fd3b6dc9700 1 -- 172.21.15.156:0/335901980 shutdown_connections 2023-09-13T04:32:11.745 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.744+0000 7fd3b6dc9700 1 -- 172.21.15.156:0/335901980 wait complete. 2023-09-13T04:32:11.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.745+0000 7fd3b6dc9700 1 Processor -- start 2023-09-13T04:32:11.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.745+0000 7fd3b6dc9700 1 -- start start 2023-09-13T04:32:11.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.745+0000 7fd3b6dc9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3a00035c0 0x7fd3b0195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:32:11.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.745+0000 7fd3b6dc9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3b0196320 0x7fd3b019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:32:11.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.745+0000 7fd3b6dc9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b01968b0 con 0x7fd3a00035c0 2023-09-13T04:32:11.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.745+0000 7fd3b6dc9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b0196a20 con 0x7fd3b0196320 2023-09-13T04:32:11.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.746+0000 7fd3affff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3b0196320 0x7fd3b019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:32:11.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.746+0000 7fd3affff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3b0196320 0x7fd3b019a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37016/0 (socket says 172.21.15.156:37016) 2023-09-13T04:32:11.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.746+0000 7fd3affff700 1 -- 172.21.15.156:0/671387067 learned_addr learned my addr 172.21.15.156:0/671387067 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-13T04:32:11.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.746+0000 7fd3affff700 1 -- 172.21.15.156:0/671387067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3b0196320 msgr2=0x7fd3b019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-09-13T04:32:11.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.746+0000 7fd3affff700 1 -- 172.21.15.156:0/671387067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3b0196320 msgr2=0x7fd3b019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-13T04:32:11.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.746+0000 7fd3affff700 1 --2- 172.21.15.156:0/671387067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3b0196320 0x7fd3b019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-13T04:32:11.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.746+0000 7fd3affff700 1 --2- 172.21.15.156:0/671387067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3b0196320 0x7fd3b019a790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-13T04:32:11.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.746+0000 7fd3b4b65700 1 --2- 172.21.15.156:0/671387067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3a00035c0 0x7fd3b0195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:32:11.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.746+0000 7fd3b4b65700 1 -- 172.21.15.156:0/671387067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3b0196320 msgr2=0x7fd3b019a790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-13T04:32:11.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.746+0000 7fd3b4b65700 1 --2- 172.21.15.156:0/671387067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3b0196320 0x7fd3b019a790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:11.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.746+0000 7fd3b4b65700 1 -- 172.21.15.156:0/671387067 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a400f900 con 0x7fd3a00035c0 2023-09-13T04:32:11.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.747+0000 7fd3b4b65700 1 --2- 172.21.15.156:0/671387067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3a00035c0 0x7fd3b0195de0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4000c00 tx=0x7fd3a40047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:32:11.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.747+0000 7fd3adffb700 1 -- 172.21.15.156:0/671387067 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fd3a4013b00 con 0x7fd3a00035c0 2023-09-13T04:32:11.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.747+0000 7fd3adffb700 1 -- 172.21.15.156:0/671387067 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3a40044f0 con 0x7fd3a00035c0 2023-09-13T04:32:11.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.747+0000 7fd3b6dc9700 1 -- 172.21.15.156:0/671387067 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b019ad30 con 0x7fd3a00035c0 2023-09-13T04:32:11.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.747+0000 7fd3adffb700 1 -- 172.21.15.156:0/671387067 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fd3a4003be0 con 0x7fd3a00035c0 2023-09-13T04:32:11.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.747+0000 7fd3b6dc9700 1 -- 172.21.15.156:0/671387067 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b019b280 con 0x7fd3a00035c0 2023-09-13T04:32:11.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.748+0000 7fd3adffb700 1 -- 172.21.15.156:0/671387067 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45386+0+0 (secure 0 0 0) 0x7fd3a4013cf0 con 0x7fd3a00035c0 2023-09-13T04:32:11.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.748+0000 7fd3b6dc9700 1 -- 172.21.15.156:0/671387067 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3b00622e0 con 0x7fd3a00035c0 2023-09-13T04:32:11.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.749+0000 7fd3adffb700 1 --2- 172.21.15.156:0/671387067 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd394038240 0x7fd39403a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:32:11.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.749+0000 7fd3adffb700 1 -- 172.21.15.156:0/671387067 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd3a404d890 con 0x7fd3a00035c0 2023-09-13T04:32:11.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.749+0000 7fd3affff700 1 --2- 172.21.15.156:0/671387067 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd394038240 0x7fd39403a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:32:11.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.749+0000 7fd3affff700 1 --2- 172.21.15.156:0/671387067 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd394038240 0x7fd39403a700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009180 tx=0x7fd39c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:32:11.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:11.754+0000 7fd3adffb700 1 -- 172.21.15.156:0/671387067 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd3a4034d90 con 0x7fd3a00035c0 2023-09-13T04:32:11.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:11 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T04:32:11.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:11 smithi002 ceph-mon[97651]: mon.a calling monitor election 2023-09-13T04:32:11.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:11 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:32:11.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:11 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:32:11.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:11 smithi002 ceph-mon[97651]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:11.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:11 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:32:11.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:11 smithi002 ceph-mon[97651]: mon.c calling monitor election 2023-09-13T04:32:11.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:11 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:32:11.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:11 smithi002 ceph-mon[97651]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:11.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:11 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:32:11.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:11 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:32:11.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:11 smithi002 ceph-mon[97651]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-13T04:32:11.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:11 smithi002 ceph-mon[97651]: monmap e2: 2 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-09-13T04:32:11.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:11 smithi002 ceph-mon[97651]: fsmap 2023-09-13T04:32:11.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:11 smithi002 ceph-mon[97651]: osdmap e4: 0 total, 0 up, 0 in 2023-09-13T04:32:11.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:11 smithi002 ceph-mon[97651]: mgrmap e13: x(active, since 47s) 2023-09-13T04:32:11.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:11 smithi002 ceph-mon[97651]: overall HEALTH_OK 2023-09-13T04:32:11.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:11 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:11.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:11 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:12.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:12.022+0000 7fd3b6dc9700 1 -- 172.21.15.156:0/671387067 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd3b01086d0 con 0x7fd3a00035c0 2023-09-13T04:32:12.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:12.023+0000 7fd3adffb700 1 -- 172.21.15.156:0/671387067 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1002 (secure 0 0 0) 0x7fd3a4034350 con 0x7fd3a00035c0 2023-09-13T04:32:12.023 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-13T04:32:12.023 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":2,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","modified":"2023-09-13T04:32:06.301546Z","created":"2023-09-13T04:30:18.425317Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-09-13T04:32:12.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:12.025+0000 7fd3b6dc9700 1 -- 172.21.15.156:0/671387067 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd394038240 msgr2=0x7fd39403a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:12.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:12.025+0000 7fd3b6dc9700 1 --2- 172.21.15.156:0/671387067 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd394038240 0x7fd39403a700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009180 tx=0x7fd39c008040 comp rx=0 tx=0).stop 2023-09-13T04:32:12.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:12.025+0000 7fd3b6dc9700 1 -- 172.21.15.156:0/671387067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3a00035c0 msgr2=0x7fd3b0195de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:12.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:12.025+0000 7fd3b6dc9700 1 --2- 172.21.15.156:0/671387067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3a00035c0 0x7fd3b0195de0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4000c00 tx=0x7fd3a40047a0 comp rx=0 tx=0).stop 2023-09-13T04:32:12.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:12.025+0000 7fd3b6dc9700 1 -- 172.21.15.156:0/671387067 shutdown_connections 2023-09-13T04:32:12.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:12.025+0000 7fd3b6dc9700 1 --2- 172.21.15.156:0/671387067 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd394038240 0x7fd39403a700 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:12.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:12.025+0000 7fd3b6dc9700 1 --2- 172.21.15.156:0/671387067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3a00035c0 0x7fd3b0195de0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:12.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:12.025+0000 7fd3b6dc9700 1 --2- 172.21.15.156:0/671387067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3b0196320 0x7fd3b019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:12.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:12.025+0000 7fd3b6dc9700 1 -- 172.21.15.156:0/671387067 >> 172.21.15.156:0/671387067 conn(0x7fd3b00fb2a0 msgr2=0x7fd3b018ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:32:12.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:12.025+0000 7fd3b6dc9700 1 -- 172.21.15.156:0/671387067 shutdown_connections 2023-09-13T04:32:12.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:12.025+0000 7fd3b6dc9700 1 -- 172.21.15.156:0/671387067 wait complete. 2023-09-13T04:32:12.027 INFO:teuthology.orchestra.run.smithi156.stderr:dumped monmap epoch 2 2023-09-13T04:32:12.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:12 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:12.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:12 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T04:32:12.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:12 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:12.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:12 smithi156 ceph-mon[133675]: Deploying daemon mon.b on smithi149 2023-09-13T04:32:12.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:12 smithi156 ceph-mon[133675]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:12.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:12 smithi156 ceph-mon[133675]: from='client.? 172.21.15.156:0/671387067' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-13T04:32:12.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:12 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:32:12.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:12 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:12.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:12 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T04:32:12.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:12 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:12.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:12 smithi002 ceph-mon[97651]: Deploying daemon mon.b on smithi149 2023-09-13T04:32:12.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:12 smithi002 ceph-mon[97651]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:12.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:12 smithi002 ceph-mon[97651]: from='client.? 172.21.15.156:0/671387067' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-13T04:32:12.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:12 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:32:13.409 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-13T04:32:13.409 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph mon dump -f json 2023-09-13T04:32:13.974 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-13T04:32:14.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:13 smithi149 systemd[1]: Starting Ceph mon.b for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:32:14.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 podman[133275]: 2023-09-13T04:32:14.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: set uid:gid to 167:167 (ceph:ceph) 2023-09-13T04:32:14.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: ceph version 18.0.0-6083-g4a754ac9 (4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5) reef (dev), process ceph-mon, pid 7 2023-09-13T04:32:14.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: pidfile_write: ignore empty --pid-file 2023-09-13T04:32:14.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 bash[133275]: 4dffc1de4c8a460556428109147f07992c5c20347f574a377dd571ef985efcc8 2023-09-13T04:32:14.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: load: jerasure load: lrc 2023-09-13T04:32:14.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: RocksDB version: 7.9.2 2023-09-13T04:32:14.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Git sha 0 2023-09-13T04:32:14.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Compile date 2023-09-12 08:33:53 2023-09-13T04:32:14.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: DB SUMMARY 2023-09-13T04:32:14.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: DB Session ID: TOZ3VDAVG330EA6GW5KR 2023-09-13T04:32:14.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: CURRENT file: CURRENT 2023-09-13T04:32:14.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: IDENTITY file: IDENTITY 2023-09-13T04:32:14.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-09-13T04:32:14.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-09-13T04:32:14.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-09-13T04:32:14.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.error_if_exists: 0 2023-09-13T04:32:14.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.create_if_missing: 0 2023-09-13T04:32:14.641 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.paranoid_checks: 1 2023-09-13T04:32:14.641 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-13T04:32:14.641 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-13T04:32:14.641 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-13T04:32:14.641 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.env: 0x5620ed717da0 2023-09-13T04:32:14.642 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.fs: PosixFileSystem 2023-09-13T04:32:14.642 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.info_log: 0x5620ee2354a0 2023-09-13T04:32:14.642 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_file_opening_threads: 16 2023-09-13T04:32:14.642 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.statistics: (nil) 2023-09-13T04:32:14.642 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.use_fsync: 0 2023-09-13T04:32:14.643 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_log_file_size: 0 2023-09-13T04:32:14.643 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-13T04:32:14.643 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-13T04:32:14.643 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.keep_log_file_num: 1000 2023-09-13T04:32:14.643 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.recycle_log_file_num: 0 2023-09-13T04:32:14.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.allow_fallocate: 1 2023-09-13T04:32:14.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.allow_mmap_reads: 0 2023-09-13T04:32:14.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.allow_mmap_writes: 0 2023-09-13T04:32:14.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.use_direct_reads: 0 2023-09-13T04:32:14.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-13T04:32:14.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.create_missing_column_families: 0 2023-09-13T04:32:14.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.db_log_dir: 2023-09-13T04:32:14.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.wal_dir: 2023-09-13T04:32:14.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-13T04:32:14.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-13T04:32:14.646 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-13T04:32:14.646 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-13T04:32:14.646 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-13T04:32:14.646 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-13T04:32:14.646 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.advise_random_on_open: 1 2023-09-13T04:32:14.647 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.db_write_buffer_size: 0 2023-09-13T04:32:14.647 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.write_buffer_manager: 0x5620ee1ef4a0 2023-09-13T04:32:14.647 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-13T04:32:14.647 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-13T04:32:14.647 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-13T04:32:14.648 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.rate_limiter: (nil) 2023-09-13T04:32:14.648 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-13T04:32:14.648 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.wal_recovery_mode: 2 2023-09-13T04:32:14.648 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.enable_thread_tracking: 0 2023-09-13T04:32:14.648 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.enable_pipelined_write: 0 2023-09-13T04:32:14.648 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.unordered_write: 0 2023-09-13T04:32:14.649 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-13T04:32:14.649 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-13T04:32:14.649 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-13T04:32:14.649 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-13T04:32:14.649 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.row_cache: None 2023-09-13T04:32:14.650 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.wal_filter: None 2023-09-13T04:32:14.650 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-13T04:32:14.650 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.allow_ingest_behind: 0 2023-09-13T04:32:14.650 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.two_write_queues: 0 2023-09-13T04:32:14.650 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.manual_wal_flush: 0 2023-09-13T04:32:14.651 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.wal_compression: 0 2023-09-13T04:32:14.651 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.atomic_flush: 0 2023-09-13T04:32:14.651 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-13T04:32:14.651 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-13T04:32:14.651 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-13T04:32:14.651 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.log_readahead_size: 0 2023-09-13T04:32:14.652 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-13T04:32:14.652 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.best_efforts_recovery: 0 2023-09-13T04:32:14.652 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-13T04:32:14.652 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-13T04:32:14.652 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.allow_data_in_errors: 0 2023-09-13T04:32:14.653 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.db_host_id: __hostname__ 2023-09-13T04:32:14.653 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-13T04:32:14.653 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_background_jobs: 2 2023-09-13T04:32:14.653 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_background_compactions: -1 2023-09-13T04:32:14.653 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_subcompactions: 1 2023-09-13T04:32:14.654 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-13T04:32:14.654 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-13T04:32:14.654 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-13T04:32:14.654 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_total_wal_size: 0 2023-09-13T04:32:14.654 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-13T04:32:14.654 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-13T04:32:14.655 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-13T04:32:14.655 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-13T04:32:14.655 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_open_files: -1 2023-09-13T04:32:14.655 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.bytes_per_sync: 0 2023-09-13T04:32:14.655 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-13T04:32:14.656 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-13T04:32:14.656 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.compaction_readahead_size: 0 2023-09-13T04:32:14.656 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_background_flushes: -1 2023-09-13T04:32:14.656 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Compression algorithms supported: 2023-09-13T04:32:14.656 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-13T04:32:14.657 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: kZSTD supported: 0 2023-09-13T04:32:14.657 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: kXpressCompression supported: 0 2023-09-13T04:32:14.657 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: kLZ4HCCompression supported: 1 2023-09-13T04:32:14.657 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: kZlibCompression supported: 1 2023-09-13T04:32:14.657 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: kSnappyCompression supported: 1 2023-09-13T04:32:14.657 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: kLZ4Compression supported: 1 2023-09-13T04:32:14.658 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: kBZip2Compression supported: 0 2023-09-13T04:32:14.658 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-13T04:32:14.658 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-13T04:32:14.658 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-09-13T04:32:14.658 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-13T04:32:14.659 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-13T04:32:14.659 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.merge_operator: 2023-09-13T04:32:14.659 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.compaction_filter: None 2023-09-13T04:32:14.659 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.compaction_filter_factory: None 2023-09-13T04:32:14.659 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.sst_partitioner_factory: None 2023-09-13T04:32:14.660 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-13T04:32:14.660 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-13T04:32:14.660 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5620ee2355a0) 2023-09-13T04:32:14.660 INFO:journalctl@ceph.mon.b.smithi149.stdout: cache_index_and_filter_blocks: 1 2023-09-13T04:32:14.660 INFO:journalctl@ceph.mon.b.smithi149.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-13T04:32:14.661 INFO:journalctl@ceph.mon.b.smithi149.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-13T04:32:14.661 INFO:journalctl@ceph.mon.b.smithi149.stdout: pin_top_level_index_and_filter: 1 2023-09-13T04:32:14.661 INFO:journalctl@ceph.mon.b.smithi149.stdout: index_type: 0 2023-09-13T04:32:14.661 INFO:journalctl@ceph.mon.b.smithi149.stdout: data_block_index_type: 0 2023-09-13T04:32:14.661 INFO:journalctl@ceph.mon.b.smithi149.stdout: index_shortening: 1 2023-09-13T04:32:14.661 INFO:journalctl@ceph.mon.b.smithi149.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-13T04:32:14.662 INFO:journalctl@ceph.mon.b.smithi149.stdout: checksum: 4 2023-09-13T04:32:14.662 INFO:journalctl@ceph.mon.b.smithi149.stdout: no_block_cache: 0 2023-09-13T04:32:14.662 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_cache: 0x5620ee28d1f0 2023-09-13T04:32:14.662 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_cache_name: BinnedLRUCache 2023-09-13T04:32:14.662 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_cache_options: 2023-09-13T04:32:14.663 INFO:journalctl@ceph.mon.b.smithi149.stdout: capacity : 536870912 2023-09-13T04:32:14.663 INFO:journalctl@ceph.mon.b.smithi149.stdout: num_shard_bits : 4 2023-09-13T04:32:14.663 INFO:journalctl@ceph.mon.b.smithi149.stdout: strict_capacity_limit : 0 2023-09-13T04:32:14.663 INFO:journalctl@ceph.mon.b.smithi149.stdout: high_pri_pool_ratio: 0.000 2023-09-13T04:32:14.663 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_cache_compressed: (nil) 2023-09-13T04:32:14.664 INFO:journalctl@ceph.mon.b.smithi149.stdout: persistent_cache: (nil) 2023-09-13T04:32:14.664 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_size: 4096 2023-09-13T04:32:14.664 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_size_deviation: 10 2023-09-13T04:32:14.664 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_restart_interval: 16 2023-09-13T04:32:14.664 INFO:journalctl@ceph.mon.b.smithi149.stdout: index_block_restart_interval: 1 2023-09-13T04:32:14.664 INFO:journalctl@ceph.mon.b.smithi149.stdout: metadata_block_size: 4096 2023-09-13T04:32:14.665 INFO:journalctl@ceph.mon.b.smithi149.stdout: partition_filters: 0 2023-09-13T04:32:14.665 INFO:journalctl@ceph.mon.b.smithi149.stdout: use_delta_encoding: 1 2023-09-13T04:32:14.665 INFO:journalctl@ceph.mon.b.smithi149.stdout: filter_policy: bloomfilter 2023-09-13T04:32:14.665 INFO:journalctl@ceph.mon.b.smithi149.stdout: whole_key_filtering: 1 2023-09-13T04:32:14.665 INFO:journalctl@ceph.mon.b.smithi149.stdout: verify_compression: 0 2023-09-13T04:32:14.666 INFO:journalctl@ceph.mon.b.smithi149.stdout: read_amp_bytes_per_bit: 0 2023-09-13T04:32:14.666 INFO:journalctl@ceph.mon.b.smithi149.stdout: format_version: 5 2023-09-13T04:32:14.666 INFO:journalctl@ceph.mon.b.smithi149.stdout: enable_index_compression: 1 2023-09-13T04:32:14.666 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_align: 0 2023-09-13T04:32:14.666 INFO:journalctl@ceph.mon.b.smithi149.stdout: max_auto_readahead_size: 262144 2023-09-13T04:32:14.667 INFO:journalctl@ceph.mon.b.smithi149.stdout: prepopulate_block_cache: 0 2023-09-13T04:32:14.667 INFO:journalctl@ceph.mon.b.smithi149.stdout: initial_auto_readahead_size: 8192 2023-09-13T04:32:14.667 INFO:journalctl@ceph.mon.b.smithi149.stdout: num_file_reads_for_auto_readahead: 2 2023-09-13T04:32:14.667 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.write_buffer_size: 33554432 2023-09-13T04:32:14.667 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_write_buffer_number: 2 2023-09-13T04:32:14.667 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.compression: NoCompression 2023-09-13T04:32:14.668 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.bottommost_compression: Disabled 2023-09-13T04:32:14.668 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.prefix_extractor: nullptr 2023-09-13T04:32:14.668 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-13T04:32:14.668 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.num_levels: 7 2023-09-13T04:32:14.668 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-13T04:32:14.669 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-13T04:32:14.669 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-13T04:32:14.669 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-13T04:32:14.669 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-13T04:32:14.669 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-13T04:32:14.670 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-13T04:32:14.670 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-13T04:32:14.670 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-13T04:32:14.670 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-13T04:32:14.670 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-13T04:32:14.671 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-13T04:32:14.671 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-13T04:32:14.671 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.compression_opts.level: 32767 2023-09-13T04:32:14.671 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.compression_opts.strategy: 0 2023-09-13T04:32:14.671 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-13T04:32:14.671 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-13T04:32:14.672 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-13T04:32:14.672 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-13T04:32:14.672 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.compression_opts.enabled: false 2023-09-13T04:32:14.672 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-13T04:32:14.672 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-13T04:32:14.673 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-13T04:32:14.673 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-13T04:32:14.673 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.target_file_size_base: 67108864 2023-09-13T04:32:14.673 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-13T04:32:14.673 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-13T04:32:14.673 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-13T04:32:14.674 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-13T04:32:14.674 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-13T04:32:14.674 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-13T04:32:14.674 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-13T04:32:14.674 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-13T04:32:14.675 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-13T04:32:14.675 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-13T04:32:14.675 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-13T04:32:14.675 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-13T04:32:14.675 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-13T04:32:14.676 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-13T04:32:14.676 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.arena_block_size: 1048576 2023-09-13T04:32:14.676 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-13T04:32:14.676 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-13T04:32:14.676 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.disable_auto_compactions: 0 2023-09-13T04:32:14.678 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-13T04:32:14.678 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-13T04:32:14.678 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-13T04:32:14.678 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-13T04:32:14.679 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-13T04:32:14.679 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-13T04:32:14.679 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-13T04:32:14.679 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-13T04:32:14.679 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-13T04:32:14.680 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-13T04:32:14.680 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.table_properties_collectors: 2023-09-13T04:32:14.680 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.inplace_update_support: 0 2023-09-13T04:32:14.680 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-13T04:32:14.680 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-13T04:32:14.681 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-13T04:32:14.681 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-13T04:32:14.681 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.bloom_locality: 0 2023-09-13T04:32:14.681 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.max_successive_merges: 0 2023-09-13T04:32:14.681 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-13T04:32:14.682 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.paranoid_file_checks: 0 2023-09-13T04:32:14.682 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.force_consistency_checks: 1 2023-09-13T04:32:14.682 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.report_bg_io_stats: 0 2023-09-13T04:32:14.682 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.ttl: 2592000 2023-09-13T04:32:14.682 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-13T04:32:14.683 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-13T04:32:14.683 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-13T04:32:14.683 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.enable_blob_files: false 2023-09-13T04:32:14.683 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.min_blob_size: 0 2023-09-13T04:32:14.683 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.blob_file_size: 268435456 2023-09-13T04:32:14.683 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-13T04:32:14.684 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-13T04:32:14.684 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-13T04:32:14.684 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-13T04:32:14.684 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-13T04:32:14.684 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.blob_file_starting_level: 0 2023-09-13T04:32:14.685 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-13T04:32:14.685 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-09-13T04:32:14.685 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-09-13T04:32:14.685 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ff082c72-d721-4265-868c-f51849edcdf6 2023-09-13T04:32:14.685 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694579534501417, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-09-13T04:32:14.686 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-09-13T04:32:14.686 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694579534501830, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1694579534, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ff082c72-d721-4265-868c-f51849edcdf6", "db_session_id": "TOZ3VDAVG330EA6GW5KR", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-09-13T04:32:14.686 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694579534501898, "job": 1, "event": "recovery_finished"} 2023-09-13T04:32:14.686 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-09-13T04:32:14.686 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-13T04:32:14.687 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5620ee32a000 2023-09-13T04:32:14.687 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: DB pointer 0x5620ee316000 2023-09-13T04:32:14.687 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-13T04:32:14.687 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-13T04:32:14.687 INFO:journalctl@ceph.mon.b.smithi149.stdout: ** DB Stats ** 2023-09-13T04:32:14.688 INFO:journalctl@ceph.mon.b.smithi149.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-13T04:32:14.688 INFO:journalctl@ceph.mon.b.smithi149.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-13T04:32:14.688 INFO:journalctl@ceph.mon.b.smithi149.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-13T04:32:14.688 INFO:journalctl@ceph.mon.b.smithi149.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-13T04:32:14.688 INFO:journalctl@ceph.mon.b.smithi149.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-13T04:32:14.689 INFO:journalctl@ceph.mon.b.smithi149.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-13T04:32:14.689 INFO:journalctl@ceph.mon.b.smithi149.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-13T04:32:14.689 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2023-09-13T04:32:14.689 INFO:journalctl@ceph.mon.b.smithi149.stdout: ** Compaction Stats [default] ** 2023-09-13T04:32:14.689 INFO:journalctl@ceph.mon.b.smithi149.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-13T04:32:14.689 INFO:journalctl@ceph.mon.b.smithi149.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-13T04:32:14.690 INFO:journalctl@ceph.mon.b.smithi149.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-13T04:32:14.690 INFO:journalctl@ceph.mon.b.smithi149.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-13T04:32:14.690 INFO:journalctl@ceph.mon.b.smithi149.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-13T04:32:14.690 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2023-09-13T04:32:14.690 INFO:journalctl@ceph.mon.b.smithi149.stdout: ** Compaction Stats [default] ** 2023-09-13T04:32:14.691 INFO:journalctl@ceph.mon.b.smithi149.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-13T04:32:14.691 INFO:journalctl@ceph.mon.b.smithi149.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-13T04:32:14.691 INFO:journalctl@ceph.mon.b.smithi149.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-13T04:32:14.691 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2023-09-13T04:32:14.691 INFO:journalctl@ceph.mon.b.smithi149.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-13T04:32:14.692 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2023-09-13T04:32:14.692 INFO:journalctl@ceph.mon.b.smithi149.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-13T04:32:14.692 INFO:journalctl@ceph.mon.b.smithi149.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-13T04:32:14.692 INFO:journalctl@ceph.mon.b.smithi149.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-13T04:32:14.692 INFO:journalctl@ceph.mon.b.smithi149.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-13T04:32:14.692 INFO:journalctl@ceph.mon.b.smithi149.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-13T04:32:14.693 INFO:journalctl@ceph.mon.b.smithi149.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-13T04:32:14.693 INFO:journalctl@ceph.mon.b.smithi149.stdout: Cumulative compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-13T04:32:14.693 INFO:journalctl@ceph.mon.b.smithi149.stdout: Interval compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-13T04:32:14.693 INFO:journalctl@ceph.mon.b.smithi149.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-13T04:32:14.693 INFO:journalctl@ceph.mon.b.smithi149.stdout: Block cache BinnedLRUCache@0x5620ee28d1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-09-13T04:32:14.694 INFO:journalctl@ceph.mon.b.smithi149.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-09-13T04:32:14.694 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2023-09-13T04:32:14.694 INFO:journalctl@ceph.mon.b.smithi149.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-13T04:32:14.694 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-09-13T04:32:14.694 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: using public_addr v2:172.21.15.149:0/0 -> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] 2023-09-13T04:32:14.695 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: starting mon.b rank -1 at public addrs [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] at bind addrs [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:32:14.695 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: mon.b@-1(???) e0 preinit fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:32:14.695 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 systemd[1]: Started Ceph mon.b for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:32:14.695 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: mon.b@-1(synchronizing).mds e1 new map 2023-09-13T04:32:14.695 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: mon.b@-1(synchronizing).mds e1 print_map 2023-09-13T04:32:14.696 INFO:journalctl@ceph.mon.b.smithi149.stdout: e1 2023-09-13T04:32:14.696 INFO:journalctl@ceph.mon.b.smithi149.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-13T04:32:14.696 INFO:journalctl@ceph.mon.b.smithi149.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-13T04:32:14.696 INFO:journalctl@ceph.mon.b.smithi149.stdout: legacy client fscid: -1 2023-09-13T04:32:14.696 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2023-09-13T04:32:14.696 INFO:journalctl@ceph.mon.b.smithi149.stdout: No filesystems configured 2023-09-13T04:32:14.697 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-09-13T04:32:14.697 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-09-13T04:32:14.697 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-09-13T04:32:14.697 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-09-13T04:32:14.697 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-09-13T04:32:14.698 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-09-13T04:32:14.698 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-13T04:32:14.698 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-13T04:32:14.698 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-13T04:32:14.698 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-13T04:32:14.699 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.699 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.699 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:14.699 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:32:14.699 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:14.700 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.700 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.700 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.700 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.700 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:32:14.701 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:14.701 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:32:14.701 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: Updating smithi149:/etc/ceph/ceph.conf 2023-09-13T04:32:14.701 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: Updating smithi149:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:32:14.701 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.702 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.702 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.702 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2023-09-13T04:32:14.702 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: Updating smithi149:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T04:32:14.702 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:32:14.703 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:14.703 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: Deploying cephadm binary to smithi156 2023-09-13T04:32:14.703 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:14.703 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.703 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: Added host smithi156 2023-09-13T04:32:14.704 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:32:14.704 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.704 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.704 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:14.704 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.705 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:32:14.705 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:14.705 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.705 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.705 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.706 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.706 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:32:14.706 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:14.706 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:32:14.707 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: Updating smithi156:/etc/ceph/ceph.conf 2023-09-13T04:32:14.707 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: Updating smithi156:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:32:14.707 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2023-09-13T04:32:14.707 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: Updating smithi156:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T04:32:14.707 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:14.708 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3106530642' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-13T04:32:14.708 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.708 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.708 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.708 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3106530642' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-13T04:32:14.709 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: osdmap e4: 0 total, 0 up, 0 in 2023-09-13T04:32:14.709 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:14.709 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi002:172.21.15.2=a;smithi149:172.21.15.149=b;smithi156:172.21.15.156=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:32:14.709 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: Saving service mon spec with placement smithi002:172.21.15.2=a;smithi149:172.21.15.149=b;smithi156:172.21.15.156=c;count:3 2023-09-13T04:32:14.709 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.710 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:32:14.710 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:14.710 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:32:14.710 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.710 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T04:32:14.711 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:14.711 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: Deploying daemon mon.c on smithi156 2023-09-13T04:32:14.711 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:14.711 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T04:32:14.711 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: mon.a calling monitor election 2023-09-13T04:32:14.711 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:32:14.712 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:32:14.712 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:14.712 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:32:14.712 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: mon.c calling monitor election 2023-09-13T04:32:14.712 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:32:14.713 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:14.713 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:32:14.713 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:32:14.713 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-13T04:32:14.713 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: monmap e2: 2 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-09-13T04:32:14.714 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: fsmap 2023-09-13T04:32:14.714 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: osdmap e4: 0 total, 0 up, 0 in 2023-09-13T04:32:14.714 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: mgrmap e13: x(active, since 47s) 2023-09-13T04:32:14.714 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: overall HEALTH_OK 2023-09-13T04:32:14.714 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.715 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.715 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:14.715 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T04:32:14.715 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:14.715 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: Deploying daemon mon.b on smithi149 2023-09-13T04:32:14.716 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:14.716 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='client.? 172.21.15.156:0/671387067' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-13T04:32:14.716 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:32:14.716 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:14.716 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-09-13T04:32:14.716 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-09-13T04:32:14.717 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: expand_channel_meta expand map: {default=false} 2023-09-13T04:32:14.717 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: expand_channel_meta from 'false' to 'false' 2023-09-13T04:32:14.717 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: expand_channel_meta expanded map: {default=false} 2023-09-13T04:32:14.717 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: expand_channel_meta expand map: {default=info} 2023-09-13T04:32:14.717 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: expand_channel_meta from 'info' to 'info' 2023-09-13T04:32:14.718 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: expand_channel_meta expanded map: {default=info} 2023-09-13T04:32:14.718 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: expand_channel_meta expand map: {default=daemon} 2023-09-13T04:32:14.718 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: expand_channel_meta from 'daemon' to 'daemon' 2023-09-13T04:32:14.718 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: expand_channel_meta expanded map: {default=daemon} 2023-09-13T04:32:14.718 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: expand_channel_meta expand map: {default=debug} 2023-09-13T04:32:14.719 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: expand_channel_meta from 'debug' to 'debug' 2023-09-13T04:32:14.719 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:14 smithi149 ceph-mon[133315]: expand_channel_meta expanded map: {default=debug} 2023-09-13T04:32:14.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:14 smithi002 ceph-mon[97651]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:14.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:14 smithi156 ceph-mon[133675]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:19.581 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.581+0000 7fa5e8656700 1 -- 172.21.15.156:0/1634304243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5cc0035c0 msgr2=0x7fa5cc005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:19.581 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.581+0000 7fa5e8656700 1 --2- 172.21.15.156:0/1634304243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5cc0035c0 0x7fa5cc005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00766e0 tx=0x7fa5d000fe60 comp rx=0 tx=0).stop 2023-09-13T04:32:19.582 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.581+0000 7fa5e8656700 1 -- 172.21.15.156:0/1634304243 shutdown_connections 2023-09-13T04:32:19.582 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.581+0000 7fa5e8656700 1 --2- 172.21.15.156:0/1634304243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5cc0035c0 0x7fa5cc005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:19.582 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.581+0000 7fa5e8656700 1 -- 172.21.15.156:0/1634304243 >> 172.21.15.156:0/1634304243 conn(0x7fa5e00752c0 msgr2=0x7fa5e00756d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:32:19.582 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.581+0000 7fa5e8656700 1 -- 172.21.15.156:0/1634304243 shutdown_connections 2023-09-13T04:32:19.582 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.581+0000 7fa5e8656700 1 -- 172.21.15.156:0/1634304243 wait complete. 2023-09-13T04:32:19.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.582+0000 7fa5e8656700 1 Processor -- start 2023-09-13T04:32:19.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.582+0000 7fa5e8656700 1 -- start start 2023-09-13T04:32:19.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.582+0000 7fa5e8656700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5cc0035c0 0x7fa5e0195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:32:19.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.582+0000 7fa5e8656700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5e0196450 0x7fa5e019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:32:19.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.582+0000 7fa5e8656700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5e019ae00 0x7fa5e019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:32:19.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.582+0000 7fa5e8656700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e0196b40 con 0x7fa5e0196450 2023-09-13T04:32:19.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.582+0000 7fa5e8656700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e01a2b70 con 0x7fa5e019ae00 2023-09-13T04:32:19.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.582+0000 7fa5e8656700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e01a2cb0 con 0x7fa5cc0035c0 2023-09-13T04:32:19.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.582+0000 7fa5e63f2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5cc0035c0 0x7fa5e0195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:32:19.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.582+0000 7fa5e63f2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5cc0035c0 0x7fa5e0195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37042/0 (socket says 172.21.15.156:37042) 2023-09-13T04:32:19.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.582+0000 7fa5e63f2700 1 -- 172.21.15.156:0/3404865898 learned_addr learned my addr 172.21.15.156:0/3404865898 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-13T04:32:19.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.582+0000 7fa5e5bf1700 1 --2- 172.21.15.156:0/3404865898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5e0196450 0x7fa5e019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:32:19.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.582+0000 7fa5e63f2700 1 -- 172.21.15.156:0/3404865898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5cc0035c0 msgr2=0x7fa5e0195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-09-13T04:32:19.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.582+0000 7fa5e63f2700 1 -- 172.21.15.156:0/3404865898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5cc0035c0 msgr2=0x7fa5e0195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-13T04:32:19.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.582+0000 7fa5e63f2700 1 --2- 172.21.15.156:0/3404865898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5cc0035c0 0x7fa5e0195f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-13T04:32:19.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.582+0000 7fa5e63f2700 1 --2- 172.21.15.156:0/3404865898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5cc0035c0 0x7fa5e0195f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-13T04:32:19.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.582+0000 7fa5e6bf3700 1 --2- 172.21.15.156:0/3404865898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5e019ae00 0x7fa5e019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:32:19.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.582+0000 7fa5e6bf3700 1 -- 172.21.15.156:0/3404865898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5e019ae00 msgr2=0x7fa5e019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-09-13T04:32:19.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.583+0000 7fa5e6bf3700 1 -- 172.21.15.156:0/3404865898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5e019ae00 msgr2=0x7fa5e019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-13T04:32:19.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.583+0000 7fa5e6bf3700 1 --2- 172.21.15.156:0/3404865898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5e019ae00 0x7fa5e019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-13T04:32:19.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.583+0000 7fa5e6bf3700 1 --2- 172.21.15.156:0/3404865898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5e019ae00 0x7fa5e019b2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-13T04:32:19.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.583+0000 7fa5e5bf1700 1 -- 172.21.15.156:0/3404865898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5cc0035c0 msgr2=0x7fa5e0195f10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-13T04:32:19.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.583+0000 7fa5e5bf1700 1 --2- 172.21.15.156:0/3404865898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5cc0035c0 0x7fa5e0195f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:19.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.583+0000 7fa5e5bf1700 1 -- 172.21.15.156:0/3404865898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5e019ae00 msgr2=0x7fa5e019b2b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-13T04:32:19.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.583+0000 7fa5e5bf1700 1 --2- 172.21.15.156:0/3404865898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5e019ae00 0x7fa5e019b2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:19.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.583+0000 7fa5e5bf1700 1 -- 172.21.15.156:0/3404865898 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d000f8b0 con 0x7fa5e0196450 2023-09-13T04:32:19.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.583+0000 7fa5e5bf1700 1 --2- 172.21.15.156:0/3404865898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5e0196450 0x7fa5e019a8c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc00b550 tx=0x7fa5dc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:32:19.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.583+0000 7fa5d77fe700 1 -- 172.21.15.156:0/3404865898 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc004230 con 0x7fa5e0196450 2023-09-13T04:32:19.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.583+0000 7fa5d77fe700 1 -- 172.21.15.156:0/3404865898 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5dc004390 con 0x7fa5e0196450 2023-09-13T04:32:19.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.583+0000 7fa5e8656700 1 -- 172.21.15.156:0/3404865898 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e01a2f90 con 0x7fa5e0196450 2023-09-13T04:32:19.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.583+0000 7fa5d77fe700 1 -- 172.21.15.156:0/3404865898 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc011af0 con 0x7fa5e0196450 2023-09-13T04:32:19.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.583+0000 7fa5e8656700 1 -- 172.21.15.156:0/3404865898 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e01a3480 con 0x7fa5e0196450 2023-09-13T04:32:19.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.584+0000 7fa5d77fe700 1 -- 172.21.15.156:0/3404865898 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45386+0+0 (secure 0 0 0) 0x7fa5dc011c50 con 0x7fa5e0196450 2023-09-13T04:32:19.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.584+0000 7fa5d77fe700 1 --2- 172.21.15.156:0/3404865898 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fa5c4038880 0x7fa5c403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:32:19.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.584+0000 7fa5d77fe700 1 -- 172.21.15.156:0/3404865898 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa5dc04b8b0 con 0x7fa5e0196450 2023-09-13T04:32:19.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.585+0000 7fa5e63f2700 1 --2- 172.21.15.156:0/3404865898 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fa5c4038880 0x7fa5c403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:32:19.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.585+0000 7fa5e8656700 1 -- 172.21.15.156:0/3404865898 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e0190330 con 0x7fa5e0196450 2023-09-13T04:32:19.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.585+0000 7fa5e63f2700 1 --2- 172.21.15.156:0/3404865898 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fa5c4038880 0x7fa5c403ad40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0005850 tx=0x7fa5d0005ad0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:32:19.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.591+0000 7fa5d77fe700 1 -- 172.21.15.156:0/3404865898 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa5dc01a400 con 0x7fa5e0196450 2023-09-13T04:32:19.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.865+0000 7fa5e8656700 1 -- 172.21.15.156:0/3404865898 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa5e01074a0 con 0x7fa5e0196450 2023-09-13T04:32:19.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.865+0000 7fa5d77fe700 1 -- 172.21.15.156:0/3404865898 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1270 (secure 0 0 0) 0x7fa5dc03e020 con 0x7fa5e0196450 2023-09-13T04:32:19.866 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-13T04:32:19.867 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":3,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","modified":"2023-09-13T04:32:14.531694Z","created":"2023-09-13T04:30:18.425317Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:3300","nonce":0},{"type":"v1","addr":"172.21.15.149:6789","nonce":0}]},"addr":"172.21.15.149:6789/0","public_addr":"172.21.15.149:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-09-13T04:32:19.868 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.867+0000 7fa5e8656700 1 -- 172.21.15.156:0/3404865898 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fa5c4038880 msgr2=0x7fa5c403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:19.868 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.867+0000 7fa5e8656700 1 --2- 172.21.15.156:0/3404865898 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fa5c4038880 0x7fa5c403ad40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0005850 tx=0x7fa5d0005ad0 comp rx=0 tx=0).stop 2023-09-13T04:32:19.868 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.867+0000 7fa5e8656700 1 -- 172.21.15.156:0/3404865898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5e0196450 msgr2=0x7fa5e019a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:19.868 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.867+0000 7fa5e8656700 1 --2- 172.21.15.156:0/3404865898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5e0196450 0x7fa5e019a8c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc00b550 tx=0x7fa5dc00b910 comp rx=0 tx=0).stop 2023-09-13T04:32:19.868 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.868+0000 7fa5e8656700 1 -- 172.21.15.156:0/3404865898 shutdown_connections 2023-09-13T04:32:19.869 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.868+0000 7fa5e8656700 1 --2- 172.21.15.156:0/3404865898 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fa5c4038880 0x7fa5c403ad40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:19.869 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.868+0000 7fa5e8656700 1 --2- 172.21.15.156:0/3404865898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5e019ae00 0x7fa5e019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:19.869 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.868+0000 7fa5e8656700 1 --2- 172.21.15.156:0/3404865898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5cc0035c0 0x7fa5e0195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:19.869 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.868+0000 7fa5e8656700 1 --2- 172.21.15.156:0/3404865898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5e0196450 0x7fa5e019a8c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:19.869 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.868+0000 7fa5e8656700 1 -- 172.21.15.156:0/3404865898 >> 172.21.15.156:0/3404865898 conn(0x7fa5e00752c0 msgr2=0x7fa5e0068620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:32:19.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.868+0000 7fa5e8656700 1 -- 172.21.15.156:0/3404865898 shutdown_connections 2023-09-13T04:32:19.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:19.868+0000 7fa5e8656700 1 -- 172.21.15.156:0/3404865898 wait complete. 2023-09-13T04:32:19.870 INFO:teuthology.orchestra.run.smithi156.stderr:dumped monmap epoch 3 2023-09-13T04:32:19.894 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T04:32:19.894 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: mon.c calling monitor election 2023-09-13T04:32:19.894 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: mon.a calling monitor election 2023-09-13T04:32:19.894 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:32:19.895 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:32:19.895 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:19.895 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:32:19.895 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:32:19.895 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:19.896 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:32:19.896 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:32:19.896 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:32:19.896 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-13T04:32:19.896 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: monmap e3: 3 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-09-13T04:32:19.897 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: fsmap 2023-09-13T04:32:19.897 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: osdmap e4: 0 total, 0 up, 0 in 2023-09-13T04:32:19.897 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: mgrmap e13: x(active, since 56s) 2023-09-13T04:32:19.897 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-09-13T04:32:19.897 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-09-13T04:32:19.898 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-09-13T04:32:19.898 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: mon.b (rank 2) addr [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] is down (out of quorum) 2023-09-13T04:32:19.898 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:19.898 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:19.898 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:19.899 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:20.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T04:32:20.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: mon.c calling monitor election 2023-09-13T04:32:20.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: mon.a calling monitor election 2023-09-13T04:32:20.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:32:20.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:32:20.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:20.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:32:20.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:32:20.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:20.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:32:20.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:32:20.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:32:20.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-13T04:32:20.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: monmap e3: 3 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-09-13T04:32:20.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: fsmap 2023-09-13T04:32:20.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: osdmap e4: 0 total, 0 up, 0 in 2023-09-13T04:32:20.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: mgrmap e13: x(active, since 56s) 2023-09-13T04:32:20.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-09-13T04:32:20.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-09-13T04:32:20.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-09-13T04:32:20.010 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: mon.b (rank 2) addr [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] is down (out of quorum) 2023-09-13T04:32:20.010 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:20.010 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:20.010 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:20.010 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:20.239 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-09-13T04:32:20.240 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph config generate-minimal-conf 2023-09-13T04:32:20.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:20 smithi156 ceph-mon[133675]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:20.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:20 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:32:20.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:20 smithi156 ceph-mon[133675]: from='client.? 172.21.15.156:0/3404865898' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-13T04:32:20.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:20 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:32:20.983 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:32:21.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:20 smithi002 ceph-mon[97651]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:21.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:20 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:32:21.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:20 smithi002 ceph-mon[97651]: from='client.? 172.21.15.156:0/3404865898' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-13T04:32:21.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:20 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:32:21.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T04:32:21.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: mon.c calling monitor election 2023-09-13T04:32:21.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: mon.a calling monitor election 2023-09-13T04:32:21.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:32:21.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:32:21.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:21.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:32:21.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:32:21.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:21.890 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:32:21.890 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:32:21.890 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:32:21.891 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-13T04:32:21.891 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: monmap e3: 3 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-09-13T04:32:21.891 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: fsmap 2023-09-13T04:32:21.891 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: osdmap e4: 0 total, 0 up, 0 in 2023-09-13T04:32:21.892 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: mgrmap e13: x(active, since 56s) 2023-09-13T04:32:21.892 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-09-13T04:32:21.892 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-09-13T04:32:21.893 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-09-13T04:32:21.893 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: mon.b (rank 2) addr [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] is down (out of quorum) 2023-09-13T04:32:21.893 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:21.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:21.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:21.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:21.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:21.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:32:21.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: from='client.? 172.21.15.156:0/3404865898' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-13T04:32:21.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:21 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:32:22.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.569+0000 7fd50c2ac700 1 -- 172.21.15.2:0/1072458987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5040730e0 msgr2=0x7fd5040734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:22.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.569+0000 7fd50c2ac700 1 --2- 172.21.15.2:0/1072458987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5040730e0 0x7fd5040734c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd500009910 tx=0x7fd500009c20 comp rx=0 tx=0).stop 2023-09-13T04:32:22.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.570+0000 7fd50c2ac700 1 -- 172.21.15.2:0/1072458987 shutdown_connections 2023-09-13T04:32:22.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.570+0000 7fd50c2ac700 1 --2- 172.21.15.2:0/1072458987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5040730e0 0x7fd5040734c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:22.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.570+0000 7fd50c2ac700 1 -- 172.21.15.2:0/1072458987 >> 172.21.15.2:0/1072458987 conn(0x7fd5040fc2a0 msgr2=0x7fd5040fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:32:22.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.570+0000 7fd50c2ac700 1 -- 172.21.15.2:0/1072458987 shutdown_connections 2023-09-13T04:32:22.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.570+0000 7fd50c2ac700 1 -- 172.21.15.2:0/1072458987 wait complete. 2023-09-13T04:32:22.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.571+0000 7fd50c2ac700 1 Processor -- start 2023-09-13T04:32:22.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.571+0000 7fd50c2ac700 1 -- start start 2023-09-13T04:32:22.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.572+0000 7fd50c2ac700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5040730e0 0x7fd504198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:32:22.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.572+0000 7fd50c2ac700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5041985b0 0x7fd50419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:32:22.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.572+0000 7fd50c2ac700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd50419cf60 0x7fd50419d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:32:22.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.572+0000 7fd50c2ac700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd504198ca0 con 0x7fd50419cf60 2023-09-13T04:32:22.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.572+0000 7fd50c2ac700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5041a4cd0 con 0x7fd5041985b0 2023-09-13T04:32:22.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.572+0000 7fd50c2ac700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5041a4e10 con 0x7fd5040730e0 2023-09-13T04:32:22.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.572+0000 7fd50a849700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd50419cf60 0x7fd50419d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:32:22.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.572+0000 7fd509847700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5041985b0 0x7fd50419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:32:22.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.572+0000 7fd50a048700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5040730e0 0x7fd504198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:32:22.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.572+0000 7fd50a849700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd50419cf60 0x7fd50419d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50034/0 (socket says 172.21.15.2:50034) 2023-09-13T04:32:22.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.572+0000 7fd50a849700 1 -- 172.21.15.2:0/3987964612 learned_addr learned my addr 172.21.15.2:0/3987964612 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:32:22.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.572+0000 7fd509847700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5041985b0 0x7fd50419ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:43582/0 (socket says 172.21.15.2:43582) 2023-09-13T04:32:22.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.572+0000 7fd50a048700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5040730e0 0x7fd504198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:40704/0 (socket says 172.21.15.2:40704) 2023-09-13T04:32:22.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.573+0000 7fd50a849700 1 -- 172.21.15.2:0/3987964612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5040730e0 msgr2=0x7fd504198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:22.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.573+0000 7fd50a849700 1 --2- 172.21.15.2:0/3987964612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5040730e0 0x7fd504198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:22.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.573+0000 7fd50a849700 1 -- 172.21.15.2:0/3987964612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5041985b0 msgr2=0x7fd50419ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:22.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.573+0000 7fd50a849700 1 --2- 172.21.15.2:0/3987964612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5041985b0 0x7fd50419ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:22.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.573+0000 7fd50a849700 1 -- 172.21.15.2:0/3987964612 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5000095f0 con 0x7fd50419cf60 2023-09-13T04:32:22.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.573+0000 7fd50a849700 1 --2- 172.21.15.2:0/3987964612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd50419cf60 0x7fd50419d410 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00ba50 tx=0x7fd4fc00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:32:22.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.573+0000 7fd4fb7fe700 1 -- 172.21.15.2:0/3987964612 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc00c930 con 0x7fd50419cf60 2023-09-13T04:32:22.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.574+0000 7fd4fb7fe700 1 -- 172.21.15.2:0/3987964612 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4fc014440 con 0x7fd50419cf60 2023-09-13T04:32:22.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.574+0000 7fd50c2ac700 1 -- 172.21.15.2:0/3987964612 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5041a50f0 con 0x7fd50419cf60 2023-09-13T04:32:22.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.574+0000 7fd4fb7fe700 1 -- 172.21.15.2:0/3987964612 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc0075b0 con 0x7fd50419cf60 2023-09-13T04:32:22.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.574+0000 7fd50c2ac700 1 -- 172.21.15.2:0/3987964612 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5041a55f0 con 0x7fd50419cf60 2023-09-13T04:32:22.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.575+0000 7fd50c2ac700 1 -- 172.21.15.2:0/3987964612 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd50410a450 con 0x7fd50419cf60 2023-09-13T04:32:22.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.576+0000 7fd4fb7fe700 1 -- 172.21.15.2:0/3987964612 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45386+0+0 (secure 0 0 0) 0x7fd4fc0145b0 con 0x7fd50419cf60 2023-09-13T04:32:22.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.576+0000 7fd4fb7fe700 1 --2- 172.21.15.2:0/3987964612 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd4f0038880 0x7fd4f003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:32:22.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.576+0000 7fd4fb7fe700 1 -- 172.21.15.2:0/3987964612 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd4fc04a590 con 0x7fd50419cf60 2023-09-13T04:32:22.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.576+0000 7fd50a048700 1 --2- 172.21.15.2:0/3987964612 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd4f0038880 0x7fd4f003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:32:22.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.577+0000 7fd50a048700 1 --2- 172.21.15.2:0/3987964612 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd4f0038880 0x7fd4f003ad40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd500006010 tx=0x7fd50000b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:32:22.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.581+0000 7fd4fb7fe700 1 -- 172.21.15.2:0/3987964612 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd4fc00cc20 con 0x7fd50419cf60 2023-09-13T04:32:22.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:22 smithi149 ceph-mon[133315]: mon.b calling monitor election 2023-09-13T04:32:22.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:22 smithi149 ceph-mon[133315]: Updating smithi002:/etc/ceph/ceph.conf 2023-09-13T04:32:22.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:22 smithi149 ceph-mon[133315]: Updating smithi149:/etc/ceph/ceph.conf 2023-09-13T04:32:22.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:22 smithi149 ceph-mon[133315]: Updating smithi156:/etc/ceph/ceph.conf 2023-09-13T04:32:22.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:22 smithi149 ceph-mon[133315]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:22.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:22 smithi149 ceph-mon[133315]: mon.b calling monitor election 2023-09-13T04:32:22.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:22 smithi149 ceph-mon[133315]: mon.c calling monitor election 2023-09-13T04:32:22.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:22 smithi149 ceph-mon[133315]: mon.a calling monitor election 2023-09-13T04:32:22.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:22 smithi149 ceph-mon[133315]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-13T04:32:22.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:22 smithi149 ceph-mon[133315]: monmap e3: 3 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-09-13T04:32:22.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:22 smithi149 ceph-mon[133315]: fsmap 2023-09-13T04:32:22.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:22 smithi149 ceph-mon[133315]: osdmap e4: 0 total, 0 up, 0 in 2023-09-13T04:32:22.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:22 smithi149 ceph-mon[133315]: mgrmap e13: x(active, since 58s) 2023-09-13T04:32:22.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:22 smithi149 ceph-mon[133315]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-09-13T04:32:22.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:22 smithi149 ceph-mon[133315]: Cluster is now healthy 2023-09-13T04:32:22.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:22 smithi149 ceph-mon[133315]: overall HEALTH_OK 2023-09-13T04:32:22.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:22 smithi149 ceph-mon[133315]: Updating smithi149:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:32:22.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:22 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:22.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:22 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:22.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:22 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:22.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:22 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:22.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:22 smithi156 ceph-mon[133675]: mon.b calling monitor election 2023-09-13T04:32:22.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:22 smithi156 ceph-mon[133675]: Updating smithi002:/etc/ceph/ceph.conf 2023-09-13T04:32:22.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:22 smithi156 ceph-mon[133675]: Updating smithi149:/etc/ceph/ceph.conf 2023-09-13T04:32:22.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:22 smithi156 ceph-mon[133675]: Updating smithi156:/etc/ceph/ceph.conf 2023-09-13T04:32:22.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:22 smithi156 ceph-mon[133675]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:22.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:22 smithi156 ceph-mon[133675]: mon.b calling monitor election 2023-09-13T04:32:22.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:22 smithi156 ceph-mon[133675]: mon.c calling monitor election 2023-09-13T04:32:22.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:22 smithi156 ceph-mon[133675]: mon.a calling monitor election 2023-09-13T04:32:22.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:22 smithi156 ceph-mon[133675]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-13T04:32:22.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:22 smithi156 ceph-mon[133675]: monmap e3: 3 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-09-13T04:32:22.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:22 smithi156 ceph-mon[133675]: fsmap 2023-09-13T04:32:22.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:22 smithi156 ceph-mon[133675]: osdmap e4: 0 total, 0 up, 0 in 2023-09-13T04:32:22.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:22 smithi156 ceph-mon[133675]: mgrmap e13: x(active, since 58s) 2023-09-13T04:32:22.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:22 smithi156 ceph-mon[133675]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-09-13T04:32:22.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:22 smithi156 ceph-mon[133675]: Cluster is now healthy 2023-09-13T04:32:22.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:22 smithi156 ceph-mon[133675]: overall HEALTH_OK 2023-09-13T04:32:22.690 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:22 smithi156 ceph-mon[133675]: Updating smithi149:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:32:22.690 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:22 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:22.690 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:22 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:22.690 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:22 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:22.690 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:22 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:22.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:22 smithi002 ceph-mon[97651]: mon.b calling monitor election 2023-09-13T04:32:22.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:22 smithi002 ceph-mon[97651]: Updating smithi002:/etc/ceph/ceph.conf 2023-09-13T04:32:22.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:22 smithi002 ceph-mon[97651]: Updating smithi149:/etc/ceph/ceph.conf 2023-09-13T04:32:22.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:22 smithi002 ceph-mon[97651]: Updating smithi156:/etc/ceph/ceph.conf 2023-09-13T04:32:22.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:22 smithi002 ceph-mon[97651]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:22.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:22 smithi002 ceph-mon[97651]: mon.b calling monitor election 2023-09-13T04:32:22.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:22 smithi002 ceph-mon[97651]: mon.c calling monitor election 2023-09-13T04:32:22.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:22 smithi002 ceph-mon[97651]: mon.a calling monitor election 2023-09-13T04:32:22.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:22 smithi002 ceph-mon[97651]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-13T04:32:22.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:22 smithi002 ceph-mon[97651]: monmap e3: 3 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-09-13T04:32:22.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:22 smithi002 ceph-mon[97651]: fsmap 2023-09-13T04:32:22.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:22 smithi002 ceph-mon[97651]: osdmap e4: 0 total, 0 up, 0 in 2023-09-13T04:32:22.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:22 smithi002 ceph-mon[97651]: mgrmap e13: x(active, since 58s) 2023-09-13T04:32:22.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:22 smithi002 ceph-mon[97651]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-09-13T04:32:22.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:22 smithi002 ceph-mon[97651]: Cluster is now healthy 2023-09-13T04:32:22.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:22 smithi002 ceph-mon[97651]: overall HEALTH_OK 2023-09-13T04:32:22.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:22 smithi002 ceph-mon[97651]: Updating smithi149:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:32:22.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:22 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:22.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:22 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:22.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:22 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:22.760 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:22 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:22.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.803+0000 7fd50c2ac700 1 -- 172.21.15.2:0/3987964612 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fd50410cc90 con 0x7fd50419cf60 2023-09-13T04:32:22.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.803+0000 7fd4fb7fe700 1 -- 172.21.15.2:0/3987964612 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7fd4fc012340 con 0x7fd50419cf60 2023-09-13T04:32:22.805 INFO:teuthology.orchestra.run.smithi002.stdout:# minimal ceph.conf for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:32:22.806 INFO:teuthology.orchestra.run.smithi002.stdout:[global] 2023-09-13T04:32:22.806 INFO:teuthology.orchestra.run.smithi002.stdout: fsid = 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:32:22.806 INFO:teuthology.orchestra.run.smithi002.stdout: mon_host = [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] 2023-09-13T04:32:22.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.806+0000 7fd50c2ac700 1 -- 172.21.15.2:0/3987964612 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd4f0038880 msgr2=0x7fd4f003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:22.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.806+0000 7fd50c2ac700 1 --2- 172.21.15.2:0/3987964612 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd4f0038880 0x7fd4f003ad40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd500006010 tx=0x7fd50000b540 comp rx=0 tx=0).stop 2023-09-13T04:32:22.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.806+0000 7fd50c2ac700 1 -- 172.21.15.2:0/3987964612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd50419cf60 msgr2=0x7fd50419d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:22.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.806+0000 7fd50c2ac700 1 --2- 172.21.15.2:0/3987964612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd50419cf60 0x7fd50419d410 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00ba50 tx=0x7fd4fc00bd60 comp rx=0 tx=0).stop 2023-09-13T04:32:22.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.806+0000 7fd50c2ac700 1 -- 172.21.15.2:0/3987964612 shutdown_connections 2023-09-13T04:32:22.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.806+0000 7fd50c2ac700 1 --2- 172.21.15.2:0/3987964612 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd4f0038880 0x7fd4f003ad40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:22.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.806+0000 7fd50c2ac700 1 --2- 172.21.15.2:0/3987964612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd50419cf60 0x7fd50419d410 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:22.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.806+0000 7fd50c2ac700 1 --2- 172.21.15.2:0/3987964612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5040730e0 0x7fd504198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:22.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.806+0000 7fd50c2ac700 1 --2- 172.21.15.2:0/3987964612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5041985b0 0x7fd50419ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:22.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.806+0000 7fd50c2ac700 1 -- 172.21.15.2:0/3987964612 >> 172.21.15.2:0/3987964612 conn(0x7fd5040fc2a0 msgr2=0x7fd504107590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:32:22.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.806+0000 7fd50c2ac700 1 -- 172.21.15.2:0/3987964612 shutdown_connections 2023-09-13T04:32:22.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:22.806+0000 7fd50c2ac700 1 -- 172.21.15.2:0/3987964612 wait complete. 2023-09-13T04:32:23.271 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-09-13T04:32:23.272 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-09-13T04:32:23.272 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.conf 2023-09-13T04:32:23.348 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-09-13T04:32:23.348 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-13T04:32:23.464 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2023-09-13T04:32:23.465 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.conf 2023-09-13T04:32:23.502 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2023-09-13T04:32:23.503 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-13T04:32:23.574 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-13T04:32:23.574 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.conf 2023-09-13T04:32:23.610 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-13T04:32:23.610 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-13T04:32:23.641 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:23 smithi002 ceph-mon[97651]: Updating smithi156:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:32:23.642 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:23 smithi002 ceph-mon[97651]: Updating smithi002:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:32:23.642 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:23.642 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:23.643 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:23.643 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:32:23.643 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:23.643 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:23.643 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:23.644 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:23.644 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T04:32:23.644 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-13T04:32:23.644 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:23.644 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:23 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3987964612' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:23.645 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:23.680 INFO:tasks.cephadm:Adding mgr.x on smithi002 2023-09-13T04:32:23.680 INFO:tasks.cephadm:Adding mgr.y on smithi149 2023-09-13T04:32:23.681 INFO:tasks.cephadm:Adding mgr.z on smithi156 2023-09-13T04:32:23.681 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph orch apply mgr '3;smithi002=x;smithi149=y;smithi156=z' 2023-09-13T04:32:23.745 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:23 smithi156 ceph-mon[133675]: Updating smithi156:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:32:23.746 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:23 smithi156 ceph-mon[133675]: Updating smithi002:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:32:23.746 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:23.746 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:23.746 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:23.747 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:32:23.747 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:23.747 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:23.747 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:23.747 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:23.748 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T04:32:23.748 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-13T04:32:23.748 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:23.748 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:23 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3987964612' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:23.748 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:23.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:23 smithi149 ceph-mon[133315]: Updating smithi156:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:32:23.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:23 smithi149 ceph-mon[133315]: Updating smithi002:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:32:23.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:23.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:23.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:23.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:32:23.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:23.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:23.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:23.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:23.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T04:32:23.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-13T04:32:23.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:23.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:23 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3987964612' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:23.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:24.274 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-13T04:32:24.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:24 smithi002 ceph-mon[97651]: Reconfiguring mon.a (unknown last config time)... 2023-09-13T04:32:24.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:24 smithi002 ceph-mon[97651]: Reconfiguring daemon mon.a on smithi002 2023-09-13T04:32:24.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:24 smithi002 ceph-mon[97651]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:24.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:24.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:24.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T04:32:24.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-13T04:32:24.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:24.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:24 smithi156 ceph-mon[133675]: Reconfiguring mon.a (unknown last config time)... 2023-09-13T04:32:24.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:24 smithi156 ceph-mon[133675]: Reconfiguring daemon mon.a on smithi002 2023-09-13T04:32:24.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:24 smithi156 ceph-mon[133675]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:24.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:24.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:24.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T04:32:24.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-13T04:32:24.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:24.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:24 smithi149 ceph-mon[133315]: Reconfiguring mon.a (unknown last config time)... 2023-09-13T04:32:24.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:24 smithi149 ceph-mon[133315]: Reconfiguring daemon mon.a on smithi002 2023-09-13T04:32:24.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:24 smithi149 ceph-mon[133315]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:24.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:24.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:24.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T04:32:24.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-13T04:32:24.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:25.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.443+0000 7f853d5a2700 1 -- 172.21.15.156:0/3852419075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8538100d80 msgr2=0x7f85381011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:25.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.443+0000 7f853d5a2700 1 --2- 172.21.15.156:0/3852419075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8538100d80 0x7f85381011a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8528009910 tx=0x7f8528009c20 comp rx=0 tx=0).stop 2023-09-13T04:32:25.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.443+0000 7f853d5a2700 1 -- 172.21.15.156:0/3852419075 shutdown_connections 2023-09-13T04:32:25.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.443+0000 7f853d5a2700 1 --2- 172.21.15.156:0/3852419075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8538100d80 0x7f85381011a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:25.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.443+0000 7f853d5a2700 1 -- 172.21.15.156:0/3852419075 >> 172.21.15.156:0/3852419075 conn(0x7f85380fa4f0 msgr2=0x7f85380fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:32:25.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.443+0000 7f853d5a2700 1 -- 172.21.15.156:0/3852419075 shutdown_connections 2023-09-13T04:32:25.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.443+0000 7f853d5a2700 1 -- 172.21.15.156:0/3852419075 wait complete. 2023-09-13T04:32:25.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.444+0000 7f853d5a2700 1 Processor -- start 2023-09-13T04:32:25.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.444+0000 7f853d5a2700 1 -- start start 2023-09-13T04:32:25.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.444+0000 7f853d5a2700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8538100d80 0x7f8538195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:32:25.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.444+0000 7f853d5a2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85381960c0 0x7f853819b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:32:25.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.444+0000 7f853d5a2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8538196540 0x7f853819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:32:25.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.444+0000 7f853d5a2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853819de60 con 0x7f85381960c0 2023-09-13T04:32:25.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.444+0000 7f853d5a2700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853819dfd0 con 0x7f8538100d80 2023-09-13T04:32:25.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.444+0000 7f853d5a2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853819e190 con 0x7f8538196540 2023-09-13T04:32:25.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.444+0000 7f85377fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8538196540 0x7f853819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:32:25.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.444+0000 7f85377fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8538196540 0x7f853819d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37056/0 (socket says 172.21.15.156:37056) 2023-09-13T04:32:25.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.444+0000 7f85377fe700 1 -- 172.21.15.156:0/705240103 learned_addr learned my addr 172.21.15.156:0/705240103 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-13T04:32:25.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.444+0000 7f85367fc700 1 --2- 172.21.15.156:0/705240103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85381960c0 0x7f853819b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:32:25.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.445+0000 7f85377fe700 1 -- 172.21.15.156:0/705240103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8538100d80 msgr2=0x7f8538195b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:25.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.445+0000 7f85377fe700 1 --2- 172.21.15.156:0/705240103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8538100d80 0x7f8538195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:25.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.445+0000 7f85377fe700 1 -- 172.21.15.156:0/705240103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85381960c0 msgr2=0x7f853819b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:25.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.445+0000 7f85377fe700 1 --2- 172.21.15.156:0/705240103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85381960c0 0x7f853819b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:25.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.445+0000 7f85377fe700 1 -- 172.21.15.156:0/705240103 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85280095f0 con 0x7f8538196540 2023-09-13T04:32:25.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.445+0000 7f85367fc700 1 --2- 172.21.15.156:0/705240103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85381960c0 0x7f853819b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:32:25.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.445+0000 7f85377fe700 1 --2- 172.21.15.156:0/705240103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8538196540 0x7f853819d6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f853000ba50 tx=0x7f853000be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:32:25.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.445+0000 7f851ffff700 1 -- 172.21.15.156:0/705240103 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853000cab0 con 0x7f8538196540 2023-09-13T04:32:25.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.445+0000 7f851ffff700 1 -- 172.21.15.156:0/705240103 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f853000cc10 con 0x7f8538196540 2023-09-13T04:32:25.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.445+0000 7f851ffff700 1 -- 172.21.15.156:0/705240103 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85300075b0 con 0x7f8538196540 2023-09-13T04:32:25.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.445+0000 7f853d5a2700 1 -- 172.21.15.156:0/705240103 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f853819e470 con 0x7f8538196540 2023-09-13T04:32:25.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.445+0000 7f853d5a2700 1 -- 172.21.15.156:0/705240103 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853819e9a0 con 0x7f8538196540 2023-09-13T04:32:25.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.446+0000 7f851ffff700 1 -- 172.21.15.156:0/705240103 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 13) v1 ==== 45386+0+0 (secure 0 0 0) 0x7f8530007710 con 0x7f8538196540 2023-09-13T04:32:25.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.446+0000 7f853d5a2700 1 -- 172.21.15.156:0/705240103 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85381054f0 con 0x7f8538196540 2023-09-13T04:32:25.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.446+0000 7f851ffff700 1 --2- 172.21.15.156:0/705240103 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8520038880 0x7f852003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:32:25.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.446+0000 7f851ffff700 1 -- 172.21.15.156:0/705240103 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f853004a830 con 0x7f8538196540 2023-09-13T04:32:25.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.446+0000 7f8536ffd700 1 --2- 172.21.15.156:0/705240103 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8520038880 0x7f852003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:32:25.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.447+0000 7f8536ffd700 1 --2- 172.21.15.156:0/705240103 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8520038880 0x7f852003ad40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8528004690 tx=0x7f8528005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:32:25.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.451+0000 7f851ffff700 1 -- 172.21.15.156:0/705240103 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f853001b8b0 con 0x7f8538196540 2023-09-13T04:32:25.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.666+0000 7f853d5a2700 1 -- 172.21.15.156:0/705240103 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi002=x;smithi149=y;smithi156=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f85380fbe90 con 0x7f8520038880 2023-09-13T04:32:25.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.673+0000 7f851ffff700 1 -- 172.21.15.156:0/705240103 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f85380fbe90 con 0x7f8520038880 2023-09-13T04:32:25.674 INFO:teuthology.orchestra.run.smithi156.stdout:Scheduled mgr update... 2023-09-13T04:32:25.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.675+0000 7f853d5a2700 1 -- 172.21.15.156:0/705240103 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8520038880 msgr2=0x7f852003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:25.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.675+0000 7f853d5a2700 1 --2- 172.21.15.156:0/705240103 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8520038880 0x7f852003ad40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8528004690 tx=0x7f8528005fb0 comp rx=0 tx=0).stop 2023-09-13T04:32:25.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.675+0000 7f853d5a2700 1 -- 172.21.15.156:0/705240103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8538196540 msgr2=0x7f853819d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:25.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.675+0000 7f853d5a2700 1 --2- 172.21.15.156:0/705240103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8538196540 0x7f853819d6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f853000ba50 tx=0x7f853000be10 comp rx=0 tx=0).stop 2023-09-13T04:32:25.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.675+0000 7f853d5a2700 1 -- 172.21.15.156:0/705240103 shutdown_connections 2023-09-13T04:32:25.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.675+0000 7f853d5a2700 1 --2- 172.21.15.156:0/705240103 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8520038880 0x7f852003ad40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:25.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.675+0000 7f853d5a2700 1 --2- 172.21.15.156:0/705240103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8538196540 0x7f853819d6c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:25.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.675+0000 7f853d5a2700 1 --2- 172.21.15.156:0/705240103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8538100d80 0x7f8538195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:25.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.675+0000 7f853d5a2700 1 --2- 172.21.15.156:0/705240103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85381960c0 0x7f853819b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:25.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.675+0000 7f853d5a2700 1 -- 172.21.15.156:0/705240103 >> 172.21.15.156:0/705240103 conn(0x7f85380fa4f0 msgr2=0x7f85380fb230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:32:25.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.676+0000 7f853d5a2700 1 -- 172.21.15.156:0/705240103 shutdown_connections 2023-09-13T04:32:25.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:32:25.676+0000 7f853d5a2700 1 -- 172.21.15.156:0/705240103 wait complete. 2023-09-13T04:32:25.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:25 smithi156 ceph-mon[133675]: Reconfiguring mon.b (monmap changed)... 2023-09-13T04:32:25.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:25 smithi156 ceph-mon[133675]: Reconfiguring daemon mon.b on smithi149 2023-09-13T04:32:25.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:25 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:25.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:25 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:25.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:25 smithi002 ceph-mon[97651]: Reconfiguring mon.b (monmap changed)... 2023-09-13T04:32:25.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:25 smithi002 ceph-mon[97651]: Reconfiguring daemon mon.b on smithi149 2023-09-13T04:32:25.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:25 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:25.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:25 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:25.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:25 smithi149 ceph-mon[133315]: Reconfiguring mon.b (monmap changed)... 2023-09-13T04:32:25.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:25 smithi149 ceph-mon[133315]: Reconfiguring daemon mon.b on smithi149 2023-09-13T04:32:25.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:25 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:25.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:25 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:26.015 DEBUG:teuthology.orchestra.run.smithi149:mgr.y> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mgr.y.service 2023-09-13T04:32:26.018 DEBUG:teuthology.orchestra.run.smithi156:mgr.z> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mgr.z.service 2023-09-13T04:32:26.048 INFO:journalctl@ceph.mgr.y.smithi149.stdout:-- Logs begin at Wed 2023-09-13 04:09:45 UTC. -- 2023-09-13T04:32:26.058 INFO:tasks.cephadm:Deploying OSDs... 2023-09-13T04:32:26.059 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-09-13T04:32:26.059 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/scratch_devs of=/dev/stdout 2023-09-13T04:32:26.086 INFO:journalctl@ceph.mgr.z.smithi156.stdout:-- Logs begin at Wed 2023-09-13 04:09:45 UTC. -- 2023-09-13T04:32:26.108 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-13T04:32:26.108 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_1 2023-09-13T04:32:26.196 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-13T04:32:26.196 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-13T04:32:26.196 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-09-13T04:32:26.197 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T04:32:26.197 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2023-09-13T04:32:26.197 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2023-09-13 04:31:36.145829900 +0000 2023-09-13T04:32:26.197 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2023-09-13 04:25:48.788457656 +0000 2023-09-13T04:32:26.197 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2023-09-13 04:25:48.788457656 +0000 2023-09-13T04:32:26.198 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2023-09-13 04:25:48.766458126 +0000 2023-09-13T04:32:26.198 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-13T04:32:26.267 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2023-09-13T04:32:26.268 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2023-09-13T04:32:26.268 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000200222 s, 2.6 MB/s 2023-09-13T04:32:26.271 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-13T04:32:26.317 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_2 2023-09-13T04:32:26.401 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-13T04:32:26.401 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-13T04:32:26.401 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-09-13T04:32:26.402 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T04:32:26.402 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2023-09-13T04:32:26.402 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2023-09-13 04:31:36.216828349 +0000 2023-09-13T04:32:26.403 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2023-09-13 04:25:49.630439686 +0000 2023-09-13T04:32:26.403 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2023-09-13 04:25:49.630439686 +0000 2023-09-13T04:32:26.403 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2023-09-13 04:25:49.609440135 +0000 2023-09-13T04:32:26.403 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-13T04:32:26.508 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2023-09-13T04:32:26.509 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2023-09-13T04:32:26.509 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000186545 s, 2.7 MB/s 2023-09-13T04:32:26.511 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-13T04:32:26.557 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_3 2023-09-13T04:32:26.641 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-13T04:32:26.642 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-13T04:32:26.643 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-09-13T04:32:26.643 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T04:32:26.643 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2023-09-13T04:32:26.643 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2023-09-13 04:31:36.286826821 +0000 2023-09-13T04:32:26.644 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2023-09-13 04:25:50.453422121 +0000 2023-09-13T04:32:26.644 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2023-09-13 04:25:50.453422121 +0000 2023-09-13T04:32:26.644 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2023-09-13 04:25:50.431422590 +0000 2023-09-13T04:32:26.645 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-13T04:32:26.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:26 smithi156 ceph-mon[133675]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:26.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:26 smithi156 ceph-mon[133675]: Reconfiguring mon.c (monmap changed)... 2023-09-13T04:32:26.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:26 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T04:32:26.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:26 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-13T04:32:26.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:26 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:26.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:26 smithi156 ceph-mon[133675]: Reconfiguring daemon mon.c on smithi156 2023-09-13T04:32:26.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:26 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:26.742 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:26 smithi002 ceph-mon[97651]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:26.742 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:26 smithi002 ceph-mon[97651]: Reconfiguring mon.c (monmap changed)... 2023-09-13T04:32:26.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:26 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T04:32:26.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:26 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-13T04:32:26.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:26 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:26.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:26 smithi002 ceph-mon[97651]: Reconfiguring daemon mon.c on smithi156 2023-09-13T04:32:26.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:26 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:26.748 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2023-09-13T04:32:26.748 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2023-09-13T04:32:26.749 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000189708 s, 2.7 MB/s 2023-09-13T04:32:26.751 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-13T04:32:26.797 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_4 2023-09-13T04:32:26.883 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-13T04:32:26.883 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-13T04:32:26.883 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-09-13T04:32:26.884 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T04:32:26.884 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2023-09-13T04:32:26.884 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2023-09-13 04:31:36.347825489 +0000 2023-09-13T04:32:26.884 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2023-09-13 04:25:51.281404449 +0000 2023-09-13T04:32:26.885 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2023-09-13 04:25:51.281404449 +0000 2023-09-13T04:32:26.885 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2023-09-13 04:25:51.260404897 +0000 2023-09-13T04:32:26.885 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-13T04:32:26.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:26 smithi149 ceph-mon[133315]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:26.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:26 smithi149 ceph-mon[133315]: Reconfiguring mon.c (monmap changed)... 2023-09-13T04:32:26.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:26 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T04:32:26.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:26 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-13T04:32:26.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:26 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:26.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:26 smithi149 ceph-mon[133315]: Reconfiguring daemon mon.c on smithi156 2023-09-13T04:32:26.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:26 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:26.989 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2023-09-13T04:32:26.989 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2023-09-13T04:32:26.989 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000175073 s, 2.9 MB/s 2023-09-13T04:32:26.991 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-13T04:32:27.038 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2023-09-13T04:32:27.038 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/scratch_devs of=/dev/stdout 2023-09-13T04:32:27.059 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-13T04:32:27.060 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_1 2023-09-13T04:32:27.120 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-13T04:32:27.121 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-13T04:32:27.121 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 6h/6d Inode: 382434 Links: 1 2023-09-13T04:32:27.121 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T04:32:27.121 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2023-09-13T04:32:27.121 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2023-09-13 04:32:13.888744396 +0000 2023-09-13T04:32:27.122 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2023-09-13 04:22:33.187524648 +0000 2023-09-13T04:32:27.122 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2023-09-13 04:22:33.187524648 +0000 2023-09-13T04:32:27.122 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: - 2023-09-13T04:32:27.122 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-13T04:32:27.191 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2023-09-13T04:32:27.192 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2023-09-13T04:32:27.192 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000263504 s, 1.9 MB/s 2023-09-13T04:32:27.193 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-13T04:32:27.254 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_2 2023-09-13T04:32:27.314 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-13T04:32:27.314 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-13T04:32:27.314 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 6h/6d Inode: 390158 Links: 1 2023-09-13T04:32:27.315 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T04:32:27.315 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2023-09-13T04:32:27.315 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2023-09-13 04:32:13.888744396 +0000 2023-09-13T04:32:27.315 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2023-09-13 04:22:33.691514650 +0000 2023-09-13T04:32:27.315 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2023-09-13 04:22:33.691514650 +0000 2023-09-13T04:32:27.316 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: - 2023-09-13T04:32:27.316 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-13T04:32:27.382 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2023-09-13T04:32:27.382 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2023-09-13T04:32:27.382 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000279818 s, 1.8 MB/s 2023-09-13T04:32:27.383 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-13T04:32:27.443 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_3 2023-09-13T04:32:27.501 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-13T04:32:27.501 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-13T04:32:27.501 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 6h/6d Inode: 391294 Links: 1 2023-09-13T04:32:27.501 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T04:32:27.502 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2023-09-13T04:32:27.502 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2023-09-13 04:32:13.888744396 +0000 2023-09-13T04:32:27.502 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2023-09-13 04:22:34.198504593 +0000 2023-09-13T04:32:27.502 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2023-09-13 04:22:34.198504593 +0000 2023-09-13T04:32:27.503 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: - 2023-09-13T04:32:27.503 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-13T04:32:27.567 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2023-09-13T04:32:27.567 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2023-09-13T04:32:27.567 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000230356 s, 2.2 MB/s 2023-09-13T04:32:27.568 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-13T04:32:27.627 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_4 2023-09-13T04:32:27.686 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-13T04:32:27.686 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-13T04:32:27.687 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 6h/6d Inode: 391368 Links: 1 2023-09-13T04:32:27.687 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T04:32:27.687 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2023-09-13T04:32:27.687 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2023-09-13 04:32:13.888744396 +0000 2023-09-13T04:32:27.687 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2023-09-13 04:22:34.702494594 +0000 2023-09-13T04:32:27.688 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2023-09-13 04:22:34.702494594 +0000 2023-09-13T04:32:27.688 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: - 2023-09-13T04:32:27.688 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-13T04:32:27.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:27 smithi149 ceph-mon[133315]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi002=x;smithi149=y;smithi156=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:32:27.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:27 smithi149 ceph-mon[133315]: Saving service mgr spec with placement smithi002=x;smithi149=y;smithi156=z;count:3 2023-09-13T04:32:27.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:27 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:27.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:27 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:27.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:27 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:32:27.752 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:27 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:27.752 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:27 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:32:27.752 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:27 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:27.752 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:27 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-13T04:32:27.752 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:27 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-13T04:32:27.753 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:27 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-13T04:32:27.753 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:27 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:27.754 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2023-09-13T04:32:27.754 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2023-09-13T04:32:27.755 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000171641 s, 3.0 MB/s 2023-09-13T04:32:27.755 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-13T04:32:27.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:27 smithi002 ceph-mon[97651]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi002=x;smithi149=y;smithi156=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:32:27.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:27 smithi002 ceph-mon[97651]: Saving service mgr spec with placement smithi002=x;smithi149=y;smithi156=z;count:3 2023-09-13T04:32:27.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:27 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:27.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:27 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:27.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:27 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:32:27.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:27 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:27.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:27 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:32:27.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:27 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:27.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:27 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-13T04:32:27.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:27 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-13T04:32:27.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:27 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-13T04:32:27.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:27 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:27.758 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:27 smithi156 ceph-mon[133675]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi002=x;smithi149=y;smithi156=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:32:27.759 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:27 smithi156 ceph-mon[133675]: Saving service mgr spec with placement smithi002=x;smithi149=y;smithi156=z;count:3 2023-09-13T04:32:27.759 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:27 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:27.759 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:27 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:27.759 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:27 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:32:27.759 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:27 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:27.760 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:27 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:32:27.760 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:27 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:27.760 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:27 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-13T04:32:27.760 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:27 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-13T04:32:27.760 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:27 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-13T04:32:27.760 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:27 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:27.813 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-13T04:32:27.814 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/scratch_devs of=/dev/stdout 2023-09-13T04:32:27.832 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-13T04:32:27.832 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_1 2023-09-13T04:32:27.890 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-13T04:32:27.890 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-13T04:32:27.890 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 389448 Links: 1 2023-09-13T04:32:27.890 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T04:32:27.891 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-09-13T04:32:27.891 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-09-13 04:32:05.505355479 +0000 2023-09-13T04:32:27.891 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-09-13 04:22:32.381486531 +0000 2023-09-13T04:32:27.891 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-09-13 04:22:32.381486531 +0000 2023-09-13T04:32:27.891 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-09-13T04:32:27.892 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-13T04:32:27.958 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-09-13T04:32:27.958 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-09-13T04:32:27.958 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000115705 s, 4.4 MB/s 2023-09-13T04:32:27.959 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-13T04:32:28.018 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_2 2023-09-13T04:32:28.083 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-13T04:32:28.083 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-13T04:32:28.083 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 384306 Links: 1 2023-09-13T04:32:28.083 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T04:32:28.084 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-09-13T04:32:28.084 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-09-13 04:32:05.505355479 +0000 2023-09-13T04:32:28.084 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-09-13 04:22:32.884478568 +0000 2023-09-13T04:32:28.084 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-09-13 04:22:32.884478568 +0000 2023-09-13T04:32:28.084 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-09-13T04:32:28.085 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-13T04:32:28.151 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-09-13T04:32:28.151 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-09-13T04:32:28.151 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000111569 s, 4.6 MB/s 2023-09-13T04:32:28.153 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-13T04:32:28.212 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_3 2023-09-13T04:32:28.273 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-13T04:32:28.273 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-13T04:32:28.273 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 384452 Links: 1 2023-09-13T04:32:28.274 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T04:32:28.274 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-09-13T04:32:28.274 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-09-13 04:32:28.227995348 +0000 2023-09-13T04:32:28.274 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-09-13 04:22:33.383470668 +0000 2023-09-13T04:32:28.274 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-09-13 04:22:33.383470668 +0000 2023-09-13T04:32:28.275 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-09-13T04:32:28.275 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-13T04:32:28.339 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:28 smithi156 systemd[1]: Starting Ceph mgr.z for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:32:28.342 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-09-13T04:32:28.342 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-09-13T04:32:28.342 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 9.7193e-05 s, 5.3 MB/s 2023-09-13T04:32:28.344 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-13T04:32:28.403 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_4 2023-09-13T04:32:28.463 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-13T04:32:28.463 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-13T04:32:28.463 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 384550 Links: 1 2023-09-13T04:32:28.464 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T04:32:28.464 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-09-13T04:32:28.464 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-09-13 04:32:28.227995348 +0000 2023-09-13T04:32:28.464 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-09-13 04:22:33.884462737 +0000 2023-09-13T04:32:28.464 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-09-13 04:22:33.884462737 +0000 2023-09-13T04:32:28.465 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-09-13T04:32:28.465 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-13T04:32:28.529 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-09-13T04:32:28.529 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-09-13T04:32:28.529 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000106563 s, 4.8 MB/s 2023-09-13T04:32:28.530 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-13T04:32:28.588 INFO:tasks.cephadm:Deploying osd.0 on smithi002 with /dev/vg_nvme/lv_4... 2023-09-13T04:32:28.589 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-13T04:32:28.604 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:28 smithi156 ceph-mon[133675]: Deploying daemon mgr.z on smithi156 2023-09-13T04:32:28.604 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:28 smithi156 ceph-mon[133675]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:28.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:28 smithi002 ceph-mon[97651]: Deploying daemon mgr.z on smithi156 2023-09-13T04:32:28.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:28 smithi002 ceph-mon[97651]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:28.862 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:28 smithi156 podman[135719]: 2023-09-13T04:32:28.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:28 smithi149 ceph-mon[133315]: Deploying daemon mgr.z on smithi156 2023-09-13T04:32:28.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:28 smithi149 ceph-mon[133315]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:29.160 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:28 smithi156 bash[135719]: 6b87df66ee6d9f026c8c45a3635e563a571da56f0edc5cd725b2d965e3bf0a7f 2023-09-13T04:32:29.160 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:28 smithi156 ceph-mgr[135821]: -- 172.21.15.156:0/495946033 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559775d2b1e0 con 0x559775d00c00 2023-09-13T04:32:29.160 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:28 smithi156 systemd[1]: Started Ceph mgr.z for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:32:29.160 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:28 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:28.992+0000 7f5e7650e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-13T04:32:29.161 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:29 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:29.070+0000 7f5e7650e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-13T04:32:29.161 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:29 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:29.158+0000 7f5e7650e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-13T04:32:29.327 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:32:29.420 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:29 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:29.250+0000 7f5e7650e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-13T04:32:29.686 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:29 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:29.419+0000 7f5e7650e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-13T04:32:30.095 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:29 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:30.095 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:29 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:30.096 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:29 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:30.096 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:29 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-13T04:32:30.096 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:29 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-13T04:32:30.097 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:29 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-13T04:32:30.097 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:29 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:30.097 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:29 smithi156 ceph-mon[133675]: Deploying daemon mgr.y on smithi149 2023-09-13T04:32:30.097 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:29 smithi156 ceph-mon[133675]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:30.098 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:29 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:29.782+0000 7f5e7650e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-13T04:32:30.098 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:29 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:29.912+0000 7f5e7650e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-13T04:32:30.098 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:30 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:30.000+0000 7f5e7650e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-13T04:32:30.098 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:30 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:30.094+0000 7f5e7650e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-13T04:32:30.219 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:29 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:30.220 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:29 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:30.220 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:29 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:30.220 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:29 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-13T04:32:30.220 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:29 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-13T04:32:30.220 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:29 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-13T04:32:30.221 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:29 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:30.221 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:29 smithi149 ceph-mon[133315]: Deploying daemon mgr.y on smithi149 2023-09-13T04:32:30.221 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:29 smithi149 ceph-mon[133315]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:30.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:29 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:30.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:29 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:30.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:29 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:30.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:29 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-13T04:32:30.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:29 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-13T04:32:30.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:29 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-13T04:32:30.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:29 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:30.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:29 smithi002 ceph-mon[97651]: Deploying daemon mgr.y on smithi149 2023-09-13T04:32:30.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:29 smithi002 ceph-mon[97651]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:30.636 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:30 smithi149 systemd[1]: Starting Ceph mgr.y for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:32:30.936 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:30 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:30.514+0000 7f5e7650e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-13T04:32:31.037 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:30 smithi149 podman[134752]: 2023-09-13T04:32:31.330 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:31 smithi149 bash[134752]: 23d3c1fec40b838c8737a53ef19e535f9914dac54aad4ba7058d3e5778996439 2023-09-13T04:32:31.330 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:31 smithi149 ceph-mgr[134792]: -- 172.21.15.149:0/1972681730 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56366de5b1e0 con 0x56366de30800 2023-09-13T04:32:31.330 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:31 smithi149 systemd[1]: Started Ceph mgr.y for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:32:31.331 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:31 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:31.155+0000 7fec4a76e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-13T04:32:31.331 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:31 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:31.240+0000 7fec4a76e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-13T04:32:31.331 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:31 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:31.328+0000 7fec4a76e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-13T04:32:31.335 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-13T04:32:31.336 INFO:teuthology.orchestra.run.smithi002.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-13T04:32:31.336 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10+0 records in 2023-09-13T04:32:31.336 INFO:teuthology.orchestra.run.smithi002.stderr:10+0 records out 2023-09-13T04:32:31.336 INFO:teuthology.orchestra.run.smithi002.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0321882 s, 326 MB/s 2023-09-13T04:32:31.336 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping successful for: 2023-09-13T04:32:31.436 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:31 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:31.180+0000 7f5e7650e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-13T04:32:31.436 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:31 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:31.263+0000 7f5e7650e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-13T04:32:31.607 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:31 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:31.425+0000 7fec4a76e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-13T04:32:31.811 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph orch daemon add osd smithi002:vg_nvme/lv_4 2023-09-13T04:32:31.886 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:31 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:31.606+0000 7fec4a76e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-13T04:32:32.325 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:32 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:32.325 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:32 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:32.325 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:32 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:32.326 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:32 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:32.326 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:32 smithi149 ceph-mon[133315]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:32.326 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:32 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:32:32.326 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:31 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:31.995+0000 7fec4a76e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-13T04:32:32.327 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:32 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:32.130+0000 7fec4a76e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-13T04:32:32.327 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:32 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:32.223+0000 7fec4a76e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-13T04:32:32.357 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:32 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:32.076+0000 7f5e7650e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-13T04:32:32.358 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:32 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:32.175+0000 7f5e7650e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-13T04:32:32.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:32 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:32.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:32 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:32.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:32 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:32.359 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:32 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:32.359 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:32 smithi156 ceph-mon[133675]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:32.359 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:32 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:32:32.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:32 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:32.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:32 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:32.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:32 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:32.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:32 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:32.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:32 smithi002 ceph-mon[97651]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:32.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:32 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:32:32.555 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:32:32.664 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:32 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:32.356+0000 7f5e7650e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-13T04:32:32.665 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:32 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:32.663+0000 7f5e7650e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-13T04:32:32.936 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:32 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:32.903+0000 7f5e7650e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-13T04:32:33.136 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:32 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:32.324+0000 7fec4a76e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-13T04:32:33.136 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:32 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:32.778+0000 7fec4a76e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-13T04:32:33.322 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:33 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:33.058+0000 7f5e7650e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-13T04:32:33.686 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:33 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:33.321+0000 7f5e7650e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-13T04:32:33.886 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:33 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:33.473+0000 7fec4a76e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-13T04:32:33.886 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:33 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:33.556+0000 7fec4a76e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-13T04:32:34.110 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:34 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:34.110 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:34 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:34.111 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:34 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:34.111 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:34 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:34.111 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:34 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:34.111 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:34 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:32:34.111 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:34 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:34.112 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:34 smithi002 ceph-mon[97651]: Reconfiguring mgr.x (unknown last config time)... 2023-09-13T04:32:34.112 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:34 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-13T04:32:34.112 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:34 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-13T04:32:34.112 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:34 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:34.113 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:34 smithi002 ceph-mon[97651]: Reconfiguring daemon mgr.x on smithi002 2023-09-13T04:32:34.113 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:34 smithi002 ceph-mon[97651]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:34.113 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:34 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:34.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.150+0000 7f1f22e0e700 1 -- 172.21.15.2:0/2838591377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f1c073710 msgr2=0x7f1f1c073b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:34.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.150+0000 7f1f22e0e700 1 --2- 172.21.15.2:0/2838591377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f1c073710 0x7f1f1c073b90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009990 tx=0x7f1f08009ca0 comp rx=0 tx=0).stop 2023-09-13T04:32:34.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.151+0000 7f1f22e0e700 1 -- 172.21.15.2:0/2838591377 shutdown_connections 2023-09-13T04:32:34.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.151+0000 7f1f22e0e700 1 --2- 172.21.15.2:0/2838591377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f1c108c40 0x7f1f1c10b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:34.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.151+0000 7f1f22e0e700 1 --2- 172.21.15.2:0/2838591377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f1c074ce0 0x7f1f1c073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:34.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.151+0000 7f1f22e0e700 1 --2- 172.21.15.2:0/2838591377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f1c073710 0x7f1f1c073b90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:34.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.151+0000 7f1f22e0e700 1 -- 172.21.15.2:0/2838591377 >> 172.21.15.2:0/2838591377 conn(0x7f1f1c0fc2d0 msgr2=0x7f1f1c0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:32:34.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.151+0000 7f1f22e0e700 1 -- 172.21.15.2:0/2838591377 shutdown_connections 2023-09-13T04:32:34.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.151+0000 7f1f22e0e700 1 -- 172.21.15.2:0/2838591377 wait complete. 2023-09-13T04:32:34.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.152+0000 7f1f22e0e700 1 Processor -- start 2023-09-13T04:32:34.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.152+0000 7f1f22e0e700 1 -- start start 2023-09-13T04:32:34.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.152+0000 7f1f22e0e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f1c073710 0x7f1f1c198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:32:34.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.152+0000 7f1f22e0e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f1c074ce0 0x7f1f1c199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:32:34.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.152+0000 7f1f22e0e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f1c108c40 0x7f1f1c1a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:32:34.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.152+0000 7f1f22e0e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c1a0840 con 0x7f1f1c073710 2023-09-13T04:32:34.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.152+0000 7f1f22e0e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c1a0980 con 0x7f1f1c074ce0 2023-09-13T04:32:34.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.152+0000 7f1f22e0e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c1a0b40 con 0x7f1f1c108c40 2023-09-13T04:32:34.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.153+0000 7f1f20baa700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f1c073710 0x7f1f1c198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:32:34.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.153+0000 7f1f20baa700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f1c073710 0x7f1f1c198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34424/0 (socket says 172.21.15.2:34424) 2023-09-13T04:32:34.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.153+0000 7f1f20baa700 1 -- 172.21.15.2:0/3096832077 learned_addr learned my addr 172.21.15.2:0/3096832077 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:32:34.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.153+0000 7f1f1bfff700 1 --2- 172.21.15.2:0/3096832077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f1c074ce0 0x7f1f1c199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:32:34.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.153+0000 7f1f213ab700 1 --2- 172.21.15.2:0/3096832077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f1c108c40 0x7f1f1c1a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:32:34.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.153+0000 7f1f20baa700 1 -- 172.21.15.2:0/3096832077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f1c108c40 msgr2=0x7f1f1c1a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:34.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.153+0000 7f1f20baa700 1 --2- 172.21.15.2:0/3096832077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f1c108c40 0x7f1f1c1a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:34.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.153+0000 7f1f20baa700 1 -- 172.21.15.2:0/3096832077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f1c074ce0 msgr2=0x7f1f1c199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:34.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.153+0000 7f1f20baa700 1 --2- 172.21.15.2:0/3096832077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f1c074ce0 0x7f1f1c199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:34.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.153+0000 7f1f20baa700 1 -- 172.21.15.2:0/3096832077 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f14009c40 con 0x7f1f1c073710 2023-09-13T04:32:34.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.153+0000 7f1f20baa700 1 --2- 172.21.15.2:0/3096832077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f1c073710 0x7f1f1c198cf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000e970 tx=0x7f1f1000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:32:34.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.153+0000 7f1f19ffb700 1 -- 172.21.15.2:0/3096832077 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10009790 con 0x7f1f1c073710 2023-09-13T04:32:34.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.153+0000 7f1f19ffb700 1 -- 172.21.15.2:0/3096832077 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f1000c7c0 con 0x7f1f1c073710 2023-09-13T04:32:34.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.153+0000 7f1f22e0e700 1 -- 172.21.15.2:0/3096832077 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f080095f0 con 0x7f1f1c073710 2023-09-13T04:32:34.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.154+0000 7f1f19ffb700 1 -- 172.21.15.2:0/3096832077 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10005460 con 0x7f1f1c073710 2023-09-13T04:32:34.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.154+0000 7f1f22e0e700 1 -- 172.21.15.2:0/3096832077 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1c1a1190 con 0x7f1f1c073710 2023-09-13T04:32:34.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.155+0000 7f1f22e0e700 1 -- 172.21.15.2:0/3096832077 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f1c1072f0 con 0x7f1f1c073710 2023-09-13T04:32:34.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.155+0000 7f1f19ffb700 1 -- 172.21.15.2:0/3096832077 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45386+0+0 (secure 0 0 0) 0x7f1f10005600 con 0x7f1f1c073710 2023-09-13T04:32:34.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.156+0000 7f1f19ffb700 1 --2- 172.21.15.2:0/3096832077 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1f0c038630 0x7f1f0c03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:32:34.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.156+0000 7f1f19ffb700 1 -- 172.21.15.2:0/3096832077 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1f10014070 con 0x7f1f1c073710 2023-09-13T04:32:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.156+0000 7f1f1bfff700 1 --2- 172.21.15.2:0/3096832077 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1f0c038630 0x7f1f0c03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:32:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.157+0000 7f1f1bfff700 1 --2- 172.21.15.2:0/3096832077 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1f0c038630 0x7f1f0c03aaf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800b2e0 tx=0x7f1f08005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:32:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.161+0000 7f1f19ffb700 1 -- 172.21.15.2:0/3096832077 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1f1001fb30 con 0x7f1f1c073710 2023-09-13T04:32:34.335 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:34 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:34.335 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:34 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:34.336 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:34 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:34.336 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:34 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:34.336 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:34 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:34.336 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:34 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:32:34.337 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:34 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:34.337 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:34 smithi149 ceph-mon[133315]: Reconfiguring mgr.x (unknown last config time)... 2023-09-13T04:32:34.337 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:34 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-13T04:32:34.337 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:34 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-13T04:32:34.337 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:34 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:34.338 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:34 smithi149 ceph-mon[133315]: Reconfiguring daemon mgr.x on smithi002 2023-09-13T04:32:34.338 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:34 smithi149 ceph-mon[133315]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:34.338 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:34 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:34.350 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:34 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:34.026+0000 7f5e7650e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-13T04:32:34.351 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:34 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:34.158+0000 7f5e7650e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-13T04:32:34.351 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:34 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:34.349+0000 7f5e7650e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-13T04:32:34.351 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:34 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:34.351 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:34 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:34.352 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:34 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:34.352 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:34 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:34.352 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:34 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:34.352 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:34 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:32:34.352 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:34 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:34.353 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:34 smithi156 ceph-mon[133675]: Reconfiguring mgr.x (unknown last config time)... 2023-09-13T04:32:34.353 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:34 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-13T04:32:34.353 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:34 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-13T04:32:34.353 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:34 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:34.353 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:34 smithi156 ceph-mon[133675]: Reconfiguring daemon mgr.x on smithi002 2023-09-13T04:32:34.354 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:34 smithi156 ceph-mon[133675]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:34.354 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:34 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:34.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:34.391+0000 7f1f22e0e700 1 -- 172.21.15.2:0/3096832077 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f1c02d0b0 con 0x7f1f0c038630 2023-09-13T04:32:34.597 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:34 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:34.334+0000 7fec4a76e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-13T04:32:34.597 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:34 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:34.424+0000 7fec4a76e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-13T04:32:34.681 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:34 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:34.463+0000 7f5e7650e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-13T04:32:34.876 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:34 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:34.597+0000 7fec4a76e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-13T04:32:34.936 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:34 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:34.680+0000 7f5e7650e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-13T04:32:35.135 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:34 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:34.876+0000 7fec4a76e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-13T04:32:35.136 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:35 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:35.112+0000 7fec4a76e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-13T04:32:35.386 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:35 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:35.260+0000 7fec4a76e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-13T04:32:35.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:35 smithi149 ceph-mon[133315]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:32:35.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:35 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:32:35.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:35 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:32:35.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:35 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:35.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:35 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:35.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:35 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:35.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:35 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:32:35.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:35 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:35.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:35 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:32:35.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:35 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:35.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:35 smithi156 ceph-mon[133675]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:32:35.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:35 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:32:35.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:35 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:32:35.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:35 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:35.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:35 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:35.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:35 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:35.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:35 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:32:35.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:35 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:35.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:35 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:32:35.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:35 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:35.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:35 smithi002 ceph-mon[97651]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:32:35.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:35 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:32:35.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:35 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:32:35.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:35 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:35.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:35 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:35.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:35 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:35.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:35 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:32:35.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:35 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:35.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:35 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:32:35.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:35 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:35.889 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:35 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:35.520+0000 7fec4a76e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-13T04:32:35.936 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:32:35 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:32:35.545+0000 7f5e7650e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-13T04:32:36.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:36.143+0000 7f1f19ffb700 1 -- 172.21.15.2:0/3096832077 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 14) v1 ==== 90294+0+0 (secure 0 0 0) 0x7f1f1000ad30 con 0x7f1f1c073710 2023-09-13T04:32:36.386 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:36 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:36.228+0000 7fec4a76e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-13T04:32:36.386 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:36 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:36.360+0000 7fec4a76e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-13T04:32:36.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:36 smithi149 ceph-mon[133315]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:36.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:36 smithi149 ceph-mon[133315]: Standby manager daemon z started 2023-09-13T04:32:36.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:36 smithi149 ceph-mon[133315]: from='mgr.? 172.21.15.156:0/3672908043' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-13T04:32:36.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:36 smithi149 ceph-mon[133315]: from='mgr.? 172.21.15.156:0/3672908043' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-13T04:32:36.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:36 smithi149 ceph-mon[133315]: from='mgr.? 172.21.15.156:0/3672908043' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-13T04:32:36.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:36 smithi149 ceph-mon[133315]: from='mgr.? 172.21.15.156:0/3672908043' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-13T04:32:36.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:36 smithi156 ceph-mon[133675]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:36.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:36 smithi156 ceph-mon[133675]: Standby manager daemon z started 2023-09-13T04:32:36.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:36 smithi156 ceph-mon[133675]: from='mgr.? 172.21.15.156:0/3672908043' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-13T04:32:36.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:36 smithi156 ceph-mon[133675]: from='mgr.? 172.21.15.156:0/3672908043' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-13T04:32:36.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:36 smithi156 ceph-mon[133675]: from='mgr.? 172.21.15.156:0/3672908043' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-13T04:32:36.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:36 smithi156 ceph-mon[133675]: from='mgr.? 172.21.15.156:0/3672908043' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-13T04:32:36.475 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:36 smithi002 ceph-mon[97651]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:36.476 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:36 smithi002 ceph-mon[97651]: Standby manager daemon z started 2023-09-13T04:32:36.476 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:36 smithi002 ceph-mon[97651]: from='mgr.? 172.21.15.156:0/3672908043' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-13T04:32:36.476 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:36 smithi002 ceph-mon[97651]: from='mgr.? 172.21.15.156:0/3672908043' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-13T04:32:36.476 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:36 smithi002 ceph-mon[97651]: from='mgr.? 172.21.15.156:0/3672908043' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-13T04:32:36.477 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:36 smithi002 ceph-mon[97651]: from='mgr.? 172.21.15.156:0/3672908043' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-13T04:32:36.886 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:36 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:36.553+0000 7fec4a76e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-13T04:32:36.886 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:36 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:36.667+0000 7fec4a76e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-13T04:32:37.141 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:36 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:36.886+0000 7fec4a76e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-13T04:32:37.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:37 smithi156 ceph-mon[133675]: mgrmap e14: x(active, since 72s), standbys: z 2023-09-13T04:32:37.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:37 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-13T04:32:37.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:37 smithi002 ceph-mon[97651]: mgrmap e14: x(active, since 72s), standbys: z 2023-09-13T04:32:37.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:37 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-13T04:32:37.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:37 smithi149 ceph-mon[133315]: mgrmap e14: x(active, since 72s), standbys: z 2023-09-13T04:32:37.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:37 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-13T04:32:38.136 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:32:37 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:32:37.755+0000 7fec4a76e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-13T04:32:38.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:38 smithi156 ceph-mon[133675]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:38.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:38 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3133565291' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9a443bc-d0b1-41ea-8409-e0cccb0116ef"}]: dispatch 2023-09-13T04:32:38.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:38 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3133565291' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9a443bc-d0b1-41ea-8409-e0cccb0116ef"}]': finished 2023-09-13T04:32:38.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:38 smithi156 ceph-mon[133675]: osdmap e5: 1 total, 0 up, 1 in 2023-09-13T04:32:38.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:38 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T04:32:38.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:38 smithi156 ceph-mon[133675]: Standby manager daemon y started 2023-09-13T04:32:38.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:38 smithi156 ceph-mon[133675]: from='mgr.? 172.21.15.149:0/4119066016' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-13T04:32:38.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:38 smithi156 ceph-mon[133675]: from='mgr.? 172.21.15.149:0/4119066016' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-13T04:32:38.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:38 smithi156 ceph-mon[133675]: from='mgr.? 172.21.15.149:0/4119066016' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-13T04:32:38.439 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:38 smithi156 ceph-mon[133675]: from='mgr.? 172.21.15.149:0/4119066016' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-13T04:32:38.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:38 smithi002 ceph-mon[97651]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:38.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:38 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3133565291' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9a443bc-d0b1-41ea-8409-e0cccb0116ef"}]: dispatch 2023-09-13T04:32:38.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:38 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3133565291' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9a443bc-d0b1-41ea-8409-e0cccb0116ef"}]': finished 2023-09-13T04:32:38.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:38 smithi002 ceph-mon[97651]: osdmap e5: 1 total, 0 up, 1 in 2023-09-13T04:32:38.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:38 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T04:32:38.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:38 smithi002 ceph-mon[97651]: Standby manager daemon y started 2023-09-13T04:32:38.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:38 smithi002 ceph-mon[97651]: from='mgr.? 172.21.15.149:0/4119066016' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-13T04:32:38.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:38 smithi002 ceph-mon[97651]: from='mgr.? 172.21.15.149:0/4119066016' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-13T04:32:38.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:38 smithi002 ceph-mon[97651]: from='mgr.? 172.21.15.149:0/4119066016' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-13T04:32:38.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:38 smithi002 ceph-mon[97651]: from='mgr.? 172.21.15.149:0/4119066016' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-13T04:32:38.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:38.528+0000 7f1f19ffb700 1 -- 172.21.15.2:0/3096832077 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mgrmap(e 15) v1 ==== 135202+0+0 (secure 0 0 0) 0x7f1f1001f340 con 0x7f1f1c073710 2023-09-13T04:32:38.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:38 smithi149 ceph-mon[133315]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:38.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:38 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3133565291' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9a443bc-d0b1-41ea-8409-e0cccb0116ef"}]: dispatch 2023-09-13T04:32:38.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:38 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3133565291' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9a443bc-d0b1-41ea-8409-e0cccb0116ef"}]': finished 2023-09-13T04:32:38.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:38 smithi149 ceph-mon[133315]: osdmap e5: 1 total, 0 up, 1 in 2023-09-13T04:32:38.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:38 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T04:32:38.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:38 smithi149 ceph-mon[133315]: Standby manager daemon y started 2023-09-13T04:32:38.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:38 smithi149 ceph-mon[133315]: from='mgr.? 172.21.15.149:0/4119066016' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-13T04:32:38.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:38 smithi149 ceph-mon[133315]: from='mgr.? 172.21.15.149:0/4119066016' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-13T04:32:38.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:38 smithi149 ceph-mon[133315]: from='mgr.? 172.21.15.149:0/4119066016' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-13T04:32:38.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:38 smithi149 ceph-mon[133315]: from='mgr.? 172.21.15.149:0/4119066016' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-13T04:32:39.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:39 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2441299450' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:32:39.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:39 smithi156 ceph-mon[133675]: mgrmap e15: x(active, since 75s), standbys: z, y 2023-09-13T04:32:39.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:39 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-13T04:32:39.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:39 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2441299450' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:32:39.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:39 smithi002 ceph-mon[97651]: mgrmap e15: x(active, since 75s), standbys: z, y 2023-09-13T04:32:39.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:39 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-13T04:32:39.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:39 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2441299450' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:32:39.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:39 smithi149 ceph-mon[133315]: mgrmap e15: x(active, since 75s), standbys: z, y 2023-09-13T04:32:39.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:39 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-13T04:32:40.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:40 smithi156 ceph-mon[133675]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:40.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:40 smithi002 ceph-mon[97651]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:40.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:40 smithi149 ceph-mon[133315]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:42.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:42 smithi002 ceph-mon[97651]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:42.954 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:42 smithi156 ceph-mon[133675]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:42.954 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:42 smithi149 ceph-mon[133315]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:44.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:44 smithi002 ceph-mon[97651]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:44.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:44 smithi149 ceph-mon[133315]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:44.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:44 smithi156 ceph-mon[133675]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:45.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:45 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-13T04:32:45.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:45 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:45.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:45 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-13T04:32:45.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:45 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:45.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:45 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-13T04:32:45.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:45 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:46.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:46 smithi002 ceph-mon[97651]: Deploying daemon osd.0 on smithi002 2023-09-13T04:32:46.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:46 smithi002 ceph-mon[97651]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:46.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:46 smithi149 ceph-mon[133315]: Deploying daemon osd.0 on smithi002 2023-09-13T04:32:46.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:46 smithi149 ceph-mon[133315]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:46.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:46 smithi156 ceph-mon[133675]: Deploying daemon osd.0 on smithi002 2023-09-13T04:32:46.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:46 smithi156 ceph-mon[133675]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:48.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:48 smithi002 ceph-mon[97651]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:48.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:48 smithi149 ceph-mon[133315]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:48.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:48 smithi156 ceph-mon[133675]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:50.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:50 smithi156 ceph-mon[133675]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:50.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:50 smithi002 ceph-mon[97651]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:50.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:50 smithi149 ceph-mon[133315]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:52.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:52 smithi002 ceph-mon[97651]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:52.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:52 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:52.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:52 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:52.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:52 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:32:52.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:52 smithi149 ceph-mon[133315]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:52.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:52 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:52.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:52 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:52.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:52 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:32:52.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:52 smithi156 ceph-mon[133675]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:52.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:52 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:52.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:52 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:52.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:52 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:32:54.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:54 smithi002 ceph-mon[97651]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:54.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:54 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:54.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:54 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:54.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:54 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:54.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:54 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:32:54.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:54 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:54.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:54 smithi149 ceph-mon[133315]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:54.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:54 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:54.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:54 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:54.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:54 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:54.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:54 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:32:54.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:54 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:54.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:54 smithi156 ceph-mon[133675]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:54.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:54 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:54.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:54 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:54.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:54 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:32:54.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:54 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:32:54.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:54 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:55.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:55.107+0000 7f1f19ffb700 1 -- 172.21.15.2:0/3096832077 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1f1c02d0b0 con 0x7f1f0c038630 2023-09-13T04:32:55.109 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 0 on host 'smithi002' 2023-09-13T04:32:55.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:55.110+0000 7f1f22e0e700 1 -- 172.21.15.2:0/3096832077 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1f0c038630 msgr2=0x7f1f0c03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:55.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:55.110+0000 7f1f22e0e700 1 --2- 172.21.15.2:0/3096832077 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1f0c038630 0x7f1f0c03aaf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800b2e0 tx=0x7f1f08005fb0 comp rx=0 tx=0).stop 2023-09-13T04:32:55.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:55.110+0000 7f1f22e0e700 1 -- 172.21.15.2:0/3096832077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f1c073710 msgr2=0x7f1f1c198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:32:55.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:55.110+0000 7f1f22e0e700 1 --2- 172.21.15.2:0/3096832077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f1c073710 0x7f1f1c198cf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000e970 tx=0x7f1f1000ec80 comp rx=0 tx=0).stop 2023-09-13T04:32:55.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:55.111+0000 7f1f22e0e700 1 -- 172.21.15.2:0/3096832077 shutdown_connections 2023-09-13T04:32:55.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:55.111+0000 7f1f22e0e700 1 --2- 172.21.15.2:0/3096832077 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1f0c038630 0x7f1f0c03aaf0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:55.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:55.111+0000 7f1f22e0e700 1 --2- 172.21.15.2:0/3096832077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f1c108c40 0x7f1f1c1a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:55.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:55.111+0000 7f1f22e0e700 1 --2- 172.21.15.2:0/3096832077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f1c073710 0x7f1f1c198cf0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:55.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:55.111+0000 7f1f22e0e700 1 --2- 172.21.15.2:0/3096832077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f1c074ce0 0x7f1f1c199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:32:55.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:55.111+0000 7f1f22e0e700 1 -- 172.21.15.2:0/3096832077 >> 172.21.15.2:0/3096832077 conn(0x7f1f1c0fc2d0 msgr2=0x7f1f1c0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:32:55.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:55.111+0000 7f1f22e0e700 1 -- 172.21.15.2:0/3096832077 shutdown_connections 2023-09-13T04:32:55.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:32:55.111+0000 7f1f22e0e700 1 -- 172.21.15.2:0/3096832077 wait complete. 2023-09-13T04:32:55.792 DEBUG:teuthology.orchestra.run.smithi002:osd.0> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.0.service 2023-09-13T04:32:55.797 INFO:tasks.cephadm:Deploying osd.1 on smithi149 with /dev/vg_nvme/lv_4... 2023-09-13T04:32:55.797 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-13T04:32:55.875 INFO:journalctl@ceph.osd.0.smithi002.stdout:-- Logs begin at Wed 2023-09-13 04:13:33 UTC. -- 2023-09-13T04:32:56.360 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.b/config 2023-09-13T04:32:56.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:56 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:56.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:56 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:56.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:56 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:32:56.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:56 smithi149 ceph-mon[133315]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:56.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:56 smithi149 ceph-mon[133315]: from='osd.0 [v2:172.21.15.2:6802/1800239506,v1:172.21.15.2:6803/1800239506]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-13T04:32:56.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:56 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:56.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:56 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:56.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:56 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:32:56.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:56 smithi156 ceph-mon[133675]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:56.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:56 smithi156 ceph-mon[133675]: from='osd.0 [v2:172.21.15.2:6802/1800239506,v1:172.21.15.2:6803/1800239506]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-13T04:32:56.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:56 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:56.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:56 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:32:56.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:56 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:32:56.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:56 smithi002 ceph-mon[97651]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:56.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:56 smithi002 ceph-mon[97651]: from='osd.0 [v2:172.21.15.2:6802/1800239506,v1:172.21.15.2:6803/1800239506]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-13T04:32:57.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:57 smithi149 ceph-mon[133315]: from='osd.0 [v2:172.21.15.2:6802/1800239506,v1:172.21.15.2:6803/1800239506]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-13T04:32:57.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:57 smithi149 ceph-mon[133315]: osdmap e6: 1 total, 0 up, 1 in 2023-09-13T04:32:57.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:57 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T04:32:57.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:57 smithi149 ceph-mon[133315]: from='osd.0 [v2:172.21.15.2:6802/1800239506,v1:172.21.15.2:6803/1800239506]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-09-13T04:32:57.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:57 smithi156 ceph-mon[133675]: from='osd.0 [v2:172.21.15.2:6802/1800239506,v1:172.21.15.2:6803/1800239506]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-13T04:32:57.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:57 smithi156 ceph-mon[133675]: osdmap e6: 1 total, 0 up, 1 in 2023-09-13T04:32:57.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:57 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T04:32:57.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:57 smithi156 ceph-mon[133675]: from='osd.0 [v2:172.21.15.2:6802/1800239506,v1:172.21.15.2:6803/1800239506]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-09-13T04:32:57.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:57 smithi002 ceph-mon[97651]: from='osd.0 [v2:172.21.15.2:6802/1800239506,v1:172.21.15.2:6803/1800239506]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-13T04:32:57.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:57 smithi002 ceph-mon[97651]: osdmap e6: 1 total, 0 up, 1 in 2023-09-13T04:32:57.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:57 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T04:32:57.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:57 smithi002 ceph-mon[97651]: from='osd.0 [v2:172.21.15.2:6802/1800239506,v1:172.21.15.2:6803/1800239506]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-09-13T04:32:57.507 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 13 04:32:57 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-0[115301]: 2023-09-13T04:32:57.105+0000 7f654fd1a700 -1 osd.0 0 waiting for initial osdmap 2023-09-13T04:32:57.507 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 13 04:32:57 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-0[115301]: 2023-09-13T04:32:57.110+0000 7f6548b09700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-13T04:32:57.969 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-13T04:32:57.969 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-13T04:32:57.969 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2023-09-13T04:32:57.970 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2023-09-13T04:32:57.970 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114374 s, 917 MB/s 2023-09-13T04:32:57.970 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2023-09-13T04:32:58.314 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:58 smithi149 ceph-mon[133315]: from='osd.0 [v2:172.21.15.2:6802/1800239506,v1:172.21.15.2:6803/1800239506]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2023-09-13T04:32:58.314 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:58 smithi149 ceph-mon[133315]: osdmap e7: 1 total, 0 up, 1 in 2023-09-13T04:32:58.315 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:58 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T04:32:58.315 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:58 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T04:32:58.315 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:58 smithi149 ceph-mon[133315]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:58.316 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph orch daemon add osd smithi149:vg_nvme/lv_4 2023-09-13T04:32:58.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:58 smithi156 ceph-mon[133675]: from='osd.0 [v2:172.21.15.2:6802/1800239506,v1:172.21.15.2:6803/1800239506]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2023-09-13T04:32:58.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:58 smithi156 ceph-mon[133675]: osdmap e7: 1 total, 0 up, 1 in 2023-09-13T04:32:58.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:58 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T04:32:58.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:58 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T04:32:58.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:58 smithi156 ceph-mon[133675]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:58.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:58 smithi002 ceph-mon[97651]: from='osd.0 [v2:172.21.15.2:6802/1800239506,v1:172.21.15.2:6803/1800239506]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2023-09-13T04:32:58.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:58 smithi002 ceph-mon[97651]: osdmap e7: 1 total, 0 up, 1 in 2023-09-13T04:32:58.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:58 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T04:32:58.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:58 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T04:32:58.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:58 smithi002 ceph-mon[97651]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T04:32:58.873 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.b/config 2023-09-13T04:32:59.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:59 smithi149 ceph-mon[133315]: purged_snaps scrub starts 2023-09-13T04:32:59.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:59 smithi149 ceph-mon[133315]: purged_snaps scrub ok 2023-09-13T04:32:59.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:59 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T04:32:59.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:59 smithi149 ceph-mon[133315]: osd.0 [v2:172.21.15.2:6802/1800239506,v1:172.21.15.2:6803/1800239506] boot 2023-09-13T04:32:59.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:59 smithi149 ceph-mon[133315]: osdmap e8: 1 total, 1 up, 1 in 2023-09-13T04:32:59.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:32:59 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T04:32:59.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:59 smithi156 ceph-mon[133675]: purged_snaps scrub starts 2023-09-13T04:32:59.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:59 smithi156 ceph-mon[133675]: purged_snaps scrub ok 2023-09-13T04:32:59.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:59 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T04:32:59.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:59 smithi156 ceph-mon[133675]: osd.0 [v2:172.21.15.2:6802/1800239506,v1:172.21.15.2:6803/1800239506] boot 2023-09-13T04:32:59.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:59 smithi156 ceph-mon[133675]: osdmap e8: 1 total, 1 up, 1 in 2023-09-13T04:32:59.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:32:59 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T04:32:59.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:59 smithi002 ceph-mon[97651]: purged_snaps scrub starts 2023-09-13T04:32:59.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:59 smithi002 ceph-mon[97651]: purged_snaps scrub ok 2023-09-13T04:32:59.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:59 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T04:32:59.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:59 smithi002 ceph-mon[97651]: osd.0 [v2:172.21.15.2:6802/1800239506,v1:172.21.15.2:6803/1800239506] boot 2023-09-13T04:32:59.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:59 smithi002 ceph-mon[97651]: osdmap e8: 1 total, 1 up, 1 in 2023-09-13T04:32:59.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:32:59 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T04:33:00.097 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.095+0000 7ff2a81d1700 1 -- 172.21.15.149:0/1099550312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2a0101810 msgr2=0x7ff2a01080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:33:00.098 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.095+0000 7ff2a81d1700 1 --2- 172.21.15.149:0/1099550312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2a0101810 0x7ff2a01080c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff29800b0d0 tx=0x7ff29800b3e0 comp rx=0 tx=0).stop 2023-09-13T04:33:00.099 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.095+0000 7ff2a81d1700 1 -- 172.21.15.149:0/1099550312 shutdown_connections 2023-09-13T04:33:00.099 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.095+0000 7ff2a81d1700 1 --2- 172.21.15.149:0/1099550312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2a0101810 0x7ff2a01080c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:33:00.100 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.095+0000 7ff2a81d1700 1 --2- 172.21.15.149:0/1099550312 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2a00ffc70 0x7ff2a0100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:33:00.101 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.095+0000 7ff2a81d1700 1 --2- 172.21.15.149:0/1099550312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a0100e70 0x7ff2a01012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:33:00.101 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.095+0000 7ff2a81d1700 1 -- 172.21.15.149:0/1099550312 >> 172.21.15.149:0/1099550312 conn(0x7ff2a00fb1f0 msgr2=0x7ff2a00fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:33:00.101 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.095+0000 7ff2a81d1700 1 -- 172.21.15.149:0/1099550312 shutdown_connections 2023-09-13T04:33:00.101 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.095+0000 7ff2a81d1700 1 -- 172.21.15.149:0/1099550312 wait complete. 2023-09-13T04:33:00.102 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.096+0000 7ff2a81d1700 1 Processor -- start 2023-09-13T04:33:00.102 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.096+0000 7ff2a81d1700 1 -- start start 2023-09-13T04:33:00.102 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.096+0000 7ff2a81d1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a00ffc70 0x7ff2a0069bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:33:00.102 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.096+0000 7ff2a81d1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2a0100e70 0x7ff2a019b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:33:00.103 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.096+0000 7ff2a81d1700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2a0101810 0x7ff2a019df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:33:00.103 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.096+0000 7ff2a81d1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a019e6c0 con 0x7ff2a0100e70 2023-09-13T04:33:00.103 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.096+0000 7ff2a81d1700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a019e830 con 0x7ff2a0101810 2023-09-13T04:33:00.103 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.096+0000 7ff2a81d1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a019e9f0 con 0x7ff2a00ffc70 2023-09-13T04:33:00.104 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.096+0000 7ff2a676e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2a0101810 0x7ff2a019df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:33:00.106 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.096+0000 7ff2a676e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2a0101810 0x7ff2a019df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:57446/0 (socket says 172.21.15.149:57446) 2023-09-13T04:33:00.107 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.096+0000 7ff2a676e700 1 -- 172.21.15.149:0/2603514887 learned_addr learned my addr 172.21.15.149:0/2603514887 (peer_addr_for_me v2:172.21.15.149:0/0) 2023-09-13T04:33:00.107 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.096+0000 7ff2a5f6d700 1 --2- 172.21.15.149:0/2603514887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a00ffc70 0x7ff2a0069bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:33:00.107 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.096+0000 7ff2a576c700 1 --2- 172.21.15.149:0/2603514887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2a0100e70 0x7ff2a019b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:33:00.107 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.096+0000 7ff2a676e700 1 -- 172.21.15.149:0/2603514887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a00ffc70 msgr2=0x7ff2a0069bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:33:00.108 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.096+0000 7ff2a676e700 1 --2- 172.21.15.149:0/2603514887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a00ffc70 0x7ff2a0069bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:33:00.108 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.096+0000 7ff2a676e700 1 -- 172.21.15.149:0/2603514887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2a0100e70 msgr2=0x7ff2a019b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:33:00.108 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.096+0000 7ff2a676e700 1 --2- 172.21.15.149:0/2603514887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2a0100e70 0x7ff2a019b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:33:00.108 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.096+0000 7ff2a676e700 1 -- 172.21.15.149:0/2603514887 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff298009d00 con 0x7ff2a0101810 2023-09-13T04:33:00.109 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.096+0000 7ff2a5f6d700 1 --2- 172.21.15.149:0/2603514887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a00ffc70 0x7ff2a0069bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:33:00.109 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.096+0000 7ff2a676e700 1 --2- 172.21.15.149:0/2603514887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2a0101810 0x7ff2a019df20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff298007a00 tx=0x7ff298009570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:33:00.109 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.097+0000 7ff296ffd700 1 -- 172.21.15.149:0/2603514887 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29800e040 con 0x7ff2a0101810 2023-09-13T04:33:00.109 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.097+0000 7ff296ffd700 1 -- 172.21.15.149:0/2603514887 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff298003e10 con 0x7ff2a0101810 2023-09-13T04:33:00.109 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.097+0000 7ff2a81d1700 1 -- 172.21.15.149:0/2603514887 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a019ec70 con 0x7ff2a0101810 2023-09-13T04:33:00.110 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.097+0000 7ff296ffd700 1 -- 172.21.15.149:0/2603514887 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff298005750 con 0x7ff2a0101810 2023-09-13T04:33:00.110 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.097+0000 7ff2a81d1700 1 -- 172.21.15.149:0/2603514887 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a019f0b0 con 0x7ff2a0101810 2023-09-13T04:33:00.110 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.098+0000 7ff2a81d1700 1 -- 172.21.15.149:0/2603514887 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a01071c0 con 0x7ff2a0101810 2023-09-13T04:33:00.110 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.099+0000 7ff296ffd700 1 -- 172.21.15.149:0/2603514887 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 15) v1 ==== 135202+0+0 (secure 0 0 0) 0x7ff298012cb0 con 0x7ff2a0101810 2023-09-13T04:33:00.111 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.099+0000 7ff296ffd700 1 --2- 172.21.15.149:0/2603514887 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff28c0a05b0 0x7ff28c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:33:00.111 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.099+0000 7ff296ffd700 1 -- 172.21.15.149:0/2603514887 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7ff2980cbe10 con 0x7ff2a0101810 2023-09-13T04:33:00.111 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.099+0000 7ff2a5f6d700 1 --2- 172.21.15.149:0/2603514887 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff28c0a05b0 0x7ff28c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:33:00.111 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.100+0000 7ff2a5f6d700 1 --2- 172.21.15.149:0/2603514887 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff28c0a05b0 0x7ff28c0a2a70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0100cd0 tx=0x7ff290005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:33:00.111 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.103+0000 7ff296ffd700 1 -- 172.21.15.149:0/2603514887 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff298097440 con 0x7ff2a0101810 2023-09-13T04:33:00.302 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:00.300+0000 7ff2a81d1700 1 -- 172.21.15.149:0/2603514887 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2a002d150 con 0x7ff28c0a05b0 2023-09-13T04:33:00.392 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:00 smithi149 ceph-mon[133315]: osdmap e9: 1 total, 1 up, 1 in 2023-09-13T04:33:00.392 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:00 smithi149 ceph-mon[133315]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:00.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:00 smithi149 ceph-mon[133315]: Detected new or changed devices on smithi002 2023-09-13T04:33:00.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:00 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:00.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:00 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:00.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:00 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:33:00.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:00 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:00.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:00 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:00.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:00 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:33:00.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:00 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:00.440 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:00 smithi156 ceph-mon[133675]: osdmap e9: 1 total, 1 up, 1 in 2023-09-13T04:33:00.440 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:00 smithi156 ceph-mon[133675]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:00.441 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:00 smithi156 ceph-mon[133675]: Detected new or changed devices on smithi002 2023-09-13T04:33:00.441 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:00 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:00.441 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:00 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:00.441 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:00 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:33:00.442 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:00 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:00.442 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:00 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:00.442 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:00 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:33:00.442 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:00 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:00.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:00 smithi002 ceph-mon[97651]: osdmap e9: 1 total, 1 up, 1 in 2023-09-13T04:33:00.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:00 smithi002 ceph-mon[97651]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:00.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:00 smithi002 ceph-mon[97651]: Detected new or changed devices on smithi002 2023-09-13T04:33:00.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:00 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:00.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:00 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:00.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:00 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:33:00.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:00 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:00.509 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:00 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:00.509 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:00 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:33:00.510 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:00 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:01.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:01 smithi149 ceph-mon[133315]: Adjusting osd_memory_target on smithi002 to 17272M 2023-09-13T04:33:01.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:01 smithi149 ceph-mon[133315]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:33:01.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:01 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:33:01.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:01 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:33:01.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:01 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:01.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:01 smithi156 ceph-mon[133675]: Adjusting osd_memory_target on smithi002 to 17272M 2023-09-13T04:33:01.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:01 smithi156 ceph-mon[133675]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:33:01.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:01 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:33:01.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:01 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:33:01.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:01 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:01.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:01 smithi002 ceph-mon[97651]: Adjusting osd_memory_target on smithi002 to 17272M 2023-09-13T04:33:01.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:01 smithi002 ceph-mon[97651]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:33:01.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:01 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:33:01.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:01 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:33:01.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:01 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:02.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:02 smithi149 ceph-mon[133315]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:02.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:02 smithi156 ceph-mon[133675]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:02.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:02 smithi002 ceph-mon[97651]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:03.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:03 smithi156 ceph-mon[133675]: from='client.? 172.21.15.149:0/1862910394' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "620d0f1a-56e1-4edd-afa4-100d96afd043"}]: dispatch 2023-09-13T04:33:03.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:03 smithi156 ceph-mon[133675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "620d0f1a-56e1-4edd-afa4-100d96afd043"}]: dispatch 2023-09-13T04:33:03.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:03 smithi156 ceph-mon[133675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "620d0f1a-56e1-4edd-afa4-100d96afd043"}]': finished 2023-09-13T04:33:03.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:03 smithi156 ceph-mon[133675]: osdmap e10: 2 total, 1 up, 2 in 2023-09-13T04:33:03.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:03 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T04:33:03.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:03 smithi002 ceph-mon[97651]: from='client.? 172.21.15.149:0/1862910394' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "620d0f1a-56e1-4edd-afa4-100d96afd043"}]: dispatch 2023-09-13T04:33:03.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:03 smithi002 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "620d0f1a-56e1-4edd-afa4-100d96afd043"}]: dispatch 2023-09-13T04:33:03.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:03 smithi002 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "620d0f1a-56e1-4edd-afa4-100d96afd043"}]': finished 2023-09-13T04:33:03.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:03 smithi002 ceph-mon[97651]: osdmap e10: 2 total, 1 up, 2 in 2023-09-13T04:33:03.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:03 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T04:33:03.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:03 smithi149 ceph-mon[133315]: from='client.? 172.21.15.149:0/1862910394' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "620d0f1a-56e1-4edd-afa4-100d96afd043"}]: dispatch 2023-09-13T04:33:03.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:03 smithi149 ceph-mon[133315]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "620d0f1a-56e1-4edd-afa4-100d96afd043"}]: dispatch 2023-09-13T04:33:03.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:03 smithi149 ceph-mon[133315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "620d0f1a-56e1-4edd-afa4-100d96afd043"}]': finished 2023-09-13T04:33:03.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:03 smithi149 ceph-mon[133315]: osdmap e10: 2 total, 1 up, 2 in 2023-09-13T04:33:03.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:03 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T04:33:04.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:04 smithi156 ceph-mon[133675]: from='client.? 172.21.15.149:0/1752277384' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:33:04.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:04 smithi156 ceph-mon[133675]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:04.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:04 smithi002 ceph-mon[97651]: from='client.? 172.21.15.149:0/1752277384' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:33:04.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:04 smithi002 ceph-mon[97651]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:04.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:04 smithi149 ceph-mon[133315]: from='client.? 172.21.15.149:0/1752277384' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:33:04.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:04 smithi149 ceph-mon[133315]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:06.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:06 smithi149 ceph-mon[133315]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:06.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:06 smithi002 ceph-mon[97651]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:06.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:06 smithi156 ceph-mon[133675]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:08.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:08 smithi002 ceph-mon[97651]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:08.881 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:08 smithi149 ceph-mon[133315]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:08.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:08 smithi156 ceph-mon[133675]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:10.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:10 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-13T04:33:10.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:10 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:10.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:10 smithi149 ceph-mon[133315]: Deploying daemon osd.1 on smithi149 2023-09-13T04:33:10.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:10 smithi149 ceph-mon[133315]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:10.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:10 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-13T04:33:10.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:10 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:10.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:10 smithi156 ceph-mon[133675]: Deploying daemon osd.1 on smithi149 2023-09-13T04:33:10.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:10 smithi156 ceph-mon[133675]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:10.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:10 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-13T04:33:10.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:10 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:10.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:10 smithi002 ceph-mon[97651]: Deploying daemon osd.1 on smithi149 2023-09-13T04:33:10.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:10 smithi002 ceph-mon[97651]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:12.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:12 smithi149 ceph-mon[133315]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:12.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:12 smithi002 ceph-mon[97651]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:12.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:12 smithi156 ceph-mon[133675]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:14.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:14 smithi002 ceph-mon[97651]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:14.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:14 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:14.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:14 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:14.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:14 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:33:14.848 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:14 smithi149 ceph-mon[133315]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:14.848 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:14.849 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:14.849 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:33:14.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:14 smithi156 ceph-mon[133675]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:14.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:14 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:14.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:14 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:14.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:14 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:33:16.415 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:16.413+0000 7ff296ffd700 1 -- 172.21.15.149:0/2603514887 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff2a002d150 con 0x7ff28c0a05b0 2023-09-13T04:33:16.415 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 1 on host 'smithi149' 2023-09-13T04:33:16.417 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:16.415+0000 7ff2a81d1700 1 -- 172.21.15.149:0/2603514887 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff28c0a05b0 msgr2=0x7ff28c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:33:16.417 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:16.416+0000 7ff2a81d1700 1 --2- 172.21.15.149:0/2603514887 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff28c0a05b0 0x7ff28c0a2a70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0100cd0 tx=0x7ff290005f90 comp rx=0 tx=0).stop 2023-09-13T04:33:16.417 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:16.416+0000 7ff2a81d1700 1 -- 172.21.15.149:0/2603514887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2a0101810 msgr2=0x7ff2a019df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:33:16.417 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:16.416+0000 7ff2a81d1700 1 --2- 172.21.15.149:0/2603514887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2a0101810 0x7ff2a019df20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff298007a00 tx=0x7ff298009570 comp rx=0 tx=0).stop 2023-09-13T04:33:16.417 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:16.416+0000 7ff2a81d1700 1 -- 172.21.15.149:0/2603514887 shutdown_connections 2023-09-13T04:33:16.418 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:16.416+0000 7ff2a81d1700 1 --2- 172.21.15.149:0/2603514887 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff28c0a05b0 0x7ff28c0a2a70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:33:16.418 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:16.416+0000 7ff2a81d1700 1 --2- 172.21.15.149:0/2603514887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2a0101810 0x7ff2a019df20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:33:16.418 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:16.416+0000 7ff2a81d1700 1 --2- 172.21.15.149:0/2603514887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a00ffc70 0x7ff2a0069bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:33:16.418 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:16.416+0000 7ff2a81d1700 1 --2- 172.21.15.149:0/2603514887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2a0100e70 0x7ff2a019b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:33:16.419 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:16.416+0000 7ff2a81d1700 1 -- 172.21.15.149:0/2603514887 >> 172.21.15.149:0/2603514887 conn(0x7ff2a00fb1f0 msgr2=0x7ff2a0104300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:33:16.419 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:16.416+0000 7ff2a81d1700 1 -- 172.21.15.149:0/2603514887 shutdown_connections 2023-09-13T04:33:16.419 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:33:16.416+0000 7ff2a81d1700 1 -- 172.21.15.149:0/2603514887 wait complete. 2023-09-13T04:33:16.656 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:16 smithi149 ceph-mon[133315]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:16.657 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:16 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:16.657 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:16 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:16.657 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:16 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:16.657 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:16 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:33:16.657 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:16 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:16.658 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:16 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:16.658 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:16 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:16.658 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:16 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:33:16.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:16 smithi002 ceph-mon[97651]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:16.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:16 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:16.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:16 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:16.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:16 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:16.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:16 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:33:16.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:16 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:16.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:16 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:16.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:16 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:16.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:16 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:33:16.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:16 smithi156 ceph-mon[133675]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:16.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:16 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:16.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:16 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:16.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:16 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:16.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:16 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:33:16.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:16 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:16.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:16 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:16.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:16 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:16.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:16 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:33:16.998 DEBUG:teuthology.orchestra.run.smithi149:osd.1> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.1.service 2023-09-13T04:33:17.001 INFO:tasks.cephadm:Deploying osd.2 on smithi156 with /dev/vg_nvme/lv_4... 2023-09-13T04:33:17.002 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-13T04:33:17.038 INFO:journalctl@ceph.osd.1.smithi149.stdout:-- Logs begin at Wed 2023-09-13 04:09:45 UTC. -- 2023-09-13T04:33:17.566 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-13T04:33:17.886 INFO:journalctl@ceph.osd.1.smithi149.stdout:Sep 13 04:33:17 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-1[138391]: 2023-09-13T04:33:17.704+0000 7f2cc96c5640 -1 osd.1 0 log_to_monitors true 2023-09-13T04:33:18.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:18 smithi149 ceph-mon[133315]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:18.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:18 smithi149 ceph-mon[133315]: from='osd.1 [v2:172.21.15.149:6800/1737673954,v1:172.21.15.149:6801/1737673954]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-13T04:33:18.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:18 smithi149 ceph-mon[133315]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-13T04:33:18.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:18 smithi156 ceph-mon[133675]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:18.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:18 smithi156 ceph-mon[133675]: from='osd.1 [v2:172.21.15.149:6800/1737673954,v1:172.21.15.149:6801/1737673954]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-13T04:33:18.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:18 smithi156 ceph-mon[133675]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-13T04:33:19.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:18 smithi002 ceph-mon[97651]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:19.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:18 smithi002 ceph-mon[97651]: from='osd.1 [v2:172.21.15.149:6800/1737673954,v1:172.21.15.149:6801/1737673954]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-13T04:33:19.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:18 smithi002 ceph-mon[97651]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-13T04:33:19.061 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-13T04:33:19.062 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-13T04:33:19.062 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2023-09-13T04:33:19.062 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2023-09-13T04:33:19.062 INFO:teuthology.orchestra.run.smithi156.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124022 s, 845 MB/s 2023-09-13T04:33:19.062 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 2023-09-13T04:33:19.063 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2023-09-13T04:33:19.421 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph orch daemon add osd smithi156:vg_nvme/lv_4 2023-09-13T04:33:19.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:19 smithi156 ceph-mon[133675]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-13T04:33:19.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:19 smithi156 ceph-mon[133675]: osdmap e11: 2 total, 1 up, 2 in 2023-09-13T04:33:19.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T04:33:19.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:19 smithi156 ceph-mon[133675]: from='osd.1 [v2:172.21.15.149:6800/1737673954,v1:172.21.15.149:6801/1737673954]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-09-13T04:33:19.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:19 smithi156 ceph-mon[133675]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-09-13T04:33:19.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:19.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:19.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:33:19.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:19.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:19.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:33:19.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:19.886 INFO:journalctl@ceph.osd.1.smithi149.stdout:Sep 13 04:33:19 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-1[138391]: 2023-09-13T04:33:19.510+0000 7f2cbed3c700 -1 osd.1 0 waiting for initial osdmap 2023-09-13T04:33:19.886 INFO:journalctl@ceph.osd.1.smithi149.stdout:Sep 13 04:33:19 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-1[138391]: 2023-09-13T04:33:19.513+0000 7f2cbab31700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-13T04:33:19.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:19 smithi149 ceph-mon[133315]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-13T04:33:19.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:19 smithi149 ceph-mon[133315]: osdmap e11: 2 total, 1 up, 2 in 2023-09-13T04:33:19.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:19 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T04:33:19.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:19 smithi149 ceph-mon[133315]: from='osd.1 [v2:172.21.15.149:6800/1737673954,v1:172.21.15.149:6801/1737673954]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-09-13T04:33:19.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:19 smithi149 ceph-mon[133315]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-09-13T04:33:19.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:19 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:19.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:19 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:19.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:19 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:33:19.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:19 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:19.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:19 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:19.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:19 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:33:19.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:19 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:19.979 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-13T04:33:20.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:19 smithi002 ceph-mon[97651]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-13T04:33:20.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:19 smithi002 ceph-mon[97651]: osdmap e11: 2 total, 1 up, 2 in 2023-09-13T04:33:20.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T04:33:20.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:19 smithi002 ceph-mon[97651]: from='osd.1 [v2:172.21.15.149:6800/1737673954,v1:172.21.15.149:6801/1737673954]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-09-13T04:33:20.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:19 smithi002 ceph-mon[97651]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-09-13T04:33:20.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:20.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:20.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:33:20.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:20.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:20.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:33:20.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:20.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:20 smithi149 ceph-mon[133315]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:20.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:20 smithi149 ceph-mon[133315]: Detected new or changed devices on smithi149 2023-09-13T04:33:20.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:20 smithi149 ceph-mon[133315]: Adjusting osd_memory_target on smithi149 to 5879M 2023-09-13T04:33:20.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:20 smithi149 ceph-mon[133315]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2023-09-13T04:33:20.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:20 smithi149 ceph-mon[133315]: osdmap e12: 2 total, 1 up, 2 in 2023-09-13T04:33:20.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:20 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T04:33:20.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:20 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T04:33:20.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:20 smithi156 ceph-mon[133675]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:20.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:20 smithi156 ceph-mon[133675]: Detected new or changed devices on smithi149 2023-09-13T04:33:20.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:20 smithi156 ceph-mon[133675]: Adjusting osd_memory_target on smithi149 to 5879M 2023-09-13T04:33:20.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:20 smithi156 ceph-mon[133675]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2023-09-13T04:33:20.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:20 smithi156 ceph-mon[133675]: osdmap e12: 2 total, 1 up, 2 in 2023-09-13T04:33:20.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:20 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T04:33:20.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:20 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T04:33:21.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:20 smithi002 ceph-mon[97651]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:21.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:20 smithi002 ceph-mon[97651]: Detected new or changed devices on smithi149 2023-09-13T04:33:21.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:20 smithi002 ceph-mon[97651]: Adjusting osd_memory_target on smithi149 to 5879M 2023-09-13T04:33:21.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:20 smithi002 ceph-mon[97651]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2023-09-13T04:33:21.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:20 smithi002 ceph-mon[97651]: osdmap e12: 2 total, 1 up, 2 in 2023-09-13T04:33:21.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:20 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T04:33:21.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:20 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T04:33:21.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.254+0000 7f26e3660700 1 -- 172.21.15.156:0/3513830323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26dc102910 msgr2=0x7f26dc1091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:33:21.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.254+0000 7f26e3660700 1 --2- 172.21.15.156:0/3513830323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26dc102910 0x7f26dc1091c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f26d400b0a0 tx=0x7f26d400b3b0 comp rx=0 tx=0).stop 2023-09-13T04:33:21.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.254+0000 7f26e3660700 1 -- 172.21.15.156:0/3513830323 shutdown_connections 2023-09-13T04:33:21.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.254+0000 7f26e3660700 1 --2- 172.21.15.156:0/3513830323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26dc102910 0x7f26dc1091c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:33:21.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.254+0000 7f26e3660700 1 --2- 172.21.15.156:0/3513830323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26dc100d70 0x7f26dc101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:33:21.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.254+0000 7f26e3660700 1 --2- 172.21.15.156:0/3513830323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26dc101f70 0x7f26dc1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:33:21.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.254+0000 7f26e3660700 1 -- 172.21.15.156:0/3513830323 >> 172.21.15.156:0/3513830323 conn(0x7f26dc0fc2d0 msgr2=0x7f26dc0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:33:21.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.255+0000 7f26e3660700 1 -- 172.21.15.156:0/3513830323 shutdown_connections 2023-09-13T04:33:21.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.255+0000 7f26e3660700 1 -- 172.21.15.156:0/3513830323 wait complete. 2023-09-13T04:33:21.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.255+0000 7f26e3660700 1 Processor -- start 2023-09-13T04:33:21.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.255+0000 7f26e3660700 1 -- start start 2023-09-13T04:33:21.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.255+0000 7f26e3660700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26dc100d70 0x7f26dc196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:33:21.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.255+0000 7f26e3660700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26dc101f70 0x7f26dc1971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:33:21.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.255+0000 7f26e3660700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26dc102910 0x7f26dc19e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:33:21.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.255+0000 7f26e3660700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc19e830 con 0x7f26dc102910 2023-09-13T04:33:21.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.255+0000 7f26e3660700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc19e970 con 0x7f26dc100d70 2023-09-13T04:33:21.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.255+0000 7f26e3660700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc19eb30 con 0x7f26dc101f70 2023-09-13T04:33:21.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.256+0000 7f26e0bfb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26dc101f70 0x7f26dc1971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:33:21.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.256+0000 7f26e0bfb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26dc101f70 0x7f26dc1971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37086/0 (socket says 172.21.15.156:37086) 2023-09-13T04:33:21.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.256+0000 7f26e0bfb700 1 -- 172.21.15.156:0/176687892 learned_addr learned my addr 172.21.15.156:0/176687892 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-13T04:33:21.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.256+0000 7f26e13fc700 1 --2- 172.21.15.156:0/176687892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26dc100d70 0x7f26dc196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:33:21.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.256+0000 7f26e0bfb700 1 -- 172.21.15.156:0/176687892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26dc100d70 msgr2=0x7f26dc196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:33:21.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.256+0000 7f26e0bfb700 1 --2- 172.21.15.156:0/176687892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26dc100d70 0x7f26dc196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:33:21.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.256+0000 7f26e0bfb700 1 -- 172.21.15.156:0/176687892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26dc102910 msgr2=0x7f26dc19e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:33:21.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.256+0000 7f26e0bfb700 1 --2- 172.21.15.156:0/176687892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26dc102910 0x7f26dc19e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:33:21.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.256+0000 7f26e0bfb700 1 -- 172.21.15.156:0/176687892 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d4009d00 con 0x7f26dc101f70 2023-09-13T04:33:21.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.256+0000 7f26e0bfb700 1 --2- 172.21.15.156:0/176687892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26dc101f70 0x7f26dc1971f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00b530 tx=0x7f26cc00b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:33:21.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.256+0000 7f26d27fc700 1 -- 172.21.15.156:0/176687892 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc011830 con 0x7f26dc101f70 2023-09-13T04:33:21.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.256+0000 7f26d27fc700 1 -- 172.21.15.156:0/176687892 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26cc011e50 con 0x7f26dc101f70 2023-09-13T04:33:21.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.256+0000 7f26e3660700 1 -- 172.21.15.156:0/176687892 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26dc19ee10 con 0x7f26dc101f70 2023-09-13T04:33:21.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.256+0000 7f26d27fc700 1 -- 172.21.15.156:0/176687892 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc00f2e0 con 0x7f26dc101f70 2023-09-13T04:33:21.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.256+0000 7f26e3660700 1 -- 172.21.15.156:0/176687892 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26dc19f350 con 0x7f26dc101f70 2023-09-13T04:33:21.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.257+0000 7f26e3660700 1 -- 172.21.15.156:0/176687892 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26dc108350 con 0x7f26dc101f70 2023-09-13T04:33:21.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.259+0000 7f26d27fc700 1 -- 172.21.15.156:0/176687892 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 15) v1 ==== 135202+0+0 (secure 0 0 0) 0x7f26cc010580 con 0x7f26dc101f70 2023-09-13T04:33:21.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.259+0000 7f26d27fc700 1 --2- 172.21.15.156:0/176687892 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f26c80a0630 0x7f26c80a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:33:21.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.259+0000 7f26d27fc700 1 -- 172.21.15.156:0/176687892 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f26cc0ca330 con 0x7f26dc101f70 2023-09-13T04:33:21.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.260+0000 7f26e13fc700 1 --2- 172.21.15.156:0/176687892 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f26c80a0630 0x7f26c80a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:33:21.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.260+0000 7f26e13fc700 1 --2- 172.21.15.156:0/176687892 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f26c80a0630 0x7f26c80a2af0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f26d8006fd0 tx=0x7f26d8008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:33:21.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.263+0000 7f26d27fc700 1 -- 172.21.15.156:0/176687892 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f26cc0956b0 con 0x7f26dc101f70 2023-09-13T04:33:21.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:21.460+0000 7f26e3660700 1 -- 172.21.15.156:0/176687892 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f26dc02d0b0 con 0x7f26c80a0630 2023-09-13T04:33:21.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:21 smithi149 ceph-mon[133315]: purged_snaps scrub starts 2023-09-13T04:33:21.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:21 smithi149 ceph-mon[133315]: purged_snaps scrub ok 2023-09-13T04:33:21.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:21 smithi149 ceph-mon[133315]: osd.1 [v2:172.21.15.149:6800/1737673954,v1:172.21.15.149:6801/1737673954] boot 2023-09-13T04:33:21.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:21 smithi149 ceph-mon[133315]: osdmap e13: 2 total, 2 up, 2 in 2023-09-13T04:33:21.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:21 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T04:33:21.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:21 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:33:21.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:21 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:33:21.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:21 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:21.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:21 smithi156 ceph-mon[133675]: purged_snaps scrub starts 2023-09-13T04:33:21.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:21 smithi156 ceph-mon[133675]: purged_snaps scrub ok 2023-09-13T04:33:21.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:21 smithi156 ceph-mon[133675]: osd.1 [v2:172.21.15.149:6800/1737673954,v1:172.21.15.149:6801/1737673954] boot 2023-09-13T04:33:21.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:21 smithi156 ceph-mon[133675]: osdmap e13: 2 total, 2 up, 2 in 2023-09-13T04:33:21.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:21 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T04:33:21.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:21 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:33:21.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:21 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:33:21.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:21 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:22.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:21 smithi002 ceph-mon[97651]: purged_snaps scrub starts 2023-09-13T04:33:22.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:21 smithi002 ceph-mon[97651]: purged_snaps scrub ok 2023-09-13T04:33:22.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:21 smithi002 ceph-mon[97651]: osd.1 [v2:172.21.15.149:6800/1737673954,v1:172.21.15.149:6801/1737673954] boot 2023-09-13T04:33:22.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:21 smithi002 ceph-mon[97651]: osdmap e13: 2 total, 2 up, 2 in 2023-09-13T04:33:22.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:21 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T04:33:22.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:21 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:33:22.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:21 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:33:22.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:21 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:22.801 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:22 smithi156 ceph-mon[133675]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:22.802 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:22 smithi156 ceph-mon[133675]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:33:22.802 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:22 smithi156 ceph-mon[133675]: osdmap e14: 2 total, 2 up, 2 in 2023-09-13T04:33:22.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:22 smithi149 ceph-mon[133315]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:22.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:22 smithi149 ceph-mon[133315]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:33:22.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:22 smithi149 ceph-mon[133315]: osdmap e14: 2 total, 2 up, 2 in 2023-09-13T04:33:23.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:22 smithi002 ceph-mon[97651]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T04:33:23.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:22 smithi002 ceph-mon[97651]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:33:23.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:22 smithi002 ceph-mon[97651]: osdmap e14: 2 total, 2 up, 2 in 2023-09-13T04:33:24.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:24 smithi149 ceph-mon[133315]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:24.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:24 smithi149 ceph-mon[133315]: from='client.? 172.21.15.156:0/216316912' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ee5587a-dd0b-42b0-ae9a-a717ae6afad9"}]: dispatch 2023-09-13T04:33:24.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:24 smithi149 ceph-mon[133315]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ee5587a-dd0b-42b0-ae9a-a717ae6afad9"}]: dispatch 2023-09-13T04:33:24.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:24 smithi149 ceph-mon[133315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ee5587a-dd0b-42b0-ae9a-a717ae6afad9"}]': finished 2023-09-13T04:33:24.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:24 smithi149 ceph-mon[133315]: osdmap e15: 3 total, 2 up, 3 in 2023-09-13T04:33:24.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T04:33:24.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:24 smithi149 ceph-mon[133315]: from='client.? 172.21.15.156:0/4016568872' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:33:24.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:24 smithi156 ceph-mon[133675]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:24.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:24 smithi156 ceph-mon[133675]: from='client.? 172.21.15.156:0/216316912' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ee5587a-dd0b-42b0-ae9a-a717ae6afad9"}]: dispatch 2023-09-13T04:33:24.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:24 smithi156 ceph-mon[133675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ee5587a-dd0b-42b0-ae9a-a717ae6afad9"}]: dispatch 2023-09-13T04:33:24.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:24 smithi156 ceph-mon[133675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ee5587a-dd0b-42b0-ae9a-a717ae6afad9"}]': finished 2023-09-13T04:33:24.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:24 smithi156 ceph-mon[133675]: osdmap e15: 3 total, 2 up, 3 in 2023-09-13T04:33:24.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T04:33:24.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:24 smithi156 ceph-mon[133675]: from='client.? 172.21.15.156:0/4016568872' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:33:25.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:24 smithi002 ceph-mon[97651]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:25.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:24 smithi002 ceph-mon[97651]: from='client.? 172.21.15.156:0/216316912' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ee5587a-dd0b-42b0-ae9a-a717ae6afad9"}]: dispatch 2023-09-13T04:33:25.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:24 smithi002 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ee5587a-dd0b-42b0-ae9a-a717ae6afad9"}]: dispatch 2023-09-13T04:33:25.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:24 smithi002 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ee5587a-dd0b-42b0-ae9a-a717ae6afad9"}]': finished 2023-09-13T04:33:25.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:24 smithi002 ceph-mon[97651]: osdmap e15: 3 total, 2 up, 3 in 2023-09-13T04:33:25.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T04:33:25.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:24 smithi002 ceph-mon[97651]: from='client.? 172.21.15.156:0/4016568872' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:33:26.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:26 smithi149 ceph-mon[133315]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:26.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:26 smithi156 ceph-mon[133675]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:27.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:26 smithi002 ceph-mon[97651]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:28.862 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:28 smithi156 ceph-mon[133675]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:28.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:28 smithi149 ceph-mon[133315]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:29.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:28 smithi002 ceph-mon[97651]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:30.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:30 smithi156 ceph-mon[133675]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:30.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:30 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-13T04:33:30.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:30 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:30.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:30 smithi002 ceph-mon[97651]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:30.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:30 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-13T04:33:30.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:30 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:30.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:30 smithi149 ceph-mon[133315]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:30.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:30 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-13T04:33:30.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:30 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:31.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:31 smithi156 ceph-mon[133675]: Deploying daemon osd.2 on smithi156 2023-09-13T04:33:31.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:31 smithi002 ceph-mon[97651]: Deploying daemon osd.2 on smithi156 2023-09-13T04:33:31.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:31 smithi149 ceph-mon[133315]: Deploying daemon osd.2 on smithi156 2023-09-13T04:33:32.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:32 smithi156 ceph-mon[133675]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:32.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:32 smithi002 ceph-mon[97651]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:32.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:32 smithi149 ceph-mon[133315]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:34.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:34 smithi002 ceph-mon[97651]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:34.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:34 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:34.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:34 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:34.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:34 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:33:34.829 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:34 smithi156 ceph-mon[133675]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:34.830 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:34 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:34.830 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:34 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:34.831 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:34 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:33:34.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:34 smithi149 ceph-mon[133315]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:34.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:34 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:34.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:34 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:34.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:34 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:33:36.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:36 smithi002 ceph-mon[97651]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:36.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:36 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:36.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:36 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:36.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:36 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:36.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:36 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:33:36.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:36 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:36.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:36 smithi149 ceph-mon[133315]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:36.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:36 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:36.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:36 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:36.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:36 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:36.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:36 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:33:36.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:36 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:36.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:36 smithi156 ceph-mon[133675]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:36.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:36 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:36.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:36 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:36.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:36 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:36.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:36 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:33:36.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:36 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:36.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:36.969+0000 7f26d27fc700 1 -- 172.21.15.156:0/176687892 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f26dc02d0b0 con 0x7f26c80a0630 2023-09-13T04:33:36.971 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 2 on host 'smithi156' 2023-09-13T04:33:36.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:36.972+0000 7f26e3660700 1 -- 172.21.15.156:0/176687892 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f26c80a0630 msgr2=0x7f26c80a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:33:36.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:36.972+0000 7f26e3660700 1 --2- 172.21.15.156:0/176687892 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f26c80a0630 0x7f26c80a2af0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f26d8006fd0 tx=0x7f26d8008040 comp rx=0 tx=0).stop 2023-09-13T04:33:36.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:36.972+0000 7f26e3660700 1 -- 172.21.15.156:0/176687892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26dc101f70 msgr2=0x7f26dc1971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:33:36.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:36.972+0000 7f26e3660700 1 --2- 172.21.15.156:0/176687892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26dc101f70 0x7f26dc1971f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00b530 tx=0x7f26cc00b8f0 comp rx=0 tx=0).stop 2023-09-13T04:33:36.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:36.972+0000 7f26e3660700 1 -- 172.21.15.156:0/176687892 shutdown_connections 2023-09-13T04:33:36.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:36.972+0000 7f26e3660700 1 --2- 172.21.15.156:0/176687892 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f26c80a0630 0x7f26c80a2af0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:33:36.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:36.972+0000 7f26e3660700 1 --2- 172.21.15.156:0/176687892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26dc102910 0x7f26dc19e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:33:36.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:36.972+0000 7f26e3660700 1 --2- 172.21.15.156:0/176687892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26dc100d70 0x7f26dc196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:33:36.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:36.972+0000 7f26e3660700 1 --2- 172.21.15.156:0/176687892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26dc101f70 0x7f26dc1971f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:33:36.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:36.972+0000 7f26e3660700 1 -- 172.21.15.156:0/176687892 >> 172.21.15.156:0/176687892 conn(0x7f26dc0fc2d0 msgr2=0x7f26dc105490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:33:36.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:36.972+0000 7f26e3660700 1 -- 172.21.15.156:0/176687892 shutdown_connections 2023-09-13T04:33:36.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:33:36.972+0000 7f26e3660700 1 -- 172.21.15.156:0/176687892 wait complete. 2023-09-13T04:33:37.365 DEBUG:teuthology.orchestra.run.smithi156:osd.2> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.2.service 2023-09-13T04:33:37.367 INFO:tasks.cephadm:Deploying osd.3 on smithi002 with /dev/vg_nvme/lv_3... 2023-09-13T04:33:37.367 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-13T04:33:37.396 INFO:journalctl@ceph.osd.2.smithi156.stdout:-- Logs begin at Wed 2023-09-13 04:09:45 UTC. -- 2023-09-13T04:33:38.108 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:33:38.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:37 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:38.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:37 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:38.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:37 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:33:38.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:37 smithi156 ceph-mon[133675]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:38.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:37 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:38.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:37 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:38.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:37 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:33:38.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:37 smithi002 ceph-mon[97651]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:38.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:37 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:38.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:37 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:38.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:37 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:33:38.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:37 smithi149 ceph-mon[133315]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:38.675 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 13 04:33:38 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-2[139457]: 2023-09-13T04:33:38.368+0000 7f1aa544b640 -1 osd.2 0 log_to_monitors true 2023-09-13T04:33:39.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:38 smithi002 ceph-mon[97651]: from='osd.2 [v2:172.21.15.156:6800/3090983787,v1:172.21.15.156:6801/3090983787]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-13T04:33:39.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:38 smithi002 ceph-mon[97651]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-13T04:33:39.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:38 smithi149 ceph-mon[133315]: from='osd.2 [v2:172.21.15.156:6800/3090983787,v1:172.21.15.156:6801/3090983787]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-13T04:33:39.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:38 smithi149 ceph-mon[133315]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-13T04:33:39.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:38 smithi156 ceph-mon[133675]: from='osd.2 [v2:172.21.15.156:6800/3090983787,v1:172.21.15.156:6801/3090983787]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-13T04:33:39.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:38 smithi156 ceph-mon[133675]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-13T04:33:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-13T04:33:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-13T04:33:40.168 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10+0 records in 2023-09-13T04:33:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:10+0 records out 2023-09-13T04:33:40.169 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0439361 s, 239 MB/s 2023-09-13T04:33:40.169 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping successful for: 2023-09-13T04:33:40.186 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 13 04:33:39 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-2[139457]: 2023-09-13T04:33:39.955+0000 7f1a9aac2700 -1 osd.2 0 waiting for initial osdmap 2023-09-13T04:33:40.186 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 13 04:33:39 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-2[139457]: 2023-09-13T04:33:39.959+0000 7f1a940b2700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-13T04:33:40.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:39 smithi156 ceph-mon[133675]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-13T04:33:40.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:39 smithi156 ceph-mon[133675]: osdmap e16: 3 total, 2 up, 3 in 2023-09-13T04:33:40.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:39 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T04:33:40.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:39 smithi156 ceph-mon[133675]: from='osd.2 [v2:172.21.15.156:6800/3090983787,v1:172.21.15.156:6801/3090983787]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-13T04:33:40.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:39 smithi156 ceph-mon[133675]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-13T04:33:40.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:39 smithi156 ceph-mon[133675]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:40.204 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:39 smithi002 ceph-mon[97651]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-13T04:33:40.204 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:39 smithi002 ceph-mon[97651]: osdmap e16: 3 total, 2 up, 3 in 2023-09-13T04:33:40.204 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:39 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T04:33:40.205 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:39 smithi002 ceph-mon[97651]: from='osd.2 [v2:172.21.15.156:6800/3090983787,v1:172.21.15.156:6801/3090983787]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-13T04:33:40.205 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:39 smithi002 ceph-mon[97651]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-13T04:33:40.205 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:39 smithi002 ceph-mon[97651]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:40.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:39 smithi149 ceph-mon[133315]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-13T04:33:40.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:39 smithi149 ceph-mon[133315]: osdmap e16: 3 total, 2 up, 3 in 2023-09-13T04:33:40.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:39 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T04:33:40.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:39 smithi149 ceph-mon[133315]: from='osd.2 [v2:172.21.15.156:6800/3090983787,v1:172.21.15.156:6801/3090983787]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-13T04:33:40.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:39 smithi149 ceph-mon[133315]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-13T04:33:40.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:39 smithi149 ceph-mon[133315]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:40.688 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph orch daemon add osd smithi002:vg_nvme/lv_3 2023-09-13T04:33:41.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:40 smithi002 ceph-mon[97651]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-09-13T04:33:41.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:40 smithi002 ceph-mon[97651]: osdmap e17: 3 total, 2 up, 3 in 2023-09-13T04:33:41.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:40 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T04:33:41.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:40 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T04:33:41.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:40 smithi002 ceph-mon[97651]: Detected new or changed devices on smithi156 2023-09-13T04:33:41.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:40 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:41.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:40 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:41.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:40 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:33:41.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:40 smithi002 ceph-mon[97651]: Adjusting osd_memory_target on smithi156 to 17168M 2023-09-13T04:33:41.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:40 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:41.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:40 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:41.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:40 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:33:41.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:40 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:41.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:40 smithi149 ceph-mon[133315]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-09-13T04:33:41.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:40 smithi149 ceph-mon[133315]: osdmap e17: 3 total, 2 up, 3 in 2023-09-13T04:33:41.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:40 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T04:33:41.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:40 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T04:33:41.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:40 smithi149 ceph-mon[133315]: Detected new or changed devices on smithi156 2023-09-13T04:33:41.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:40 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:41.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:40 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:41.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:40 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:33:41.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:40 smithi149 ceph-mon[133315]: Adjusting osd_memory_target on smithi156 to 17168M 2023-09-13T04:33:41.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:40 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:41.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:40 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:41.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:40 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:33:41.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:40 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:41.420 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:33:41.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:40 smithi156 ceph-mon[133675]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-09-13T04:33:41.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:40 smithi156 ceph-mon[133675]: osdmap e17: 3 total, 2 up, 3 in 2023-09-13T04:33:41.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:40 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T04:33:41.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:40 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T04:33:41.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:40 smithi156 ceph-mon[133675]: Detected new or changed devices on smithi156 2023-09-13T04:33:41.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:40 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:41.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:40 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:41.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:40 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:33:41.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:40 smithi156 ceph-mon[133675]: Adjusting osd_memory_target on smithi156 to 17168M 2023-09-13T04:33:41.439 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:40 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:41.439 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:40 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:41.439 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:40 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:33:41.439 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:40 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:42.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:42 smithi002 ceph-mon[97651]: purged_snaps scrub starts 2023-09-13T04:33:42.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:42 smithi002 ceph-mon[97651]: purged_snaps scrub ok 2023-09-13T04:33:42.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:42 smithi002 ceph-mon[97651]: osd.2 [v2:172.21.15.156:6800/3090983787,v1:172.21.15.156:6801/3090983787] boot 2023-09-13T04:33:42.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:42 smithi002 ceph-mon[97651]: osdmap e18: 3 total, 3 up, 3 in 2023-09-13T04:33:42.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:42 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T04:33:42.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:42 smithi149 ceph-mon[133315]: purged_snaps scrub starts 2023-09-13T04:33:42.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:42 smithi149 ceph-mon[133315]: purged_snaps scrub ok 2023-09-13T04:33:42.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:42 smithi149 ceph-mon[133315]: osd.2 [v2:172.21.15.156:6800/3090983787,v1:172.21.15.156:6801/3090983787] boot 2023-09-13T04:33:42.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:42 smithi149 ceph-mon[133315]: osdmap e18: 3 total, 3 up, 3 in 2023-09-13T04:33:42.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:42 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T04:33:42.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:42 smithi156 ceph-mon[133675]: purged_snaps scrub starts 2023-09-13T04:33:42.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:42 smithi156 ceph-mon[133675]: purged_snaps scrub ok 2023-09-13T04:33:42.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:42 smithi156 ceph-mon[133675]: osd.2 [v2:172.21.15.156:6800/3090983787,v1:172.21.15.156:6801/3090983787] boot 2023-09-13T04:33:42.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:42 smithi156 ceph-mon[133675]: osdmap e18: 3 total, 3 up, 3 in 2023-09-13T04:33:42.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:42 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T04:33:42.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.944+0000 7f75a6ed8700 1 -- 172.21.15.2:0/2742150871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75a0074ce0 msgr2=0x7f75a0073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:33:42.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.944+0000 7f75a6ed8700 1 --2- 172.21.15.2:0/2742150871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75a0074ce0 0x7f75a0073140 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7594009990 tx=0x7f7594009ca0 comp rx=0 tx=0).stop 2023-09-13T04:33:42.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.945+0000 7f75a6ed8700 1 -- 172.21.15.2:0/2742150871 shutdown_connections 2023-09-13T04:33:42.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.945+0000 7f75a6ed8700 1 --2- 172.21.15.2:0/2742150871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75a0108c40 0x7f75a010b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:33:42.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.945+0000 7f75a6ed8700 1 --2- 172.21.15.2:0/2742150871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75a0074ce0 0x7f75a0073140 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:33:42.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.945+0000 7f75a6ed8700 1 --2- 172.21.15.2:0/2742150871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75a0073710 0x7f75a0073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:33:42.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.945+0000 7f75a6ed8700 1 -- 172.21.15.2:0/2742150871 >> 172.21.15.2:0/2742150871 conn(0x7f75a00fc2d0 msgr2=0x7f75a00fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:33:42.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.945+0000 7f75a6ed8700 1 -- 172.21.15.2:0/2742150871 shutdown_connections 2023-09-13T04:33:42.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.945+0000 7f75a6ed8700 1 -- 172.21.15.2:0/2742150871 wait complete. 2023-09-13T04:33:42.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.946+0000 7f75a6ed8700 1 Processor -- start 2023-09-13T04:33:42.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.946+0000 7f75a6ed8700 1 -- start start 2023-09-13T04:33:42.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.946+0000 7f75a6ed8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75a0073710 0x7f75a0198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:33:42.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.946+0000 7f75a6ed8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75a0074ce0 0x7f75a01991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:33:42.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.946+0000 7f75a6ed8700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75a0108c40 0x7f75a01a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:33:42.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.946+0000 7f75a6ed8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a01a07b0 con 0x7f75a0073710 2023-09-13T04:33:42.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.946+0000 7f75a6ed8700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a01a08f0 con 0x7f75a0108c40 2023-09-13T04:33:42.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.946+0000 7f75a6ed8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a01a0ab0 con 0x7f75a0074ce0 2023-09-13T04:33:42.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.946+0000 7f75a4c74700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75a0073710 0x7f75a0198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:33:42.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.947+0000 7f75a4c74700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75a0073710 0x7f75a0198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44462/0 (socket says 172.21.15.2:44462) 2023-09-13T04:33:42.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.947+0000 7f75a4c74700 1 -- 172.21.15.2:0/2791000412 learned_addr learned my addr 172.21.15.2:0/2791000412 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:33:42.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.947+0000 7f75a5475700 1 --2- 172.21.15.2:0/2791000412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75a0108c40 0x7f75a01a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:33:42.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.947+0000 7f759ffff700 1 --2- 172.21.15.2:0/2791000412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75a0074ce0 0x7f75a01991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:33:42.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.947+0000 7f75a4c74700 1 -- 172.21.15.2:0/2791000412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75a0074ce0 msgr2=0x7f75a01991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:33:42.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.947+0000 7f75a4c74700 1 --2- 172.21.15.2:0/2791000412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75a0074ce0 0x7f75a01991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:33:42.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.947+0000 7f75a4c74700 1 -- 172.21.15.2:0/2791000412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75a0108c40 msgr2=0x7f75a01a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:33:42.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.947+0000 7f75a4c74700 1 --2- 172.21.15.2:0/2791000412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75a0108c40 0x7f75a01a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:33:42.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.947+0000 7f75a4c74700 1 -- 172.21.15.2:0/2791000412 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75940095f0 con 0x7f75a0073710 2023-09-13T04:33:42.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.947+0000 7f75a4c74700 1 --2- 172.21.15.2:0/2791000412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75a0073710 0x7f75a0198c60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7594004710 tx=0x7f7594009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:33:42.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.947+0000 7f759dffb700 1 -- 172.21.15.2:0/2791000412 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759401d070 con 0x7f75a0073710 2023-09-13T04:33:42.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.947+0000 7f759dffb700 1 -- 172.21.15.2:0/2791000412 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7594021410 con 0x7f75a0073710 2023-09-13T04:33:42.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.947+0000 7f75a6ed8700 1 -- 172.21.15.2:0/2791000412 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a01a0d30 con 0x7f75a0073710 2023-09-13T04:33:42.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.947+0000 7f759dffb700 1 -- 172.21.15.2:0/2791000412 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7594017650 con 0x7f75a0073710 2023-09-13T04:33:42.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.948+0000 7f75a6ed8700 1 -- 172.21.15.2:0/2791000412 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a01a1140 con 0x7f75a0073710 2023-09-13T04:33:42.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.949+0000 7f75a6ed8700 1 -- 172.21.15.2:0/2791000412 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a01072f0 con 0x7f75a0073710 2023-09-13T04:33:42.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.951+0000 7f759dffb700 1 -- 172.21.15.2:0/2791000412 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 15) v1 ==== 135202+0+0 (secure 0 0 0) 0x7f7594021580 con 0x7f75a0073710 2023-09-13T04:33:42.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.952+0000 7f759dffb700 1 --2- 172.21.15.2:0/2791000412 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f75900a0630 0x7f75900a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:33:42.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.952+0000 7f759dffb700 1 -- 172.21.15.2:0/2791000412 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3226+0+0 (secure 0 0 0) 0x7f75940cbde0 con 0x7f75a0073710 2023-09-13T04:33:42.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.952+0000 7f759ffff700 1 --2- 172.21.15.2:0/2791000412 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f75900a0630 0x7f75900a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:33:42.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.953+0000 7f759ffff700 1 --2- 172.21.15.2:0/2791000412 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f75900a0630 0x7f75900a2af0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f75a0074a10 tx=0x7f758c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:33:42.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:42.955+0000 7f759dffb700 1 -- 172.21.15.2:0/2791000412 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7594096dc0 con 0x7f75a0073710 2023-09-13T04:33:43.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:43.188+0000 7f75a6ed8700 1 -- 172.21.15.2:0/2791000412 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f75a002d0d0 con 0x7f75900a0630 2023-09-13T04:33:43.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:43 smithi149 ceph-mon[133315]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:43.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:43 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-13T04:33:43.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:43 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-13T04:33:43.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:43 smithi149 ceph-mon[133315]: osdmap e19: 3 total, 3 up, 3 in 2023-09-13T04:33:43.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:43 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-13T04:33:43.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:43 smithi156 ceph-mon[133675]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:43.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:43 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-13T04:33:43.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:43 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-13T04:33:43.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:43 smithi156 ceph-mon[133675]: osdmap e19: 3 total, 3 up, 3 in 2023-09-13T04:33:43.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:43 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-13T04:33:43.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:43 smithi002 ceph-mon[97651]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T04:33:43.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:43 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-13T04:33:43.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:43 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-13T04:33:43.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:43 smithi002 ceph-mon[97651]: osdmap e19: 3 total, 3 up, 3 in 2023-09-13T04:33:43.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:43 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-13T04:33:44.005 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 13 04:33:43 smithi002 sudo[120725]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-13T04:33:44.006 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 13 04:33:43 smithi002 sudo[120725]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-13T04:33:44.006 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 13 04:33:43 smithi002 sudo[120725]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-13T04:33:44.006 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 13 04:33:43 smithi002 sudo[120725]: pam_unix(sudo:session): session closed for user root 2023-09-13T04:33:44.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:33:44.054+0000 7f759dffb700 1 -- 172.21.15.2:0/2791000412 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f7594097190 con 0x7f75a0073710 2023-09-13T04:33:44.329 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 13 04:33:44 smithi002 sudo[120770]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-13T04:33:44.329 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 13 04:33:44 smithi002 sudo[120770]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-13T04:33:44.329 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 13 04:33:44 smithi002 sudo[120770]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-13T04:33:44.329 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 13 04:33:44 smithi002 sudo[120770]: pam_unix(sudo:session): session closed for user root 2023-09-13T04:33:44.330 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:44 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-13T04:33:44.330 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:44 smithi002 ceph-mon[97651]: osdmap e20: 3 total, 3 up, 3 in 2023-09-13T04:33:44.330 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:44 smithi002 ceph-mon[97651]: from='client.14259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:33:44.330 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:44 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:33:44.331 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:44 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:33:44.331 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:44 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:44.331 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:44 smithi002 ceph-mon[97651]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-13T04:33:44.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:44 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-13T04:33:44.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:44 smithi149 ceph-mon[133315]: osdmap e20: 3 total, 3 up, 3 in 2023-09-13T04:33:44.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:44 smithi149 ceph-mon[133315]: from='client.14259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:33:44.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:44 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:33:44.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:44 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:33:44.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:44 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:44.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:44 smithi149 ceph-mon[133315]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-13T04:33:44.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:44 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-13T04:33:44.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:44 smithi156 ceph-mon[133675]: osdmap e20: 3 total, 3 up, 3 in 2023-09-13T04:33:44.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:44 smithi156 ceph-mon[133675]: from='client.14259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:33:44.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:44 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:33:44.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:44 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:33:44.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:44 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:44.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:44 smithi156 ceph-mon[133675]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-13T04:33:44.789 INFO:journalctl@ceph.osd.1.smithi149.stdout:Sep 13 04:33:44 smithi149 sudo[143714]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-13T04:33:44.790 INFO:journalctl@ceph.osd.1.smithi149.stdout:Sep 13 04:33:44 smithi149 sudo[143714]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-13T04:33:44.790 INFO:journalctl@ceph.osd.1.smithi149.stdout:Sep 13 04:33:44 smithi149 sudo[143714]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-13T04:33:44.790 INFO:journalctl@ceph.osd.1.smithi149.stdout:Sep 13 04:33:44 smithi149 sudo[143714]: pam_unix(sudo:session): session closed for user root 2023-09-13T04:33:45.047 INFO:journalctl@ceph.osd.1.smithi149.stdout:Sep 13 04:33:44 smithi149 sudo[143717]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-13T04:33:45.047 INFO:journalctl@ceph.osd.1.smithi149.stdout:Sep 13 04:33:44 smithi149 sudo[143717]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-13T04:33:45.048 INFO:journalctl@ceph.osd.1.smithi149.stdout:Sep 13 04:33:44 smithi149 sudo[143717]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-13T04:33:45.048 INFO:journalctl@ceph.osd.1.smithi149.stdout:Sep 13 04:33:44 smithi149 sudo[143717]: pam_unix(sudo:session): session closed for user root 2023-09-13T04:33:45.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:45 smithi002 ceph-mon[97651]: osdmap e21: 3 total, 3 up, 3 in 2023-09-13T04:33:45.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:45 smithi002 ceph-mon[97651]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-09-13T04:33:45.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:45 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:45.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:45 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:45.309 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:45 smithi156 ceph-mon[133675]: osdmap e21: 3 total, 3 up, 3 in 2023-09-13T04:33:45.310 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:45 smithi156 ceph-mon[133675]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-09-13T04:33:45.310 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:45 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:45.310 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:45 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:45.311 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 13 04:33:45 smithi156 sudo[144099]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-13T04:33:45.311 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 13 04:33:45 smithi156 sudo[144099]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-13T04:33:45.311 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 13 04:33:45 smithi156 sudo[144099]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-13T04:33:45.312 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 13 04:33:45 smithi156 sudo[144099]: pam_unix(sudo:session): session closed for user root 2023-09-13T04:33:45.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:45 smithi149 ceph-mon[133315]: osdmap e21: 3 total, 3 up, 3 in 2023-09-13T04:33:45.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:45 smithi149 ceph-mon[133315]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-09-13T04:33:45.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:45 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:45.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:45 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:45.686 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 13 04:33:45 smithi156 sudo[144102]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-13T04:33:45.686 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 13 04:33:45 smithi156 sudo[144102]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-13T04:33:45.687 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 13 04:33:45 smithi156 sudo[144102]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-13T04:33:45.687 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 13 04:33:45 smithi156 sudo[144102]: pam_unix(sudo:session): session closed for user root 2023-09-13T04:33:46.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:45 smithi002 sudo[120884]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-13T04:33:46.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:45 smithi002 sudo[120884]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-13T04:33:46.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:45 smithi002 sudo[120884]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-13T04:33:46.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:45 smithi002 sudo[120884]: pam_unix(sudo:session): session closed for user root 2023-09-13T04:33:46.678 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:46 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:46.678 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:46 smithi156 sudo[144105]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-13T04:33:46.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:46 smithi156 sudo[144105]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-13T04:33:46.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:46 smithi156 sudo[144105]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-13T04:33:46.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:46 smithi156 sudo[144105]: pam_unix(sudo:session): session closed for user root 2023-09-13T04:33:46.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:46 smithi002 sudo[120912]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-13T04:33:46.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:46 smithi002 sudo[120912]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-13T04:33:46.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:46 smithi002 sudo[120912]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-13T04:33:46.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:46 smithi002 sudo[120912]: pam_unix(sudo:session): session closed for user root 2023-09-13T04:33:46.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:46 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:46.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:46 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:33:46.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:46 smithi156 sudo[144108]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-13T04:33:46.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:46 smithi156 sudo[144108]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-13T04:33:46.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:46 smithi156 sudo[144108]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-13T04:33:46.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:46 smithi156 sudo[144108]: pam_unix(sudo:session): session closed for user root 2023-09-13T04:33:47.409 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 sudo[143720]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-13T04:33:47.409 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 sudo[143720]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-13T04:33:47.409 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 sudo[143720]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-13T04:33:47.410 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 sudo[143720]: pam_unix(sudo:session): session closed for user root 2023-09-13T04:33:47.410 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 sudo[143723]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-13T04:33:47.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:47 smithi156 ceph-mon[133675]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-13T04:33:47.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:47 smithi156 ceph-mon[133675]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-13T04:33:47.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:47 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/145582825' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93397693-f57f-4e1a-a1f8-10e3651e5e75"}]: dispatch 2023-09-13T04:33:47.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:47 smithi156 ceph-mon[133675]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-13T04:33:47.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:47 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T04:33:47.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:47 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:33:47.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:47 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:33:47.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:47 smithi156 ceph-mon[133675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93397693-f57f-4e1a-a1f8-10e3651e5e75"}]: dispatch 2023-09-13T04:33:47.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:47 smithi156 ceph-mon[133675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93397693-f57f-4e1a-a1f8-10e3651e5e75"}]': finished 2023-09-13T04:33:47.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:47 smithi156 ceph-mon[133675]: osdmap e22: 4 total, 3 up, 4 in 2023-09-13T04:33:47.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:47 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T04:33:47.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:47 smithi156 ceph-mon[133675]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-13T04:33:47.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:47 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T04:33:47.690 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:47 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:33:47.690 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:47 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:33:47.690 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:47 smithi156 ceph-mon[133675]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-13T04:33:47.691 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:47 smithi156 ceph-mon[133675]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-13T04:33:47.691 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:47 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T04:33:47.691 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:47 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:33:47.691 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:47 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:33:47.692 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:47 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3673521772' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:33:47.692 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:47 smithi156 ceph-mon[133675]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-13T04:33:47.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:47 smithi002 ceph-mon[97651]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-13T04:33:47.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:47 smithi002 ceph-mon[97651]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-13T04:33:47.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:47 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/145582825' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93397693-f57f-4e1a-a1f8-10e3651e5e75"}]: dispatch 2023-09-13T04:33:47.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:47 smithi002 ceph-mon[97651]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-13T04:33:47.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:47 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T04:33:47.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:47 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:33:47.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:47 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:33:47.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:47 smithi002 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93397693-f57f-4e1a-a1f8-10e3651e5e75"}]: dispatch 2023-09-13T04:33:47.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:47 smithi002 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93397693-f57f-4e1a-a1f8-10e3651e5e75"}]': finished 2023-09-13T04:33:47.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:47 smithi002 ceph-mon[97651]: osdmap e22: 4 total, 3 up, 4 in 2023-09-13T04:33:47.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:47 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T04:33:47.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:47 smithi002 ceph-mon[97651]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-13T04:33:47.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:47 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T04:33:47.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:47 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:33:47.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:47 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:33:47.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:47 smithi002 ceph-mon[97651]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-13T04:33:47.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:47 smithi002 ceph-mon[97651]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-13T04:33:47.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:47 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T04:33:47.760 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:47 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:33:47.760 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:47 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:33:47.760 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:47 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3673521772' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:33:47.760 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:47 smithi002 ceph-mon[97651]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-13T04:33:47.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 sudo[143723]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-13T04:33:47.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 sudo[143723]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-13T04:33:47.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 sudo[143723]: pam_unix(sudo:session): session closed for user root 2023-09-13T04:33:47.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 ceph-mon[133315]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-13T04:33:47.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 ceph-mon[133315]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-13T04:33:47.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/145582825' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93397693-f57f-4e1a-a1f8-10e3651e5e75"}]: dispatch 2023-09-13T04:33:47.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 ceph-mon[133315]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-13T04:33:47.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T04:33:47.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:33:47.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:33:47.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 ceph-mon[133315]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93397693-f57f-4e1a-a1f8-10e3651e5e75"}]: dispatch 2023-09-13T04:33:47.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 ceph-mon[133315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93397693-f57f-4e1a-a1f8-10e3651e5e75"}]': finished 2023-09-13T04:33:47.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 ceph-mon[133315]: osdmap e22: 4 total, 3 up, 4 in 2023-09-13T04:33:47.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T04:33:47.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 ceph-mon[133315]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-13T04:33:47.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T04:33:47.890 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:33:47.890 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:33:47.890 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 ceph-mon[133315]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-13T04:33:47.890 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 ceph-mon[133315]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-13T04:33:47.890 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T04:33:47.891 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:33:47.891 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:33:47.891 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3673521772' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:33:47.891 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:47 smithi149 ceph-mon[133315]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-13T04:33:48.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:48 smithi156 ceph-mon[133675]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-13T04:33:48.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:48 smithi002 ceph-mon[97651]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-13T04:33:48.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:48 smithi149 ceph-mon[133315]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-13T04:33:50.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:50 smithi156 ceph-mon[133675]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-13T04:33:50.441 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:50 smithi002 ceph-mon[97651]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-13T04:33:50.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:50 smithi149 ceph-mon[133315]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-13T04:33:52.621 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:52 smithi002 ceph-mon[97651]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-13T04:33:52.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:52 smithi149 ceph-mon[133315]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-13T04:33:52.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:52 smithi156 ceph-mon[133675]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-13T04:33:54.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:54 smithi002 ceph-mon[97651]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-09-13T04:33:54.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:54 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-13T04:33:54.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:54 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:54.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:54 smithi149 ceph-mon[133315]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-09-13T04:33:54.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:54 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-13T04:33:54.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:54 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:54.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:54 smithi156 ceph-mon[133675]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-09-13T04:33:54.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:54 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-13T04:33:54.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:54 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:33:55.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:55 smithi149 ceph-mon[133315]: Deploying daemon osd.3 on smithi002 2023-09-13T04:33:55.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:55 smithi156 ceph-mon[133675]: Deploying daemon osd.3 on smithi002 2023-09-13T04:33:56.001 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:55 smithi002 ceph-mon[97651]: Deploying daemon osd.3 on smithi002 2023-09-13T04:33:56.705 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:56 smithi002 ceph-mon[97651]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-09-13T04:33:56.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:56 smithi149 ceph-mon[133315]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-09-13T04:33:56.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:56 smithi156 ceph-mon[133675]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-09-13T04:33:58.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:33:58 smithi002 ceph-mon[97651]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-09-13T04:33:58.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:33:58 smithi149 ceph-mon[133315]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-09-13T04:33:58.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:33:58 smithi156 ceph-mon[133675]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-09-13T04:34:00.338 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:00 smithi002 ceph-mon[97651]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-13T04:34:00.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:00 smithi156 ceph-mon[133675]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-13T04:34:00.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:00 smithi149 ceph-mon[133315]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-13T04:34:01.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:01 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:01.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:01 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:01.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:01 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:34:01.723 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:01 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:01.723 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:01 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:01.724 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:01 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:34:01.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:01 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:01.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:01 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:01.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:01 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:34:02.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:02 smithi156 ceph-mon[133675]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-13T04:34:02.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:02 smithi002 ceph-mon[97651]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-13T04:34:02.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:02 smithi149 ceph-mon[133315]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-13T04:34:04.104 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 3 on host 'smithi002' 2023-09-13T04:34:04.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:04.103+0000 7f759dffb700 1 -- 172.21.15.2:0/2791000412 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f75a002d0d0 con 0x7f75900a0630 2023-09-13T04:34:04.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:04.107+0000 7f75a6ed8700 1 -- 172.21.15.2:0/2791000412 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f75900a0630 msgr2=0x7f75900a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:34:04.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:04.107+0000 7f75a6ed8700 1 --2- 172.21.15.2:0/2791000412 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f75900a0630 0x7f75900a2af0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f75a0074a10 tx=0x7f758c009260 comp rx=0 tx=0).stop 2023-09-13T04:34:04.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:04.107+0000 7f75a6ed8700 1 -- 172.21.15.2:0/2791000412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75a0073710 msgr2=0x7f75a0198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:34:04.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:04.107+0000 7f75a6ed8700 1 --2- 172.21.15.2:0/2791000412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75a0073710 0x7f75a0198c60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7594004710 tx=0x7f7594009580 comp rx=0 tx=0).stop 2023-09-13T04:34:04.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:04.107+0000 7f75a6ed8700 1 -- 172.21.15.2:0/2791000412 shutdown_connections 2023-09-13T04:34:04.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:04.107+0000 7f75a6ed8700 1 --2- 172.21.15.2:0/2791000412 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f75900a0630 0x7f75900a2af0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:04.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:04.107+0000 7f75a6ed8700 1 --2- 172.21.15.2:0/2791000412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75a0108c40 0x7f75a01a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:04.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:04.107+0000 7f75a6ed8700 1 --2- 172.21.15.2:0/2791000412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75a0073710 0x7f75a0198c60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:04.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:04.107+0000 7f75a6ed8700 1 --2- 172.21.15.2:0/2791000412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75a0074ce0 0x7f75a01991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:04.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:04.107+0000 7f75a6ed8700 1 -- 172.21.15.2:0/2791000412 >> 172.21.15.2:0/2791000412 conn(0x7f75a00fc2d0 msgr2=0x7f75a00fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:34:04.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:04.107+0000 7f75a6ed8700 1 -- 172.21.15.2:0/2791000412 shutdown_connections 2023-09-13T04:34:04.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:04.107+0000 7f75a6ed8700 1 -- 172.21.15.2:0/2791000412 wait complete. 2023-09-13T04:34:04.283 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:04 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:04.283 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:04 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:04.283 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:04 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:04.283 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:04 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:34:04.284 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:04 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:04.284 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:04 smithi002 ceph-mon[97651]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-13T04:34:04.284 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:04 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:04.284 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:04 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:04.284 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:04 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:34:04.285 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:04 smithi002 ceph-mon[97651]: from='osd.3 [v2:172.21.15.2:6810/3764424070,v1:172.21.15.2:6811/3764424070]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-13T04:34:04.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:04 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:04.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:04 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:04.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:04 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:04.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:04 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:34:04.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:04 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:04.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:04 smithi149 ceph-mon[133315]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-13T04:34:04.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:04 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:04.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:04 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:04.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:04 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:34:04.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:04 smithi149 ceph-mon[133315]: from='osd.3 [v2:172.21.15.2:6810/3764424070,v1:172.21.15.2:6811/3764424070]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-13T04:34:04.647 DEBUG:teuthology.orchestra.run.smithi002:osd.3> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.3.service 2023-09-13T04:34:04.651 INFO:tasks.cephadm:Deploying osd.4 on smithi149 with /dev/vg_nvme/lv_3... 2023-09-13T04:34:04.652 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-13T04:34:04.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:04 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:04.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:04 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:04.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:04 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:04.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:04 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:34:04.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:04 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:04.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:04 smithi156 ceph-mon[133675]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-13T04:34:04.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:04 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:04.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:04 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:04.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:04 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:34:04.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:04 smithi156 ceph-mon[133675]: from='osd.3 [v2:172.21.15.2:6810/3764424070,v1:172.21.15.2:6811/3764424070]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-13T04:34:04.728 INFO:journalctl@ceph.osd.3.smithi002.stdout:-- Logs begin at Wed 2023-09-13 04:13:33 UTC. -- 2023-09-13T04:34:05.215 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.b/config 2023-09-13T04:34:06.167 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:06 smithi002 ceph-mon[97651]: from='osd.3 [v2:172.21.15.2:6810/3764424070,v1:172.21.15.2:6811/3764424070]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-13T04:34:06.167 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:06 smithi002 ceph-mon[97651]: osdmap e23: 4 total, 3 up, 4 in 2023-09-13T04:34:06.167 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:06 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T04:34:06.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:06 smithi002 ceph-mon[97651]: from='osd.3 [v2:172.21.15.2:6810/3764424070,v1:172.21.15.2:6811/3764424070]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-09-13T04:34:06.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:06 smithi002 ceph-mon[97651]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-13T04:34:06.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:06 smithi149 ceph-mon[133315]: from='osd.3 [v2:172.21.15.2:6810/3764424070,v1:172.21.15.2:6811/3764424070]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-13T04:34:06.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:06 smithi149 ceph-mon[133315]: osdmap e23: 4 total, 3 up, 4 in 2023-09-13T04:34:06.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:06 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T04:34:06.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:06 smithi149 ceph-mon[133315]: from='osd.3 [v2:172.21.15.2:6810/3764424070,v1:172.21.15.2:6811/3764424070]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-09-13T04:34:06.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:06 smithi149 ceph-mon[133315]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-13T04:34:06.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:06 smithi156 ceph-mon[133675]: from='osd.3 [v2:172.21.15.2:6810/3764424070,v1:172.21.15.2:6811/3764424070]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-13T04:34:06.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:06 smithi156 ceph-mon[133675]: osdmap e23: 4 total, 3 up, 4 in 2023-09-13T04:34:06.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T04:34:06.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:06 smithi156 ceph-mon[133675]: from='osd.3 [v2:172.21.15.2:6810/3764424070,v1:172.21.15.2:6811/3764424070]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-09-13T04:34:06.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:06 smithi156 ceph-mon[133675]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-13T04:34:06.506 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 13 04:34:06 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-3[123164]: 2023-09-13T04:34:06.164+0000 7f109e3d6700 -1 osd.3 0 waiting for initial osdmap 2023-09-13T04:34:06.506 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 13 04:34:06 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-3[123164]: 2023-09-13T04:34:06.171+0000 7f109a1cb700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-13T04:34:06.730 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-13T04:34:06.730 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-13T04:34:06.730 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2023-09-13T04:34:06.730 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2023-09-13T04:34:06.731 INFO:teuthology.orchestra.run.smithi149.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0148988 s, 704 MB/s 2023-09-13T04:34:06.731 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2023-09-13T04:34:07.168 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph orch daemon add osd smithi149:vg_nvme/lv_3 2023-09-13T04:34:07.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:07 smithi156 ceph-mon[133675]: purged_snaps scrub starts 2023-09-13T04:34:07.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:07 smithi156 ceph-mon[133675]: purged_snaps scrub ok 2023-09-13T04:34:07.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:07 smithi156 ceph-mon[133675]: from='osd.3 [v2:172.21.15.2:6810/3764424070,v1:172.21.15.2:6811/3764424070]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2023-09-13T04:34:07.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:07 smithi156 ceph-mon[133675]: osdmap e24: 4 total, 3 up, 4 in 2023-09-13T04:34:07.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:07 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T04:34:07.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:07 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T04:34:07.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:07 smithi002 ceph-mon[97651]: purged_snaps scrub starts 2023-09-13T04:34:07.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:07 smithi002 ceph-mon[97651]: purged_snaps scrub ok 2023-09-13T04:34:07.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:07 smithi002 ceph-mon[97651]: from='osd.3 [v2:172.21.15.2:6810/3764424070,v1:172.21.15.2:6811/3764424070]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2023-09-13T04:34:07.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:07 smithi002 ceph-mon[97651]: osdmap e24: 4 total, 3 up, 4 in 2023-09-13T04:34:07.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:07 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T04:34:07.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:07 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T04:34:07.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:07 smithi149 ceph-mon[133315]: purged_snaps scrub starts 2023-09-13T04:34:07.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:07 smithi149 ceph-mon[133315]: purged_snaps scrub ok 2023-09-13T04:34:07.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:07 smithi149 ceph-mon[133315]: from='osd.3 [v2:172.21.15.2:6810/3764424070,v1:172.21.15.2:6811/3764424070]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2023-09-13T04:34:07.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:07 smithi149 ceph-mon[133315]: osdmap e24: 4 total, 3 up, 4 in 2023-09-13T04:34:07.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:07 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T04:34:07.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:07 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T04:34:07.720 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.b/config 2023-09-13T04:34:08.223 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:08 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T04:34:08.223 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:08 smithi149 ceph-mon[133315]: osd.3 [v2:172.21.15.2:6810/3764424070,v1:172.21.15.2:6811/3764424070] boot 2023-09-13T04:34:08.223 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:08 smithi149 ceph-mon[133315]: osdmap e25: 4 total, 4 up, 4 in 2023-09-13T04:34:08.224 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:08 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T04:34:08.224 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:08 smithi149 ceph-mon[133315]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-13T04:34:08.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:08 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T04:34:08.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:08 smithi002 ceph-mon[97651]: osd.3 [v2:172.21.15.2:6810/3764424070,v1:172.21.15.2:6811/3764424070] boot 2023-09-13T04:34:08.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:08 smithi002 ceph-mon[97651]: osdmap e25: 4 total, 4 up, 4 in 2023-09-13T04:34:08.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:08 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T04:34:08.406 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:08 smithi002 ceph-mon[97651]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-13T04:34:08.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:08 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T04:34:08.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:08 smithi156 ceph-mon[133675]: osd.3 [v2:172.21.15.2:6810/3764424070,v1:172.21.15.2:6811/3764424070] boot 2023-09-13T04:34:08.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:08 smithi156 ceph-mon[133675]: osdmap e25: 4 total, 4 up, 4 in 2023-09-13T04:34:08.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:08 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T04:34:08.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:08 smithi156 ceph-mon[133675]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-13T04:34:08.936 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.935+0000 7fa1a1907700 1 -- 172.21.15.149:0/3629458960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa19c100650 msgr2=0x7fa19c100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:34:08.936 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.935+0000 7fa1a1907700 1 --2- 172.21.15.149:0/3629458960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa19c100650 0x7fa19c100ab0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0099c0 tx=0x7fa18c009cd0 comp rx=0 tx=0).stop 2023-09-13T04:34:08.937 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.935+0000 7fa1a1907700 1 -- 172.21.15.149:0/3629458960 shutdown_connections 2023-09-13T04:34:08.937 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.935+0000 7fa1a1907700 1 --2- 172.21.15.149:0/3629458960 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa19c100ff0 0x7fa19c1078a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:08.937 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.935+0000 7fa1a1907700 1 --2- 172.21.15.149:0/3629458960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa19c0ff450 0x7fa19c0ff870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:08.938 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.935+0000 7fa1a1907700 1 --2- 172.21.15.149:0/3629458960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa19c100650 0x7fa19c100ab0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:08.938 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.935+0000 7fa1a1907700 1 -- 172.21.15.149:0/3629458960 >> 172.21.15.149:0/3629458960 conn(0x7fa19c0fa9d0 msgr2=0x7fa19c0fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:34:08.938 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.935+0000 7fa1a1907700 1 -- 172.21.15.149:0/3629458960 shutdown_connections 2023-09-13T04:34:08.938 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.935+0000 7fa1a1907700 1 -- 172.21.15.149:0/3629458960 wait complete. 2023-09-13T04:34:08.941 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.936+0000 7fa1a1907700 1 Processor -- start 2023-09-13T04:34:08.941 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.936+0000 7fa1a1907700 1 -- start start 2023-09-13T04:34:08.941 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.936+0000 7fa1a1907700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa19c0ff450 0x7fa19c0fcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:34:08.941 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.936+0000 7fa1a1907700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa19c100650 0x7fa19c19b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:34:08.942 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.936+0000 7fa1a1907700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa19c100ff0 0x7fa19c19dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:34:08.942 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.936+0000 7fa1a1907700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19c19e500 con 0x7fa19c0ff450 2023-09-13T04:34:08.942 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.936+0000 7fa1a1907700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19c19e670 con 0x7fa19c100650 2023-09-13T04:34:08.942 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.936+0000 7fa1a1907700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19c19e830 con 0x7fa19c100ff0 2023-09-13T04:34:08.943 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.936+0000 7fa19a7fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa19c100650 0x7fa19c19b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:34:08.943 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.936+0000 7fa19a7fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa19c100650 0x7fa19c19b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:57522/0 (socket says 172.21.15.149:57522) 2023-09-13T04:34:08.943 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.936+0000 7fa19a7fc700 1 -- 172.21.15.149:0/662277502 learned_addr learned my addr 172.21.15.149:0/662277502 (peer_addr_for_me v2:172.21.15.149:0/0) 2023-09-13T04:34:08.943 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.936+0000 7fa19b7fe700 1 --2- 172.21.15.149:0/662277502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa19c100ff0 0x7fa19c19dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:34:08.943 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.936+0000 7fa19a7fc700 1 -- 172.21.15.149:0/662277502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa19c100ff0 msgr2=0x7fa19c19dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:34:08.944 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.936+0000 7fa19a7fc700 1 --2- 172.21.15.149:0/662277502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa19c100ff0 0x7fa19c19dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:08.944 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.936+0000 7fa19a7fc700 1 -- 172.21.15.149:0/662277502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa19c0ff450 msgr2=0x7fa19c0fcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:34:08.944 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.936+0000 7fa19a7fc700 1 --2- 172.21.15.149:0/662277502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa19c0ff450 0x7fa19c0fcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:08.945 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.936+0000 7fa19a7fc700 1 -- 172.21.15.149:0/662277502 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18c0095f0 con 0x7fa19c100650 2023-09-13T04:34:08.945 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.936+0000 7fa19b7fe700 1 --2- 172.21.15.149:0/662277502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa19c100ff0 0x7fa19c19dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:34:08.945 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.936+0000 7fa19a7fc700 1 --2- 172.21.15.149:0/662277502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa19c100650 0x7fa19c19b720 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa18c009990 tx=0x7fa18c005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:34:08.945 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.937+0000 7fa1a0905700 1 -- 172.21.15.149:0/662277502 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18c01d070 con 0x7fa19c100650 2023-09-13T04:34:08.946 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.937+0000 7fa1a0905700 1 -- 172.21.15.149:0/662277502 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa18c022470 con 0x7fa19c100650 2023-09-13T04:34:08.946 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.937+0000 7fa1a1907700 1 -- 172.21.15.149:0/662277502 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa19c19eab0 con 0x7fa19c100650 2023-09-13T04:34:08.946 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.937+0000 7fa1a0905700 1 -- 172.21.15.149:0/662277502 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18c004230 con 0x7fa19c100650 2023-09-13T04:34:08.946 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.937+0000 7fa1a1907700 1 -- 172.21.15.149:0/662277502 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa19c19efa0 con 0x7fa19c100650 2023-09-13T04:34:08.946 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.938+0000 7fa1a1907700 1 -- 172.21.15.149:0/662277502 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa19c106fd0 con 0x7fa19c100650 2023-09-13T04:34:08.947 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.939+0000 7fa1a0905700 1 -- 172.21.15.149:0/662277502 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fa18c0225e0 con 0x7fa19c100650 2023-09-13T04:34:08.947 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.939+0000 7fa1a0905700 1 --2- 172.21.15.149:0/662277502 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fa1840a06c0 0x7fa1840a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:34:08.947 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.940+0000 7fa1a0905700 1 -- 172.21.15.149:0/662277502 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3698+0+0 (secure 0 0 0) 0x7fa18c0cbf20 con 0x7fa19c100650 2023-09-13T04:34:08.947 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.940+0000 7fa19affd700 1 --2- 172.21.15.149:0/662277502 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fa1840a06c0 0x7fa1840a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:34:08.948 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.940+0000 7fa19affd700 1 --2- 172.21.15.149:0/662277502 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fa1840a06c0 0x7fa1840a2b80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa19c1004b0 tx=0x7fa190009470 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:34:08.948 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:08.943+0000 7fa1a0905700 1 -- 172.21.15.149:0/662277502 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa18c096d60 con 0x7fa19c100650 2023-09-13T04:34:09.141 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:09.140+0000 7fa1a1907700 1 -- 172.21.15.149:0/662277502 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa19c02d010 con 0x7fa1840a06c0 2023-09-13T04:34:09.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:09 smithi156 ceph-mon[133675]: osdmap e26: 4 total, 4 up, 4 in 2023-09-13T04:34:09.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:09 smithi156 ceph-mon[133675]: Detected new or changed devices on smithi002 2023-09-13T04:34:09.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:09 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:09.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:09 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:09.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:09 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:34:09.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:09 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:34:09.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:09 smithi156 ceph-mon[133675]: Adjusting osd_memory_target on smithi002 to 8636M 2023-09-13T04:34:09.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:09 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:09.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:09 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:09.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:09 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:34:09.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:09 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:09.439 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:09 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:34:09.439 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:09 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:34:09.439 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:09 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:09.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:09 smithi002 ceph-mon[97651]: osdmap e26: 4 total, 4 up, 4 in 2023-09-13T04:34:09.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:09 smithi002 ceph-mon[97651]: Detected new or changed devices on smithi002 2023-09-13T04:34:09.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:09 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:09.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:09 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:09.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:09 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:34:09.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:09 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:34:09.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:09 smithi002 ceph-mon[97651]: Adjusting osd_memory_target on smithi002 to 8636M 2023-09-13T04:34:09.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:09 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:09.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:09 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:09.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:09 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:34:09.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:09 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:09.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:09 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:34:09.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:09 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:34:09.509 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:09 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:09.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:09 smithi149 ceph-mon[133315]: osdmap e26: 4 total, 4 up, 4 in 2023-09-13T04:34:09.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:09 smithi149 ceph-mon[133315]: Detected new or changed devices on smithi002 2023-09-13T04:34:09.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:09 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:09.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:09 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:09.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:09 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:34:09.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:09 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:34:09.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:09 smithi149 ceph-mon[133315]: Adjusting osd_memory_target on smithi002 to 8636M 2023-09-13T04:34:09.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:09 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:09.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:09 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:09.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:09 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:34:09.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:09 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:09.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:09 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:34:09.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:09 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:34:09.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:09 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:10.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:10 smithi149 ceph-mon[133315]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:34:10.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:10 smithi149 ceph-mon[133315]: osdmap e27: 4 total, 4 up, 4 in 2023-09-13T04:34:10.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:10 smithi149 ceph-mon[133315]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T04:34:10.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:10 smithi156 ceph-mon[133675]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:34:10.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:10 smithi156 ceph-mon[133675]: osdmap e27: 4 total, 4 up, 4 in 2023-09-13T04:34:10.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:10 smithi156 ceph-mon[133675]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T04:34:10.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:10 smithi002 ceph-mon[97651]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:34:10.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:10 smithi002 ceph-mon[97651]: osdmap e27: 4 total, 4 up, 4 in 2023-09-13T04:34:10.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:10 smithi002 ceph-mon[97651]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T04:34:11.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:11 smithi002 ceph-mon[97651]: osdmap e28: 4 total, 4 up, 4 in 2023-09-13T04:34:11.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:11 smithi149 ceph-mon[133315]: osdmap e28: 4 total, 4 up, 4 in 2023-09-13T04:34:11.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:11 smithi156 ceph-mon[133675]: osdmap e28: 4 total, 4 up, 4 in 2023-09-13T04:34:12.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:12 smithi002 ceph-mon[97651]: osdmap e29: 4 total, 4 up, 4 in 2023-09-13T04:34:12.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:12 smithi002 ceph-mon[97651]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T04:34:12.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:12 smithi002 ceph-mon[97651]: from='client.? 172.21.15.149:0/2234212625' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09ed1cc4-cce9-454b-bced-dbe54d622f97"}]: dispatch 2023-09-13T04:34:12.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:12 smithi002 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09ed1cc4-cce9-454b-bced-dbe54d622f97"}]: dispatch 2023-09-13T04:34:12.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:12 smithi002 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09ed1cc4-cce9-454b-bced-dbe54d622f97"}]': finished 2023-09-13T04:34:12.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:12 smithi002 ceph-mon[97651]: osdmap e30: 5 total, 4 up, 5 in 2023-09-13T04:34:12.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:12 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T04:34:12.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:12 smithi149 ceph-mon[133315]: osdmap e29: 4 total, 4 up, 4 in 2023-09-13T04:34:12.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:12 smithi149 ceph-mon[133315]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T04:34:12.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:12 smithi149 ceph-mon[133315]: from='client.? 172.21.15.149:0/2234212625' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09ed1cc4-cce9-454b-bced-dbe54d622f97"}]: dispatch 2023-09-13T04:34:12.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:12 smithi149 ceph-mon[133315]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09ed1cc4-cce9-454b-bced-dbe54d622f97"}]: dispatch 2023-09-13T04:34:12.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:12 smithi149 ceph-mon[133315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09ed1cc4-cce9-454b-bced-dbe54d622f97"}]': finished 2023-09-13T04:34:12.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:12 smithi149 ceph-mon[133315]: osdmap e30: 5 total, 4 up, 5 in 2023-09-13T04:34:12.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:12 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T04:34:12.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:12 smithi156 ceph-mon[133675]: osdmap e29: 4 total, 4 up, 4 in 2023-09-13T04:34:12.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:12 smithi156 ceph-mon[133675]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T04:34:12.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:12 smithi156 ceph-mon[133675]: from='client.? 172.21.15.149:0/2234212625' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09ed1cc4-cce9-454b-bced-dbe54d622f97"}]: dispatch 2023-09-13T04:34:12.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:12 smithi156 ceph-mon[133675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09ed1cc4-cce9-454b-bced-dbe54d622f97"}]: dispatch 2023-09-13T04:34:12.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:12 smithi156 ceph-mon[133675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09ed1cc4-cce9-454b-bced-dbe54d622f97"}]': finished 2023-09-13T04:34:12.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:12 smithi156 ceph-mon[133675]: osdmap e30: 5 total, 4 up, 5 in 2023-09-13T04:34:12.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:12 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T04:34:13.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:13 smithi002 ceph-mon[97651]: from='client.? 172.21.15.149:0/2319243400' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:34:13.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:13 smithi149 ceph-mon[133315]: from='client.? 172.21.15.149:0/2319243400' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:34:13.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:13 smithi156 ceph-mon[133675]: from='client.? 172.21.15.149:0/2319243400' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:34:14.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:14 smithi002 ceph-mon[97651]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 135 KiB/s, 0 objects/s recovering 2023-09-13T04:34:14.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:14 smithi149 ceph-mon[133315]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 135 KiB/s, 0 objects/s recovering 2023-09-13T04:34:14.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:14 smithi156 ceph-mon[133675]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 135 KiB/s, 0 objects/s recovering 2023-09-13T04:34:16.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:16 smithi002 ceph-mon[97651]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-13T04:34:16.838 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:16 smithi149 ceph-mon[133315]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-13T04:34:16.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:16 smithi156 ceph-mon[133675]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-13T04:34:18.774 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:18 smithi149 ceph-mon[133315]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-09-13T04:34:18.774 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:18 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-13T04:34:18.774 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:18 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:18.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:18 smithi156 ceph-mon[133675]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-09-13T04:34:18.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:18 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-13T04:34:18.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:18 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:19.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:18 smithi002 ceph-mon[97651]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-09-13T04:34:19.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:18 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-13T04:34:19.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:18 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:19.561 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:19 smithi149 ceph-mon[133315]: Deploying daemon osd.4 on smithi149 2023-09-13T04:34:19.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:19 smithi156 ceph-mon[133675]: Deploying daemon osd.4 on smithi149 2023-09-13T04:34:20.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:19 smithi002 ceph-mon[97651]: Deploying daemon osd.4 on smithi149 2023-09-13T04:34:20.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:20 smithi149 ceph-mon[133315]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-09-13T04:34:20.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:20 smithi156 ceph-mon[133675]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-09-13T04:34:21.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:20 smithi002 ceph-mon[97651]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-09-13T04:34:22.578 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:22 smithi149 ceph-mon[133315]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-13T04:34:22.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:22 smithi156 ceph-mon[133675]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-13T04:34:23.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:22 smithi002 ceph-mon[97651]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-13T04:34:23.801 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:23.801 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:23.802 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:34:23.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:23.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:23.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:34:24.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:24.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:24.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:34:24.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:24 smithi149 ceph-mon[133315]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T04:34:24.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:24 smithi156 ceph-mon[133675]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T04:34:25.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:24 smithi002 ceph-mon[97651]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T04:34:25.663 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 4 on host 'smithi149' 2023-09-13T04:34:25.664 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:25.663+0000 7fa1a0905700 1 -- 172.21.15.149:0/662277502 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa19c02d010 con 0x7fa1840a06c0 2023-09-13T04:34:25.666 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:25.666+0000 7fa1a1907700 1 -- 172.21.15.149:0/662277502 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fa1840a06c0 msgr2=0x7fa1840a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:34:25.666 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:25.666+0000 7fa1a1907700 1 --2- 172.21.15.149:0/662277502 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fa1840a06c0 0x7fa1840a2b80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa19c1004b0 tx=0x7fa190009470 comp rx=0 tx=0).stop 2023-09-13T04:34:25.667 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:25.666+0000 7fa1a1907700 1 -- 172.21.15.149:0/662277502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa19c100650 msgr2=0x7fa19c19b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:34:25.667 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:25.666+0000 7fa1a1907700 1 --2- 172.21.15.149:0/662277502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa19c100650 0x7fa19c19b720 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa18c009990 tx=0x7fa18c005e80 comp rx=0 tx=0).stop 2023-09-13T04:34:25.667 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:25.666+0000 7fa1a1907700 1 -- 172.21.15.149:0/662277502 shutdown_connections 2023-09-13T04:34:25.667 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:25.666+0000 7fa1a1907700 1 --2- 172.21.15.149:0/662277502 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fa1840a06c0 0x7fa1840a2b80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:25.667 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:25.666+0000 7fa1a1907700 1 --2- 172.21.15.149:0/662277502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa19c100ff0 0x7fa19c19dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:25.668 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:25.666+0000 7fa1a1907700 1 --2- 172.21.15.149:0/662277502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa19c0ff450 0x7fa19c0fcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:25.668 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:25.666+0000 7fa1a1907700 1 --2- 172.21.15.149:0/662277502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa19c100650 0x7fa19c19b720 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:25.668 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:25.666+0000 7fa1a1907700 1 -- 172.21.15.149:0/662277502 >> 172.21.15.149:0/662277502 conn(0x7fa19c0fa9d0 msgr2=0x7fa19c104110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:34:25.668 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:25.666+0000 7fa1a1907700 1 -- 172.21.15.149:0/662277502 shutdown_connections 2023-09-13T04:34:25.669 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:34:25.666+0000 7fa1a1907700 1 -- 172.21.15.149:0/662277502 wait complete. 2023-09-13T04:34:26.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:25 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:26.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:25 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:26.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:25 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:26.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:25 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:34:26.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:25 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:26.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:25 smithi149 ceph-mon[133315]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T04:34:26.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:25 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:26.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:25 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:26.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:25 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:34:26.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:25 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:26.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:25 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:26.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:25 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:26.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:25 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:34:26.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:25 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:26.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:25 smithi156 ceph-mon[133675]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T04:34:26.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:25 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:26.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:25 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:26.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:25 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:34:26.220 DEBUG:teuthology.orchestra.run.smithi149:osd.4> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.4.service 2023-09-13T04:34:26.223 INFO:tasks.cephadm:Deploying osd.5 on smithi156 with /dev/vg_nvme/lv_3... 2023-09-13T04:34:26.223 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-13T04:34:26.254 INFO:journalctl@ceph.osd.4.smithi149.stdout:-- Logs begin at Wed 2023-09-13 04:09:45 UTC. -- 2023-09-13T04:34:26.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:25 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:26.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:25 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:26.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:25 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:26.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:25 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:34:26.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:25 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:26.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:25 smithi002 ceph-mon[97651]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T04:34:26.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:25 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:26.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:25 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:26.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:25 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:34:26.787 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-13T04:34:26.862 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 13 04:34:26 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-4[146911]: 2023-09-13T04:34:26.507+0000 7fe522ad8640 -1 osd.4 0 log_to_monitors true 2023-09-13T04:34:27.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:26 smithi149 ceph-mon[133315]: from='osd.4 [v2:172.21.15.149:6808/3095161596,v1:172.21.15.149:6809/3095161596]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-13T04:34:27.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:26 smithi149 ceph-mon[133315]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-13T04:34:27.150 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:26 smithi156 ceph-mon[133675]: from='osd.4 [v2:172.21.15.149:6808/3095161596,v1:172.21.15.149:6809/3095161596]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-13T04:34:27.150 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:26 smithi156 ceph-mon[133675]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-13T04:34:27.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:26 smithi002 ceph-mon[97651]: from='osd.4 [v2:172.21.15.149:6808/3095161596,v1:172.21.15.149:6809/3095161596]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-13T04:34:27.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:26 smithi002 ceph-mon[97651]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-13T04:34:28.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:27 smithi156 ceph-mon[133675]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-13T04:34:28.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:27 smithi156 ceph-mon[133675]: osdmap e31: 5 total, 4 up, 5 in 2023-09-13T04:34:28.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:27 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T04:34:28.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:27 smithi156 ceph-mon[133675]: from='osd.4 [v2:172.21.15.149:6808/3095161596,v1:172.21.15.149:6809/3095161596]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-09-13T04:34:28.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:27 smithi156 ceph-mon[133675]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-09-13T04:34:28.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:27 smithi156 ceph-mon[133675]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T04:34:28.211 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 13 04:34:27 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-4[146911]: 2023-09-13T04:34:27.908+0000 7fe51814f700 -1 osd.4 0 waiting for initial osdmap 2023-09-13T04:34:28.212 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 13 04:34:27 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-4[146911]: 2023-09-13T04:34:27.913+0000 7fe510f3e700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-13T04:34:28.212 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:27 smithi149 ceph-mon[133315]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-13T04:34:28.213 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:27 smithi149 ceph-mon[133315]: osdmap e31: 5 total, 4 up, 5 in 2023-09-13T04:34:28.213 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:27 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T04:34:28.213 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:27 smithi149 ceph-mon[133315]: from='osd.4 [v2:172.21.15.149:6808/3095161596,v1:172.21.15.149:6809/3095161596]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-09-13T04:34:28.213 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:27 smithi149 ceph-mon[133315]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-09-13T04:34:28.213 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:27 smithi149 ceph-mon[133315]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T04:34:28.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:27 smithi002 ceph-mon[97651]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-13T04:34:28.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:27 smithi002 ceph-mon[97651]: osdmap e31: 5 total, 4 up, 5 in 2023-09-13T04:34:28.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:27 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T04:34:28.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:27 smithi002 ceph-mon[97651]: from='osd.4 [v2:172.21.15.149:6808/3095161596,v1:172.21.15.149:6809/3095161596]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-09-13T04:34:28.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:27 smithi002 ceph-mon[97651]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-09-13T04:34:28.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:27 smithi002 ceph-mon[97651]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T04:34:28.330 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-13T04:34:28.331 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-13T04:34:28.331 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2023-09-13T04:34:28.331 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2023-09-13T04:34:28.331 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00821726 s, 1.3 GB/s 2023-09-13T04:34:28.332 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2023-09-13T04:34:28.743 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph orch daemon add osd smithi156:vg_nvme/lv_3 2023-09-13T04:34:29.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:28 smithi156 ceph-mon[133675]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2023-09-13T04:34:29.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:28 smithi156 ceph-mon[133675]: osdmap e32: 5 total, 4 up, 5 in 2023-09-13T04:34:29.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:28 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T04:34:29.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:28 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T04:34:29.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:28 smithi156 ceph-mon[133675]: Detected new or changed devices on smithi149 2023-09-13T04:34:29.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:28 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:29.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:28 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:29.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:28 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:34:29.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:28 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:34:29.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:28 smithi156 ceph-mon[133675]: Adjusting osd_memory_target on smithi149 to 2939M 2023-09-13T04:34:29.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:28 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:29.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:28 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:29.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:28 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:34:29.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:28 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:29.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:28 smithi002 ceph-mon[97651]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2023-09-13T04:34:29.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:28 smithi002 ceph-mon[97651]: osdmap e32: 5 total, 4 up, 5 in 2023-09-13T04:34:29.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:28 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T04:34:29.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:28 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T04:34:29.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:28 smithi002 ceph-mon[97651]: Detected new or changed devices on smithi149 2023-09-13T04:34:29.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:28 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:29.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:28 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:29.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:28 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:34:29.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:28 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:34:29.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:28 smithi002 ceph-mon[97651]: Adjusting osd_memory_target on smithi149 to 2939M 2023-09-13T04:34:29.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:28 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:29.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:28 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:29.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:28 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:34:29.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:28 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:29.300 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-13T04:34:29.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:28 smithi149 ceph-mon[133315]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2023-09-13T04:34:29.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:28 smithi149 ceph-mon[133315]: osdmap e32: 5 total, 4 up, 5 in 2023-09-13T04:34:29.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:28 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T04:34:29.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:28 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T04:34:29.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:28 smithi149 ceph-mon[133315]: Detected new or changed devices on smithi149 2023-09-13T04:34:29.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:28 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:29.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:28 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:29.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:28 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:34:29.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:28 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:34:29.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:28 smithi149 ceph-mon[133315]: Adjusting osd_memory_target on smithi149 to 2939M 2023-09-13T04:34:29.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:28 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:29.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:28 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:29.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:28 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:34:29.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:28 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:30.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:29 smithi156 ceph-mon[133675]: purged_snaps scrub starts 2023-09-13T04:34:30.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:29 smithi156 ceph-mon[133675]: purged_snaps scrub ok 2023-09-13T04:34:30.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:29 smithi156 ceph-mon[133675]: osd.4 [v2:172.21.15.149:6808/3095161596,v1:172.21.15.149:6809/3095161596] boot 2023-09-13T04:34:30.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:29 smithi156 ceph-mon[133675]: osdmap e33: 5 total, 5 up, 5 in 2023-09-13T04:34:30.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:29 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T04:34:30.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:29 smithi156 ceph-mon[133675]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T04:34:30.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:29 smithi002 ceph-mon[97651]: purged_snaps scrub starts 2023-09-13T04:34:30.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:29 smithi002 ceph-mon[97651]: purged_snaps scrub ok 2023-09-13T04:34:30.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:29 smithi002 ceph-mon[97651]: osd.4 [v2:172.21.15.149:6808/3095161596,v1:172.21.15.149:6809/3095161596] boot 2023-09-13T04:34:30.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:29 smithi002 ceph-mon[97651]: osdmap e33: 5 total, 5 up, 5 in 2023-09-13T04:34:30.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:29 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T04:34:30.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:29 smithi002 ceph-mon[97651]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T04:34:30.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:29 smithi149 ceph-mon[133315]: purged_snaps scrub starts 2023-09-13T04:34:30.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:29 smithi149 ceph-mon[133315]: purged_snaps scrub ok 2023-09-13T04:34:30.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:29 smithi149 ceph-mon[133315]: osd.4 [v2:172.21.15.149:6808/3095161596,v1:172.21.15.149:6809/3095161596] boot 2023-09-13T04:34:30.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:29 smithi149 ceph-mon[133315]: osdmap e33: 5 total, 5 up, 5 in 2023-09-13T04:34:30.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:29 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T04:34:30.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:29 smithi149 ceph-mon[133315]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T04:34:30.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.490+0000 7f460b069700 1 -- 172.21.15.156:0/2937551307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46040ffc80 msgr2=0x7f46041000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:34:30.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.490+0000 7f460b069700 1 --2- 172.21.15.156:0/2937551307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46040ffc80 0x7f46041000a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009990 tx=0x7f45f8009ca0 comp rx=0 tx=0).stop 2023-09-13T04:34:30.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.490+0000 7f460b069700 1 -- 172.21.15.156:0/2937551307 shutdown_connections 2023-09-13T04:34:30.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.490+0000 7f460b069700 1 --2- 172.21.15.156:0/2937551307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4604101820 0x7f46041080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:30.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.490+0000 7f460b069700 1 --2- 172.21.15.156:0/2937551307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46040ffc80 0x7f46041000a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:30.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.490+0000 7f460b069700 1 --2- 172.21.15.156:0/2937551307 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4604100e80 0x7f46041012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:30.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.490+0000 7f460b069700 1 -- 172.21.15.156:0/2937551307 >> 172.21.15.156:0/2937551307 conn(0x7f46040fb220 msgr2=0x7f46040fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:34:30.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.490+0000 7f460b069700 1 -- 172.21.15.156:0/2937551307 shutdown_connections 2023-09-13T04:34:30.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.490+0000 7f460b069700 1 -- 172.21.15.156:0/2937551307 wait complete. 2023-09-13T04:34:30.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.490+0000 7f460b069700 1 Processor -- start 2023-09-13T04:34:30.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.490+0000 7f460b069700 1 -- start start 2023-09-13T04:34:30.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.491+0000 7f460b069700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46040ffc80 0x7f460410a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:34:30.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.491+0000 7f460b069700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4604100e80 0x7f4604108be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:34:30.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.491+0000 7f460b069700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4604101820 0x7f4604109120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:34:30.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.491+0000 7f460b069700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46041098a0 con 0x7f46040ffc80 2023-09-13T04:34:30.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.491+0000 7f460b069700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4604109a10 con 0x7f4604100e80 2023-09-13T04:34:30.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.491+0000 7f460b069700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46041a0b40 con 0x7f4604101820 2023-09-13T04:34:30.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.491+0000 7f4609606700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4604101820 0x7f4604109120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:34:30.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.491+0000 7f4609606700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4604101820 0x7f4604109120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37170/0 (socket says 172.21.15.156:37170) 2023-09-13T04:34:30.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.491+0000 7f4609606700 1 -- 172.21.15.156:0/3707538997 learned_addr learned my addr 172.21.15.156:0/3707538997 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-13T04:34:30.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.491+0000 7f4603fff700 1 --2- 172.21.15.156:0/3707538997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4604100e80 0x7f4604108be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:34:30.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.491+0000 7f4608e05700 1 --2- 172.21.15.156:0/3707538997 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46040ffc80 0x7f460410a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:34:30.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.491+0000 7f4609606700 1 -- 172.21.15.156:0/3707538997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4604100e80 msgr2=0x7f4604108be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:34:30.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.491+0000 7f4609606700 1 --2- 172.21.15.156:0/3707538997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4604100e80 0x7f4604108be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:30.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.491+0000 7f4609606700 1 -- 172.21.15.156:0/3707538997 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46040ffc80 msgr2=0x7f460410a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:34:30.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.491+0000 7f4609606700 1 --2- 172.21.15.156:0/3707538997 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46040ffc80 0x7f460410a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:30.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.491+0000 7f4609606700 1 -- 172.21.15.156:0/3707538997 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f80095f0 con 0x7f4604101820 2023-09-13T04:34:30.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.491+0000 7f4609606700 1 --2- 172.21.15.156:0/3707538997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4604101820 0x7f4604109120 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00b970 tx=0x7f45fc00bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:34:30.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.491+0000 7f4601ffb700 1 -- 172.21.15.156:0/3707538997 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc00c930 con 0x7f4604101820 2023-09-13T04:34:30.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.491+0000 7f460b069700 1 -- 172.21.15.156:0/3707538997 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46041a0e20 con 0x7f4604101820 2023-09-13T04:34:30.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.491+0000 7f4601ffb700 1 -- 172.21.15.156:0/3707538997 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45fc014440 con 0x7f4604101820 2023-09-13T04:34:30.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.491+0000 7f4601ffb700 1 -- 172.21.15.156:0/3707538997 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc0075d0 con 0x7f4604101820 2023-09-13T04:34:30.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.492+0000 7f460b069700 1 -- 172.21.15.156:0/3707538997 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46041a12c0 con 0x7f4604101820 2023-09-13T04:34:30.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.492+0000 7f460b069700 1 -- 172.21.15.156:0/3707538997 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f460410c650 con 0x7f4604101820 2023-09-13T04:34:30.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.493+0000 7f4601ffb700 1 -- 172.21.15.156:0/3707538997 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f45fc0145b0 con 0x7f4604101820 2023-09-13T04:34:30.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.494+0000 7f4601ffb700 1 --2- 172.21.15.156:0/3707538997 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f45f40a0640 0x7f45f40a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:34:30.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.494+0000 7f4601ffb700 1 -- 172.21.15.156:0/3707538997 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f45fc006e40 con 0x7f4604101820 2023-09-13T04:34:30.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.495+0000 7f4608e05700 1 --2- 172.21.15.156:0/3707538997 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f45f40a0640 0x7f45f40a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:34:30.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.495+0000 7f4608e05700 1 --2- 172.21.15.156:0/3707538997 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f45f40a0640 0x7f45f40a2b00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f45f80036f0 tx=0x7f45f8003680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:34:30.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.499+0000 7f4601ffb700 1 -- 172.21.15.156:0/3707538997 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f45fc04ee10 con 0x7f4604101820 2023-09-13T04:34:30.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:30.693+0000 7f460b069700 1 -- 172.21.15.156:0/3707538997 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f46040610f0 con 0x7f45f40a0640 2023-09-13T04:34:31.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:31 smithi156 ceph-mon[133675]: osdmap e34: 5 total, 5 up, 5 in 2023-09-13T04:34:31.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:31 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:34:31.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:31 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:34:31.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:31 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:31.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:31 smithi002 ceph-mon[97651]: osdmap e34: 5 total, 5 up, 5 in 2023-09-13T04:34:31.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:31 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:34:31.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:31 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:34:31.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:31 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:31.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:31 smithi149 ceph-mon[133315]: osdmap e34: 5 total, 5 up, 5 in 2023-09-13T04:34:31.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:31 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:34:31.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:31 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:34:31.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:31 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:32.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:32 smithi156 ceph-mon[133675]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:34:32.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:32 smithi156 ceph-mon[133675]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:32.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:32 smithi002 ceph-mon[97651]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:34:32.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:32 smithi002 ceph-mon[97651]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:32.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:32 smithi149 ceph-mon[133315]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:34:32.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:32 smithi149 ceph-mon[133315]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:33.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:33 smithi156 ceph-mon[133675]: from='client.? 172.21.15.156:0/1072289828' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ede570f-f574-4d3f-934d-245afd175128"}]: dispatch 2023-09-13T04:34:33.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:33 smithi156 ceph-mon[133675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ede570f-f574-4d3f-934d-245afd175128"}]: dispatch 2023-09-13T04:34:33.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:33 smithi156 ceph-mon[133675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ede570f-f574-4d3f-934d-245afd175128"}]': finished 2023-09-13T04:34:33.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:33 smithi156 ceph-mon[133675]: osdmap e35: 6 total, 5 up, 6 in 2023-09-13T04:34:33.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:33 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T04:34:33.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:33 smithi002 ceph-mon[97651]: from='client.? 172.21.15.156:0/1072289828' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ede570f-f574-4d3f-934d-245afd175128"}]: dispatch 2023-09-13T04:34:33.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:33 smithi002 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ede570f-f574-4d3f-934d-245afd175128"}]: dispatch 2023-09-13T04:34:33.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:33 smithi002 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ede570f-f574-4d3f-934d-245afd175128"}]': finished 2023-09-13T04:34:33.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:33 smithi002 ceph-mon[97651]: osdmap e35: 6 total, 5 up, 6 in 2023-09-13T04:34:33.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:33 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T04:34:33.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:33 smithi149 ceph-mon[133315]: from='client.? 172.21.15.156:0/1072289828' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ede570f-f574-4d3f-934d-245afd175128"}]: dispatch 2023-09-13T04:34:33.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:33 smithi149 ceph-mon[133315]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ede570f-f574-4d3f-934d-245afd175128"}]: dispatch 2023-09-13T04:34:33.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:33 smithi149 ceph-mon[133315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ede570f-f574-4d3f-934d-245afd175128"}]': finished 2023-09-13T04:34:33.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:33 smithi149 ceph-mon[133315]: osdmap e35: 6 total, 5 up, 6 in 2023-09-13T04:34:33.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:33 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T04:34:34.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:34 smithi156 ceph-mon[133675]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:34.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:34 smithi156 ceph-mon[133675]: from='client.? 172.21.15.156:0/297198909' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:34:34.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:34 smithi002 ceph-mon[97651]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:34.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:34 smithi002 ceph-mon[97651]: from='client.? 172.21.15.156:0/297198909' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:34:34.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:34 smithi149 ceph-mon[133315]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:34.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:34 smithi149 ceph-mon[133315]: from='client.? 172.21.15.156:0/297198909' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:34:36.843 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:36 smithi156 ceph-mon[133675]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:36.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:36 smithi149 ceph-mon[133315]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:37.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:36 smithi002 ceph-mon[97651]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:38.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:38 smithi149 ceph-mon[133315]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:38.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:38 smithi156 ceph-mon[133675]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:39.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:38 smithi002 ceph-mon[97651]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:40.384 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:40 smithi156 ceph-mon[133675]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:40.384 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:40 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-13T04:34:40.384 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:40 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:40.384 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:40 smithi156 ceph-mon[133675]: Deploying daemon osd.5 on smithi156 2023-09-13T04:34:40.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:40 smithi002 ceph-mon[97651]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:40.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:40 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-13T04:34:40.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:40 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:40.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:40 smithi002 ceph-mon[97651]: Deploying daemon osd.5 on smithi156 2023-09-13T04:34:40.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:40 smithi149 ceph-mon[133315]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:40.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:40 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-13T04:34:40.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:40 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:40.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:40 smithi149 ceph-mon[133315]: Deploying daemon osd.5 on smithi156 2023-09-13T04:34:42.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:42 smithi149 ceph-mon[133315]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:42.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:42 smithi156 ceph-mon[133675]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:43.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:42 smithi002 ceph-mon[97651]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:44.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:44 smithi156 ceph-mon[133675]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:44.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:44 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:44.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:44 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:34:44.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:44 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:44.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:44 smithi149 ceph-mon[133315]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:44.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:44 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:44.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:44 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:34:44.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:44 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:45.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:44 smithi002 ceph-mon[97651]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:45.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:44 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:45.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:44 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:34:45.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:44 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:46.648 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:46 smithi156 ceph-mon[133675]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:46.648 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:46 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:46.649 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:46 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:46.649 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:46 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:46.649 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:46 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:34:46.649 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:46 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:46.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:46.704+0000 7f4601ffb700 1 -- 172.21.15.156:0/3707538997 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f46040610f0 con 0x7f45f40a0640 2023-09-13T04:34:46.707 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 5 on host 'smithi156' 2023-09-13T04:34:46.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:46.707+0000 7f460b069700 1 -- 172.21.15.156:0/3707538997 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f45f40a0640 msgr2=0x7f45f40a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:34:46.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:46.707+0000 7f460b069700 1 --2- 172.21.15.156:0/3707538997 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f45f40a0640 0x7f45f40a2b00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f45f80036f0 tx=0x7f45f8003680 comp rx=0 tx=0).stop 2023-09-13T04:34:46.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:46.707+0000 7f460b069700 1 -- 172.21.15.156:0/3707538997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4604101820 msgr2=0x7f4604109120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:34:46.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:46.707+0000 7f460b069700 1 --2- 172.21.15.156:0/3707538997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4604101820 0x7f4604109120 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00b970 tx=0x7f45fc00bc80 comp rx=0 tx=0).stop 2023-09-13T04:34:46.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:46.707+0000 7f460b069700 1 -- 172.21.15.156:0/3707538997 shutdown_connections 2023-09-13T04:34:46.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:46.707+0000 7f460b069700 1 --2- 172.21.15.156:0/3707538997 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f45f40a0640 0x7f45f40a2b00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:46.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:46.707+0000 7f460b069700 1 --2- 172.21.15.156:0/3707538997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4604101820 0x7f4604109120 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:46.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:46.707+0000 7f460b069700 1 --2- 172.21.15.156:0/3707538997 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46040ffc80 0x7f460410a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:46.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:46.707+0000 7f460b069700 1 --2- 172.21.15.156:0/3707538997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4604100e80 0x7f4604108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:46.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:46.707+0000 7f460b069700 1 -- 172.21.15.156:0/3707538997 >> 172.21.15.156:0/3707538997 conn(0x7f46040fb220 msgr2=0x7f46040fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:34:46.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:46.707+0000 7f460b069700 1 -- 172.21.15.156:0/3707538997 shutdown_connections 2023-09-13T04:34:46.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:34:46.707+0000 7f460b069700 1 -- 172.21.15.156:0/3707538997 wait complete. 2023-09-13T04:34:46.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:46 smithi149 ceph-mon[133315]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:46.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:46 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:46.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:46 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:46.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:46 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:46.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:46 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:34:46.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:46 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:47.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:46 smithi002 ceph-mon[97651]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:47.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:46 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:47.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:46 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:47.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:46 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:47.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:46 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:34:47.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:46 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:47.088 DEBUG:teuthology.orchestra.run.smithi156:osd.5> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.5.service 2023-09-13T04:34:47.091 INFO:tasks.cephadm:Deploying osd.6 on smithi002 with /dev/vg_nvme/lv_2... 2023-09-13T04:34:47.092 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-13T04:34:47.144 INFO:journalctl@ceph.osd.5.smithi156.stdout:-- Logs begin at Wed 2023-09-13 04:09:45 UTC. -- 2023-09-13T04:34:47.850 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:34:47.900 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:47 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:47.900 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:47 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:47.900 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:47 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:34:47.901 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:47 smithi156 ceph-mon[133675]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:47.901 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:34:47 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:34:47.828+0000 7f4e38c59640 -1 osd.5 0 log_to_monitors true 2023-09-13T04:34:48.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:47 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:48.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:47 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:48.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:47 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:34:48.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:47 smithi002 ceph-mon[97651]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:48.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:47 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:48.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:47 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:48.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:47 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:34:48.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:47 smithi149 ceph-mon[133315]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:48.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:48 smithi002 ceph-mon[97651]: from='osd.5 [v2:172.21.15.156:6808/1709817635,v1:172.21.15.156:6809/1709817635]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-13T04:34:48.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:48 smithi002 ceph-mon[97651]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-13T04:34:49.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:48 smithi149 ceph-mon[133315]: from='osd.5 [v2:172.21.15.156:6808/1709817635,v1:172.21.15.156:6809/1709817635]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-13T04:34:49.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:48 smithi149 ceph-mon[133315]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-13T04:34:49.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:48 smithi156 ceph-mon[133675]: from='osd.5 [v2:172.21.15.156:6808/1709817635,v1:172.21.15.156:6809/1709817635]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-13T04:34:49.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:48 smithi156 ceph-mon[133675]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-13T04:34:49.869 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-13T04:34:49.870 INFO:teuthology.orchestra.run.smithi002.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-13T04:34:49.870 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10+0 records in 2023-09-13T04:34:49.870 INFO:teuthology.orchestra.run.smithi002.stderr:10+0 records out 2023-09-13T04:34:49.870 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0325189 s, 322 MB/s 2023-09-13T04:34:49.871 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping successful for: 2023-09-13T04:34:50.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:49 smithi002 ceph-mon[97651]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-13T04:34:50.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:49 smithi002 ceph-mon[97651]: osdmap e36: 6 total, 5 up, 6 in 2023-09-13T04:34:50.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:49 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T04:34:50.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:49 smithi002 ceph-mon[97651]: from='osd.5 [v2:172.21.15.156:6808/1709817635,v1:172.21.15.156:6809/1709817635]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-13T04:34:50.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:49 smithi002 ceph-mon[97651]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-13T04:34:50.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:49 smithi002 ceph-mon[97651]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:50.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:49 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:50.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:49 smithi149 ceph-mon[133315]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-13T04:34:50.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:49 smithi149 ceph-mon[133315]: osdmap e36: 6 total, 5 up, 6 in 2023-09-13T04:34:50.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:49 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T04:34:50.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:49 smithi149 ceph-mon[133315]: from='osd.5 [v2:172.21.15.156:6808/1709817635,v1:172.21.15.156:6809/1709817635]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-13T04:34:50.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:49 smithi149 ceph-mon[133315]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-13T04:34:50.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:49 smithi149 ceph-mon[133315]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:50.139 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:49 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:50.186 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:34:49 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:34:49.707+0000 7f4e2e2d0700 -1 osd.5 0 waiting for initial osdmap 2023-09-13T04:34:50.186 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:34:49 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:34:49.715+0000 7f4e288c2700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-13T04:34:50.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:49 smithi156 ceph-mon[133675]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-13T04:34:50.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:49 smithi156 ceph-mon[133675]: osdmap e36: 6 total, 5 up, 6 in 2023-09-13T04:34:50.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:49 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T04:34:50.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:49 smithi156 ceph-mon[133675]: from='osd.5 [v2:172.21.15.156:6808/1709817635,v1:172.21.15.156:6809/1709817635]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-13T04:34:50.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:49 smithi156 ceph-mon[133675]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-13T04:34:50.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:49 smithi156 ceph-mon[133675]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T04:34:50.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:49 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:50.336 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph orch daemon add osd smithi002:vg_nvme/lv_2 2023-09-13T04:34:51.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:50 smithi002 ceph-mon[97651]: Detected new or changed devices on smithi156 2023-09-13T04:34:51.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:50 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:51.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:50 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:34:51.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:50 smithi002 ceph-mon[97651]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-09-13T04:34:51.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:50 smithi002 ceph-mon[97651]: osdmap e37: 6 total, 5 up, 6 in 2023-09-13T04:34:51.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:50 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:34:51.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:50 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T04:34:51.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:50 smithi002 ceph-mon[97651]: Adjusting osd_memory_target on smithi156 to 8584M 2023-09-13T04:34:51.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:50 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:51.010 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:50 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T04:34:51.010 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:50 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:51.010 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:50 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:34:51.011 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:50 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:51.069 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:34:51.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:50 smithi149 ceph-mon[133315]: Detected new or changed devices on smithi156 2023-09-13T04:34:51.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:50 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:51.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:50 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:34:51.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:50 smithi149 ceph-mon[133315]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-09-13T04:34:51.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:50 smithi149 ceph-mon[133315]: osdmap e37: 6 total, 5 up, 6 in 2023-09-13T04:34:51.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:50 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:34:51.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:50 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T04:34:51.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:50 smithi149 ceph-mon[133315]: Adjusting osd_memory_target on smithi156 to 8584M 2023-09-13T04:34:51.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:50 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:51.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:50 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T04:34:51.139 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:50 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:51.139 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:50 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:34:51.139 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:50 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:51.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:50 smithi156 ceph-mon[133675]: Detected new or changed devices on smithi156 2023-09-13T04:34:51.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:50 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:51.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:50 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:34:51.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:50 smithi156 ceph-mon[133675]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-09-13T04:34:51.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:50 smithi156 ceph-mon[133675]: osdmap e37: 6 total, 5 up, 6 in 2023-09-13T04:34:51.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:50 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:34:51.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:50 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T04:34:51.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:50 smithi156 ceph-mon[133675]: Adjusting osd_memory_target on smithi156 to 8584M 2023-09-13T04:34:51.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:50 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:51.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:50 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T04:34:51.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:50 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:51.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:50 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:34:51.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:50 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:34:51.712 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:51 smithi002 ceph-mon[97651]: purged_snaps scrub starts 2023-09-13T04:34:51.713 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:51 smithi002 ceph-mon[97651]: purged_snaps scrub ok 2023-09-13T04:34:52.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:51 smithi002 ceph-mon[97651]: osd.5 [v2:172.21.15.156:6808/1709817635,v1:172.21.15.156:6809/1709817635] boot 2023-09-13T04:34:52.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:51 smithi002 ceph-mon[97651]: osdmap e38: 6 total, 6 up, 6 in 2023-09-13T04:34:52.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:51 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T04:34:52.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:51 smithi002 ceph-mon[97651]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T04:34:52.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:51 smithi149 ceph-mon[133315]: purged_snaps scrub starts 2023-09-13T04:34:52.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:51 smithi149 ceph-mon[133315]: purged_snaps scrub ok 2023-09-13T04:34:52.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:51 smithi149 ceph-mon[133315]: osd.5 [v2:172.21.15.156:6808/1709817635,v1:172.21.15.156:6809/1709817635] boot 2023-09-13T04:34:52.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:51 smithi149 ceph-mon[133315]: osdmap e38: 6 total, 6 up, 6 in 2023-09-13T04:34:52.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:51 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T04:34:52.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:51 smithi149 ceph-mon[133315]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T04:34:52.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:51 smithi156 ceph-mon[133675]: purged_snaps scrub starts 2023-09-13T04:34:52.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:51 smithi156 ceph-mon[133675]: purged_snaps scrub ok 2023-09-13T04:34:52.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:51 smithi156 ceph-mon[133675]: osd.5 [v2:172.21.15.156:6808/1709817635,v1:172.21.15.156:6809/1709817635] boot 2023-09-13T04:34:52.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:51 smithi156 ceph-mon[133675]: osdmap e38: 6 total, 6 up, 6 in 2023-09-13T04:34:52.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:51 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T04:34:52.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:51 smithi156 ceph-mon[133675]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T04:34:52.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.682+0000 7fb668d17700 1 -- 172.21.15.2:0/116375751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb664100d70 msgr2=0x7fb664101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:34:52.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.682+0000 7fb668d17700 1 --2- 172.21.15.2:0/116375751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb664100d70 0x7fb664101190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb654009920 tx=0x7fb654009c30 comp rx=0 tx=0).stop 2023-09-13T04:34:52.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.682+0000 7fb668d17700 1 -- 172.21.15.2:0/116375751 shutdown_connections 2023-09-13T04:34:52.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.682+0000 7fb668d17700 1 --2- 172.21.15.2:0/116375751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb664102910 0x7fb6641091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:52.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.682+0000 7fb668d17700 1 --2- 172.21.15.2:0/116375751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb664100d70 0x7fb664101190 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:52.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.682+0000 7fb668d17700 1 --2- 172.21.15.2:0/116375751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb664101f70 0x7fb6641023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:52.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.682+0000 7fb668d17700 1 -- 172.21.15.2:0/116375751 >> 172.21.15.2:0/116375751 conn(0x7fb6640fc2d0 msgr2=0x7fb6640fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:34:52.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.682+0000 7fb668d17700 1 -- 172.21.15.2:0/116375751 shutdown_connections 2023-09-13T04:34:52.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.682+0000 7fb668d17700 1 -- 172.21.15.2:0/116375751 wait complete. 2023-09-13T04:34:52.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.683+0000 7fb668d17700 1 Processor -- start 2023-09-13T04:34:52.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.683+0000 7fb668d17700 1 -- start start 2023-09-13T04:34:52.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.684+0000 7fb668d17700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb664100d70 0x7fb664074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:34:52.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.684+0000 7fb668d17700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb664101f70 0x7fb6640730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:34:52.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.684+0000 7fb668d17700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb664102910 0x7fb6640735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:34:52.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.684+0000 7fb668d17700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb664073d60 con 0x7fb664102910 2023-09-13T04:34:52.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.684+0000 7fb668d17700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb664073ed0 con 0x7fb664101f70 2023-09-13T04:34:52.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.684+0000 7fb668d17700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6641a0b70 con 0x7fb664100d70 2023-09-13T04:34:52.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.684+0000 7fb662d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb664102910 0x7fb6640735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:34:52.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.684+0000 7fb662d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb664102910 0x7fb6640735e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43440/0 (socket says 172.21.15.2:43440) 2023-09-13T04:34:52.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.684+0000 7fb662d9d700 1 -- 172.21.15.2:0/2162891359 learned_addr learned my addr 172.21.15.2:0/2162891359 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:34:52.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.684+0000 7fb66259c700 1 --2- 172.21.15.2:0/2162891359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb664100d70 0x7fb664074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:34:52.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.684+0000 7fb661d9b700 1 --2- 172.21.15.2:0/2162891359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb664101f70 0x7fb6640730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:34:52.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.685+0000 7fb662d9d700 1 -- 172.21.15.2:0/2162891359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb664100d70 msgr2=0x7fb664074a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:34:52.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.685+0000 7fb662d9d700 1 --2- 172.21.15.2:0/2162891359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb664100d70 0x7fb664074a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:52.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.685+0000 7fb662d9d700 1 -- 172.21.15.2:0/2162891359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb664101f70 msgr2=0x7fb6640730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:34:52.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.685+0000 7fb662d9d700 1 --2- 172.21.15.2:0/2162891359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb664101f70 0x7fb6640730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:34:52.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.685+0000 7fb662d9d700 1 -- 172.21.15.2:0/2162891359 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6580095f0 con 0x7fb664102910 2023-09-13T04:34:52.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.685+0000 7fb662d9d700 1 --2- 172.21.15.2:0/2162891359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb664102910 0x7fb6640735e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb65c00ef30 tx=0x7fb65c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:34:52.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.685+0000 7fb6537fe700 1 -- 172.21.15.2:0/2162891359 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c018070 con 0x7fb664102910 2023-09-13T04:34:52.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.685+0000 7fb6537fe700 1 -- 172.21.15.2:0/2162891359 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb65c00f040 con 0x7fb664102910 2023-09-13T04:34:52.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.685+0000 7fb668d17700 1 -- 172.21.15.2:0/2162891359 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb654009520 con 0x7fb664102910 2023-09-13T04:34:52.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.685+0000 7fb6537fe700 1 -- 172.21.15.2:0/2162891359 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c00cd00 con 0x7fb664102910 2023-09-13T04:34:52.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.685+0000 7fb668d17700 1 -- 172.21.15.2:0/2162891359 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6641a1110 con 0x7fb664102910 2023-09-13T04:34:52.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.686+0000 7fb668d17700 1 -- 172.21.15.2:0/2162891359 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb664108440 con 0x7fb664102910 2023-09-13T04:34:52.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.690+0000 7fb6537fe700 1 -- 172.21.15.2:0/2162891359 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fb65c003a00 con 0x7fb664102910 2023-09-13T04:34:52.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.691+0000 7fb6537fe700 1 --2- 172.21.15.2:0/2162891359 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb64c0a0640 0x7fb64c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:34:52.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.691+0000 7fb6537fe700 1 -- 172.21.15.2:0/2162891359 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4562+0+0 (secure 0 0 0) 0x7fb65c0cb2a0 con 0x7fb664102910 2023-09-13T04:34:52.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.692+0000 7fb66259c700 1 --2- 172.21.15.2:0/2162891359 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb64c0a0640 0x7fb64c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:34:52.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.692+0000 7fb66259c700 1 --2- 172.21.15.2:0/2162891359 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb64c0a0640 0x7fb64c0a2b00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb6540098f0 tx=0x7fb654019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:34:52.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.694+0000 7fb6537fe700 1 -- 172.21.15.2:0/2162891359 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb65c095c60 con 0x7fb664102910 2023-09-13T04:34:52.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:34:52.917+0000 7fb668d17700 1 -- 172.21.15.2:0/2162891359 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb66402d0b0 con 0x7fb64c0a0640 2023-09-13T04:34:53.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:52 smithi002 ceph-mon[97651]: osdmap e39: 6 total, 6 up, 6 in 2023-09-13T04:34:53.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:52 smithi149 ceph-mon[133315]: osdmap e39: 6 total, 6 up, 6 in 2023-09-13T04:34:53.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:52 smithi156 ceph-mon[133675]: osdmap e39: 6 total, 6 up, 6 in 2023-09-13T04:34:53.997 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:53 smithi002 ceph-mon[97651]: osdmap e40: 6 total, 6 up, 6 in 2023-09-13T04:34:53.997 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:53 smithi002 ceph-mon[97651]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:34:53.997 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:53 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:34:53.998 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:53 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:34:53.998 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:53 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:53.998 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:53 smithi002 ceph-mon[97651]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T04:34:54.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:53 smithi149 ceph-mon[133315]: osdmap e40: 6 total, 6 up, 6 in 2023-09-13T04:34:54.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:53 smithi149 ceph-mon[133315]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:34:54.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:53 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:34:54.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:53 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:34:54.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:53 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:54.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:53 smithi149 ceph-mon[133315]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T04:34:54.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:53 smithi156 ceph-mon[133675]: osdmap e40: 6 total, 6 up, 6 in 2023-09-13T04:34:54.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:53 smithi156 ceph-mon[133675]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:34:54.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:53 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:34:54.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:53 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:34:54.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:53 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:34:54.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:53 smithi156 ceph-mon[133675]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T04:34:55.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:54 smithi002 ceph-mon[97651]: osdmap e41: 6 total, 6 up, 6 in 2023-09-13T04:34:55.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:54 smithi149 ceph-mon[133315]: osdmap e41: 6 total, 6 up, 6 in 2023-09-13T04:34:55.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:54 smithi156 ceph-mon[133675]: osdmap e41: 6 total, 6 up, 6 in 2023-09-13T04:34:56.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:55 smithi002 ceph-mon[97651]: osdmap e42: 6 total, 6 up, 6 in 2023-09-13T04:34:56.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:55 smithi002 ceph-mon[97651]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T04:34:56.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:55 smithi149 ceph-mon[133315]: osdmap e42: 6 total, 6 up, 6 in 2023-09-13T04:34:56.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:55 smithi149 ceph-mon[133315]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T04:34:56.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:55 smithi156 ceph-mon[133675]: osdmap e42: 6 total, 6 up, 6 in 2023-09-13T04:34:56.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:55 smithi156 ceph-mon[133675]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T04:34:57.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:56 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/249167886' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46e6e453-ced8-44b7-8ec7-e38497f8d4ab"}]: dispatch 2023-09-13T04:34:57.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:56 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/249167886' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46e6e453-ced8-44b7-8ec7-e38497f8d4ab"}]': finished 2023-09-13T04:34:57.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:56 smithi002 ceph-mon[97651]: osdmap e43: 7 total, 6 up, 7 in 2023-09-13T04:34:57.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:56 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T04:34:57.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:56 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/249167886' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46e6e453-ced8-44b7-8ec7-e38497f8d4ab"}]: dispatch 2023-09-13T04:34:57.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:56 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/249167886' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46e6e453-ced8-44b7-8ec7-e38497f8d4ab"}]': finished 2023-09-13T04:34:57.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:56 smithi149 ceph-mon[133315]: osdmap e43: 7 total, 6 up, 7 in 2023-09-13T04:34:57.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:56 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T04:34:57.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:56 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/249167886' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46e6e453-ced8-44b7-8ec7-e38497f8d4ab"}]: dispatch 2023-09-13T04:34:57.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:56 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/249167886' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46e6e453-ced8-44b7-8ec7-e38497f8d4ab"}]': finished 2023-09-13T04:34:57.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:56 smithi156 ceph-mon[133675]: osdmap e43: 7 total, 6 up, 7 in 2023-09-13T04:34:57.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:56 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T04:34:58.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:57 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3937898165' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:34:58.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:34:57 smithi002 ceph-mon[97651]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 122 KiB/s, 0 objects/s recovering 2023-09-13T04:34:58.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:57 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3937898165' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:34:58.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:34:57 smithi149 ceph-mon[133315]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 122 KiB/s, 0 objects/s recovering 2023-09-13T04:34:58.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:57 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3937898165' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:34:58.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:34:57 smithi156 ceph-mon[133675]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 122 KiB/s, 0 objects/s recovering 2023-09-13T04:35:00.428 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:00 smithi002 ceph-mon[97651]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-13T04:35:00.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:00 smithi156 ceph-mon[133675]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-13T04:35:00.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:00 smithi149 ceph-mon[133315]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-13T04:35:02.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:02 smithi149 ceph-mon[133315]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2023-09-13T04:35:02.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:02 smithi156 ceph-mon[133675]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2023-09-13T04:35:03.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:02 smithi002 ceph-mon[97651]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2023-09-13T04:35:04.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:04 smithi002 ceph-mon[97651]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-09-13T04:35:04.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:04 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-13T04:35:04.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:04 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:04.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:04 smithi002 ceph-mon[97651]: Deploying daemon osd.6 on smithi002 2023-09-13T04:35:04.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:04 smithi149 ceph-mon[133315]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-09-13T04:35:04.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:04 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-13T04:35:04.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:04 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:04.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:04 smithi149 ceph-mon[133315]: Deploying daemon osd.6 on smithi002 2023-09-13T04:35:04.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:04 smithi156 ceph-mon[133675]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-09-13T04:35:04.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:04 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-13T04:35:04.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:04 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:04.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:04 smithi156 ceph-mon[133675]: Deploying daemon osd.6 on smithi002 2023-09-13T04:35:06.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:06 smithi002 ceph-mon[97651]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-13T04:35:06.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:06 smithi149 ceph-mon[133315]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-13T04:35:06.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:06 smithi156 ceph-mon[133675]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-13T04:35:08.778 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:08 smithi002 ceph-mon[97651]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-09-13T04:35:08.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:08 smithi149 ceph-mon[133315]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-09-13T04:35:08.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:08 smithi156 ceph-mon[133675]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-09-13T04:35:10.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:10 smithi156 ceph-mon[133675]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T04:35:10.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:10 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:10.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:10 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:10.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:10 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:35:10.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:10 smithi002 ceph-mon[97651]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T04:35:10.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:10 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:10.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:10 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:10.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:10 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:35:10.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:10 smithi149 ceph-mon[133315]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T04:35:10.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:10 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:10.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:10 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:10.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:10 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:35:12.667 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:12 smithi002 ceph-mon[97651]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T04:35:12.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:12 smithi149 ceph-mon[133315]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T04:35:12.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:12 smithi156 ceph-mon[133675]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T04:35:14.084 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 6 on host 'smithi002' 2023-09-13T04:35:14.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:35:14.084+0000 7fb6537fe700 1 -- 172.21.15.2:0/2162891359 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb66402d0b0 con 0x7fb64c0a0640 2023-09-13T04:35:14.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:35:14.087+0000 7fb668d17700 1 -- 172.21.15.2:0/2162891359 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb64c0a0640 msgr2=0x7fb64c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:35:14.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:35:14.087+0000 7fb668d17700 1 --2- 172.21.15.2:0/2162891359 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb64c0a0640 0x7fb64c0a2b00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb6540098f0 tx=0x7fb654019040 comp rx=0 tx=0).stop 2023-09-13T04:35:14.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:35:14.087+0000 7fb668d17700 1 -- 172.21.15.2:0/2162891359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb664102910 msgr2=0x7fb6640735e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:35:14.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:35:14.087+0000 7fb668d17700 1 --2- 172.21.15.2:0/2162891359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb664102910 0x7fb6640735e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb65c00ef30 tx=0x7fb65c00c3a0 comp rx=0 tx=0).stop 2023-09-13T04:35:14.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:35:14.087+0000 7fb668d17700 1 -- 172.21.15.2:0/2162891359 shutdown_connections 2023-09-13T04:35:14.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:35:14.087+0000 7fb668d17700 1 --2- 172.21.15.2:0/2162891359 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb64c0a0640 0x7fb64c0a2b00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:35:14.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:35:14.087+0000 7fb668d17700 1 --2- 172.21.15.2:0/2162891359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb664102910 0x7fb6640735e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:35:14.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:35:14.087+0000 7fb668d17700 1 --2- 172.21.15.2:0/2162891359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb664100d70 0x7fb664074a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:35:14.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:35:14.087+0000 7fb668d17700 1 --2- 172.21.15.2:0/2162891359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb664101f70 0x7fb6640730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:35:14.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:35:14.087+0000 7fb668d17700 1 -- 172.21.15.2:0/2162891359 >> 172.21.15.2:0/2162891359 conn(0x7fb6640fc2d0 msgr2=0x7fb664105580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:35:14.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:35:14.088+0000 7fb668d17700 1 -- 172.21.15.2:0/2162891359 shutdown_connections 2023-09-13T04:35:14.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:35:14.088+0000 7fb668d17700 1 -- 172.21.15.2:0/2162891359 wait complete. 2023-09-13T04:35:14.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:14 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:14.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:14 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:14.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:14 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:14.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:14 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:35:14.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:14 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:14.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:14 smithi156 ceph-mon[133675]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T04:35:14.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:14 smithi156 ceph-mon[133675]: from='osd.6 [v2:172.21.15.2:6818/3548993414,v1:172.21.15.2:6819/3548993414]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-13T04:35:14.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:14 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:14.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:14 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:14.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:14 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:35:14.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:14 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:14.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:14 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:14.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:14 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:14.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:14 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:35:14.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:14 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:14.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:14 smithi002 ceph-mon[97651]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T04:35:14.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:14 smithi002 ceph-mon[97651]: from='osd.6 [v2:172.21.15.2:6818/3548993414,v1:172.21.15.2:6819/3548993414]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-13T04:35:14.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:14 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:14.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:14 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:14.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:14 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:35:14.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:14.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:14.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:14.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:35:14.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:14.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:14 smithi149 ceph-mon[133315]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T04:35:14.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:14 smithi149 ceph-mon[133315]: from='osd.6 [v2:172.21.15.2:6818/3548993414,v1:172.21.15.2:6819/3548993414]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-13T04:35:14.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:14.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:14.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:35:14.800 DEBUG:teuthology.orchestra.run.smithi002:osd.6> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.6.service 2023-09-13T04:35:14.804 INFO:tasks.cephadm:Deploying osd.7 on smithi149 with /dev/vg_nvme/lv_2... 2023-09-13T04:35:14.805 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-13T04:35:14.879 INFO:journalctl@ceph.osd.6.smithi002.stdout:-- Logs begin at Wed 2023-09-13 04:13:33 UTC. -- 2023-09-13T04:35:15.379 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.b/config 2023-09-13T04:35:15.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:15 smithi156 ceph-mon[133675]: from='osd.6 [v2:172.21.15.2:6818/3548993414,v1:172.21.15.2:6819/3548993414]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-13T04:35:15.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:15 smithi156 ceph-mon[133675]: osdmap e44: 7 total, 6 up, 7 in 2023-09-13T04:35:15.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:15 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T04:35:15.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:15 smithi156 ceph-mon[133675]: from='osd.6 [v2:172.21.15.2:6818/3548993414,v1:172.21.15.2:6819/3548993414]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-09-13T04:35:15.443 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:15 smithi002 ceph-mon[97651]: from='osd.6 [v2:172.21.15.2:6818/3548993414,v1:172.21.15.2:6819/3548993414]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-13T04:35:15.444 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:15 smithi002 ceph-mon[97651]: osdmap e44: 7 total, 6 up, 7 in 2023-09-13T04:35:15.444 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:15 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T04:35:15.444 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:15 smithi002 ceph-mon[97651]: from='osd.6 [v2:172.21.15.2:6818/3548993414,v1:172.21.15.2:6819/3548993414]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-09-13T04:35:15.444 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:35:15 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:35:15.183+0000 7efcce4a9700 -1 osd.6 0 waiting for initial osdmap 2023-09-13T04:35:15.445 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:35:15 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:35:15.195+0000 7efcc6a97700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-13T04:35:15.569 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:15 smithi149 ceph-mon[133315]: from='osd.6 [v2:172.21.15.2:6818/3548993414,v1:172.21.15.2:6819/3548993414]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-13T04:35:15.569 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:15 smithi149 ceph-mon[133315]: osdmap e44: 7 total, 6 up, 7 in 2023-09-13T04:35:15.569 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:15 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T04:35:15.569 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:15 smithi149 ceph-mon[133315]: from='osd.6 [v2:172.21.15.2:6818/3548993414,v1:172.21.15.2:6819/3548993414]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-09-13T04:35:16.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:16 smithi002 ceph-mon[97651]: from='osd.6 [v2:172.21.15.2:6818/3548993414,v1:172.21.15.2:6819/3548993414]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2023-09-13T04:35:16.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:16 smithi002 ceph-mon[97651]: osdmap e45: 7 total, 6 up, 7 in 2023-09-13T04:35:16.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:16 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T04:35:16.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:16 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T04:35:16.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:16 smithi002 ceph-mon[97651]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T04:35:16.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:16 smithi156 ceph-mon[133675]: from='osd.6 [v2:172.21.15.2:6818/3548993414,v1:172.21.15.2:6819/3548993414]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2023-09-13T04:35:16.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:16 smithi156 ceph-mon[133675]: osdmap e45: 7 total, 6 up, 7 in 2023-09-13T04:35:16.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:16 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T04:35:16.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:16 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T04:35:16.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:16 smithi156 ceph-mon[133675]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T04:35:16.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:16 smithi149 ceph-mon[133315]: from='osd.6 [v2:172.21.15.2:6818/3548993414,v1:172.21.15.2:6819/3548993414]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2023-09-13T04:35:16.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:16 smithi149 ceph-mon[133315]: osdmap e45: 7 total, 6 up, 7 in 2023-09-13T04:35:16.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:16 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T04:35:16.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:16 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T04:35:16.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:16 smithi149 ceph-mon[133315]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T04:35:16.877 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-13T04:35:16.878 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-13T04:35:16.878 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2023-09-13T04:35:16.879 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2023-09-13T04:35:16.879 INFO:teuthology.orchestra.run.smithi149.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0147115 s, 713 MB/s 2023-09-13T04:35:16.880 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 2023-09-13T04:35:16.880 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2023-09-13T04:35:17.328 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph orch daemon add osd smithi149:vg_nvme/lv_2 2023-09-13T04:35:17.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:17 smithi002 ceph-mon[97651]: purged_snaps scrub starts 2023-09-13T04:35:17.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:17 smithi002 ceph-mon[97651]: purged_snaps scrub ok 2023-09-13T04:35:17.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:17 smithi002 ceph-mon[97651]: osd.6 [v2:172.21.15.2:6818/3548993414,v1:172.21.15.2:6819/3548993414] boot 2023-09-13T04:35:17.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:17 smithi002 ceph-mon[97651]: osdmap e46: 7 total, 7 up, 7 in 2023-09-13T04:35:17.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:17 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T04:35:17.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:17 smithi149 ceph-mon[133315]: purged_snaps scrub starts 2023-09-13T04:35:17.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:17 smithi149 ceph-mon[133315]: purged_snaps scrub ok 2023-09-13T04:35:17.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:17 smithi149 ceph-mon[133315]: osd.6 [v2:172.21.15.2:6818/3548993414,v1:172.21.15.2:6819/3548993414] boot 2023-09-13T04:35:17.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:17 smithi149 ceph-mon[133315]: osdmap e46: 7 total, 7 up, 7 in 2023-09-13T04:35:17.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:17 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T04:35:17.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:17 smithi156 ceph-mon[133675]: purged_snaps scrub starts 2023-09-13T04:35:17.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:17 smithi156 ceph-mon[133675]: purged_snaps scrub ok 2023-09-13T04:35:17.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:17 smithi156 ceph-mon[133675]: osd.6 [v2:172.21.15.2:6818/3548993414,v1:172.21.15.2:6819/3548993414] boot 2023-09-13T04:35:17.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:17 smithi156 ceph-mon[133675]: osdmap e46: 7 total, 7 up, 7 in 2023-09-13T04:35:17.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:17 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T04:35:17.883 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.b/config 2023-09-13T04:35:18.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:18 smithi149 ceph-mon[133315]: osdmap e47: 7 total, 7 up, 7 in 2023-09-13T04:35:18.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:18 smithi149 ceph-mon[133315]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:18.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:18 smithi002 ceph-mon[97651]: osdmap e47: 7 total, 7 up, 7 in 2023-09-13T04:35:18.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:18 smithi002 ceph-mon[97651]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:18.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:18 smithi156 ceph-mon[133675]: osdmap e47: 7 total, 7 up, 7 in 2023-09-13T04:35:18.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:18 smithi156 ceph-mon[133675]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:19.188 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.186+0000 7fe4a8f47700 1 -- 172.21.15.149:0/224918591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4a4101820 msgr2=0x7fe4a41080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:35:19.188 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.186+0000 7fe4a8f47700 1 --2- 172.21.15.149:0/224918591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4a4101820 0x7fe4a41080d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00b0d0 tx=0x7fe49c00b3e0 comp rx=0 tx=0).stop 2023-09-13T04:35:19.188 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.186+0000 7fe4a8f47700 1 -- 172.21.15.149:0/224918591 shutdown_connections 2023-09-13T04:35:19.189 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.186+0000 7fe4a8f47700 1 --2- 172.21.15.149:0/224918591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4a4101820 0x7fe4a41080d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:35:19.189 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.186+0000 7fe4a8f47700 1 --2- 172.21.15.149:0/224918591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4a40ffc80 0x7fe4a41000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:35:19.189 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.186+0000 7fe4a8f47700 1 --2- 172.21.15.149:0/224918591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4a4100e80 0x7fe4a41012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:35:19.189 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.186+0000 7fe4a8f47700 1 -- 172.21.15.149:0/224918591 >> 172.21.15.149:0/224918591 conn(0x7fe4a40fb220 msgr2=0x7fe4a40fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:35:19.189 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.186+0000 7fe4a8f47700 1 -- 172.21.15.149:0/224918591 shutdown_connections 2023-09-13T04:35:19.190 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.186+0000 7fe4a8f47700 1 -- 172.21.15.149:0/224918591 wait complete. 2023-09-13T04:35:19.190 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.187+0000 7fe4a8f47700 1 Processor -- start 2023-09-13T04:35:19.190 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.187+0000 7fe4a8f47700 1 -- start start 2023-09-13T04:35:19.190 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.187+0000 7fe4a8f47700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4a40ffc80 0x7fe4a4195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:35:19.190 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.187+0000 7fe4a8f47700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4a4100e80 0x7fe4a4195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:35:19.191 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.187+0000 7fe4a8f47700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4a4101820 0x7fe4a419d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:35:19.191 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.187+0000 7fe4a8f47700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a419d600 con 0x7fe4a4100e80 2023-09-13T04:35:19.191 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.187+0000 7fe4a8f47700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a419d740 con 0x7fe4a40ffc80 2023-09-13T04:35:19.191 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.187+0000 7fe4a8f47700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a419d8d0 con 0x7fe4a4101820 2023-09-13T04:35:19.191 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.187+0000 7fe4a259c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4a40ffc80 0x7fe4a4195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:35:19.192 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.187+0000 7fe4a259c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4a40ffc80 0x7fe4a4195a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:57634/0 (socket says 172.21.15.149:57634) 2023-09-13T04:35:19.192 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.187+0000 7fe4a259c700 1 -- 172.21.15.149:0/3465135683 learned_addr learned my addr 172.21.15.149:0/3465135683 (peer_addr_for_me v2:172.21.15.149:0/0) 2023-09-13T04:35:19.192 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.187+0000 7fe4a1d9b700 1 --2- 172.21.15.149:0/3465135683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4a4100e80 0x7fe4a4195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:35:19.192 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.187+0000 7fe4a259c700 1 -- 172.21.15.149:0/3465135683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4a4101820 msgr2=0x7fe4a419d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:35:19.193 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.187+0000 7fe4a259c700 1 --2- 172.21.15.149:0/3465135683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4a4101820 0x7fe4a419d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:35:19.193 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.187+0000 7fe4a259c700 1 -- 172.21.15.149:0/3465135683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4a4100e80 msgr2=0x7fe4a4195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:35:19.193 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.187+0000 7fe4a259c700 1 --2- 172.21.15.149:0/3465135683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4a4100e80 0x7fe4a4195fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:35:19.193 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.187+0000 7fe4a259c700 1 -- 172.21.15.149:0/3465135683 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c009d00 con 0x7fe4a40ffc80 2023-09-13T04:35:19.193 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.188+0000 7fe4a259c700 1 --2- 172.21.15.149:0/3465135683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4a40ffc80 0x7fe4a4195a70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe49400d980 tx=0x7fe49400dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:35:19.194 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.188+0000 7fe4937fe700 1 -- 172.21.15.149:0/3465135683 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49400de60 con 0x7fe4a40ffc80 2023-09-13T04:35:19.194 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.188+0000 7fe4937fe700 1 -- 172.21.15.149:0/3465135683 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4940044e0 con 0x7fe4a40ffc80 2023-09-13T04:35:19.194 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.188+0000 7fe4937fe700 1 -- 172.21.15.149:0/3465135683 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe494003bc0 con 0x7fe4a40ffc80 2023-09-13T04:35:19.194 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.188+0000 7fe4a8f47700 1 -- 172.21.15.149:0/3465135683 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a419dbb0 con 0x7fe4a40ffc80 2023-09-13T04:35:19.195 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.188+0000 7fe4a8f47700 1 -- 172.21.15.149:0/3465135683 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a419e060 con 0x7fe4a40ffc80 2023-09-13T04:35:19.196 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.189+0000 7fe4a8f47700 1 -- 172.21.15.149:0/3465135683 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a410b500 con 0x7fe4a40ffc80 2023-09-13T04:35:19.196 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.190+0000 7fe4937fe700 1 -- 172.21.15.149:0/3465135683 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fe494009bb0 con 0x7fe4a40ffc80 2023-09-13T04:35:19.196 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.190+0000 7fe4937fe700 1 --2- 172.21.15.149:0/3465135683 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe48c0a0640 0x7fe48c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:35:19.196 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.190+0000 7fe4937fe700 1 -- 172.21.15.149:0/3465135683 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fe4940cabe0 con 0x7fe4a40ffc80 2023-09-13T04:35:19.197 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.191+0000 7fe4a1d9b700 1 --2- 172.21.15.149:0/3465135683 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe48c0a0640 0x7fe48c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:35:19.197 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.191+0000 7fe4a1d9b700 1 --2- 172.21.15.149:0/3465135683 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe48c0a0640 0x7fe48c0a2b00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe498006fd0 tx=0x7fe498008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:35:19.197 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.195+0000 7fe4937fe700 1 -- 172.21.15.149:0/3465135683 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe494095420 con 0x7fe4a40ffc80 2023-09-13T04:35:19.391 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:19.391+0000 7fe4a8f47700 1 -- 172.21.15.149:0/3465135683 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4a40610f0 con 0x7fe48c0a0640 2023-09-13T04:35:19.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:19 smithi149 ceph-mon[133315]: Detected new or changed devices on smithi002 2023-09-13T04:35:19.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:19 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:19.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:19 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:19.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:19 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:35:19.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:19 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:35:19.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:19 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:35:19.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:19 smithi149 ceph-mon[133315]: Adjusting osd_memory_target on smithi002 to 5757M 2023-09-13T04:35:19.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:19 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:19.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:19 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:19.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:19 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:35:19.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:19 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:19.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:19 smithi156 ceph-mon[133675]: Detected new or changed devices on smithi002 2023-09-13T04:35:19.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:19.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:19.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:35:19.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:35:19.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:35:19.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:19 smithi156 ceph-mon[133675]: Adjusting osd_memory_target on smithi002 to 5757M 2023-09-13T04:35:19.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:19.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:19.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:35:19.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:19 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:19.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:19 smithi002 ceph-mon[97651]: Detected new or changed devices on smithi002 2023-09-13T04:35:19.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:19.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:19.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:35:19.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:35:19.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:35:19.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:19 smithi002 ceph-mon[97651]: Adjusting osd_memory_target on smithi002 to 5757M 2023-09-13T04:35:19.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:19.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:19.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:35:19.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:19 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:20.562 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:20 smithi149 ceph-mon[133315]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:35:20.562 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:20 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:35:20.562 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:20 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:35:20.562 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:20 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:20.563 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:20 smithi149 ceph-mon[133315]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:20.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:20 smithi156 ceph-mon[133675]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:35:20.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:20 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:35:20.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:20 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:35:20.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:20 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:20.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:20 smithi156 ceph-mon[133675]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:20.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:20 smithi002 ceph-mon[97651]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:35:20.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:20 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:35:20.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:20 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:35:20.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:20 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:20.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:20 smithi002 ceph-mon[97651]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:22.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:22 smithi149 ceph-mon[133315]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:22.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:22 smithi149 ceph-mon[133315]: from='client.? 172.21.15.149:0/1946808851' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d82c256-1078-4815-83d9-6fcdb98746ff"}]: dispatch 2023-09-13T04:35:22.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:22 smithi149 ceph-mon[133315]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d82c256-1078-4815-83d9-6fcdb98746ff"}]: dispatch 2023-09-13T04:35:22.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:22 smithi149 ceph-mon[133315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d82c256-1078-4815-83d9-6fcdb98746ff"}]': finished 2023-09-13T04:35:22.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:22 smithi149 ceph-mon[133315]: osdmap e48: 8 total, 7 up, 8 in 2023-09-13T04:35:22.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:22 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T04:35:22.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:22 smithi156 ceph-mon[133675]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:22.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:22 smithi156 ceph-mon[133675]: from='client.? 172.21.15.149:0/1946808851' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d82c256-1078-4815-83d9-6fcdb98746ff"}]: dispatch 2023-09-13T04:35:22.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:22 smithi156 ceph-mon[133675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d82c256-1078-4815-83d9-6fcdb98746ff"}]: dispatch 2023-09-13T04:35:22.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:22 smithi156 ceph-mon[133675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d82c256-1078-4815-83d9-6fcdb98746ff"}]': finished 2023-09-13T04:35:22.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:22 smithi156 ceph-mon[133675]: osdmap e48: 8 total, 7 up, 8 in 2023-09-13T04:35:22.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:22 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T04:35:23.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:22 smithi002 ceph-mon[97651]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:23.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:22 smithi002 ceph-mon[97651]: from='client.? 172.21.15.149:0/1946808851' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d82c256-1078-4815-83d9-6fcdb98746ff"}]: dispatch 2023-09-13T04:35:23.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:22 smithi002 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d82c256-1078-4815-83d9-6fcdb98746ff"}]: dispatch 2023-09-13T04:35:23.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:22 smithi002 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d82c256-1078-4815-83d9-6fcdb98746ff"}]': finished 2023-09-13T04:35:23.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:22 smithi002 ceph-mon[97651]: osdmap e48: 8 total, 7 up, 8 in 2023-09-13T04:35:23.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:22 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T04:35:23.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:23 smithi149 ceph-mon[133315]: from='client.? 172.21.15.149:0/2126071068' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:35:23.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:23 smithi156 ceph-mon[133675]: from='client.? 172.21.15.149:0/2126071068' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:35:24.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:23 smithi002 ceph-mon[97651]: from='client.? 172.21.15.149:0/2126071068' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:35:24.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:24 smithi149 ceph-mon[133315]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:24.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:24 smithi156 ceph-mon[133675]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:25.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:24 smithi002 ceph-mon[97651]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:26.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:26 smithi149 ceph-mon[133315]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:26.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:26 smithi156 ceph-mon[133675]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:27.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:26 smithi002 ceph-mon[97651]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:28.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:28 smithi149 ceph-mon[133315]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:28.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:28 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-13T04:35:28.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:28 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:28.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:28 smithi156 ceph-mon[133675]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:28.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:28 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-13T04:35:28.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:28 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:29.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:28 smithi002 ceph-mon[97651]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:29.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:28 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-13T04:35:29.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:28 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:29.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:29 smithi149 ceph-mon[133315]: Deploying daemon osd.7 on smithi149 2023-09-13T04:35:29.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:29 smithi156 ceph-mon[133675]: Deploying daemon osd.7 on smithi149 2023-09-13T04:35:30.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:29 smithi002 ceph-mon[97651]: Deploying daemon osd.7 on smithi149 2023-09-13T04:35:30.867 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:30 smithi149 ceph-mon[133315]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:30.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:30 smithi156 ceph-mon[133675]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:31.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:30 smithi002 ceph-mon[97651]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:32.736 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:32 smithi149 ceph-mon[133315]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:32.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:32 smithi156 ceph-mon[133675]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:33.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:32 smithi002 ceph-mon[97651]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:33.967 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:33 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:33.968 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:33 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:33.968 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:33 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:35:33.968 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:33 smithi149 ceph-mon[133315]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:34.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:33 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:34.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:33 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:34.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:33 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:35:34.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:33 smithi156 ceph-mon[133675]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:34.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:33 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:34.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:33 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:34.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:33 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:35:34.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:33 smithi002 ceph-mon[97651]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:35.611 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:35.610+0000 7fe4937fe700 1 -- 172.21.15.149:0/3465135683 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe4a40610f0 con 0x7fe48c0a0640 2023-09-13T04:35:35.611 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 7 on host 'smithi149' 2023-09-13T04:35:35.613 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:35.612+0000 7fe4a8f47700 1 -- 172.21.15.149:0/3465135683 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe48c0a0640 msgr2=0x7fe48c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:35:35.613 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:35.612+0000 7fe4a8f47700 1 --2- 172.21.15.149:0/3465135683 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe48c0a0640 0x7fe48c0a2b00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe498006fd0 tx=0x7fe498008040 comp rx=0 tx=0).stop 2023-09-13T04:35:35.614 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:35.612+0000 7fe4a8f47700 1 -- 172.21.15.149:0/3465135683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4a40ffc80 msgr2=0x7fe4a4195a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:35:35.614 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:35.612+0000 7fe4a8f47700 1 --2- 172.21.15.149:0/3465135683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4a40ffc80 0x7fe4a4195a70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe49400d980 tx=0x7fe49400dd40 comp rx=0 tx=0).stop 2023-09-13T04:35:35.614 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:35.612+0000 7fe4a8f47700 1 -- 172.21.15.149:0/3465135683 shutdown_connections 2023-09-13T04:35:35.614 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:35.612+0000 7fe4a8f47700 1 --2- 172.21.15.149:0/3465135683 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe48c0a0640 0x7fe48c0a2b00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:35:35.614 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:35.612+0000 7fe4a8f47700 1 --2- 172.21.15.149:0/3465135683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4a4101820 0x7fe4a419d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:35:35.615 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:35.612+0000 7fe4a8f47700 1 --2- 172.21.15.149:0/3465135683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4a40ffc80 0x7fe4a4195a70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:35:35.615 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:35.612+0000 7fe4a8f47700 1 --2- 172.21.15.149:0/3465135683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4a4100e80 0x7fe4a4195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:35:35.615 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:35.612+0000 7fe4a8f47700 1 -- 172.21.15.149:0/3465135683 >> 172.21.15.149:0/3465135683 conn(0x7fe4a40fb220 msgr2=0x7fe4a40fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:35:35.615 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:35.612+0000 7fe4a8f47700 1 -- 172.21.15.149:0/3465135683 shutdown_connections 2023-09-13T04:35:35.615 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:35:35.612+0000 7fe4a8f47700 1 -- 172.21.15.149:0/3465135683 wait complete. 2023-09-13T04:35:36.066 DEBUG:teuthology.orchestra.run.smithi149:osd.7> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.7.service 2023-09-13T04:35:36.068 INFO:tasks.cephadm:Deploying osd.8 on smithi156 with /dev/vg_nvme/lv_2... 2023-09-13T04:35:36.069 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-13T04:35:36.099 INFO:journalctl@ceph.osd.7.smithi149.stdout:-- Logs begin at Wed 2023-09-13 04:09:45 UTC. -- 2023-09-13T04:35:36.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:36 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:36.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:36 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:36.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:36 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:36.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:36 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:35:36.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:36 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:36.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:36 smithi149 ceph-mon[133315]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:36.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:36 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:36.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:36 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:36.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:36 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:35:36.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:36 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:36.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:36 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:36.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:36 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:36.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:36 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:35:36.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:36 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:36.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:36 smithi156 ceph-mon[133675]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:36.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:36 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:36.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:36 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:36.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:36 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:35:36.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:36 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:36.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:36 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:36.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:36 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:36.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:36 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:35:36.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:36 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:36.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:36 smithi002 ceph-mon[97651]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:36.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:36 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:36.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:36 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:36.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:36 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:35:36.636 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-13T04:35:36.834 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:35:36 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:35:36.709+0000 7f5327930640 -1 osd.7 0 log_to_monitors true 2023-09-13T04:35:37.441 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:37 smithi156 ceph-mon[133675]: from='osd.7 [v2:172.21.15.149:6816/3932319846,v1:172.21.15.149:6817/3932319846]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-13T04:35:37.442 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:37 smithi156 ceph-mon[133675]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-13T04:35:37.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:37 smithi002 ceph-mon[97651]: from='osd.7 [v2:172.21.15.149:6816/3932319846,v1:172.21.15.149:6817/3932319846]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-13T04:35:37.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:37 smithi002 ceph-mon[97651]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-13T04:35:37.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:37 smithi149 ceph-mon[133315]: from='osd.7 [v2:172.21.15.149:6816/3932319846,v1:172.21.15.149:6817/3932319846]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-13T04:35:37.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:37 smithi149 ceph-mon[133315]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-13T04:35:38.222 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-13T04:35:38.222 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-13T04:35:38.222 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2023-09-13T04:35:38.222 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2023-09-13T04:35:38.222 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00831921 s, 1.3 GB/s 2023-09-13T04:35:38.223 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2023-09-13T04:35:38.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:38 smithi002 ceph-mon[97651]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-13T04:35:38.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:38 smithi002 ceph-mon[97651]: osdmap e49: 8 total, 7 up, 8 in 2023-09-13T04:35:38.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:38 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T04:35:38.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:38 smithi002 ceph-mon[97651]: from='osd.7 [v2:172.21.15.149:6816/3932319846,v1:172.21.15.149:6817/3932319846]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-09-13T04:35:38.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:38 smithi002 ceph-mon[97651]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-09-13T04:35:38.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:38 smithi002 ceph-mon[97651]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:38.548 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:35:38 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:35:38.254+0000 7f531cfa7700 -1 osd.7 0 waiting for initial osdmap 2023-09-13T04:35:38.548 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:35:38 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:35:38.265+0000 7f5316d98700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-13T04:35:38.548 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:38 smithi149 ceph-mon[133315]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-13T04:35:38.548 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:38 smithi149 ceph-mon[133315]: osdmap e49: 8 total, 7 up, 8 in 2023-09-13T04:35:38.549 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:38 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T04:35:38.549 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:38 smithi149 ceph-mon[133315]: from='osd.7 [v2:172.21.15.149:6816/3932319846,v1:172.21.15.149:6817/3932319846]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-09-13T04:35:38.549 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:38 smithi149 ceph-mon[133315]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-09-13T04:35:38.549 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:38 smithi149 ceph-mon[133315]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:38.577 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:38 smithi156 ceph-mon[133675]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-13T04:35:38.577 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:38 smithi156 ceph-mon[133675]: osdmap e49: 8 total, 7 up, 8 in 2023-09-13T04:35:38.578 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:38 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T04:35:38.578 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:38 smithi156 ceph-mon[133675]: from='osd.7 [v2:172.21.15.149:6816/3932319846,v1:172.21.15.149:6817/3932319846]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-09-13T04:35:38.578 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:38 smithi156 ceph-mon[133675]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-09-13T04:35:38.578 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:38 smithi156 ceph-mon[133675]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T04:35:38.579 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph orch daemon add osd smithi156:vg_nvme/lv_2 2023-09-13T04:35:39.142 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-13T04:35:39.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:39 smithi002 ceph-mon[97651]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2023-09-13T04:35:39.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:39 smithi002 ceph-mon[97651]: osdmap e50: 8 total, 7 up, 8 in 2023-09-13T04:35:39.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:39 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T04:35:39.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:39 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T04:35:39.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:39 smithi002 ceph-mon[97651]: Detected new or changed devices on smithi149 2023-09-13T04:35:39.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:39 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:39.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:39 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:39.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:39 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:35:39.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:39 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:35:39.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:39 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:35:39.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:39 smithi002 ceph-mon[97651]: Adjusting osd_memory_target on smithi149 to 1959M 2023-09-13T04:35:39.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:39 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:39.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:39 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:39.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:39 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:35:39.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:39 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:39.509 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:39 smithi002 ceph-mon[97651]: osd.7 [v2:172.21.15.149:6816/3932319846,v1:172.21.15.149:6817/3932319846] boot 2023-09-13T04:35:39.509 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:39 smithi002 ceph-mon[97651]: osdmap e51: 8 total, 8 up, 8 in 2023-09-13T04:35:39.509 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:39 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T04:35:39.552 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:39 smithi156 ceph-mon[133675]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2023-09-13T04:35:39.552 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:39 smithi156 ceph-mon[133675]: osdmap e50: 8 total, 7 up, 8 in 2023-09-13T04:35:39.553 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:39 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T04:35:39.553 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:39 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T04:35:39.553 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:39 smithi156 ceph-mon[133675]: Detected new or changed devices on smithi149 2023-09-13T04:35:39.553 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:39 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:39.553 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:39 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:39.554 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:39 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:35:39.554 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:39 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:35:39.554 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:39 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:35:39.554 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:39 smithi156 ceph-mon[133675]: Adjusting osd_memory_target on smithi149 to 1959M 2023-09-13T04:35:39.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:39 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:39.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:39 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:39.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:39 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:35:39.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:39 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:39.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:39 smithi156 ceph-mon[133675]: osd.7 [v2:172.21.15.149:6816/3932319846,v1:172.21.15.149:6817/3932319846] boot 2023-09-13T04:35:39.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:39 smithi156 ceph-mon[133675]: osdmap e51: 8 total, 8 up, 8 in 2023-09-13T04:35:39.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:39 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T04:35:39.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:39 smithi149 ceph-mon[133315]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2023-09-13T04:35:39.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:39 smithi149 ceph-mon[133315]: osdmap e50: 8 total, 7 up, 8 in 2023-09-13T04:35:39.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:39 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T04:35:39.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:39 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T04:35:39.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:39 smithi149 ceph-mon[133315]: Detected new or changed devices on smithi149 2023-09-13T04:35:39.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:39 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:39.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:39 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:39.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:39 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:35:39.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:39 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:35:39.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:39 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:35:39.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:39 smithi149 ceph-mon[133315]: Adjusting osd_memory_target on smithi149 to 1959M 2023-09-13T04:35:39.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:39 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:39.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:39 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:39.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:39 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:35:39.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:39 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:39.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:39 smithi149 ceph-mon[133315]: osd.7 [v2:172.21.15.149:6816/3932319846,v1:172.21.15.149:6817/3932319846] boot 2023-09-13T04:35:39.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:39 smithi149 ceph-mon[133315]: osdmap e51: 8 total, 8 up, 8 in 2023-09-13T04:35:39.641 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:39 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T04:35:40.356 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.354+0000 7f43e564b700 1 -- 172.21.15.156:0/35187671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43e0104190 msgr2=0x7f43e0106580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:35:40.356 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.354+0000 7f43e564b700 1 --2- 172.21.15.156:0/35187671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43e0104190 0x7f43e0106580 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009990 tx=0x7f43d0009ca0 comp rx=0 tx=0).stop 2023-09-13T04:35:40.356 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.354+0000 7f43e564b700 1 -- 172.21.15.156:0/35187671 shutdown_connections 2023-09-13T04:35:40.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.354+0000 7f43e564b700 1 --2- 172.21.15.156:0/35187671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43e0106ac0 0x7f43e0108ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:35:40.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.354+0000 7f43e564b700 1 --2- 172.21.15.156:0/35187671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43e0101860 0x7f43e0103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:35:40.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.354+0000 7f43e564b700 1 --2- 172.21.15.156:0/35187671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43e0104190 0x7f43e0106580 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:35:40.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.354+0000 7f43e564b700 1 -- 172.21.15.156:0/35187671 >> 172.21.15.156:0/35187671 conn(0x7f43e00fb1f0 msgr2=0x7f43e00fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:35:40.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.354+0000 7f43e564b700 1 -- 172.21.15.156:0/35187671 shutdown_connections 2023-09-13T04:35:40.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.354+0000 7f43e564b700 1 -- 172.21.15.156:0/35187671 wait complete. 2023-09-13T04:35:40.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.355+0000 7f43e564b700 1 Processor -- start 2023-09-13T04:35:40.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.355+0000 7f43e564b700 1 -- start start 2023-09-13T04:35:40.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.355+0000 7f43e564b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43e0101860 0x7f43e01969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:35:40.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.355+0000 7f43e564b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43e0104190 0x7f43e0196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:35:40.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.355+0000 7f43e564b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43e0106ac0 0x7f43e019df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:35:40.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.355+0000 7f43e564b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e019e560 con 0x7f43e0101860 2023-09-13T04:35:40.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.355+0000 7f43e564b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e019e6a0 con 0x7f43e0104190 2023-09-13T04:35:40.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.355+0000 7f43e564b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e019e830 con 0x7f43e0106ac0 2023-09-13T04:35:40.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.356+0000 7f43df7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43e0106ac0 0x7f43e019df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:35:40.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.356+0000 7f43df7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43e0106ac0 0x7f43e019df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37280/0 (socket says 172.21.15.156:37280) 2023-09-13T04:35:40.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.356+0000 7f43df7fe700 1 -- 172.21.15.156:0/4090469560 learned_addr learned my addr 172.21.15.156:0/4090469560 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-13T04:35:40.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.356+0000 7f43de7fc700 1 --2- 172.21.15.156:0/4090469560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43e0104190 0x7f43e0196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:35:40.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.356+0000 7f43df7fe700 1 -- 172.21.15.156:0/4090469560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43e0104190 msgr2=0x7f43e0196f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:35:40.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.356+0000 7f43df7fe700 1 --2- 172.21.15.156:0/4090469560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43e0104190 0x7f43e0196f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:35:40.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.356+0000 7f43df7fe700 1 -- 172.21.15.156:0/4090469560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43e0101860 msgr2=0x7f43e01969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:35:40.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.356+0000 7f43df7fe700 1 --2- 172.21.15.156:0/4090469560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43e0101860 0x7f43e01969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:35:40.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.356+0000 7f43df7fe700 1 -- 172.21.15.156:0/4090469560 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d00095f0 con 0x7f43e0106ac0 2023-09-13T04:35:40.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.356+0000 7f43df7fe700 1 --2- 172.21.15.156:0/4090469560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43e0106ac0 0x7f43e019df90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f43d800add0 tx=0x7f43d800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:35:40.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.356+0000 7f43c7fff700 1 -- 172.21.15.156:0/4090469560 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d800f070 con 0x7f43e0106ac0 2023-09-13T04:35:40.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.356+0000 7f43c7fff700 1 -- 172.21.15.156:0/4090469560 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43d8007b00 con 0x7f43e0106ac0 2023-09-13T04:35:40.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.356+0000 7f43e564b700 1 -- 172.21.15.156:0/4090469560 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e019eb10 con 0x7f43e0106ac0 2023-09-13T04:35:40.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.356+0000 7f43c7fff700 1 -- 172.21.15.156:0/4090469560 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d80053a0 con 0x7f43e0106ac0 2023-09-13T04:35:40.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.356+0000 7f43e564b700 1 -- 172.21.15.156:0/4090469560 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e019efc0 con 0x7f43e0106ac0 2023-09-13T04:35:40.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.357+0000 7f43e564b700 1 -- 172.21.15.156:0/4090469560 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e00fcd70 con 0x7f43e0106ac0 2023-09-13T04:35:40.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.359+0000 7f43c7fff700 1 -- 172.21.15.156:0/4090469560 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f43d800c940 con 0x7f43e0106ac0 2023-09-13T04:35:40.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.359+0000 7f43c7fff700 1 --2- 172.21.15.156:0/4090469560 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f43c80a0640 0x7f43c80a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:35:40.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.359+0000 7f43c7fff700 1 -- 172.21.15.156:0/4090469560 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f43d80cac60 con 0x7f43e0106ac0 2023-09-13T04:35:40.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.360+0000 7f43deffd700 1 --2- 172.21.15.156:0/4090469560 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f43c80a0640 0x7f43c80a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:35:40.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.360+0000 7f43deffd700 1 --2- 172.21.15.156:0/4090469560 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f43c80a0640 0x7f43c80a2b00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f43d4005fd0 tx=0x7f43d4005d00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:35:40.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.362+0000 7f43c7fff700 1 -- 172.21.15.156:0/4090469560 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f43d8095360 con 0x7f43e0106ac0 2023-09-13T04:35:40.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:40 smithi156 ceph-mon[133675]: purged_snaps scrub starts 2023-09-13T04:35:40.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:40 smithi156 ceph-mon[133675]: purged_snaps scrub ok 2023-09-13T04:35:40.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:40 smithi156 ceph-mon[133675]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:40.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:40 smithi156 ceph-mon[133675]: osdmap e52: 8 total, 8 up, 8 in 2023-09-13T04:35:40.560 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:40.559+0000 7f43e564b700 1 -- 172.21.15.156:0/4090469560 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f43e002d030 con 0x7f43c80a0640 2023-09-13T04:35:40.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:40 smithi149 ceph-mon[133315]: purged_snaps scrub starts 2023-09-13T04:35:40.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:40 smithi149 ceph-mon[133315]: purged_snaps scrub ok 2023-09-13T04:35:40.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:40 smithi149 ceph-mon[133315]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:40.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:40 smithi149 ceph-mon[133315]: osdmap e52: 8 total, 8 up, 8 in 2023-09-13T04:35:40.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:40 smithi002 ceph-mon[97651]: purged_snaps scrub starts 2023-09-13T04:35:40.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:40 smithi002 ceph-mon[97651]: purged_snaps scrub ok 2023-09-13T04:35:40.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:40 smithi002 ceph-mon[97651]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:40.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:40 smithi002 ceph-mon[97651]: osdmap e52: 8 total, 8 up, 8 in 2023-09-13T04:35:41.568 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:41 smithi156 ceph-mon[133675]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:35:41.569 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:41 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:35:41.569 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:41 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:35:41.569 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:41 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:41.569 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:41 smithi156 ceph-mon[133675]: osdmap e53: 8 total, 8 up, 8 in 2023-09-13T04:35:41.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:41 smithi149 ceph-mon[133315]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:35:41.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:41 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:35:41.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:41 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:35:41.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:41 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:41.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:41 smithi149 ceph-mon[133315]: osdmap e53: 8 total, 8 up, 8 in 2023-09-13T04:35:41.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:41 smithi002 ceph-mon[97651]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:35:41.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:41 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:35:41.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:41 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:35:41.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:41 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:41.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:41 smithi002 ceph-mon[97651]: osdmap e53: 8 total, 8 up, 8 in 2023-09-13T04:35:42.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:42 smithi156 ceph-mon[133675]: pgmap v172: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:42.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:42 smithi149 ceph-mon[133315]: pgmap v172: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:42.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:42 smithi002 ceph-mon[97651]: pgmap v172: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:43.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:43 smithi149 ceph-mon[133315]: osdmap e54: 8 total, 8 up, 8 in 2023-09-13T04:35:43.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:43 smithi149 ceph-mon[133315]: from='client.? 172.21.15.156:0/2224281811' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbb7a301-2a19-4239-a27d-d07410dd84c7"}]: dispatch 2023-09-13T04:35:43.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:43 smithi149 ceph-mon[133315]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbb7a301-2a19-4239-a27d-d07410dd84c7"}]: dispatch 2023-09-13T04:35:43.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:43 smithi149 ceph-mon[133315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fbb7a301-2a19-4239-a27d-d07410dd84c7"}]': finished 2023-09-13T04:35:43.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:43 smithi149 ceph-mon[133315]: osdmap e55: 9 total, 8 up, 9 in 2023-09-13T04:35:43.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:43 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T04:35:43.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:43 smithi156 ceph-mon[133675]: osdmap e54: 8 total, 8 up, 8 in 2023-09-13T04:35:43.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:43 smithi156 ceph-mon[133675]: from='client.? 172.21.15.156:0/2224281811' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbb7a301-2a19-4239-a27d-d07410dd84c7"}]: dispatch 2023-09-13T04:35:43.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:43 smithi156 ceph-mon[133675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbb7a301-2a19-4239-a27d-d07410dd84c7"}]: dispatch 2023-09-13T04:35:43.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:43 smithi156 ceph-mon[133675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fbb7a301-2a19-4239-a27d-d07410dd84c7"}]': finished 2023-09-13T04:35:43.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:43 smithi156 ceph-mon[133675]: osdmap e55: 9 total, 8 up, 9 in 2023-09-13T04:35:43.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:43 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T04:35:43.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:43 smithi002 ceph-mon[97651]: osdmap e54: 8 total, 8 up, 8 in 2023-09-13T04:35:43.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:43 smithi002 ceph-mon[97651]: from='client.? 172.21.15.156:0/2224281811' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbb7a301-2a19-4239-a27d-d07410dd84c7"}]: dispatch 2023-09-13T04:35:43.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:43 smithi002 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbb7a301-2a19-4239-a27d-d07410dd84c7"}]: dispatch 2023-09-13T04:35:43.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:43 smithi002 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fbb7a301-2a19-4239-a27d-d07410dd84c7"}]': finished 2023-09-13T04:35:43.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:43 smithi002 ceph-mon[97651]: osdmap e55: 9 total, 8 up, 9 in 2023-09-13T04:35:43.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:43 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T04:35:44.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:44 smithi149 ceph-mon[133315]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:44.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:44 smithi149 ceph-mon[133315]: from='client.? 172.21.15.156:0/1726618057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:35:44.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:44 smithi156 ceph-mon[133675]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:44.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:44 smithi156 ceph-mon[133675]: from='client.? 172.21.15.156:0/1726618057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:35:44.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:44 smithi002 ceph-mon[97651]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:44.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:44 smithi002 ceph-mon[97651]: from='client.? 172.21.15.156:0/1726618057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:35:46.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:46 smithi149 ceph-mon[133315]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:46.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:46 smithi156 ceph-mon[133675]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:47.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:46 smithi002 ceph-mon[97651]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:48.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:48 smithi149 ceph-mon[133315]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:48.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:48 smithi156 ceph-mon[133675]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:49.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:48 smithi002 ceph-mon[97651]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:50.272 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:50 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-13T04:35:50.272 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:50 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:50.273 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:50 smithi156 ceph-mon[133675]: Deploying daemon osd.8 on smithi156 2023-09-13T04:35:50.273 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:50 smithi156 ceph-mon[133675]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:50.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:50 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-13T04:35:50.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:50 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:50.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:50 smithi002 ceph-mon[97651]: Deploying daemon osd.8 on smithi156 2023-09-13T04:35:50.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:50 smithi002 ceph-mon[97651]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:50.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:50 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-13T04:35:50.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:50 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:50.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:50 smithi149 ceph-mon[133315]: Deploying daemon osd.8 on smithi156 2023-09-13T04:35:50.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:50 smithi149 ceph-mon[133315]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:52.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:52 smithi156 ceph-mon[133675]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:52.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:52 smithi149 ceph-mon[133315]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:53.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:52 smithi002 ceph-mon[97651]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:54.817 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:54 smithi156 ceph-mon[133675]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:54.817 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:54 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:54.817 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:54 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:35:54.817 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:54 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:54.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:54 smithi149 ceph-mon[133315]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:54.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:54 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:54.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:54 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:35:54.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:54 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:55.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:54 smithi002 ceph-mon[97651]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:55.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:54 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:55.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:54 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:35:55.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:54 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:56.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:56 smithi156 ceph-mon[133675]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:56.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:56 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:56.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:56 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:56.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:56 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:56.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:56 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:35:56.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:56 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:56.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:56.725+0000 7f43c7fff700 1 -- 172.21.15.156:0/4090469560 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f43e002d030 con 0x7f43c80a0640 2023-09-13T04:35:56.727 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 8 on host 'smithi156' 2023-09-13T04:35:56.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:56.728+0000 7f43e564b700 1 -- 172.21.15.156:0/4090469560 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f43c80a0640 msgr2=0x7f43c80a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:35:56.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:56.728+0000 7f43e564b700 1 --2- 172.21.15.156:0/4090469560 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f43c80a0640 0x7f43c80a2b00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f43d4005fd0 tx=0x7f43d4005d00 comp rx=0 tx=0).stop 2023-09-13T04:35:56.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:56.728+0000 7f43e564b700 1 -- 172.21.15.156:0/4090469560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43e0106ac0 msgr2=0x7f43e019df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:35:56.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:56.728+0000 7f43e564b700 1 --2- 172.21.15.156:0/4090469560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43e0106ac0 0x7f43e019df90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f43d800add0 tx=0x7f43d800c330 comp rx=0 tx=0).stop 2023-09-13T04:35:56.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:56.728+0000 7f43e564b700 1 -- 172.21.15.156:0/4090469560 shutdown_connections 2023-09-13T04:35:56.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:56.728+0000 7f43e564b700 1 --2- 172.21.15.156:0/4090469560 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f43c80a0640 0x7f43c80a2b00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:35:56.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:56.728+0000 7f43e564b700 1 --2- 172.21.15.156:0/4090469560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43e0106ac0 0x7f43e019df90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:35:56.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:56.728+0000 7f43e564b700 1 --2- 172.21.15.156:0/4090469560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43e0101860 0x7f43e01969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:35:56.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:56.728+0000 7f43e564b700 1 --2- 172.21.15.156:0/4090469560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43e0104190 0x7f43e0196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:35:56.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:56.728+0000 7f43e564b700 1 -- 172.21.15.156:0/4090469560 >> 172.21.15.156:0/4090469560 conn(0x7f43e00fb1f0 msgr2=0x7f43e01024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:35:56.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:56.728+0000 7f43e564b700 1 -- 172.21.15.156:0/4090469560 shutdown_connections 2023-09-13T04:35:56.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:35:56.728+0000 7f43e564b700 1 -- 172.21.15.156:0/4090469560 wait complete. 2023-09-13T04:35:56.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:56 smithi149 ceph-mon[133315]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:56.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:56 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:56.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:56 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:56.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:56 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:56.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:56 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:35:56.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:56 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:57.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:56 smithi002 ceph-mon[97651]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:57.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:56 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:57.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:56 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:57.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:56 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:35:57.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:56 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:35:57.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:56 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:57.119 DEBUG:teuthology.orchestra.run.smithi156:osd.8> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.8.service 2023-09-13T04:35:57.122 INFO:tasks.cephadm:Deploying osd.9 on smithi002 with /dev/vg_nvme/lv_1... 2023-09-13T04:35:57.122 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-13T04:35:57.153 INFO:journalctl@ceph.osd.8.smithi156.stdout:-- Logs begin at Wed 2023-09-13 04:09:45 UTC. -- 2023-09-13T04:35:57.864 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:35:57.936 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:35:57 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:35:57.626+0000 7f79aa646640 -1 osd.8 0 log_to_monitors true 2023-09-13T04:35:57.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:57 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:57.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:57 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:57.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:57 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:35:57.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:57 smithi156 ceph-mon[133675]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:57.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:57 smithi156 ceph-mon[133675]: from='osd.8 [v2:172.21.15.156:6816/1139967383,v1:172.21.15.156:6817/1139967383]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-13T04:35:57.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:57 smithi156 ceph-mon[133675]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-13T04:35:58.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:57 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:58.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:57 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:58.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:57 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:35:58.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:57 smithi002 ceph-mon[97651]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:58.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:57 smithi002 ceph-mon[97651]: from='osd.8 [v2:172.21.15.156:6816/1139967383,v1:172.21.15.156:6817/1139967383]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-13T04:35:58.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:57 smithi002 ceph-mon[97651]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-13T04:35:58.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:57 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:58.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:57 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:35:58.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:57 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:35:58.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:57 smithi149 ceph-mon[133315]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:58.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:57 smithi149 ceph-mon[133315]: from='osd.8 [v2:172.21.15.156:6816/1139967383,v1:172.21.15.156:6817/1139967383]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-13T04:35:58.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:57 smithi149 ceph-mon[133315]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-13T04:35:58.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:58 smithi156 ceph-mon[133675]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-13T04:35:58.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:58 smithi156 ceph-mon[133675]: osdmap e56: 9 total, 8 up, 9 in 2023-09-13T04:35:58.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:58 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T04:35:58.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:58 smithi156 ceph-mon[133675]: from='osd.8 [v2:172.21.15.156:6816/1139967383,v1:172.21.15.156:6817/1139967383]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-13T04:35:58.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:58 smithi156 ceph-mon[133675]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-13T04:35:58.938 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:35:58 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:35:58.708+0000 7f799fcbd700 -1 osd.8 0 waiting for initial osdmap 2023-09-13T04:35:58.938 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:35:58 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:35:58.717+0000 7f7998aac700 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-13T04:35:59.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:58 smithi002 ceph-mon[97651]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-13T04:35:59.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:58 smithi002 ceph-mon[97651]: osdmap e56: 9 total, 8 up, 9 in 2023-09-13T04:35:59.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:58 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T04:35:59.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:58 smithi002 ceph-mon[97651]: from='osd.8 [v2:172.21.15.156:6816/1139967383,v1:172.21.15.156:6817/1139967383]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-13T04:35:59.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:58 smithi002 ceph-mon[97651]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-13T04:35:59.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:58 smithi149 ceph-mon[133315]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-13T04:35:59.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:58 smithi149 ceph-mon[133315]: osdmap e56: 9 total, 8 up, 9 in 2023-09-13T04:35:59.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:58 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T04:35:59.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:58 smithi149 ceph-mon[133315]: from='osd.8 [v2:172.21.15.156:6816/1139967383,v1:172.21.15.156:6817/1139967383]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-13T04:35:59.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:58 smithi149 ceph-mon[133315]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-13T04:35:59.781 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:59 smithi156 ceph-mon[133675]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-09-13T04:35:59.782 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:59 smithi156 ceph-mon[133675]: osdmap e57: 9 total, 8 up, 9 in 2023-09-13T04:35:59.782 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:59 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T04:35:59.782 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:59 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T04:35:59.782 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:59 smithi156 ceph-mon[133675]: osd.8 [v2:172.21.15.156:6816/1139967383,v1:172.21.15.156:6817/1139967383] boot 2023-09-13T04:35:59.783 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:59 smithi156 ceph-mon[133675]: osdmap e58: 9 total, 9 up, 9 in 2023-09-13T04:35:59.783 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:59 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T04:35:59.783 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:35:59 smithi156 ceph-mon[133675]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:35:59.901 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-13T04:35:59.901 INFO:teuthology.orchestra.run.smithi002.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-13T04:35:59.901 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10+0 records in 2023-09-13T04:35:59.901 INFO:teuthology.orchestra.run.smithi002.stderr:10+0 records out 2023-09-13T04:35:59.902 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0306018 s, 343 MB/s 2023-09-13T04:35:59.902 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping successful for: 2023-09-13T04:36:00.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:59 smithi002 ceph-mon[97651]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-09-13T04:36:00.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:59 smithi002 ceph-mon[97651]: osdmap e57: 9 total, 8 up, 9 in 2023-09-13T04:36:00.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:59 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T04:36:00.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:59 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T04:36:00.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:59 smithi002 ceph-mon[97651]: osd.8 [v2:172.21.15.156:6816/1139967383,v1:172.21.15.156:6817/1139967383] boot 2023-09-13T04:36:00.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:59 smithi002 ceph-mon[97651]: osdmap e58: 9 total, 9 up, 9 in 2023-09-13T04:36:00.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:59 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T04:36:00.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:35:59 smithi002 ceph-mon[97651]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:36:00.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:59 smithi149 ceph-mon[133315]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-09-13T04:36:00.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:59 smithi149 ceph-mon[133315]: osdmap e57: 9 total, 8 up, 9 in 2023-09-13T04:36:00.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:59 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T04:36:00.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:59 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T04:36:00.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:59 smithi149 ceph-mon[133315]: osd.8 [v2:172.21.15.156:6816/1139967383,v1:172.21.15.156:6817/1139967383] boot 2023-09-13T04:36:00.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:59 smithi149 ceph-mon[133315]: osdmap e58: 9 total, 9 up, 9 in 2023-09-13T04:36:00.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:59 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T04:36:00.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:35:59 smithi149 ceph-mon[133315]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T04:36:00.545 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph orch daemon add osd smithi002:vg_nvme/lv_1 2023-09-13T04:36:01.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:00 smithi156 ceph-mon[133675]: purged_snaps scrub starts 2023-09-13T04:36:01.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:00 smithi156 ceph-mon[133675]: purged_snaps scrub ok 2023-09-13T04:36:01.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:00 smithi156 ceph-mon[133675]: Detected new or changed devices on smithi156 2023-09-13T04:36:01.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:00 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:01.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:00 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:01.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:00 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:01.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:00 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:01.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:00 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:01.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:00 smithi156 ceph-mon[133675]: Adjusting osd_memory_target on smithi156 to 5722M 2023-09-13T04:36:01.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:00 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:01.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:00 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:01.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:00 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:36:01.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:00 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:01.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:00 smithi156 ceph-mon[133675]: osdmap e59: 9 total, 9 up, 9 in 2023-09-13T04:36:01.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:00 smithi002 ceph-mon[97651]: purged_snaps scrub starts 2023-09-13T04:36:01.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:00 smithi002 ceph-mon[97651]: purged_snaps scrub ok 2023-09-13T04:36:01.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:00 smithi002 ceph-mon[97651]: Detected new or changed devices on smithi156 2023-09-13T04:36:01.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:00 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:01.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:00 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:01.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:00 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:01.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:00 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:01.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:00 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:01.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:00 smithi002 ceph-mon[97651]: Adjusting osd_memory_target on smithi156 to 5722M 2023-09-13T04:36:01.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:00 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:01.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:00 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:01.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:00 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:36:01.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:00 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:01.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:00 smithi002 ceph-mon[97651]: osdmap e59: 9 total, 9 up, 9 in 2023-09-13T04:36:01.276 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:36:01.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:00 smithi149 ceph-mon[133315]: purged_snaps scrub starts 2023-09-13T04:36:01.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:00 smithi149 ceph-mon[133315]: purged_snaps scrub ok 2023-09-13T04:36:01.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:00 smithi149 ceph-mon[133315]: Detected new or changed devices on smithi156 2023-09-13T04:36:01.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:00 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:01.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:00 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:01.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:00 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:01.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:00 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:01.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:00 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:01.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:00 smithi149 ceph-mon[133315]: Adjusting osd_memory_target on smithi156 to 5722M 2023-09-13T04:36:01.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:00 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:01.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:00 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:01.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:00 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:36:01.390 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:00 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:01.390 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:00 smithi149 ceph-mon[133315]: osdmap e59: 9 total, 9 up, 9 in 2023-09-13T04:36:02.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:02 smithi002 ceph-mon[97651]: osdmap e60: 9 total, 9 up, 9 in 2023-09-13T04:36:02.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:02 smithi002 ceph-mon[97651]: pgmap v189: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-13T04:36:02.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:02 smithi149 ceph-mon[133315]: osdmap e60: 9 total, 9 up, 9 in 2023-09-13T04:36:02.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:02 smithi149 ceph-mon[133315]: pgmap v189: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-13T04:36:02.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:02 smithi156 ceph-mon[133675]: osdmap e60: 9 total, 9 up, 9 in 2023-09-13T04:36:02.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:02 smithi156 ceph-mon[133675]: pgmap v189: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-13T04:36:02.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.937+0000 7f48375ef700 1 -- 172.21.15.2:0/95710948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4830074ce0 msgr2=0x7f4830073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:36:02.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.937+0000 7f48375ef700 1 --2- 172.21.15.2:0/95710948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4830074ce0 0x7f4830073140 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f482c009990 tx=0x7f482c009ca0 comp rx=0 tx=0).stop 2023-09-13T04:36:02.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.937+0000 7f48375ef700 1 -- 172.21.15.2:0/95710948 shutdown_connections 2023-09-13T04:36:02.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.937+0000 7f48375ef700 1 --2- 172.21.15.2:0/95710948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4830108c40 0x7f483010b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:36:02.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.937+0000 7f48375ef700 1 --2- 172.21.15.2:0/95710948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4830074ce0 0x7f4830073140 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:36:02.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.937+0000 7f48375ef700 1 --2- 172.21.15.2:0/95710948 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4830073710 0x7f4830073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:36:02.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.937+0000 7f48375ef700 1 -- 172.21.15.2:0/95710948 >> 172.21.15.2:0/95710948 conn(0x7f48300fc2d0 msgr2=0x7f48300fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:36:02.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.937+0000 7f48375ef700 1 -- 172.21.15.2:0/95710948 shutdown_connections 2023-09-13T04:36:02.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.937+0000 7f48375ef700 1 -- 172.21.15.2:0/95710948 wait complete. 2023-09-13T04:36:02.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.938+0000 7f48375ef700 1 Processor -- start 2023-09-13T04:36:02.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.939+0000 7f48375ef700 1 -- start start 2023-09-13T04:36:02.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.939+0000 7f48375ef700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4830073710 0x7f4830198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:36:02.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.939+0000 7f48375ef700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4830074ce0 0x7f4830199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:36:02.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.939+0000 7f48375ef700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4830108c40 0x7f48301a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:36:02.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.939+0000 7f48375ef700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48301a07a0 con 0x7f4830108c40 2023-09-13T04:36:02.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.939+0000 7f48375ef700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48301a08e0 con 0x7f4830073710 2023-09-13T04:36:02.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.939+0000 7f48375ef700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48301a0aa0 con 0x7f4830074ce0 2023-09-13T04:36:02.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.939+0000 7f4834b8a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4830074ce0 0x7f4830199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:36:02.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.939+0000 7f4835b8c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4830108c40 0x7f48301a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:36:02.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.939+0000 7f4834b8a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4830074ce0 0x7f4830199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:35326/0 (socket says 172.21.15.2:35326) 2023-09-13T04:36:02.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.939+0000 7f4834b8a700 1 -- 172.21.15.2:0/2459038106 learned_addr learned my addr 172.21.15.2:0/2459038106 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:36:02.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.939+0000 7f483538b700 1 --2- 172.21.15.2:0/2459038106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4830073710 0x7f4830198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:36:02.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.940+0000 7f4835b8c700 1 -- 172.21.15.2:0/2459038106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4830074ce0 msgr2=0x7f4830199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:36:02.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.940+0000 7f4835b8c700 1 --2- 172.21.15.2:0/2459038106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4830074ce0 0x7f4830199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:36:02.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.940+0000 7f4835b8c700 1 -- 172.21.15.2:0/2459038106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4830073710 msgr2=0x7f4830198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:36:02.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.940+0000 7f4835b8c700 1 --2- 172.21.15.2:0/2459038106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4830073710 0x7f4830198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:36:02.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.940+0000 7f4835b8c700 1 -- 172.21.15.2:0/2459038106 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c0095f0 con 0x7f4830108c40 2023-09-13T04:36:02.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.940+0000 7f4835b8c700 1 --2- 172.21.15.2:0/2459038106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4830108c40 0x7f48301a01d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f482800efc0 tx=0x7f482800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:36:02.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.940+0000 7f48267fc700 1 -- 172.21.15.2:0/2459038106 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4828018070 con 0x7f4830108c40 2023-09-13T04:36:02.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.940+0000 7f48267fc700 1 -- 172.21.15.2:0/2459038106 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f482800f040 con 0x7f4830108c40 2023-09-13T04:36:02.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.940+0000 7f48267fc700 1 -- 172.21.15.2:0/2459038106 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482800cd40 con 0x7f4830108c40 2023-09-13T04:36:02.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.940+0000 7f48375ef700 1 -- 172.21.15.2:0/2459038106 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48301a0d20 con 0x7f4830108c40 2023-09-13T04:36:02.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.940+0000 7f48375ef700 1 -- 172.21.15.2:0/2459038106 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48301a1170 con 0x7f4830108c40 2023-09-13T04:36:02.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.942+0000 7f48375ef700 1 -- 172.21.15.2:0/2459038106 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48301072f0 con 0x7f4830108c40 2023-09-13T04:36:02.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.943+0000 7f48267fc700 1 -- 172.21.15.2:0/2459038106 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f4828003a00 con 0x7f4830108c40 2023-09-13T04:36:02.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.945+0000 7f48267fc700 1 --2- 172.21.15.2:0/2459038106 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f481c0a06c0 0x7f481c0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:36:02.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.945+0000 7f48267fc700 1 -- 172.21.15.2:0/2459038106 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5829+0+0 (secure 0 0 0) 0x7f48280cada0 con 0x7f4830108c40 2023-09-13T04:36:02.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.945+0000 7f483538b700 1 --2- 172.21.15.2:0/2459038106 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f481c0a06c0 0x7f481c0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:36:02.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.945+0000 7f483538b700 1 --2- 172.21.15.2:0/2459038106 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f481c0a06c0 0x7f481c0a2b80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f482c000c00 tx=0x7f482c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:36:02.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:02.949+0000 7f48267fc700 1 -- 172.21.15.2:0/2459038106 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4828095270 con 0x7f4830108c40 2023-09-13T04:36:03.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:03.168+0000 7f48375ef700 1 -- 172.21.15.2:0/2459038106 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f483010e7c0 con 0x7f481c0a06c0 2023-09-13T04:36:03.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:03 smithi002 ceph-mon[97651]: osdmap e61: 9 total, 9 up, 9 in 2023-09-13T04:36:03.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:03 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:36:03.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:03 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:36:03.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:03 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:03.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:03 smithi149 ceph-mon[133315]: osdmap e61: 9 total, 9 up, 9 in 2023-09-13T04:36:03.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:03 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:36:03.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:03 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:36:03.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:03 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:03.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:03 smithi156 ceph-mon[133675]: osdmap e61: 9 total, 9 up, 9 in 2023-09-13T04:36:03.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:03 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:36:03.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:03 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:36:03.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:03 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:04.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:04 smithi002 ceph-mon[97651]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:36:04.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:04 smithi002 ceph-mon[97651]: osdmap e62: 9 total, 9 up, 9 in 2023-09-13T04:36:04.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:04 smithi002 ceph-mon[97651]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-13T04:36:04.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:04 smithi149 ceph-mon[133315]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:36:04.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:04 smithi149 ceph-mon[133315]: osdmap e62: 9 total, 9 up, 9 in 2023-09-13T04:36:04.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:04 smithi149 ceph-mon[133315]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-13T04:36:04.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:04 smithi156 ceph-mon[133675]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:36:04.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:04 smithi156 ceph-mon[133675]: osdmap e62: 9 total, 9 up, 9 in 2023-09-13T04:36:04.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:04 smithi156 ceph-mon[133675]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-13T04:36:06.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:06 smithi149 ceph-mon[133315]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2023-09-13T04:36:06.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:06 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1816862305' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb50721f-5036-47d4-89f8-0fa09538266d"}]: dispatch 2023-09-13T04:36:06.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:06 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1816862305' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb50721f-5036-47d4-89f8-0fa09538266d"}]': finished 2023-09-13T04:36:06.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:06 smithi149 ceph-mon[133315]: osdmap e63: 10 total, 9 up, 10 in 2023-09-13T04:36:06.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:06 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T04:36:06.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:06 smithi156 ceph-mon[133675]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2023-09-13T04:36:06.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:06 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1816862305' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb50721f-5036-47d4-89f8-0fa09538266d"}]: dispatch 2023-09-13T04:36:06.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:06 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1816862305' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb50721f-5036-47d4-89f8-0fa09538266d"}]': finished 2023-09-13T04:36:06.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:06 smithi156 ceph-mon[133675]: osdmap e63: 10 total, 9 up, 10 in 2023-09-13T04:36:06.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T04:36:07.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:06 smithi002 ceph-mon[97651]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2023-09-13T04:36:07.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:06 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1816862305' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb50721f-5036-47d4-89f8-0fa09538266d"}]: dispatch 2023-09-13T04:36:07.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:06 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1816862305' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb50721f-5036-47d4-89f8-0fa09538266d"}]': finished 2023-09-13T04:36:07.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:06 smithi002 ceph-mon[97651]: osdmap e63: 10 total, 9 up, 10 in 2023-09-13T04:36:07.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:06 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T04:36:07.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:07 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/762345343' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:36:07.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:07 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/762345343' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:36:08.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:07 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/762345343' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:36:08.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:08 smithi149 ceph-mon[133315]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-13T04:36:08.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:08 smithi156 ceph-mon[133675]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-13T04:36:09.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:08 smithi002 ceph-mon[97651]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-13T04:36:10.497 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:10 smithi002 ceph-mon[97651]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-09-13T04:36:10.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:10 smithi149 ceph-mon[133315]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-09-13T04:36:10.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:10 smithi156 ceph-mon[133675]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-09-13T04:36:12.637 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:12 smithi002 ceph-mon[97651]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-09-13T04:36:12.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:12 smithi149 ceph-mon[133315]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-09-13T04:36:12.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:12 smithi156 ceph-mon[133675]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-09-13T04:36:14.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:14 smithi149 ceph-mon[133315]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-13T04:36:14.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-13T04:36:14.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:14 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:14.897 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:14 smithi002 ceph-mon[97651]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-13T04:36:14.898 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:14 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-13T04:36:14.898 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:14 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:14.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:14 smithi156 ceph-mon[133675]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-13T04:36:14.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:14 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-13T04:36:14.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:14 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:15.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:15 smithi149 ceph-mon[133315]: Deploying daemon osd.9 on smithi002 2023-09-13T04:36:15.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:15 smithi156 ceph-mon[133675]: Deploying daemon osd.9 on smithi002 2023-09-13T04:36:16.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:15 smithi002 ceph-mon[97651]: Deploying daemon osd.9 on smithi002 2023-09-13T04:36:16.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:16 smithi149 ceph-mon[133315]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T04:36:16.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:16 smithi156 ceph-mon[133675]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T04:36:16.973 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:16 smithi002 ceph-mon[97651]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T04:36:18.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:18 smithi002 ceph-mon[97651]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T04:36:18.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:18 smithi149 ceph-mon[133315]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T04:36:18.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:18 smithi156 ceph-mon[133675]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T04:36:20.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:20 smithi002 ceph-mon[97651]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T04:36:20.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:20 smithi149 ceph-mon[133315]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T04:36:20.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:20 smithi156 ceph-mon[133675]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T04:36:21.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:21 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:21.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:21 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:36:21.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:21 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:21.666 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:21 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:21.666 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:21 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:36:21.667 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:21 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:21.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:21 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:21.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:21 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:36:21.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:21 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:22.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:22 smithi156 ceph-mon[133675]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T04:36:22.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:22 smithi002 ceph-mon[97651]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T04:36:22.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:22 smithi149 ceph-mon[133315]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T04:36:24.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:24.136+0000 7f48267fc700 1 -- 172.21.15.2:0/2459038106 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f483010e7c0 con 0x7f481c0a06c0 2023-09-13T04:36:24.138 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 9 on host 'smithi002' 2023-09-13T04:36:24.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:24.139+0000 7f48375ef700 1 -- 172.21.15.2:0/2459038106 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f481c0a06c0 msgr2=0x7f481c0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:36:24.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:24.139+0000 7f48375ef700 1 --2- 172.21.15.2:0/2459038106 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f481c0a06c0 0x7f481c0a2b80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f482c000c00 tx=0x7f482c005fb0 comp rx=0 tx=0).stop 2023-09-13T04:36:24.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:24.139+0000 7f48375ef700 1 -- 172.21.15.2:0/2459038106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4830108c40 msgr2=0x7f48301a01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:36:24.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:24.139+0000 7f48375ef700 1 --2- 172.21.15.2:0/2459038106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4830108c40 0x7f48301a01d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f482800efc0 tx=0x7f482800c3a0 comp rx=0 tx=0).stop 2023-09-13T04:36:24.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:24.139+0000 7f48375ef700 1 -- 172.21.15.2:0/2459038106 shutdown_connections 2023-09-13T04:36:24.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:24.139+0000 7f48375ef700 1 --2- 172.21.15.2:0/2459038106 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f481c0a06c0 0x7f481c0a2b80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:36:24.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:24.139+0000 7f48375ef700 1 --2- 172.21.15.2:0/2459038106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4830108c40 0x7f48301a01d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:36:24.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:24.139+0000 7f48375ef700 1 --2- 172.21.15.2:0/2459038106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4830073710 0x7f4830198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:36:24.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:24.139+0000 7f48375ef700 1 --2- 172.21.15.2:0/2459038106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4830074ce0 0x7f4830199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:36:24.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:24.139+0000 7f48375ef700 1 -- 172.21.15.2:0/2459038106 >> 172.21.15.2:0/2459038106 conn(0x7f48300fc2d0 msgr2=0x7f48300fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:36:24.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:24.139+0000 7f48375ef700 1 -- 172.21.15.2:0/2459038106 shutdown_connections 2023-09-13T04:36:24.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:36:24.139+0000 7f48375ef700 1 -- 172.21.15.2:0/2459038106 wait complete. 2023-09-13T04:36:24.599 DEBUG:teuthology.orchestra.run.smithi002:osd.9> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.9.service 2023-09-13T04:36:24.605 INFO:tasks.cephadm:Deploying osd.10 on smithi149 with /dev/vg_nvme/lv_1... 2023-09-13T04:36:24.606 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-13T04:36:24.677 INFO:journalctl@ceph.osd.9.smithi002.stdout:-- Logs begin at Wed 2023-09-13 04:13:33 UTC. -- 2023-09-13T04:36:24.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:24 smithi149 ceph-mon[133315]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T04:36:24.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:24.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:24.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:24.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:36:24.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:24.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:24.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:24.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:24 smithi149 ceph-mon[133315]: from='osd.9 [v2:172.21.15.2:6826/3097876180,v1:172.21.15.2:6827/3097876180]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-13T04:36:24.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:36:24.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:24 smithi156 ceph-mon[133675]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T04:36:24.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:24.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:24.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:24.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:36:24.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:24.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:24.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:24.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:24 smithi156 ceph-mon[133675]: from='osd.9 [v2:172.21.15.2:6826/3097876180,v1:172.21.15.2:6827/3097876180]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-13T04:36:24.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:36:25.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:24 smithi002 ceph-mon[97651]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T04:36:25.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:25.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:25.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:25.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:36:25.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:25.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:25.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:25.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:24 smithi002 ceph-mon[97651]: from='osd.9 [v2:172.21.15.2:6826/3097876180,v1:172.21.15.2:6827/3097876180]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-13T04:36:25.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:36:25.175 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.b/config 2023-09-13T04:36:26.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:26 smithi149 ceph-mon[133315]: from='osd.9 [v2:172.21.15.2:6826/3097876180,v1:172.21.15.2:6827/3097876180]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-13T04:36:26.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:26 smithi149 ceph-mon[133315]: osdmap e64: 10 total, 9 up, 10 in 2023-09-13T04:36:26.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:26 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T04:36:26.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:26 smithi149 ceph-mon[133315]: from='osd.9 [v2:172.21.15.2:6826/3097876180,v1:172.21.15.2:6827/3097876180]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-09-13T04:36:26.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:26 smithi149 ceph-mon[133315]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T04:36:26.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:26 smithi156 ceph-mon[133675]: from='osd.9 [v2:172.21.15.2:6826/3097876180,v1:172.21.15.2:6827/3097876180]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-13T04:36:26.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:26 smithi156 ceph-mon[133675]: osdmap e64: 10 total, 9 up, 10 in 2023-09-13T04:36:26.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:26 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T04:36:26.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:26 smithi156 ceph-mon[133675]: from='osd.9 [v2:172.21.15.2:6826/3097876180,v1:172.21.15.2:6827/3097876180]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-09-13T04:36:26.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:26 smithi156 ceph-mon[133675]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T04:36:26.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:26 smithi002 ceph-mon[97651]: from='osd.9 [v2:172.21.15.2:6826/3097876180,v1:172.21.15.2:6827/3097876180]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-13T04:36:26.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:26 smithi002 ceph-mon[97651]: osdmap e64: 10 total, 9 up, 10 in 2023-09-13T04:36:26.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:26 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T04:36:26.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:26 smithi002 ceph-mon[97651]: from='osd.9 [v2:172.21.15.2:6826/3097876180,v1:172.21.15.2:6827/3097876180]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-09-13T04:36:26.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:26 smithi002 ceph-mon[97651]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T04:36:26.507 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:36:26 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:36:26.113+0000 7f6dd1761700 -1 osd.9 0 waiting for initial osdmap 2023-09-13T04:36:26.507 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:36:26 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:36:26.130+0000 7f6dcb552700 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-13T04:36:26.701 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-13T04:36:26.701 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-13T04:36:26.701 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2023-09-13T04:36:26.701 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2023-09-13T04:36:26.702 INFO:teuthology.orchestra.run.smithi149.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110488 s, 949 MB/s 2023-09-13T04:36:26.702 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 2023-09-13T04:36:26.702 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2023-09-13T04:36:27.130 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph orch daemon add osd smithi149:vg_nvme/lv_1 2023-09-13T04:36:27.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:27 smithi149 ceph-mon[133315]: from='osd.9 [v2:172.21.15.2:6826/3097876180,v1:172.21.15.2:6827/3097876180]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2023-09-13T04:36:27.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:27 smithi149 ceph-mon[133315]: osdmap e65: 10 total, 9 up, 10 in 2023-09-13T04:36:27.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:27 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T04:36:27.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:27 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T04:36:27.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:27 smithi156 ceph-mon[133675]: from='osd.9 [v2:172.21.15.2:6826/3097876180,v1:172.21.15.2:6827/3097876180]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2023-09-13T04:36:27.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:27 smithi156 ceph-mon[133675]: osdmap e65: 10 total, 9 up, 10 in 2023-09-13T04:36:27.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:27 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T04:36:27.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:27 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T04:36:27.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:27 smithi002 ceph-mon[97651]: from='osd.9 [v2:172.21.15.2:6826/3097876180,v1:172.21.15.2:6827/3097876180]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2023-09-13T04:36:27.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:27 smithi002 ceph-mon[97651]: osdmap e65: 10 total, 9 up, 10 in 2023-09-13T04:36:27.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:27 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T04:36:27.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:27 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T04:36:27.689 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.b/config 2023-09-13T04:36:28.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:28 smithi149 ceph-mon[133315]: purged_snaps scrub starts 2023-09-13T04:36:28.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:28 smithi149 ceph-mon[133315]: purged_snaps scrub ok 2023-09-13T04:36:28.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:28 smithi149 ceph-mon[133315]: osd.9 [v2:172.21.15.2:6826/3097876180,v1:172.21.15.2:6827/3097876180] boot 2023-09-13T04:36:28.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:28 smithi149 ceph-mon[133315]: osdmap e66: 10 total, 10 up, 10 in 2023-09-13T04:36:28.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:28 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T04:36:28.253 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:28 smithi149 ceph-mon[133315]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:28.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:28 smithi002 ceph-mon[97651]: purged_snaps scrub starts 2023-09-13T04:36:28.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:28 smithi002 ceph-mon[97651]: purged_snaps scrub ok 2023-09-13T04:36:28.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:28 smithi002 ceph-mon[97651]: osd.9 [v2:172.21.15.2:6826/3097876180,v1:172.21.15.2:6827/3097876180] boot 2023-09-13T04:36:28.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:28 smithi002 ceph-mon[97651]: osdmap e66: 10 total, 10 up, 10 in 2023-09-13T04:36:28.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:28 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T04:36:28.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:28 smithi002 ceph-mon[97651]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:28.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:28 smithi156 ceph-mon[133675]: purged_snaps scrub starts 2023-09-13T04:36:28.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:28 smithi156 ceph-mon[133675]: purged_snaps scrub ok 2023-09-13T04:36:28.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:28 smithi156 ceph-mon[133675]: osd.9 [v2:172.21.15.2:6826/3097876180,v1:172.21.15.2:6827/3097876180] boot 2023-09-13T04:36:28.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:28 smithi156 ceph-mon[133675]: osdmap e66: 10 total, 10 up, 10 in 2023-09-13T04:36:28.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:28 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T04:36:28.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:28 smithi156 ceph-mon[133675]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:28.989 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.987+0000 7f18d5db4700 1 -- 172.21.15.149:0/1888733385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18d0101810 msgr2=0x7f18d01080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:36:28.989 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.987+0000 7f18d5db4700 1 --2- 172.21.15.149:0/1888733385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18d0101810 0x7f18d01080c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f18c800b0d0 tx=0x7f18c800b3e0 comp rx=0 tx=0).stop 2023-09-13T04:36:28.989 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.987+0000 7f18d5db4700 1 -- 172.21.15.149:0/1888733385 shutdown_connections 2023-09-13T04:36:28.989 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.987+0000 7f18d5db4700 1 --2- 172.21.15.149:0/1888733385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18d0101810 0x7f18d01080c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:36:28.990 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.987+0000 7f18d5db4700 1 --2- 172.21.15.149:0/1888733385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18d00ffc70 0x7f18d0100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:36:28.990 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.987+0000 7f18d5db4700 1 --2- 172.21.15.149:0/1888733385 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18d0100e70 0x7f18d01012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:36:28.990 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.987+0000 7f18d5db4700 1 -- 172.21.15.149:0/1888733385 >> 172.21.15.149:0/1888733385 conn(0x7f18d00fb1f0 msgr2=0x7f18d00fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:36:28.990 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.987+0000 7f18d5db4700 1 -- 172.21.15.149:0/1888733385 shutdown_connections 2023-09-13T04:36:28.990 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.987+0000 7f18d5db4700 1 -- 172.21.15.149:0/1888733385 wait complete. 2023-09-13T04:36:28.991 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.988+0000 7f18d5db4700 1 Processor -- start 2023-09-13T04:36:28.992 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.988+0000 7f18d5db4700 1 -- start start 2023-09-13T04:36:28.992 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.988+0000 7f18d5db4700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18d00ffc70 0x7f18d0196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:36:28.992 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.988+0000 7f18d5db4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18d0100e70 0x7f18d0196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:36:28.993 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.988+0000 7f18d5db4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18d0101810 0x7f18d019e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:36:28.993 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.988+0000 7f18d5db4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d019e620 con 0x7f18d0100e70 2023-09-13T04:36:28.993 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.988+0000 7f18d5db4700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d019e760 con 0x7f18d00ffc70 2023-09-13T04:36:28.993 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.988+0000 7f18d5db4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d019e8f0 con 0x7f18d0101810 2023-09-13T04:36:28.993 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.988+0000 7f18cf7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18d00ffc70 0x7f18d0196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:36:28.994 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.988+0000 7f18cf7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18d00ffc70 0x7f18d0196a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:57774/0 (socket says 172.21.15.149:57774) 2023-09-13T04:36:28.994 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.988+0000 7f18cf7fe700 1 -- 172.21.15.149:0/3472979834 learned_addr learned my addr 172.21.15.149:0/3472979834 (peer_addr_for_me v2:172.21.15.149:0/0) 2023-09-13T04:36:28.994 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.989+0000 7f18cffff700 1 --2- 172.21.15.149:0/3472979834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18d0101810 0x7f18d019e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:36:28.994 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.989+0000 7f18cf7fe700 1 -- 172.21.15.149:0/3472979834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18d0101810 msgr2=0x7f18d019e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:36:28.994 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.989+0000 7f18cf7fe700 1 --2- 172.21.15.149:0/3472979834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18d0101810 0x7f18d019e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:36:28.994 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.989+0000 7f18cf7fe700 1 -- 172.21.15.149:0/3472979834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18d0100e70 msgr2=0x7f18d0196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:36:28.995 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.989+0000 7f18cf7fe700 1 --2- 172.21.15.149:0/3472979834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18d0100e70 0x7f18d0196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:36:28.995 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.989+0000 7f18cf7fe700 1 -- 172.21.15.149:0/3472979834 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c8009d00 con 0x7f18d00ffc70 2023-09-13T04:36:28.995 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.989+0000 7f18cf7fe700 1 --2- 172.21.15.149:0/3472979834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18d00ffc70 0x7f18d0196a90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f18b800d980 tx=0x7f18b800dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:36:28.995 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.989+0000 7f18ccff9700 1 -- 172.21.15.149:0/3472979834 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b800de60 con 0x7f18d00ffc70 2023-09-13T04:36:28.995 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.989+0000 7f18d5db4700 1 -- 172.21.15.149:0/3472979834 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d019ebd0 con 0x7f18d00ffc70 2023-09-13T04:36:28.996 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.989+0000 7f18ccff9700 1 -- 172.21.15.149:0/3472979834 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18b80044e0 con 0x7f18d00ffc70 2023-09-13T04:36:28.996 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.989+0000 7f18ccff9700 1 -- 172.21.15.149:0/3472979834 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8003bc0 con 0x7f18d00ffc70 2023-09-13T04:36:28.996 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.989+0000 7f18d5db4700 1 -- 172.21.15.149:0/3472979834 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d019f0d0 con 0x7f18d00ffc70 2023-09-13T04:36:28.996 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.990+0000 7f18d5db4700 1 -- 172.21.15.149:0/3472979834 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d010c640 con 0x7f18d00ffc70 2023-09-13T04:36:28.996 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.991+0000 7f18ccff9700 1 -- 172.21.15.149:0/3472979834 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f18b8009bb0 con 0x7f18d00ffc70 2023-09-13T04:36:28.997 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.991+0000 7f18ccff9700 1 --2- 172.21.15.149:0/3472979834 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f18bc0a0640 0x7f18bc0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:36:28.997 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.992+0000 7f18ccff9700 1 -- 172.21.15.149:0/3472979834 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f18b80cb0f0 con 0x7f18d00ffc70 2023-09-13T04:36:28.997 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.992+0000 7f18ceffd700 1 --2- 172.21.15.149:0/3472979834 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f18bc0a0640 0x7f18bc0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:36:28.998 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.993+0000 7f18ceffd700 1 --2- 172.21.15.149:0/3472979834 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f18bc0a0640 0x7f18bc0a2b00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f18c0005fd0 tx=0x7f18c0005d00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:36:28.998 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:28.996+0000 7f18ccff9700 1 -- 172.21.15.149:0/3472979834 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f18b8095420 con 0x7f18d00ffc70 2023-09-13T04:36:29.197 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:29.196+0000 7f18d5db4700 1 -- 172.21.15.149:0/3472979834 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f18d00610f0 con 0x7f18bc0a0640 2023-09-13T04:36:29.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:29 smithi149 ceph-mon[133315]: osdmap e67: 10 total, 10 up, 10 in 2023-09-13T04:36:29.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:29 smithi149 ceph-mon[133315]: Detected new or changed devices on smithi002 2023-09-13T04:36:29.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:29 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:29.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:29 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:29.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:29 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:29.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:29 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:29.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:29 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:29.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:29 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:29.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:29 smithi149 ceph-mon[133315]: Adjusting osd_memory_target on smithi002 to 4318M 2023-09-13T04:36:29.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:29 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:29.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:29 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:29.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:29 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:36:29.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:29 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:29.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:29 smithi156 ceph-mon[133675]: osdmap e67: 10 total, 10 up, 10 in 2023-09-13T04:36:29.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:29 smithi156 ceph-mon[133675]: Detected new or changed devices on smithi002 2023-09-13T04:36:29.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:29 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:29.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:29 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:29.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:29 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:29.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:29 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:29.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:29 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:29.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:29 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:29.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:29 smithi156 ceph-mon[133675]: Adjusting osd_memory_target on smithi002 to 4318M 2023-09-13T04:36:29.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:29 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:29.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:29 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:29.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:29 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:36:29.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:29 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:29.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:29 smithi002 ceph-mon[97651]: osdmap e67: 10 total, 10 up, 10 in 2023-09-13T04:36:29.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:29 smithi002 ceph-mon[97651]: Detected new or changed devices on smithi002 2023-09-13T04:36:29.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:29 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:29.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:29 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:29.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:29 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:29.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:29 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:29.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:29 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:29.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:29 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:29.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:29 smithi002 ceph-mon[97651]: Adjusting osd_memory_target on smithi002 to 4318M 2023-09-13T04:36:29.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:29 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:29.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:29 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:29.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:29 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:36:29.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:29 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:30.301 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:30 smithi149 ceph-mon[133315]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:36:30.301 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:30 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:36:30.301 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:30 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:36:30.302 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:30 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:30.302 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:30 smithi149 ceph-mon[133315]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:30.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:30 smithi002 ceph-mon[97651]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:36:30.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:30 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:36:30.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:30 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:36:30.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:30 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:30.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:30 smithi002 ceph-mon[97651]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:30.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:30 smithi156 ceph-mon[133675]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:36:30.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:30 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:36:30.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:30 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:36:30.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:30 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:30.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:30 smithi156 ceph-mon[133675]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:32.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:32 smithi149 ceph-mon[133315]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:32.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:32 smithi149 ceph-mon[133315]: from='client.? 172.21.15.149:0/3863572817' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23ee78a4-63ef-48b5-9383-046730f45bf6"}]: dispatch 2023-09-13T04:36:32.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:32 smithi149 ceph-mon[133315]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23ee78a4-63ef-48b5-9383-046730f45bf6"}]: dispatch 2023-09-13T04:36:32.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:32 smithi149 ceph-mon[133315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23ee78a4-63ef-48b5-9383-046730f45bf6"}]': finished 2023-09-13T04:36:32.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:32 smithi149 ceph-mon[133315]: osdmap e68: 11 total, 10 up, 11 in 2023-09-13T04:36:32.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:32 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T04:36:32.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:32 smithi149 ceph-mon[133315]: from='client.? 172.21.15.149:0/2410694904' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:36:32.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:32 smithi156 ceph-mon[133675]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:32.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:32 smithi156 ceph-mon[133675]: from='client.? 172.21.15.149:0/3863572817' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23ee78a4-63ef-48b5-9383-046730f45bf6"}]: dispatch 2023-09-13T04:36:32.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:32 smithi156 ceph-mon[133675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23ee78a4-63ef-48b5-9383-046730f45bf6"}]: dispatch 2023-09-13T04:36:32.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:32 smithi156 ceph-mon[133675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23ee78a4-63ef-48b5-9383-046730f45bf6"}]': finished 2023-09-13T04:36:32.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:32 smithi156 ceph-mon[133675]: osdmap e68: 11 total, 10 up, 11 in 2023-09-13T04:36:32.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:32 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T04:36:32.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:32 smithi156 ceph-mon[133675]: from='client.? 172.21.15.149:0/2410694904' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:36:33.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:32 smithi002 ceph-mon[97651]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:33.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:32 smithi002 ceph-mon[97651]: from='client.? 172.21.15.149:0/3863572817' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23ee78a4-63ef-48b5-9383-046730f45bf6"}]: dispatch 2023-09-13T04:36:33.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:32 smithi002 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23ee78a4-63ef-48b5-9383-046730f45bf6"}]: dispatch 2023-09-13T04:36:33.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:32 smithi002 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23ee78a4-63ef-48b5-9383-046730f45bf6"}]': finished 2023-09-13T04:36:33.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:32 smithi002 ceph-mon[97651]: osdmap e68: 11 total, 10 up, 11 in 2023-09-13T04:36:33.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:32 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T04:36:33.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:32 smithi002 ceph-mon[97651]: from='client.? 172.21.15.149:0/2410694904' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:36:34.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:34 smithi149 ceph-mon[133315]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:34.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:34 smithi156 ceph-mon[133675]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:35.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:34 smithi002 ceph-mon[97651]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:36.775 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:36 smithi149 ceph-mon[133315]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:36.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:36 smithi156 ceph-mon[133675]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:37.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:36 smithi002 ceph-mon[97651]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:38.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:38 smithi149 ceph-mon[133315]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:38.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:38 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-13T04:36:38.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:38 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:38.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:38 smithi156 ceph-mon[133675]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:38.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:38 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-13T04:36:38.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:38 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:39.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:38 smithi002 ceph-mon[97651]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:39.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:38 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-13T04:36:39.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:38 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:39.849 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:39 smithi149 ceph-mon[133315]: Deploying daemon osd.10 on smithi149 2023-09-13T04:36:39.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:39 smithi156 ceph-mon[133675]: Deploying daemon osd.10 on smithi149 2023-09-13T04:36:40.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:39 smithi002 ceph-mon[97651]: Deploying daemon osd.10 on smithi149 2023-09-13T04:36:40.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:40 smithi149 ceph-mon[133315]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:40.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:40 smithi156 ceph-mon[133675]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:41.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:40 smithi002 ceph-mon[97651]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:42.883 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:42 smithi149 ceph-mon[133315]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:42.883 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:42 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:42.884 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:42 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:42.884 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:42 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:36:42.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:42 smithi156 ceph-mon[133675]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:42.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:42 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:42.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:42 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:42.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:42 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:36:43.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:42 smithi002 ceph-mon[97651]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:43.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:42 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:43.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:42 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:43.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:42 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:36:44.815 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:44 smithi149 ceph-mon[133315]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:44.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:44 smithi156 ceph-mon[133675]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:45.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:44 smithi002 ceph-mon[97651]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:45.160 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:45.160+0000 7f18ccff9700 1 -- 172.21.15.149:0/3472979834 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f18d00610f0 con 0x7f18bc0a0640 2023-09-13T04:36:45.160 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 10 on host 'smithi149' 2023-09-13T04:36:45.162 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:45.162+0000 7f18d5db4700 1 -- 172.21.15.149:0/3472979834 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f18bc0a0640 msgr2=0x7f18bc0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:36:45.162 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:45.162+0000 7f18d5db4700 1 --2- 172.21.15.149:0/3472979834 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f18bc0a0640 0x7f18bc0a2b00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f18c0005fd0 tx=0x7f18c0005d00 comp rx=0 tx=0).stop 2023-09-13T04:36:45.163 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:45.162+0000 7f18d5db4700 1 -- 172.21.15.149:0/3472979834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18d00ffc70 msgr2=0x7f18d0196a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:36:45.163 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:45.162+0000 7f18d5db4700 1 --2- 172.21.15.149:0/3472979834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18d00ffc70 0x7f18d0196a90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f18b800d980 tx=0x7f18b800dd40 comp rx=0 tx=0).stop 2023-09-13T04:36:45.163 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:45.162+0000 7f18d5db4700 1 -- 172.21.15.149:0/3472979834 shutdown_connections 2023-09-13T04:36:45.163 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:45.162+0000 7f18d5db4700 1 --2- 172.21.15.149:0/3472979834 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f18bc0a0640 0x7f18bc0a2b00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:36:45.164 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:45.162+0000 7f18d5db4700 1 --2- 172.21.15.149:0/3472979834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18d0101810 0x7f18d019e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:36:45.164 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:45.162+0000 7f18d5db4700 1 --2- 172.21.15.149:0/3472979834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18d00ffc70 0x7f18d0196a90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:36:45.164 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:45.162+0000 7f18d5db4700 1 --2- 172.21.15.149:0/3472979834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18d0100e70 0x7f18d0196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:36:45.164 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:45.162+0000 7f18d5db4700 1 -- 172.21.15.149:0/3472979834 >> 172.21.15.149:0/3472979834 conn(0x7f18d00fb1f0 msgr2=0x7f18d00fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:36:45.164 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:45.162+0000 7f18d5db4700 1 -- 172.21.15.149:0/3472979834 shutdown_connections 2023-09-13T04:36:45.164 INFO:teuthology.orchestra.run.smithi149.stderr:2023-09-13T04:36:45.162+0000 7f18d5db4700 1 -- 172.21.15.149:0/3472979834 wait complete. 2023-09-13T04:36:45.875 DEBUG:teuthology.orchestra.run.smithi149:osd.10> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.10.service 2023-09-13T04:36:45.877 INFO:tasks.cephadm:Deploying osd.11 on smithi156 with /dev/vg_nvme/lv_1... 2023-09-13T04:36:45.878 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-13T04:36:45.907 INFO:journalctl@ceph.osd.10.smithi149.stdout:-- Logs begin at Wed 2023-09-13 04:09:45 UTC. -- 2023-09-13T04:36:46.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:46 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:46.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:46 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:46.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:46 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:46.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:46 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:46.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:46 smithi156 ceph-mon[133675]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:46.379 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:46 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:46.379 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:46 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:46.379 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:46 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:46.380 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:46 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:46.380 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:46 smithi149 ceph-mon[133315]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:46.444 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-13T04:36:46.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:46 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:46.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:46 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:46.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:46 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:46.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:46 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:46.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:46 smithi002 ceph-mon[97651]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:46.636 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:36:46 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:36:46.416+0000 7f586c1c5640 -1 osd.10 0 log_to_monitors true 2023-09-13T04:36:47.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:47 smithi149 ceph-mon[133315]: from='osd.10 [v2:172.21.15.149:6824/3663849505,v1:172.21.15.149:6825/3663849505]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-13T04:36:47.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:47 smithi149 ceph-mon[133315]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-13T04:36:47.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:47 smithi156 ceph-mon[133675]: from='osd.10 [v2:172.21.15.149:6824/3663849505,v1:172.21.15.149:6825/3663849505]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-13T04:36:47.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:47 smithi156 ceph-mon[133675]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-13T04:36:47.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:47 smithi002 ceph-mon[97651]: from='osd.10 [v2:172.21.15.149:6824/3663849505,v1:172.21.15.149:6825/3663849505]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-13T04:36:47.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:47 smithi002 ceph-mon[97651]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-13T04:36:48.030 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-13T04:36:48.030 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-13T04:36:48.030 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2023-09-13T04:36:48.030 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2023-09-13T04:36:48.031 INFO:teuthology.orchestra.run.smithi156.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00805894 s, 1.3 GB/s 2023-09-13T04:36:48.031 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2023-09-13T04:36:48.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:48 smithi149 ceph-mon[133315]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-13T04:36:48.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:48 smithi149 ceph-mon[133315]: osdmap e69: 11 total, 10 up, 11 in 2023-09-13T04:36:48.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:48 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T04:36:48.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:48 smithi149 ceph-mon[133315]: from='osd.10 [v2:172.21.15.149:6824/3663849505,v1:172.21.15.149:6825/3663849505]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-09-13T04:36:48.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:48 smithi149 ceph-mon[133315]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-09-13T04:36:48.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:48 smithi149 ceph-mon[133315]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:48.388 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:36:48 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:36:48.145+0000 7f586183c700 -1 osd.10 0 waiting for initial osdmap 2023-09-13T04:36:48.389 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:36:48 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:36:48.157+0000 7f585ae2c700 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-13T04:36:48.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:48 smithi156 ceph-mon[133675]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-13T04:36:48.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:48 smithi156 ceph-mon[133675]: osdmap e69: 11 total, 10 up, 11 in 2023-09-13T04:36:48.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:48 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T04:36:48.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:48 smithi156 ceph-mon[133675]: from='osd.10 [v2:172.21.15.149:6824/3663849505,v1:172.21.15.149:6825/3663849505]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-09-13T04:36:48.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:48 smithi156 ceph-mon[133675]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-09-13T04:36:48.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:48 smithi156 ceph-mon[133675]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:48.500 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph orch daemon add osd smithi156:vg_nvme/lv_1 2023-09-13T04:36:48.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:48 smithi002 ceph-mon[97651]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-13T04:36:48.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:48 smithi002 ceph-mon[97651]: osdmap e69: 11 total, 10 up, 11 in 2023-09-13T04:36:48.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:48 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T04:36:48.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:48 smithi002 ceph-mon[97651]: from='osd.10 [v2:172.21.15.149:6824/3663849505,v1:172.21.15.149:6825/3663849505]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-09-13T04:36:48.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:48 smithi002 ceph-mon[97651]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-09-13T04:36:48.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:48 smithi002 ceph-mon[97651]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:49.057 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-13T04:36:49.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:49 smithi156 ceph-mon[133675]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2023-09-13T04:36:49.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:49 smithi156 ceph-mon[133675]: osdmap e70: 11 total, 10 up, 11 in 2023-09-13T04:36:49.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:49 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T04:36:49.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:49 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T04:36:49.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:49 smithi156 ceph-mon[133675]: Detected new or changed devices on smithi149 2023-09-13T04:36:49.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:49 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:49.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:49 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:49.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:49 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:49.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:49 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:49.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:49 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:49.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:49 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:49.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:49 smithi156 ceph-mon[133675]: Adjusting osd_memory_target on smithi149 to 1469M 2023-09-13T04:36:49.439 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:49 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:49.439 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:49 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:49.439 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:49 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:36:49.439 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:49 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:49.439 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:49 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:36:49.440 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:49 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:49.440 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:49 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:36:49.440 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:49 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:49.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:49 smithi002 ceph-mon[97651]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2023-09-13T04:36:49.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:49 smithi002 ceph-mon[97651]: osdmap e70: 11 total, 10 up, 11 in 2023-09-13T04:36:49.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:49 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T04:36:49.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:49 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T04:36:49.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:49 smithi002 ceph-mon[97651]: Detected new or changed devices on smithi149 2023-09-13T04:36:49.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:49 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:49.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:49 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:49.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:49 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:49.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:49 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:49.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:49 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:49.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:49 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:49.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:49 smithi002 ceph-mon[97651]: Adjusting osd_memory_target on smithi149 to 1469M 2023-09-13T04:36:49.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:49 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:49.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:49 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:49.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:49 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:36:49.509 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:49 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:49.509 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:49 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:36:49.509 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:49 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:49.509 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:49 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:36:49.509 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:49 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:49.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:49 smithi149 ceph-mon[133315]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2023-09-13T04:36:49.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:49 smithi149 ceph-mon[133315]: osdmap e70: 11 total, 10 up, 11 in 2023-09-13T04:36:49.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:49 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T04:36:49.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:49 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T04:36:49.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:49 smithi149 ceph-mon[133315]: Detected new or changed devices on smithi149 2023-09-13T04:36:49.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:49 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:49.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:49 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:49.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:49 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:49.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:49 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:49.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:49 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:49.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:49 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:36:49.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:49 smithi149 ceph-mon[133315]: Adjusting osd_memory_target on smithi149 to 1469M 2023-09-13T04:36:49.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:49 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:49.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:49 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:49.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:49 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:36:49.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:49 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:49.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:49 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:36:49.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:49 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:49.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:49 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:36:49.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:49 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:36:50.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:50 smithi156 ceph-mon[133675]: purged_snaps scrub starts 2023-09-13T04:36:50.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:50 smithi156 ceph-mon[133675]: purged_snaps scrub ok 2023-09-13T04:36:50.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:50 smithi156 ceph-mon[133675]: osd.10 [v2:172.21.15.149:6824/3663849505,v1:172.21.15.149:6825/3663849505] boot 2023-09-13T04:36:50.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:50 smithi156 ceph-mon[133675]: osdmap e71: 11 total, 11 up, 11 in 2023-09-13T04:36:50.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:50 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T04:36:50.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:50 smithi156 ceph-mon[133675]: osdmap e72: 11 total, 11 up, 11 in 2023-09-13T04:36:50.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:50 smithi156 ceph-mon[133675]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:50.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.333+0000 7f8c7a749700 1 -- 172.21.15.156:0/674461639 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c74101810 msgr2=0x7f8c741080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:36:50.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.333+0000 7f8c7a749700 1 --2- 172.21.15.156:0/674461639 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c74101810 0x7f8c741080c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00b0a0 tx=0x7f8c6c00b3b0 comp rx=0 tx=0).stop 2023-09-13T04:36:50.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.334+0000 7f8c7a749700 1 -- 172.21.15.156:0/674461639 shutdown_connections 2023-09-13T04:36:50.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.334+0000 7f8c7a749700 1 --2- 172.21.15.156:0/674461639 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c74101810 0x7f8c741080c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:36:50.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.334+0000 7f8c7a749700 1 --2- 172.21.15.156:0/674461639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c740ffc70 0x7f8c74100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:36:50.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.334+0000 7f8c7a749700 1 --2- 172.21.15.156:0/674461639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c74100e70 0x7f8c741012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:36:50.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.334+0000 7f8c7a749700 1 -- 172.21.15.156:0/674461639 >> 172.21.15.156:0/674461639 conn(0x7f8c740fb1f0 msgr2=0x7f8c740fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:36:50.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.334+0000 7f8c7a749700 1 -- 172.21.15.156:0/674461639 shutdown_connections 2023-09-13T04:36:50.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.334+0000 7f8c7a749700 1 -- 172.21.15.156:0/674461639 wait complete. 2023-09-13T04:36:50.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.334+0000 7f8c7a749700 1 Processor -- start 2023-09-13T04:36:50.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.334+0000 7f8c7a749700 1 -- start start 2023-09-13T04:36:50.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.335+0000 7f8c7a749700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c740ffc70 0x7f8c74069bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:36:50.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.335+0000 7f8c7a749700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c74100e70 0x7f8c7419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:36:50.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.335+0000 7f8c7a749700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c74101810 0x7f8c7419e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:36:50.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.335+0000 7f8c7a749700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7419e7d0 con 0x7f8c740ffc70 2023-09-13T04:36:50.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.335+0000 7f8c7a749700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7419e940 con 0x7f8c74100e70 2023-09-13T04:36:50.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.335+0000 7f8c7a749700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7419eb00 con 0x7f8c74101810 2023-09-13T04:36:50.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.335+0000 7f8c78ce6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c74101810 0x7f8c7419e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:36:50.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.335+0000 7f8c78ce6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c74101810 0x7f8c7419e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37430/0 (socket says 172.21.15.156:37430) 2023-09-13T04:36:50.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.335+0000 7f8c78ce6700 1 -- 172.21.15.156:0/572859689 learned_addr learned my addr 172.21.15.156:0/572859689 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-13T04:36:50.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.335+0000 7f8c737fe700 1 --2- 172.21.15.156:0/572859689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c74100e70 0x7f8c7419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:36:50.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.335+0000 7f8c78ce6700 1 -- 172.21.15.156:0/572859689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c74100e70 msgr2=0x7f8c7419b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:36:50.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.335+0000 7f8c78ce6700 1 --2- 172.21.15.156:0/572859689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c74100e70 0x7f8c7419b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:36:50.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.335+0000 7f8c78ce6700 1 -- 172.21.15.156:0/572859689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c740ffc70 msgr2=0x7f8c74069bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:36:50.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.335+0000 7f8c78ce6700 1 --2- 172.21.15.156:0/572859689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c740ffc70 0x7f8c74069bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:36:50.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.335+0000 7f8c78ce6700 1 -- 172.21.15.156:0/572859689 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6c009d00 con 0x7f8c74101810 2023-09-13T04:36:50.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.335+0000 7f8c78ce6700 1 --2- 172.21.15.156:0/572859689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c74101810 0x7f8c7419e030 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00b070 tx=0x7f8c6c004db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:36:50.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.336+0000 7f8c717fa700 1 -- 172.21.15.156:0/572859689 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c00e040 con 0x7f8c74101810 2023-09-13T04:36:50.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.336+0000 7f8c7a749700 1 -- 172.21.15.156:0/572859689 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c7419ed80 con 0x7f8c74101810 2023-09-13T04:36:50.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.336+0000 7f8c717fa700 1 -- 172.21.15.156:0/572859689 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c6c01bd20 con 0x7f8c74101810 2023-09-13T04:36:50.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.336+0000 7f8c717fa700 1 -- 172.21.15.156:0/572859689 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c0058e0 con 0x7f8c74101810 2023-09-13T04:36:50.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.336+0000 7f8c7a749700 1 -- 172.21.15.156:0/572859689 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c7419f210 con 0x7f8c74101810 2023-09-13T04:36:50.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.337+0000 7f8c7a749700 1 -- 172.21.15.156:0/572859689 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c74107270 con 0x7f8c74101810 2023-09-13T04:36:50.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.338+0000 7f8c717fa700 1 -- 172.21.15.156:0/572859689 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f8c6c005040 con 0x7f8c74101810 2023-09-13T04:36:50.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.339+0000 7f8c717fa700 1 --2- 172.21.15.156:0/572859689 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8c5c0a0640 0x7f8c5c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:36:50.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.339+0000 7f8c717fa700 1 -- 172.21.15.156:0/572859689 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f8c6c0cda40 con 0x7f8c74101810 2023-09-13T04:36:50.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.339+0000 7f8c73fff700 1 --2- 172.21.15.156:0/572859689 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8c5c0a0640 0x7f8c5c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:36:50.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.340+0000 7f8c73fff700 1 --2- 172.21.15.156:0/572859689 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8c5c0a0640 0x7f8c5c0a2b00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8c74100cd0 tx=0x7f8c64006d20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:36:50.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.342+0000 7f8c717fa700 1 -- 172.21.15.156:0/572859689 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8c6c097c30 con 0x7f8c74101810 2023-09-13T04:36:50.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:50 smithi002 ceph-mon[97651]: purged_snaps scrub starts 2023-09-13T04:36:50.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:50 smithi002 ceph-mon[97651]: purged_snaps scrub ok 2023-09-13T04:36:50.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:50 smithi002 ceph-mon[97651]: osd.10 [v2:172.21.15.149:6824/3663849505,v1:172.21.15.149:6825/3663849505] boot 2023-09-13T04:36:50.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:50 smithi002 ceph-mon[97651]: osdmap e71: 11 total, 11 up, 11 in 2023-09-13T04:36:50.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:50 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T04:36:50.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:50 smithi002 ceph-mon[97651]: osdmap e72: 11 total, 11 up, 11 in 2023-09-13T04:36:50.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:50 smithi002 ceph-mon[97651]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:50.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:36:50.538+0000 7f8c7a749700 1 -- 172.21.15.156:0/572859689 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c7410b590 con 0x7f8c5c0a0640 2023-09-13T04:36:50.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:50 smithi149 ceph-mon[133315]: purged_snaps scrub starts 2023-09-13T04:36:50.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:50 smithi149 ceph-mon[133315]: purged_snaps scrub ok 2023-09-13T04:36:50.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:50 smithi149 ceph-mon[133315]: osd.10 [v2:172.21.15.149:6824/3663849505,v1:172.21.15.149:6825/3663849505] boot 2023-09-13T04:36:50.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:50 smithi149 ceph-mon[133315]: osdmap e71: 11 total, 11 up, 11 in 2023-09-13T04:36:50.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:50 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T04:36:50.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:50 smithi149 ceph-mon[133315]: osdmap e72: 11 total, 11 up, 11 in 2023-09-13T04:36:50.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:50 smithi149 ceph-mon[133315]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T04:36:51.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:51 smithi002 ceph-mon[97651]: osdmap e73: 11 total, 11 up, 11 in 2023-09-13T04:36:51.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:51 smithi002 ceph-mon[97651]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:36:51.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:51 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:36:51.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:51 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:36:51.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:51 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:51.550 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:51 smithi156 ceph-mon[133675]: osdmap e73: 11 total, 11 up, 11 in 2023-09-13T04:36:51.551 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:51 smithi156 ceph-mon[133675]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:36:51.551 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:51 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:36:51.551 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:51 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:36:51.552 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:51 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:51.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:51 smithi149 ceph-mon[133315]: osdmap e73: 11 total, 11 up, 11 in 2023-09-13T04:36:51.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:51 smithi149 ceph-mon[133315]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:36:51.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:51 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T04:36:51.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:51 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T04:36:51.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:51 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:36:52.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:52 smithi156 ceph-mon[133675]: osdmap e74: 11 total, 11 up, 11 in 2023-09-13T04:36:52.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:52 smithi156 ceph-mon[133675]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-13T04:36:52.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:52 smithi002 ceph-mon[97651]: osdmap e74: 11 total, 11 up, 11 in 2023-09-13T04:36:52.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:52 smithi002 ceph-mon[97651]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-13T04:36:52.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:52 smithi149 ceph-mon[133315]: osdmap e74: 11 total, 11 up, 11 in 2023-09-13T04:36:52.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:52 smithi149 ceph-mon[133315]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-13T04:36:53.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:53 smithi002 ceph-mon[97651]: osdmap e75: 11 total, 11 up, 11 in 2023-09-13T04:36:53.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:53 smithi002 ceph-mon[97651]: from='client.? 172.21.15.156:0/3801070161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41b43893-e5cc-422c-9efd-0f25edbf5d2a"}]: dispatch 2023-09-13T04:36:53.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:53 smithi002 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41b43893-e5cc-422c-9efd-0f25edbf5d2a"}]: dispatch 2023-09-13T04:36:53.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:53 smithi002 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41b43893-e5cc-422c-9efd-0f25edbf5d2a"}]': finished 2023-09-13T04:36:53.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:53 smithi002 ceph-mon[97651]: osdmap e76: 12 total, 11 up, 12 in 2023-09-13T04:36:53.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:53 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T04:36:53.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:53 smithi149 ceph-mon[133315]: osdmap e75: 11 total, 11 up, 11 in 2023-09-13T04:36:53.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:53 smithi149 ceph-mon[133315]: from='client.? 172.21.15.156:0/3801070161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41b43893-e5cc-422c-9efd-0f25edbf5d2a"}]: dispatch 2023-09-13T04:36:53.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:53 smithi149 ceph-mon[133315]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41b43893-e5cc-422c-9efd-0f25edbf5d2a"}]: dispatch 2023-09-13T04:36:53.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:53 smithi149 ceph-mon[133315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41b43893-e5cc-422c-9efd-0f25edbf5d2a"}]': finished 2023-09-13T04:36:53.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:53 smithi149 ceph-mon[133315]: osdmap e76: 12 total, 11 up, 12 in 2023-09-13T04:36:53.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:53 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T04:36:53.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:53 smithi156 ceph-mon[133675]: osdmap e75: 11 total, 11 up, 11 in 2023-09-13T04:36:53.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:53 smithi156 ceph-mon[133675]: from='client.? 172.21.15.156:0/3801070161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41b43893-e5cc-422c-9efd-0f25edbf5d2a"}]: dispatch 2023-09-13T04:36:53.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:53 smithi156 ceph-mon[133675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41b43893-e5cc-422c-9efd-0f25edbf5d2a"}]: dispatch 2023-09-13T04:36:53.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:53 smithi156 ceph-mon[133675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41b43893-e5cc-422c-9efd-0f25edbf5d2a"}]': finished 2023-09-13T04:36:53.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:53 smithi156 ceph-mon[133675]: osdmap e76: 12 total, 11 up, 12 in 2023-09-13T04:36:53.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:53 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T04:36:54.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:54 smithi002 ceph-mon[97651]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-13T04:36:54.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:54 smithi002 ceph-mon[97651]: from='client.? 172.21.15.156:0/307224818' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:36:54.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:54 smithi149 ceph-mon[133315]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-13T04:36:54.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:54 smithi149 ceph-mon[133315]: from='client.? 172.21.15.156:0/307224818' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:36:54.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:54 smithi156 ceph-mon[133675]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-13T04:36:54.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:54 smithi156 ceph-mon[133675]: from='client.? 172.21.15.156:0/307224818' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T04:36:56.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:56 smithi149 ceph-mon[133315]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 109 KiB/s, 0 objects/s recovering 2023-09-13T04:36:56.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:56 smithi156 ceph-mon[133675]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 109 KiB/s, 0 objects/s recovering 2023-09-13T04:36:57.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:56 smithi002 ceph-mon[97651]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 109 KiB/s, 0 objects/s recovering 2023-09-13T04:36:58.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:36:58 smithi156 ceph-mon[133675]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2023-09-13T04:36:58.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:36:58 smithi149 ceph-mon[133315]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2023-09-13T04:36:59.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:36:58 smithi002 ceph-mon[97651]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2023-09-13T04:37:00.294 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:00 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-13T04:37:00.295 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:00 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:00.295 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:00 smithi156 ceph-mon[133675]: Deploying daemon osd.11 on smithi156 2023-09-13T04:37:00.295 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:00 smithi156 ceph-mon[133675]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-13T04:37:00.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:00 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-13T04:37:00.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:00 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:00.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:00 smithi002 ceph-mon[97651]: Deploying daemon osd.11 on smithi156 2023-09-13T04:37:00.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:00 smithi002 ceph-mon[97651]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-13T04:37:00.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:00 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-13T04:37:00.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:00 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:00.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:00 smithi149 ceph-mon[133315]: Deploying daemon osd.11 on smithi156 2023-09-13T04:37:00.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:00 smithi149 ceph-mon[133315]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-13T04:37:02.820 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:02 smithi156 ceph-mon[133675]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-09-13T04:37:02.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:02 smithi149 ceph-mon[133315]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-09-13T04:37:03.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:02 smithi002 ceph-mon[97651]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-09-13T04:37:04.649 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:04 smithi156 ceph-mon[133675]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-09-13T04:37:04.649 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:04 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:04.649 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:04 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:04.649 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:04 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:37:04.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:04 smithi149 ceph-mon[133315]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-09-13T04:37:04.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:04 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:04.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:04 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:04.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:04 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:37:05.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:04 smithi002 ceph-mon[97651]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-09-13T04:37:05.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:04 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:05.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:04 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:05.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:04 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:37:06.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:06.352+0000 7f8c717fa700 1 -- 172.21.15.156:0/572859689 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f8c7410b590 con 0x7f8c5c0a0640 2023-09-13T04:37:06.354 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 11 on host 'smithi156' 2023-09-13T04:37:06.356 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:06.354+0000 7f8c7a749700 1 -- 172.21.15.156:0/572859689 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8c5c0a0640 msgr2=0x7f8c5c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:06.356 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:06.354+0000 7f8c7a749700 1 --2- 172.21.15.156:0/572859689 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8c5c0a0640 0x7f8c5c0a2b00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8c74100cd0 tx=0x7f8c64006d20 comp rx=0 tx=0).stop 2023-09-13T04:37:06.356 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:06.354+0000 7f8c7a749700 1 -- 172.21.15.156:0/572859689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c74101810 msgr2=0x7f8c7419e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:06.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:06.355+0000 7f8c7a749700 1 --2- 172.21.15.156:0/572859689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c74101810 0x7f8c7419e030 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00b070 tx=0x7f8c6c004db0 comp rx=0 tx=0).stop 2023-09-13T04:37:06.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:06.355+0000 7f8c7a749700 1 -- 172.21.15.156:0/572859689 shutdown_connections 2023-09-13T04:37:06.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:06.355+0000 7f8c7a749700 1 --2- 172.21.15.156:0/572859689 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8c5c0a0640 0x7f8c5c0a2b00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:06.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:06.355+0000 7f8c7a749700 1 --2- 172.21.15.156:0/572859689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c74101810 0x7f8c7419e030 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:06.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:06.355+0000 7f8c7a749700 1 --2- 172.21.15.156:0/572859689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c740ffc70 0x7f8c74069bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:06.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:06.355+0000 7f8c7a749700 1 --2- 172.21.15.156:0/572859689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c74100e70 0x7f8c7419b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:06.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:06.355+0000 7f8c7a749700 1 -- 172.21.15.156:0/572859689 >> 172.21.15.156:0/572859689 conn(0x7f8c740fb1f0 msgr2=0x7f8c741043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:06.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:06.355+0000 7f8c7a749700 1 -- 172.21.15.156:0/572859689 shutdown_connections 2023-09-13T04:37:06.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:06.355+0000 7f8c7a749700 1 -- 172.21.15.156:0/572859689 wait complete. 2023-09-13T04:37:06.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:06 smithi156 ceph-mon[133675]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-13T04:37:06.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:06.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:06.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:06.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:37:06.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:06.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:37:06.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:06.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:06 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:06.745 DEBUG:teuthology.orchestra.run.smithi156:osd.11> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.11.service 2023-09-13T04:37:06.747 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-09-13T04:37:06.747 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph osd stat -f json 2023-09-13T04:37:06.784 INFO:journalctl@ceph.osd.11.smithi156.stdout:-- Logs begin at Wed 2023-09-13 04:09:45 UTC. -- 2023-09-13T04:37:06.818 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:06 smithi002 ceph-mon[97651]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-13T04:37:06.818 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:06 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:06.819 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:06 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:06.819 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:06 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:06.819 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:06 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:37:06.819 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:06 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:06.819 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:06 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:37:06.820 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:06 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:06.820 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:06 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:06.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:06 smithi149 ceph-mon[133315]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-13T04:37:06.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:06 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:06.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:06 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:06.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:06 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:06.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:06 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:37:06.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:06 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:06.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:06 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:37:06.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:06 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:06.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:06 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:07.494 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:37:07.752 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:37:07 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:37:07.490+0000 7fc5d4510640 -1 osd.11 0 log_to_monitors true 2023-09-13T04:37:07.753 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:07 smithi156 ceph-mon[133675]: from='osd.11 [v2:172.21.15.156:6824/215467467,v1:172.21.15.156:6825/215467467]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-13T04:37:07.753 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:07 smithi156 ceph-mon[133675]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-13T04:37:07.844 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:07 smithi002 ceph-mon[97651]: from='osd.11 [v2:172.21.15.156:6824/215467467,v1:172.21.15.156:6825/215467467]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-13T04:37:07.845 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:07 smithi002 ceph-mon[97651]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-13T04:37:07.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:07 smithi149 ceph-mon[133315]: from='osd.11 [v2:172.21.15.156:6824/215467467,v1:172.21.15.156:6825/215467467]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-13T04:37:07.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:07 smithi149 ceph-mon[133315]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-13T04:37:08.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:08 smithi002 ceph-mon[97651]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-13T04:37:08.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:08 smithi002 ceph-mon[97651]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-13T04:37:08.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:08 smithi002 ceph-mon[97651]: osdmap e77: 12 total, 11 up, 12 in 2023-09-13T04:37:08.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:08 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T04:37:08.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:08 smithi002 ceph-mon[97651]: from='osd.11 [v2:172.21.15.156:6824/215467467,v1:172.21.15.156:6825/215467467]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-13T04:37:08.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:08 smithi002 ceph-mon[97651]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-13T04:37:08.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:08 smithi149 ceph-mon[133315]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-13T04:37:08.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:08 smithi149 ceph-mon[133315]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-13T04:37:08.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:08 smithi149 ceph-mon[133315]: osdmap e77: 12 total, 11 up, 12 in 2023-09-13T04:37:08.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:08 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T04:37:08.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:08 smithi149 ceph-mon[133315]: from='osd.11 [v2:172.21.15.156:6824/215467467,v1:172.21.15.156:6825/215467467]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-13T04:37:08.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:08 smithi149 ceph-mon[133315]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-13T04:37:08.936 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:37:08 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:37:08.566+0000 7fc5c9b87700 -1 osd.11 0 waiting for initial osdmap 2023-09-13T04:37:08.937 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:37:08 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:37:08.581+0000 7fc5c2976700 -1 osd.11 78 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-13T04:37:08.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:08 smithi156 ceph-mon[133675]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-13T04:37:08.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:08 smithi156 ceph-mon[133675]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-13T04:37:08.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:08 smithi156 ceph-mon[133675]: osdmap e77: 12 total, 11 up, 12 in 2023-09-13T04:37:08.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:08 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T04:37:08.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:08 smithi156 ceph-mon[133675]: from='osd.11 [v2:172.21.15.156:6824/215467467,v1:172.21.15.156:6825/215467467]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-13T04:37:08.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:08 smithi156 ceph-mon[133675]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-13T04:37:09.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.159+0000 7efefb82e700 1 -- 172.21.15.2:0/3466204137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efef4102910 msgr2=0x7efef41091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:09.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.159+0000 7efefb82e700 1 --2- 172.21.15.2:0/3466204137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efef4102910 0x7efef41091c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efeec00b0d0 tx=0x7efeec00b3e0 comp rx=0 tx=0).stop 2023-09-13T04:37:09.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.159+0000 7efefb82e700 1 -- 172.21.15.2:0/3466204137 shutdown_connections 2023-09-13T04:37:09.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.159+0000 7efefb82e700 1 --2- 172.21.15.2:0/3466204137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efef4102910 0x7efef41091c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:09.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.159+0000 7efefb82e700 1 --2- 172.21.15.2:0/3466204137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efef4100d70 0x7efef4101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:09.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.159+0000 7efefb82e700 1 --2- 172.21.15.2:0/3466204137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efef4101f70 0x7efef41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:09.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.159+0000 7efefb82e700 1 -- 172.21.15.2:0/3466204137 >> 172.21.15.2:0/3466204137 conn(0x7efef40fc2d0 msgr2=0x7efef40fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:09.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.159+0000 7efefb82e700 1 -- 172.21.15.2:0/3466204137 shutdown_connections 2023-09-13T04:37:09.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.159+0000 7efefb82e700 1 -- 172.21.15.2:0/3466204137 wait complete. 2023-09-13T04:37:09.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.160+0000 7efefb82e700 1 Processor -- start 2023-09-13T04:37:09.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.160+0000 7efefb82e700 1 -- start start 2023-09-13T04:37:09.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.161+0000 7efefb82e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efef4100d70 0x7efef4074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:09.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.161+0000 7efefb82e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efef4101f70 0x7efef40730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:09.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.161+0000 7efefb82e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efef4102910 0x7efef40735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:09.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.161+0000 7efefb82e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef4073d60 con 0x7efef4100d70 2023-09-13T04:37:09.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.161+0000 7efefb82e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef4073ed0 con 0x7efef4101f70 2023-09-13T04:37:09.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.161+0000 7efefb82e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef41a0b80 con 0x7efef4102910 2023-09-13T04:37:09.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.161+0000 7efef95ca700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efef4100d70 0x7efef4074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:09.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.161+0000 7efef95ca700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efef4100d70 0x7efef4074a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33354/0 (socket says 172.21.15.2:33354) 2023-09-13T04:37:09.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.161+0000 7efef95ca700 1 -- 172.21.15.2:0/2572707230 learned_addr learned my addr 172.21.15.2:0/2572707230 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:09.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.161+0000 7efef8dc9700 1 --2- 172.21.15.2:0/2572707230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efef4101f70 0x7efef40730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:09.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.161+0000 7efef9dcb700 1 --2- 172.21.15.2:0/2572707230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efef4102910 0x7efef40735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:09.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.162+0000 7efef95ca700 1 -- 172.21.15.2:0/2572707230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efef4102910 msgr2=0x7efef40735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:09.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.162+0000 7efef95ca700 1 --2- 172.21.15.2:0/2572707230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efef4102910 0x7efef40735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:09.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.162+0000 7efef95ca700 1 -- 172.21.15.2:0/2572707230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efef4101f70 msgr2=0x7efef40730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:09.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.162+0000 7efef95ca700 1 --2- 172.21.15.2:0/2572707230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efef4101f70 0x7efef40730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:09.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.162+0000 7efef95ca700 1 -- 172.21.15.2:0/2572707230 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec009d00 con 0x7efef4100d70 2023-09-13T04:37:09.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.162+0000 7efef9dcb700 1 --2- 172.21.15.2:0/2572707230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efef4102910 0x7efef40735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:37:09.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.162+0000 7efef95ca700 1 --2- 172.21.15.2:0/2572707230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efef4100d70 0x7efef4074a50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efef000d980 tx=0x7efef000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:09.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.162+0000 7efeea7fc700 1 -- 172.21.15.2:0/2572707230 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef000b950 con 0x7efef4100d70 2023-09-13T04:37:09.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.162+0000 7efeea7fc700 1 -- 172.21.15.2:0/2572707230 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efef0007620 con 0x7efef4100d70 2023-09-13T04:37:09.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.162+0000 7efefb82e700 1 -- 172.21.15.2:0/2572707230 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef41a0e60 con 0x7efef4100d70 2023-09-13T04:37:09.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.162+0000 7efeea7fc700 1 -- 172.21.15.2:0/2572707230 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef000f3c0 con 0x7efef4100d70 2023-09-13T04:37:09.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.162+0000 7efefb82e700 1 -- 172.21.15.2:0/2572707230 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef41a1270 con 0x7efef4100d70 2023-09-13T04:37:09.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.163+0000 7efefb82e700 1 -- 172.21.15.2:0/2572707230 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef41082d0 con 0x7efef4100d70 2023-09-13T04:37:09.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.165+0000 7efeea7fc700 1 -- 172.21.15.2:0/2572707230 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7efef0010640 con 0x7efef4100d70 2023-09-13T04:37:09.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.166+0000 7efeea7fc700 1 --2- 172.21.15.2:0/2572707230 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7efee00a0640 0x7efee00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:09.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.167+0000 7efeea7fc700 1 -- 172.21.15.2:0/2572707230 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6847+0+0 (secure 0 0 0) 0x7efef00cb570 con 0x7efef4100d70 2023-09-13T04:37:09.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.167+0000 7efef8dc9700 1 --2- 172.21.15.2:0/2572707230 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7efee00a0640 0x7efee00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:09.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.167+0000 7efef8dc9700 1 --2- 172.21.15.2:0/2572707230 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7efee00a0640 0x7efee00a2b00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efee4005950 tx=0x7efee400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:09.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.171+0000 7efeea7fc700 1 -- 172.21.15.2:0/2572707230 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7efef00955c0 con 0x7efef4100d70 2023-09-13T04:37:09.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.388+0000 7efefb82e700 1 -- 172.21.15.2:0/2572707230 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7efef410c640 con 0x7efef4100d70 2023-09-13T04:37:09.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.389+0000 7efeea7fc700 1 -- 172.21.15.2:0/2572707230 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) v1 ==== 74+0+133 (secure 0 0 0) 0x7efef0098be0 con 0x7efef4100d70 2023-09-13T04:37:09.391 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:37:09.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.391+0000 7efefb82e700 1 -- 172.21.15.2:0/2572707230 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7efee00a0640 msgr2=0x7efee00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:09.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.391+0000 7efefb82e700 1 --2- 172.21.15.2:0/2572707230 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7efee00a0640 0x7efee00a2b00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efee4005950 tx=0x7efee400b410 comp rx=0 tx=0).stop 2023-09-13T04:37:09.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.391+0000 7efefb82e700 1 -- 172.21.15.2:0/2572707230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efef4100d70 msgr2=0x7efef4074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:09.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.391+0000 7efefb82e700 1 --2- 172.21.15.2:0/2572707230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efef4100d70 0x7efef4074a50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efef000d980 tx=0x7efef000dc90 comp rx=0 tx=0).stop 2023-09-13T04:37:09.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.391+0000 7efefb82e700 1 -- 172.21.15.2:0/2572707230 shutdown_connections 2023-09-13T04:37:09.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.391+0000 7efefb82e700 1 --2- 172.21.15.2:0/2572707230 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7efee00a0640 0x7efee00a2b00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:09.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.391+0000 7efefb82e700 1 --2- 172.21.15.2:0/2572707230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efef4102910 0x7efef40735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:09.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.391+0000 7efefb82e700 1 --2- 172.21.15.2:0/2572707230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efef4100d70 0x7efef4074a50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:09.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.391+0000 7efefb82e700 1 --2- 172.21.15.2:0/2572707230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efef4101f70 0x7efef40730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:09.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.391+0000 7efefb82e700 1 -- 172.21.15.2:0/2572707230 >> 172.21.15.2:0/2572707230 conn(0x7efef40fc2d0 msgr2=0x7efef4105410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:09.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.391+0000 7efefb82e700 1 -- 172.21.15.2:0/2572707230 shutdown_connections 2023-09-13T04:37:09.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:09.392+0000 7efefb82e700 1 -- 172.21.15.2:0/2572707230 wait complete. 2023-09-13T04:37:09.573 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:09 smithi156 ceph-mon[133675]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-09-13T04:37:09.573 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:09 smithi156 ceph-mon[133675]: osdmap e78: 12 total, 11 up, 12 in 2023-09-13T04:37:09.573 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:09 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T04:37:09.573 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:09 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T04:37:09.574 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:09 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:09.574 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:09 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:09.574 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:09 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:37:09.574 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:09 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:37:09.575 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:09 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:37:09.575 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:09 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:37:09.575 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:09 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:09.575 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:09 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:09.575 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:09 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:37:09.576 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:09 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:09.576 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:09 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2572707230' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-13T04:37:09.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:09 smithi002 ceph-mon[97651]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-09-13T04:37:09.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:09 smithi002 ceph-mon[97651]: osdmap e78: 12 total, 11 up, 12 in 2023-09-13T04:37:09.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:09 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T04:37:09.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:09 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T04:37:09.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:09 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:09.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:09 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:09.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:09 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:37:09.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:09 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:37:09.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:09 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:37:09.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:09 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:37:09.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:09 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:09.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:09 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:09.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:09 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:37:09.760 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:09 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:09.760 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:09 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2572707230' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-13T04:37:09.882 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":78,"num_osds":12,"num_up_osds":11,"osd_up_since":1694579809,"num_in_osds":12,"osd_in_since":1694579812,"num_remapped_pgs":0} 2023-09-13T04:37:09.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:09 smithi149 ceph-mon[133315]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-09-13T04:37:09.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:09 smithi149 ceph-mon[133315]: osdmap e78: 12 total, 11 up, 12 in 2023-09-13T04:37:09.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:09 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T04:37:09.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:09 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T04:37:09.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:09 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:09.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:09 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:09.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:09 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:37:09.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:09 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:37:09.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:09 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:37:09.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:09 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:37:09.890 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:09 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:09.890 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:09 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:09.890 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:09 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:37:09.891 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:09 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:09.891 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:09 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2572707230' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-13T04:37:10.884 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph osd stat -f json 2023-09-13T04:37:10.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:10 smithi149 ceph-mon[133315]: purged_snaps scrub starts 2023-09-13T04:37:10.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:10 smithi149 ceph-mon[133315]: purged_snaps scrub ok 2023-09-13T04:37:10.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:10 smithi149 ceph-mon[133315]: Detected new or changed devices on smithi156 2023-09-13T04:37:10.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:10 smithi149 ceph-mon[133315]: Adjusting osd_memory_target on smithi156 to 4292M 2023-09-13T04:37:10.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:10 smithi149 ceph-mon[133315]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-13T04:37:10.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:10 smithi149 ceph-mon[133315]: osd.11 [v2:172.21.15.156:6824/215467467,v1:172.21.15.156:6825/215467467] boot 2023-09-13T04:37:10.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:10 smithi149 ceph-mon[133315]: osdmap e79: 12 total, 12 up, 12 in 2023-09-13T04:37:10.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:10 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T04:37:10.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:10 smithi156 ceph-mon[133675]: purged_snaps scrub starts 2023-09-13T04:37:10.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:10 smithi156 ceph-mon[133675]: purged_snaps scrub ok 2023-09-13T04:37:10.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:10 smithi156 ceph-mon[133675]: Detected new or changed devices on smithi156 2023-09-13T04:37:10.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:10 smithi156 ceph-mon[133675]: Adjusting osd_memory_target on smithi156 to 4292M 2023-09-13T04:37:10.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:10 smithi156 ceph-mon[133675]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-13T04:37:10.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:10 smithi156 ceph-mon[133675]: osd.11 [v2:172.21.15.156:6824/215467467,v1:172.21.15.156:6825/215467467] boot 2023-09-13T04:37:10.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:10 smithi156 ceph-mon[133675]: osdmap e79: 12 total, 12 up, 12 in 2023-09-13T04:37:10.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:10 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T04:37:10.956 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:10 smithi002 ceph-mon[97651]: purged_snaps scrub starts 2023-09-13T04:37:10.956 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:10 smithi002 ceph-mon[97651]: purged_snaps scrub ok 2023-09-13T04:37:10.956 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:10 smithi002 ceph-mon[97651]: Detected new or changed devices on smithi156 2023-09-13T04:37:10.956 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:10 smithi002 ceph-mon[97651]: Adjusting osd_memory_target on smithi156 to 4292M 2023-09-13T04:37:10.956 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:10 smithi002 ceph-mon[97651]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-13T04:37:10.957 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:10 smithi002 ceph-mon[97651]: osd.11 [v2:172.21.15.156:6824/215467467,v1:172.21.15.156:6825/215467467] boot 2023-09-13T04:37:10.957 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:10 smithi002 ceph-mon[97651]: osdmap e79: 12 total, 12 up, 12 in 2023-09-13T04:37:10.957 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:10 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T04:37:11.632 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:37:12.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:12 smithi002 ceph-mon[97651]: osdmap e80: 12 total, 12 up, 12 in 2023-09-13T04:37:12.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:12 smithi002 ceph-mon[97651]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:12.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:12 smithi149 ceph-mon[133315]: osdmap e80: 12 total, 12 up, 12 in 2023-09-13T04:37:12.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:12 smithi149 ceph-mon[133315]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:12.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:12 smithi156 ceph-mon[133675]: osdmap e80: 12 total, 12 up, 12 in 2023-09-13T04:37:12.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:12 smithi156 ceph-mon[133675]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:13.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.284+0000 7f2ce5c46700 1 -- 172.21.15.2:0/3580374441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ce0100d70 msgr2=0x7f2ce0101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:13.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.284+0000 7f2ce5c46700 1 --2- 172.21.15.2:0/3580374441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ce0100d70 0x7f2ce0101190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00099c0 tx=0x7f2cd0009cd0 comp rx=0 tx=0).stop 2023-09-13T04:37:13.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.284+0000 7f2ce5c46700 1 -- 172.21.15.2:0/3580374441 shutdown_connections 2023-09-13T04:37:13.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.284+0000 7f2ce5c46700 1 --2- 172.21.15.2:0/3580374441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ce0102910 0x7f2ce01091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:13.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.284+0000 7f2ce5c46700 1 --2- 172.21.15.2:0/3580374441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ce0100d70 0x7f2ce0101190 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:13.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.284+0000 7f2ce5c46700 1 --2- 172.21.15.2:0/3580374441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ce0101f70 0x7f2ce01023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:13.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.284+0000 7f2ce5c46700 1 -- 172.21.15.2:0/3580374441 >> 172.21.15.2:0/3580374441 conn(0x7f2ce00fc2d0 msgr2=0x7f2ce00fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:13.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.284+0000 7f2ce5c46700 1 -- 172.21.15.2:0/3580374441 shutdown_connections 2023-09-13T04:37:13.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.284+0000 7f2ce5c46700 1 -- 172.21.15.2:0/3580374441 wait complete. 2023-09-13T04:37:13.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.285+0000 7f2ce5c46700 1 Processor -- start 2023-09-13T04:37:13.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.286+0000 7f2ce5c46700 1 -- start start 2023-09-13T04:37:13.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.286+0000 7f2ce5c46700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ce0100d70 0x7f2ce0196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:13.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.286+0000 7f2ce5c46700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ce0101f70 0x7f2ce0196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:13.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.286+0000 7f2ce5c46700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ce0102910 0x7f2ce019e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:13.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.286+0000 7f2ce5c46700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce019e610 con 0x7f2ce0100d70 2023-09-13T04:37:13.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.286+0000 7f2ce5c46700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce019e750 con 0x7f2ce0102910 2023-09-13T04:37:13.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.286+0000 7f2ce5c46700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce019e8e0 con 0x7f2ce0101f70 2023-09-13T04:37:13.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.286+0000 7f2cdf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ce0100d70 0x7f2ce0196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:13.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.286+0000 7f2cdf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ce0100d70 0x7f2ce0196a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33378/0 (socket says 172.21.15.2:33378) 2023-09-13T04:37:13.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.286+0000 7f2cdf7fe700 1 -- 172.21.15.2:0/7308145 learned_addr learned my addr 172.21.15.2:0/7308145 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:13.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.286+0000 7f2cdeffd700 1 --2- 172.21.15.2:0/7308145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ce0101f70 0x7f2ce0196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:13.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.286+0000 7f2cdffff700 1 --2- 172.21.15.2:0/7308145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ce0102910 0x7f2ce019e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:13.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.287+0000 7f2cdf7fe700 1 -- 172.21.15.2:0/7308145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ce0101f70 msgr2=0x7f2ce0196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:13.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.287+0000 7f2cdf7fe700 1 --2- 172.21.15.2:0/7308145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ce0101f70 0x7f2ce0196fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:13.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.287+0000 7f2cdf7fe700 1 -- 172.21.15.2:0/7308145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ce0102910 msgr2=0x7f2ce019e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:13.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.287+0000 7f2cdf7fe700 1 --2- 172.21.15.2:0/7308145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ce0102910 0x7f2ce019e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:13.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.287+0000 7f2cdf7fe700 1 -- 172.21.15.2:0/7308145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd00095f0 con 0x7f2ce0100d70 2023-09-13T04:37:13.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.287+0000 7f2cdeffd700 1 --2- 172.21.15.2:0/7308145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ce0101f70 0x7f2ce0196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:37:13.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.287+0000 7f2cdffff700 1 --2- 172.21.15.2:0/7308145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ce0102910 0x7f2ce019e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:37:13.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.287+0000 7f2cdf7fe700 1 --2- 172.21.15.2:0/7308145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ce0100d70 0x7f2ce0196a80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00098e0 tx=0x7f2cd000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:13.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.287+0000 7f2cdcff9700 1 -- 172.21.15.2:0/7308145 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd001d070 con 0x7f2ce0100d70 2023-09-13T04:37:13.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.287+0000 7f2cdcff9700 1 -- 172.21.15.2:0/7308145 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cd00054f0 con 0x7f2ce0100d70 2023-09-13T04:37:13.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.287+0000 7f2cdcff9700 1 -- 172.21.15.2:0/7308145 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0004bd0 con 0x7f2ce0100d70 2023-09-13T04:37:13.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.287+0000 7f2ce5c46700 1 -- 172.21.15.2:0/7308145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce019eb60 con 0x7f2ce0100d70 2023-09-13T04:37:13.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.287+0000 7f2ce5c46700 1 -- 172.21.15.2:0/7308145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce019ef20 con 0x7f2ce0100d70 2023-09-13T04:37:13.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.289+0000 7f2ce5c46700 1 -- 172.21.15.2:0/7308145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce0108360 con 0x7f2ce0100d70 2023-09-13T04:37:13.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.290+0000 7f2cdcff9700 1 -- 172.21.15.2:0/7308145 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f2cd000fbf0 con 0x7f2ce0100d70 2023-09-13T04:37:13.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.293+0000 7f2cdcff9700 1 --2- 172.21.15.2:0/7308145 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2cc80a06c0 0x7f2cc80a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:13.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.293+0000 7f2cdcff9700 1 -- 172.21.15.2:0/7308145 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2cd00cc560 con 0x7f2ce0100d70 2023-09-13T04:37:13.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.293+0000 7f2cdeffd700 1 --2- 172.21.15.2:0/7308145 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2cc80a06c0 0x7f2cc80a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:13.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.294+0000 7f2cdeffd700 1 --2- 172.21.15.2:0/7308145 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2cc80a06c0 0x7f2cc80a2b80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009cb0 tx=0x7f2cd4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:13.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.297+0000 7f2cdcff9700 1 -- 172.21.15.2:0/7308145 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2cd0096520 con 0x7f2ce0100d70 2023-09-13T04:37:13.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.508+0000 7f2ce5c46700 1 -- 172.21.15.2:0/7308145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f2ce010c5c0 con 0x7f2ce0100d70 2023-09-13T04:37:13.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.509+0000 7f2cdcff9700 1 -- 172.21.15.2:0/7308145 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7f2cd0099b40 con 0x7f2ce0100d70 2023-09-13T04:37:13.510 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:37:13.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.511+0000 7f2ce5c46700 1 -- 172.21.15.2:0/7308145 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2cc80a06c0 msgr2=0x7f2cc80a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:13.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.511+0000 7f2ce5c46700 1 --2- 172.21.15.2:0/7308145 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2cc80a06c0 0x7f2cc80a2b80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009cb0 tx=0x7f2cd4009260 comp rx=0 tx=0).stop 2023-09-13T04:37:13.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.511+0000 7f2ce5c46700 1 -- 172.21.15.2:0/7308145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ce0100d70 msgr2=0x7f2ce0196a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:13.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.511+0000 7f2ce5c46700 1 --2- 172.21.15.2:0/7308145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ce0100d70 0x7f2ce0196a80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00098e0 tx=0x7f2cd000f740 comp rx=0 tx=0).stop 2023-09-13T04:37:13.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.511+0000 7f2ce5c46700 1 -- 172.21.15.2:0/7308145 shutdown_connections 2023-09-13T04:37:13.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.511+0000 7f2ce5c46700 1 --2- 172.21.15.2:0/7308145 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2cc80a06c0 0x7f2cc80a2b80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:13.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.511+0000 7f2ce5c46700 1 --2- 172.21.15.2:0/7308145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ce0102910 0x7f2ce019e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:13.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.511+0000 7f2ce5c46700 1 --2- 172.21.15.2:0/7308145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ce0100d70 0x7f2ce0196a80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:13.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.511+0000 7f2ce5c46700 1 --2- 172.21.15.2:0/7308145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ce0101f70 0x7f2ce0196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:13.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.511+0000 7f2ce5c46700 1 -- 172.21.15.2:0/7308145 >> 172.21.15.2:0/7308145 conn(0x7f2ce00fc2d0 msgr2=0x7f2ce01054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:13.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.511+0000 7f2ce5c46700 1 -- 172.21.15.2:0/7308145 shutdown_connections 2023-09-13T04:37:13.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:13.511+0000 7f2ce5c46700 1 -- 172.21.15.2:0/7308145 wait complete. 2023-09-13T04:37:13.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:13 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/7308145' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-13T04:37:13.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:13 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/7308145' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-13T04:37:13.941 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:13 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/7308145' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-13T04:37:14.013 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1694579829,"num_in_osds":12,"osd_in_since":1694579812,"num_remapped_pgs":0} 2023-09-13T04:37:14.013 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph osd dump --format=json 2023-09-13T04:37:14.750 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:37:14.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:14 smithi149 ceph-mon[133315]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:14.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:14 smithi156 ceph-mon[133675]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:15.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:14 smithi002 ceph-mon[97651]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:16.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.284+0000 7fcae3e70700 1 -- 172.21.15.2:0/3363285422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcadc106d90 msgr2=0x7fcadc107170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:16.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.284+0000 7fcae3e70700 1 --2- 172.21.15.2:0/3363285422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcadc106d90 0x7fcadc107170 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcad80099c0 tx=0x7fcad8009cd0 comp rx=0 tx=0).stop 2023-09-13T04:37:16.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.284+0000 7fcae3e70700 1 -- 172.21.15.2:0/3363285422 shutdown_connections 2023-09-13T04:37:16.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.284+0000 7fcae3e70700 1 --2- 172.21.15.2:0/3363285422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcadc101740 0x7fcadc10dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:16.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.284+0000 7fcae3e70700 1 --2- 172.21.15.2:0/3363285422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcadc106d90 0x7fcadc107170 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:16.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.284+0000 7fcae3e70700 1 --2- 172.21.15.2:0/3363285422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcadc100d80 0x7fcadc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:16.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.284+0000 7fcae3e70700 1 -- 172.21.15.2:0/3363285422 >> 172.21.15.2:0/3363285422 conn(0x7fcadc077ea0 msgr2=0x7fcadc0782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:16.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.285+0000 7fcae3e70700 1 -- 172.21.15.2:0/3363285422 shutdown_connections 2023-09-13T04:37:16.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.285+0000 7fcae3e70700 1 -- 172.21.15.2:0/3363285422 wait complete. 2023-09-13T04:37:16.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.286+0000 7fcae3e70700 1 Processor -- start 2023-09-13T04:37:16.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.286+0000 7fcae3e70700 1 -- start start 2023-09-13T04:37:16.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.287+0000 7fcae3e70700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcadc100d80 0x7fcadc199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:16.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.287+0000 7fcae3e70700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcadc101740 0x7fcadc199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:16.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.287+0000 7fcae3e70700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcadc106d90 0x7fcadc19d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:16.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.287+0000 7fcae3e70700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc19dfc0 con 0x7fcadc106d90 2023-09-13T04:37:16.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.287+0000 7fcae3e70700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc19e130 con 0x7fcadc100d80 2023-09-13T04:37:16.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.287+0000 7fcae3e70700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc19e2f0 con 0x7fcadc101740 2023-09-13T04:37:16.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.287+0000 7fcae240d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcadc106d90 0x7fcadc19d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:16.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.287+0000 7fcae1c0c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcadc100d80 0x7fcadc199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:16.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.287+0000 7fcae240d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcadc106d90 0x7fcadc19d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46234/0 (socket says 172.21.15.2:46234) 2023-09-13T04:37:16.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.287+0000 7fcae240d700 1 -- 172.21.15.2:0/1363504032 learned_addr learned my addr 172.21.15.2:0/1363504032 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:16.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.287+0000 7fcae140b700 1 --2- 172.21.15.2:0/1363504032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcadc101740 0x7fcadc199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:16.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.288+0000 7fcae240d700 1 -- 172.21.15.2:0/1363504032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcadc101740 msgr2=0x7fcadc199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:16.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.288+0000 7fcae240d700 1 --2- 172.21.15.2:0/1363504032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcadc101740 0x7fcadc199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:16.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.288+0000 7fcae240d700 1 -- 172.21.15.2:0/1363504032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcadc100d80 msgr2=0x7fcadc199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:16.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.288+0000 7fcae240d700 1 --2- 172.21.15.2:0/1363504032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcadc100d80 0x7fcadc199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:16.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.288+0000 7fcae240d700 1 -- 172.21.15.2:0/1363504032 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad80095f0 con 0x7fcadc106d90 2023-09-13T04:37:16.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.288+0000 7fcae1c0c700 1 --2- 172.21.15.2:0/1363504032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcadc100d80 0x7fcadc199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:37:16.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.288+0000 7fcae240d700 1 --2- 172.21.15.2:0/1363504032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcadc106d90 0x7fcadc19d9b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcad400efc0 tx=0x7fcad400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:16.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.288+0000 7fcad2ffd700 1 -- 172.21.15.2:0/1363504032 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad4018070 con 0x7fcadc106d90 2023-09-13T04:37:16.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.288+0000 7fcae3e70700 1 -- 172.21.15.2:0/1363504032 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcadc19e570 con 0x7fcadc106d90 2023-09-13T04:37:16.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.288+0000 7fcad2ffd700 1 -- 172.21.15.2:0/1363504032 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcad400f040 con 0x7fcadc106d90 2023-09-13T04:37:16.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.288+0000 7fcad2ffd700 1 -- 172.21.15.2:0/1363504032 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad400cbc0 con 0x7fcadc106d90 2023-09-13T04:37:16.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.288+0000 7fcae3e70700 1 -- 172.21.15.2:0/1363504032 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcadc1a5e30 con 0x7fcadc106d90 2023-09-13T04:37:16.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.289+0000 7fcae3e70700 1 -- 172.21.15.2:0/1363504032 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcadc04e9b0 con 0x7fcadc106d90 2023-09-13T04:37:16.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.291+0000 7fcad2ffd700 1 -- 172.21.15.2:0/1363504032 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fcad400cd40 con 0x7fcadc106d90 2023-09-13T04:37:16.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.293+0000 7fcad2ffd700 1 --2- 172.21.15.2:0/1363504032 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fcac80a0640 0x7fcac80a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:16.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.293+0000 7fcad2ffd700 1 -- 172.21.15.2:0/1363504032 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcad40cb9c0 con 0x7fcadc106d90 2023-09-13T04:37:16.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.293+0000 7fcae1c0c700 1 --2- 172.21.15.2:0/1363504032 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fcac80a0640 0x7fcac80a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:16.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.294+0000 7fcae1c0c700 1 --2- 172.21.15.2:0/1363504032 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fcac80a0640 0x7fcac80a2b00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcad8000c00 tx=0x7fcad8019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:16.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.295+0000 7fcad2ffd700 1 -- 172.21.15.2:0/1363504032 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcad4095980 con 0x7fcadc106d90 2023-09-13T04:37:16.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.504+0000 7fcae3e70700 1 -- 172.21.15.2:0/1363504032 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcadc110d50 con 0x7fcadc106d90 2023-09-13T04:37:16.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.505+0000 7fcad2ffd700 1 -- 172.21.15.2:0/1363504032 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18389 (secure 0 0 0) 0x7fcad4098fa0 con 0x7fcadc106d90 2023-09-13T04:37:16.506 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:37:16.506 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":80,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:37:11.321587+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:37:16.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.508+0000 7fcae3e70700 1 -- 172.21.15.2:0/1363504032 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fcac80a0640 msgr2=0x7fcac80a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:16.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.508+0000 7fcae3e70700 1 --2- 172.21.15.2:0/1363504032 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fcac80a0640 0x7fcac80a2b00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcad8000c00 tx=0x7fcad8019040 comp rx=0 tx=0).stop 2023-09-13T04:37:16.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.508+0000 7fcae3e70700 1 -- 172.21.15.2:0/1363504032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcadc106d90 msgr2=0x7fcadc19d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:16.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.508+0000 7fcae3e70700 1 --2- 172.21.15.2:0/1363504032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcadc106d90 0x7fcadc19d9b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcad400efc0 tx=0x7fcad400c3a0 comp rx=0 tx=0).stop 2023-09-13T04:37:16.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.508+0000 7fcae3e70700 1 -- 172.21.15.2:0/1363504032 shutdown_connections 2023-09-13T04:37:16.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.508+0000 7fcae3e70700 1 --2- 172.21.15.2:0/1363504032 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fcac80a0640 0x7fcac80a2b00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:16.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.508+0000 7fcae3e70700 1 --2- 172.21.15.2:0/1363504032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcadc106d90 0x7fcadc19d9b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:16.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.508+0000 7fcae3e70700 1 --2- 172.21.15.2:0/1363504032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcadc100d80 0x7fcadc199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:16.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.508+0000 7fcae3e70700 1 --2- 172.21.15.2:0/1363504032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcadc101740 0x7fcadc199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:16.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.508+0000 7fcae3e70700 1 -- 172.21.15.2:0/1363504032 >> 172.21.15.2:0/1363504032 conn(0x7fcadc077ea0 msgr2=0x7fcadc10bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:16.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.508+0000 7fcae3e70700 1 -- 172.21.15.2:0/1363504032 shutdown_connections 2023-09-13T04:37:16.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:16.508+0000 7fcae3e70700 1 -- 172.21.15.2:0/1363504032 wait complete. 2023-09-13T04:37:16.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:16 smithi149 ceph-mon[133315]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:16.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:16 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1363504032' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:37:16.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:16 smithi156 ceph-mon[133675]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:16.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:16 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1363504032' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:37:16.991 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:16 smithi002 ceph-mon[97651]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:16.991 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:16 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1363504032' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:37:17.034 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T04:33:42.025714+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-13T04:37:17.034 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph osd pool get .mgr pg_num 2023-09-13T04:37:17.767 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:37:18.681 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:18 smithi002 ceph-mon[97651]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:18.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:18 smithi149 ceph-mon[133315]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:18.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:18 smithi156 ceph-mon[133675]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:19.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.394+0000 7fc52373c700 1 -- 172.21.15.2:0/1945268203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc51c072ee0 msgr2=0x7fc51c0732c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:19.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.394+0000 7fc52373c700 1 --2- 172.21.15.2:0/1945268203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc51c072ee0 0x7fc51c0732c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc518009990 tx=0x7fc518009ca0 comp rx=0 tx=0).stop 2023-09-13T04:37:19.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.395+0000 7fc52373c700 1 -- 172.21.15.2:0/1945268203 shutdown_connections 2023-09-13T04:37:19.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.395+0000 7fc52373c700 1 --2- 172.21.15.2:0/1945268203 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc51c10d730 0x7fc51c10fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:19.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.395+0000 7fc52373c700 1 --2- 172.21.15.2:0/1945268203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc51c072ee0 0x7fc51c0732c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:19.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.395+0000 7fc52373c700 1 --2- 172.21.15.2:0/1945268203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc51c073800 0x7fc51c10d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:19.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.395+0000 7fc52373c700 1 -- 172.21.15.2:0/1945268203 >> 172.21.15.2:0/1945268203 conn(0x7fc51c077ea0 msgr2=0x7fc51c0782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:19.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.395+0000 7fc52373c700 1 -- 172.21.15.2:0/1945268203 shutdown_connections 2023-09-13T04:37:19.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.395+0000 7fc52373c700 1 -- 172.21.15.2:0/1945268203 wait complete. 2023-09-13T04:37:19.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.396+0000 7fc52373c700 1 Processor -- start 2023-09-13T04:37:19.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.396+0000 7fc52373c700 1 -- start start 2023-09-13T04:37:19.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.396+0000 7fc52373c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc51c072ee0 0x7fc51c199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:19.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.396+0000 7fc52373c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc51c073800 0x7fc51c199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:19.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.396+0000 7fc52373c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc51c10d730 0x7fc51c19da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:19.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.396+0000 7fc52373c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c19e010 con 0x7fc51c072ee0 2023-09-13T04:37:19.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.396+0000 7fc52373c700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c19e180 con 0x7fc51c10d730 2023-09-13T04:37:19.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.396+0000 7fc52373c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c19e340 con 0x7fc51c073800 2023-09-13T04:37:19.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.397+0000 7fc5214d8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc51c072ee0 0x7fc51c199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:19.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.397+0000 7fc5214d8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc51c072ee0 0x7fc51c199050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46252/0 (socket says 172.21.15.2:46252) 2023-09-13T04:37:19.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.397+0000 7fc5214d8700 1 -- 172.21.15.2:0/2135309663 learned_addr learned my addr 172.21.15.2:0/2135309663 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:19.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.397+0000 7fc521cd9700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc51c10d730 0x7fc51c19da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:19.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.397+0000 7fc520cd7700 1 --2- 172.21.15.2:0/2135309663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc51c073800 0x7fc51c199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:19.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.397+0000 7fc5214d8700 1 -- 172.21.15.2:0/2135309663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc51c073800 msgr2=0x7fc51c199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:19.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.397+0000 7fc5214d8700 1 --2- 172.21.15.2:0/2135309663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc51c073800 0x7fc51c199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:19.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.397+0000 7fc5214d8700 1 -- 172.21.15.2:0/2135309663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc51c10d730 msgr2=0x7fc51c19da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:19.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.397+0000 7fc5214d8700 1 --2- 172.21.15.2:0/2135309663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc51c10d730 0x7fc51c19da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:19.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.397+0000 7fc5214d8700 1 -- 172.21.15.2:0/2135309663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5180095f0 con 0x7fc51c072ee0 2023-09-13T04:37:19.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.397+0000 7fc5214d8700 1 --2- 172.21.15.2:0/2135309663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc51c072ee0 0x7fc51c199050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc518000c00 tx=0x7fc518004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:19.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.397+0000 7fc5127fc700 1 -- 172.21.15.2:0/2135309663 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51801d070 con 0x7fc51c072ee0 2023-09-13T04:37:19.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.398+0000 7fc52373c700 1 -- 172.21.15.2:0/2135309663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc51c19e5c0 con 0x7fc51c072ee0 2023-09-13T04:37:19.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.398+0000 7fc5127fc700 1 -- 172.21.15.2:0/2135309663 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc518021410 con 0x7fc51c072ee0 2023-09-13T04:37:19.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.398+0000 7fc5127fc700 1 -- 172.21.15.2:0/2135309663 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc518017650 con 0x7fc51c072ee0 2023-09-13T04:37:19.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.398+0000 7fc52373c700 1 -- 172.21.15.2:0/2135309663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc51c1a5e80 con 0x7fc51c072ee0 2023-09-13T04:37:19.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.399+0000 7fc52373c700 1 -- 172.21.15.2:0/2135309663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc51c04e9b0 con 0x7fc51c072ee0 2023-09-13T04:37:19.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.400+0000 7fc5127fc700 1 -- 172.21.15.2:0/2135309663 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fc518021580 con 0x7fc51c072ee0 2023-09-13T04:37:19.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.402+0000 7fc5127fc700 1 --2- 172.21.15.2:0/2135309663 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc5080a06c0 0x7fc5080a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:19.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.402+0000 7fc5127fc700 1 -- 172.21.15.2:0/2135309663 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc5180cca90 con 0x7fc51c072ee0 2023-09-13T04:37:19.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.402+0000 7fc520cd7700 1 --2- 172.21.15.2:0/2135309663 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc5080a06c0 0x7fc5080a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:19.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.403+0000 7fc520cd7700 1 --2- 172.21.15.2:0/2135309663 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc5080a06c0 0x7fc5080a2b80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc51c19a670 tx=0x7fc50c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:19.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.406+0000 7fc5127fc700 1 -- 172.21.15.2:0/2135309663 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc518096a50 con 0x7fc51c072ee0 2023-09-13T04:37:19.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.614+0000 7fc52373c700 1 -- 172.21.15.2:0/2135309663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc51c19a3b0 con 0x7fc51c072ee0 2023-09-13T04:37:19.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.615+0000 7fc5127fc700 1 -- 172.21.15.2:0/2135309663 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc51809a070 con 0x7fc51c072ee0 2023-09-13T04:37:19.616 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-09-13T04:37:19.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.617+0000 7fc52373c700 1 -- 172.21.15.2:0/2135309663 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc5080a06c0 msgr2=0x7fc5080a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:19.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.617+0000 7fc52373c700 1 --2- 172.21.15.2:0/2135309663 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc5080a06c0 0x7fc5080a2b80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc51c19a670 tx=0x7fc50c009260 comp rx=0 tx=0).stop 2023-09-13T04:37:19.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.617+0000 7fc52373c700 1 -- 172.21.15.2:0/2135309663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc51c072ee0 msgr2=0x7fc51c199050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:19.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.617+0000 7fc52373c700 1 --2- 172.21.15.2:0/2135309663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc51c072ee0 0x7fc51c199050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc518000c00 tx=0x7fc518004790 comp rx=0 tx=0).stop 2023-09-13T04:37:19.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.617+0000 7fc52373c700 1 -- 172.21.15.2:0/2135309663 shutdown_connections 2023-09-13T04:37:19.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.617+0000 7fc52373c700 1 --2- 172.21.15.2:0/2135309663 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc5080a06c0 0x7fc5080a2b80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:19.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.617+0000 7fc52373c700 1 --2- 172.21.15.2:0/2135309663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc51c10d730 0x7fc51c19da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:19.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.617+0000 7fc52373c700 1 --2- 172.21.15.2:0/2135309663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc51c072ee0 0x7fc51c199050 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:19.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.617+0000 7fc52373c700 1 --2- 172.21.15.2:0/2135309663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc51c073800 0x7fc51c199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:19.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.617+0000 7fc52373c700 1 -- 172.21.15.2:0/2135309663 >> 172.21.15.2:0/2135309663 conn(0x7fc51c077ea0 msgr2=0x7fc51c0fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:19.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.617+0000 7fc52373c700 1 -- 172.21.15.2:0/2135309663 shutdown_connections 2023-09-13T04:37:19.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:19.617+0000 7fc52373c700 1 -- 172.21.15.2:0/2135309663 wait complete. 2023-09-13T04:37:20.146 INFO:tasks.cephadm:Adding mds.a on smithi002 2023-09-13T04:37:20.146 INFO:tasks.cephadm:Adding mds.d on smithi002 2023-09-13T04:37:20.147 INFO:tasks.cephadm:Adding mds.g on smithi002 2023-09-13T04:37:20.147 INFO:tasks.cephadm:Adding mds.j on smithi002 2023-09-13T04:37:20.147 INFO:tasks.cephadm:Adding mds.b on smithi149 2023-09-13T04:37:20.147 INFO:tasks.cephadm:Adding mds.e on smithi149 2023-09-13T04:37:20.148 INFO:tasks.cephadm:Adding mds.h on smithi149 2023-09-13T04:37:20.148 INFO:tasks.cephadm:Adding mds.k on smithi149 2023-09-13T04:37:20.148 INFO:tasks.cephadm:Adding mds.c on smithi156 2023-09-13T04:37:20.148 INFO:tasks.cephadm:Adding mds.f on smithi156 2023-09-13T04:37:20.149 INFO:tasks.cephadm:Adding mds.i on smithi156 2023-09-13T04:37:20.149 INFO:tasks.cephadm:Adding mds.l on smithi156 2023-09-13T04:37:20.149 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph orch apply mds all '12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l' 2023-09-13T04:37:20.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:20 smithi156 ceph-mon[133675]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:20.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:20 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2135309663' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:37:20.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:20 smithi002 ceph-mon[97651]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:20.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:20 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2135309663' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:37:20.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:20 smithi149 ceph-mon[133315]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:20.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:20 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2135309663' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:37:20.720 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-13T04:37:21.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.903+0000 7f3f7eb11700 1 -- 172.21.15.156:0/284842650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f780ffec0 msgr2=0x7f3f78100340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:21.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.903+0000 7f3f7eb11700 1 --2- 172.21.15.156:0/284842650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f780ffec0 0x7f3f78100340 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3f64009990 tx=0x7f3f64009ca0 comp rx=0 tx=0).stop 2023-09-13T04:37:21.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.903+0000 7f3f7eb11700 1 -- 172.21.15.156:0/284842650 shutdown_connections 2023-09-13T04:37:21.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.903+0000 7f3f7eb11700 1 --2- 172.21.15.156:0/284842650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f78100880 0x7f3f78108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:21.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.903+0000 7f3f7eb11700 1 --2- 172.21.15.156:0/284842650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f781012f0 0x7f3f78101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:21.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.903+0000 7f3f7eb11700 1 --2- 172.21.15.156:0/284842650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f780ffec0 0x7f3f78100340 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:21.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.903+0000 7f3f7eb11700 1 -- 172.21.15.156:0/284842650 >> 172.21.15.156:0/284842650 conn(0x7f3f780fa970 msgr2=0x7f3f780fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:21.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.904+0000 7f3f7eb11700 1 -- 172.21.15.156:0/284842650 shutdown_connections 2023-09-13T04:37:21.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.904+0000 7f3f7eb11700 1 -- 172.21.15.156:0/284842650 wait complete. 2023-09-13T04:37:21.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.904+0000 7f3f7eb11700 1 Processor -- start 2023-09-13T04:37:21.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.904+0000 7f3f7eb11700 1 -- start start 2023-09-13T04:37:21.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.904+0000 7f3f7eb11700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f780ffec0 0x7f3f78103c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:21.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.904+0000 7f3f7eb11700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f78100880 0x7f3f781022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:21.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.904+0000 7f3f7eb11700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f781012f0 0x7f3f781027f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:21.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.904+0000 7f3f7eb11700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f78102f70 con 0x7f3f78100880 2023-09-13T04:37:21.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.904+0000 7f3f7eb11700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f781030e0 con 0x7f3f781012f0 2023-09-13T04:37:21.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.904+0000 7f3f7eb11700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f781a0aa0 con 0x7f3f780ffec0 2023-09-13T04:37:21.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.904+0000 7f3f7c8ad700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f780ffec0 0x7f3f78103c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:21.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.904+0000 7f3f7c8ad700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f780ffec0 0x7f3f78103c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37550/0 (socket says 172.21.15.156:37550) 2023-09-13T04:37:21.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.904+0000 7f3f7c8ad700 1 -- 172.21.15.156:0/3590865289 learned_addr learned my addr 172.21.15.156:0/3590865289 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-13T04:37:21.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.905+0000 7f3f7d0ae700 1 --2- 172.21.15.156:0/3590865289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f781012f0 0x7f3f781027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:21.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.905+0000 7f3f7c8ad700 1 -- 172.21.15.156:0/3590865289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f781012f0 msgr2=0x7f3f781027f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:21.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.905+0000 7f3f7c8ad700 1 --2- 172.21.15.156:0/3590865289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f781012f0 0x7f3f781027f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:21.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.905+0000 7f3f77fff700 1 --2- 172.21.15.156:0/3590865289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f78100880 0x7f3f781022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:21.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.905+0000 7f3f7c8ad700 1 -- 172.21.15.156:0/3590865289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f78100880 msgr2=0x7f3f781022b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:21.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.905+0000 7f3f7c8ad700 1 --2- 172.21.15.156:0/3590865289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f78100880 0x7f3f781022b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:21.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.905+0000 7f3f7c8ad700 1 -- 172.21.15.156:0/3590865289 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f640095f0 con 0x7f3f780ffec0 2023-09-13T04:37:21.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.905+0000 7f3f7d0ae700 1 --2- 172.21.15.156:0/3590865289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f781012f0 0x7f3f781027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:37:21.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.905+0000 7f3f7c8ad700 1 --2- 172.21.15.156:0/3590865289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f780ffec0 0x7f3f78103c60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00d950 tx=0x7f3f6c00dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:21.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.905+0000 7f3f75ffb700 1 -- 172.21.15.156:0/3590865289 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c009b40 con 0x7f3f780ffec0 2023-09-13T04:37:21.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.905+0000 7f3f75ffb700 1 -- 172.21.15.156:0/3590865289 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f6c009ca0 con 0x7f3f780ffec0 2023-09-13T04:37:21.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.905+0000 7f3f7eb11700 1 -- 172.21.15.156:0/3590865289 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f781a0d80 con 0x7f3f780ffec0 2023-09-13T04:37:21.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.905+0000 7f3f75ffb700 1 -- 172.21.15.156:0/3590865289 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c00f620 con 0x7f3f780ffec0 2023-09-13T04:37:21.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.905+0000 7f3f7eb11700 1 -- 172.21.15.156:0/3590865289 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f781a1210 con 0x7f3f780ffec0 2023-09-13T04:37:21.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.906+0000 7f3f7eb11700 1 -- 172.21.15.156:0/3590865289 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f7810b4f0 con 0x7f3f780ffec0 2023-09-13T04:37:21.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.907+0000 7f3f75ffb700 1 -- 172.21.15.156:0/3590865289 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f3f6c010460 con 0x7f3f780ffec0 2023-09-13T04:37:21.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.907+0000 7f3f75ffb700 1 --2- 172.21.15.156:0/3590865289 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f3f680a06c0 0x7f3f680a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:21.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.908+0000 7f3f75ffb700 1 -- 172.21.15.156:0/3590865289 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3f6c0cbab0 con 0x7f3f780ffec0 2023-09-13T04:37:21.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.908+0000 7f3f77fff700 1 --2- 172.21.15.156:0/3590865289 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f3f680a06c0 0x7f3f680a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:21.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.909+0000 7f3f77fff700 1 --2- 172.21.15.156:0/3590865289 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f3f680a06c0 0x7f3f680a2b80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3f640095c0 tx=0x7f3f64009530 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:21.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:21.912+0000 7f3f75ffb700 1 -- 172.21.15.156:0/3590865289 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3f6c0959e0 con 0x7f3f780ffec0 2023-09-13T04:37:22.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:22.113+0000 7f3f7eb11700 1 -- 172.21.15.156:0/3590865289 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f780ff640 con 0x7f3f680a06c0 2023-09-13T04:37:22.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:22.121+0000 7f3f75ffb700 1 -- 172.21.15.156:0/3590865289 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f3f780ff640 con 0x7f3f680a06c0 2023-09-13T04:37:22.122 INFO:teuthology.orchestra.run.smithi156.stdout:Scheduled mds.all update... 2023-09-13T04:37:22.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:22.124+0000 7f3f7eb11700 1 -- 172.21.15.156:0/3590865289 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f3f680a06c0 msgr2=0x7f3f680a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:22.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:22.124+0000 7f3f7eb11700 1 --2- 172.21.15.156:0/3590865289 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f3f680a06c0 0x7f3f680a2b80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3f640095c0 tx=0x7f3f64009530 comp rx=0 tx=0).stop 2023-09-13T04:37:22.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:22.124+0000 7f3f7eb11700 1 -- 172.21.15.156:0/3590865289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f780ffec0 msgr2=0x7f3f78103c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:22.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:22.124+0000 7f3f7eb11700 1 --2- 172.21.15.156:0/3590865289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f780ffec0 0x7f3f78103c60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00d950 tx=0x7f3f6c00dd10 comp rx=0 tx=0).stop 2023-09-13T04:37:22.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:22.124+0000 7f3f7eb11700 1 -- 172.21.15.156:0/3590865289 shutdown_connections 2023-09-13T04:37:22.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:22.124+0000 7f3f7eb11700 1 --2- 172.21.15.156:0/3590865289 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f3f680a06c0 0x7f3f680a2b80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:22.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:22.124+0000 7f3f7eb11700 1 --2- 172.21.15.156:0/3590865289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f781012f0 0x7f3f781027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:22.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:22.124+0000 7f3f7eb11700 1 --2- 172.21.15.156:0/3590865289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f780ffec0 0x7f3f78103c60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:22.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:22.124+0000 7f3f7eb11700 1 --2- 172.21.15.156:0/3590865289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f78100880 0x7f3f781022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:22.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:22.124+0000 7f3f7eb11700 1 -- 172.21.15.156:0/3590865289 >> 172.21.15.156:0/3590865289 conn(0x7f3f780fa970 msgr2=0x7f3f780fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:22.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:22.124+0000 7f3f7eb11700 1 -- 172.21.15.156:0/3590865289 shutdown_connections 2023-09-13T04:37:22.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-13T04:37:22.124+0000 7f3f7eb11700 1 -- 172.21.15.156:0/3590865289 wait complete. 2023-09-13T04:37:22.477 DEBUG:teuthology.orchestra.run.smithi002:mds.a> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.a.service 2023-09-13T04:37:22.483 DEBUG:teuthology.orchestra.run.smithi002:mds.d> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.d.service 2023-09-13T04:37:22.528 DEBUG:teuthology.orchestra.run.smithi002:mds.g> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.g.service 2023-09-13T04:37:22.566 INFO:journalctl@ceph.mds.a.smithi002.stdout:-- Logs begin at Wed 2023-09-13 04:13:33 UTC. -- 2023-09-13T04:37:22.570 DEBUG:teuthology.orchestra.run.smithi002:mds.j> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.j.service 2023-09-13T04:37:22.605 INFO:journalctl@ceph.mds.d.smithi002.stdout:-- Logs begin at Wed 2023-09-13 04:13:33 UTC. -- 2023-09-13T04:37:22.608 DEBUG:teuthology.orchestra.run.smithi149:mds.b> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.b.service 2023-09-13T04:37:22.611 DEBUG:teuthology.orchestra.run.smithi149:mds.e> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.e.service 2023-09-13T04:37:22.641 INFO:journalctl@ceph.mds.b.smithi149.stdout:-- Logs begin at Wed 2023-09-13 04:09:45 UTC. -- 2023-09-13T04:37:22.642 DEBUG:teuthology.orchestra.run.smithi149:mds.h> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.h.service 2023-09-13T04:37:22.645 INFO:journalctl@ceph.mds.g.smithi002.stdout:-- Logs begin at Wed 2023-09-13 04:13:33 UTC. -- 2023-09-13T04:37:22.667 INFO:journalctl@ceph.mds.e.smithi149.stdout:-- Logs begin at Wed 2023-09-13 04:09:45 UTC. -- 2023-09-13T04:37:22.668 DEBUG:teuthology.orchestra.run.smithi149:mds.k> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.k.service 2023-09-13T04:37:22.679 INFO:journalctl@ceph.mds.j.smithi002.stdout:-- Logs begin at Wed 2023-09-13 04:13:33 UTC. -- 2023-09-13T04:37:22.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:22 smithi156 ceph-mon[133675]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:22.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:22 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:22.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:22 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:37:22.694 INFO:journalctl@ceph.mds.h.smithi149.stdout:-- Logs begin at Wed 2023-09-13 04:09:45 UTC. -- 2023-09-13T04:37:22.695 DEBUG:teuthology.orchestra.run.smithi156:mds.c> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.c.service 2023-09-13T04:37:22.697 DEBUG:teuthology.orchestra.run.smithi156:mds.f> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.f.service 2023-09-13T04:37:22.721 INFO:journalctl@ceph.mds.k.smithi149.stdout:-- Logs begin at Wed 2023-09-13 04:09:45 UTC. -- 2023-09-13T04:37:22.728 INFO:journalctl@ceph.mds.c.smithi156.stdout:-- Logs begin at Wed 2023-09-13 04:09:45 UTC. -- 2023-09-13T04:37:22.729 DEBUG:teuthology.orchestra.run.smithi156:mds.i> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.i.service 2023-09-13T04:37:22.755 INFO:journalctl@ceph.mds.f.smithi156.stdout:-- Logs begin at Wed 2023-09-13 04:09:45 UTC. -- 2023-09-13T04:37:22.756 DEBUG:teuthology.orchestra.run.smithi156:mds.l> sudo journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.l.service 2023-09-13T04:37:22.781 INFO:journalctl@ceph.mds.i.smithi156.stdout:-- Logs begin at Wed 2023-09-13 04:09:45 UTC. -- 2023-09-13T04:37:22.782 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-09-13T04:37:22.783 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T04:37:22.807 INFO:journalctl@ceph.mds.l.smithi156.stdout:-- Logs begin at Wed 2023-09-13 04:09:45 UTC. -- 2023-09-13T04:37:22.845 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:22 smithi002 ceph-mon[97651]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:22.845 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:22 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:22.845 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:22 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:37:22.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:22 smithi149 ceph-mon[133315]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:22.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:22 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:22.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:22 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:37:23.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.000+0000 7f8a319ff700 1 -- 172.21.15.2:0/2084345299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a2c102880 msgr2=0x7f8a2c109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:23.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.000+0000 7f8a319ff700 1 --2- 172.21.15.2:0/2084345299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a2c102880 0x7f8a2c109130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009fc0 tx=0x7f8a2400b330 comp rx=0 tx=0).stop 2023-09-13T04:37:23.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.000+0000 7f8a319ff700 1 -- 172.21.15.2:0/2084345299 shutdown_connections 2023-09-13T04:37:23.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.000+0000 7f8a319ff700 1 --2- 172.21.15.2:0/2084345299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a2c102880 0x7f8a2c109130 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.000+0000 7f8a319ff700 1 --2- 172.21.15.2:0/2084345299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a2c100cf0 0x7f8a2c101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.000+0000 7f8a319ff700 1 --2- 172.21.15.2:0/2084345299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a2c101ee0 0x7f8a2c102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.000+0000 7f8a319ff700 1 -- 172.21.15.2:0/2084345299 >> 172.21.15.2:0/2084345299 conn(0x7f8a2c0fc220 msgr2=0x7f8a2c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:23.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.001+0000 7f8a319ff700 1 -- 172.21.15.2:0/2084345299 shutdown_connections 2023-09-13T04:37:23.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.001+0000 7f8a319ff700 1 -- 172.21.15.2:0/2084345299 wait complete. 2023-09-13T04:37:23.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.002+0000 7f8a319ff700 1 Processor -- start 2023-09-13T04:37:23.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.002+0000 7f8a319ff700 1 -- start start 2023-09-13T04:37:23.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.003+0000 7f8a319ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a2c100cf0 0x7f8a2c074a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:23.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.003+0000 7f8a319ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a2c101ee0 0x7f8a2c0730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:23.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.003+0000 7f8a319ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a2c102880 0x7f8a2c0735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:23.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.003+0000 7f8a319ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2c073d80 con 0x7f8a2c102880 2023-09-13T04:37:23.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.003+0000 7f8a319ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2c073ef0 con 0x7f8a2c101ee0 2023-09-13T04:37:23.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.003+0000 7f8a319ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2c1a0b10 con 0x7f8a2c100cf0 2023-09-13T04:37:23.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.003+0000 7f8a2b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a2c102880 0x7f8a2c0735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:23.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.003+0000 7f8a2affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a2c100cf0 0x7f8a2c074a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:23.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.003+0000 7f8a2affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a2c100cf0 0x7f8a2c074a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:50662/0 (socket says 172.21.15.2:50662) 2023-09-13T04:37:23.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.003+0000 7f8a2affd700 1 -- 172.21.15.2:0/3565992488 learned_addr learned my addr 172.21.15.2:0/3565992488 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:23.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.003+0000 7f8a2a7fc700 1 --2- 172.21.15.2:0/3565992488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a2c101ee0 0x7f8a2c0730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:23.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.003+0000 7f8a2affd700 1 -- 172.21.15.2:0/3565992488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a2c101ee0 msgr2=0x7f8a2c0730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:23.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.003+0000 7f8a2affd700 1 --2- 172.21.15.2:0/3565992488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a2c101ee0 0x7f8a2c0730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.003+0000 7f8a2affd700 1 -- 172.21.15.2:0/3565992488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a2c102880 msgr2=0x7f8a2c0735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:23.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.003+0000 7f8a2affd700 1 --2- 172.21.15.2:0/3565992488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a2c102880 0x7f8a2c0735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.003+0000 7f8a2affd700 1 -- 172.21.15.2:0/3565992488 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a24009c40 con 0x7f8a2c100cf0 2023-09-13T04:37:23.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.003+0000 7f8a2b7fe700 1 --2- 172.21.15.2:0/3565992488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a2c102880 0x7f8a2c0735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:37:23.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.004+0000 7f8a2affd700 1 --2- 172.21.15.2:0/3565992488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a2c100cf0 0x7f8a2c074a60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00e970 tx=0x7f8a1c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:23.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.004+0000 7f8a309fd700 1 -- 172.21.15.2:0/3565992488 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c009790 con 0x7f8a2c100cf0 2023-09-13T04:37:23.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.004+0000 7f8a309fd700 1 -- 172.21.15.2:0/3565992488 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a1c00c7c0 con 0x7f8a2c100cf0 2023-09-13T04:37:23.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.004+0000 7f8a309fd700 1 -- 172.21.15.2:0/3565992488 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c003c20 con 0x7f8a2c100cf0 2023-09-13T04:37:23.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.004+0000 7f8a319ff700 1 -- 172.21.15.2:0/3565992488 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a2c1a0df0 con 0x7f8a2c100cf0 2023-09-13T04:37:23.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.004+0000 7f8a319ff700 1 -- 172.21.15.2:0/3565992488 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a2c1a1220 con 0x7f8a2c100cf0 2023-09-13T04:37:23.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.005+0000 7f8a319ff700 1 -- 172.21.15.2:0/3565992488 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a2c108190 con 0x7f8a2c100cf0 2023-09-13T04:37:23.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.007+0000 7f8a309fd700 1 -- 172.21.15.2:0/3565992488 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f8a1c00c930 con 0x7f8a2c100cf0 2023-09-13T04:37:23.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.008+0000 7f8a309fd700 1 --2- 172.21.15.2:0/3565992488 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8a140a0710 0x7f8a140a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:23.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.008+0000 7f8a309fd700 1 -- 172.21.15.2:0/3565992488 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8a1c014070 con 0x7f8a2c100cf0 2023-09-13T04:37:23.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.008+0000 7f8a2a7fc700 1 --2- 172.21.15.2:0/3565992488 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8a140a0710 0x7f8a140a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:23.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.009+0000 7f8a2a7fc700 1 --2- 172.21.15.2:0/3565992488 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8a140a0710 0x7f8a140a2bd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8a20005fd0 tx=0x7f8a20009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:23.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.010+0000 7f8a309fd700 1 -- 172.21.15.2:0/3565992488 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8a1c095a30 con 0x7f8a2c100cf0 2023-09-13T04:37:23.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.223+0000 7f8a319ff700 1 -- 172.21.15.2:0/3565992488 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a2c02cd70 con 0x7f8a2c100cf0 2023-09-13T04:37:23.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.224+0000 7f8a309fd700 1 -- 172.21.15.2:0/3565992488 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18389 (secure 0 0 0) 0x7f8a1c099050 con 0x7f8a2c100cf0 2023-09-13T04:37:23.225 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:37:23.225 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":80,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:37:11.321587+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:37:23.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.227+0000 7f8a319ff700 1 -- 172.21.15.2:0/3565992488 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8a140a0710 msgr2=0x7f8a140a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:23.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.227+0000 7f8a319ff700 1 --2- 172.21.15.2:0/3565992488 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8a140a0710 0x7f8a140a2bd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8a20005fd0 tx=0x7f8a20009310 comp rx=0 tx=0).stop 2023-09-13T04:37:23.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.227+0000 7f8a319ff700 1 -- 172.21.15.2:0/3565992488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a2c100cf0 msgr2=0x7f8a2c074a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:23.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.227+0000 7f8a319ff700 1 --2- 172.21.15.2:0/3565992488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a2c100cf0 0x7f8a2c074a60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00e970 tx=0x7f8a1c00ec80 comp rx=0 tx=0).stop 2023-09-13T04:37:23.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.227+0000 7f8a319ff700 1 -- 172.21.15.2:0/3565992488 shutdown_connections 2023-09-13T04:37:23.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.227+0000 7f8a319ff700 1 --2- 172.21.15.2:0/3565992488 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8a140a0710 0x7f8a140a2bd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.227+0000 7f8a319ff700 1 --2- 172.21.15.2:0/3565992488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a2c102880 0x7f8a2c0735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.227+0000 7f8a319ff700 1 --2- 172.21.15.2:0/3565992488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a2c100cf0 0x7f8a2c074a60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.227+0000 7f8a319ff700 1 --2- 172.21.15.2:0/3565992488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a2c101ee0 0x7f8a2c0730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.227+0000 7f8a319ff700 1 -- 172.21.15.2:0/3565992488 >> 172.21.15.2:0/3565992488 conn(0x7f8a2c0fc220 msgr2=0x7f8a2c1052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:23.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.227+0000 7f8a319ff700 1 -- 172.21.15.2:0/3565992488 shutdown_connections 2023-09-13T04:37:23.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.227+0000 7f8a319ff700 1 -- 172.21.15.2:0/3565992488 wait complete. 2023-09-13T04:37:23.247 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T04:33:42.025714+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-13T04:37:23.247 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-13T04:37:23.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.453+0000 7f7b085ff700 1 -- 172.21.15.2:0/693293696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b00073710 msgr2=0x7f7b00073b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:23.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.453+0000 7f7b085ff700 1 --2- 172.21.15.2:0/693293696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b00073710 0x7f7b00073b90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7af0009990 tx=0x7f7af0009ca0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.453+0000 7f7b085ff700 1 -- 172.21.15.2:0/693293696 shutdown_connections 2023-09-13T04:37:23.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.453+0000 7f7b085ff700 1 --2- 172.21.15.2:0/693293696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b00108ba0 0x7f7b0010b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.453+0000 7f7b085ff700 1 --2- 172.21.15.2:0/693293696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b00074cf0 0x7f7b00073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.453+0000 7f7b085ff700 1 --2- 172.21.15.2:0/693293696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b00073710 0x7f7b00073b90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.453+0000 7f7b085ff700 1 -- 172.21.15.2:0/693293696 >> 172.21.15.2:0/693293696 conn(0x7f7b000fc230 msgr2=0x7f7b000fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:23.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.453+0000 7f7b085ff700 1 -- 172.21.15.2:0/693293696 shutdown_connections 2023-09-13T04:37:23.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.453+0000 7f7b085ff700 1 -- 172.21.15.2:0/693293696 wait complete. 2023-09-13T04:37:23.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.454+0000 7f7b085ff700 1 Processor -- start 2023-09-13T04:37:23.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.454+0000 7f7b085ff700 1 -- start start 2023-09-13T04:37:23.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.455+0000 7f7b085ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b00073710 0x7f7b00198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:23.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.455+0000 7f7b085ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b00074cf0 0x7f7b00199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:23.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.455+0000 7f7b085ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b00108ba0 0x7f7b001a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:23.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.455+0000 7f7b085ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b001a0770 con 0x7f7b00108ba0 2023-09-13T04:37:23.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.455+0000 7f7b085ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b001a08b0 con 0x7f7b00073710 2023-09-13T04:37:23.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.455+0000 7f7b085ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b001a0a40 con 0x7f7b00074cf0 2023-09-13T04:37:23.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.455+0000 7f7b05b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b00074cf0 0x7f7b00199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:23.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.455+0000 7f7b0639b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b00073710 0x7f7b00198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:23.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.455+0000 7f7b05b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b00074cf0 0x7f7b00199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:50692/0 (socket says 172.21.15.2:50692) 2023-09-13T04:37:23.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.455+0000 7f7b05b9a700 1 -- 172.21.15.2:0/1592182012 learned_addr learned my addr 172.21.15.2:0/1592182012 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:23.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.455+0000 7f7b06b9c700 1 --2- 172.21.15.2:0/1592182012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b00108ba0 0x7f7b001a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:23.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.455+0000 7f7b05b9a700 1 -- 172.21.15.2:0/1592182012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b00073710 msgr2=0x7f7b00198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:23.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.455+0000 7f7b05b9a700 1 --2- 172.21.15.2:0/1592182012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b00073710 0x7f7b00198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.455+0000 7f7b05b9a700 1 -- 172.21.15.2:0/1592182012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b00108ba0 msgr2=0x7f7b001a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:23.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.455+0000 7f7b05b9a700 1 --2- 172.21.15.2:0/1592182012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b00108ba0 0x7f7b001a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.455+0000 7f7b05b9a700 1 -- 172.21.15.2:0/1592182012 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af8009c40 con 0x7f7b00074cf0 2023-09-13T04:37:23.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.455+0000 7f7b0639b700 1 --2- 172.21.15.2:0/1592182012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b00073710 0x7f7b00198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:37:23.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.456+0000 7f7b05b9a700 1 --2- 172.21.15.2:0/1592182012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b00074cf0 0x7f7b00199130 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7af0000c00 tx=0x7f7af0005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:23.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.456+0000 7f7af77fe700 1 -- 172.21.15.2:0/1592182012 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af001d070 con 0x7f7b00074cf0 2023-09-13T04:37:23.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.456+0000 7f7af77fe700 1 -- 172.21.15.2:0/1592182012 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7af0022470 con 0x7f7b00074cf0 2023-09-13T04:37:23.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.456+0000 7f7b085ff700 1 -- 172.21.15.2:0/1592182012 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af00095f0 con 0x7f7b00074cf0 2023-09-13T04:37:23.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.456+0000 7f7af77fe700 1 -- 172.21.15.2:0/1592182012 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af0017650 con 0x7f7b00074cf0 2023-09-13T04:37:23.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.456+0000 7f7b085ff700 1 -- 172.21.15.2:0/1592182012 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b001a0fb0 con 0x7f7b00074cf0 2023-09-13T04:37:23.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.457+0000 7f7b085ff700 1 -- 172.21.15.2:0/1592182012 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b00107250 con 0x7f7b00074cf0 2023-09-13T04:37:23.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.459+0000 7f7af77fe700 1 -- 172.21.15.2:0/1592182012 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f7af00225e0 con 0x7f7b00074cf0 2023-09-13T04:37:23.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.460+0000 7f7af77fe700 1 --2- 172.21.15.2:0/1592182012 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f7aec0a06c0 0x7f7aec0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:23.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.460+0000 7f7af77fe700 1 -- 172.21.15.2:0/1592182012 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7af00ccf10 con 0x7f7b00074cf0 2023-09-13T04:37:23.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.460+0000 7f7b0639b700 1 --2- 172.21.15.2:0/1592182012 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f7aec0a06c0 0x7f7aec0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:23.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.461+0000 7f7b0639b700 1 --2- 172.21.15.2:0/1592182012 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f7aec0a06c0 0x7f7aec0a2b80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7afc011490 tx=0x7f7afc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:23.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.462+0000 7f7af77fe700 1 -- 172.21.15.2:0/1592182012 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7af0096ed0 con 0x7f7b00074cf0 2023-09-13T04:37:23.591 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:23 smithi156 ceph-mon[133675]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:37:23.591 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:23 smithi156 ceph-mon[133675]: Saving service mds.all spec with placement smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l;count:12 2023-09-13T04:37:23.591 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:23.592 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:37:23.592 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:23.592 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:23.592 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:23.592 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:23.593 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:23 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3565992488' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:37:23.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.673+0000 7f7b085ff700 1 -- 172.21.15.2:0/1592182012 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7b0010e860 con 0x7f7b00074cf0 2023-09-13T04:37:23.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.674+0000 7f7af77fe700 1 -- 172.21.15.2:0/1592182012 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7af009a4f0 con 0x7f7b00074cf0 2023-09-13T04:37:23.674 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-09-13T04:37:23.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.676+0000 7f7b085ff700 1 -- 172.21.15.2:0/1592182012 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f7aec0a06c0 msgr2=0x7f7aec0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:23.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.676+0000 7f7b085ff700 1 --2- 172.21.15.2:0/1592182012 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f7aec0a06c0 0x7f7aec0a2b80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7afc011490 tx=0x7f7afc009260 comp rx=0 tx=0).stop 2023-09-13T04:37:23.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.676+0000 7f7b085ff700 1 -- 172.21.15.2:0/1592182012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b00074cf0 msgr2=0x7f7b00199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:23.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.676+0000 7f7b085ff700 1 --2- 172.21.15.2:0/1592182012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b00074cf0 0x7f7b00199130 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7af0000c00 tx=0x7f7af0005720 comp rx=0 tx=0).stop 2023-09-13T04:37:23.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.676+0000 7f7b085ff700 1 -- 172.21.15.2:0/1592182012 shutdown_connections 2023-09-13T04:37:23.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.676+0000 7f7b085ff700 1 --2- 172.21.15.2:0/1592182012 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f7aec0a06c0 0x7f7aec0a2b80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.676+0000 7f7b085ff700 1 --2- 172.21.15.2:0/1592182012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b00108ba0 0x7f7b001a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.676+0000 7f7b085ff700 1 --2- 172.21.15.2:0/1592182012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b00073710 0x7f7b00198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.676+0000 7f7b085ff700 1 --2- 172.21.15.2:0/1592182012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b00074cf0 0x7f7b00199130 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.676+0000 7f7b085ff700 1 -- 172.21.15.2:0/1592182012 >> 172.21.15.2:0/1592182012 conn(0x7f7b000fc230 msgr2=0x7f7b000fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:23.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.676+0000 7f7b085ff700 1 -- 172.21.15.2:0/1592182012 shutdown_connections 2023-09-13T04:37:23.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.677+0000 7f7b085ff700 1 -- 172.21.15.2:0/1592182012 wait complete. 2023-09-13T04:37:23.696 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:37:23.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:23 smithi149 ceph-mon[133315]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:37:23.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:23 smithi149 ceph-mon[133315]: Saving service mds.all spec with placement smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l;count:12 2023-09-13T04:37:23.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:23.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:37:23.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:23.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:23.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:23.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:23.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:23 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3565992488' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:37:23.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.900+0000 7fe9d05ff700 1 -- 172.21.15.2:0/1036525616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9c8100cf0 msgr2=0x7fe9c8101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:23.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.900+0000 7fe9d05ff700 1 --2- 172.21.15.2:0/1036525616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9c8100cf0 0x7fe9c8101110 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40099c0 tx=0x7fe9c4009cd0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.900+0000 7fe9d05ff700 1 -- 172.21.15.2:0/1036525616 shutdown_connections 2023-09-13T04:37:23.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.900+0000 7fe9d05ff700 1 --2- 172.21.15.2:0/1036525616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9c8102880 0x7fe9c8109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.900+0000 7fe9d05ff700 1 --2- 172.21.15.2:0/1036525616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9c8100cf0 0x7fe9c8101110 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.900+0000 7fe9d05ff700 1 --2- 172.21.15.2:0/1036525616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9c8101ee0 0x7fe9c8102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.900+0000 7fe9d05ff700 1 -- 172.21.15.2:0/1036525616 >> 172.21.15.2:0/1036525616 conn(0x7fe9c80fc220 msgr2=0x7fe9c80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:23.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.900+0000 7fe9d05ff700 1 -- 172.21.15.2:0/1036525616 shutdown_connections 2023-09-13T04:37:23.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.900+0000 7fe9d05ff700 1 -- 172.21.15.2:0/1036525616 wait complete. 2023-09-13T04:37:23.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.901+0000 7fe9d05ff700 1 Processor -- start 2023-09-13T04:37:23.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.901+0000 7fe9d05ff700 1 -- start start 2023-09-13T04:37:23.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.901+0000 7fe9d05ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9c8100cf0 0x7fe9c8198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:23.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.901+0000 7fe9d05ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9c8101ee0 0x7fe9c8199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:23.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.901+0000 7fe9d05ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9c8102880 0x7fe9c81a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:23.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.901+0000 7fe9d05ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c81a0760 con 0x7fe9c8100cf0 2023-09-13T04:37:23.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.901+0000 7fe9d05ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c81a08a0 con 0x7fe9c8101ee0 2023-09-13T04:37:23.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.901+0000 7fe9d05ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c81a0a30 con 0x7fe9c8102880 2023-09-13T04:37:23.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.902+0000 7fe9ce39b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9c8100cf0 0x7fe9c8198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:23.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.902+0000 7fe9cdb9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9c8101ee0 0x7fe9c8199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:23.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.902+0000 7fe9ceb9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9c8102880 0x7fe9c81a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:23.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.902+0000 7fe9ce39b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9c8100cf0 0x7fe9c8198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46310/0 (socket says 172.21.15.2:46310) 2023-09-13T04:37:23.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.902+0000 7fe9cdb9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9c8101ee0 0x7fe9c8199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:47456/0 (socket says 172.21.15.2:47456) 2023-09-13T04:37:23.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.902+0000 7fe9ce39b700 1 -- 172.21.15.2:0/993449090 learned_addr learned my addr 172.21.15.2:0/993449090 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:23.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.902+0000 7fe9ce39b700 1 -- 172.21.15.2:0/993449090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9c8102880 msgr2=0x7fe9c81a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:23.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.902+0000 7fe9ce39b700 1 --2- 172.21.15.2:0/993449090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9c8102880 0x7fe9c81a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.902+0000 7fe9ce39b700 1 -- 172.21.15.2:0/993449090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9c8101ee0 msgr2=0x7fe9c8199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:23.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.902+0000 7fe9ce39b700 1 --2- 172.21.15.2:0/993449090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9c8101ee0 0x7fe9c8199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:23.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.902+0000 7fe9ce39b700 1 -- 172.21.15.2:0/993449090 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c40095f0 con 0x7fe9c8100cf0 2023-09-13T04:37:23.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.902+0000 7fe9ceb9c700 1 --2- 172.21.15.2:0/993449090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9c8102880 0x7fe9c81a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:37:23.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.902+0000 7fe9ce39b700 1 --2- 172.21.15.2:0/993449090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9c8100cf0 0x7fe9c8198be0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4009990 tx=0x7fe9c400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:23.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.903+0000 7fe9bf7fe700 1 -- 172.21.15.2:0/993449090 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c401d070 con 0x7fe9c8100cf0 2023-09-13T04:37:23.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.903+0000 7fe9bf7fe700 1 -- 172.21.15.2:0/993449090 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9c4005030 con 0x7fe9c8100cf0 2023-09-13T04:37:23.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.903+0000 7fe9d05ff700 1 -- 172.21.15.2:0/993449090 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c81a0cb0 con 0x7fe9c8100cf0 2023-09-13T04:37:23.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.903+0000 7fe9bf7fe700 1 -- 172.21.15.2:0/993449090 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4004a70 con 0x7fe9c8100cf0 2023-09-13T04:37:23.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.903+0000 7fe9d05ff700 1 -- 172.21.15.2:0/993449090 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c81a1130 con 0x7fe9c8100cf0 2023-09-13T04:37:23.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.904+0000 7fe9d05ff700 1 -- 172.21.15.2:0/993449090 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c8193340 con 0x7fe9c8100cf0 2023-09-13T04:37:23.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.906+0000 7fe9bf7fe700 1 -- 172.21.15.2:0/993449090 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fe9c4005410 con 0x7fe9c8100cf0 2023-09-13T04:37:23.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.908+0000 7fe9bf7fe700 1 --2- 172.21.15.2:0/993449090 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe9b40a0710 0x7fe9b40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:23.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.908+0000 7fe9bf7fe700 1 -- 172.21.15.2:0/993449090 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe9c40cc430 con 0x7fe9c8100cf0 2023-09-13T04:37:23.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.908+0000 7fe9cdb9a700 1 --2- 172.21.15.2:0/993449090 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe9b40a0710 0x7fe9b40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:23.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.909+0000 7fe9cdb9a700 1 --2- 172.21.15.2:0/993449090 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe9b40a0710 0x7fe9b40a2bd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009c60 tx=0x7fe9b8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:23.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:23.912+0000 7fe9bf7fe700 1 -- 172.21.15.2:0/993449090 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe9c40963f0 con 0x7fe9c8100cf0 2023-09-13T04:37:24.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:23 smithi002 ceph-mon[97651]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:37:24.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:23 smithi002 ceph-mon[97651]: Saving service mds.all spec with placement smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l;count:12 2023-09-13T04:37:24.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:24.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:37:24.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:24.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:24.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:24.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:24.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:23 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3565992488' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:37:24.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:24.166+0000 7fe9d05ff700 1 -- 172.21.15.2:0/993449090 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9c810c550 con 0x7fe9c8100cf0 2023-09-13T04:37:24.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:24.167+0000 7fe9bf7fe700 1 -- 172.21.15.2:0/993449090 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7fe9c4099a10 con 0x7fe9c8100cf0 2023-09-13T04:37:24.167 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:37:24.168 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-09-13T04:37:24.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:24.169+0000 7fe9d05ff700 1 -- 172.21.15.2:0/993449090 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe9b40a0710 msgr2=0x7fe9b40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:24.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:24.169+0000 7fe9d05ff700 1 --2- 172.21.15.2:0/993449090 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe9b40a0710 0x7fe9b40a2bd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009c60 tx=0x7fe9b8009260 comp rx=0 tx=0).stop 2023-09-13T04:37:24.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:24.169+0000 7fe9d05ff700 1 -- 172.21.15.2:0/993449090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9c8100cf0 msgr2=0x7fe9c8198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:24.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:24.169+0000 7fe9d05ff700 1 --2- 172.21.15.2:0/993449090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9c8100cf0 0x7fe9c8198be0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4009990 tx=0x7fe9c400f690 comp rx=0 tx=0).stop 2023-09-13T04:37:24.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:24.169+0000 7fe9d05ff700 1 -- 172.21.15.2:0/993449090 shutdown_connections 2023-09-13T04:37:24.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:24.169+0000 7fe9d05ff700 1 --2- 172.21.15.2:0/993449090 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe9b40a0710 0x7fe9b40a2bd0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:24.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:24.169+0000 7fe9d05ff700 1 --2- 172.21.15.2:0/993449090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9c8102880 0x7fe9c81a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:24.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:24.169+0000 7fe9d05ff700 1 --2- 172.21.15.2:0/993449090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9c8100cf0 0x7fe9c8198be0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:24.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:24.169+0000 7fe9d05ff700 1 --2- 172.21.15.2:0/993449090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9c8101ee0 0x7fe9c8199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:24.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:24.169+0000 7fe9d05ff700 1 -- 172.21.15.2:0/993449090 >> 172.21.15.2:0/993449090 conn(0x7fe9c80fc220 msgr2=0x7fe9c81077e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:24.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:24.169+0000 7fe9d05ff700 1 -- 172.21.15.2:0/993449090 shutdown_connections 2023-09-13T04:37:24.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:24.169+0000 7fe9d05ff700 1 -- 172.21.15.2:0/993449090 wait complete. 2023-09-13T04:37:24.172 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 1 2023-09-13T04:37:24.186 INFO:journalctl@ceph.mds.l.smithi156.stdout:Sep 13 04:37:24 smithi156 systemd[1]: Starting Ceph mds.l for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:37:24.746 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:24 smithi156 ceph-mon[133675]: Deploying daemon mds.l on smithi156 2023-09-13T04:37:24.747 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:24 smithi156 ceph-mon[133675]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:24.747 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:24 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1592182012' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:37:24.747 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:24 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/993449090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:24.747 INFO:journalctl@ceph.mds.l.smithi156.stdout:Sep 13 04:37:24 smithi156 podman[171004]: 2023-09-13T04:37:24.748 INFO:journalctl@ceph.mds.l.smithi156.stdout:Sep 13 04:37:24 smithi156 ceph-mds[171044]: -- 172.21.15.156:0/4014931518 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5605ae2a71e0 con 0x5605ae27d000 2023-09-13T04:37:24.748 INFO:journalctl@ceph.mds.l.smithi156.stdout:Sep 13 04:37:24 smithi156 bash[171004]: 8620f51fa9365231c147256894fe79b13e56df84b4fdd52b87ea20fd71f2f3bc 2023-09-13T04:37:24.748 INFO:journalctl@ceph.mds.l.smithi156.stdout:Sep 13 04:37:24 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-l[171020]: starting mds.l at 2023-09-13T04:37:24.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:24 smithi149 ceph-mon[133315]: Deploying daemon mds.l on smithi156 2023-09-13T04:37:24.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:24 smithi149 ceph-mon[133315]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:24.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:24 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1592182012' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:37:24.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:24 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/993449090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:25.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:24 smithi002 ceph-mon[97651]: Deploying daemon mds.l on smithi156 2023-09-13T04:37:25.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:24 smithi002 ceph-mon[97651]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:25.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:24 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1592182012' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:37:25.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:24 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/993449090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:25.186 INFO:journalctl@ceph.mds.l.smithi156.stdout:Sep 13 04:37:24 smithi156 systemd[1]: Started Ceph mds.l for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:37:26.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:25 smithi002 ceph-mon[97651]: mds.? [v2:172.21.15.156:6832/4209382763,v1:172.21.15.156:6833/4209382763] up:boot 2023-09-13T04:37:26.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:25 smithi002 ceph-mon[97651]: fsmap 1 up:standby 2023-09-13T04:37:26.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:25 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-13T04:37:26.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:25 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:26.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:25 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:26.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:25 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:26.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:25 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:26.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:25 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:26.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:25 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:26.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:25 smithi002 ceph-mon[97651]: Deploying daemon mds.k on smithi149 2023-09-13T04:37:26.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:25 smithi002 ceph-mon[97651]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:26.064 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:25 smithi149 ceph-mon[133315]: mds.? [v2:172.21.15.156:6832/4209382763,v1:172.21.15.156:6833/4209382763] up:boot 2023-09-13T04:37:26.064 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:25 smithi149 ceph-mon[133315]: fsmap 1 up:standby 2023-09-13T04:37:26.064 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:25 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-13T04:37:26.064 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:25 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:26.064 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:25 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:26.065 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:25 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:26.065 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:25 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:26.065 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:25 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:26.065 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:25 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:26.065 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:25 smithi149 ceph-mon[133315]: Deploying daemon mds.k on smithi149 2023-09-13T04:37:26.066 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:25 smithi149 ceph-mon[133315]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:26.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:25 smithi156 ceph-mon[133675]: mds.? [v2:172.21.15.156:6832/4209382763,v1:172.21.15.156:6833/4209382763] up:boot 2023-09-13T04:37:26.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:25 smithi156 ceph-mon[133675]: fsmap 1 up:standby 2023-09-13T04:37:26.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:25 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-13T04:37:26.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:25 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:26.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:25 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:26.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:25 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:26.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:25 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:26.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:25 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:26.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:25 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:26.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:25 smithi156 ceph-mon[133675]: Deploying daemon mds.k on smithi149 2023-09-13T04:37:26.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:25 smithi156 ceph-mon[133675]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:26.189 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:37:26.387 INFO:journalctl@ceph.mds.k.smithi149.stdout:Sep 13 04:37:26 smithi149 systemd[1]: Starting Ceph mds.k for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:37:26.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.404+0000 7f54ba7ff700 1 -- 172.21.15.2:0/3679820110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54b4073710 msgr2=0x7f54b4073b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:26.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.404+0000 7f54ba7ff700 1 --2- 172.21.15.2:0/3679820110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54b4073710 0x7f54b4073b90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009990 tx=0x7f54a8009ca0 comp rx=0 tx=0).stop 2023-09-13T04:37:26.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.404+0000 7f54ba7ff700 1 -- 172.21.15.2:0/3679820110 shutdown_connections 2023-09-13T04:37:26.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.404+0000 7f54ba7ff700 1 --2- 172.21.15.2:0/3679820110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54b4108bb0 0x7f54b410b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:26.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.404+0000 7f54ba7ff700 1 --2- 172.21.15.2:0/3679820110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54b4074cf0 0x7f54b4073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:26.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.404+0000 7f54ba7ff700 1 --2- 172.21.15.2:0/3679820110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54b4073710 0x7f54b4073b90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:26.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.404+0000 7f54ba7ff700 1 -- 172.21.15.2:0/3679820110 >> 172.21.15.2:0/3679820110 conn(0x7f54b40fc220 msgr2=0x7f54b40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:26.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.404+0000 7f54ba7ff700 1 -- 172.21.15.2:0/3679820110 shutdown_connections 2023-09-13T04:37:26.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.404+0000 7f54ba7ff700 1 -- 172.21.15.2:0/3679820110 wait complete. 2023-09-13T04:37:26.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.405+0000 7f54ba7ff700 1 Processor -- start 2023-09-13T04:37:26.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.405+0000 7f54ba7ff700 1 -- start start 2023-09-13T04:37:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.406+0000 7f54ba7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54b4073710 0x7f54b4198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.406+0000 7f54ba7ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54b4074cf0 0x7f54b4199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.406+0000 7f54ba7ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54b4108bb0 0x7f54b41a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.406+0000 7f54ba7ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b41a0770 con 0x7f54b4108bb0 2023-09-13T04:37:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.406+0000 7f54ba7ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b41a08b0 con 0x7f54b4074cf0 2023-09-13T04:37:26.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.406+0000 7f54ba7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b41a0a40 con 0x7f54b4073710 2023-09-13T04:37:26.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.406+0000 7f54b37fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54b4074cf0 0x7f54b4199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:26.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.406+0000 7f54b37fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54b4074cf0 0x7f54b4199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:57328/0 (socket says 172.21.15.2:57328) 2023-09-13T04:37:26.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.406+0000 7f54b37fe700 1 -- 172.21.15.2:0/3750853908 learned_addr learned my addr 172.21.15.2:0/3750853908 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:26.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.406+0000 7f54b3fff700 1 --2- 172.21.15.2:0/3750853908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54b4073710 0x7f54b4198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:26.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.406+0000 7f54b8d9c700 1 --2- 172.21.15.2:0/3750853908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54b4108bb0 0x7f54b41a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:26.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.406+0000 7f54b37fe700 1 -- 172.21.15.2:0/3750853908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54b4073710 msgr2=0x7f54b4198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:26.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.406+0000 7f54b37fe700 1 --2- 172.21.15.2:0/3750853908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54b4073710 0x7f54b4198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:26.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.406+0000 7f54b37fe700 1 -- 172.21.15.2:0/3750853908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54b4108bb0 msgr2=0x7f54b41a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:26.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.406+0000 7f54b37fe700 1 --2- 172.21.15.2:0/3750853908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54b4108bb0 0x7f54b41a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:26.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.406+0000 7f54b37fe700 1 -- 172.21.15.2:0/3750853908 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a4009520 con 0x7f54b4074cf0 2023-09-13T04:37:26.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.406+0000 7f54b37fe700 1 --2- 172.21.15.2:0/3750853908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54b4074cf0 0x7f54b4199130 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f54a800b2e0 tx=0x7f54a8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:26.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.407+0000 7f54b17fa700 1 -- 172.21.15.2:0/3750853908 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a801d070 con 0x7f54b4074cf0 2023-09-13T04:37:26.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.407+0000 7f54b17fa700 1 -- 172.21.15.2:0/3750853908 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54a8005030 con 0x7f54b4074cf0 2023-09-13T04:37:26.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.407+0000 7f54b17fa700 1 -- 172.21.15.2:0/3750853908 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8004c20 con 0x7f54b4074cf0 2023-09-13T04:37:26.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.407+0000 7f54ba7ff700 1 -- 172.21.15.2:0/3750853908 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a80095f0 con 0x7f54b4074cf0 2023-09-13T04:37:26.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.407+0000 7f54ba7ff700 1 -- 172.21.15.2:0/3750853908 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b41a0ef0 con 0x7f54b4074cf0 2023-09-13T04:37:26.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.408+0000 7f54ba7ff700 1 -- 172.21.15.2:0/3750853908 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b4107260 con 0x7f54b4074cf0 2023-09-13T04:37:26.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.410+0000 7f54b17fa700 1 -- 172.21.15.2:0/3750853908 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f54a8004d80 con 0x7f54b4074cf0 2023-09-13T04:37:26.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.411+0000 7f54b17fa700 1 --2- 172.21.15.2:0/3750853908 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f549c0a0710 0x7f549c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:26.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.411+0000 7f54b17fa700 1 -- 172.21.15.2:0/3750853908 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f54a80ccda0 con 0x7f54b4074cf0 2023-09-13T04:37:26.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.411+0000 7f54b3fff700 1 --2- 172.21.15.2:0/3750853908 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f549c0a0710 0x7f549c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:26.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.412+0000 7f54b3fff700 1 --2- 172.21.15.2:0/3750853908 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f549c0a0710 0x7f549c0a2bd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f54a4009df0 tx=0x7f54a4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:26.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.414+0000 7f54b17fa700 1 -- 172.21.15.2:0/3750853908 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f54a8096d60 con 0x7f54b4074cf0 2023-09-13T04:37:26.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.674+0000 7f54ba7ff700 1 -- 172.21.15.2:0/3750853908 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54b410e980 con 0x7f54b4074cf0 2023-09-13T04:37:26.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.674+0000 7f54b17fa700 1 -- 172.21.15.2:0/3750853908 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f54a8005c00 con 0x7f54b4074cf0 2023-09-13T04:37:26.675 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:37:26.675 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-09-13T04:37:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.677+0000 7f54ba7ff700 1 -- 172.21.15.2:0/3750853908 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f549c0a0710 msgr2=0x7f549c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.677+0000 7f54ba7ff700 1 --2- 172.21.15.2:0/3750853908 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f549c0a0710 0x7f549c0a2bd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f54a4009df0 tx=0x7f54a4009260 comp rx=0 tx=0).stop 2023-09-13T04:37:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.677+0000 7f54ba7ff700 1 -- 172.21.15.2:0/3750853908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54b4074cf0 msgr2=0x7f54b4199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.677+0000 7f54ba7ff700 1 --2- 172.21.15.2:0/3750853908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54b4074cf0 0x7f54b4199130 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f54a800b2e0 tx=0x7f54a8004730 comp rx=0 tx=0).stop 2023-09-13T04:37:26.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.677+0000 7f54ba7ff700 1 -- 172.21.15.2:0/3750853908 shutdown_connections 2023-09-13T04:37:26.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.677+0000 7f54ba7ff700 1 --2- 172.21.15.2:0/3750853908 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f549c0a0710 0x7f549c0a2bd0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:26.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.677+0000 7f54ba7ff700 1 --2- 172.21.15.2:0/3750853908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54b4108bb0 0x7f54b41a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:26.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.677+0000 7f54ba7ff700 1 --2- 172.21.15.2:0/3750853908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54b4073710 0x7f54b4198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:26.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.677+0000 7f54ba7ff700 1 --2- 172.21.15.2:0/3750853908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54b4074cf0 0x7f54b4199130 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:26.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.677+0000 7f54ba7ff700 1 -- 172.21.15.2:0/3750853908 >> 172.21.15.2:0/3750853908 conn(0x7f54b40fc220 msgr2=0x7f54b40fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:26.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.677+0000 7f54ba7ff700 1 -- 172.21.15.2:0/3750853908 shutdown_connections 2023-09-13T04:37:26.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:26.677+0000 7f54ba7ff700 1 -- 172.21.15.2:0/3750853908 wait complete. 2023-09-13T04:37:26.681 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 2 2023-09-13T04:37:26.857 INFO:journalctl@ceph.mds.k.smithi149.stdout:Sep 13 04:37:26 smithi149 podman[168909]: 2023-09-13T04:37:26.857 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:26 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3750853908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:27.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:26 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3750853908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:27.136 INFO:journalctl@ceph.mds.k.smithi149.stdout:Sep 13 04:37:26 smithi149 ceph-mds[168949]: -- 172.21.15.149:0/240692066 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561c677231e0 con 0x561c676f9000 2023-09-13T04:37:27.136 INFO:journalctl@ceph.mds.k.smithi149.stdout:Sep 13 04:37:26 smithi149 bash[168909]: 4532367ab42a163eee42e7bc5c7fbda77b6e8b2f60ef0a1b7fd93148b2f4714f 2023-09-13T04:37:27.137 INFO:journalctl@ceph.mds.k.smithi149.stdout:Sep 13 04:37:26 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-k[168925]: starting mds.k at 2023-09-13T04:37:27.137 INFO:journalctl@ceph.mds.k.smithi149.stdout:Sep 13 04:37:26 smithi149 systemd[1]: Started Ceph mds.k for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:37:27.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:26 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3750853908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:28.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:27 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:28.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:27 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:28.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:27 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:28.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:27 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:28.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:27 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:28.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:27 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:28.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:27 smithi002 ceph-mon[97651]: Deploying daemon mds.j on smithi002 2023-09-13T04:37:28.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:27 smithi002 ceph-mon[97651]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:28.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:27 smithi002 ceph-mon[97651]: mds.? [v2:172.21.15.149:6832/3368344346,v1:172.21.15.149:6833/3368344346] up:boot 2023-09-13T04:37:28.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:27 smithi002 ceph-mon[97651]: fsmap 2 up:standby 2023-09-13T04:37:28.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:27 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-13T04:37:28.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:27 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:28.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:27 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:28.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:27 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:28.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:27 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:28.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:27 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:28.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:27 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:28.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:27 smithi149 ceph-mon[133315]: Deploying daemon mds.j on smithi002 2023-09-13T04:37:28.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:27 smithi149 ceph-mon[133315]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:28.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:27 smithi149 ceph-mon[133315]: mds.? [v2:172.21.15.149:6832/3368344346,v1:172.21.15.149:6833/3368344346] up:boot 2023-09-13T04:37:28.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:27 smithi149 ceph-mon[133315]: fsmap 2 up:standby 2023-09-13T04:37:28.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:27 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-13T04:37:28.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:27 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:28.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:27 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:28.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:27 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:28.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:27 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:28.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:27 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:28.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:27 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:28.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:27 smithi156 ceph-mon[133675]: Deploying daemon mds.j on smithi002 2023-09-13T04:37:28.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:27 smithi156 ceph-mon[133675]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:28.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:27 smithi156 ceph-mon[133675]: mds.? [v2:172.21.15.149:6832/3368344346,v1:172.21.15.149:6833/3368344346] up:boot 2023-09-13T04:37:28.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:27 smithi156 ceph-mon[133675]: fsmap 2 up:standby 2023-09-13T04:37:28.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:27 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-13T04:37:28.697 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:37:29.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.144+0000 7f854cbff700 1 -- 172.21.15.2:0/1605688072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8548100cd0 msgr2=0x7f85481010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:29.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.144+0000 7f854cbff700 1 --2- 172.21.15.2:0/1605688072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8548100cd0 0x7f85481010f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f85380099c0 tx=0x7f8538009cd0 comp rx=0 tx=0).stop 2023-09-13T04:37:29.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.144+0000 7f854cbff700 1 -- 172.21.15.2:0/1605688072 shutdown_connections 2023-09-13T04:37:29.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.144+0000 7f854cbff700 1 --2- 172.21.15.2:0/1605688072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8548102860 0x7f8548109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:29.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.144+0000 7f854cbff700 1 --2- 172.21.15.2:0/1605688072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8548100cd0 0x7f85481010f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:29.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.144+0000 7f854cbff700 1 --2- 172.21.15.2:0/1605688072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8548101ec0 0x7f8548102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:29.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.144+0000 7f854cbff700 1 -- 172.21.15.2:0/1605688072 >> 172.21.15.2:0/1605688072 conn(0x7f85480fc220 msgr2=0x7f85480fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:29.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.144+0000 7f854cbff700 1 -- 172.21.15.2:0/1605688072 shutdown_connections 2023-09-13T04:37:29.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.144+0000 7f854cbff700 1 -- 172.21.15.2:0/1605688072 wait complete. 2023-09-13T04:37:29.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.146+0000 7f854cbff700 1 Processor -- start 2023-09-13T04:37:29.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.146+0000 7f854cbff700 1 -- start start 2023-09-13T04:37:29.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.146+0000 7f854cbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8548100cd0 0x7f8548074a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:29.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.146+0000 7f854cbff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8548101ec0 0x7f85480730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:29.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.146+0000 7f854cbff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8548102860 0x7f85480735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:29.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.146+0000 7f854cbff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8548073d80 con 0x7f8548102860 2023-09-13T04:37:29.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.146+0000 7f854cbff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8548073ef0 con 0x7f8548101ec0 2023-09-13T04:37:29.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.146+0000 7f854cbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85481a0950 con 0x7f8548100cd0 2023-09-13T04:37:29.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.146+0000 7f8546d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8548102860 0x7f85480735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:29.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.147+0000 7f8546d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8548102860 0x7f85480735e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59078/0 (socket says 172.21.15.2:59078) 2023-09-13T04:37:29.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.147+0000 7f8546d9d700 1 -- 172.21.15.2:0/1501186542 learned_addr learned my addr 172.21.15.2:0/1501186542 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:29.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.147+0000 7f854659c700 1 --2- 172.21.15.2:0/1501186542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8548100cd0 0x7f8548074a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:29.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.147+0000 7f8545d9b700 1 --2- 172.21.15.2:0/1501186542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8548101ec0 0x7f85480730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:29.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.147+0000 7f8546d9d700 1 -- 172.21.15.2:0/1501186542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8548100cd0 msgr2=0x7f8548074a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:29.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.147+0000 7f8546d9d700 1 --2- 172.21.15.2:0/1501186542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8548100cd0 0x7f8548074a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:29.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.147+0000 7f8546d9d700 1 -- 172.21.15.2:0/1501186542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8548101ec0 msgr2=0x7f85480730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:29.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.147+0000 7f8546d9d700 1 --2- 172.21.15.2:0/1501186542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8548101ec0 0x7f85480730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:29.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.147+0000 7f8546d9d700 1 -- 172.21.15.2:0/1501186542 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85380095f0 con 0x7f8548102860 2023-09-13T04:37:29.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.147+0000 7f8546d9d700 1 --2- 172.21.15.2:0/1501186542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8548102860 0x7f85480735e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f854000ada0 tx=0x7f854000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:29.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.147+0000 7f85377fe700 1 -- 172.21.15.2:0/1501186542 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854000f070 con 0x7f8548102860 2023-09-13T04:37:29.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.148+0000 7f854cbff700 1 -- 172.21.15.2:0/1501186542 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85481a0c30 con 0x7f8548102860 2023-09-13T04:37:29.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.148+0000 7f85377fe700 1 -- 172.21.15.2:0/1501186542 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8540007b30 con 0x7f8548102860 2023-09-13T04:37:29.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.148+0000 7f854cbff700 1 -- 172.21.15.2:0/1501186542 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85481a1140 con 0x7f8548102860 2023-09-13T04:37:29.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.148+0000 7f85377fe700 1 -- 172.21.15.2:0/1501186542 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8540005280 con 0x7f8548102860 2023-09-13T04:37:29.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.149+0000 7f854cbff700 1 -- 172.21.15.2:0/1501186542 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8548107bb0 con 0x7f8548102860 2023-09-13T04:37:29.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.151+0000 7f85377fe700 1 -- 172.21.15.2:0/1501186542 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f854000c8d0 con 0x7f8548102860 2023-09-13T04:37:29.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.152+0000 7f85377fe700 1 --2- 172.21.15.2:0/1501186542 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f85300a0690 0x7f85300a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:29.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.152+0000 7f85377fe700 1 -- 172.21.15.2:0/1501186542 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f85400cace0 con 0x7f8548102860 2023-09-13T04:37:29.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.153+0000 7f854659c700 1 --2- 172.21.15.2:0/1501186542 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f85300a0690 0x7f85300a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:29.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.153+0000 7f854659c700 1 --2- 172.21.15.2:0/1501186542 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f85300a0690 0x7f85300a2b50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8538009f50 tx=0x7f853800b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:29.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.157+0000 7f85377fe700 1 -- 172.21.15.2:0/1501186542 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8540098730 con 0x7f8548102860 2023-09-13T04:37:29.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.428+0000 7f854cbff700 1 -- 172.21.15.2:0/1501186542 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f854810c4e0 con 0x7f8548102860 2023-09-13T04:37:29.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.429+0000 7f85377fe700 1 -- 172.21.15.2:0/1501186542 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7f85400982c0 con 0x7f8548102860 2023-09-13T04:37:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:37:29.430 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-09-13T04:37:29.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.432+0000 7f854cbff700 1 -- 172.21.15.2:0/1501186542 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f85300a0690 msgr2=0x7f85300a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:29.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.432+0000 7f854cbff700 1 --2- 172.21.15.2:0/1501186542 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f85300a0690 0x7f85300a2b50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8538009f50 tx=0x7f853800b540 comp rx=0 tx=0).stop 2023-09-13T04:37:29.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.432+0000 7f854cbff700 1 -- 172.21.15.2:0/1501186542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8548102860 msgr2=0x7f85480735e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:29.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.432+0000 7f854cbff700 1 --2- 172.21.15.2:0/1501186542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8548102860 0x7f85480735e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f854000ada0 tx=0x7f854000c330 comp rx=0 tx=0).stop 2023-09-13T04:37:29.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.432+0000 7f854cbff700 1 -- 172.21.15.2:0/1501186542 shutdown_connections 2023-09-13T04:37:29.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.432+0000 7f854cbff700 1 --2- 172.21.15.2:0/1501186542 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f85300a0690 0x7f85300a2b50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:29.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.432+0000 7f854cbff700 1 --2- 172.21.15.2:0/1501186542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8548102860 0x7f85480735e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:29.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.432+0000 7f854cbff700 1 --2- 172.21.15.2:0/1501186542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8548100cd0 0x7f8548074a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:29.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.432+0000 7f854cbff700 1 --2- 172.21.15.2:0/1501186542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8548101ec0 0x7f85480730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:29.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.432+0000 7f854cbff700 1 -- 172.21.15.2:0/1501186542 >> 172.21.15.2:0/1501186542 conn(0x7f85480fc220 msgr2=0x7f85480fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:29.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.432+0000 7f854cbff700 1 -- 172.21.15.2:0/1501186542 shutdown_connections 2023-09-13T04:37:29.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:29.432+0000 7f854cbff700 1 -- 172.21.15.2:0/1501186542 wait complete. 2023-09-13T04:37:29.435 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 3 2023-09-13T04:37:29.756 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 13 04:37:29 smithi002 systemd[1]: Starting Ceph mds.j for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:37:30.092 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 13 04:37:29 smithi002 podman[147420]: 2023-09-13T04:37:30.092 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 13 04:37:30 smithi002 bash[147420]: 703b451296b96deffe8dd184f8000c8702a7342260254b327f9bafb4fdc6174b 2023-09-13T04:37:30.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:30 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1501186542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:30.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:30 smithi002 ceph-mon[97651]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:30.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:30 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:30.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:30 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:30.507 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 13 04:37:30 smithi002 ceph-mds[147460]: -- 172.21.15.2:0/1696242653 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55835232f1e0 con 0x558352305000 2023-09-13T04:37:30.507 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 13 04:37:30 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-j[147436]: starting mds.j at 2023-09-13T04:37:30.507 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 13 04:37:30 smithi002 systemd[1]: Started Ceph mds.j for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:37:30.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:30 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1501186542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:30.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:30 smithi149 ceph-mon[133315]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:30.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:30 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:30.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:30 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:30.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:30 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1501186542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:30.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:30 smithi156 ceph-mon[133675]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:30.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:30 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:30.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:30 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:31.453 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:37:31.501 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:31 smithi002 ceph-mon[97651]: mds.? [v2:172.21.15.2:6834/2478119463,v1:172.21.15.2:6835/2478119463] up:boot 2023-09-13T04:37:31.501 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:31 smithi002 ceph-mon[97651]: fsmap 3 up:standby 2023-09-13T04:37:31.501 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:31 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-13T04:37:31.501 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:31 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:31.502 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:31 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:31.502 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:31 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:31.502 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:31 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:31.502 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:31 smithi002 ceph-mon[97651]: Deploying daemon mds.d on smithi002 2023-09-13T04:37:31.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:31 smithi149 ceph-mon[133315]: mds.? [v2:172.21.15.2:6834/2478119463,v1:172.21.15.2:6835/2478119463] up:boot 2023-09-13T04:37:31.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:31 smithi149 ceph-mon[133315]: fsmap 3 up:standby 2023-09-13T04:37:31.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:31 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-13T04:37:31.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:31 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:31.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:31 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:31.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:31 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:31.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:31 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:31.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:31 smithi149 ceph-mon[133315]: Deploying daemon mds.d on smithi002 2023-09-13T04:37:31.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.669+0000 7f1907bff700 1 -- 172.21.15.2:0/3626306152 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1900074cf0 msgr2=0x7f1900073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:31.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.669+0000 7f1907bff700 1 --2- 172.21.15.2:0/3626306152 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1900074cf0 0x7f1900073140 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009990 tx=0x7f18fc009ca0 comp rx=0 tx=0).stop 2023-09-13T04:37:31.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.669+0000 7f1907bff700 1 -- 172.21.15.2:0/3626306152 shutdown_connections 2023-09-13T04:37:31.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.669+0000 7f1907bff700 1 --2- 172.21.15.2:0/3626306152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1900108bb0 0x7f190010b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:31.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.669+0000 7f1907bff700 1 --2- 172.21.15.2:0/3626306152 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1900074cf0 0x7f1900073140 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:31.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.669+0000 7f1907bff700 1 --2- 172.21.15.2:0/3626306152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1900073710 0x7f1900073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:31.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.669+0000 7f1907bff700 1 -- 172.21.15.2:0/3626306152 >> 172.21.15.2:0/3626306152 conn(0x7f19000fc220 msgr2=0x7f19000fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:31.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.669+0000 7f1907bff700 1 -- 172.21.15.2:0/3626306152 shutdown_connections 2023-09-13T04:37:31.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.669+0000 7f1907bff700 1 -- 172.21.15.2:0/3626306152 wait complete. 2023-09-13T04:37:31.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.670+0000 7f1907bff700 1 Processor -- start 2023-09-13T04:37:31.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.670+0000 7f1907bff700 1 -- start start 2023-09-13T04:37:31.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.671+0000 7f1907bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1900073710 0x7f1900198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:31.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.671+0000 7f1907bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1900074cf0 0x7f1900199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:31.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.671+0000 7f1907bff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1900108bb0 0x7f19001a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:31.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.671+0000 7f1907bff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19001a0750 con 0x7f1900108bb0 2023-09-13T04:37:31.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.671+0000 7f1907bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19001a0890 con 0x7f1900074cf0 2023-09-13T04:37:31.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.671+0000 7f1907bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19001a0a20 con 0x7f1900073710 2023-09-13T04:37:31.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.671+0000 7f190619c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1900108bb0 0x7f19001a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:31.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.671+0000 7f190619c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1900108bb0 0x7f19001a0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59126/0 (socket says 172.21.15.2:59126) 2023-09-13T04:37:31.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.671+0000 7f190619c700 1 -- 172.21.15.2:0/3825348333 learned_addr learned my addr 172.21.15.2:0/3825348333 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:31.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.671+0000 7f190519a700 1 --2- 172.21.15.2:0/3825348333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1900074cf0 0x7f1900199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:31.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.671+0000 7f190599b700 1 --2- 172.21.15.2:0/3825348333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1900073710 0x7f1900198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:31.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.672+0000 7f190619c700 1 -- 172.21.15.2:0/3825348333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1900073710 msgr2=0x7f1900198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:31.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.672+0000 7f190619c700 1 --2- 172.21.15.2:0/3825348333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1900073710 0x7f1900198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:31.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.672+0000 7f190619c700 1 -- 172.21.15.2:0/3825348333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1900074cf0 msgr2=0x7f1900199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:31.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.672+0000 7f190619c700 1 --2- 172.21.15.2:0/3825348333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1900074cf0 0x7f1900199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:31.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.672+0000 7f190619c700 1 -- 172.21.15.2:0/3825348333 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc0095f0 con 0x7f1900108bb0 2023-09-13T04:37:31.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.672+0000 7f190519a700 1 --2- 172.21.15.2:0/3825348333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1900074cf0 0x7f1900199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:37:31.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.672+0000 7f190619c700 1 --2- 172.21.15.2:0/3825348333 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1900108bb0 0x7f19001a0180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f18f800ef30 tx=0x7f18f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:31.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.672+0000 7f18f6ffd700 1 -- 172.21.15.2:0/3825348333 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f8018070 con 0x7f1900108bb0 2023-09-13T04:37:31.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.672+0000 7f18f6ffd700 1 -- 172.21.15.2:0/3825348333 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18f800f040 con 0x7f1900108bb0 2023-09-13T04:37:31.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.672+0000 7f1907bff700 1 -- 172.21.15.2:0/3825348333 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19001a0ca0 con 0x7f1900108bb0 2023-09-13T04:37:31.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.672+0000 7f18f6ffd700 1 -- 172.21.15.2:0/3825348333 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f800cd00 con 0x7f1900108bb0 2023-09-13T04:37:31.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.672+0000 7f1907bff700 1 -- 172.21.15.2:0/3825348333 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19001a1160 con 0x7f1900108bb0 2023-09-13T04:37:31.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.673+0000 7f1907bff700 1 -- 172.21.15.2:0/3825348333 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1900107260 con 0x7f1900108bb0 2023-09-13T04:37:31.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.675+0000 7f18f6ffd700 1 -- 172.21.15.2:0/3825348333 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f18f8003a00 con 0x7f1900108bb0 2023-09-13T04:37:31.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.677+0000 7f18f6ffd700 1 --2- 172.21.15.2:0/3825348333 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f18ec0a0690 0x7f18ec0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:31.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.677+0000 7f18f6ffd700 1 -- 172.21.15.2:0/3825348333 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f18f80cb2f0 con 0x7f1900108bb0 2023-09-13T04:37:31.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.677+0000 7f190599b700 1 --2- 172.21.15.2:0/3825348333 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f18ec0a0690 0x7f18ec0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:31.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.677+0000 7f190599b700 1 --2- 172.21.15.2:0/3825348333 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f18ec0a0690 0x7f18ec0a2b50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00b2e0 tx=0x7f18fc005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:31.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.681+0000 7f18f6ffd700 1 -- 172.21.15.2:0/3825348333 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f18f80952b0 con 0x7f1900108bb0 2023-09-13T04:37:31.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:31 smithi156 ceph-mon[133675]: mds.? [v2:172.21.15.2:6834/2478119463,v1:172.21.15.2:6835/2478119463] up:boot 2023-09-13T04:37:31.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:31 smithi156 ceph-mon[133675]: fsmap 3 up:standby 2023-09-13T04:37:31.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:31 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-13T04:37:31.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:31 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:31.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:31 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:31.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:31 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:31.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:31 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:31.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:31 smithi156 ceph-mon[133675]: Deploying daemon mds.d on smithi002 2023-09-13T04:37:31.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.962+0000 7f1907bff700 1 -- 172.21.15.2:0/3825348333 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f190010e730 con 0x7f1900108bb0 2023-09-13T04:37:31.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.963+0000 7f18f6ffd700 1 -- 172.21.15.2:0/3825348333 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7f18f80988d0 con 0x7f1900108bb0 2023-09-13T04:37:31.966 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:37:31.967 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-09-13T04:37:31.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.965+0000 7f1907bff700 1 -- 172.21.15.2:0/3825348333 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f18ec0a0690 msgr2=0x7f18ec0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:31.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.965+0000 7f1907bff700 1 --2- 172.21.15.2:0/3825348333 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f18ec0a0690 0x7f18ec0a2b50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00b2e0 tx=0x7f18fc005fb0 comp rx=0 tx=0).stop 2023-09-13T04:37:31.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.965+0000 7f1907bff700 1 -- 172.21.15.2:0/3825348333 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1900108bb0 msgr2=0x7f19001a0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:31.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.965+0000 7f1907bff700 1 --2- 172.21.15.2:0/3825348333 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1900108bb0 0x7f19001a0180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f18f800ef30 tx=0x7f18f800c3a0 comp rx=0 tx=0).stop 2023-09-13T04:37:31.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.965+0000 7f1907bff700 1 -- 172.21.15.2:0/3825348333 shutdown_connections 2023-09-13T04:37:31.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.965+0000 7f1907bff700 1 --2- 172.21.15.2:0/3825348333 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f18ec0a0690 0x7f18ec0a2b50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:31.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.965+0000 7f1907bff700 1 --2- 172.21.15.2:0/3825348333 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1900108bb0 0x7f19001a0180 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:31.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.965+0000 7f1907bff700 1 --2- 172.21.15.2:0/3825348333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1900073710 0x7f1900198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:31.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.966+0000 7f1907bff700 1 --2- 172.21.15.2:0/3825348333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1900074cf0 0x7f1900199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:31.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.966+0000 7f1907bff700 1 -- 172.21.15.2:0/3825348333 >> 172.21.15.2:0/3825348333 conn(0x7f19000fc220 msgr2=0x7f19000fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:31.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.966+0000 7f1907bff700 1 -- 172.21.15.2:0/3825348333 shutdown_connections 2023-09-13T04:37:31.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:31.966+0000 7f1907bff700 1 -- 172.21.15.2:0/3825348333 wait complete. 2023-09-13T04:37:31.970 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 4 2023-09-13T04:37:32.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:32 smithi002 ceph-mon[97651]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:32.452 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:32 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3825348333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:32.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:32 smithi149 ceph-mon[133315]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:32.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:32 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3825348333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:32.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:32 smithi156 ceph-mon[133675]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:32.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:32 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3825348333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:32.755 INFO:journalctl@ceph.mds.d.smithi002.stdout:Sep 13 04:37:32 smithi002 systemd[1]: Starting Ceph mds.d for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:37:33.223 INFO:journalctl@ceph.mds.d.smithi002.stdout:Sep 13 04:37:32 smithi002 podman[147805]: 2023-09-13T04:37:33.505 INFO:journalctl@ceph.mds.d.smithi002.stdout:Sep 13 04:37:33 smithi002 bash[147805]: 4ff4715e5eed64b894ebbf23cf99406dc206bf08ac5980c0d64f1bcb54149f7a 2023-09-13T04:37:33.506 INFO:journalctl@ceph.mds.d.smithi002.stdout:Sep 13 04:37:33 smithi002 ceph-mds[147846]: -- 172.21.15.2:0/3188370764 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561846f251e0 con 0x561846efac00 2023-09-13T04:37:33.506 INFO:journalctl@ceph.mds.d.smithi002.stdout:Sep 13 04:37:33 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-d[147821]: starting mds.d at 2023-09-13T04:37:33.506 INFO:journalctl@ceph.mds.d.smithi002.stdout:Sep 13 04:37:33 smithi002 systemd[1]: Started Ceph mds.d for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:37:33.987 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:37:34.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.204+0000 7f89ad5ff700 1 -- 172.21.15.2:0/3830931143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a8108bc0 msgr2=0x7f89a810b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:34.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.204+0000 7f89ad5ff700 1 --2- 172.21.15.2:0/3830931143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a8108bc0 0x7f89a810b030 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f89a000b0a0 tx=0x7f89a000b3b0 comp rx=0 tx=0).stop 2023-09-13T04:37:34.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.205+0000 7f89ad5ff700 1 -- 172.21.15.2:0/3830931143 shutdown_connections 2023-09-13T04:37:34.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.205+0000 7f89ad5ff700 1 --2- 172.21.15.2:0/3830931143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a8108bc0 0x7f89a810b030 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:34.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.205+0000 7f89ad5ff700 1 --2- 172.21.15.2:0/3830931143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89a8074c90 0x7f89a80730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:34.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.205+0000 7f89ad5ff700 1 --2- 172.21.15.2:0/3830931143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a80736b0 0x7f89a8073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:34.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.205+0000 7f89ad5ff700 1 -- 172.21.15.2:0/3830931143 >> 172.21.15.2:0/3830931143 conn(0x7f89a80fc220 msgr2=0x7f89a80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:34.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.205+0000 7f89ad5ff700 1 -- 172.21.15.2:0/3830931143 shutdown_connections 2023-09-13T04:37:34.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.205+0000 7f89ad5ff700 1 -- 172.21.15.2:0/3830931143 wait complete. 2023-09-13T04:37:34.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.206+0000 7f89ad5ff700 1 Processor -- start 2023-09-13T04:37:34.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.206+0000 7f89ad5ff700 1 -- start start 2023-09-13T04:37:34.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.206+0000 7f89ad5ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a80736b0 0x7f89a8198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:34.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.206+0000 7f89ad5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89a8074c90 0x7f89a81991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:34.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.206+0000 7f89ad5ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a8108bc0 0x7f89a81a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:34.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.206+0000 7f89ad5ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a81a07e0 con 0x7f89a8108bc0 2023-09-13T04:37:34.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.206+0000 7f89ad5ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a81a0950 con 0x7f89a80736b0 2023-09-13T04:37:34.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.206+0000 7f89ad5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a81a0b10 con 0x7f89a8074c90 2023-09-13T04:37:34.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.206+0000 7f89a77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a8108bc0 0x7f89a81a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:34.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.206+0000 7f89a77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a8108bc0 0x7f89a81a0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53784/0 (socket says 172.21.15.2:53784) 2023-09-13T04:37:34.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.206+0000 7f89a77fe700 1 -- 172.21.15.2:0/1269335978 learned_addr learned my addr 172.21.15.2:0/1269335978 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:34.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.206+0000 7f89a67fc700 1 --2- 172.21.15.2:0/1269335978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89a8074c90 0x7f89a81991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:34.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.207+0000 7f89a6ffd700 1 --2- 172.21.15.2:0/1269335978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a80736b0 0x7f89a8198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:34.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.207+0000 7f89a77fe700 1 -- 172.21.15.2:0/1269335978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89a8074c90 msgr2=0x7f89a81991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:34.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.207+0000 7f89a77fe700 1 --2- 172.21.15.2:0/1269335978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89a8074c90 0x7f89a81991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:34.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.207+0000 7f89a77fe700 1 -- 172.21.15.2:0/1269335978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a80736b0 msgr2=0x7f89a8198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:34.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.207+0000 7f89a77fe700 1 --2- 172.21.15.2:0/1269335978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a80736b0 0x7f89a8198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:34.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.207+0000 7f89a77fe700 1 -- 172.21.15.2:0/1269335978 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a0009d00 con 0x7f89a8108bc0 2023-09-13T04:37:34.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.207+0000 7f89a77fe700 1 --2- 172.21.15.2:0/1269335978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a8108bc0 0x7f89a81a0210 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009fd0 tx=0x7f89a000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:34.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.207+0000 7f898ffff700 1 -- 172.21.15.2:0/1269335978 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a000e040 con 0x7f89a8108bc0 2023-09-13T04:37:34.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.207+0000 7f89ad5ff700 1 -- 172.21.15.2:0/1269335978 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a81a0d90 con 0x7f89a8108bc0 2023-09-13T04:37:34.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.207+0000 7f898ffff700 1 -- 172.21.15.2:0/1269335978 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89a0003e10 con 0x7f89a8108bc0 2023-09-13T04:37:34.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.207+0000 7f898ffff700 1 -- 172.21.15.2:0/1269335978 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a001b6a0 con 0x7f89a8108bc0 2023-09-13T04:37:34.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.208+0000 7f89ad5ff700 1 -- 172.21.15.2:0/1269335978 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a81a11c0 con 0x7f89a8108bc0 2023-09-13T04:37:34.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.209+0000 7f89ad5ff700 1 -- 172.21.15.2:0/1269335978 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a81072c0 con 0x7f89a8108bc0 2023-09-13T04:37:34.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.210+0000 7f898ffff700 1 -- 172.21.15.2:0/1269335978 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f89a001d470 con 0x7f89a8108bc0 2023-09-13T04:37:34.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.212+0000 7f898ffff700 1 --2- 172.21.15.2:0/1269335978 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f89900a0710 0x7f89900a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:34.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.212+0000 7f898ffff700 1 -- 172.21.15.2:0/1269335978 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f89a00cc880 con 0x7f89a8108bc0 2023-09-13T04:37:34.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.212+0000 7f89a6ffd700 1 --2- 172.21.15.2:0/1269335978 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f89900a0710 0x7f89900a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:34.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.213+0000 7f89a6ffd700 1 --2- 172.21.15.2:0/1269335978 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f89900a0710 0x7f89900a2bd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f89a8199b40 tx=0x7f8998009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:34.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.216+0000 7f898ffff700 1 -- 172.21.15.2:0/1269335978 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f89a0096840 con 0x7f89a8108bc0 2023-09-13T04:37:34.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.479+0000 7f89ad5ff700 1 -- 172.21.15.2:0/1269335978 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89a810e720 con 0x7f89a8108bc0 2023-09-13T04:37:34.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.479+0000 7f898ffff700 1 -- 172.21.15.2:0/1269335978 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3695 (secure 0 0 0) 0x7f89a0099e60 con 0x7f89a8108bc0 2023-09-13T04:37:34.480 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:37:34.480 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[]} 2023-09-13T04:37:34.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.481+0000 7f89ad5ff700 1 -- 172.21.15.2:0/1269335978 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f89900a0710 msgr2=0x7f89900a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:34.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.481+0000 7f89ad5ff700 1 --2- 172.21.15.2:0/1269335978 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f89900a0710 0x7f89900a2bd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f89a8199b40 tx=0x7f8998009260 comp rx=0 tx=0).stop 2023-09-13T04:37:34.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.481+0000 7f89ad5ff700 1 -- 172.21.15.2:0/1269335978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a8108bc0 msgr2=0x7f89a81a0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:34.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.481+0000 7f89ad5ff700 1 --2- 172.21.15.2:0/1269335978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a8108bc0 0x7f89a81a0210 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009fd0 tx=0x7f89a000bce0 comp rx=0 tx=0).stop 2023-09-13T04:37:34.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.482+0000 7f89ad5ff700 1 -- 172.21.15.2:0/1269335978 shutdown_connections 2023-09-13T04:37:34.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.482+0000 7f89ad5ff700 1 --2- 172.21.15.2:0/1269335978 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f89900a0710 0x7f89900a2bd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:34.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.482+0000 7f89ad5ff700 1 --2- 172.21.15.2:0/1269335978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a8108bc0 0x7f89a81a0210 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:34.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.482+0000 7f89ad5ff700 1 --2- 172.21.15.2:0/1269335978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a80736b0 0x7f89a8198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:34.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.482+0000 7f89ad5ff700 1 --2- 172.21.15.2:0/1269335978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89a8074c90 0x7f89a81991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:34.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.482+0000 7f89ad5ff700 1 -- 172.21.15.2:0/1269335978 >> 172.21.15.2:0/1269335978 conn(0x7f89a80fc220 msgr2=0x7f89a810adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:34.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.482+0000 7f89ad5ff700 1 -- 172.21.15.2:0/1269335978 shutdown_connections 2023-09-13T04:37:34.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:34.482+0000 7f89ad5ff700 1 -- 172.21.15.2:0/1269335978 wait complete. 2023-09-13T04:37:34.485 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 5 2023-09-13T04:37:34.499 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:34 smithi002 ceph-mon[97651]: mds.? [v2:172.21.15.2:6836/258142559,v1:172.21.15.2:6837/258142559] up:boot 2023-09-13T04:37:34.499 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:34 smithi002 ceph-mon[97651]: fsmap 4 up:standby 2023-09-13T04:37:34.499 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:34 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-13T04:37:34.499 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:34 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:34.500 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:34 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:34.500 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:34 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:34.500 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:34 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:34.500 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:34 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:34.501 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:34 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:34.501 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:34 smithi002 ceph-mon[97651]: Deploying daemon mds.e on smithi149 2023-09-13T04:37:34.501 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:34 smithi002 ceph-mon[97651]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:34.629 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:34 smithi149 ceph-mon[133315]: mds.? [v2:172.21.15.2:6836/258142559,v1:172.21.15.2:6837/258142559] up:boot 2023-09-13T04:37:34.629 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:34 smithi149 ceph-mon[133315]: fsmap 4 up:standby 2023-09-13T04:37:34.629 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:34 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-13T04:37:34.629 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:34 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:34.630 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:34 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:34.630 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:34 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:34.630 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:34 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:34.630 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:34 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:34.631 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:34 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:34.631 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:34 smithi149 ceph-mon[133315]: Deploying daemon mds.e on smithi149 2023-09-13T04:37:34.631 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:34 smithi149 ceph-mon[133315]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:34.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:34 smithi156 ceph-mon[133675]: mds.? [v2:172.21.15.2:6836/258142559,v1:172.21.15.2:6837/258142559] up:boot 2023-09-13T04:37:34.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:34 smithi156 ceph-mon[133675]: fsmap 4 up:standby 2023-09-13T04:37:34.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:34 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-13T04:37:34.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:34 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:34.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:34 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:34.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:34 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:34.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:34 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:34.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:34 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:34.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:34 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:34.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:34 smithi156 ceph-mon[133675]: Deploying daemon mds.e on smithi149 2023-09-13T04:37:34.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:34 smithi156 ceph-mon[133675]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:35.300 INFO:journalctl@ceph.mds.e.smithi149.stdout:Sep 13 04:37:34 smithi149 systemd[1]: Starting Ceph mds.e for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:37:35.572 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:35 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1269335978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:35.572 INFO:journalctl@ceph.mds.e.smithi149.stdout:Sep 13 04:37:35 smithi149 podman[169250]: 2023-09-13T04:37:35.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:35 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1269335978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:35.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:35 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1269335978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:35.886 INFO:journalctl@ceph.mds.e.smithi149.stdout:Sep 13 04:37:35 smithi149 ceph-mds[169290]: -- 172.21.15.149:0/3800511666 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563fd81111e0 con 0x563fd80e6c00 2023-09-13T04:37:35.886 INFO:journalctl@ceph.mds.e.smithi149.stdout:Sep 13 04:37:35 smithi149 bash[169250]: 0618f1c6f3133ed306960e8a0ac20e7fc5b6d0e990c2f23f60e9df58a00312cc 2023-09-13T04:37:35.887 INFO:journalctl@ceph.mds.e.smithi149.stdout:Sep 13 04:37:35 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-e[169266]: starting mds.e at 2023-09-13T04:37:35.887 INFO:journalctl@ceph.mds.e.smithi149.stdout:Sep 13 04:37:35 smithi149 systemd[1]: Started Ceph mds.e for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:37:36.503 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:37:36.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.717+0000 7f1f9cbff700 1 -- 172.21.15.2:0/3602522601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f98102880 msgr2=0x7f1f98109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:36.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.717+0000 7f1f9cbff700 1 --2- 172.21.15.2:0/3602522601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f98102880 0x7f1f98109130 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1f9000b0d0 tx=0x7f1f9000b3e0 comp rx=0 tx=0).stop 2023-09-13T04:37:36.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.717+0000 7f1f9cbff700 1 -- 172.21.15.2:0/3602522601 shutdown_connections 2023-09-13T04:37:36.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.717+0000 7f1f9cbff700 1 --2- 172.21.15.2:0/3602522601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f98102880 0x7f1f98109130 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:36.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.717+0000 7f1f9cbff700 1 --2- 172.21.15.2:0/3602522601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f98100cf0 0x7f1f98101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:36.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.717+0000 7f1f9cbff700 1 --2- 172.21.15.2:0/3602522601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f98101ee0 0x7f1f98102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:36.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.717+0000 7f1f9cbff700 1 -- 172.21.15.2:0/3602522601 >> 172.21.15.2:0/3602522601 conn(0x7f1f980fc220 msgr2=0x7f1f980fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:36.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.717+0000 7f1f9cbff700 1 -- 172.21.15.2:0/3602522601 shutdown_connections 2023-09-13T04:37:36.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.717+0000 7f1f9cbff700 1 -- 172.21.15.2:0/3602522601 wait complete. 2023-09-13T04:37:36.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.719+0000 7f1f9cbff700 1 Processor -- start 2023-09-13T04:37:36.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.719+0000 7f1f9cbff700 1 -- start start 2023-09-13T04:37:36.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.719+0000 7f1f9cbff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f98100cf0 0x7f1f98196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:36.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.719+0000 7f1f9cbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f98101ee0 0x7f1f98196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:36.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.719+0000 7f1f9cbff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f98102880 0x7f1f9819e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:36.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.719+0000 7f1f9cbff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9819e5d0 con 0x7f1f98102880 2023-09-13T04:37:36.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.719+0000 7f1f9cbff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9819e710 con 0x7f1f98100cf0 2023-09-13T04:37:36.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.719+0000 7f1f9cbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9819e8a0 con 0x7f1f98101ee0 2023-09-13T04:37:36.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.720+0000 7f1f96d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f98102880 0x7f1f9819e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:36.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.720+0000 7f1f9659c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f98100cf0 0x7f1f98196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:36.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.720+0000 7f1f96d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f98102880 0x7f1f9819e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53800/0 (socket says 172.21.15.2:53800) 2023-09-13T04:37:36.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.720+0000 7f1f9659c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f98100cf0 0x7f1f98196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:51030/0 (socket says 172.21.15.2:51030) 2023-09-13T04:37:36.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.720+0000 7f1f96d9d700 1 -- 172.21.15.2:0/664462160 learned_addr learned my addr 172.21.15.2:0/664462160 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:36.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.720+0000 7f1f95d9b700 1 --2- 172.21.15.2:0/664462160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f98101ee0 0x7f1f98196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:36.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.720+0000 7f1f96d9d700 1 -- 172.21.15.2:0/664462160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f98101ee0 msgr2=0x7f1f98196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:36.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.720+0000 7f1f96d9d700 1 --2- 172.21.15.2:0/664462160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f98101ee0 0x7f1f98196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:36.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.720+0000 7f1f96d9d700 1 -- 172.21.15.2:0/664462160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f98100cf0 msgr2=0x7f1f98196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:36.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.720+0000 7f1f96d9d700 1 --2- 172.21.15.2:0/664462160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f98100cf0 0x7f1f98196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:36.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.720+0000 7f1f96d9d700 1 -- 172.21.15.2:0/664462160 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f90009d00 con 0x7f1f98102880 2023-09-13T04:37:36.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.720+0000 7f1f96d9d700 1 --2- 172.21.15.2:0/664462160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f98102880 0x7f1f9819e000 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1f90008ec0 tx=0x7f1f90008ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:36.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.720+0000 7f1f877fe700 1 -- 172.21.15.2:0/664462160 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9000e040 con 0x7f1f98102880 2023-09-13T04:37:36.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.721+0000 7f1f9cbff700 1 -- 172.21.15.2:0/664462160 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f9819eb20 con 0x7f1f98102880 2023-09-13T04:37:36.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.721+0000 7f1f877fe700 1 -- 172.21.15.2:0/664462160 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f90005070 con 0x7f1f98102880 2023-09-13T04:37:36.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.721+0000 7f1f877fe700 1 -- 172.21.15.2:0/664462160 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9001b6c0 con 0x7f1f98102880 2023-09-13T04:37:36.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.721+0000 7f1f9cbff700 1 -- 172.21.15.2:0/664462160 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f9819ef50 con 0x7f1f98102880 2023-09-13T04:37:36.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.722+0000 7f1f9cbff700 1 -- 172.21.15.2:0/664462160 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f981081f0 con 0x7f1f98102880 2023-09-13T04:37:36.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.724+0000 7f1f877fe700 1 -- 172.21.15.2:0/664462160 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f1f900051e0 con 0x7f1f98102880 2023-09-13T04:37:36.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.725+0000 7f1f877fe700 1 --2- 172.21.15.2:0/664462160 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1f800a0690 0x7f1f800a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:36.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.725+0000 7f1f877fe700 1 -- 172.21.15.2:0/664462160 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1f900cc850 con 0x7f1f98102880 2023-09-13T04:37:36.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.725+0000 7f1f9659c700 1 --2- 172.21.15.2:0/664462160 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1f800a0690 0x7f1f800a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:36.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.726+0000 7f1f9659c700 1 --2- 172.21.15.2:0/664462160 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1f800a0690 0x7f1f800a2b50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1f98101d40 tx=0x7f1f88009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:36.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.729+0000 7f1f877fe700 1 -- 172.21.15.2:0/664462160 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1f90096810 con 0x7f1f98102880 2023-09-13T04:37:36.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:36 smithi149 ceph-mon[133315]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:36.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:36 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:36.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:36 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:36.890 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:36 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:36.890 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:36 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:36.890 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:36 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:36.890 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:36 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:36.890 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:36 smithi149 ceph-mon[133315]: mds.? [v2:172.21.15.149:6834/725547291,v1:172.21.15.149:6835/725547291] up:boot 2023-09-13T04:37:36.891 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:36 smithi149 ceph-mon[133315]: fsmap 5 up:standby 2023-09-13T04:37:36.891 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:36 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-13T04:37:36.921 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:36 smithi156 ceph-mon[133675]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:36.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:36 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:36.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:36 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:36.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:36 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:36.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:36 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:36.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:36 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:36.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:36 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:36.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:36 smithi156 ceph-mon[133675]: mds.? [v2:172.21.15.149:6834/725547291,v1:172.21.15.149:6835/725547291] up:boot 2023-09-13T04:37:36.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:36 smithi156 ceph-mon[133675]: fsmap 5 up:standby 2023-09-13T04:37:36.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:36 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-13T04:37:36.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.987+0000 7f1f9cbff700 1 -- 172.21.15.2:0/664462160 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f9810c510 con 0x7f1f98102880 2023-09-13T04:37:36.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.988+0000 7f1f877fe700 1 -- 172.21.15.2:0/664462160 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4486 (secure 0 0 0) 0x7f1f90099e30 con 0x7f1f98102880 2023-09-13T04:37:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:37:36.990 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-09-13T04:37:36.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.990+0000 7f1f9cbff700 1 -- 172.21.15.2:0/664462160 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1f800a0690 msgr2=0x7f1f800a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:36.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.990+0000 7f1f9cbff700 1 --2- 172.21.15.2:0/664462160 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1f800a0690 0x7f1f800a2b50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1f98101d40 tx=0x7f1f88009260 comp rx=0 tx=0).stop 2023-09-13T04:37:36.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.990+0000 7f1f9cbff700 1 -- 172.21.15.2:0/664462160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f98102880 msgr2=0x7f1f9819e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:36.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.990+0000 7f1f9cbff700 1 --2- 172.21.15.2:0/664462160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f98102880 0x7f1f9819e000 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1f90008ec0 tx=0x7f1f90008ef0 comp rx=0 tx=0).stop 2023-09-13T04:37:36.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.990+0000 7f1f9cbff700 1 -- 172.21.15.2:0/664462160 shutdown_connections 2023-09-13T04:37:36.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.990+0000 7f1f9cbff700 1 --2- 172.21.15.2:0/664462160 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1f800a0690 0x7f1f800a2b50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:36.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.990+0000 7f1f9cbff700 1 --2- 172.21.15.2:0/664462160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f98102880 0x7f1f9819e000 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:36.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.990+0000 7f1f9cbff700 1 --2- 172.21.15.2:0/664462160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f98100cf0 0x7f1f98196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:36.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.990+0000 7f1f9cbff700 1 --2- 172.21.15.2:0/664462160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f98101ee0 0x7f1f98196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:36.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.990+0000 7f1f9cbff700 1 -- 172.21.15.2:0/664462160 >> 172.21.15.2:0/664462160 conn(0x7f1f980fc220 msgr2=0x7f1f98105330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:36.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.991+0000 7f1f9cbff700 1 -- 172.21.15.2:0/664462160 shutdown_connections 2023-09-13T04:37:36.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:36.991+0000 7f1f9cbff700 1 -- 172.21.15.2:0/664462160 wait complete. 2023-09-13T04:37:36.994 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 6 2023-09-13T04:37:37.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:36 smithi002 ceph-mon[97651]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:37.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:36 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:37.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:36 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:37.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:36 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:37.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:36 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:37.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:36 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:37.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:36 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:37.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:36 smithi002 ceph-mon[97651]: mds.? [v2:172.21.15.149:6834/725547291,v1:172.21.15.149:6835/725547291] up:boot 2023-09-13T04:37:37.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:36 smithi002 ceph-mon[97651]: fsmap 5 up:standby 2023-09-13T04:37:37.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:36 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-13T04:37:37.482 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 13 04:37:37 smithi156 systemd[1]: Starting Ceph mds.f for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:37:37.746 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:37 smithi156 ceph-mon[133675]: Deploying daemon mds.f on smithi156 2023-09-13T04:37:37.746 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:37 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/664462160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:37.746 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 13 04:37:37 smithi156 podman[171345]: 2023-09-13T04:37:37.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:37 smithi149 ceph-mon[133315]: Deploying daemon mds.f on smithi156 2023-09-13T04:37:37.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:37 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/664462160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:38.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:37 smithi002 ceph-mon[97651]: Deploying daemon mds.f on smithi156 2023-09-13T04:37:38.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:37 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/664462160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:38.186 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 13 04:37:37 smithi156 ceph-mds[171385]: -- 172.21.15.156:0/1787504324 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b22d0c71e0 con 0x55b22d09d000 2023-09-13T04:37:38.186 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 13 04:37:37 smithi156 bash[171345]: f215d1637903a3c7f7fdd610961f0785169712b576e08a53ef266f695c550f07 2023-09-13T04:37:38.187 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 13 04:37:37 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-f[171361]: starting mds.f at 2023-09-13T04:37:38.187 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 13 04:37:37 smithi156 systemd[1]: Started Ceph mds.f for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:37:38.747 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:38 smithi156 ceph-mon[133675]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:38.747 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:38 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:38.748 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:38 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:38.748 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:38 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:38.748 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:38 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:38.748 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:38 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:38.748 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:38 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:39.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:38 smithi002 ceph-mon[97651]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:39.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:38 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:39.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:38 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:39.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:38 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:39.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:38 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:39.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:38 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:39.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:38 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:39.011 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:37:39.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:38 smithi149 ceph-mon[133315]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:39.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:38 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:39.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:38 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:39.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:38 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:39.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:38 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:39.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:38 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:39.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:38 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:39.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.224+0000 7fe15b5ff700 1 -- 172.21.15.2:0/3364588793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe154100cf0 msgr2=0x7fe154101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:39.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.225+0000 7fe15b5ff700 1 --2- 172.21.15.2:0/3364588793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe154100cf0 0x7fe154101110 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe150009990 tx=0x7fe150009ca0 comp rx=0 tx=0).stop 2023-09-13T04:37:39.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.225+0000 7fe15b5ff700 1 -- 172.21.15.2:0/3364588793 shutdown_connections 2023-09-13T04:37:39.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.225+0000 7fe15b5ff700 1 --2- 172.21.15.2:0/3364588793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe154102880 0x7fe154109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:39.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.225+0000 7fe15b5ff700 1 --2- 172.21.15.2:0/3364588793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe154100cf0 0x7fe154101110 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:39.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.225+0000 7fe15b5ff700 1 --2- 172.21.15.2:0/3364588793 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe154101ee0 0x7fe154102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:39.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.225+0000 7fe15b5ff700 1 -- 172.21.15.2:0/3364588793 >> 172.21.15.2:0/3364588793 conn(0x7fe1540fc220 msgr2=0x7fe1540fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:39.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.225+0000 7fe15b5ff700 1 -- 172.21.15.2:0/3364588793 shutdown_connections 2023-09-13T04:37:39.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.225+0000 7fe15b5ff700 1 -- 172.21.15.2:0/3364588793 wait complete. 2023-09-13T04:37:39.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.226+0000 7fe15b5ff700 1 Processor -- start 2023-09-13T04:37:39.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.226+0000 7fe15b5ff700 1 -- start start 2023-09-13T04:37:39.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.226+0000 7fe15b5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe154100cf0 0x7fe154074a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:39.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.226+0000 7fe15b5ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe154101ee0 0x7fe1540730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:39.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.226+0000 7fe15b5ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe154102880 0x7fe1540735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:39.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.226+0000 7fe15b5ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe154073d80 con 0x7fe154101ee0 2023-09-13T04:37:39.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.226+0000 7fe15b5ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe154073ef0 con 0x7fe154102880 2023-09-13T04:37:39.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.226+0000 7fe15b5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1541a0a10 con 0x7fe154100cf0 2023-09-13T04:37:39.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.227+0000 7fe158b9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe154101ee0 0x7fe1540730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:39.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.227+0000 7fe15939b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe154100cf0 0x7fe154074a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:39.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.227+0000 7fe15939b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe154100cf0 0x7fe154074a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:49446/0 (socket says 172.21.15.2:49446) 2023-09-13T04:37:39.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.227+0000 7fe15939b700 1 -- 172.21.15.2:0/1978474740 learned_addr learned my addr 172.21.15.2:0/1978474740 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:39.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.227+0000 7fe159b9c700 1 --2- 172.21.15.2:0/1978474740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe154102880 0x7fe1540735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:39.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.227+0000 7fe15939b700 1 -- 172.21.15.2:0/1978474740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe154102880 msgr2=0x7fe1540735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:39.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.227+0000 7fe15939b700 1 --2- 172.21.15.2:0/1978474740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe154102880 0x7fe1540735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:39.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.227+0000 7fe15939b700 1 -- 172.21.15.2:0/1978474740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe154101ee0 msgr2=0x7fe1540730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:39.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.227+0000 7fe15939b700 1 --2- 172.21.15.2:0/1978474740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe154101ee0 0x7fe1540730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:39.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.227+0000 7fe15939b700 1 -- 172.21.15.2:0/1978474740 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe144009520 con 0x7fe154100cf0 2023-09-13T04:37:39.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.227+0000 7fe158b9a700 1 --2- 172.21.15.2:0/1978474740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe154101ee0 0x7fe1540730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:37:39.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.227+0000 7fe15939b700 1 --2- 172.21.15.2:0/1978474740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe154100cf0 0x7fe154074a60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe150004710 tx=0x7fe150009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:39.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.227+0000 7fe14a7fc700 1 -- 172.21.15.2:0/1978474740 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15001d070 con 0x7fe154100cf0 2023-09-13T04:37:39.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.227+0000 7fe14a7fc700 1 -- 172.21.15.2:0/1978474740 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe150021410 con 0x7fe154100cf0 2023-09-13T04:37:39.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.227+0000 7fe15b5ff700 1 -- 172.21.15.2:0/1978474740 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1500095f0 con 0x7fe154100cf0 2023-09-13T04:37:39.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.227+0000 7fe14a7fc700 1 -- 172.21.15.2:0/1978474740 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe150017650 con 0x7fe154100cf0 2023-09-13T04:37:39.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.228+0000 7fe15b5ff700 1 -- 172.21.15.2:0/1978474740 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1541a0f80 con 0x7fe154100cf0 2023-09-13T04:37:39.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.228+0000 7fe15b5ff700 1 -- 172.21.15.2:0/1978474740 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15410c550 con 0x7fe154100cf0 2023-09-13T04:37:39.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.230+0000 7fe14a7fc700 1 -- 172.21.15.2:0/1978474740 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fe150021580 con 0x7fe154100cf0 2023-09-13T04:37:39.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.232+0000 7fe14a7fc700 1 --2- 172.21.15.2:0/1978474740 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe1400a0710 0x7fe1400a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:39.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.232+0000 7fe14a7fc700 1 -- 172.21.15.2:0/1978474740 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe1500cd3a0 con 0x7fe154100cf0 2023-09-13T04:37:39.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.232+0000 7fe158b9a700 1 --2- 172.21.15.2:0/1978474740 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe1400a0710 0x7fe1400a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:39.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.233+0000 7fe158b9a700 1 --2- 172.21.15.2:0/1978474740 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe1400a0710 0x7fe1400a2bd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe144009fd0 tx=0x7fe144009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:39.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.234+0000 7fe14a7fc700 1 -- 172.21.15.2:0/1978474740 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe150097360 con 0x7fe154100cf0 2023-09-13T04:37:39.343 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:37:39 smithi156 systemd[1]: Starting Ceph mds.c for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:37:39.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.488+0000 7fe15b5ff700 1 -- 172.21.15.2:0/1978474740 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe154107ba0 con 0x7fe154100cf0 2023-09-13T04:37:39.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.489+0000 7fe14a7fc700 1 -- 172.21.15.2:0/1978474740 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5277 (secure 0 0 0) 0x7fe15009a980 con 0x7fe154100cf0 2023-09-13T04:37:39.491 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:37:39.491 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/846579077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":846579077},{"type":"v1","addr":"172.21.15.156:6835","nonce":846579077}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2023-09-13T04:37:39.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.492+0000 7fe15b5ff700 1 -- 172.21.15.2:0/1978474740 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe1400a0710 msgr2=0x7fe1400a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:39.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.492+0000 7fe15b5ff700 1 --2- 172.21.15.2:0/1978474740 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe1400a0710 0x7fe1400a2bd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe144009fd0 tx=0x7fe144009260 comp rx=0 tx=0).stop 2023-09-13T04:37:39.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.492+0000 7fe15b5ff700 1 -- 172.21.15.2:0/1978474740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe154100cf0 msgr2=0x7fe154074a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:39.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.492+0000 7fe15b5ff700 1 --2- 172.21.15.2:0/1978474740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe154100cf0 0x7fe154074a60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe150004710 tx=0x7fe150009580 comp rx=0 tx=0).stop 2023-09-13T04:37:39.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.492+0000 7fe15b5ff700 1 -- 172.21.15.2:0/1978474740 shutdown_connections 2023-09-13T04:37:39.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.492+0000 7fe15b5ff700 1 --2- 172.21.15.2:0/1978474740 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe1400a0710 0x7fe1400a2bd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:39.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.492+0000 7fe15b5ff700 1 --2- 172.21.15.2:0/1978474740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe154102880 0x7fe1540735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:39.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.492+0000 7fe15b5ff700 1 --2- 172.21.15.2:0/1978474740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe154100cf0 0x7fe154074a60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:39.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.492+0000 7fe15b5ff700 1 --2- 172.21.15.2:0/1978474740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe154101ee0 0x7fe1540730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:39.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.492+0000 7fe15b5ff700 1 -- 172.21.15.2:0/1978474740 >> 172.21.15.2:0/1978474740 conn(0x7fe1540fc220 msgr2=0x7fe1540fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:39.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.492+0000 7fe15b5ff700 1 -- 172.21.15.2:0/1978474740 shutdown_connections 2023-09-13T04:37:39.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:39.492+0000 7fe15b5ff700 1 -- 172.21.15.2:0/1978474740 wait complete. 2023-09-13T04:37:39.496 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 7 2023-09-13T04:37:39.903 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:37:39 smithi156 podman[171684]: 2023-09-13T04:37:39.903 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:39 smithi156 ceph-mon[133675]: Deploying daemon mds.c on smithi156 2023-09-13T04:37:39.903 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:39 smithi156 ceph-mon[133675]: mds.? [v2:172.21.15.156:6834/846579077,v1:172.21.15.156:6835/846579077] up:boot 2023-09-13T04:37:39.904 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:39 smithi156 ceph-mon[133675]: fsmap 6 up:standby 2023-09-13T04:37:39.904 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:39 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-13T04:37:39.904 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:39 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1978474740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:40.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:39 smithi002 ceph-mon[97651]: Deploying daemon mds.c on smithi156 2023-09-13T04:37:40.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:39 smithi002 ceph-mon[97651]: mds.? [v2:172.21.15.156:6834/846579077,v1:172.21.15.156:6835/846579077] up:boot 2023-09-13T04:37:40.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:39 smithi002 ceph-mon[97651]: fsmap 6 up:standby 2023-09-13T04:37:40.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:39 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-13T04:37:40.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:39 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1978474740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:40.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:39 smithi149 ceph-mon[133315]: Deploying daemon mds.c on smithi156 2023-09-13T04:37:40.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:39 smithi149 ceph-mon[133315]: mds.? [v2:172.21.15.156:6834/846579077,v1:172.21.15.156:6835/846579077] up:boot 2023-09-13T04:37:40.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:39 smithi149 ceph-mon[133315]: fsmap 6 up:standby 2023-09-13T04:37:40.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:39 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-13T04:37:40.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:39 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1978474740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:40.186 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:37:39 smithi156 ceph-mds[171724]: -- 172.21.15.156:0/200184533 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55951eff71e0 con 0x55951efccc00 2023-09-13T04:37:40.186 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:37:39 smithi156 bash[171684]: ddc6fedb9be48e61b87626a6d07943a2582ab896ec0ba6af1a2ae53b690c92c4 2023-09-13T04:37:40.187 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:37:39 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[171700]: starting mds.c at 2023-09-13T04:37:40.187 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:37:39 smithi156 systemd[1]: Started Ceph mds.c for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:37:40.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:40 smithi149 ceph-mon[133315]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:40.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:40 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:40.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:40 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:40.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:40 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:40.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:40 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:40.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:40 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:40.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:40 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:40.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:40 smithi156 ceph-mon[133675]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:40.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:40 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:40.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:40 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:40.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:40 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:40.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:40 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:40.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:40 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:40.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:40 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:41.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:40 smithi002 ceph-mon[97651]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:41.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:40 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:41.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:40 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:41.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:40 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:41.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:40 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:41.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:40 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:41.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:40 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:41.503 INFO:journalctl@ceph.mds.h.smithi149.stdout:Sep 13 04:37:41 smithi149 systemd[1]: Starting Ceph mds.h for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:37:41.513 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:37:41.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.730+0000 7f50a71ff700 1 -- 172.21.15.2:0/901902829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50a0073710 msgr2=0x7f50a0073b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:41.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.730+0000 7f50a71ff700 1 --2- 172.21.15.2:0/901902829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50a0073710 0x7f50a0073b90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5090009990 tx=0x7f5090009ca0 comp rx=0 tx=0).stop 2023-09-13T04:37:41.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.730+0000 7f50a71ff700 1 -- 172.21.15.2:0/901902829 shutdown_connections 2023-09-13T04:37:41.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.730+0000 7f50a71ff700 1 --2- 172.21.15.2:0/901902829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50a0108bb0 0x7f50a010b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:41.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.730+0000 7f50a71ff700 1 --2- 172.21.15.2:0/901902829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a0074cf0 0x7f50a0073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:41.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.730+0000 7f50a71ff700 1 --2- 172.21.15.2:0/901902829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50a0073710 0x7f50a0073b90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:41.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.730+0000 7f50a71ff700 1 -- 172.21.15.2:0/901902829 >> 172.21.15.2:0/901902829 conn(0x7f50a00fc220 msgr2=0x7f50a00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:41.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.730+0000 7f50a71ff700 1 -- 172.21.15.2:0/901902829 shutdown_connections 2023-09-13T04:37:41.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.730+0000 7f50a71ff700 1 -- 172.21.15.2:0/901902829 wait complete. 2023-09-13T04:37:41.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.731+0000 7f50a71ff700 1 Processor -- start 2023-09-13T04:37:41.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.731+0000 7f50a71ff700 1 -- start start 2023-09-13T04:37:41.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.732+0000 7f50a71ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50a0073710 0x7f50a0198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:41.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.732+0000 7f50a71ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50a0074cf0 0x7f50a0199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:41.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.732+0000 7f50a71ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a0108bb0 0x7f50a01a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:41.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.732+0000 7f50a71ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a01a0760 con 0x7f50a0073710 2023-09-13T04:37:41.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.732+0000 7f50a71ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a01a08d0 con 0x7f50a0108bb0 2023-09-13T04:37:41.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.732+0000 7f50a71ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a01a0a90 con 0x7f50a0074cf0 2023-09-13T04:37:41.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.732+0000 7f50a4f9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50a0073710 0x7f50a0198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:41.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.732+0000 7f50a4f9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50a0073710 0x7f50a0198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53832/0 (socket says 172.21.15.2:53832) 2023-09-13T04:37:41.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.732+0000 7f50a4f9b700 1 -- 172.21.15.2:0/3067338934 learned_addr learned my addr 172.21.15.2:0/3067338934 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:41.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.732+0000 7f50a579c700 1 --2- 172.21.15.2:0/3067338934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a0108bb0 0x7f50a01a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:41.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.732+0000 7f5097fff700 1 --2- 172.21.15.2:0/3067338934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50a0074cf0 0x7f50a0199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:41.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.733+0000 7f50a579c700 1 -- 172.21.15.2:0/3067338934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50a0074cf0 msgr2=0x7f50a0199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:41.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.733+0000 7f50a579c700 1 --2- 172.21.15.2:0/3067338934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50a0074cf0 0x7f50a0199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:41.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.733+0000 7f50a579c700 1 -- 172.21.15.2:0/3067338934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50a0073710 msgr2=0x7f50a0198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:41.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.733+0000 7f50a579c700 1 --2- 172.21.15.2:0/3067338934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50a0073710 0x7f50a0198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:41.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.733+0000 7f50a579c700 1 -- 172.21.15.2:0/3067338934 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c009520 con 0x7f50a0108bb0 2023-09-13T04:37:41.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.733+0000 7f50a4f9b700 1 --2- 172.21.15.2:0/3067338934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50a0073710 0x7f50a0198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:37:41.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.733+0000 7f5097fff700 1 --2- 172.21.15.2:0/3067338934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50a0074cf0 0x7f50a0199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:37:41.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.733+0000 7f50a579c700 1 --2- 172.21.15.2:0/3067338934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a0108bb0 0x7f50a01a0190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f509800efc0 tx=0x7f509800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:41.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.733+0000 7f5095ffb700 1 -- 172.21.15.2:0/3067338934 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5098018070 con 0x7f50a0108bb0 2023-09-13T04:37:41.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.733+0000 7f5095ffb700 1 -- 172.21.15.2:0/3067338934 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f509800f040 con 0x7f50a0108bb0 2023-09-13T04:37:41.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.733+0000 7f50a71ff700 1 -- 172.21.15.2:0/3067338934 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50900095f0 con 0x7f50a0108bb0 2023-09-13T04:37:41.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.733+0000 7f5095ffb700 1 -- 172.21.15.2:0/3067338934 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509800cdf0 con 0x7f50a0108bb0 2023-09-13T04:37:41.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.733+0000 7f50a71ff700 1 -- 172.21.15.2:0/3067338934 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a01a1040 con 0x7f50a0108bb0 2023-09-13T04:37:41.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.734+0000 7f50a71ff700 1 -- 172.21.15.2:0/3067338934 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a0107260 con 0x7f50a0108bb0 2023-09-13T04:37:41.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.736+0000 7f5095ffb700 1 -- 172.21.15.2:0/3067338934 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f5098003a00 con 0x7f50a0108bb0 2023-09-13T04:37:41.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.737+0000 7f5095ffb700 1 --2- 172.21.15.2:0/3067338934 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f50880a0690 0x7f50880a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:41.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.738+0000 7f5095ffb700 1 -- 172.21.15.2:0/3067338934 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f50980cbab0 con 0x7f50a0108bb0 2023-09-13T04:37:41.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.738+0000 7f50a4f9b700 1 --2- 172.21.15.2:0/3067338934 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f50880a0690 0x7f50880a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:41.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.738+0000 7f50a4f9b700 1 --2- 172.21.15.2:0/3067338934 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f50880a0690 0x7f50880a2b50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f50a019a0f0 tx=0x7f509c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:41.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.740+0000 7f5095ffb700 1 -- 172.21.15.2:0/3067338934 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5098095a70 con 0x7f50a0108bb0 2023-09-13T04:37:41.774 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:41 smithi149 ceph-mon[133315]: Deploying daemon mds.h on smithi149 2023-09-13T04:37:41.774 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:41 smithi149 ceph-mon[133315]: mds.? [v2:172.21.15.156:6836/524563090,v1:172.21.15.156:6837/524563090] up:boot 2023-09-13T04:37:41.774 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:41 smithi149 ceph-mon[133315]: fsmap 7 up:standby 2023-09-13T04:37:41.774 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:41 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-13T04:37:41.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:41 smithi156 ceph-mon[133675]: Deploying daemon mds.h on smithi149 2023-09-13T04:37:41.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:41 smithi156 ceph-mon[133675]: mds.? [v2:172.21.15.156:6836/524563090,v1:172.21.15.156:6837/524563090] up:boot 2023-09-13T04:37:41.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:41 smithi156 ceph-mon[133675]: fsmap 7 up:standby 2023-09-13T04:37:41.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:41 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-13T04:37:42.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.999+0000 7f50a71ff700 1 -- 172.21.15.2:0/3067338934 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50a010e980 con 0x7f50a0108bb0 2023-09-13T04:37:42.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:41.999+0000 7f5095ffb700 1 -- 172.21.15.2:0/3067338934 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6068 (secure 0 0 0) 0x7f5098099090 con 0x7f50a0108bb0 2023-09-13T04:37:42.001 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:37:42.001 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/846579077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":846579077},{"type":"v1","addr":"172.21.15.156:6835","nonce":846579077}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/524563090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":524563090},{"type":"v1","addr":"172.21.15.156:6837","nonce":524563090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-09-13T04:37:42.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:42.002+0000 7f50a71ff700 1 -- 172.21.15.2:0/3067338934 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f50880a0690 msgr2=0x7f50880a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:42.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:42.002+0000 7f50a71ff700 1 --2- 172.21.15.2:0/3067338934 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f50880a0690 0x7f50880a2b50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f50a019a0f0 tx=0x7f509c009260 comp rx=0 tx=0).stop 2023-09-13T04:37:42.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:42.002+0000 7f50a71ff700 1 -- 172.21.15.2:0/3067338934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a0108bb0 msgr2=0x7f50a01a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:42.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:42.002+0000 7f50a71ff700 1 --2- 172.21.15.2:0/3067338934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a0108bb0 0x7f50a01a0190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f509800efc0 tx=0x7f509800c3a0 comp rx=0 tx=0).stop 2023-09-13T04:37:42.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:42.002+0000 7f50a71ff700 1 -- 172.21.15.2:0/3067338934 shutdown_connections 2023-09-13T04:37:42.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:42.002+0000 7f50a71ff700 1 --2- 172.21.15.2:0/3067338934 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f50880a0690 0x7f50880a2b50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:42.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:42.002+0000 7f50a71ff700 1 --2- 172.21.15.2:0/3067338934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a0108bb0 0x7f50a01a0190 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:42.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:42.002+0000 7f50a71ff700 1 --2- 172.21.15.2:0/3067338934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50a0073710 0x7f50a0198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:42.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:42.002+0000 7f50a71ff700 1 --2- 172.21.15.2:0/3067338934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50a0074cf0 0x7f50a0199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:42.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:42.002+0000 7f50a71ff700 1 -- 172.21.15.2:0/3067338934 >> 172.21.15.2:0/3067338934 conn(0x7f50a00fc220 msgr2=0x7f50a00fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:42.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:42.002+0000 7f50a71ff700 1 -- 172.21.15.2:0/3067338934 shutdown_connections 2023-09-13T04:37:42.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:42.002+0000 7f50a71ff700 1 -- 172.21.15.2:0/3067338934 wait complete. 2023-09-13T04:37:42.006 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 8 2023-09-13T04:37:42.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:41 smithi002 ceph-mon[97651]: Deploying daemon mds.h on smithi149 2023-09-13T04:37:42.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:41 smithi002 ceph-mon[97651]: mds.? [v2:172.21.15.156:6836/524563090,v1:172.21.15.156:6837/524563090] up:boot 2023-09-13T04:37:42.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:41 smithi002 ceph-mon[97651]: fsmap 7 up:standby 2023-09-13T04:37:42.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:41 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-13T04:37:42.038 INFO:journalctl@ceph.mds.h.smithi149.stdout:Sep 13 04:37:41 smithi149 podman[169590]: 2023-09-13T04:37:42.386 INFO:journalctl@ceph.mds.h.smithi149.stdout:Sep 13 04:37:42 smithi149 ceph-mds[169630]: -- 172.21.15.149:0/2251246053 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b0264211e0 con 0x55b0263f6c00 2023-09-13T04:37:42.386 INFO:journalctl@ceph.mds.h.smithi149.stdout:Sep 13 04:37:42 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-h[169606]: starting mds.h at 2023-09-13T04:37:42.387 INFO:journalctl@ceph.mds.h.smithi149.stdout:Sep 13 04:37:42 smithi149 bash[169590]: ee34900a5f4a13a8026e919f0baf4f00a13b6dd18fb4d1a35752fa2adb82f0c5 2023-09-13T04:37:42.387 INFO:journalctl@ceph.mds.h.smithi149.stdout:Sep 13 04:37:42 smithi149 systemd[1]: Started Ceph mds.h for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:37:42.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:42 smithi156 ceph-mon[133675]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:42.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:42 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3067338934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:42.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:42 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:42.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:42 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:42.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:42 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:42.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:42 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:42.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:42 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:42.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:42 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:43.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:42 smithi002 ceph-mon[97651]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:43.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:42 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3067338934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:43.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:42 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:43.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:42 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:43.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:42 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:43.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:42 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:43.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:42 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:43.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:42 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:43.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:42 smithi149 ceph-mon[133315]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:43.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:42 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3067338934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:43.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:42 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:43.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:42 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:43.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:42 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:43.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:42 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:43.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:42 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:43.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:42 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:43.924 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:43 smithi156 ceph-mon[133675]: Deploying daemon mds.i on smithi156 2023-09-13T04:37:43.925 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:43 smithi156 ceph-mon[133675]: mds.? [v2:172.21.15.149:6836/1896833690,v1:172.21.15.149:6837/1896833690] up:boot 2023-09-13T04:37:43.925 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:43 smithi156 ceph-mon[133675]: fsmap 8 up:standby 2023-09-13T04:37:43.925 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:43 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-13T04:37:43.925 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 13 04:37:43 smithi156 systemd[1]: Starting Ceph mds.i for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:37:44.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:43 smithi002 ceph-mon[97651]: Deploying daemon mds.i on smithi156 2023-09-13T04:37:44.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:43 smithi002 ceph-mon[97651]: mds.? [v2:172.21.15.149:6836/1896833690,v1:172.21.15.149:6837/1896833690] up:boot 2023-09-13T04:37:44.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:43 smithi002 ceph-mon[97651]: fsmap 8 up:standby 2023-09-13T04:37:44.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:43 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-13T04:37:44.023 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:37:44.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:43 smithi149 ceph-mon[133315]: Deploying daemon mds.i on smithi156 2023-09-13T04:37:44.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:43 smithi149 ceph-mon[133315]: mds.? [v2:172.21.15.149:6836/1896833690,v1:172.21.15.149:6837/1896833690] up:boot 2023-09-13T04:37:44.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:43 smithi149 ceph-mon[133315]: fsmap 8 up:standby 2023-09-13T04:37:44.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:43 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-13T04:37:44.186 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 13 04:37:43 smithi156 podman[172022]: 2023-09-13T04:37:44.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.238+0000 7f01f25ff700 1 -- 172.21.15.2:0/4058089557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01ec102880 msgr2=0x7f01ec109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:44.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.238+0000 7f01f25ff700 1 --2- 172.21.15.2:0/4058089557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01ec102880 0x7f01ec109130 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f01e400b0d0 tx=0x7f01e400b3e0 comp rx=0 tx=0).stop 2023-09-13T04:37:44.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.239+0000 7f01f25ff700 1 -- 172.21.15.2:0/4058089557 shutdown_connections 2023-09-13T04:37:44.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.239+0000 7f01f25ff700 1 --2- 172.21.15.2:0/4058089557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01ec102880 0x7f01ec109130 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:44.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.239+0000 7f01f25ff700 1 --2- 172.21.15.2:0/4058089557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ec100cf0 0x7f01ec101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:44.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.239+0000 7f01f25ff700 1 --2- 172.21.15.2:0/4058089557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01ec101ee0 0x7f01ec102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:44.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.239+0000 7f01f25ff700 1 -- 172.21.15.2:0/4058089557 >> 172.21.15.2:0/4058089557 conn(0x7f01ec0fc220 msgr2=0x7f01ec0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:44.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.239+0000 7f01f25ff700 1 -- 172.21.15.2:0/4058089557 shutdown_connections 2023-09-13T04:37:44.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.239+0000 7f01f25ff700 1 -- 172.21.15.2:0/4058089557 wait complete. 2023-09-13T04:37:44.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.240+0000 7f01f25ff700 1 Processor -- start 2023-09-13T04:37:44.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.240+0000 7f01f25ff700 1 -- start start 2023-09-13T04:37:44.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.241+0000 7f01f25ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ec100cf0 0x7f01ec196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:44.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.241+0000 7f01f25ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01ec101ee0 0x7f01ec197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:44.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.241+0000 7f01f25ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01ec102880 0x7f01ec19e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:44.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.241+0000 7f01f25ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec19e690 con 0x7f01ec101ee0 2023-09-13T04:37:44.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.241+0000 7f01f25ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec19e7d0 con 0x7f01ec102880 2023-09-13T04:37:44.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.241+0000 7f01f25ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec19e960 con 0x7f01ec100cf0 2023-09-13T04:37:44.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.241+0000 7f01eb7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01ec101ee0 0x7f01ec197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:44.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.241+0000 7f01ebfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ec100cf0 0x7f01ec196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:44.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.241+0000 7f01ebfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ec100cf0 0x7f01ec196b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:42050/0 (socket says 172.21.15.2:42050) 2023-09-13T04:37:44.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.241+0000 7f01ebfff700 1 -- 172.21.15.2:0/1556266284 learned_addr learned my addr 172.21.15.2:0/1556266284 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:44.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.241+0000 7f01f0b9c700 1 --2- 172.21.15.2:0/1556266284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01ec102880 0x7f01ec19e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:44.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.242+0000 7f01ebfff700 1 -- 172.21.15.2:0/1556266284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01ec102880 msgr2=0x7f01ec19e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:44.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.242+0000 7f01ebfff700 1 --2- 172.21.15.2:0/1556266284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01ec102880 0x7f01ec19e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:44.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.242+0000 7f01ebfff700 1 -- 172.21.15.2:0/1556266284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01ec101ee0 msgr2=0x7f01ec197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:44.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.242+0000 7f01ebfff700 1 --2- 172.21.15.2:0/1556266284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01ec101ee0 0x7f01ec197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:44.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.242+0000 7f01ebfff700 1 -- 172.21.15.2:0/1556266284 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e4009d00 con 0x7f01ec100cf0 2023-09-13T04:37:44.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.242+0000 7f01f0b9c700 1 --2- 172.21.15.2:0/1556266284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01ec102880 0x7f01ec19e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:37:44.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.242+0000 7f01eb7fe700 1 --2- 172.21.15.2:0/1556266284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01ec101ee0 0x7f01ec197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:37:44.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.242+0000 7f01ebfff700 1 --2- 172.21.15.2:0/1556266284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ec100cf0 0x7f01ec196b10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00e970 tx=0x7f01dc00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:44.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.242+0000 7f01e97fa700 1 -- 172.21.15.2:0/1556266284 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc009790 con 0x7f01ec100cf0 2023-09-13T04:37:44.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.242+0000 7f01e97fa700 1 -- 172.21.15.2:0/1556266284 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01dc00eec0 con 0x7f01ec100cf0 2023-09-13T04:37:44.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.242+0000 7f01f25ff700 1 -- 172.21.15.2:0/1556266284 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ec19ec40 con 0x7f01ec100cf0 2023-09-13T04:37:44.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.242+0000 7f01e97fa700 1 -- 172.21.15.2:0/1556266284 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc003cf0 con 0x7f01ec100cf0 2023-09-13T04:37:44.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.242+0000 7f01f25ff700 1 -- 172.21.15.2:0/1556266284 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ec19f150 con 0x7f01ec100cf0 2023-09-13T04:37:44.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.243+0000 7f01f25ff700 1 -- 172.21.15.2:0/1556266284 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ec1081e0 con 0x7f01ec100cf0 2023-09-13T04:37:44.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.245+0000 7f01e97fa700 1 -- 172.21.15.2:0/1556266284 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f01dc00ccd0 con 0x7f01ec100cf0 2023-09-13T04:37:44.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.246+0000 7f01e97fa700 1 --2- 172.21.15.2:0/1556266284 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f01d40a0690 0x7f01d40a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:44.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.246+0000 7f01e97fa700 1 -- 172.21.15.2:0/1556266284 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f01dc014070 con 0x7f01ec100cf0 2023-09-13T04:37:44.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.247+0000 7f01eb7fe700 1 --2- 172.21.15.2:0/1556266284 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f01d40a0690 0x7f01d40a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:44.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.247+0000 7f01eb7fe700 1 --2- 172.21.15.2:0/1556266284 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f01d40a0690 0x7f01d40a2b50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009fd0 tx=0x7f01e0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:44.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.250+0000 7f01e97fa700 1 -- 172.21.15.2:0/1556266284 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f01dc095940 con 0x7f01ec100cf0 2023-09-13T04:37:44.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.527+0000 7f01f25ff700 1 -- 172.21.15.2:0/1556266284 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01ec10c560 con 0x7f01ec100cf0 2023-09-13T04:37:44.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.527+0000 7f01e97fa700 1 -- 172.21.15.2:0/1556266284 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6862 (secure 0 0 0) 0x7f01dc098f60 con 0x7f01ec100cf0 2023-09-13T04:37:44.529 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:37:44.529 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/846579077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":846579077},{"type":"v1","addr":"172.21.15.156:6835","nonce":846579077}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1896833690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1896833690},{"type":"v1","addr":"172.21.15.149:6837","nonce":1896833690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/524563090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":524563090},{"type":"v1","addr":"172.21.15.156:6837","nonce":524563090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-09-13T04:37:44.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.530+0000 7f01f25ff700 1 -- 172.21.15.2:0/1556266284 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f01d40a0690 msgr2=0x7f01d40a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:44.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.530+0000 7f01f25ff700 1 --2- 172.21.15.2:0/1556266284 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f01d40a0690 0x7f01d40a2b50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009fd0 tx=0x7f01e0009260 comp rx=0 tx=0).stop 2023-09-13T04:37:44.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.530+0000 7f01f25ff700 1 -- 172.21.15.2:0/1556266284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ec100cf0 msgr2=0x7f01ec196b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:44.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.530+0000 7f01f25ff700 1 --2- 172.21.15.2:0/1556266284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ec100cf0 0x7f01ec196b10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00e970 tx=0x7f01dc00ec80 comp rx=0 tx=0).stop 2023-09-13T04:37:44.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.530+0000 7f01f25ff700 1 -- 172.21.15.2:0/1556266284 shutdown_connections 2023-09-13T04:37:44.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.530+0000 7f01f25ff700 1 --2- 172.21.15.2:0/1556266284 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f01d40a0690 0x7f01d40a2b50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:44.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.530+0000 7f01f25ff700 1 --2- 172.21.15.2:0/1556266284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01ec102880 0x7f01ec19e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:44.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.530+0000 7f01f25ff700 1 --2- 172.21.15.2:0/1556266284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ec100cf0 0x7f01ec196b10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:44.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.530+0000 7f01f25ff700 1 --2- 172.21.15.2:0/1556266284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01ec101ee0 0x7f01ec197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:44.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.530+0000 7f01f25ff700 1 -- 172.21.15.2:0/1556266284 >> 172.21.15.2:0/1556266284 conn(0x7f01ec0fc220 msgr2=0x7f01ec105320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:44.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.530+0000 7f01f25ff700 1 -- 172.21.15.2:0/1556266284 shutdown_connections 2023-09-13T04:37:44.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:44.530+0000 7f01f25ff700 1 -- 172.21.15.2:0/1556266284 wait complete. 2023-09-13T04:37:44.534 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 9 2023-09-13T04:37:44.669 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 13 04:37:44 smithi156 ceph-mds[172063]: -- 172.21.15.156:0/3418893803 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fafdbb11e0 con 0x55fafdb86800 2023-09-13T04:37:44.670 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 13 04:37:44 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-i[172038]: starting mds.i at 2023-09-13T04:37:44.670 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 13 04:37:44 smithi156 bash[172022]: cbdd722ea508ee433ea34dbabcc55607801ab64da9c30965f43843fd4b79488b 2023-09-13T04:37:44.670 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 13 04:37:44 smithi156 systemd[1]: Started Ceph mds.i for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:37:44.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:44 smithi156 ceph-mon[133675]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:44.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:44 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:44.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:44 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:44.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:44 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:44.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:44 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:44.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:44 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:44.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:44 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:44.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:44 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1556266284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:45.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:44 smithi002 ceph-mon[97651]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:45.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:44 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:45.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:44 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:45.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:44 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:45.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:44 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:45.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:44 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:45.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:44 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:45.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:44 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1556266284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:45.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:44 smithi149 ceph-mon[133315]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:45.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:44 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:45.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:44 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:45.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:44 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:45.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:44 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:45.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:44 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:45.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:44 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:45.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:44 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1556266284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:45.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:45 smithi156 ceph-mon[133675]: Deploying daemon mds.g on smithi002 2023-09-13T04:37:45.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:45 smithi156 ceph-mon[133675]: mds.? [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] up:boot 2023-09-13T04:37:45.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:45 smithi156 ceph-mon[133675]: fsmap 9 up:standby 2023-09-13T04:37:45.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:45 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-13T04:37:45.980 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:45 smithi002 ceph-mon[97651]: Deploying daemon mds.g on smithi002 2023-09-13T04:37:45.980 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:45 smithi002 ceph-mon[97651]: mds.? [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] up:boot 2023-09-13T04:37:45.981 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:45 smithi002 ceph-mon[97651]: fsmap 9 up:standby 2023-09-13T04:37:45.981 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:45 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-13T04:37:46.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:45 smithi149 ceph-mon[133315]: Deploying daemon mds.g on smithi002 2023-09-13T04:37:46.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:45 smithi149 ceph-mon[133315]: mds.? [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] up:boot 2023-09-13T04:37:46.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:45 smithi149 ceph-mon[133315]: fsmap 9 up:standby 2023-09-13T04:37:46.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:45 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-13T04:37:46.554 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:37:46.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.801+0000 7fb0283ff700 1 -- 172.21.15.2:0/3472564734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb020104060 msgr2=0x7fb0201044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:46.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.801+0000 7fb0283ff700 1 --2- 172.21.15.2:0/3472564734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb020104060 0x7fb0201044e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb010009990 tx=0x7fb010009ca0 comp rx=0 tx=0).stop 2023-09-13T04:37:46.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.801+0000 7fb0283ff700 1 -- 172.21.15.2:0/3472564734 shutdown_connections 2023-09-13T04:37:46.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.801+0000 7fb0283ff700 1 --2- 172.21.15.2:0/3472564734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb020104a20 0x7fb02010b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:46.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.801+0000 7fb0283ff700 1 --2- 172.21.15.2:0/3472564734 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb020102f10 0x7fb020103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:46.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.801+0000 7fb0283ff700 1 --2- 172.21.15.2:0/3472564734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb020104060 0x7fb0201044e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:46.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.801+0000 7fb0283ff700 1 -- 172.21.15.2:0/3472564734 >> 172.21.15.2:0/3472564734 conn(0x7fb0200fe460 msgr2=0x7fb0201008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:46.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.801+0000 7fb0283ff700 1 -- 172.21.15.2:0/3472564734 shutdown_connections 2023-09-13T04:37:46.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.801+0000 7fb0283ff700 1 -- 172.21.15.2:0/3472564734 wait complete. 2023-09-13T04:37:46.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.803+0000 7fb0283ff700 1 Processor -- start 2023-09-13T04:37:46.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.803+0000 7fb0283ff700 1 -- start start 2023-09-13T04:37:46.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.803+0000 7fb0283ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb020102f10 0x7fb020198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:46.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.803+0000 7fb0283ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb020104060 0x7fb020199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:46.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.803+0000 7fb0283ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb020104a20 0x7fb0201a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:46.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.803+0000 7fb0283ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0201a06b0 con 0x7fb020102f10 2023-09-13T04:37:46.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.803+0000 7fb0283ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0201a07f0 con 0x7fb020104060 2023-09-13T04:37:46.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.803+0000 7fb0283ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0201a0980 con 0x7fb020104a20 2023-09-13T04:37:46.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.803+0000 7fb02619b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb020102f10 0x7fb020198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:46.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.803+0000 7fb02619b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb020102f10 0x7fb020198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38716/0 (socket says 172.21.15.2:38716) 2023-09-13T04:37:46.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.803+0000 7fb02619b700 1 -- 172.21.15.2:0/833100393 learned_addr learned my addr 172.21.15.2:0/833100393 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:46.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.803+0000 7fb02599a700 1 --2- 172.21.15.2:0/833100393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb020104060 0x7fb020199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:46.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.804+0000 7fb02699c700 1 --2- 172.21.15.2:0/833100393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb020104a20 0x7fb0201a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:46.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.804+0000 7fb02619b700 1 -- 172.21.15.2:0/833100393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb020104a20 msgr2=0x7fb0201a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:46.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.804+0000 7fb02619b700 1 --2- 172.21.15.2:0/833100393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb020104a20 0x7fb0201a00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:46.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.804+0000 7fb02619b700 1 -- 172.21.15.2:0/833100393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb020104060 msgr2=0x7fb020199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:46.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.804+0000 7fb02619b700 1 --2- 172.21.15.2:0/833100393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb020104060 0x7fb020199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:46.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.804+0000 7fb02619b700 1 -- 172.21.15.2:0/833100393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0100095f0 con 0x7fb020102f10 2023-09-13T04:37:46.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.804+0000 7fb02599a700 1 --2- 172.21.15.2:0/833100393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb020104060 0x7fb020199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:37:46.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.804+0000 7fb02619b700 1 --2- 172.21.15.2:0/833100393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb020102f10 0x7fb020198b30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00e9a0 tx=0x7fb01c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:46.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.804+0000 7fb0177fe700 1 -- 172.21.15.2:0/833100393 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c009790 con 0x7fb020102f10 2023-09-13T04:37:46.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.804+0000 7fb0177fe700 1 -- 172.21.15.2:0/833100393 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb01c010430 con 0x7fb020102f10 2023-09-13T04:37:46.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.804+0000 7fb0283ff700 1 -- 172.21.15.2:0/833100393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0201a0c60 con 0x7fb020102f10 2023-09-13T04:37:46.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.804+0000 7fb0177fe700 1 -- 172.21.15.2:0/833100393 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c018680 con 0x7fb020102f10 2023-09-13T04:37:46.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.805+0000 7fb0283ff700 1 -- 172.21.15.2:0/833100393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0201a1170 con 0x7fb020102f10 2023-09-13T04:37:46.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.806+0000 7fb0283ff700 1 -- 172.21.15.2:0/833100393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb02010a440 con 0x7fb020102f10 2023-09-13T04:37:46.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.808+0000 7fb0177fe700 1 -- 172.21.15.2:0/833100393 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fb01c0105a0 con 0x7fb020102f10 2023-09-13T04:37:46.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.810+0000 7fb0177fe700 1 --2- 172.21.15.2:0/833100393 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb00c0a0690 0x7fb00c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:46.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.810+0000 7fb0177fe700 1 -- 172.21.15.2:0/833100393 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb01c014070 con 0x7fb020102f10 2023-09-13T04:37:46.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.810+0000 7fb02599a700 1 --2- 172.21.15.2:0/833100393 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb00c0a0690 0x7fb00c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:46.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.811+0000 7fb02599a700 1 --2- 172.21.15.2:0/833100393 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb00c0a0690 0x7fb00c0a2b50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb01000b2e0 tx=0x7fb010005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:46.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:46.814+0000 7fb0177fe700 1 -- 172.21.15.2:0/833100393 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb01c051e10 con 0x7fb020102f10 2023-09-13T04:37:46.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:46 smithi156 ceph-mon[133675]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:47.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:46 smithi002 ceph-mon[97651]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:47.006 INFO:journalctl@ceph.mds.g.smithi002.stdout:Sep 13 04:37:46 smithi002 systemd[1]: Starting Ceph mds.g for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:37:47.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:47.085+0000 7fb0283ff700 1 -- 172.21.15.2:0/833100393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb02010e980 con 0x7fb020102f10 2023-09-13T04:37:47.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:47.086+0000 7fb0177fe700 1 -- 172.21.15.2:0/833100393 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7658 (secure 0 0 0) 0x7fb01c095850 con 0x7fb020102f10 2023-09-13T04:37:47.087 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:37:47.088 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/846579077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":846579077},{"type":"v1","addr":"172.21.15.156:6835","nonce":846579077}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1896833690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1896833690},{"type":"v1","addr":"172.21.15.149:6837","nonce":1896833690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/524563090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":524563090},{"type":"v1","addr":"172.21.15.156:6837","nonce":524563090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/1785833227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1785833227},{"type":"v1","addr":"172.21.15.156:6839","nonce":1785833227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-09-13T04:37:47.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:47.088+0000 7fb0283ff700 1 -- 172.21.15.2:0/833100393 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb00c0a0690 msgr2=0x7fb00c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:47.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:47.088+0000 7fb0283ff700 1 --2- 172.21.15.2:0/833100393 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb00c0a0690 0x7fb00c0a2b50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb01000b2e0 tx=0x7fb010005fb0 comp rx=0 tx=0).stop 2023-09-13T04:37:47.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:47.088+0000 7fb0283ff700 1 -- 172.21.15.2:0/833100393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb020102f10 msgr2=0x7fb020198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:47.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:47.088+0000 7fb0283ff700 1 --2- 172.21.15.2:0/833100393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb020102f10 0x7fb020198b30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00e9a0 tx=0x7fb01c00ecb0 comp rx=0 tx=0).stop 2023-09-13T04:37:47.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:47.088+0000 7fb0283ff700 1 -- 172.21.15.2:0/833100393 shutdown_connections 2023-09-13T04:37:47.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:47.088+0000 7fb0283ff700 1 --2- 172.21.15.2:0/833100393 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb00c0a0690 0x7fb00c0a2b50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:47.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:47.088+0000 7fb0283ff700 1 --2- 172.21.15.2:0/833100393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb020104a20 0x7fb0201a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:47.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:47.088+0000 7fb0283ff700 1 --2- 172.21.15.2:0/833100393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb020102f10 0x7fb020198b30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:47.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:47.088+0000 7fb0283ff700 1 --2- 172.21.15.2:0/833100393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb020104060 0x7fb020199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:47.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:47.088+0000 7fb0283ff700 1 -- 172.21.15.2:0/833100393 >> 172.21.15.2:0/833100393 conn(0x7fb0200fe460 msgr2=0x7fb020107580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:47.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:47.089+0000 7fb0283ff700 1 -- 172.21.15.2:0/833100393 shutdown_connections 2023-09-13T04:37:47.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:47.089+0000 7fb0283ff700 1 -- 172.21.15.2:0/833100393 wait complete. 2023-09-13T04:37:47.093 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 10 2023-09-13T04:37:47.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:46 smithi149 ceph-mon[133315]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:47.422 INFO:journalctl@ceph.mds.g.smithi002.stdout:Sep 13 04:37:47 smithi002 podman[148418]: 2023-09-13T04:37:47.689 INFO:journalctl@ceph.mds.g.smithi002.stdout:Sep 13 04:37:47 smithi002 ceph-mds[148460]: -- 172.21.15.2:0/603115093 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5645138ff1e0 con 0x5645138d4800 2023-09-13T04:37:47.689 INFO:journalctl@ceph.mds.g.smithi002.stdout:Sep 13 04:37:47 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-g[148436]: starting mds.g at 2023-09-13T04:37:47.689 INFO:journalctl@ceph.mds.g.smithi002.stdout:Sep 13 04:37:47 smithi002 bash[148418]: 2d8fb06de5367019a672714d8304948dd39a0abb755f88020ba6d8040c15a463 2023-09-13T04:37:47.689 INFO:journalctl@ceph.mds.g.smithi002.stdout:Sep 13 04:37:47 smithi002 systemd[1]: Started Ceph mds.g for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:37:48.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:47 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/833100393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:48.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:47 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:48.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:47 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:48.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:47 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:48.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:47 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:48.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:47 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:48.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:47 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:48.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:47 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/833100393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:48.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:47 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:48.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:47 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:48.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:47 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:48.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:47 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:48.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:47 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:48.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:47 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:48.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:47 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/833100393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:48.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:47 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:48.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:47 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:48.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:47 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:48.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:47 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:48.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:47 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:48.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:47 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:48.823 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:48 smithi002 ceph-mon[97651]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:48.823 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:48 smithi002 ceph-mon[97651]: Deploying daemon mds.a on smithi002 2023-09-13T04:37:48.823 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:48 smithi002 ceph-mon[97651]: mds.? [v2:172.21.15.2:6838/2469307307,v1:172.21.15.2:6839/2469307307] up:boot 2023-09-13T04:37:48.824 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:48 smithi002 ceph-mon[97651]: fsmap 10 up:standby 2023-09-13T04:37:48.824 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:48 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-13T04:37:49.110 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:37:49.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:48 smithi149 ceph-mon[133315]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:49.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:48 smithi149 ceph-mon[133315]: Deploying daemon mds.a on smithi002 2023-09-13T04:37:49.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:48 smithi149 ceph-mon[133315]: mds.? [v2:172.21.15.2:6838/2469307307,v1:172.21.15.2:6839/2469307307] up:boot 2023-09-13T04:37:49.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:48 smithi149 ceph-mon[133315]: fsmap 10 up:standby 2023-09-13T04:37:49.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:48 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-13T04:37:49.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:48 smithi156 ceph-mon[133675]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:49.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:48 smithi156 ceph-mon[133675]: Deploying daemon mds.a on smithi002 2023-09-13T04:37:49.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:48 smithi156 ceph-mon[133675]: mds.? [v2:172.21.15.2:6838/2469307307,v1:172.21.15.2:6839/2469307307] up:boot 2023-09-13T04:37:49.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:48 smithi156 ceph-mon[133675]: fsmap 10 up:standby 2023-09-13T04:37:49.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:48 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-13T04:37:49.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.337+0000 7fbc04dff700 1 -- 172.21.15.2:0/3149047788 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc001040d0 msgr2=0x7fbc001064c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:49.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.337+0000 7fbc04dff700 1 --2- 172.21.15.2:0/3149047788 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc001040d0 0x7fbc001064c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009990 tx=0x7fbbf4009ca0 comp rx=0 tx=0).stop 2023-09-13T04:37:49.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.337+0000 7fbc04dff700 1 -- 172.21.15.2:0/3149047788 shutdown_connections 2023-09-13T04:37:49.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.337+0000 7fbc04dff700 1 --2- 172.21.15.2:0/3149047788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc00106a00 0x7fbc00108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:49.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.337+0000 7fbc04dff700 1 --2- 172.21.15.2:0/3149047788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc001017a0 0x7fbc00103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:49.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.337+0000 7fbc04dff700 1 --2- 172.21.15.2:0/3149047788 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc001040d0 0x7fbc001064c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:49.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.337+0000 7fbc04dff700 1 -- 172.21.15.2:0/3149047788 >> 172.21.15.2:0/3149047788 conn(0x7fbc000fb1b0 msgr2=0x7fbc000fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:49.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.337+0000 7fbc04dff700 1 -- 172.21.15.2:0/3149047788 shutdown_connections 2023-09-13T04:37:49.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.337+0000 7fbc04dff700 1 -- 172.21.15.2:0/3149047788 wait complete. 2023-09-13T04:37:49.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.339+0000 7fbc04dff700 1 Processor -- start 2023-09-13T04:37:49.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.339+0000 7fbc04dff700 1 -- start start 2023-09-13T04:37:49.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.339+0000 7fbc04dff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc001017a0 0x7fbc00196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:49.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.339+0000 7fbc04dff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc001040d0 0x7fbc00196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:49.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.340+0000 7fbc04dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc00106a00 0x7fbc0019def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:49.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.340+0000 7fbc04dff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0019e4c0 con 0x7fbc001040d0 2023-09-13T04:37:49.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.340+0000 7fbc04dff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0019e600 con 0x7fbc001017a0 2023-09-13T04:37:49.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.340+0000 7fbc04dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0019e790 con 0x7fbc00106a00 2023-09-13T04:37:49.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.340+0000 7fbbfe59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc001017a0 0x7fbc00196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:49.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.340+0000 7fbbfe59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc001017a0 0x7fbc00196940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:45572/0 (socket says 172.21.15.2:45572) 2023-09-13T04:37:49.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.340+0000 7fbbfe59c700 1 -- 172.21.15.2:0/4135333584 learned_addr learned my addr 172.21.15.2:0/4135333584 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:49.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.340+0000 7fbbfdd9b700 1 --2- 172.21.15.2:0/4135333584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc001040d0 0x7fbc00196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:49.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.340+0000 7fbbfed9d700 1 --2- 172.21.15.2:0/4135333584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc00106a00 0x7fbc0019def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:49.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.340+0000 7fbbfe59c700 1 -- 172.21.15.2:0/4135333584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc00106a00 msgr2=0x7fbc0019def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:49.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.340+0000 7fbbfe59c700 1 --2- 172.21.15.2:0/4135333584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc00106a00 0x7fbc0019def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:49.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.340+0000 7fbbfe59c700 1 -- 172.21.15.2:0/4135333584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc001040d0 msgr2=0x7fbc00196e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:49.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.340+0000 7fbbfe59c700 1 --2- 172.21.15.2:0/4135333584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc001040d0 0x7fbc00196e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:49.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.340+0000 7fbbfe59c700 1 -- 172.21.15.2:0/4135333584 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf40095f0 con 0x7fbc001017a0 2023-09-13T04:37:49.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.340+0000 7fbbfe59c700 1 --2- 172.21.15.2:0/4135333584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc001017a0 0x7fbc00196940 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000e9a0 tx=0x7fbbf000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:49.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.341+0000 7fbbef7fe700 1 -- 172.21.15.2:0/4135333584 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0009790 con 0x7fbc001017a0 2023-09-13T04:37:49.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.341+0000 7fbbef7fe700 1 -- 172.21.15.2:0/4135333584 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbf0010430 con 0x7fbc001017a0 2023-09-13T04:37:49.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.341+0000 7fbc04dff700 1 -- 172.21.15.2:0/4135333584 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc0019ea70 con 0x7fbc001017a0 2023-09-13T04:37:49.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.341+0000 7fbbef7fe700 1 -- 172.21.15.2:0/4135333584 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0018680 con 0x7fbc001017a0 2023-09-13T04:37:49.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.341+0000 7fbc04dff700 1 -- 172.21.15.2:0/4135333584 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc0019efd0 con 0x7fbc001017a0 2023-09-13T04:37:49.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.342+0000 7fbc04dff700 1 -- 172.21.15.2:0/4135333584 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc000fcd30 con 0x7fbc001017a0 2023-09-13T04:37:49.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.344+0000 7fbbef7fe700 1 -- 172.21.15.2:0/4135333584 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fbbf00105a0 con 0x7fbc001017a0 2023-09-13T04:37:49.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.346+0000 7fbbef7fe700 1 --2- 172.21.15.2:0/4135333584 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fbbe80a0710 0x7fbbe80a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:49.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.346+0000 7fbbef7fe700 1 -- 172.21.15.2:0/4135333584 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbbf0014070 con 0x7fbc001017a0 2023-09-13T04:37:49.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.347+0000 7fbbfdd9b700 1 --2- 172.21.15.2:0/4135333584 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fbbe80a0710 0x7fbbe80a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:49.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.347+0000 7fbbfdd9b700 1 --2- 172.21.15.2:0/4135333584 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fbbe80a0710 0x7fbbe80a2bd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400b2e0 tx=0x7fbbf4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:49.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.350+0000 7fbbef7fe700 1 -- 172.21.15.2:0/4135333584 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fbbf0051e10 con 0x7fbc001017a0 2023-09-13T04:37:49.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.611+0000 7fbc04dff700 1 -- 172.21.15.2:0/4135333584 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc0010c560 con 0x7fbc001017a0 2023-09-13T04:37:49.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.612+0000 7fbbef7fe700 1 -- 172.21.15.2:0/4135333584 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8447 (secure 0 0 0) 0x7fbbf0095820 con 0x7fbc001017a0 2023-09-13T04:37:49.614 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:37:49.614 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2469307307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2469307307},{"type":"v1","addr":"172.21.15.2:6839","nonce":2469307307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/846579077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":846579077},{"type":"v1","addr":"172.21.15.156:6835","nonce":846579077}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1896833690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1896833690},{"type":"v1","addr":"172.21.15.149:6837","nonce":1896833690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/524563090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":524563090},{"type":"v1","addr":"172.21.15.156:6837","nonce":524563090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/1785833227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1785833227},{"type":"v1","addr":"172.21.15.156:6839","nonce":1785833227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-09-13T04:37:49.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.616+0000 7fbc04dff700 1 -- 172.21.15.2:0/4135333584 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fbbe80a0710 msgr2=0x7fbbe80a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:49.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.616+0000 7fbc04dff700 1 --2- 172.21.15.2:0/4135333584 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fbbe80a0710 0x7fbbe80a2bd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400b2e0 tx=0x7fbbf4005fb0 comp rx=0 tx=0).stop 2023-09-13T04:37:49.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.616+0000 7fbc04dff700 1 -- 172.21.15.2:0/4135333584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc001017a0 msgr2=0x7fbc00196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:49.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.616+0000 7fbc04dff700 1 --2- 172.21.15.2:0/4135333584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc001017a0 0x7fbc00196940 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000e9a0 tx=0x7fbbf000ecb0 comp rx=0 tx=0).stop 2023-09-13T04:37:49.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.616+0000 7fbc04dff700 1 -- 172.21.15.2:0/4135333584 shutdown_connections 2023-09-13T04:37:49.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.616+0000 7fbc04dff700 1 --2- 172.21.15.2:0/4135333584 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fbbe80a0710 0x7fbbe80a2bd0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:49.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.616+0000 7fbc04dff700 1 --2- 172.21.15.2:0/4135333584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc00106a00 0x7fbc0019def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:49.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.616+0000 7fbc04dff700 1 --2- 172.21.15.2:0/4135333584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc001017a0 0x7fbc00196940 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:49.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.616+0000 7fbc04dff700 1 --2- 172.21.15.2:0/4135333584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc001040d0 0x7fbc00196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:49.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.616+0000 7fbc04dff700 1 -- 172.21.15.2:0/4135333584 >> 172.21.15.2:0/4135333584 conn(0x7fbc000fb1b0 msgr2=0x7fbc00102410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:49.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.616+0000 7fbc04dff700 1 -- 172.21.15.2:0/4135333584 shutdown_connections 2023-09-13T04:37:49.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:49.616+0000 7fbc04dff700 1 -- 172.21.15.2:0/4135333584 wait complete. 2023-09-13T04:37:49.621 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 11 2023-09-13T04:37:50.228 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:37:49 smithi002 systemd[1]: Starting Ceph mds.a for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:37:50.483 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:50 smithi002 ceph-mon[97651]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:50.484 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:50 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/4135333584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:50.484 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:37:50 smithi002 podman[148810]: 2023-09-13T04:37:50.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:50 smithi149 ceph-mon[133315]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:50.689 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:50 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/4135333584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:50.690 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:50 smithi156 ceph-mon[133675]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:50.690 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:50 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/4135333584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:51.005 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:37:50 smithi002 bash[148810]: 9244ceb730c1656d24a2b09f44ebc44a5b2e39223dc653420fbb3925747cf194 2023-09-13T04:37:51.006 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:37:50 smithi002 ceph-mds[148850]: -- 172.21.15.2:0/40409923 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5648bb60f1e0 con 0x5648bb5e4c00 2023-09-13T04:37:51.006 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:37:50 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: starting mds.a at 2023-09-13T04:37:51.006 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:37:50 smithi002 systemd[1]: Started Ceph mds.a for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:37:51.646 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:37:51.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.867+0000 7fc6357ff700 1 -- 172.21.15.2:0/3947779365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc630100c70 msgr2=0x7fc630101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:51.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.867+0000 7fc6357ff700 1 --2- 172.21.15.2:0/3947779365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc630100c70 0x7fc630101090 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc6200099c0 tx=0x7fc620009cd0 comp rx=0 tx=0).stop 2023-09-13T04:37:51.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.867+0000 7fc6357ff700 1 -- 172.21.15.2:0/3947779365 shutdown_connections 2023-09-13T04:37:51.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.867+0000 7fc6357ff700 1 --2- 172.21.15.2:0/3947779365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc630102800 0x7fc6301090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:51.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.867+0000 7fc6357ff700 1 --2- 172.21.15.2:0/3947779365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc630100c70 0x7fc630101090 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:51.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.867+0000 7fc6357ff700 1 --2- 172.21.15.2:0/3947779365 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc630101e60 0x7fc6301022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:51.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.867+0000 7fc6357ff700 1 -- 172.21.15.2:0/3947779365 >> 172.21.15.2:0/3947779365 conn(0x7fc6300fc1c0 msgr2=0x7fc6300fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:51.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.868+0000 7fc6357ff700 1 -- 172.21.15.2:0/3947779365 shutdown_connections 2023-09-13T04:37:51.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.868+0000 7fc6357ff700 1 -- 172.21.15.2:0/3947779365 wait complete. 2023-09-13T04:37:51.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.869+0000 7fc6357ff700 1 Processor -- start 2023-09-13T04:37:51.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.869+0000 7fc6357ff700 1 -- start start 2023-09-13T04:37:51.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.869+0000 7fc6357ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc630100c70 0x7fc630198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:51.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.869+0000 7fc6357ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc630101e60 0x7fc630199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:51.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.869+0000 7fc6357ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc630102800 0x7fc6301a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:51.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.869+0000 7fc6357ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6301a06b0 con 0x7fc630101e60 2023-09-13T04:37:51.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.869+0000 7fc6357ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6301a07f0 con 0x7fc630102800 2023-09-13T04:37:51.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.869+0000 7fc6357ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6301a0980 con 0x7fc630100c70 2023-09-13T04:37:51.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.869+0000 7fc62e7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc630101e60 0x7fc630199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:51.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.869+0000 7fc62e7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc630101e60 0x7fc630199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38774/0 (socket says 172.21.15.2:38774) 2023-09-13T04:37:51.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.869+0000 7fc62e7fc700 1 -- 172.21.15.2:0/1624552977 learned_addr learned my addr 172.21.15.2:0/1624552977 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:51.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.869+0000 7fc62effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc630100c70 0x7fc630198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:51.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.869+0000 7fc62f7fe700 1 --2- 172.21.15.2:0/1624552977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc630102800 0x7fc6301a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:51.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.870+0000 7fc62e7fc700 1 -- 172.21.15.2:0/1624552977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc630100c70 msgr2=0x7fc630198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:51.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.870+0000 7fc62e7fc700 1 --2- 172.21.15.2:0/1624552977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc630100c70 0x7fc630198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:51.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.870+0000 7fc62e7fc700 1 -- 172.21.15.2:0/1624552977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc630102800 msgr2=0x7fc6301a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:51.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.870+0000 7fc62e7fc700 1 --2- 172.21.15.2:0/1624552977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc630102800 0x7fc6301a00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:51.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.870+0000 7fc62e7fc700 1 -- 172.21.15.2:0/1624552977 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6200095f0 con 0x7fc630101e60 2023-09-13T04:37:51.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.870+0000 7fc62e7fc700 1 --2- 172.21.15.2:0/1624552977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc630101e60 0x7fc630199070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc62400d980 tx=0x7fc62400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:51.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.870+0000 7fc617fff700 1 -- 172.21.15.2:0/1624552977 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc624004220 con 0x7fc630101e60 2023-09-13T04:37:51.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.870+0000 7fc617fff700 1 -- 172.21.15.2:0/1624552977 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc624007640 con 0x7fc630101e60 2023-09-13T04:37:51.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.870+0000 7fc6357ff700 1 -- 172.21.15.2:0/1624552977 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6301a0c60 con 0x7fc630101e60 2023-09-13T04:37:51.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.870+0000 7fc617fff700 1 -- 172.21.15.2:0/1624552977 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc624005690 con 0x7fc630101e60 2023-09-13T04:37:51.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.870+0000 7fc6357ff700 1 -- 172.21.15.2:0/1624552977 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6301a1170 con 0x7fc630101e60 2023-09-13T04:37:51.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.871+0000 7fc6357ff700 1 -- 172.21.15.2:0/1624552977 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6301932c0 con 0x7fc630101e60 2023-09-13T04:37:51.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.873+0000 7fc617fff700 1 -- 172.21.15.2:0/1624552977 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fc624004750 con 0x7fc630101e60 2023-09-13T04:37:51.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.874+0000 7fc617fff700 1 --2- 172.21.15.2:0/1624552977 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc6180a0690 0x7fc6180a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:51.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.875+0000 7fc617fff700 1 -- 172.21.15.2:0/1624552977 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc6240cb5c0 con 0x7fc630101e60 2023-09-13T04:37:51.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.875+0000 7fc62effd700 1 --2- 172.21.15.2:0/1624552977 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc6180a0690 0x7fc6180a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:51.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.875+0000 7fc62effd700 1 --2- 172.21.15.2:0/1624552977 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc6180a0690 0x7fc6180a2b50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc620009990 tx=0x7fc620005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:51.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:51.879+0000 7fc617fff700 1 -- 172.21.15.2:0/1624552977 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc6240954f0 con 0x7fc630101e60 2023-09-13T04:37:52.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:51 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:52.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:51 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:52.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:51 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:52.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:51 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:52.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:51 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:52.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:51 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:52.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:51 smithi002 ceph-mon[97651]: Deploying daemon mds.b on smithi149 2023-09-13T04:37:52.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:51 smithi002 ceph-mon[97651]: mds.? [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] up:boot 2023-09-13T04:37:52.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:51 smithi002 ceph-mon[97651]: fsmap 11 up:standby 2023-09-13T04:37:52.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:51 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-13T04:37:52.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:51 smithi002 ceph-mon[97651]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:52.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:51 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:52.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:51 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:52.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:51 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:52.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:51 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:52.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:51 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:52.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:51 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:52.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:51 smithi149 ceph-mon[133315]: Deploying daemon mds.b on smithi149 2023-09-13T04:37:52.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:51 smithi149 ceph-mon[133315]: mds.? [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] up:boot 2023-09-13T04:37:52.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:51 smithi149 ceph-mon[133315]: fsmap 11 up:standby 2023-09-13T04:37:52.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:51 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-13T04:37:52.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:51 smithi149 ceph-mon[133315]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:52.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:52.139+0000 7fc6357ff700 1 -- 172.21.15.2:0/1624552977 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc63010c4d0 con 0x7fc630101e60 2023-09-13T04:37:52.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:52.139+0000 7fc617fff700 1 -- 172.21.15.2:0/1624552977 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9233 (secure 0 0 0) 0x7fc624098b10 con 0x7fc630101e60 2023-09-13T04:37:52.141 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:37:52.141 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2469307307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2469307307},{"type":"v1","addr":"172.21.15.2:6839","nonce":2469307307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/846579077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":846579077},{"type":"v1","addr":"172.21.15.156:6835","nonce":846579077}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1896833690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1896833690},{"type":"v1","addr":"172.21.15.149:6837","nonce":1896833690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/524563090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":524563090},{"type":"v1","addr":"172.21.15.156:6837","nonce":524563090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/1785833227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1785833227},{"type":"v1","addr":"172.21.15.156:6839","nonce":1785833227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/482545391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":482545391},{"type":"v1","addr":"172.21.15.2:6841","nonce":482545391}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[]} 2023-09-13T04:37:52.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:52.142+0000 7fc6357ff700 1 -- 172.21.15.2:0/1624552977 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc6180a0690 msgr2=0x7fc6180a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:52.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:52.142+0000 7fc6357ff700 1 --2- 172.21.15.2:0/1624552977 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc6180a0690 0x7fc6180a2b50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc620009990 tx=0x7fc620005fb0 comp rx=0 tx=0).stop 2023-09-13T04:37:52.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:52.142+0000 7fc6357ff700 1 -- 172.21.15.2:0/1624552977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc630101e60 msgr2=0x7fc630199070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:52.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:52.142+0000 7fc6357ff700 1 --2- 172.21.15.2:0/1624552977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc630101e60 0x7fc630199070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc62400d980 tx=0x7fc62400dc90 comp rx=0 tx=0).stop 2023-09-13T04:37:52.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:52.142+0000 7fc6357ff700 1 -- 172.21.15.2:0/1624552977 shutdown_connections 2023-09-13T04:37:52.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:52.142+0000 7fc6357ff700 1 --2- 172.21.15.2:0/1624552977 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc6180a0690 0x7fc6180a2b50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:52.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:52.142+0000 7fc6357ff700 1 --2- 172.21.15.2:0/1624552977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc630102800 0x7fc6301a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:52.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:52.142+0000 7fc6357ff700 1 --2- 172.21.15.2:0/1624552977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc630100c70 0x7fc630198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:52.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:52.142+0000 7fc6357ff700 1 --2- 172.21.15.2:0/1624552977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc630101e60 0x7fc630199070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:52.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:52.142+0000 7fc6357ff700 1 -- 172.21.15.2:0/1624552977 >> 172.21.15.2:0/1624552977 conn(0x7fc6300fc1c0 msgr2=0x7fc630107750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:52.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:52.142+0000 7fc6357ff700 1 -- 172.21.15.2:0/1624552977 shutdown_connections 2023-09-13T04:37:52.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:52.142+0000 7fc6357ff700 1 -- 172.21.15.2:0/1624552977 wait complete. 2023-09-13T04:37:52.146 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 12 2023-09-13T04:37:52.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:51 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:52.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:51 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:52.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:51 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:52.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:51 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T04:37:52.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:51 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T04:37:52.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:51 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:52.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:51 smithi156 ceph-mon[133675]: Deploying daemon mds.b on smithi149 2023-09-13T04:37:52.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:51 smithi156 ceph-mon[133675]: mds.? [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] up:boot 2023-09-13T04:37:52.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:51 smithi156 ceph-mon[133675]: fsmap 11 up:standby 2023-09-13T04:37:52.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:51 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-13T04:37:52.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:51 smithi156 ceph-mon[133675]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:52.262 INFO:journalctl@ceph.mds.b.smithi149.stdout:Sep 13 04:37:52 smithi149 systemd[1]: Starting Ceph mds.b for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:37:52.853 INFO:journalctl@ceph.mds.b.smithi149.stdout:Sep 13 04:37:52 smithi149 podman[169938]: 2023-09-13T04:37:52.854 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:52 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1624552977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:53.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:52 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1624552977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:53.136 INFO:journalctl@ceph.mds.b.smithi149.stdout:Sep 13 04:37:52 smithi149 bash[169938]: 5cb269cd01364a64ffd71c3968903736bf1bfe1d8b37e3a61558b87ea4452ff8 2023-09-13T04:37:53.136 INFO:journalctl@ceph.mds.b.smithi149.stdout:Sep 13 04:37:52 smithi149 ceph-mds[169978]: -- 172.21.15.149:0/4090497633 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560c3ae231e0 con 0x560c3adf8800 2023-09-13T04:37:53.137 INFO:journalctl@ceph.mds.b.smithi149.stdout:Sep 13 04:37:52 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-b[169954]: starting mds.b at 2023-09-13T04:37:53.137 INFO:journalctl@ceph.mds.b.smithi149.stdout:Sep 13 04:37:52 smithi149 systemd[1]: Started Ceph mds.b for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:37:53.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:52 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1624552977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:54.164 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:37:54.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:53 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:54.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:53 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:54.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:53 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:54.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:53 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:54.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:53 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:54.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:54 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:37:54.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:54 smithi002 ceph-mon[97651]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:54.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:54 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:54.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:54 smithi002 ceph-mon[97651]: mds.? [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] up:boot 2023-09-13T04:37:54.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:54 smithi002 ceph-mon[97651]: fsmap 12 up:standby 2023-09-13T04:37:54.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:54 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-13T04:37:54.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.384+0000 7f1044fff700 1 -- 172.21.15.2:0/2694742944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1040100ce0 msgr2=0x7f1040101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:54.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.384+0000 7f1044fff700 1 --2- 172.21.15.2:0/2694742944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1040100ce0 0x7f1040101100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f10300098f0 tx=0x7f1030009c00 comp rx=0 tx=0).stop 2023-09-13T04:37:54.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.384+0000 7f1044fff700 1 -- 172.21.15.2:0/2694742944 shutdown_connections 2023-09-13T04:37:54.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.384+0000 7f1044fff700 1 --2- 172.21.15.2:0/2694742944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1040102870 0x7f1040109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:54.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.384+0000 7f1044fff700 1 --2- 172.21.15.2:0/2694742944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1040100ce0 0x7f1040101100 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:54.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.384+0000 7f1044fff700 1 --2- 172.21.15.2:0/2694742944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1040101ed0 0x7f1040102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:54.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.384+0000 7f1044fff700 1 -- 172.21.15.2:0/2694742944 >> 172.21.15.2:0/2694742944 conn(0x7f10400fc230 msgr2=0x7f10400fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:54.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.384+0000 7f1044fff700 1 -- 172.21.15.2:0/2694742944 shutdown_connections 2023-09-13T04:37:54.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.384+0000 7f1044fff700 1 -- 172.21.15.2:0/2694742944 wait complete. 2023-09-13T04:37:54.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:54 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:54.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:54 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:54.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:54 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:54.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:54 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:54.390 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:54 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:54.390 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:54 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:37:54.390 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:54 smithi149 ceph-mon[133315]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:54.390 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:54 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:54.390 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:54 smithi149 ceph-mon[133315]: mds.? [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] up:boot 2023-09-13T04:37:54.391 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:54 smithi149 ceph-mon[133315]: fsmap 12 up:standby 2023-09-13T04:37:54.391 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:54 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-13T04:37:54.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.385+0000 7f1044fff700 1 Processor -- start 2023-09-13T04:37:54.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.386+0000 7f1044fff700 1 -- start start 2023-09-13T04:37:54.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.386+0000 7f1044fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1040100ce0 0x7f1040198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:54.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.386+0000 7f1044fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1040101ed0 0x7f1040199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:54.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.386+0000 7f1044fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1040102870 0x7f10401a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:54.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.386+0000 7f1044fff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10401a0780 con 0x7f1040101ed0 2023-09-13T04:37:54.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.386+0000 7f1044fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10401a08f0 con 0x7f1040100ce0 2023-09-13T04:37:54.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.386+0000 7f1044fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10401a0ab0 con 0x7f1040102870 2023-09-13T04:37:54.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.386+0000 7f103e59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1040100ce0 0x7f1040198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:54.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.386+0000 7f103e59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1040100ce0 0x7f1040198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:43208/0 (socket says 172.21.15.2:43208) 2023-09-13T04:37:54.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.386+0000 7f103e59c700 1 -- 172.21.15.2:0/787371690 learned_addr learned my addr 172.21.15.2:0/787371690 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:54.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.386+0000 7f103dd9b700 1 --2- 172.21.15.2:0/787371690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1040101ed0 0x7f1040199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:54.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.386+0000 7f103ed9d700 1 --2- 172.21.15.2:0/787371690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1040102870 0x7f10401a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:54.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.387+0000 7f103e59c700 1 -- 172.21.15.2:0/787371690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1040102870 msgr2=0x7f10401a01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:54.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.387+0000 7f103e59c700 1 --2- 172.21.15.2:0/787371690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1040102870 0x7f10401a01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:54.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.387+0000 7f103e59c700 1 -- 172.21.15.2:0/787371690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1040101ed0 msgr2=0x7f1040199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:54.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.387+0000 7f103e59c700 1 --2- 172.21.15.2:0/787371690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1040101ed0 0x7f1040199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:54.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.387+0000 7f103e59c700 1 -- 172.21.15.2:0/787371690 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1030009520 con 0x7f1040100ce0 2023-09-13T04:37:54.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.387+0000 7f103e59c700 1 --2- 172.21.15.2:0/787371690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1040100ce0 0x7f1040198c40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1030009fd0 tx=0x7f103000b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:54.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.387+0000 7f102f7fe700 1 -- 172.21.15.2:0/787371690 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103001c070 con 0x7f1040100ce0 2023-09-13T04:37:54.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.387+0000 7f102f7fe700 1 -- 172.21.15.2:0/787371690 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1030005040 con 0x7f1040100ce0 2023-09-13T04:37:54.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.387+0000 7f1044fff700 1 -- 172.21.15.2:0/787371690 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10401a0d30 con 0x7f1040100ce0 2023-09-13T04:37:54.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.387+0000 7f102f7fe700 1 -- 172.21.15.2:0/787371690 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1030003990 con 0x7f1040100ce0 2023-09-13T04:37:54.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.388+0000 7f1044fff700 1 -- 172.21.15.2:0/787371690 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10401a1150 con 0x7f1040100ce0 2023-09-13T04:37:54.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.389+0000 7f1044fff700 1 -- 172.21.15.2:0/787371690 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1040193330 con 0x7f1040100ce0 2023-09-13T04:37:54.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.392+0000 7f102f7fe700 1 -- 172.21.15.2:0/787371690 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f1030020030 con 0x7f1040100ce0 2023-09-13T04:37:54.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.393+0000 7f102f7fe700 1 --2- 172.21.15.2:0/787371690 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f10280a0690 0x7f10280a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:54.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.394+0000 7f102f7fe700 1 -- 172.21.15.2:0/787371690 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f10300cc720 con 0x7f1040100ce0 2023-09-13T04:37:54.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.394+0000 7f103dd9b700 1 --2- 172.21.15.2:0/787371690 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f10280a0690 0x7f10280a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:54.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.394+0000 7f103dd9b700 1 --2- 172.21.15.2:0/787371690 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f10280a0690 0x7f10280a2b50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1034005fd0 tx=0x7f1034009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:54.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.398+0000 7f102f7fe700 1 -- 172.21.15.2:0/787371690 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f10300966e0 con 0x7f1040100ce0 2023-09-13T04:37:54.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:54 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:54.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:54 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:54.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:54 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:54.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:54 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:54.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:54 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:54.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:54 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:37:54.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:54 smithi156 ceph-mon[133675]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:54.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:54 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:54.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:54 smithi156 ceph-mon[133675]: mds.? [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] up:boot 2023-09-13T04:37:54.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:54 smithi156 ceph-mon[133675]: fsmap 12 up:standby 2023-09-13T04:37:54.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:54 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-13T04:37:54.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.665+0000 7f1044fff700 1 -- 172.21.15.2:0/787371690 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f104010c540 con 0x7f1040100ce0 2023-09-13T04:37:54.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.666+0000 7f102f7fe700 1 -- 172.21.15.2:0/787371690 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10025 (secure 0 0 0) 0x7f1030099d00 con 0x7f1040100ce0 2023-09-13T04:37:54.667 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:37:54.667 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2469307307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2469307307},{"type":"v1","addr":"172.21.15.2:6839","nonce":2469307307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/846579077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":846579077},{"type":"v1","addr":"172.21.15.156:6835","nonce":846579077}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1896833690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1896833690},{"type":"v1","addr":"172.21.15.149:6837","nonce":1896833690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/524563090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":524563090},{"type":"v1","addr":"172.21.15.156:6837","nonce":524563090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/1785833227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1785833227},{"type":"v1","addr":"172.21.15.156:6839","nonce":1785833227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/482545391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":482545391},{"type":"v1","addr":"172.21.15.2:6841","nonce":482545391}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/389167841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":389167841},{"type":"v1","addr":"172.21.15.149:6839","nonce":389167841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-09-13T04:37:54.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.668+0000 7f1044fff700 1 -- 172.21.15.2:0/787371690 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f10280a0690 msgr2=0x7f10280a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:54.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.668+0000 7f1044fff700 1 --2- 172.21.15.2:0/787371690 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f10280a0690 0x7f10280a2b50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1034005fd0 tx=0x7f1034009310 comp rx=0 tx=0).stop 2023-09-13T04:37:54.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.668+0000 7f1044fff700 1 -- 172.21.15.2:0/787371690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1040100ce0 msgr2=0x7f1040198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:54.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.668+0000 7f1044fff700 1 --2- 172.21.15.2:0/787371690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1040100ce0 0x7f1040198c40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1030009fd0 tx=0x7f103000b560 comp rx=0 tx=0).stop 2023-09-13T04:37:54.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.668+0000 7f1044fff700 1 -- 172.21.15.2:0/787371690 shutdown_connections 2023-09-13T04:37:54.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.668+0000 7f1044fff700 1 --2- 172.21.15.2:0/787371690 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f10280a0690 0x7f10280a2b50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:54.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.668+0000 7f1044fff700 1 --2- 172.21.15.2:0/787371690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1040102870 0x7f10401a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:54.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.668+0000 7f1044fff700 1 --2- 172.21.15.2:0/787371690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1040100ce0 0x7f1040198c40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:54.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.668+0000 7f1044fff700 1 --2- 172.21.15.2:0/787371690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1040101ed0 0x7f1040199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:54.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.668+0000 7f1044fff700 1 -- 172.21.15.2:0/787371690 >> 172.21.15.2:0/787371690 conn(0x7f10400fc230 msgr2=0x7f1040107810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:54.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.668+0000 7f1044fff700 1 -- 172.21.15.2:0/787371690 shutdown_connections 2023-09-13T04:37:54.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.669+0000 7f1044fff700 1 -- 172.21.15.2:0/787371690 wait complete. 2023-09-13T04:37:54.673 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 13 2023-09-13T04:37:54.691 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T04:37:54.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.905+0000 7f629bfff700 1 -- 172.21.15.2:0/2184652370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62941040d0 msgr2=0x7f62941064c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:54.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.905+0000 7f629bfff700 1 --2- 172.21.15.2:0/2184652370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62941040d0 0x7f62941064c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6284009990 tx=0x7f6284009ca0 comp rx=0 tx=0).stop 2023-09-13T04:37:54.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.905+0000 7f629bfff700 1 -- 172.21.15.2:0/2184652370 shutdown_connections 2023-09-13T04:37:54.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.905+0000 7f629bfff700 1 --2- 172.21.15.2:0/2184652370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6294106a00 0x7f6294108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:54.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.905+0000 7f629bfff700 1 --2- 172.21.15.2:0/2184652370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62941017a0 0x7f6294103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:54.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.905+0000 7f629bfff700 1 --2- 172.21.15.2:0/2184652370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62941040d0 0x7f62941064c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:54.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.905+0000 7f629bfff700 1 -- 172.21.15.2:0/2184652370 >> 172.21.15.2:0/2184652370 conn(0x7f62940fb1b0 msgr2=0x7f62940fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:54.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.905+0000 7f629bfff700 1 -- 172.21.15.2:0/2184652370 shutdown_connections 2023-09-13T04:37:54.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.905+0000 7f629bfff700 1 -- 172.21.15.2:0/2184652370 wait complete. 2023-09-13T04:37:54.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.907+0000 7f629bfff700 1 Processor -- start 2023-09-13T04:37:54.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.907+0000 7f629bfff700 1 -- start start 2023-09-13T04:37:54.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.907+0000 7f629bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62941017a0 0x7f6294198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:54.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.907+0000 7f629bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62941040d0 0x7f62941990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:54.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.908+0000 7f629bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6294106a00 0x7f62941a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:54.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.908+0000 7f629bfff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62941a0710 con 0x7f6294106a00 2023-09-13T04:37:54.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.908+0000 7f629bfff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62941a0850 con 0x7f62941017a0 2023-09-13T04:37:54.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.908+0000 7f629bfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62941a09e0 con 0x7f62941040d0 2023-09-13T04:37:54.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.908+0000 7f6299d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62941017a0 0x7f6294198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:54.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.908+0000 7f629959a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62941040d0 0x7f62941990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:54.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.908+0000 7f629959a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62941040d0 0x7f62941990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:58618/0 (socket says 172.21.15.2:58618) 2023-09-13T04:37:54.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.908+0000 7f629959a700 1 -- 172.21.15.2:0/873558337 learned_addr learned my addr 172.21.15.2:0/873558337 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:54.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.908+0000 7f6299d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62941017a0 0x7f6294198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:43226/0 (socket says 172.21.15.2:43226) 2023-09-13T04:37:54.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.908+0000 7f629a59c700 1 --2- 172.21.15.2:0/873558337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6294106a00 0x7f62941a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:54.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.908+0000 7f629959a700 1 -- 172.21.15.2:0/873558337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62941017a0 msgr2=0x7f6294198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:54.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.908+0000 7f629959a700 1 --2- 172.21.15.2:0/873558337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62941017a0 0x7f6294198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:54.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.908+0000 7f629959a700 1 -- 172.21.15.2:0/873558337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6294106a00 msgr2=0x7f62941a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:54.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.908+0000 7f629959a700 1 --2- 172.21.15.2:0/873558337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6294106a00 0x7f62941a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:54.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.908+0000 7f629959a700 1 -- 172.21.15.2:0/873558337 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6290009520 con 0x7f62941040d0 2023-09-13T04:37:54.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.908+0000 7f629a59c700 1 --2- 172.21.15.2:0/873558337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6294106a00 0x7f62941a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:37:54.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.908+0000 7f629959a700 1 --2- 172.21.15.2:0/873558337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62941040d0 0x7f62941990d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6284004740 tx=0x7f6284009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:54.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.909+0000 7f628affd700 1 -- 172.21.15.2:0/873558337 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628401d070 con 0x7f62941040d0 2023-09-13T04:37:54.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.909+0000 7f628affd700 1 -- 172.21.15.2:0/873558337 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6284004830 con 0x7f62941040d0 2023-09-13T04:37:54.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.909+0000 7f628affd700 1 -- 172.21.15.2:0/873558337 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6284005450 con 0x7f62941040d0 2023-09-13T04:37:54.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.909+0000 7f629bfff700 1 -- 172.21.15.2:0/873558337 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628c009c40 con 0x7f62941040d0 2023-09-13T04:37:54.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.909+0000 7f629bfff700 1 -- 172.21.15.2:0/873558337 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62840095f0 con 0x7f62941040d0 2023-09-13T04:37:54.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.910+0000 7f629bfff700 1 -- 172.21.15.2:0/873558337 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62941085d0 con 0x7f62941040d0 2023-09-13T04:37:54.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.912+0000 7f628affd700 1 -- 172.21.15.2:0/873558337 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f628400fb00 con 0x7f62941040d0 2023-09-13T04:37:54.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.914+0000 7f628affd700 1 --2- 172.21.15.2:0/873558337 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f62800a0710 0x7f62800a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:54.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.914+0000 7f628affd700 1 -- 172.21.15.2:0/873558337 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f62840ccec0 con 0x7f62941040d0 2023-09-13T04:37:54.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.914+0000 7f6299d9b700 1 --2- 172.21.15.2:0/873558337 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f62800a0710 0x7f62800a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:54.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.915+0000 7f6299d9b700 1 --2- 172.21.15.2:0/873558337 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f62800a0710 0x7f62800a2bd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6290009df0 tx=0x7f6290009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:54.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:54.915+0000 7f628affd700 1 -- 172.21.15.2:0/873558337 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6284096e80 con 0x7f62941040d0 2023-09-13T04:37:55.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.144+0000 7f629bfff700 1 -- 172.21.15.2:0/873558337 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f629410c720 con 0x7f62941040d0 2023-09-13T04:37:55.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.147+0000 7f628affd700 1 -- 172.21.15.2:0/873558337 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18389 (secure 0 0 0) 0x7f628409a4a0 con 0x7f62941040d0 2023-09-13T04:37:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:37:55.148 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":80,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:37:11.321587+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:37:55.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.149+0000 7f629bfff700 1 -- 172.21.15.2:0/873558337 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f62800a0710 msgr2=0x7f62800a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:55.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.149+0000 7f629bfff700 1 --2- 172.21.15.2:0/873558337 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f62800a0710 0x7f62800a2bd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6290009df0 tx=0x7f6290009260 comp rx=0 tx=0).stop 2023-09-13T04:37:55.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.149+0000 7f629bfff700 1 -- 172.21.15.2:0/873558337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62941040d0 msgr2=0x7f62941990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:55.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.149+0000 7f629bfff700 1 --2- 172.21.15.2:0/873558337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62941040d0 0x7f62941990d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6284004740 tx=0x7f6284009580 comp rx=0 tx=0).stop 2023-09-13T04:37:55.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.149+0000 7f629bfff700 1 -- 172.21.15.2:0/873558337 shutdown_connections 2023-09-13T04:37:55.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.149+0000 7f629bfff700 1 --2- 172.21.15.2:0/873558337 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f62800a0710 0x7f62800a2bd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:55.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.149+0000 7f629bfff700 1 --2- 172.21.15.2:0/873558337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6294106a00 0x7f62941a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:55.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.149+0000 7f629bfff700 1 --2- 172.21.15.2:0/873558337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62941017a0 0x7f6294198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:55.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.149+0000 7f629bfff700 1 --2- 172.21.15.2:0/873558337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62941040d0 0x7f62941990d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:55.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.149+0000 7f629bfff700 1 -- 172.21.15.2:0/873558337 >> 172.21.15.2:0/873558337 conn(0x7f62940fb1b0 msgr2=0x7f62940ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:55.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.149+0000 7f629bfff700 1 -- 172.21.15.2:0/873558337 shutdown_connections 2023-09-13T04:37:55.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.149+0000 7f629bfff700 1 -- 172.21.15.2:0/873558337 wait complete. 2023-09-13T04:37:55.169 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T04:33:42.025714+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-13T04:37:55.170 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-13T04:37:55.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.389+0000 7f15683ff700 1 -- 172.21.15.2:0/4230491223 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1560101ee0 msgr2=0x7f1560102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:55.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.389+0000 7f15683ff700 1 --2- 172.21.15.2:0/4230491223 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1560101ee0 0x7f1560102340 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f15500099c0 tx=0x7f1550009cd0 comp rx=0 tx=0).stop 2023-09-13T04:37:55.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.389+0000 7f15683ff700 1 -- 172.21.15.2:0/4230491223 shutdown_connections 2023-09-13T04:37:55.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.389+0000 7f15683ff700 1 --2- 172.21.15.2:0/4230491223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1560102880 0x7f1560109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:55.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.389+0000 7f15683ff700 1 --2- 172.21.15.2:0/4230491223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1560100cf0 0x7f1560101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:55.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.389+0000 7f15683ff700 1 --2- 172.21.15.2:0/4230491223 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1560101ee0 0x7f1560102340 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:55.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.389+0000 7f15683ff700 1 -- 172.21.15.2:0/4230491223 >> 172.21.15.2:0/4230491223 conn(0x7f15600fc260 msgr2=0x7f15600fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:55.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.389+0000 7f15683ff700 1 -- 172.21.15.2:0/4230491223 shutdown_connections 2023-09-13T04:37:55.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.389+0000 7f15683ff700 1 -- 172.21.15.2:0/4230491223 wait complete. 2023-09-13T04:37:55.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.390+0000 7f15683ff700 1 Processor -- start 2023-09-13T04:37:55.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.390+0000 7f15683ff700 1 -- start start 2023-09-13T04:37:55.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.390+0000 7f15683ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1560100cf0 0x7f1560198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:55.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.390+0000 7f15683ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1560101ee0 0x7f15601991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:55.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.390+0000 7f15683ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1560102880 0x7f15601a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:55.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.390+0000 7f15683ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15601a07a0 con 0x7f1560100cf0 2023-09-13T04:37:55.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.390+0000 7f15683ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15601a0910 con 0x7f1560102880 2023-09-13T04:37:55.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.390+0000 7f15683ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15601a0ad0 con 0x7f1560101ee0 2023-09-13T04:37:55.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.391+0000 7f156599a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1560101ee0 0x7f15601991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:55.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.391+0000 7f156599a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1560101ee0 0x7f15601991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:58638/0 (socket says 172.21.15.2:58638) 2023-09-13T04:37:55.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.391+0000 7f156619b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1560100cf0 0x7f1560198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:55.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.391+0000 7f156599a700 1 -- 172.21.15.2:0/2810922752 learned_addr learned my addr 172.21.15.2:0/2810922752 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:55.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.391+0000 7f156699c700 1 --2- 172.21.15.2:0/2810922752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1560102880 0x7f15601a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:55.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.391+0000 7f156599a700 1 -- 172.21.15.2:0/2810922752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1560102880 msgr2=0x7f15601a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:55.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.391+0000 7f156599a700 1 --2- 172.21.15.2:0/2810922752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1560102880 0x7f15601a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:55.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.391+0000 7f156599a700 1 -- 172.21.15.2:0/2810922752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1560100cf0 msgr2=0x7f1560198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:55.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.391+0000 7f156599a700 1 --2- 172.21.15.2:0/2810922752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1560100cf0 0x7f1560198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:55.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.391+0000 7f156599a700 1 -- 172.21.15.2:0/2810922752 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15500095f0 con 0x7f1560101ee0 2023-09-13T04:37:55.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.391+0000 7f156699c700 1 --2- 172.21.15.2:0/2810922752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1560102880 0x7f15601a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:37:55.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.391+0000 7f156599a700 1 --2- 172.21.15.2:0/2810922752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1560101ee0 0x7f15601991a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1550009fd0 tx=0x7f155000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:55.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.391+0000 7f15577fe700 1 -- 172.21.15.2:0/2810922752 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155001d070 con 0x7f1560101ee0 2023-09-13T04:37:55.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.392+0000 7f15683ff700 1 -- 172.21.15.2:0/2810922752 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15601a0d50 con 0x7f1560101ee0 2023-09-13T04:37:55.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.392+0000 7f15577fe700 1 -- 172.21.15.2:0/2810922752 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15500054f0 con 0x7f1560101ee0 2023-09-13T04:37:55.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.392+0000 7f15577fe700 1 -- 172.21.15.2:0/2810922752 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1550004bd0 con 0x7f1560101ee0 2023-09-13T04:37:55.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.392+0000 7f15683ff700 1 -- 172.21.15.2:0/2810922752 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15601a1170 con 0x7f1560101ee0 2023-09-13T04:37:55.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.393+0000 7f15683ff700 1 -- 172.21.15.2:0/2810922752 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1560193390 con 0x7f1560101ee0 2023-09-13T04:37:55.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.395+0000 7f15577fe700 1 -- 172.21.15.2:0/2810922752 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f155000fb80 con 0x7f1560101ee0 2023-09-13T04:37:55.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.396+0000 7f15577fe700 1 --2- 172.21.15.2:0/2810922752 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f154c0a0640 0x7f154c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:55.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.396+0000 7f15577fe700 1 -- 172.21.15.2:0/2810922752 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f15500cc6e0 con 0x7f1560101ee0 2023-09-13T04:37:55.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.396+0000 7f156619b700 1 --2- 172.21.15.2:0/2810922752 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f154c0a0640 0x7f154c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:55.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.397+0000 7f156619b700 1 --2- 172.21.15.2:0/2810922752 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f154c0a0640 0x7f154c0a2b00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f156019a130 tx=0x7f155c009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:55.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.400+0000 7f15577fe700 1 -- 172.21.15.2:0/2810922752 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f15500966a0 con 0x7f1560101ee0 2023-09-13T04:37:55.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.619+0000 7f15683ff700 1 -- 172.21.15.2:0/2810922752 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f156002d000 con 0x7f1560101ee0 2023-09-13T04:37:55.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.620+0000 7f15577fe700 1 -- 172.21.15.2:0/2810922752 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1550099cc0 con 0x7f1560101ee0 2023-09-13T04:37:55.621 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-09-13T04:37:55.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.622+0000 7f15683ff700 1 -- 172.21.15.2:0/2810922752 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f154c0a0640 msgr2=0x7f154c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:55.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.622+0000 7f15683ff700 1 --2- 172.21.15.2:0/2810922752 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f154c0a0640 0x7f154c0a2b00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f156019a130 tx=0x7f155c009190 comp rx=0 tx=0).stop 2023-09-13T04:37:55.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.622+0000 7f15683ff700 1 -- 172.21.15.2:0/2810922752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1560101ee0 msgr2=0x7f15601991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:55.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.622+0000 7f15683ff700 1 --2- 172.21.15.2:0/2810922752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1560101ee0 0x7f15601991a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1550009fd0 tx=0x7f155000f690 comp rx=0 tx=0).stop 2023-09-13T04:37:55.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.623+0000 7f15683ff700 1 -- 172.21.15.2:0/2810922752 shutdown_connections 2023-09-13T04:37:55.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.623+0000 7f15683ff700 1 --2- 172.21.15.2:0/2810922752 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f154c0a0640 0x7f154c0a2b00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:55.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.623+0000 7f15683ff700 1 --2- 172.21.15.2:0/2810922752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1560102880 0x7f15601a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:55.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.623+0000 7f15683ff700 1 --2- 172.21.15.2:0/2810922752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1560100cf0 0x7f1560198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:55.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.623+0000 7f15683ff700 1 --2- 172.21.15.2:0/2810922752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1560101ee0 0x7f15601991a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:55.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.623+0000 7f15683ff700 1 -- 172.21.15.2:0/2810922752 >> 172.21.15.2:0/2810922752 conn(0x7f15600fc260 msgr2=0x7f15601077b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:55.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.623+0000 7f15683ff700 1 -- 172.21.15.2:0/2810922752 shutdown_connections 2023-09-13T04:37:55.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.623+0000 7f15683ff700 1 -- 172.21.15.2:0/2810922752 wait complete. 2023-09-13T04:37:55.643 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-13T04:37:55.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:55 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/787371690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:55.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:55 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/873558337' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:37:55.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.853+0000 7f45593ff700 1 -- 172.21.15.2:0/3713245481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4554074c90 msgr2=0x7f45540730e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:55.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.853+0000 7f45593ff700 1 --2- 172.21.15.2:0/3713245481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4554074c90 0x7f45540730e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4544009990 tx=0x7f4544009ca0 comp rx=0 tx=0).stop 2023-09-13T04:37:55.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.853+0000 7f45593ff700 1 -- 172.21.15.2:0/3713245481 shutdown_connections 2023-09-13T04:37:55.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.853+0000 7f45593ff700 1 --2- 172.21.15.2:0/3713245481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4554108ba0 0x7f455410b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:55.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.853+0000 7f45593ff700 1 --2- 172.21.15.2:0/3713245481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4554074c90 0x7f45540730e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:55.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.853+0000 7f45593ff700 1 --2- 172.21.15.2:0/3713245481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45540736b0 0x7f4554073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:55.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.853+0000 7f45593ff700 1 -- 172.21.15.2:0/3713245481 >> 172.21.15.2:0/3713245481 conn(0x7f45540fc220 msgr2=0x7f45540fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:55.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.853+0000 7f45593ff700 1 -- 172.21.15.2:0/3713245481 shutdown_connections 2023-09-13T04:37:55.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.853+0000 7f45593ff700 1 -- 172.21.15.2:0/3713245481 wait complete. 2023-09-13T04:37:55.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.854+0000 7f45593ff700 1 Processor -- start 2023-09-13T04:37:55.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.854+0000 7f45593ff700 1 -- start start 2023-09-13T04:37:55.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.855+0000 7f45593ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45540736b0 0x7f4554198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:55.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.855+0000 7f45593ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4554074c90 0x7f4554199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:55.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.855+0000 7f45593ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4554108ba0 0x7f45541a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:55.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.855+0000 7f45593ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45541a06c0 con 0x7f45540736b0 2023-09-13T04:37:55.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.855+0000 7f45593ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45541a0800 con 0x7f4554074c90 2023-09-13T04:37:55.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.855+0000 7f45593ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45541a0990 con 0x7f4554108ba0 2023-09-13T04:37:55.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.855+0000 7f4552ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45540736b0 0x7f4554198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:55.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.855+0000 7f45527fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4554074c90 0x7f4554199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:55.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.855+0000 7f4552ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45540736b0 0x7f4554198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48062/0 (socket says 172.21.15.2:48062) 2023-09-13T04:37:55.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.855+0000 7f4552ffd700 1 -- 172.21.15.2:0/3755084732 learned_addr learned my addr 172.21.15.2:0/3755084732 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:55.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.855+0000 7f45537fe700 1 --2- 172.21.15.2:0/3755084732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4554108ba0 0x7f45541a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:55.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.855+0000 7f45527fc700 1 -- 172.21.15.2:0/3755084732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4554108ba0 msgr2=0x7f45541a00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:55.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.855+0000 7f45527fc700 1 --2- 172.21.15.2:0/3755084732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4554108ba0 0x7f45541a00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:55.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.855+0000 7f45527fc700 1 -- 172.21.15.2:0/3755084732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45540736b0 msgr2=0x7f4554198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:55.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.855+0000 7f45527fc700 1 --2- 172.21.15.2:0/3755084732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45540736b0 0x7f4554198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:55.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.855+0000 7f45527fc700 1 -- 172.21.15.2:0/3755084732 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45440095f0 con 0x7f4554074c90 2023-09-13T04:37:55.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.856+0000 7f45527fc700 1 --2- 172.21.15.2:0/3755084732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4554074c90 0x7f4554199080 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f454800e9f0 tx=0x7f454800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:55.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.856+0000 7f453bfff700 1 -- 172.21.15.2:0/3755084732 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454800c9c0 con 0x7f4554074c90 2023-09-13T04:37:55.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.856+0000 7f453bfff700 1 -- 172.21.15.2:0/3755084732 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4548010430 con 0x7f4554074c90 2023-09-13T04:37:55.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.856+0000 7f453bfff700 1 -- 172.21.15.2:0/3755084732 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4548018680 con 0x7f4554074c90 2023-09-13T04:37:55.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.856+0000 7f45593ff700 1 -- 172.21.15.2:0/3755084732 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45541a0c70 con 0x7f4554074c90 2023-09-13T04:37:55.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.856+0000 7f45593ff700 1 -- 172.21.15.2:0/3755084732 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45541a1130 con 0x7f4554074c90 2023-09-13T04:37:55.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.857+0000 7f45593ff700 1 -- 172.21.15.2:0/3755084732 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45541079b0 con 0x7f4554074c90 2023-09-13T04:37:55.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.859+0000 7f453bfff700 1 -- 172.21.15.2:0/3755084732 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f4548010860 con 0x7f4554074c90 2023-09-13T04:37:55.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.860+0000 7f453bfff700 1 --2- 172.21.15.2:0/3755084732 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f453c0a0710 0x7f453c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:55.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.861+0000 7f453bfff700 1 -- 172.21.15.2:0/3755084732 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4548014070 con 0x7f4554074c90 2023-09-13T04:37:55.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.861+0000 7f4552ffd700 1 --2- 172.21.15.2:0/3755084732 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f453c0a0710 0x7f453c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:55.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.861+0000 7f4552ffd700 1 --2- 172.21.15.2:0/3755084732 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f453c0a0710 0x7f453c0a2bd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4544006010 tx=0x7f454400b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:55.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:55.862+0000 7f453bfff700 1 -- 172.21.15.2:0/3755084732 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4548095990 con 0x7f4554074c90 2023-09-13T04:37:55.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:55 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/787371690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:55.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:55 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/873558337' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:37:55.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:55 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/787371690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:37:55.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:55 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/873558337' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:37:56.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.086+0000 7f45593ff700 1 -- 172.21.15.2:0/3755084732 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f455410e750 con 0x7f4554074c90 2023-09-13T04:37:56.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.087+0000 7f453bfff700 1 -- 172.21.15.2:0/3755084732 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4548098fb0 con 0x7f4554074c90 2023-09-13T04:37:56.089 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:37:56.089 INFO:teuthology.orchestra.run.smithi002.stdout:[ 2023-09-13T04:37:56.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:37:56.090 INFO:teuthology.orchestra.run.smithi002.stdout: "poolnum": 1, 2023-09-13T04:37:56.090 INFO:teuthology.orchestra.run.smithi002.stdout: "poolname": ".mgr" 2023-09-13T04:37:56.090 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:37:56.090 INFO:teuthology.orchestra.run.smithi002.stdout:] 2023-09-13T04:37:56.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.090+0000 7f45593ff700 1 -- 172.21.15.2:0/3755084732 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f453c0a0710 msgr2=0x7f453c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:56.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.090+0000 7f45593ff700 1 --2- 172.21.15.2:0/3755084732 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f453c0a0710 0x7f453c0a2bd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4544006010 tx=0x7f454400b260 comp rx=0 tx=0).stop 2023-09-13T04:37:56.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.090+0000 7f45593ff700 1 -- 172.21.15.2:0/3755084732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4554074c90 msgr2=0x7f4554199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:56.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.090+0000 7f45593ff700 1 --2- 172.21.15.2:0/3755084732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4554074c90 0x7f4554199080 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f454800e9f0 tx=0x7f454800ed00 comp rx=0 tx=0).stop 2023-09-13T04:37:56.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.090+0000 7f45593ff700 1 -- 172.21.15.2:0/3755084732 shutdown_connections 2023-09-13T04:37:56.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.090+0000 7f45593ff700 1 --2- 172.21.15.2:0/3755084732 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f453c0a0710 0x7f453c0a2bd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:56.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.090+0000 7f45593ff700 1 --2- 172.21.15.2:0/3755084732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4554108ba0 0x7f45541a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:56.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.090+0000 7f45593ff700 1 --2- 172.21.15.2:0/3755084732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45540736b0 0x7f4554198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:56.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.090+0000 7f45593ff700 1 --2- 172.21.15.2:0/3755084732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4554074c90 0x7f4554199080 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:56.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.090+0000 7f45593ff700 1 -- 172.21.15.2:0/3755084732 >> 172.21.15.2:0/3755084732 conn(0x7f45540fc220 msgr2=0x7f4554104af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:56.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.090+0000 7f45593ff700 1 -- 172.21.15.2:0/3755084732 shutdown_connections 2023-09-13T04:37:56.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.090+0000 7f45593ff700 1 -- 172.21.15.2:0/3755084732 wait complete. 2023-09-13T04:37:56.115 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-13T04:37:56.116 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-13T04:37:56.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.341+0000 7fcddcfff700 1 -- 172.21.15.2:0/856893467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcdd8101ee0 msgr2=0x7fcdd8102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:56.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.341+0000 7fcddcfff700 1 --2- 172.21.15.2:0/856893467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcdd8101ee0 0x7fcdd8102340 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0099c0 tx=0x7fcdcc009cd0 comp rx=0 tx=0).stop 2023-09-13T04:37:56.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.341+0000 7fcddcfff700 1 -- 172.21.15.2:0/856893467 shutdown_connections 2023-09-13T04:37:56.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.341+0000 7fcddcfff700 1 --2- 172.21.15.2:0/856893467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdd8102880 0x7fcdd8109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:56.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.341+0000 7fcddcfff700 1 --2- 172.21.15.2:0/856893467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdd8100cf0 0x7fcdd8101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:56.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.341+0000 7fcddcfff700 1 --2- 172.21.15.2:0/856893467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcdd8101ee0 0x7fcdd8102340 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:56.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.341+0000 7fcddcfff700 1 -- 172.21.15.2:0/856893467 >> 172.21.15.2:0/856893467 conn(0x7fcdd80fc260 msgr2=0x7fcdd80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:56.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.341+0000 7fcddcfff700 1 -- 172.21.15.2:0/856893467 shutdown_connections 2023-09-13T04:37:56.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.341+0000 7fcddcfff700 1 -- 172.21.15.2:0/856893467 wait complete. 2023-09-13T04:37:56.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.342+0000 7fcddcfff700 1 Processor -- start 2023-09-13T04:37:56.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.342+0000 7fcddcfff700 1 -- start start 2023-09-13T04:37:56.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.342+0000 7fcddcfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcdd8100cf0 0x7fcdd8074a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:56.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.342+0000 7fcddcfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdd8101ee0 0x7fcdd80730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:56.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.342+0000 7fcddcfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdd8102880 0x7fcdd80735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:56.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.342+0000 7fcddcfff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd8073d80 con 0x7fcdd8100cf0 2023-09-13T04:37:56.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.342+0000 7fcddcfff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd8073ef0 con 0x7fcdd8102880 2023-09-13T04:37:56.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.342+0000 7fcddcfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd81a0a80 con 0x7fcdd8101ee0 2023-09-13T04:37:56.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.343+0000 7fcdd659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcdd8100cf0 0x7fcdd8074a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:56.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.343+0000 7fcdd6d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdd8102880 0x7fcdd80735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:56.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.343+0000 7fcdd5d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdd8101ee0 0x7fcdd80730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:56.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.343+0000 7fcdd659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcdd8100cf0 0x7fcdd8074a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48068/0 (socket says 172.21.15.2:48068) 2023-09-13T04:37:56.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.343+0000 7fcdd659c700 1 -- 172.21.15.2:0/1997410669 learned_addr learned my addr 172.21.15.2:0/1997410669 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:56.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.343+0000 7fcdd5d9b700 1 -- 172.21.15.2:0/1997410669 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdd8102880 msgr2=0x7fcdd80735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:56.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.343+0000 7fcdd5d9b700 1 --2- 172.21.15.2:0/1997410669 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdd8102880 0x7fcdd80735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:56.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.343+0000 7fcdd5d9b700 1 -- 172.21.15.2:0/1997410669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcdd8100cf0 msgr2=0x7fcdd8074a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:56.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.343+0000 7fcdd5d9b700 1 --2- 172.21.15.2:0/1997410669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcdd8100cf0 0x7fcdd8074a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:56.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.343+0000 7fcdd5d9b700 1 -- 172.21.15.2:0/1997410669 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdcc0095f0 con 0x7fcdd8101ee0 2023-09-13T04:37:56.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.343+0000 7fcdd6d9d700 1 --2- 172.21.15.2:0/1997410669 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdd8102880 0x7fcdd80735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:37:56.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.343+0000 7fcdd659c700 1 --2- 172.21.15.2:0/1997410669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcdd8100cf0 0x7fcdd8074a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:37:56.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.343+0000 7fcdd5d9b700 1 --2- 172.21.15.2:0/1997410669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdd8101ee0 0x7fcdd80730a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0098e0 tx=0x7fcdcc00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:56.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.344+0000 7fcdc77fe700 1 -- 172.21.15.2:0/1997410669 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc01d070 con 0x7fcdd8101ee0 2023-09-13T04:37:56.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.344+0000 7fcdc77fe700 1 -- 172.21.15.2:0/1997410669 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdcc005020 con 0x7fcdd8101ee0 2023-09-13T04:37:56.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.344+0000 7fcdc77fe700 1 -- 172.21.15.2:0/1997410669 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc004a80 con 0x7fcdd8101ee0 2023-09-13T04:37:56.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.344+0000 7fcddcfff700 1 -- 172.21.15.2:0/1997410669 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd81a0d00 con 0x7fcdd8101ee0 2023-09-13T04:37:56.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.344+0000 7fcddcfff700 1 -- 172.21.15.2:0/1997410669 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd81a1070 con 0x7fcdd8101ee0 2023-09-13T04:37:56.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.345+0000 7fcddcfff700 1 -- 172.21.15.2:0/1997410669 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd810c5e0 con 0x7fcdd8101ee0 2023-09-13T04:37:56.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.347+0000 7fcdc77fe700 1 -- 172.21.15.2:0/1997410669 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fcdcc005400 con 0x7fcdd8101ee0 2023-09-13T04:37:56.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.348+0000 7fcdc77fe700 1 --2- 172.21.15.2:0/1997410669 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fcdc00a06c0 0x7fcdc00a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:56.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.348+0000 7fcdc77fe700 1 -- 172.21.15.2:0/1997410669 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcdcc0cc310 con 0x7fcdd8101ee0 2023-09-13T04:37:56.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.348+0000 7fcdd659c700 1 --2- 172.21.15.2:0/1997410669 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fcdc00a06c0 0x7fcdc00a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:56.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.349+0000 7fcdd659c700 1 --2- 172.21.15.2:0/1997410669 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fcdc00a06c0 0x7fcdc00a2b80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcdc80104e0 tx=0x7fcdc800b450 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:56.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.350+0000 7fcdc77fe700 1 -- 172.21.15.2:0/1997410669 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcdcc096380 con 0x7fcdd8101ee0 2023-09-13T04:37:56.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:56.574+0000 7fcddcfff700 1 -- 172.21.15.2:0/1997410669 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcdd8107bf0 con 0x7fcdd8101ee0 2023-09-13T04:37:56.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:56 smithi149 ceph-mon[133315]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:56.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:56 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2810922752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:37:56.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:56 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:56.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:56 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:56.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:56 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3755084732' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-13T04:37:56.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:56 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:56.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:56 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:56.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:56 smithi156 ceph-mon[133675]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:56.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:56 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2810922752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:37:56.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:56 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:56.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:56 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:56.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:56 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3755084732' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-13T04:37:56.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:56 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:56.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:56 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:57.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:56 smithi002 ceph-mon[97651]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:57.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:56 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2810922752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:37:57.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:56 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:57.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:56 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:57.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:56 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3755084732' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-13T04:37:57.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:56 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:57.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:56 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:57.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.516+0000 7fcdc77fe700 1 -- 172.21.15.2:0/1997410669 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcdcc005190 con 0x7fcdd8101ee0 2023-09-13T04:37:57.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.521+0000 7fcddcfff700 1 -- 172.21.15.2:0/1997410669 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fcdc00a06c0 msgr2=0x7fcdc00a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:57.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.521+0000 7fcddcfff700 1 --2- 172.21.15.2:0/1997410669 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fcdc00a06c0 0x7fcdc00a2b80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcdc80104e0 tx=0x7fcdc800b450 comp rx=0 tx=0).stop 2023-09-13T04:37:57.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.521+0000 7fcddcfff700 1 -- 172.21.15.2:0/1997410669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdd8101ee0 msgr2=0x7fcdd80730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:57.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.521+0000 7fcddcfff700 1 --2- 172.21.15.2:0/1997410669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdd8101ee0 0x7fcdd80730a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0098e0 tx=0x7fcdcc00f690 comp rx=0 tx=0).stop 2023-09-13T04:37:57.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.522+0000 7fcddcfff700 1 -- 172.21.15.2:0/1997410669 shutdown_connections 2023-09-13T04:37:57.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.522+0000 7fcddcfff700 1 --2- 172.21.15.2:0/1997410669 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fcdc00a06c0 0x7fcdc00a2b80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:57.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.522+0000 7fcddcfff700 1 --2- 172.21.15.2:0/1997410669 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdd8102880 0x7fcdd80735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:57.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.522+0000 7fcddcfff700 1 --2- 172.21.15.2:0/1997410669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcdd8100cf0 0x7fcdd8074a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:57.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.522+0000 7fcddcfff700 1 --2- 172.21.15.2:0/1997410669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdd8101ee0 0x7fcdd80730a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:57.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.522+0000 7fcddcfff700 1 -- 172.21.15.2:0/1997410669 >> 172.21.15.2:0/1997410669 conn(0x7fcdd80fc260 msgr2=0x7fcdd80fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:57.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.524+0000 7fcddcfff700 1 -- 172.21.15.2:0/1997410669 shutdown_connections 2023-09-13T04:37:57.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.524+0000 7fcddcfff700 1 -- 172.21.15.2:0/1997410669 wait complete. 2023-09-13T04:37:57.526 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_metadata' created 2023-09-13T04:37:57.546 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-09-13T04:37:57.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.764+0000 7f85d2bff700 1 -- 172.21.15.2:0/909107842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85cc100ce0 msgr2=0x7f85cc101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:57.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.764+0000 7f85d2bff700 1 --2- 172.21.15.2:0/909107842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85cc100ce0 0x7f85cc101100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f85c800b0d0 tx=0x7f85c800b3e0 comp rx=0 tx=0).stop 2023-09-13T04:37:57.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.764+0000 7f85d2bff700 1 -- 172.21.15.2:0/909107842 shutdown_connections 2023-09-13T04:37:57.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.764+0000 7f85d2bff700 1 --2- 172.21.15.2:0/909107842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85cc102870 0x7f85cc109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:57.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.764+0000 7f85d2bff700 1 --2- 172.21.15.2:0/909107842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85cc100ce0 0x7f85cc101100 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:57.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.764+0000 7f85d2bff700 1 --2- 172.21.15.2:0/909107842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85cc101ed0 0x7f85cc102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:57.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.764+0000 7f85d2bff700 1 -- 172.21.15.2:0/909107842 >> 172.21.15.2:0/909107842 conn(0x7f85cc0fc230 msgr2=0x7f85cc0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:57.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.764+0000 7f85d2bff700 1 -- 172.21.15.2:0/909107842 shutdown_connections 2023-09-13T04:37:57.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.764+0000 7f85d2bff700 1 -- 172.21.15.2:0/909107842 wait complete. 2023-09-13T04:37:57.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.765+0000 7f85d2bff700 1 Processor -- start 2023-09-13T04:37:57.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.765+0000 7f85d2bff700 1 -- start start 2023-09-13T04:37:57.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.766+0000 7f85d2bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85cc100ce0 0x7f85cc198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:57.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.766+0000 7f85d2bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85cc101ed0 0x7f85cc199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:57.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.766+0000 7f85d2bff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85cc102870 0x7f85cc1a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:57.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.766+0000 7f85d2bff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc1a0880 con 0x7f85cc102870 2023-09-13T04:37:57.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.766+0000 7f85d2bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc1a09f0 con 0x7f85cc100ce0 2023-09-13T04:37:57.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.766+0000 7f85d2bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc1a0bb0 con 0x7f85cc101ed0 2023-09-13T04:37:57.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.766+0000 7f85d119c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85cc102870 0x7f85cc1a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:57.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.766+0000 7f85c3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85cc101ed0 0x7f85cc199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:57.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.766+0000 7f85d119c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85cc102870 0x7f85cc1a02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48090/0 (socket says 172.21.15.2:48090) 2023-09-13T04:37:57.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.766+0000 7f85d119c700 1 -- 172.21.15.2:0/2761592784 learned_addr learned my addr 172.21.15.2:0/2761592784 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:57.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.766+0000 7f85d099b700 1 --2- 172.21.15.2:0/2761592784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85cc100ce0 0x7f85cc198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:57.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.766+0000 7f85c3fff700 1 -- 172.21.15.2:0/2761592784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85cc100ce0 msgr2=0x7f85cc198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:57.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.766+0000 7f85c3fff700 1 --2- 172.21.15.2:0/2761592784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85cc100ce0 0x7f85cc198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:57.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.766+0000 7f85c3fff700 1 -- 172.21.15.2:0/2761592784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85cc102870 msgr2=0x7f85cc1a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:57.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.766+0000 7f85c3fff700 1 --2- 172.21.15.2:0/2761592784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85cc102870 0x7f85cc1a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:57.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.766+0000 7f85c3fff700 1 -- 172.21.15.2:0/2761592784 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c8009d00 con 0x7f85cc101ed0 2023-09-13T04:37:57.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.766+0000 7f85d119c700 1 --2- 172.21.15.2:0/2761592784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85cc102870 0x7f85cc1a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:37:57.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.767+0000 7f85c3fff700 1 --2- 172.21.15.2:0/2761592784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85cc101ed0 0x7f85cc199280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00d750 tx=0x7f85bc00da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:57.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.767+0000 7f85c1ffb700 1 -- 172.21.15.2:0/2761592784 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc004020 con 0x7f85cc101ed0 2023-09-13T04:37:57.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.767+0000 7f85c1ffb700 1 -- 172.21.15.2:0/2761592784 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85bc004830 con 0x7f85cc101ed0 2023-09-13T04:37:57.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.767+0000 7f85d2bff700 1 -- 172.21.15.2:0/2761592784 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85cc1a0e90 con 0x7f85cc101ed0 2023-09-13T04:37:57.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.767+0000 7f85c1ffb700 1 -- 172.21.15.2:0/2761592784 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc0054c0 con 0x7f85cc101ed0 2023-09-13T04:37:57.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.767+0000 7f85d2bff700 1 -- 172.21.15.2:0/2761592784 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85cc1a12b0 con 0x7f85cc101ed0 2023-09-13T04:37:57.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.768+0000 7f85d2bff700 1 -- 172.21.15.2:0/2761592784 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85cc1932e0 con 0x7f85cc101ed0 2023-09-13T04:37:57.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.770+0000 7f85c1ffb700 1 -- 172.21.15.2:0/2761592784 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f85bc0049a0 con 0x7f85cc101ed0 2023-09-13T04:37:57.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.771+0000 7f85c1ffb700 1 --2- 172.21.15.2:0/2761592784 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f85b40a0640 0x7f85b40a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:57.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.771+0000 7f85c1ffb700 1 -- 172.21.15.2:0/2761592784 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f85bc0cb270 con 0x7f85cc101ed0 2023-09-13T04:37:57.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.772+0000 7f85d099b700 1 --2- 172.21.15.2:0/2761592784 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f85b40a0640 0x7f85b40a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:57.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.772+0000 7f85d099b700 1 --2- 172.21.15.2:0/2761592784 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f85b40a0640 0x7f85b40a2b00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f85cc19a210 tx=0x7f85c800b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:57.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.775+0000 7f85c1ffb700 1 -- 172.21.15.2:0/2761592784 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f85bc095030 con 0x7f85cc101ed0 2023-09-13T04:37:57.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:57 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1997410669' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-13T04:37:57.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:57 smithi149 ceph-mon[133315]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-13T04:37:57.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:57 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:57.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:57 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:57.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:57 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:57.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:57 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:37:57.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:57 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:57.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:57 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:37:57.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:57 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1997410669' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-13T04:37:57.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:57 smithi156 ceph-mon[133675]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-13T04:37:57.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:57 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:57.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:57 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:57.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:57 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:57.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:57 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:37:57.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:57 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:57.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:57 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:37:57.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:57.992+0000 7f85d2bff700 1 -- 172.21.15.2:0/2761592784 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f85cc10c4f0 con 0x7f85cc101ed0 2023-09-13T04:37:58.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:57 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1997410669' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-13T04:37:58.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:57 smithi002 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-13T04:37:58.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:57 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:58.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:57 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:58.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:57 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:37:58.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:57 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:37:58.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:57 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:37:58.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:57 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:37:58.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.576+0000 7f85c1ffb700 1 -- 172.21.15.2:0/2761592784 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v82) v1 ==== 171+0+0 (secure 0 0 0) 0x7f85bc098650 con 0x7f85cc101ed0 2023-09-13T04:37:58.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.594+0000 7f85d2bff700 1 -- 172.21.15.2:0/2761592784 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f85b40a0640 msgr2=0x7f85b40a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:58.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.594+0000 7f85d2bff700 1 --2- 172.21.15.2:0/2761592784 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f85b40a0640 0x7f85b40a2b00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f85cc19a210 tx=0x7f85c800b040 comp rx=0 tx=0).stop 2023-09-13T04:37:58.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.594+0000 7f85d2bff700 1 -- 172.21.15.2:0/2761592784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85cc101ed0 msgr2=0x7f85cc199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:58.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.594+0000 7f85d2bff700 1 --2- 172.21.15.2:0/2761592784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85cc101ed0 0x7f85cc199280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00d750 tx=0x7f85bc00da60 comp rx=0 tx=0).stop 2023-09-13T04:37:58.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.595+0000 7f85d2bff700 1 -- 172.21.15.2:0/2761592784 shutdown_connections 2023-09-13T04:37:58.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.595+0000 7f85d2bff700 1 --2- 172.21.15.2:0/2761592784 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f85b40a0640 0x7f85b40a2b00 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f85cc19a210 tx=0x7f85c800b040 comp rx=0 tx=0).stop 2023-09-13T04:37:58.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.595+0000 7f85d2bff700 1 --2- 172.21.15.2:0/2761592784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85cc102870 0x7f85cc1a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:58.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.595+0000 7f85d2bff700 1 --2- 172.21.15.2:0/2761592784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85cc100ce0 0x7f85cc198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:58.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.595+0000 7f85d2bff700 1 --2- 172.21.15.2:0/2761592784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85cc101ed0 0x7f85cc199280 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:58.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.595+0000 7f85d2bff700 1 -- 172.21.15.2:0/2761592784 >> 172.21.15.2:0/2761592784 conn(0x7f85cc0fc230 msgr2=0x7f85cc107810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:58.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.596+0000 7f85d2bff700 1 -- 172.21.15.2:0/2761592784 shutdown_connections 2023-09-13T04:37:58.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.596+0000 7f85d2bff700 1 -- 172.21.15.2:0/2761592784 wait complete. 2023-09-13T04:37:58.600 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data' created 2023-09-13T04:37:58.619 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-13T04:37:58.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.835+0000 7f2c299ff700 1 -- 172.21.15.2:0/1729074414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c24100cf0 msgr2=0x7f2c24101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:58.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.835+0000 7f2c299ff700 1 --2- 172.21.15.2:0/1729074414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c24100cf0 0x7f2c24101110 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2c14009990 tx=0x7f2c14009ca0 comp rx=0 tx=0).stop 2023-09-13T04:37:58.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.835+0000 7f2c299ff700 1 -- 172.21.15.2:0/1729074414 shutdown_connections 2023-09-13T04:37:58.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.835+0000 7f2c299ff700 1 --2- 172.21.15.2:0/1729074414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c24102880 0x7f2c24109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:58.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.835+0000 7f2c299ff700 1 --2- 172.21.15.2:0/1729074414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c24100cf0 0x7f2c24101110 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:58.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.835+0000 7f2c299ff700 1 --2- 172.21.15.2:0/1729074414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c24101ee0 0x7f2c24102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:58.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.835+0000 7f2c299ff700 1 -- 172.21.15.2:0/1729074414 >> 172.21.15.2:0/1729074414 conn(0x7f2c240fc260 msgr2=0x7f2c240fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:58.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.835+0000 7f2c299ff700 1 -- 172.21.15.2:0/1729074414 shutdown_connections 2023-09-13T04:37:58.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.835+0000 7f2c299ff700 1 -- 172.21.15.2:0/1729074414 wait complete. 2023-09-13T04:37:58.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.836+0000 7f2c299ff700 1 Processor -- start 2023-09-13T04:37:58.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.836+0000 7f2c299ff700 1 -- start start 2023-09-13T04:37:58.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.836+0000 7f2c299ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c24100cf0 0x7f2c24198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:58.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.837+0000 7f2c299ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c24101ee0 0x7f2c241991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:58.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.837+0000 7f2c299ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c24102880 0x7f2c241a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:58.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.837+0000 7f2c299ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c241a07a0 con 0x7f2c24102880 2023-09-13T04:37:58.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.837+0000 7f2c299ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c241a0910 con 0x7f2c24100cf0 2023-09-13T04:37:58.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.837+0000 7f2c299ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c241a0ad0 con 0x7f2c24101ee0 2023-09-13T04:37:58.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.837+0000 7f2c227fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c24101ee0 0x7f2c241991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:58.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.837+0000 7f2c227fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c24101ee0 0x7f2c241991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:58682/0 (socket says 172.21.15.2:58682) 2023-09-13T04:37:58.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.837+0000 7f2c227fc700 1 -- 172.21.15.2:0/908046939 learned_addr learned my addr 172.21.15.2:0/908046939 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:58.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.837+0000 7f2c237fe700 1 --2- 172.21.15.2:0/908046939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c24102880 0x7f2c241a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:58.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.837+0000 7f2c22ffd700 1 --2- 172.21.15.2:0/908046939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c24100cf0 0x7f2c24198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:58.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.837+0000 7f2c237fe700 1 -- 172.21.15.2:0/908046939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c24101ee0 msgr2=0x7f2c241991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:58.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.837+0000 7f2c237fe700 1 --2- 172.21.15.2:0/908046939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c24101ee0 0x7f2c241991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:58.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.837+0000 7f2c237fe700 1 -- 172.21.15.2:0/908046939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c24100cf0 msgr2=0x7f2c24198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:58.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.837+0000 7f2c237fe700 1 --2- 172.21.15.2:0/908046939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c24100cf0 0x7f2c24198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:58.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.837+0000 7f2c237fe700 1 -- 172.21.15.2:0/908046939 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c140095f0 con 0x7f2c24102880 2023-09-13T04:37:58.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.837+0000 7f2c237fe700 1 --2- 172.21.15.2:0/908046939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c24102880 0x7f2c241a01d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00efc0 tx=0x7f2c1c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:58.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.838+0000 7f2c289fd700 1 -- 172.21.15.2:0/908046939 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c018070 con 0x7f2c24102880 2023-09-13T04:37:58.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.838+0000 7f2c299ff700 1 -- 172.21.15.2:0/908046939 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c241a0d50 con 0x7f2c24102880 2023-09-13T04:37:58.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.838+0000 7f2c289fd700 1 -- 172.21.15.2:0/908046939 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c1c00f040 con 0x7f2c24102880 2023-09-13T04:37:58.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.838+0000 7f2c289fd700 1 -- 172.21.15.2:0/908046939 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c00cd40 con 0x7f2c24102880 2023-09-13T04:37:58.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.838+0000 7f2c299ff700 1 -- 172.21.15.2:0/908046939 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c241a11b0 con 0x7f2c24102880 2023-09-13T04:37:58.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.839+0000 7f2c299ff700 1 -- 172.21.15.2:0/908046939 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c2410c550 con 0x7f2c24102880 2023-09-13T04:37:58.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.843+0000 7f2c289fd700 1 -- 172.21.15.2:0/908046939 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f2c1c003a00 con 0x7f2c24102880 2023-09-13T04:37:58.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.844+0000 7f2c289fd700 1 --2- 172.21.15.2:0/908046939 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2c0c0a0640 0x7f2c0c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:58.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.844+0000 7f2c289fd700 1 -- 172.21.15.2:0/908046939 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f2c1c0cbfe0 con 0x7f2c24102880 2023-09-13T04:37:58.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.844+0000 7f2c22ffd700 1 --2- 172.21.15.2:0/908046939 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2c0c0a0640 0x7f2c0c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:58.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.845+0000 7f2c22ffd700 1 --2- 172.21.15.2:0/908046939 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2c0c0a0640 0x7f2c0c0a2b00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2c14000c00 tx=0x7f2c14005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:58.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:58.847+0000 7f2c289fd700 1 -- 172.21.15.2:0/908046939 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2c1c095cd0 con 0x7f2c24102880 2023-09-13T04:37:58.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:58 smithi149 ceph-mon[133315]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:58.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:58 smithi149 ceph-mon[133315]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-13T04:37:58.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:58 smithi149 ceph-mon[133315]: osdmap e81: 12 total, 12 up, 12 in 2023-09-13T04:37:58.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:58 smithi156 ceph-mon[133675]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:58.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:58 smithi156 ceph-mon[133675]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-13T04:37:58.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:58 smithi156 ceph-mon[133675]: osdmap e81: 12 total, 12 up, 12 in 2023-09-13T04:37:59.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:58 smithi002 ceph-mon[97651]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:37:59.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:58 smithi002 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-13T04:37:59.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:58 smithi002 ceph-mon[97651]: osdmap e81: 12 total, 12 up, 12 in 2023-09-13T04:37:59.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.108+0000 7f2c299ff700 1 -- 172.21.15.2:0/908046939 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2c2410c6e0 con 0x7f2c24102880 2023-09-13T04:37:59.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.575+0000 7f2c289fd700 1 -- 172.21.15.2:0/908046939 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2c1c0992f0 con 0x7f2c24102880 2023-09-13T04:37:59.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.579+0000 7f2c299ff700 1 -- 172.21.15.2:0/908046939 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2c0c0a0640 msgr2=0x7f2c0c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:59.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.579+0000 7f2c299ff700 1 --2- 172.21.15.2:0/908046939 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2c0c0a0640 0x7f2c0c0a2b00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2c14000c00 tx=0x7f2c14005fb0 comp rx=0 tx=0).stop 2023-09-13T04:37:59.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.579+0000 7f2c299ff700 1 -- 172.21.15.2:0/908046939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c24102880 msgr2=0x7f2c241a01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:59.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.579+0000 7f2c299ff700 1 --2- 172.21.15.2:0/908046939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c24102880 0x7f2c241a01d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00efc0 tx=0x7f2c1c00c3a0 comp rx=0 tx=0).stop 2023-09-13T04:37:59.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.583+0000 7f2c299ff700 1 -- 172.21.15.2:0/908046939 shutdown_connections 2023-09-13T04:37:59.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.583+0000 7f2c299ff700 1 --2- 172.21.15.2:0/908046939 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2c0c0a0640 0x7f2c0c0a2b00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:59.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.583+0000 7f2c299ff700 1 --2- 172.21.15.2:0/908046939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c24102880 0x7f2c241a01d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:59.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.583+0000 7f2c299ff700 1 --2- 172.21.15.2:0/908046939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c24100cf0 0x7f2c24198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:59.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.583+0000 7f2c299ff700 1 --2- 172.21.15.2:0/908046939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c24101ee0 0x7f2c241991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:59.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.583+0000 7f2c299ff700 1 -- 172.21.15.2:0/908046939 >> 172.21.15.2:0/908046939 conn(0x7f2c240fc260 msgr2=0x7f2c240fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:59.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.583+0000 7f2c299ff700 1 -- 172.21.15.2:0/908046939 shutdown_connections 2023-09-13T04:37:59.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.583+0000 7f2c299ff700 1 -- 172.21.15.2:0/908046939 wait complete. 2023-09-13T04:37:59.586 INFO:teuthology.orchestra.run.smithi002.stderr:new fs with metadata pool 2 and data pool 3 2023-09-13T04:37:59.604 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-09-13T04:37:59.605 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-09-13T04:37:59.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.829+0000 7f4305bff700 1 -- 172.21.15.2:0/504696815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4300074cf0 msgr2=0x7f4300073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:59.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.829+0000 7f4305bff700 1 --2- 172.21.15.2:0/504696815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4300074cf0 0x7f4300073140 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f42f00098f0 tx=0x7f42f0009c00 comp rx=0 tx=0).stop 2023-09-13T04:37:59.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.829+0000 7f4305bff700 1 -- 172.21.15.2:0/504696815 shutdown_connections 2023-09-13T04:37:59.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.829+0000 7f4305bff700 1 --2- 172.21.15.2:0/504696815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4300108bb0 0x7f430010b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:59.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.829+0000 7f4305bff700 1 --2- 172.21.15.2:0/504696815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4300074cf0 0x7f4300073140 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:59.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.829+0000 7f4305bff700 1 --2- 172.21.15.2:0/504696815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4300073710 0x7f4300073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:59.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.829+0000 7f4305bff700 1 -- 172.21.15.2:0/504696815 >> 172.21.15.2:0/504696815 conn(0x7f43000fc260 msgr2=0x7f43000fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:37:59.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.829+0000 7f4305bff700 1 -- 172.21.15.2:0/504696815 shutdown_connections 2023-09-13T04:37:59.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.829+0000 7f4305bff700 1 -- 172.21.15.2:0/504696815 wait complete. 2023-09-13T04:37:59.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.830+0000 7f4305bff700 1 Processor -- start 2023-09-13T04:37:59.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.831+0000 7f4305bff700 1 -- start start 2023-09-13T04:37:59.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.831+0000 7f4305bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4300073710 0x7f4300198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:59.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.831+0000 7f4305bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4300074cf0 0x7f43001991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:59.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.831+0000 7f4305bff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4300108bb0 0x7f43001a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:59.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.831+0000 7f4305bff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43001a07b0 con 0x7f4300108bb0 2023-09-13T04:37:59.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.831+0000 7f4305bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43001a0920 con 0x7f4300074cf0 2023-09-13T04:37:59.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.831+0000 7f4305bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43001a0ae0 con 0x7f4300073710 2023-09-13T04:37:59.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.831+0000 7f42feffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4300074cf0 0x7f43001991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:59.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.831+0000 7f42ff7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4300073710 0x7f4300198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:59.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.831+0000 7f42fffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4300108bb0 0x7f43001a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:59.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.831+0000 7f42ff7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4300073710 0x7f4300198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:58706/0 (socket says 172.21.15.2:58706) 2023-09-13T04:37:59.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.831+0000 7f42ff7fe700 1 -- 172.21.15.2:0/2142483093 learned_addr learned my addr 172.21.15.2:0/2142483093 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:37:59.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.831+0000 7f42feffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4300074cf0 0x7f43001991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:43348/0 (socket says 172.21.15.2:43348) 2023-09-13T04:37:59.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.832+0000 7f42ff7fe700 1 -- 172.21.15.2:0/2142483093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4300074cf0 msgr2=0x7f43001991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:59.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.832+0000 7f42ff7fe700 1 --2- 172.21.15.2:0/2142483093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4300074cf0 0x7f43001991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:59.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.832+0000 7f42ff7fe700 1 -- 172.21.15.2:0/2142483093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4300108bb0 msgr2=0x7f43001a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:37:59.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.832+0000 7f42ff7fe700 1 --2- 172.21.15.2:0/2142483093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4300108bb0 0x7f43001a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:37:59.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.832+0000 7f42ff7fe700 1 -- 172.21.15.2:0/2142483093 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f40095f0 con 0x7f4300073710 2023-09-13T04:37:59.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.832+0000 7f42ff7fe700 1 --2- 172.21.15.2:0/2142483093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4300073710 0x7f4300198c70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009fd0 tx=0x7f42f000b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:59.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.832+0000 7f42fcff9700 1 -- 172.21.15.2:0/2142483093 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f001c070 con 0x7f4300073710 2023-09-13T04:37:59.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.832+0000 7f42fcff9700 1 -- 172.21.15.2:0/2142483093 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42f000f7d0 con 0x7f4300073710 2023-09-13T04:37:59.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.832+0000 7f4305bff700 1 -- 172.21.15.2:0/2142483093 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f0009520 con 0x7f4300073710 2023-09-13T04:37:59.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.832+0000 7f42fcff9700 1 -- 172.21.15.2:0/2142483093 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0004ad0 con 0x7f4300073710 2023-09-13T04:37:59.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.832+0000 7f4305bff700 1 -- 172.21.15.2:0/2142483093 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43001a0ff0 con 0x7f4300073710 2023-09-13T04:37:59.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.833+0000 7f4305bff700 1 -- 172.21.15.2:0/2142483093 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4300107260 con 0x7f4300073710 2023-09-13T04:37:59.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.835+0000 7f42fcff9700 1 -- 172.21.15.2:0/2142483093 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f42f0020070 con 0x7f4300073710 2023-09-13T04:37:59.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.837+0000 7f42fcff9700 1 --2- 172.21.15.2:0/2142483093 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f42e80a0640 0x7f42e80a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:37:59.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.837+0000 7f42fcff9700 1 -- 172.21.15.2:0/2142483093 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f42f00cc970 con 0x7f4300073710 2023-09-13T04:37:59.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.837+0000 7f42feffd700 1 --2- 172.21.15.2:0/2142483093 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f42e80a0640 0x7f42e80a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:37:59.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.837+0000 7f42feffd700 1 --2- 172.21.15.2:0/2142483093 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f42e80a0640 0x7f42e80a2b00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4300074a10 tx=0x7f42f4009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:37:59.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:37:59.839+0000 7f42fcff9700 1 -- 172.21.15.2:0/2142483093 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f42f00965e0 con 0x7f4300073710 2023-09-13T04:37:59.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:59 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2761592784' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-09-13T04:37:59.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:59 smithi149 ceph-mon[133315]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-09-13T04:37:59.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:59 smithi149 ceph-mon[133315]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-09-13T04:37:59.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:59 smithi149 ceph-mon[133315]: osdmap e82: 12 total, 12 up, 12 in 2023-09-13T04:37:59.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:59 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/908046939' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-13T04:37:59.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:59 smithi149 ceph-mon[133315]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-09-13T04:37:59.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:37:59 smithi149 ceph-mon[133315]: osdmap e83: 12 total, 12 up, 12 in 2023-09-13T04:37:59.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:59 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2761592784' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-09-13T04:37:59.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:59 smithi156 ceph-mon[133675]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-09-13T04:37:59.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:59 smithi156 ceph-mon[133675]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-09-13T04:37:59.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:59 smithi156 ceph-mon[133675]: osdmap e82: 12 total, 12 up, 12 in 2023-09-13T04:37:59.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:59 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/908046939' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-13T04:37:59.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:59 smithi156 ceph-mon[133675]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-09-13T04:37:59.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:37:59 smithi156 ceph-mon[133675]: osdmap e83: 12 total, 12 up, 12 in 2023-09-13T04:38:00.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:59 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2761592784' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-09-13T04:38:00.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:59 smithi002 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-09-13T04:38:00.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:59 smithi002 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-09-13T04:38:00.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:59 smithi002 ceph-mon[97651]: osdmap e82: 12 total, 12 up, 12 in 2023-09-13T04:38:00.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:59 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/908046939' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-13T04:38:00.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:59 smithi002 ceph-mon[97651]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-09-13T04:38:00.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:37:59 smithi002 ceph-mon[97651]: osdmap e83: 12 total, 12 up, 12 in 2023-09-13T04:38:00.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.075+0000 7f4305bff700 1 -- 172.21.15.2:0/2142483093 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f430010e730 con 0x7f4300073710 2023-09-13T04:38:00.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.231+0000 7f42fcff9700 1 -- 172.21.15.2:0/2142483093 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v84) v1 ==== 157+0+0 (secure 0 0 0) 0x7f42f0099c00 con 0x7f4300073710 2023-09-13T04:38:00.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.233+0000 7f4305bff700 1 -- 172.21.15.2:0/2142483093 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f42e80a0640 msgr2=0x7f42e80a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:00.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.233+0000 7f4305bff700 1 --2- 172.21.15.2:0/2142483093 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f42e80a0640 0x7f42e80a2b00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4300074a10 tx=0x7f42f4009310 comp rx=0 tx=0).stop 2023-09-13T04:38:00.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.233+0000 7f4305bff700 1 -- 172.21.15.2:0/2142483093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4300073710 msgr2=0x7f4300198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:00.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.233+0000 7f4305bff700 1 --2- 172.21.15.2:0/2142483093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4300073710 0x7f4300198c70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009fd0 tx=0x7f42f000b790 comp rx=0 tx=0).stop 2023-09-13T04:38:00.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.233+0000 7f4305bff700 1 -- 172.21.15.2:0/2142483093 shutdown_connections 2023-09-13T04:38:00.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.233+0000 7f4305bff700 1 --2- 172.21.15.2:0/2142483093 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f42e80a0640 0x7f42e80a2b00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:00.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.233+0000 7f4305bff700 1 --2- 172.21.15.2:0/2142483093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4300108bb0 0x7f43001a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:00.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.233+0000 7f4305bff700 1 --2- 172.21.15.2:0/2142483093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4300073710 0x7f4300198c70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:00.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.233+0000 7f4305bff700 1 --2- 172.21.15.2:0/2142483093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4300074cf0 0x7f43001991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:00.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.233+0000 7f4305bff700 1 -- 172.21.15.2:0/2142483093 >> 172.21.15.2:0/2142483093 conn(0x7f43000fc260 msgr2=0x7f43000fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:00.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.233+0000 7f4305bff700 1 -- 172.21.15.2:0/2142483093 shutdown_connections 2023-09-13T04:38:00.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.233+0000 7f4305bff700 1 -- 172.21.15.2:0/2142483093 wait complete. 2023-09-13T04:38:00.253 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-09-13T04:38:00.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.473+0000 7fef181ff700 1 -- 172.21.15.2:0/3704206009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef10074cf0 msgr2=0x7fef10073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:00.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.473+0000 7fef181ff700 1 --2- 172.21.15.2:0/3704206009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef10074cf0 0x7fef10073140 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fef0c009990 tx=0x7fef0c009ca0 comp rx=0 tx=0).stop 2023-09-13T04:38:00.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.474+0000 7fef181ff700 1 -- 172.21.15.2:0/3704206009 shutdown_connections 2023-09-13T04:38:00.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.474+0000 7fef181ff700 1 --2- 172.21.15.2:0/3704206009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef10108bb0 0x7fef1010b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:00.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.474+0000 7fef181ff700 1 --2- 172.21.15.2:0/3704206009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef10074cf0 0x7fef10073140 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:00.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.474+0000 7fef181ff700 1 --2- 172.21.15.2:0/3704206009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef10073710 0x7fef10073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:00.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.474+0000 7fef181ff700 1 -- 172.21.15.2:0/3704206009 >> 172.21.15.2:0/3704206009 conn(0x7fef100fc220 msgr2=0x7fef100fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:00.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.474+0000 7fef181ff700 1 -- 172.21.15.2:0/3704206009 shutdown_connections 2023-09-13T04:38:00.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.474+0000 7fef181ff700 1 -- 172.21.15.2:0/3704206009 wait complete. 2023-09-13T04:38:00.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.475+0000 7fef181ff700 1 Processor -- start 2023-09-13T04:38:00.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.475+0000 7fef181ff700 1 -- start start 2023-09-13T04:38:00.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.475+0000 7fef181ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef10073710 0x7fef10198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:00.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.475+0000 7fef181ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef10074cf0 0x7fef101990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:00.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.475+0000 7fef181ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef10108bb0 0x7fef101a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:00.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.475+0000 7fef181ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef101a06f0 con 0x7fef10073710 2023-09-13T04:38:00.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.475+0000 7fef181ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef101a0830 con 0x7fef10074cf0 2023-09-13T04:38:00.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.475+0000 7fef181ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef101a09c0 con 0x7fef10108bb0 2023-09-13T04:38:00.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.475+0000 7fef15f9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef10073710 0x7fef10198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:00.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.475+0000 7fef15f9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef10073710 0x7fef10198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48128/0 (socket says 172.21.15.2:48128) 2023-09-13T04:38:00.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.475+0000 7fef1579a700 1 --2- 172.21.15.2:0/2339507111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef10074cf0 0x7fef101990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:00.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.475+0000 7fef15f9b700 1 -- 172.21.15.2:0/2339507111 learned_addr learned my addr 172.21.15.2:0/2339507111 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:00.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.476+0000 7fef1679c700 1 --2- 172.21.15.2:0/2339507111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef10108bb0 0x7fef101a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:00.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.476+0000 7fef1579a700 1 -- 172.21.15.2:0/2339507111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef10108bb0 msgr2=0x7fef101a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:00.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.476+0000 7fef1579a700 1 --2- 172.21.15.2:0/2339507111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef10108bb0 0x7fef101a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:00.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.476+0000 7fef1579a700 1 -- 172.21.15.2:0/2339507111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef10073710 msgr2=0x7fef10198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:00.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.476+0000 7fef1579a700 1 --2- 172.21.15.2:0/2339507111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef10073710 0x7fef10198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:00.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.476+0000 7fef1579a700 1 -- 172.21.15.2:0/2339507111 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0c0095f0 con 0x7fef10074cf0 2023-09-13T04:38:00.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.476+0000 7fef1579a700 1 --2- 172.21.15.2:0/2339507111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef10074cf0 0x7fef101990b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fef0000e9c0 tx=0x7fef0000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:00.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.476+0000 7fef06ffd700 1 -- 172.21.15.2:0/2339507111 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef00004220 con 0x7fef10074cf0 2023-09-13T04:38:00.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.476+0000 7fef06ffd700 1 -- 172.21.15.2:0/2339507111 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef00004380 con 0x7fef10074cf0 2023-09-13T04:38:00.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.476+0000 7fef06ffd700 1 -- 172.21.15.2:0/2339507111 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef00003be0 con 0x7fef10074cf0 2023-09-13T04:38:00.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.476+0000 7fef181ff700 1 -- 172.21.15.2:0/2339507111 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef101a0ca0 con 0x7fef10074cf0 2023-09-13T04:38:00.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.477+0000 7fef181ff700 1 -- 172.21.15.2:0/2339507111 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef101a11b0 con 0x7fef10074cf0 2023-09-13T04:38:00.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.477+0000 7fef181ff700 1 -- 172.21.15.2:0/2339507111 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef10107260 con 0x7fef10074cf0 2023-09-13T04:38:00.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.479+0000 7fef06ffd700 1 -- 172.21.15.2:0/2339507111 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fef00003d40 con 0x7fef10074cf0 2023-09-13T04:38:00.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.481+0000 7fef06ffd700 1 --2- 172.21.15.2:0/2339507111 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7feefc0a0760 0x7feefc0a2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:00.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.481+0000 7fef06ffd700 1 -- 172.21.15.2:0/2339507111 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8205+0+0 (secure 0 0 0) 0x7fef00014070 con 0x7fef10074cf0 2023-09-13T04:38:00.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.481+0000 7fef15f9b700 1 --2- 172.21.15.2:0/2339507111 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7feefc0a0760 0x7feefc0a2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:00.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.481+0000 7fef15f9b700 1 --2- 172.21.15.2:0/2339507111 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7feefc0a0760 0x7feefc0a2c20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fef0c006010 tx=0x7fef0c00b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:00.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.483+0000 7fef06ffd700 1 -- 172.21.15.2:0/2339507111 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fef00095b00 con 0x7fef10074cf0 2023-09-13T04:38:00.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:00.700+0000 7fef181ff700 1 -- 172.21.15.2:0/2339507111 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fef1002d0a0 con 0x7fef10074cf0 2023-09-13T04:38:00.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:00 smithi149 ceph-mon[133315]: pgmap v271: 97 pgs: 12 active+clean, 10 creating+peering, 75 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:38:00.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:00 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/908046939' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-13T04:38:00.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:00 smithi149 ceph-mon[133315]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-09-13T04:38:00.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:00 smithi149 ceph-mon[133315]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-09-13T04:38:00.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:00 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:38:00.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:00 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:38:00.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:00 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2142483093' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-13T04:38:00.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:00 smithi149 ceph-mon[133315]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-13T04:38:00.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:00 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:38:00.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:00 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:38:00.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:00 smithi149 ceph-mon[133315]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-09-13T04:38:00.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:00 smithi149 ceph-mon[133315]: osdmap e84: 12 total, 12 up, 12 in 2023-09-13T04:38:00.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:00 smithi156 ceph-mon[133675]: pgmap v271: 97 pgs: 12 active+clean, 10 creating+peering, 75 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:38:00.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:00 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/908046939' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-13T04:38:00.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:00 smithi156 ceph-mon[133675]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-09-13T04:38:00.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:00 smithi156 ceph-mon[133675]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-09-13T04:38:00.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:00 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:38:00.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:00 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:38:00.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:00 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2142483093' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-13T04:38:00.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:00 smithi156 ceph-mon[133675]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-13T04:38:00.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:00 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:38:00.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:00 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:38:00.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:00 smithi156 ceph-mon[133675]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-09-13T04:38:00.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:00 smithi156 ceph-mon[133675]: osdmap e84: 12 total, 12 up, 12 in 2023-09-13T04:38:01.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:00 smithi002 ceph-mon[97651]: pgmap v271: 97 pgs: 12 active+clean, 10 creating+peering, 75 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:38:01.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:00 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/908046939' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-13T04:38:01.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:00 smithi002 ceph-mon[97651]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-09-13T04:38:01.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:00 smithi002 ceph-mon[97651]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-09-13T04:38:01.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:00 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:38:01.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:00 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:38:01.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:00 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2142483093' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-13T04:38:01.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:00 smithi002 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-13T04:38:01.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:00 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:38:01.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:00 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:38:01.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:00 smithi002 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-09-13T04:38:01.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:00 smithi002 ceph-mon[97651]: osdmap e84: 12 total, 12 up, 12 in 2023-09-13T04:38:02.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:02 smithi149 ceph-mon[133315]: mds.? [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] up:active 2023-09-13T04:38:02.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:02 smithi149 ceph-mon[133315]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-13T04:38:02.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:02 smithi149 ceph-mon[133315]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-13T04:38:02.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:02 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2339507111' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-13T04:38:02.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:02 smithi149 ceph-mon[133315]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-13T04:38:02.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:02 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:38:02.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:02 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:38:02.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:02 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:38:02.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:02 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:38:02.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:02 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:38:02.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:02 smithi156 ceph-mon[133675]: mds.? [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] up:active 2023-09-13T04:38:02.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:02 smithi156 ceph-mon[133675]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-13T04:38:02.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:02 smithi156 ceph-mon[133675]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-13T04:38:02.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:02 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2339507111' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-13T04:38:02.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:02 smithi156 ceph-mon[133675]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-13T04:38:02.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:02 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:38:02.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:02 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:38:02.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:02 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:38:02.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:02 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:38:02.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:02 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:38:02.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:02 smithi002 ceph-mon[97651]: mds.? [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] up:active 2023-09-13T04:38:02.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:02 smithi002 ceph-mon[97651]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-13T04:38:02.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:02 smithi002 ceph-mon[97651]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-13T04:38:02.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:02 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2339507111' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-13T04:38:02.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:02 smithi002 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-13T04:38:02.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:02 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:38:02.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:02 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:38:02.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:02 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:38:02.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:02 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:38:02.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:02 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:38:02.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.594+0000 7fef06ffd700 1 -- 172.21.15.2:0/2339507111 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v86) v1 ==== 228+0+0 (secure 0 0 0) 0x7fef00099120 con 0x7fef10074cf0 2023-09-13T04:38:02.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.602+0000 7fef181ff700 1 -- 172.21.15.2:0/2339507111 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7feefc0a0760 msgr2=0x7feefc0a2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:02.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.602+0000 7fef181ff700 1 --2- 172.21.15.2:0/2339507111 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7feefc0a0760 0x7feefc0a2c20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fef0c006010 tx=0x7fef0c00b260 comp rx=0 tx=0).stop 2023-09-13T04:38:02.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.602+0000 7fef181ff700 1 -- 172.21.15.2:0/2339507111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef10074cf0 msgr2=0x7fef101990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:02.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.602+0000 7fef181ff700 1 --2- 172.21.15.2:0/2339507111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef10074cf0 0x7fef101990b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fef0000e9c0 tx=0x7fef0000ecd0 comp rx=0 tx=0).stop 2023-09-13T04:38:02.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.603+0000 7fef181ff700 1 -- 172.21.15.2:0/2339507111 shutdown_connections 2023-09-13T04:38:02.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.603+0000 7fef181ff700 1 --2- 172.21.15.2:0/2339507111 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7feefc0a0760 0x7feefc0a2c20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:02.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.603+0000 7fef181ff700 1 --2- 172.21.15.2:0/2339507111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef10108bb0 0x7fef101a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.603+0000 7fef181ff700 1 --2- 172.21.15.2:0/2339507111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef10073710 0x7fef10198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.603+0000 7fef181ff700 1 --2- 172.21.15.2:0/2339507111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef10074cf0 0x7fef101990b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.603+0000 7fef181ff700 1 -- 172.21.15.2:0/2339507111 >> 172.21.15.2:0/2339507111 conn(0x7fef100fc220 msgr2=0x7fef100fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.603+0000 7fef181ff700 1 -- 172.21.15.2:0/2339507111 shutdown_connections 2023-09-13T04:38:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.603+0000 7fef181ff700 1 -- 172.21.15.2:0/2339507111 wait complete. 2023-09-13T04:38:02.606 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data_ec' created 2023-09-13T04:38:02.633 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-09-13T04:38:02.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.841+0000 7f5ecf7ff700 1 -- 172.21.15.2:0/745253926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec8102870 msgr2=0x7f5ec8109120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:02.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.841+0000 7f5ecf7ff700 1 --2- 172.21.15.2:0/745253926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec8102870 0x7f5ec8109120 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000b0d0 tx=0x7f5ec000b3e0 comp rx=0 tx=0).stop 2023-09-13T04:38:02.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.842+0000 7f5ecf7ff700 1 -- 172.21.15.2:0/745253926 shutdown_connections 2023-09-13T04:38:02.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.842+0000 7f5ecf7ff700 1 --2- 172.21.15.2:0/745253926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec8102870 0x7f5ec8109120 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:02.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.842+0000 7f5ecf7ff700 1 --2- 172.21.15.2:0/745253926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ec8100ce0 0x7f5ec8101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:02.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.842+0000 7f5ecf7ff700 1 --2- 172.21.15.2:0/745253926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ec8101ed0 0x7f5ec8102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:02.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.842+0000 7f5ecf7ff700 1 -- 172.21.15.2:0/745253926 >> 172.21.15.2:0/745253926 conn(0x7f5ec80fc230 msgr2=0x7f5ec80fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:02.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.842+0000 7f5ecf7ff700 1 -- 172.21.15.2:0/745253926 shutdown_connections 2023-09-13T04:38:02.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.842+0000 7f5ecf7ff700 1 -- 172.21.15.2:0/745253926 wait complete. 2023-09-13T04:38:02.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.843+0000 7f5ecf7ff700 1 Processor -- start 2023-09-13T04:38:02.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.843+0000 7f5ecf7ff700 1 -- start start 2023-09-13T04:38:02.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.843+0000 7f5ecf7ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ec8100ce0 0x7f5ec8074a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:02.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.843+0000 7f5ecf7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ec8101ed0 0x7f5ec80730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:02.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.843+0000 7f5ecf7ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec8102870 0x7f5ec80735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:02.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.843+0000 7f5ecf7ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec8073d80 con 0x7f5ec8102870 2023-09-13T04:38:02.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.843+0000 7f5ecf7ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec8073ef0 con 0x7f5ec8100ce0 2023-09-13T04:38:02.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.843+0000 7f5ecf7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec81a0ad0 con 0x7f5ec8101ed0 2023-09-13T04:38:02.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.843+0000 7f5ecdd9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec8102870 0x7f5ec80735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:02.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.843+0000 7f5ecdd9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec8102870 0x7f5ec80735e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48148/0 (socket says 172.21.15.2:48148) 2023-09-13T04:38:02.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.843+0000 7f5ecdd9c700 1 -- 172.21.15.2:0/631078756 learned_addr learned my addr 172.21.15.2:0/631078756 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:02.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.843+0000 7f5ecd59b700 1 --2- 172.21.15.2:0/631078756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ec8100ce0 0x7f5ec8074a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:02.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.843+0000 7f5eccd9a700 1 --2- 172.21.15.2:0/631078756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ec8101ed0 0x7f5ec80730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:02.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.844+0000 7f5ecdd9c700 1 -- 172.21.15.2:0/631078756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ec8101ed0 msgr2=0x7f5ec80730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:02.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.844+0000 7f5ecdd9c700 1 --2- 172.21.15.2:0/631078756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ec8101ed0 0x7f5ec80730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:02.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.844+0000 7f5ecdd9c700 1 -- 172.21.15.2:0/631078756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ec8100ce0 msgr2=0x7f5ec8074a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:02.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.844+0000 7f5ecdd9c700 1 --2- 172.21.15.2:0/631078756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ec8100ce0 0x7f5ec8074a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:02.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.844+0000 7f5ecdd9c700 1 -- 172.21.15.2:0/631078756 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec0009d00 con 0x7f5ec8102870 2023-09-13T04:38:02.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.844+0000 7f5ecdd9c700 1 --2- 172.21.15.2:0/631078756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec8102870 0x7f5ec80735e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0008ec0 tx=0x7f5ec0008ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:02.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.844+0000 7f5ebe7fc700 1 -- 172.21.15.2:0/631078756 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000e040 con 0x7f5ec8102870 2023-09-13T04:38:02.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.844+0000 7f5ecf7ff700 1 -- 172.21.15.2:0/631078756 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec81a0d50 con 0x7f5ec8102870 2023-09-13T04:38:02.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.844+0000 7f5ebe7fc700 1 -- 172.21.15.2:0/631078756 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ec0005070 con 0x7f5ec8102870 2023-09-13T04:38:02.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.844+0000 7f5ebe7fc700 1 -- 172.21.15.2:0/631078756 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec001b6c0 con 0x7f5ec8102870 2023-09-13T04:38:02.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.844+0000 7f5ecf7ff700 1 -- 172.21.15.2:0/631078756 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec81a1180 con 0x7f5ec8102870 2023-09-13T04:38:02.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.845+0000 7f5ecf7ff700 1 -- 172.21.15.2:0/631078756 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec8108220 con 0x7f5ec8102870 2023-09-13T04:38:02.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.847+0000 7f5ebe7fc700 1 -- 172.21.15.2:0/631078756 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f5ec00051e0 con 0x7f5ec8102870 2023-09-13T04:38:02.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.850+0000 7f5ebe7fc700 1 --2- 172.21.15.2:0/631078756 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f5eb40a0640 0x7f5eb40a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:02.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.850+0000 7f5ebe7fc700 1 -- 172.21.15.2:0/631078756 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f5ec00cce70 con 0x7f5ec8102870 2023-09-13T04:38:02.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.850+0000 7f5ecd59b700 1 --2- 172.21.15.2:0/631078756 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f5eb40a0640 0x7f5eb40a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:02.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.850+0000 7f5ecd59b700 1 --2- 172.21.15.2:0/631078756 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f5eb40a0640 0x7f5eb40a2b00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8101d30 tx=0x7f5ec4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:02.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:02.854+0000 7f5ebe7fc700 1 -- 172.21.15.2:0/631078756 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5ec0096800 con 0x7f5ec8102870 2023-09-13T04:38:03.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.064+0000 7f5ecf7ff700 1 -- 172.21.15.2:0/631078756 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f5ec810c500 con 0x7f5ec8102870 2023-09-13T04:38:03.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:03 smithi149 ceph-mon[133315]: pgmap v273: 97 pgs: 60 active+clean, 10 creating+peering, 27 unknown; 578 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 9 op/s 2023-09-13T04:38:03.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:03 smithi149 ceph-mon[133315]: osdmap e85: 12 total, 12 up, 12 in 2023-09-13T04:38:03.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:03 smithi149 ceph-mon[133315]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-13T04:38:03.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:03 smithi149 ceph-mon[133315]: osdmap e86: 12 total, 12 up, 12 in 2023-09-13T04:38:03.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:03 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/631078756' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-09-13T04:38:03.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:03 smithi156 ceph-mon[133675]: pgmap v273: 97 pgs: 60 active+clean, 10 creating+peering, 27 unknown; 578 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 9 op/s 2023-09-13T04:38:03.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:03 smithi156 ceph-mon[133675]: osdmap e85: 12 total, 12 up, 12 in 2023-09-13T04:38:03.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:03 smithi156 ceph-mon[133675]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-13T04:38:03.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:03 smithi156 ceph-mon[133675]: osdmap e86: 12 total, 12 up, 12 in 2023-09-13T04:38:03.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:03 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/631078756' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-09-13T04:38:03.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:03 smithi002 ceph-mon[97651]: pgmap v273: 97 pgs: 60 active+clean, 10 creating+peering, 27 unknown; 578 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 9 op/s 2023-09-13T04:38:03.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:03 smithi002 ceph-mon[97651]: osdmap e85: 12 total, 12 up, 12 in 2023-09-13T04:38:03.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:03 smithi002 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-13T04:38:03.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:03 smithi002 ceph-mon[97651]: osdmap e86: 12 total, 12 up, 12 in 2023-09-13T04:38:03.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:03 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/631078756' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-09-13T04:38:03.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.595+0000 7f5ebe7fc700 1 -- 172.21.15.2:0/631078756 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5ec0099e20 con 0x7f5ec8102870 2023-09-13T04:38:03.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.608+0000 7f5ecf7ff700 1 -- 172.21.15.2:0/631078756 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f5eb40a0640 msgr2=0x7f5eb40a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:03.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.608+0000 7f5ecf7ff700 1 --2- 172.21.15.2:0/631078756 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f5eb40a0640 0x7f5eb40a2b00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8101d30 tx=0x7f5ec4009260 comp rx=0 tx=0).stop 2023-09-13T04:38:03.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.608+0000 7f5ecf7ff700 1 -- 172.21.15.2:0/631078756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec8102870 msgr2=0x7f5ec80735e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:03.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.608+0000 7f5ecf7ff700 1 --2- 172.21.15.2:0/631078756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec8102870 0x7f5ec80735e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0008ec0 tx=0x7f5ec0008ef0 comp rx=0 tx=0).stop 2023-09-13T04:38:03.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.619+0000 7f5ecf7ff700 1 -- 172.21.15.2:0/631078756 shutdown_connections 2023-09-13T04:38:03.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.619+0000 7f5ecf7ff700 1 --2- 172.21.15.2:0/631078756 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f5eb40a0640 0x7f5eb40a2b00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:03.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.619+0000 7f5ecf7ff700 1 --2- 172.21.15.2:0/631078756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec8102870 0x7f5ec80735e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:03.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.620+0000 7f5ecf7ff700 1 --2- 172.21.15.2:0/631078756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ec8100ce0 0x7f5ec8074a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:03.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.620+0000 7f5ecf7ff700 1 --2- 172.21.15.2:0/631078756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ec8101ed0 0x7f5ec80730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:03.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.620+0000 7f5ecf7ff700 1 -- 172.21.15.2:0/631078756 >> 172.21.15.2:0/631078756 conn(0x7f5ec80fc230 msgr2=0x7f5ec8105360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:03.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.620+0000 7f5ecf7ff700 1 -- 172.21.15.2:0/631078756 shutdown_connections 2023-09-13T04:38:03.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.621+0000 7f5ecf7ff700 1 -- 172.21.15.2:0/631078756 wait complete. 2023-09-13T04:38:03.623 INFO:teuthology.orchestra.run.smithi002.stderr:set pool 4 allow_ec_overwrites to true 2023-09-13T04:38:03.642 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-09-13T04:38:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.852+0000 7f6f165ff700 1 -- 172.21.15.2:0/1173211306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f10102870 msgr2=0x7f6f10109120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:03.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.852+0000 7f6f165ff700 1 --2- 172.21.15.2:0/1173211306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f10102870 0x7f6f10109120 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6f0800b0d0 tx=0x7f6f0800b3e0 comp rx=0 tx=0).stop 2023-09-13T04:38:03.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.852+0000 7f6f165ff700 1 -- 172.21.15.2:0/1173211306 shutdown_connections 2023-09-13T04:38:03.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.852+0000 7f6f165ff700 1 --2- 172.21.15.2:0/1173211306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f10102870 0x7f6f10109120 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:03.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.853+0000 7f6f165ff700 1 --2- 172.21.15.2:0/1173211306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f10100ce0 0x7f6f10101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:03.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.853+0000 7f6f165ff700 1 --2- 172.21.15.2:0/1173211306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f10101ed0 0x7f6f10102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:03.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.853+0000 7f6f165ff700 1 -- 172.21.15.2:0/1173211306 >> 172.21.15.2:0/1173211306 conn(0x7f6f100fc230 msgr2=0x7f6f100fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:03.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.853+0000 7f6f165ff700 1 -- 172.21.15.2:0/1173211306 shutdown_connections 2023-09-13T04:38:03.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.853+0000 7f6f165ff700 1 -- 172.21.15.2:0/1173211306 wait complete. 2023-09-13T04:38:03.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.854+0000 7f6f165ff700 1 Processor -- start 2023-09-13T04:38:03.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.854+0000 7f6f165ff700 1 -- start start 2023-09-13T04:38:03.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.854+0000 7f6f165ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f10100ce0 0x7f6f10198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:03.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.854+0000 7f6f165ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f10101ed0 0x7f6f10199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:03.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.854+0000 7f6f165ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f10102870 0x7f6f101a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:03.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.854+0000 7f6f165ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f101a0830 con 0x7f6f10100ce0 2023-09-13T04:38:03.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.854+0000 7f6f165ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f101a09a0 con 0x7f6f10102870 2023-09-13T04:38:03.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.854+0000 7f6f165ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f101a0b60 con 0x7f6f10101ed0 2023-09-13T04:38:03.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.854+0000 7f6f0ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f10100ce0 0x7f6f10198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:03.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.854+0000 7f6f0ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f10100ce0 0x7f6f10198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48170/0 (socket says 172.21.15.2:48170) 2023-09-13T04:38:03.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.854+0000 7f6f0ffff700 1 -- 172.21.15.2:0/1253590282 learned_addr learned my addr 172.21.15.2:0/1253590282 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:03.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.854+0000 7f6f0f7fe700 1 --2- 172.21.15.2:0/1253590282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f10101ed0 0x7f6f10199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:03.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.854+0000 7f6f14b9c700 1 --2- 172.21.15.2:0/1253590282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f10102870 0x7f6f101a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:03.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.855+0000 7f6f0ffff700 1 -- 172.21.15.2:0/1253590282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f10101ed0 msgr2=0x7f6f10199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:03.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.855+0000 7f6f0ffff700 1 --2- 172.21.15.2:0/1253590282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f10101ed0 0x7f6f10199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:03.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.855+0000 7f6f0ffff700 1 -- 172.21.15.2:0/1253590282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f10102870 msgr2=0x7f6f101a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:03.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.855+0000 7f6f0ffff700 1 --2- 172.21.15.2:0/1253590282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f10102870 0x7f6f101a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:03.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.855+0000 7f6f0ffff700 1 -- 172.21.15.2:0/1253590282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f08009d00 con 0x7f6f10100ce0 2023-09-13T04:38:03.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.855+0000 7f6f0ffff700 1 --2- 172.21.15.2:0/1253590282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f10100ce0 0x7f6f10198cf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000e970 tx=0x7f6f0000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:03.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.855+0000 7f6f0d7fa700 1 -- 172.21.15.2:0/1253590282 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f00009790 con 0x7f6f10100ce0 2023-09-13T04:38:03.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.855+0000 7f6f0d7fa700 1 -- 172.21.15.2:0/1253590282 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f0000c7e0 con 0x7f6f10100ce0 2023-09-13T04:38:03.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.855+0000 7f6f0d7fa700 1 -- 172.21.15.2:0/1253590282 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f00003c70 con 0x7f6f10100ce0 2023-09-13T04:38:03.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.855+0000 7f6f165ff700 1 -- 172.21.15.2:0/1253590282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f101a0e40 con 0x7f6f10100ce0 2023-09-13T04:38:03.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.855+0000 7f6f165ff700 1 -- 172.21.15.2:0/1253590282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f101a1210 con 0x7f6f10100ce0 2023-09-13T04:38:03.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.856+0000 7f6f165ff700 1 -- 172.21.15.2:0/1253590282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f1010c550 con 0x7f6f10100ce0 2023-09-13T04:38:03.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.858+0000 7f6f0d7fa700 1 -- 172.21.15.2:0/1253590282 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f6f0000cc50 con 0x7f6f10100ce0 2023-09-13T04:38:03.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.860+0000 7f6f0d7fa700 1 --2- 172.21.15.2:0/1253590282 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f6ef80a06c0 0x7f6ef80a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:03.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.861+0000 7f6f0d7fa700 1 -- 172.21.15.2:0/1253590282 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f6f00014070 con 0x7f6f10100ce0 2023-09-13T04:38:03.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.861+0000 7f6f0f7fe700 1 --2- 172.21.15.2:0/1253590282 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f6ef80a06c0 0x7f6ef80a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:03.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.861+0000 7f6f0f7fe700 1 --2- 172.21.15.2:0/1253590282 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f6ef80a06c0 0x7f6ef80a2b80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009c60 tx=0x7f6f04009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:03.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:03.865+0000 7f6f0d7fa700 1 -- 172.21.15.2:0/1253590282 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6f00095840 con 0x7f6f10100ce0 2023-09-13T04:38:04.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:04.131+0000 7f6f165ff700 1 -- 172.21.15.2:0/1253590282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f6f1010c760 con 0x7f6f10100ce0 2023-09-13T04:38:04.136 INFO:journalctl@ceph.mds.b.smithi149.stdout:Sep 13 04:38:03 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-b[169954]: 2023-09-13T04:38:03.706+0000 7f5780775700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-13T04:38:04.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:04 smithi149 ceph-mon[133315]: pgmap v276: 129 pgs: 89 active+clean, 10 creating+peering, 30 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s wr, 10 op/s 2023-09-13T04:38:04.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:04 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/631078756' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-09-13T04:38:04.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:04 smithi149 ceph-mon[133315]: osdmap e87: 12 total, 12 up, 12 in 2023-09-13T04:38:04.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:04 smithi149 ceph-mon[133315]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-13T04:38:04.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:04 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1253590282' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-09-13T04:38:04.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:04 smithi149 ceph-mon[133315]: osdmap e88: 12 total, 12 up, 12 in 2023-09-13T04:38:04.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:04 smithi156 ceph-mon[133675]: pgmap v276: 129 pgs: 89 active+clean, 10 creating+peering, 30 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s wr, 10 op/s 2023-09-13T04:38:04.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:04 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/631078756' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-09-13T04:38:04.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:04 smithi156 ceph-mon[133675]: osdmap e87: 12 total, 12 up, 12 in 2023-09-13T04:38:04.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:04 smithi156 ceph-mon[133675]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-13T04:38:04.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:04 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1253590282' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-09-13T04:38:04.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:04 smithi156 ceph-mon[133675]: osdmap e88: 12 total, 12 up, 12 in 2023-09-13T04:38:05.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:04 smithi002 ceph-mon[97651]: pgmap v276: 129 pgs: 89 active+clean, 10 creating+peering, 30 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s wr, 10 op/s 2023-09-13T04:38:05.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:04 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/631078756' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-09-13T04:38:05.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:04 smithi002 ceph-mon[97651]: osdmap e87: 12 total, 12 up, 12 in 2023-09-13T04:38:05.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:04 smithi002 ceph-mon[97651]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-13T04:38:05.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:04 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1253590282' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-09-13T04:38:05.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:04 smithi002 ceph-mon[97651]: osdmap e88: 12 total, 12 up, 12 in 2023-09-13T04:38:05.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.106+0000 7f6f0d7fa700 1 -- 172.21.15.2:0/1253590282 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f6f00003dd0 con 0x7f6f10100ce0 2023-09-13T04:38:05.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.109+0000 7f6f165ff700 1 -- 172.21.15.2:0/1253590282 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f6ef80a06c0 msgr2=0x7f6ef80a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:05.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.109+0000 7f6f165ff700 1 --2- 172.21.15.2:0/1253590282 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f6ef80a06c0 0x7f6ef80a2b80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009c60 tx=0x7f6f04009260 comp rx=0 tx=0).stop 2023-09-13T04:38:05.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.109+0000 7f6f165ff700 1 -- 172.21.15.2:0/1253590282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f10100ce0 msgr2=0x7f6f10198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:05.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.109+0000 7f6f165ff700 1 --2- 172.21.15.2:0/1253590282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f10100ce0 0x7f6f10198cf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000e970 tx=0x7f6f0000ec80 comp rx=0 tx=0).stop 2023-09-13T04:38:05.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.109+0000 7f6f165ff700 1 -- 172.21.15.2:0/1253590282 shutdown_connections 2023-09-13T04:38:05.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.110+0000 7f6f165ff700 1 --2- 172.21.15.2:0/1253590282 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f6ef80a06c0 0x7f6ef80a2b80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:05.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.110+0000 7f6f165ff700 1 --2- 172.21.15.2:0/1253590282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f10102870 0x7f6f101a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:05.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.110+0000 7f6f165ff700 1 --2- 172.21.15.2:0/1253590282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f10100ce0 0x7f6f10198cf0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:05.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.110+0000 7f6f165ff700 1 --2- 172.21.15.2:0/1253590282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f10101ed0 0x7f6f10199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:05.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.110+0000 7f6f165ff700 1 -- 172.21.15.2:0/1253590282 >> 172.21.15.2:0/1253590282 conn(0x7f6f100fc230 msgr2=0x7f6f10107770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:05.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.110+0000 7f6f165ff700 1 -- 172.21.15.2:0/1253590282 shutdown_connections 2023-09-13T04:38:05.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.110+0000 7f6f165ff700 1 -- 172.21.15.2:0/1253590282 wait complete. 2023-09-13T04:38:05.113 INFO:teuthology.orchestra.run.smithi002.stderr:added data pool 4 to fsmap 2023-09-13T04:38:05.132 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:38:05.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.340+0000 7f1c1c1ff700 1 -- 172.21.15.2:0/2584222760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c141069a0 msgr2=0x7f1c14108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:05.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.340+0000 7f1c1c1ff700 1 --2- 172.21.15.2:0/2584222760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c141069a0 0x7f1c14108e10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c00b0a0 tx=0x7f1c0c00b3b0 comp rx=0 tx=0).stop 2023-09-13T04:38:05.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.340+0000 7f1c1c1ff700 1 -- 172.21.15.2:0/2584222760 shutdown_connections 2023-09-13T04:38:05.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.340+0000 7f1c1c1ff700 1 --2- 172.21.15.2:0/2584222760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c141069a0 0x7f1c14108e10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:05.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.340+0000 7f1c1c1ff700 1 --2- 172.21.15.2:0/2584222760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c14101740 0x7f1c14103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:05.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.340+0000 7f1c1c1ff700 1 --2- 172.21.15.2:0/2584222760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c14104070 0x7f1c14106460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:05.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.340+0000 7f1c1c1ff700 1 -- 172.21.15.2:0/2584222760 >> 172.21.15.2:0/2584222760 conn(0x7f1c140fb110 msgr2=0x7f1c140fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:05.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.340+0000 7f1c1c1ff700 1 -- 172.21.15.2:0/2584222760 shutdown_connections 2023-09-13T04:38:05.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.340+0000 7f1c1c1ff700 1 -- 172.21.15.2:0/2584222760 wait complete. 2023-09-13T04:38:05.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.341+0000 7f1c1c1ff700 1 Processor -- start 2023-09-13T04:38:05.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.341+0000 7f1c1c1ff700 1 -- start start 2023-09-13T04:38:05.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.342+0000 7f1c1c1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c14101740 0x7f1c14198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:05.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.342+0000 7f1c1c1ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c14104070 0x7f1c141991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:05.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.342+0000 7f1c1c1ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c141069a0 0x7f1c141a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:05.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.342+0000 7f1c1c1ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c141a07e0 con 0x7f1c14104070 2023-09-13T04:38:05.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.342+0000 7f1c1c1ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c141a0950 con 0x7f1c141069a0 2023-09-13T04:38:05.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.342+0000 7f1c1c1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c141a0b10 con 0x7f1c14101740 2023-09-13T04:38:05.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.342+0000 7f1c1979a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c14104070 0x7f1c141991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:05.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.342+0000 7f1c1979a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c14104070 0x7f1c141991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46330/0 (socket says 172.21.15.2:46330) 2023-09-13T04:38:05.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.342+0000 7f1c1979a700 1 -- 172.21.15.2:0/224518323 learned_addr learned my addr 172.21.15.2:0/224518323 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:05.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.342+0000 7f1c19f9b700 1 --2- 172.21.15.2:0/224518323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c14101740 0x7f1c14198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:05.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.342+0000 7f1c1a79c700 1 --2- 172.21.15.2:0/224518323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c141069a0 0x7f1c141a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:05.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.342+0000 7f1c1979a700 1 -- 172.21.15.2:0/224518323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c14101740 msgr2=0x7f1c14198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:05.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.342+0000 7f1c1979a700 1 --2- 172.21.15.2:0/224518323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c14101740 0x7f1c14198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:05.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.342+0000 7f1c1979a700 1 -- 172.21.15.2:0/224518323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c141069a0 msgr2=0x7f1c141a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:05.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.342+0000 7f1c1979a700 1 --2- 172.21.15.2:0/224518323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c141069a0 0x7f1c141a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:05.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.343+0000 7f1c1979a700 1 -- 172.21.15.2:0/224518323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0c009d00 con 0x7f1c14104070 2023-09-13T04:38:05.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.343+0000 7f1c1979a700 1 --2- 172.21.15.2:0/224518323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c14104070 0x7f1c141991b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1c0400e9a0 tx=0x7f1c0400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:05.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.343+0000 7f1c0affd700 1 -- 172.21.15.2:0/224518323 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c04009790 con 0x7f1c14104070 2023-09-13T04:38:05.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.343+0000 7f1c0affd700 1 -- 172.21.15.2:0/224518323 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c04010430 con 0x7f1c14104070 2023-09-13T04:38:05.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.343+0000 7f1c1c1ff700 1 -- 172.21.15.2:0/224518323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c141a0df0 con 0x7f1c14104070 2023-09-13T04:38:05.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.343+0000 7f1c0affd700 1 -- 172.21.15.2:0/224518323 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c04018680 con 0x7f1c14104070 2023-09-13T04:38:05.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.343+0000 7f1c1c1ff700 1 -- 172.21.15.2:0/224518323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c141a1270 con 0x7f1c14104070 2023-09-13T04:38:05.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.344+0000 7f1c1c1ff700 1 -- 172.21.15.2:0/224518323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c14108520 con 0x7f1c14104070 2023-09-13T04:38:05.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.346+0000 7f1c0affd700 1 -- 172.21.15.2:0/224518323 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f1c040105a0 con 0x7f1c14104070 2023-09-13T04:38:05.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.348+0000 7f1c0affd700 1 --2- 172.21.15.2:0/224518323 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1c000a0690 0x7f1c000a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:05.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.348+0000 7f1c0affd700 1 -- 172.21.15.2:0/224518323 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1c04014070 con 0x7f1c14104070 2023-09-13T04:38:05.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.348+0000 7f1c19f9b700 1 --2- 172.21.15.2:0/224518323 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1c000a0690 0x7f1c000a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:05.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.348+0000 7f1c19f9b700 1 --2- 172.21.15.2:0/224518323 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1c000a0690 0x7f1c000a2b50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1c14199b40 tx=0x7f1c10009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:05.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.351+0000 7f1c0affd700 1 -- 172.21.15.2:0/224518323 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1c04051e10 con 0x7f1c14104070 2023-09-13T04:38:05.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.607+0000 7f1c1c1ff700 1 -- 172.21.15.2:0/224518323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c1410c4f0 con 0x7f1c14104070 2023-09-13T04:38:05.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.608+0000 7f1c0affd700 1 -- 172.21.15.2:0/224518323 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11243 (secure 0 0 0) 0x7f1c04095840 con 0x7f1c14104070 2023-09-13T04:38:05.608 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:38:05.608 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2469307307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2469307307},{"type":"v1","addr":"172.21.15.2:6839","nonce":2469307307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/846579077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":846579077},{"type":"v1","addr":"172.21.15.156:6835","nonce":846579077}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1896833690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1896833690},{"type":"v1","addr":"172.21.15.149:6837","nonce":1896833690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/524563090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":524563090},{"type":"v1","addr":"172.21.15.156:6837","nonce":524563090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/1785833227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1785833227},{"type":"v1","addr":"172.21.15.156:6839","nonce":1785833227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/482545391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":482545391},{"type":"v1","addr":"172.21.15.2:6841","nonce":482545391}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-13T04:37:59.110701+0000","modified":"2023-09-13T04:38:04.132862+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24566},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/389167841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":389167841},{"type":"v1","addr":"172.21.15.149:6839","nonce":389167841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-09-13T04:38:05.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.610+0000 7f1c1c1ff700 1 -- 172.21.15.2:0/224518323 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1c000a0690 msgr2=0x7f1c000a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:05.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.610+0000 7f1c1c1ff700 1 --2- 172.21.15.2:0/224518323 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1c000a0690 0x7f1c000a2b50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1c14199b40 tx=0x7f1c10009260 comp rx=0 tx=0).stop 2023-09-13T04:38:05.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.610+0000 7f1c1c1ff700 1 -- 172.21.15.2:0/224518323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c14104070 msgr2=0x7f1c141991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:05.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.610+0000 7f1c1c1ff700 1 --2- 172.21.15.2:0/224518323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c14104070 0x7f1c141991b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1c0400e9a0 tx=0x7f1c0400ecb0 comp rx=0 tx=0).stop 2023-09-13T04:38:05.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.610+0000 7f1c1c1ff700 1 -- 172.21.15.2:0/224518323 shutdown_connections 2023-09-13T04:38:05.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.610+0000 7f1c1c1ff700 1 --2- 172.21.15.2:0/224518323 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1c000a0690 0x7f1c000a2b50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:05.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.610+0000 7f1c1c1ff700 1 --2- 172.21.15.2:0/224518323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c141069a0 0x7f1c141a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:05.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.610+0000 7f1c1c1ff700 1 --2- 172.21.15.2:0/224518323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c14101740 0x7f1c14198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:05.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.610+0000 7f1c1c1ff700 1 --2- 172.21.15.2:0/224518323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c14104070 0x7f1c141991b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:05.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.610+0000 7f1c1c1ff700 1 -- 172.21.15.2:0/224518323 >> 172.21.15.2:0/224518323 conn(0x7f1c140fb110 msgr2=0x7f1c140ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:05.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.611+0000 7f1c1c1ff700 1 -- 172.21.15.2:0/224518323 shutdown_connections 2023-09-13T04:38:05.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.611+0000 7f1c1c1ff700 1 -- 172.21.15.2:0/224518323 wait complete. 2023-09-13T04:38:05.614 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2023-09-13T04:38:05.629 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T04:38:05.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.834+0000 7fe4dc1ff700 1 -- 172.21.15.2:0/3081344100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4d4073710 msgr2=0x7fe4d4073b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:05.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.834+0000 7fe4dc1ff700 1 --2- 172.21.15.2:0/3081344100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4d4073710 0x7fe4d4073b90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4009990 tx=0x7fe4c4009ca0 comp rx=0 tx=0).stop 2023-09-13T04:38:05.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.834+0000 7fe4dc1ff700 1 -- 172.21.15.2:0/3081344100 shutdown_connections 2023-09-13T04:38:05.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.834+0000 7fe4dc1ff700 1 --2- 172.21.15.2:0/3081344100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4d4108bb0 0x7fe4d410b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:05.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.834+0000 7fe4dc1ff700 1 --2- 172.21.15.2:0/3081344100 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4d4074cf0 0x7fe4d4073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:05.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.834+0000 7fe4dc1ff700 1 --2- 172.21.15.2:0/3081344100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4d4073710 0x7fe4d4073b90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:05.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.834+0000 7fe4dc1ff700 1 -- 172.21.15.2:0/3081344100 >> 172.21.15.2:0/3081344100 conn(0x7fe4d40fc220 msgr2=0x7fe4d40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:05.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.834+0000 7fe4dc1ff700 1 -- 172.21.15.2:0/3081344100 shutdown_connections 2023-09-13T04:38:05.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.834+0000 7fe4dc1ff700 1 -- 172.21.15.2:0/3081344100 wait complete. 2023-09-13T04:38:05.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.836+0000 7fe4dc1ff700 1 Processor -- start 2023-09-13T04:38:05.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.836+0000 7fe4dc1ff700 1 -- start start 2023-09-13T04:38:05.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.836+0000 7fe4dc1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4d4073710 0x7fe4d4198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:05.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.837+0000 7fe4dc1ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4d4074cf0 0x7fe4d4199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:05.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.837+0000 7fe4dc1ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4d4108bb0 0x7fe4d41a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:05.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.837+0000 7fe4dc1ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d41a0770 con 0x7fe4d4108bb0 2023-09-13T04:38:05.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.837+0000 7fe4dc1ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d41a08b0 con 0x7fe4d4074cf0 2023-09-13T04:38:05.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.837+0000 7fe4dc1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d41a0a40 con 0x7fe4d4073710 2023-09-13T04:38:05.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.837+0000 7fe4d9f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4d4073710 0x7fe4d4198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:05.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.837+0000 7fe4d9f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4d4073710 0x7fe4d4198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:55486/0 (socket says 172.21.15.2:55486) 2023-09-13T04:38:05.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.837+0000 7fe4d9f9b700 1 -- 172.21.15.2:0/301044677 learned_addr learned my addr 172.21.15.2:0/301044677 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:05.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.837+0000 7fe4d979a700 1 --2- 172.21.15.2:0/301044677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4d4074cf0 0x7fe4d4199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:05.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.837+0000 7fe4da79c700 1 --2- 172.21.15.2:0/301044677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4d4108bb0 0x7fe4d41a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:05.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.837+0000 7fe4d979a700 1 -- 172.21.15.2:0/301044677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4d4073710 msgr2=0x7fe4d4198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:05.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.837+0000 7fe4d979a700 1 --2- 172.21.15.2:0/301044677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4d4073710 0x7fe4d4198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:05.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.837+0000 7fe4d979a700 1 -- 172.21.15.2:0/301044677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4d4108bb0 msgr2=0x7fe4d41a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:05.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.837+0000 7fe4d979a700 1 --2- 172.21.15.2:0/301044677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4d4108bb0 0x7fe4d41a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:05.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.837+0000 7fe4d979a700 1 -- 172.21.15.2:0/301044677 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d0009520 con 0x7fe4d4074cf0 2023-09-13T04:38:05.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.837+0000 7fe4da79c700 1 --2- 172.21.15.2:0/301044677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4d4108bb0 0x7fe4d41a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:38:05.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.838+0000 7fe4d979a700 1 --2- 172.21.15.2:0/301044677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4d4074cf0 0x7fe4d4199130 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400b680 tx=0x7fe4c4009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:05.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.838+0000 7fe4caffd700 1 -- 172.21.15.2:0/301044677 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c401d070 con 0x7fe4d4074cf0 2023-09-13T04:38:05.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.838+0000 7fe4dc1ff700 1 -- 172.21.15.2:0/301044677 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c40095f0 con 0x7fe4d4074cf0 2023-09-13T04:38:05.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.838+0000 7fe4dc1ff700 1 -- 172.21.15.2:0/301044677 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d41a0fb0 con 0x7fe4d4074cf0 2023-09-13T04:38:05.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.838+0000 7fe4caffd700 1 -- 172.21.15.2:0/301044677 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4c4021410 con 0x7fe4d4074cf0 2023-09-13T04:38:05.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.838+0000 7fe4caffd700 1 -- 172.21.15.2:0/301044677 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4021ea0 con 0x7fe4d4074cf0 2023-09-13T04:38:05.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.839+0000 7fe4dc1ff700 1 -- 172.21.15.2:0/301044677 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d4107260 con 0x7fe4d4074cf0 2023-09-13T04:38:05.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.841+0000 7fe4caffd700 1 -- 172.21.15.2:0/301044677 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fe4c4021830 con 0x7fe4d4074cf0 2023-09-13T04:38:05.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.843+0000 7fe4caffd700 1 --2- 172.21.15.2:0/301044677 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe4c00a0710 0x7fe4c00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:05.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.843+0000 7fe4caffd700 1 -- 172.21.15.2:0/301044677 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe4c40cd360 con 0x7fe4d4074cf0 2023-09-13T04:38:05.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.843+0000 7fe4d9f9b700 1 --2- 172.21.15.2:0/301044677 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe4c00a0710 0x7fe4c00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:05.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.843+0000 7fe4d9f9b700 1 --2- 172.21.15.2:0/301044677 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe4c00a0710 0x7fe4c00a2bd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009ef0 tx=0x7fe4d0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:05.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:05.845+0000 7fe4caffd700 1 -- 172.21.15.2:0/301044677 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe4c4096cd0 con 0x7fe4d4074cf0 2023-09-13T04:38:06.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.057+0000 7fe4dc1ff700 1 -- 172.21.15.2:0/301044677 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4d410e8b0 con 0x7fe4d4074cf0 2023-09-13T04:38:06.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.059+0000 7fe4caffd700 1 -- 172.21.15.2:0/301044677 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23927 (secure 0 0 0) 0x7fe4c409a2f0 con 0x7fe4d4074cf0 2023-09-13T04:38:06.059 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:38:06.060 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":88,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:38:04.132627+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T04:37:57.094802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T04:37:58.564487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-13T04:38:02.096895+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:38:06.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.061+0000 7fe4dc1ff700 1 -- 172.21.15.2:0/301044677 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe4c00a0710 msgr2=0x7fe4c00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:06.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.061+0000 7fe4dc1ff700 1 --2- 172.21.15.2:0/301044677 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe4c00a0710 0x7fe4c00a2bd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009ef0 tx=0x7fe4d0009260 comp rx=0 tx=0).stop 2023-09-13T04:38:06.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.061+0000 7fe4dc1ff700 1 -- 172.21.15.2:0/301044677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4d4074cf0 msgr2=0x7fe4d4199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:06.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.061+0000 7fe4dc1ff700 1 --2- 172.21.15.2:0/301044677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4d4074cf0 0x7fe4d4199130 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400b680 tx=0x7fe4c4009580 comp rx=0 tx=0).stop 2023-09-13T04:38:06.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.061+0000 7fe4dc1ff700 1 -- 172.21.15.2:0/301044677 shutdown_connections 2023-09-13T04:38:06.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.061+0000 7fe4dc1ff700 1 --2- 172.21.15.2:0/301044677 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe4c00a0710 0x7fe4c00a2bd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:06.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.061+0000 7fe4dc1ff700 1 --2- 172.21.15.2:0/301044677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4d4108bb0 0x7fe4d41a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:06.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.061+0000 7fe4dc1ff700 1 --2- 172.21.15.2:0/301044677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4d4073710 0x7fe4d4198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:06.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.061+0000 7fe4dc1ff700 1 --2- 172.21.15.2:0/301044677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4d4074cf0 0x7fe4d4199130 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:06.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.061+0000 7fe4dc1ff700 1 -- 172.21.15.2:0/301044677 >> 172.21.15.2:0/301044677 conn(0x7fe4d40fc220 msgr2=0x7fe4d40fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:06.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.062+0000 7fe4dc1ff700 1 -- 172.21.15.2:0/301044677 shutdown_connections 2023-09-13T04:38:06.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.062+0000 7fe4dc1ff700 1 -- 172.21.15.2:0/301044677 wait complete. 2023-09-13T04:38:06.081 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T04:38:06.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.286+0000 7f62711ff700 1 -- 172.21.15.2:0/650708257 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f626c101ee0 msgr2=0x7f626c102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:06.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.286+0000 7f62711ff700 1 --2- 172.21.15.2:0/650708257 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f626c101ee0 0x7f626c102340 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f62600099c0 tx=0x7f6260009cd0 comp rx=0 tx=0).stop 2023-09-13T04:38:06.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.286+0000 7f62711ff700 1 -- 172.21.15.2:0/650708257 shutdown_connections 2023-09-13T04:38:06.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.286+0000 7f62711ff700 1 --2- 172.21.15.2:0/650708257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f626c102880 0x7f626c109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:06.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.286+0000 7f62711ff700 1 --2- 172.21.15.2:0/650708257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f626c100cf0 0x7f626c101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:06.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.286+0000 7f62711ff700 1 --2- 172.21.15.2:0/650708257 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f626c101ee0 0x7f626c102340 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:06.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.286+0000 7f62711ff700 1 -- 172.21.15.2:0/650708257 >> 172.21.15.2:0/650708257 conn(0x7f626c0fc220 msgr2=0x7f626c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:06.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.286+0000 7f62711ff700 1 -- 172.21.15.2:0/650708257 shutdown_connections 2023-09-13T04:38:06.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.287+0000 7f62711ff700 1 -- 172.21.15.2:0/650708257 wait complete. 2023-09-13T04:38:06.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.287+0000 7f62711ff700 1 Processor -- start 2023-09-13T04:38:06.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.288+0000 7f62711ff700 1 -- start start 2023-09-13T04:38:06.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.288+0000 7f62711ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f626c100cf0 0x7f626c198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:06.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.288+0000 7f62711ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f626c101ee0 0x7f626c199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:06.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.288+0000 7f62711ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f626c102880 0x7f626c1a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:06.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.288+0000 7f62711ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626c1a0770 con 0x7f626c101ee0 2023-09-13T04:38:06.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.288+0000 7f62711ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626c1a08b0 con 0x7f626c102880 2023-09-13T04:38:06.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.288+0000 7f62711ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626c1a0a40 con 0x7f626c100cf0 2023-09-13T04:38:06.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.288+0000 7f626a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f626c101ee0 0x7f626c199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:06.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.288+0000 7f626b59e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f626c102880 0x7f626c1a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:06.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.288+0000 7f626a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f626c101ee0 0x7f626c199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46358/0 (socket says 172.21.15.2:46358) 2023-09-13T04:38:06.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.288+0000 7f626a59c700 1 -- 172.21.15.2:0/3070575562 learned_addr learned my addr 172.21.15.2:0/3070575562 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:06.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.288+0000 7f626ad9d700 1 --2- 172.21.15.2:0/3070575562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f626c100cf0 0x7f626c198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:06.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.289+0000 7f626b59e700 1 -- 172.21.15.2:0/3070575562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f626c100cf0 msgr2=0x7f626c198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:06.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.289+0000 7f626b59e700 1 --2- 172.21.15.2:0/3070575562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f626c100cf0 0x7f626c198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:06.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.289+0000 7f626b59e700 1 -- 172.21.15.2:0/3070575562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f626c101ee0 msgr2=0x7f626c199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:06.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.289+0000 7f626b59e700 1 --2- 172.21.15.2:0/3070575562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f626c101ee0 0x7f626c199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:06.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.289+0000 7f626b59e700 1 -- 172.21.15.2:0/3070575562 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62600095f0 con 0x7f626c102880 2023-09-13T04:38:06.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.289+0000 7f626ad9d700 1 --2- 172.21.15.2:0/3070575562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f626c100cf0 0x7f626c198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:38:06.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.289+0000 7f626a59c700 1 --2- 172.21.15.2:0/3070575562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f626c101ee0 0x7f626c199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:38:06.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.289+0000 7f626b59e700 1 --2- 172.21.15.2:0/3070575562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f626c102880 0x7f626c1a01a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f626400efc0 tx=0x7f626400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:06.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.289+0000 7f6253fff700 1 -- 172.21.15.2:0/3070575562 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6264018070 con 0x7f626c102880 2023-09-13T04:38:06.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.289+0000 7f6253fff700 1 -- 172.21.15.2:0/3070575562 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f626400f040 con 0x7f626c102880 2023-09-13T04:38:06.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.289+0000 7f6253fff700 1 -- 172.21.15.2:0/3070575562 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626400cdf0 con 0x7f626c102880 2023-09-13T04:38:06.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.289+0000 7f62711ff700 1 -- 172.21.15.2:0/3070575562 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f626c1a0cc0 con 0x7f626c102880 2023-09-13T04:38:06.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.289+0000 7f62711ff700 1 -- 172.21.15.2:0/3070575562 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f626c1a1180 con 0x7f626c102880 2023-09-13T04:38:06.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.290+0000 7f62711ff700 1 -- 172.21.15.2:0/3070575562 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f626c193340 con 0x7f626c102880 2023-09-13T04:38:06.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.292+0000 7f6253fff700 1 -- 172.21.15.2:0/3070575562 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f6264003a00 con 0x7f626c102880 2023-09-13T04:38:06.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.293+0000 7f6253fff700 1 --2- 172.21.15.2:0/3070575562 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f62540a0710 0x7f62540a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:06.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.294+0000 7f6253fff700 1 -- 172.21.15.2:0/3070575562 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f62640cc0b0 con 0x7f626c102880 2023-09-13T04:38:06.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.294+0000 7f626ad9d700 1 --2- 172.21.15.2:0/3070575562 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f62540a0710 0x7f62540a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:06.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.294+0000 7f626ad9d700 1 --2- 172.21.15.2:0/3070575562 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f62540a0710 0x7f62540a2bd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f626c101d40 tx=0x7f625c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:06.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.296+0000 7f6253fff700 1 -- 172.21.15.2:0/3070575562 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6264095a20 con 0x7f626c102880 2023-09-13T04:38:06.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:06 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1253590282' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-09-13T04:38:06.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:06 smithi149 ceph-mon[133315]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-13T04:38:06.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:06 smithi149 ceph-mon[133315]: pgmap v279: 129 pgs: 108 active+clean, 2 creating+peering, 19 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 1 op/s 2023-09-13T04:38:06.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:06 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/224518323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:38:06.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:06 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/301044677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:06.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:06 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1253590282' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-09-13T04:38:06.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:06 smithi156 ceph-mon[133675]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-13T04:38:06.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:06 smithi156 ceph-mon[133675]: pgmap v279: 129 pgs: 108 active+clean, 2 creating+peering, 19 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 1 op/s 2023-09-13T04:38:06.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:06 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/224518323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:38:06.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:06 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/301044677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:06.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:06 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1253590282' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-09-13T04:38:06.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:06 smithi002 ceph-mon[97651]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-13T04:38:06.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:06 smithi002 ceph-mon[97651]: pgmap v279: 129 pgs: 108 active+clean, 2 creating+peering, 19 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 1 op/s 2023-09-13T04:38:06.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:06 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/224518323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:38:06.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:06 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/301044677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:06.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.512+0000 7f62711ff700 1 -- 172.21.15.2:0/3070575562 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f626c10c550 con 0x7f626c102880 2023-09-13T04:38:06.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.514+0000 7f6253fff700 1 -- 172.21.15.2:0/3070575562 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23927 (secure 0 0 0) 0x7f6264099040 con 0x7f626c102880 2023-09-13T04:38:06.515 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:38:06.515 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":88,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:38:04.132627+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T04:37:57.094802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T04:37:58.564487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-13T04:38:02.096895+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:38:06.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.516+0000 7f62711ff700 1 -- 172.21.15.2:0/3070575562 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f62540a0710 msgr2=0x7f62540a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:06.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.516+0000 7f62711ff700 1 --2- 172.21.15.2:0/3070575562 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f62540a0710 0x7f62540a2bd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f626c101d40 tx=0x7f625c009260 comp rx=0 tx=0).stop 2023-09-13T04:38:06.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.516+0000 7f62711ff700 1 -- 172.21.15.2:0/3070575562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f626c102880 msgr2=0x7f626c1a01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:06.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.516+0000 7f62711ff700 1 --2- 172.21.15.2:0/3070575562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f626c102880 0x7f626c1a01a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f626400efc0 tx=0x7f626400c3a0 comp rx=0 tx=0).stop 2023-09-13T04:38:06.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.516+0000 7f62711ff700 1 -- 172.21.15.2:0/3070575562 shutdown_connections 2023-09-13T04:38:06.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.516+0000 7f62711ff700 1 --2- 172.21.15.2:0/3070575562 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f62540a0710 0x7f62540a2bd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:06.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.516+0000 7f62711ff700 1 --2- 172.21.15.2:0/3070575562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f626c102880 0x7f626c1a01a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:06.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.516+0000 7f62711ff700 1 --2- 172.21.15.2:0/3070575562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f626c100cf0 0x7f626c198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:06.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.516+0000 7f62711ff700 1 --2- 172.21.15.2:0/3070575562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f626c101ee0 0x7f626c199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:06.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.516+0000 7f62711ff700 1 -- 172.21.15.2:0/3070575562 >> 172.21.15.2:0/3070575562 conn(0x7f626c0fc220 msgr2=0x7f626c1077b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:06.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.516+0000 7f62711ff700 1 -- 172.21.15.2:0/3070575562 shutdown_connections 2023-09-13T04:38:06.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.516+0000 7f62711ff700 1 -- 172.21.15.2:0/3070575562 wait complete. 2023-09-13T04:38:06.551 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:38:06.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.756+0000 7faf881ff700 1 -- 172.21.15.2:0/2401656956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf80102880 msgr2=0x7faf80109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:06.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.756+0000 7faf881ff700 1 --2- 172.21.15.2:0/2401656956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf80102880 0x7faf80109130 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faf7800b0d0 tx=0x7faf7800b3e0 comp rx=0 tx=0).stop 2023-09-13T04:38:06.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.756+0000 7faf881ff700 1 -- 172.21.15.2:0/2401656956 shutdown_connections 2023-09-13T04:38:06.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.756+0000 7faf881ff700 1 --2- 172.21.15.2:0/2401656956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf80102880 0x7faf80109130 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:06.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.756+0000 7faf881ff700 1 --2- 172.21.15.2:0/2401656956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf80100cf0 0x7faf80101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:06.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.756+0000 7faf881ff700 1 --2- 172.21.15.2:0/2401656956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf80101ee0 0x7faf80102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:06.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.756+0000 7faf881ff700 1 -- 172.21.15.2:0/2401656956 >> 172.21.15.2:0/2401656956 conn(0x7faf800fc220 msgr2=0x7faf800fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:06.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.756+0000 7faf881ff700 1 -- 172.21.15.2:0/2401656956 shutdown_connections 2023-09-13T04:38:06.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.756+0000 7faf881ff700 1 -- 172.21.15.2:0/2401656956 wait complete. 2023-09-13T04:38:06.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.757+0000 7faf881ff700 1 Processor -- start 2023-09-13T04:38:06.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.758+0000 7faf881ff700 1 -- start start 2023-09-13T04:38:06.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.758+0000 7faf881ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf80100cf0 0x7faf80198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:06.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.758+0000 7faf881ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf80101ee0 0x7faf80199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:06.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.758+0000 7faf881ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf80102880 0x7faf801a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:06.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.758+0000 7faf881ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf801a0830 con 0x7faf80101ee0 2023-09-13T04:38:06.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.758+0000 7faf881ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf801a09a0 con 0x7faf80100cf0 2023-09-13T04:38:06.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.758+0000 7faf881ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf801a0b60 con 0x7faf80102880 2023-09-13T04:38:06.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.758+0000 7faf8579a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf80101ee0 0x7faf80199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:06.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.758+0000 7faf8579a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf80101ee0 0x7faf80199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46378/0 (socket says 172.21.15.2:46378) 2023-09-13T04:38:06.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.758+0000 7faf8579a700 1 -- 172.21.15.2:0/1612943796 learned_addr learned my addr 172.21.15.2:0/1612943796 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:06.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.758+0000 7faf85f9b700 1 --2- 172.21.15.2:0/1612943796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf80100cf0 0x7faf80198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:06.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.758+0000 7faf8679c700 1 --2- 172.21.15.2:0/1612943796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf80102880 0x7faf801a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:06.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.759+0000 7faf8579a700 1 -- 172.21.15.2:0/1612943796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf80102880 msgr2=0x7faf801a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:06.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.759+0000 7faf8579a700 1 --2- 172.21.15.2:0/1612943796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf80102880 0x7faf801a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:06.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.759+0000 7faf8579a700 1 -- 172.21.15.2:0/1612943796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf80100cf0 msgr2=0x7faf80198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:06.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.759+0000 7faf8579a700 1 --2- 172.21.15.2:0/1612943796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf80100cf0 0x7faf80198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:06.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.759+0000 7faf8579a700 1 -- 172.21.15.2:0/1612943796 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf78009d00 con 0x7faf80101ee0 2023-09-13T04:38:06.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.759+0000 7faf8579a700 1 --2- 172.21.15.2:0/1612943796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf80101ee0 0x7faf80199200 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faf7000e9a0 tx=0x7faf7000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:06.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.759+0000 7faf76ffd700 1 -- 172.21.15.2:0/1612943796 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70009790 con 0x7faf80101ee0 2023-09-13T04:38:06.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.759+0000 7faf76ffd700 1 -- 172.21.15.2:0/1612943796 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf7000cbd0 con 0x7faf80101ee0 2023-09-13T04:38:06.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.759+0000 7faf881ff700 1 -- 172.21.15.2:0/1612943796 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf801a0e40 con 0x7faf80101ee0 2023-09-13T04:38:06.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.759+0000 7faf76ffd700 1 -- 172.21.15.2:0/1612943796 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70018680 con 0x7faf80101ee0 2023-09-13T04:38:06.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.759+0000 7faf881ff700 1 -- 172.21.15.2:0/1612943796 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf801a1310 con 0x7faf80101ee0 2023-09-13T04:38:06.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.760+0000 7faf881ff700 1 -- 172.21.15.2:0/1612943796 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf8010c560 con 0x7faf80101ee0 2023-09-13T04:38:06.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.762+0000 7faf76ffd700 1 -- 172.21.15.2:0/1612943796 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7faf7000cd40 con 0x7faf80101ee0 2023-09-13T04:38:06.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.764+0000 7faf76ffd700 1 --2- 172.21.15.2:0/1612943796 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7faf6c0a0710 0x7faf6c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:06.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.764+0000 7faf76ffd700 1 -- 172.21.15.2:0/1612943796 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7faf70014070 con 0x7faf80101ee0 2023-09-13T04:38:06.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.764+0000 7faf85f9b700 1 --2- 172.21.15.2:0/1612943796 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7faf6c0a0710 0x7faf6c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:06.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.764+0000 7faf85f9b700 1 --2- 172.21.15.2:0/1612943796 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7faf6c0a0710 0x7faf6c0a2bd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faf80199b90 tx=0x7faf7c00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:06.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:06.766+0000 7faf76ffd700 1 -- 172.21.15.2:0/1612943796 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7faf70095be0 con 0x7faf80101ee0 2023-09-13T04:38:07.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.024+0000 7faf881ff700 1 -- 172.21.15.2:0/1612943796 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf8010c750 con 0x7faf80101ee0 2023-09-13T04:38:07.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.025+0000 7faf76ffd700 1 -- 172.21.15.2:0/1612943796 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11243 (secure 0 0 0) 0x7faf70099200 con 0x7faf80101ee0 2023-09-13T04:38:07.026 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:38:07.026 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2469307307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2469307307},{"type":"v1","addr":"172.21.15.2:6839","nonce":2469307307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/846579077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":846579077},{"type":"v1","addr":"172.21.15.156:6835","nonce":846579077}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1896833690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1896833690},{"type":"v1","addr":"172.21.15.149:6837","nonce":1896833690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/524563090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":524563090},{"type":"v1","addr":"172.21.15.156:6837","nonce":524563090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/1785833227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1785833227},{"type":"v1","addr":"172.21.15.156:6839","nonce":1785833227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/482545391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":482545391},{"type":"v1","addr":"172.21.15.2:6841","nonce":482545391}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-13T04:37:59.110701+0000","modified":"2023-09-13T04:38:04.132862+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24566},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/389167841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":389167841},{"type":"v1","addr":"172.21.15.149:6839","nonce":389167841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-09-13T04:38:07.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.027+0000 7faf881ff700 1 -- 172.21.15.2:0/1612943796 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7faf6c0a0710 msgr2=0x7faf6c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:07.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.027+0000 7faf881ff700 1 --2- 172.21.15.2:0/1612943796 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7faf6c0a0710 0x7faf6c0a2bd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faf80199b90 tx=0x7faf7c00b410 comp rx=0 tx=0).stop 2023-09-13T04:38:07.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.027+0000 7faf881ff700 1 -- 172.21.15.2:0/1612943796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf80101ee0 msgr2=0x7faf80199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:07.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.027+0000 7faf881ff700 1 --2- 172.21.15.2:0/1612943796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf80101ee0 0x7faf80199200 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faf7000e9a0 tx=0x7faf7000ecb0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.027+0000 7faf881ff700 1 -- 172.21.15.2:0/1612943796 shutdown_connections 2023-09-13T04:38:07.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.027+0000 7faf881ff700 1 --2- 172.21.15.2:0/1612943796 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7faf6c0a0710 0x7faf6c0a2bd0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.027+0000 7faf881ff700 1 --2- 172.21.15.2:0/1612943796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf80102880 0x7faf801a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.027+0000 7faf881ff700 1 --2- 172.21.15.2:0/1612943796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf80100cf0 0x7faf80198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.027+0000 7faf881ff700 1 --2- 172.21.15.2:0/1612943796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf80101ee0 0x7faf80199200 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.027+0000 7faf881ff700 1 -- 172.21.15.2:0/1612943796 >> 172.21.15.2:0/1612943796 conn(0x7faf800fc220 msgr2=0x7faf80107740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:07.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.028+0000 7faf881ff700 1 -- 172.21.15.2:0/1612943796 shutdown_connections 2023-09-13T04:38:07.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.028+0000 7faf881ff700 1 -- 172.21.15.2:0/1612943796 wait complete. 2023-09-13T04:38:07.031 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2023-09-13T04:38:07.047 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T04:38:07.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.254+0000 7fd0523ff700 1 -- 172.21.15.2:0/3740842895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd04c108bb0 msgr2=0x7fd04c10b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:07.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.254+0000 7fd0523ff700 1 --2- 172.21.15.2:0/3740842895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd04c108bb0 0x7fd04c10b020 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd04400b0a0 tx=0x7fd04400b3b0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.254+0000 7fd0523ff700 1 -- 172.21.15.2:0/3740842895 shutdown_connections 2023-09-13T04:38:07.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.254+0000 7fd0523ff700 1 --2- 172.21.15.2:0/3740842895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd04c108bb0 0x7fd04c10b020 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.254+0000 7fd0523ff700 1 --2- 172.21.15.2:0/3740842895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd04c074c90 0x7fd04c0730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.254+0000 7fd0523ff700 1 --2- 172.21.15.2:0/3740842895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd04c0736b0 0x7fd04c073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.254+0000 7fd0523ff700 1 -- 172.21.15.2:0/3740842895 >> 172.21.15.2:0/3740842895 conn(0x7fd04c0fc230 msgr2=0x7fd04c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:07.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.254+0000 7fd0523ff700 1 -- 172.21.15.2:0/3740842895 shutdown_connections 2023-09-13T04:38:07.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.254+0000 7fd0523ff700 1 -- 172.21.15.2:0/3740842895 wait complete. 2023-09-13T04:38:07.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.255+0000 7fd0523ff700 1 Processor -- start 2023-09-13T04:38:07.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.255+0000 7fd0523ff700 1 -- start start 2023-09-13T04:38:07.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.255+0000 7fd0523ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd04c0736b0 0x7fd04c198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:07.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.255+0000 7fd0523ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd04c074c90 0x7fd04c1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:07.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.255+0000 7fd0523ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd04c108bb0 0x7fd04c1a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:07.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.255+0000 7fd0523ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c1a0810 con 0x7fd04c108bb0 2023-09-13T04:38:07.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.255+0000 7fd0523ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c1a0980 con 0x7fd04c0736b0 2023-09-13T04:38:07.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.256+0000 7fd0523ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c1a0b40 con 0x7fd04c074c90 2023-09-13T04:38:07.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.256+0000 7fd05099c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd04c108bb0 0x7fd04c1a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:07.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.256+0000 7fd04bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd04c0736b0 0x7fd04c198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:07.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.256+0000 7fd05099c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd04c108bb0 0x7fd04c1a0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46392/0 (socket says 172.21.15.2:46392) 2023-09-13T04:38:07.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.256+0000 7fd05099c700 1 -- 172.21.15.2:0/3582515109 learned_addr learned my addr 172.21.15.2:0/3582515109 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:07.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.256+0000 7fd04b7fe700 1 --2- 172.21.15.2:0/3582515109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd04c074c90 0x7fd04c1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:07.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.256+0000 7fd05099c700 1 -- 172.21.15.2:0/3582515109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd04c074c90 msgr2=0x7fd04c1991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:07.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.256+0000 7fd05099c700 1 --2- 172.21.15.2:0/3582515109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd04c074c90 0x7fd04c1991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.256+0000 7fd05099c700 1 -- 172.21.15.2:0/3582515109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd04c0736b0 msgr2=0x7fd04c198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:07.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.256+0000 7fd05099c700 1 --2- 172.21.15.2:0/3582515109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd04c0736b0 0x7fd04c198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.256+0000 7fd05099c700 1 -- 172.21.15.2:0/3582515109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03c009520 con 0x7fd04c108bb0 2023-09-13T04:38:07.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.256+0000 7fd05099c700 1 --2- 172.21.15.2:0/3582515109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd04c108bb0 0x7fd04c1a0240 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd044009fd0 tx=0x7fd04400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:07.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.257+0000 7fd0497fa700 1 -- 172.21.15.2:0/3582515109 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04400e040 con 0x7fd04c108bb0 2023-09-13T04:38:07.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.257+0000 7fd0523ff700 1 -- 172.21.15.2:0/3582515109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd044009d00 con 0x7fd04c108bb0 2023-09-13T04:38:07.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.257+0000 7fd0497fa700 1 -- 172.21.15.2:0/3582515109 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0440094a0 con 0x7fd04c108bb0 2023-09-13T04:38:07.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.257+0000 7fd0497fa700 1 -- 172.21.15.2:0/3582515109 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04401b6c0 con 0x7fd04c108bb0 2023-09-13T04:38:07.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.257+0000 7fd0523ff700 1 -- 172.21.15.2:0/3582515109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04c1a10b0 con 0x7fd04c108bb0 2023-09-13T04:38:07.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.258+0000 7fd0523ff700 1 -- 172.21.15.2:0/3582515109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd04c1072b0 con 0x7fd04c108bb0 2023-09-13T04:38:07.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.260+0000 7fd0497fa700 1 -- 172.21.15.2:0/3582515109 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fd04401b820 con 0x7fd04c108bb0 2023-09-13T04:38:07.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.261+0000 7fd0497fa700 1 --2- 172.21.15.2:0/3582515109 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd0340a0690 0x7fd0340a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:07.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.262+0000 7fd0497fa700 1 -- 172.21.15.2:0/3582515109 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd0440cd380 con 0x7fd04c108bb0 2023-09-13T04:38:07.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.262+0000 7fd04bfff700 1 --2- 172.21.15.2:0/3582515109 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd0340a0690 0x7fd0340a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:07.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.262+0000 7fd04bfff700 1 --2- 172.21.15.2:0/3582515109 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd0340a0690 0x7fd0340a2b50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd04c199b70 tx=0x7fd03c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:07.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.263+0000 7fd0497fa700 1 -- 172.21.15.2:0/3582515109 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd044096cf0 con 0x7fd04c108bb0 2023-09-13T04:38:07.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:07 smithi149 ceph-mon[133315]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-09-13T04:38:07.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:07 smithi149 ceph-mon[133315]: Cluster is now healthy 2023-09-13T04:38:07.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:07 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3070575562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:07.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:07 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1612943796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:38:07.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:07 smithi156 ceph-mon[133675]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-09-13T04:38:07.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:07 smithi156 ceph-mon[133675]: Cluster is now healthy 2023-09-13T04:38:07.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:07 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3070575562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:07.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:07 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1612943796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:38:07.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.478+0000 7fd0523ff700 1 -- 172.21.15.2:0/3582515109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd04c10e960 con 0x7fd04c108bb0 2023-09-13T04:38:07.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.480+0000 7fd0497fa700 1 -- 172.21.15.2:0/3582515109 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23927 (secure 0 0 0) 0x7fd04409a310 con 0x7fd04c108bb0 2023-09-13T04:38:07.480 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:38:07.481 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":88,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:38:04.132627+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T04:37:57.094802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T04:37:58.564487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-13T04:38:02.096895+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:38:07.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.482+0000 7fd0523ff700 1 -- 172.21.15.2:0/3582515109 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd0340a0690 msgr2=0x7fd0340a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:07.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.482+0000 7fd0523ff700 1 --2- 172.21.15.2:0/3582515109 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd0340a0690 0x7fd0340a2b50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd04c199b70 tx=0x7fd03c009260 comp rx=0 tx=0).stop 2023-09-13T04:38:07.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.482+0000 7fd0523ff700 1 -- 172.21.15.2:0/3582515109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd04c108bb0 msgr2=0x7fd04c1a0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:07.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.482+0000 7fd0523ff700 1 --2- 172.21.15.2:0/3582515109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd04c108bb0 0x7fd04c1a0240 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd044009fd0 tx=0x7fd04400bce0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.483+0000 7fd0523ff700 1 -- 172.21.15.2:0/3582515109 shutdown_connections 2023-09-13T04:38:07.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.483+0000 7fd0523ff700 1 --2- 172.21.15.2:0/3582515109 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd0340a0690 0x7fd0340a2b50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.483+0000 7fd0523ff700 1 --2- 172.21.15.2:0/3582515109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd04c108bb0 0x7fd04c1a0240 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.483+0000 7fd0523ff700 1 --2- 172.21.15.2:0/3582515109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd04c0736b0 0x7fd04c198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.483+0000 7fd0523ff700 1 --2- 172.21.15.2:0/3582515109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd04c074c90 0x7fd04c1991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.483+0000 7fd0523ff700 1 -- 172.21.15.2:0/3582515109 >> 172.21.15.2:0/3582515109 conn(0x7fd04c0fc230 msgr2=0x7fd04c0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:07.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.483+0000 7fd0523ff700 1 -- 172.21.15.2:0/3582515109 shutdown_connections 2023-09-13T04:38:07.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.483+0000 7fd0523ff700 1 -- 172.21.15.2:0/3582515109 wait complete. 2023-09-13T04:38:07.499 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:07 smithi002 ceph-mon[97651]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-09-13T04:38:07.500 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:07 smithi002 ceph-mon[97651]: Cluster is now healthy 2023-09-13T04:38:07.500 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:07 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3070575562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:07.500 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:07 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1612943796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:38:07.502 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-13T04:37:59.110701+0000', 'modified': '2023-09-13T04:38:04.132862+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24566}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24566': {'gid': 24566, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.149:6839/389167841', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 389167841}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 389167841}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-09-13T04:38:07.503 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-13T04:38:07.503 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-13T04:38:07.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.709+0000 7f4edc3ff700 1 -- 172.21.15.2:0/743126139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ed4100ce0 msgr2=0x7f4ed4101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:07.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.709+0000 7f4edc3ff700 1 --2- 172.21.15.2:0/743126139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ed4100ce0 0x7f4ed4101100 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00099c0 tx=0x7f4ed0009cd0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.709+0000 7f4edc3ff700 1 -- 172.21.15.2:0/743126139 shutdown_connections 2023-09-13T04:38:07.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.709+0000 7f4edc3ff700 1 --2- 172.21.15.2:0/743126139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ed4102870 0x7f4ed4109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.709+0000 7f4edc3ff700 1 --2- 172.21.15.2:0/743126139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ed4100ce0 0x7f4ed4101100 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.709+0000 7f4edc3ff700 1 --2- 172.21.15.2:0/743126139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ed4101ed0 0x7f4ed4102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.709+0000 7f4edc3ff700 1 -- 172.21.15.2:0/743126139 >> 172.21.15.2:0/743126139 conn(0x7f4ed40fc230 msgr2=0x7f4ed40fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:07.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.709+0000 7f4edc3ff700 1 -- 172.21.15.2:0/743126139 shutdown_connections 2023-09-13T04:38:07.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.709+0000 7f4edc3ff700 1 -- 172.21.15.2:0/743126139 wait complete. 2023-09-13T04:38:07.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.710+0000 7f4edc3ff700 1 Processor -- start 2023-09-13T04:38:07.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.710+0000 7f4edc3ff700 1 -- start start 2023-09-13T04:38:07.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.710+0000 7f4edc3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ed4100ce0 0x7f4ed4198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:07.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.711+0000 7f4edc3ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ed4101ed0 0x7f4ed4199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:07.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.711+0000 7f4edc3ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ed4102870 0x7f4ed41a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:07.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.711+0000 7f4edc3ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed41a0770 con 0x7f4ed4101ed0 2023-09-13T04:38:07.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.711+0000 7f4edc3ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed41a08e0 con 0x7f4ed4102870 2023-09-13T04:38:07.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.711+0000 7f4edc3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed41a0aa0 con 0x7f4ed4100ce0 2023-09-13T04:38:07.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.711+0000 7f4ed999a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ed4101ed0 0x7f4ed4199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:07.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.711+0000 7f4eda99c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ed4102870 0x7f4ed41a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:07.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.711+0000 7f4eda19b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ed4100ce0 0x7f4ed4198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:07.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.711+0000 7f4ed999a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ed4101ed0 0x7f4ed4199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46402/0 (socket says 172.21.15.2:46402) 2023-09-13T04:38:07.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.711+0000 7f4ed999a700 1 -- 172.21.15.2:0/240886550 learned_addr learned my addr 172.21.15.2:0/240886550 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:07.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.711+0000 7f4ed999a700 1 -- 172.21.15.2:0/240886550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ed4100ce0 msgr2=0x7f4ed4198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:07.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.711+0000 7f4ed999a700 1 --2- 172.21.15.2:0/240886550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ed4100ce0 0x7f4ed4198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.711+0000 7f4ed999a700 1 -- 172.21.15.2:0/240886550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ed4102870 msgr2=0x7f4ed41a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:07.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.711+0000 7f4ed999a700 1 --2- 172.21.15.2:0/240886550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ed4102870 0x7f4ed41a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.711+0000 7f4ed999a700 1 -- 172.21.15.2:0/240886550 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed00095f0 con 0x7f4ed4101ed0 2023-09-13T04:38:07.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.712+0000 7f4ed999a700 1 --2- 172.21.15.2:0/240886550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ed4101ed0 0x7f4ed4199170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400e9a0 tx=0x7f4ec400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:07.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.712+0000 7f4ecb7fe700 1 -- 172.21.15.2:0/240886550 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4009790 con 0x7f4ed4101ed0 2023-09-13T04:38:07.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.712+0000 7f4ecb7fe700 1 -- 172.21.15.2:0/240886550 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ec4010430 con 0x7f4ed4101ed0 2023-09-13T04:38:07.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.712+0000 7f4edc3ff700 1 -- 172.21.15.2:0/240886550 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed41a0d80 con 0x7f4ed4101ed0 2023-09-13T04:38:07.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.712+0000 7f4ecb7fe700 1 -- 172.21.15.2:0/240886550 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4018680 con 0x7f4ed4101ed0 2023-09-13T04:38:07.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.712+0000 7f4edc3ff700 1 -- 172.21.15.2:0/240886550 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed41a11d0 con 0x7f4ed4101ed0 2023-09-13T04:38:07.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.715+0000 7f4ecb7fe700 1 -- 172.21.15.2:0/240886550 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f4ec40105a0 con 0x7f4ed4101ed0 2023-09-13T04:38:07.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.717+0000 7f4eefcdeb80 1 Processor -- start 2023-09-13T04:38:07.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.718+0000 7f4ecb7fe700 1 --2- 172.21.15.2:0/240886550 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4ec00a0640 0x7f4ec00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:07.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.718+0000 7f4eefcdeb80 1 -- start start 2023-09-13T04:38:07.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.718+0000 7f4ecb7fe700 1 -- 172.21.15.2:0/240886550 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4ec4014070 con 0x7f4ed4101ed0 2023-09-13T04:38:07.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.718+0000 7f4eda99c700 1 --2- 172.21.15.2:0/240886550 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4ec00a0640 0x7f4ec00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:07.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.718+0000 7f4eefcdeb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42576b0 0x5567c4257ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:07.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.718+0000 7f4eefcdeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567c4258010 0x5567c4256470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:07.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.718+0000 7f4eefcdeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567c42569b0 0x5567c4256e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:07.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.718+0000 7f4eefcdeb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567c41af730 con 0x5567c42576b0 2023-09-13T04:38:07.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.718+0000 7f4eefcdeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567c41ad050 con 0x5567c4258010 2023-09-13T04:38:07.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.718+0000 7f4eefcdeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567c41a8940 con 0x5567c42569b0 2023-09-13T04:38:07.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.719+0000 7f4eda19b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42576b0 0x5567c4257ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:07.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.719+0000 7f4ed999a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567c4258010 0x5567c4256470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:07.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.719+0000 7f4eda19b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42576b0 0x5567c4257ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46406/0 (socket says 172.21.15.2:46406) 2023-09-13T04:38:07.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.719+0000 7f4eda19b700 1 -- 172.21.15.2:0/2470487800 learned_addr learned my addr 172.21.15.2:0/2470487800 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:07.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.719+0000 7f4eda99c700 1 --2- 172.21.15.2:0/2470487800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567c42569b0 0x5567c4256e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:07.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.719+0000 7f4eda99c700 1 --2- 172.21.15.2:0/240886550 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4ec00a0640 0x7f4ec00a2b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00f590 tx=0x7f4ecc00b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:07.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.719+0000 7f4ed999a700 1 -- 172.21.15.2:0/2470487800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567c42569b0 msgr2=0x5567c4256e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:38:07.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.719+0000 7f4ed999a700 1 --2- 172.21.15.2:0/2470487800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567c42569b0 0x5567c4256e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.719+0000 7f4ed999a700 1 -- 172.21.15.2:0/2470487800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42576b0 msgr2=0x5567c4257ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:38:07.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.719+0000 7f4ed999a700 1 --2- 172.21.15.2:0/2470487800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42576b0 0x5567c4257ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.719+0000 7f4ed999a700 1 -- 172.21.15.2:0/2470487800 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567c40285e0 con 0x5567c4258010 2023-09-13T04:38:07.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.719+0000 7f4ed999a700 1 --2- 172.21.15.2:0/2470487800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567c4258010 0x5567c4256470 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400c2d0 tx=0x7f4ec4098e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb9e0bf29bd0a17a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:07.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.719+0000 7f4ec8bfe700 1 -- 172.21.15.2:0/2470487800 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec409d120 con 0x5567c4258010 2023-09-13T04:38:07.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.719+0000 7f4ec8bfe700 1 -- 172.21.15.2:0/2470487800 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ec4099510 con 0x5567c4258010 2023-09-13T04:38:07.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.719+0000 7f4ec8bfe700 1 -- 172.21.15.2:0/2470487800 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4098a00 con 0x5567c4258010 2023-09-13T04:38:07.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.720+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/2470487800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567c4258010 msgr2=0x5567c4256470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:07.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.720+0000 7f4eefcdeb80 1 --2- 172.21.15.2:0/2470487800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567c4258010 0x5567c4256470 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400c2d0 tx=0x7f4ec4098e70 comp rx=0 tx=0).stop 2023-09-13T04:38:07.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.720+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/2470487800 shutdown_connections 2023-09-13T04:38:07.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.720+0000 7f4eefcdeb80 1 --2- 172.21.15.2:0/2470487800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567c42569b0 0x5567c4256e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.720+0000 7f4eefcdeb80 1 --2- 172.21.15.2:0/2470487800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42576b0 0x5567c4257ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.720+0000 7f4eefcdeb80 1 --2- 172.21.15.2:0/2470487800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567c4258010 0x5567c4256470 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.720+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/2470487800 >> 172.21.15.2:0/2470487800 conn(0x5567c41b4c30 msgr2=0x5567c4250ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:07.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.720+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/2470487800 shutdown_connections 2023-09-13T04:38:07.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.720+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/2470487800 wait complete. 2023-09-13T04:38:07.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.721+0000 7f4eefcdeb80 1 -- start start 2023-09-13T04:38:07.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.721+0000 7f4eefcdeb80 1 Processor -- start 2023-09-13T04:38:07.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.722+0000 7f4eefcdeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567c42569b0 0x5567c425ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:07.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.722+0000 7f4eefcdeb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42576b0 0x5567c42604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:07.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.722+0000 7f4eefcdeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567c4258010 0x5567c42674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:07.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.722+0000 7f4eefcdeb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567c4146b60 con 0x5567c42576b0 2023-09-13T04:38:07.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.722+0000 7f4eefcdeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567c4146860 con 0x5567c42569b0 2023-09-13T04:38:07.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.722+0000 7f4eefcdeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567c402a8d0 con 0x5567c4258010 2023-09-13T04:38:07.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.722+0000 7f4eefcdeb80 10 client.0 _ll_register_callbacks cb 0x5567c4144630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:38:07.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.722+0000 7f4eda19b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567c42569b0 0x5567c425ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:07.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.722+0000 7f4ed999a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42576b0 0x5567c42604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:07.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.722+0000 7f4eda19b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567c42569b0 0x5567c425ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:41874/0 (socket says 172.21.15.2:41874) 2023-09-13T04:38:07.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.722+0000 7f4eda19b700 1 -- 172.21.15.2:0/2578984325 learned_addr learned my addr 172.21.15.2:0/2578984325 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:07.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.722+0000 7f4ed999a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42576b0 0x5567c42604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46422/0 (socket says 172.21.15.2:46422) 2023-09-13T04:38:07.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.722+0000 7f4eda99c700 1 --2- 172.21.15.2:0/2578984325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567c4258010 0x5567c42674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:07.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.722+0000 7f4ed999a700 1 -- 172.21.15.2:0/2578984325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567c4258010 msgr2=0x5567c42674f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:38:07.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.722+0000 7f4ed999a700 1 --2- 172.21.15.2:0/2578984325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567c4258010 0x5567c42674f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.722+0000 7f4ed999a700 1 -- 172.21.15.2:0/2578984325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567c42569b0 msgr2=0x5567c425ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:38:07.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.722+0000 7f4ed999a700 1 --2- 172.21.15.2:0/2578984325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567c42569b0 0x5567c425ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.722+0000 7f4ed999a700 1 -- 172.21.15.2:0/2578984325 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed41a11d0 con 0x5567c42576b0 2023-09-13T04:38:07.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.722+0000 7f4eda99c700 1 --2- 172.21.15.2:0/2578984325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567c4258010 0x5567c42674f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:38:07.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.722+0000 7f4eda19b700 1 --2- 172.21.15.2:0/2578984325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567c42569b0 0x5567c425ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:38:07.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.723+0000 7f4ed999a700 1 --2- 172.21.15.2:0/2578984325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42576b0 0x5567c42604b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x5567c425e720 tx=0x7f4ec4010b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5696033acb057f78 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:07.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.723+0000 7f4ebdffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:38:07.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.723+0000 7f4ebdffb700 1 -- 172.21.15.2:0/2578984325 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec409d340 con 0x5567c42576b0 2023-09-13T04:38:07.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.723+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/2578984325 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567c4249620 con 0x5567c42576b0 2023-09-13T04:38:07.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.723+0000 7f4ebdffb700 1 -- 172.21.15.2:0/2578984325 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4ec40a1580 con 0x5567c42576b0 2023-09-13T04:38:07.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.723+0000 7f4ebdffb700 1 -- 172.21.15.2:0/2578984325 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ec40b73e0 con 0x5567c42576b0 2023-09-13T04:38:07.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.723+0000 7f4eefcdeb80 10 client.14643 fetch_fsmap learned FSMap version 17 2023-09-13T04:38:07.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.723+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/2578984325 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x5567c41c0750 con 0x5567c42576b0 2023-09-13T04:38:07.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.723+0000 7f4ebdffb700 1 -- 172.21.15.2:0/2578984325 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5567c4249620 con 0x5567c42576b0 2023-09-13T04:38:07.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.723+0000 7f4ebdffb700 1 -- 172.21.15.2:0/2578984325 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f4ec40650e0 con 0x5567c42576b0 2023-09-13T04:38:07.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.723+0000 7f4eefcdeb80 10 client.14643 fetch_fsmap finished waiting for FSMap version 17 2023-09-13T04:38:07.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.723+0000 7f4eefcdeb80 10 client.14643 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-13T04:38:07.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.723+0000 7f4eefcdeb80 20 client.14643 populate_metadata read hostname 'smithi002' 2023-09-13T04:38:07.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.723+0000 7f4eefcdeb80 1 --2- 172.21.15.2:0/2578984325 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5567c426b780 0x5567c426a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:07.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.724+0000 7f4eefcdeb80 4 client.14643 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:38:07.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.724+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/2578984325 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567c41a1920 con 0x5567c426b780 2023-09-13T04:38:07.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.724+0000 7f4eda19b700 1 --2- 172.21.15.2:0/2578984325 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5567c426b780 0x5567c426a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:07.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.724+0000 7f4eda19b700 1 --2- 172.21.15.2:0/2578984325 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5567c426b780 0x5567c426a420 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54ab50eea3f0cde3 server_cookie=574a1390a36e3b0d in_seq=0 out_seq=0 2023-09-13T04:38:07.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.724+0000 7f4ebdffb700 10 client.14643 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:38:07.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.725+0000 7f4ebdffb700 1 -- 172.21.15.2:0/2578984325 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5567c41a1920 con 0x5567c426b780 2023-09-13T04:38:07.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.725+0000 7f4ebdffb700 10 client.14643 handle_command_reply: tid=0 2023-09-13T04:38:07.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.725+0000 7f4eefcdeb80 1 client.14643 shutdown 2023-09-13T04:38:07.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.726+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/2578984325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42576b0 msgr2=0x5567c42604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:07.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.726+0000 7f4eefcdeb80 1 --2- 172.21.15.2:0/2578984325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42576b0 0x5567c42604b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x5567c425e720 tx=0x7f4ec4010b80 comp rx=0 tx=0).stop 2023-09-13T04:38:07.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.726+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/2578984325 shutdown_connections 2023-09-13T04:38:07.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.726+0000 7f4eefcdeb80 1 --2- 172.21.15.2:0/2578984325 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5567c426b780 0x5567c426a420 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.726+0000 7f4eefcdeb80 1 --2- 172.21.15.2:0/2578984325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567c4258010 0x5567c42674f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.726+0000 7f4eefcdeb80 1 --2- 172.21.15.2:0/2578984325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567c42569b0 0x5567c425ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.726+0000 7f4ebdffb700 0 client.14643 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:38:07.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.726+0000 7f4eefcdeb80 1 --2- 172.21.15.2:0/2578984325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42576b0 0x5567c42604b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.726+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/2578984325 >> 172.21.15.2:0/2578984325 conn(0x5567c41b4c30 msgr2=0x5567c424b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:07.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.726+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/2578984325 shutdown_connections 2023-09-13T04:38:07.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.726+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/2578984325 wait complete. 2023-09-13T04:38:07.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.726+0000 7f4eefcdeb80 20 client.14643 trim_cache size 0 max 16384 2023-09-13T04:38:07.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.744+0000 7f4eefcdeb80 1 Processor -- start 2023-09-13T04:38:07.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.744+0000 7f4eefcdeb80 1 -- start start 2023-09-13T04:38:07.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.745+0000 7f4eefcdeb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42569b0 0x5567c4256470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:07.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.745+0000 7f4eefcdeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567c42576b0 0x5567c42674b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:07.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.745+0000 7f4eefcdeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567c4258010 0x5567c425a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:07.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.745+0000 7f4eefcdeb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567c425fb40 con 0x5567c42569b0 2023-09-13T04:38:07.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.745+0000 7f4eefcdeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567c4036de0 con 0x5567c4258010 2023-09-13T04:38:07.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.745+0000 7f4eefcdeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567c4164f00 con 0x5567c42576b0 2023-09-13T04:38:07.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.745+0000 7f4eda19b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42569b0 0x5567c4256470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:07.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.745+0000 7f4eda19b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42569b0 0x5567c4256470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46426/0 (socket says 172.21.15.2:46426) 2023-09-13T04:38:07.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.745+0000 7f4eda19b700 1 -- 172.21.15.2:0/1107901293 learned_addr learned my addr 172.21.15.2:0/1107901293 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:07.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.745+0000 7f4ed999a700 1 --2- 172.21.15.2:0/1107901293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567c42576b0 0x5567c42674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:07.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.745+0000 7f4eda99c700 1 --2- 172.21.15.2:0/1107901293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567c4258010 0x5567c425a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:07.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.745+0000 7f4eda19b700 1 -- 172.21.15.2:0/1107901293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567c42576b0 msgr2=0x5567c42674b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:38:07.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.745+0000 7f4eda19b700 1 --2- 172.21.15.2:0/1107901293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567c42576b0 0x5567c42674b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.745+0000 7f4eda19b700 1 -- 172.21.15.2:0/1107901293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567c4258010 msgr2=0x5567c425a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:38:07.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.745+0000 7f4eda19b700 1 --2- 172.21.15.2:0/1107901293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567c4258010 0x5567c425a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.745+0000 7f4eda19b700 1 -- 172.21.15.2:0/1107901293 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567c4251160 con 0x5567c42569b0 2023-09-13T04:38:07.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.745+0000 7f4eda19b700 1 --2- 172.21.15.2:0/1107901293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42569b0 0x5567c4256470 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000a010 tx=0x7f4ed0005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef10b582a7ba06a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:07.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.746+0000 7f4ec93ff700 1 -- 172.21.15.2:0/1107901293 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0028ce0 con 0x5567c42569b0 2023-09-13T04:38:07.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.746+0000 7f4ec93ff700 1 -- 172.21.15.2:0/1107901293 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ed001e040 con 0x5567c42569b0 2023-09-13T04:38:07.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.746+0000 7f4ec93ff700 1 -- 172.21.15.2:0/1107901293 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed00037d0 con 0x5567c42569b0 2023-09-13T04:38:07.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.746+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/1107901293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42569b0 msgr2=0x5567c4256470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:07.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.746+0000 7f4eefcdeb80 1 --2- 172.21.15.2:0/1107901293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42569b0 0x5567c4256470 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000a010 tx=0x7f4ed0005fb0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.746+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/1107901293 shutdown_connections 2023-09-13T04:38:07.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.746+0000 7f4eefcdeb80 1 --2- 172.21.15.2:0/1107901293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567c4258010 0x5567c425a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.746+0000 7f4eefcdeb80 1 --2- 172.21.15.2:0/1107901293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42569b0 0x5567c4256470 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.746+0000 7f4eefcdeb80 1 --2- 172.21.15.2:0/1107901293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567c42576b0 0x5567c42674b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.746+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/1107901293 >> 172.21.15.2:0/1107901293 conn(0x5567c41b4c30 msgr2=0x5567c425dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:07.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.746+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/1107901293 shutdown_connections 2023-09-13T04:38:07.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.746+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/1107901293 wait complete. 2023-09-13T04:38:07.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.747+0000 7f4eefcdeb80 1 -- start start 2023-09-13T04:38:07.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.747+0000 7f4eefcdeb80 1 Processor -- start 2023-09-13T04:38:07.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.747+0000 7f4eefcdeb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42569b0 0x5567c4261b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:07.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.747+0000 7f4eefcdeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567c42576b0 0x5567c4262050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:07.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.747+0000 7f4eefcdeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567c4258010 0x5567c4266f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:07.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.747+0000 7f4eefcdeb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567c4267440 con 0x5567c42569b0 2023-09-13T04:38:07.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.747+0000 7f4eefcdeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567c42675b0 con 0x5567c42576b0 2023-09-13T04:38:07.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.747+0000 7f4eefcdeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567c4267770 con 0x5567c4258010 2023-09-13T04:38:07.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.747+0000 7f4eefcdeb80 10 client.0 _ll_register_callbacks cb 0x5567c4144630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:38:07.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.748+0000 7f4eda19b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42569b0 0x5567c4261b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:07.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.748+0000 7f4ed999a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567c42576b0 0x5567c4262050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:07.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.748+0000 7f4eda19b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42569b0 0x5567c4261b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46436/0 (socket says 172.21.15.2:46436) 2023-09-13T04:38:07.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.748+0000 7f4eda19b700 1 -- 172.21.15.2:0/4290763743 learned_addr learned my addr 172.21.15.2:0/4290763743 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:07.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.748+0000 7f4eda99c700 1 --2- 172.21.15.2:0/4290763743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567c4258010 0x5567c4266f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:07.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.748+0000 7f4eda19b700 1 -- 172.21.15.2:0/4290763743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567c4258010 msgr2=0x5567c4266f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:38:07.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.748+0000 7f4eda19b700 1 --2- 172.21.15.2:0/4290763743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567c4258010 0x5567c4266f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.748+0000 7f4eda19b700 1 -- 172.21.15.2:0/4290763743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567c42576b0 msgr2=0x5567c4262050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:38:07.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.748+0000 7f4eda19b700 1 --2- 172.21.15.2:0/4290763743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567c42576b0 0x5567c4262050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.748+0000 7f4eda19b700 1 -- 172.21.15.2:0/4290763743 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed4109a70 con 0x5567c42569b0 2023-09-13T04:38:07.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.748+0000 7f4ed999a700 1 --2- 172.21.15.2:0/4290763743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567c42576b0 0x5567c4262050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:38:07.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.748+0000 7f4eda19b700 1 --2- 172.21.15.2:0/4290763743 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42569b0 0x5567c4261b10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00098e0 tx=0x7f4ed0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7de8e52092be37f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:07.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.748+0000 7f4ebdffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:38:07.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.749+0000 7f4ebdffb700 1 -- 172.21.15.2:0/4290763743 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0028ce0 con 0x5567c42569b0 2023-09-13T04:38:07.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.749+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/4290763743 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ec4099510 con 0x5567c42569b0 2023-09-13T04:38:07.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.749+0000 7f4ebdffb700 1 -- 172.21.15.2:0/4290763743 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4ed0028e40 con 0x5567c42569b0 2023-09-13T04:38:07.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.749+0000 7f4ebdffb700 1 -- 172.21.15.2:0/4290763743 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ed0005760 con 0x5567c42569b0 2023-09-13T04:38:07.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.749+0000 7f4eefcdeb80 10 client.14649 fetch_fsmap learned FSMap version 17 2023-09-13T04:38:07.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.749+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/4290763743 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x5567c3fd4d90 con 0x5567c42569b0 2023-09-13T04:38:07.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.749+0000 7f4ebdffb700 1 -- 172.21.15.2:0/4290763743 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ec4099510 con 0x5567c42569b0 2023-09-13T04:38:07.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.749+0000 7f4ebdffb700 1 -- 172.21.15.2:0/4290763743 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f4ed000f460 con 0x5567c42569b0 2023-09-13T04:38:07.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.749+0000 7f4eefcdeb80 10 client.14649 fetch_fsmap finished waiting for FSMap version 17 2023-09-13T04:38:07.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.749+0000 7f4eefcdeb80 10 client.14649 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-13T04:38:07.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.749+0000 7f4eefcdeb80 20 client.14649 populate_metadata read hostname 'smithi002' 2023-09-13T04:38:07.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.749+0000 7f4eefcdeb80 1 --2- 172.21.15.2:0/4290763743 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5567c42c3c60 0x5567c42c0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:07.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.749+0000 7f4eefcdeb80 4 client.14649 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "status"}] 2023-09-13T04:38:07.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.749+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/4290763743 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5567c4162000 con 0x5567c42c3c60 2023-09-13T04:38:07.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.750+0000 7f4ed999a700 1 --2- 172.21.15.2:0/4290763743 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5567c42c3c60 0x5567c42c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:07.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.750+0000 7f4ed999a700 1 --2- 172.21.15.2:0/4290763743 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5567c42c3c60 0x5567c42c0130 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4d14d749f25a71 server_cookie=62fb2c0e242793d2 in_seq=0 out_seq=0 2023-09-13T04:38:07.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.750+0000 7f4ebdffb700 10 client.14649 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:38:07.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.750+0000 7f4ebdffb700 1 -- 172.21.15.2:0/4290763743 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5567c4162000 con 0x5567c42c3c60 2023-09-13T04:38:07.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.750+0000 7f4ebdffb700 10 client.14649 handle_command_reply: tid=0 2023-09-13T04:38:07.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.750+0000 7f4eefcdeb80 1 client.14649 shutdown 2023-09-13T04:38:07.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.751+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/4290763743 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42569b0 msgr2=0x5567c4261b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:07.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.751+0000 7f4eefcdeb80 1 --2- 172.21.15.2:0/4290763743 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42569b0 0x5567c4261b10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00098e0 tx=0x7f4ed0000bc0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.751+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/4290763743 shutdown_connections 2023-09-13T04:38:07.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.751+0000 7f4eefcdeb80 1 --2- 172.21.15.2:0/4290763743 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5567c42c3c60 0x5567c42c0130 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.751+0000 7f4eefcdeb80 1 --2- 172.21.15.2:0/4290763743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567c4258010 0x5567c4266f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.751+0000 7f4eefcdeb80 1 --2- 172.21.15.2:0/4290763743 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567c42569b0 0x5567c4261b10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.751+0000 7f4ebdffb700 0 client.14649 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:38:07.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.751+0000 7f4eefcdeb80 1 --2- 172.21.15.2:0/4290763743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567c42576b0 0x5567c4262050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.751+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/4290763743 >> 172.21.15.2:0/4290763743 conn(0x5567c41b4c30 msgr2=0x5567c425a930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:07.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.751+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/4290763743 shutdown_connections 2023-09-13T04:38:07.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.751+0000 7f4eefcdeb80 1 -- 172.21.15.2:0/4290763743 wait complete. 2023-09-13T04:38:07.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.751+0000 7f4eefcdeb80 20 client.14649 trim_cache size 0 max 16384 2023-09-13T04:38:07.769 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:38:07.769 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da", 2023-09-13T04:38:07.769 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 0, 2023-09-13T04:38:07.769 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24566, 2023-09-13T04:38:07.769 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2023-09-13T04:38:07.770 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2023-09-13T04:38:07.770 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2023-09-13T04:38:07.770 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 8.1759647799999993, 2023-09-13T04:38:07.770 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 17, 2023-09-13T04:38:07.771 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 88, 2023-09-13T04:38:07.771 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 88, 2023-09-13T04:38:07.771 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 14.894481928999999 2023-09-13T04:38:07.771 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:38:07.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.752+0000 7f4edc3ff700 1 -- 172.21.15.2:0/240886550 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4ec00a0640 msgr2=0x7f4ec00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:07.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.752+0000 7f4edc3ff700 1 --2- 172.21.15.2:0/240886550 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4ec00a0640 0x7f4ec00a2b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00f590 tx=0x7f4ecc00b040 comp rx=0 tx=0).stop 2023-09-13T04:38:07.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.752+0000 7f4edc3ff700 1 -- 172.21.15.2:0/240886550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ed4101ed0 msgr2=0x7f4ed4199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:07.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.752+0000 7f4edc3ff700 1 --2- 172.21.15.2:0/240886550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ed4101ed0 0x7f4ed4199170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400e9a0 tx=0x7f4ec400ecb0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.752+0000 7f4edc3ff700 1 -- 172.21.15.2:0/240886550 shutdown_connections 2023-09-13T04:38:07.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.752+0000 7f4edc3ff700 1 --2- 172.21.15.2:0/240886550 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4ec00a0640 0x7f4ec00a2b00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.752+0000 7f4edc3ff700 1 --2- 172.21.15.2:0/240886550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ed4102870 0x7f4ed41a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.752+0000 7f4edc3ff700 1 --2- 172.21.15.2:0/240886550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ed4100ce0 0x7f4ed4198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.752+0000 7f4edc3ff700 1 --2- 172.21.15.2:0/240886550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ed4101ed0 0x7f4ed4199170 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:07.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.752+0000 7f4edc3ff700 1 -- 172.21.15.2:0/240886550 >> 172.21.15.2:0/240886550 conn(0x7f4ed40fc230 msgr2=0x7f4ed40fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:07.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.752+0000 7f4edc3ff700 1 -- 172.21.15.2:0/240886550 shutdown_connections 2023-09-13T04:38:07.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:07.752+0000 7f4edc3ff700 1 -- 172.21.15.2:0/240886550 wait complete. 2023-09-13T04:38:07.775 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-13T04:38:07.776 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-09-13T04:38:07.777 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph fs ls 2023-09-13T04:38:08.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:08 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3582515109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:08.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:08 smithi149 ceph-mon[133315]: pgmap v280: 129 pgs: 127 active+clean, 2 creating+peering; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 1 op/s 2023-09-13T04:38:08.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:08 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3582515109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:08.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:08 smithi156 ceph-mon[133675]: pgmap v280: 129 pgs: 127 active+clean, 2 creating+peering; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 1 op/s 2023-09-13T04:38:08.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:08 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3582515109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:08.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:08 smithi002 ceph-mon[97651]: pgmap v280: 129 pgs: 127 active+clean, 2 creating+peering; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 1 op/s 2023-09-13T04:38:08.506 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:38:10.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.033+0000 7f1ad9c7c700 1 -- 172.21.15.2:0/1510803445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ad4106d90 msgr2=0x7f1ad4107170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:10.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.033+0000 7f1ad9c7c700 1 --2- 172.21.15.2:0/1510803445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ad4106d90 0x7f1ad4107170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40099c0 tx=0x7f1ac4009cd0 comp rx=0 tx=0).stop 2023-09-13T04:38:10.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.033+0000 7f1ad9c7c700 1 -- 172.21.15.2:0/1510803445 shutdown_connections 2023-09-13T04:38:10.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.033+0000 7f1ad9c7c700 1 --2- 172.21.15.2:0/1510803445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ad4101740 0x7f1ad410dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:10.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.033+0000 7f1ad9c7c700 1 --2- 172.21.15.2:0/1510803445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ad4106d90 0x7f1ad4107170 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:10.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.033+0000 7f1ad9c7c700 1 --2- 172.21.15.2:0/1510803445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad4100d80 0x7f1ad4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:10.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.033+0000 7f1ad9c7c700 1 -- 172.21.15.2:0/1510803445 >> 172.21.15.2:0/1510803445 conn(0x7f1ad4077ea0 msgr2=0x7f1ad40782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:10.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.033+0000 7f1ad9c7c700 1 -- 172.21.15.2:0/1510803445 shutdown_connections 2023-09-13T04:38:10.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.033+0000 7f1ad9c7c700 1 -- 172.21.15.2:0/1510803445 wait complete. 2023-09-13T04:38:10.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.035+0000 7f1ad9c7c700 1 Processor -- start 2023-09-13T04:38:10.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.035+0000 7f1ad9c7c700 1 -- start start 2023-09-13T04:38:10.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.036+0000 7f1ad9c7c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ad4100d80 0x7f1ad4196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:10.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.036+0000 7f1ad9c7c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad4101740 0x7f1ad4197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:10.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.036+0000 7f1ad9c7c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ad4106d90 0x7f1ad419b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:10.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.036+0000 7f1ad9c7c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad419bd50 con 0x7f1ad4100d80 2023-09-13T04:38:10.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.036+0000 7f1ad9c7c700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad419bec0 con 0x7f1ad4101740 2023-09-13T04:38:10.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.036+0000 7f1ad9c7c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad419c080 con 0x7f1ad4106d90 2023-09-13T04:38:10.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.036+0000 7f1ad37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ad4100d80 0x7f1ad4196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:10.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.036+0000 7f1ad37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ad4100d80 0x7f1ad4196de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46450/0 (socket says 172.21.15.2:46450) 2023-09-13T04:38:10.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.036+0000 7f1ad37fe700 1 -- 172.21.15.2:0/1118694286 learned_addr learned my addr 172.21.15.2:0/1118694286 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:10.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.036+0000 7f1ad2ffd700 1 --2- 172.21.15.2:0/1118694286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad4101740 0x7f1ad4197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:10.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.036+0000 7f1ad3fff700 1 --2- 172.21.15.2:0/1118694286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ad4106d90 0x7f1ad419b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:10.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.036+0000 7f1ad37fe700 1 -- 172.21.15.2:0/1118694286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ad4106d90 msgr2=0x7f1ad419b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:10.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.036+0000 7f1ad37fe700 1 --2- 172.21.15.2:0/1118694286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ad4106d90 0x7f1ad419b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:10.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.036+0000 7f1ad37fe700 1 -- 172.21.15.2:0/1118694286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad4101740 msgr2=0x7f1ad4197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:10.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.036+0000 7f1ad37fe700 1 --2- 172.21.15.2:0/1118694286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad4101740 0x7f1ad4197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:10.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.036+0000 7f1ad37fe700 1 -- 172.21.15.2:0/1118694286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac40095f0 con 0x7f1ad4100d80 2023-09-13T04:38:10.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.036+0000 7f1ad2ffd700 1 --2- 172.21.15.2:0/1118694286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad4101740 0x7f1ad4197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:38:10.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.036+0000 7f1ad3fff700 1 --2- 172.21.15.2:0/1118694286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ad4106d90 0x7f1ad419b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:38:10.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.037+0000 7f1ad37fe700 1 --2- 172.21.15.2:0/1118694286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ad4100d80 0x7f1ad4196de0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4000c00 tx=0x7f1ac400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:10.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.037+0000 7f1ad0ff9700 1 -- 172.21.15.2:0/1118694286 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac401d070 con 0x7f1ad4100d80 2023-09-13T04:38:10.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.037+0000 7f1ad0ff9700 1 -- 172.21.15.2:0/1118694286 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ac4005510 con 0x7f1ad4100d80 2023-09-13T04:38:10.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.037+0000 7f1ad9c7c700 1 -- 172.21.15.2:0/1118694286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad419c300 con 0x7f1ad4100d80 2023-09-13T04:38:10.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.037+0000 7f1ad0ff9700 1 -- 172.21.15.2:0/1118694286 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4004cb0 con 0x7f1ad4100d80 2023-09-13T04:38:10.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.037+0000 7f1ad9c7c700 1 -- 172.21.15.2:0/1118694286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad41a3bc0 con 0x7f1ad4100d80 2023-09-13T04:38:10.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.038+0000 7f1ad9c7c700 1 -- 172.21.15.2:0/1118694286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad404e9b0 con 0x7f1ad4100d80 2023-09-13T04:38:10.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.040+0000 7f1ad0ff9700 1 -- 172.21.15.2:0/1118694286 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f1ac400fc30 con 0x7f1ad4100d80 2023-09-13T04:38:10.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.041+0000 7f1ad0ff9700 1 --2- 172.21.15.2:0/1118694286 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1abc0a06c0 0x7f1abc0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:10.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.042+0000 7f1ad0ff9700 1 -- 172.21.15.2:0/1118694286 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1ac40ccbc0 con 0x7f1ad4100d80 2023-09-13T04:38:10.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.042+0000 7f1ad2ffd700 1 --2- 172.21.15.2:0/1118694286 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1abc0a06c0 0x7f1abc0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:10.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.042+0000 7f1ad2ffd700 1 --2- 172.21.15.2:0/1118694286 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1abc0a06c0 0x7f1abc0a2b80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1ad41983b0 tx=0x7f1ac8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:10.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.046+0000 7f1ad0ff9700 1 -- 172.21.15.2:0/1118694286 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1ac4096530 con 0x7f1ad4100d80 2023-09-13T04:38:10.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.305+0000 7f1ad9c7c700 1 -- 172.21.15.2:0/1118694286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1ad41980f0 con 0x7f1ad4100d80 2023-09-13T04:38:10.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.305+0000 7f1ad0ff9700 1 -- 172.21.15.2:0/1118694286 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7f1ac4099b50 con 0x7f1ad4100d80 2023-09-13T04:38:10.306 INFO:teuthology.orchestra.run.smithi002.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-09-13T04:38:10.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.308+0000 7f1ad9c7c700 1 -- 172.21.15.2:0/1118694286 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1abc0a06c0 msgr2=0x7f1abc0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:10.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.308+0000 7f1ad9c7c700 1 --2- 172.21.15.2:0/1118694286 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1abc0a06c0 0x7f1abc0a2b80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1ad41983b0 tx=0x7f1ac8009260 comp rx=0 tx=0).stop 2023-09-13T04:38:10.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.308+0000 7f1ad9c7c700 1 -- 172.21.15.2:0/1118694286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ad4100d80 msgr2=0x7f1ad4196de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:10.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.308+0000 7f1ad9c7c700 1 --2- 172.21.15.2:0/1118694286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ad4100d80 0x7f1ad4196de0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4000c00 tx=0x7f1ac400f740 comp rx=0 tx=0).stop 2023-09-13T04:38:10.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.308+0000 7f1ad9c7c700 1 -- 172.21.15.2:0/1118694286 shutdown_connections 2023-09-13T04:38:10.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.308+0000 7f1ad9c7c700 1 --2- 172.21.15.2:0/1118694286 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1abc0a06c0 0x7f1abc0a2b80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:10.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.308+0000 7f1ad9c7c700 1 --2- 172.21.15.2:0/1118694286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ad4106d90 0x7f1ad419b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:10.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.308+0000 7f1ad9c7c700 1 --2- 172.21.15.2:0/1118694286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ad4100d80 0x7f1ad4196de0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:10.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.308+0000 7f1ad9c7c700 1 --2- 172.21.15.2:0/1118694286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad4101740 0x7f1ad4197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:10.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.308+0000 7f1ad9c7c700 1 -- 172.21.15.2:0/1118694286 >> 172.21.15.2:0/1118694286 conn(0x7f1ad4077ea0 msgr2=0x7f1ad40ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:10.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.308+0000 7f1ad9c7c700 1 -- 172.21.15.2:0/1118694286 shutdown_connections 2023-09-13T04:38:10.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:10.308+0000 7f1ad9c7c700 1 -- 172.21.15.2:0/1118694286 wait complete. 2023-09-13T04:38:10.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:10 smithi002 ceph-mon[97651]: pgmap v281: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-13T04:38:10.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:10 smithi149 ceph-mon[133315]: pgmap v281: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-13T04:38:10.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:10 smithi156 ceph-mon[133675]: pgmap v281: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-13T04:38:10.789 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-09-13T04:38:10.789 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-13T04:38:10.789 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi002.front.sepia.ceph.com 2023-09-13T04:38:10.790 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-09-13T04:38:10.790 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-09-13T04:38:10.790 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-09-13T04:38:10.791 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-13T04:38:10.791 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-13T04:38:10.791 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-09-13T04:38:10.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:38:10.792 DEBUG:teuthology.orchestra.run.smithi002:> ip addr 2023-09-13T04:38:10.837 INFO:teuthology.orchestra.run.smithi002.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-13T04:38:10.837 INFO:teuthology.orchestra.run.smithi002.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-13T04:38:10.837 INFO:teuthology.orchestra.run.smithi002.stdout: inet 127.0.0.1/8 scope host lo 2023-09-13T04:38:10.837 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-09-13T04:38:10.838 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 ::1/128 scope host 2023-09-13T04:38:10.838 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-09-13T04:38:10.838 INFO:teuthology.orchestra.run.smithi002.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-13T04:38:10.838 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:02 brd ff:ff:ff:ff:ff:ff 2023-09-13T04:38:10.839 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp7s0f0 2023-09-13T04:38:10.839 INFO:teuthology.orchestra.run.smithi002.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-13T04:38:10.839 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:03 brd ff:ff:ff:ff:ff:ff 2023-09-13T04:38:10.839 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp7s0f1 2023-09-13T04:38:10.839 INFO:teuthology.orchestra.run.smithi002.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-13T04:38:10.839 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:20 brd ff:ff:ff:ff:ff:ff 2023-09-13T04:38:10.840 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp1s0f0 2023-09-13T04:38:10.840 INFO:teuthology.orchestra.run.smithi002.stdout: inet 172.21.15.2/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-09-13T04:38:10.840 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft 41741sec preferred_lft 41741sec 2023-09-13T04:38:10.840 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 fe80::ec4:7aff:febd:1920/64 scope link 2023-09-13T04:38:10.841 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-09-13T04:38:10.841 INFO:teuthology.orchestra.run.smithi002.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-13T04:38:10.841 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:21 brd ff:ff:ff:ff:ff:ff 2023-09-13T04:38:10.841 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp1s0f1 2023-09-13T04:38:10.842 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-13T04:38:10.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:38:10.843 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-09-13T04:38:10.844 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-09-13T04:38:10.844 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add name ceph-brx type bridge 2023-09-13T04:38:10.844 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr flush dev ceph-brx 2023-09-13T04:38:10.845 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set ceph-brx up 2023-09-13T04:38:10.845 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-13T04:38:10.845 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-09-13T04:38:11.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:38:11.326 DEBUG:teuthology.orchestra.run.smithi002:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-13T04:38:11.400 INFO:teuthology.orchestra.run.smithi002.stdout:1 2023-09-13T04:38:11.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:38:11.404 DEBUG:teuthology.orchestra.run.smithi002:> route 2023-09-13T04:38:11.464 INFO:teuthology.orchestra.run.smithi002.stdout:Kernel IP routing table 2023-09-13T04:38:11.464 INFO:teuthology.orchestra.run.smithi002.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-13T04:38:11.464 INFO:teuthology.orchestra.run.smithi002.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-09-13T04:38:11.465 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-09-13T04:38:11.465 INFO:teuthology.orchestra.run.smithi002.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-13T04:38:11.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:38:11.466 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-09-13T04:38:11.467 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-09-13T04:38:11.467 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-09-13T04:38:11.467 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-09-13T04:38:11.468 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-09-13T04:38:11.468 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-09-13T04:38:11.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:11 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1118694286' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-13T04:38:11.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:11 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1118694286' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-13T04:38:11.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:11 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1118694286' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-13T04:38:12.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:38:12.192 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2023-09-13T04:38:12.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:38:12.279 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list-id 2023-09-13T04:38:12.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:38:12.364 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-09-13T04:38:12.364 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-09-13T04:38:12.365 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-13T04:38:12.365 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-09-13T04:38:12.366 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-09-13T04:38:12.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:12 smithi002 ceph-mon[97651]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-13T04:38:12.549 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-09-13T04:38:12.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:38:12.550 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-09-13T04:38:12.550 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-09-13T04:38:12.551 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-09-13T04:38:12.551 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-13T04:38:12.553 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-09-13T04:38:12.553 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-09-13T04:38:12.554 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-09-13T04:38:12.554 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-09-13T04:38:12.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:12 smithi149 ceph-mon[133315]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-13T04:38:12.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:12 smithi156 ceph-mon[133675]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-13T04:38:12.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:38:12.844 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-09-13T04:38:12.845 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-09-13T04:38:12.845 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set brx.0 up 2023-09-13T04:38:12.845 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set dev brx.0 master ceph-brx 2023-09-13T04:38:12.846 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-09-13T04:38:13.023 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-09-13T04:38:13.024 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-13T04:38:13.024 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-09-13T04:38:13.068 INFO:teuthology.orchestra.run.smithi002.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-09-13T04:38:13.068 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-13T04:38:13.069 DEBUG:teuthology.orchestra.run.smithi002:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-09-13T04:38:13.152 DEBUG:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2023-09-13T04:38:13.264 DEBUG:teuthology.orchestra.run.smithi002:> cat /proc/self/mounts | awk '{print $2}' 2023-09-13T04:38:13.310 INFO:teuthology.orchestra.run.smithi002.stdout:/sys 2023-09-13T04:38:13.310 INFO:teuthology.orchestra.run.smithi002.stdout:/proc 2023-09-13T04:38:13.310 INFO:teuthology.orchestra.run.smithi002.stdout:/dev 2023-09-13T04:38:13.310 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/security 2023-09-13T04:38:13.311 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/shm 2023-09-13T04:38:13.311 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/pts 2023-09-13T04:38:13.311 INFO:teuthology.orchestra.run.smithi002.stdout:/run 2023-09-13T04:38:13.311 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup 2023-09-13T04:38:13.312 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/systemd 2023-09-13T04:38:13.312 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/pstore 2023-09-13T04:38:13.312 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/freezer 2023-09-13T04:38:13.312 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-09-13T04:38:13.312 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-09-13T04:38:13.312 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/devices 2023-09-13T04:38:13.313 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/blkio 2023-09-13T04:38:13.313 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/hugetlb 2023-09-13T04:38:13.313 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/cpuset 2023-09-13T04:38:13.313 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/pids 2023-09-13T04:38:13.314 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/memory 2023-09-13T04:38:13.314 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/perf_event 2023-09-13T04:38:13.314 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/tracing 2023-09-13T04:38:13.314 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/config 2023-09-13T04:38:13.315 INFO:teuthology.orchestra.run.smithi002.stdout:/ 2023-09-13T04:38:13.315 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/selinux 2023-09-13T04:38:13.316 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2023-09-13T04:38:13.316 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/debug 2023-09-13T04:38:13.316 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/mqueue 2023-09-13T04:38:13.316 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/hugepages 2023-09-13T04:38:13.316 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/fuse/connections 2023-09-13T04:38:13.316 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/nfs/rpc_pipefs 2023-09-13T04:38:13.317 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/1000 2023-09-13T04:38:13.317 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/ceph 2023-09-13T04:38:13.317 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay 2023-09-13T04:38:13.317 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/9448a226694c6fe7a193249f6dd32aab9105638574e1246c5b3cd22f59f71649/merged 2023-09-13T04:38:13.318 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/58feb3c7e9ad2a0512f9a35b7ad9df270c61c59f8e78af605514375a95ee7e6a/merged 2023-09-13T04:38:13.318 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/0 2023-09-13T04:38:13.318 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2023-09-13T04:38:13.318 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/e83a788fc6fb020faa0be35123baf0da922bd5f0e780c2767388ddb6c99f4dff/merged 2023-09-13T04:38:13.318 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/3a00de84dd0e0706da447387696d07cf88d2120c94e07f43396ba3affe4382ff/merged 2023-09-13T04:38:13.319 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/f2f304f45f9c1a673a53decf5a776041f31dab34a5363c43d112fc45f4aed334/merged 2023-09-13T04:38:13.319 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/9fd66c4afe6ab40f577eb0d9d8167c1694cf834437fbd7109564537e464060a5/merged 2023-09-13T04:38:13.319 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/f325f7624707328c01c9c829501b27a2e08eaa497f0715ac225706c1202fde12/merged 2023-09-13T04:38:13.319 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/ac7a04b9d720dfab2fbe51856f099d131d5819a18dfebd9bca8a59fafb292be9/merged 2023-09-13T04:38:13.319 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/e30cc29750d8f91ad8e23b5943642edf1ea85c91aa34ba77ca818f81ba801f01/merged 2023-09-13T04:38:13.320 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/95a8d67b7d0e016a611b9ff813c1091ad0e4a6e14e0c6d3cd1d80659a610324d/merged 2023-09-13T04:38:13.320 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns 2023-09-13T04:38:13.320 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-13T04:38:13.320 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-13T04:38:13.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:38:13.322 DEBUG:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2023-09-13T04:38:13.405 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-13T04:38:13.406 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-09-13T04:38:13.450 DEBUG:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2023-09-13T04:38:13.523 DEBUG:teuthology.orchestra.run.smithi002:> cat /proc/self/mounts | awk '{print $2}' 2023-09-13T04:38:13.574 INFO:teuthology.orchestra.run.smithi002.stdout:/sys 2023-09-13T04:38:13.574 INFO:teuthology.orchestra.run.smithi002.stdout:/proc 2023-09-13T04:38:13.574 INFO:teuthology.orchestra.run.smithi002.stdout:/dev 2023-09-13T04:38:13.575 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/security 2023-09-13T04:38:13.575 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/shm 2023-09-13T04:38:13.575 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/pts 2023-09-13T04:38:13.575 INFO:teuthology.orchestra.run.smithi002.stdout:/run 2023-09-13T04:38:13.576 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup 2023-09-13T04:38:13.576 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/systemd 2023-09-13T04:38:13.576 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/pstore 2023-09-13T04:38:13.576 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/freezer 2023-09-13T04:38:13.576 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-09-13T04:38:13.577 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-09-13T04:38:13.577 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/devices 2023-09-13T04:38:13.577 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/blkio 2023-09-13T04:38:13.577 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/hugetlb 2023-09-13T04:38:13.577 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/cpuset 2023-09-13T04:38:13.578 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/pids 2023-09-13T04:38:13.578 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/memory 2023-09-13T04:38:13.578 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/perf_event 2023-09-13T04:38:13.578 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/tracing 2023-09-13T04:38:13.579 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/config 2023-09-13T04:38:13.579 INFO:teuthology.orchestra.run.smithi002.stdout:/ 2023-09-13T04:38:13.579 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/selinux 2023-09-13T04:38:13.579 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2023-09-13T04:38:13.579 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/debug 2023-09-13T04:38:13.580 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/mqueue 2023-09-13T04:38:13.580 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/hugepages 2023-09-13T04:38:13.580 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/fuse/connections 2023-09-13T04:38:13.580 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/nfs/rpc_pipefs 2023-09-13T04:38:13.580 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/1000 2023-09-13T04:38:13.581 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/ceph 2023-09-13T04:38:13.581 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay 2023-09-13T04:38:13.581 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/9448a226694c6fe7a193249f6dd32aab9105638574e1246c5b3cd22f59f71649/merged 2023-09-13T04:38:13.581 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/58feb3c7e9ad2a0512f9a35b7ad9df270c61c59f8e78af605514375a95ee7e6a/merged 2023-09-13T04:38:13.582 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/0 2023-09-13T04:38:13.582 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2023-09-13T04:38:13.582 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/e83a788fc6fb020faa0be35123baf0da922bd5f0e780c2767388ddb6c99f4dff/merged 2023-09-13T04:38:13.582 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/3a00de84dd0e0706da447387696d07cf88d2120c94e07f43396ba3affe4382ff/merged 2023-09-13T04:38:13.582 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/f2f304f45f9c1a673a53decf5a776041f31dab34a5363c43d112fc45f4aed334/merged 2023-09-13T04:38:13.583 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/9fd66c4afe6ab40f577eb0d9d8167c1694cf834437fbd7109564537e464060a5/merged 2023-09-13T04:38:13.583 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/f325f7624707328c01c9c829501b27a2e08eaa497f0715ac225706c1202fde12/merged 2023-09-13T04:38:13.583 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/ac7a04b9d720dfab2fbe51856f099d131d5819a18dfebd9bca8a59fafb292be9/merged 2023-09-13T04:38:13.583 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/e30cc29750d8f91ad8e23b5943642edf1ea85c91aa34ba77ca818f81ba801f01/merged 2023-09-13T04:38:13.583 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/95a8d67b7d0e016a611b9ff813c1091ad0e4a6e14e0c6d3cd1d80659a610324d/merged 2023-09-13T04:38:13.584 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns 2023-09-13T04:38:13.584 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-13T04:38:13.584 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-13T04:38:13.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:38:13.585 DEBUG:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2023-09-13T04:38:13.671 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi002.stderr:2023-09-13T04:38:13.671+0000 7f49d62655c0 -1 init, newargv = 0x563952f8a7f0 newargc=15 2023-09-13T04:38:13.672 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi002.stderr:ceph-fuse[151313]: starting ceph client 2023-09-13T04:38:13.681 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi002.stderr:ceph-fuse[151313]: starting fuse 2023-09-13T04:38:14.730 DEBUG:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2023-09-13T04:38:14.808 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:14 smithi002 ceph-mon[97651]: pgmap v283: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-13T04:38:14.810 DEBUG:teuthology.orchestra.run.smithi002:> cat /proc/self/mounts | awk '{print $2}' 2023-09-13T04:38:14.856 INFO:teuthology.orchestra.run.smithi002.stdout:/sys 2023-09-13T04:38:14.857 INFO:teuthology.orchestra.run.smithi002.stdout:/proc 2023-09-13T04:38:14.857 INFO:teuthology.orchestra.run.smithi002.stdout:/dev 2023-09-13T04:38:14.857 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/security 2023-09-13T04:38:14.857 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/shm 2023-09-13T04:38:14.858 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/pts 2023-09-13T04:38:14.858 INFO:teuthology.orchestra.run.smithi002.stdout:/run 2023-09-13T04:38:14.858 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup 2023-09-13T04:38:14.858 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/systemd 2023-09-13T04:38:14.858 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/pstore 2023-09-13T04:38:14.859 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/freezer 2023-09-13T04:38:14.859 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-09-13T04:38:14.859 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-09-13T04:38:14.859 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/devices 2023-09-13T04:38:14.859 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/blkio 2023-09-13T04:38:14.859 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/hugetlb 2023-09-13T04:38:14.860 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/cpuset 2023-09-13T04:38:14.860 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/pids 2023-09-13T04:38:14.860 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/memory 2023-09-13T04:38:14.860 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup/perf_event 2023-09-13T04:38:14.860 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/tracing 2023-09-13T04:38:14.861 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/config 2023-09-13T04:38:14.861 INFO:teuthology.orchestra.run.smithi002.stdout:/ 2023-09-13T04:38:14.861 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/selinux 2023-09-13T04:38:14.861 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2023-09-13T04:38:14.861 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/debug 2023-09-13T04:38:14.861 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/mqueue 2023-09-13T04:38:14.862 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/hugepages 2023-09-13T04:38:14.862 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/fuse/connections 2023-09-13T04:38:14.862 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/nfs/rpc_pipefs 2023-09-13T04:38:14.862 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/1000 2023-09-13T04:38:14.862 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/ceph 2023-09-13T04:38:14.863 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay 2023-09-13T04:38:14.863 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/9448a226694c6fe7a193249f6dd32aab9105638574e1246c5b3cd22f59f71649/merged 2023-09-13T04:38:14.863 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/58feb3c7e9ad2a0512f9a35b7ad9df270c61c59f8e78af605514375a95ee7e6a/merged 2023-09-13T04:38:14.863 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/0 2023-09-13T04:38:14.863 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2023-09-13T04:38:14.864 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/e83a788fc6fb020faa0be35123baf0da922bd5f0e780c2767388ddb6c99f4dff/merged 2023-09-13T04:38:14.864 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/3a00de84dd0e0706da447387696d07cf88d2120c94e07f43396ba3affe4382ff/merged 2023-09-13T04:38:14.864 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/f2f304f45f9c1a673a53decf5a776041f31dab34a5363c43d112fc45f4aed334/merged 2023-09-13T04:38:14.864 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/9fd66c4afe6ab40f577eb0d9d8167c1694cf834437fbd7109564537e464060a5/merged 2023-09-13T04:38:14.864 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/f325f7624707328c01c9c829501b27a2e08eaa497f0715ac225706c1202fde12/merged 2023-09-13T04:38:14.864 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/ac7a04b9d720dfab2fbe51856f099d131d5819a18dfebd9bca8a59fafb292be9/merged 2023-09-13T04:38:14.865 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/e30cc29750d8f91ad8e23b5943642edf1ea85c91aa34ba77ca818f81ba801f01/merged 2023-09-13T04:38:14.865 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/95a8d67b7d0e016a611b9ff813c1091ad0e4a6e14e0c6d3cd1d80659a610324d/merged 2023-09-13T04:38:14.865 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns 2023-09-13T04:38:14.865 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-13T04:38:14.865 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-13T04:38:14.866 INFO:teuthology.orchestra.run.smithi002.stdout:/home/ubuntu/cephtest/mnt.admin 2023-09-13T04:38:14.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:38:14.867 DEBUG:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2023-09-13T04:38:14.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:14 smithi149 ceph-mon[133315]: pgmap v283: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-13T04:38:14.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:14 smithi156 ceph-mon[133675]: pgmap v283: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-13T04:38:14.951 INFO:teuthology.orchestra.run.smithi002.stdout:121 2023-09-13T04:38:14.951 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2023-09-13T04:38:14.952 DEBUG:teuthology.orchestra.run.smithi002:> sudo stdin-killer -- python3 -c ' 2023-09-13T04:38:14.952 DEBUG:teuthology.orchestra.run.smithi002:> import glob 2023-09-13T04:38:14.952 DEBUG:teuthology.orchestra.run.smithi002:> import re 2023-09-13T04:38:14.953 DEBUG:teuthology.orchestra.run.smithi002:> import os 2023-09-13T04:38:14.953 DEBUG:teuthology.orchestra.run.smithi002:> import subprocess 2023-09-13T04:38:14.953 DEBUG:teuthology.orchestra.run.smithi002:> 2023-09-13T04:38:14.954 DEBUG:teuthology.orchestra.run.smithi002:> def _find_admin_socket(client_name): 2023-09-13T04:38:14.954 DEBUG:teuthology.orchestra.run.smithi002:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-09-13T04:38:14.954 DEBUG:teuthology.orchestra.run.smithi002:> files = glob.glob(asok_path) 2023-09-13T04:38:14.955 DEBUG:teuthology.orchestra.run.smithi002:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-09-13T04:38:14.955 DEBUG:teuthology.orchestra.run.smithi002:> 2023-09-13T04:38:14.955 DEBUG:teuthology.orchestra.run.smithi002:> # Given a non-glob path, it better be there 2023-09-13T04:38:14.956 DEBUG:teuthology.orchestra.run.smithi002:> if "*" not in asok_path: 2023-09-13T04:38:14.956 DEBUG:teuthology.orchestra.run.smithi002:> assert(len(files) == 1) 2023-09-13T04:38:14.956 DEBUG:teuthology.orchestra.run.smithi002:> return files[0] 2023-09-13T04:38:14.957 DEBUG:teuthology.orchestra.run.smithi002:> 2023-09-13T04:38:14.957 DEBUG:teuthology.orchestra.run.smithi002:> for f in files: 2023-09-13T04:38:14.957 DEBUG:teuthology.orchestra.run.smithi002:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-13T04:38:14.958 DEBUG:teuthology.orchestra.run.smithi002:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-13T04:38:14.958 DEBUG:teuthology.orchestra.run.smithi002:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-13T04:38:14.958 DEBUG:teuthology.orchestra.run.smithi002:> contents = proc_f.read() 2023-09-13T04:38:14.959 DEBUG:teuthology.orchestra.run.smithi002:> if mountpoint in contents: 2023-09-13T04:38:14.959 DEBUG:teuthology.orchestra.run.smithi002:> return f 2023-09-13T04:38:14.959 DEBUG:teuthology.orchestra.run.smithi002:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-13T04:38:14.960 DEBUG:teuthology.orchestra.run.smithi002:> 2023-09-13T04:38:14.960 DEBUG:teuthology.orchestra.run.smithi002:> print(_find_admin_socket("client.admin")) 2023-09-13T04:38:14.960 DEBUG:teuthology.orchestra.run.smithi002:> ' 2023-09-13T04:38:15.135 INFO:teuthology.orchestra.run.smithi002.stdout:/var/run/ceph/ceph-client.admin.151313.asok 2023-09-13T04:38:15.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-13T04:38:15.149 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.151313.asok 2023-09-13T04:38:15.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:38:15.150 DEBUG:teuthology.orchestra.run.smithi002:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.151313.asok status 2023-09-13T04:38:15.383 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:38:15.383 INFO:teuthology.orchestra.run.smithi002.stdout: "metadata": { 2023-09-13T04:38:15.384 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_sha1": "4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5", 2023-09-13T04:38:15.384 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_version": "ceph version 18.0.0-6083-g4a754ac9 (4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5) reef (dev)", 2023-09-13T04:38:15.384 INFO:teuthology.orchestra.run.smithi002.stdout: "entity_id": "admin", 2023-09-13T04:38:15.384 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": "smithi002", 2023-09-13T04:38:15.385 INFO:teuthology.orchestra.run.smithi002.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-09-13T04:38:15.385 INFO:teuthology.orchestra.run.smithi002.stdout: "pid": "151313", 2023-09-13T04:38:15.385 INFO:teuthology.orchestra.run.smithi002.stdout: "root": "/" 2023-09-13T04:38:15.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:38:15.385 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_count": 0, 2023-09-13T04:38:15.386 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_pinned_count": 0, 2023-09-13T04:38:15.386 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 14661, 2023-09-13T04:38:15.386 INFO:teuthology.orchestra.run.smithi002.stdout: "inst": { 2023-09-13T04:38:15.386 INFO:teuthology.orchestra.run.smithi002.stdout: "name": { 2023-09-13T04:38:15.387 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "client", 2023-09-13T04:38:15.387 INFO:teuthology.orchestra.run.smithi002.stdout: "num": 14661 2023-09-13T04:38:15.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:38:15.387 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2023-09-13T04:38:15.387 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2023-09-13T04:38:15.388 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "192.168.0.1:0", 2023-09-13T04:38:15.388 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 580991495 2023-09-13T04:38:15.388 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:38:15.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:38:15.389 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2023-09-13T04:38:15.389 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2023-09-13T04:38:15.389 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "192.168.0.1:0", 2023-09-13T04:38:15.389 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 580991495 2023-09-13T04:38:15.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:38:15.390 INFO:teuthology.orchestra.run.smithi002.stdout: "inst_str": "client.14661 192.168.0.1:0/580991495", 2023-09-13T04:38:15.390 INFO:teuthology.orchestra.run.smithi002.stdout: "addr_str": "192.168.0.1:0/580991495", 2023-09-13T04:38:15.390 INFO:teuthology.orchestra.run.smithi002.stdout: "inode_count": 1, 2023-09-13T04:38:15.391 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_epoch": 17, 2023-09-13T04:38:15.391 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch": 88, 2023-09-13T04:38:15.391 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch_barrier": 0, 2023-09-13T04:38:15.391 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklisted": false, 2023-09-13T04:38:15.392 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs" 2023-09-13T04:38:15.392 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:38:15.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:38:15.399 DEBUG:teuthology.orchestra.run.smithi002:> stat --file-system '--printf=%T 2023-09-13T04:38:15.399 DEBUG:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-09-13T04:38:15.445 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2023-09-13T04:38:15.445 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-09-13T04:38:15.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:38:15.446 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-09-13T04:38:15.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:38:15.558 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-09-13T04:38:15.678 INFO:teuthology.orchestra.run.smithi002.stdout:# file: . 2023-09-13T04:38:15.678 INFO:teuthology.orchestra.run.smithi002.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-09-13T04:38:15.679 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:38:15.680 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-09-13T04:38:15.680 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-13T04:38:15.725 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T04:38:15.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.968+0000 7f4aafdff700 1 -- 172.21.15.2:0/3015121364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4aa80ffe90 msgr2=0x7f4aa8100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:15.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.968+0000 7f4aafdff700 1 --2- 172.21.15.2:0/3015121364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4aa80ffe90 0x7f4aa8100310 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009990 tx=0x7f4a98009ca0 comp rx=0 tx=0).stop 2023-09-13T04:38:15.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.968+0000 7f4aafdff700 1 -- 172.21.15.2:0/3015121364 shutdown_connections 2023-09-13T04:38:15.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.968+0000 7f4aafdff700 1 --2- 172.21.15.2:0/3015121364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4aa8100850 0x7f4aa8100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:15.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.968+0000 7f4aafdff700 1 --2- 172.21.15.2:0/3015121364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4aa80ff530 0x7f4aa80ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:15.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.968+0000 7f4aafdff700 1 --2- 172.21.15.2:0/3015121364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4aa80ffe90 0x7f4aa8100310 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:15.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.968+0000 7f4aafdff700 1 -- 172.21.15.2:0/3015121364 >> 172.21.15.2:0/3015121364 conn(0x7f4aa80fb110 msgr2=0x7f4aa80fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:15.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.968+0000 7f4aafdff700 1 -- 172.21.15.2:0/3015121364 shutdown_connections 2023-09-13T04:38:15.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.968+0000 7f4aafdff700 1 -- 172.21.15.2:0/3015121364 wait complete. 2023-09-13T04:38:15.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.969+0000 7f4aafdff700 1 Processor -- start 2023-09-13T04:38:15.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.970+0000 7f4aafdff700 1 -- start start 2023-09-13T04:38:15.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.970+0000 7f4aafdff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4aa80ff530 0x7f4aa8198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:15.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.970+0000 7f4aafdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4aa80ffe90 0x7f4aa81990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:15.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.970+0000 7f4aafdff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4aa8100850 0x7f4aa81a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:15.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.970+0000 7f4aafdff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa81a0710 con 0x7f4aa8100850 2023-09-13T04:38:15.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.970+0000 7f4aafdff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa81a0850 con 0x7f4aa80ff530 2023-09-13T04:38:15.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.970+0000 7f4aafdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa81a09e0 con 0x7f4aa80ffe90 2023-09-13T04:38:15.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.970+0000 7f4aadb9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4aa80ff530 0x7f4aa8198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:15.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.970+0000 7f4aae39c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4aa8100850 0x7f4aa81a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:15.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.970+0000 7f4aadb9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4aa80ff530 0x7f4aa8198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:34834/0 (socket says 172.21.15.2:34834) 2023-09-13T04:38:15.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.970+0000 7f4aadb9b700 1 -- 172.21.15.2:0/410464484 learned_addr learned my addr 172.21.15.2:0/410464484 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:15.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.971+0000 7f4aad39a700 1 --2- 172.21.15.2:0/410464484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4aa80ffe90 0x7f4aa81990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:15.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.971+0000 7f4aadb9b700 1 -- 172.21.15.2:0/410464484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4aa80ffe90 msgr2=0x7f4aa81990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:15.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.971+0000 7f4aadb9b700 1 --2- 172.21.15.2:0/410464484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4aa80ffe90 0x7f4aa81990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:15.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.971+0000 7f4aadb9b700 1 -- 172.21.15.2:0/410464484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4aa8100850 msgr2=0x7f4aa81a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:15.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.971+0000 7f4aadb9b700 1 --2- 172.21.15.2:0/410464484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4aa8100850 0x7f4aa81a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:15.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.971+0000 7f4aadb9b700 1 -- 172.21.15.2:0/410464484 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a980095f0 con 0x7f4aa80ff530 2023-09-13T04:38:15.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.971+0000 7f4aae39c700 1 --2- 172.21.15.2:0/410464484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4aa8100850 0x7f4aa81a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:38:15.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.971+0000 7f4aadb9b700 1 --2- 172.21.15.2:0/410464484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4aa80ff530 0x7f4aa8198b90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400e9a0 tx=0x7f4aa400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:15.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.971+0000 7f4a9effd700 1 -- 172.21.15.2:0/410464484 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4009790 con 0x7f4aa80ff530 2023-09-13T04:38:15.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.971+0000 7f4aafdff700 1 -- 172.21.15.2:0/410464484 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa81a0cc0 con 0x7f4aa80ff530 2023-09-13T04:38:15.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.971+0000 7f4a9effd700 1 -- 172.21.15.2:0/410464484 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4aa4010430 con 0x7f4aa80ff530 2023-09-13T04:38:15.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.971+0000 7f4a9effd700 1 -- 172.21.15.2:0/410464484 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4018680 con 0x7f4aa80ff530 2023-09-13T04:38:15.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.972+0000 7f4aafdff700 1 -- 172.21.15.2:0/410464484 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa81a11d0 con 0x7f4aa80ff530 2023-09-13T04:38:15.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.972+0000 7f4aafdff700 1 -- 172.21.15.2:0/410464484 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aa810a310 con 0x7f4aa80ff530 2023-09-13T04:38:15.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.975+0000 7f4a9effd700 1 -- 172.21.15.2:0/410464484 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f4aa40105a0 con 0x7f4aa80ff530 2023-09-13T04:38:15.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.976+0000 7f4a9effd700 1 --2- 172.21.15.2:0/410464484 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4a940a0690 0x7f4a940a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:15.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.976+0000 7f4a9effd700 1 -- 172.21.15.2:0/410464484 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4aa4014070 con 0x7f4aa80ff530 2023-09-13T04:38:15.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.976+0000 7f4aad39a700 1 --2- 172.21.15.2:0/410464484 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4a940a0690 0x7f4a940a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:15.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.977+0000 7f4aad39a700 1 --2- 172.21.15.2:0/410464484 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4a940a0690 0x7f4a940a2b50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800b2e0 tx=0x7f4a98005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:15.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:15.978+0000 7f4a9effd700 1 -- 172.21.15.2:0/410464484 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4aa4095a80 con 0x7f4aa80ff530 2023-09-13T04:38:16.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.192+0000 7f4aafdff700 1 -- 172.21.15.2:0/410464484 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4aa810e8e0 con 0x7f4aa80ff530 2023-09-13T04:38:16.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.195+0000 7f4a9effd700 1 -- 172.21.15.2:0/410464484 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23927 (secure 0 0 0) 0x7f4aa4026090 con 0x7f4aa80ff530 2023-09-13T04:38:16.195 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:38:16.196 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":88,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:38:04.132627+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T04:37:57.094802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T04:37:58.564487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-13T04:38:02.096895+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:38:16.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.197+0000 7f4aafdff700 1 -- 172.21.15.2:0/410464484 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4a940a0690 msgr2=0x7f4a940a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:16.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.197+0000 7f4aafdff700 1 --2- 172.21.15.2:0/410464484 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4a940a0690 0x7f4a940a2b50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800b2e0 tx=0x7f4a98005fb0 comp rx=0 tx=0).stop 2023-09-13T04:38:16.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.197+0000 7f4aafdff700 1 -- 172.21.15.2:0/410464484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4aa80ff530 msgr2=0x7f4aa8198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:16.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.197+0000 7f4aafdff700 1 --2- 172.21.15.2:0/410464484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4aa80ff530 0x7f4aa8198b90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400e9a0 tx=0x7f4aa400ecb0 comp rx=0 tx=0).stop 2023-09-13T04:38:16.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.197+0000 7f4aafdff700 1 -- 172.21.15.2:0/410464484 shutdown_connections 2023-09-13T04:38:16.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.197+0000 7f4aafdff700 1 --2- 172.21.15.2:0/410464484 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4a940a0690 0x7f4a940a2b50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:16.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.197+0000 7f4aafdff700 1 --2- 172.21.15.2:0/410464484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4aa8100850 0x7f4aa81a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:16.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.197+0000 7f4aafdff700 1 --2- 172.21.15.2:0/410464484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4aa80ff530 0x7f4aa8198b90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:16.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.197+0000 7f4aafdff700 1 --2- 172.21.15.2:0/410464484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4aa80ffe90 0x7f4aa81990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:16.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.197+0000 7f4aafdff700 1 -- 172.21.15.2:0/410464484 >> 172.21.15.2:0/410464484 conn(0x7f4aa80fb110 msgr2=0x7f4aa80fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:16.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.197+0000 7f4aafdff700 1 -- 172.21.15.2:0/410464484 shutdown_connections 2023-09-13T04:38:16.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.197+0000 7f4aafdff700 1 -- 172.21.15.2:0/410464484 wait complete. 2023-09-13T04:38:16.216 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T04:33:42.025714+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-13T04:37:57.094802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-13T04:37:58.564487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-13T04:38:02.096895+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-13T04:38:16.217 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-13T04:38:16.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.423+0000 7f43bddff700 1 -- 172.21.15.2:0/1298494605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43b8104070 msgr2=0x7f43b8106460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:16.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.423+0000 7f43bddff700 1 --2- 172.21.15.2:0/1298494605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43b8104070 0x7f43b8106460 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009990 tx=0x7f43ac009ca0 comp rx=0 tx=0).stop 2023-09-13T04:38:16.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.423+0000 7f43bddff700 1 -- 172.21.15.2:0/1298494605 shutdown_connections 2023-09-13T04:38:16.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.423+0000 7f43bddff700 1 --2- 172.21.15.2:0/1298494605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43b81069a0 0x7f43b8108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:16.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.423+0000 7f43bddff700 1 --2- 172.21.15.2:0/1298494605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43b8101740 0x7f43b8103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:16.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.423+0000 7f43bddff700 1 --2- 172.21.15.2:0/1298494605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43b8104070 0x7f43b8106460 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:16.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.423+0000 7f43bddff700 1 -- 172.21.15.2:0/1298494605 >> 172.21.15.2:0/1298494605 conn(0x7f43b80fb130 msgr2=0x7f43b80fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:16.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.423+0000 7f43bddff700 1 -- 172.21.15.2:0/1298494605 shutdown_connections 2023-09-13T04:38:16.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.423+0000 7f43bddff700 1 -- 172.21.15.2:0/1298494605 wait complete. 2023-09-13T04:38:16.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.424+0000 7f43bddff700 1 Processor -- start 2023-09-13T04:38:16.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.424+0000 7f43bddff700 1 -- start start 2023-09-13T04:38:16.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.425+0000 7f43bddff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43b8101740 0x7f43b8198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:16.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.425+0000 7f43bddff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43b8104070 0x7f43b8199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:16.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.425+0000 7f43bddff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43b81069a0 0x7f43b81a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:16.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.425+0000 7f43bddff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b81a0780 con 0x7f43b81069a0 2023-09-13T04:38:16.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.425+0000 7f43bddff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b81a08f0 con 0x7f43b8101740 2023-09-13T04:38:16.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.425+0000 7f43bddff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b81a0ab0 con 0x7f43b8104070 2023-09-13T04:38:16.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.425+0000 7f43b7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43b81069a0 0x7f43b81a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:16.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.425+0000 7f43b7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43b81069a0 0x7f43b81a01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43468/0 (socket says 172.21.15.2:43468) 2023-09-13T04:38:16.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.425+0000 7f43b7fff700 1 -- 172.21.15.2:0/145768745 learned_addr learned my addr 172.21.15.2:0/145768745 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:16.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.425+0000 7f43b77fe700 1 --2- 172.21.15.2:0/145768745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43b8101740 0x7f43b8198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:16.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.425+0000 7f43b6ffd700 1 --2- 172.21.15.2:0/145768745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43b8104070 0x7f43b8199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:16.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.426+0000 7f43b7fff700 1 -- 172.21.15.2:0/145768745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43b8104070 msgr2=0x7f43b8199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:16.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.426+0000 7f43b7fff700 1 --2- 172.21.15.2:0/145768745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43b8104070 0x7f43b8199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:16.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.426+0000 7f43b7fff700 1 -- 172.21.15.2:0/145768745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43b8101740 msgr2=0x7f43b8198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:16.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.426+0000 7f43b7fff700 1 --2- 172.21.15.2:0/145768745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43b8101740 0x7f43b8198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:16.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.426+0000 7f43b7fff700 1 -- 172.21.15.2:0/145768745 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a8009520 con 0x7f43b81069a0 2023-09-13T04:38:16.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.426+0000 7f43b6ffd700 1 --2- 172.21.15.2:0/145768745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43b8104070 0x7f43b8199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:38:16.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.426+0000 7f43b7fff700 1 --2- 172.21.15.2:0/145768745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43b81069a0 0x7f43b81a01b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f43b000ef30 tx=0x7f43b000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:16.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.426+0000 7f43b4ff9700 1 -- 172.21.15.2:0/145768745 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b0018070 con 0x7f43b81069a0 2023-09-13T04:38:16.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.426+0000 7f43b4ff9700 1 -- 172.21.15.2:0/145768745 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43b000f040 con 0x7f43b81069a0 2023-09-13T04:38:16.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.426+0000 7f43bddff700 1 -- 172.21.15.2:0/145768745 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43ac0095f0 con 0x7f43b81069a0 2023-09-13T04:38:16.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.426+0000 7f43b4ff9700 1 -- 172.21.15.2:0/145768745 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b000cdb0 con 0x7f43b81069a0 2023-09-13T04:38:16.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.426+0000 7f43bddff700 1 -- 172.21.15.2:0/145768745 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b81a1060 con 0x7f43b81069a0 2023-09-13T04:38:16.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.427+0000 7f43bddff700 1 -- 172.21.15.2:0/145768745 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b8108570 con 0x7f43b81069a0 2023-09-13T04:38:16.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.429+0000 7f43b4ff9700 1 -- 172.21.15.2:0/145768745 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f43b0003a00 con 0x7f43b81069a0 2023-09-13T04:38:16.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.430+0000 7f43b4ff9700 1 --2- 172.21.15.2:0/145768745 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f43a00a0640 0x7f43a00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:16.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.431+0000 7f43b4ff9700 1 -- 172.21.15.2:0/145768745 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f43b00cba60 con 0x7f43b81069a0 2023-09-13T04:38:16.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.431+0000 7f43b77fe700 1 --2- 172.21.15.2:0/145768745 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f43a00a0640 0x7f43a00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:16.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.431+0000 7f43b77fe700 1 --2- 172.21.15.2:0/145768745 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f43a00a0640 0x7f43a00a2b00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f43b819a110 tx=0x7f43a8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:16.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.435+0000 7f43b4ff9700 1 -- 172.21.15.2:0/145768745 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f43b00953d0 con 0x7f43b81069a0 2023-09-13T04:38:16.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.648+0000 7f43bddff700 1 -- 172.21.15.2:0/145768745 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f43b802cf70 con 0x7f43b81069a0 2023-09-13T04:38:16.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.649+0000 7f43b4ff9700 1 -- 172.21.15.2:0/145768745 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f43b00989f0 con 0x7f43b81069a0 2023-09-13T04:38:16.650 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-09-13T04:38:16.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.652+0000 7f43bddff700 1 -- 172.21.15.2:0/145768745 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f43a00a0640 msgr2=0x7f43a00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:16.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.652+0000 7f43bddff700 1 --2- 172.21.15.2:0/145768745 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f43a00a0640 0x7f43a00a2b00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f43b819a110 tx=0x7f43a8009260 comp rx=0 tx=0).stop 2023-09-13T04:38:16.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.652+0000 7f43bddff700 1 -- 172.21.15.2:0/145768745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43b81069a0 msgr2=0x7f43b81a01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:16.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.652+0000 7f43bddff700 1 --2- 172.21.15.2:0/145768745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43b81069a0 0x7f43b81a01b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f43b000ef30 tx=0x7f43b000c3a0 comp rx=0 tx=0).stop 2023-09-13T04:38:16.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.652+0000 7f43bddff700 1 -- 172.21.15.2:0/145768745 shutdown_connections 2023-09-13T04:38:16.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.652+0000 7f43bddff700 1 --2- 172.21.15.2:0/145768745 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f43a00a0640 0x7f43a00a2b00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:16.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.652+0000 7f43bddff700 1 --2- 172.21.15.2:0/145768745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43b81069a0 0x7f43b81a01b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:16.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.652+0000 7f43bddff700 1 --2- 172.21.15.2:0/145768745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43b8101740 0x7f43b8198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:16.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.652+0000 7f43bddff700 1 --2- 172.21.15.2:0/145768745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43b8104070 0x7f43b8199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:16.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.652+0000 7f43bddff700 1 -- 172.21.15.2:0/145768745 >> 172.21.15.2:0/145768745 conn(0x7f43b80fb130 msgr2=0x7f43b80ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:16.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.652+0000 7f43bddff700 1 -- 172.21.15.2:0/145768745 shutdown_connections 2023-09-13T04:38:16.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.652+0000 7f43bddff700 1 -- 172.21.15.2:0/145768745 wait complete. 2023-09-13T04:38:16.670 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-13T04:38:16.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.877+0000 7f95cb5ff700 1 -- 172.21.15.2:0/4118966095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95c4100ce0 msgr2=0x7f95c4101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:16.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.877+0000 7f95cb5ff700 1 --2- 172.21.15.2:0/4118966095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95c4100ce0 0x7f95c4101100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f95c00099c0 tx=0x7f95c0009cd0 comp rx=0 tx=0).stop 2023-09-13T04:38:16.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.877+0000 7f95cb5ff700 1 -- 172.21.15.2:0/4118966095 shutdown_connections 2023-09-13T04:38:16.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.877+0000 7f95cb5ff700 1 --2- 172.21.15.2:0/4118966095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95c4102870 0x7f95c4109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:16.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.877+0000 7f95cb5ff700 1 --2- 172.21.15.2:0/4118966095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95c4100ce0 0x7f95c4101100 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:16.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.877+0000 7f95cb5ff700 1 --2- 172.21.15.2:0/4118966095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95c4101ed0 0x7f95c4102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:16.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.877+0000 7f95cb5ff700 1 -- 172.21.15.2:0/4118966095 >> 172.21.15.2:0/4118966095 conn(0x7f95c40fc230 msgr2=0x7f95c40fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:16.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.877+0000 7f95cb5ff700 1 -- 172.21.15.2:0/4118966095 shutdown_connections 2023-09-13T04:38:16.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.877+0000 7f95cb5ff700 1 -- 172.21.15.2:0/4118966095 wait complete. 2023-09-13T04:38:16.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.878+0000 7f95cb5ff700 1 Processor -- start 2023-09-13T04:38:16.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.878+0000 7f95cb5ff700 1 -- start start 2023-09-13T04:38:16.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.878+0000 7f95cb5ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95c4100ce0 0x7f95c41969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:16.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.878+0000 7f95cb5ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95c4101ed0 0x7f95c4196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:16.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.879+0000 7f95cb5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95c4102870 0x7f95c419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:16.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.879+0000 7f95cb5ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c419e560 con 0x7f95c4101ed0 2023-09-13T04:38:16.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.879+0000 7f95cb5ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c419e6a0 con 0x7f95c4100ce0 2023-09-13T04:38:16.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.879+0000 7f95cb5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c419e830 con 0x7f95c4102870 2023-09-13T04:38:16.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.879+0000 7f95c8b9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95c4101ed0 0x7f95c4196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:16.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.879+0000 7f95c8b9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95c4101ed0 0x7f95c4196f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43482/0 (socket says 172.21.15.2:43482) 2023-09-13T04:38:16.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.879+0000 7f95c8b9a700 1 -- 172.21.15.2:0/2171609116 learned_addr learned my addr 172.21.15.2:0/2171609116 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:16.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.879+0000 7f95c939b700 1 --2- 172.21.15.2:0/2171609116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95c4100ce0 0x7f95c41969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:16.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.879+0000 7f95c9b9c700 1 --2- 172.21.15.2:0/2171609116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95c4102870 0x7f95c419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:16.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.879+0000 7f95c8b9a700 1 -- 172.21.15.2:0/2171609116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95c4102870 msgr2=0x7f95c419df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:16.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.879+0000 7f95c8b9a700 1 --2- 172.21.15.2:0/2171609116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95c4102870 0x7f95c419df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:16.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.879+0000 7f95c8b9a700 1 -- 172.21.15.2:0/2171609116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95c4100ce0 msgr2=0x7f95c41969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:16.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.879+0000 7f95c8b9a700 1 --2- 172.21.15.2:0/2171609116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95c4100ce0 0x7f95c41969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:16.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.879+0000 7f95c8b9a700 1 -- 172.21.15.2:0/2171609116 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b4009520 con 0x7f95c4101ed0 2023-09-13T04:38:16.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.880+0000 7f95c8b9a700 1 --2- 172.21.15.2:0/2171609116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95c4101ed0 0x7f95c4196f20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f95b400eb30 tx=0x7f95b400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:16.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.880+0000 7f95ba7fc700 1 -- 172.21.15.2:0/2171609116 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b4009880 con 0x7f95c4101ed0 2023-09-13T04:38:16.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.880+0000 7f95ba7fc700 1 -- 172.21.15.2:0/2171609116 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95b40044e0 con 0x7f95c4101ed0 2023-09-13T04:38:16.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.880+0000 7f95cb5ff700 1 -- 172.21.15.2:0/2171609116 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c00095f0 con 0x7f95c4101ed0 2023-09-13T04:38:16.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.880+0000 7f95ba7fc700 1 -- 172.21.15.2:0/2171609116 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b4003d80 con 0x7f95c4101ed0 2023-09-13T04:38:16.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.880+0000 7f95cb5ff700 1 -- 172.21.15.2:0/2171609116 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c419eee0 con 0x7f95c4101ed0 2023-09-13T04:38:16.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.881+0000 7f95cb5ff700 1 -- 172.21.15.2:0/2171609116 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c4107c60 con 0x7f95c4101ed0 2023-09-13T04:38:16.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:16 smithi149 ceph-mon[133315]: pgmap v284: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 90 B/s wr, 0 op/s 2023-09-13T04:38:16.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:16 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/410464484' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:16.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.883+0000 7f95ba7fc700 1 -- 172.21.15.2:0/2171609116 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f95b4003f00 con 0x7f95c4101ed0 2023-09-13T04:38:16.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.884+0000 7f95ba7fc700 1 --2- 172.21.15.2:0/2171609116 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f95b00a06c0 0x7f95b00a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:16.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.884+0000 7f95ba7fc700 1 -- 172.21.15.2:0/2171609116 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f95b4014070 con 0x7f95c4101ed0 2023-09-13T04:38:16.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.885+0000 7f95c939b700 1 --2- 172.21.15.2:0/2171609116 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f95b00a06c0 0x7f95b00a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:16.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.885+0000 7f95c939b700 1 --2- 172.21.15.2:0/2171609116 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f95b00a06c0 0x7f95b00a2b80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009990 tx=0x7f95c0005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:16.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:16.886+0000 7f95ba7fc700 1 -- 172.21.15.2:0/2171609116 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f95b4099780 con 0x7f95c4101ed0 2023-09-13T04:38:16.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:16 smithi156 ceph-mon[133675]: pgmap v284: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 90 B/s wr, 0 op/s 2023-09-13T04:38:16.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:16 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/410464484' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:17.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:16 smithi002 ceph-mon[97651]: pgmap v284: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 90 B/s wr, 0 op/s 2023-09-13T04:38:17.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:16 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/410464484' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:17.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.097+0000 7f95cb5ff700 1 -- 172.21.15.2:0/2171609116 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f95c410c700 con 0x7f95c4101ed0 2023-09-13T04:38:17.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.098+0000 7f95ba7fc700 1 -- 172.21.15.2:0/2171609116 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f95b400fe60 con 0x7f95c4101ed0 2023-09-13T04:38:17.099 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-09-13T04:38:17.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.100+0000 7f95cb5ff700 1 -- 172.21.15.2:0/2171609116 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f95b00a06c0 msgr2=0x7f95b00a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:17.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.100+0000 7f95cb5ff700 1 --2- 172.21.15.2:0/2171609116 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f95b00a06c0 0x7f95b00a2b80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009990 tx=0x7f95c0005fb0 comp rx=0 tx=0).stop 2023-09-13T04:38:17.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.100+0000 7f95cb5ff700 1 -- 172.21.15.2:0/2171609116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95c4101ed0 msgr2=0x7f95c4196f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:17.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.100+0000 7f95cb5ff700 1 --2- 172.21.15.2:0/2171609116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95c4101ed0 0x7f95c4196f20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f95b400eb30 tx=0x7f95b400ee40 comp rx=0 tx=0).stop 2023-09-13T04:38:17.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.100+0000 7f95cb5ff700 1 -- 172.21.15.2:0/2171609116 shutdown_connections 2023-09-13T04:38:17.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.100+0000 7f95cb5ff700 1 --2- 172.21.15.2:0/2171609116 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f95b00a06c0 0x7f95b00a2b80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:17.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.100+0000 7f95cb5ff700 1 --2- 172.21.15.2:0/2171609116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95c4102870 0x7f95c419df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:17.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.100+0000 7f95cb5ff700 1 --2- 172.21.15.2:0/2171609116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95c4100ce0 0x7f95c41969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:17.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.100+0000 7f95cb5ff700 1 --2- 172.21.15.2:0/2171609116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95c4101ed0 0x7f95c4196f20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:17.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.100+0000 7f95cb5ff700 1 -- 172.21.15.2:0/2171609116 >> 172.21.15.2:0/2171609116 conn(0x7f95c40fc230 msgr2=0x7f95c40fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:17.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.100+0000 7f95cb5ff700 1 -- 172.21.15.2:0/2171609116 shutdown_connections 2023-09-13T04:38:17.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.100+0000 7f95cb5ff700 1 -- 172.21.15.2:0/2171609116 wait complete. 2023-09-13T04:38:17.119 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-13T04:38:17.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.327+0000 7f942ddff700 1 -- 172.21.15.2:0/3953211373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9428101ed0 msgr2=0x7f9428102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:17.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.327+0000 7f942ddff700 1 --2- 172.21.15.2:0/3953211373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9428101ed0 0x7f9428102330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f941c009990 tx=0x7f941c009ca0 comp rx=0 tx=0).stop 2023-09-13T04:38:17.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.327+0000 7f942ddff700 1 -- 172.21.15.2:0/3953211373 shutdown_connections 2023-09-13T04:38:17.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.327+0000 7f942ddff700 1 --2- 172.21.15.2:0/3953211373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9428102870 0x7f9428109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:17.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.327+0000 7f942ddff700 1 --2- 172.21.15.2:0/3953211373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9428100ce0 0x7f9428101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:17.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.327+0000 7f942ddff700 1 --2- 172.21.15.2:0/3953211373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9428101ed0 0x7f9428102330 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:17.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.327+0000 7f942ddff700 1 -- 172.21.15.2:0/3953211373 >> 172.21.15.2:0/3953211373 conn(0x7f94280fc230 msgr2=0x7f94280fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:17.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.327+0000 7f942ddff700 1 -- 172.21.15.2:0/3953211373 shutdown_connections 2023-09-13T04:38:17.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.327+0000 7f942ddff700 1 -- 172.21.15.2:0/3953211373 wait complete. 2023-09-13T04:38:17.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.328+0000 7f942ddff700 1 Processor -- start 2023-09-13T04:38:17.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.328+0000 7f942ddff700 1 -- start start 2023-09-13T04:38:17.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.329+0000 7f942ddff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9428100ce0 0x7f9428196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:17.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.329+0000 7f942ddff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9428101ed0 0x7f9428196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:17.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.329+0000 7f942ddff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9428102870 0x7f942819df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:17.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.329+0000 7f942ddff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942819e510 con 0x7f9428101ed0 2023-09-13T04:38:17.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.329+0000 7f942ddff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942819e650 con 0x7f9428100ce0 2023-09-13T04:38:17.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.329+0000 7f942ddff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942819e7e0 con 0x7f9428102870 2023-09-13T04:38:17.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.329+0000 7f9426ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9428101ed0 0x7f9428196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:17.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.329+0000 7f9426ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9428101ed0 0x7f9428196ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43492/0 (socket says 172.21.15.2:43492) 2023-09-13T04:38:17.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.329+0000 7f9426ffd700 1 -- 172.21.15.2:0/1235453044 learned_addr learned my addr 172.21.15.2:0/1235453044 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:17.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.329+0000 7f94277fe700 1 --2- 172.21.15.2:0/1235453044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9428100ce0 0x7f9428196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:17.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.329+0000 7f9427fff700 1 --2- 172.21.15.2:0/1235453044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9428102870 0x7f942819df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:17.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.330+0000 7f9426ffd700 1 -- 172.21.15.2:0/1235453044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9428102870 msgr2=0x7f942819df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:17.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.330+0000 7f9426ffd700 1 --2- 172.21.15.2:0/1235453044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9428102870 0x7f942819df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:17.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.330+0000 7f9426ffd700 1 -- 172.21.15.2:0/1235453044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9428100ce0 msgr2=0x7f9428196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:17.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.330+0000 7f9426ffd700 1 --2- 172.21.15.2:0/1235453044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9428100ce0 0x7f9428196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:17.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.330+0000 7f9426ffd700 1 -- 172.21.15.2:0/1235453044 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c0095f0 con 0x7f9428101ed0 2023-09-13T04:38:17.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.330+0000 7f94277fe700 1 --2- 172.21.15.2:0/1235453044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9428100ce0 0x7f9428196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:38:17.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.330+0000 7f9426ffd700 1 --2- 172.21.15.2:0/1235453044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9428101ed0 0x7f9428196ed0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f941c004740 tx=0x7f941c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:17.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.330+0000 7f9424ff9700 1 -- 172.21.15.2:0/1235453044 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c01d070 con 0x7f9428101ed0 2023-09-13T04:38:17.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.330+0000 7f942ddff700 1 -- 172.21.15.2:0/1235453044 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942819ea60 con 0x7f9428101ed0 2023-09-13T04:38:17.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.330+0000 7f9424ff9700 1 -- 172.21.15.2:0/1235453044 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f941c004830 con 0x7f9428101ed0 2023-09-13T04:38:17.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.330+0000 7f9424ff9700 1 -- 172.21.15.2:0/1235453044 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c005450 con 0x7f9428101ed0 2023-09-13T04:38:17.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.330+0000 7f942ddff700 1 -- 172.21.15.2:0/1235453044 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942819ee80 con 0x7f9428101ed0 2023-09-13T04:38:17.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.331+0000 7f942ddff700 1 -- 172.21.15.2:0/1235453044 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f942810c540 con 0x7f9428101ed0 2023-09-13T04:38:17.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.333+0000 7f9424ff9700 1 -- 172.21.15.2:0/1235453044 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f941c00fb00 con 0x7f9428101ed0 2023-09-13T04:38:17.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.335+0000 7f9424ff9700 1 --2- 172.21.15.2:0/1235453044 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f94100a0640 0x7f94100a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:17.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.335+0000 7f9424ff9700 1 -- 172.21.15.2:0/1235453044 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f941c0ccfe0 con 0x7f9428101ed0 2023-09-13T04:38:17.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.335+0000 7f94277fe700 1 --2- 172.21.15.2:0/1235453044 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f94100a0640 0x7f94100a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:17.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.336+0000 7f94277fe700 1 --2- 172.21.15.2:0/1235453044 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f94100a0640 0x7f94100a2b00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9428101d30 tx=0x7f9418009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:17.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.339+0000 7f9424ff9700 1 -- 172.21.15.2:0/1235453044 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f941c05ae10 con 0x7f9428101ed0 2023-09-13T04:38:17.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.550+0000 7f942ddff700 1 -- 172.21.15.2:0/1235453044 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9428107c00 con 0x7f9428101ed0 2023-09-13T04:38:17.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.551+0000 7f9424ff9700 1 -- 172.21.15.2:0/1235453044 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f941c096990 con 0x7f9428101ed0 2023-09-13T04:38:17.551 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-09-13T04:38:17.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.553+0000 7f942ddff700 1 -- 172.21.15.2:0/1235453044 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f94100a0640 msgr2=0x7f94100a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:17.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.553+0000 7f942ddff700 1 --2- 172.21.15.2:0/1235453044 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f94100a0640 0x7f94100a2b00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9428101d30 tx=0x7f9418009260 comp rx=0 tx=0).stop 2023-09-13T04:38:17.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.553+0000 7f942ddff700 1 -- 172.21.15.2:0/1235453044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9428101ed0 msgr2=0x7f9428196ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:17.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.553+0000 7f942ddff700 1 --2- 172.21.15.2:0/1235453044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9428101ed0 0x7f9428196ed0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f941c004740 tx=0x7f941c009580 comp rx=0 tx=0).stop 2023-09-13T04:38:17.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.553+0000 7f942ddff700 1 -- 172.21.15.2:0/1235453044 shutdown_connections 2023-09-13T04:38:17.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.553+0000 7f942ddff700 1 --2- 172.21.15.2:0/1235453044 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f94100a0640 0x7f94100a2b00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:17.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.553+0000 7f942ddff700 1 --2- 172.21.15.2:0/1235453044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9428102870 0x7f942819df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:17.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.553+0000 7f942ddff700 1 --2- 172.21.15.2:0/1235453044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9428100ce0 0x7f9428196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:17.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.553+0000 7f942ddff700 1 --2- 172.21.15.2:0/1235453044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9428101ed0 0x7f9428196ed0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:17.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.553+0000 7f942ddff700 1 -- 172.21.15.2:0/1235453044 >> 172.21.15.2:0/1235453044 conn(0x7f94280fc230 msgr2=0x7f94280fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:17.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.553+0000 7f942ddff700 1 -- 172.21.15.2:0/1235453044 shutdown_connections 2023-09-13T04:38:17.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.553+0000 7f942ddff700 1 -- 172.21.15.2:0/1235453044 wait complete. 2023-09-13T04:38:17.575 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-13T04:38:17.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.782+0000 7f04e51ff700 1 -- 172.21.15.2:0/404663794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04e0100ce0 msgr2=0x7f04e0101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:17.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.782+0000 7f04e51ff700 1 --2- 172.21.15.2:0/404663794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04e0100ce0 0x7f04e0101100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f04d00098f0 tx=0x7f04d0009c00 comp rx=0 tx=0).stop 2023-09-13T04:38:17.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.782+0000 7f04e51ff700 1 -- 172.21.15.2:0/404663794 shutdown_connections 2023-09-13T04:38:17.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.782+0000 7f04e51ff700 1 --2- 172.21.15.2:0/404663794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04e0102870 0x7f04e0109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:17.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.782+0000 7f04e51ff700 1 --2- 172.21.15.2:0/404663794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04e0100ce0 0x7f04e0101100 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:17.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.782+0000 7f04e51ff700 1 --2- 172.21.15.2:0/404663794 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04e0101ed0 0x7f04e0102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:17.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.782+0000 7f04e51ff700 1 -- 172.21.15.2:0/404663794 >> 172.21.15.2:0/404663794 conn(0x7f04e00fc230 msgr2=0x7f04e00fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:17.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.782+0000 7f04e51ff700 1 -- 172.21.15.2:0/404663794 shutdown_connections 2023-09-13T04:38:17.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.783+0000 7f04e51ff700 1 -- 172.21.15.2:0/404663794 wait complete. 2023-09-13T04:38:17.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.783+0000 7f04e51ff700 1 Processor -- start 2023-09-13T04:38:17.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.784+0000 7f04e51ff700 1 -- start start 2023-09-13T04:38:17.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.784+0000 7f04e51ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04e0100ce0 0x7f04e0074a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:17.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.784+0000 7f04e51ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04e0101ed0 0x7f04e00730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:17.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.784+0000 7f04e51ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04e0102870 0x7f04e00735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:17.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.784+0000 7f04e51ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e0073d80 con 0x7f04e0101ed0 2023-09-13T04:38:17.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.784+0000 7f04e51ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e0073ef0 con 0x7f04e0100ce0 2023-09-13T04:38:17.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.784+0000 7f04e51ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e01a0b40 con 0x7f04e0102870 2023-09-13T04:38:17.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.784+0000 7f04de59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04e0101ed0 0x7f04e00730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:17.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.784+0000 7f04de59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04e0101ed0 0x7f04e00730a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43510/0 (socket says 172.21.15.2:43510) 2023-09-13T04:38:17.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.784+0000 7f04de59c700 1 -- 172.21.15.2:0/3582019791 learned_addr learned my addr 172.21.15.2:0/3582019791 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:17.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.784+0000 7f04ded9d700 1 --2- 172.21.15.2:0/3582019791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04e0100ce0 0x7f04e0074a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:17.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.784+0000 7f04df59e700 1 --2- 172.21.15.2:0/3582019791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04e0102870 0x7f04e00735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:17.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.785+0000 7f04de59c700 1 -- 172.21.15.2:0/3582019791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04e0102870 msgr2=0x7f04e00735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:17.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.785+0000 7f04de59c700 1 --2- 172.21.15.2:0/3582019791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04e0102870 0x7f04e00735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:17.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.785+0000 7f04de59c700 1 -- 172.21.15.2:0/3582019791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04e0100ce0 msgr2=0x7f04e0074a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:17.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.785+0000 7f04de59c700 1 --2- 172.21.15.2:0/3582019791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04e0100ce0 0x7f04e0074a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:17.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.785+0000 7f04de59c700 1 -- 172.21.15.2:0/3582019791 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d40095f0 con 0x7f04e0101ed0 2023-09-13T04:38:17.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.785+0000 7f04df59e700 1 --2- 172.21.15.2:0/3582019791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04e0102870 0x7f04e00735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:38:17.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.785+0000 7f04ded9d700 1 --2- 172.21.15.2:0/3582019791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04e0100ce0 0x7f04e0074a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:38:17.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.785+0000 7f04de59c700 1 --2- 172.21.15.2:0/3582019791 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04e0101ed0 0x7f04e00730a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f04d400ec10 tx=0x7f04d400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:17.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.785+0000 7f04c7fff700 1 -- 172.21.15.2:0/3582019791 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d400cd40 con 0x7f04e0101ed0 2023-09-13T04:38:17.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.785+0000 7f04e51ff700 1 -- 172.21.15.2:0/3582019791 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d0009520 con 0x7f04e0101ed0 2023-09-13T04:38:17.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.785+0000 7f04c7fff700 1 -- 172.21.15.2:0/3582019791 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04d40044e0 con 0x7f04e0101ed0 2023-09-13T04:38:17.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.785+0000 7f04c7fff700 1 -- 172.21.15.2:0/3582019791 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d4010690 con 0x7f04e0101ed0 2023-09-13T04:38:17.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.785+0000 7f04e51ff700 1 -- 172.21.15.2:0/3582019791 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e01a11a0 con 0x7f04e0101ed0 2023-09-13T04:38:17.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.786+0000 7f04e51ff700 1 -- 172.21.15.2:0/3582019791 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e010c540 con 0x7f04e0101ed0 2023-09-13T04:38:17.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.788+0000 7f04c7fff700 1 -- 172.21.15.2:0/3582019791 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f04d4003710 con 0x7f04e0101ed0 2023-09-13T04:38:17.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.790+0000 7f04c7fff700 1 --2- 172.21.15.2:0/3582019791 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f04c80a06c0 0x7f04c80a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:17.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.790+0000 7f04c7fff700 1 -- 172.21.15.2:0/3582019791 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f04d4014070 con 0x7f04e0101ed0 2023-09-13T04:38:17.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.790+0000 7f04ded9d700 1 --2- 172.21.15.2:0/3582019791 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f04c80a06c0 0x7f04c80a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:17.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.790+0000 7f04ded9d700 1 --2- 172.21.15.2:0/3582019791 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f04c80a06c0 0x7f04c80a2b80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009fd0 tx=0x7f04d0019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:17.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:17.793+0000 7f04c7fff700 1 -- 172.21.15.2:0/3582019791 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f04d4095c10 con 0x7f04e0101ed0 2023-09-13T04:38:17.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:17 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/145768745' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:38:17.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:17 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2171609116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:38:17.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:17 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1235453044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:38:17.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:17 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/145768745' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:38:17.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:17 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2171609116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:38:17.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:17 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1235453044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:38:18.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:17 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/145768745' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:38:18.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:17 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2171609116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:38:18.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:17 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1235453044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:38:18.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.007+0000 7f04e51ff700 1 -- 172.21.15.2:0/3582019791 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f04e0107cc0 con 0x7f04e0101ed0 2023-09-13T04:38:18.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.008+0000 7f04c7fff700 1 -- 172.21.15.2:0/3582019791 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f04d4099230 con 0x7f04e0101ed0 2023-09-13T04:38:18.008 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-09-13T04:38:18.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.010+0000 7f04e51ff700 1 -- 172.21.15.2:0/3582019791 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f04c80a06c0 msgr2=0x7f04c80a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:18.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.010+0000 7f04e51ff700 1 --2- 172.21.15.2:0/3582019791 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f04c80a06c0 0x7f04c80a2b80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009fd0 tx=0x7f04d0019040 comp rx=0 tx=0).stop 2023-09-13T04:38:18.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.010+0000 7f04e51ff700 1 -- 172.21.15.2:0/3582019791 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04e0101ed0 msgr2=0x7f04e00730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:18.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.010+0000 7f04e51ff700 1 --2- 172.21.15.2:0/3582019791 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04e0101ed0 0x7f04e00730a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f04d400ec10 tx=0x7f04d400ef20 comp rx=0 tx=0).stop 2023-09-13T04:38:18.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.011+0000 7f04e51ff700 1 -- 172.21.15.2:0/3582019791 shutdown_connections 2023-09-13T04:38:18.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.011+0000 7f04e51ff700 1 --2- 172.21.15.2:0/3582019791 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f04c80a06c0 0x7f04c80a2b80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:18.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.011+0000 7f04e51ff700 1 --2- 172.21.15.2:0/3582019791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04e0102870 0x7f04e00735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:18.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.011+0000 7f04e51ff700 1 --2- 172.21.15.2:0/3582019791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04e0100ce0 0x7f04e0074a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:18.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.011+0000 7f04e51ff700 1 --2- 172.21.15.2:0/3582019791 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04e0101ed0 0x7f04e00730a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:18.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.011+0000 7f04e51ff700 1 -- 172.21.15.2:0/3582019791 >> 172.21.15.2:0/3582019791 conn(0x7f04e00fc230 msgr2=0x7f04e00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:18.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.011+0000 7f04e51ff700 1 -- 172.21.15.2:0/3582019791 shutdown_connections 2023-09-13T04:38:18.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.011+0000 7f04e51ff700 1 -- 172.21.15.2:0/3582019791 wait complete. 2023-09-13T04:38:18.030 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-13T04:38:18.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.235+0000 7fb66c1ff700 1 -- 172.21.15.2:0/1486659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb664101ed0 msgr2=0x7fb664102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:18.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.235+0000 7fb66c1ff700 1 --2- 172.21.15.2:0/1486659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb664101ed0 0x7fb664102330 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb6540099c0 tx=0x7fb654009cd0 comp rx=0 tx=0).stop 2023-09-13T04:38:18.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.235+0000 7fb66c1ff700 1 -- 172.21.15.2:0/1486659 shutdown_connections 2023-09-13T04:38:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.235+0000 7fb66c1ff700 1 --2- 172.21.15.2:0/1486659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb664102870 0x7fb664109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.235+0000 7fb66c1ff700 1 --2- 172.21.15.2:0/1486659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb664100ce0 0x7fb664101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.235+0000 7fb66c1ff700 1 --2- 172.21.15.2:0/1486659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb664101ed0 0x7fb664102330 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:18.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.235+0000 7fb66c1ff700 1 -- 172.21.15.2:0/1486659 >> 172.21.15.2:0/1486659 conn(0x7fb6640fc230 msgr2=0x7fb6640fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:18.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.235+0000 7fb66c1ff700 1 -- 172.21.15.2:0/1486659 shutdown_connections 2023-09-13T04:38:18.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.235+0000 7fb66c1ff700 1 -- 172.21.15.2:0/1486659 wait complete. 2023-09-13T04:38:18.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.236+0000 7fb66c1ff700 1 Processor -- start 2023-09-13T04:38:18.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.236+0000 7fb66c1ff700 1 -- start start 2023-09-13T04:38:18.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.237+0000 7fb66c1ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb664100ce0 0x7fb664198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:18.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.237+0000 7fb66c1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb664101ed0 0x7fb664199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:18.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.237+0000 7fb66c1ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb664102870 0x7fb6641a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:18.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.237+0000 7fb66c1ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6641a0770 con 0x7fb664102870 2023-09-13T04:38:18.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.237+0000 7fb66c1ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6641a08e0 con 0x7fb664100ce0 2023-09-13T04:38:18.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.237+0000 7fb66c1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6641a0aa0 con 0x7fb664101ed0 2023-09-13T04:38:18.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.237+0000 7fb66a79c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb664102870 0x7fb6641a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:18.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.237+0000 7fb66a79c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb664102870 0x7fb6641a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43524/0 (socket says 172.21.15.2:43524) 2023-09-13T04:38:18.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.237+0000 7fb66a79c700 1 -- 172.21.15.2:0/25341420 learned_addr learned my addr 172.21.15.2:0/25341420 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:18.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.237+0000 7fb669f9b700 1 --2- 172.21.15.2:0/25341420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb664100ce0 0x7fb664198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:18.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.237+0000 7fb66979a700 1 --2- 172.21.15.2:0/25341420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb664101ed0 0x7fb664199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:18.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.237+0000 7fb66a79c700 1 -- 172.21.15.2:0/25341420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb664101ed0 msgr2=0x7fb664199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:18.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.237+0000 7fb66a79c700 1 --2- 172.21.15.2:0/25341420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb664101ed0 0x7fb664199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:18.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.237+0000 7fb66a79c700 1 -- 172.21.15.2:0/25341420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb664100ce0 msgr2=0x7fb664198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:18.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.237+0000 7fb66a79c700 1 --2- 172.21.15.2:0/25341420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb664100ce0 0x7fb664198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:18.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.237+0000 7fb66a79c700 1 -- 172.21.15.2:0/25341420 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6540095f0 con 0x7fb664102870 2023-09-13T04:38:18.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.237+0000 7fb669f9b700 1 --2- 172.21.15.2:0/25341420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb664100ce0 0x7fb664198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:38:18.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.238+0000 7fb66a79c700 1 --2- 172.21.15.2:0/25341420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb664102870 0x7fb6641a01a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb65c00e940 tx=0x7fb65c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:18.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.238+0000 7fb65affd700 1 -- 172.21.15.2:0/25341420 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c0097b0 con 0x7fb664102870 2023-09-13T04:38:18.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.238+0000 7fb65affd700 1 -- 172.21.15.2:0/25341420 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb65c005020 con 0x7fb664102870 2023-09-13T04:38:18.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.238+0000 7fb66c1ff700 1 -- 172.21.15.2:0/25341420 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6641a0d80 con 0x7fb664102870 2023-09-13T04:38:18.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.238+0000 7fb65affd700 1 -- 172.21.15.2:0/25341420 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c00cc40 con 0x7fb664102870 2023-09-13T04:38:18.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.238+0000 7fb66c1ff700 1 -- 172.21.15.2:0/25341420 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6641a1180 con 0x7fb664102870 2023-09-13T04:38:18.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.239+0000 7fb66c1ff700 1 -- 172.21.15.2:0/25341420 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb66410c540 con 0x7fb664102870 2023-09-13T04:38:18.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.241+0000 7fb65affd700 1 -- 172.21.15.2:0/25341420 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fb65c005400 con 0x7fb664102870 2023-09-13T04:38:18.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.243+0000 7fb65affd700 1 --2- 172.21.15.2:0/25341420 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb6500a0640 0x7fb6500a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:18.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.243+0000 7fb65affd700 1 -- 172.21.15.2:0/25341420 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb65c014070 con 0x7fb664102870 2023-09-13T04:38:18.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.243+0000 7fb669f9b700 1 --2- 172.21.15.2:0/25341420 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb6500a0640 0x7fb6500a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:18.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.244+0000 7fb669f9b700 1 --2- 172.21.15.2:0/25341420 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb6500a0640 0x7fb6500a2b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb66419a100 tx=0x7fb66000b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:18.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.247+0000 7fb65affd700 1 -- 172.21.15.2:0/25341420 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb65c0958c0 con 0x7fb664102870 2023-09-13T04:38:18.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.472+0000 7fb66c1ff700 1 -- 172.21.15.2:0/25341420 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb6641a13d0 con 0x7fb664102870 2023-09-13T04:38:18.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.473+0000 7fb65affd700 1 -- 172.21.15.2:0/25341420 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+581 (secure 0 0 0) 0x7fb65c005190 con 0x7fb664102870 2023-09-13T04:38:18.473 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/1393377368 2023-09-14T04:31:23.417691+0000 2023-09-13T04:38:18.474 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/2772434518 2023-09-14T04:30:54.963309+0000 2023-09-13T04:38:18.474 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3810974822 2023-09-14T04:31:23.417691+0000 2023-09-13T04:38:18.474 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/2772434518 2023-09-14T04:30:54.963309+0000 2023-09-13T04:38:18.474 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1907805835 2023-09-14T04:30:54.963309+0000 2023-09-13T04:38:18.474 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2811866029 2023-09-14T04:30:54.963309+0000 2023-09-13T04:38:18.475 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2538614772 2023-09-14T04:31:23.417691+0000 2023-09-13T04:38:18.475 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/1393377368 2023-09-14T04:31:23.417691+0000 2023-09-13T04:38:18.475 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3722465120 2023-09-14T04:31:23.417691+0000 2023-09-13T04:38:18.475 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/516951274 2023-09-14T04:30:54.963309+0000 2023-09-13T04:38:18.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.475+0000 7fb66c1ff700 1 -- 172.21.15.2:0/25341420 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb6500a0640 msgr2=0x7fb6500a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:18.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.475+0000 7fb66c1ff700 1 --2- 172.21.15.2:0/25341420 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb6500a0640 0x7fb6500a2b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb66419a100 tx=0x7fb66000b040 comp rx=0 tx=0).stop 2023-09-13T04:38:18.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.475+0000 7fb66c1ff700 1 -- 172.21.15.2:0/25341420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb664102870 msgr2=0x7fb6641a01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:18.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.475+0000 7fb66c1ff700 1 --2- 172.21.15.2:0/25341420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb664102870 0x7fb6641a01a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb65c00e940 tx=0x7fb65c00ec50 comp rx=0 tx=0).stop 2023-09-13T04:38:18.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.475+0000 7fb66c1ff700 1 -- 172.21.15.2:0/25341420 shutdown_connections 2023-09-13T04:38:18.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.475+0000 7fb66c1ff700 1 --2- 172.21.15.2:0/25341420 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb6500a0640 0x7fb6500a2b00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:18.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.475+0000 7fb66c1ff700 1 --2- 172.21.15.2:0/25341420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb664102870 0x7fb6641a01a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:18.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.475+0000 7fb66c1ff700 1 --2- 172.21.15.2:0/25341420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb664100ce0 0x7fb664198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:18.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.475+0000 7fb66c1ff700 1 --2- 172.21.15.2:0/25341420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb664101ed0 0x7fb664199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:18.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.475+0000 7fb66c1ff700 1 -- 172.21.15.2:0/25341420 >> 172.21.15.2:0/25341420 conn(0x7fb6640fc230 msgr2=0x7fb6640fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:18.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.475+0000 7fb66c1ff700 1 -- 172.21.15.2:0/25341420 shutdown_connections 2023-09-13T04:38:18.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.475+0000 7fb66c1ff700 1 -- 172.21.15.2:0/25341420 wait complete. 2023-09-13T04:38:18.479 INFO:teuthology.orchestra.run.smithi002.stderr:listed 10 entries 2023-09-13T04:38:18.494 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-09-13T04:38:18.495 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-13T04:38:18.539 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T04:38:18.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:18 smithi002 ceph-mon[97651]: pgmap v285: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-09-13T04:38:18.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:18 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3582019791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:38:18.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:18 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/25341420' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-13T04:38:18.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.782+0000 7fc10f3ff700 1 -- 172.21.15.2:0/2637137351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc108073710 msgr2=0x7fc108073b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:18.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.782+0000 7fc10f3ff700 1 --2- 172.21.15.2:0/2637137351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc108073710 0x7fc108073b90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009990 tx=0x7fc0f8009ca0 comp rx=0 tx=0).stop 2023-09-13T04:38:18.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.782+0000 7fc10f3ff700 1 -- 172.21.15.2:0/2637137351 shutdown_connections 2023-09-13T04:38:18.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.782+0000 7fc10f3ff700 1 --2- 172.21.15.2:0/2637137351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc108108bb0 0x7fc10810b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:18.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.782+0000 7fc10f3ff700 1 --2- 172.21.15.2:0/2637137351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc108074cf0 0x7fc108073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:18.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.782+0000 7fc10f3ff700 1 --2- 172.21.15.2:0/2637137351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc108073710 0x7fc108073b90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:18.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.782+0000 7fc10f3ff700 1 -- 172.21.15.2:0/2637137351 >> 172.21.15.2:0/2637137351 conn(0x7fc1080fc220 msgr2=0x7fc1080fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:18.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.782+0000 7fc10f3ff700 1 -- 172.21.15.2:0/2637137351 shutdown_connections 2023-09-13T04:38:18.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.782+0000 7fc10f3ff700 1 -- 172.21.15.2:0/2637137351 wait complete. 2023-09-13T04:38:18.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.783+0000 7fc10f3ff700 1 Processor -- start 2023-09-13T04:38:18.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.784+0000 7fc10f3ff700 1 -- start start 2023-09-13T04:38:18.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.784+0000 7fc10f3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc108073710 0x7fc108198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:18.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.784+0000 7fc10f3ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc108074cf0 0x7fc108199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:18.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.784+0000 7fc10f3ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc108108bb0 0x7fc1081a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:18.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.784+0000 7fc10f3ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1081a0760 con 0x7fc108108bb0 2023-09-13T04:38:18.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.784+0000 7fc10f3ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1081a08d0 con 0x7fc108074cf0 2023-09-13T04:38:18.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.784+0000 7fc10f3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1081a0a90 con 0x7fc108073710 2023-09-13T04:38:18.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.784+0000 7fc10d99c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc108108bb0 0x7fc1081a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:18.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.784+0000 7fc10d99c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc108108bb0 0x7fc1081a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43556/0 (socket says 172.21.15.2:43556) 2023-09-13T04:38:18.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.784+0000 7fc10d99c700 1 -- 172.21.15.2:0/3094162301 learned_addr learned my addr 172.21.15.2:0/3094162301 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:18.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.784+0000 7fc10c99a700 1 --2- 172.21.15.2:0/3094162301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc108074cf0 0x7fc108199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:18.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.784+0000 7fc10d19b700 1 --2- 172.21.15.2:0/3094162301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc108073710 0x7fc108198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:18.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.784+0000 7fc10d99c700 1 -- 172.21.15.2:0/3094162301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc108073710 msgr2=0x7fc108198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:18.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.784+0000 7fc10d99c700 1 --2- 172.21.15.2:0/3094162301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc108073710 0x7fc108198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:18.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.785+0000 7fc10d99c700 1 -- 172.21.15.2:0/3094162301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc108074cf0 msgr2=0x7fc108199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:18.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.785+0000 7fc10d99c700 1 --2- 172.21.15.2:0/3094162301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc108074cf0 0x7fc108199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:18.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.785+0000 7fc10d99c700 1 -- 172.21.15.2:0/3094162301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f80095f0 con 0x7fc108108bb0 2023-09-13T04:38:18.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.785+0000 7fc10c99a700 1 --2- 172.21.15.2:0/3094162301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc108074cf0 0x7fc108199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:38:18.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.785+0000 7fc10d99c700 1 --2- 172.21.15.2:0/3094162301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc108108bb0 0x7fc1081a0190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc10000ef30 tx=0x7fc10000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:18.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.785+0000 7fc0fe7fc700 1 -- 172.21.15.2:0/3094162301 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc100018070 con 0x7fc108108bb0 2023-09-13T04:38:18.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.785+0000 7fc0fe7fc700 1 -- 172.21.15.2:0/3094162301 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc10000f040 con 0x7fc108108bb0 2023-09-13T04:38:18.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.785+0000 7fc10f3ff700 1 -- 172.21.15.2:0/3094162301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1081a0d10 con 0x7fc108108bb0 2023-09-13T04:38:18.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.785+0000 7fc0fe7fc700 1 -- 172.21.15.2:0/3094162301 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10000cd00 con 0x7fc108108bb0 2023-09-13T04:38:18.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.785+0000 7fc10f3ff700 1 -- 172.21.15.2:0/3094162301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1081a11d0 con 0x7fc108108bb0 2023-09-13T04:38:18.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.786+0000 7fc10f3ff700 1 -- 172.21.15.2:0/3094162301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc108107260 con 0x7fc108108bb0 2023-09-13T04:38:18.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.788+0000 7fc0fe7fc700 1 -- 172.21.15.2:0/3094162301 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fc100003a00 con 0x7fc108108bb0 2023-09-13T04:38:18.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.790+0000 7fc0fe7fc700 1 --2- 172.21.15.2:0/3094162301 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc0f40a0710 0x7fc0f40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:18.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.790+0000 7fc0fe7fc700 1 -- 172.21.15.2:0/3094162301 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc1000cb8c0 con 0x7fc108108bb0 2023-09-13T04:38:18.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.790+0000 7fc10d19b700 1 --2- 172.21.15.2:0/3094162301 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc0f40a0710 0x7fc0f40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:18.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.790+0000 7fc10d19b700 1 --2- 172.21.15.2:0/3094162301 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc0f40a0710 0x7fc0f40a2bd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc10819a0f0 tx=0x7fc104009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:18.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:18.794+0000 7fc0fe7fc700 1 -- 172.21.15.2:0/3094162301 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc100095230 con 0x7fc108108bb0 2023-09-13T04:38:18.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:18 smithi149 ceph-mon[133315]: pgmap v285: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-09-13T04:38:18.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:18 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3582019791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:38:18.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:18 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/25341420' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-13T04:38:18.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:18 smithi156 ceph-mon[133675]: pgmap v285: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-09-13T04:38:18.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:18 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3582019791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:38:18.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:18 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/25341420' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-13T04:38:19.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.006+0000 7fc10f3ff700 1 -- 172.21.15.2:0/3094162301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc10810e730 con 0x7fc108108bb0 2023-09-13T04:38:19.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.008+0000 7fc0fe7fc700 1 -- 172.21.15.2:0/3094162301 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23927 (secure 0 0 0) 0x7fc100098850 con 0x7fc108108bb0 2023-09-13T04:38:19.009 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:38:19.009 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":88,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:38:04.132627+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T04:37:57.094802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T04:37:58.564487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-13T04:38:02.096895+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:38:19.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.010+0000 7fc10f3ff700 1 -- 172.21.15.2:0/3094162301 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc0f40a0710 msgr2=0x7fc0f40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:19.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.010+0000 7fc10f3ff700 1 --2- 172.21.15.2:0/3094162301 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc0f40a0710 0x7fc0f40a2bd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc10819a0f0 tx=0x7fc104009260 comp rx=0 tx=0).stop 2023-09-13T04:38:19.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.010+0000 7fc10f3ff700 1 -- 172.21.15.2:0/3094162301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc108108bb0 msgr2=0x7fc1081a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:19.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.010+0000 7fc10f3ff700 1 --2- 172.21.15.2:0/3094162301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc108108bb0 0x7fc1081a0190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc10000ef30 tx=0x7fc10000c3a0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.010+0000 7fc10f3ff700 1 -- 172.21.15.2:0/3094162301 shutdown_connections 2023-09-13T04:38:19.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.010+0000 7fc10f3ff700 1 --2- 172.21.15.2:0/3094162301 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc0f40a0710 0x7fc0f40a2bd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.010+0000 7fc10f3ff700 1 --2- 172.21.15.2:0/3094162301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc108108bb0 0x7fc1081a0190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.010+0000 7fc10f3ff700 1 --2- 172.21.15.2:0/3094162301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc108073710 0x7fc108198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.010+0000 7fc10f3ff700 1 --2- 172.21.15.2:0/3094162301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc108074cf0 0x7fc108199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.010+0000 7fc10f3ff700 1 -- 172.21.15.2:0/3094162301 >> 172.21.15.2:0/3094162301 conn(0x7fc1080fc220 msgr2=0x7fc1080fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:19.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.011+0000 7fc10f3ff700 1 -- 172.21.15.2:0/3094162301 shutdown_connections 2023-09-13T04:38:19.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.011+0000 7fc10f3ff700 1 -- 172.21.15.2:0/3094162301 wait complete. 2023-09-13T04:38:19.030 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T04:33:42.025714+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-13T04:37:57.094802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-13T04:37:58.564487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-13T04:38:02.096895+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-13T04:38:19.031 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-13T04:38:19.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.237+0000 7f21cbbff700 1 -- 172.21.15.2:0/693366156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21c40ff570 msgr2=0x7f21c40ff990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:19.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.237+0000 7f21cbbff700 1 --2- 172.21.15.2:0/693366156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21c40ff570 0x7f21c40ff990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f21c0009990 tx=0x7f21c0009ca0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.237+0000 7f21cbbff700 1 -- 172.21.15.2:0/693366156 shutdown_connections 2023-09-13T04:38:19.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.237+0000 7f21cbbff700 1 --2- 172.21.15.2:0/693366156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c4100890 0x7f21c4100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.237+0000 7f21cbbff700 1 --2- 172.21.15.2:0/693366156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21c40ff570 0x7f21c40ff990 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.237+0000 7f21cbbff700 1 --2- 172.21.15.2:0/693366156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21c40ffed0 0x7f21c4100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.237+0000 7f21cbbff700 1 -- 172.21.15.2:0/693366156 >> 172.21.15.2:0/693366156 conn(0x7f21c40fb170 msgr2=0x7f21c40fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:19.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.238+0000 7f21cbbff700 1 -- 172.21.15.2:0/693366156 shutdown_connections 2023-09-13T04:38:19.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.238+0000 7f21cbbff700 1 -- 172.21.15.2:0/693366156 wait complete. 2023-09-13T04:38:19.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.239+0000 7f21cbbff700 1 Processor -- start 2023-09-13T04:38:19.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.239+0000 7f21cbbff700 1 -- start start 2023-09-13T04:38:19.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.239+0000 7f21cbbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21c40ff570 0x7f21c4198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:19.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.239+0000 7f21cbbff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21c40ffed0 0x7f21c4199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:19.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.239+0000 7f21cbbff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c4100890 0x7f21c41a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:19.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.239+0000 7f21cbbff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c41a0710 con 0x7f21c40ffed0 2023-09-13T04:38:19.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.239+0000 7f21cbbff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c41a0850 con 0x7f21c4100890 2023-09-13T04:38:19.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.239+0000 7f21cbbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c41a09e0 con 0x7f21c40ff570 2023-09-13T04:38:19.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.239+0000 7f21c919a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21c40ffed0 0x7f21c4199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:19.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.239+0000 7f21c919a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21c40ffed0 0x7f21c4199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43576/0 (socket says 172.21.15.2:43576) 2023-09-13T04:38:19.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.239+0000 7f21c919a700 1 -- 172.21.15.2:0/2560446972 learned_addr learned my addr 172.21.15.2:0/2560446972 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:19.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.240+0000 7f21c999b700 1 --2- 172.21.15.2:0/2560446972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21c40ff570 0x7f21c4198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:19.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.240+0000 7f21ca19c700 1 --2- 172.21.15.2:0/2560446972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c4100890 0x7f21c41a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:19.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.240+0000 7f21c919a700 1 -- 172.21.15.2:0/2560446972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21c40ff570 msgr2=0x7f21c4198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:19.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.240+0000 7f21c919a700 1 --2- 172.21.15.2:0/2560446972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21c40ff570 0x7f21c4198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.240+0000 7f21c919a700 1 -- 172.21.15.2:0/2560446972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c4100890 msgr2=0x7f21c41a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:19.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.240+0000 7f21c919a700 1 --2- 172.21.15.2:0/2560446972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c4100890 0x7f21c41a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.240+0000 7f21c919a700 1 -- 172.21.15.2:0/2560446972 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c00095f0 con 0x7f21c40ffed0 2023-09-13T04:38:19.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.240+0000 7f21ca19c700 1 --2- 172.21.15.2:0/2560446972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c4100890 0x7f21c41a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:38:19.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.240+0000 7f21c919a700 1 --2- 172.21.15.2:0/2560446972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21c40ffed0 0x7f21c4199120 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f21b400e9a0 tx=0x7f21b400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:19.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.241+0000 7f21baffd700 1 -- 172.21.15.2:0/2560446972 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b4009790 con 0x7f21c40ffed0 2023-09-13T04:38:19.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.241+0000 7f21baffd700 1 -- 172.21.15.2:0/2560446972 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21b4010430 con 0x7f21c40ffed0 2023-09-13T04:38:19.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.241+0000 7f21baffd700 1 -- 172.21.15.2:0/2560446972 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b4018680 con 0x7f21c40ffed0 2023-09-13T04:38:19.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.241+0000 7f21cbbff700 1 -- 172.21.15.2:0/2560446972 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c41a0cc0 con 0x7f21c40ffed0 2023-09-13T04:38:19.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.241+0000 7f21cbbff700 1 -- 172.21.15.2:0/2560446972 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c41a11d0 con 0x7f21c40ffed0 2023-09-13T04:38:19.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.242+0000 7f21cbbff700 1 -- 172.21.15.2:0/2560446972 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c410a350 con 0x7f21c40ffed0 2023-09-13T04:38:19.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.244+0000 7f21baffd700 1 -- 172.21.15.2:0/2560446972 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f21b40105a0 con 0x7f21c40ffed0 2023-09-13T04:38:19.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.246+0000 7f21baffd700 1 --2- 172.21.15.2:0/2560446972 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f21b00a06c0 0x7f21b00a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:19.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.246+0000 7f21baffd700 1 -- 172.21.15.2:0/2560446972 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f21b4014070 con 0x7f21c40ffed0 2023-09-13T04:38:19.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.246+0000 7f21c999b700 1 --2- 172.21.15.2:0/2560446972 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f21b00a06c0 0x7f21b00a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:19.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.247+0000 7f21c999b700 1 --2- 172.21.15.2:0/2560446972 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f21b00a06c0 0x7f21b00a2b80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f21c000b680 tx=0x7f21c0005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:19.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.248+0000 7f21baffd700 1 -- 172.21.15.2:0/2560446972 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f21b4095a80 con 0x7f21c40ffed0 2023-09-13T04:38:19.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.457+0000 7f21cbbff700 1 -- 172.21.15.2:0/2560446972 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f21c410e870 con 0x7f21c40ffed0 2023-09-13T04:38:19.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.458+0000 7f21baffd700 1 -- 172.21.15.2:0/2560446972 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f21b4026020 con 0x7f21c40ffed0 2023-09-13T04:38:19.458 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-09-13T04:38:19.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.460+0000 7f21cbbff700 1 -- 172.21.15.2:0/2560446972 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f21b00a06c0 msgr2=0x7f21b00a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:19.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.460+0000 7f21cbbff700 1 --2- 172.21.15.2:0/2560446972 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f21b00a06c0 0x7f21b00a2b80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f21c000b680 tx=0x7f21c0005fd0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.460+0000 7f21cbbff700 1 -- 172.21.15.2:0/2560446972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21c40ffed0 msgr2=0x7f21c4199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:19.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.460+0000 7f21cbbff700 1 --2- 172.21.15.2:0/2560446972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21c40ffed0 0x7f21c4199120 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f21b400e9a0 tx=0x7f21b400ecb0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.460+0000 7f21cbbff700 1 -- 172.21.15.2:0/2560446972 shutdown_connections 2023-09-13T04:38:19.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.460+0000 7f21cbbff700 1 --2- 172.21.15.2:0/2560446972 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f21b00a06c0 0x7f21b00a2b80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.460+0000 7f21cbbff700 1 --2- 172.21.15.2:0/2560446972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c4100890 0x7f21c41a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.460+0000 7f21cbbff700 1 --2- 172.21.15.2:0/2560446972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21c40ff570 0x7f21c4198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.460+0000 7f21cbbff700 1 --2- 172.21.15.2:0/2560446972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21c40ffed0 0x7f21c4199120 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.460+0000 7f21cbbff700 1 -- 172.21.15.2:0/2560446972 >> 172.21.15.2:0/2560446972 conn(0x7f21c40fb170 msgr2=0x7f21c40fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:19.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.460+0000 7f21cbbff700 1 -- 172.21.15.2:0/2560446972 shutdown_connections 2023-09-13T04:38:19.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.460+0000 7f21cbbff700 1 -- 172.21.15.2:0/2560446972 wait complete. 2023-09-13T04:38:19.479 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-13T04:38:19.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.692+0000 7faef85ff700 1 -- 172.21.15.2:0/1464706355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faef0101ed0 msgr2=0x7faef0102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:19.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.692+0000 7faef85ff700 1 --2- 172.21.15.2:0/1464706355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faef0101ed0 0x7faef0102330 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faee00099c0 tx=0x7faee0009cd0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.692+0000 7faef85ff700 1 -- 172.21.15.2:0/1464706355 shutdown_connections 2023-09-13T04:38:19.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.692+0000 7faef85ff700 1 --2- 172.21.15.2:0/1464706355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faef0102870 0x7faef0109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.692+0000 7faef85ff700 1 --2- 172.21.15.2:0/1464706355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faef0100ce0 0x7faef0101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.692+0000 7faef85ff700 1 --2- 172.21.15.2:0/1464706355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faef0101ed0 0x7faef0102330 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.692+0000 7faef85ff700 1 -- 172.21.15.2:0/1464706355 >> 172.21.15.2:0/1464706355 conn(0x7faef00fc230 msgr2=0x7faef00fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:19.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.692+0000 7faef85ff700 1 -- 172.21.15.2:0/1464706355 shutdown_connections 2023-09-13T04:38:19.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.692+0000 7faef85ff700 1 -- 172.21.15.2:0/1464706355 wait complete. 2023-09-13T04:38:19.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.693+0000 7faef85ff700 1 Processor -- start 2023-09-13T04:38:19.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.693+0000 7faef85ff700 1 -- start start 2023-09-13T04:38:19.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.694+0000 7faef85ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faef0100ce0 0x7faef0198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:19.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.694+0000 7faef85ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faef0101ed0 0x7faef0199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:19.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.694+0000 7faef85ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faef0102870 0x7faef01a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:19.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.694+0000 7faef85ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef01a0790 con 0x7faef0101ed0 2023-09-13T04:38:19.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.694+0000 7faef85ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef01a0900 con 0x7faef0100ce0 2023-09-13T04:38:19.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.694+0000 7faef85ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef01a0ac0 con 0x7faef0102870 2023-09-13T04:38:19.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.694+0000 7faef5b9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faef0101ed0 0x7faef0199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:19.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.694+0000 7faef5b9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faef0101ed0 0x7faef0199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43590/0 (socket says 172.21.15.2:43590) 2023-09-13T04:38:19.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.694+0000 7faef639b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faef0100ce0 0x7faef0198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:19.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.694+0000 7faef5b9a700 1 -- 172.21.15.2:0/1578045064 learned_addr learned my addr 172.21.15.2:0/1578045064 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:19.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.694+0000 7faef6b9c700 1 --2- 172.21.15.2:0/1578045064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faef0102870 0x7faef01a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:19.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.694+0000 7faef5b9a700 1 -- 172.21.15.2:0/1578045064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faef0102870 msgr2=0x7faef01a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:19.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.694+0000 7faef5b9a700 1 --2- 172.21.15.2:0/1578045064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faef0102870 0x7faef01a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.694+0000 7faef5b9a700 1 -- 172.21.15.2:0/1578045064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faef0100ce0 msgr2=0x7faef0198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:19.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.694+0000 7faef5b9a700 1 --2- 172.21.15.2:0/1578045064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faef0100ce0 0x7faef0198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.694+0000 7faef5b9a700 1 -- 172.21.15.2:0/1578045064 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee8009c40 con 0x7faef0101ed0 2023-09-13T04:38:19.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.694+0000 7faef639b700 1 --2- 172.21.15.2:0/1578045064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faef0100ce0 0x7faef0198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:38:19.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.695+0000 7faef5b9a700 1 --2- 172.21.15.2:0/1578045064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faef0101ed0 0x7faef0199190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faee0009fd0 tx=0x7faee000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:19.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.695+0000 7faee77fe700 1 -- 172.21.15.2:0/1578045064 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee001d070 con 0x7faef0101ed0 2023-09-13T04:38:19.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.695+0000 7faef85ff700 1 -- 172.21.15.2:0/1578045064 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee00095f0 con 0x7faef0101ed0 2023-09-13T04:38:19.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.695+0000 7faee77fe700 1 -- 172.21.15.2:0/1578045064 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faee0005510 con 0x7faef0101ed0 2023-09-13T04:38:19.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.695+0000 7faee77fe700 1 -- 172.21.15.2:0/1578045064 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee0004a20 con 0x7faef0101ed0 2023-09-13T04:38:19.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.695+0000 7faef85ff700 1 -- 172.21.15.2:0/1578045064 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef01a0fd0 con 0x7faef0101ed0 2023-09-13T04:38:19.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.696+0000 7faef85ff700 1 -- 172.21.15.2:0/1578045064 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef010c540 con 0x7faef0101ed0 2023-09-13T04:38:19.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.698+0000 7faee77fe700 1 -- 172.21.15.2:0/1578045064 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7faee0004c20 con 0x7faef0101ed0 2023-09-13T04:38:19.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.699+0000 7faee77fe700 1 --2- 172.21.15.2:0/1578045064 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7faedc0a0640 0x7faedc0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:19.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.700+0000 7faee77fe700 1 -- 172.21.15.2:0/1578045064 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7faee00ccb90 con 0x7faef0101ed0 2023-09-13T04:38:19.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.700+0000 7faef639b700 1 --2- 172.21.15.2:0/1578045064 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7faedc0a0640 0x7faedc0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:19.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.700+0000 7faef639b700 1 --2- 172.21.15.2:0/1578045064 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7faedc0a0640 0x7faedc0a2b00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faef019a120 tx=0x7faeec009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:19.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.703+0000 7faee77fe700 1 -- 172.21.15.2:0/1578045064 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7faee0096620 con 0x7faef0101ed0 2023-09-13T04:38:19.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:19 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3094162301' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:19.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:19 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2560446972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:38:19.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:19 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3094162301' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:19.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:19 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2560446972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:38:19.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.914+0000 7faef85ff700 1 -- 172.21.15.2:0/1578045064 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faef01a1160 con 0x7faef0101ed0 2023-09-13T04:38:19.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.914+0000 7faee77fe700 1 -- 172.21.15.2:0/1578045064 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7faee0099c40 con 0x7faef0101ed0 2023-09-13T04:38:19.915 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-09-13T04:38:19.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.916+0000 7faef85ff700 1 -- 172.21.15.2:0/1578045064 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7faedc0a0640 msgr2=0x7faedc0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:19.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.917+0000 7faef85ff700 1 --2- 172.21.15.2:0/1578045064 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7faedc0a0640 0x7faedc0a2b00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faef019a120 tx=0x7faeec009260 comp rx=0 tx=0).stop 2023-09-13T04:38:19.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.917+0000 7faef85ff700 1 -- 172.21.15.2:0/1578045064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faef0101ed0 msgr2=0x7faef0199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:19.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.917+0000 7faef85ff700 1 --2- 172.21.15.2:0/1578045064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faef0101ed0 0x7faef0199190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faee0009fd0 tx=0x7faee000f690 comp rx=0 tx=0).stop 2023-09-13T04:38:19.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.917+0000 7faef85ff700 1 -- 172.21.15.2:0/1578045064 shutdown_connections 2023-09-13T04:38:19.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.917+0000 7faef85ff700 1 --2- 172.21.15.2:0/1578045064 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7faedc0a0640 0x7faedc0a2b00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.917+0000 7faef85ff700 1 --2- 172.21.15.2:0/1578045064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faef0102870 0x7faef01a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.917+0000 7faef85ff700 1 --2- 172.21.15.2:0/1578045064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faef0100ce0 0x7faef0198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.917+0000 7faef85ff700 1 --2- 172.21.15.2:0/1578045064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faef0101ed0 0x7faef0199190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:19.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.917+0000 7faef85ff700 1 -- 172.21.15.2:0/1578045064 >> 172.21.15.2:0/1578045064 conn(0x7faef00fc230 msgr2=0x7faef00fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:19.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.917+0000 7faef85ff700 1 -- 172.21.15.2:0/1578045064 shutdown_connections 2023-09-13T04:38:19.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:19.917+0000 7faef85ff700 1 -- 172.21.15.2:0/1578045064 wait complete. 2023-09-13T04:38:19.935 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-13T04:38:19.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:19 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3094162301' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:19.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:19 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2560446972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:38:20.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.144+0000 7f23619ff700 1 -- 172.21.15.2:0/1349434174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f235c101ed0 msgr2=0x7f235c102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:20.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.144+0000 7f23619ff700 1 --2- 172.21.15.2:0/1349434174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f235c101ed0 0x7f235c102330 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f23500099c0 tx=0x7f2350009cd0 comp rx=0 tx=0).stop 2023-09-13T04:38:20.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.144+0000 7f23619ff700 1 -- 172.21.15.2:0/1349434174 shutdown_connections 2023-09-13T04:38:20.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.144+0000 7f23619ff700 1 --2- 172.21.15.2:0/1349434174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f235c102870 0x7f235c109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:20.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.144+0000 7f23619ff700 1 --2- 172.21.15.2:0/1349434174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f235c100ce0 0x7f235c101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:20.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.144+0000 7f23619ff700 1 --2- 172.21.15.2:0/1349434174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f235c101ed0 0x7f235c102330 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:20.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.144+0000 7f23619ff700 1 -- 172.21.15.2:0/1349434174 >> 172.21.15.2:0/1349434174 conn(0x7f235c0fc230 msgr2=0x7f235c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:20.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.144+0000 7f23619ff700 1 -- 172.21.15.2:0/1349434174 shutdown_connections 2023-09-13T04:38:20.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.144+0000 7f23619ff700 1 -- 172.21.15.2:0/1349434174 wait complete. 2023-09-13T04:38:20.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.145+0000 7f23619ff700 1 Processor -- start 2023-09-13T04:38:20.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.145+0000 7f23619ff700 1 -- start start 2023-09-13T04:38:20.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.145+0000 7f23619ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f235c100ce0 0x7f235c196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:20.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.145+0000 7f23619ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f235c101ed0 0x7f235c196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:20.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.145+0000 7f23619ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f235c102870 0x7f235c19dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:20.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.145+0000 7f23619ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c19e580 con 0x7f235c102870 2023-09-13T04:38:20.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.145+0000 7f23619ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c19e6c0 con 0x7f235c100ce0 2023-09-13T04:38:20.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.145+0000 7f23619ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c19e850 con 0x7f235c101ed0 2023-09-13T04:38:20.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.146+0000 7f235affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f235c100ce0 0x7f235c196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:20.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.146+0000 7f235affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f235c100ce0 0x7f235c196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:35002/0 (socket says 172.21.15.2:35002) 2023-09-13T04:38:20.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.146+0000 7f235affd700 1 -- 172.21.15.2:0/3204429356 learned_addr learned my addr 172.21.15.2:0/3204429356 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:20.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.146+0000 7f235a7fc700 1 --2- 172.21.15.2:0/3204429356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f235c101ed0 0x7f235c196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:20.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.146+0000 7f235b7fe700 1 --2- 172.21.15.2:0/3204429356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f235c102870 0x7f235c19dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:20.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.146+0000 7f235affd700 1 -- 172.21.15.2:0/3204429356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f235c101ed0 msgr2=0x7f235c196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:20.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.146+0000 7f235affd700 1 --2- 172.21.15.2:0/3204429356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f235c101ed0 0x7f235c196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:20.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.146+0000 7f235affd700 1 -- 172.21.15.2:0/3204429356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f235c102870 msgr2=0x7f235c19dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:20.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.146+0000 7f235affd700 1 --2- 172.21.15.2:0/3204429356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f235c102870 0x7f235c19dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:20.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.146+0000 7f235affd700 1 -- 172.21.15.2:0/3204429356 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23500095f0 con 0x7f235c100ce0 2023-09-13T04:38:20.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.146+0000 7f235affd700 1 --2- 172.21.15.2:0/3204429356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f235c100ce0 0x7f235c196a00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f234c00e970 tx=0x7f234c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:20.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.147+0000 7f23609fd700 1 -- 172.21.15.2:0/3204429356 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c009790 con 0x7f235c100ce0 2023-09-13T04:38:20.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.147+0000 7f23619ff700 1 -- 172.21.15.2:0/3204429356 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f235c19eb30 con 0x7f235c100ce0 2023-09-13T04:38:20.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.147+0000 7f23609fd700 1 -- 172.21.15.2:0/3204429356 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f234c00c7e0 con 0x7f235c100ce0 2023-09-13T04:38:20.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.147+0000 7f23609fd700 1 -- 172.21.15.2:0/3204429356 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c003c70 con 0x7f235c100ce0 2023-09-13T04:38:20.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.147+0000 7f23619ff700 1 -- 172.21.15.2:0/3204429356 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235c19f040 con 0x7f235c100ce0 2023-09-13T04:38:20.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.148+0000 7f23619ff700 1 -- 172.21.15.2:0/3204429356 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f235c10c540 con 0x7f235c100ce0 2023-09-13T04:38:20.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.150+0000 7f23609fd700 1 -- 172.21.15.2:0/3204429356 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f234c00cc50 con 0x7f235c100ce0 2023-09-13T04:38:20.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.151+0000 7f23609fd700 1 --2- 172.21.15.2:0/3204429356 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f23440a0640 0x7f23440a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:20.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.151+0000 7f23609fd700 1 -- 172.21.15.2:0/3204429356 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f234c014070 con 0x7f235c100ce0 2023-09-13T04:38:20.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.152+0000 7f235a7fc700 1 --2- 172.21.15.2:0/3204429356 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f23440a0640 0x7f23440a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:20.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.152+0000 7f235a7fc700 1 --2- 172.21.15.2:0/3204429356 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f23440a0640 0x7f23440a2b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f235000b580 tx=0x7f2350005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:20.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.154+0000 7f23609fd700 1 -- 172.21.15.2:0/3204429356 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f234c0959f0 con 0x7f235c100ce0 2023-09-13T04:38:20.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.362+0000 7f23619ff700 1 -- 172.21.15.2:0/3204429356 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f235c107c30 con 0x7f235c100ce0 2023-09-13T04:38:20.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.363+0000 7f23609fd700 1 -- 172.21.15.2:0/3204429356 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f234c099010 con 0x7f235c100ce0 2023-09-13T04:38:20.364 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-09-13T04:38:20.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.366+0000 7f23619ff700 1 -- 172.21.15.2:0/3204429356 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f23440a0640 msgr2=0x7f23440a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:20.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.366+0000 7f23619ff700 1 --2- 172.21.15.2:0/3204429356 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f23440a0640 0x7f23440a2b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f235000b580 tx=0x7f2350005fb0 comp rx=0 tx=0).stop 2023-09-13T04:38:20.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.366+0000 7f23619ff700 1 -- 172.21.15.2:0/3204429356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f235c100ce0 msgr2=0x7f235c196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:20.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.366+0000 7f23619ff700 1 --2- 172.21.15.2:0/3204429356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f235c100ce0 0x7f235c196a00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f234c00e970 tx=0x7f234c00ec80 comp rx=0 tx=0).stop 2023-09-13T04:38:20.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.366+0000 7f23619ff700 1 -- 172.21.15.2:0/3204429356 shutdown_connections 2023-09-13T04:38:20.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.366+0000 7f23619ff700 1 --2- 172.21.15.2:0/3204429356 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f23440a0640 0x7f23440a2b00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:20.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.366+0000 7f23619ff700 1 --2- 172.21.15.2:0/3204429356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f235c102870 0x7f235c19dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:20.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.366+0000 7f23619ff700 1 --2- 172.21.15.2:0/3204429356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f235c100ce0 0x7f235c196a00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:20.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.366+0000 7f23619ff700 1 --2- 172.21.15.2:0/3204429356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f235c101ed0 0x7f235c196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:20.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.366+0000 7f23619ff700 1 -- 172.21.15.2:0/3204429356 >> 172.21.15.2:0/3204429356 conn(0x7f235c0fc230 msgr2=0x7f235c0fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:20.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.366+0000 7f23619ff700 1 -- 172.21.15.2:0/3204429356 shutdown_connections 2023-09-13T04:38:20.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.366+0000 7f23619ff700 1 -- 172.21.15.2:0/3204429356 wait complete. 2023-09-13T04:38:20.384 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-13T04:38:20.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.592+0000 7fd5435ff700 1 -- 172.21.15.2:0/1845154474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd53c101ed0 msgr2=0x7fd53c102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:20.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.592+0000 7fd5435ff700 1 --2- 172.21.15.2:0/1845154474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd53c101ed0 0x7fd53c102330 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd52c0099c0 tx=0x7fd52c009cd0 comp rx=0 tx=0).stop 2023-09-13T04:38:20.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.592+0000 7fd5435ff700 1 -- 172.21.15.2:0/1845154474 shutdown_connections 2023-09-13T04:38:20.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.592+0000 7fd5435ff700 1 --2- 172.21.15.2:0/1845154474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd53c102870 0x7fd53c109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:20.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.592+0000 7fd5435ff700 1 --2- 172.21.15.2:0/1845154474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd53c100ce0 0x7fd53c101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:20.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.592+0000 7fd5435ff700 1 --2- 172.21.15.2:0/1845154474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd53c101ed0 0x7fd53c102330 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:20.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.592+0000 7fd5435ff700 1 -- 172.21.15.2:0/1845154474 >> 172.21.15.2:0/1845154474 conn(0x7fd53c0fc230 msgr2=0x7fd53c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:20.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.592+0000 7fd5435ff700 1 -- 172.21.15.2:0/1845154474 shutdown_connections 2023-09-13T04:38:20.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.592+0000 7fd5435ff700 1 -- 172.21.15.2:0/1845154474 wait complete. 2023-09-13T04:38:20.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.593+0000 7fd5435ff700 1 Processor -- start 2023-09-13T04:38:20.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.593+0000 7fd5435ff700 1 -- start start 2023-09-13T04:38:20.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.593+0000 7fd5435ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd53c100ce0 0x7fd53c198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:20.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.593+0000 7fd5435ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd53c101ed0 0x7fd53c1990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:20.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.593+0000 7fd5435ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd53c102870 0x7fd53c1a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:20.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.593+0000 7fd5435ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c1a0700 con 0x7fd53c101ed0 2023-09-13T04:38:20.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.593+0000 7fd5435ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c1a0840 con 0x7fd53c100ce0 2023-09-13T04:38:20.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.593+0000 7fd5435ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c1a09d0 con 0x7fd53c102870 2023-09-13T04:38:20.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.594+0000 7fd540b9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd53c101ed0 0x7fd53c1990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:20.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.594+0000 7fd540b9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd53c101ed0 0x7fd53c1990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43620/0 (socket says 172.21.15.2:43620) 2023-09-13T04:38:20.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.594+0000 7fd540b9a700 1 -- 172.21.15.2:0/3252833254 learned_addr learned my addr 172.21.15.2:0/3252833254 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:20.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.594+0000 7fd54139b700 1 --2- 172.21.15.2:0/3252833254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd53c100ce0 0x7fd53c198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:20.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.594+0000 7fd541b9c700 1 --2- 172.21.15.2:0/3252833254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd53c102870 0x7fd53c1a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:20.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.594+0000 7fd540b9a700 1 -- 172.21.15.2:0/3252833254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd53c102870 msgr2=0x7fd53c1a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:20.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.594+0000 7fd540b9a700 1 --2- 172.21.15.2:0/3252833254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd53c102870 0x7fd53c1a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:20.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.594+0000 7fd540b9a700 1 -- 172.21.15.2:0/3252833254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd53c100ce0 msgr2=0x7fd53c198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:20.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.594+0000 7fd540b9a700 1 --2- 172.21.15.2:0/3252833254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd53c100ce0 0x7fd53c198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:20.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.594+0000 7fd540b9a700 1 -- 172.21.15.2:0/3252833254 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd538009520 con 0x7fd53c101ed0 2023-09-13T04:38:20.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.594+0000 7fd541b9c700 1 --2- 172.21.15.2:0/3252833254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd53c102870 0x7fd53c1a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:38:20.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.594+0000 7fd540b9a700 1 --2- 172.21.15.2:0/3252833254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd53c101ed0 0x7fd53c1990c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd52c00c010 tx=0x7fd52c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:20.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.595+0000 7fd5327fc700 1 -- 172.21.15.2:0/3252833254 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c01d070 con 0x7fd53c101ed0 2023-09-13T04:38:20.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.595+0000 7fd5435ff700 1 -- 172.21.15.2:0/3252833254 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52c0095f0 con 0x7fd53c101ed0 2023-09-13T04:38:20.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.595+0000 7fd5327fc700 1 -- 172.21.15.2:0/3252833254 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd52c005050 con 0x7fd53c101ed0 2023-09-13T04:38:20.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.595+0000 7fd5327fc700 1 -- 172.21.15.2:0/3252833254 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c004b70 con 0x7fd53c101ed0 2023-09-13T04:38:20.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.595+0000 7fd5435ff700 1 -- 172.21.15.2:0/3252833254 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd53c1a0f40 con 0x7fd53c101ed0 2023-09-13T04:38:20.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.596+0000 7fd5435ff700 1 -- 172.21.15.2:0/3252833254 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd53c10c4f0 con 0x7fd53c101ed0 2023-09-13T04:38:20.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.598+0000 7fd5327fc700 1 -- 172.21.15.2:0/3252833254 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fd52c0051c0 con 0x7fd53c101ed0 2023-09-13T04:38:20.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.599+0000 7fd5327fc700 1 --2- 172.21.15.2:0/3252833254 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd5280a0640 0x7fd5280a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:20.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.599+0000 7fd5327fc700 1 -- 172.21.15.2:0/3252833254 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd52c0cd0d0 con 0x7fd53c101ed0 2023-09-13T04:38:20.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.599+0000 7fd54139b700 1 --2- 172.21.15.2:0/3252833254 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd5280a0640 0x7fd5280a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:20.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.600+0000 7fd54139b700 1 --2- 172.21.15.2:0/3252833254 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd5280a0640 0x7fd5280a2b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd53c101d30 tx=0x7fd538009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:20.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.601+0000 7fd5327fc700 1 -- 172.21.15.2:0/3252833254 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd52c096a40 con 0x7fd53c101ed0 2023-09-13T04:38:20.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:20 smithi002 ceph-mon[97651]: pgmap v286: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-09-13T04:38:20.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:20 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1578045064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:38:20.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:20 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3204429356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:38:20.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.817+0000 7fd5435ff700 1 -- 172.21.15.2:0/3252833254 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd53c1a1130 con 0x7fd53c101ed0 2023-09-13T04:38:20.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.818+0000 7fd5327fc700 1 -- 172.21.15.2:0/3252833254 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd52c09a060 con 0x7fd53c101ed0 2023-09-13T04:38:20.818 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-09-13T04:38:20.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.820+0000 7fd5435ff700 1 -- 172.21.15.2:0/3252833254 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd5280a0640 msgr2=0x7fd5280a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:20.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.820+0000 7fd5435ff700 1 --2- 172.21.15.2:0/3252833254 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd5280a0640 0x7fd5280a2b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd53c101d30 tx=0x7fd538009260 comp rx=0 tx=0).stop 2023-09-13T04:38:20.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.820+0000 7fd5435ff700 1 -- 172.21.15.2:0/3252833254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd53c101ed0 msgr2=0x7fd53c1990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:20.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.820+0000 7fd5435ff700 1 --2- 172.21.15.2:0/3252833254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd53c101ed0 0x7fd53c1990c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd52c00c010 tx=0x7fd52c00f690 comp rx=0 tx=0).stop 2023-09-13T04:38:20.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.820+0000 7fd5435ff700 1 -- 172.21.15.2:0/3252833254 shutdown_connections 2023-09-13T04:38:20.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.820+0000 7fd5435ff700 1 --2- 172.21.15.2:0/3252833254 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd5280a0640 0x7fd5280a2b00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:20.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.820+0000 7fd5435ff700 1 --2- 172.21.15.2:0/3252833254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd53c102870 0x7fd53c1a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:20.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.820+0000 7fd5435ff700 1 --2- 172.21.15.2:0/3252833254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd53c100ce0 0x7fd53c198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:20.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.820+0000 7fd5435ff700 1 --2- 172.21.15.2:0/3252833254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd53c101ed0 0x7fd53c1990c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:20.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.820+0000 7fd5435ff700 1 -- 172.21.15.2:0/3252833254 >> 172.21.15.2:0/3252833254 conn(0x7fd53c0fc230 msgr2=0x7fd53c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:20.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.820+0000 7fd5435ff700 1 -- 172.21.15.2:0/3252833254 shutdown_connections 2023-09-13T04:38:20.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:20.820+0000 7fd5435ff700 1 -- 172.21.15.2:0/3252833254 wait complete. 2023-09-13T04:38:20.839 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-13T04:38:20.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:20 smithi149 ceph-mon[133315]: pgmap v286: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-09-13T04:38:20.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:20 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1578045064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:38:20.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:20 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3204429356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:38:20.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:20 smithi156 ceph-mon[133675]: pgmap v286: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-09-13T04:38:20.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:20 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1578045064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:38:20.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:20 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3204429356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:38:21.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.044+0000 7f51fb5ff700 1 -- 172.21.15.2:0/1446805324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51f4102870 msgr2=0x7f51f4109120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:21.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.044+0000 7f51fb5ff700 1 --2- 172.21.15.2:0/1446805324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51f4102870 0x7f51f4109120 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0099c0 tx=0x7f51ec009cd0 comp rx=0 tx=0).stop 2023-09-13T04:38:21.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.044+0000 7f51fb5ff700 1 -- 172.21.15.2:0/1446805324 shutdown_connections 2023-09-13T04:38:21.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.044+0000 7f51fb5ff700 1 --2- 172.21.15.2:0/1446805324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51f4102870 0x7f51f4109120 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:21.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.044+0000 7f51fb5ff700 1 --2- 172.21.15.2:0/1446805324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51f4100ce0 0x7f51f4101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:21.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.044+0000 7f51fb5ff700 1 --2- 172.21.15.2:0/1446805324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51f4101ed0 0x7f51f4102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:21.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.044+0000 7f51fb5ff700 1 -- 172.21.15.2:0/1446805324 >> 172.21.15.2:0/1446805324 conn(0x7f51f40fc230 msgr2=0x7f51f40fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:21.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.044+0000 7f51fb5ff700 1 -- 172.21.15.2:0/1446805324 shutdown_connections 2023-09-13T04:38:21.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.044+0000 7f51fb5ff700 1 -- 172.21.15.2:0/1446805324 wait complete. 2023-09-13T04:38:21.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.045+0000 7f51fb5ff700 1 Processor -- start 2023-09-13T04:38:21.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.046+0000 7f51fb5ff700 1 -- start start 2023-09-13T04:38:21.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.046+0000 7f51fb5ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51f4100ce0 0x7f51f4074a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:21.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.046+0000 7f51fb5ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51f4101ed0 0x7f51f40730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:21.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.046+0000 7f51fb5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51f4102870 0x7f51f40735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:21.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.046+0000 7f51fb5ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f4073d80 con 0x7f51f4101ed0 2023-09-13T04:38:21.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.046+0000 7f51fb5ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f4073ef0 con 0x7f51f4100ce0 2023-09-13T04:38:21.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.046+0000 7f51fb5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f41a09c0 con 0x7f51f4102870 2023-09-13T04:38:21.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.046+0000 7f51f8b9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51f4101ed0 0x7f51f40730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:21.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.046+0000 7f51f8b9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51f4101ed0 0x7f51f40730a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43626/0 (socket says 172.21.15.2:43626) 2023-09-13T04:38:21.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.046+0000 7f51f8b9a700 1 -- 172.21.15.2:0/3089061211 learned_addr learned my addr 172.21.15.2:0/3089061211 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:21.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.046+0000 7f51f939b700 1 --2- 172.21.15.2:0/3089061211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51f4100ce0 0x7f51f4074a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:21.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.046+0000 7f51f9b9c700 1 --2- 172.21.15.2:0/3089061211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51f4102870 0x7f51f40735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:21.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.047+0000 7f51f8b9a700 1 -- 172.21.15.2:0/3089061211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51f4102870 msgr2=0x7f51f40735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:21.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.047+0000 7f51f8b9a700 1 --2- 172.21.15.2:0/3089061211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51f4102870 0x7f51f40735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:21.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.047+0000 7f51f8b9a700 1 -- 172.21.15.2:0/3089061211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51f4100ce0 msgr2=0x7f51f4074a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:21.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.047+0000 7f51f8b9a700 1 --2- 172.21.15.2:0/3089061211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51f4100ce0 0x7f51f4074a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:21.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.047+0000 7f51f8b9a700 1 -- 172.21.15.2:0/3089061211 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f0009c40 con 0x7f51f4101ed0 2023-09-13T04:38:21.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.047+0000 7f51f8b9a700 1 --2- 172.21.15.2:0/3089061211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51f4101ed0 0x7f51f40730a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f51e400e9a0 tx=0x7f51e400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:21.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.047+0000 7f51ea7fc700 1 -- 172.21.15.2:0/3089061211 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4009790 con 0x7f51f4101ed0 2023-09-13T04:38:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.047+0000 7f51ea7fc700 1 -- 172.21.15.2:0/3089061211 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51e4010430 con 0x7f51f4101ed0 2023-09-13T04:38:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.047+0000 7f51ea7fc700 1 -- 172.21.15.2:0/3089061211 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4018680 con 0x7f51f4101ed0 2023-09-13T04:38:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.047+0000 7f51fb5ff700 1 -- 172.21.15.2:0/3089061211 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ec0095f0 con 0x7f51f4101ed0 2023-09-13T04:38:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.047+0000 7f51fb5ff700 1 -- 172.21.15.2:0/3089061211 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f41a0fa0 con 0x7f51f4101ed0 2023-09-13T04:38:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.048+0000 7f51fb5ff700 1 -- 172.21.15.2:0/3089061211 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f4107b70 con 0x7f51f4101ed0 2023-09-13T04:38:21.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.050+0000 7f51ea7fc700 1 -- 172.21.15.2:0/3089061211 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f51e40105a0 con 0x7f51f4101ed0 2023-09-13T04:38:21.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.051+0000 7f51ea7fc700 1 --2- 172.21.15.2:0/3089061211 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f51e00a0640 0x7f51e00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.052+0000 7f51ea7fc700 1 -- 172.21.15.2:0/3089061211 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f51e4014070 con 0x7f51f4101ed0 2023-09-13T04:38:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.052+0000 7f51f939b700 1 --2- 172.21.15.2:0/3089061211 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f51e00a0640 0x7f51e00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.052+0000 7f51f939b700 1 --2- 172.21.15.2:0/3089061211 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f51e00a0640 0x7f51e00a2b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f51f4101d30 tx=0x7f51f000b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.055+0000 7f51ea7fc700 1 -- 172.21.15.2:0/3089061211 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f51e4051e10 con 0x7f51f4101ed0 2023-09-13T04:38:21.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.282+0000 7f51fb5ff700 1 -- 172.21.15.2:0/3089061211 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f51f402ce20 con 0x7f51f4101ed0 2023-09-13T04:38:21.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.282+0000 7f51ea7fc700 1 -- 172.21.15.2:0/3089061211 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+581 (secure 0 0 0) 0x7f51e4095870 con 0x7f51f4101ed0 2023-09-13T04:38:21.284 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/1393377368 2023-09-14T04:31:23.417691+0000 2023-09-13T04:38:21.285 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/2772434518 2023-09-14T04:30:54.963309+0000 2023-09-13T04:38:21.285 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3810974822 2023-09-14T04:31:23.417691+0000 2023-09-13T04:38:21.285 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/2772434518 2023-09-14T04:30:54.963309+0000 2023-09-13T04:38:21.286 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1907805835 2023-09-14T04:30:54.963309+0000 2023-09-13T04:38:21.286 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2811866029 2023-09-14T04:30:54.963309+0000 2023-09-13T04:38:21.286 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2538614772 2023-09-14T04:31:23.417691+0000 2023-09-13T04:38:21.287 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/1393377368 2023-09-14T04:31:23.417691+0000 2023-09-13T04:38:21.287 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3722465120 2023-09-14T04:31:23.417691+0000 2023-09-13T04:38:21.287 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/516951274 2023-09-14T04:30:54.963309+0000 2023-09-13T04:38:21.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.285+0000 7f51fb5ff700 1 -- 172.21.15.2:0/3089061211 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f51e00a0640 msgr2=0x7f51e00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:21.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.285+0000 7f51fb5ff700 1 --2- 172.21.15.2:0/3089061211 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f51e00a0640 0x7f51e00a2b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f51f4101d30 tx=0x7f51f000b040 comp rx=0 tx=0).stop 2023-09-13T04:38:21.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.285+0000 7f51fb5ff700 1 -- 172.21.15.2:0/3089061211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51f4101ed0 msgr2=0x7f51f40730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:21.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.285+0000 7f51fb5ff700 1 --2- 172.21.15.2:0/3089061211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51f4101ed0 0x7f51f40730a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f51e400e9a0 tx=0x7f51e400ecb0 comp rx=0 tx=0).stop 2023-09-13T04:38:21.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.285+0000 7f51fb5ff700 1 -- 172.21.15.2:0/3089061211 shutdown_connections 2023-09-13T04:38:21.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.285+0000 7f51fb5ff700 1 --2- 172.21.15.2:0/3089061211 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f51e00a0640 0x7f51e00a2b00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:21.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.285+0000 7f51fb5ff700 1 --2- 172.21.15.2:0/3089061211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51f4102870 0x7f51f40735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:21.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.285+0000 7f51fb5ff700 1 --2- 172.21.15.2:0/3089061211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51f4100ce0 0x7f51f4074a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:21.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.285+0000 7f51fb5ff700 1 --2- 172.21.15.2:0/3089061211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51f4101ed0 0x7f51f40730a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:21.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.285+0000 7f51fb5ff700 1 -- 172.21.15.2:0/3089061211 >> 172.21.15.2:0/3089061211 conn(0x7f51f40fc230 msgr2=0x7f51f40fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:21.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.285+0000 7f51fb5ff700 1 -- 172.21.15.2:0/3089061211 shutdown_connections 2023-09-13T04:38:21.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.285+0000 7f51fb5ff700 1 -- 172.21.15.2:0/3089061211 wait complete. 2023-09-13T04:38:21.291 INFO:teuthology.orchestra.run.smithi002.stderr:listed 10 entries 2023-09-13T04:38:21.304 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi002.front.sepia.ceph.com... 2023-09-13T04:38:21.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:38:21.305 DEBUG:teuthology.orchestra.run.smithi002:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-09-13T04:38:21.372 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi002.stderr:ceph-fuse[151313]: fuse finished with error 0 and tester_r 0 2023-09-13T04:38:21.377 INFO:teuthology.orchestra.run:waiting for 300 2023-09-13T04:38:21.455 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi002.front.sepia.ceph.com 2023-09-13T04:38:21.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:38:21.457 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-09-13T04:38:21.514 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T04:38:21.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.719+0000 7f227dbff700 1 -- 172.21.15.2:0/3279037769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2278102880 msgr2=0x7f2278109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:21.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.719+0000 7f227dbff700 1 --2- 172.21.15.2:0/3279037769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2278102880 0x7f2278109130 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f227000b0d0 tx=0x7f227000b3e0 comp rx=0 tx=0).stop 2023-09-13T04:38:21.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.719+0000 7f227dbff700 1 -- 172.21.15.2:0/3279037769 shutdown_connections 2023-09-13T04:38:21.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.719+0000 7f227dbff700 1 --2- 172.21.15.2:0/3279037769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2278102880 0x7f2278109130 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:21.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.719+0000 7f227dbff700 1 --2- 172.21.15.2:0/3279037769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2278100cf0 0x7f2278101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:21.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.719+0000 7f227dbff700 1 --2- 172.21.15.2:0/3279037769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2278101ee0 0x7f2278102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:21.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.719+0000 7f227dbff700 1 -- 172.21.15.2:0/3279037769 >> 172.21.15.2:0/3279037769 conn(0x7f22780fc220 msgr2=0x7f22780fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:21.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.719+0000 7f227dbff700 1 -- 172.21.15.2:0/3279037769 shutdown_connections 2023-09-13T04:38:21.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.719+0000 7f227dbff700 1 -- 172.21.15.2:0/3279037769 wait complete. 2023-09-13T04:38:21.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.721+0000 7f227dbff700 1 Processor -- start 2023-09-13T04:38:21.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.721+0000 7f227dbff700 1 -- start start 2023-09-13T04:38:21.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.722+0000 7f227dbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2278100cf0 0x7f2278074a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:21.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.722+0000 7f227dbff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2278101ee0 0x7f22780730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:21.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.722+0000 7f227dbff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2278102880 0x7f22780735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:21.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.722+0000 7f227dbff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2278073d80 con 0x7f2278101ee0 2023-09-13T04:38:21.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.722+0000 7f227dbff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2278073ef0 con 0x7f2278102880 2023-09-13T04:38:21.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.722+0000 7f227dbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22781a0af0 con 0x7f2278100cf0 2023-09-13T04:38:21.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.722+0000 7f2276ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2278101ee0 0x7f22780730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:21.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.722+0000 7f22777fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2278100cf0 0x7f2278074a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:21.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.722+0000 7f2276ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2278101ee0 0x7f22780730a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43644/0 (socket says 172.21.15.2:43644) 2023-09-13T04:38:21.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.722+0000 7f2276ffd700 1 -- 172.21.15.2:0/2065859905 learned_addr learned my addr 172.21.15.2:0/2065859905 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:21.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.722+0000 7f2277fff700 1 --2- 172.21.15.2:0/2065859905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2278102880 0x7f22780735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:21.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.722+0000 7f2276ffd700 1 -- 172.21.15.2:0/2065859905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2278100cf0 msgr2=0x7f2278074a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:21.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.722+0000 7f2276ffd700 1 --2- 172.21.15.2:0/2065859905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2278100cf0 0x7f2278074a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:21.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.722+0000 7f2276ffd700 1 -- 172.21.15.2:0/2065859905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2278102880 msgr2=0x7f22780735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:21.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.722+0000 7f2276ffd700 1 --2- 172.21.15.2:0/2065859905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2278102880 0x7f22780735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:21.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.722+0000 7f2276ffd700 1 -- 172.21.15.2:0/2065859905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2270009d00 con 0x7f2278101ee0 2023-09-13T04:38:21.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.722+0000 7f2277fff700 1 --2- 172.21.15.2:0/2065859905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2278102880 0x7f22780735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:38:21.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.722+0000 7f22777fe700 1 --2- 172.21.15.2:0/2065859905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2278100cf0 0x7f2278074a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:38:21.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.723+0000 7f2276ffd700 1 --2- 172.21.15.2:0/2065859905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2278101ee0 0x7f22780730a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f226c00e9a0 tx=0x7f226c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:21.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.723+0000 7f2274ff9700 1 -- 172.21.15.2:0/2065859905 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c009790 con 0x7f2278101ee0 2023-09-13T04:38:21.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.723+0000 7f2274ff9700 1 -- 172.21.15.2:0/2065859905 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f226c010430 con 0x7f2278101ee0 2023-09-13T04:38:21.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.723+0000 7f227dbff700 1 -- 172.21.15.2:0/2065859905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22781a0dd0 con 0x7f2278101ee0 2023-09-13T04:38:21.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.723+0000 7f2274ff9700 1 -- 172.21.15.2:0/2065859905 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c018680 con 0x7f2278101ee0 2023-09-13T04:38:21.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.723+0000 7f227dbff700 1 -- 172.21.15.2:0/2065859905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22781a12b0 con 0x7f2278101ee0 2023-09-13T04:38:21.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.724+0000 7f227dbff700 1 -- 172.21.15.2:0/2065859905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f227810c560 con 0x7f2278101ee0 2023-09-13T04:38:21.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.726+0000 7f2274ff9700 1 -- 172.21.15.2:0/2065859905 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f226c0105a0 con 0x7f2278101ee0 2023-09-13T04:38:21.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.727+0000 7f2274ff9700 1 --2- 172.21.15.2:0/2065859905 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f22600a0710 0x7f22600a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:21.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.727+0000 7f2274ff9700 1 -- 172.21.15.2:0/2065859905 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f226c014070 con 0x7f2278101ee0 2023-09-13T04:38:21.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.728+0000 7f22777fe700 1 --2- 172.21.15.2:0/2065859905 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f22600a0710 0x7f22600a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:21.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.728+0000 7f22777fe700 1 --2- 172.21.15.2:0/2065859905 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f22600a0710 0x7f22600a2bd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2278101d40 tx=0x7f2268009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:21.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.731+0000 7f2274ff9700 1 -- 172.21.15.2:0/2065859905 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f226c051e10 con 0x7f2278101ee0 2023-09-13T04:38:21.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:21 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3252833254' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:38:21.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:21 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3089061211' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-13T04:38:21.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:21 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3252833254' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:38:21.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:21 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3089061211' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-13T04:38:21.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.944+0000 7f227dbff700 1 -- 172.21.15.2:0/2065859905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2278107b50 con 0x7f2278101ee0 2023-09-13T04:38:21.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.946+0000 7f2274ff9700 1 -- 172.21.15.2:0/2065859905 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23927 (secure 0 0 0) 0x7f226c095810 con 0x7f2278101ee0 2023-09-13T04:38:21.947 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:38:21.947 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":88,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:38:04.132627+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T04:37:57.094802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T04:37:58.564487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-13T04:38:02.096895+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:38:21.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.948+0000 7f227dbff700 1 -- 172.21.15.2:0/2065859905 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f22600a0710 msgr2=0x7f22600a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:21.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.948+0000 7f227dbff700 1 --2- 172.21.15.2:0/2065859905 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f22600a0710 0x7f22600a2bd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2278101d40 tx=0x7f2268009260 comp rx=0 tx=0).stop 2023-09-13T04:38:21.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.948+0000 7f227dbff700 1 -- 172.21.15.2:0/2065859905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2278101ee0 msgr2=0x7f22780730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:21.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.948+0000 7f227dbff700 1 --2- 172.21.15.2:0/2065859905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2278101ee0 0x7f22780730a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f226c00e9a0 tx=0x7f226c00ecb0 comp rx=0 tx=0).stop 2023-09-13T04:38:21.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.948+0000 7f227dbff700 1 -- 172.21.15.2:0/2065859905 shutdown_connections 2023-09-13T04:38:21.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.948+0000 7f227dbff700 1 --2- 172.21.15.2:0/2065859905 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f22600a0710 0x7f22600a2bd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:21.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.948+0000 7f227dbff700 1 --2- 172.21.15.2:0/2065859905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2278102880 0x7f22780735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:21.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.948+0000 7f227dbff700 1 --2- 172.21.15.2:0/2065859905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2278100cf0 0x7f2278074a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:21.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.948+0000 7f227dbff700 1 --2- 172.21.15.2:0/2065859905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2278101ee0 0x7f22780730a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:21.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.948+0000 7f227dbff700 1 -- 172.21.15.2:0/2065859905 >> 172.21.15.2:0/2065859905 conn(0x7f22780fc220 msgr2=0x7f22780fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:21.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.948+0000 7f227dbff700 1 -- 172.21.15.2:0/2065859905 shutdown_connections 2023-09-13T04:38:21.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:21.948+0000 7f227dbff700 1 -- 172.21.15.2:0/2065859905 wait complete. 2023-09-13T04:38:21.966 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:21 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3252833254' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:38:21.966 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:21 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3089061211' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-13T04:38:21.970 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T04:38:22.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.174+0000 7f306adff700 1 -- 172.21.15.2:0/1324182959 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3064100cf0 msgr2=0x7f3064101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:22.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.174+0000 7f306adff700 1 --2- 172.21.15.2:0/1324182959 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3064100cf0 0x7f3064101110 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f30580099c0 tx=0x7f3058009cd0 comp rx=0 tx=0).stop 2023-09-13T04:38:22.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.174+0000 7f306adff700 1 -- 172.21.15.2:0/1324182959 shutdown_connections 2023-09-13T04:38:22.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.174+0000 7f306adff700 1 --2- 172.21.15.2:0/1324182959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3064102880 0x7f3064109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:22.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.174+0000 7f306adff700 1 --2- 172.21.15.2:0/1324182959 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3064100cf0 0x7f3064101110 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:22.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.174+0000 7f306adff700 1 --2- 172.21.15.2:0/1324182959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3064101ee0 0x7f3064102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:22.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.174+0000 7f306adff700 1 -- 172.21.15.2:0/1324182959 >> 172.21.15.2:0/1324182959 conn(0x7f30640fc220 msgr2=0x7f30640fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:22.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.174+0000 7f306adff700 1 -- 172.21.15.2:0/1324182959 shutdown_connections 2023-09-13T04:38:22.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.174+0000 7f306adff700 1 -- 172.21.15.2:0/1324182959 wait complete. 2023-09-13T04:38:22.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.175+0000 7f306adff700 1 Processor -- start 2023-09-13T04:38:22.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.175+0000 7f306adff700 1 -- start start 2023-09-13T04:38:22.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.175+0000 7f306adff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3064100cf0 0x7f3064198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:22.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.175+0000 7f306adff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3064101ee0 0x7f3064199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:22.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.175+0000 7f306adff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3064102880 0x7f30641a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:22.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.175+0000 7f306adff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30641a0770 con 0x7f3064100cf0 2023-09-13T04:38:22.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.175+0000 7f306adff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30641a08b0 con 0x7f3064102880 2023-09-13T04:38:22.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.175+0000 7f306adff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30641a0a40 con 0x7f3064101ee0 2023-09-13T04:38:22.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.176+0000 7f3068b9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3064100cf0 0x7f3064198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:22.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.176+0000 7f306939c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3064102880 0x7f30641a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:22.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.176+0000 7f306939c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3064102880 0x7f30641a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:35070/0 (socket says 172.21.15.2:35070) 2023-09-13T04:38:22.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.176+0000 7f3068b9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3064100cf0 0x7f3064198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43662/0 (socket says 172.21.15.2:43662) 2023-09-13T04:38:22.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.176+0000 7f306939c700 1 -- 172.21.15.2:0/1818350740 learned_addr learned my addr 172.21.15.2:0/1818350740 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:22.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.176+0000 7f3063fff700 1 --2- 172.21.15.2:0/1818350740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3064101ee0 0x7f3064199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:22.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.176+0000 7f306939c700 1 -- 172.21.15.2:0/1818350740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3064101ee0 msgr2=0x7f3064199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:22.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.176+0000 7f306939c700 1 --2- 172.21.15.2:0/1818350740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3064101ee0 0x7f3064199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:22.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.176+0000 7f306939c700 1 -- 172.21.15.2:0/1818350740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3064100cf0 msgr2=0x7f3064198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:22.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.176+0000 7f306939c700 1 --2- 172.21.15.2:0/1818350740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3064100cf0 0x7f3064198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:22.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.176+0000 7f306939c700 1 -- 172.21.15.2:0/1818350740 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30580095f0 con 0x7f3064102880 2023-09-13T04:38:22.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.176+0000 7f3068b9b700 1 --2- 172.21.15.2:0/1818350740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3064100cf0 0x7f3064198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:38:22.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.176+0000 7f306939c700 1 --2- 172.21.15.2:0/1818350740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3064102880 0x7f30641a01a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f305c00ef30 tx=0x7f305c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:22.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.177+0000 7f3061ffb700 1 -- 172.21.15.2:0/1818350740 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c018070 con 0x7f3064102880 2023-09-13T04:38:22.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.177+0000 7f306adff700 1 -- 172.21.15.2:0/1818350740 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30641a0cc0 con 0x7f3064102880 2023-09-13T04:38:22.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.177+0000 7f3061ffb700 1 -- 172.21.15.2:0/1818350740 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f305c00f040 con 0x7f3064102880 2023-09-13T04:38:22.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.177+0000 7f306adff700 1 -- 172.21.15.2:0/1818350740 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30641a11d0 con 0x7f3064102880 2023-09-13T04:38:22.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.177+0000 7f3061ffb700 1 -- 172.21.15.2:0/1818350740 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c00ccb0 con 0x7f3064102880 2023-09-13T04:38:22.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.178+0000 7f306adff700 1 -- 172.21.15.2:0/1818350740 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3064193340 con 0x7f3064102880 2023-09-13T04:38:22.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.180+0000 7f3061ffb700 1 -- 172.21.15.2:0/1818350740 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f305c003a00 con 0x7f3064102880 2023-09-13T04:38:22.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.181+0000 7f3061ffb700 1 --2- 172.21.15.2:0/1818350740 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f30540a0710 0x7f30540a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:22.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.182+0000 7f3061ffb700 1 -- 172.21.15.2:0/1818350740 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f305c0cc1c0 con 0x7f3064102880 2023-09-13T04:38:22.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.182+0000 7f3068b9b700 1 --2- 172.21.15.2:0/1818350740 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f30540a0710 0x7f30540a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:22.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.182+0000 7f3068b9b700 1 --2- 172.21.15.2:0/1818350740 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f30540a0710 0x7f30540a2bd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3058009990 tx=0x7f3058005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:22.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.183+0000 7f3061ffb700 1 -- 172.21.15.2:0/1818350740 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f305c0995c0 con 0x7f3064102880 2023-09-13T04:38:22.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.397+0000 7f306adff700 1 -- 172.21.15.2:0/1818350740 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f306410c550 con 0x7f3064102880 2023-09-13T04:38:22.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.399+0000 7f3061ffb700 1 -- 172.21.15.2:0/1818350740 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23927 (secure 0 0 0) 0x7f305c099150 con 0x7f3064102880 2023-09-13T04:38:22.400 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:38:22.401 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":88,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:38:04.132627+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T04:37:57.094802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T04:37:58.564487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-13T04:38:02.096895+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:38:22.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.401+0000 7f306adff700 1 -- 172.21.15.2:0/1818350740 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f30540a0710 msgr2=0x7f30540a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:22.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.401+0000 7f306adff700 1 --2- 172.21.15.2:0/1818350740 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f30540a0710 0x7f30540a2bd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3058009990 tx=0x7f3058005fb0 comp rx=0 tx=0).stop 2023-09-13T04:38:22.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.401+0000 7f306adff700 1 -- 172.21.15.2:0/1818350740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3064102880 msgr2=0x7f30641a01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:22.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.401+0000 7f306adff700 1 --2- 172.21.15.2:0/1818350740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3064102880 0x7f30641a01a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f305c00ef30 tx=0x7f305c00c3a0 comp rx=0 tx=0).stop 2023-09-13T04:38:22.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.401+0000 7f306adff700 1 -- 172.21.15.2:0/1818350740 shutdown_connections 2023-09-13T04:38:22.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.401+0000 7f306adff700 1 --2- 172.21.15.2:0/1818350740 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f30540a0710 0x7f30540a2bd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:22.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.401+0000 7f306adff700 1 --2- 172.21.15.2:0/1818350740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3064102880 0x7f30641a01a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:22.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.401+0000 7f306adff700 1 --2- 172.21.15.2:0/1818350740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3064100cf0 0x7f3064198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:22.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.401+0000 7f306adff700 1 --2- 172.21.15.2:0/1818350740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3064101ee0 0x7f3064199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:22.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.401+0000 7f306adff700 1 -- 172.21.15.2:0/1818350740 >> 172.21.15.2:0/1818350740 conn(0x7f30640fc220 msgr2=0x7f30641077e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:22.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.401+0000 7f306adff700 1 -- 172.21.15.2:0/1818350740 shutdown_connections 2023-09-13T04:38:22.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.402+0000 7f306adff700 1 -- 172.21.15.2:0/1818350740 wait complete. 2023-09-13T04:38:22.422 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-13T04:38:22.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.631+0000 7fd41efff700 1 -- 172.21.15.2:0/4000793303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd418100ce0 msgr2=0x7fd418101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:22.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.631+0000 7fd41efff700 1 --2- 172.21.15.2:0/4000793303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd418100ce0 0x7fd418101100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009920 tx=0x7fd40c009c30 comp rx=0 tx=0).stop 2023-09-13T04:38:22.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.631+0000 7fd41efff700 1 -- 172.21.15.2:0/4000793303 shutdown_connections 2023-09-13T04:38:22.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.631+0000 7fd41efff700 1 --2- 172.21.15.2:0/4000793303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd418102870 0x7fd418109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:22.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.631+0000 7fd41efff700 1 --2- 172.21.15.2:0/4000793303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd418100ce0 0x7fd418101100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:22.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.631+0000 7fd41efff700 1 --2- 172.21.15.2:0/4000793303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd418101ed0 0x7fd418102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:22.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.631+0000 7fd41efff700 1 -- 172.21.15.2:0/4000793303 >> 172.21.15.2:0/4000793303 conn(0x7fd4180fc230 msgr2=0x7fd4180fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:22.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.631+0000 7fd41efff700 1 -- 172.21.15.2:0/4000793303 shutdown_connections 2023-09-13T04:38:22.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.632+0000 7fd41efff700 1 -- 172.21.15.2:0/4000793303 wait complete. 2023-09-13T04:38:22.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.632+0000 7fd41efff700 1 Processor -- start 2023-09-13T04:38:22.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.633+0000 7fd41efff700 1 -- start start 2023-09-13T04:38:22.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.633+0000 7fd41efff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd418100ce0 0x7fd418074a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:22.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.633+0000 7fd41efff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd418101ed0 0x7fd4180730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:22.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.633+0000 7fd41efff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd418102870 0x7fd4180735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:22.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.633+0000 7fd41efff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd418073d80 con 0x7fd418100ce0 2023-09-13T04:38:22.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.633+0000 7fd41efff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd418073ef0 con 0x7fd418102870 2023-09-13T04:38:22.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.633+0000 7fd41efff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4181a0b20 con 0x7fd418101ed0 2023-09-13T04:38:22.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.633+0000 7fd41cd9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd418100ce0 0x7fd418074a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:22.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.633+0000 7fd41cd9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd418100ce0 0x7fd418074a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43692/0 (socket says 172.21.15.2:43692) 2023-09-13T04:38:22.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.633+0000 7fd41cd9b700 1 -- 172.21.15.2:0/939282833 learned_addr learned my addr 172.21.15.2:0/939282833 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:22.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.633+0000 7fd41d59c700 1 --2- 172.21.15.2:0/939282833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd418102870 0x7fd4180735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:22.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.633+0000 7fd417fff700 1 --2- 172.21.15.2:0/939282833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd418101ed0 0x7fd4180730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:22.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.634+0000 7fd41cd9b700 1 -- 172.21.15.2:0/939282833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd418101ed0 msgr2=0x7fd4180730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:22.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.634+0000 7fd41cd9b700 1 --2- 172.21.15.2:0/939282833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd418101ed0 0x7fd4180730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:22.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.634+0000 7fd41cd9b700 1 -- 172.21.15.2:0/939282833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd418102870 msgr2=0x7fd4180735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:22.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.634+0000 7fd41cd9b700 1 --2- 172.21.15.2:0/939282833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd418102870 0x7fd4180735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:22.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.634+0000 7fd41cd9b700 1 -- 172.21.15.2:0/939282833 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c009520 con 0x7fd418100ce0 2023-09-13T04:38:22.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.634+0000 7fd41cd9b700 1 --2- 172.21.15.2:0/939282833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd418100ce0 0x7fd418074a60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009230 tx=0x7fd40c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:22.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.634+0000 7fd415ffb700 1 -- 172.21.15.2:0/939282833 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c01c070 con 0x7fd418100ce0 2023-09-13T04:38:22.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.634+0000 7fd41efff700 1 -- 172.21.15.2:0/939282833 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4181a0da0 con 0x7fd418100ce0 2023-09-13T04:38:22.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.634+0000 7fd415ffb700 1 -- 172.21.15.2:0/939282833 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd40c00f7d0 con 0x7fd418100ce0 2023-09-13T04:38:22.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.634+0000 7fd415ffb700 1 -- 172.21.15.2:0/939282833 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c004990 con 0x7fd418100ce0 2023-09-13T04:38:22.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.634+0000 7fd41efff700 1 -- 172.21.15.2:0/939282833 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4181a1170 con 0x7fd418100ce0 2023-09-13T04:38:22.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.635+0000 7fd41efff700 1 -- 172.21.15.2:0/939282833 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd418107d00 con 0x7fd418100ce0 2023-09-13T04:38:22.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.637+0000 7fd415ffb700 1 -- 172.21.15.2:0/939282833 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fd40c020070 con 0x7fd418100ce0 2023-09-13T04:38:22.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.638+0000 7fd415ffb700 1 --2- 172.21.15.2:0/939282833 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd4080a0640 0x7fd4080a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:22.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.639+0000 7fd415ffb700 1 -- 172.21.15.2:0/939282833 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd40c0ccbf0 con 0x7fd418100ce0 2023-09-13T04:38:22.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.639+0000 7fd417fff700 1 --2- 172.21.15.2:0/939282833 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd4080a0640 0x7fd4080a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:22.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.639+0000 7fd417fff700 1 --2- 172.21.15.2:0/939282833 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd4080a0640 0x7fd4080a2b00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd404009bf0 tx=0x7fd404009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:22.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.642+0000 7fd415ffb700 1 -- 172.21.15.2:0/939282833 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd40c096560 con 0x7fd418100ce0 2023-09-13T04:38:22.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:22 smithi002 ceph-mon[97651]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-09-13T04:38:22.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:22 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2065859905' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:22.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:22 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1818350740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:22.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:22 smithi149 ceph-mon[133315]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-09-13T04:38:22.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:22 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2065859905' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:22.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:22 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1818350740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:22.905+0000 7fd41efff700 1 -- 172.21.15.2:0/939282833 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd41802ce90 con 0x7fd418100ce0 2023-09-13T04:38:22.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:22 smithi156 ceph-mon[133675]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-09-13T04:38:22.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:22 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2065859905' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:22.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:22 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1818350740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:23.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.588+0000 7fd415ffb700 1 -- 172.21.15.2:0/939282833 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd40c099b80 con 0x7fd418100ce0 2023-09-13T04:38:23.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.595+0000 7fd41efff700 1 -- 172.21.15.2:0/939282833 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd4080a0640 msgr2=0x7fd4080a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:23.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.595+0000 7fd41efff700 1 --2- 172.21.15.2:0/939282833 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd4080a0640 0x7fd4080a2b00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd404009bf0 tx=0x7fd404009260 comp rx=0 tx=0).stop 2023-09-13T04:38:23.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.595+0000 7fd41efff700 1 -- 172.21.15.2:0/939282833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd418100ce0 msgr2=0x7fd418074a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:23.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.595+0000 7fd41efff700 1 --2- 172.21.15.2:0/939282833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd418100ce0 0x7fd418074a60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009230 tx=0x7fd40c00b790 comp rx=0 tx=0).stop 2023-09-13T04:38:23.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.596+0000 7fd41efff700 1 -- 172.21.15.2:0/939282833 shutdown_connections 2023-09-13T04:38:23.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.596+0000 7fd41efff700 1 --2- 172.21.15.2:0/939282833 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd4080a0640 0x7fd4080a2b00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:23.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.596+0000 7fd41efff700 1 --2- 172.21.15.2:0/939282833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd418102870 0x7fd4180735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:23.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.596+0000 7fd41efff700 1 --2- 172.21.15.2:0/939282833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd418100ce0 0x7fd418074a60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:23.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.596+0000 7fd41efff700 1 --2- 172.21.15.2:0/939282833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd418101ed0 0x7fd4180730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:23.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.596+0000 7fd41efff700 1 -- 172.21.15.2:0/939282833 >> 172.21.15.2:0/939282833 conn(0x7fd4180fc230 msgr2=0x7fd4180fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:23.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.597+0000 7fd41efff700 1 -- 172.21.15.2:0/939282833 shutdown_connections 2023-09-13T04:38:23.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.597+0000 7fd41efff700 1 -- 172.21.15.2:0/939282833 wait complete. 2023-09-13T04:38:23.621 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True} 2023-09-13T04:38:23.621 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-09-13T04:38:23.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.833+0000 7f4dbc1ff700 1 -- 172.21.15.2:0/449017221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4db4101ed0 msgr2=0x7f4db4102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:23.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.833+0000 7f4dbc1ff700 1 --2- 172.21.15.2:0/449017221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4db4101ed0 0x7f4db4102330 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4da40099c0 tx=0x7f4da4009cd0 comp rx=0 tx=0).stop 2023-09-13T04:38:23.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.833+0000 7f4dbc1ff700 1 -- 172.21.15.2:0/449017221 shutdown_connections 2023-09-13T04:38:23.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.833+0000 7f4dbc1ff700 1 --2- 172.21.15.2:0/449017221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4db4102870 0x7f4db4109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:23.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.833+0000 7f4dbc1ff700 1 --2- 172.21.15.2:0/449017221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4db4100ce0 0x7f4db4101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:23.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.833+0000 7f4dbc1ff700 1 --2- 172.21.15.2:0/449017221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4db4101ed0 0x7f4db4102330 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:23.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.833+0000 7f4dbc1ff700 1 -- 172.21.15.2:0/449017221 >> 172.21.15.2:0/449017221 conn(0x7f4db40fc230 msgr2=0x7f4db40fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:23.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.833+0000 7f4dbc1ff700 1 -- 172.21.15.2:0/449017221 shutdown_connections 2023-09-13T04:38:23.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.833+0000 7f4dbc1ff700 1 -- 172.21.15.2:0/449017221 wait complete. 2023-09-13T04:38:23.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.834+0000 7f4dbc1ff700 1 Processor -- start 2023-09-13T04:38:23.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.834+0000 7f4dbc1ff700 1 -- start start 2023-09-13T04:38:23.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.835+0000 7f4dbc1ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4db4100ce0 0x7f4db4198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:23.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.835+0000 7f4dbc1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4db4101ed0 0x7f4db41990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:23.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.835+0000 7f4dbc1ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4db4102870 0x7f4db41a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:23.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.835+0000 7f4dbc1ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db41a0700 con 0x7f4db4100ce0 2023-09-13T04:38:23.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.835+0000 7f4dbc1ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db41a0840 con 0x7f4db4102870 2023-09-13T04:38:23.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.835+0000 7f4dbc1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db41a09d0 con 0x7f4db4101ed0 2023-09-13T04:38:23.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.835+0000 7f4db9f9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4db4100ce0 0x7f4db4198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:23.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.835+0000 7f4db9f9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4db4100ce0 0x7f4db4198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43704/0 (socket says 172.21.15.2:43704) 2023-09-13T04:38:23.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.835+0000 7f4db9f9b700 1 -- 172.21.15.2:0/406342826 learned_addr learned my addr 172.21.15.2:0/406342826 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:23.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.835+0000 7f4db979a700 1 --2- 172.21.15.2:0/406342826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4db4101ed0 0x7f4db41990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:23.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.835+0000 7f4dba79c700 1 --2- 172.21.15.2:0/406342826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4db4102870 0x7f4db41a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:23.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.835+0000 7f4db9f9b700 1 -- 172.21.15.2:0/406342826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4db4101ed0 msgr2=0x7f4db41990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:23.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.835+0000 7f4db9f9b700 1 --2- 172.21.15.2:0/406342826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4db4101ed0 0x7f4db41990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:23.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.835+0000 7f4db9f9b700 1 -- 172.21.15.2:0/406342826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4db4102870 msgr2=0x7f4db41a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:23.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.835+0000 7f4db9f9b700 1 --2- 172.21.15.2:0/406342826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4db4102870 0x7f4db41a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:23.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.835+0000 7f4db9f9b700 1 -- 172.21.15.2:0/406342826 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da40095f0 con 0x7f4db4100ce0 2023-09-13T04:38:23.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.836+0000 7f4db9f9b700 1 --2- 172.21.15.2:0/406342826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4db4100ce0 0x7f4db4198b80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4db000e970 tx=0x7f4db000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:23.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.836+0000 7f4daaffd700 1 -- 172.21.15.2:0/406342826 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db0009790 con 0x7f4db4100ce0 2023-09-13T04:38:23.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.836+0000 7f4daaffd700 1 -- 172.21.15.2:0/406342826 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4db000eec0 con 0x7f4db4100ce0 2023-09-13T04:38:23.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.836+0000 7f4dbc1ff700 1 -- 172.21.15.2:0/406342826 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db41a0cb0 con 0x7f4db4100ce0 2023-09-13T04:38:23.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.836+0000 7f4daaffd700 1 -- 172.21.15.2:0/406342826 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db0003cf0 con 0x7f4db4100ce0 2023-09-13T04:38:23.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.836+0000 7f4dbc1ff700 1 -- 172.21.15.2:0/406342826 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db41a11c0 con 0x7f4db4100ce0 2023-09-13T04:38:23.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.837+0000 7f4dbc1ff700 1 -- 172.21.15.2:0/406342826 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4db410c540 con 0x7f4db4100ce0 2023-09-13T04:38:23.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.839+0000 7f4daaffd700 1 -- 172.21.15.2:0/406342826 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f4db000ccd0 con 0x7f4db4100ce0 2023-09-13T04:38:23.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.840+0000 7f4daaffd700 1 --2- 172.21.15.2:0/406342826 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4da00a06c0 0x7f4da00a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:23.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.841+0000 7f4daaffd700 1 -- 172.21.15.2:0/406342826 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4db0014070 con 0x7f4db4100ce0 2023-09-13T04:38:23.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.841+0000 7f4db979a700 1 --2- 172.21.15.2:0/406342826 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4da00a06c0 0x7f4da00a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:23.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.841+0000 7f4db979a700 1 --2- 172.21.15.2:0/406342826 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4da00a06c0 0x7f4da00a2b80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4da400b580 tx=0x7f4da4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:23.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:23.844+0000 7f4daaffd700 1 -- 172.21.15.2:0/406342826 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4db00958b0 con 0x7f4db4100ce0 2023-09-13T04:38:23.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:23 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/939282833' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-13T04:38:23.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-09-13T04:38:23.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-09-13T04:38:23.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2023-09-13T04:38:23.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2023-09-13T04:38:23.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2023-09-13T04:38:23.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-09-13T04:38:23.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-09-13T04:38:23.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]: dispatch 2023-09-13T04:38:23.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2023-09-13T04:38:23.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:23 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 10]}]: dispatch 2023-09-13T04:38:23.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:23 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/939282833' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-13T04:38:23.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-09-13T04:38:23.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-09-13T04:38:23.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2023-09-13T04:38:23.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2023-09-13T04:38:23.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2023-09-13T04:38:23.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-09-13T04:38:23.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-09-13T04:38:23.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]: dispatch 2023-09-13T04:38:23.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2023-09-13T04:38:23.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:23 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 10]}]: dispatch 2023-09-13T04:38:24.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:23 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/939282833' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-13T04:38:24.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-09-13T04:38:24.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-09-13T04:38:24.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2023-09-13T04:38:24.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2023-09-13T04:38:24.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2023-09-13T04:38:24.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-09-13T04:38:24.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-09-13T04:38:24.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]: dispatch 2023-09-13T04:38:24.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2023-09-13T04:38:24.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:23 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 10]}]: dispatch 2023-09-13T04:38:24.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.102+0000 7f4dbc1ff700 1 -- 172.21.15.2:0/406342826 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f4db41a13b0 con 0x7f4db4100ce0 2023-09-13T04:38:24.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.232+0000 7f4daaffd700 1 -- 172.21.15.2:0/406342826 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f4db0098ed0 con 0x7f4db4100ce0 2023-09-13T04:38:24.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.237+0000 7f4dbc1ff700 1 -- 172.21.15.2:0/406342826 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4da00a06c0 msgr2=0x7f4da00a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:24.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.237+0000 7f4dbc1ff700 1 --2- 172.21.15.2:0/406342826 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4da00a06c0 0x7f4da00a2b80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4da400b580 tx=0x7f4da4005fb0 comp rx=0 tx=0).stop 2023-09-13T04:38:24.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.237+0000 7f4dbc1ff700 1 -- 172.21.15.2:0/406342826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4db4100ce0 msgr2=0x7f4db4198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:24.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.237+0000 7f4dbc1ff700 1 --2- 172.21.15.2:0/406342826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4db4100ce0 0x7f4db4198b80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4db000e970 tx=0x7f4db000ec80 comp rx=0 tx=0).stop 2023-09-13T04:38:24.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.237+0000 7f4dbc1ff700 1 -- 172.21.15.2:0/406342826 shutdown_connections 2023-09-13T04:38:24.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.237+0000 7f4dbc1ff700 1 --2- 172.21.15.2:0/406342826 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4da00a06c0 0x7f4da00a2b80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:24.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.237+0000 7f4dbc1ff700 1 --2- 172.21.15.2:0/406342826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4db4102870 0x7f4db41a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:24.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.237+0000 7f4dbc1ff700 1 --2- 172.21.15.2:0/406342826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4db4100ce0 0x7f4db4198b80 secure :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4db000e970 tx=0x7f4db000ec80 comp rx=0 tx=0).stop 2023-09-13T04:38:24.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.237+0000 7f4dbc1ff700 1 --2- 172.21.15.2:0/406342826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4db4101ed0 0x7f4db41990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:24.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.237+0000 7f4dbc1ff700 1 -- 172.21.15.2:0/406342826 >> 172.21.15.2:0/406342826 conn(0x7f4db40fc230 msgr2=0x7f4db40fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:24.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.238+0000 7f4dbc1ff700 1 -- 172.21.15.2:0/406342826 shutdown_connections 2023-09-13T04:38:24.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.238+0000 7f4dbc1ff700 1 -- 172.21.15.2:0/406342826 wait complete. 2023-09-13T04:38:24.262 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-09-13T04:38:24.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.470+0000 7f0afa9ff700 1 -- 172.21.15.2:0/1313644344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0af4075670 msgr2=0x7f0af4075a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:24.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.470+0000 7f0afa9ff700 1 --2- 172.21.15.2:0/1313644344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0af4075670 0x7f0af4075a90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4009990 tx=0x7f0ae4009ca0 comp rx=0 tx=0).stop 2023-09-13T04:38:24.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.470+0000 7f0afa9ff700 1 -- 172.21.15.2:0/1313644344 shutdown_connections 2023-09-13T04:38:24.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.470+0000 7f0afa9ff700 1 --2- 172.21.15.2:0/1313644344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0af4077270 0x7f0af4077af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:24.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.470+0000 7f0afa9ff700 1 --2- 172.21.15.2:0/1313644344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0af4075670 0x7f0af4075a90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:24.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.470+0000 7f0afa9ff700 1 --2- 172.21.15.2:0/1313644344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0af40768b0 0x7f0af4076d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:24.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.470+0000 7f0afa9ff700 1 -- 172.21.15.2:0/1313644344 >> 172.21.15.2:0/1313644344 conn(0x7f0af40fe460 msgr2=0x7f0af41008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:24.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.470+0000 7f0afa9ff700 1 -- 172.21.15.2:0/1313644344 shutdown_connections 2023-09-13T04:38:24.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.470+0000 7f0afa9ff700 1 -- 172.21.15.2:0/1313644344 wait complete. 2023-09-13T04:38:24.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.471+0000 7f0afa9ff700 1 Processor -- start 2023-09-13T04:38:24.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.472+0000 7f0afa9ff700 1 -- start start 2023-09-13T04:38:24.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.472+0000 7f0afa9ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0af4075670 0x7f0af419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:24.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.472+0000 7f0afa9ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0af40768b0 0x7f0af419d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:24.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.472+0000 7f0afa9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0af4077270 0x7f0af41a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:24.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.472+0000 7f0afa9ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af41a4b90 con 0x7f0af40768b0 2023-09-13T04:38:24.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.472+0000 7f0afa9ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af41a4cd0 con 0x7f0af4075670 2023-09-13T04:38:24.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.472+0000 7f0afa9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af41a4e60 con 0x7f0af4077270 2023-09-13T04:38:24.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.472+0000 7f0af37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0af40768b0 0x7f0af419d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:24.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.472+0000 7f0af37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0af40768b0 0x7f0af419d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38030/0 (socket says 172.21.15.2:38030) 2023-09-13T04:38:24.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.472+0000 7f0af37fe700 1 -- 172.21.15.2:0/2666637494 learned_addr learned my addr 172.21.15.2:0/2666637494 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:24.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.472+0000 7f0af3fff700 1 --2- 172.21.15.2:0/2666637494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0af4075670 0x7f0af419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:24.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.473+0000 7f0af8f9c700 1 --2- 172.21.15.2:0/2666637494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0af4077270 0x7f0af41a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:24.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.473+0000 7f0af37fe700 1 -- 172.21.15.2:0/2666637494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0af4077270 msgr2=0x7f0af41a45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:24.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.473+0000 7f0af37fe700 1 --2- 172.21.15.2:0/2666637494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0af4077270 0x7f0af41a45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:24.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.473+0000 7f0af37fe700 1 -- 172.21.15.2:0/2666637494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0af4075670 msgr2=0x7f0af419d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:24.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.473+0000 7f0af37fe700 1 --2- 172.21.15.2:0/2666637494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0af4075670 0x7f0af419d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:24.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.473+0000 7f0af37fe700 1 -- 172.21.15.2:0/2666637494 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae40095f0 con 0x7f0af40768b0 2023-09-13T04:38:24.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.473+0000 7f0af37fe700 1 --2- 172.21.15.2:0/2666637494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0af40768b0 0x7f0af419d550 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800e9a0 tx=0x7f0ae800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:24.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.473+0000 7f0af17fa700 1 -- 172.21.15.2:0/2666637494 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8009790 con 0x7f0af40768b0 2023-09-13T04:38:24.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.473+0000 7f0af17fa700 1 -- 172.21.15.2:0/2666637494 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ae8010430 con 0x7f0af40768b0 2023-09-13T04:38:24.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.473+0000 7f0afa9ff700 1 -- 172.21.15.2:0/2666637494 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af41a5140 con 0x7f0af40768b0 2023-09-13T04:38:24.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.473+0000 7f0af17fa700 1 -- 172.21.15.2:0/2666637494 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8018680 con 0x7f0af40768b0 2023-09-13T04:38:24.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.474+0000 7f0afa9ff700 1 -- 172.21.15.2:0/2666637494 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af41a5650 con 0x7f0af40768b0 2023-09-13T04:38:24.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.475+0000 7f0afa9ff700 1 -- 172.21.15.2:0/2666637494 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af410f910 con 0x7f0af40768b0 2023-09-13T04:38:24.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.477+0000 7f0af17fa700 1 -- 172.21.15.2:0/2666637494 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f0ae80105a0 con 0x7f0af40768b0 2023-09-13T04:38:24.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.478+0000 7f0af17fa700 1 --2- 172.21.15.2:0/2666637494 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f0adc0a0640 0x7f0adc0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:24.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.478+0000 7f0af17fa700 1 -- 172.21.15.2:0/2666637494 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0ae8014070 con 0x7f0af40768b0 2023-09-13T04:38:24.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.478+0000 7f0af3fff700 1 --2- 172.21.15.2:0/2666637494 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f0adc0a0640 0x7f0adc0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:24.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.479+0000 7f0af3fff700 1 --2- 172.21.15.2:0/2666637494 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f0adc0a0640 0x7f0adc0a2b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0ae400b2e0 tx=0x7f0ae4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:24.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.480+0000 7f0af17fa700 1 -- 172.21.15.2:0/2666637494 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0ae8095a50 con 0x7f0af40768b0 2023-09-13T04:38:24.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:24.752+0000 7f0afa9ff700 1 -- 172.21.15.2:0/2666637494 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f0af4112ba0 con 0x7f0af40768b0 2023-09-13T04:38:24.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:24 smithi149 ceph-mon[133315]: pgmap v288: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-09-13T04:38:24.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:24 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/939282833' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-13T04:38:24.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:24 smithi149 ceph-mon[133315]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-13T04:38:24.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-09-13T04:38:24.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-09-13T04:38:24.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2023-09-13T04:38:24.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2023-09-13T04:38:24.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2023-09-13T04:38:24.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-09-13T04:38:24.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-09-13T04:38:24.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]': finished 2023-09-13T04:38:24.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2023-09-13T04:38:24.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 10]}]': finished 2023-09-13T04:38:24.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:24 smithi149 ceph-mon[133315]: osdmap e89: 12 total, 12 up, 12 in 2023-09-13T04:38:24.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:24 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/406342826' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-09-13T04:38:24.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:24 smithi149 ceph-mon[133315]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-13T04:38:24.890 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:24 smithi149 ceph-mon[133315]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-13T04:38:24.890 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:24 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/406342826' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-09-13T04:38:24.890 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:24 smithi149 ceph-mon[133315]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2023-09-13T04:38:24.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:24 smithi156 ceph-mon[133675]: pgmap v288: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-09-13T04:38:24.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:24 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/939282833' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-13T04:38:24.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:24 smithi156 ceph-mon[133675]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-13T04:38:24.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-09-13T04:38:24.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-09-13T04:38:24.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2023-09-13T04:38:24.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2023-09-13T04:38:24.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2023-09-13T04:38:24.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-09-13T04:38:24.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-09-13T04:38:24.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]': finished 2023-09-13T04:38:24.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2023-09-13T04:38:24.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 10]}]': finished 2023-09-13T04:38:24.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:24 smithi156 ceph-mon[133675]: osdmap e89: 12 total, 12 up, 12 in 2023-09-13T04:38:24.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:24 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/406342826' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-09-13T04:38:24.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:24 smithi156 ceph-mon[133675]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-13T04:38:24.940 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:24 smithi156 ceph-mon[133675]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-13T04:38:24.940 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:24 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/406342826' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-09-13T04:38:24.940 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:24 smithi156 ceph-mon[133675]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2023-09-13T04:38:25.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:24 smithi002 ceph-mon[97651]: pgmap v288: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-09-13T04:38:25.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:24 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/939282833' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-13T04:38:25.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:24 smithi002 ceph-mon[97651]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-13T04:38:25.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-09-13T04:38:25.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-09-13T04:38:25.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2023-09-13T04:38:25.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2023-09-13T04:38:25.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2023-09-13T04:38:25.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-09-13T04:38:25.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-09-13T04:38:25.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]': finished 2023-09-13T04:38:25.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2023-09-13T04:38:25.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 10]}]': finished 2023-09-13T04:38:25.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:24 smithi002 ceph-mon[97651]: osdmap e89: 12 total, 12 up, 12 in 2023-09-13T04:38:25.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:24 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/406342826' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-09-13T04:38:25.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:24 smithi002 ceph-mon[97651]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-13T04:38:25.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:24 smithi002 ceph-mon[97651]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-13T04:38:25.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:24 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/406342826' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-09-13T04:38:25.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:24 smithi002 ceph-mon[97651]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2023-09-13T04:38:25.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.602+0000 7f0af17fa700 1 -- 172.21.15.2:0/2666637494 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f0ae80260c0 con 0x7f0af40768b0 2023-09-13T04:38:25.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.605+0000 7f0afa9ff700 1 -- 172.21.15.2:0/2666637494 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f0adc0a0640 msgr2=0x7f0adc0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:25.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.605+0000 7f0afa9ff700 1 --2- 172.21.15.2:0/2666637494 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f0adc0a0640 0x7f0adc0a2b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0ae400b2e0 tx=0x7f0ae4005fb0 comp rx=0 tx=0).stop 2023-09-13T04:38:25.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.605+0000 7f0afa9ff700 1 -- 172.21.15.2:0/2666637494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0af40768b0 msgr2=0x7f0af419d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:25.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.605+0000 7f0afa9ff700 1 --2- 172.21.15.2:0/2666637494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0af40768b0 0x7f0af419d550 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800e9a0 tx=0x7f0ae800ecb0 comp rx=0 tx=0).stop 2023-09-13T04:38:25.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.605+0000 7f0afa9ff700 1 -- 172.21.15.2:0/2666637494 shutdown_connections 2023-09-13T04:38:25.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.605+0000 7f0afa9ff700 1 --2- 172.21.15.2:0/2666637494 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f0adc0a0640 0x7f0adc0a2b00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:25.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.605+0000 7f0afa9ff700 1 --2- 172.21.15.2:0/2666637494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0af4077270 0x7f0af41a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:25.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.605+0000 7f0afa9ff700 1 --2- 172.21.15.2:0/2666637494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0af4075670 0x7f0af419d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:25.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.605+0000 7f0afa9ff700 1 --2- 172.21.15.2:0/2666637494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0af40768b0 0x7f0af419d550 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:25.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.605+0000 7f0afa9ff700 1 -- 172.21.15.2:0/2666637494 >> 172.21.15.2:0/2666637494 conn(0x7f0af40fe460 msgr2=0x7f0af41001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:25.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.606+0000 7f0afa9ff700 1 -- 172.21.15.2:0/2666637494 shutdown_connections 2023-09-13T04:38:25.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.606+0000 7f0afa9ff700 1 -- 172.21.15.2:0/2666637494 wait complete. 2023-09-13T04:38:25.634 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-09-13T04:38:25.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.844+0000 7fc4171ff700 1 -- 172.21.15.2:0/1783336442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc410100ce0 msgr2=0x7fc410101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:25.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.844+0000 7fc4171ff700 1 --2- 172.21.15.2:0/1783336442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc410100ce0 0x7fc410101100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc4040099c0 tx=0x7fc404009cd0 comp rx=0 tx=0).stop 2023-09-13T04:38:25.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.844+0000 7fc4171ff700 1 -- 172.21.15.2:0/1783336442 shutdown_connections 2023-09-13T04:38:25.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.844+0000 7fc4171ff700 1 --2- 172.21.15.2:0/1783336442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc410102870 0x7fc410109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:25.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.844+0000 7fc4171ff700 1 --2- 172.21.15.2:0/1783336442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc410100ce0 0x7fc410101100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:25.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.844+0000 7fc4171ff700 1 --2- 172.21.15.2:0/1783336442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc410101ed0 0x7fc410102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:25.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.844+0000 7fc4171ff700 1 -- 172.21.15.2:0/1783336442 >> 172.21.15.2:0/1783336442 conn(0x7fc4100fc230 msgr2=0x7fc4100fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:25.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.845+0000 7fc4171ff700 1 -- 172.21.15.2:0/1783336442 shutdown_connections 2023-09-13T04:38:25.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.845+0000 7fc4171ff700 1 -- 172.21.15.2:0/1783336442 wait complete. 2023-09-13T04:38:25.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.845+0000 7fc4171ff700 1 Processor -- start 2023-09-13T04:38:25.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.846+0000 7fc4171ff700 1 -- start start 2023-09-13T04:38:25.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.846+0000 7fc4171ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc410100ce0 0x7fc410198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:25.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.846+0000 7fc4171ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc410101ed0 0x7fc410199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:25.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.846+0000 7fc4171ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc410102870 0x7fc4101a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:25.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.846+0000 7fc4171ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4101a0780 con 0x7fc410102870 2023-09-13T04:38:25.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.846+0000 7fc4171ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4101a08f0 con 0x7fc410101ed0 2023-09-13T04:38:25.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.846+0000 7fc4171ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4101a0ab0 con 0x7fc410100ce0 2023-09-13T04:38:25.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.846+0000 7fc41579c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc410102870 0x7fc4101a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:25.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.846+0000 7fc41579c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc410102870 0x7fc4101a01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38044/0 (socket says 172.21.15.2:38044) 2023-09-13T04:38:25.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.846+0000 7fc41579c700 1 -- 172.21.15.2:0/1828402985 learned_addr learned my addr 172.21.15.2:0/1828402985 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:25.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.846+0000 7fc414f9b700 1 --2- 172.21.15.2:0/1828402985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc410100ce0 0x7fc410198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:25.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.846+0000 7fc40ffff700 1 --2- 172.21.15.2:0/1828402985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc410101ed0 0x7fc410199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:25.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.847+0000 7fc41579c700 1 -- 172.21.15.2:0/1828402985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc410100ce0 msgr2=0x7fc410198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:25.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.847+0000 7fc41579c700 1 --2- 172.21.15.2:0/1828402985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc410100ce0 0x7fc410198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:25.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.847+0000 7fc41579c700 1 -- 172.21.15.2:0/1828402985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc410101ed0 msgr2=0x7fc410199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:25.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.847+0000 7fc41579c700 1 --2- 172.21.15.2:0/1828402985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc410101ed0 0x7fc410199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:25.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.847+0000 7fc41579c700 1 -- 172.21.15.2:0/1828402985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4040095f0 con 0x7fc410102870 2023-09-13T04:38:25.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.847+0000 7fc41579c700 1 --2- 172.21.15.2:0/1828402985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc410102870 0x7fc4101a01b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc40800ef30 tx=0x7fc40800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:25.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.847+0000 7fc40dffb700 1 -- 172.21.15.2:0/1828402985 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc408018070 con 0x7fc410102870 2023-09-13T04:38:25.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.847+0000 7fc40dffb700 1 -- 172.21.15.2:0/1828402985 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc40800f040 con 0x7fc410102870 2023-09-13T04:38:25.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.847+0000 7fc4171ff700 1 -- 172.21.15.2:0/1828402985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4101a0d30 con 0x7fc410102870 2023-09-13T04:38:25.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.847+0000 7fc40dffb700 1 -- 172.21.15.2:0/1828402985 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40800cd00 con 0x7fc410102870 2023-09-13T04:38:25.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.848+0000 7fc4171ff700 1 -- 172.21.15.2:0/1828402985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4101a11c0 con 0x7fc410102870 2023-09-13T04:38:25.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.849+0000 7fc4171ff700 1 -- 172.21.15.2:0/1828402985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc410193330 con 0x7fc410102870 2023-09-13T04:38:25.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.850+0000 7fc40dffb700 1 -- 172.21.15.2:0/1828402985 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fc408003a00 con 0x7fc410102870 2023-09-13T04:38:25.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.852+0000 7fc40dffb700 1 --2- 172.21.15.2:0/1828402985 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc4000a06c0 0x7fc4000a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:25.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.852+0000 7fc40dffb700 1 -- 172.21.15.2:0/1828402985 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc4080cba00 con 0x7fc410102870 2023-09-13T04:38:25.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.852+0000 7fc414f9b700 1 --2- 172.21.15.2:0/1828402985 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc4000a06c0 0x7fc4000a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:25.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.853+0000 7fc414f9b700 1 --2- 172.21.15.2:0/1828402985 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc4000a06c0 0x7fc4000a2b80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc404009990 tx=0x7fc40401a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:25.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:25.856+0000 7fc40dffb700 1 -- 172.21.15.2:0/1828402985 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc408095250 con 0x7fc410102870 2023-09-13T04:38:25.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:25 smithi149 ceph-mon[133315]: osdmap e90: 12 total, 12 up, 12 in 2023-09-13T04:38:25.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:25 smithi149 ceph-mon[133315]: daemon mds.a is now active in filesystem cephfs as rank 1 2023-09-13T04:38:25.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:25 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2666637494' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-13T04:38:25.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:25 smithi156 ceph-mon[133675]: osdmap e90: 12 total, 12 up, 12 in 2023-09-13T04:38:25.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:25 smithi156 ceph-mon[133675]: daemon mds.a is now active in filesystem cephfs as rank 1 2023-09-13T04:38:25.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:25 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2666637494' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-13T04:38:26.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:25 smithi002 ceph-mon[97651]: osdmap e90: 12 total, 12 up, 12 in 2023-09-13T04:38:26.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:25 smithi002 ceph-mon[97651]: daemon mds.a is now active in filesystem cephfs as rank 1 2023-09-13T04:38:26.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:25 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2666637494' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-13T04:38:26.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.119+0000 7fc4171ff700 1 -- 172.21.15.2:0/1828402985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fc41010c540 con 0x7fc410102870 2023-09-13T04:38:26.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.608+0000 7fc40dffb700 1 -- 172.21.15.2:0/1828402985 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7fc408098870 con 0x7fc410102870 2023-09-13T04:38:26.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.611+0000 7fc4171ff700 1 -- 172.21.15.2:0/1828402985 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc4000a06c0 msgr2=0x7fc4000a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:26.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.611+0000 7fc4171ff700 1 --2- 172.21.15.2:0/1828402985 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc4000a06c0 0x7fc4000a2b80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc404009990 tx=0x7fc40401a040 comp rx=0 tx=0).stop 2023-09-13T04:38:26.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.611+0000 7fc4171ff700 1 -- 172.21.15.2:0/1828402985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc410102870 msgr2=0x7fc4101a01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:26.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.611+0000 7fc4171ff700 1 --2- 172.21.15.2:0/1828402985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc410102870 0x7fc4101a01b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc40800ef30 tx=0x7fc40800c3a0 comp rx=0 tx=0).stop 2023-09-13T04:38:26.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.611+0000 7fc4171ff700 1 -- 172.21.15.2:0/1828402985 shutdown_connections 2023-09-13T04:38:26.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.611+0000 7fc4171ff700 1 --2- 172.21.15.2:0/1828402985 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc4000a06c0 0x7fc4000a2b80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:26.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.611+0000 7fc4171ff700 1 --2- 172.21.15.2:0/1828402985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc410102870 0x7fc4101a01b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:26.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.611+0000 7fc4171ff700 1 --2- 172.21.15.2:0/1828402985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc410100ce0 0x7fc410198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:26.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.611+0000 7fc4171ff700 1 --2- 172.21.15.2:0/1828402985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc410101ed0 0x7fc410199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:26.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.611+0000 7fc4171ff700 1 -- 172.21.15.2:0/1828402985 >> 172.21.15.2:0/1828402985 conn(0x7fc4100fc230 msgr2=0x7fc410107810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:26.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.612+0000 7fc4171ff700 1 -- 172.21.15.2:0/1828402985 shutdown_connections 2023-09-13T04:38:26.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.612+0000 7fc4171ff700 1 -- 172.21.15.2:0/1828402985 wait complete. 2023-09-13T04:38:26.633 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-13T04:38:26.634 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:38:26.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.837+0000 7f090bdff700 1 -- 172.21.15.2:0/1246185174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0904101ee0 msgr2=0x7f0904102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:26.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.837+0000 7f090bdff700 1 --2- 172.21.15.2:0/1246185174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0904101ee0 0x7f0904102340 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f08f40099c0 tx=0x7f08f4009cd0 comp rx=0 tx=0).stop 2023-09-13T04:38:26.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.838+0000 7f090bdff700 1 -- 172.21.15.2:0/1246185174 shutdown_connections 2023-09-13T04:38:26.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.838+0000 7f090bdff700 1 --2- 172.21.15.2:0/1246185174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0904102880 0x7f0904109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:26.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.838+0000 7f090bdff700 1 --2- 172.21.15.2:0/1246185174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0904100cf0 0x7f0904101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:26.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.838+0000 7f090bdff700 1 --2- 172.21.15.2:0/1246185174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0904101ee0 0x7f0904102340 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:26.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.838+0000 7f090bdff700 1 -- 172.21.15.2:0/1246185174 >> 172.21.15.2:0/1246185174 conn(0x7f09040fc220 msgr2=0x7f09040fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:26.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.838+0000 7f090bdff700 1 -- 172.21.15.2:0/1246185174 shutdown_connections 2023-09-13T04:38:26.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.838+0000 7f090bdff700 1 -- 172.21.15.2:0/1246185174 wait complete. 2023-09-13T04:38:26.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.839+0000 7f090bdff700 1 Processor -- start 2023-09-13T04:38:26.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.839+0000 7f090bdff700 1 -- start start 2023-09-13T04:38:26.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.839+0000 7f090bdff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0904100cf0 0x7f0904198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:26.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.839+0000 7f090bdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0904101ee0 0x7f09041990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:26.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.839+0000 7f090bdff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0904102880 0x7f09041a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:26.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.839+0000 7f090bdff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09041a0730 con 0x7f0904100cf0 2023-09-13T04:38:26.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.839+0000 7f090bdff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09041a0870 con 0x7f0904102880 2023-09-13T04:38:26.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.839+0000 7f090bdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09041a0a00 con 0x7f0904101ee0 2023-09-13T04:38:26.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.840+0000 7f0909b9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0904100cf0 0x7f0904198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:26.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.840+0000 7f0909b9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0904100cf0 0x7f0904198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38070/0 (socket says 172.21.15.2:38070) 2023-09-13T04:38:26.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.840+0000 7f0909b9b700 1 -- 172.21.15.2:0/1138186360 learned_addr learned my addr 172.21.15.2:0/1138186360 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:26.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.840+0000 7f090939a700 1 --2- 172.21.15.2:0/1138186360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0904101ee0 0x7f09041990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:26.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.840+0000 7f090a39c700 1 --2- 172.21.15.2:0/1138186360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0904102880 0x7f09041a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:26.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.840+0000 7f090939a700 1 -- 172.21.15.2:0/1138186360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0904102880 msgr2=0x7f09041a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:26.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.840+0000 7f090939a700 1 --2- 172.21.15.2:0/1138186360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0904102880 0x7f09041a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:26.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.840+0000 7f090939a700 1 -- 172.21.15.2:0/1138186360 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0904100cf0 msgr2=0x7f0904198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:26.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.840+0000 7f090939a700 1 --2- 172.21.15.2:0/1138186360 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0904100cf0 0x7f0904198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:26.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.840+0000 7f090939a700 1 -- 172.21.15.2:0/1138186360 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f40095f0 con 0x7f0904101ee0 2023-09-13T04:38:26.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.840+0000 7f0909b9b700 1 --2- 172.21.15.2:0/1138186360 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0904100cf0 0x7f0904198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:38:26.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.840+0000 7f090939a700 1 --2- 172.21.15.2:0/1138186360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0904101ee0 0x7f09041990f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f08f40058e0 tx=0x7f08f400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:26.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.840+0000 7f08faffd700 1 -- 172.21.15.2:0/1138186360 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f401d070 con 0x7f0904101ee0 2023-09-13T04:38:26.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.841+0000 7f08faffd700 1 -- 172.21.15.2:0/1138186360 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08f4005050 con 0x7f0904101ee0 2023-09-13T04:38:26.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.841+0000 7f090bdff700 1 -- 172.21.15.2:0/1138186360 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09041a0c80 con 0x7f0904101ee0 2023-09-13T04:38:26.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.841+0000 7f08faffd700 1 -- 172.21.15.2:0/1138186360 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f4004a90 con 0x7f0904101ee0 2023-09-13T04:38:26.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.841+0000 7f090bdff700 1 -- 172.21.15.2:0/1138186360 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09041a1100 con 0x7f0904101ee0 2023-09-13T04:38:26.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.842+0000 7f090bdff700 1 -- 172.21.15.2:0/1138186360 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f090410c500 con 0x7f0904101ee0 2023-09-13T04:38:26.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.844+0000 7f08faffd700 1 -- 172.21.15.2:0/1138186360 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f08f4005430 con 0x7f0904101ee0 2023-09-13T04:38:26.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.845+0000 7f08faffd700 1 --2- 172.21.15.2:0/1138186360 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f08f00a0710 0x7f08f00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:26.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.845+0000 7f08faffd700 1 -- 172.21.15.2:0/1138186360 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f08f40ccdd0 con 0x7f0904101ee0 2023-09-13T04:38:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.846+0000 7f0909b9b700 1 --2- 172.21.15.2:0/1138186360 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f08f00a0710 0x7f08f00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.846+0000 7f0909b9b700 1 --2- 172.21.15.2:0/1138186360 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f08f00a0710 0x7f08f00a2bd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0900005950 tx=0x7f0900009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:26.848+0000 7f08faffd700 1 -- 172.21.15.2:0/1138186360 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f08f4096620 con 0x7f0904101ee0 2023-09-13T04:38:26.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:26 smithi149 ceph-mon[133315]: pgmap v291: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 2 op/s 2023-09-13T04:38:26.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:26 smithi149 ceph-mon[133315]: mds.? [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] up:active 2023-09-13T04:38:26.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:26 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2666637494' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-13T04:38:26.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:26 smithi149 ceph-mon[133315]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-13T04:38:26.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:26 smithi149 ceph-mon[133315]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-13T04:38:26.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:26 smithi149 ceph-mon[133315]: Cluster is now healthy 2023-09-13T04:38:26.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:26 smithi149 ceph-mon[133315]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2023-09-13T04:38:26.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:26 smithi149 ceph-mon[133315]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2023-09-13T04:38:26.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:26 smithi149 ceph-mon[133315]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-09-13T04:38:26.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:26 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1828402985' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-13T04:38:26.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:26 smithi156 ceph-mon[133675]: pgmap v291: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 2 op/s 2023-09-13T04:38:26.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:26 smithi156 ceph-mon[133675]: mds.? [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] up:active 2023-09-13T04:38:26.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:26 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2666637494' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-13T04:38:26.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:26 smithi156 ceph-mon[133675]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-13T04:38:26.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:26 smithi156 ceph-mon[133675]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-13T04:38:26.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:26 smithi156 ceph-mon[133675]: Cluster is now healthy 2023-09-13T04:38:26.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:26 smithi156 ceph-mon[133675]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2023-09-13T04:38:26.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:26 smithi156 ceph-mon[133675]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2023-09-13T04:38:26.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:26 smithi156 ceph-mon[133675]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-09-13T04:38:26.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:26 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1828402985' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-13T04:38:27.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:26 smithi002 ceph-mon[97651]: pgmap v291: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 2 op/s 2023-09-13T04:38:27.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:26 smithi002 ceph-mon[97651]: mds.? [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] up:active 2023-09-13T04:38:27.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:26 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2666637494' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-13T04:38:27.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:26 smithi002 ceph-mon[97651]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-13T04:38:27.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:26 smithi002 ceph-mon[97651]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-13T04:38:27.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:26 smithi002 ceph-mon[97651]: Cluster is now healthy 2023-09-13T04:38:27.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:26 smithi002 ceph-mon[97651]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2023-09-13T04:38:27.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:26 smithi002 ceph-mon[97651]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2023-09-13T04:38:27.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:26 smithi002 ceph-mon[97651]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-09-13T04:38:27.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:26 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1828402985' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-13T04:38:27.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.106+0000 7f090bdff700 1 -- 172.21.15.2:0/1138186360 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09041a1290 con 0x7f0904101ee0 2023-09-13T04:38:27.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.107+0000 7f08faffd700 1 -- 172.21.15.2:0/1138186360 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11303 (secure 0 0 0) 0x7f08f4099c40 con 0x7f0904101ee0 2023-09-13T04:38:27.109 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:38:27.109 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2469307307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2469307307},{"type":"v1","addr":"172.21.15.2:6839","nonce":2469307307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T04:37:59.110701+0000","modified":"2023-09-13T04:38:26.610260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24554,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/846579077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":846579077},{"type":"v1","addr":"172.21.15.156:6835","nonce":846579077}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1896833690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1896833690},{"type":"v1","addr":"172.21.15.149:6837","nonce":1896833690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/524563090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":524563090},{"type":"v1","addr":"172.21.15.156:6837","nonce":524563090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6839/1785833227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1785833227},{"type":"v1","addr":"172.21.15.156:6839","nonce":1785833227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.2:6841/482545391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":482545391},{"type":"v1","addr":"172.21.15.2:6841","nonce":482545391}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/389167841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":389167841},{"type":"v1","addr":"172.21.15.149:6839","nonce":389167841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T04:38:27.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.109+0000 7f090bdff700 1 -- 172.21.15.2:0/1138186360 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f08f00a0710 msgr2=0x7f08f00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:27.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.109+0000 7f090bdff700 1 --2- 172.21.15.2:0/1138186360 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f08f00a0710 0x7f08f00a2bd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0900005950 tx=0x7f0900009260 comp rx=0 tx=0).stop 2023-09-13T04:38:27.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.109+0000 7f090bdff700 1 -- 172.21.15.2:0/1138186360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0904101ee0 msgr2=0x7f09041990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:27.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.109+0000 7f090bdff700 1 --2- 172.21.15.2:0/1138186360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0904101ee0 0x7f09041990f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f08f40058e0 tx=0x7f08f400f690 comp rx=0 tx=0).stop 2023-09-13T04:38:27.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.110+0000 7f090bdff700 1 -- 172.21.15.2:0/1138186360 shutdown_connections 2023-09-13T04:38:27.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.110+0000 7f090bdff700 1 --2- 172.21.15.2:0/1138186360 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f08f00a0710 0x7f08f00a2bd0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:27.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.110+0000 7f090bdff700 1 --2- 172.21.15.2:0/1138186360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0904102880 0x7f09041a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:27.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.110+0000 7f090bdff700 1 --2- 172.21.15.2:0/1138186360 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0904100cf0 0x7f0904198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:27.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.110+0000 7f090bdff700 1 --2- 172.21.15.2:0/1138186360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0904101ee0 0x7f09041990f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:27.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.110+0000 7f090bdff700 1 -- 172.21.15.2:0/1138186360 >> 172.21.15.2:0/1138186360 conn(0x7f09040fc220 msgr2=0x7f09040fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:27.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.110+0000 7f090bdff700 1 -- 172.21.15.2:0/1138186360 shutdown_connections 2023-09-13T04:38:27.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.110+0000 7f090bdff700 1 -- 172.21.15.2:0/1138186360 wait complete. 2023-09-13T04:38:27.114 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 23 2023-09-13T04:38:27.130 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T04:38:27.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.333+0000 7f92defff700 1 -- 172.21.15.2:0/777313798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92d8101ee0 msgr2=0x7f92d8102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:27.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.333+0000 7f92defff700 1 --2- 172.21.15.2:0/777313798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92d8101ee0 0x7f92d8102340 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f92c40099c0 tx=0x7f92c4009cd0 comp rx=0 tx=0).stop 2023-09-13T04:38:27.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.333+0000 7f92defff700 1 -- 172.21.15.2:0/777313798 shutdown_connections 2023-09-13T04:38:27.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.333+0000 7f92defff700 1 --2- 172.21.15.2:0/777313798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92d8102880 0x7f92d8109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:27.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.333+0000 7f92defff700 1 --2- 172.21.15.2:0/777313798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92d8100cf0 0x7f92d8101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:27.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.333+0000 7f92defff700 1 --2- 172.21.15.2:0/777313798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92d8101ee0 0x7f92d8102340 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:27.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.333+0000 7f92defff700 1 -- 172.21.15.2:0/777313798 >> 172.21.15.2:0/777313798 conn(0x7f92d80fc220 msgr2=0x7f92d80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:27.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.333+0000 7f92defff700 1 -- 172.21.15.2:0/777313798 shutdown_connections 2023-09-13T04:38:27.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.333+0000 7f92defff700 1 -- 172.21.15.2:0/777313798 wait complete. 2023-09-13T04:38:27.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.334+0000 7f92defff700 1 Processor -- start 2023-09-13T04:38:27.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.334+0000 7f92defff700 1 -- start start 2023-09-13T04:38:27.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.334+0000 7f92defff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92d8100cf0 0x7f92d81969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:27.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.334+0000 7f92defff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92d8101ee0 0x7f92d8196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:27.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.334+0000 7f92defff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92d8102880 0x7f92d819df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:27.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.334+0000 7f92defff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d819e530 con 0x7f92d8100cf0 2023-09-13T04:38:27.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.334+0000 7f92defff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d819e670 con 0x7f92d8101ee0 2023-09-13T04:38:27.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.334+0000 7f92defff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d819e800 con 0x7f92d8102880 2023-09-13T04:38:27.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.334+0000 7f92dcd9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92d8100cf0 0x7f92d81969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:27.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.335+0000 7f92dcd9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92d8100cf0 0x7f92d81969b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38084/0 (socket says 172.21.15.2:38084) 2023-09-13T04:38:27.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.335+0000 7f92dcd9b700 1 -- 172.21.15.2:0/3944598609 learned_addr learned my addr 172.21.15.2:0/3944598609 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:27.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.335+0000 7f92d7fff700 1 --2- 172.21.15.2:0/3944598609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92d8101ee0 0x7f92d8196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:27.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.335+0000 7f92dd59c700 1 --2- 172.21.15.2:0/3944598609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92d8102880 0x7f92d819df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:27.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.335+0000 7f92dcd9b700 1 -- 172.21.15.2:0/3944598609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92d8102880 msgr2=0x7f92d819df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:27.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.335+0000 7f92dcd9b700 1 --2- 172.21.15.2:0/3944598609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92d8102880 0x7f92d819df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:27.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.335+0000 7f92dcd9b700 1 -- 172.21.15.2:0/3944598609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92d8101ee0 msgr2=0x7f92d8196ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:27.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.335+0000 7f92dcd9b700 1 --2- 172.21.15.2:0/3944598609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92d8101ee0 0x7f92d8196ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:27.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.335+0000 7f92dcd9b700 1 -- 172.21.15.2:0/3944598609 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c40095f0 con 0x7f92d8100cf0 2023-09-13T04:38:27.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.335+0000 7f92dcd9b700 1 --2- 172.21.15.2:0/3944598609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92d8100cf0 0x7f92d81969b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f92cc00e970 tx=0x7f92cc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:27.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.335+0000 7f92d5ffb700 1 -- 172.21.15.2:0/3944598609 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92cc009790 con 0x7f92d8100cf0 2023-09-13T04:38:27.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.335+0000 7f92d5ffb700 1 -- 172.21.15.2:0/3944598609 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92cc00eec0 con 0x7f92d8100cf0 2023-09-13T04:38:27.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.335+0000 7f92defff700 1 -- 172.21.15.2:0/3944598609 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d819eae0 con 0x7f92d8100cf0 2023-09-13T04:38:27.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.335+0000 7f92d5ffb700 1 -- 172.21.15.2:0/3944598609 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92cc003cf0 con 0x7f92d8100cf0 2023-09-13T04:38:27.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.336+0000 7f92defff700 1 -- 172.21.15.2:0/3944598609 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d819f040 con 0x7f92d8100cf0 2023-09-13T04:38:27.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.337+0000 7f92defff700 1 -- 172.21.15.2:0/3944598609 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92d810c550 con 0x7f92d8100cf0 2023-09-13T04:38:27.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.338+0000 7f92d5ffb700 1 -- 172.21.15.2:0/3944598609 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f92cc00ccd0 con 0x7f92d8100cf0 2023-09-13T04:38:27.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.340+0000 7f92d5ffb700 1 --2- 172.21.15.2:0/3944598609 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f92c80a0710 0x7f92c80a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:27.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.340+0000 7f92d5ffb700 1 -- 172.21.15.2:0/3944598609 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f92cc014070 con 0x7f92d8100cf0 2023-09-13T04:38:27.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.340+0000 7f92d7fff700 1 --2- 172.21.15.2:0/3944598609 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f92c80a0710 0x7f92c80a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:27.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.341+0000 7f92d7fff700 1 --2- 172.21.15.2:0/3944598609 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f92c80a0710 0x7f92c80a2bd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f92c400b580 tx=0x7f92c4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:27.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.344+0000 7f92d5ffb700 1 -- 172.21.15.2:0/3944598609 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f92cc0958b0 con 0x7f92d8100cf0 2023-09-13T04:38:27.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.556+0000 7f92defff700 1 -- 172.21.15.2:0/3944598609 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92d810c740 con 0x7f92d8100cf0 2023-09-13T04:38:27.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.558+0000 7f92d5ffb700 1 -- 172.21.15.2:0/3944598609 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24456 (secure 0 0 0) 0x7f92cc098ed0 con 0x7f92d8100cf0 2023-09-13T04:38:27.559 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:38:27.559 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":90,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:38:24.584729+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T04:37:57.094802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T04:37:58.564487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-13T04:38:02.096895+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:38:27.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.560+0000 7f92defff700 1 -- 172.21.15.2:0/3944598609 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f92c80a0710 msgr2=0x7f92c80a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:27.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.560+0000 7f92defff700 1 --2- 172.21.15.2:0/3944598609 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f92c80a0710 0x7f92c80a2bd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f92c400b580 tx=0x7f92c4005fb0 comp rx=0 tx=0).stop 2023-09-13T04:38:27.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.560+0000 7f92defff700 1 -- 172.21.15.2:0/3944598609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92d8100cf0 msgr2=0x7f92d81969b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:27.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.560+0000 7f92defff700 1 --2- 172.21.15.2:0/3944598609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92d8100cf0 0x7f92d81969b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f92cc00e970 tx=0x7f92cc00ec80 comp rx=0 tx=0).stop 2023-09-13T04:38:27.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.560+0000 7f92defff700 1 -- 172.21.15.2:0/3944598609 shutdown_connections 2023-09-13T04:38:27.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.560+0000 7f92defff700 1 --2- 172.21.15.2:0/3944598609 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f92c80a0710 0x7f92c80a2bd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:27.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.560+0000 7f92defff700 1 --2- 172.21.15.2:0/3944598609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92d8102880 0x7f92d819df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:27.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.560+0000 7f92defff700 1 --2- 172.21.15.2:0/3944598609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92d8100cf0 0x7f92d81969b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:27.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.560+0000 7f92defff700 1 --2- 172.21.15.2:0/3944598609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92d8101ee0 0x7f92d8196ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:27.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.560+0000 7f92defff700 1 -- 172.21.15.2:0/3944598609 >> 172.21.15.2:0/3944598609 conn(0x7f92d80fc220 msgr2=0x7f92d80fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:27.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.560+0000 7f92defff700 1 -- 172.21.15.2:0/3944598609 shutdown_connections 2023-09-13T04:38:27.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.560+0000 7f92defff700 1 -- 172.21.15.2:0/3944598609 wait complete. 2023-09-13T04:38:27.581 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-13T04:38:27.582 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-13T04:38:27.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.788+0000 7f17565ff700 1 -- 172.21.15.2:0/2961368094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1750100cf0 msgr2=0x7f1750101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.788+0000 7f17565ff700 1 --2- 172.21.15.2:0/2961368094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1750100cf0 0x7f1750101110 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f17400099c0 tx=0x7f1740009cd0 comp rx=0 tx=0).stop 2023-09-13T04:38:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.788+0000 7f17565ff700 1 -- 172.21.15.2:0/2961368094 shutdown_connections 2023-09-13T04:38:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.788+0000 7f17565ff700 1 --2- 172.21.15.2:0/2961368094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1750102880 0x7f1750109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.788+0000 7f17565ff700 1 --2- 172.21.15.2:0/2961368094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1750100cf0 0x7f1750101110 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.788+0000 7f17565ff700 1 --2- 172.21.15.2:0/2961368094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1750101ee0 0x7f1750102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:27.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.788+0000 7f17565ff700 1 -- 172.21.15.2:0/2961368094 >> 172.21.15.2:0/2961368094 conn(0x7f17500fc220 msgr2=0x7f17500fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:27.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.788+0000 7f17565ff700 1 -- 172.21.15.2:0/2961368094 shutdown_connections 2023-09-13T04:38:27.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.788+0000 7f17565ff700 1 -- 172.21.15.2:0/2961368094 wait complete. 2023-09-13T04:38:27.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.789+0000 7f17565ff700 1 Processor -- start 2023-09-13T04:38:27.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.789+0000 7f17565ff700 1 -- start start 2023-09-13T04:38:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.789+0000 7f17565ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1750100cf0 0x7f1750198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.789+0000 7f17565ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1750101ee0 0x7f1750199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.789+0000 7f17565ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1750102880 0x7f17501a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.790+0000 7f17565ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17501a0770 con 0x7f1750100cf0 2023-09-13T04:38:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.790+0000 7f17565ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17501a08b0 con 0x7f1750101ee0 2023-09-13T04:38:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.790+0000 7f17565ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17501a0a40 con 0x7f1750102880 2023-09-13T04:38:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.790+0000 7f174ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1750100cf0 0x7f1750198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.790+0000 7f174ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1750100cf0 0x7f1750198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38102/0 (socket says 172.21.15.2:38102) 2023-09-13T04:38:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.790+0000 7f174ffff700 1 -- 172.21.15.2:0/1536044145 learned_addr learned my addr 172.21.15.2:0/1536044145 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.790+0000 7f1754b9c700 1 --2- 172.21.15.2:0/1536044145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1750102880 0x7f17501a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:27.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.790+0000 7f174f7fe700 1 --2- 172.21.15.2:0/1536044145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1750101ee0 0x7f1750199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:27.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.790+0000 7f174ffff700 1 -- 172.21.15.2:0/1536044145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1750102880 msgr2=0x7f17501a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:27.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.790+0000 7f174ffff700 1 --2- 172.21.15.2:0/1536044145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1750102880 0x7f17501a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:27.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.790+0000 7f174ffff700 1 -- 172.21.15.2:0/1536044145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1750101ee0 msgr2=0x7f1750199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:27.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.790+0000 7f174ffff700 1 --2- 172.21.15.2:0/1536044145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1750101ee0 0x7f1750199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:27.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.790+0000 7f174ffff700 1 -- 172.21.15.2:0/1536044145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17400095f0 con 0x7f1750100cf0 2023-09-13T04:38:27.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.790+0000 7f174ffff700 1 --2- 172.21.15.2:0/1536044145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1750100cf0 0x7f1750198bf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1740009fd0 tx=0x7f174000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:27.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.791+0000 7f174d7fa700 1 -- 172.21.15.2:0/1536044145 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174001d070 con 0x7f1750100cf0 2023-09-13T04:38:27.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.791+0000 7f174d7fa700 1 -- 172.21.15.2:0/1536044145 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1740005510 con 0x7f1750100cf0 2023-09-13T04:38:27.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.791+0000 7f17565ff700 1 -- 172.21.15.2:0/1536044145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17501a0cc0 con 0x7f1750100cf0 2023-09-13T04:38:27.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.791+0000 7f174d7fa700 1 -- 172.21.15.2:0/1536044145 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1740004bb0 con 0x7f1750100cf0 2023-09-13T04:38:27.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.791+0000 7f17565ff700 1 -- 172.21.15.2:0/1536044145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17501a10e0 con 0x7f1750100cf0 2023-09-13T04:38:27.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.792+0000 7f17565ff700 1 -- 172.21.15.2:0/1536044145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1750193340 con 0x7f1750100cf0 2023-09-13T04:38:27.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.794+0000 7f174d7fa700 1 -- 172.21.15.2:0/1536044145 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f1740004d50 con 0x7f1750100cf0 2023-09-13T04:38:27.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.796+0000 7f174d7fa700 1 --2- 172.21.15.2:0/1536044145 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f17380a0690 0x7f17380a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:27.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.796+0000 7f174d7fa700 1 -- 172.21.15.2:0/1536044145 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f17400ccdf0 con 0x7f1750100cf0 2023-09-13T04:38:27.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.796+0000 7f174f7fe700 1 --2- 172.21.15.2:0/1536044145 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f17380a0690 0x7f17380a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:27.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.797+0000 7f174f7fe700 1 --2- 172.21.15.2:0/1536044145 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f17380a0690 0x7f17380a2b50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1744009bf0 tx=0x7f1744009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:27.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:27.800+0000 7f174d7fa700 1 -- 172.21.15.2:0/1536044145 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1740096640 con 0x7f1750100cf0 2023-09-13T04:38:27.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:27 smithi149 ceph-mon[133315]: mds.? [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] up:active 2023-09-13T04:38:27.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:27 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1828402985' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-13T04:38:27.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:27 smithi149 ceph-mon[133315]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2023-09-13T04:38:27.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:27 smithi149 ceph-mon[133315]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:38:27.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:27 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1138186360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:38:27.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:27 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3944598609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:27.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:27 smithi156 ceph-mon[133675]: mds.? [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] up:active 2023-09-13T04:38:27.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:27 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1828402985' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-13T04:38:27.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:27 smithi156 ceph-mon[133675]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2023-09-13T04:38:27.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:27 smithi156 ceph-mon[133675]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:38:27.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:27 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1138186360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:38:27.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:27 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3944598609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:28.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:27 smithi002 ceph-mon[97651]: mds.? [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] up:active 2023-09-13T04:38:28.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:27 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1828402985' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-13T04:38:28.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:27 smithi002 ceph-mon[97651]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2023-09-13T04:38:28.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:27 smithi002 ceph-mon[97651]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:38:28.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:27 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1138186360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:38:28.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:27 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3944598609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:28.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.010+0000 7f17565ff700 1 -- 172.21.15.2:0/1536044145 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f175002d0d0 con 0x7f17380a0690 2023-09-13T04:38:28.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.027+0000 7f174d7fa700 1 -- 172.21.15.2:0/1536044145 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422580 (secure 0 0 0) 0x7f175002d0d0 con 0x7f17380a0690 2023-09-13T04:38:28.034 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:38:28.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.032+0000 7f17565ff700 1 -- 172.21.15.2:0/1536044145 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f17380a0690 msgr2=0x7f17380a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:28.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.032+0000 7f17565ff700 1 --2- 172.21.15.2:0/1536044145 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f17380a0690 0x7f17380a2b50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1744009bf0 tx=0x7f1744009260 comp rx=0 tx=0).stop 2023-09-13T04:38:28.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.032+0000 7f17565ff700 1 -- 172.21.15.2:0/1536044145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1750100cf0 msgr2=0x7f1750198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:28.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.032+0000 7f17565ff700 1 --2- 172.21.15.2:0/1536044145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1750100cf0 0x7f1750198bf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1740009fd0 tx=0x7f174000f740 comp rx=0 tx=0).stop 2023-09-13T04:38:28.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.032+0000 7f17565ff700 1 -- 172.21.15.2:0/1536044145 shutdown_connections 2023-09-13T04:38:28.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.032+0000 7f17565ff700 1 --2- 172.21.15.2:0/1536044145 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f17380a0690 0x7f17380a2b50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:28.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.032+0000 7f17565ff700 1 --2- 172.21.15.2:0/1536044145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1750102880 0x7f17501a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:28.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.032+0000 7f17565ff700 1 --2- 172.21.15.2:0/1536044145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1750100cf0 0x7f1750198bf0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:28.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.032+0000 7f17565ff700 1 --2- 172.21.15.2:0/1536044145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1750101ee0 0x7f1750199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:28.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.032+0000 7f17565ff700 1 -- 172.21.15.2:0/1536044145 >> 172.21.15.2:0/1536044145 conn(0x7f17500fc220 msgr2=0x7f17501077e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:28.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.032+0000 7f17565ff700 1 -- 172.21.15.2:0/1536044145 shutdown_connections 2023-09-13T04:38:28.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.032+0000 7f17565ff700 1 -- 172.21.15.2:0/1536044145 wait complete. 2023-09-13T04:38:28.040 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-09-13T04:38:28.053 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2023-09-13T04:38:27.505931+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596996,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":138,"num_read_kb":121,"num_write":286,"num_write_kb":2970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":416,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337796,"kb_used_data":13532,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604028,"statfs":{"total":1151940427776,"available":1151594524672,"internally_reserved":0,"allocated":13856768,"data_stored":7620918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3769,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":59,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002540"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592832+0000","last_change":"2023-09-13T04:37:59.244279+0000","last_active":"2023-09-13T04:38:23.592832+0000","last_peered":"2023-09-13T04:38:23.592832+0000","last_clean":"2023-09-13T04:38:23.592832+0000","last_became_active":"2023-09-13T04:37:59.243976+0000","last_became_peered":"2023-09-13T04:37:59.243976+0000","last_unstale":"2023-09-13T04:38:23.592832+0000","last_undegraded":"2023-09-13T04:38:23.592832+0000","last_fullsized":"2023-09-13T04:38:23.592832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:21:52.809411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406851+0000","last_change":"2023-09-13T04:37:59.243391+0000","last_active":"2023-09-13T04:38:04.406851+0000","last_peered":"2023-09-13T04:38:04.406851+0000","last_clean":"2023-09-13T04:38:04.406851+0000","last_became_active":"2023-09-13T04:37:59.243044+0000","last_became_peered":"2023-09-13T04:37:59.243044+0000","last_unstale":"2023-09-13T04:38:04.406851+0000","last_undegraded":"2023-09-13T04:38:04.406851+0000","last_fullsized":"2023-09-13T04:38:04.406851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:02:17.806638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406414+0000","last_change":"2023-09-13T04:37:59.242826+0000","last_active":"2023-09-13T04:38:04.406414+0000","last_peered":"2023-09-13T04:38:04.406414+0000","last_clean":"2023-09-13T04:38:04.406414+0000","last_became_active":"2023-09-13T04:37:59.242125+0000","last_became_peered":"2023-09-13T04:37:59.242125+0000","last_unstale":"2023-09-13T04:38:04.406414+0000","last_undegraded":"2023-09-13T04:38:04.406414+0000","last_fullsized":"2023-09-13T04:38:04.406414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:08:29.216093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590536+0000","last_change":"2023-09-13T04:37:59.253322+0000","last_active":"2023-09-13T04:38:23.590536+0000","last_peered":"2023-09-13T04:38:23.590536+0000","last_clean":"2023-09-13T04:38:23.590536+0000","last_became_active":"2023-09-13T04:37:59.253050+0000","last_became_peered":"2023-09-13T04:37:59.253050+0000","last_unstale":"2023-09-13T04:38:23.590536+0000","last_undegraded":"2023-09-13T04:38:23.590536+0000","last_fullsized":"2023-09-13T04:38:23.590536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:45:22.357879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593487+0000","last_change":"2023-09-13T04:37:59.250089+0000","last_active":"2023-09-13T04:38:23.593487+0000","last_peered":"2023-09-13T04:38:23.593487+0000","last_clean":"2023-09-13T04:38:23.593487+0000","last_became_active":"2023-09-13T04:37:59.249771+0000","last_became_peered":"2023-09-13T04:37:59.249771+0000","last_unstale":"2023-09-13T04:38:23.593487+0000","last_undegraded":"2023-09-13T04:38:23.593487+0000","last_fullsized":"2023-09-13T04:38:23.593487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:46:59.917756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595139+0000","last_change":"2023-09-13T04:37:59.247259+0000","last_active":"2023-09-13T04:38:23.595139+0000","last_peered":"2023-09-13T04:38:23.595139+0000","last_clean":"2023-09-13T04:38:23.595139+0000","last_became_active":"2023-09-13T04:37:59.246911+0000","last_became_peered":"2023-09-13T04:37:59.246911+0000","last_unstale":"2023-09-13T04:38:23.595139+0000","last_undegraded":"2023-09-13T04:38:23.595139+0000","last_fullsized":"2023-09-13T04:38:23.595139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:00:39.742392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.407650+0000","last_change":"2023-09-13T04:37:59.244083+0000","last_active":"2023-09-13T04:38:04.407650+0000","last_peered":"2023-09-13T04:38:04.407650+0000","last_clean":"2023-09-13T04:38:04.407650+0000","last_became_active":"2023-09-13T04:37:59.243789+0000","last_became_peered":"2023-09-13T04:37:59.243789+0000","last_unstale":"2023-09-13T04:38:04.407650+0000","last_undegraded":"2023-09-13T04:38:04.407650+0000","last_fullsized":"2023-09-13T04:38:04.407650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:24:00.622790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.602097+0000","last_change":"2023-09-13T04:38:24.601265+0000","last_active":"2023-09-13T04:38:24.602097+0000","last_peered":"2023-09-13T04:38:24.602097+0000","last_clean":"2023-09-13T04:38:24.602097+0000","last_became_active":"2023-09-13T04:38:24.600939+0000","last_became_peered":"2023-09-13T04:38:24.600939+0000","last_unstale":"2023-09-13T04:38:24.602097+0000","last_undegraded":"2023-09-13T04:38:24.602097+0000","last_fullsized":"2023-09-13T04:38:24.602097+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:18:23.642333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594759+0000","last_change":"2023-09-13T04:37:59.238053+0000","last_active":"2023-09-13T04:38:23.594759+0000","last_peered":"2023-09-13T04:38:23.594759+0000","last_clean":"2023-09-13T04:38:23.594759+0000","last_became_active":"2023-09-13T04:37:59.237777+0000","last_became_peered":"2023-09-13T04:37:59.237777+0000","last_unstale":"2023-09-13T04:38:23.594759+0000","last_undegraded":"2023-09-13T04:38:23.594759+0000","last_fullsized":"2023-09-13T04:38:23.594759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:45:39.835659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594929+0000","last_change":"2023-09-13T04:37:59.252605+0000","last_active":"2023-09-13T04:38:23.594929+0000","last_peered":"2023-09-13T04:38:23.594929+0000","last_clean":"2023-09-13T04:38:23.594929+0000","last_became_active":"2023-09-13T04:37:59.252309+0000","last_became_peered":"2023-09-13T04:37:59.252309+0000","last_unstale":"2023-09-13T04:38:23.594929+0000","last_undegraded":"2023-09-13T04:38:23.594929+0000","last_fullsized":"2023-09-13T04:38:23.594929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:41:45.064230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.408085+0000","last_change":"2023-09-13T04:37:59.243345+0000","last_active":"2023-09-13T04:38:04.408085+0000","last_peered":"2023-09-13T04:38:04.408085+0000","last_clean":"2023-09-13T04:38:04.408085+0000","last_became_active":"2023-09-13T04:37:59.242726+0000","last_became_peered":"2023-09-13T04:37:59.242726+0000","last_unstale":"2023-09-13T04:38:04.408085+0000","last_undegraded":"2023-09-13T04:38:04.408085+0000","last_fullsized":"2023-09-13T04:38:04.408085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:39:55.115281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590601+0000","last_change":"2023-09-13T04:37:59.247415+0000","last_active":"2023-09-13T04:38:23.590601+0000","last_peered":"2023-09-13T04:38:23.590601+0000","last_clean":"2023-09-13T04:38:23.590601+0000","last_became_active":"2023-09-13T04:37:59.247050+0000","last_became_peered":"2023-09-13T04:37:59.247050+0000","last_unstale":"2023-09-13T04:38:23.590601+0000","last_undegraded":"2023-09-13T04:38:23.590601+0000","last_fullsized":"2023-09-13T04:38:23.590601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:03:44.793628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597902+0000","last_change":"2023-09-13T04:37:59.248935+0000","last_active":"2023-09-13T04:38:23.597902+0000","last_peered":"2023-09-13T04:38:23.597902+0000","last_clean":"2023-09-13T04:38:23.597902+0000","last_became_active":"2023-09-13T04:37:59.248554+0000","last_became_peered":"2023-09-13T04:37:59.248554+0000","last_unstale":"2023-09-13T04:38:23.597902+0000","last_undegraded":"2023-09-13T04:38:23.597902+0000","last_fullsized":"2023-09-13T04:38:23.597902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:53:31.882567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406214+0000","last_change":"2023-09-13T04:37:59.242851+0000","last_active":"2023-09-13T04:38:04.406214+0000","last_peered":"2023-09-13T04:38:04.406214+0000","last_clean":"2023-09-13T04:38:04.406214+0000","last_became_active":"2023-09-13T04:37:59.242384+0000","last_became_peered":"2023-09-13T04:37:59.242384+0000","last_unstale":"2023-09-13T04:38:04.406214+0000","last_undegraded":"2023-09-13T04:38:04.406214+0000","last_fullsized":"2023-09-13T04:38:04.406214+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:59:36.508808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"peering","last_fresh":"2023-09-13T04:38:04.406264+0000","last_change":"2023-09-13T04:38:23.591313+0000","last_active":"2023-09-13T04:38:23.591313+0000","last_peered":"2023-09-13T04:38:04.406264+0000","last_clean":"2023-09-13T04:38:04.406264+0000","last_became_active":"2023-09-13T04:37:59.252899+0000","last_became_peered":"2023-09-13T04:37:59.252899+0000","last_unstale":"2023-09-13T04:38:04.406264+0000","last_undegraded":"2023-09-13T04:38:04.406264+0000","last_fullsized":"2023-09-13T04:38:04.406264+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4,6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592657+0000","last_change":"2023-09-13T04:37:59.249107+0000","last_active":"2023-09-13T04:38:23.592657+0000","last_peered":"2023-09-13T04:38:23.592657+0000","last_clean":"2023-09-13T04:38:23.592657+0000","last_became_active":"2023-09-13T04:37:59.248803+0000","last_became_peered":"2023-09-13T04:37:59.248803+0000","last_unstale":"2023-09-13T04:38:23.592657+0000","last_undegraded":"2023-09-13T04:38:23.592657+0000","last_fullsized":"2023-09-13T04:38:23.592657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:50:52.481194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592743+0000","last_change":"2023-09-13T04:37:59.243724+0000","last_active":"2023-09-13T04:38:23.592743+0000","last_peered":"2023-09-13T04:38:23.592743+0000","last_clean":"2023-09-13T04:38:23.592743+0000","last_became_active":"2023-09-13T04:37:59.243385+0000","last_became_peered":"2023-09-13T04:37:59.243385+0000","last_unstale":"2023-09-13T04:38:23.592743+0000","last_undegraded":"2023-09-13T04:38:23.592743+0000","last_fullsized":"2023-09-13T04:38:23.592743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:55:55.203940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588550+0000","last_change":"2023-09-13T04:37:59.238986+0000","last_active":"2023-09-13T04:38:23.588550+0000","last_peered":"2023-09-13T04:38:23.588550+0000","last_clean":"2023-09-13T04:38:23.588550+0000","last_became_active":"2023-09-13T04:37:59.238668+0000","last_became_peered":"2023-09-13T04:37:59.238668+0000","last_unstale":"2023-09-13T04:38:23.588550+0000","last_undegraded":"2023-09-13T04:38:23.588550+0000","last_fullsized":"2023-09-13T04:38:23.588550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:21:10.225223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588585+0000","last_change":"2023-09-13T04:37:59.245598+0000","last_active":"2023-09-13T04:38:23.588585+0000","last_peered":"2023-09-13T04:38:23.588585+0000","last_clean":"2023-09-13T04:38:23.588585+0000","last_became_active":"2023-09-13T04:37:59.244652+0000","last_became_peered":"2023-09-13T04:37:59.244652+0000","last_unstale":"2023-09-13T04:38:23.588585+0000","last_undegraded":"2023-09-13T04:38:23.588585+0000","last_fullsized":"2023-09-13T04:38:23.588585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:34:43.671168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588238+0000","last_change":"2023-09-13T04:38:03.617708+0000","last_active":"2023-09-13T04:38:23.588238+0000","last_peered":"2023-09-13T04:38:23.588238+0000","last_clean":"2023-09-13T04:38:23.588238+0000","last_became_active":"2023-09-13T04:38:03.617277+0000","last_became_peered":"2023-09-13T04:38:03.617277+0000","last_unstale":"2023-09-13T04:38:23.588238+0000","last_undegraded":"2023-09-13T04:38:23.588238+0000","last_fullsized":"2023-09-13T04:38:23.588238+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:26:44.318030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.621267+0000","last_change":"2023-09-13T04:37:58.595156+0000","last_active":"2023-09-13T04:38:25.621267+0000","last_peered":"2023-09-13T04:38:25.621267+0000","last_clean":"2023-09-13T04:38:25.621267+0000","last_became_active":"2023-09-13T04:37:58.594863+0000","last_became_peered":"2023-09-13T04:37:58.594863+0000","last_unstale":"2023-09-13T04:38:25.621267+0000","last_undegraded":"2023-09-13T04:38:25.621267+0000","last_fullsized":"2023-09-13T04:38:25.621267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:27:14.420495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.602369+0000","last_change":"2023-09-13T04:37:59.250252+0000","last_active":"2023-09-13T04:38:03.602369+0000","last_peered":"2023-09-13T04:38:03.602369+0000","last_clean":"2023-09-13T04:38:03.602369+0000","last_became_active":"2023-09-13T04:37:59.249927+0000","last_became_peered":"2023-09-13T04:37:59.249927+0000","last_unstale":"2023-09-13T04:38:03.602369+0000","last_undegraded":"2023-09-13T04:38:03.602369+0000","last_fullsized":"2023-09-13T04:38:03.602369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:18:06.757040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.599683+0000","last_change":"2023-09-13T04:38:24.599683+0000","last_active":"2023-09-13T04:38:24.599683+0000","last_peered":"2023-09-13T04:38:24.599683+0000","last_clean":"2023-09-13T04:38:24.599683+0000","last_became_active":"2023-09-13T04:38:24.599230+0000","last_became_peered":"2023-09-13T04:38:24.599230+0000","last_unstale":"2023-09-13T04:38:24.599683+0000","last_undegraded":"2023-09-13T04:38:24.599683+0000","last_fullsized":"2023-09-13T04:38:24.599683+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:48:22.584735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.602561+0000","last_change":"2023-09-13T04:37:58.597785+0000","last_active":"2023-09-13T04:38:03.602561+0000","last_peered":"2023-09-13T04:38:03.602561+0000","last_clean":"2023-09-13T04:38:03.602561+0000","last_became_active":"2023-09-13T04:37:58.597485+0000","last_became_peered":"2023-09-13T04:37:58.597485+0000","last_unstale":"2023-09-13T04:38:03.602561+0000","last_undegraded":"2023-09-13T04:38:03.602561+0000","last_fullsized":"2023-09-13T04:38:03.602561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:39:57.156038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406705+0000","last_change":"2023-09-13T04:37:59.241840+0000","last_active":"2023-09-13T04:38:04.406705+0000","last_peered":"2023-09-13T04:38:04.406705+0000","last_clean":"2023-09-13T04:38:04.406705+0000","last_became_active":"2023-09-13T04:37:59.241545+0000","last_became_peered":"2023-09-13T04:37:59.241545+0000","last_unstale":"2023-09-13T04:38:04.406705+0000","last_undegraded":"2023-09-13T04:38:04.406705+0000","last_fullsized":"2023-09-13T04:38:04.406705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:04:17.439520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591101+0000","last_change":"2023-09-13T04:38:03.621792+0000","last_active":"2023-09-13T04:38:23.591101+0000","last_peered":"2023-09-13T04:38:23.591101+0000","last_clean":"2023-09-13T04:38:23.591101+0000","last_became_active":"2023-09-13T04:38:03.621433+0000","last_became_peered":"2023-09-13T04:38:03.621433+0000","last_unstale":"2023-09-13T04:38:23.591101+0000","last_undegraded":"2023-09-13T04:38:23.591101+0000","last_fullsized":"2023-09-13T04:38:23.591101+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:52:05.926733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594362+0000","last_change":"2023-09-13T04:37:58.590817+0000","last_active":"2023-09-13T04:38:23.594362+0000","last_peered":"2023-09-13T04:38:23.594362+0000","last_clean":"2023-09-13T04:38:23.594362+0000","last_became_active":"2023-09-13T04:37:58.590348+0000","last_became_peered":"2023-09-13T04:37:58.590348+0000","last_unstale":"2023-09-13T04:38:23.594362+0000","last_undegraded":"2023-09-13T04:38:23.594362+0000","last_fullsized":"2023-09-13T04:38:23.594362+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:38:35.680905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594252+0000","last_change":"2023-09-13T04:37:59.237304+0000","last_active":"2023-09-13T04:38:23.594252+0000","last_peered":"2023-09-13T04:38:23.594252+0000","last_clean":"2023-09-13T04:38:23.594252+0000","last_became_active":"2023-09-13T04:37:59.237008+0000","last_became_peered":"2023-09-13T04:37:59.237008+0000","last_unstale":"2023-09-13T04:38:23.594252+0000","last_undegraded":"2023-09-13T04:38:23.594252+0000","last_fullsized":"2023-09-13T04:38:23.594252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:28:14.254219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590193+0000","last_change":"2023-09-13T04:38:03.606749+0000","last_active":"2023-09-13T04:38:23.590193+0000","last_peered":"2023-09-13T04:38:23.590193+0000","last_clean":"2023-09-13T04:38:23.590193+0000","last_became_active":"2023-09-13T04:38:03.606276+0000","last_became_peered":"2023-09-13T04:38:03.606276+0000","last_unstale":"2023-09-13T04:38:23.590193+0000","last_undegraded":"2023-09-13T04:38:23.590193+0000","last_fullsized":"2023-09-13T04:38:23.590193+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:30:53.572543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593692+0000","last_change":"2023-09-13T04:37:58.590937+0000","last_active":"2023-09-13T04:38:23.593692+0000","last_peered":"2023-09-13T04:38:23.593692+0000","last_clean":"2023-09-13T04:38:23.593692+0000","last_became_active":"2023-09-13T04:37:58.590606+0000","last_became_peered":"2023-09-13T04:37:58.590606+0000","last_unstale":"2023-09-13T04:38:23.593692+0000","last_undegraded":"2023-09-13T04:38:23.593692+0000","last_fullsized":"2023-09-13T04:38:23.593692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:30:03.354670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591224+0000","last_change":"2023-09-13T04:37:59.251899+0000","last_active":"2023-09-13T04:38:23.591224+0000","last_peered":"2023-09-13T04:38:23.591224+0000","last_clean":"2023-09-13T04:38:23.591224+0000","last_became_active":"2023-09-13T04:37:59.251557+0000","last_became_peered":"2023-09-13T04:37:59.251557+0000","last_unstale":"2023-09-13T04:38:23.591224+0000","last_undegraded":"2023-09-13T04:38:23.591224+0000","last_fullsized":"2023-09-13T04:38:23.591224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:57:45.241188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588439+0000","last_change":"2023-09-13T04:38:03.606076+0000","last_active":"2023-09-13T04:38:23.588439+0000","last_peered":"2023-09-13T04:38:23.588439+0000","last_clean":"2023-09-13T04:38:23.588439+0000","last_became_active":"2023-09-13T04:38:03.605556+0000","last_became_peered":"2023-09-13T04:38:03.605556+0000","last_unstale":"2023-09-13T04:38:23.588439+0000","last_undegraded":"2023-09-13T04:38:23.588439+0000","last_fullsized":"2023-09-13T04:38:23.588439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:03:32.344625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.246702+0000","last_change":"2023-09-13T04:37:58.588170+0000","last_active":"2023-09-13T04:38:24.246702+0000","last_peered":"2023-09-13T04:38:24.246702+0000","last_clean":"2023-09-13T04:38:24.246702+0000","last_became_active":"2023-09-13T04:37:58.587860+0000","last_became_peered":"2023-09-13T04:37:58.587860+0000","last_unstale":"2023-09-13T04:38:24.246702+0000","last_undegraded":"2023-09-13T04:38:24.246702+0000","last_fullsized":"2023-09-13T04:38:24.246702+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:38:40.048150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595472+0000","last_change":"2023-09-13T04:37:59.240248+0000","last_active":"2023-09-13T04:38:23.595472+0000","last_peered":"2023-09-13T04:38:23.595472+0000","last_clean":"2023-09-13T04:38:23.595472+0000","last_became_active":"2023-09-13T04:37:59.240008+0000","last_became_peered":"2023-09-13T04:37:59.240008+0000","last_unstale":"2023-09-13T04:38:23.595472+0000","last_undegraded":"2023-09-13T04:38:23.595472+0000","last_fullsized":"2023-09-13T04:38:23.595472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:15:13.275110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587821+0000","last_change":"2023-09-13T04:38:03.616607+0000","last_active":"2023-09-13T04:38:23.587821+0000","last_peered":"2023-09-13T04:38:23.587821+0000","last_clean":"2023-09-13T04:38:23.587821+0000","last_became_active":"2023-09-13T04:38:03.616209+0000","last_became_peered":"2023-09-13T04:38:03.616209+0000","last_unstale":"2023-09-13T04:38:23.587821+0000","last_undegraded":"2023-09-13T04:38:23.587821+0000","last_fullsized":"2023-09-13T04:38:23.587821+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:24:21.934053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595102+0000","last_change":"2023-09-13T04:37:58.589104+0000","last_active":"2023-09-13T04:38:23.595102+0000","last_peered":"2023-09-13T04:38:23.595102+0000","last_clean":"2023-09-13T04:38:23.595102+0000","last_became_active":"2023-09-13T04:37:58.588780+0000","last_became_peered":"2023-09-13T04:37:58.588780+0000","last_unstale":"2023-09-13T04:38:23.595102+0000","last_undegraded":"2023-09-13T04:38:23.595102+0000","last_fullsized":"2023-09-13T04:38:23.595102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:01:32.839729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.600030+0000","last_change":"2023-09-13T04:38:24.598663+0000","last_active":"2023-09-13T04:38:24.600030+0000","last_peered":"2023-09-13T04:38:24.600030+0000","last_clean":"2023-09-13T04:38:24.600030+0000","last_became_active":"2023-09-13T04:38:24.598258+0000","last_became_peered":"2023-09-13T04:38:24.598258+0000","last_unstale":"2023-09-13T04:38:24.600030+0000","last_undegraded":"2023-09-13T04:38:24.600030+0000","last_fullsized":"2023-09-13T04:38:24.600030+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:50:28.157135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592633+0000","last_change":"2023-09-13T04:38:03.619930+0000","last_active":"2023-09-13T04:38:23.592633+0000","last_peered":"2023-09-13T04:38:23.592633+0000","last_clean":"2023-09-13T04:38:23.592633+0000","last_became_active":"2023-09-13T04:38:03.619560+0000","last_became_peered":"2023-09-13T04:38:03.619560+0000","last_unstale":"2023-09-13T04:38:23.592633+0000","last_undegraded":"2023-09-13T04:38:23.592633+0000","last_fullsized":"2023-09-13T04:38:23.592633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:48:06.867602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.619713+0000","last_change":"2023-09-13T04:37:58.590396+0000","last_active":"2023-09-13T04:38:25.619713+0000","last_peered":"2023-09-13T04:38:25.619713+0000","last_clean":"2023-09-13T04:38:25.619713+0000","last_became_active":"2023-09-13T04:37:58.589929+0000","last_became_peered":"2023-09-13T04:37:58.589929+0000","last_unstale":"2023-09-13T04:38:25.619713+0000","last_undegraded":"2023-09-13T04:38:25.619713+0000","last_fullsized":"2023-09-13T04:38:25.619713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:28:36.322096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.600166+0000","last_change":"2023-09-13T04:37:59.246033+0000","last_active":"2023-09-13T04:38:03.600166+0000","last_peered":"2023-09-13T04:38:03.600166+0000","last_clean":"2023-09-13T04:38:03.600166+0000","last_became_active":"2023-09-13T04:37:59.245664+0000","last_became_peered":"2023-09-13T04:37:59.245664+0000","last_unstale":"2023-09-13T04:38:03.600166+0000","last_undegraded":"2023-09-13T04:38:03.600166+0000","last_fullsized":"2023-09-13T04:38:03.600166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:24:17.145055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406894+0000","last_change":"2023-09-13T04:38:03.608504+0000","last_active":"2023-09-13T04:38:04.406894+0000","last_peered":"2023-09-13T04:38:04.406894+0000","last_clean":"2023-09-13T04:38:04.406894+0000","last_became_active":"2023-09-13T04:38:03.608058+0000","last_became_peered":"2023-09-13T04:38:03.608058+0000","last_unstale":"2023-09-13T04:38:04.406894+0000","last_undegraded":"2023-09-13T04:38:04.406894+0000","last_fullsized":"2023-09-13T04:38:04.406894+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:42:43.485453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244691+0000","last_change":"2023-09-13T04:37:58.588208+0000","last_active":"2023-09-13T04:38:24.244691+0000","last_peered":"2023-09-13T04:38:24.244691+0000","last_clean":"2023-09-13T04:38:24.244691+0000","last_became_active":"2023-09-13T04:37:58.587840+0000","last_became_peered":"2023-09-13T04:37:58.587840+0000","last_unstale":"2023-09-13T04:38:24.244691+0000","last_undegraded":"2023-09-13T04:38:24.244691+0000","last_fullsized":"2023-09-13T04:38:24.244691+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:44:58.012233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587683+0000","last_change":"2023-09-13T04:37:59.245102+0000","last_active":"2023-09-13T04:38:23.587683+0000","last_peered":"2023-09-13T04:38:23.587683+0000","last_clean":"2023-09-13T04:38:23.587683+0000","last_became_active":"2023-09-13T04:37:59.244799+0000","last_became_peered":"2023-09-13T04:37:59.244799+0000","last_unstale":"2023-09-13T04:38:23.587683+0000","last_undegraded":"2023-09-13T04:38:23.587683+0000","last_fullsized":"2023-09-13T04:38:23.587683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:57:16.295623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.454268+0000","last_change":"2023-09-13T04:38:03.607021+0000","last_active":"2023-09-13T04:38:04.454268+0000","last_peered":"2023-09-13T04:38:04.454268+0000","last_clean":"2023-09-13T04:38:04.454268+0000","last_became_active":"2023-09-13T04:38:03.606611+0000","last_became_peered":"2023-09-13T04:38:03.606611+0000","last_unstale":"2023-09-13T04:38:04.454268+0000","last_undegraded":"2023-09-13T04:38:04.454268+0000","last_fullsized":"2023-09-13T04:38:04.454268+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:33:20.551479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244009+0000","last_change":"2023-09-13T04:37:58.588731+0000","last_active":"2023-09-13T04:38:24.244009+0000","last_peered":"2023-09-13T04:38:24.244009+0000","last_clean":"2023-09-13T04:38:24.244009+0000","last_became_active":"2023-09-13T04:37:58.588398+0000","last_became_peered":"2023-09-13T04:37:58.588398+0000","last_unstale":"2023-09-13T04:38:24.244009+0000","last_undegraded":"2023-09-13T04:38:24.244009+0000","last_fullsized":"2023-09-13T04:38:24.244009+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:13:10.724085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594790+0000","last_change":"2023-09-13T04:37:59.244757+0000","last_active":"2023-09-13T04:38:23.594790+0000","last_peered":"2023-09-13T04:38:23.594790+0000","last_clean":"2023-09-13T04:38:23.594790+0000","last_became_active":"2023-09-13T04:37:59.244423+0000","last_became_peered":"2023-09-13T04:37:59.244423+0000","last_unstale":"2023-09-13T04:38:23.594790+0000","last_undegraded":"2023-09-13T04:38:23.594790+0000","last_fullsized":"2023-09-13T04:38:23.594790+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:14:37.973058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587303+0000","last_change":"2023-09-13T04:38:03.618358+0000","last_active":"2023-09-13T04:38:23.587303+0000","last_peered":"2023-09-13T04:38:23.587303+0000","last_clean":"2023-09-13T04:38:23.587303+0000","last_became_active":"2023-09-13T04:38:03.617291+0000","last_became_peered":"2023-09-13T04:38:03.617291+0000","last_unstale":"2023-09-13T04:38:23.587303+0000","last_undegraded":"2023-09-13T04:38:23.587303+0000","last_fullsized":"2023-09-13T04:38:23.587303+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:33:13.955353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244399+0000","last_change":"2023-09-13T04:37:58.590978+0000","last_active":"2023-09-13T04:38:24.244399+0000","last_peered":"2023-09-13T04:38:24.244399+0000","last_clean":"2023-09-13T04:38:24.244399+0000","last_became_active":"2023-09-13T04:37:58.590673+0000","last_became_peered":"2023-09-13T04:37:58.590673+0000","last_unstale":"2023-09-13T04:38:24.244399+0000","last_undegraded":"2023-09-13T04:38:24.244399+0000","last_fullsized":"2023-09-13T04:38:24.244399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:10:32.570029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406610+0000","last_change":"2023-09-13T04:37:59.251283+0000","last_active":"2023-09-13T04:38:04.406610+0000","last_peered":"2023-09-13T04:38:04.406610+0000","last_clean":"2023-09-13T04:38:04.406610+0000","last_became_active":"2023-09-13T04:37:59.250119+0000","last_became_peered":"2023-09-13T04:37:59.250119+0000","last_unstale":"2023-09-13T04:38:04.406610+0000","last_undegraded":"2023-09-13T04:38:04.406610+0000","last_fullsized":"2023-09-13T04:38:04.406610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:48:54.389970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.596107+0000","last_change":"2023-09-13T04:38:24.596107+0000","last_active":"2023-09-13T04:38:24.596107+0000","last_peered":"2023-09-13T04:38:24.596107+0000","last_clean":"2023-09-13T04:38:24.596107+0000","last_became_active":"2023-09-13T04:38:24.595653+0000","last_became_peered":"2023-09-13T04:38:24.595653+0000","last_unstale":"2023-09-13T04:38:24.596107+0000","last_undegraded":"2023-09-13T04:38:24.596107+0000","last_fullsized":"2023-09-13T04:38:24.596107+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:02:53.708658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.243198+0000","last_change":"2023-09-13T04:37:58.584909+0000","last_active":"2023-09-13T04:38:24.243198+0000","last_peered":"2023-09-13T04:38:24.243198+0000","last_clean":"2023-09-13T04:38:24.243198+0000","last_became_active":"2023-09-13T04:37:58.584630+0000","last_became_peered":"2023-09-13T04:37:58.584630+0000","last_unstale":"2023-09-13T04:38:24.243198+0000","last_undegraded":"2023-09-13T04:38:24.243198+0000","last_fullsized":"2023-09-13T04:38:24.243198+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:57:56.567499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594349+0000","last_change":"2023-09-13T04:37:59.254448+0000","last_active":"2023-09-13T04:38:23.594349+0000","last_peered":"2023-09-13T04:38:23.594349+0000","last_clean":"2023-09-13T04:38:23.594349+0000","last_became_active":"2023-09-13T04:37:59.254138+0000","last_became_peered":"2023-09-13T04:37:59.254138+0000","last_unstale":"2023-09-13T04:38:23.594349+0000","last_undegraded":"2023-09-13T04:38:23.594349+0000","last_fullsized":"2023-09-13T04:38:23.594349+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:36:14.939881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588357+0000","last_change":"2023-09-13T04:38:03.613746+0000","last_active":"2023-09-13T04:38:23.588357+0000","last_peered":"2023-09-13T04:38:23.588357+0000","last_clean":"2023-09-13T04:38:23.588357+0000","last_became_active":"2023-09-13T04:38:03.613354+0000","last_became_peered":"2023-09-13T04:38:03.613354+0000","last_unstale":"2023-09-13T04:38:23.588357+0000","last_undegraded":"2023-09-13T04:38:23.588357+0000","last_fullsized":"2023-09-13T04:38:23.588357+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:41:01.294912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.242914+0000","last_change":"2023-09-13T04:37:58.591196+0000","last_active":"2023-09-13T04:38:24.242914+0000","last_peered":"2023-09-13T04:38:24.242914+0000","last_clean":"2023-09-13T04:38:24.242914+0000","last_became_active":"2023-09-13T04:37:58.590883+0000","last_became_peered":"2023-09-13T04:37:58.590883+0000","last_unstale":"2023-09-13T04:38:24.242914+0000","last_undegraded":"2023-09-13T04:38:24.242914+0000","last_fullsized":"2023-09-13T04:38:24.242914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:46:58.530270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595499+0000","last_change":"2023-09-13T04:37:59.243483+0000","last_active":"2023-09-13T04:38:23.595499+0000","last_peered":"2023-09-13T04:38:23.595499+0000","last_clean":"2023-09-13T04:38:23.595499+0000","last_became_active":"2023-09-13T04:37:59.243088+0000","last_became_peered":"2023-09-13T04:37:59.243088+0000","last_unstale":"2023-09-13T04:38:23.595499+0000","last_undegraded":"2023-09-13T04:38:23.595499+0000","last_fullsized":"2023-09-13T04:38:23.595499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:56:14.093819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591697+0000","last_change":"2023-09-13T04:38:03.608820+0000","last_active":"2023-09-13T04:38:23.591697+0000","last_peered":"2023-09-13T04:38:23.591697+0000","last_clean":"2023-09-13T04:38:23.591697+0000","last_became_active":"2023-09-13T04:38:03.608200+0000","last_became_peered":"2023-09-13T04:38:03.608200+0000","last_unstale":"2023-09-13T04:38:23.591697+0000","last_undegraded":"2023-09-13T04:38:23.591697+0000","last_fullsized":"2023-09-13T04:38:23.591697+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:54:52.416501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.607235+0000","last_change":"2023-09-13T04:38:24.606820+0000","last_active":"2023-09-13T04:38:24.607235+0000","last_peered":"2023-09-13T04:38:24.607235+0000","last_clean":"2023-09-13T04:38:24.607235+0000","last_became_active":"2023-09-13T04:38:24.599059+0000","last_became_peered":"2023-09-13T04:38:24.599059+0000","last_unstale":"2023-09-13T04:38:24.607235+0000","last_undegraded":"2023-09-13T04:38:24.607235+0000","last_fullsized":"2023-09-13T04:38:24.607235+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:08:12.634357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597568+0000","last_change":"2023-09-13T04:37:59.250666+0000","last_active":"2023-09-13T04:38:23.597568+0000","last_peered":"2023-09-13T04:38:23.597568+0000","last_clean":"2023-09-13T04:38:23.597568+0000","last_became_active":"2023-09-13T04:37:59.250240+0000","last_became_peered":"2023-09-13T04:37:59.250240+0000","last_unstale":"2023-09-13T04:38:23.597568+0000","last_undegraded":"2023-09-13T04:38:23.597568+0000","last_fullsized":"2023-09-13T04:38:23.597568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:44:31.324222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592271+0000","last_change":"2023-09-13T04:38:03.616296+0000","last_active":"2023-09-13T04:38:23.592271+0000","last_peered":"2023-09-13T04:38:23.592271+0000","last_clean":"2023-09-13T04:38:23.592271+0000","last_became_active":"2023-09-13T04:38:03.615934+0000","last_became_peered":"2023-09-13T04:38:03.615934+0000","last_unstale":"2023-09-13T04:38:23.592271+0000","last_undegraded":"2023-09-13T04:38:23.592271+0000","last_fullsized":"2023-09-13T04:38:23.592271+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:51:18.994472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244805+0000","last_change":"2023-09-13T04:37:58.594782+0000","last_active":"2023-09-13T04:38:24.244805+0000","last_peered":"2023-09-13T04:38:24.244805+0000","last_clean":"2023-09-13T04:38:24.244805+0000","last_became_active":"2023-09-13T04:37:58.594480+0000","last_became_peered":"2023-09-13T04:37:58.594480+0000","last_unstale":"2023-09-13T04:38:24.244805+0000","last_undegraded":"2023-09-13T04:38:24.244805+0000","last_fullsized":"2023-09-13T04:38:24.244805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:40:19.635672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.407475+0000","last_change":"2023-09-13T04:37:59.252575+0000","last_active":"2023-09-13T04:38:04.407475+0000","last_peered":"2023-09-13T04:38:04.407475+0000","last_clean":"2023-09-13T04:38:04.407475+0000","last_became_active":"2023-09-13T04:37:59.252268+0000","last_became_peered":"2023-09-13T04:37:59.252268+0000","last_unstale":"2023-09-13T04:38:04.407475+0000","last_undegraded":"2023-09-13T04:38:04.407475+0000","last_fullsized":"2023-09-13T04:38:04.407475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:27:10.814930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587908+0000","last_change":"2023-09-13T04:38:03.618835+0000","last_active":"2023-09-13T04:38:23.587908+0000","last_peered":"2023-09-13T04:38:23.587908+0000","last_clean":"2023-09-13T04:38:23.587908+0000","last_became_active":"2023-09-13T04:38:03.618226+0000","last_became_peered":"2023-09-13T04:38:03.618226+0000","last_unstale":"2023-09-13T04:38:23.587908+0000","last_undegraded":"2023-09-13T04:38:23.587908+0000","last_fullsized":"2023-09-13T04:38:23.587908+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:01:28.157490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590730+0000","last_change":"2023-09-13T04:37:58.588545+0000","last_active":"2023-09-13T04:38:23.590730+0000","last_peered":"2023-09-13T04:38:23.590730+0000","last_clean":"2023-09-13T04:38:23.590730+0000","last_became_active":"2023-09-13T04:37:58.587995+0000","last_became_peered":"2023-09-13T04:37:58.587995+0000","last_unstale":"2023-09-13T04:38:23.590730+0000","last_undegraded":"2023-09-13T04:38:23.590730+0000","last_fullsized":"2023-09-13T04:38:23.590730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:40:08.797724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.407438+0000","last_change":"2023-09-13T04:37:59.249100+0000","last_active":"2023-09-13T04:38:04.407438+0000","last_peered":"2023-09-13T04:38:04.407438+0000","last_clean":"2023-09-13T04:38:04.407438+0000","last_became_active":"2023-09-13T04:37:59.248771+0000","last_became_peered":"2023-09-13T04:37:59.248771+0000","last_unstale":"2023-09-13T04:38:04.407438+0000","last_undegraded":"2023-09-13T04:38:04.407438+0000","last_fullsized":"2023-09-13T04:38:04.407438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:27:37.964595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597364+0000","last_change":"2023-09-13T04:38:03.616988+0000","last_active":"2023-09-13T04:38:23.597364+0000","last_peered":"2023-09-13T04:38:23.597364+0000","last_clean":"2023-09-13T04:38:23.597364+0000","last_became_active":"2023-09-13T04:38:03.616524+0000","last_became_peered":"2023-09-13T04:38:03.616524+0000","last_unstale":"2023-09-13T04:38:23.597364+0000","last_undegraded":"2023-09-13T04:38:23.597364+0000","last_fullsized":"2023-09-13T04:38:23.597364+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:42:41.431115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'10","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.613710+0000","last_change":"2023-09-13T04:38:24.613384+0000","last_active":"2023-09-13T04:38:24.613710+0000","last_peered":"2023-09-13T04:38:24.613710+0000","last_clean":"2023-09-13T04:38:24.613710+0000","last_became_active":"2023-09-13T04:38:24.599349+0000","last_became_peered":"2023-09-13T04:38:24.599349+0000","last_unstale":"2023-09-13T04:38:24.613710+0000","last_undegraded":"2023-09-13T04:38:24.613710+0000","last_fullsized":"2023-09-13T04:38:24.613710+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:23:23.479715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593329+0000","last_change":"2023-09-13T04:37:59.252237+0000","last_active":"2023-09-13T04:38:23.593329+0000","last_peered":"2023-09-13T04:38:23.593329+0000","last_clean":"2023-09-13T04:38:23.593329+0000","last_became_active":"2023-09-13T04:37:59.251718+0000","last_became_peered":"2023-09-13T04:37:59.251718+0000","last_unstale":"2023-09-13T04:38:23.593329+0000","last_undegraded":"2023-09-13T04:38:23.593329+0000","last_fullsized":"2023-09-13T04:38:23.593329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:45:48.405226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593276+0000","last_change":"2023-09-13T04:38:03.607096+0000","last_active":"2023-09-13T04:38:23.593276+0000","last_peered":"2023-09-13T04:38:23.593276+0000","last_clean":"2023-09-13T04:38:23.593276+0000","last_became_active":"2023-09-13T04:38:03.606656+0000","last_became_peered":"2023-09-13T04:38:03.606656+0000","last_unstale":"2023-09-13T04:38:23.593276+0000","last_undegraded":"2023-09-13T04:38:23.593276+0000","last_fullsized":"2023-09-13T04:38:23.593276+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:59:26.140360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595200+0000","last_change":"2023-09-13T04:37:58.591047+0000","last_active":"2023-09-13T04:38:23.595200+0000","last_peered":"2023-09-13T04:38:23.595200+0000","last_clean":"2023-09-13T04:38:23.595200+0000","last_became_active":"2023-09-13T04:37:58.590740+0000","last_became_peered":"2023-09-13T04:37:58.590740+0000","last_unstale":"2023-09-13T04:38:23.595200+0000","last_undegraded":"2023-09-13T04:38:23.595200+0000","last_fullsized":"2023-09-13T04:38:23.595200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:14:42.280416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592406+0000","last_change":"2023-09-13T04:37:59.249362+0000","last_active":"2023-09-13T04:38:23.592406+0000","last_peered":"2023-09-13T04:38:23.592406+0000","last_clean":"2023-09-13T04:38:23.592406+0000","last_became_active":"2023-09-13T04:37:59.249041+0000","last_became_peered":"2023-09-13T04:37:59.249041+0000","last_unstale":"2023-09-13T04:38:23.592406+0000","last_undegraded":"2023-09-13T04:38:23.592406+0000","last_fullsized":"2023-09-13T04:38:23.592406+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:06:33.822157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.598370+0000","last_change":"2023-09-13T04:38:24.598370+0000","last_active":"2023-09-13T04:38:24.598370+0000","last_peered":"2023-09-13T04:38:24.598370+0000","last_clean":"2023-09-13T04:38:24.598370+0000","last_became_active":"2023-09-13T04:38:24.597931+0000","last_became_peered":"2023-09-13T04:38:24.597931+0000","last_unstale":"2023-09-13T04:38:24.598370+0000","last_undegraded":"2023-09-13T04:38:24.598370+0000","last_fullsized":"2023-09-13T04:38:24.598370+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:14:28.911329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588100+0000","last_change":"2023-09-13T04:37:58.592496+0000","last_active":"2023-09-13T04:38:23.588100+0000","last_peered":"2023-09-13T04:38:23.588100+0000","last_clean":"2023-09-13T04:38:23.588100+0000","last_became_active":"2023-09-13T04:37:58.592106+0000","last_became_peered":"2023-09-13T04:37:58.592106+0000","last_unstale":"2023-09-13T04:38:23.588100+0000","last_undegraded":"2023-09-13T04:38:23.588100+0000","last_fullsized":"2023-09-13T04:38:23.588100+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:28:55.618563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594212+0000","last_change":"2023-09-13T04:37:59.250019+0000","last_active":"2023-09-13T04:38:23.594212+0000","last_peered":"2023-09-13T04:38:23.594212+0000","last_clean":"2023-09-13T04:38:23.594212+0000","last_became_active":"2023-09-13T04:37:59.249278+0000","last_became_peered":"2023-09-13T04:37:59.249278+0000","last_unstale":"2023-09-13T04:38:23.594212+0000","last_undegraded":"2023-09-13T04:38:23.594212+0000","last_fullsized":"2023-09-13T04:38:23.594212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:15:31.217236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597640+0000","last_change":"2023-09-13T04:38:03.614542+0000","last_active":"2023-09-13T04:38:23.597640+0000","last_peered":"2023-09-13T04:38:23.597640+0000","last_clean":"2023-09-13T04:38:23.597640+0000","last_became_active":"2023-09-13T04:38:03.614103+0000","last_became_peered":"2023-09-13T04:38:03.614103+0000","last_unstale":"2023-09-13T04:38:23.597640+0000","last_undegraded":"2023-09-13T04:38:23.597640+0000","last_fullsized":"2023-09-13T04:38:23.597640+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:06:48.426843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.407747+0000","last_change":"2023-09-13T04:37:58.591864+0000","last_active":"2023-09-13T04:38:04.407747+0000","last_peered":"2023-09-13T04:38:04.407747+0000","last_clean":"2023-09-13T04:38:04.407747+0000","last_became_active":"2023-09-13T04:37:58.591562+0000","last_became_peered":"2023-09-13T04:37:58.591562+0000","last_unstale":"2023-09-13T04:38:04.407747+0000","last_undegraded":"2023-09-13T04:38:04.407747+0000","last_fullsized":"2023-09-13T04:38:04.407747+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:31:27.280489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594567+0000","last_change":"2023-09-13T04:37:59.252600+0000","last_active":"2023-09-13T04:38:23.594567+0000","last_peered":"2023-09-13T04:38:23.594567+0000","last_clean":"2023-09-13T04:38:23.594567+0000","last_became_active":"2023-09-13T04:37:59.252306+0000","last_became_peered":"2023-09-13T04:37:59.252306+0000","last_unstale":"2023-09-13T04:38:23.594567+0000","last_undegraded":"2023-09-13T04:38:23.594567+0000","last_fullsized":"2023-09-13T04:38:23.594567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:18:55.686847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.596920+0000","last_change":"2023-09-13T04:38:03.742502+0000","last_active":"2023-09-13T04:38:23.596920+0000","last_peered":"2023-09-13T04:38:23.596920+0000","last_clean":"2023-09-13T04:38:23.596920+0000","last_became_active":"2023-09-13T04:38:03.742054+0000","last_became_peered":"2023-09-13T04:38:03.742054+0000","last_unstale":"2023-09-13T04:38:23.596920+0000","last_undegraded":"2023-09-13T04:38:23.596920+0000","last_fullsized":"2023-09-13T04:38:23.596920+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:55:54.601145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593527+0000","last_change":"2023-09-13T04:37:59.245644+0000","last_active":"2023-09-13T04:38:23.593527+0000","last_peered":"2023-09-13T04:38:23.593527+0000","last_clean":"2023-09-13T04:38:23.593527+0000","last_became_active":"2023-09-13T04:37:59.245284+0000","last_became_peered":"2023-09-13T04:37:59.245284+0000","last_unstale":"2023-09-13T04:38:23.593527+0000","last_undegraded":"2023-09-13T04:38:23.593527+0000","last_fullsized":"2023-09-13T04:38:23.593527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:19:04.002414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.621561+0000","last_change":"2023-09-13T04:37:58.590249+0000","last_active":"2023-09-13T04:38:25.621561+0000","last_peered":"2023-09-13T04:38:25.621561+0000","last_clean":"2023-09-13T04:38:25.621561+0000","last_became_active":"2023-09-13T04:37:58.589521+0000","last_became_peered":"2023-09-13T04:37:58.589521+0000","last_unstale":"2023-09-13T04:38:25.621561+0000","last_undegraded":"2023-09-13T04:38:25.621561+0000","last_fullsized":"2023-09-13T04:38:25.621561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:11:57.542149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"peering","last_fresh":"2023-09-13T04:38:04.408074+0000","last_change":"2023-09-13T04:38:23.588693+0000","last_active":"2023-09-13T04:38:23.588693+0000","last_peered":"2023-09-13T04:38:04.408074+0000","last_clean":"2023-09-13T04:38:04.408074+0000","last_became_active":"2023-09-13T04:38:03.608060+0000","last_became_peered":"2023-09-13T04:38:03.608060+0000","last_unstale":"2023-09-13T04:38:04.408074+0000","last_undegraded":"2023-09-13T04:38:04.408074+0000","last_fullsized":"2023-09-13T04:38:04.408074+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,8,9,11],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.597408+0000","last_change":"2023-09-13T04:37:59.243794+0000","last_active":"2023-09-13T04:38:03.597408+0000","last_peered":"2023-09-13T04:38:03.597408+0000","last_clean":"2023-09-13T04:38:03.597408+0000","last_became_active":"2023-09-13T04:37:59.243368+0000","last_became_peered":"2023-09-13T04:37:59.243368+0000","last_unstale":"2023-09-13T04:38:03.597408+0000","last_undegraded":"2023-09-13T04:38:03.597408+0000","last_fullsized":"2023-09-13T04:38:03.597408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:06:45.318737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.245630+0000","last_change":"2023-09-13T04:37:58.589195+0000","last_active":"2023-09-13T04:38:24.245630+0000","last_peered":"2023-09-13T04:38:24.245630+0000","last_clean":"2023-09-13T04:38:24.245630+0000","last_became_active":"2023-09-13T04:37:58.588905+0000","last_became_peered":"2023-09-13T04:37:58.588905+0000","last_unstale":"2023-09-13T04:38:24.245630+0000","last_undegraded":"2023-09-13T04:38:24.245630+0000","last_fullsized":"2023-09-13T04:38:24.245630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:13:15.978891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":325,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.407034+0000","last_change":"2023-09-13T04:36:52.227264+0000","last_active":"2023-09-13T04:38:04.407034+0000","last_peered":"2023-09-13T04:38:04.407034+0000","last_clean":"2023-09-13T04:38:04.407034+0000","last_became_active":"2023-09-13T04:36:52.226773+0000","last_became_peered":"2023-09-13T04:36:52.226773+0000","last_unstale":"2023-09-13T04:38:04.407034+0000","last_undegraded":"2023-09-13T04:38:04.407034+0000","last_fullsized":"2023-09-13T04:38:04.407034+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:33:42.026499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:33:42.026499+0000","last_clean_scrub_stamp":"2023-09-13T04:33:42.026499+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:28:55.806587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592890+0000","last_change":"2023-09-13T04:38:03.742790+0000","last_active":"2023-09-13T04:38:23.592890+0000","last_peered":"2023-09-13T04:38:23.592890+0000","last_clean":"2023-09-13T04:38:23.592890+0000","last_became_active":"2023-09-13T04:38:03.742338+0000","last_became_peered":"2023-09-13T04:38:03.742338+0000","last_unstale":"2023-09-13T04:38:23.592890+0000","last_undegraded":"2023-09-13T04:38:23.592890+0000","last_fullsized":"2023-09-13T04:38:23.592890+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:01:38.910623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597412+0000","last_change":"2023-09-13T04:37:59.252669+0000","last_active":"2023-09-13T04:38:23.597412+0000","last_peered":"2023-09-13T04:38:23.597412+0000","last_clean":"2023-09-13T04:38:23.597412+0000","last_became_active":"2023-09-13T04:37:59.252223+0000","last_became_peered":"2023-09-13T04:37:59.252223+0000","last_unstale":"2023-09-13T04:38:23.597412+0000","last_undegraded":"2023-09-13T04:38:23.597412+0000","last_fullsized":"2023-09-13T04:38:23.597412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:57:21.090083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587916+0000","last_change":"2023-09-13T04:37:58.590540+0000","last_active":"2023-09-13T04:38:23.587916+0000","last_peered":"2023-09-13T04:38:23.587916+0000","last_clean":"2023-09-13T04:38:23.587916+0000","last_became_active":"2023-09-13T04:37:58.590061+0000","last_became_peered":"2023-09-13T04:37:58.590061+0000","last_unstale":"2023-09-13T04:38:23.587916+0000","last_undegraded":"2023-09-13T04:38:23.587916+0000","last_fullsized":"2023-09-13T04:38:23.587916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:57:17.133461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592719+0000","last_change":"2023-09-13T04:38:03.614615+0000","last_active":"2023-09-13T04:38:23.592719+0000","last_peered":"2023-09-13T04:38:23.592719+0000","last_clean":"2023-09-13T04:38:23.592719+0000","last_became_active":"2023-09-13T04:38:03.614086+0000","last_became_peered":"2023-09-13T04:38:03.614086+0000","last_unstale":"2023-09-13T04:38:23.592719+0000","last_undegraded":"2023-09-13T04:38:23.592719+0000","last_fullsized":"2023-09-13T04:38:23.592719+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:25:35.525513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.243719+0000","last_change":"2023-09-13T04:37:58.592096+0000","last_active":"2023-09-13T04:38:24.243719+0000","last_peered":"2023-09-13T04:38:24.243719+0000","last_clean":"2023-09-13T04:38:24.243719+0000","last_became_active":"2023-09-13T04:37:58.591872+0000","last_became_peered":"2023-09-13T04:37:58.591872+0000","last_unstale":"2023-09-13T04:38:24.243719+0000","last_undegraded":"2023-09-13T04:38:24.243719+0000","last_fullsized":"2023-09-13T04:38:24.243719+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:05:23.936105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597816+0000","last_change":"2023-09-13T04:37:59.240759+0000","last_active":"2023-09-13T04:38:23.597816+0000","last_peered":"2023-09-13T04:38:23.597816+0000","last_clean":"2023-09-13T04:38:23.597816+0000","last_became_active":"2023-09-13T04:37:59.240414+0000","last_became_peered":"2023-09-13T04:37:59.240414+0000","last_unstale":"2023-09-13T04:38:23.597816+0000","last_undegraded":"2023-09-13T04:38:23.597816+0000","last_fullsized":"2023-09-13T04:38:23.597816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:37:20.301938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.454201+0000","last_change":"2023-09-13T04:38:03.742519+0000","last_active":"2023-09-13T04:38:04.454201+0000","last_peered":"2023-09-13T04:38:04.454201+0000","last_clean":"2023-09-13T04:38:04.454201+0000","last_became_active":"2023-09-13T04:38:03.742075+0000","last_became_peered":"2023-09-13T04:38:03.742075+0000","last_unstale":"2023-09-13T04:38:04.454201+0000","last_undegraded":"2023-09-13T04:38:04.454201+0000","last_fullsized":"2023-09-13T04:38:04.454201+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:14:33.155837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.619899+0000","last_change":"2023-09-13T04:37:58.593891+0000","last_active":"2023-09-13T04:38:25.619899+0000","last_peered":"2023-09-13T04:38:25.619899+0000","last_clean":"2023-09-13T04:38:25.619899+0000","last_became_active":"2023-09-13T04:37:58.593497+0000","last_became_peered":"2023-09-13T04:37:58.593497+0000","last_unstale":"2023-09-13T04:38:25.619899+0000","last_undegraded":"2023-09-13T04:38:25.619899+0000","last_fullsized":"2023-09-13T04:38:25.619899+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:25:53.652601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.601755+0000","last_change":"2023-09-13T04:37:59.247151+0000","last_active":"2023-09-13T04:38:03.601755+0000","last_peered":"2023-09-13T04:38:03.601755+0000","last_clean":"2023-09-13T04:38:03.601755+0000","last_became_active":"2023-09-13T04:37:59.246853+0000","last_became_peered":"2023-09-13T04:37:59.246853+0000","last_unstale":"2023-09-13T04:38:03.601755+0000","last_undegraded":"2023-09-13T04:38:03.601755+0000","last_fullsized":"2023-09-13T04:38:03.601755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:15:22.659365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587493+0000","last_change":"2023-09-13T04:38:03.606594+0000","last_active":"2023-09-13T04:38:23.587493+0000","last_peered":"2023-09-13T04:38:23.587493+0000","last_clean":"2023-09-13T04:38:23.587493+0000","last_became_active":"2023-09-13T04:38:03.606058+0000","last_became_peered":"2023-09-13T04:38:03.606058+0000","last_unstale":"2023-09-13T04:38:23.587493+0000","last_undegraded":"2023-09-13T04:38:23.587493+0000","last_fullsized":"2023-09-13T04:38:23.587493+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:57:39.237520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.246057+0000","last_change":"2023-09-13T04:37:58.588362+0000","last_active":"2023-09-13T04:38:24.246057+0000","last_peered":"2023-09-13T04:38:24.246057+0000","last_clean":"2023-09-13T04:38:24.246057+0000","last_became_active":"2023-09-13T04:37:58.588065+0000","last_became_peered":"2023-09-13T04:37:58.588065+0000","last_unstale":"2023-09-13T04:38:24.246057+0000","last_undegraded":"2023-09-13T04:38:24.246057+0000","last_fullsized":"2023-09-13T04:38:24.246057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:32:04.255654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406766+0000","last_change":"2023-09-13T04:37:59.251883+0000","last_active":"2023-09-13T04:38:04.406766+0000","last_peered":"2023-09-13T04:38:04.406766+0000","last_clean":"2023-09-13T04:38:04.406766+0000","last_became_active":"2023-09-13T04:37:59.251522+0000","last_became_peered":"2023-09-13T04:37:59.251522+0000","last_unstale":"2023-09-13T04:38:04.406766+0000","last_undegraded":"2023-09-13T04:38:04.406766+0000","last_fullsized":"2023-09-13T04:38:04.406766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:05:06.420799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590729+0000","last_change":"2023-09-13T04:38:03.742726+0000","last_active":"2023-09-13T04:38:23.590729+0000","last_peered":"2023-09-13T04:38:23.590729+0000","last_clean":"2023-09-13T04:38:23.590729+0000","last_became_active":"2023-09-13T04:38:03.742293+0000","last_became_peered":"2023-09-13T04:38:03.742293+0000","last_unstale":"2023-09-13T04:38:23.590729+0000","last_undegraded":"2023-09-13T04:38:23.590729+0000","last_fullsized":"2023-09-13T04:38:23.590729+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:52:15.089764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.243314+0000","last_change":"2023-09-13T04:37:58.584923+0000","last_active":"2023-09-13T04:38:24.243314+0000","last_peered":"2023-09-13T04:38:24.243314+0000","last_clean":"2023-09-13T04:38:24.243314+0000","last_became_active":"2023-09-13T04:37:58.584320+0000","last_became_peered":"2023-09-13T04:37:58.584320+0000","last_unstale":"2023-09-13T04:38:24.243314+0000","last_undegraded":"2023-09-13T04:38:24.243314+0000","last_fullsized":"2023-09-13T04:38:24.243314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:16:53.670534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592586+0000","last_change":"2023-09-13T04:37:59.243029+0000","last_active":"2023-09-13T04:38:23.592586+0000","last_peered":"2023-09-13T04:38:23.592586+0000","last_clean":"2023-09-13T04:38:23.592586+0000","last_became_active":"2023-09-13T04:37:59.242733+0000","last_became_peered":"2023-09-13T04:37:59.242733+0000","last_unstale":"2023-09-13T04:38:23.592586+0000","last_undegraded":"2023-09-13T04:38:23.592586+0000","last_fullsized":"2023-09-13T04:38:23.592586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:48:21.430026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588037+0000","last_change":"2023-09-13T04:38:03.616735+0000","last_active":"2023-09-13T04:38:23.588037+0000","last_peered":"2023-09-13T04:38:23.588037+0000","last_clean":"2023-09-13T04:38:23.588037+0000","last_became_active":"2023-09-13T04:38:03.616275+0000","last_became_peered":"2023-09-13T04:38:03.616275+0000","last_unstale":"2023-09-13T04:38:23.588037+0000","last_undegraded":"2023-09-13T04:38:23.588037+0000","last_fullsized":"2023-09-13T04:38:23.588037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:12:27.267918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.598266+0000","last_change":"2023-09-13T04:37:58.595040+0000","last_active":"2023-09-13T04:38:03.598266+0000","last_peered":"2023-09-13T04:38:03.598266+0000","last_clean":"2023-09-13T04:38:03.598266+0000","last_became_active":"2023-09-13T04:37:58.591403+0000","last_became_peered":"2023-09-13T04:37:58.591403+0000","last_unstale":"2023-09-13T04:38:03.598266+0000","last_undegraded":"2023-09-13T04:38:03.598266+0000","last_fullsized":"2023-09-13T04:38:03.598266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:42:06.561066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592175+0000","last_change":"2023-09-13T04:37:59.248837+0000","last_active":"2023-09-13T04:38:23.592175+0000","last_peered":"2023-09-13T04:38:23.592175+0000","last_clean":"2023-09-13T04:38:23.592175+0000","last_became_active":"2023-09-13T04:37:59.248573+0000","last_became_peered":"2023-09-13T04:37:59.248573+0000","last_unstale":"2023-09-13T04:38:23.592175+0000","last_undegraded":"2023-09-13T04:38:23.592175+0000","last_fullsized":"2023-09-13T04:38:23.592175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:56:22.980484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590405+0000","last_change":"2023-09-13T04:38:03.616370+0000","last_active":"2023-09-13T04:38:23.590405+0000","last_peered":"2023-09-13T04:38:23.590405+0000","last_clean":"2023-09-13T04:38:23.590405+0000","last_became_active":"2023-09-13T04:38:03.615996+0000","last_became_peered":"2023-09-13T04:38:03.615996+0000","last_unstale":"2023-09-13T04:38:23.590405+0000","last_undegraded":"2023-09-13T04:38:23.590405+0000","last_fullsized":"2023-09-13T04:38:23.590405+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:56:59.924205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.590955+0000","last_change":"2023-09-13T04:37:58.597902+0000","last_active":"2023-09-13T04:38:24.590955+0000","last_peered":"2023-09-13T04:38:24.590955+0000","last_clean":"2023-09-13T04:38:24.590955+0000","last_became_active":"2023-09-13T04:37:58.597578+0000","last_became_peered":"2023-09-13T04:37:58.597578+0000","last_unstale":"2023-09-13T04:38:24.590955+0000","last_undegraded":"2023-09-13T04:38:24.590955+0000","last_fullsized":"2023-09-13T04:38:24.590955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:10:09.931949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.604302+0000","last_change":"2023-09-13T04:37:59.248924+0000","last_active":"2023-09-13T04:38:03.604302+0000","last_peered":"2023-09-13T04:38:03.604302+0000","last_clean":"2023-09-13T04:38:03.604302+0000","last_became_active":"2023-09-13T04:37:59.248483+0000","last_became_peered":"2023-09-13T04:37:59.248483+0000","last_unstale":"2023-09-13T04:38:03.604302+0000","last_undegraded":"2023-09-13T04:38:03.604302+0000","last_fullsized":"2023-09-13T04:38:03.604302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:49:56.078544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590322+0000","last_change":"2023-09-13T04:38:03.742260+0000","last_active":"2023-09-13T04:38:23.590322+0000","last_peered":"2023-09-13T04:38:23.590322+0000","last_clean":"2023-09-13T04:38:23.590322+0000","last_became_active":"2023-09-13T04:38:03.741893+0000","last_became_peered":"2023-09-13T04:38:03.741893+0000","last_unstale":"2023-09-13T04:38:23.590322+0000","last_undegraded":"2023-09-13T04:38:23.590322+0000","last_fullsized":"2023-09-13T04:38:23.590322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:59:00.653545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.245764+0000","last_change":"2023-09-13T04:37:58.586898+0000","last_active":"2023-09-13T04:38:24.245764+0000","last_peered":"2023-09-13T04:38:24.245764+0000","last_clean":"2023-09-13T04:38:24.245764+0000","last_became_active":"2023-09-13T04:37:58.586594+0000","last_became_peered":"2023-09-13T04:37:58.586594+0000","last_unstale":"2023-09-13T04:38:24.245764+0000","last_undegraded":"2023-09-13T04:38:24.245764+0000","last_fullsized":"2023-09-13T04:38:24.245764+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:29:43.718522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.407855+0000","last_change":"2023-09-13T04:37:59.244749+0000","last_active":"2023-09-13T04:38:04.407855+0000","last_peered":"2023-09-13T04:38:04.407855+0000","last_clean":"2023-09-13T04:38:04.407855+0000","last_became_active":"2023-09-13T04:37:59.244391+0000","last_became_peered":"2023-09-13T04:37:59.244391+0000","last_unstale":"2023-09-13T04:38:04.407855+0000","last_undegraded":"2023-09-13T04:38:04.407855+0000","last_fullsized":"2023-09-13T04:38:04.407855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:33:02.195729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592524+0000","last_change":"2023-09-13T04:38:03.609766+0000","last_active":"2023-09-13T04:38:23.592524+0000","last_peered":"2023-09-13T04:38:23.592524+0000","last_clean":"2023-09-13T04:38:23.592524+0000","last_became_active":"2023-09-13T04:38:03.609432+0000","last_became_peered":"2023-09-13T04:38:03.609432+0000","last_unstale":"2023-09-13T04:38:23.592524+0000","last_undegraded":"2023-09-13T04:38:23.592524+0000","last_fullsized":"2023-09-13T04:38:23.592524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:30:56.115483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.620943+0000","last_change":"2023-09-13T04:37:58.589119+0000","last_active":"2023-09-13T04:38:25.620943+0000","last_peered":"2023-09-13T04:38:25.620943+0000","last_clean":"2023-09-13T04:38:25.620943+0000","last_became_active":"2023-09-13T04:37:58.588803+0000","last_became_peered":"2023-09-13T04:37:58.588803+0000","last_unstale":"2023-09-13T04:38:25.620943+0000","last_undegraded":"2023-09-13T04:38:25.620943+0000","last_fullsized":"2023-09-13T04:38:25.620943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:51:20.888666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.408147+0000","last_change":"2023-09-13T04:37:59.243506+0000","last_active":"2023-09-13T04:38:04.408147+0000","last_peered":"2023-09-13T04:38:04.408147+0000","last_clean":"2023-09-13T04:38:04.408147+0000","last_became_active":"2023-09-13T04:37:59.243082+0000","last_became_peered":"2023-09-13T04:37:59.243082+0000","last_unstale":"2023-09-13T04:38:04.408147+0000","last_undegraded":"2023-09-13T04:38:04.408147+0000","last_fullsized":"2023-09-13T04:38:04.408147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:01:41.179286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406373+0000","last_change":"2023-09-13T04:38:03.609042+0000","last_active":"2023-09-13T04:38:04.406373+0000","last_peered":"2023-09-13T04:38:04.406373+0000","last_clean":"2023-09-13T04:38:04.406373+0000","last_became_active":"2023-09-13T04:38:03.608553+0000","last_became_peered":"2023-09-13T04:38:03.608553+0000","last_unstale":"2023-09-13T04:38:04.406373+0000","last_undegraded":"2023-09-13T04:38:04.406373+0000","last_fullsized":"2023-09-13T04:38:04.406373+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:05:56.648475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591325+0000","last_change":"2023-09-13T04:37:58.598056+0000","last_active":"2023-09-13T04:38:23.591325+0000","last_peered":"2023-09-13T04:38:23.591325+0000","last_clean":"2023-09-13T04:38:23.591325+0000","last_became_active":"2023-09-13T04:37:58.597724+0000","last_became_peered":"2023-09-13T04:37:58.597724+0000","last_unstale":"2023-09-13T04:38:23.591325+0000","last_undegraded":"2023-09-13T04:38:23.591325+0000","last_fullsized":"2023-09-13T04:38:23.591325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:43:08.296866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.407964+0000","last_change":"2023-09-13T04:37:59.246505+0000","last_active":"2023-09-13T04:38:04.407964+0000","last_peered":"2023-09-13T04:38:04.407964+0000","last_clean":"2023-09-13T04:38:04.407964+0000","last_became_active":"2023-09-13T04:37:59.246024+0000","last_became_peered":"2023-09-13T04:37:59.246024+0000","last_unstale":"2023-09-13T04:38:04.407964+0000","last_undegraded":"2023-09-13T04:38:04.407964+0000","last_fullsized":"2023-09-13T04:38:04.407964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:36:58.008066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590871+0000","last_change":"2023-09-13T04:38:03.605354+0000","last_active":"2023-09-13T04:38:23.590871+0000","last_peered":"2023-09-13T04:38:23.590871+0000","last_clean":"2023-09-13T04:38:23.590871+0000","last_became_active":"2023-09-13T04:38:03.604840+0000","last_became_peered":"2023-09-13T04:38:03.604840+0000","last_unstale":"2023-09-13T04:38:23.590871+0000","last_undegraded":"2023-09-13T04:38:23.590871+0000","last_fullsized":"2023-09-13T04:38:23.590871+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:46:29.867914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597380+0000","last_change":"2023-09-13T04:37:58.589052+0000","last_active":"2023-09-13T04:38:23.597380+0000","last_peered":"2023-09-13T04:38:23.597380+0000","last_clean":"2023-09-13T04:38:23.597380+0000","last_became_active":"2023-09-13T04:37:58.588707+0000","last_became_peered":"2023-09-13T04:37:58.588707+0000","last_unstale":"2023-09-13T04:38:23.597380+0000","last_undegraded":"2023-09-13T04:38:23.597380+0000","last_fullsized":"2023-09-13T04:38:23.597380+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:25:23.966046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.408021+0000","last_change":"2023-09-13T04:37:59.241157+0000","last_active":"2023-09-13T04:38:04.408021+0000","last_peered":"2023-09-13T04:38:04.408021+0000","last_clean":"2023-09-13T04:38:04.408021+0000","last_became_active":"2023-09-13T04:37:59.240857+0000","last_became_peered":"2023-09-13T04:37:59.240857+0000","last_unstale":"2023-09-13T04:38:04.408021+0000","last_undegraded":"2023-09-13T04:38:04.408021+0000","last_fullsized":"2023-09-13T04:38:04.408021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:04:44.986908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592376+0000","last_change":"2023-09-13T04:37:59.242447+0000","last_active":"2023-09-13T04:38:23.592376+0000","last_peered":"2023-09-13T04:38:23.592376+0000","last_clean":"2023-09-13T04:38:23.592376+0000","last_became_active":"2023-09-13T04:37:59.242130+0000","last_became_peered":"2023-09-13T04:37:59.242130+0000","last_unstale":"2023-09-13T04:38:23.592376+0000","last_undegraded":"2023-09-13T04:38:23.592376+0000","last_fullsized":"2023-09-13T04:38:23.592376+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:12:12.844888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.595646+0000","last_change":"2023-09-13T04:38:24.595646+0000","last_active":"2023-09-13T04:38:24.595646+0000","last_peered":"2023-09-13T04:38:24.595646+0000","last_clean":"2023-09-13T04:38:24.595646+0000","last_became_active":"2023-09-13T04:38:24.595295+0000","last_became_peered":"2023-09-13T04:38:24.595295+0000","last_unstale":"2023-09-13T04:38:24.595646+0000","last_undegraded":"2023-09-13T04:38:24.595646+0000","last_fullsized":"2023-09-13T04:38:24.595646+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:24:27.174979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587574+0000","last_change":"2023-09-13T04:37:59.245358+0000","last_active":"2023-09-13T04:38:23.587574+0000","last_peered":"2023-09-13T04:38:23.587574+0000","last_clean":"2023-09-13T04:38:23.587574+0000","last_became_active":"2023-09-13T04:37:59.245052+0000","last_became_peered":"2023-09-13T04:37:59.245052+0000","last_unstale":"2023-09-13T04:38:23.587574+0000","last_undegraded":"2023-09-13T04:38:23.587574+0000","last_fullsized":"2023-09-13T04:38:23.587574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:05:42.561716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406504+0000","last_change":"2023-09-13T04:37:59.254053+0000","last_active":"2023-09-13T04:38:04.406504+0000","last_peered":"2023-09-13T04:38:04.406504+0000","last_clean":"2023-09-13T04:38:04.406504+0000","last_became_active":"2023-09-13T04:37:59.253544+0000","last_became_peered":"2023-09-13T04:37:59.253544+0000","last_unstale":"2023-09-13T04:38:04.406504+0000","last_undegraded":"2023-09-13T04:38:04.406504+0000","last_fullsized":"2023-09-13T04:38:04.406504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:18:54.689208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597920+0000","last_change":"2023-09-13T04:37:59.253403+0000","last_active":"2023-09-13T04:38:23.597920+0000","last_peered":"2023-09-13T04:38:23.597920+0000","last_clean":"2023-09-13T04:38:23.597920+0000","last_became_active":"2023-09-13T04:37:59.253062+0000","last_became_peered":"2023-09-13T04:37:59.253062+0000","last_unstale":"2023-09-13T04:38:23.597920+0000","last_undegraded":"2023-09-13T04:38:23.597920+0000","last_fullsized":"2023-09-13T04:38:23.597920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:31:09.898886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592570+0000","last_change":"2023-09-13T04:37:59.240916+0000","last_active":"2023-09-13T04:38:23.592570+0000","last_peered":"2023-09-13T04:38:23.592570+0000","last_clean":"2023-09-13T04:38:23.592570+0000","last_became_active":"2023-09-13T04:37:59.240605+0000","last_became_peered":"2023-09-13T04:37:59.240605+0000","last_unstale":"2023-09-13T04:38:23.592570+0000","last_undegraded":"2023-09-13T04:38:23.592570+0000","last_fullsized":"2023-09-13T04:38:23.592570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:15:48.241587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588778+0000","last_change":"2023-09-13T04:37:59.238569+0000","last_active":"2023-09-13T04:38:23.588778+0000","last_peered":"2023-09-13T04:38:23.588778+0000","last_clean":"2023-09-13T04:38:23.588778+0000","last_became_active":"2023-09-13T04:37:59.238255+0000","last_became_peered":"2023-09-13T04:37:59.238255+0000","last_unstale":"2023-09-13T04:38:23.588778+0000","last_undegraded":"2023-09-13T04:38:23.588778+0000","last_fullsized":"2023-09-13T04:38:23.588778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:19:03.433404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.601850+0000","last_change":"2023-09-13T04:37:59.252243+0000","last_active":"2023-09-13T04:38:03.601850+0000","last_peered":"2023-09-13T04:38:03.601850+0000","last_clean":"2023-09-13T04:38:03.601850+0000","last_became_active":"2023-09-13T04:37:59.251902+0000","last_became_peered":"2023-09-13T04:37:59.251902+0000","last_unstale":"2023-09-13T04:38:03.601850+0000","last_undegraded":"2023-09-13T04:38:03.601850+0000","last_fullsized":"2023-09-13T04:38:03.601850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:35:57.395627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593414+0000","last_change":"2023-09-13T04:37:59.245847+0000","last_active":"2023-09-13T04:38:23.593414+0000","last_peered":"2023-09-13T04:38:23.593414+0000","last_clean":"2023-09-13T04:38:23.593414+0000","last_became_active":"2023-09-13T04:37:59.245542+0000","last_became_peered":"2023-09-13T04:37:59.245542+0000","last_unstale":"2023-09-13T04:38:23.593414+0000","last_undegraded":"2023-09-13T04:38:23.593414+0000","last_fullsized":"2023-09-13T04:38:23.593414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:24:16.813333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.598419+0000","last_change":"2023-09-13T04:37:59.246963+0000","last_active":"2023-09-13T04:38:03.598419+0000","last_peered":"2023-09-13T04:38:03.598419+0000","last_clean":"2023-09-13T04:38:03.598419+0000","last_became_active":"2023-09-13T04:37:59.246448+0000","last_became_peered":"2023-09-13T04:37:59.246448+0000","last_unstale":"2023-09-13T04:38:03.598419+0000","last_undegraded":"2023-09-13T04:38:03.598419+0000","last_fullsized":"2023-09-13T04:38:03.598419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:25:45.418778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594086+0000","last_change":"2023-09-13T04:37:59.245399+0000","last_active":"2023-09-13T04:38:23.594086+0000","last_peered":"2023-09-13T04:38:23.594086+0000","last_clean":"2023-09-13T04:38:23.594086+0000","last_became_active":"2023-09-13T04:37:59.245060+0000","last_became_peered":"2023-09-13T04:37:59.245060+0000","last_unstale":"2023-09-13T04:38:23.594086+0000","last_undegraded":"2023-09-13T04:38:23.594086+0000","last_fullsized":"2023-09-13T04:38:23.594086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:25:41.078238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597807+0000","last_change":"2023-09-13T04:37:59.244776+0000","last_active":"2023-09-13T04:38:23.597807+0000","last_peered":"2023-09-13T04:38:23.597807+0000","last_clean":"2023-09-13T04:38:23.597807+0000","last_became_active":"2023-09-13T04:37:59.244409+0000","last_became_peered":"2023-09-13T04:37:59.244409+0000","last_unstale":"2023-09-13T04:38:23.597807+0000","last_undegraded":"2023-09-13T04:38:23.597807+0000","last_fullsized":"2023-09-13T04:38:23.597807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:04:02.561414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592391+0000","last_change":"2023-09-13T04:37:59.248153+0000","last_active":"2023-09-13T04:38:23.592391+0000","last_peered":"2023-09-13T04:38:23.592391+0000","last_clean":"2023-09-13T04:38:23.592391+0000","last_became_active":"2023-09-13T04:37:59.247840+0000","last_became_peered":"2023-09-13T04:37:59.247840+0000","last_unstale":"2023-09-13T04:38:23.592391+0000","last_undegraded":"2023-09-13T04:38:23.592391+0000","last_fullsized":"2023-09-13T04:38:23.592391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:52:53.021219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6628,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":105,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":18262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90,"ondisk_log_size":90,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":490076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.52100000000000002}]},{"osd":1,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.495},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.63700000000000001}]},{"osd":2,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.59599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":4,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.79000000000000004}]},{"osd":5,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.77500000000000002}]},{"osd":6,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.75800000000000001}]},{"osd":7,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.86399999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":9,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.878}]},{"osd":10,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.92400000000000004}]}]},{"osd":10,"up_from":71,"seq":304942678037,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1626112,"data_stored":1084927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":1,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.42299999999999999}]},{"osd":2,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.57199999999999995}]},{"osd":3,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.626},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.754}]},{"osd":4,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.44600000000000001}]},{"osd":5,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.69899999999999995}]},{"osd":6,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":3.4129999999999998,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.84499999999999997}]},{"osd":7,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.64500000000000002}]},{"osd":8,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.59399999999999997}]},{"osd":9,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.80500000000000005}]},{"osd":11,"last update":"Wed Sep 13 04:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.67800000000000005}]}]},{"osd":9,"up_from":66,"seq":283467841562,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":490860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.36299999999999999}]},{"osd":1,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.56499999999999995}]},{"osd":2,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.73499999999999999}]},{"osd":3,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.45200000000000001}]},{"osd":4,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.63},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.59199999999999997}]},{"osd":5,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.75}]},{"osd":6,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.72099999999999997}]},{"osd":7,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.60899999999999999}]},{"osd":8,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.64800000000000002}]},{"osd":10,"last update":"Wed Sep 13 04:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.68300000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]}]},{"osd":8,"up_from":58,"seq":249108103198,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":490337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.33200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56999999999999995,"15min":0.56499999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.68100000000000005}]},{"osd":1,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.54800000000000004,"15min":0.53600000000000003},"min":{"1min":0.38400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.58199999999999996},"min":{"1min":0.374,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.78600000000000003}]},{"osd":2,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.47499999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.48899999999999999,"15min":0.48599999999999999},"min":{"1min":0.29799999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.48399999999999999}]},{"osd":3,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.434},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64600000000000002,"15min":0.64300000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.61799999999999999}]},{"osd":4,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58899999999999997,"15min":0.58399999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.41099999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.73699999999999999}]},{"osd":5,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.34499999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80700000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.54400000000000004}]},{"osd":6,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.88200000000000001}]},{"osd":7,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71199999999999997,"15min":0.70999999999999996},"min":{"1min":0.48399999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69099999999999995,"15min":0.68700000000000006},"min":{"1min":0.503,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.81200000000000006}]},{"osd":9,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.85399999999999998}]},{"osd":10,"last update":"Wed Sep 13 04:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.82499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]}]},{"osd":7,"up_from":51,"seq":219043332131,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1564672,"data_stored":1061916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.46500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.746,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60499999999999998,"15min":0.60799999999999998},"min":{"1min":0.41599999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.40200000000000002}]},{"osd":1,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.40899999999999997,"15min":0.40300000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42099999999999999,"15min":0.41999999999999998},"min":{"1min":0.30199999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65800000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.34200000000000003}]},{"osd":2,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.42599999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83899999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.57799999999999996,"15min":0.57099999999999995},"min":{"1min":0.443,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.70599999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.53700000000000003}]},{"osd":3,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.5},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66600000000000004,"15min":0.66900000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82799999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.52100000000000002}]},{"osd":4,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76600000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51200000000000001,"15min":0.51500000000000001},"min":{"1min":0.34499999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.376}]},{"osd":5,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.628,"15min":0.626},"min":{"1min":0.47799999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64600000000000002,"15min":0.64200000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.60299999999999998}]},{"osd":6,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72199999999999998,"15min":0.72099999999999997},"min":{"1min":0.50600000000000001,"5min":0.496,"15min":0.496},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.62},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.48799999999999999}]},{"osd":8,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70299999999999996,"15min":0.70399999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.51700000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.88100000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.68200000000000005}]},{"osd":9,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.72299999999999998}]},{"osd":10,"last update":"Wed Sep 13 04:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.434},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.56499999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":490132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46100000000000002,"15min":0.45100000000000001},"min":{"1min":0.34699999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.503,"15min":0.48999999999999999},"min":{"1min":0.38100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.42999999999999999}]},{"osd":1,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.47099999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82499999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.70999999999999996,"15min":0.67900000000000005},"min":{"1min":0.60399999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.94699999999999995}]},{"osd":2,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.52400000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0129999999999999,"5min":1.054,"15min":1.054},"last":0.83199999999999996}]},{"osd":3,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.66100000000000003,"15min":0.64100000000000001},"min":{"1min":0.47799999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83599999999999997,"5min":1.113,"15min":1.113},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.626,"15min":0.61799999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89900000000000002,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.496}]},{"osd":4,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76400000000000001,"15min":0.747},"min":{"1min":0.67900000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0509999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.75900000000000001,"15min":0.77100000000000002},"min":{"1min":0.52600000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92900000000000005,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.61599999999999999}]},{"osd":5,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.80800000000000005,"15min":0.79200000000000004},"min":{"1min":0.56100000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1950000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.79200000000000004,"15min":0.80800000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.80200000000000005}]},{"osd":7,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.83999999999999997,"15min":0.82399999999999995},"min":{"1min":0.63500000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.79800000000000004,"15min":0.78800000000000003},"min":{"1min":0.58999999999999997,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.57899999999999996}]},{"osd":8,"last update":"Wed Sep 13 04:38:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.81599999999999995,"15min":0.82399999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96599999999999997,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.91900000000000004,"15min":0.92400000000000004},"min":{"1min":0.61199999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2629999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.032}]},{"osd":9,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.0089999999999999}]},{"osd":10,"last update":"Wed Sep 13 04:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.754}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":493884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55600000000000005,"15min":0.53700000000000003},"min":{"1min":0.33700000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56999999999999995,"15min":0.55200000000000005},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.85199999999999998}]},{"osd":2,"last update":"Wed Sep 13 04:37:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.55900000000000005,"15min":0.53800000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.54800000000000004,"15min":0.51800000000000002},"min":{"1min":0.42299999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.73199999999999998}]},{"osd":3,"last update":"Wed Sep 13 04:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83099999999999996,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.623,"15min":0.60899999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91600000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67800000000000005}]},{"osd":4,"last update":"Wed Sep 13 04:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.46200000000000002,"15min":0.45300000000000001},"min":{"1min":0.29599999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.45800000000000002,"15min":0.44800000000000001},"min":{"1min":0.29199999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.497}]},{"osd":5,"last update":"Wed Sep 13 04:37:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.44800000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89000000000000001,"5min":1.3,"15min":1.3},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61499999999999999,"15min":0.58299999999999996},"min":{"1min":0.435,"5min":0.37,"15min":0.37},"max":{"1min":1,"5min":1.264,"15min":1.264},"last":0.78800000000000003}]},{"osd":6,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.99199999999999999,"5min":1.482,"15min":1.482},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95399999999999996,"5min":1.375,"15min":1.375},"last":0.71699999999999997}]},{"osd":7,"last update":"Wed Sep 13 04:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.60699999999999998,"15min":0.58699999999999997},"min":{"1min":0.39900000000000002,"5min":0.372,"15min":0.372},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.65900000000000003,"15min":0.64300000000000002},"min":{"1min":0.47399999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.75800000000000001}]},{"osd":8,"last update":"Wed Sep 13 04:38:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.48899999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2010000000000001,"5min":1.393,"15min":1.393},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.093,"5min":1.224,"15min":1.224},"last":0.77100000000000002}]},{"osd":9,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.81100000000000005}]},{"osd":10,"last update":"Wed Sep 13 04:37:53 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.878}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":480566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 13 04:37:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":0.66000000000000003,"15min":0.58299999999999996},"min":{"1min":0.42199999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.2359999999999998,"5min":6.2359999999999998,"15min":6.2359999999999998},"last":0.745},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.58699999999999997,"15min":0.55200000000000005},"min":{"1min":0.47899999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.77300000000000002}]},{"osd":2,"last update":"Wed Sep 13 04:37:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.624,"15min":0.59899999999999998},"min":{"1min":0.42899999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93899999999999995,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63700000000000001,"15min":0.60299999999999998},"min":{"1min":0.439,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.96899999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.82399999999999995}]},{"osd":3,"last update":"Wed Sep 13 04:36:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.57099999999999995,"15min":0.55500000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.86499999999999999}]},{"osd":4,"last update":"Wed Sep 13 04:37:30 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.65900000000000003,"15min":0.61299999999999999},"min":{"1min":0.58699999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":1.079},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.64900000000000002,"15min":0.621},"min":{"1min":0.51400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.84599999999999997}]},{"osd":5,"last update":"Wed Sep 13 04:38:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68300000000000005,"15min":0.67000000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.70699999999999996,"15min":0.68300000000000005},"min":{"1min":0.55100000000000005,"5min":0.435,"15min":0.435},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.91900000000000004}]},{"osd":6,"last update":"Wed Sep 13 04:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.44600000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94899999999999995,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88700000000000001,"5min":1.165,"15min":1.165},"last":0.79800000000000004}]},{"osd":7,"last update":"Wed Sep 13 04:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.79500000000000004,"15min":0.78500000000000003},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.81599999999999995,"15min":0.80200000000000005},"min":{"1min":0.60099999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":1.26}]},{"osd":8,"last update":"Wed Sep 13 04:38:00 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":0.90600000000000003,"15min":0.85999999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":6.2089999999999996,"5min":6.2089999999999996,"15min":6.2089999999999996},"last":0.874},{"interface":"front","average":{"1min":1.171,"5min":0.87,"15min":0.81999999999999995},"min":{"1min":0.52400000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":6.1699999999999999,"5min":6.1699999999999999,"15min":6.1699999999999999},"last":1.143}]},{"osd":9,"last update":"Wed Sep 13 04:37:30 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":1.171}]},{"osd":10,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":6.1859999999999999,"5min":6.1859999999999999,"15min":6.1859999999999999},"last":0.996}]},{"osd":11,"last update":"Wed Sep 13 04:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.2330000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":1.044}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":495200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:55 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.34599999999999997,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79100000000000004,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51900000000000002,"15min":0.51800000000000002},"min":{"1min":0.371,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78300000000000003,"5min":1.429,"15min":1.429},"last":0.45100000000000001}]},{"osd":1,"last update":"Wed Sep 13 04:37:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.53500000000000003,"15min":0.496},"min":{"1min":0.39000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80700000000000005,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54300000000000004,"15min":0.50700000000000001},"min":{"1min":0.39100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70799999999999996,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.755}]},{"osd":3,"last update":"Wed Sep 13 04:38:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.36499999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.747,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.441,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81399999999999995,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.66600000000000004}]},{"osd":4,"last update":"Wed Sep 13 04:37:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.52700000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78000000000000003,"5min":1.244,"15min":1.244},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55800000000000005,"15min":0.53000000000000003},"min":{"1min":0.436,"5min":0.374,"15min":0.374},"max":{"1min":0.85499999999999998,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.79400000000000004}]},{"osd":5,"last update":"Wed Sep 13 04:37:57 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.28699999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.70899999999999996,"5min":1.796,"15min":1.796},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.30599999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.99299999999999999,"5min":1.546,"15min":1.546},"last":0.49399999999999999}]},{"osd":6,"last update":"Wed Sep 13 04:38:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.69199999999999995,"15min":0.69899999999999995},"min":{"1min":0.48499999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81599999999999995,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66700000000000004,"15min":0.67200000000000004},"min":{"1min":0.48199999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85899999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.77100000000000002}]},{"osd":7,"last update":"Wed Sep 13 04:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65400000000000003,"15min":0.65800000000000003},"min":{"1min":0.42499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94699999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.497,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92500000000000004,"5min":1.147,"15min":1.147},"last":0.88300000000000001}]},{"osd":8,"last update":"Wed Sep 13 04:38:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61699999999999999,"15min":0.61899999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.96599999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.38400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.82699999999999996}]},{"osd":9,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.84199999999999997}]},{"osd":10,"last update":"Wed Sep 13 04:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.98199999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":974848,"data_stored":471070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.42699999999999999,"15min":0.39000000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.46500000000000002,"15min":0.435},"min":{"1min":0.375,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78500000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.38400000000000001}]},{"osd":1,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.57699999999999996},"min":{"1min":0.45700000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64200000000000002,"15min":0.57699999999999996},"min":{"1min":0.436,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92200000000000004,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.69799999999999995}]},{"osd":2,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.67400000000000004,"15min":0.629},"min":{"1min":0.53000000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64500000000000002,"15min":0.621},"min":{"1min":0.45800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.55100000000000005}]},{"osd":4,"last update":"Wed Sep 13 04:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.64300000000000002,"15min":0.60099999999999998},"min":{"1min":0.52900000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.64100000000000001,"15min":0.60699999999999998},"min":{"1min":0.44500000000000001,"5min":0.439,"15min":0.439},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.68799999999999994}]},{"osd":5,"last update":"Wed Sep 13 04:38:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.68999999999999995,"15min":0.65000000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.024,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.873,"5min":0.72399999999999998,"15min":0.66600000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":0.71199999999999997}]},{"osd":6,"last update":"Wed Sep 13 04:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.71599999999999997,"15min":0.69499999999999995},"min":{"1min":0.51200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.75900000000000001,"15min":0.74199999999999999},"min":{"1min":0.52400000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.622}]},{"osd":7,"last update":"Wed Sep 13 04:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.86299999999999999,"15min":0.86399999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.85399999999999998,"15min":0.84999999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.80500000000000005}]},{"osd":8,"last update":"Wed Sep 13 04:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.875}]},{"osd":9,"last update":"Wed Sep 13 04:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.63800000000000001}]},{"osd":10,"last update":"Wed Sep 13 04:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.77200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":490064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.56599999999999995,"15min":0.54400000000000004},"min":{"1min":0.43099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.753},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76500000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.68999999999999995}]},{"osd":1,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.373,"15min":0.34699999999999998},"min":{"1min":0.28399999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.38700000000000001,"15min":0.36399999999999999},"min":{"1min":0.28100000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.48499999999999999}]},{"osd":2,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58299999999999996,"15min":0.56100000000000005},"min":{"1min":0.41099999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56599999999999995,"15min":0.53800000000000003},"min":{"1min":0.41399999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.64300000000000002}]},{"osd":3,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.59399999999999997,"15min":0.56499999999999995},"min":{"1min":0.41799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59399999999999997,"15min":0.57699999999999996},"min":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.68400000000000005}]},{"osd":5,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60299999999999998,"15min":0.58599999999999997},"min":{"1min":0.45100000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90300000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.39700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.75800000000000001}]},{"osd":6,"last update":"Wed Sep 13 04:38:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.68000000000000005,"15min":0.68400000000000005},"min":{"1min":0.47099999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81599999999999995,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.69799999999999995,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.84599999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52400000000000002,"15min":0.51900000000000002},"min":{"1min":0.36199999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73599999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.70099999999999996}]},{"osd":8,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.70699999999999996,"15min":0.69199999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76700000000000002,"15min":0.76300000000000001},"min":{"1min":0.53000000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.77600000000000002}]},{"osd":9,"last update":"Wed Sep 13 04:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.86199999999999999}]},{"osd":10,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.82899999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1622016,"data_stored":1081886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53000000000000003,"15min":0.52400000000000002},"min":{"1min":0.34399999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72099999999999997,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.317,"5min":0.315,"15min":0.315},"max":{"1min":0.70299999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.84799999999999998}]},{"osd":1,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.40300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78500000000000003,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.54300000000000004,"15min":0.52400000000000002},"min":{"1min":0.34799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89600000000000002,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.60399999999999998}]},{"osd":2,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.38,"15min":0.36099999999999999},"min":{"1min":0.26900000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.37},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.39600000000000002,"15min":0.378},"min":{"1min":0.28299999999999997,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.41199999999999998}]},{"osd":3,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.372,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92600000000000005,"5min":1.036,"15min":1.036},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.78700000000000003,"15min":0.86899999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.1259999999999999,"5min":8.0009999999999994,"15min":8.0009999999999994},"last":0.77700000000000002}]},{"osd":4,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.58299999999999996,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82299999999999995,"5min":1.599,"15min":1.599},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.57699999999999996,"15min":0.55500000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.86599999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.623}]},{"osd":6,"last update":"Wed Sep 13 04:38:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83499999999999996,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81899999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.72999999999999998}]},{"osd":7,"last update":"Wed Sep 13 04:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.878,"5min":1.488,"15min":1.488},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94899999999999995,"5min":1.51,"15min":1.51},"last":0.63900000000000001}]},{"osd":8,"last update":"Wed Sep 13 04:38:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.38900000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.60599999999999998,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.56999999999999995}]},{"osd":9,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.70099999999999996}]},{"osd":10,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.68500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-13T04:38:28.064 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-13T04:38:28.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.297+0000 7f4fe63ff700 1 -- 172.21.15.2:0/3346247869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fe0108bb0 msgr2=0x7f4fe010b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:28.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.297+0000 7f4fe63ff700 1 --2- 172.21.15.2:0/3346247869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fe0108bb0 0x7f4fe010b040 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800b0a0 tx=0x7f4fd800b3b0 comp rx=0 tx=0).stop 2023-09-13T04:38:28.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.297+0000 7f4fe63ff700 1 -- 172.21.15.2:0/3346247869 shutdown_connections 2023-09-13T04:38:28.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.297+0000 7f4fe63ff700 1 --2- 172.21.15.2:0/3346247869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fe0108bb0 0x7f4fe010b040 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:28.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.297+0000 7f4fe63ff700 1 --2- 172.21.15.2:0/3346247869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe0074cf0 0x7f4fe0073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:28.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.297+0000 7f4fe63ff700 1 --2- 172.21.15.2:0/3346247869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fe0073710 0x7f4fe0073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:28.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.297+0000 7f4fe63ff700 1 -- 172.21.15.2:0/3346247869 >> 172.21.15.2:0/3346247869 conn(0x7f4fe00fc220 msgr2=0x7f4fe00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:28.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.297+0000 7f4fe63ff700 1 -- 172.21.15.2:0/3346247869 shutdown_connections 2023-09-13T04:38:28.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.297+0000 7f4fe63ff700 1 -- 172.21.15.2:0/3346247869 wait complete. 2023-09-13T04:38:28.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.298+0000 7f4fe63ff700 1 Processor -- start 2023-09-13T04:38:28.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.298+0000 7f4fe63ff700 1 -- start start 2023-09-13T04:38:28.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.298+0000 7f4fe63ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe0073710 0x7f4fe0198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:28.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.298+0000 7f4fe63ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fe0074cf0 0x7f4fe01991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:28.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.298+0000 7f4fe63ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fe0108bb0 0x7f4fe01a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:28.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.298+0000 7f4fe63ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe01a0810 con 0x7f4fe0108bb0 2023-09-13T04:38:28.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.298+0000 7f4fe63ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe01a0980 con 0x7f4fe0073710 2023-09-13T04:38:28.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.298+0000 7f4fe63ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe01a0b40 con 0x7f4fe0074cf0 2023-09-13T04:38:28.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.299+0000 7f4fdffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe0073710 0x7f4fe0198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:28.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.299+0000 7f4fdffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe0073710 0x7f4fe0198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:56086/0 (socket says 172.21.15.2:56086) 2023-09-13T04:38:28.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.299+0000 7f4fdffff700 1 -- 172.21.15.2:0/2432936789 learned_addr learned my addr 172.21.15.2:0/2432936789 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:28.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.299+0000 7f4fe499c700 1 --2- 172.21.15.2:0/2432936789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fe0108bb0 0x7f4fe01a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:28.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.299+0000 7f4fdf7fe700 1 --2- 172.21.15.2:0/2432936789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fe0074cf0 0x7f4fe01991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:28.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.299+0000 7f4fdffff700 1 -- 172.21.15.2:0/2432936789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fe0074cf0 msgr2=0x7f4fe01991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:28.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.299+0000 7f4fdffff700 1 --2- 172.21.15.2:0/2432936789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fe0074cf0 0x7f4fe01991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:28.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.299+0000 7f4fdffff700 1 -- 172.21.15.2:0/2432936789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fe0108bb0 msgr2=0x7f4fe01a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:28.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.299+0000 7f4fdffff700 1 --2- 172.21.15.2:0/2432936789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fe0108bb0 0x7f4fe01a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:28.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.299+0000 7f4fdffff700 1 -- 172.21.15.2:0/2432936789 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd8009d00 con 0x7f4fe0073710 2023-09-13T04:38:28.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.299+0000 7f4fdffff700 1 --2- 172.21.15.2:0/2432936789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe0073710 0x7f4fe0198ca0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000e9a0 tx=0x7f4fd000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:28.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.299+0000 7f4fdd7fa700 1 -- 172.21.15.2:0/2432936789 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0009790 con 0x7f4fe0073710 2023-09-13T04:38:28.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.300+0000 7f4fe63ff700 1 -- 172.21.15.2:0/2432936789 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe01a0e20 con 0x7f4fe0073710 2023-09-13T04:38:28.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.300+0000 7f4fdd7fa700 1 -- 172.21.15.2:0/2432936789 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fd000cbd0 con 0x7f4fe0073710 2023-09-13T04:38:28.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.300+0000 7f4fdd7fa700 1 -- 172.21.15.2:0/2432936789 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0018680 con 0x7f4fe0073710 2023-09-13T04:38:28.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.300+0000 7f4fe63ff700 1 -- 172.21.15.2:0/2432936789 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe01a1250 con 0x7f4fe0073710 2023-09-13T04:38:28.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.301+0000 7f4fe63ff700 1 -- 172.21.15.2:0/2432936789 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe0107260 con 0x7f4fe0073710 2023-09-13T04:38:28.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.303+0000 7f4fdd7fa700 1 -- 172.21.15.2:0/2432936789 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f4fd000cd40 con 0x7f4fe0073710 2023-09-13T04:38:28.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.304+0000 7f4fdd7fa700 1 --2- 172.21.15.2:0/2432936789 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4fc80a0690 0x7f4fc80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:28.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.304+0000 7f4fdd7fa700 1 -- 172.21.15.2:0/2432936789 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4fd0014070 con 0x7f4fe0073710 2023-09-13T04:38:28.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.305+0000 7f4fdf7fe700 1 --2- 172.21.15.2:0/2432936789 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4fc80a0690 0x7f4fc80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:28.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.305+0000 7f4fdf7fe700 1 --2- 172.21.15.2:0/2432936789 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4fc80a0690 0x7f4fc80a2b50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4fe019a1a0 tx=0x7f4fd4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:28.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.306+0000 7f4fdd7fa700 1 -- 172.21.15.2:0/2432936789 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4fd0095bf0 con 0x7f4fe0073710 2023-09-13T04:38:28.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.515+0000 7f4fe63ff700 1 -- 172.21.15.2:0/2432936789 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4fe002d090 con 0x7f4fc80a0690 2023-09-13T04:38:28.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.528+0000 7f4fdd7fa700 1 -- 172.21.15.2:0/2432936789 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422580 (secure 0 0 0) 0x7f4fe002d090 con 0x7f4fc80a0690 2023-09-13T04:38:28.536 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:38:28.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.534+0000 7f4fe63ff700 1 -- 172.21.15.2:0/2432936789 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4fc80a0690 msgr2=0x7f4fc80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:28.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.534+0000 7f4fe63ff700 1 --2- 172.21.15.2:0/2432936789 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4fc80a0690 0x7f4fc80a2b50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4fe019a1a0 tx=0x7f4fd4009260 comp rx=0 tx=0).stop 2023-09-13T04:38:28.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.534+0000 7f4fe63ff700 1 -- 172.21.15.2:0/2432936789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe0073710 msgr2=0x7f4fe0198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:28.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.534+0000 7f4fe63ff700 1 --2- 172.21.15.2:0/2432936789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe0073710 0x7f4fe0198ca0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000e9a0 tx=0x7f4fd000ecb0 comp rx=0 tx=0).stop 2023-09-13T04:38:28.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.534+0000 7f4fe63ff700 1 -- 172.21.15.2:0/2432936789 shutdown_connections 2023-09-13T04:38:28.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.534+0000 7f4fe63ff700 1 --2- 172.21.15.2:0/2432936789 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4fc80a0690 0x7f4fc80a2b50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:28.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.534+0000 7f4fe63ff700 1 --2- 172.21.15.2:0/2432936789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fe0108bb0 0x7f4fe01a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:28.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.534+0000 7f4fe63ff700 1 --2- 172.21.15.2:0/2432936789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe0073710 0x7f4fe0198ca0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:28.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.534+0000 7f4fe63ff700 1 --2- 172.21.15.2:0/2432936789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fe0074cf0 0x7f4fe01991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:28.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.534+0000 7f4fe63ff700 1 -- 172.21.15.2:0/2432936789 >> 172.21.15.2:0/2432936789 conn(0x7f4fe00fc220 msgr2=0x7f4fe00fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:28.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.534+0000 7f4fe63ff700 1 -- 172.21.15.2:0/2432936789 shutdown_connections 2023-09-13T04:38:28.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.534+0000 7f4fe63ff700 1 -- 172.21.15.2:0/2432936789 wait complete. 2023-09-13T04:38:28.542 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-09-13T04:38:28.553 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2023-09-13T04:38:27.505931+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596996,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":138,"num_read_kb":121,"num_write":286,"num_write_kb":2970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":416,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337796,"kb_used_data":13532,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604028,"statfs":{"total":1151940427776,"available":1151594524672,"internally_reserved":0,"allocated":13856768,"data_stored":7620918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3769,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":59,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002540"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592832+0000","last_change":"2023-09-13T04:37:59.244279+0000","last_active":"2023-09-13T04:38:23.592832+0000","last_peered":"2023-09-13T04:38:23.592832+0000","last_clean":"2023-09-13T04:38:23.592832+0000","last_became_active":"2023-09-13T04:37:59.243976+0000","last_became_peered":"2023-09-13T04:37:59.243976+0000","last_unstale":"2023-09-13T04:38:23.592832+0000","last_undegraded":"2023-09-13T04:38:23.592832+0000","last_fullsized":"2023-09-13T04:38:23.592832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:21:52.809411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406851+0000","last_change":"2023-09-13T04:37:59.243391+0000","last_active":"2023-09-13T04:38:04.406851+0000","last_peered":"2023-09-13T04:38:04.406851+0000","last_clean":"2023-09-13T04:38:04.406851+0000","last_became_active":"2023-09-13T04:37:59.243044+0000","last_became_peered":"2023-09-13T04:37:59.243044+0000","last_unstale":"2023-09-13T04:38:04.406851+0000","last_undegraded":"2023-09-13T04:38:04.406851+0000","last_fullsized":"2023-09-13T04:38:04.406851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:02:17.806638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406414+0000","last_change":"2023-09-13T04:37:59.242826+0000","last_active":"2023-09-13T04:38:04.406414+0000","last_peered":"2023-09-13T04:38:04.406414+0000","last_clean":"2023-09-13T04:38:04.406414+0000","last_became_active":"2023-09-13T04:37:59.242125+0000","last_became_peered":"2023-09-13T04:37:59.242125+0000","last_unstale":"2023-09-13T04:38:04.406414+0000","last_undegraded":"2023-09-13T04:38:04.406414+0000","last_fullsized":"2023-09-13T04:38:04.406414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:08:29.216093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590536+0000","last_change":"2023-09-13T04:37:59.253322+0000","last_active":"2023-09-13T04:38:23.590536+0000","last_peered":"2023-09-13T04:38:23.590536+0000","last_clean":"2023-09-13T04:38:23.590536+0000","last_became_active":"2023-09-13T04:37:59.253050+0000","last_became_peered":"2023-09-13T04:37:59.253050+0000","last_unstale":"2023-09-13T04:38:23.590536+0000","last_undegraded":"2023-09-13T04:38:23.590536+0000","last_fullsized":"2023-09-13T04:38:23.590536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:45:22.357879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593487+0000","last_change":"2023-09-13T04:37:59.250089+0000","last_active":"2023-09-13T04:38:23.593487+0000","last_peered":"2023-09-13T04:38:23.593487+0000","last_clean":"2023-09-13T04:38:23.593487+0000","last_became_active":"2023-09-13T04:37:59.249771+0000","last_became_peered":"2023-09-13T04:37:59.249771+0000","last_unstale":"2023-09-13T04:38:23.593487+0000","last_undegraded":"2023-09-13T04:38:23.593487+0000","last_fullsized":"2023-09-13T04:38:23.593487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:46:59.917756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595139+0000","last_change":"2023-09-13T04:37:59.247259+0000","last_active":"2023-09-13T04:38:23.595139+0000","last_peered":"2023-09-13T04:38:23.595139+0000","last_clean":"2023-09-13T04:38:23.595139+0000","last_became_active":"2023-09-13T04:37:59.246911+0000","last_became_peered":"2023-09-13T04:37:59.246911+0000","last_unstale":"2023-09-13T04:38:23.595139+0000","last_undegraded":"2023-09-13T04:38:23.595139+0000","last_fullsized":"2023-09-13T04:38:23.595139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:00:39.742392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.407650+0000","last_change":"2023-09-13T04:37:59.244083+0000","last_active":"2023-09-13T04:38:04.407650+0000","last_peered":"2023-09-13T04:38:04.407650+0000","last_clean":"2023-09-13T04:38:04.407650+0000","last_became_active":"2023-09-13T04:37:59.243789+0000","last_became_peered":"2023-09-13T04:37:59.243789+0000","last_unstale":"2023-09-13T04:38:04.407650+0000","last_undegraded":"2023-09-13T04:38:04.407650+0000","last_fullsized":"2023-09-13T04:38:04.407650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:24:00.622790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.602097+0000","last_change":"2023-09-13T04:38:24.601265+0000","last_active":"2023-09-13T04:38:24.602097+0000","last_peered":"2023-09-13T04:38:24.602097+0000","last_clean":"2023-09-13T04:38:24.602097+0000","last_became_active":"2023-09-13T04:38:24.600939+0000","last_became_peered":"2023-09-13T04:38:24.600939+0000","last_unstale":"2023-09-13T04:38:24.602097+0000","last_undegraded":"2023-09-13T04:38:24.602097+0000","last_fullsized":"2023-09-13T04:38:24.602097+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:18:23.642333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594759+0000","last_change":"2023-09-13T04:37:59.238053+0000","last_active":"2023-09-13T04:38:23.594759+0000","last_peered":"2023-09-13T04:38:23.594759+0000","last_clean":"2023-09-13T04:38:23.594759+0000","last_became_active":"2023-09-13T04:37:59.237777+0000","last_became_peered":"2023-09-13T04:37:59.237777+0000","last_unstale":"2023-09-13T04:38:23.594759+0000","last_undegraded":"2023-09-13T04:38:23.594759+0000","last_fullsized":"2023-09-13T04:38:23.594759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:45:39.835659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594929+0000","last_change":"2023-09-13T04:37:59.252605+0000","last_active":"2023-09-13T04:38:23.594929+0000","last_peered":"2023-09-13T04:38:23.594929+0000","last_clean":"2023-09-13T04:38:23.594929+0000","last_became_active":"2023-09-13T04:37:59.252309+0000","last_became_peered":"2023-09-13T04:37:59.252309+0000","last_unstale":"2023-09-13T04:38:23.594929+0000","last_undegraded":"2023-09-13T04:38:23.594929+0000","last_fullsized":"2023-09-13T04:38:23.594929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:41:45.064230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.408085+0000","last_change":"2023-09-13T04:37:59.243345+0000","last_active":"2023-09-13T04:38:04.408085+0000","last_peered":"2023-09-13T04:38:04.408085+0000","last_clean":"2023-09-13T04:38:04.408085+0000","last_became_active":"2023-09-13T04:37:59.242726+0000","last_became_peered":"2023-09-13T04:37:59.242726+0000","last_unstale":"2023-09-13T04:38:04.408085+0000","last_undegraded":"2023-09-13T04:38:04.408085+0000","last_fullsized":"2023-09-13T04:38:04.408085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:39:55.115281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590601+0000","last_change":"2023-09-13T04:37:59.247415+0000","last_active":"2023-09-13T04:38:23.590601+0000","last_peered":"2023-09-13T04:38:23.590601+0000","last_clean":"2023-09-13T04:38:23.590601+0000","last_became_active":"2023-09-13T04:37:59.247050+0000","last_became_peered":"2023-09-13T04:37:59.247050+0000","last_unstale":"2023-09-13T04:38:23.590601+0000","last_undegraded":"2023-09-13T04:38:23.590601+0000","last_fullsized":"2023-09-13T04:38:23.590601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:03:44.793628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597902+0000","last_change":"2023-09-13T04:37:59.248935+0000","last_active":"2023-09-13T04:38:23.597902+0000","last_peered":"2023-09-13T04:38:23.597902+0000","last_clean":"2023-09-13T04:38:23.597902+0000","last_became_active":"2023-09-13T04:37:59.248554+0000","last_became_peered":"2023-09-13T04:37:59.248554+0000","last_unstale":"2023-09-13T04:38:23.597902+0000","last_undegraded":"2023-09-13T04:38:23.597902+0000","last_fullsized":"2023-09-13T04:38:23.597902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:53:31.882567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406214+0000","last_change":"2023-09-13T04:37:59.242851+0000","last_active":"2023-09-13T04:38:04.406214+0000","last_peered":"2023-09-13T04:38:04.406214+0000","last_clean":"2023-09-13T04:38:04.406214+0000","last_became_active":"2023-09-13T04:37:59.242384+0000","last_became_peered":"2023-09-13T04:37:59.242384+0000","last_unstale":"2023-09-13T04:38:04.406214+0000","last_undegraded":"2023-09-13T04:38:04.406214+0000","last_fullsized":"2023-09-13T04:38:04.406214+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:59:36.508808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"peering","last_fresh":"2023-09-13T04:38:04.406264+0000","last_change":"2023-09-13T04:38:23.591313+0000","last_active":"2023-09-13T04:38:23.591313+0000","last_peered":"2023-09-13T04:38:04.406264+0000","last_clean":"2023-09-13T04:38:04.406264+0000","last_became_active":"2023-09-13T04:37:59.252899+0000","last_became_peered":"2023-09-13T04:37:59.252899+0000","last_unstale":"2023-09-13T04:38:04.406264+0000","last_undegraded":"2023-09-13T04:38:04.406264+0000","last_fullsized":"2023-09-13T04:38:04.406264+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4,6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592657+0000","last_change":"2023-09-13T04:37:59.249107+0000","last_active":"2023-09-13T04:38:23.592657+0000","last_peered":"2023-09-13T04:38:23.592657+0000","last_clean":"2023-09-13T04:38:23.592657+0000","last_became_active":"2023-09-13T04:37:59.248803+0000","last_became_peered":"2023-09-13T04:37:59.248803+0000","last_unstale":"2023-09-13T04:38:23.592657+0000","last_undegraded":"2023-09-13T04:38:23.592657+0000","last_fullsized":"2023-09-13T04:38:23.592657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:50:52.481194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592743+0000","last_change":"2023-09-13T04:37:59.243724+0000","last_active":"2023-09-13T04:38:23.592743+0000","last_peered":"2023-09-13T04:38:23.592743+0000","last_clean":"2023-09-13T04:38:23.592743+0000","last_became_active":"2023-09-13T04:37:59.243385+0000","last_became_peered":"2023-09-13T04:37:59.243385+0000","last_unstale":"2023-09-13T04:38:23.592743+0000","last_undegraded":"2023-09-13T04:38:23.592743+0000","last_fullsized":"2023-09-13T04:38:23.592743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:55:55.203940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588550+0000","last_change":"2023-09-13T04:37:59.238986+0000","last_active":"2023-09-13T04:38:23.588550+0000","last_peered":"2023-09-13T04:38:23.588550+0000","last_clean":"2023-09-13T04:38:23.588550+0000","last_became_active":"2023-09-13T04:37:59.238668+0000","last_became_peered":"2023-09-13T04:37:59.238668+0000","last_unstale":"2023-09-13T04:38:23.588550+0000","last_undegraded":"2023-09-13T04:38:23.588550+0000","last_fullsized":"2023-09-13T04:38:23.588550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:21:10.225223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588585+0000","last_change":"2023-09-13T04:37:59.245598+0000","last_active":"2023-09-13T04:38:23.588585+0000","last_peered":"2023-09-13T04:38:23.588585+0000","last_clean":"2023-09-13T04:38:23.588585+0000","last_became_active":"2023-09-13T04:37:59.244652+0000","last_became_peered":"2023-09-13T04:37:59.244652+0000","last_unstale":"2023-09-13T04:38:23.588585+0000","last_undegraded":"2023-09-13T04:38:23.588585+0000","last_fullsized":"2023-09-13T04:38:23.588585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:34:43.671168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588238+0000","last_change":"2023-09-13T04:38:03.617708+0000","last_active":"2023-09-13T04:38:23.588238+0000","last_peered":"2023-09-13T04:38:23.588238+0000","last_clean":"2023-09-13T04:38:23.588238+0000","last_became_active":"2023-09-13T04:38:03.617277+0000","last_became_peered":"2023-09-13T04:38:03.617277+0000","last_unstale":"2023-09-13T04:38:23.588238+0000","last_undegraded":"2023-09-13T04:38:23.588238+0000","last_fullsized":"2023-09-13T04:38:23.588238+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:26:44.318030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.621267+0000","last_change":"2023-09-13T04:37:58.595156+0000","last_active":"2023-09-13T04:38:25.621267+0000","last_peered":"2023-09-13T04:38:25.621267+0000","last_clean":"2023-09-13T04:38:25.621267+0000","last_became_active":"2023-09-13T04:37:58.594863+0000","last_became_peered":"2023-09-13T04:37:58.594863+0000","last_unstale":"2023-09-13T04:38:25.621267+0000","last_undegraded":"2023-09-13T04:38:25.621267+0000","last_fullsized":"2023-09-13T04:38:25.621267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:27:14.420495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.602369+0000","last_change":"2023-09-13T04:37:59.250252+0000","last_active":"2023-09-13T04:38:03.602369+0000","last_peered":"2023-09-13T04:38:03.602369+0000","last_clean":"2023-09-13T04:38:03.602369+0000","last_became_active":"2023-09-13T04:37:59.249927+0000","last_became_peered":"2023-09-13T04:37:59.249927+0000","last_unstale":"2023-09-13T04:38:03.602369+0000","last_undegraded":"2023-09-13T04:38:03.602369+0000","last_fullsized":"2023-09-13T04:38:03.602369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:18:06.757040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.599683+0000","last_change":"2023-09-13T04:38:24.599683+0000","last_active":"2023-09-13T04:38:24.599683+0000","last_peered":"2023-09-13T04:38:24.599683+0000","last_clean":"2023-09-13T04:38:24.599683+0000","last_became_active":"2023-09-13T04:38:24.599230+0000","last_became_peered":"2023-09-13T04:38:24.599230+0000","last_unstale":"2023-09-13T04:38:24.599683+0000","last_undegraded":"2023-09-13T04:38:24.599683+0000","last_fullsized":"2023-09-13T04:38:24.599683+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:48:22.584735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.602561+0000","last_change":"2023-09-13T04:37:58.597785+0000","last_active":"2023-09-13T04:38:03.602561+0000","last_peered":"2023-09-13T04:38:03.602561+0000","last_clean":"2023-09-13T04:38:03.602561+0000","last_became_active":"2023-09-13T04:37:58.597485+0000","last_became_peered":"2023-09-13T04:37:58.597485+0000","last_unstale":"2023-09-13T04:38:03.602561+0000","last_undegraded":"2023-09-13T04:38:03.602561+0000","last_fullsized":"2023-09-13T04:38:03.602561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:39:57.156038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406705+0000","last_change":"2023-09-13T04:37:59.241840+0000","last_active":"2023-09-13T04:38:04.406705+0000","last_peered":"2023-09-13T04:38:04.406705+0000","last_clean":"2023-09-13T04:38:04.406705+0000","last_became_active":"2023-09-13T04:37:59.241545+0000","last_became_peered":"2023-09-13T04:37:59.241545+0000","last_unstale":"2023-09-13T04:38:04.406705+0000","last_undegraded":"2023-09-13T04:38:04.406705+0000","last_fullsized":"2023-09-13T04:38:04.406705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:04:17.439520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591101+0000","last_change":"2023-09-13T04:38:03.621792+0000","last_active":"2023-09-13T04:38:23.591101+0000","last_peered":"2023-09-13T04:38:23.591101+0000","last_clean":"2023-09-13T04:38:23.591101+0000","last_became_active":"2023-09-13T04:38:03.621433+0000","last_became_peered":"2023-09-13T04:38:03.621433+0000","last_unstale":"2023-09-13T04:38:23.591101+0000","last_undegraded":"2023-09-13T04:38:23.591101+0000","last_fullsized":"2023-09-13T04:38:23.591101+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:52:05.926733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594362+0000","last_change":"2023-09-13T04:37:58.590817+0000","last_active":"2023-09-13T04:38:23.594362+0000","last_peered":"2023-09-13T04:38:23.594362+0000","last_clean":"2023-09-13T04:38:23.594362+0000","last_became_active":"2023-09-13T04:37:58.590348+0000","last_became_peered":"2023-09-13T04:37:58.590348+0000","last_unstale":"2023-09-13T04:38:23.594362+0000","last_undegraded":"2023-09-13T04:38:23.594362+0000","last_fullsized":"2023-09-13T04:38:23.594362+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:38:35.680905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594252+0000","last_change":"2023-09-13T04:37:59.237304+0000","last_active":"2023-09-13T04:38:23.594252+0000","last_peered":"2023-09-13T04:38:23.594252+0000","last_clean":"2023-09-13T04:38:23.594252+0000","last_became_active":"2023-09-13T04:37:59.237008+0000","last_became_peered":"2023-09-13T04:37:59.237008+0000","last_unstale":"2023-09-13T04:38:23.594252+0000","last_undegraded":"2023-09-13T04:38:23.594252+0000","last_fullsized":"2023-09-13T04:38:23.594252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:28:14.254219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590193+0000","last_change":"2023-09-13T04:38:03.606749+0000","last_active":"2023-09-13T04:38:23.590193+0000","last_peered":"2023-09-13T04:38:23.590193+0000","last_clean":"2023-09-13T04:38:23.590193+0000","last_became_active":"2023-09-13T04:38:03.606276+0000","last_became_peered":"2023-09-13T04:38:03.606276+0000","last_unstale":"2023-09-13T04:38:23.590193+0000","last_undegraded":"2023-09-13T04:38:23.590193+0000","last_fullsized":"2023-09-13T04:38:23.590193+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:30:53.572543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593692+0000","last_change":"2023-09-13T04:37:58.590937+0000","last_active":"2023-09-13T04:38:23.593692+0000","last_peered":"2023-09-13T04:38:23.593692+0000","last_clean":"2023-09-13T04:38:23.593692+0000","last_became_active":"2023-09-13T04:37:58.590606+0000","last_became_peered":"2023-09-13T04:37:58.590606+0000","last_unstale":"2023-09-13T04:38:23.593692+0000","last_undegraded":"2023-09-13T04:38:23.593692+0000","last_fullsized":"2023-09-13T04:38:23.593692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:30:03.354670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591224+0000","last_change":"2023-09-13T04:37:59.251899+0000","last_active":"2023-09-13T04:38:23.591224+0000","last_peered":"2023-09-13T04:38:23.591224+0000","last_clean":"2023-09-13T04:38:23.591224+0000","last_became_active":"2023-09-13T04:37:59.251557+0000","last_became_peered":"2023-09-13T04:37:59.251557+0000","last_unstale":"2023-09-13T04:38:23.591224+0000","last_undegraded":"2023-09-13T04:38:23.591224+0000","last_fullsized":"2023-09-13T04:38:23.591224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:57:45.241188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588439+0000","last_change":"2023-09-13T04:38:03.606076+0000","last_active":"2023-09-13T04:38:23.588439+0000","last_peered":"2023-09-13T04:38:23.588439+0000","last_clean":"2023-09-13T04:38:23.588439+0000","last_became_active":"2023-09-13T04:38:03.605556+0000","last_became_peered":"2023-09-13T04:38:03.605556+0000","last_unstale":"2023-09-13T04:38:23.588439+0000","last_undegraded":"2023-09-13T04:38:23.588439+0000","last_fullsized":"2023-09-13T04:38:23.588439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:03:32.344625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.246702+0000","last_change":"2023-09-13T04:37:58.588170+0000","last_active":"2023-09-13T04:38:24.246702+0000","last_peered":"2023-09-13T04:38:24.246702+0000","last_clean":"2023-09-13T04:38:24.246702+0000","last_became_active":"2023-09-13T04:37:58.587860+0000","last_became_peered":"2023-09-13T04:37:58.587860+0000","last_unstale":"2023-09-13T04:38:24.246702+0000","last_undegraded":"2023-09-13T04:38:24.246702+0000","last_fullsized":"2023-09-13T04:38:24.246702+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:38:40.048150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595472+0000","last_change":"2023-09-13T04:37:59.240248+0000","last_active":"2023-09-13T04:38:23.595472+0000","last_peered":"2023-09-13T04:38:23.595472+0000","last_clean":"2023-09-13T04:38:23.595472+0000","last_became_active":"2023-09-13T04:37:59.240008+0000","last_became_peered":"2023-09-13T04:37:59.240008+0000","last_unstale":"2023-09-13T04:38:23.595472+0000","last_undegraded":"2023-09-13T04:38:23.595472+0000","last_fullsized":"2023-09-13T04:38:23.595472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:15:13.275110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587821+0000","last_change":"2023-09-13T04:38:03.616607+0000","last_active":"2023-09-13T04:38:23.587821+0000","last_peered":"2023-09-13T04:38:23.587821+0000","last_clean":"2023-09-13T04:38:23.587821+0000","last_became_active":"2023-09-13T04:38:03.616209+0000","last_became_peered":"2023-09-13T04:38:03.616209+0000","last_unstale":"2023-09-13T04:38:23.587821+0000","last_undegraded":"2023-09-13T04:38:23.587821+0000","last_fullsized":"2023-09-13T04:38:23.587821+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:24:21.934053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595102+0000","last_change":"2023-09-13T04:37:58.589104+0000","last_active":"2023-09-13T04:38:23.595102+0000","last_peered":"2023-09-13T04:38:23.595102+0000","last_clean":"2023-09-13T04:38:23.595102+0000","last_became_active":"2023-09-13T04:37:58.588780+0000","last_became_peered":"2023-09-13T04:37:58.588780+0000","last_unstale":"2023-09-13T04:38:23.595102+0000","last_undegraded":"2023-09-13T04:38:23.595102+0000","last_fullsized":"2023-09-13T04:38:23.595102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:01:32.839729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.600030+0000","last_change":"2023-09-13T04:38:24.598663+0000","last_active":"2023-09-13T04:38:24.600030+0000","last_peered":"2023-09-13T04:38:24.600030+0000","last_clean":"2023-09-13T04:38:24.600030+0000","last_became_active":"2023-09-13T04:38:24.598258+0000","last_became_peered":"2023-09-13T04:38:24.598258+0000","last_unstale":"2023-09-13T04:38:24.600030+0000","last_undegraded":"2023-09-13T04:38:24.600030+0000","last_fullsized":"2023-09-13T04:38:24.600030+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:50:28.157135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592633+0000","last_change":"2023-09-13T04:38:03.619930+0000","last_active":"2023-09-13T04:38:23.592633+0000","last_peered":"2023-09-13T04:38:23.592633+0000","last_clean":"2023-09-13T04:38:23.592633+0000","last_became_active":"2023-09-13T04:38:03.619560+0000","last_became_peered":"2023-09-13T04:38:03.619560+0000","last_unstale":"2023-09-13T04:38:23.592633+0000","last_undegraded":"2023-09-13T04:38:23.592633+0000","last_fullsized":"2023-09-13T04:38:23.592633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:48:06.867602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.619713+0000","last_change":"2023-09-13T04:37:58.590396+0000","last_active":"2023-09-13T04:38:25.619713+0000","last_peered":"2023-09-13T04:38:25.619713+0000","last_clean":"2023-09-13T04:38:25.619713+0000","last_became_active":"2023-09-13T04:37:58.589929+0000","last_became_peered":"2023-09-13T04:37:58.589929+0000","last_unstale":"2023-09-13T04:38:25.619713+0000","last_undegraded":"2023-09-13T04:38:25.619713+0000","last_fullsized":"2023-09-13T04:38:25.619713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:28:36.322096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.600166+0000","last_change":"2023-09-13T04:37:59.246033+0000","last_active":"2023-09-13T04:38:03.600166+0000","last_peered":"2023-09-13T04:38:03.600166+0000","last_clean":"2023-09-13T04:38:03.600166+0000","last_became_active":"2023-09-13T04:37:59.245664+0000","last_became_peered":"2023-09-13T04:37:59.245664+0000","last_unstale":"2023-09-13T04:38:03.600166+0000","last_undegraded":"2023-09-13T04:38:03.600166+0000","last_fullsized":"2023-09-13T04:38:03.600166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:24:17.145055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406894+0000","last_change":"2023-09-13T04:38:03.608504+0000","last_active":"2023-09-13T04:38:04.406894+0000","last_peered":"2023-09-13T04:38:04.406894+0000","last_clean":"2023-09-13T04:38:04.406894+0000","last_became_active":"2023-09-13T04:38:03.608058+0000","last_became_peered":"2023-09-13T04:38:03.608058+0000","last_unstale":"2023-09-13T04:38:04.406894+0000","last_undegraded":"2023-09-13T04:38:04.406894+0000","last_fullsized":"2023-09-13T04:38:04.406894+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:42:43.485453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244691+0000","last_change":"2023-09-13T04:37:58.588208+0000","last_active":"2023-09-13T04:38:24.244691+0000","last_peered":"2023-09-13T04:38:24.244691+0000","last_clean":"2023-09-13T04:38:24.244691+0000","last_became_active":"2023-09-13T04:37:58.587840+0000","last_became_peered":"2023-09-13T04:37:58.587840+0000","last_unstale":"2023-09-13T04:38:24.244691+0000","last_undegraded":"2023-09-13T04:38:24.244691+0000","last_fullsized":"2023-09-13T04:38:24.244691+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:44:58.012233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587683+0000","last_change":"2023-09-13T04:37:59.245102+0000","last_active":"2023-09-13T04:38:23.587683+0000","last_peered":"2023-09-13T04:38:23.587683+0000","last_clean":"2023-09-13T04:38:23.587683+0000","last_became_active":"2023-09-13T04:37:59.244799+0000","last_became_peered":"2023-09-13T04:37:59.244799+0000","last_unstale":"2023-09-13T04:38:23.587683+0000","last_undegraded":"2023-09-13T04:38:23.587683+0000","last_fullsized":"2023-09-13T04:38:23.587683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:57:16.295623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.454268+0000","last_change":"2023-09-13T04:38:03.607021+0000","last_active":"2023-09-13T04:38:04.454268+0000","last_peered":"2023-09-13T04:38:04.454268+0000","last_clean":"2023-09-13T04:38:04.454268+0000","last_became_active":"2023-09-13T04:38:03.606611+0000","last_became_peered":"2023-09-13T04:38:03.606611+0000","last_unstale":"2023-09-13T04:38:04.454268+0000","last_undegraded":"2023-09-13T04:38:04.454268+0000","last_fullsized":"2023-09-13T04:38:04.454268+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:33:20.551479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244009+0000","last_change":"2023-09-13T04:37:58.588731+0000","last_active":"2023-09-13T04:38:24.244009+0000","last_peered":"2023-09-13T04:38:24.244009+0000","last_clean":"2023-09-13T04:38:24.244009+0000","last_became_active":"2023-09-13T04:37:58.588398+0000","last_became_peered":"2023-09-13T04:37:58.588398+0000","last_unstale":"2023-09-13T04:38:24.244009+0000","last_undegraded":"2023-09-13T04:38:24.244009+0000","last_fullsized":"2023-09-13T04:38:24.244009+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:13:10.724085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594790+0000","last_change":"2023-09-13T04:37:59.244757+0000","last_active":"2023-09-13T04:38:23.594790+0000","last_peered":"2023-09-13T04:38:23.594790+0000","last_clean":"2023-09-13T04:38:23.594790+0000","last_became_active":"2023-09-13T04:37:59.244423+0000","last_became_peered":"2023-09-13T04:37:59.244423+0000","last_unstale":"2023-09-13T04:38:23.594790+0000","last_undegraded":"2023-09-13T04:38:23.594790+0000","last_fullsized":"2023-09-13T04:38:23.594790+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:14:37.973058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587303+0000","last_change":"2023-09-13T04:38:03.618358+0000","last_active":"2023-09-13T04:38:23.587303+0000","last_peered":"2023-09-13T04:38:23.587303+0000","last_clean":"2023-09-13T04:38:23.587303+0000","last_became_active":"2023-09-13T04:38:03.617291+0000","last_became_peered":"2023-09-13T04:38:03.617291+0000","last_unstale":"2023-09-13T04:38:23.587303+0000","last_undegraded":"2023-09-13T04:38:23.587303+0000","last_fullsized":"2023-09-13T04:38:23.587303+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:33:13.955353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244399+0000","last_change":"2023-09-13T04:37:58.590978+0000","last_active":"2023-09-13T04:38:24.244399+0000","last_peered":"2023-09-13T04:38:24.244399+0000","last_clean":"2023-09-13T04:38:24.244399+0000","last_became_active":"2023-09-13T04:37:58.590673+0000","last_became_peered":"2023-09-13T04:37:58.590673+0000","last_unstale":"2023-09-13T04:38:24.244399+0000","last_undegraded":"2023-09-13T04:38:24.244399+0000","last_fullsized":"2023-09-13T04:38:24.244399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:10:32.570029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406610+0000","last_change":"2023-09-13T04:37:59.251283+0000","last_active":"2023-09-13T04:38:04.406610+0000","last_peered":"2023-09-13T04:38:04.406610+0000","last_clean":"2023-09-13T04:38:04.406610+0000","last_became_active":"2023-09-13T04:37:59.250119+0000","last_became_peered":"2023-09-13T04:37:59.250119+0000","last_unstale":"2023-09-13T04:38:04.406610+0000","last_undegraded":"2023-09-13T04:38:04.406610+0000","last_fullsized":"2023-09-13T04:38:04.406610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:48:54.389970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.596107+0000","last_change":"2023-09-13T04:38:24.596107+0000","last_active":"2023-09-13T04:38:24.596107+0000","last_peered":"2023-09-13T04:38:24.596107+0000","last_clean":"2023-09-13T04:38:24.596107+0000","last_became_active":"2023-09-13T04:38:24.595653+0000","last_became_peered":"2023-09-13T04:38:24.595653+0000","last_unstale":"2023-09-13T04:38:24.596107+0000","last_undegraded":"2023-09-13T04:38:24.596107+0000","last_fullsized":"2023-09-13T04:38:24.596107+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:02:53.708658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.243198+0000","last_change":"2023-09-13T04:37:58.584909+0000","last_active":"2023-09-13T04:38:24.243198+0000","last_peered":"2023-09-13T04:38:24.243198+0000","last_clean":"2023-09-13T04:38:24.243198+0000","last_became_active":"2023-09-13T04:37:58.584630+0000","last_became_peered":"2023-09-13T04:37:58.584630+0000","last_unstale":"2023-09-13T04:38:24.243198+0000","last_undegraded":"2023-09-13T04:38:24.243198+0000","last_fullsized":"2023-09-13T04:38:24.243198+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:57:56.567499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594349+0000","last_change":"2023-09-13T04:37:59.254448+0000","last_active":"2023-09-13T04:38:23.594349+0000","last_peered":"2023-09-13T04:38:23.594349+0000","last_clean":"2023-09-13T04:38:23.594349+0000","last_became_active":"2023-09-13T04:37:59.254138+0000","last_became_peered":"2023-09-13T04:37:59.254138+0000","last_unstale":"2023-09-13T04:38:23.594349+0000","last_undegraded":"2023-09-13T04:38:23.594349+0000","last_fullsized":"2023-09-13T04:38:23.594349+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:36:14.939881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588357+0000","last_change":"2023-09-13T04:38:03.613746+0000","last_active":"2023-09-13T04:38:23.588357+0000","last_peered":"2023-09-13T04:38:23.588357+0000","last_clean":"2023-09-13T04:38:23.588357+0000","last_became_active":"2023-09-13T04:38:03.613354+0000","last_became_peered":"2023-09-13T04:38:03.613354+0000","last_unstale":"2023-09-13T04:38:23.588357+0000","last_undegraded":"2023-09-13T04:38:23.588357+0000","last_fullsized":"2023-09-13T04:38:23.588357+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:41:01.294912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.242914+0000","last_change":"2023-09-13T04:37:58.591196+0000","last_active":"2023-09-13T04:38:24.242914+0000","last_peered":"2023-09-13T04:38:24.242914+0000","last_clean":"2023-09-13T04:38:24.242914+0000","last_became_active":"2023-09-13T04:37:58.590883+0000","last_became_peered":"2023-09-13T04:37:58.590883+0000","last_unstale":"2023-09-13T04:38:24.242914+0000","last_undegraded":"2023-09-13T04:38:24.242914+0000","last_fullsized":"2023-09-13T04:38:24.242914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:46:58.530270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595499+0000","last_change":"2023-09-13T04:37:59.243483+0000","last_active":"2023-09-13T04:38:23.595499+0000","last_peered":"2023-09-13T04:38:23.595499+0000","last_clean":"2023-09-13T04:38:23.595499+0000","last_became_active":"2023-09-13T04:37:59.243088+0000","last_became_peered":"2023-09-13T04:37:59.243088+0000","last_unstale":"2023-09-13T04:38:23.595499+0000","last_undegraded":"2023-09-13T04:38:23.595499+0000","last_fullsized":"2023-09-13T04:38:23.595499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:56:14.093819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591697+0000","last_change":"2023-09-13T04:38:03.608820+0000","last_active":"2023-09-13T04:38:23.591697+0000","last_peered":"2023-09-13T04:38:23.591697+0000","last_clean":"2023-09-13T04:38:23.591697+0000","last_became_active":"2023-09-13T04:38:03.608200+0000","last_became_peered":"2023-09-13T04:38:03.608200+0000","last_unstale":"2023-09-13T04:38:23.591697+0000","last_undegraded":"2023-09-13T04:38:23.591697+0000","last_fullsized":"2023-09-13T04:38:23.591697+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:54:52.416501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.607235+0000","last_change":"2023-09-13T04:38:24.606820+0000","last_active":"2023-09-13T04:38:24.607235+0000","last_peered":"2023-09-13T04:38:24.607235+0000","last_clean":"2023-09-13T04:38:24.607235+0000","last_became_active":"2023-09-13T04:38:24.599059+0000","last_became_peered":"2023-09-13T04:38:24.599059+0000","last_unstale":"2023-09-13T04:38:24.607235+0000","last_undegraded":"2023-09-13T04:38:24.607235+0000","last_fullsized":"2023-09-13T04:38:24.607235+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:08:12.634357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597568+0000","last_change":"2023-09-13T04:37:59.250666+0000","last_active":"2023-09-13T04:38:23.597568+0000","last_peered":"2023-09-13T04:38:23.597568+0000","last_clean":"2023-09-13T04:38:23.597568+0000","last_became_active":"2023-09-13T04:37:59.250240+0000","last_became_peered":"2023-09-13T04:37:59.250240+0000","last_unstale":"2023-09-13T04:38:23.597568+0000","last_undegraded":"2023-09-13T04:38:23.597568+0000","last_fullsized":"2023-09-13T04:38:23.597568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:44:31.324222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592271+0000","last_change":"2023-09-13T04:38:03.616296+0000","last_active":"2023-09-13T04:38:23.592271+0000","last_peered":"2023-09-13T04:38:23.592271+0000","last_clean":"2023-09-13T04:38:23.592271+0000","last_became_active":"2023-09-13T04:38:03.615934+0000","last_became_peered":"2023-09-13T04:38:03.615934+0000","last_unstale":"2023-09-13T04:38:23.592271+0000","last_undegraded":"2023-09-13T04:38:23.592271+0000","last_fullsized":"2023-09-13T04:38:23.592271+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:51:18.994472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244805+0000","last_change":"2023-09-13T04:37:58.594782+0000","last_active":"2023-09-13T04:38:24.244805+0000","last_peered":"2023-09-13T04:38:24.244805+0000","last_clean":"2023-09-13T04:38:24.244805+0000","last_became_active":"2023-09-13T04:37:58.594480+0000","last_became_peered":"2023-09-13T04:37:58.594480+0000","last_unstale":"2023-09-13T04:38:24.244805+0000","last_undegraded":"2023-09-13T04:38:24.244805+0000","last_fullsized":"2023-09-13T04:38:24.244805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:40:19.635672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.407475+0000","last_change":"2023-09-13T04:37:59.252575+0000","last_active":"2023-09-13T04:38:04.407475+0000","last_peered":"2023-09-13T04:38:04.407475+0000","last_clean":"2023-09-13T04:38:04.407475+0000","last_became_active":"2023-09-13T04:37:59.252268+0000","last_became_peered":"2023-09-13T04:37:59.252268+0000","last_unstale":"2023-09-13T04:38:04.407475+0000","last_undegraded":"2023-09-13T04:38:04.407475+0000","last_fullsized":"2023-09-13T04:38:04.407475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:27:10.814930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587908+0000","last_change":"2023-09-13T04:38:03.618835+0000","last_active":"2023-09-13T04:38:23.587908+0000","last_peered":"2023-09-13T04:38:23.587908+0000","last_clean":"2023-09-13T04:38:23.587908+0000","last_became_active":"2023-09-13T04:38:03.618226+0000","last_became_peered":"2023-09-13T04:38:03.618226+0000","last_unstale":"2023-09-13T04:38:23.587908+0000","last_undegraded":"2023-09-13T04:38:23.587908+0000","last_fullsized":"2023-09-13T04:38:23.587908+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:01:28.157490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590730+0000","last_change":"2023-09-13T04:37:58.588545+0000","last_active":"2023-09-13T04:38:23.590730+0000","last_peered":"2023-09-13T04:38:23.590730+0000","last_clean":"2023-09-13T04:38:23.590730+0000","last_became_active":"2023-09-13T04:37:58.587995+0000","last_became_peered":"2023-09-13T04:37:58.587995+0000","last_unstale":"2023-09-13T04:38:23.590730+0000","last_undegraded":"2023-09-13T04:38:23.590730+0000","last_fullsized":"2023-09-13T04:38:23.590730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:40:08.797724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.407438+0000","last_change":"2023-09-13T04:37:59.249100+0000","last_active":"2023-09-13T04:38:04.407438+0000","last_peered":"2023-09-13T04:38:04.407438+0000","last_clean":"2023-09-13T04:38:04.407438+0000","last_became_active":"2023-09-13T04:37:59.248771+0000","last_became_peered":"2023-09-13T04:37:59.248771+0000","last_unstale":"2023-09-13T04:38:04.407438+0000","last_undegraded":"2023-09-13T04:38:04.407438+0000","last_fullsized":"2023-09-13T04:38:04.407438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:27:37.964595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597364+0000","last_change":"2023-09-13T04:38:03.616988+0000","last_active":"2023-09-13T04:38:23.597364+0000","last_peered":"2023-09-13T04:38:23.597364+0000","last_clean":"2023-09-13T04:38:23.597364+0000","last_became_active":"2023-09-13T04:38:03.616524+0000","last_became_peered":"2023-09-13T04:38:03.616524+0000","last_unstale":"2023-09-13T04:38:23.597364+0000","last_undegraded":"2023-09-13T04:38:23.597364+0000","last_fullsized":"2023-09-13T04:38:23.597364+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:42:41.431115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'10","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.613710+0000","last_change":"2023-09-13T04:38:24.613384+0000","last_active":"2023-09-13T04:38:24.613710+0000","last_peered":"2023-09-13T04:38:24.613710+0000","last_clean":"2023-09-13T04:38:24.613710+0000","last_became_active":"2023-09-13T04:38:24.599349+0000","last_became_peered":"2023-09-13T04:38:24.599349+0000","last_unstale":"2023-09-13T04:38:24.613710+0000","last_undegraded":"2023-09-13T04:38:24.613710+0000","last_fullsized":"2023-09-13T04:38:24.613710+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:23:23.479715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593329+0000","last_change":"2023-09-13T04:37:59.252237+0000","last_active":"2023-09-13T04:38:23.593329+0000","last_peered":"2023-09-13T04:38:23.593329+0000","last_clean":"2023-09-13T04:38:23.593329+0000","last_became_active":"2023-09-13T04:37:59.251718+0000","last_became_peered":"2023-09-13T04:37:59.251718+0000","last_unstale":"2023-09-13T04:38:23.593329+0000","last_undegraded":"2023-09-13T04:38:23.593329+0000","last_fullsized":"2023-09-13T04:38:23.593329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:45:48.405226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593276+0000","last_change":"2023-09-13T04:38:03.607096+0000","last_active":"2023-09-13T04:38:23.593276+0000","last_peered":"2023-09-13T04:38:23.593276+0000","last_clean":"2023-09-13T04:38:23.593276+0000","last_became_active":"2023-09-13T04:38:03.606656+0000","last_became_peered":"2023-09-13T04:38:03.606656+0000","last_unstale":"2023-09-13T04:38:23.593276+0000","last_undegraded":"2023-09-13T04:38:23.593276+0000","last_fullsized":"2023-09-13T04:38:23.593276+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:59:26.140360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595200+0000","last_change":"2023-09-13T04:37:58.591047+0000","last_active":"2023-09-13T04:38:23.595200+0000","last_peered":"2023-09-13T04:38:23.595200+0000","last_clean":"2023-09-13T04:38:23.595200+0000","last_became_active":"2023-09-13T04:37:58.590740+0000","last_became_peered":"2023-09-13T04:37:58.590740+0000","last_unstale":"2023-09-13T04:38:23.595200+0000","last_undegraded":"2023-09-13T04:38:23.595200+0000","last_fullsized":"2023-09-13T04:38:23.595200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:14:42.280416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592406+0000","last_change":"2023-09-13T04:37:59.249362+0000","last_active":"2023-09-13T04:38:23.592406+0000","last_peered":"2023-09-13T04:38:23.592406+0000","last_clean":"2023-09-13T04:38:23.592406+0000","last_became_active":"2023-09-13T04:37:59.249041+0000","last_became_peered":"2023-09-13T04:37:59.249041+0000","last_unstale":"2023-09-13T04:38:23.592406+0000","last_undegraded":"2023-09-13T04:38:23.592406+0000","last_fullsized":"2023-09-13T04:38:23.592406+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:06:33.822157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.598370+0000","last_change":"2023-09-13T04:38:24.598370+0000","last_active":"2023-09-13T04:38:24.598370+0000","last_peered":"2023-09-13T04:38:24.598370+0000","last_clean":"2023-09-13T04:38:24.598370+0000","last_became_active":"2023-09-13T04:38:24.597931+0000","last_became_peered":"2023-09-13T04:38:24.597931+0000","last_unstale":"2023-09-13T04:38:24.598370+0000","last_undegraded":"2023-09-13T04:38:24.598370+0000","last_fullsized":"2023-09-13T04:38:24.598370+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:14:28.911329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588100+0000","last_change":"2023-09-13T04:37:58.592496+0000","last_active":"2023-09-13T04:38:23.588100+0000","last_peered":"2023-09-13T04:38:23.588100+0000","last_clean":"2023-09-13T04:38:23.588100+0000","last_became_active":"2023-09-13T04:37:58.592106+0000","last_became_peered":"2023-09-13T04:37:58.592106+0000","last_unstale":"2023-09-13T04:38:23.588100+0000","last_undegraded":"2023-09-13T04:38:23.588100+0000","last_fullsized":"2023-09-13T04:38:23.588100+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:28:55.618563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594212+0000","last_change":"2023-09-13T04:37:59.250019+0000","last_active":"2023-09-13T04:38:23.594212+0000","last_peered":"2023-09-13T04:38:23.594212+0000","last_clean":"2023-09-13T04:38:23.594212+0000","last_became_active":"2023-09-13T04:37:59.249278+0000","last_became_peered":"2023-09-13T04:37:59.249278+0000","last_unstale":"2023-09-13T04:38:23.594212+0000","last_undegraded":"2023-09-13T04:38:23.594212+0000","last_fullsized":"2023-09-13T04:38:23.594212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:15:31.217236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597640+0000","last_change":"2023-09-13T04:38:03.614542+0000","last_active":"2023-09-13T04:38:23.597640+0000","last_peered":"2023-09-13T04:38:23.597640+0000","last_clean":"2023-09-13T04:38:23.597640+0000","last_became_active":"2023-09-13T04:38:03.614103+0000","last_became_peered":"2023-09-13T04:38:03.614103+0000","last_unstale":"2023-09-13T04:38:23.597640+0000","last_undegraded":"2023-09-13T04:38:23.597640+0000","last_fullsized":"2023-09-13T04:38:23.597640+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:06:48.426843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.407747+0000","last_change":"2023-09-13T04:37:58.591864+0000","last_active":"2023-09-13T04:38:04.407747+0000","last_peered":"2023-09-13T04:38:04.407747+0000","last_clean":"2023-09-13T04:38:04.407747+0000","last_became_active":"2023-09-13T04:37:58.591562+0000","last_became_peered":"2023-09-13T04:37:58.591562+0000","last_unstale":"2023-09-13T04:38:04.407747+0000","last_undegraded":"2023-09-13T04:38:04.407747+0000","last_fullsized":"2023-09-13T04:38:04.407747+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:31:27.280489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594567+0000","last_change":"2023-09-13T04:37:59.252600+0000","last_active":"2023-09-13T04:38:23.594567+0000","last_peered":"2023-09-13T04:38:23.594567+0000","last_clean":"2023-09-13T04:38:23.594567+0000","last_became_active":"2023-09-13T04:37:59.252306+0000","last_became_peered":"2023-09-13T04:37:59.252306+0000","last_unstale":"2023-09-13T04:38:23.594567+0000","last_undegraded":"2023-09-13T04:38:23.594567+0000","last_fullsized":"2023-09-13T04:38:23.594567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:18:55.686847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.596920+0000","last_change":"2023-09-13T04:38:03.742502+0000","last_active":"2023-09-13T04:38:23.596920+0000","last_peered":"2023-09-13T04:38:23.596920+0000","last_clean":"2023-09-13T04:38:23.596920+0000","last_became_active":"2023-09-13T04:38:03.742054+0000","last_became_peered":"2023-09-13T04:38:03.742054+0000","last_unstale":"2023-09-13T04:38:23.596920+0000","last_undegraded":"2023-09-13T04:38:23.596920+0000","last_fullsized":"2023-09-13T04:38:23.596920+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:55:54.601145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593527+0000","last_change":"2023-09-13T04:37:59.245644+0000","last_active":"2023-09-13T04:38:23.593527+0000","last_peered":"2023-09-13T04:38:23.593527+0000","last_clean":"2023-09-13T04:38:23.593527+0000","last_became_active":"2023-09-13T04:37:59.245284+0000","last_became_peered":"2023-09-13T04:37:59.245284+0000","last_unstale":"2023-09-13T04:38:23.593527+0000","last_undegraded":"2023-09-13T04:38:23.593527+0000","last_fullsized":"2023-09-13T04:38:23.593527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:19:04.002414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.621561+0000","last_change":"2023-09-13T04:37:58.590249+0000","last_active":"2023-09-13T04:38:25.621561+0000","last_peered":"2023-09-13T04:38:25.621561+0000","last_clean":"2023-09-13T04:38:25.621561+0000","last_became_active":"2023-09-13T04:37:58.589521+0000","last_became_peered":"2023-09-13T04:37:58.589521+0000","last_unstale":"2023-09-13T04:38:25.621561+0000","last_undegraded":"2023-09-13T04:38:25.621561+0000","last_fullsized":"2023-09-13T04:38:25.621561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:11:57.542149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"peering","last_fresh":"2023-09-13T04:38:04.408074+0000","last_change":"2023-09-13T04:38:23.588693+0000","last_active":"2023-09-13T04:38:23.588693+0000","last_peered":"2023-09-13T04:38:04.408074+0000","last_clean":"2023-09-13T04:38:04.408074+0000","last_became_active":"2023-09-13T04:38:03.608060+0000","last_became_peered":"2023-09-13T04:38:03.608060+0000","last_unstale":"2023-09-13T04:38:04.408074+0000","last_undegraded":"2023-09-13T04:38:04.408074+0000","last_fullsized":"2023-09-13T04:38:04.408074+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,8,9,11],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.597408+0000","last_change":"2023-09-13T04:37:59.243794+0000","last_active":"2023-09-13T04:38:03.597408+0000","last_peered":"2023-09-13T04:38:03.597408+0000","last_clean":"2023-09-13T04:38:03.597408+0000","last_became_active":"2023-09-13T04:37:59.243368+0000","last_became_peered":"2023-09-13T04:37:59.243368+0000","last_unstale":"2023-09-13T04:38:03.597408+0000","last_undegraded":"2023-09-13T04:38:03.597408+0000","last_fullsized":"2023-09-13T04:38:03.597408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:06:45.318737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.245630+0000","last_change":"2023-09-13T04:37:58.589195+0000","last_active":"2023-09-13T04:38:24.245630+0000","last_peered":"2023-09-13T04:38:24.245630+0000","last_clean":"2023-09-13T04:38:24.245630+0000","last_became_active":"2023-09-13T04:37:58.588905+0000","last_became_peered":"2023-09-13T04:37:58.588905+0000","last_unstale":"2023-09-13T04:38:24.245630+0000","last_undegraded":"2023-09-13T04:38:24.245630+0000","last_fullsized":"2023-09-13T04:38:24.245630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:13:15.978891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":325,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.407034+0000","last_change":"2023-09-13T04:36:52.227264+0000","last_active":"2023-09-13T04:38:04.407034+0000","last_peered":"2023-09-13T04:38:04.407034+0000","last_clean":"2023-09-13T04:38:04.407034+0000","last_became_active":"2023-09-13T04:36:52.226773+0000","last_became_peered":"2023-09-13T04:36:52.226773+0000","last_unstale":"2023-09-13T04:38:04.407034+0000","last_undegraded":"2023-09-13T04:38:04.407034+0000","last_fullsized":"2023-09-13T04:38:04.407034+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:33:42.026499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:33:42.026499+0000","last_clean_scrub_stamp":"2023-09-13T04:33:42.026499+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:28:55.806587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592890+0000","last_change":"2023-09-13T04:38:03.742790+0000","last_active":"2023-09-13T04:38:23.592890+0000","last_peered":"2023-09-13T04:38:23.592890+0000","last_clean":"2023-09-13T04:38:23.592890+0000","last_became_active":"2023-09-13T04:38:03.742338+0000","last_became_peered":"2023-09-13T04:38:03.742338+0000","last_unstale":"2023-09-13T04:38:23.592890+0000","last_undegraded":"2023-09-13T04:38:23.592890+0000","last_fullsized":"2023-09-13T04:38:23.592890+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:01:38.910623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597412+0000","last_change":"2023-09-13T04:37:59.252669+0000","last_active":"2023-09-13T04:38:23.597412+0000","last_peered":"2023-09-13T04:38:23.597412+0000","last_clean":"2023-09-13T04:38:23.597412+0000","last_became_active":"2023-09-13T04:37:59.252223+0000","last_became_peered":"2023-09-13T04:37:59.252223+0000","last_unstale":"2023-09-13T04:38:23.597412+0000","last_undegraded":"2023-09-13T04:38:23.597412+0000","last_fullsized":"2023-09-13T04:38:23.597412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:57:21.090083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587916+0000","last_change":"2023-09-13T04:37:58.590540+0000","last_active":"2023-09-13T04:38:23.587916+0000","last_peered":"2023-09-13T04:38:23.587916+0000","last_clean":"2023-09-13T04:38:23.587916+0000","last_became_active":"2023-09-13T04:37:58.590061+0000","last_became_peered":"2023-09-13T04:37:58.590061+0000","last_unstale":"2023-09-13T04:38:23.587916+0000","last_undegraded":"2023-09-13T04:38:23.587916+0000","last_fullsized":"2023-09-13T04:38:23.587916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:57:17.133461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592719+0000","last_change":"2023-09-13T04:38:03.614615+0000","last_active":"2023-09-13T04:38:23.592719+0000","last_peered":"2023-09-13T04:38:23.592719+0000","last_clean":"2023-09-13T04:38:23.592719+0000","last_became_active":"2023-09-13T04:38:03.614086+0000","last_became_peered":"2023-09-13T04:38:03.614086+0000","last_unstale":"2023-09-13T04:38:23.592719+0000","last_undegraded":"2023-09-13T04:38:23.592719+0000","last_fullsized":"2023-09-13T04:38:23.592719+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:25:35.525513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.243719+0000","last_change":"2023-09-13T04:37:58.592096+0000","last_active":"2023-09-13T04:38:24.243719+0000","last_peered":"2023-09-13T04:38:24.243719+0000","last_clean":"2023-09-13T04:38:24.243719+0000","last_became_active":"2023-09-13T04:37:58.591872+0000","last_became_peered":"2023-09-13T04:37:58.591872+0000","last_unstale":"2023-09-13T04:38:24.243719+0000","last_undegraded":"2023-09-13T04:38:24.243719+0000","last_fullsized":"2023-09-13T04:38:24.243719+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:05:23.936105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597816+0000","last_change":"2023-09-13T04:37:59.240759+0000","last_active":"2023-09-13T04:38:23.597816+0000","last_peered":"2023-09-13T04:38:23.597816+0000","last_clean":"2023-09-13T04:38:23.597816+0000","last_became_active":"2023-09-13T04:37:59.240414+0000","last_became_peered":"2023-09-13T04:37:59.240414+0000","last_unstale":"2023-09-13T04:38:23.597816+0000","last_undegraded":"2023-09-13T04:38:23.597816+0000","last_fullsized":"2023-09-13T04:38:23.597816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:37:20.301938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.454201+0000","last_change":"2023-09-13T04:38:03.742519+0000","last_active":"2023-09-13T04:38:04.454201+0000","last_peered":"2023-09-13T04:38:04.454201+0000","last_clean":"2023-09-13T04:38:04.454201+0000","last_became_active":"2023-09-13T04:38:03.742075+0000","last_became_peered":"2023-09-13T04:38:03.742075+0000","last_unstale":"2023-09-13T04:38:04.454201+0000","last_undegraded":"2023-09-13T04:38:04.454201+0000","last_fullsized":"2023-09-13T04:38:04.454201+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:14:33.155837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.619899+0000","last_change":"2023-09-13T04:37:58.593891+0000","last_active":"2023-09-13T04:38:25.619899+0000","last_peered":"2023-09-13T04:38:25.619899+0000","last_clean":"2023-09-13T04:38:25.619899+0000","last_became_active":"2023-09-13T04:37:58.593497+0000","last_became_peered":"2023-09-13T04:37:58.593497+0000","last_unstale":"2023-09-13T04:38:25.619899+0000","last_undegraded":"2023-09-13T04:38:25.619899+0000","last_fullsized":"2023-09-13T04:38:25.619899+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:25:53.652601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.601755+0000","last_change":"2023-09-13T04:37:59.247151+0000","last_active":"2023-09-13T04:38:03.601755+0000","last_peered":"2023-09-13T04:38:03.601755+0000","last_clean":"2023-09-13T04:38:03.601755+0000","last_became_active":"2023-09-13T04:37:59.246853+0000","last_became_peered":"2023-09-13T04:37:59.246853+0000","last_unstale":"2023-09-13T04:38:03.601755+0000","last_undegraded":"2023-09-13T04:38:03.601755+0000","last_fullsized":"2023-09-13T04:38:03.601755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:15:22.659365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587493+0000","last_change":"2023-09-13T04:38:03.606594+0000","last_active":"2023-09-13T04:38:23.587493+0000","last_peered":"2023-09-13T04:38:23.587493+0000","last_clean":"2023-09-13T04:38:23.587493+0000","last_became_active":"2023-09-13T04:38:03.606058+0000","last_became_peered":"2023-09-13T04:38:03.606058+0000","last_unstale":"2023-09-13T04:38:23.587493+0000","last_undegraded":"2023-09-13T04:38:23.587493+0000","last_fullsized":"2023-09-13T04:38:23.587493+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:57:39.237520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.246057+0000","last_change":"2023-09-13T04:37:58.588362+0000","last_active":"2023-09-13T04:38:24.246057+0000","last_peered":"2023-09-13T04:38:24.246057+0000","last_clean":"2023-09-13T04:38:24.246057+0000","last_became_active":"2023-09-13T04:37:58.588065+0000","last_became_peered":"2023-09-13T04:37:58.588065+0000","last_unstale":"2023-09-13T04:38:24.246057+0000","last_undegraded":"2023-09-13T04:38:24.246057+0000","last_fullsized":"2023-09-13T04:38:24.246057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:32:04.255654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406766+0000","last_change":"2023-09-13T04:37:59.251883+0000","last_active":"2023-09-13T04:38:04.406766+0000","last_peered":"2023-09-13T04:38:04.406766+0000","last_clean":"2023-09-13T04:38:04.406766+0000","last_became_active":"2023-09-13T04:37:59.251522+0000","last_became_peered":"2023-09-13T04:37:59.251522+0000","last_unstale":"2023-09-13T04:38:04.406766+0000","last_undegraded":"2023-09-13T04:38:04.406766+0000","last_fullsized":"2023-09-13T04:38:04.406766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:05:06.420799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590729+0000","last_change":"2023-09-13T04:38:03.742726+0000","last_active":"2023-09-13T04:38:23.590729+0000","last_peered":"2023-09-13T04:38:23.590729+0000","last_clean":"2023-09-13T04:38:23.590729+0000","last_became_active":"2023-09-13T04:38:03.742293+0000","last_became_peered":"2023-09-13T04:38:03.742293+0000","last_unstale":"2023-09-13T04:38:23.590729+0000","last_undegraded":"2023-09-13T04:38:23.590729+0000","last_fullsized":"2023-09-13T04:38:23.590729+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:52:15.089764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.243314+0000","last_change":"2023-09-13T04:37:58.584923+0000","last_active":"2023-09-13T04:38:24.243314+0000","last_peered":"2023-09-13T04:38:24.243314+0000","last_clean":"2023-09-13T04:38:24.243314+0000","last_became_active":"2023-09-13T04:37:58.584320+0000","last_became_peered":"2023-09-13T04:37:58.584320+0000","last_unstale":"2023-09-13T04:38:24.243314+0000","last_undegraded":"2023-09-13T04:38:24.243314+0000","last_fullsized":"2023-09-13T04:38:24.243314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:16:53.670534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592586+0000","last_change":"2023-09-13T04:37:59.243029+0000","last_active":"2023-09-13T04:38:23.592586+0000","last_peered":"2023-09-13T04:38:23.592586+0000","last_clean":"2023-09-13T04:38:23.592586+0000","last_became_active":"2023-09-13T04:37:59.242733+0000","last_became_peered":"2023-09-13T04:37:59.242733+0000","last_unstale":"2023-09-13T04:38:23.592586+0000","last_undegraded":"2023-09-13T04:38:23.592586+0000","last_fullsized":"2023-09-13T04:38:23.592586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:48:21.430026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588037+0000","last_change":"2023-09-13T04:38:03.616735+0000","last_active":"2023-09-13T04:38:23.588037+0000","last_peered":"2023-09-13T04:38:23.588037+0000","last_clean":"2023-09-13T04:38:23.588037+0000","last_became_active":"2023-09-13T04:38:03.616275+0000","last_became_peered":"2023-09-13T04:38:03.616275+0000","last_unstale":"2023-09-13T04:38:23.588037+0000","last_undegraded":"2023-09-13T04:38:23.588037+0000","last_fullsized":"2023-09-13T04:38:23.588037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:12:27.267918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.598266+0000","last_change":"2023-09-13T04:37:58.595040+0000","last_active":"2023-09-13T04:38:03.598266+0000","last_peered":"2023-09-13T04:38:03.598266+0000","last_clean":"2023-09-13T04:38:03.598266+0000","last_became_active":"2023-09-13T04:37:58.591403+0000","last_became_peered":"2023-09-13T04:37:58.591403+0000","last_unstale":"2023-09-13T04:38:03.598266+0000","last_undegraded":"2023-09-13T04:38:03.598266+0000","last_fullsized":"2023-09-13T04:38:03.598266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:42:06.561066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592175+0000","last_change":"2023-09-13T04:37:59.248837+0000","last_active":"2023-09-13T04:38:23.592175+0000","last_peered":"2023-09-13T04:38:23.592175+0000","last_clean":"2023-09-13T04:38:23.592175+0000","last_became_active":"2023-09-13T04:37:59.248573+0000","last_became_peered":"2023-09-13T04:37:59.248573+0000","last_unstale":"2023-09-13T04:38:23.592175+0000","last_undegraded":"2023-09-13T04:38:23.592175+0000","last_fullsized":"2023-09-13T04:38:23.592175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:56:22.980484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590405+0000","last_change":"2023-09-13T04:38:03.616370+0000","last_active":"2023-09-13T04:38:23.590405+0000","last_peered":"2023-09-13T04:38:23.590405+0000","last_clean":"2023-09-13T04:38:23.590405+0000","last_became_active":"2023-09-13T04:38:03.615996+0000","last_became_peered":"2023-09-13T04:38:03.615996+0000","last_unstale":"2023-09-13T04:38:23.590405+0000","last_undegraded":"2023-09-13T04:38:23.590405+0000","last_fullsized":"2023-09-13T04:38:23.590405+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:56:59.924205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.590955+0000","last_change":"2023-09-13T04:37:58.597902+0000","last_active":"2023-09-13T04:38:24.590955+0000","last_peered":"2023-09-13T04:38:24.590955+0000","last_clean":"2023-09-13T04:38:24.590955+0000","last_became_active":"2023-09-13T04:37:58.597578+0000","last_became_peered":"2023-09-13T04:37:58.597578+0000","last_unstale":"2023-09-13T04:38:24.590955+0000","last_undegraded":"2023-09-13T04:38:24.590955+0000","last_fullsized":"2023-09-13T04:38:24.590955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:10:09.931949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.604302+0000","last_change":"2023-09-13T04:37:59.248924+0000","last_active":"2023-09-13T04:38:03.604302+0000","last_peered":"2023-09-13T04:38:03.604302+0000","last_clean":"2023-09-13T04:38:03.604302+0000","last_became_active":"2023-09-13T04:37:59.248483+0000","last_became_peered":"2023-09-13T04:37:59.248483+0000","last_unstale":"2023-09-13T04:38:03.604302+0000","last_undegraded":"2023-09-13T04:38:03.604302+0000","last_fullsized":"2023-09-13T04:38:03.604302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:49:56.078544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590322+0000","last_change":"2023-09-13T04:38:03.742260+0000","last_active":"2023-09-13T04:38:23.590322+0000","last_peered":"2023-09-13T04:38:23.590322+0000","last_clean":"2023-09-13T04:38:23.590322+0000","last_became_active":"2023-09-13T04:38:03.741893+0000","last_became_peered":"2023-09-13T04:38:03.741893+0000","last_unstale":"2023-09-13T04:38:23.590322+0000","last_undegraded":"2023-09-13T04:38:23.590322+0000","last_fullsized":"2023-09-13T04:38:23.590322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:59:00.653545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.245764+0000","last_change":"2023-09-13T04:37:58.586898+0000","last_active":"2023-09-13T04:38:24.245764+0000","last_peered":"2023-09-13T04:38:24.245764+0000","last_clean":"2023-09-13T04:38:24.245764+0000","last_became_active":"2023-09-13T04:37:58.586594+0000","last_became_peered":"2023-09-13T04:37:58.586594+0000","last_unstale":"2023-09-13T04:38:24.245764+0000","last_undegraded":"2023-09-13T04:38:24.245764+0000","last_fullsized":"2023-09-13T04:38:24.245764+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:29:43.718522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.407855+0000","last_change":"2023-09-13T04:37:59.244749+0000","last_active":"2023-09-13T04:38:04.407855+0000","last_peered":"2023-09-13T04:38:04.407855+0000","last_clean":"2023-09-13T04:38:04.407855+0000","last_became_active":"2023-09-13T04:37:59.244391+0000","last_became_peered":"2023-09-13T04:37:59.244391+0000","last_unstale":"2023-09-13T04:38:04.407855+0000","last_undegraded":"2023-09-13T04:38:04.407855+0000","last_fullsized":"2023-09-13T04:38:04.407855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:33:02.195729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592524+0000","last_change":"2023-09-13T04:38:03.609766+0000","last_active":"2023-09-13T04:38:23.592524+0000","last_peered":"2023-09-13T04:38:23.592524+0000","last_clean":"2023-09-13T04:38:23.592524+0000","last_became_active":"2023-09-13T04:38:03.609432+0000","last_became_peered":"2023-09-13T04:38:03.609432+0000","last_unstale":"2023-09-13T04:38:23.592524+0000","last_undegraded":"2023-09-13T04:38:23.592524+0000","last_fullsized":"2023-09-13T04:38:23.592524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:30:56.115483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.620943+0000","last_change":"2023-09-13T04:37:58.589119+0000","last_active":"2023-09-13T04:38:25.620943+0000","last_peered":"2023-09-13T04:38:25.620943+0000","last_clean":"2023-09-13T04:38:25.620943+0000","last_became_active":"2023-09-13T04:37:58.588803+0000","last_became_peered":"2023-09-13T04:37:58.588803+0000","last_unstale":"2023-09-13T04:38:25.620943+0000","last_undegraded":"2023-09-13T04:38:25.620943+0000","last_fullsized":"2023-09-13T04:38:25.620943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:51:20.888666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.408147+0000","last_change":"2023-09-13T04:37:59.243506+0000","last_active":"2023-09-13T04:38:04.408147+0000","last_peered":"2023-09-13T04:38:04.408147+0000","last_clean":"2023-09-13T04:38:04.408147+0000","last_became_active":"2023-09-13T04:37:59.243082+0000","last_became_peered":"2023-09-13T04:37:59.243082+0000","last_unstale":"2023-09-13T04:38:04.408147+0000","last_undegraded":"2023-09-13T04:38:04.408147+0000","last_fullsized":"2023-09-13T04:38:04.408147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:01:41.179286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406373+0000","last_change":"2023-09-13T04:38:03.609042+0000","last_active":"2023-09-13T04:38:04.406373+0000","last_peered":"2023-09-13T04:38:04.406373+0000","last_clean":"2023-09-13T04:38:04.406373+0000","last_became_active":"2023-09-13T04:38:03.608553+0000","last_became_peered":"2023-09-13T04:38:03.608553+0000","last_unstale":"2023-09-13T04:38:04.406373+0000","last_undegraded":"2023-09-13T04:38:04.406373+0000","last_fullsized":"2023-09-13T04:38:04.406373+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:05:56.648475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591325+0000","last_change":"2023-09-13T04:37:58.598056+0000","last_active":"2023-09-13T04:38:23.591325+0000","last_peered":"2023-09-13T04:38:23.591325+0000","last_clean":"2023-09-13T04:38:23.591325+0000","last_became_active":"2023-09-13T04:37:58.597724+0000","last_became_peered":"2023-09-13T04:37:58.597724+0000","last_unstale":"2023-09-13T04:38:23.591325+0000","last_undegraded":"2023-09-13T04:38:23.591325+0000","last_fullsized":"2023-09-13T04:38:23.591325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:43:08.296866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.407964+0000","last_change":"2023-09-13T04:37:59.246505+0000","last_active":"2023-09-13T04:38:04.407964+0000","last_peered":"2023-09-13T04:38:04.407964+0000","last_clean":"2023-09-13T04:38:04.407964+0000","last_became_active":"2023-09-13T04:37:59.246024+0000","last_became_peered":"2023-09-13T04:37:59.246024+0000","last_unstale":"2023-09-13T04:38:04.407964+0000","last_undegraded":"2023-09-13T04:38:04.407964+0000","last_fullsized":"2023-09-13T04:38:04.407964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:36:58.008066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590871+0000","last_change":"2023-09-13T04:38:03.605354+0000","last_active":"2023-09-13T04:38:23.590871+0000","last_peered":"2023-09-13T04:38:23.590871+0000","last_clean":"2023-09-13T04:38:23.590871+0000","last_became_active":"2023-09-13T04:38:03.604840+0000","last_became_peered":"2023-09-13T04:38:03.604840+0000","last_unstale":"2023-09-13T04:38:23.590871+0000","last_undegraded":"2023-09-13T04:38:23.590871+0000","last_fullsized":"2023-09-13T04:38:23.590871+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:46:29.867914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597380+0000","last_change":"2023-09-13T04:37:58.589052+0000","last_active":"2023-09-13T04:38:23.597380+0000","last_peered":"2023-09-13T04:38:23.597380+0000","last_clean":"2023-09-13T04:38:23.597380+0000","last_became_active":"2023-09-13T04:37:58.588707+0000","last_became_peered":"2023-09-13T04:37:58.588707+0000","last_unstale":"2023-09-13T04:38:23.597380+0000","last_undegraded":"2023-09-13T04:38:23.597380+0000","last_fullsized":"2023-09-13T04:38:23.597380+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:25:23.966046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.408021+0000","last_change":"2023-09-13T04:37:59.241157+0000","last_active":"2023-09-13T04:38:04.408021+0000","last_peered":"2023-09-13T04:38:04.408021+0000","last_clean":"2023-09-13T04:38:04.408021+0000","last_became_active":"2023-09-13T04:37:59.240857+0000","last_became_peered":"2023-09-13T04:37:59.240857+0000","last_unstale":"2023-09-13T04:38:04.408021+0000","last_undegraded":"2023-09-13T04:38:04.408021+0000","last_fullsized":"2023-09-13T04:38:04.408021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:04:44.986908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592376+0000","last_change":"2023-09-13T04:37:59.242447+0000","last_active":"2023-09-13T04:38:23.592376+0000","last_peered":"2023-09-13T04:38:23.592376+0000","last_clean":"2023-09-13T04:38:23.592376+0000","last_became_active":"2023-09-13T04:37:59.242130+0000","last_became_peered":"2023-09-13T04:37:59.242130+0000","last_unstale":"2023-09-13T04:38:23.592376+0000","last_undegraded":"2023-09-13T04:38:23.592376+0000","last_fullsized":"2023-09-13T04:38:23.592376+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:12:12.844888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.595646+0000","last_change":"2023-09-13T04:38:24.595646+0000","last_active":"2023-09-13T04:38:24.595646+0000","last_peered":"2023-09-13T04:38:24.595646+0000","last_clean":"2023-09-13T04:38:24.595646+0000","last_became_active":"2023-09-13T04:38:24.595295+0000","last_became_peered":"2023-09-13T04:38:24.595295+0000","last_unstale":"2023-09-13T04:38:24.595646+0000","last_undegraded":"2023-09-13T04:38:24.595646+0000","last_fullsized":"2023-09-13T04:38:24.595646+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:24:27.174979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587574+0000","last_change":"2023-09-13T04:37:59.245358+0000","last_active":"2023-09-13T04:38:23.587574+0000","last_peered":"2023-09-13T04:38:23.587574+0000","last_clean":"2023-09-13T04:38:23.587574+0000","last_became_active":"2023-09-13T04:37:59.245052+0000","last_became_peered":"2023-09-13T04:37:59.245052+0000","last_unstale":"2023-09-13T04:38:23.587574+0000","last_undegraded":"2023-09-13T04:38:23.587574+0000","last_fullsized":"2023-09-13T04:38:23.587574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:05:42.561716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406504+0000","last_change":"2023-09-13T04:37:59.254053+0000","last_active":"2023-09-13T04:38:04.406504+0000","last_peered":"2023-09-13T04:38:04.406504+0000","last_clean":"2023-09-13T04:38:04.406504+0000","last_became_active":"2023-09-13T04:37:59.253544+0000","last_became_peered":"2023-09-13T04:37:59.253544+0000","last_unstale":"2023-09-13T04:38:04.406504+0000","last_undegraded":"2023-09-13T04:38:04.406504+0000","last_fullsized":"2023-09-13T04:38:04.406504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:18:54.689208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597920+0000","last_change":"2023-09-13T04:37:59.253403+0000","last_active":"2023-09-13T04:38:23.597920+0000","last_peered":"2023-09-13T04:38:23.597920+0000","last_clean":"2023-09-13T04:38:23.597920+0000","last_became_active":"2023-09-13T04:37:59.253062+0000","last_became_peered":"2023-09-13T04:37:59.253062+0000","last_unstale":"2023-09-13T04:38:23.597920+0000","last_undegraded":"2023-09-13T04:38:23.597920+0000","last_fullsized":"2023-09-13T04:38:23.597920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:31:09.898886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592570+0000","last_change":"2023-09-13T04:37:59.240916+0000","last_active":"2023-09-13T04:38:23.592570+0000","last_peered":"2023-09-13T04:38:23.592570+0000","last_clean":"2023-09-13T04:38:23.592570+0000","last_became_active":"2023-09-13T04:37:59.240605+0000","last_became_peered":"2023-09-13T04:37:59.240605+0000","last_unstale":"2023-09-13T04:38:23.592570+0000","last_undegraded":"2023-09-13T04:38:23.592570+0000","last_fullsized":"2023-09-13T04:38:23.592570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:15:48.241587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588778+0000","last_change":"2023-09-13T04:37:59.238569+0000","last_active":"2023-09-13T04:38:23.588778+0000","last_peered":"2023-09-13T04:38:23.588778+0000","last_clean":"2023-09-13T04:38:23.588778+0000","last_became_active":"2023-09-13T04:37:59.238255+0000","last_became_peered":"2023-09-13T04:37:59.238255+0000","last_unstale":"2023-09-13T04:38:23.588778+0000","last_undegraded":"2023-09-13T04:38:23.588778+0000","last_fullsized":"2023-09-13T04:38:23.588778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:19:03.433404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.601850+0000","last_change":"2023-09-13T04:37:59.252243+0000","last_active":"2023-09-13T04:38:03.601850+0000","last_peered":"2023-09-13T04:38:03.601850+0000","last_clean":"2023-09-13T04:38:03.601850+0000","last_became_active":"2023-09-13T04:37:59.251902+0000","last_became_peered":"2023-09-13T04:37:59.251902+0000","last_unstale":"2023-09-13T04:38:03.601850+0000","last_undegraded":"2023-09-13T04:38:03.601850+0000","last_fullsized":"2023-09-13T04:38:03.601850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:35:57.395627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593414+0000","last_change":"2023-09-13T04:37:59.245847+0000","last_active":"2023-09-13T04:38:23.593414+0000","last_peered":"2023-09-13T04:38:23.593414+0000","last_clean":"2023-09-13T04:38:23.593414+0000","last_became_active":"2023-09-13T04:37:59.245542+0000","last_became_peered":"2023-09-13T04:37:59.245542+0000","last_unstale":"2023-09-13T04:38:23.593414+0000","last_undegraded":"2023-09-13T04:38:23.593414+0000","last_fullsized":"2023-09-13T04:38:23.593414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:24:16.813333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.598419+0000","last_change":"2023-09-13T04:37:59.246963+0000","last_active":"2023-09-13T04:38:03.598419+0000","last_peered":"2023-09-13T04:38:03.598419+0000","last_clean":"2023-09-13T04:38:03.598419+0000","last_became_active":"2023-09-13T04:37:59.246448+0000","last_became_peered":"2023-09-13T04:37:59.246448+0000","last_unstale":"2023-09-13T04:38:03.598419+0000","last_undegraded":"2023-09-13T04:38:03.598419+0000","last_fullsized":"2023-09-13T04:38:03.598419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:25:45.418778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594086+0000","last_change":"2023-09-13T04:37:59.245399+0000","last_active":"2023-09-13T04:38:23.594086+0000","last_peered":"2023-09-13T04:38:23.594086+0000","last_clean":"2023-09-13T04:38:23.594086+0000","last_became_active":"2023-09-13T04:37:59.245060+0000","last_became_peered":"2023-09-13T04:37:59.245060+0000","last_unstale":"2023-09-13T04:38:23.594086+0000","last_undegraded":"2023-09-13T04:38:23.594086+0000","last_fullsized":"2023-09-13T04:38:23.594086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:25:41.078238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597807+0000","last_change":"2023-09-13T04:37:59.244776+0000","last_active":"2023-09-13T04:38:23.597807+0000","last_peered":"2023-09-13T04:38:23.597807+0000","last_clean":"2023-09-13T04:38:23.597807+0000","last_became_active":"2023-09-13T04:37:59.244409+0000","last_became_peered":"2023-09-13T04:37:59.244409+0000","last_unstale":"2023-09-13T04:38:23.597807+0000","last_undegraded":"2023-09-13T04:38:23.597807+0000","last_fullsized":"2023-09-13T04:38:23.597807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:04:02.561414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592391+0000","last_change":"2023-09-13T04:37:59.248153+0000","last_active":"2023-09-13T04:38:23.592391+0000","last_peered":"2023-09-13T04:38:23.592391+0000","last_clean":"2023-09-13T04:38:23.592391+0000","last_became_active":"2023-09-13T04:37:59.247840+0000","last_became_peered":"2023-09-13T04:37:59.247840+0000","last_unstale":"2023-09-13T04:38:23.592391+0000","last_undegraded":"2023-09-13T04:38:23.592391+0000","last_fullsized":"2023-09-13T04:38:23.592391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:52:53.021219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6628,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":105,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":18262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90,"ondisk_log_size":90,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":490076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.52100000000000002}]},{"osd":1,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.495},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.63700000000000001}]},{"osd":2,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.59599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":4,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.79000000000000004}]},{"osd":5,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.77500000000000002}]},{"osd":6,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.75800000000000001}]},{"osd":7,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.86399999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":9,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.878}]},{"osd":10,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.92400000000000004}]}]},{"osd":10,"up_from":71,"seq":304942678037,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1626112,"data_stored":1084927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":1,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.42299999999999999}]},{"osd":2,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.57199999999999995}]},{"osd":3,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.626},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.754}]},{"osd":4,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.44600000000000001}]},{"osd":5,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.69899999999999995}]},{"osd":6,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":3.4129999999999998,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.84499999999999997}]},{"osd":7,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.64500000000000002}]},{"osd":8,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.59399999999999997}]},{"osd":9,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.80500000000000005}]},{"osd":11,"last update":"Wed Sep 13 04:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.67800000000000005}]}]},{"osd":9,"up_from":66,"seq":283467841562,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":490860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.36299999999999999}]},{"osd":1,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.56499999999999995}]},{"osd":2,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.73499999999999999}]},{"osd":3,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.45200000000000001}]},{"osd":4,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.63},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.59199999999999997}]},{"osd":5,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.75}]},{"osd":6,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.72099999999999997}]},{"osd":7,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.60899999999999999}]},{"osd":8,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.64800000000000002}]},{"osd":10,"last update":"Wed Sep 13 04:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.68300000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]}]},{"osd":8,"up_from":58,"seq":249108103198,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":490337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.33200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56999999999999995,"15min":0.56499999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.68100000000000005}]},{"osd":1,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.54800000000000004,"15min":0.53600000000000003},"min":{"1min":0.38400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.58199999999999996},"min":{"1min":0.374,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.78600000000000003}]},{"osd":2,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.47499999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.48899999999999999,"15min":0.48599999999999999},"min":{"1min":0.29799999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.48399999999999999}]},{"osd":3,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.434},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64600000000000002,"15min":0.64300000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.61799999999999999}]},{"osd":4,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58899999999999997,"15min":0.58399999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.41099999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.73699999999999999}]},{"osd":5,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.34499999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80700000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.54400000000000004}]},{"osd":6,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.88200000000000001}]},{"osd":7,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71199999999999997,"15min":0.70999999999999996},"min":{"1min":0.48399999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69099999999999995,"15min":0.68700000000000006},"min":{"1min":0.503,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.81200000000000006}]},{"osd":9,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.85399999999999998}]},{"osd":10,"last update":"Wed Sep 13 04:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.82499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]}]},{"osd":7,"up_from":51,"seq":219043332131,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1564672,"data_stored":1061916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.46500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.746,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60499999999999998,"15min":0.60799999999999998},"min":{"1min":0.41599999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.40200000000000002}]},{"osd":1,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.40899999999999997,"15min":0.40300000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42099999999999999,"15min":0.41999999999999998},"min":{"1min":0.30199999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65800000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.34200000000000003}]},{"osd":2,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.42599999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83899999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.57799999999999996,"15min":0.57099999999999995},"min":{"1min":0.443,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.70599999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.53700000000000003}]},{"osd":3,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.5},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66600000000000004,"15min":0.66900000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82799999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.52100000000000002}]},{"osd":4,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76600000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51200000000000001,"15min":0.51500000000000001},"min":{"1min":0.34499999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.376}]},{"osd":5,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.628,"15min":0.626},"min":{"1min":0.47799999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64600000000000002,"15min":0.64200000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.60299999999999998}]},{"osd":6,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72199999999999998,"15min":0.72099999999999997},"min":{"1min":0.50600000000000001,"5min":0.496,"15min":0.496},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.62},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.48799999999999999}]},{"osd":8,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70299999999999996,"15min":0.70399999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.51700000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.88100000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.68200000000000005}]},{"osd":9,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.72299999999999998}]},{"osd":10,"last update":"Wed Sep 13 04:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.434},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.56499999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":490132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46100000000000002,"15min":0.45100000000000001},"min":{"1min":0.34699999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.503,"15min":0.48999999999999999},"min":{"1min":0.38100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.42999999999999999}]},{"osd":1,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.47099999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82499999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.70999999999999996,"15min":0.67900000000000005},"min":{"1min":0.60399999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.94699999999999995}]},{"osd":2,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.52400000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0129999999999999,"5min":1.054,"15min":1.054},"last":0.83199999999999996}]},{"osd":3,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.66100000000000003,"15min":0.64100000000000001},"min":{"1min":0.47799999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83599999999999997,"5min":1.113,"15min":1.113},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.626,"15min":0.61799999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89900000000000002,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.496}]},{"osd":4,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76400000000000001,"15min":0.747},"min":{"1min":0.67900000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0509999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.75900000000000001,"15min":0.77100000000000002},"min":{"1min":0.52600000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92900000000000005,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.61599999999999999}]},{"osd":5,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.80800000000000005,"15min":0.79200000000000004},"min":{"1min":0.56100000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1950000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.79200000000000004,"15min":0.80800000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.80200000000000005}]},{"osd":7,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.83999999999999997,"15min":0.82399999999999995},"min":{"1min":0.63500000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.79800000000000004,"15min":0.78800000000000003},"min":{"1min":0.58999999999999997,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.57899999999999996}]},{"osd":8,"last update":"Wed Sep 13 04:38:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.81599999999999995,"15min":0.82399999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96599999999999997,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.91900000000000004,"15min":0.92400000000000004},"min":{"1min":0.61199999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2629999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.032}]},{"osd":9,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.0089999999999999}]},{"osd":10,"last update":"Wed Sep 13 04:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.754}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":493884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55600000000000005,"15min":0.53700000000000003},"min":{"1min":0.33700000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56999999999999995,"15min":0.55200000000000005},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.85199999999999998}]},{"osd":2,"last update":"Wed Sep 13 04:37:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.55900000000000005,"15min":0.53800000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.54800000000000004,"15min":0.51800000000000002},"min":{"1min":0.42299999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.73199999999999998}]},{"osd":3,"last update":"Wed Sep 13 04:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83099999999999996,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.623,"15min":0.60899999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91600000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67800000000000005}]},{"osd":4,"last update":"Wed Sep 13 04:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.46200000000000002,"15min":0.45300000000000001},"min":{"1min":0.29599999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.45800000000000002,"15min":0.44800000000000001},"min":{"1min":0.29199999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.497}]},{"osd":5,"last update":"Wed Sep 13 04:37:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.44800000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89000000000000001,"5min":1.3,"15min":1.3},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61499999999999999,"15min":0.58299999999999996},"min":{"1min":0.435,"5min":0.37,"15min":0.37},"max":{"1min":1,"5min":1.264,"15min":1.264},"last":0.78800000000000003}]},{"osd":6,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.99199999999999999,"5min":1.482,"15min":1.482},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95399999999999996,"5min":1.375,"15min":1.375},"last":0.71699999999999997}]},{"osd":7,"last update":"Wed Sep 13 04:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.60699999999999998,"15min":0.58699999999999997},"min":{"1min":0.39900000000000002,"5min":0.372,"15min":0.372},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.65900000000000003,"15min":0.64300000000000002},"min":{"1min":0.47399999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.75800000000000001}]},{"osd":8,"last update":"Wed Sep 13 04:38:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.48899999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2010000000000001,"5min":1.393,"15min":1.393},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.093,"5min":1.224,"15min":1.224},"last":0.77100000000000002}]},{"osd":9,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.81100000000000005}]},{"osd":10,"last update":"Wed Sep 13 04:37:53 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.878}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":480566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 13 04:37:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":0.66000000000000003,"15min":0.58299999999999996},"min":{"1min":0.42199999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.2359999999999998,"5min":6.2359999999999998,"15min":6.2359999999999998},"last":0.745},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.58699999999999997,"15min":0.55200000000000005},"min":{"1min":0.47899999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.77300000000000002}]},{"osd":2,"last update":"Wed Sep 13 04:37:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.624,"15min":0.59899999999999998},"min":{"1min":0.42899999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93899999999999995,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63700000000000001,"15min":0.60299999999999998},"min":{"1min":0.439,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.96899999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.82399999999999995}]},{"osd":3,"last update":"Wed Sep 13 04:36:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.57099999999999995,"15min":0.55500000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.86499999999999999}]},{"osd":4,"last update":"Wed Sep 13 04:37:30 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.65900000000000003,"15min":0.61299999999999999},"min":{"1min":0.58699999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":1.079},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.64900000000000002,"15min":0.621},"min":{"1min":0.51400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.84599999999999997}]},{"osd":5,"last update":"Wed Sep 13 04:38:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68300000000000005,"15min":0.67000000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.70699999999999996,"15min":0.68300000000000005},"min":{"1min":0.55100000000000005,"5min":0.435,"15min":0.435},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.91900000000000004}]},{"osd":6,"last update":"Wed Sep 13 04:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.44600000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94899999999999995,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88700000000000001,"5min":1.165,"15min":1.165},"last":0.79800000000000004}]},{"osd":7,"last update":"Wed Sep 13 04:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.79500000000000004,"15min":0.78500000000000003},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.81599999999999995,"15min":0.80200000000000005},"min":{"1min":0.60099999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":1.26}]},{"osd":8,"last update":"Wed Sep 13 04:38:00 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":0.90600000000000003,"15min":0.85999999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":6.2089999999999996,"5min":6.2089999999999996,"15min":6.2089999999999996},"last":0.874},{"interface":"front","average":{"1min":1.171,"5min":0.87,"15min":0.81999999999999995},"min":{"1min":0.52400000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":6.1699999999999999,"5min":6.1699999999999999,"15min":6.1699999999999999},"last":1.143}]},{"osd":9,"last update":"Wed Sep 13 04:37:30 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":1.171}]},{"osd":10,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":6.1859999999999999,"5min":6.1859999999999999,"15min":6.1859999999999999},"last":0.996}]},{"osd":11,"last update":"Wed Sep 13 04:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.2330000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":1.044}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":495200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:55 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.34599999999999997,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79100000000000004,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51900000000000002,"15min":0.51800000000000002},"min":{"1min":0.371,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78300000000000003,"5min":1.429,"15min":1.429},"last":0.45100000000000001}]},{"osd":1,"last update":"Wed Sep 13 04:37:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.53500000000000003,"15min":0.496},"min":{"1min":0.39000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80700000000000005,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54300000000000004,"15min":0.50700000000000001},"min":{"1min":0.39100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70799999999999996,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.755}]},{"osd":3,"last update":"Wed Sep 13 04:38:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.36499999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.747,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.441,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81399999999999995,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.66600000000000004}]},{"osd":4,"last update":"Wed Sep 13 04:37:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.52700000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78000000000000003,"5min":1.244,"15min":1.244},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55800000000000005,"15min":0.53000000000000003},"min":{"1min":0.436,"5min":0.374,"15min":0.374},"max":{"1min":0.85499999999999998,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.79400000000000004}]},{"osd":5,"last update":"Wed Sep 13 04:37:57 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.28699999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.70899999999999996,"5min":1.796,"15min":1.796},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.30599999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.99299999999999999,"5min":1.546,"15min":1.546},"last":0.49399999999999999}]},{"osd":6,"last update":"Wed Sep 13 04:38:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.69199999999999995,"15min":0.69899999999999995},"min":{"1min":0.48499999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81599999999999995,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66700000000000004,"15min":0.67200000000000004},"min":{"1min":0.48199999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85899999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.77100000000000002}]},{"osd":7,"last update":"Wed Sep 13 04:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65400000000000003,"15min":0.65800000000000003},"min":{"1min":0.42499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94699999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.497,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92500000000000004,"5min":1.147,"15min":1.147},"last":0.88300000000000001}]},{"osd":8,"last update":"Wed Sep 13 04:38:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61699999999999999,"15min":0.61899999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.96599999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.38400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.82699999999999996}]},{"osd":9,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.84199999999999997}]},{"osd":10,"last update":"Wed Sep 13 04:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.98199999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":974848,"data_stored":471070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.42699999999999999,"15min":0.39000000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.46500000000000002,"15min":0.435},"min":{"1min":0.375,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78500000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.38400000000000001}]},{"osd":1,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.57699999999999996},"min":{"1min":0.45700000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64200000000000002,"15min":0.57699999999999996},"min":{"1min":0.436,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92200000000000004,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.69799999999999995}]},{"osd":2,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.67400000000000004,"15min":0.629},"min":{"1min":0.53000000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64500000000000002,"15min":0.621},"min":{"1min":0.45800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.55100000000000005}]},{"osd":4,"last update":"Wed Sep 13 04:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.64300000000000002,"15min":0.60099999999999998},"min":{"1min":0.52900000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.64100000000000001,"15min":0.60699999999999998},"min":{"1min":0.44500000000000001,"5min":0.439,"15min":0.439},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.68799999999999994}]},{"osd":5,"last update":"Wed Sep 13 04:38:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.68999999999999995,"15min":0.65000000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.024,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.873,"5min":0.72399999999999998,"15min":0.66600000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":0.71199999999999997}]},{"osd":6,"last update":"Wed Sep 13 04:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.71599999999999997,"15min":0.69499999999999995},"min":{"1min":0.51200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.75900000000000001,"15min":0.74199999999999999},"min":{"1min":0.52400000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.622}]},{"osd":7,"last update":"Wed Sep 13 04:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.86299999999999999,"15min":0.86399999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.85399999999999998,"15min":0.84999999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.80500000000000005}]},{"osd":8,"last update":"Wed Sep 13 04:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.875}]},{"osd":9,"last update":"Wed Sep 13 04:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.63800000000000001}]},{"osd":10,"last update":"Wed Sep 13 04:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.77200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":490064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.56599999999999995,"15min":0.54400000000000004},"min":{"1min":0.43099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.753},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76500000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.68999999999999995}]},{"osd":1,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.373,"15min":0.34699999999999998},"min":{"1min":0.28399999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.38700000000000001,"15min":0.36399999999999999},"min":{"1min":0.28100000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.48499999999999999}]},{"osd":2,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58299999999999996,"15min":0.56100000000000005},"min":{"1min":0.41099999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56599999999999995,"15min":0.53800000000000003},"min":{"1min":0.41399999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.64300000000000002}]},{"osd":3,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.59399999999999997,"15min":0.56499999999999995},"min":{"1min":0.41799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59399999999999997,"15min":0.57699999999999996},"min":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.68400000000000005}]},{"osd":5,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60299999999999998,"15min":0.58599999999999997},"min":{"1min":0.45100000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90300000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.39700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.75800000000000001}]},{"osd":6,"last update":"Wed Sep 13 04:38:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.68000000000000005,"15min":0.68400000000000005},"min":{"1min":0.47099999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81599999999999995,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.69799999999999995,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.84599999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52400000000000002,"15min":0.51900000000000002},"min":{"1min":0.36199999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73599999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.70099999999999996}]},{"osd":8,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.70699999999999996,"15min":0.69199999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76700000000000002,"15min":0.76300000000000001},"min":{"1min":0.53000000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.77600000000000002}]},{"osd":9,"last update":"Wed Sep 13 04:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.86199999999999999}]},{"osd":10,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.82899999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1622016,"data_stored":1081886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53000000000000003,"15min":0.52400000000000002},"min":{"1min":0.34399999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72099999999999997,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.317,"5min":0.315,"15min":0.315},"max":{"1min":0.70299999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.84799999999999998}]},{"osd":1,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.40300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78500000000000003,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.54300000000000004,"15min":0.52400000000000002},"min":{"1min":0.34799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89600000000000002,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.60399999999999998}]},{"osd":2,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.38,"15min":0.36099999999999999},"min":{"1min":0.26900000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.37},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.39600000000000002,"15min":0.378},"min":{"1min":0.28299999999999997,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.41199999999999998}]},{"osd":3,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.372,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92600000000000005,"5min":1.036,"15min":1.036},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.78700000000000003,"15min":0.86899999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.1259999999999999,"5min":8.0009999999999994,"15min":8.0009999999999994},"last":0.77700000000000002}]},{"osd":4,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.58299999999999996,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82299999999999995,"5min":1.599,"15min":1.599},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.57699999999999996,"15min":0.55500000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.86599999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.623}]},{"osd":6,"last update":"Wed Sep 13 04:38:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83499999999999996,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81899999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.72999999999999998}]},{"osd":7,"last update":"Wed Sep 13 04:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.878,"5min":1.488,"15min":1.488},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94899999999999995,"5min":1.51,"15min":1.51},"last":0.63900000000000001}]},{"osd":8,"last update":"Wed Sep 13 04:38:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.38900000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.60599999999999998,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.56999999999999995}]},{"osd":9,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.70099999999999996}]},{"osd":10,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.68500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-13T04:38:28.562 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-13T04:38:28.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.798+0000 7ff3bb3ff700 1 -- 172.21.15.2:0/1731819254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3b4101ee0 msgr2=0x7ff3b4102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:28.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.798+0000 7ff3bb3ff700 1 --2- 172.21.15.2:0/1731819254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3b4101ee0 0x7ff3b4102340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40099c0 tx=0x7ff3a4009cd0 comp rx=0 tx=0).stop 2023-09-13T04:38:28.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.798+0000 7ff3bb3ff700 1 -- 172.21.15.2:0/1731819254 shutdown_connections 2023-09-13T04:38:28.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.798+0000 7ff3bb3ff700 1 --2- 172.21.15.2:0/1731819254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3b4102880 0x7ff3b4109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:28.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.798+0000 7ff3bb3ff700 1 --2- 172.21.15.2:0/1731819254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3b4100cf0 0x7ff3b4101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:28.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.798+0000 7ff3bb3ff700 1 --2- 172.21.15.2:0/1731819254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3b4101ee0 0x7ff3b4102340 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:28.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.798+0000 7ff3bb3ff700 1 -- 172.21.15.2:0/1731819254 >> 172.21.15.2:0/1731819254 conn(0x7ff3b40fc220 msgr2=0x7ff3b40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:28.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.798+0000 7ff3bb3ff700 1 -- 172.21.15.2:0/1731819254 shutdown_connections 2023-09-13T04:38:28.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.798+0000 7ff3bb3ff700 1 -- 172.21.15.2:0/1731819254 wait complete. 2023-09-13T04:38:28.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.799+0000 7ff3bb3ff700 1 Processor -- start 2023-09-13T04:38:28.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.799+0000 7ff3bb3ff700 1 -- start start 2023-09-13T04:38:28.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.799+0000 7ff3bb3ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3b4100cf0 0x7ff3b4198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:28.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.799+0000 7ff3bb3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3b4101ee0 0x7ff3b41990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:28.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.799+0000 7ff3bb3ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3b4102880 0x7ff3b41a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:28.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.799+0000 7ff3bb3ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b41a0730 con 0x7ff3b4102880 2023-09-13T04:38:28.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.799+0000 7ff3bb3ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b41a0870 con 0x7ff3b4100cf0 2023-09-13T04:38:28.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.799+0000 7ff3bb3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b41a0a00 con 0x7ff3b4101ee0 2023-09-13T04:38:28.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.800+0000 7ff3b999c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3b4102880 0x7ff3b41a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:28.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.800+0000 7ff3b899a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3b4101ee0 0x7ff3b41990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:28.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.800+0000 7ff3b919b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3b4100cf0 0x7ff3b4198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:28.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.800+0000 7ff3b899a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3b4101ee0 0x7ff3b41990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:55714/0 (socket says 172.21.15.2:55714) 2023-09-13T04:38:28.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.800+0000 7ff3b899a700 1 -- 172.21.15.2:0/1817155287 learned_addr learned my addr 172.21.15.2:0/1817155287 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:28.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.800+0000 7ff3b899a700 1 -- 172.21.15.2:0/1817155287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3b4100cf0 msgr2=0x7ff3b4198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:28.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.800+0000 7ff3b899a700 1 --2- 172.21.15.2:0/1817155287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3b4100cf0 0x7ff3b4198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:28.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.800+0000 7ff3b899a700 1 -- 172.21.15.2:0/1817155287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3b4102880 msgr2=0x7ff3b41a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:28.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.800+0000 7ff3b899a700 1 --2- 172.21.15.2:0/1817155287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3b4102880 0x7ff3b41a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:28.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.800+0000 7ff3b899a700 1 -- 172.21.15.2:0/1817155287 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a40095f0 con 0x7ff3b4101ee0 2023-09-13T04:38:28.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.800+0000 7ff3b999c700 1 --2- 172.21.15.2:0/1817155287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3b4102880 0x7ff3b41a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:38:28.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.801+0000 7ff3b899a700 1 --2- 172.21.15.2:0/1817155287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3b4101ee0 0x7ff3b41990f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009fd0 tx=0x7ff3a400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:28.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.801+0000 7ff3aa7fc700 1 -- 172.21.15.2:0/1817155287 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a401d070 con 0x7ff3b4101ee0 2023-09-13T04:38:28.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.801+0000 7ff3aa7fc700 1 -- 172.21.15.2:0/1817155287 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3a4005050 con 0x7ff3b4101ee0 2023-09-13T04:38:28.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.801+0000 7ff3bb3ff700 1 -- 172.21.15.2:0/1817155287 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b41a0c80 con 0x7ff3b4101ee0 2023-09-13T04:38:28.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.801+0000 7ff3aa7fc700 1 -- 172.21.15.2:0/1817155287 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4004b30 con 0x7ff3b4101ee0 2023-09-13T04:38:28.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.801+0000 7ff3bb3ff700 1 -- 172.21.15.2:0/1817155287 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b41a10a0 con 0x7ff3b4101ee0 2023-09-13T04:38:28.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.802+0000 7ff3bb3ff700 1 -- 172.21.15.2:0/1817155287 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b410c5c0 con 0x7ff3b4101ee0 2023-09-13T04:38:28.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.804+0000 7ff3aa7fc700 1 -- 172.21.15.2:0/1817155287 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7ff3a4004c90 con 0x7ff3b4101ee0 2023-09-13T04:38:28.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.805+0000 7ff3aa7fc700 1 --2- 172.21.15.2:0/1817155287 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff3a00a0690 0x7ff3a00a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:28.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.805+0000 7ff3aa7fc700 1 -- 172.21.15.2:0/1817155287 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff3a40cd190 con 0x7ff3b4101ee0 2023-09-13T04:38:28.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.805+0000 7ff3b919b700 1 --2- 172.21.15.2:0/1817155287 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff3a00a0690 0x7ff3a00a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:28.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.806+0000 7ff3b919b700 1 --2- 172.21.15.2:0/1817155287 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff3a00a0690 0x7ff3a00a2b50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4101d40 tx=0x7ff3b0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:28.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:28.807+0000 7ff3aa7fc700 1 -- 172.21.15.2:0/1817155287 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff3a40969e0 con 0x7ff3b4101ee0 2023-09-13T04:38:28.886 INFO:journalctl@ceph.mds.b.smithi149.stdout:Sep 13 04:38:28 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-b[169954]: 2023-09-13T04:38:28.709+0000 7f5780775700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-09-13T04:38:28.887 INFO:journalctl@ceph.mds.b.smithi149.stdout:Sep 13 04:38:28 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-b[169954]: 2023-09-13T04:38:28.709+0000 7f5780775700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-09-13T04:38:28.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:28 smithi149 ceph-mon[133315]: pgmap v292: 129 pgs: 2 peering, 127 active+clean; 583 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 2.1 KiB/s wr, 7 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-09-13T04:38:28.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:28 smithi156 ceph-mon[133675]: pgmap v292: 129 pgs: 2 peering, 127 active+clean; 583 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 2.1 KiB/s wr, 7 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-09-13T04:38:29.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:28 smithi002 ceph-mon[97651]: pgmap v292: 129 pgs: 2 peering, 127 active+clean; 583 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 2.1 KiB/s wr, 7 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-09-13T04:38:29.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.015+0000 7ff3bb3ff700 1 -- 172.21.15.2:0/1817155287 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff3b41087e0 con 0x7ff3a00a0690 2023-09-13T04:38:29.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.028+0000 7ff3aa7fc700 1 -- 172.21.15.2:0/1817155287 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422580 (secure 0 0 0) 0x7ff3b41087e0 con 0x7ff3a00a0690 2023-09-13T04:38:29.034 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:38:29.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.032+0000 7ff3bb3ff700 1 -- 172.21.15.2:0/1817155287 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff3a00a0690 msgr2=0x7ff3a00a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:29.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.032+0000 7ff3bb3ff700 1 --2- 172.21.15.2:0/1817155287 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff3a00a0690 0x7ff3a00a2b50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4101d40 tx=0x7ff3b0009260 comp rx=0 tx=0).stop 2023-09-13T04:38:29.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.032+0000 7ff3bb3ff700 1 -- 172.21.15.2:0/1817155287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3b4101ee0 msgr2=0x7ff3b41990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:29.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.032+0000 7ff3bb3ff700 1 --2- 172.21.15.2:0/1817155287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3b4101ee0 0x7ff3b41990f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009fd0 tx=0x7ff3a400f690 comp rx=0 tx=0).stop 2023-09-13T04:38:29.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.033+0000 7ff3bb3ff700 1 -- 172.21.15.2:0/1817155287 shutdown_connections 2023-09-13T04:38:29.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.033+0000 7ff3bb3ff700 1 --2- 172.21.15.2:0/1817155287 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff3a00a0690 0x7ff3a00a2b50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:29.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.033+0000 7ff3bb3ff700 1 --2- 172.21.15.2:0/1817155287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3b4102880 0x7ff3b41a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:29.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.033+0000 7ff3bb3ff700 1 --2- 172.21.15.2:0/1817155287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3b4100cf0 0x7ff3b4198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:29.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.033+0000 7ff3bb3ff700 1 --2- 172.21.15.2:0/1817155287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3b4101ee0 0x7ff3b41990f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:29.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.033+0000 7ff3bb3ff700 1 -- 172.21.15.2:0/1817155287 >> 172.21.15.2:0/1817155287 conn(0x7ff3b40fc220 msgr2=0x7ff3b40fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:29.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.033+0000 7ff3bb3ff700 1 -- 172.21.15.2:0/1817155287 shutdown_connections 2023-09-13T04:38:29.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.033+0000 7ff3bb3ff700 1 -- 172.21.15.2:0/1817155287 wait complete. 2023-09-13T04:38:29.040 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-09-13T04:38:29.053 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2023-09-13T04:38:27.505931+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596996,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":138,"num_read_kb":121,"num_write":286,"num_write_kb":2970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":416,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337796,"kb_used_data":13532,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604028,"statfs":{"total":1151940427776,"available":1151594524672,"internally_reserved":0,"allocated":13856768,"data_stored":7620918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3769,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":59,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002540"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592832+0000","last_change":"2023-09-13T04:37:59.244279+0000","last_active":"2023-09-13T04:38:23.592832+0000","last_peered":"2023-09-13T04:38:23.592832+0000","last_clean":"2023-09-13T04:38:23.592832+0000","last_became_active":"2023-09-13T04:37:59.243976+0000","last_became_peered":"2023-09-13T04:37:59.243976+0000","last_unstale":"2023-09-13T04:38:23.592832+0000","last_undegraded":"2023-09-13T04:38:23.592832+0000","last_fullsized":"2023-09-13T04:38:23.592832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:21:52.809411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406851+0000","last_change":"2023-09-13T04:37:59.243391+0000","last_active":"2023-09-13T04:38:04.406851+0000","last_peered":"2023-09-13T04:38:04.406851+0000","last_clean":"2023-09-13T04:38:04.406851+0000","last_became_active":"2023-09-13T04:37:59.243044+0000","last_became_peered":"2023-09-13T04:37:59.243044+0000","last_unstale":"2023-09-13T04:38:04.406851+0000","last_undegraded":"2023-09-13T04:38:04.406851+0000","last_fullsized":"2023-09-13T04:38:04.406851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:02:17.806638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406414+0000","last_change":"2023-09-13T04:37:59.242826+0000","last_active":"2023-09-13T04:38:04.406414+0000","last_peered":"2023-09-13T04:38:04.406414+0000","last_clean":"2023-09-13T04:38:04.406414+0000","last_became_active":"2023-09-13T04:37:59.242125+0000","last_became_peered":"2023-09-13T04:37:59.242125+0000","last_unstale":"2023-09-13T04:38:04.406414+0000","last_undegraded":"2023-09-13T04:38:04.406414+0000","last_fullsized":"2023-09-13T04:38:04.406414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:08:29.216093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590536+0000","last_change":"2023-09-13T04:37:59.253322+0000","last_active":"2023-09-13T04:38:23.590536+0000","last_peered":"2023-09-13T04:38:23.590536+0000","last_clean":"2023-09-13T04:38:23.590536+0000","last_became_active":"2023-09-13T04:37:59.253050+0000","last_became_peered":"2023-09-13T04:37:59.253050+0000","last_unstale":"2023-09-13T04:38:23.590536+0000","last_undegraded":"2023-09-13T04:38:23.590536+0000","last_fullsized":"2023-09-13T04:38:23.590536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:45:22.357879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593487+0000","last_change":"2023-09-13T04:37:59.250089+0000","last_active":"2023-09-13T04:38:23.593487+0000","last_peered":"2023-09-13T04:38:23.593487+0000","last_clean":"2023-09-13T04:38:23.593487+0000","last_became_active":"2023-09-13T04:37:59.249771+0000","last_became_peered":"2023-09-13T04:37:59.249771+0000","last_unstale":"2023-09-13T04:38:23.593487+0000","last_undegraded":"2023-09-13T04:38:23.593487+0000","last_fullsized":"2023-09-13T04:38:23.593487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:46:59.917756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595139+0000","last_change":"2023-09-13T04:37:59.247259+0000","last_active":"2023-09-13T04:38:23.595139+0000","last_peered":"2023-09-13T04:38:23.595139+0000","last_clean":"2023-09-13T04:38:23.595139+0000","last_became_active":"2023-09-13T04:37:59.246911+0000","last_became_peered":"2023-09-13T04:37:59.246911+0000","last_unstale":"2023-09-13T04:38:23.595139+0000","last_undegraded":"2023-09-13T04:38:23.595139+0000","last_fullsized":"2023-09-13T04:38:23.595139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:00:39.742392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.407650+0000","last_change":"2023-09-13T04:37:59.244083+0000","last_active":"2023-09-13T04:38:04.407650+0000","last_peered":"2023-09-13T04:38:04.407650+0000","last_clean":"2023-09-13T04:38:04.407650+0000","last_became_active":"2023-09-13T04:37:59.243789+0000","last_became_peered":"2023-09-13T04:37:59.243789+0000","last_unstale":"2023-09-13T04:38:04.407650+0000","last_undegraded":"2023-09-13T04:38:04.407650+0000","last_fullsized":"2023-09-13T04:38:04.407650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:24:00.622790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.602097+0000","last_change":"2023-09-13T04:38:24.601265+0000","last_active":"2023-09-13T04:38:24.602097+0000","last_peered":"2023-09-13T04:38:24.602097+0000","last_clean":"2023-09-13T04:38:24.602097+0000","last_became_active":"2023-09-13T04:38:24.600939+0000","last_became_peered":"2023-09-13T04:38:24.600939+0000","last_unstale":"2023-09-13T04:38:24.602097+0000","last_undegraded":"2023-09-13T04:38:24.602097+0000","last_fullsized":"2023-09-13T04:38:24.602097+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:18:23.642333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594759+0000","last_change":"2023-09-13T04:37:59.238053+0000","last_active":"2023-09-13T04:38:23.594759+0000","last_peered":"2023-09-13T04:38:23.594759+0000","last_clean":"2023-09-13T04:38:23.594759+0000","last_became_active":"2023-09-13T04:37:59.237777+0000","last_became_peered":"2023-09-13T04:37:59.237777+0000","last_unstale":"2023-09-13T04:38:23.594759+0000","last_undegraded":"2023-09-13T04:38:23.594759+0000","last_fullsized":"2023-09-13T04:38:23.594759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:45:39.835659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594929+0000","last_change":"2023-09-13T04:37:59.252605+0000","last_active":"2023-09-13T04:38:23.594929+0000","last_peered":"2023-09-13T04:38:23.594929+0000","last_clean":"2023-09-13T04:38:23.594929+0000","last_became_active":"2023-09-13T04:37:59.252309+0000","last_became_peered":"2023-09-13T04:37:59.252309+0000","last_unstale":"2023-09-13T04:38:23.594929+0000","last_undegraded":"2023-09-13T04:38:23.594929+0000","last_fullsized":"2023-09-13T04:38:23.594929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:41:45.064230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.408085+0000","last_change":"2023-09-13T04:37:59.243345+0000","last_active":"2023-09-13T04:38:04.408085+0000","last_peered":"2023-09-13T04:38:04.408085+0000","last_clean":"2023-09-13T04:38:04.408085+0000","last_became_active":"2023-09-13T04:37:59.242726+0000","last_became_peered":"2023-09-13T04:37:59.242726+0000","last_unstale":"2023-09-13T04:38:04.408085+0000","last_undegraded":"2023-09-13T04:38:04.408085+0000","last_fullsized":"2023-09-13T04:38:04.408085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:39:55.115281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590601+0000","last_change":"2023-09-13T04:37:59.247415+0000","last_active":"2023-09-13T04:38:23.590601+0000","last_peered":"2023-09-13T04:38:23.590601+0000","last_clean":"2023-09-13T04:38:23.590601+0000","last_became_active":"2023-09-13T04:37:59.247050+0000","last_became_peered":"2023-09-13T04:37:59.247050+0000","last_unstale":"2023-09-13T04:38:23.590601+0000","last_undegraded":"2023-09-13T04:38:23.590601+0000","last_fullsized":"2023-09-13T04:38:23.590601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:03:44.793628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597902+0000","last_change":"2023-09-13T04:37:59.248935+0000","last_active":"2023-09-13T04:38:23.597902+0000","last_peered":"2023-09-13T04:38:23.597902+0000","last_clean":"2023-09-13T04:38:23.597902+0000","last_became_active":"2023-09-13T04:37:59.248554+0000","last_became_peered":"2023-09-13T04:37:59.248554+0000","last_unstale":"2023-09-13T04:38:23.597902+0000","last_undegraded":"2023-09-13T04:38:23.597902+0000","last_fullsized":"2023-09-13T04:38:23.597902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:53:31.882567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406214+0000","last_change":"2023-09-13T04:37:59.242851+0000","last_active":"2023-09-13T04:38:04.406214+0000","last_peered":"2023-09-13T04:38:04.406214+0000","last_clean":"2023-09-13T04:38:04.406214+0000","last_became_active":"2023-09-13T04:37:59.242384+0000","last_became_peered":"2023-09-13T04:37:59.242384+0000","last_unstale":"2023-09-13T04:38:04.406214+0000","last_undegraded":"2023-09-13T04:38:04.406214+0000","last_fullsized":"2023-09-13T04:38:04.406214+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:59:36.508808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"peering","last_fresh":"2023-09-13T04:38:04.406264+0000","last_change":"2023-09-13T04:38:23.591313+0000","last_active":"2023-09-13T04:38:23.591313+0000","last_peered":"2023-09-13T04:38:04.406264+0000","last_clean":"2023-09-13T04:38:04.406264+0000","last_became_active":"2023-09-13T04:37:59.252899+0000","last_became_peered":"2023-09-13T04:37:59.252899+0000","last_unstale":"2023-09-13T04:38:04.406264+0000","last_undegraded":"2023-09-13T04:38:04.406264+0000","last_fullsized":"2023-09-13T04:38:04.406264+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4,6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592657+0000","last_change":"2023-09-13T04:37:59.249107+0000","last_active":"2023-09-13T04:38:23.592657+0000","last_peered":"2023-09-13T04:38:23.592657+0000","last_clean":"2023-09-13T04:38:23.592657+0000","last_became_active":"2023-09-13T04:37:59.248803+0000","last_became_peered":"2023-09-13T04:37:59.248803+0000","last_unstale":"2023-09-13T04:38:23.592657+0000","last_undegraded":"2023-09-13T04:38:23.592657+0000","last_fullsized":"2023-09-13T04:38:23.592657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:50:52.481194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592743+0000","last_change":"2023-09-13T04:37:59.243724+0000","last_active":"2023-09-13T04:38:23.592743+0000","last_peered":"2023-09-13T04:38:23.592743+0000","last_clean":"2023-09-13T04:38:23.592743+0000","last_became_active":"2023-09-13T04:37:59.243385+0000","last_became_peered":"2023-09-13T04:37:59.243385+0000","last_unstale":"2023-09-13T04:38:23.592743+0000","last_undegraded":"2023-09-13T04:38:23.592743+0000","last_fullsized":"2023-09-13T04:38:23.592743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:55:55.203940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588550+0000","last_change":"2023-09-13T04:37:59.238986+0000","last_active":"2023-09-13T04:38:23.588550+0000","last_peered":"2023-09-13T04:38:23.588550+0000","last_clean":"2023-09-13T04:38:23.588550+0000","last_became_active":"2023-09-13T04:37:59.238668+0000","last_became_peered":"2023-09-13T04:37:59.238668+0000","last_unstale":"2023-09-13T04:38:23.588550+0000","last_undegraded":"2023-09-13T04:38:23.588550+0000","last_fullsized":"2023-09-13T04:38:23.588550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:21:10.225223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588585+0000","last_change":"2023-09-13T04:37:59.245598+0000","last_active":"2023-09-13T04:38:23.588585+0000","last_peered":"2023-09-13T04:38:23.588585+0000","last_clean":"2023-09-13T04:38:23.588585+0000","last_became_active":"2023-09-13T04:37:59.244652+0000","last_became_peered":"2023-09-13T04:37:59.244652+0000","last_unstale":"2023-09-13T04:38:23.588585+0000","last_undegraded":"2023-09-13T04:38:23.588585+0000","last_fullsized":"2023-09-13T04:38:23.588585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:34:43.671168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588238+0000","last_change":"2023-09-13T04:38:03.617708+0000","last_active":"2023-09-13T04:38:23.588238+0000","last_peered":"2023-09-13T04:38:23.588238+0000","last_clean":"2023-09-13T04:38:23.588238+0000","last_became_active":"2023-09-13T04:38:03.617277+0000","last_became_peered":"2023-09-13T04:38:03.617277+0000","last_unstale":"2023-09-13T04:38:23.588238+0000","last_undegraded":"2023-09-13T04:38:23.588238+0000","last_fullsized":"2023-09-13T04:38:23.588238+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:26:44.318030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.621267+0000","last_change":"2023-09-13T04:37:58.595156+0000","last_active":"2023-09-13T04:38:25.621267+0000","last_peered":"2023-09-13T04:38:25.621267+0000","last_clean":"2023-09-13T04:38:25.621267+0000","last_became_active":"2023-09-13T04:37:58.594863+0000","last_became_peered":"2023-09-13T04:37:58.594863+0000","last_unstale":"2023-09-13T04:38:25.621267+0000","last_undegraded":"2023-09-13T04:38:25.621267+0000","last_fullsized":"2023-09-13T04:38:25.621267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:27:14.420495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.602369+0000","last_change":"2023-09-13T04:37:59.250252+0000","last_active":"2023-09-13T04:38:03.602369+0000","last_peered":"2023-09-13T04:38:03.602369+0000","last_clean":"2023-09-13T04:38:03.602369+0000","last_became_active":"2023-09-13T04:37:59.249927+0000","last_became_peered":"2023-09-13T04:37:59.249927+0000","last_unstale":"2023-09-13T04:38:03.602369+0000","last_undegraded":"2023-09-13T04:38:03.602369+0000","last_fullsized":"2023-09-13T04:38:03.602369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:18:06.757040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.599683+0000","last_change":"2023-09-13T04:38:24.599683+0000","last_active":"2023-09-13T04:38:24.599683+0000","last_peered":"2023-09-13T04:38:24.599683+0000","last_clean":"2023-09-13T04:38:24.599683+0000","last_became_active":"2023-09-13T04:38:24.599230+0000","last_became_peered":"2023-09-13T04:38:24.599230+0000","last_unstale":"2023-09-13T04:38:24.599683+0000","last_undegraded":"2023-09-13T04:38:24.599683+0000","last_fullsized":"2023-09-13T04:38:24.599683+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:48:22.584735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.602561+0000","last_change":"2023-09-13T04:37:58.597785+0000","last_active":"2023-09-13T04:38:03.602561+0000","last_peered":"2023-09-13T04:38:03.602561+0000","last_clean":"2023-09-13T04:38:03.602561+0000","last_became_active":"2023-09-13T04:37:58.597485+0000","last_became_peered":"2023-09-13T04:37:58.597485+0000","last_unstale":"2023-09-13T04:38:03.602561+0000","last_undegraded":"2023-09-13T04:38:03.602561+0000","last_fullsized":"2023-09-13T04:38:03.602561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:39:57.156038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406705+0000","last_change":"2023-09-13T04:37:59.241840+0000","last_active":"2023-09-13T04:38:04.406705+0000","last_peered":"2023-09-13T04:38:04.406705+0000","last_clean":"2023-09-13T04:38:04.406705+0000","last_became_active":"2023-09-13T04:37:59.241545+0000","last_became_peered":"2023-09-13T04:37:59.241545+0000","last_unstale":"2023-09-13T04:38:04.406705+0000","last_undegraded":"2023-09-13T04:38:04.406705+0000","last_fullsized":"2023-09-13T04:38:04.406705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:04:17.439520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591101+0000","last_change":"2023-09-13T04:38:03.621792+0000","last_active":"2023-09-13T04:38:23.591101+0000","last_peered":"2023-09-13T04:38:23.591101+0000","last_clean":"2023-09-13T04:38:23.591101+0000","last_became_active":"2023-09-13T04:38:03.621433+0000","last_became_peered":"2023-09-13T04:38:03.621433+0000","last_unstale":"2023-09-13T04:38:23.591101+0000","last_undegraded":"2023-09-13T04:38:23.591101+0000","last_fullsized":"2023-09-13T04:38:23.591101+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:52:05.926733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594362+0000","last_change":"2023-09-13T04:37:58.590817+0000","last_active":"2023-09-13T04:38:23.594362+0000","last_peered":"2023-09-13T04:38:23.594362+0000","last_clean":"2023-09-13T04:38:23.594362+0000","last_became_active":"2023-09-13T04:37:58.590348+0000","last_became_peered":"2023-09-13T04:37:58.590348+0000","last_unstale":"2023-09-13T04:38:23.594362+0000","last_undegraded":"2023-09-13T04:38:23.594362+0000","last_fullsized":"2023-09-13T04:38:23.594362+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:38:35.680905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594252+0000","last_change":"2023-09-13T04:37:59.237304+0000","last_active":"2023-09-13T04:38:23.594252+0000","last_peered":"2023-09-13T04:38:23.594252+0000","last_clean":"2023-09-13T04:38:23.594252+0000","last_became_active":"2023-09-13T04:37:59.237008+0000","last_became_peered":"2023-09-13T04:37:59.237008+0000","last_unstale":"2023-09-13T04:38:23.594252+0000","last_undegraded":"2023-09-13T04:38:23.594252+0000","last_fullsized":"2023-09-13T04:38:23.594252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:28:14.254219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590193+0000","last_change":"2023-09-13T04:38:03.606749+0000","last_active":"2023-09-13T04:38:23.590193+0000","last_peered":"2023-09-13T04:38:23.590193+0000","last_clean":"2023-09-13T04:38:23.590193+0000","last_became_active":"2023-09-13T04:38:03.606276+0000","last_became_peered":"2023-09-13T04:38:03.606276+0000","last_unstale":"2023-09-13T04:38:23.590193+0000","last_undegraded":"2023-09-13T04:38:23.590193+0000","last_fullsized":"2023-09-13T04:38:23.590193+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:30:53.572543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593692+0000","last_change":"2023-09-13T04:37:58.590937+0000","last_active":"2023-09-13T04:38:23.593692+0000","last_peered":"2023-09-13T04:38:23.593692+0000","last_clean":"2023-09-13T04:38:23.593692+0000","last_became_active":"2023-09-13T04:37:58.590606+0000","last_became_peered":"2023-09-13T04:37:58.590606+0000","last_unstale":"2023-09-13T04:38:23.593692+0000","last_undegraded":"2023-09-13T04:38:23.593692+0000","last_fullsized":"2023-09-13T04:38:23.593692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:30:03.354670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591224+0000","last_change":"2023-09-13T04:37:59.251899+0000","last_active":"2023-09-13T04:38:23.591224+0000","last_peered":"2023-09-13T04:38:23.591224+0000","last_clean":"2023-09-13T04:38:23.591224+0000","last_became_active":"2023-09-13T04:37:59.251557+0000","last_became_peered":"2023-09-13T04:37:59.251557+0000","last_unstale":"2023-09-13T04:38:23.591224+0000","last_undegraded":"2023-09-13T04:38:23.591224+0000","last_fullsized":"2023-09-13T04:38:23.591224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:57:45.241188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588439+0000","last_change":"2023-09-13T04:38:03.606076+0000","last_active":"2023-09-13T04:38:23.588439+0000","last_peered":"2023-09-13T04:38:23.588439+0000","last_clean":"2023-09-13T04:38:23.588439+0000","last_became_active":"2023-09-13T04:38:03.605556+0000","last_became_peered":"2023-09-13T04:38:03.605556+0000","last_unstale":"2023-09-13T04:38:23.588439+0000","last_undegraded":"2023-09-13T04:38:23.588439+0000","last_fullsized":"2023-09-13T04:38:23.588439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:03:32.344625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.246702+0000","last_change":"2023-09-13T04:37:58.588170+0000","last_active":"2023-09-13T04:38:24.246702+0000","last_peered":"2023-09-13T04:38:24.246702+0000","last_clean":"2023-09-13T04:38:24.246702+0000","last_became_active":"2023-09-13T04:37:58.587860+0000","last_became_peered":"2023-09-13T04:37:58.587860+0000","last_unstale":"2023-09-13T04:38:24.246702+0000","last_undegraded":"2023-09-13T04:38:24.246702+0000","last_fullsized":"2023-09-13T04:38:24.246702+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:38:40.048150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595472+0000","last_change":"2023-09-13T04:37:59.240248+0000","last_active":"2023-09-13T04:38:23.595472+0000","last_peered":"2023-09-13T04:38:23.595472+0000","last_clean":"2023-09-13T04:38:23.595472+0000","last_became_active":"2023-09-13T04:37:59.240008+0000","last_became_peered":"2023-09-13T04:37:59.240008+0000","last_unstale":"2023-09-13T04:38:23.595472+0000","last_undegraded":"2023-09-13T04:38:23.595472+0000","last_fullsized":"2023-09-13T04:38:23.595472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:15:13.275110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587821+0000","last_change":"2023-09-13T04:38:03.616607+0000","last_active":"2023-09-13T04:38:23.587821+0000","last_peered":"2023-09-13T04:38:23.587821+0000","last_clean":"2023-09-13T04:38:23.587821+0000","last_became_active":"2023-09-13T04:38:03.616209+0000","last_became_peered":"2023-09-13T04:38:03.616209+0000","last_unstale":"2023-09-13T04:38:23.587821+0000","last_undegraded":"2023-09-13T04:38:23.587821+0000","last_fullsized":"2023-09-13T04:38:23.587821+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:24:21.934053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595102+0000","last_change":"2023-09-13T04:37:58.589104+0000","last_active":"2023-09-13T04:38:23.595102+0000","last_peered":"2023-09-13T04:38:23.595102+0000","last_clean":"2023-09-13T04:38:23.595102+0000","last_became_active":"2023-09-13T04:37:58.588780+0000","last_became_peered":"2023-09-13T04:37:58.588780+0000","last_unstale":"2023-09-13T04:38:23.595102+0000","last_undegraded":"2023-09-13T04:38:23.595102+0000","last_fullsized":"2023-09-13T04:38:23.595102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:01:32.839729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.600030+0000","last_change":"2023-09-13T04:38:24.598663+0000","last_active":"2023-09-13T04:38:24.600030+0000","last_peered":"2023-09-13T04:38:24.600030+0000","last_clean":"2023-09-13T04:38:24.600030+0000","last_became_active":"2023-09-13T04:38:24.598258+0000","last_became_peered":"2023-09-13T04:38:24.598258+0000","last_unstale":"2023-09-13T04:38:24.600030+0000","last_undegraded":"2023-09-13T04:38:24.600030+0000","last_fullsized":"2023-09-13T04:38:24.600030+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:50:28.157135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592633+0000","last_change":"2023-09-13T04:38:03.619930+0000","last_active":"2023-09-13T04:38:23.592633+0000","last_peered":"2023-09-13T04:38:23.592633+0000","last_clean":"2023-09-13T04:38:23.592633+0000","last_became_active":"2023-09-13T04:38:03.619560+0000","last_became_peered":"2023-09-13T04:38:03.619560+0000","last_unstale":"2023-09-13T04:38:23.592633+0000","last_undegraded":"2023-09-13T04:38:23.592633+0000","last_fullsized":"2023-09-13T04:38:23.592633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:48:06.867602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.619713+0000","last_change":"2023-09-13T04:37:58.590396+0000","last_active":"2023-09-13T04:38:25.619713+0000","last_peered":"2023-09-13T04:38:25.619713+0000","last_clean":"2023-09-13T04:38:25.619713+0000","last_became_active":"2023-09-13T04:37:58.589929+0000","last_became_peered":"2023-09-13T04:37:58.589929+0000","last_unstale":"2023-09-13T04:38:25.619713+0000","last_undegraded":"2023-09-13T04:38:25.619713+0000","last_fullsized":"2023-09-13T04:38:25.619713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:28:36.322096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.600166+0000","last_change":"2023-09-13T04:37:59.246033+0000","last_active":"2023-09-13T04:38:03.600166+0000","last_peered":"2023-09-13T04:38:03.600166+0000","last_clean":"2023-09-13T04:38:03.600166+0000","last_became_active":"2023-09-13T04:37:59.245664+0000","last_became_peered":"2023-09-13T04:37:59.245664+0000","last_unstale":"2023-09-13T04:38:03.600166+0000","last_undegraded":"2023-09-13T04:38:03.600166+0000","last_fullsized":"2023-09-13T04:38:03.600166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:24:17.145055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406894+0000","last_change":"2023-09-13T04:38:03.608504+0000","last_active":"2023-09-13T04:38:04.406894+0000","last_peered":"2023-09-13T04:38:04.406894+0000","last_clean":"2023-09-13T04:38:04.406894+0000","last_became_active":"2023-09-13T04:38:03.608058+0000","last_became_peered":"2023-09-13T04:38:03.608058+0000","last_unstale":"2023-09-13T04:38:04.406894+0000","last_undegraded":"2023-09-13T04:38:04.406894+0000","last_fullsized":"2023-09-13T04:38:04.406894+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:42:43.485453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244691+0000","last_change":"2023-09-13T04:37:58.588208+0000","last_active":"2023-09-13T04:38:24.244691+0000","last_peered":"2023-09-13T04:38:24.244691+0000","last_clean":"2023-09-13T04:38:24.244691+0000","last_became_active":"2023-09-13T04:37:58.587840+0000","last_became_peered":"2023-09-13T04:37:58.587840+0000","last_unstale":"2023-09-13T04:38:24.244691+0000","last_undegraded":"2023-09-13T04:38:24.244691+0000","last_fullsized":"2023-09-13T04:38:24.244691+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:44:58.012233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587683+0000","last_change":"2023-09-13T04:37:59.245102+0000","last_active":"2023-09-13T04:38:23.587683+0000","last_peered":"2023-09-13T04:38:23.587683+0000","last_clean":"2023-09-13T04:38:23.587683+0000","last_became_active":"2023-09-13T04:37:59.244799+0000","last_became_peered":"2023-09-13T04:37:59.244799+0000","last_unstale":"2023-09-13T04:38:23.587683+0000","last_undegraded":"2023-09-13T04:38:23.587683+0000","last_fullsized":"2023-09-13T04:38:23.587683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:57:16.295623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.454268+0000","last_change":"2023-09-13T04:38:03.607021+0000","last_active":"2023-09-13T04:38:04.454268+0000","last_peered":"2023-09-13T04:38:04.454268+0000","last_clean":"2023-09-13T04:38:04.454268+0000","last_became_active":"2023-09-13T04:38:03.606611+0000","last_became_peered":"2023-09-13T04:38:03.606611+0000","last_unstale":"2023-09-13T04:38:04.454268+0000","last_undegraded":"2023-09-13T04:38:04.454268+0000","last_fullsized":"2023-09-13T04:38:04.454268+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:33:20.551479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244009+0000","last_change":"2023-09-13T04:37:58.588731+0000","last_active":"2023-09-13T04:38:24.244009+0000","last_peered":"2023-09-13T04:38:24.244009+0000","last_clean":"2023-09-13T04:38:24.244009+0000","last_became_active":"2023-09-13T04:37:58.588398+0000","last_became_peered":"2023-09-13T04:37:58.588398+0000","last_unstale":"2023-09-13T04:38:24.244009+0000","last_undegraded":"2023-09-13T04:38:24.244009+0000","last_fullsized":"2023-09-13T04:38:24.244009+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:13:10.724085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594790+0000","last_change":"2023-09-13T04:37:59.244757+0000","last_active":"2023-09-13T04:38:23.594790+0000","last_peered":"2023-09-13T04:38:23.594790+0000","last_clean":"2023-09-13T04:38:23.594790+0000","last_became_active":"2023-09-13T04:37:59.244423+0000","last_became_peered":"2023-09-13T04:37:59.244423+0000","last_unstale":"2023-09-13T04:38:23.594790+0000","last_undegraded":"2023-09-13T04:38:23.594790+0000","last_fullsized":"2023-09-13T04:38:23.594790+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:14:37.973058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587303+0000","last_change":"2023-09-13T04:38:03.618358+0000","last_active":"2023-09-13T04:38:23.587303+0000","last_peered":"2023-09-13T04:38:23.587303+0000","last_clean":"2023-09-13T04:38:23.587303+0000","last_became_active":"2023-09-13T04:38:03.617291+0000","last_became_peered":"2023-09-13T04:38:03.617291+0000","last_unstale":"2023-09-13T04:38:23.587303+0000","last_undegraded":"2023-09-13T04:38:23.587303+0000","last_fullsized":"2023-09-13T04:38:23.587303+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:33:13.955353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244399+0000","last_change":"2023-09-13T04:37:58.590978+0000","last_active":"2023-09-13T04:38:24.244399+0000","last_peered":"2023-09-13T04:38:24.244399+0000","last_clean":"2023-09-13T04:38:24.244399+0000","last_became_active":"2023-09-13T04:37:58.590673+0000","last_became_peered":"2023-09-13T04:37:58.590673+0000","last_unstale":"2023-09-13T04:38:24.244399+0000","last_undegraded":"2023-09-13T04:38:24.244399+0000","last_fullsized":"2023-09-13T04:38:24.244399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:10:32.570029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406610+0000","last_change":"2023-09-13T04:37:59.251283+0000","last_active":"2023-09-13T04:38:04.406610+0000","last_peered":"2023-09-13T04:38:04.406610+0000","last_clean":"2023-09-13T04:38:04.406610+0000","last_became_active":"2023-09-13T04:37:59.250119+0000","last_became_peered":"2023-09-13T04:37:59.250119+0000","last_unstale":"2023-09-13T04:38:04.406610+0000","last_undegraded":"2023-09-13T04:38:04.406610+0000","last_fullsized":"2023-09-13T04:38:04.406610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:48:54.389970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.596107+0000","last_change":"2023-09-13T04:38:24.596107+0000","last_active":"2023-09-13T04:38:24.596107+0000","last_peered":"2023-09-13T04:38:24.596107+0000","last_clean":"2023-09-13T04:38:24.596107+0000","last_became_active":"2023-09-13T04:38:24.595653+0000","last_became_peered":"2023-09-13T04:38:24.595653+0000","last_unstale":"2023-09-13T04:38:24.596107+0000","last_undegraded":"2023-09-13T04:38:24.596107+0000","last_fullsized":"2023-09-13T04:38:24.596107+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:02:53.708658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.243198+0000","last_change":"2023-09-13T04:37:58.584909+0000","last_active":"2023-09-13T04:38:24.243198+0000","last_peered":"2023-09-13T04:38:24.243198+0000","last_clean":"2023-09-13T04:38:24.243198+0000","last_became_active":"2023-09-13T04:37:58.584630+0000","last_became_peered":"2023-09-13T04:37:58.584630+0000","last_unstale":"2023-09-13T04:38:24.243198+0000","last_undegraded":"2023-09-13T04:38:24.243198+0000","last_fullsized":"2023-09-13T04:38:24.243198+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:57:56.567499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594349+0000","last_change":"2023-09-13T04:37:59.254448+0000","last_active":"2023-09-13T04:38:23.594349+0000","last_peered":"2023-09-13T04:38:23.594349+0000","last_clean":"2023-09-13T04:38:23.594349+0000","last_became_active":"2023-09-13T04:37:59.254138+0000","last_became_peered":"2023-09-13T04:37:59.254138+0000","last_unstale":"2023-09-13T04:38:23.594349+0000","last_undegraded":"2023-09-13T04:38:23.594349+0000","last_fullsized":"2023-09-13T04:38:23.594349+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:36:14.939881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588357+0000","last_change":"2023-09-13T04:38:03.613746+0000","last_active":"2023-09-13T04:38:23.588357+0000","last_peered":"2023-09-13T04:38:23.588357+0000","last_clean":"2023-09-13T04:38:23.588357+0000","last_became_active":"2023-09-13T04:38:03.613354+0000","last_became_peered":"2023-09-13T04:38:03.613354+0000","last_unstale":"2023-09-13T04:38:23.588357+0000","last_undegraded":"2023-09-13T04:38:23.588357+0000","last_fullsized":"2023-09-13T04:38:23.588357+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:41:01.294912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.242914+0000","last_change":"2023-09-13T04:37:58.591196+0000","last_active":"2023-09-13T04:38:24.242914+0000","last_peered":"2023-09-13T04:38:24.242914+0000","last_clean":"2023-09-13T04:38:24.242914+0000","last_became_active":"2023-09-13T04:37:58.590883+0000","last_became_peered":"2023-09-13T04:37:58.590883+0000","last_unstale":"2023-09-13T04:38:24.242914+0000","last_undegraded":"2023-09-13T04:38:24.242914+0000","last_fullsized":"2023-09-13T04:38:24.242914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:46:58.530270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595499+0000","last_change":"2023-09-13T04:37:59.243483+0000","last_active":"2023-09-13T04:38:23.595499+0000","last_peered":"2023-09-13T04:38:23.595499+0000","last_clean":"2023-09-13T04:38:23.595499+0000","last_became_active":"2023-09-13T04:37:59.243088+0000","last_became_peered":"2023-09-13T04:37:59.243088+0000","last_unstale":"2023-09-13T04:38:23.595499+0000","last_undegraded":"2023-09-13T04:38:23.595499+0000","last_fullsized":"2023-09-13T04:38:23.595499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:56:14.093819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591697+0000","last_change":"2023-09-13T04:38:03.608820+0000","last_active":"2023-09-13T04:38:23.591697+0000","last_peered":"2023-09-13T04:38:23.591697+0000","last_clean":"2023-09-13T04:38:23.591697+0000","last_became_active":"2023-09-13T04:38:03.608200+0000","last_became_peered":"2023-09-13T04:38:03.608200+0000","last_unstale":"2023-09-13T04:38:23.591697+0000","last_undegraded":"2023-09-13T04:38:23.591697+0000","last_fullsized":"2023-09-13T04:38:23.591697+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:54:52.416501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.607235+0000","last_change":"2023-09-13T04:38:24.606820+0000","last_active":"2023-09-13T04:38:24.607235+0000","last_peered":"2023-09-13T04:38:24.607235+0000","last_clean":"2023-09-13T04:38:24.607235+0000","last_became_active":"2023-09-13T04:38:24.599059+0000","last_became_peered":"2023-09-13T04:38:24.599059+0000","last_unstale":"2023-09-13T04:38:24.607235+0000","last_undegraded":"2023-09-13T04:38:24.607235+0000","last_fullsized":"2023-09-13T04:38:24.607235+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:08:12.634357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597568+0000","last_change":"2023-09-13T04:37:59.250666+0000","last_active":"2023-09-13T04:38:23.597568+0000","last_peered":"2023-09-13T04:38:23.597568+0000","last_clean":"2023-09-13T04:38:23.597568+0000","last_became_active":"2023-09-13T04:37:59.250240+0000","last_became_peered":"2023-09-13T04:37:59.250240+0000","last_unstale":"2023-09-13T04:38:23.597568+0000","last_undegraded":"2023-09-13T04:38:23.597568+0000","last_fullsized":"2023-09-13T04:38:23.597568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:44:31.324222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592271+0000","last_change":"2023-09-13T04:38:03.616296+0000","last_active":"2023-09-13T04:38:23.592271+0000","last_peered":"2023-09-13T04:38:23.592271+0000","last_clean":"2023-09-13T04:38:23.592271+0000","last_became_active":"2023-09-13T04:38:03.615934+0000","last_became_peered":"2023-09-13T04:38:03.615934+0000","last_unstale":"2023-09-13T04:38:23.592271+0000","last_undegraded":"2023-09-13T04:38:23.592271+0000","last_fullsized":"2023-09-13T04:38:23.592271+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:51:18.994472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244805+0000","last_change":"2023-09-13T04:37:58.594782+0000","last_active":"2023-09-13T04:38:24.244805+0000","last_peered":"2023-09-13T04:38:24.244805+0000","last_clean":"2023-09-13T04:38:24.244805+0000","last_became_active":"2023-09-13T04:37:58.594480+0000","last_became_peered":"2023-09-13T04:37:58.594480+0000","last_unstale":"2023-09-13T04:38:24.244805+0000","last_undegraded":"2023-09-13T04:38:24.244805+0000","last_fullsized":"2023-09-13T04:38:24.244805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:40:19.635672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.407475+0000","last_change":"2023-09-13T04:37:59.252575+0000","last_active":"2023-09-13T04:38:04.407475+0000","last_peered":"2023-09-13T04:38:04.407475+0000","last_clean":"2023-09-13T04:38:04.407475+0000","last_became_active":"2023-09-13T04:37:59.252268+0000","last_became_peered":"2023-09-13T04:37:59.252268+0000","last_unstale":"2023-09-13T04:38:04.407475+0000","last_undegraded":"2023-09-13T04:38:04.407475+0000","last_fullsized":"2023-09-13T04:38:04.407475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:27:10.814930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587908+0000","last_change":"2023-09-13T04:38:03.618835+0000","last_active":"2023-09-13T04:38:23.587908+0000","last_peered":"2023-09-13T04:38:23.587908+0000","last_clean":"2023-09-13T04:38:23.587908+0000","last_became_active":"2023-09-13T04:38:03.618226+0000","last_became_peered":"2023-09-13T04:38:03.618226+0000","last_unstale":"2023-09-13T04:38:23.587908+0000","last_undegraded":"2023-09-13T04:38:23.587908+0000","last_fullsized":"2023-09-13T04:38:23.587908+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:01:28.157490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590730+0000","last_change":"2023-09-13T04:37:58.588545+0000","last_active":"2023-09-13T04:38:23.590730+0000","last_peered":"2023-09-13T04:38:23.590730+0000","last_clean":"2023-09-13T04:38:23.590730+0000","last_became_active":"2023-09-13T04:37:58.587995+0000","last_became_peered":"2023-09-13T04:37:58.587995+0000","last_unstale":"2023-09-13T04:38:23.590730+0000","last_undegraded":"2023-09-13T04:38:23.590730+0000","last_fullsized":"2023-09-13T04:38:23.590730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:40:08.797724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.407438+0000","last_change":"2023-09-13T04:37:59.249100+0000","last_active":"2023-09-13T04:38:04.407438+0000","last_peered":"2023-09-13T04:38:04.407438+0000","last_clean":"2023-09-13T04:38:04.407438+0000","last_became_active":"2023-09-13T04:37:59.248771+0000","last_became_peered":"2023-09-13T04:37:59.248771+0000","last_unstale":"2023-09-13T04:38:04.407438+0000","last_undegraded":"2023-09-13T04:38:04.407438+0000","last_fullsized":"2023-09-13T04:38:04.407438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:27:37.964595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597364+0000","last_change":"2023-09-13T04:38:03.616988+0000","last_active":"2023-09-13T04:38:23.597364+0000","last_peered":"2023-09-13T04:38:23.597364+0000","last_clean":"2023-09-13T04:38:23.597364+0000","last_became_active":"2023-09-13T04:38:03.616524+0000","last_became_peered":"2023-09-13T04:38:03.616524+0000","last_unstale":"2023-09-13T04:38:23.597364+0000","last_undegraded":"2023-09-13T04:38:23.597364+0000","last_fullsized":"2023-09-13T04:38:23.597364+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:42:41.431115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'10","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.613710+0000","last_change":"2023-09-13T04:38:24.613384+0000","last_active":"2023-09-13T04:38:24.613710+0000","last_peered":"2023-09-13T04:38:24.613710+0000","last_clean":"2023-09-13T04:38:24.613710+0000","last_became_active":"2023-09-13T04:38:24.599349+0000","last_became_peered":"2023-09-13T04:38:24.599349+0000","last_unstale":"2023-09-13T04:38:24.613710+0000","last_undegraded":"2023-09-13T04:38:24.613710+0000","last_fullsized":"2023-09-13T04:38:24.613710+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:23:23.479715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593329+0000","last_change":"2023-09-13T04:37:59.252237+0000","last_active":"2023-09-13T04:38:23.593329+0000","last_peered":"2023-09-13T04:38:23.593329+0000","last_clean":"2023-09-13T04:38:23.593329+0000","last_became_active":"2023-09-13T04:37:59.251718+0000","last_became_peered":"2023-09-13T04:37:59.251718+0000","last_unstale":"2023-09-13T04:38:23.593329+0000","last_undegraded":"2023-09-13T04:38:23.593329+0000","last_fullsized":"2023-09-13T04:38:23.593329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:45:48.405226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593276+0000","last_change":"2023-09-13T04:38:03.607096+0000","last_active":"2023-09-13T04:38:23.593276+0000","last_peered":"2023-09-13T04:38:23.593276+0000","last_clean":"2023-09-13T04:38:23.593276+0000","last_became_active":"2023-09-13T04:38:03.606656+0000","last_became_peered":"2023-09-13T04:38:03.606656+0000","last_unstale":"2023-09-13T04:38:23.593276+0000","last_undegraded":"2023-09-13T04:38:23.593276+0000","last_fullsized":"2023-09-13T04:38:23.593276+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:59:26.140360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595200+0000","last_change":"2023-09-13T04:37:58.591047+0000","last_active":"2023-09-13T04:38:23.595200+0000","last_peered":"2023-09-13T04:38:23.595200+0000","last_clean":"2023-09-13T04:38:23.595200+0000","last_became_active":"2023-09-13T04:37:58.590740+0000","last_became_peered":"2023-09-13T04:37:58.590740+0000","last_unstale":"2023-09-13T04:38:23.595200+0000","last_undegraded":"2023-09-13T04:38:23.595200+0000","last_fullsized":"2023-09-13T04:38:23.595200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:14:42.280416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592406+0000","last_change":"2023-09-13T04:37:59.249362+0000","last_active":"2023-09-13T04:38:23.592406+0000","last_peered":"2023-09-13T04:38:23.592406+0000","last_clean":"2023-09-13T04:38:23.592406+0000","last_became_active":"2023-09-13T04:37:59.249041+0000","last_became_peered":"2023-09-13T04:37:59.249041+0000","last_unstale":"2023-09-13T04:38:23.592406+0000","last_undegraded":"2023-09-13T04:38:23.592406+0000","last_fullsized":"2023-09-13T04:38:23.592406+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:06:33.822157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.598370+0000","last_change":"2023-09-13T04:38:24.598370+0000","last_active":"2023-09-13T04:38:24.598370+0000","last_peered":"2023-09-13T04:38:24.598370+0000","last_clean":"2023-09-13T04:38:24.598370+0000","last_became_active":"2023-09-13T04:38:24.597931+0000","last_became_peered":"2023-09-13T04:38:24.597931+0000","last_unstale":"2023-09-13T04:38:24.598370+0000","last_undegraded":"2023-09-13T04:38:24.598370+0000","last_fullsized":"2023-09-13T04:38:24.598370+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:14:28.911329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588100+0000","last_change":"2023-09-13T04:37:58.592496+0000","last_active":"2023-09-13T04:38:23.588100+0000","last_peered":"2023-09-13T04:38:23.588100+0000","last_clean":"2023-09-13T04:38:23.588100+0000","last_became_active":"2023-09-13T04:37:58.592106+0000","last_became_peered":"2023-09-13T04:37:58.592106+0000","last_unstale":"2023-09-13T04:38:23.588100+0000","last_undegraded":"2023-09-13T04:38:23.588100+0000","last_fullsized":"2023-09-13T04:38:23.588100+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:28:55.618563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594212+0000","last_change":"2023-09-13T04:37:59.250019+0000","last_active":"2023-09-13T04:38:23.594212+0000","last_peered":"2023-09-13T04:38:23.594212+0000","last_clean":"2023-09-13T04:38:23.594212+0000","last_became_active":"2023-09-13T04:37:59.249278+0000","last_became_peered":"2023-09-13T04:37:59.249278+0000","last_unstale":"2023-09-13T04:38:23.594212+0000","last_undegraded":"2023-09-13T04:38:23.594212+0000","last_fullsized":"2023-09-13T04:38:23.594212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:15:31.217236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597640+0000","last_change":"2023-09-13T04:38:03.614542+0000","last_active":"2023-09-13T04:38:23.597640+0000","last_peered":"2023-09-13T04:38:23.597640+0000","last_clean":"2023-09-13T04:38:23.597640+0000","last_became_active":"2023-09-13T04:38:03.614103+0000","last_became_peered":"2023-09-13T04:38:03.614103+0000","last_unstale":"2023-09-13T04:38:23.597640+0000","last_undegraded":"2023-09-13T04:38:23.597640+0000","last_fullsized":"2023-09-13T04:38:23.597640+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:06:48.426843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.407747+0000","last_change":"2023-09-13T04:37:58.591864+0000","last_active":"2023-09-13T04:38:04.407747+0000","last_peered":"2023-09-13T04:38:04.407747+0000","last_clean":"2023-09-13T04:38:04.407747+0000","last_became_active":"2023-09-13T04:37:58.591562+0000","last_became_peered":"2023-09-13T04:37:58.591562+0000","last_unstale":"2023-09-13T04:38:04.407747+0000","last_undegraded":"2023-09-13T04:38:04.407747+0000","last_fullsized":"2023-09-13T04:38:04.407747+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:31:27.280489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594567+0000","last_change":"2023-09-13T04:37:59.252600+0000","last_active":"2023-09-13T04:38:23.594567+0000","last_peered":"2023-09-13T04:38:23.594567+0000","last_clean":"2023-09-13T04:38:23.594567+0000","last_became_active":"2023-09-13T04:37:59.252306+0000","last_became_peered":"2023-09-13T04:37:59.252306+0000","last_unstale":"2023-09-13T04:38:23.594567+0000","last_undegraded":"2023-09-13T04:38:23.594567+0000","last_fullsized":"2023-09-13T04:38:23.594567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:18:55.686847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.596920+0000","last_change":"2023-09-13T04:38:03.742502+0000","last_active":"2023-09-13T04:38:23.596920+0000","last_peered":"2023-09-13T04:38:23.596920+0000","last_clean":"2023-09-13T04:38:23.596920+0000","last_became_active":"2023-09-13T04:38:03.742054+0000","last_became_peered":"2023-09-13T04:38:03.742054+0000","last_unstale":"2023-09-13T04:38:23.596920+0000","last_undegraded":"2023-09-13T04:38:23.596920+0000","last_fullsized":"2023-09-13T04:38:23.596920+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:55:54.601145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593527+0000","last_change":"2023-09-13T04:37:59.245644+0000","last_active":"2023-09-13T04:38:23.593527+0000","last_peered":"2023-09-13T04:38:23.593527+0000","last_clean":"2023-09-13T04:38:23.593527+0000","last_became_active":"2023-09-13T04:37:59.245284+0000","last_became_peered":"2023-09-13T04:37:59.245284+0000","last_unstale":"2023-09-13T04:38:23.593527+0000","last_undegraded":"2023-09-13T04:38:23.593527+0000","last_fullsized":"2023-09-13T04:38:23.593527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:19:04.002414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.621561+0000","last_change":"2023-09-13T04:37:58.590249+0000","last_active":"2023-09-13T04:38:25.621561+0000","last_peered":"2023-09-13T04:38:25.621561+0000","last_clean":"2023-09-13T04:38:25.621561+0000","last_became_active":"2023-09-13T04:37:58.589521+0000","last_became_peered":"2023-09-13T04:37:58.589521+0000","last_unstale":"2023-09-13T04:38:25.621561+0000","last_undegraded":"2023-09-13T04:38:25.621561+0000","last_fullsized":"2023-09-13T04:38:25.621561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:11:57.542149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"peering","last_fresh":"2023-09-13T04:38:04.408074+0000","last_change":"2023-09-13T04:38:23.588693+0000","last_active":"2023-09-13T04:38:23.588693+0000","last_peered":"2023-09-13T04:38:04.408074+0000","last_clean":"2023-09-13T04:38:04.408074+0000","last_became_active":"2023-09-13T04:38:03.608060+0000","last_became_peered":"2023-09-13T04:38:03.608060+0000","last_unstale":"2023-09-13T04:38:04.408074+0000","last_undegraded":"2023-09-13T04:38:04.408074+0000","last_fullsized":"2023-09-13T04:38:04.408074+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,8,9,11],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.597408+0000","last_change":"2023-09-13T04:37:59.243794+0000","last_active":"2023-09-13T04:38:03.597408+0000","last_peered":"2023-09-13T04:38:03.597408+0000","last_clean":"2023-09-13T04:38:03.597408+0000","last_became_active":"2023-09-13T04:37:59.243368+0000","last_became_peered":"2023-09-13T04:37:59.243368+0000","last_unstale":"2023-09-13T04:38:03.597408+0000","last_undegraded":"2023-09-13T04:38:03.597408+0000","last_fullsized":"2023-09-13T04:38:03.597408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:06:45.318737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.245630+0000","last_change":"2023-09-13T04:37:58.589195+0000","last_active":"2023-09-13T04:38:24.245630+0000","last_peered":"2023-09-13T04:38:24.245630+0000","last_clean":"2023-09-13T04:38:24.245630+0000","last_became_active":"2023-09-13T04:37:58.588905+0000","last_became_peered":"2023-09-13T04:37:58.588905+0000","last_unstale":"2023-09-13T04:38:24.245630+0000","last_undegraded":"2023-09-13T04:38:24.245630+0000","last_fullsized":"2023-09-13T04:38:24.245630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:13:15.978891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":325,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.407034+0000","last_change":"2023-09-13T04:36:52.227264+0000","last_active":"2023-09-13T04:38:04.407034+0000","last_peered":"2023-09-13T04:38:04.407034+0000","last_clean":"2023-09-13T04:38:04.407034+0000","last_became_active":"2023-09-13T04:36:52.226773+0000","last_became_peered":"2023-09-13T04:36:52.226773+0000","last_unstale":"2023-09-13T04:38:04.407034+0000","last_undegraded":"2023-09-13T04:38:04.407034+0000","last_fullsized":"2023-09-13T04:38:04.407034+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:33:42.026499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:33:42.026499+0000","last_clean_scrub_stamp":"2023-09-13T04:33:42.026499+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:28:55.806587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592890+0000","last_change":"2023-09-13T04:38:03.742790+0000","last_active":"2023-09-13T04:38:23.592890+0000","last_peered":"2023-09-13T04:38:23.592890+0000","last_clean":"2023-09-13T04:38:23.592890+0000","last_became_active":"2023-09-13T04:38:03.742338+0000","last_became_peered":"2023-09-13T04:38:03.742338+0000","last_unstale":"2023-09-13T04:38:23.592890+0000","last_undegraded":"2023-09-13T04:38:23.592890+0000","last_fullsized":"2023-09-13T04:38:23.592890+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:01:38.910623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597412+0000","last_change":"2023-09-13T04:37:59.252669+0000","last_active":"2023-09-13T04:38:23.597412+0000","last_peered":"2023-09-13T04:38:23.597412+0000","last_clean":"2023-09-13T04:38:23.597412+0000","last_became_active":"2023-09-13T04:37:59.252223+0000","last_became_peered":"2023-09-13T04:37:59.252223+0000","last_unstale":"2023-09-13T04:38:23.597412+0000","last_undegraded":"2023-09-13T04:38:23.597412+0000","last_fullsized":"2023-09-13T04:38:23.597412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:57:21.090083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587916+0000","last_change":"2023-09-13T04:37:58.590540+0000","last_active":"2023-09-13T04:38:23.587916+0000","last_peered":"2023-09-13T04:38:23.587916+0000","last_clean":"2023-09-13T04:38:23.587916+0000","last_became_active":"2023-09-13T04:37:58.590061+0000","last_became_peered":"2023-09-13T04:37:58.590061+0000","last_unstale":"2023-09-13T04:38:23.587916+0000","last_undegraded":"2023-09-13T04:38:23.587916+0000","last_fullsized":"2023-09-13T04:38:23.587916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:57:17.133461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592719+0000","last_change":"2023-09-13T04:38:03.614615+0000","last_active":"2023-09-13T04:38:23.592719+0000","last_peered":"2023-09-13T04:38:23.592719+0000","last_clean":"2023-09-13T04:38:23.592719+0000","last_became_active":"2023-09-13T04:38:03.614086+0000","last_became_peered":"2023-09-13T04:38:03.614086+0000","last_unstale":"2023-09-13T04:38:23.592719+0000","last_undegraded":"2023-09-13T04:38:23.592719+0000","last_fullsized":"2023-09-13T04:38:23.592719+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:25:35.525513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.243719+0000","last_change":"2023-09-13T04:37:58.592096+0000","last_active":"2023-09-13T04:38:24.243719+0000","last_peered":"2023-09-13T04:38:24.243719+0000","last_clean":"2023-09-13T04:38:24.243719+0000","last_became_active":"2023-09-13T04:37:58.591872+0000","last_became_peered":"2023-09-13T04:37:58.591872+0000","last_unstale":"2023-09-13T04:38:24.243719+0000","last_undegraded":"2023-09-13T04:38:24.243719+0000","last_fullsized":"2023-09-13T04:38:24.243719+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:05:23.936105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597816+0000","last_change":"2023-09-13T04:37:59.240759+0000","last_active":"2023-09-13T04:38:23.597816+0000","last_peered":"2023-09-13T04:38:23.597816+0000","last_clean":"2023-09-13T04:38:23.597816+0000","last_became_active":"2023-09-13T04:37:59.240414+0000","last_became_peered":"2023-09-13T04:37:59.240414+0000","last_unstale":"2023-09-13T04:38:23.597816+0000","last_undegraded":"2023-09-13T04:38:23.597816+0000","last_fullsized":"2023-09-13T04:38:23.597816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:37:20.301938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.454201+0000","last_change":"2023-09-13T04:38:03.742519+0000","last_active":"2023-09-13T04:38:04.454201+0000","last_peered":"2023-09-13T04:38:04.454201+0000","last_clean":"2023-09-13T04:38:04.454201+0000","last_became_active":"2023-09-13T04:38:03.742075+0000","last_became_peered":"2023-09-13T04:38:03.742075+0000","last_unstale":"2023-09-13T04:38:04.454201+0000","last_undegraded":"2023-09-13T04:38:04.454201+0000","last_fullsized":"2023-09-13T04:38:04.454201+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:14:33.155837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.619899+0000","last_change":"2023-09-13T04:37:58.593891+0000","last_active":"2023-09-13T04:38:25.619899+0000","last_peered":"2023-09-13T04:38:25.619899+0000","last_clean":"2023-09-13T04:38:25.619899+0000","last_became_active":"2023-09-13T04:37:58.593497+0000","last_became_peered":"2023-09-13T04:37:58.593497+0000","last_unstale":"2023-09-13T04:38:25.619899+0000","last_undegraded":"2023-09-13T04:38:25.619899+0000","last_fullsized":"2023-09-13T04:38:25.619899+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:25:53.652601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.601755+0000","last_change":"2023-09-13T04:37:59.247151+0000","last_active":"2023-09-13T04:38:03.601755+0000","last_peered":"2023-09-13T04:38:03.601755+0000","last_clean":"2023-09-13T04:38:03.601755+0000","last_became_active":"2023-09-13T04:37:59.246853+0000","last_became_peered":"2023-09-13T04:37:59.246853+0000","last_unstale":"2023-09-13T04:38:03.601755+0000","last_undegraded":"2023-09-13T04:38:03.601755+0000","last_fullsized":"2023-09-13T04:38:03.601755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:15:22.659365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587493+0000","last_change":"2023-09-13T04:38:03.606594+0000","last_active":"2023-09-13T04:38:23.587493+0000","last_peered":"2023-09-13T04:38:23.587493+0000","last_clean":"2023-09-13T04:38:23.587493+0000","last_became_active":"2023-09-13T04:38:03.606058+0000","last_became_peered":"2023-09-13T04:38:03.606058+0000","last_unstale":"2023-09-13T04:38:23.587493+0000","last_undegraded":"2023-09-13T04:38:23.587493+0000","last_fullsized":"2023-09-13T04:38:23.587493+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:57:39.237520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.246057+0000","last_change":"2023-09-13T04:37:58.588362+0000","last_active":"2023-09-13T04:38:24.246057+0000","last_peered":"2023-09-13T04:38:24.246057+0000","last_clean":"2023-09-13T04:38:24.246057+0000","last_became_active":"2023-09-13T04:37:58.588065+0000","last_became_peered":"2023-09-13T04:37:58.588065+0000","last_unstale":"2023-09-13T04:38:24.246057+0000","last_undegraded":"2023-09-13T04:38:24.246057+0000","last_fullsized":"2023-09-13T04:38:24.246057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:32:04.255654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406766+0000","last_change":"2023-09-13T04:37:59.251883+0000","last_active":"2023-09-13T04:38:04.406766+0000","last_peered":"2023-09-13T04:38:04.406766+0000","last_clean":"2023-09-13T04:38:04.406766+0000","last_became_active":"2023-09-13T04:37:59.251522+0000","last_became_peered":"2023-09-13T04:37:59.251522+0000","last_unstale":"2023-09-13T04:38:04.406766+0000","last_undegraded":"2023-09-13T04:38:04.406766+0000","last_fullsized":"2023-09-13T04:38:04.406766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:05:06.420799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590729+0000","last_change":"2023-09-13T04:38:03.742726+0000","last_active":"2023-09-13T04:38:23.590729+0000","last_peered":"2023-09-13T04:38:23.590729+0000","last_clean":"2023-09-13T04:38:23.590729+0000","last_became_active":"2023-09-13T04:38:03.742293+0000","last_became_peered":"2023-09-13T04:38:03.742293+0000","last_unstale":"2023-09-13T04:38:23.590729+0000","last_undegraded":"2023-09-13T04:38:23.590729+0000","last_fullsized":"2023-09-13T04:38:23.590729+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:52:15.089764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.243314+0000","last_change":"2023-09-13T04:37:58.584923+0000","last_active":"2023-09-13T04:38:24.243314+0000","last_peered":"2023-09-13T04:38:24.243314+0000","last_clean":"2023-09-13T04:38:24.243314+0000","last_became_active":"2023-09-13T04:37:58.584320+0000","last_became_peered":"2023-09-13T04:37:58.584320+0000","last_unstale":"2023-09-13T04:38:24.243314+0000","last_undegraded":"2023-09-13T04:38:24.243314+0000","last_fullsized":"2023-09-13T04:38:24.243314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:16:53.670534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592586+0000","last_change":"2023-09-13T04:37:59.243029+0000","last_active":"2023-09-13T04:38:23.592586+0000","last_peered":"2023-09-13T04:38:23.592586+0000","last_clean":"2023-09-13T04:38:23.592586+0000","last_became_active":"2023-09-13T04:37:59.242733+0000","last_became_peered":"2023-09-13T04:37:59.242733+0000","last_unstale":"2023-09-13T04:38:23.592586+0000","last_undegraded":"2023-09-13T04:38:23.592586+0000","last_fullsized":"2023-09-13T04:38:23.592586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:48:21.430026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588037+0000","last_change":"2023-09-13T04:38:03.616735+0000","last_active":"2023-09-13T04:38:23.588037+0000","last_peered":"2023-09-13T04:38:23.588037+0000","last_clean":"2023-09-13T04:38:23.588037+0000","last_became_active":"2023-09-13T04:38:03.616275+0000","last_became_peered":"2023-09-13T04:38:03.616275+0000","last_unstale":"2023-09-13T04:38:23.588037+0000","last_undegraded":"2023-09-13T04:38:23.588037+0000","last_fullsized":"2023-09-13T04:38:23.588037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:12:27.267918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.598266+0000","last_change":"2023-09-13T04:37:58.595040+0000","last_active":"2023-09-13T04:38:03.598266+0000","last_peered":"2023-09-13T04:38:03.598266+0000","last_clean":"2023-09-13T04:38:03.598266+0000","last_became_active":"2023-09-13T04:37:58.591403+0000","last_became_peered":"2023-09-13T04:37:58.591403+0000","last_unstale":"2023-09-13T04:38:03.598266+0000","last_undegraded":"2023-09-13T04:38:03.598266+0000","last_fullsized":"2023-09-13T04:38:03.598266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:42:06.561066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592175+0000","last_change":"2023-09-13T04:37:59.248837+0000","last_active":"2023-09-13T04:38:23.592175+0000","last_peered":"2023-09-13T04:38:23.592175+0000","last_clean":"2023-09-13T04:38:23.592175+0000","last_became_active":"2023-09-13T04:37:59.248573+0000","last_became_peered":"2023-09-13T04:37:59.248573+0000","last_unstale":"2023-09-13T04:38:23.592175+0000","last_undegraded":"2023-09-13T04:38:23.592175+0000","last_fullsized":"2023-09-13T04:38:23.592175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:56:22.980484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590405+0000","last_change":"2023-09-13T04:38:03.616370+0000","last_active":"2023-09-13T04:38:23.590405+0000","last_peered":"2023-09-13T04:38:23.590405+0000","last_clean":"2023-09-13T04:38:23.590405+0000","last_became_active":"2023-09-13T04:38:03.615996+0000","last_became_peered":"2023-09-13T04:38:03.615996+0000","last_unstale":"2023-09-13T04:38:23.590405+0000","last_undegraded":"2023-09-13T04:38:23.590405+0000","last_fullsized":"2023-09-13T04:38:23.590405+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:56:59.924205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.590955+0000","last_change":"2023-09-13T04:37:58.597902+0000","last_active":"2023-09-13T04:38:24.590955+0000","last_peered":"2023-09-13T04:38:24.590955+0000","last_clean":"2023-09-13T04:38:24.590955+0000","last_became_active":"2023-09-13T04:37:58.597578+0000","last_became_peered":"2023-09-13T04:37:58.597578+0000","last_unstale":"2023-09-13T04:38:24.590955+0000","last_undegraded":"2023-09-13T04:38:24.590955+0000","last_fullsized":"2023-09-13T04:38:24.590955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:10:09.931949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.604302+0000","last_change":"2023-09-13T04:37:59.248924+0000","last_active":"2023-09-13T04:38:03.604302+0000","last_peered":"2023-09-13T04:38:03.604302+0000","last_clean":"2023-09-13T04:38:03.604302+0000","last_became_active":"2023-09-13T04:37:59.248483+0000","last_became_peered":"2023-09-13T04:37:59.248483+0000","last_unstale":"2023-09-13T04:38:03.604302+0000","last_undegraded":"2023-09-13T04:38:03.604302+0000","last_fullsized":"2023-09-13T04:38:03.604302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:49:56.078544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590322+0000","last_change":"2023-09-13T04:38:03.742260+0000","last_active":"2023-09-13T04:38:23.590322+0000","last_peered":"2023-09-13T04:38:23.590322+0000","last_clean":"2023-09-13T04:38:23.590322+0000","last_became_active":"2023-09-13T04:38:03.741893+0000","last_became_peered":"2023-09-13T04:38:03.741893+0000","last_unstale":"2023-09-13T04:38:23.590322+0000","last_undegraded":"2023-09-13T04:38:23.590322+0000","last_fullsized":"2023-09-13T04:38:23.590322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:59:00.653545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.245764+0000","last_change":"2023-09-13T04:37:58.586898+0000","last_active":"2023-09-13T04:38:24.245764+0000","last_peered":"2023-09-13T04:38:24.245764+0000","last_clean":"2023-09-13T04:38:24.245764+0000","last_became_active":"2023-09-13T04:37:58.586594+0000","last_became_peered":"2023-09-13T04:37:58.586594+0000","last_unstale":"2023-09-13T04:38:24.245764+0000","last_undegraded":"2023-09-13T04:38:24.245764+0000","last_fullsized":"2023-09-13T04:38:24.245764+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:29:43.718522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.407855+0000","last_change":"2023-09-13T04:37:59.244749+0000","last_active":"2023-09-13T04:38:04.407855+0000","last_peered":"2023-09-13T04:38:04.407855+0000","last_clean":"2023-09-13T04:38:04.407855+0000","last_became_active":"2023-09-13T04:37:59.244391+0000","last_became_peered":"2023-09-13T04:37:59.244391+0000","last_unstale":"2023-09-13T04:38:04.407855+0000","last_undegraded":"2023-09-13T04:38:04.407855+0000","last_fullsized":"2023-09-13T04:38:04.407855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:33:02.195729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592524+0000","last_change":"2023-09-13T04:38:03.609766+0000","last_active":"2023-09-13T04:38:23.592524+0000","last_peered":"2023-09-13T04:38:23.592524+0000","last_clean":"2023-09-13T04:38:23.592524+0000","last_became_active":"2023-09-13T04:38:03.609432+0000","last_became_peered":"2023-09-13T04:38:03.609432+0000","last_unstale":"2023-09-13T04:38:23.592524+0000","last_undegraded":"2023-09-13T04:38:23.592524+0000","last_fullsized":"2023-09-13T04:38:23.592524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:30:56.115483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.620943+0000","last_change":"2023-09-13T04:37:58.589119+0000","last_active":"2023-09-13T04:38:25.620943+0000","last_peered":"2023-09-13T04:38:25.620943+0000","last_clean":"2023-09-13T04:38:25.620943+0000","last_became_active":"2023-09-13T04:37:58.588803+0000","last_became_peered":"2023-09-13T04:37:58.588803+0000","last_unstale":"2023-09-13T04:38:25.620943+0000","last_undegraded":"2023-09-13T04:38:25.620943+0000","last_fullsized":"2023-09-13T04:38:25.620943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:51:20.888666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.408147+0000","last_change":"2023-09-13T04:37:59.243506+0000","last_active":"2023-09-13T04:38:04.408147+0000","last_peered":"2023-09-13T04:38:04.408147+0000","last_clean":"2023-09-13T04:38:04.408147+0000","last_became_active":"2023-09-13T04:37:59.243082+0000","last_became_peered":"2023-09-13T04:37:59.243082+0000","last_unstale":"2023-09-13T04:38:04.408147+0000","last_undegraded":"2023-09-13T04:38:04.408147+0000","last_fullsized":"2023-09-13T04:38:04.408147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:01:41.179286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406373+0000","last_change":"2023-09-13T04:38:03.609042+0000","last_active":"2023-09-13T04:38:04.406373+0000","last_peered":"2023-09-13T04:38:04.406373+0000","last_clean":"2023-09-13T04:38:04.406373+0000","last_became_active":"2023-09-13T04:38:03.608553+0000","last_became_peered":"2023-09-13T04:38:03.608553+0000","last_unstale":"2023-09-13T04:38:04.406373+0000","last_undegraded":"2023-09-13T04:38:04.406373+0000","last_fullsized":"2023-09-13T04:38:04.406373+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:05:56.648475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591325+0000","last_change":"2023-09-13T04:37:58.598056+0000","last_active":"2023-09-13T04:38:23.591325+0000","last_peered":"2023-09-13T04:38:23.591325+0000","last_clean":"2023-09-13T04:38:23.591325+0000","last_became_active":"2023-09-13T04:37:58.597724+0000","last_became_peered":"2023-09-13T04:37:58.597724+0000","last_unstale":"2023-09-13T04:38:23.591325+0000","last_undegraded":"2023-09-13T04:38:23.591325+0000","last_fullsized":"2023-09-13T04:38:23.591325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:43:08.296866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.407964+0000","last_change":"2023-09-13T04:37:59.246505+0000","last_active":"2023-09-13T04:38:04.407964+0000","last_peered":"2023-09-13T04:38:04.407964+0000","last_clean":"2023-09-13T04:38:04.407964+0000","last_became_active":"2023-09-13T04:37:59.246024+0000","last_became_peered":"2023-09-13T04:37:59.246024+0000","last_unstale":"2023-09-13T04:38:04.407964+0000","last_undegraded":"2023-09-13T04:38:04.407964+0000","last_fullsized":"2023-09-13T04:38:04.407964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:36:58.008066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590871+0000","last_change":"2023-09-13T04:38:03.605354+0000","last_active":"2023-09-13T04:38:23.590871+0000","last_peered":"2023-09-13T04:38:23.590871+0000","last_clean":"2023-09-13T04:38:23.590871+0000","last_became_active":"2023-09-13T04:38:03.604840+0000","last_became_peered":"2023-09-13T04:38:03.604840+0000","last_unstale":"2023-09-13T04:38:23.590871+0000","last_undegraded":"2023-09-13T04:38:23.590871+0000","last_fullsized":"2023-09-13T04:38:23.590871+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:46:29.867914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597380+0000","last_change":"2023-09-13T04:37:58.589052+0000","last_active":"2023-09-13T04:38:23.597380+0000","last_peered":"2023-09-13T04:38:23.597380+0000","last_clean":"2023-09-13T04:38:23.597380+0000","last_became_active":"2023-09-13T04:37:58.588707+0000","last_became_peered":"2023-09-13T04:37:58.588707+0000","last_unstale":"2023-09-13T04:38:23.597380+0000","last_undegraded":"2023-09-13T04:38:23.597380+0000","last_fullsized":"2023-09-13T04:38:23.597380+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:25:23.966046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.408021+0000","last_change":"2023-09-13T04:37:59.241157+0000","last_active":"2023-09-13T04:38:04.408021+0000","last_peered":"2023-09-13T04:38:04.408021+0000","last_clean":"2023-09-13T04:38:04.408021+0000","last_became_active":"2023-09-13T04:37:59.240857+0000","last_became_peered":"2023-09-13T04:37:59.240857+0000","last_unstale":"2023-09-13T04:38:04.408021+0000","last_undegraded":"2023-09-13T04:38:04.408021+0000","last_fullsized":"2023-09-13T04:38:04.408021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:04:44.986908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592376+0000","last_change":"2023-09-13T04:37:59.242447+0000","last_active":"2023-09-13T04:38:23.592376+0000","last_peered":"2023-09-13T04:38:23.592376+0000","last_clean":"2023-09-13T04:38:23.592376+0000","last_became_active":"2023-09-13T04:37:59.242130+0000","last_became_peered":"2023-09-13T04:37:59.242130+0000","last_unstale":"2023-09-13T04:38:23.592376+0000","last_undegraded":"2023-09-13T04:38:23.592376+0000","last_fullsized":"2023-09-13T04:38:23.592376+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:12:12.844888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.595646+0000","last_change":"2023-09-13T04:38:24.595646+0000","last_active":"2023-09-13T04:38:24.595646+0000","last_peered":"2023-09-13T04:38:24.595646+0000","last_clean":"2023-09-13T04:38:24.595646+0000","last_became_active":"2023-09-13T04:38:24.595295+0000","last_became_peered":"2023-09-13T04:38:24.595295+0000","last_unstale":"2023-09-13T04:38:24.595646+0000","last_undegraded":"2023-09-13T04:38:24.595646+0000","last_fullsized":"2023-09-13T04:38:24.595646+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:24:27.174979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587574+0000","last_change":"2023-09-13T04:37:59.245358+0000","last_active":"2023-09-13T04:38:23.587574+0000","last_peered":"2023-09-13T04:38:23.587574+0000","last_clean":"2023-09-13T04:38:23.587574+0000","last_became_active":"2023-09-13T04:37:59.245052+0000","last_became_peered":"2023-09-13T04:37:59.245052+0000","last_unstale":"2023-09-13T04:38:23.587574+0000","last_undegraded":"2023-09-13T04:38:23.587574+0000","last_fullsized":"2023-09-13T04:38:23.587574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:05:42.561716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-13T04:38:04.406504+0000","last_change":"2023-09-13T04:37:59.254053+0000","last_active":"2023-09-13T04:38:04.406504+0000","last_peered":"2023-09-13T04:38:04.406504+0000","last_clean":"2023-09-13T04:38:04.406504+0000","last_became_active":"2023-09-13T04:37:59.253544+0000","last_became_peered":"2023-09-13T04:37:59.253544+0000","last_unstale":"2023-09-13T04:38:04.406504+0000","last_undegraded":"2023-09-13T04:38:04.406504+0000","last_fullsized":"2023-09-13T04:38:04.406504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:18:54.689208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597920+0000","last_change":"2023-09-13T04:37:59.253403+0000","last_active":"2023-09-13T04:38:23.597920+0000","last_peered":"2023-09-13T04:38:23.597920+0000","last_clean":"2023-09-13T04:38:23.597920+0000","last_became_active":"2023-09-13T04:37:59.253062+0000","last_became_peered":"2023-09-13T04:37:59.253062+0000","last_unstale":"2023-09-13T04:38:23.597920+0000","last_undegraded":"2023-09-13T04:38:23.597920+0000","last_fullsized":"2023-09-13T04:38:23.597920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:31:09.898886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592570+0000","last_change":"2023-09-13T04:37:59.240916+0000","last_active":"2023-09-13T04:38:23.592570+0000","last_peered":"2023-09-13T04:38:23.592570+0000","last_clean":"2023-09-13T04:38:23.592570+0000","last_became_active":"2023-09-13T04:37:59.240605+0000","last_became_peered":"2023-09-13T04:37:59.240605+0000","last_unstale":"2023-09-13T04:38:23.592570+0000","last_undegraded":"2023-09-13T04:38:23.592570+0000","last_fullsized":"2023-09-13T04:38:23.592570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:15:48.241587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588778+0000","last_change":"2023-09-13T04:37:59.238569+0000","last_active":"2023-09-13T04:38:23.588778+0000","last_peered":"2023-09-13T04:38:23.588778+0000","last_clean":"2023-09-13T04:38:23.588778+0000","last_became_active":"2023-09-13T04:37:59.238255+0000","last_became_peered":"2023-09-13T04:37:59.238255+0000","last_unstale":"2023-09-13T04:38:23.588778+0000","last_undegraded":"2023-09-13T04:38:23.588778+0000","last_fullsized":"2023-09-13T04:38:23.588778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:19:03.433404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.601850+0000","last_change":"2023-09-13T04:37:59.252243+0000","last_active":"2023-09-13T04:38:03.601850+0000","last_peered":"2023-09-13T04:38:03.601850+0000","last_clean":"2023-09-13T04:38:03.601850+0000","last_became_active":"2023-09-13T04:37:59.251902+0000","last_became_peered":"2023-09-13T04:37:59.251902+0000","last_unstale":"2023-09-13T04:38:03.601850+0000","last_undegraded":"2023-09-13T04:38:03.601850+0000","last_fullsized":"2023-09-13T04:38:03.601850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:35:57.395627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593414+0000","last_change":"2023-09-13T04:37:59.245847+0000","last_active":"2023-09-13T04:38:23.593414+0000","last_peered":"2023-09-13T04:38:23.593414+0000","last_clean":"2023-09-13T04:38:23.593414+0000","last_became_active":"2023-09-13T04:37:59.245542+0000","last_became_peered":"2023-09-13T04:37:59.245542+0000","last_unstale":"2023-09-13T04:38:23.593414+0000","last_undegraded":"2023-09-13T04:38:23.593414+0000","last_fullsized":"2023-09-13T04:38:23.593414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:24:16.813333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-13T04:38:03.598419+0000","last_change":"2023-09-13T04:37:59.246963+0000","last_active":"2023-09-13T04:38:03.598419+0000","last_peered":"2023-09-13T04:38:03.598419+0000","last_clean":"2023-09-13T04:38:03.598419+0000","last_became_active":"2023-09-13T04:37:59.246448+0000","last_became_peered":"2023-09-13T04:37:59.246448+0000","last_unstale":"2023-09-13T04:38:03.598419+0000","last_undegraded":"2023-09-13T04:38:03.598419+0000","last_fullsized":"2023-09-13T04:38:03.598419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:25:45.418778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594086+0000","last_change":"2023-09-13T04:37:59.245399+0000","last_active":"2023-09-13T04:38:23.594086+0000","last_peered":"2023-09-13T04:38:23.594086+0000","last_clean":"2023-09-13T04:38:23.594086+0000","last_became_active":"2023-09-13T04:37:59.245060+0000","last_became_peered":"2023-09-13T04:37:59.245060+0000","last_unstale":"2023-09-13T04:38:23.594086+0000","last_undegraded":"2023-09-13T04:38:23.594086+0000","last_fullsized":"2023-09-13T04:38:23.594086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:25:41.078238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597807+0000","last_change":"2023-09-13T04:37:59.244776+0000","last_active":"2023-09-13T04:38:23.597807+0000","last_peered":"2023-09-13T04:38:23.597807+0000","last_clean":"2023-09-13T04:38:23.597807+0000","last_became_active":"2023-09-13T04:37:59.244409+0000","last_became_peered":"2023-09-13T04:37:59.244409+0000","last_unstale":"2023-09-13T04:38:23.597807+0000","last_undegraded":"2023-09-13T04:38:23.597807+0000","last_fullsized":"2023-09-13T04:38:23.597807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:04:02.561414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592391+0000","last_change":"2023-09-13T04:37:59.248153+0000","last_active":"2023-09-13T04:38:23.592391+0000","last_peered":"2023-09-13T04:38:23.592391+0000","last_clean":"2023-09-13T04:38:23.592391+0000","last_became_active":"2023-09-13T04:37:59.247840+0000","last_became_peered":"2023-09-13T04:37:59.247840+0000","last_unstale":"2023-09-13T04:38:23.592391+0000","last_undegraded":"2023-09-13T04:38:23.592391+0000","last_fullsized":"2023-09-13T04:38:23.592391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:52:53.021219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6628,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":105,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":18262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90,"ondisk_log_size":90,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":490076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.52100000000000002}]},{"osd":1,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.495},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.63700000000000001}]},{"osd":2,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.59599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":4,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.79000000000000004}]},{"osd":5,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.77500000000000002}]},{"osd":6,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.75800000000000001}]},{"osd":7,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.86399999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":9,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.878}]},{"osd":10,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.92400000000000004}]}]},{"osd":10,"up_from":71,"seq":304942678037,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1626112,"data_stored":1084927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":1,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.42299999999999999}]},{"osd":2,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.57199999999999995}]},{"osd":3,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.626},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.754}]},{"osd":4,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.44600000000000001}]},{"osd":5,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.69899999999999995}]},{"osd":6,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":3.4129999999999998,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.84499999999999997}]},{"osd":7,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.64500000000000002}]},{"osd":8,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.59399999999999997}]},{"osd":9,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.80500000000000005}]},{"osd":11,"last update":"Wed Sep 13 04:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.67800000000000005}]}]},{"osd":9,"up_from":66,"seq":283467841562,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":490860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.36299999999999999}]},{"osd":1,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.56499999999999995}]},{"osd":2,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.73499999999999999}]},{"osd":3,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.45200000000000001}]},{"osd":4,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.63},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.59199999999999997}]},{"osd":5,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.75}]},{"osd":6,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.72099999999999997}]},{"osd":7,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.60899999999999999}]},{"osd":8,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.64800000000000002}]},{"osd":10,"last update":"Wed Sep 13 04:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.68300000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]}]},{"osd":8,"up_from":58,"seq":249108103198,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":490337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.33200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56999999999999995,"15min":0.56499999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.68100000000000005}]},{"osd":1,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.54800000000000004,"15min":0.53600000000000003},"min":{"1min":0.38400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.58199999999999996},"min":{"1min":0.374,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.78600000000000003}]},{"osd":2,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.47499999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.48899999999999999,"15min":0.48599999999999999},"min":{"1min":0.29799999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.48399999999999999}]},{"osd":3,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.434},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64600000000000002,"15min":0.64300000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.61799999999999999}]},{"osd":4,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58899999999999997,"15min":0.58399999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.41099999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.73699999999999999}]},{"osd":5,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.34499999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80700000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.54400000000000004}]},{"osd":6,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.88200000000000001}]},{"osd":7,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71199999999999997,"15min":0.70999999999999996},"min":{"1min":0.48399999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69099999999999995,"15min":0.68700000000000006},"min":{"1min":0.503,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.81200000000000006}]},{"osd":9,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.85399999999999998}]},{"osd":10,"last update":"Wed Sep 13 04:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.82499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]}]},{"osd":7,"up_from":51,"seq":219043332131,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1564672,"data_stored":1061916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.46500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.746,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60499999999999998,"15min":0.60799999999999998},"min":{"1min":0.41599999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.40200000000000002}]},{"osd":1,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.40899999999999997,"15min":0.40300000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42099999999999999,"15min":0.41999999999999998},"min":{"1min":0.30199999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65800000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.34200000000000003}]},{"osd":2,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.42599999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83899999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.57799999999999996,"15min":0.57099999999999995},"min":{"1min":0.443,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.70599999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.53700000000000003}]},{"osd":3,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.5},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66600000000000004,"15min":0.66900000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82799999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.52100000000000002}]},{"osd":4,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76600000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51200000000000001,"15min":0.51500000000000001},"min":{"1min":0.34499999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.376}]},{"osd":5,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.628,"15min":0.626},"min":{"1min":0.47799999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64600000000000002,"15min":0.64200000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.60299999999999998}]},{"osd":6,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72199999999999998,"15min":0.72099999999999997},"min":{"1min":0.50600000000000001,"5min":0.496,"15min":0.496},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.62},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.48799999999999999}]},{"osd":8,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70299999999999996,"15min":0.70399999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.51700000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.88100000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.68200000000000005}]},{"osd":9,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.72299999999999998}]},{"osd":10,"last update":"Wed Sep 13 04:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.434},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.56499999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":490132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46100000000000002,"15min":0.45100000000000001},"min":{"1min":0.34699999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.503,"15min":0.48999999999999999},"min":{"1min":0.38100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.42999999999999999}]},{"osd":1,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.47099999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82499999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.70999999999999996,"15min":0.67900000000000005},"min":{"1min":0.60399999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.94699999999999995}]},{"osd":2,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.52400000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0129999999999999,"5min":1.054,"15min":1.054},"last":0.83199999999999996}]},{"osd":3,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.66100000000000003,"15min":0.64100000000000001},"min":{"1min":0.47799999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83599999999999997,"5min":1.113,"15min":1.113},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.626,"15min":0.61799999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89900000000000002,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.496}]},{"osd":4,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76400000000000001,"15min":0.747},"min":{"1min":0.67900000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0509999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.75900000000000001,"15min":0.77100000000000002},"min":{"1min":0.52600000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92900000000000005,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.61599999999999999}]},{"osd":5,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.80800000000000005,"15min":0.79200000000000004},"min":{"1min":0.56100000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1950000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.79200000000000004,"15min":0.80800000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.80200000000000005}]},{"osd":7,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.83999999999999997,"15min":0.82399999999999995},"min":{"1min":0.63500000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.79800000000000004,"15min":0.78800000000000003},"min":{"1min":0.58999999999999997,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.57899999999999996}]},{"osd":8,"last update":"Wed Sep 13 04:38:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.81599999999999995,"15min":0.82399999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96599999999999997,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.91900000000000004,"15min":0.92400000000000004},"min":{"1min":0.61199999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2629999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.032}]},{"osd":9,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.0089999999999999}]},{"osd":10,"last update":"Wed Sep 13 04:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.754}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":493884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55600000000000005,"15min":0.53700000000000003},"min":{"1min":0.33700000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56999999999999995,"15min":0.55200000000000005},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.85199999999999998}]},{"osd":2,"last update":"Wed Sep 13 04:37:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.55900000000000005,"15min":0.53800000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.54800000000000004,"15min":0.51800000000000002},"min":{"1min":0.42299999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.73199999999999998}]},{"osd":3,"last update":"Wed Sep 13 04:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83099999999999996,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.623,"15min":0.60899999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91600000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67800000000000005}]},{"osd":4,"last update":"Wed Sep 13 04:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.46200000000000002,"15min":0.45300000000000001},"min":{"1min":0.29599999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.45800000000000002,"15min":0.44800000000000001},"min":{"1min":0.29199999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.497}]},{"osd":5,"last update":"Wed Sep 13 04:37:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.44800000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89000000000000001,"5min":1.3,"15min":1.3},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61499999999999999,"15min":0.58299999999999996},"min":{"1min":0.435,"5min":0.37,"15min":0.37},"max":{"1min":1,"5min":1.264,"15min":1.264},"last":0.78800000000000003}]},{"osd":6,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.99199999999999999,"5min":1.482,"15min":1.482},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95399999999999996,"5min":1.375,"15min":1.375},"last":0.71699999999999997}]},{"osd":7,"last update":"Wed Sep 13 04:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.60699999999999998,"15min":0.58699999999999997},"min":{"1min":0.39900000000000002,"5min":0.372,"15min":0.372},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.65900000000000003,"15min":0.64300000000000002},"min":{"1min":0.47399999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.75800000000000001}]},{"osd":8,"last update":"Wed Sep 13 04:38:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.48899999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2010000000000001,"5min":1.393,"15min":1.393},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.093,"5min":1.224,"15min":1.224},"last":0.77100000000000002}]},{"osd":9,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.81100000000000005}]},{"osd":10,"last update":"Wed Sep 13 04:37:53 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.878}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":480566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 13 04:37:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":0.66000000000000003,"15min":0.58299999999999996},"min":{"1min":0.42199999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.2359999999999998,"5min":6.2359999999999998,"15min":6.2359999999999998},"last":0.745},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.58699999999999997,"15min":0.55200000000000005},"min":{"1min":0.47899999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.77300000000000002}]},{"osd":2,"last update":"Wed Sep 13 04:37:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.624,"15min":0.59899999999999998},"min":{"1min":0.42899999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93899999999999995,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63700000000000001,"15min":0.60299999999999998},"min":{"1min":0.439,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.96899999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.82399999999999995}]},{"osd":3,"last update":"Wed Sep 13 04:36:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.57099999999999995,"15min":0.55500000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.86499999999999999}]},{"osd":4,"last update":"Wed Sep 13 04:37:30 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.65900000000000003,"15min":0.61299999999999999},"min":{"1min":0.58699999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":1.079},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.64900000000000002,"15min":0.621},"min":{"1min":0.51400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.84599999999999997}]},{"osd":5,"last update":"Wed Sep 13 04:38:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68300000000000005,"15min":0.67000000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.70699999999999996,"15min":0.68300000000000005},"min":{"1min":0.55100000000000005,"5min":0.435,"15min":0.435},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.91900000000000004}]},{"osd":6,"last update":"Wed Sep 13 04:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.44600000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94899999999999995,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88700000000000001,"5min":1.165,"15min":1.165},"last":0.79800000000000004}]},{"osd":7,"last update":"Wed Sep 13 04:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.79500000000000004,"15min":0.78500000000000003},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.81599999999999995,"15min":0.80200000000000005},"min":{"1min":0.60099999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":1.26}]},{"osd":8,"last update":"Wed Sep 13 04:38:00 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":0.90600000000000003,"15min":0.85999999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":6.2089999999999996,"5min":6.2089999999999996,"15min":6.2089999999999996},"last":0.874},{"interface":"front","average":{"1min":1.171,"5min":0.87,"15min":0.81999999999999995},"min":{"1min":0.52400000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":6.1699999999999999,"5min":6.1699999999999999,"15min":6.1699999999999999},"last":1.143}]},{"osd":9,"last update":"Wed Sep 13 04:37:30 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":1.171}]},{"osd":10,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":6.1859999999999999,"5min":6.1859999999999999,"15min":6.1859999999999999},"last":0.996}]},{"osd":11,"last update":"Wed Sep 13 04:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.2330000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":1.044}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":495200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:55 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.34599999999999997,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79100000000000004,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51900000000000002,"15min":0.51800000000000002},"min":{"1min":0.371,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78300000000000003,"5min":1.429,"15min":1.429},"last":0.45100000000000001}]},{"osd":1,"last update":"Wed Sep 13 04:37:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.53500000000000003,"15min":0.496},"min":{"1min":0.39000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80700000000000005,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54300000000000004,"15min":0.50700000000000001},"min":{"1min":0.39100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70799999999999996,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.755}]},{"osd":3,"last update":"Wed Sep 13 04:38:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.36499999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.747,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.441,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81399999999999995,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.66600000000000004}]},{"osd":4,"last update":"Wed Sep 13 04:37:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.52700000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78000000000000003,"5min":1.244,"15min":1.244},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55800000000000005,"15min":0.53000000000000003},"min":{"1min":0.436,"5min":0.374,"15min":0.374},"max":{"1min":0.85499999999999998,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.79400000000000004}]},{"osd":5,"last update":"Wed Sep 13 04:37:57 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.28699999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.70899999999999996,"5min":1.796,"15min":1.796},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.30599999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.99299999999999999,"5min":1.546,"15min":1.546},"last":0.49399999999999999}]},{"osd":6,"last update":"Wed Sep 13 04:38:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.69199999999999995,"15min":0.69899999999999995},"min":{"1min":0.48499999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81599999999999995,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66700000000000004,"15min":0.67200000000000004},"min":{"1min":0.48199999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85899999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.77100000000000002}]},{"osd":7,"last update":"Wed Sep 13 04:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65400000000000003,"15min":0.65800000000000003},"min":{"1min":0.42499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94699999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.497,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92500000000000004,"5min":1.147,"15min":1.147},"last":0.88300000000000001}]},{"osd":8,"last update":"Wed Sep 13 04:38:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61699999999999999,"15min":0.61899999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.96599999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.38400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.82699999999999996}]},{"osd":9,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.84199999999999997}]},{"osd":10,"last update":"Wed Sep 13 04:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.98199999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":974848,"data_stored":471070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.42699999999999999,"15min":0.39000000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.46500000000000002,"15min":0.435},"min":{"1min":0.375,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78500000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.38400000000000001}]},{"osd":1,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.57699999999999996},"min":{"1min":0.45700000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64200000000000002,"15min":0.57699999999999996},"min":{"1min":0.436,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92200000000000004,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.69799999999999995}]},{"osd":2,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.67400000000000004,"15min":0.629},"min":{"1min":0.53000000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64500000000000002,"15min":0.621},"min":{"1min":0.45800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.55100000000000005}]},{"osd":4,"last update":"Wed Sep 13 04:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.64300000000000002,"15min":0.60099999999999998},"min":{"1min":0.52900000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.64100000000000001,"15min":0.60699999999999998},"min":{"1min":0.44500000000000001,"5min":0.439,"15min":0.439},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.68799999999999994}]},{"osd":5,"last update":"Wed Sep 13 04:38:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.68999999999999995,"15min":0.65000000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.024,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.873,"5min":0.72399999999999998,"15min":0.66600000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":0.71199999999999997}]},{"osd":6,"last update":"Wed Sep 13 04:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.71599999999999997,"15min":0.69499999999999995},"min":{"1min":0.51200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.75900000000000001,"15min":0.74199999999999999},"min":{"1min":0.52400000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.622}]},{"osd":7,"last update":"Wed Sep 13 04:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.86299999999999999,"15min":0.86399999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.85399999999999998,"15min":0.84999999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.80500000000000005}]},{"osd":8,"last update":"Wed Sep 13 04:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.875}]},{"osd":9,"last update":"Wed Sep 13 04:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.63800000000000001}]},{"osd":10,"last update":"Wed Sep 13 04:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.77200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":490064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.56599999999999995,"15min":0.54400000000000004},"min":{"1min":0.43099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.753},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76500000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.68999999999999995}]},{"osd":1,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.373,"15min":0.34699999999999998},"min":{"1min":0.28399999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.38700000000000001,"15min":0.36399999999999999},"min":{"1min":0.28100000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.48499999999999999}]},{"osd":2,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58299999999999996,"15min":0.56100000000000005},"min":{"1min":0.41099999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56599999999999995,"15min":0.53800000000000003},"min":{"1min":0.41399999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.64300000000000002}]},{"osd":3,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.59399999999999997,"15min":0.56499999999999995},"min":{"1min":0.41799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59399999999999997,"15min":0.57699999999999996},"min":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.68400000000000005}]},{"osd":5,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60299999999999998,"15min":0.58599999999999997},"min":{"1min":0.45100000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90300000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.39700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.75800000000000001}]},{"osd":6,"last update":"Wed Sep 13 04:38:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.68000000000000005,"15min":0.68400000000000005},"min":{"1min":0.47099999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81599999999999995,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.69799999999999995,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.84599999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52400000000000002,"15min":0.51900000000000002},"min":{"1min":0.36199999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73599999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.70099999999999996}]},{"osd":8,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.70699999999999996,"15min":0.69199999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76700000000000002,"15min":0.76300000000000001},"min":{"1min":0.53000000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.77600000000000002}]},{"osd":9,"last update":"Wed Sep 13 04:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.86199999999999999}]},{"osd":10,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.82899999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1622016,"data_stored":1081886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53000000000000003,"15min":0.52400000000000002},"min":{"1min":0.34399999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72099999999999997,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.317,"5min":0.315,"15min":0.315},"max":{"1min":0.70299999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.84799999999999998}]},{"osd":1,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.40300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78500000000000003,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.54300000000000004,"15min":0.52400000000000002},"min":{"1min":0.34799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89600000000000002,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.60399999999999998}]},{"osd":2,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.38,"15min":0.36099999999999999},"min":{"1min":0.26900000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.37},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.39600000000000002,"15min":0.378},"min":{"1min":0.28299999999999997,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.41199999999999998}]},{"osd":3,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.372,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92600000000000005,"5min":1.036,"15min":1.036},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.78700000000000003,"15min":0.86899999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.1259999999999999,"5min":8.0009999999999994,"15min":8.0009999999999994},"last":0.77700000000000002}]},{"osd":4,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.58299999999999996,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82299999999999995,"5min":1.599,"15min":1.599},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.57699999999999996,"15min":0.55500000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.86599999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.623}]},{"osd":6,"last update":"Wed Sep 13 04:38:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83499999999999996,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81899999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.72999999999999998}]},{"osd":7,"last update":"Wed Sep 13 04:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.878,"5min":1.488,"15min":1.488},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94899999999999995,"5min":1.51,"15min":1.51},"last":0.63900000000000001}]},{"osd":8,"last update":"Wed Sep 13 04:38:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.38900000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.60599999999999998,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.56999999999999995}]},{"osd":9,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.70099999999999996}]},{"osd":10,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.68500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-13T04:38:29.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.31 is not active+clean 2023-09-13T04:38:29.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 16, 'reported_epoch': 88, 'state': 'peering', 'last_fresh': '2023-09-13T04:38:04.406264+0000', 'last_change': '2023-09-13T04:38:23.591313+0000', 'last_active': '2023-09-13T04:38:23.591313+0000', 'last_peered': '2023-09-13T04:38:04.406264+0000', 'last_clean': '2023-09-13T04:38:04.406264+0000', 'last_became_active': '2023-09-13T04:37:59.252899+0000', 'last_became_peered': '2023-09-13T04:37:59.252899+0000', 'last_unstale': '2023-09-13T04:38:04.406264+0000', 'last_undegraded': '2023-09-13T04:38:04.406264+0000', 'last_fullsized': '2023-09-13T04:38:04.406264+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 83, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-13T04:37:58.565164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-13T04:37:58.565164+0000', 'last_clean_scrub_stamp': '2023-09-13T04:37:58.565164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6, 2], 'acting': [0, 6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2, 4, 6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-09-13T04:38:29.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.5 is not active+clean 2023-09-13T04:38:29.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.5', 'version': "0'0", 'reported_seq': 14, 'reported_epoch': 88, 'state': 'peering', 'last_fresh': '2023-09-13T04:38:04.408074+0000', 'last_change': '2023-09-13T04:38:23.588693+0000', 'last_active': '2023-09-13T04:38:23.588693+0000', 'last_peered': '2023-09-13T04:38:04.408074+0000', 'last_clean': '2023-09-13T04:38:04.408074+0000', 'last_became_active': '2023-09-13T04:38:03.608060+0000', 'last_became_peered': '2023-09-13T04:38:03.608060+0000', 'last_unstale': '2023-09-13T04:38:04.408074+0000', 'last_undegraded': '2023-09-13T04:38:04.408074+0000', 'last_fullsized': '2023-09-13T04:38:04.408074+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 86, 'last_epoch_clean': 87, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-13T04:38:02.587125+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-13T04:38:02.587125+0000', 'last_clean_scrub_stamp': '2023-09-13T04:38:02.587125+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 8, 11, 9], 'acting': [7, 8, 11, 9], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5, 8, 9, 11], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-09-13T04:38:29.064 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-09-13T04:38:29.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.297+0000 7f15b57ff700 1 -- 172.21.15.2:0/1229735539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15b0100cf0 msgr2=0x7f15b0101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:29.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.297+0000 7f15b57ff700 1 --2- 172.21.15.2:0/1229735539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15b0100cf0 0x7f15b0101110 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f15a00099c0 tx=0x7f15a0009cd0 comp rx=0 tx=0).stop 2023-09-13T04:38:29.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.297+0000 7f15b57ff700 1 -- 172.21.15.2:0/1229735539 shutdown_connections 2023-09-13T04:38:29.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.297+0000 7f15b57ff700 1 --2- 172.21.15.2:0/1229735539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15b0102880 0x7f15b0109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:29.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.297+0000 7f15b57ff700 1 --2- 172.21.15.2:0/1229735539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15b0100cf0 0x7f15b0101110 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:29.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.297+0000 7f15b57ff700 1 --2- 172.21.15.2:0/1229735539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15b0101ee0 0x7f15b0102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:29.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.297+0000 7f15b57ff700 1 -- 172.21.15.2:0/1229735539 >> 172.21.15.2:0/1229735539 conn(0x7f15b00fc220 msgr2=0x7f15b00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:29.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.297+0000 7f15b57ff700 1 -- 172.21.15.2:0/1229735539 shutdown_connections 2023-09-13T04:38:29.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.297+0000 7f15b57ff700 1 -- 172.21.15.2:0/1229735539 wait complete. 2023-09-13T04:38:29.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.298+0000 7f15b57ff700 1 Processor -- start 2023-09-13T04:38:29.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.298+0000 7f15b57ff700 1 -- start start 2023-09-13T04:38:29.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.298+0000 7f15b57ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15b0100cf0 0x7f15b0198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:29.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.298+0000 7f15b57ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15b0101ee0 0x7f15b0199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:29.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.298+0000 7f15b57ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15b0102880 0x7f15b01a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:29.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.298+0000 7f15b57ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b01a0750 con 0x7f15b0102880 2023-09-13T04:38:29.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.298+0000 7f15b57ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b01a0890 con 0x7f15b0101ee0 2023-09-13T04:38:29.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.298+0000 7f15b57ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b01a0a20 con 0x7f15b0100cf0 2023-09-13T04:38:29.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.299+0000 7f15af7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15b0102880 0x7f15b01a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:29.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.299+0000 7f15af7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15b0102880 0x7f15b01a0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38160/0 (socket says 172.21.15.2:38160) 2023-09-13T04:38:29.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.299+0000 7f15af7fe700 1 -- 172.21.15.2:0/1394934962 learned_addr learned my addr 172.21.15.2:0/1394934962 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:29.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.299+0000 7f15aeffd700 1 --2- 172.21.15.2:0/1394934962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15b0100cf0 0x7f15b0198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:29.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.299+0000 7f15ae7fc700 1 --2- 172.21.15.2:0/1394934962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15b0101ee0 0x7f15b0199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:29.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.299+0000 7f15af7fe700 1 -- 172.21.15.2:0/1394934962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15b0100cf0 msgr2=0x7f15b0198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:29.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.299+0000 7f15af7fe700 1 --2- 172.21.15.2:0/1394934962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15b0100cf0 0x7f15b0198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:29.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.299+0000 7f15af7fe700 1 -- 172.21.15.2:0/1394934962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15b0101ee0 msgr2=0x7f15b0199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:29.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.299+0000 7f15af7fe700 1 --2- 172.21.15.2:0/1394934962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15b0101ee0 0x7f15b0199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:29.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.299+0000 7f15af7fe700 1 -- 172.21.15.2:0/1394934962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a00095f0 con 0x7f15b0102880 2023-09-13T04:38:29.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.299+0000 7f15af7fe700 1 --2- 172.21.15.2:0/1394934962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15b0102880 0x7f15b01a0180 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f15a800ef30 tx=0x7f15a800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:29.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.299+0000 7f1597fff700 1 -- 172.21.15.2:0/1394934962 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a8018070 con 0x7f15b0102880 2023-09-13T04:38:29.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.300+0000 7f15b57ff700 1 -- 172.21.15.2:0/1394934962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b01a0ca0 con 0x7f15b0102880 2023-09-13T04:38:29.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.300+0000 7f1597fff700 1 -- 172.21.15.2:0/1394934962 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15a800f040 con 0x7f15b0102880 2023-09-13T04:38:29.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.300+0000 7f1597fff700 1 -- 172.21.15.2:0/1394934962 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a800cd00 con 0x7f15b0102880 2023-09-13T04:38:29.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.300+0000 7f15b57ff700 1 -- 172.21.15.2:0/1394934962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b01a1160 con 0x7f15b0102880 2023-09-13T04:38:29.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.301+0000 7f15b57ff700 1 -- 172.21.15.2:0/1394934962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b0193340 con 0x7f15b0102880 2023-09-13T04:38:29.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.303+0000 7f1597fff700 1 -- 172.21.15.2:0/1394934962 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f15a8003a00 con 0x7f15b0102880 2023-09-13T04:38:29.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.304+0000 7f1597fff700 1 --2- 172.21.15.2:0/1394934962 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f15980a0690 0x7f15980a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:29.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.304+0000 7f1597fff700 1 -- 172.21.15.2:0/1394934962 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f15a80cc410 con 0x7f15b0102880 2023-09-13T04:38:29.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.305+0000 7f15aeffd700 1 --2- 172.21.15.2:0/1394934962 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f15980a0690 0x7f15980a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:29.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.305+0000 7f15aeffd700 1 --2- 172.21.15.2:0/1394934962 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f15980a0690 0x7f15980a2b50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f15a0009990 tx=0x7f15a0005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:29.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.306+0000 7f1597fff700 1 -- 172.21.15.2:0/1394934962 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f15a8095c60 con 0x7f15b0102880 2023-09-13T04:38:29.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.607+0000 7f15b57ff700 1 -- 172.21.15.2:0/1394934962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f15b002cd70 con 0x7f15b0102880 2023-09-13T04:38:29.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.607+0000 7f1597fff700 1 -- 172.21.15.2:0/1394934962 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3184 (secure 0 0 0) 0x7f15a8099280 con 0x7f15b0102880 2023-09-13T04:38:29.609 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:38:29.609 INFO:teuthology.orchestra.run.smithi002.stdout:{"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":368,"monmap":{"epoch":3,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":90,"num_osds":12,"num_up_osds":12,"osd_up_since":1694579829,"num_in_osds":12,"osd_in_since":1694579812,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":127},{"state_name":"peering","count":2}],"num_pgs":129,"num_pools":4,"num_objects":49,"data_bytes":596996,"bytes_used":345903104,"bytes_avail":1151594524672,"bytes_total":1151940427776,"inactive_pgs_ratio":0.015503875911235809,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":4,"recovering_keys_per_sec":1,"num_objects_recovered":6,"num_bytes_recovered":34,"num_keys_recovered":10,"read_bytes_sec":255,"write_bytes_sec":2175,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":23,"id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"f","status":"up:standby-replay","gid":24511},{"filesystem_id":1,"rank":1,"name":"h","status":"up:standby-replay","gid":24521},{"filesystem_id":1,"rank":0,"name":"c","status":"up:standby-replay","gid":24523},{"filesystem_id":1,"rank":2,"name":"i","status":"up:active","gid":24541},{"filesystem_id":1,"rank":1,"name":"a","status":"up:active","gid":24554},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24566}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.2:8443/"}},"servicemap":{"epoch":19,"modified":"2023-09-13T04:38:25.505562+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"g":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"j":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"k":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-09-13T04:38:29.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.610+0000 7f15b57ff700 1 -- 172.21.15.2:0/1394934962 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f15980a0690 msgr2=0x7f15980a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:29.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.610+0000 7f15b57ff700 1 --2- 172.21.15.2:0/1394934962 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f15980a0690 0x7f15980a2b50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f15a0009990 tx=0x7f15a0005fb0 comp rx=0 tx=0).stop 2023-09-13T04:38:29.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.610+0000 7f15b57ff700 1 -- 172.21.15.2:0/1394934962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15b0102880 msgr2=0x7f15b01a0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:29.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.610+0000 7f15b57ff700 1 --2- 172.21.15.2:0/1394934962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15b0102880 0x7f15b01a0180 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f15a800ef30 tx=0x7f15a800c3a0 comp rx=0 tx=0).stop 2023-09-13T04:38:29.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.610+0000 7f15b57ff700 1 -- 172.21.15.2:0/1394934962 shutdown_connections 2023-09-13T04:38:29.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.610+0000 7f15b57ff700 1 --2- 172.21.15.2:0/1394934962 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f15980a0690 0x7f15980a2b50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:29.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.610+0000 7f15b57ff700 1 --2- 172.21.15.2:0/1394934962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15b0102880 0x7f15b01a0180 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:29.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.610+0000 7f15b57ff700 1 --2- 172.21.15.2:0/1394934962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15b0100cf0 0x7f15b0198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:29.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.610+0000 7f15b57ff700 1 --2- 172.21.15.2:0/1394934962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15b0101ee0 0x7f15b0199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:29.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.610+0000 7f15b57ff700 1 -- 172.21.15.2:0/1394934962 >> 172.21.15.2:0/1394934962 conn(0x7f15b00fc220 msgr2=0x7f15b01077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:29.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.610+0000 7f15b57ff700 1 -- 172.21.15.2:0/1394934962 shutdown_connections 2023-09-13T04:38:29.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.610+0000 7f15b57ff700 1 -- 172.21.15.2:0/1394934962 wait complete. 2023-09-13T04:38:29.631 INFO:tasks.cephfs.filesystem.ceph_manager:making progress, resetting timeout 2023-09-13T04:38:29.632 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-13T04:38:29.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.836+0000 7f27013ff700 1 -- 172.21.15.2:0/3135844171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26fc102880 msgr2=0x7f26fc109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:29.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.836+0000 7f27013ff700 1 --2- 172.21.15.2:0/3135844171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26fc102880 0x7f26fc109130 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f26f400b0d0 tx=0x7f26f400b3e0 comp rx=0 tx=0).stop 2023-09-13T04:38:29.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.836+0000 7f27013ff700 1 -- 172.21.15.2:0/3135844171 shutdown_connections 2023-09-13T04:38:29.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.836+0000 7f27013ff700 1 --2- 172.21.15.2:0/3135844171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26fc102880 0x7f26fc109130 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:29.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.836+0000 7f27013ff700 1 --2- 172.21.15.2:0/3135844171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26fc100cf0 0x7f26fc101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:29.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.836+0000 7f27013ff700 1 --2- 172.21.15.2:0/3135844171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26fc101ee0 0x7f26fc102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:29.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.836+0000 7f27013ff700 1 -- 172.21.15.2:0/3135844171 >> 172.21.15.2:0/3135844171 conn(0x7f26fc0fc220 msgr2=0x7f26fc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:29.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.836+0000 7f27013ff700 1 -- 172.21.15.2:0/3135844171 shutdown_connections 2023-09-13T04:38:29.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.836+0000 7f27013ff700 1 -- 172.21.15.2:0/3135844171 wait complete. 2023-09-13T04:38:29.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.837+0000 7f27013ff700 1 Processor -- start 2023-09-13T04:38:29.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.837+0000 7f27013ff700 1 -- start start 2023-09-13T04:38:29.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.838+0000 7f27013ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26fc100cf0 0x7f26fc074a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:29.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.838+0000 7f27013ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26fc101ee0 0x7f26fc0730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:29.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.838+0000 7f27013ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26fc102880 0x7f26fc0735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:29.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.838+0000 7f27013ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc073d80 con 0x7f26fc100cf0 2023-09-13T04:38:29.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.838+0000 7f27013ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc073ef0 con 0x7f26fc102880 2023-09-13T04:38:29.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.838+0000 7f27013ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc1a0b00 con 0x7f26fc101ee0 2023-09-13T04:38:29.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.838+0000 7f26faffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26fc100cf0 0x7f26fc074a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:29.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.838+0000 7f26faffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26fc100cf0 0x7f26fc074a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38172/0 (socket says 172.21.15.2:38172) 2023-09-13T04:38:29.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.838+0000 7f26faffd700 1 -- 172.21.15.2:0/2345453121 learned_addr learned my addr 172.21.15.2:0/2345453121 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:29.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.838+0000 7f26fa7fc700 1 --2- 172.21.15.2:0/2345453121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26fc101ee0 0x7f26fc0730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:29.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.838+0000 7f26fb7fe700 1 --2- 172.21.15.2:0/2345453121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26fc102880 0x7f26fc0735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:29.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.838+0000 7f26faffd700 1 -- 172.21.15.2:0/2345453121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26fc101ee0 msgr2=0x7f26fc0730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:29.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.838+0000 7f26faffd700 1 --2- 172.21.15.2:0/2345453121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26fc101ee0 0x7f26fc0730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:29.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.838+0000 7f26faffd700 1 -- 172.21.15.2:0/2345453121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26fc102880 msgr2=0x7f26fc0735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:29.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.838+0000 7f26faffd700 1 --2- 172.21.15.2:0/2345453121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26fc102880 0x7f26fc0735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:29.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.838+0000 7f26faffd700 1 -- 172.21.15.2:0/2345453121 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f4009d00 con 0x7f26fc100cf0 2023-09-13T04:38:29.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.839+0000 7f26faffd700 1 --2- 172.21.15.2:0/2345453121 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26fc100cf0 0x7f26fc074a60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00e970 tx=0x7f26ec00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:29.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.839+0000 7f26e3fff700 1 -- 172.21.15.2:0/2345453121 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec009790 con 0x7f26fc100cf0 2023-09-13T04:38:29.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.839+0000 7f26e3fff700 1 -- 172.21.15.2:0/2345453121 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26ec00c7e0 con 0x7f26fc100cf0 2023-09-13T04:38:29.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.839+0000 7f26e3fff700 1 -- 172.21.15.2:0/2345453121 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec003c70 con 0x7f26fc100cf0 2023-09-13T04:38:29.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.839+0000 7f27013ff700 1 -- 172.21.15.2:0/2345453121 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26fc1a0de0 con 0x7f26fc100cf0 2023-09-13T04:38:29.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.839+0000 7f27013ff700 1 -- 172.21.15.2:0/2345453121 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26fc1a1210 con 0x7f26fc100cf0 2023-09-13T04:38:29.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.840+0000 7f27013ff700 1 -- 172.21.15.2:0/2345453121 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26fc1081f0 con 0x7f26fc100cf0 2023-09-13T04:38:29.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.842+0000 7f26e3fff700 1 -- 172.21.15.2:0/2345453121 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f26ec00cc50 con 0x7f26fc100cf0 2023-09-13T04:38:29.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.844+0000 7f26e3fff700 1 --2- 172.21.15.2:0/2345453121 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f26e40a0710 0x7f26e40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:29.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.844+0000 7f26e3fff700 1 -- 172.21.15.2:0/2345453121 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f26ec014070 con 0x7f26fc100cf0 2023-09-13T04:38:29.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.844+0000 7f26fa7fc700 1 --2- 172.21.15.2:0/2345453121 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f26e40a0710 0x7f26e40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:29.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.845+0000 7f26fa7fc700 1 --2- 172.21.15.2:0/2345453121 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f26e40a0710 0x7f26e40a2bd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f26f0007be0 tx=0x7f26f00073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:29.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:29.848+0000 7f26e3fff700 1 -- 172.21.15.2:0/2345453121 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f26ec095840 con 0x7f26fc100cf0 2023-09-13T04:38:29.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:29 smithi149 ceph-mon[133315]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:38:29.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:29 smithi149 ceph-mon[133315]: from='client.24821 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:38:29.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:29 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1394934962' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-09-13T04:38:29.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:29 smithi156 ceph-mon[133675]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:38:29.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:29 smithi156 ceph-mon[133675]: from='client.24821 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:38:29.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:29 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1394934962' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-09-13T04:38:30.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:29 smithi002 ceph-mon[97651]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:38:30.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:29 smithi002 ceph-mon[97651]: from='client.24821 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:38:30.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:29 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1394934962' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-09-13T04:38:30.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:30.057+0000 7f27013ff700 1 -- 172.21.15.2:0/2345453121 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f26fc10c560 con 0x7f26e40a0710 2023-09-13T04:38:30.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:30.070+0000 7f26e3fff700 1 -- 172.21.15.2:0/2345453121 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422719 (secure 0 0 0) 0x7f26fc10c560 con 0x7f26e40a0710 2023-09-13T04:38:30.076 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:38:30.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:30.075+0000 7f27013ff700 1 -- 172.21.15.2:0/2345453121 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f26e40a0710 msgr2=0x7f26e40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:30.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:30.075+0000 7f27013ff700 1 --2- 172.21.15.2:0/2345453121 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f26e40a0710 0x7f26e40a2bd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f26f0007be0 tx=0x7f26f00073d0 comp rx=0 tx=0).stop 2023-09-13T04:38:30.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:30.075+0000 7f27013ff700 1 -- 172.21.15.2:0/2345453121 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26fc100cf0 msgr2=0x7f26fc074a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:30.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:30.075+0000 7f27013ff700 1 --2- 172.21.15.2:0/2345453121 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26fc100cf0 0x7f26fc074a60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00e970 tx=0x7f26ec00ec80 comp rx=0 tx=0).stop 2023-09-13T04:38:30.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:30.075+0000 7f27013ff700 1 -- 172.21.15.2:0/2345453121 shutdown_connections 2023-09-13T04:38:30.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:30.075+0000 7f27013ff700 1 --2- 172.21.15.2:0/2345453121 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f26e40a0710 0x7f26e40a2bd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:30.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:30.075+0000 7f27013ff700 1 --2- 172.21.15.2:0/2345453121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26fc102880 0x7f26fc0735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:30.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:30.075+0000 7f27013ff700 1 --2- 172.21.15.2:0/2345453121 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26fc100cf0 0x7f26fc074a60 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:30.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:30.075+0000 7f27013ff700 1 --2- 172.21.15.2:0/2345453121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26fc101ee0 0x7f26fc0730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:30.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:30.075+0000 7f27013ff700 1 -- 172.21.15.2:0/2345453121 >> 172.21.15.2:0/2345453121 conn(0x7f26fc0fc220 msgr2=0x7f26fc105330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:30.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:30.075+0000 7f27013ff700 1 -- 172.21.15.2:0/2345453121 shutdown_connections 2023-09-13T04:38:30.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:30.075+0000 7f27013ff700 1 -- 172.21.15.2:0/2345453121 wait complete. 2023-09-13T04:38:30.082 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-09-13T04:38:30.095 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2023-09-13T04:38:29.506476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597086,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":138,"num_read_kb":121,"num_write":293,"num_write_kb":2971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":415,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337908,"kb_used_data":13656,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603916,"statfs":{"total":1151940427776,"available":1151594409984,"internally_reserved":0,"allocated":13983744,"data_stored":7671414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2219,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":64,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002525"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592832+0000","last_change":"2023-09-13T04:37:59.244279+0000","last_active":"2023-09-13T04:38:23.592832+0000","last_peered":"2023-09-13T04:38:23.592832+0000","last_clean":"2023-09-13T04:38:23.592832+0000","last_became_active":"2023-09-13T04:37:59.243976+0000","last_became_peered":"2023-09-13T04:37:59.243976+0000","last_unstale":"2023-09-13T04:38:23.592832+0000","last_undegraded":"2023-09-13T04:38:23.592832+0000","last_fullsized":"2023-09-13T04:38:23.592832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:21:52.809411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592579+0000","last_change":"2023-09-13T04:37:59.243391+0000","last_active":"2023-09-13T04:38:23.592579+0000","last_peered":"2023-09-13T04:38:23.592579+0000","last_clean":"2023-09-13T04:38:23.592579+0000","last_became_active":"2023-09-13T04:37:59.243044+0000","last_became_peered":"2023-09-13T04:37:59.243044+0000","last_unstale":"2023-09-13T04:38:23.592579+0000","last_undegraded":"2023-09-13T04:38:23.592579+0000","last_fullsized":"2023-09-13T04:38:23.592579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:02:17.806638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592756+0000","last_change":"2023-09-13T04:37:59.242826+0000","last_active":"2023-09-13T04:38:23.592756+0000","last_peered":"2023-09-13T04:38:23.592756+0000","last_clean":"2023-09-13T04:38:23.592756+0000","last_became_active":"2023-09-13T04:37:59.242125+0000","last_became_peered":"2023-09-13T04:37:59.242125+0000","last_unstale":"2023-09-13T04:38:23.592756+0000","last_undegraded":"2023-09-13T04:38:23.592756+0000","last_fullsized":"2023-09-13T04:38:23.592756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:08:29.216093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590536+0000","last_change":"2023-09-13T04:37:59.253322+0000","last_active":"2023-09-13T04:38:23.590536+0000","last_peered":"2023-09-13T04:38:23.590536+0000","last_clean":"2023-09-13T04:38:23.590536+0000","last_became_active":"2023-09-13T04:37:59.253050+0000","last_became_peered":"2023-09-13T04:37:59.253050+0000","last_unstale":"2023-09-13T04:38:23.590536+0000","last_undegraded":"2023-09-13T04:38:23.590536+0000","last_fullsized":"2023-09-13T04:38:23.590536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:45:22.357879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593487+0000","last_change":"2023-09-13T04:37:59.250089+0000","last_active":"2023-09-13T04:38:23.593487+0000","last_peered":"2023-09-13T04:38:23.593487+0000","last_clean":"2023-09-13T04:38:23.593487+0000","last_became_active":"2023-09-13T04:37:59.249771+0000","last_became_peered":"2023-09-13T04:37:59.249771+0000","last_unstale":"2023-09-13T04:38:23.593487+0000","last_undegraded":"2023-09-13T04:38:23.593487+0000","last_fullsized":"2023-09-13T04:38:23.593487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:46:59.917756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595139+0000","last_change":"2023-09-13T04:37:59.247259+0000","last_active":"2023-09-13T04:38:23.595139+0000","last_peered":"2023-09-13T04:38:23.595139+0000","last_clean":"2023-09-13T04:38:23.595139+0000","last_became_active":"2023-09-13T04:37:59.246911+0000","last_became_peered":"2023-09-13T04:37:59.246911+0000","last_unstale":"2023-09-13T04:38:23.595139+0000","last_undegraded":"2023-09-13T04:38:23.595139+0000","last_fullsized":"2023-09-13T04:38:23.595139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:00:39.742392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587857+0000","last_change":"2023-09-13T04:37:59.244083+0000","last_active":"2023-09-13T04:38:23.587857+0000","last_peered":"2023-09-13T04:38:23.587857+0000","last_clean":"2023-09-13T04:38:23.587857+0000","last_became_active":"2023-09-13T04:37:59.243789+0000","last_became_peered":"2023-09-13T04:37:59.243789+0000","last_unstale":"2023-09-13T04:38:23.587857+0000","last_undegraded":"2023-09-13T04:38:23.587857+0000","last_fullsized":"2023-09-13T04:38:23.587857+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:24:00.622790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.602097+0000","last_change":"2023-09-13T04:38:24.601265+0000","last_active":"2023-09-13T04:38:24.602097+0000","last_peered":"2023-09-13T04:38:24.602097+0000","last_clean":"2023-09-13T04:38:24.602097+0000","last_became_active":"2023-09-13T04:38:24.600939+0000","last_became_peered":"2023-09-13T04:38:24.600939+0000","last_unstale":"2023-09-13T04:38:24.602097+0000","last_undegraded":"2023-09-13T04:38:24.602097+0000","last_fullsized":"2023-09-13T04:38:24.602097+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:18:23.642333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594759+0000","last_change":"2023-09-13T04:37:59.238053+0000","last_active":"2023-09-13T04:38:23.594759+0000","last_peered":"2023-09-13T04:38:23.594759+0000","last_clean":"2023-09-13T04:38:23.594759+0000","last_became_active":"2023-09-13T04:37:59.237777+0000","last_became_peered":"2023-09-13T04:37:59.237777+0000","last_unstale":"2023-09-13T04:38:23.594759+0000","last_undegraded":"2023-09-13T04:38:23.594759+0000","last_fullsized":"2023-09-13T04:38:23.594759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:45:39.835659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594929+0000","last_change":"2023-09-13T04:37:59.252605+0000","last_active":"2023-09-13T04:38:23.594929+0000","last_peered":"2023-09-13T04:38:23.594929+0000","last_clean":"2023-09-13T04:38:23.594929+0000","last_became_active":"2023-09-13T04:37:59.252309+0000","last_became_peered":"2023-09-13T04:37:59.252309+0000","last_unstale":"2023-09-13T04:38:23.594929+0000","last_undegraded":"2023-09-13T04:38:23.594929+0000","last_fullsized":"2023-09-13T04:38:23.594929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:41:45.064230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588277+0000","last_change":"2023-09-13T04:37:59.243345+0000","last_active":"2023-09-13T04:38:23.588277+0000","last_peered":"2023-09-13T04:38:23.588277+0000","last_clean":"2023-09-13T04:38:23.588277+0000","last_became_active":"2023-09-13T04:37:59.242726+0000","last_became_peered":"2023-09-13T04:37:59.242726+0000","last_unstale":"2023-09-13T04:38:23.588277+0000","last_undegraded":"2023-09-13T04:38:23.588277+0000","last_fullsized":"2023-09-13T04:38:23.588277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:39:55.115281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590601+0000","last_change":"2023-09-13T04:37:59.247415+0000","last_active":"2023-09-13T04:38:23.590601+0000","last_peered":"2023-09-13T04:38:23.590601+0000","last_clean":"2023-09-13T04:38:23.590601+0000","last_became_active":"2023-09-13T04:37:59.247050+0000","last_became_peered":"2023-09-13T04:37:59.247050+0000","last_unstale":"2023-09-13T04:38:23.590601+0000","last_undegraded":"2023-09-13T04:38:23.590601+0000","last_fullsized":"2023-09-13T04:38:23.590601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:03:44.793628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597902+0000","last_change":"2023-09-13T04:37:59.248935+0000","last_active":"2023-09-13T04:38:23.597902+0000","last_peered":"2023-09-13T04:38:23.597902+0000","last_clean":"2023-09-13T04:38:23.597902+0000","last_became_active":"2023-09-13T04:37:59.248554+0000","last_became_peered":"2023-09-13T04:37:59.248554+0000","last_unstale":"2023-09-13T04:38:23.597902+0000","last_undegraded":"2023-09-13T04:38:23.597902+0000","last_fullsized":"2023-09-13T04:38:23.597902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:53:31.882567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592020+0000","last_change":"2023-09-13T04:37:59.242851+0000","last_active":"2023-09-13T04:38:23.592020+0000","last_peered":"2023-09-13T04:38:23.592020+0000","last_clean":"2023-09-13T04:38:23.592020+0000","last_became_active":"2023-09-13T04:37:59.242384+0000","last_became_peered":"2023-09-13T04:37:59.242384+0000","last_unstale":"2023-09-13T04:38:23.592020+0000","last_undegraded":"2023-09-13T04:38:23.592020+0000","last_fullsized":"2023-09-13T04:38:23.592020+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:59:36.508808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.599392+0000","last_change":"2023-09-13T04:38:24.599392+0000","last_active":"2023-09-13T04:38:24.599392+0000","last_peered":"2023-09-13T04:38:24.599392+0000","last_clean":"2023-09-13T04:38:24.599392+0000","last_became_active":"2023-09-13T04:38:24.599029+0000","last_became_peered":"2023-09-13T04:38:24.599029+0000","last_unstale":"2023-09-13T04:38:24.599392+0000","last_undegraded":"2023-09-13T04:38:24.599392+0000","last_fullsized":"2023-09-13T04:38:24.599392+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:23:03.703387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592657+0000","last_change":"2023-09-13T04:37:59.249107+0000","last_active":"2023-09-13T04:38:23.592657+0000","last_peered":"2023-09-13T04:38:23.592657+0000","last_clean":"2023-09-13T04:38:23.592657+0000","last_became_active":"2023-09-13T04:37:59.248803+0000","last_became_peered":"2023-09-13T04:37:59.248803+0000","last_unstale":"2023-09-13T04:38:23.592657+0000","last_undegraded":"2023-09-13T04:38:23.592657+0000","last_fullsized":"2023-09-13T04:38:23.592657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:50:52.481194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592743+0000","last_change":"2023-09-13T04:37:59.243724+0000","last_active":"2023-09-13T04:38:23.592743+0000","last_peered":"2023-09-13T04:38:23.592743+0000","last_clean":"2023-09-13T04:38:23.592743+0000","last_became_active":"2023-09-13T04:37:59.243385+0000","last_became_peered":"2023-09-13T04:37:59.243385+0000","last_unstale":"2023-09-13T04:38:23.592743+0000","last_undegraded":"2023-09-13T04:38:23.592743+0000","last_fullsized":"2023-09-13T04:38:23.592743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:55:55.203940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588550+0000","last_change":"2023-09-13T04:37:59.238986+0000","last_active":"2023-09-13T04:38:23.588550+0000","last_peered":"2023-09-13T04:38:23.588550+0000","last_clean":"2023-09-13T04:38:23.588550+0000","last_became_active":"2023-09-13T04:37:59.238668+0000","last_became_peered":"2023-09-13T04:37:59.238668+0000","last_unstale":"2023-09-13T04:38:23.588550+0000","last_undegraded":"2023-09-13T04:38:23.588550+0000","last_fullsized":"2023-09-13T04:38:23.588550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:21:10.225223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588585+0000","last_change":"2023-09-13T04:37:59.245598+0000","last_active":"2023-09-13T04:38:23.588585+0000","last_peered":"2023-09-13T04:38:23.588585+0000","last_clean":"2023-09-13T04:38:23.588585+0000","last_became_active":"2023-09-13T04:37:59.244652+0000","last_became_peered":"2023-09-13T04:37:59.244652+0000","last_unstale":"2023-09-13T04:38:23.588585+0000","last_undegraded":"2023-09-13T04:38:23.588585+0000","last_fullsized":"2023-09-13T04:38:23.588585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:34:43.671168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588238+0000","last_change":"2023-09-13T04:38:03.617708+0000","last_active":"2023-09-13T04:38:23.588238+0000","last_peered":"2023-09-13T04:38:23.588238+0000","last_clean":"2023-09-13T04:38:23.588238+0000","last_became_active":"2023-09-13T04:38:03.617277+0000","last_became_peered":"2023-09-13T04:38:03.617277+0000","last_unstale":"2023-09-13T04:38:23.588238+0000","last_undegraded":"2023-09-13T04:38:23.588238+0000","last_fullsized":"2023-09-13T04:38:23.588238+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:26:44.318030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.621267+0000","last_change":"2023-09-13T04:37:58.595156+0000","last_active":"2023-09-13T04:38:25.621267+0000","last_peered":"2023-09-13T04:38:25.621267+0000","last_clean":"2023-09-13T04:38:25.621267+0000","last_became_active":"2023-09-13T04:37:58.594863+0000","last_became_peered":"2023-09-13T04:37:58.594863+0000","last_unstale":"2023-09-13T04:38:25.621267+0000","last_undegraded":"2023-09-13T04:38:25.621267+0000","last_fullsized":"2023-09-13T04:38:25.621267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:27:14.420495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595813+0000","last_change":"2023-09-13T04:37:59.250252+0000","last_active":"2023-09-13T04:38:23.595813+0000","last_peered":"2023-09-13T04:38:23.595813+0000","last_clean":"2023-09-13T04:38:23.595813+0000","last_became_active":"2023-09-13T04:37:59.249927+0000","last_became_peered":"2023-09-13T04:37:59.249927+0000","last_unstale":"2023-09-13T04:38:23.595813+0000","last_undegraded":"2023-09-13T04:38:23.595813+0000","last_fullsized":"2023-09-13T04:38:23.595813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:18:06.757040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.599683+0000","last_change":"2023-09-13T04:38:24.599683+0000","last_active":"2023-09-13T04:38:24.599683+0000","last_peered":"2023-09-13T04:38:24.599683+0000","last_clean":"2023-09-13T04:38:24.599683+0000","last_became_active":"2023-09-13T04:38:24.599230+0000","last_became_peered":"2023-09-13T04:38:24.599230+0000","last_unstale":"2023-09-13T04:38:24.599683+0000","last_undegraded":"2023-09-13T04:38:24.599683+0000","last_fullsized":"2023-09-13T04:38:24.599683+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:48:22.584735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.596874+0000","last_change":"2023-09-13T04:37:58.597785+0000","last_active":"2023-09-13T04:38:24.596874+0000","last_peered":"2023-09-13T04:38:24.596874+0000","last_clean":"2023-09-13T04:38:24.596874+0000","last_became_active":"2023-09-13T04:37:58.597485+0000","last_became_peered":"2023-09-13T04:37:58.597485+0000","last_unstale":"2023-09-13T04:38:24.596874+0000","last_undegraded":"2023-09-13T04:38:24.596874+0000","last_fullsized":"2023-09-13T04:38:24.596874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:39:57.156038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592830+0000","last_change":"2023-09-13T04:37:59.241840+0000","last_active":"2023-09-13T04:38:23.592830+0000","last_peered":"2023-09-13T04:38:23.592830+0000","last_clean":"2023-09-13T04:38:23.592830+0000","last_became_active":"2023-09-13T04:37:59.241545+0000","last_became_peered":"2023-09-13T04:37:59.241545+0000","last_unstale":"2023-09-13T04:38:23.592830+0000","last_undegraded":"2023-09-13T04:38:23.592830+0000","last_fullsized":"2023-09-13T04:38:23.592830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:04:17.439520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591101+0000","last_change":"2023-09-13T04:38:03.621792+0000","last_active":"2023-09-13T04:38:23.591101+0000","last_peered":"2023-09-13T04:38:23.591101+0000","last_clean":"2023-09-13T04:38:23.591101+0000","last_became_active":"2023-09-13T04:38:03.621433+0000","last_became_peered":"2023-09-13T04:38:03.621433+0000","last_unstale":"2023-09-13T04:38:23.591101+0000","last_undegraded":"2023-09-13T04:38:23.591101+0000","last_fullsized":"2023-09-13T04:38:23.591101+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:52:05.926733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594362+0000","last_change":"2023-09-13T04:37:58.590817+0000","last_active":"2023-09-13T04:38:23.594362+0000","last_peered":"2023-09-13T04:38:23.594362+0000","last_clean":"2023-09-13T04:38:23.594362+0000","last_became_active":"2023-09-13T04:37:58.590348+0000","last_became_peered":"2023-09-13T04:37:58.590348+0000","last_unstale":"2023-09-13T04:38:23.594362+0000","last_undegraded":"2023-09-13T04:38:23.594362+0000","last_fullsized":"2023-09-13T04:38:23.594362+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:38:35.680905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594252+0000","last_change":"2023-09-13T04:37:59.237304+0000","last_active":"2023-09-13T04:38:23.594252+0000","last_peered":"2023-09-13T04:38:23.594252+0000","last_clean":"2023-09-13T04:38:23.594252+0000","last_became_active":"2023-09-13T04:37:59.237008+0000","last_became_peered":"2023-09-13T04:37:59.237008+0000","last_unstale":"2023-09-13T04:38:23.594252+0000","last_undegraded":"2023-09-13T04:38:23.594252+0000","last_fullsized":"2023-09-13T04:38:23.594252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:28:14.254219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590193+0000","last_change":"2023-09-13T04:38:03.606749+0000","last_active":"2023-09-13T04:38:23.590193+0000","last_peered":"2023-09-13T04:38:23.590193+0000","last_clean":"2023-09-13T04:38:23.590193+0000","last_became_active":"2023-09-13T04:38:03.606276+0000","last_became_peered":"2023-09-13T04:38:03.606276+0000","last_unstale":"2023-09-13T04:38:23.590193+0000","last_undegraded":"2023-09-13T04:38:23.590193+0000","last_fullsized":"2023-09-13T04:38:23.590193+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:30:53.572543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593692+0000","last_change":"2023-09-13T04:37:58.590937+0000","last_active":"2023-09-13T04:38:23.593692+0000","last_peered":"2023-09-13T04:38:23.593692+0000","last_clean":"2023-09-13T04:38:23.593692+0000","last_became_active":"2023-09-13T04:37:58.590606+0000","last_became_peered":"2023-09-13T04:37:58.590606+0000","last_unstale":"2023-09-13T04:38:23.593692+0000","last_undegraded":"2023-09-13T04:38:23.593692+0000","last_fullsized":"2023-09-13T04:38:23.593692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:30:03.354670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591224+0000","last_change":"2023-09-13T04:37:59.251899+0000","last_active":"2023-09-13T04:38:23.591224+0000","last_peered":"2023-09-13T04:38:23.591224+0000","last_clean":"2023-09-13T04:38:23.591224+0000","last_became_active":"2023-09-13T04:37:59.251557+0000","last_became_peered":"2023-09-13T04:37:59.251557+0000","last_unstale":"2023-09-13T04:38:23.591224+0000","last_undegraded":"2023-09-13T04:38:23.591224+0000","last_fullsized":"2023-09-13T04:38:23.591224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:57:45.241188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588439+0000","last_change":"2023-09-13T04:38:03.606076+0000","last_active":"2023-09-13T04:38:23.588439+0000","last_peered":"2023-09-13T04:38:23.588439+0000","last_clean":"2023-09-13T04:38:23.588439+0000","last_became_active":"2023-09-13T04:38:03.605556+0000","last_became_peered":"2023-09-13T04:38:03.605556+0000","last_unstale":"2023-09-13T04:38:23.588439+0000","last_undegraded":"2023-09-13T04:38:23.588439+0000","last_fullsized":"2023-09-13T04:38:23.588439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:03:32.344625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.246702+0000","last_change":"2023-09-13T04:37:58.588170+0000","last_active":"2023-09-13T04:38:24.246702+0000","last_peered":"2023-09-13T04:38:24.246702+0000","last_clean":"2023-09-13T04:38:24.246702+0000","last_became_active":"2023-09-13T04:37:58.587860+0000","last_became_peered":"2023-09-13T04:37:58.587860+0000","last_unstale":"2023-09-13T04:38:24.246702+0000","last_undegraded":"2023-09-13T04:38:24.246702+0000","last_fullsized":"2023-09-13T04:38:24.246702+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:38:40.048150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595472+0000","last_change":"2023-09-13T04:37:59.240248+0000","last_active":"2023-09-13T04:38:23.595472+0000","last_peered":"2023-09-13T04:38:23.595472+0000","last_clean":"2023-09-13T04:38:23.595472+0000","last_became_active":"2023-09-13T04:37:59.240008+0000","last_became_peered":"2023-09-13T04:37:59.240008+0000","last_unstale":"2023-09-13T04:38:23.595472+0000","last_undegraded":"2023-09-13T04:38:23.595472+0000","last_fullsized":"2023-09-13T04:38:23.595472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:15:13.275110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587821+0000","last_change":"2023-09-13T04:38:03.616607+0000","last_active":"2023-09-13T04:38:23.587821+0000","last_peered":"2023-09-13T04:38:23.587821+0000","last_clean":"2023-09-13T04:38:23.587821+0000","last_became_active":"2023-09-13T04:38:03.616209+0000","last_became_peered":"2023-09-13T04:38:03.616209+0000","last_unstale":"2023-09-13T04:38:23.587821+0000","last_undegraded":"2023-09-13T04:38:23.587821+0000","last_fullsized":"2023-09-13T04:38:23.587821+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:24:21.934053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595102+0000","last_change":"2023-09-13T04:37:58.589104+0000","last_active":"2023-09-13T04:38:23.595102+0000","last_peered":"2023-09-13T04:38:23.595102+0000","last_clean":"2023-09-13T04:38:23.595102+0000","last_became_active":"2023-09-13T04:37:58.588780+0000","last_became_peered":"2023-09-13T04:37:58.588780+0000","last_unstale":"2023-09-13T04:38:23.595102+0000","last_undegraded":"2023-09-13T04:38:23.595102+0000","last_fullsized":"2023-09-13T04:38:23.595102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:01:32.839729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.600030+0000","last_change":"2023-09-13T04:38:24.598663+0000","last_active":"2023-09-13T04:38:24.600030+0000","last_peered":"2023-09-13T04:38:24.600030+0000","last_clean":"2023-09-13T04:38:24.600030+0000","last_became_active":"2023-09-13T04:38:24.598258+0000","last_became_peered":"2023-09-13T04:38:24.598258+0000","last_unstale":"2023-09-13T04:38:24.600030+0000","last_undegraded":"2023-09-13T04:38:24.600030+0000","last_fullsized":"2023-09-13T04:38:24.600030+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:50:28.157135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592633+0000","last_change":"2023-09-13T04:38:03.619930+0000","last_active":"2023-09-13T04:38:23.592633+0000","last_peered":"2023-09-13T04:38:23.592633+0000","last_clean":"2023-09-13T04:38:23.592633+0000","last_became_active":"2023-09-13T04:38:03.619560+0000","last_became_peered":"2023-09-13T04:38:03.619560+0000","last_unstale":"2023-09-13T04:38:23.592633+0000","last_undegraded":"2023-09-13T04:38:23.592633+0000","last_fullsized":"2023-09-13T04:38:23.592633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:48:06.867602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.619713+0000","last_change":"2023-09-13T04:37:58.590396+0000","last_active":"2023-09-13T04:38:25.619713+0000","last_peered":"2023-09-13T04:38:25.619713+0000","last_clean":"2023-09-13T04:38:25.619713+0000","last_became_active":"2023-09-13T04:37:58.589929+0000","last_became_peered":"2023-09-13T04:37:58.589929+0000","last_unstale":"2023-09-13T04:38:25.619713+0000","last_undegraded":"2023-09-13T04:38:25.619713+0000","last_fullsized":"2023-09-13T04:38:25.619713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:28:36.322096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597620+0000","last_change":"2023-09-13T04:37:59.246033+0000","last_active":"2023-09-13T04:38:23.597620+0000","last_peered":"2023-09-13T04:38:23.597620+0000","last_clean":"2023-09-13T04:38:23.597620+0000","last_became_active":"2023-09-13T04:37:59.245664+0000","last_became_peered":"2023-09-13T04:37:59.245664+0000","last_unstale":"2023-09-13T04:38:23.597620+0000","last_undegraded":"2023-09-13T04:38:23.597620+0000","last_fullsized":"2023-09-13T04:38:23.597620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:24:17.145055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592344+0000","last_change":"2023-09-13T04:38:03.608504+0000","last_active":"2023-09-13T04:38:23.592344+0000","last_peered":"2023-09-13T04:38:23.592344+0000","last_clean":"2023-09-13T04:38:23.592344+0000","last_became_active":"2023-09-13T04:38:03.608058+0000","last_became_peered":"2023-09-13T04:38:03.608058+0000","last_unstale":"2023-09-13T04:38:23.592344+0000","last_undegraded":"2023-09-13T04:38:23.592344+0000","last_fullsized":"2023-09-13T04:38:23.592344+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:42:43.485453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244691+0000","last_change":"2023-09-13T04:37:58.588208+0000","last_active":"2023-09-13T04:38:24.244691+0000","last_peered":"2023-09-13T04:38:24.244691+0000","last_clean":"2023-09-13T04:38:24.244691+0000","last_became_active":"2023-09-13T04:37:58.587840+0000","last_became_peered":"2023-09-13T04:37:58.587840+0000","last_unstale":"2023-09-13T04:38:24.244691+0000","last_undegraded":"2023-09-13T04:38:24.244691+0000","last_fullsized":"2023-09-13T04:38:24.244691+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:44:58.012233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587683+0000","last_change":"2023-09-13T04:37:59.245102+0000","last_active":"2023-09-13T04:38:23.587683+0000","last_peered":"2023-09-13T04:38:23.587683+0000","last_clean":"2023-09-13T04:38:23.587683+0000","last_became_active":"2023-09-13T04:37:59.244799+0000","last_became_peered":"2023-09-13T04:37:59.244799+0000","last_unstale":"2023-09-13T04:38:23.587683+0000","last_undegraded":"2023-09-13T04:38:23.587683+0000","last_fullsized":"2023-09-13T04:38:23.587683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:57:16.295623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.596645+0000","last_change":"2023-09-13T04:38:03.607021+0000","last_active":"2023-09-13T04:38:23.596645+0000","last_peered":"2023-09-13T04:38:23.596645+0000","last_clean":"2023-09-13T04:38:23.596645+0000","last_became_active":"2023-09-13T04:38:03.606611+0000","last_became_peered":"2023-09-13T04:38:03.606611+0000","last_unstale":"2023-09-13T04:38:23.596645+0000","last_undegraded":"2023-09-13T04:38:23.596645+0000","last_fullsized":"2023-09-13T04:38:23.596645+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:33:20.551479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244009+0000","last_change":"2023-09-13T04:37:58.588731+0000","last_active":"2023-09-13T04:38:24.244009+0000","last_peered":"2023-09-13T04:38:24.244009+0000","last_clean":"2023-09-13T04:38:24.244009+0000","last_became_active":"2023-09-13T04:37:58.588398+0000","last_became_peered":"2023-09-13T04:37:58.588398+0000","last_unstale":"2023-09-13T04:38:24.244009+0000","last_undegraded":"2023-09-13T04:38:24.244009+0000","last_fullsized":"2023-09-13T04:38:24.244009+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:13:10.724085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594790+0000","last_change":"2023-09-13T04:37:59.244757+0000","last_active":"2023-09-13T04:38:23.594790+0000","last_peered":"2023-09-13T04:38:23.594790+0000","last_clean":"2023-09-13T04:38:23.594790+0000","last_became_active":"2023-09-13T04:37:59.244423+0000","last_became_peered":"2023-09-13T04:37:59.244423+0000","last_unstale":"2023-09-13T04:38:23.594790+0000","last_undegraded":"2023-09-13T04:38:23.594790+0000","last_fullsized":"2023-09-13T04:38:23.594790+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:14:37.973058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587303+0000","last_change":"2023-09-13T04:38:03.618358+0000","last_active":"2023-09-13T04:38:23.587303+0000","last_peered":"2023-09-13T04:38:23.587303+0000","last_clean":"2023-09-13T04:38:23.587303+0000","last_became_active":"2023-09-13T04:38:03.617291+0000","last_became_peered":"2023-09-13T04:38:03.617291+0000","last_unstale":"2023-09-13T04:38:23.587303+0000","last_undegraded":"2023-09-13T04:38:23.587303+0000","last_fullsized":"2023-09-13T04:38:23.587303+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:33:13.955353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244519+0000","last_change":"2023-09-13T04:37:58.590978+0000","last_active":"2023-09-13T04:38:24.244519+0000","last_peered":"2023-09-13T04:38:24.244519+0000","last_clean":"2023-09-13T04:38:24.244519+0000","last_became_active":"2023-09-13T04:37:58.590673+0000","last_became_peered":"2023-09-13T04:37:58.590673+0000","last_unstale":"2023-09-13T04:38:24.244519+0000","last_undegraded":"2023-09-13T04:38:24.244519+0000","last_fullsized":"2023-09-13T04:38:24.244519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:10:32.570029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592737+0000","last_change":"2023-09-13T04:37:59.251283+0000","last_active":"2023-09-13T04:38:23.592737+0000","last_peered":"2023-09-13T04:38:23.592737+0000","last_clean":"2023-09-13T04:38:23.592737+0000","last_became_active":"2023-09-13T04:37:59.250119+0000","last_became_peered":"2023-09-13T04:37:59.250119+0000","last_unstale":"2023-09-13T04:38:23.592737+0000","last_undegraded":"2023-09-13T04:38:23.592737+0000","last_fullsized":"2023-09-13T04:38:23.592737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:48:54.389970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.596107+0000","last_change":"2023-09-13T04:38:24.596107+0000","last_active":"2023-09-13T04:38:24.596107+0000","last_peered":"2023-09-13T04:38:24.596107+0000","last_clean":"2023-09-13T04:38:24.596107+0000","last_became_active":"2023-09-13T04:38:24.595653+0000","last_became_peered":"2023-09-13T04:38:24.595653+0000","last_unstale":"2023-09-13T04:38:24.596107+0000","last_undegraded":"2023-09-13T04:38:24.596107+0000","last_fullsized":"2023-09-13T04:38:24.596107+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:02:53.708658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.243198+0000","last_change":"2023-09-13T04:37:58.584909+0000","last_active":"2023-09-13T04:38:24.243198+0000","last_peered":"2023-09-13T04:38:24.243198+0000","last_clean":"2023-09-13T04:38:24.243198+0000","last_became_active":"2023-09-13T04:37:58.584630+0000","last_became_peered":"2023-09-13T04:37:58.584630+0000","last_unstale":"2023-09-13T04:38:24.243198+0000","last_undegraded":"2023-09-13T04:38:24.243198+0000","last_fullsized":"2023-09-13T04:38:24.243198+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:57:56.567499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594349+0000","last_change":"2023-09-13T04:37:59.254448+0000","last_active":"2023-09-13T04:38:23.594349+0000","last_peered":"2023-09-13T04:38:23.594349+0000","last_clean":"2023-09-13T04:38:23.594349+0000","last_became_active":"2023-09-13T04:37:59.254138+0000","last_became_peered":"2023-09-13T04:37:59.254138+0000","last_unstale":"2023-09-13T04:38:23.594349+0000","last_undegraded":"2023-09-13T04:38:23.594349+0000","last_fullsized":"2023-09-13T04:38:23.594349+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:36:14.939881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588357+0000","last_change":"2023-09-13T04:38:03.613746+0000","last_active":"2023-09-13T04:38:23.588357+0000","last_peered":"2023-09-13T04:38:23.588357+0000","last_clean":"2023-09-13T04:38:23.588357+0000","last_became_active":"2023-09-13T04:38:03.613354+0000","last_became_peered":"2023-09-13T04:38:03.613354+0000","last_unstale":"2023-09-13T04:38:23.588357+0000","last_undegraded":"2023-09-13T04:38:23.588357+0000","last_fullsized":"2023-09-13T04:38:23.588357+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:41:01.294912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.242914+0000","last_change":"2023-09-13T04:37:58.591196+0000","last_active":"2023-09-13T04:38:24.242914+0000","last_peered":"2023-09-13T04:38:24.242914+0000","last_clean":"2023-09-13T04:38:24.242914+0000","last_became_active":"2023-09-13T04:37:58.590883+0000","last_became_peered":"2023-09-13T04:37:58.590883+0000","last_unstale":"2023-09-13T04:38:24.242914+0000","last_undegraded":"2023-09-13T04:38:24.242914+0000","last_fullsized":"2023-09-13T04:38:24.242914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:46:58.530270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595499+0000","last_change":"2023-09-13T04:37:59.243483+0000","last_active":"2023-09-13T04:38:23.595499+0000","last_peered":"2023-09-13T04:38:23.595499+0000","last_clean":"2023-09-13T04:38:23.595499+0000","last_became_active":"2023-09-13T04:37:59.243088+0000","last_became_peered":"2023-09-13T04:37:59.243088+0000","last_unstale":"2023-09-13T04:38:23.595499+0000","last_undegraded":"2023-09-13T04:38:23.595499+0000","last_fullsized":"2023-09-13T04:38:23.595499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:56:14.093819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591697+0000","last_change":"2023-09-13T04:38:03.608820+0000","last_active":"2023-09-13T04:38:23.591697+0000","last_peered":"2023-09-13T04:38:23.591697+0000","last_clean":"2023-09-13T04:38:23.591697+0000","last_became_active":"2023-09-13T04:38:03.608200+0000","last_became_peered":"2023-09-13T04:38:03.608200+0000","last_unstale":"2023-09-13T04:38:23.591697+0000","last_undegraded":"2023-09-13T04:38:23.591697+0000","last_fullsized":"2023-09-13T04:38:23.591697+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:54:52.416501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.607235+0000","last_change":"2023-09-13T04:38:24.606820+0000","last_active":"2023-09-13T04:38:24.607235+0000","last_peered":"2023-09-13T04:38:24.607235+0000","last_clean":"2023-09-13T04:38:24.607235+0000","last_became_active":"2023-09-13T04:38:24.599059+0000","last_became_peered":"2023-09-13T04:38:24.599059+0000","last_unstale":"2023-09-13T04:38:24.607235+0000","last_undegraded":"2023-09-13T04:38:24.607235+0000","last_fullsized":"2023-09-13T04:38:24.607235+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:08:12.634357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597568+0000","last_change":"2023-09-13T04:37:59.250666+0000","last_active":"2023-09-13T04:38:23.597568+0000","last_peered":"2023-09-13T04:38:23.597568+0000","last_clean":"2023-09-13T04:38:23.597568+0000","last_became_active":"2023-09-13T04:37:59.250240+0000","last_became_peered":"2023-09-13T04:37:59.250240+0000","last_unstale":"2023-09-13T04:38:23.597568+0000","last_undegraded":"2023-09-13T04:38:23.597568+0000","last_fullsized":"2023-09-13T04:38:23.597568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:44:31.324222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592271+0000","last_change":"2023-09-13T04:38:03.616296+0000","last_active":"2023-09-13T04:38:23.592271+0000","last_peered":"2023-09-13T04:38:23.592271+0000","last_clean":"2023-09-13T04:38:23.592271+0000","last_became_active":"2023-09-13T04:38:03.615934+0000","last_became_peered":"2023-09-13T04:38:03.615934+0000","last_unstale":"2023-09-13T04:38:23.592271+0000","last_undegraded":"2023-09-13T04:38:23.592271+0000","last_fullsized":"2023-09-13T04:38:23.592271+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:51:18.994472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244805+0000","last_change":"2023-09-13T04:37:58.594782+0000","last_active":"2023-09-13T04:38:24.244805+0000","last_peered":"2023-09-13T04:38:24.244805+0000","last_clean":"2023-09-13T04:38:24.244805+0000","last_became_active":"2023-09-13T04:37:58.594480+0000","last_became_peered":"2023-09-13T04:37:58.594480+0000","last_unstale":"2023-09-13T04:38:24.244805+0000","last_undegraded":"2023-09-13T04:38:24.244805+0000","last_fullsized":"2023-09-13T04:38:24.244805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:40:19.635672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588001+0000","last_change":"2023-09-13T04:37:59.252575+0000","last_active":"2023-09-13T04:38:23.588001+0000","last_peered":"2023-09-13T04:38:23.588001+0000","last_clean":"2023-09-13T04:38:23.588001+0000","last_became_active":"2023-09-13T04:37:59.252268+0000","last_became_peered":"2023-09-13T04:37:59.252268+0000","last_unstale":"2023-09-13T04:38:23.588001+0000","last_undegraded":"2023-09-13T04:38:23.588001+0000","last_fullsized":"2023-09-13T04:38:23.588001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:27:10.814930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587908+0000","last_change":"2023-09-13T04:38:03.618835+0000","last_active":"2023-09-13T04:38:23.587908+0000","last_peered":"2023-09-13T04:38:23.587908+0000","last_clean":"2023-09-13T04:38:23.587908+0000","last_became_active":"2023-09-13T04:38:03.618226+0000","last_became_peered":"2023-09-13T04:38:03.618226+0000","last_unstale":"2023-09-13T04:38:23.587908+0000","last_undegraded":"2023-09-13T04:38:23.587908+0000","last_fullsized":"2023-09-13T04:38:23.587908+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:01:28.157490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590730+0000","last_change":"2023-09-13T04:37:58.588545+0000","last_active":"2023-09-13T04:38:23.590730+0000","last_peered":"2023-09-13T04:38:23.590730+0000","last_clean":"2023-09-13T04:38:23.590730+0000","last_became_active":"2023-09-13T04:37:58.587995+0000","last_became_peered":"2023-09-13T04:37:58.587995+0000","last_unstale":"2023-09-13T04:38:23.590730+0000","last_undegraded":"2023-09-13T04:38:23.590730+0000","last_fullsized":"2023-09-13T04:38:23.590730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:40:08.797724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587793+0000","last_change":"2023-09-13T04:37:59.249100+0000","last_active":"2023-09-13T04:38:23.587793+0000","last_peered":"2023-09-13T04:38:23.587793+0000","last_clean":"2023-09-13T04:38:23.587793+0000","last_became_active":"2023-09-13T04:37:59.248771+0000","last_became_peered":"2023-09-13T04:37:59.248771+0000","last_unstale":"2023-09-13T04:38:23.587793+0000","last_undegraded":"2023-09-13T04:38:23.587793+0000","last_fullsized":"2023-09-13T04:38:23.587793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:27:37.964595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597364+0000","last_change":"2023-09-13T04:38:03.616988+0000","last_active":"2023-09-13T04:38:23.597364+0000","last_peered":"2023-09-13T04:38:23.597364+0000","last_clean":"2023-09-13T04:38:23.597364+0000","last_became_active":"2023-09-13T04:38:03.616524+0000","last_became_peered":"2023-09-13T04:38:03.616524+0000","last_unstale":"2023-09-13T04:38:23.597364+0000","last_undegraded":"2023-09-13T04:38:23.597364+0000","last_fullsized":"2023-09-13T04:38:23.597364+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:42:41.431115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'10","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.613710+0000","last_change":"2023-09-13T04:38:24.613384+0000","last_active":"2023-09-13T04:38:24.613710+0000","last_peered":"2023-09-13T04:38:24.613710+0000","last_clean":"2023-09-13T04:38:24.613710+0000","last_became_active":"2023-09-13T04:38:24.599349+0000","last_became_peered":"2023-09-13T04:38:24.599349+0000","last_unstale":"2023-09-13T04:38:24.613710+0000","last_undegraded":"2023-09-13T04:38:24.613710+0000","last_fullsized":"2023-09-13T04:38:24.613710+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:23:23.479715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593329+0000","last_change":"2023-09-13T04:37:59.252237+0000","last_active":"2023-09-13T04:38:23.593329+0000","last_peered":"2023-09-13T04:38:23.593329+0000","last_clean":"2023-09-13T04:38:23.593329+0000","last_became_active":"2023-09-13T04:37:59.251718+0000","last_became_peered":"2023-09-13T04:37:59.251718+0000","last_unstale":"2023-09-13T04:38:23.593329+0000","last_undegraded":"2023-09-13T04:38:23.593329+0000","last_fullsized":"2023-09-13T04:38:23.593329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:45:48.405226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593276+0000","last_change":"2023-09-13T04:38:03.607096+0000","last_active":"2023-09-13T04:38:23.593276+0000","last_peered":"2023-09-13T04:38:23.593276+0000","last_clean":"2023-09-13T04:38:23.593276+0000","last_became_active":"2023-09-13T04:38:03.606656+0000","last_became_peered":"2023-09-13T04:38:03.606656+0000","last_unstale":"2023-09-13T04:38:23.593276+0000","last_undegraded":"2023-09-13T04:38:23.593276+0000","last_fullsized":"2023-09-13T04:38:23.593276+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:59:26.140360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595200+0000","last_change":"2023-09-13T04:37:58.591047+0000","last_active":"2023-09-13T04:38:23.595200+0000","last_peered":"2023-09-13T04:38:23.595200+0000","last_clean":"2023-09-13T04:38:23.595200+0000","last_became_active":"2023-09-13T04:37:58.590740+0000","last_became_peered":"2023-09-13T04:37:58.590740+0000","last_unstale":"2023-09-13T04:38:23.595200+0000","last_undegraded":"2023-09-13T04:38:23.595200+0000","last_fullsized":"2023-09-13T04:38:23.595200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:14:42.280416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592406+0000","last_change":"2023-09-13T04:37:59.249362+0000","last_active":"2023-09-13T04:38:23.592406+0000","last_peered":"2023-09-13T04:38:23.592406+0000","last_clean":"2023-09-13T04:38:23.592406+0000","last_became_active":"2023-09-13T04:37:59.249041+0000","last_became_peered":"2023-09-13T04:37:59.249041+0000","last_unstale":"2023-09-13T04:38:23.592406+0000","last_undegraded":"2023-09-13T04:38:23.592406+0000","last_fullsized":"2023-09-13T04:38:23.592406+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:06:33.822157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.598370+0000","last_change":"2023-09-13T04:38:24.598370+0000","last_active":"2023-09-13T04:38:24.598370+0000","last_peered":"2023-09-13T04:38:24.598370+0000","last_clean":"2023-09-13T04:38:24.598370+0000","last_became_active":"2023-09-13T04:38:24.597931+0000","last_became_peered":"2023-09-13T04:38:24.597931+0000","last_unstale":"2023-09-13T04:38:24.598370+0000","last_undegraded":"2023-09-13T04:38:24.598370+0000","last_fullsized":"2023-09-13T04:38:24.598370+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:14:28.911329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588100+0000","last_change":"2023-09-13T04:37:58.592496+0000","last_active":"2023-09-13T04:38:23.588100+0000","last_peered":"2023-09-13T04:38:23.588100+0000","last_clean":"2023-09-13T04:38:23.588100+0000","last_became_active":"2023-09-13T04:37:58.592106+0000","last_became_peered":"2023-09-13T04:37:58.592106+0000","last_unstale":"2023-09-13T04:38:23.588100+0000","last_undegraded":"2023-09-13T04:38:23.588100+0000","last_fullsized":"2023-09-13T04:38:23.588100+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:28:55.618563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594212+0000","last_change":"2023-09-13T04:37:59.250019+0000","last_active":"2023-09-13T04:38:23.594212+0000","last_peered":"2023-09-13T04:38:23.594212+0000","last_clean":"2023-09-13T04:38:23.594212+0000","last_became_active":"2023-09-13T04:37:59.249278+0000","last_became_peered":"2023-09-13T04:37:59.249278+0000","last_unstale":"2023-09-13T04:38:23.594212+0000","last_undegraded":"2023-09-13T04:38:23.594212+0000","last_fullsized":"2023-09-13T04:38:23.594212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:15:31.217236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597640+0000","last_change":"2023-09-13T04:38:03.614542+0000","last_active":"2023-09-13T04:38:23.597640+0000","last_peered":"2023-09-13T04:38:23.597640+0000","last_clean":"2023-09-13T04:38:23.597640+0000","last_became_active":"2023-09-13T04:38:03.614103+0000","last_became_peered":"2023-09-13T04:38:03.614103+0000","last_unstale":"2023-09-13T04:38:23.597640+0000","last_undegraded":"2023-09-13T04:38:23.597640+0000","last_fullsized":"2023-09-13T04:38:23.597640+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:06:48.426843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.592720+0000","last_change":"2023-09-13T04:37:58.591864+0000","last_active":"2023-09-13T04:38:24.592720+0000","last_peered":"2023-09-13T04:38:24.592720+0000","last_clean":"2023-09-13T04:38:24.592720+0000","last_became_active":"2023-09-13T04:37:58.591562+0000","last_became_peered":"2023-09-13T04:37:58.591562+0000","last_unstale":"2023-09-13T04:38:24.592720+0000","last_undegraded":"2023-09-13T04:38:24.592720+0000","last_fullsized":"2023-09-13T04:38:24.592720+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:31:27.280489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594567+0000","last_change":"2023-09-13T04:37:59.252600+0000","last_active":"2023-09-13T04:38:23.594567+0000","last_peered":"2023-09-13T04:38:23.594567+0000","last_clean":"2023-09-13T04:38:23.594567+0000","last_became_active":"2023-09-13T04:37:59.252306+0000","last_became_peered":"2023-09-13T04:37:59.252306+0000","last_unstale":"2023-09-13T04:38:23.594567+0000","last_undegraded":"2023-09-13T04:38:23.594567+0000","last_fullsized":"2023-09-13T04:38:23.594567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:18:55.686847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.596920+0000","last_change":"2023-09-13T04:38:03.742502+0000","last_active":"2023-09-13T04:38:23.596920+0000","last_peered":"2023-09-13T04:38:23.596920+0000","last_clean":"2023-09-13T04:38:23.596920+0000","last_became_active":"2023-09-13T04:38:03.742054+0000","last_became_peered":"2023-09-13T04:38:03.742054+0000","last_unstale":"2023-09-13T04:38:23.596920+0000","last_undegraded":"2023-09-13T04:38:23.596920+0000","last_fullsized":"2023-09-13T04:38:23.596920+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:55:54.601145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593527+0000","last_change":"2023-09-13T04:37:59.245644+0000","last_active":"2023-09-13T04:38:23.593527+0000","last_peered":"2023-09-13T04:38:23.593527+0000","last_clean":"2023-09-13T04:38:23.593527+0000","last_became_active":"2023-09-13T04:37:59.245284+0000","last_became_peered":"2023-09-13T04:37:59.245284+0000","last_unstale":"2023-09-13T04:38:23.593527+0000","last_undegraded":"2023-09-13T04:38:23.593527+0000","last_fullsized":"2023-09-13T04:38:23.593527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:19:04.002414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.621561+0000","last_change":"2023-09-13T04:37:58.590249+0000","last_active":"2023-09-13T04:38:25.621561+0000","last_peered":"2023-09-13T04:38:25.621561+0000","last_clean":"2023-09-13T04:38:25.621561+0000","last_became_active":"2023-09-13T04:37:58.589521+0000","last_became_peered":"2023-09-13T04:37:58.589521+0000","last_unstale":"2023-09-13T04:38:25.621561+0000","last_undegraded":"2023-09-13T04:38:25.621561+0000","last_fullsized":"2023-09-13T04:38:25.621561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:11:57.542149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.598613+0000","last_change":"2023-09-13T04:38:24.598613+0000","last_active":"2023-09-13T04:38:24.598613+0000","last_peered":"2023-09-13T04:38:24.598613+0000","last_clean":"2023-09-13T04:38:24.598613+0000","last_became_active":"2023-09-13T04:38:24.598150+0000","last_became_peered":"2023-09-13T04:38:24.598150+0000","last_unstale":"2023-09-13T04:38:24.598613+0000","last_undegraded":"2023-09-13T04:38:24.598613+0000","last_fullsized":"2023-09-13T04:38:24.598613+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:02:29.894446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595221+0000","last_change":"2023-09-13T04:37:59.243794+0000","last_active":"2023-09-13T04:38:23.595221+0000","last_peered":"2023-09-13T04:38:23.595221+0000","last_clean":"2023-09-13T04:38:23.595221+0000","last_became_active":"2023-09-13T04:37:59.243368+0000","last_became_peered":"2023-09-13T04:37:59.243368+0000","last_unstale":"2023-09-13T04:38:23.595221+0000","last_undegraded":"2023-09-13T04:38:23.595221+0000","last_fullsized":"2023-09-13T04:38:23.595221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:06:45.318737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.245630+0000","last_change":"2023-09-13T04:37:58.589195+0000","last_active":"2023-09-13T04:38:24.245630+0000","last_peered":"2023-09-13T04:38:24.245630+0000","last_clean":"2023-09-13T04:38:24.245630+0000","last_became_active":"2023-09-13T04:37:58.588905+0000","last_became_peered":"2023-09-13T04:37:58.588905+0000","last_unstale":"2023-09-13T04:38:24.245630+0000","last_undegraded":"2023-09-13T04:38:24.245630+0000","last_fullsized":"2023-09-13T04:38:24.245630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:13:15.978891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":326,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587638+0000","last_change":"2023-09-13T04:36:52.227264+0000","last_active":"2023-09-13T04:38:23.587638+0000","last_peered":"2023-09-13T04:38:23.587638+0000","last_clean":"2023-09-13T04:38:23.587638+0000","last_became_active":"2023-09-13T04:36:52.226773+0000","last_became_peered":"2023-09-13T04:36:52.226773+0000","last_unstale":"2023-09-13T04:38:23.587638+0000","last_undegraded":"2023-09-13T04:38:23.587638+0000","last_fullsized":"2023-09-13T04:38:23.587638+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:33:42.026499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:33:42.026499+0000","last_clean_scrub_stamp":"2023-09-13T04:33:42.026499+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:28:55.806587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592890+0000","last_change":"2023-09-13T04:38:03.742790+0000","last_active":"2023-09-13T04:38:23.592890+0000","last_peered":"2023-09-13T04:38:23.592890+0000","last_clean":"2023-09-13T04:38:23.592890+0000","last_became_active":"2023-09-13T04:38:03.742338+0000","last_became_peered":"2023-09-13T04:38:03.742338+0000","last_unstale":"2023-09-13T04:38:23.592890+0000","last_undegraded":"2023-09-13T04:38:23.592890+0000","last_fullsized":"2023-09-13T04:38:23.592890+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:01:38.910623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597412+0000","last_change":"2023-09-13T04:37:59.252669+0000","last_active":"2023-09-13T04:38:23.597412+0000","last_peered":"2023-09-13T04:38:23.597412+0000","last_clean":"2023-09-13T04:38:23.597412+0000","last_became_active":"2023-09-13T04:37:59.252223+0000","last_became_peered":"2023-09-13T04:37:59.252223+0000","last_unstale":"2023-09-13T04:38:23.597412+0000","last_undegraded":"2023-09-13T04:38:23.597412+0000","last_fullsized":"2023-09-13T04:38:23.597412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:57:21.090083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.615697+0000","last_change":"2023-09-13T04:37:58.590540+0000","last_active":"2023-09-13T04:38:25.615697+0000","last_peered":"2023-09-13T04:38:25.615697+0000","last_clean":"2023-09-13T04:38:25.615697+0000","last_became_active":"2023-09-13T04:37:58.590061+0000","last_became_peered":"2023-09-13T04:37:58.590061+0000","last_unstale":"2023-09-13T04:38:25.615697+0000","last_undegraded":"2023-09-13T04:38:25.615697+0000","last_fullsized":"2023-09-13T04:38:25.615697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:57:17.133461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592719+0000","last_change":"2023-09-13T04:38:03.614615+0000","last_active":"2023-09-13T04:38:23.592719+0000","last_peered":"2023-09-13T04:38:23.592719+0000","last_clean":"2023-09-13T04:38:23.592719+0000","last_became_active":"2023-09-13T04:38:03.614086+0000","last_became_peered":"2023-09-13T04:38:03.614086+0000","last_unstale":"2023-09-13T04:38:23.592719+0000","last_undegraded":"2023-09-13T04:38:23.592719+0000","last_fullsized":"2023-09-13T04:38:23.592719+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:25:35.525513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.243719+0000","last_change":"2023-09-13T04:37:58.592096+0000","last_active":"2023-09-13T04:38:24.243719+0000","last_peered":"2023-09-13T04:38:24.243719+0000","last_clean":"2023-09-13T04:38:24.243719+0000","last_became_active":"2023-09-13T04:37:58.591872+0000","last_became_peered":"2023-09-13T04:37:58.591872+0000","last_unstale":"2023-09-13T04:38:24.243719+0000","last_undegraded":"2023-09-13T04:38:24.243719+0000","last_fullsized":"2023-09-13T04:38:24.243719+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:05:23.936105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597816+0000","last_change":"2023-09-13T04:37:59.240759+0000","last_active":"2023-09-13T04:38:23.597816+0000","last_peered":"2023-09-13T04:38:23.597816+0000","last_clean":"2023-09-13T04:38:23.597816+0000","last_became_active":"2023-09-13T04:37:59.240414+0000","last_became_peered":"2023-09-13T04:37:59.240414+0000","last_unstale":"2023-09-13T04:38:23.597816+0000","last_undegraded":"2023-09-13T04:38:23.597816+0000","last_fullsized":"2023-09-13T04:38:23.597816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:37:20.301938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597533+0000","last_change":"2023-09-13T04:38:03.742519+0000","last_active":"2023-09-13T04:38:23.597533+0000","last_peered":"2023-09-13T04:38:23.597533+0000","last_clean":"2023-09-13T04:38:23.597533+0000","last_became_active":"2023-09-13T04:38:03.742075+0000","last_became_peered":"2023-09-13T04:38:03.742075+0000","last_unstale":"2023-09-13T04:38:23.597533+0000","last_undegraded":"2023-09-13T04:38:23.597533+0000","last_fullsized":"2023-09-13T04:38:23.597533+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:14:33.155837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.619899+0000","last_change":"2023-09-13T04:37:58.593891+0000","last_active":"2023-09-13T04:38:25.619899+0000","last_peered":"2023-09-13T04:38:25.619899+0000","last_clean":"2023-09-13T04:38:25.619899+0000","last_became_active":"2023-09-13T04:37:58.593497+0000","last_became_peered":"2023-09-13T04:37:58.593497+0000","last_unstale":"2023-09-13T04:38:25.619899+0000","last_undegraded":"2023-09-13T04:38:25.619899+0000","last_fullsized":"2023-09-13T04:38:25.619899+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:25:53.652601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595975+0000","last_change":"2023-09-13T04:37:59.247151+0000","last_active":"2023-09-13T04:38:23.595975+0000","last_peered":"2023-09-13T04:38:23.595975+0000","last_clean":"2023-09-13T04:38:23.595975+0000","last_became_active":"2023-09-13T04:37:59.246853+0000","last_became_peered":"2023-09-13T04:37:59.246853+0000","last_unstale":"2023-09-13T04:38:23.595975+0000","last_undegraded":"2023-09-13T04:38:23.595975+0000","last_fullsized":"2023-09-13T04:38:23.595975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:15:22.659365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587493+0000","last_change":"2023-09-13T04:38:03.606594+0000","last_active":"2023-09-13T04:38:23.587493+0000","last_peered":"2023-09-13T04:38:23.587493+0000","last_clean":"2023-09-13T04:38:23.587493+0000","last_became_active":"2023-09-13T04:38:03.606058+0000","last_became_peered":"2023-09-13T04:38:03.606058+0000","last_unstale":"2023-09-13T04:38:23.587493+0000","last_undegraded":"2023-09-13T04:38:23.587493+0000","last_fullsized":"2023-09-13T04:38:23.587493+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:57:39.237520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.246057+0000","last_change":"2023-09-13T04:37:58.588362+0000","last_active":"2023-09-13T04:38:24.246057+0000","last_peered":"2023-09-13T04:38:24.246057+0000","last_clean":"2023-09-13T04:38:24.246057+0000","last_became_active":"2023-09-13T04:37:58.588065+0000","last_became_peered":"2023-09-13T04:37:58.588065+0000","last_unstale":"2023-09-13T04:38:24.246057+0000","last_undegraded":"2023-09-13T04:38:24.246057+0000","last_fullsized":"2023-09-13T04:38:24.246057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:32:04.255654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592468+0000","last_change":"2023-09-13T04:37:59.251883+0000","last_active":"2023-09-13T04:38:23.592468+0000","last_peered":"2023-09-13T04:38:23.592468+0000","last_clean":"2023-09-13T04:38:23.592468+0000","last_became_active":"2023-09-13T04:37:59.251522+0000","last_became_peered":"2023-09-13T04:37:59.251522+0000","last_unstale":"2023-09-13T04:38:23.592468+0000","last_undegraded":"2023-09-13T04:38:23.592468+0000","last_fullsized":"2023-09-13T04:38:23.592468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:05:06.420799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590729+0000","last_change":"2023-09-13T04:38:03.742726+0000","last_active":"2023-09-13T04:38:23.590729+0000","last_peered":"2023-09-13T04:38:23.590729+0000","last_clean":"2023-09-13T04:38:23.590729+0000","last_became_active":"2023-09-13T04:38:03.742293+0000","last_became_peered":"2023-09-13T04:38:03.742293+0000","last_unstale":"2023-09-13T04:38:23.590729+0000","last_undegraded":"2023-09-13T04:38:23.590729+0000","last_fullsized":"2023-09-13T04:38:23.590729+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:52:15.089764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.243314+0000","last_change":"2023-09-13T04:37:58.584923+0000","last_active":"2023-09-13T04:38:24.243314+0000","last_peered":"2023-09-13T04:38:24.243314+0000","last_clean":"2023-09-13T04:38:24.243314+0000","last_became_active":"2023-09-13T04:37:58.584320+0000","last_became_peered":"2023-09-13T04:37:58.584320+0000","last_unstale":"2023-09-13T04:38:24.243314+0000","last_undegraded":"2023-09-13T04:38:24.243314+0000","last_fullsized":"2023-09-13T04:38:24.243314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:16:53.670534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592586+0000","last_change":"2023-09-13T04:37:59.243029+0000","last_active":"2023-09-13T04:38:23.592586+0000","last_peered":"2023-09-13T04:38:23.592586+0000","last_clean":"2023-09-13T04:38:23.592586+0000","last_became_active":"2023-09-13T04:37:59.242733+0000","last_became_peered":"2023-09-13T04:37:59.242733+0000","last_unstale":"2023-09-13T04:38:23.592586+0000","last_undegraded":"2023-09-13T04:38:23.592586+0000","last_fullsized":"2023-09-13T04:38:23.592586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:48:21.430026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588037+0000","last_change":"2023-09-13T04:38:03.616735+0000","last_active":"2023-09-13T04:38:23.588037+0000","last_peered":"2023-09-13T04:38:23.588037+0000","last_clean":"2023-09-13T04:38:23.588037+0000","last_became_active":"2023-09-13T04:38:03.616275+0000","last_became_peered":"2023-09-13T04:38:03.616275+0000","last_unstale":"2023-09-13T04:38:23.588037+0000","last_undegraded":"2023-09-13T04:38:23.588037+0000","last_fullsized":"2023-09-13T04:38:23.588037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:12:27.267918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244170+0000","last_change":"2023-09-13T04:37:58.595040+0000","last_active":"2023-09-13T04:38:24.244170+0000","last_peered":"2023-09-13T04:38:24.244170+0000","last_clean":"2023-09-13T04:38:24.244170+0000","last_became_active":"2023-09-13T04:37:58.591403+0000","last_became_peered":"2023-09-13T04:37:58.591403+0000","last_unstale":"2023-09-13T04:38:24.244170+0000","last_undegraded":"2023-09-13T04:38:24.244170+0000","last_fullsized":"2023-09-13T04:38:24.244170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:42:06.561066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592175+0000","last_change":"2023-09-13T04:37:59.248837+0000","last_active":"2023-09-13T04:38:23.592175+0000","last_peered":"2023-09-13T04:38:23.592175+0000","last_clean":"2023-09-13T04:38:23.592175+0000","last_became_active":"2023-09-13T04:37:59.248573+0000","last_became_peered":"2023-09-13T04:37:59.248573+0000","last_unstale":"2023-09-13T04:38:23.592175+0000","last_undegraded":"2023-09-13T04:38:23.592175+0000","last_fullsized":"2023-09-13T04:38:23.592175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:56:22.980484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590405+0000","last_change":"2023-09-13T04:38:03.616370+0000","last_active":"2023-09-13T04:38:23.590405+0000","last_peered":"2023-09-13T04:38:23.590405+0000","last_clean":"2023-09-13T04:38:23.590405+0000","last_became_active":"2023-09-13T04:38:03.615996+0000","last_became_peered":"2023-09-13T04:38:03.615996+0000","last_unstale":"2023-09-13T04:38:23.590405+0000","last_undegraded":"2023-09-13T04:38:23.590405+0000","last_fullsized":"2023-09-13T04:38:23.590405+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:56:59.924205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.590955+0000","last_change":"2023-09-13T04:37:58.597902+0000","last_active":"2023-09-13T04:38:24.590955+0000","last_peered":"2023-09-13T04:38:24.590955+0000","last_clean":"2023-09-13T04:38:24.590955+0000","last_became_active":"2023-09-13T04:37:58.597578+0000","last_became_peered":"2023-09-13T04:37:58.597578+0000","last_unstale":"2023-09-13T04:38:24.590955+0000","last_undegraded":"2023-09-13T04:38:24.590955+0000","last_fullsized":"2023-09-13T04:38:24.590955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:10:09.931949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597024+0000","last_change":"2023-09-13T04:37:59.248924+0000","last_active":"2023-09-13T04:38:23.597024+0000","last_peered":"2023-09-13T04:38:23.597024+0000","last_clean":"2023-09-13T04:38:23.597024+0000","last_became_active":"2023-09-13T04:37:59.248483+0000","last_became_peered":"2023-09-13T04:37:59.248483+0000","last_unstale":"2023-09-13T04:38:23.597024+0000","last_undegraded":"2023-09-13T04:38:23.597024+0000","last_fullsized":"2023-09-13T04:38:23.597024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:49:56.078544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590322+0000","last_change":"2023-09-13T04:38:03.742260+0000","last_active":"2023-09-13T04:38:23.590322+0000","last_peered":"2023-09-13T04:38:23.590322+0000","last_clean":"2023-09-13T04:38:23.590322+0000","last_became_active":"2023-09-13T04:38:03.741893+0000","last_became_peered":"2023-09-13T04:38:03.741893+0000","last_unstale":"2023-09-13T04:38:23.590322+0000","last_undegraded":"2023-09-13T04:38:23.590322+0000","last_fullsized":"2023-09-13T04:38:23.590322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:59:00.653545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.245764+0000","last_change":"2023-09-13T04:37:58.586898+0000","last_active":"2023-09-13T04:38:24.245764+0000","last_peered":"2023-09-13T04:38:24.245764+0000","last_clean":"2023-09-13T04:38:24.245764+0000","last_became_active":"2023-09-13T04:37:58.586594+0000","last_became_peered":"2023-09-13T04:37:58.586594+0000","last_unstale":"2023-09-13T04:38:24.245764+0000","last_undegraded":"2023-09-13T04:38:24.245764+0000","last_fullsized":"2023-09-13T04:38:24.245764+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:29:43.718522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587969+0000","last_change":"2023-09-13T04:37:59.244749+0000","last_active":"2023-09-13T04:38:23.587969+0000","last_peered":"2023-09-13T04:38:23.587969+0000","last_clean":"2023-09-13T04:38:23.587969+0000","last_became_active":"2023-09-13T04:37:59.244391+0000","last_became_peered":"2023-09-13T04:37:59.244391+0000","last_unstale":"2023-09-13T04:38:23.587969+0000","last_undegraded":"2023-09-13T04:38:23.587969+0000","last_fullsized":"2023-09-13T04:38:23.587969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:33:02.195729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592524+0000","last_change":"2023-09-13T04:38:03.609766+0000","last_active":"2023-09-13T04:38:23.592524+0000","last_peered":"2023-09-13T04:38:23.592524+0000","last_clean":"2023-09-13T04:38:23.592524+0000","last_became_active":"2023-09-13T04:38:03.609432+0000","last_became_peered":"2023-09-13T04:38:03.609432+0000","last_unstale":"2023-09-13T04:38:23.592524+0000","last_undegraded":"2023-09-13T04:38:23.592524+0000","last_fullsized":"2023-09-13T04:38:23.592524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:30:56.115483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.620943+0000","last_change":"2023-09-13T04:37:58.589119+0000","last_active":"2023-09-13T04:38:25.620943+0000","last_peered":"2023-09-13T04:38:25.620943+0000","last_clean":"2023-09-13T04:38:25.620943+0000","last_became_active":"2023-09-13T04:37:58.588803+0000","last_became_peered":"2023-09-13T04:37:58.588803+0000","last_unstale":"2023-09-13T04:38:25.620943+0000","last_undegraded":"2023-09-13T04:38:25.620943+0000","last_fullsized":"2023-09-13T04:38:25.620943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:51:20.888666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588392+0000","last_change":"2023-09-13T04:37:59.243506+0000","last_active":"2023-09-13T04:38:23.588392+0000","last_peered":"2023-09-13T04:38:23.588392+0000","last_clean":"2023-09-13T04:38:23.588392+0000","last_became_active":"2023-09-13T04:37:59.243082+0000","last_became_peered":"2023-09-13T04:37:59.243082+0000","last_unstale":"2023-09-13T04:38:23.588392+0000","last_undegraded":"2023-09-13T04:38:23.588392+0000","last_fullsized":"2023-09-13T04:38:23.588392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:01:41.179286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592019+0000","last_change":"2023-09-13T04:38:03.609042+0000","last_active":"2023-09-13T04:38:23.592019+0000","last_peered":"2023-09-13T04:38:23.592019+0000","last_clean":"2023-09-13T04:38:23.592019+0000","last_became_active":"2023-09-13T04:38:03.608553+0000","last_became_peered":"2023-09-13T04:38:03.608553+0000","last_unstale":"2023-09-13T04:38:23.592019+0000","last_undegraded":"2023-09-13T04:38:23.592019+0000","last_fullsized":"2023-09-13T04:38:23.592019+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:05:56.648475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591325+0000","last_change":"2023-09-13T04:37:58.598056+0000","last_active":"2023-09-13T04:38:23.591325+0000","last_peered":"2023-09-13T04:38:23.591325+0000","last_clean":"2023-09-13T04:38:23.591325+0000","last_became_active":"2023-09-13T04:37:58.597724+0000","last_became_peered":"2023-09-13T04:37:58.597724+0000","last_unstale":"2023-09-13T04:38:23.591325+0000","last_undegraded":"2023-09-13T04:38:23.591325+0000","last_fullsized":"2023-09-13T04:38:23.591325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:43:08.296866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588462+0000","last_change":"2023-09-13T04:37:59.246505+0000","last_active":"2023-09-13T04:38:23.588462+0000","last_peered":"2023-09-13T04:38:23.588462+0000","last_clean":"2023-09-13T04:38:23.588462+0000","last_became_active":"2023-09-13T04:37:59.246024+0000","last_became_peered":"2023-09-13T04:37:59.246024+0000","last_unstale":"2023-09-13T04:38:23.588462+0000","last_undegraded":"2023-09-13T04:38:23.588462+0000","last_fullsized":"2023-09-13T04:38:23.588462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:36:58.008066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590871+0000","last_change":"2023-09-13T04:38:03.605354+0000","last_active":"2023-09-13T04:38:23.590871+0000","last_peered":"2023-09-13T04:38:23.590871+0000","last_clean":"2023-09-13T04:38:23.590871+0000","last_became_active":"2023-09-13T04:38:03.604840+0000","last_became_peered":"2023-09-13T04:38:03.604840+0000","last_unstale":"2023-09-13T04:38:23.590871+0000","last_undegraded":"2023-09-13T04:38:23.590871+0000","last_fullsized":"2023-09-13T04:38:23.590871+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:46:29.867914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597380+0000","last_change":"2023-09-13T04:37:58.589052+0000","last_active":"2023-09-13T04:38:23.597380+0000","last_peered":"2023-09-13T04:38:23.597380+0000","last_clean":"2023-09-13T04:38:23.597380+0000","last_became_active":"2023-09-13T04:37:58.588707+0000","last_became_peered":"2023-09-13T04:37:58.588707+0000","last_unstale":"2023-09-13T04:38:23.597380+0000","last_undegraded":"2023-09-13T04:38:23.597380+0000","last_fullsized":"2023-09-13T04:38:23.597380+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:25:23.966046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588356+0000","last_change":"2023-09-13T04:37:59.241157+0000","last_active":"2023-09-13T04:38:23.588356+0000","last_peered":"2023-09-13T04:38:23.588356+0000","last_clean":"2023-09-13T04:38:23.588356+0000","last_became_active":"2023-09-13T04:37:59.240857+0000","last_became_peered":"2023-09-13T04:37:59.240857+0000","last_unstale":"2023-09-13T04:38:23.588356+0000","last_undegraded":"2023-09-13T04:38:23.588356+0000","last_fullsized":"2023-09-13T04:38:23.588356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:04:44.986908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592376+0000","last_change":"2023-09-13T04:37:59.242447+0000","last_active":"2023-09-13T04:38:23.592376+0000","last_peered":"2023-09-13T04:38:23.592376+0000","last_clean":"2023-09-13T04:38:23.592376+0000","last_became_active":"2023-09-13T04:37:59.242130+0000","last_became_peered":"2023-09-13T04:37:59.242130+0000","last_unstale":"2023-09-13T04:38:23.592376+0000","last_undegraded":"2023-09-13T04:38:23.592376+0000","last_fullsized":"2023-09-13T04:38:23.592376+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:12:12.844888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.595646+0000","last_change":"2023-09-13T04:38:24.595646+0000","last_active":"2023-09-13T04:38:24.595646+0000","last_peered":"2023-09-13T04:38:24.595646+0000","last_clean":"2023-09-13T04:38:24.595646+0000","last_became_active":"2023-09-13T04:38:24.595295+0000","last_became_peered":"2023-09-13T04:38:24.595295+0000","last_unstale":"2023-09-13T04:38:24.595646+0000","last_undegraded":"2023-09-13T04:38:24.595646+0000","last_fullsized":"2023-09-13T04:38:24.595646+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:24:27.174979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587574+0000","last_change":"2023-09-13T04:37:59.245358+0000","last_active":"2023-09-13T04:38:23.587574+0000","last_peered":"2023-09-13T04:38:23.587574+0000","last_clean":"2023-09-13T04:38:23.587574+0000","last_became_active":"2023-09-13T04:37:59.245052+0000","last_became_peered":"2023-09-13T04:37:59.245052+0000","last_unstale":"2023-09-13T04:38:23.587574+0000","last_undegraded":"2023-09-13T04:38:23.587574+0000","last_fullsized":"2023-09-13T04:38:23.587574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:05:42.561716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592358+0000","last_change":"2023-09-13T04:37:59.254053+0000","last_active":"2023-09-13T04:38:23.592358+0000","last_peered":"2023-09-13T04:38:23.592358+0000","last_clean":"2023-09-13T04:38:23.592358+0000","last_became_active":"2023-09-13T04:37:59.253544+0000","last_became_peered":"2023-09-13T04:37:59.253544+0000","last_unstale":"2023-09-13T04:38:23.592358+0000","last_undegraded":"2023-09-13T04:38:23.592358+0000","last_fullsized":"2023-09-13T04:38:23.592358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:18:54.689208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597920+0000","last_change":"2023-09-13T04:37:59.253403+0000","last_active":"2023-09-13T04:38:23.597920+0000","last_peered":"2023-09-13T04:38:23.597920+0000","last_clean":"2023-09-13T04:38:23.597920+0000","last_became_active":"2023-09-13T04:37:59.253062+0000","last_became_peered":"2023-09-13T04:37:59.253062+0000","last_unstale":"2023-09-13T04:38:23.597920+0000","last_undegraded":"2023-09-13T04:38:23.597920+0000","last_fullsized":"2023-09-13T04:38:23.597920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:31:09.898886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592570+0000","last_change":"2023-09-13T04:37:59.240916+0000","last_active":"2023-09-13T04:38:23.592570+0000","last_peered":"2023-09-13T04:38:23.592570+0000","last_clean":"2023-09-13T04:38:23.592570+0000","last_became_active":"2023-09-13T04:37:59.240605+0000","last_became_peered":"2023-09-13T04:37:59.240605+0000","last_unstale":"2023-09-13T04:38:23.592570+0000","last_undegraded":"2023-09-13T04:38:23.592570+0000","last_fullsized":"2023-09-13T04:38:23.592570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:15:48.241587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588778+0000","last_change":"2023-09-13T04:37:59.238569+0000","last_active":"2023-09-13T04:38:23.588778+0000","last_peered":"2023-09-13T04:38:23.588778+0000","last_clean":"2023-09-13T04:38:23.588778+0000","last_became_active":"2023-09-13T04:37:59.238255+0000","last_became_peered":"2023-09-13T04:37:59.238255+0000","last_unstale":"2023-09-13T04:38:23.588778+0000","last_undegraded":"2023-09-13T04:38:23.588778+0000","last_fullsized":"2023-09-13T04:38:23.588778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:19:03.433404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595875+0000","last_change":"2023-09-13T04:37:59.252243+0000","last_active":"2023-09-13T04:38:23.595875+0000","last_peered":"2023-09-13T04:38:23.595875+0000","last_clean":"2023-09-13T04:38:23.595875+0000","last_became_active":"2023-09-13T04:37:59.251902+0000","last_became_peered":"2023-09-13T04:37:59.251902+0000","last_unstale":"2023-09-13T04:38:23.595875+0000","last_undegraded":"2023-09-13T04:38:23.595875+0000","last_fullsized":"2023-09-13T04:38:23.595875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:35:57.395627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593414+0000","last_change":"2023-09-13T04:37:59.245847+0000","last_active":"2023-09-13T04:38:23.593414+0000","last_peered":"2023-09-13T04:38:23.593414+0000","last_clean":"2023-09-13T04:38:23.593414+0000","last_became_active":"2023-09-13T04:37:59.245542+0000","last_became_peered":"2023-09-13T04:37:59.245542+0000","last_unstale":"2023-09-13T04:38:23.593414+0000","last_undegraded":"2023-09-13T04:38:23.593414+0000","last_fullsized":"2023-09-13T04:38:23.593414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:24:16.813333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.596961+0000","last_change":"2023-09-13T04:37:59.246963+0000","last_active":"2023-09-13T04:38:23.596961+0000","last_peered":"2023-09-13T04:38:23.596961+0000","last_clean":"2023-09-13T04:38:23.596961+0000","last_became_active":"2023-09-13T04:37:59.246448+0000","last_became_peered":"2023-09-13T04:37:59.246448+0000","last_unstale":"2023-09-13T04:38:23.596961+0000","last_undegraded":"2023-09-13T04:38:23.596961+0000","last_fullsized":"2023-09-13T04:38:23.596961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:25:45.418778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594086+0000","last_change":"2023-09-13T04:37:59.245399+0000","last_active":"2023-09-13T04:38:23.594086+0000","last_peered":"2023-09-13T04:38:23.594086+0000","last_clean":"2023-09-13T04:38:23.594086+0000","last_became_active":"2023-09-13T04:37:59.245060+0000","last_became_peered":"2023-09-13T04:37:59.245060+0000","last_unstale":"2023-09-13T04:38:23.594086+0000","last_undegraded":"2023-09-13T04:38:23.594086+0000","last_fullsized":"2023-09-13T04:38:23.594086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:25:41.078238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597807+0000","last_change":"2023-09-13T04:37:59.244776+0000","last_active":"2023-09-13T04:38:23.597807+0000","last_peered":"2023-09-13T04:38:23.597807+0000","last_clean":"2023-09-13T04:38:23.597807+0000","last_became_active":"2023-09-13T04:37:59.244409+0000","last_became_peered":"2023-09-13T04:37:59.244409+0000","last_unstale":"2023-09-13T04:38:23.597807+0000","last_undegraded":"2023-09-13T04:38:23.597807+0000","last_fullsized":"2023-09-13T04:38:23.597807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:04:02.561414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592391+0000","last_change":"2023-09-13T04:37:59.248153+0000","last_active":"2023-09-13T04:38:23.592391+0000","last_peered":"2023-09-13T04:38:23.592391+0000","last_clean":"2023-09-13T04:38:23.592391+0000","last_became_active":"2023-09-13T04:37:59.247840+0000","last_became_peered":"2023-09-13T04:37:59.247840+0000","last_unstale":"2023-09-13T04:38:23.592391+0000","last_undegraded":"2023-09-13T04:38:23.592391+0000","last_fullsized":"2023-09-13T04:38:23.592391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:52:53.021219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6718,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":112,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":217088,"data_stored":20418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":490076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.52100000000000002}]},{"osd":1,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.495},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.63700000000000001}]},{"osd":2,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.59599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":4,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.79000000000000004}]},{"osd":5,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.77500000000000002}]},{"osd":6,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.75800000000000001}]},{"osd":7,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.86399999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":9,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.878}]},{"osd":10,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.92400000000000004}]}]},{"osd":10,"up_from":71,"seq":304942678037,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1626112,"data_stored":1084927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":1,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.42299999999999999}]},{"osd":2,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.57199999999999995}]},{"osd":3,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.626},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.754}]},{"osd":4,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.44600000000000001}]},{"osd":5,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.69899999999999995}]},{"osd":6,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":3.4129999999999998,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.84499999999999997}]},{"osd":7,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.64500000000000002}]},{"osd":8,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.59399999999999997}]},{"osd":9,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.80500000000000005}]},{"osd":11,"last update":"Wed Sep 13 04:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.67800000000000005}]}]},{"osd":9,"up_from":66,"seq":283467841562,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":490860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.36299999999999999}]},{"osd":1,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.56499999999999995}]},{"osd":2,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.73499999999999999}]},{"osd":3,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.45200000000000001}]},{"osd":4,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.63},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.59199999999999997}]},{"osd":5,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.75}]},{"osd":6,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.72099999999999997}]},{"osd":7,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.60899999999999999}]},{"osd":8,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.64800000000000002}]},{"osd":10,"last update":"Wed Sep 13 04:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.68300000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]}]},{"osd":8,"up_from":58,"seq":249108103198,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":490337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.33200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56999999999999995,"15min":0.56499999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.68100000000000005}]},{"osd":1,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.54800000000000004,"15min":0.53600000000000003},"min":{"1min":0.38400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.58199999999999996},"min":{"1min":0.374,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.78600000000000003}]},{"osd":2,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.47499999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.48899999999999999,"15min":0.48599999999999999},"min":{"1min":0.29799999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.48399999999999999}]},{"osd":3,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.434},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64600000000000002,"15min":0.64300000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.61799999999999999}]},{"osd":4,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58899999999999997,"15min":0.58399999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.41099999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.73699999999999999}]},{"osd":5,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.34499999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80700000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.54400000000000004}]},{"osd":6,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.88200000000000001}]},{"osd":7,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71199999999999997,"15min":0.70999999999999996},"min":{"1min":0.48399999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69099999999999995,"15min":0.68700000000000006},"min":{"1min":0.503,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.81200000000000006}]},{"osd":9,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.85399999999999998}]},{"osd":10,"last update":"Wed Sep 13 04:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.82499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]}]},{"osd":7,"up_from":51,"seq":219043332132,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1609728,"data_stored":1082202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.46500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.746,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60499999999999998,"15min":0.60799999999999998},"min":{"1min":0.41599999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.40200000000000002}]},{"osd":1,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.40899999999999997,"15min":0.40300000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42099999999999999,"15min":0.41999999999999998},"min":{"1min":0.30199999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65800000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.43099999999999999}]},{"osd":2,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.42599999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83899999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.57799999999999996,"15min":0.57099999999999995},"min":{"1min":0.443,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.70599999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.80200000000000005}]},{"osd":3,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66600000000000004,"15min":0.66900000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82799999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.64900000000000002}]},{"osd":4,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76600000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.502},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51200000000000001,"15min":0.51500000000000001},"min":{"1min":0.34499999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.47599999999999998}]},{"osd":5,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.628,"15min":0.626},"min":{"1min":0.47799999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64600000000000002,"15min":0.64200000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.85399999999999998}]},{"osd":6,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72199999999999998,"15min":0.72099999999999997},"min":{"1min":0.50600000000000001,"5min":0.496,"15min":0.496},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.58199999999999996}]},{"osd":8,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70299999999999996,"15min":0.70399999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.752},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.51700000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.88100000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.88800000000000001}]},{"osd":9,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.77900000000000003}]},{"osd":10,"last update":"Wed Sep 13 04:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.63}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]}]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":490132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46100000000000002,"15min":0.45100000000000001},"min":{"1min":0.34699999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.503,"15min":0.48999999999999999},"min":{"1min":0.38100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.42999999999999999}]},{"osd":1,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.47099999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82499999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.70999999999999996,"15min":0.67900000000000005},"min":{"1min":0.60399999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.94699999999999995}]},{"osd":2,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.52400000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0129999999999999,"5min":1.054,"15min":1.054},"last":0.83199999999999996}]},{"osd":3,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.66100000000000003,"15min":0.64100000000000001},"min":{"1min":0.47799999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83599999999999997,"5min":1.113,"15min":1.113},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.626,"15min":0.61799999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89900000000000002,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.496}]},{"osd":4,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76400000000000001,"15min":0.747},"min":{"1min":0.67900000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0509999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.75900000000000001,"15min":0.77100000000000002},"min":{"1min":0.52600000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92900000000000005,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.61599999999999999}]},{"osd":5,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.80800000000000005,"15min":0.79200000000000004},"min":{"1min":0.56100000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1950000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.79200000000000004,"15min":0.80800000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.80200000000000005}]},{"osd":7,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.83999999999999997,"15min":0.82399999999999995},"min":{"1min":0.63500000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.79800000000000004,"15min":0.78800000000000003},"min":{"1min":0.58999999999999997,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.57899999999999996}]},{"osd":8,"last update":"Wed Sep 13 04:38:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.81599999999999995,"15min":0.82399999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96599999999999997,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.91900000000000004,"15min":0.92400000000000004},"min":{"1min":0.61199999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2629999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.032}]},{"osd":9,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.0089999999999999}]},{"osd":10,"last update":"Wed Sep 13 04:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.754}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":493884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55600000000000005,"15min":0.53700000000000003},"min":{"1min":0.33700000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56999999999999995,"15min":0.55200000000000005},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.85199999999999998}]},{"osd":2,"last update":"Wed Sep 13 04:37:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.55900000000000005,"15min":0.53800000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.54800000000000004,"15min":0.51800000000000002},"min":{"1min":0.42299999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.73199999999999998}]},{"osd":3,"last update":"Wed Sep 13 04:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83099999999999996,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.623,"15min":0.60899999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91600000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67800000000000005}]},{"osd":4,"last update":"Wed Sep 13 04:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.46200000000000002,"15min":0.45300000000000001},"min":{"1min":0.29599999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.45800000000000002,"15min":0.44800000000000001},"min":{"1min":0.29199999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.497}]},{"osd":5,"last update":"Wed Sep 13 04:37:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.44800000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89000000000000001,"5min":1.3,"15min":1.3},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61499999999999999,"15min":0.58299999999999996},"min":{"1min":0.435,"5min":0.37,"15min":0.37},"max":{"1min":1,"5min":1.264,"15min":1.264},"last":0.78800000000000003}]},{"osd":6,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.99199999999999999,"5min":1.482,"15min":1.482},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95399999999999996,"5min":1.375,"15min":1.375},"last":0.71699999999999997}]},{"osd":7,"last update":"Wed Sep 13 04:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.60699999999999998,"15min":0.58699999999999997},"min":{"1min":0.39900000000000002,"5min":0.372,"15min":0.372},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.65900000000000003,"15min":0.64300000000000002},"min":{"1min":0.47399999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.75800000000000001}]},{"osd":8,"last update":"Wed Sep 13 04:38:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.48899999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2010000000000001,"5min":1.393,"15min":1.393},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.093,"5min":1.224,"15min":1.224},"last":0.77100000000000002}]},{"osd":9,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.81100000000000005}]},{"osd":10,"last update":"Wed Sep 13 04:37:53 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.878}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":491520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 13 04:37:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":0.66000000000000003,"15min":0.58299999999999996},"min":{"1min":0.42199999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.2359999999999998,"5min":6.2359999999999998,"15min":6.2359999999999998},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.58699999999999997,"15min":0.55200000000000005},"min":{"1min":0.47899999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.67000000000000004}]},{"osd":2,"last update":"Wed Sep 13 04:37:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.624,"15min":0.59899999999999998},"min":{"1min":0.42899999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93899999999999995,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63700000000000001,"15min":0.60299999999999998},"min":{"1min":0.439,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.96899999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.89900000000000002}]},{"osd":3,"last update":"Wed Sep 13 04:36:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.57099999999999995,"15min":0.55500000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.45500000000000002}]},{"osd":4,"last update":"Wed Sep 13 04:37:30 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.65900000000000003,"15min":0.61299999999999999},"min":{"1min":0.58699999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.64900000000000002,"15min":0.621},"min":{"1min":0.51400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.75900000000000001}]},{"osd":5,"last update":"Wed Sep 13 04:38:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68300000000000005,"15min":0.67000000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.70699999999999996,"15min":0.68300000000000005},"min":{"1min":0.55100000000000005,"5min":0.435,"15min":0.435},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.85199999999999998}]},{"osd":6,"last update":"Wed Sep 13 04:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.44600000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94899999999999995,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88700000000000001,"5min":1.165,"15min":1.165},"last":0.56999999999999995}]},{"osd":7,"last update":"Wed Sep 13 04:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.79500000000000004,"15min":0.78500000000000003},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.81599999999999995,"15min":0.80200000000000005},"min":{"1min":0.60099999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":1.0109999999999999}]},{"osd":8,"last update":"Wed Sep 13 04:38:00 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":0.90600000000000003,"15min":0.85999999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":6.2089999999999996,"5min":6.2089999999999996,"15min":6.2089999999999996},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.171,"5min":0.87,"15min":0.81999999999999995},"min":{"1min":0.52400000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":6.1699999999999999,"5min":6.1699999999999999,"15min":6.1699999999999999},"last":0.98899999999999999}]},{"osd":9,"last update":"Wed Sep 13 04:37:30 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.874},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.96699999999999997}]},{"osd":10,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":1.048},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":6.1859999999999999,"5min":6.1859999999999999,"15min":6.1859999999999999},"last":0.83499999999999996}]},{"osd":11,"last update":"Wed Sep 13 04:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.79900000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":495200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:55 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.34599999999999997,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79100000000000004,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51900000000000002,"15min":0.51800000000000002},"min":{"1min":0.371,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78300000000000003,"5min":1.429,"15min":1.429},"last":0.45100000000000001}]},{"osd":1,"last update":"Wed Sep 13 04:37:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.53500000000000003,"15min":0.496},"min":{"1min":0.39000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80700000000000005,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54300000000000004,"15min":0.50700000000000001},"min":{"1min":0.39100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70799999999999996,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.755}]},{"osd":3,"last update":"Wed Sep 13 04:38:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.36499999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.747,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.441,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81399999999999995,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.66600000000000004}]},{"osd":4,"last update":"Wed Sep 13 04:37:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.52700000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78000000000000003,"5min":1.244,"15min":1.244},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55800000000000005,"15min":0.53000000000000003},"min":{"1min":0.436,"5min":0.374,"15min":0.374},"max":{"1min":0.85499999999999998,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.79400000000000004}]},{"osd":5,"last update":"Wed Sep 13 04:37:57 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.28699999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.70899999999999996,"5min":1.796,"15min":1.796},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.30599999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.99299999999999999,"5min":1.546,"15min":1.546},"last":0.49399999999999999}]},{"osd":6,"last update":"Wed Sep 13 04:38:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.69199999999999995,"15min":0.69899999999999995},"min":{"1min":0.48499999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81599999999999995,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66700000000000004,"15min":0.67200000000000004},"min":{"1min":0.48199999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85899999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.77100000000000002}]},{"osd":7,"last update":"Wed Sep 13 04:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65400000000000003,"15min":0.65800000000000003},"min":{"1min":0.42499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94699999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.497,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92500000000000004,"5min":1.147,"15min":1.147},"last":0.88300000000000001}]},{"osd":8,"last update":"Wed Sep 13 04:38:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61699999999999999,"15min":0.61899999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.96599999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.38400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.82699999999999996}]},{"osd":9,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.84199999999999997}]},{"osd":10,"last update":"Wed Sep 13 04:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.98199999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":490326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.42699999999999999,"15min":0.39000000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.46500000000000002,"15min":0.435},"min":{"1min":0.375,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78500000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46100000000000002}]},{"osd":1,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.57699999999999996},"min":{"1min":0.45700000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64200000000000002,"15min":0.57699999999999996},"min":{"1min":0.436,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92200000000000004,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.76300000000000001}]},{"osd":2,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.67400000000000004,"15min":0.629},"min":{"1min":0.53000000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64500000000000002,"15min":0.621},"min":{"1min":0.45800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.59799999999999998}]},{"osd":4,"last update":"Wed Sep 13 04:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.64300000000000002,"15min":0.60099999999999998},"min":{"1min":0.52900000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.64100000000000001,"15min":0.60699999999999998},"min":{"1min":0.44500000000000001,"5min":0.439,"15min":0.439},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.70399999999999996}]},{"osd":5,"last update":"Wed Sep 13 04:38:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.68999999999999995,"15min":0.65000000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.024,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.873,"5min":0.72399999999999998,"15min":0.66600000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":0.82099999999999995}]},{"osd":6,"last update":"Wed Sep 13 04:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.74099999999999999,"15min":0.70299999999999996},"min":{"1min":0.47399999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1060000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.76800000000000002,"15min":0.745},"min":{"1min":0.504,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0529999999999999,"5min":1.109,"15min":1.109},"last":0.879}]},{"osd":7,"last update":"Wed Sep 13 04:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.86299999999999999,"15min":0.86399999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.85399999999999998,"15min":0.84999999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.90800000000000003}]},{"osd":8,"last update":"Wed Sep 13 04:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.875}]},{"osd":9,"last update":"Wed Sep 13 04:38:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.94699999999999995,"15min":0.96399999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.2030000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.84099999999999997,"15min":0.84899999999999998},"min":{"1min":0.52100000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.01,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.65900000000000003}]},{"osd":10,"last update":"Wed Sep 13 04:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.94599999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":490064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.56599999999999995,"15min":0.54400000000000004},"min":{"1min":0.43099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.753},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76500000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.68999999999999995}]},{"osd":1,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.373,"15min":0.34699999999999998},"min":{"1min":0.28399999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.38700000000000001,"15min":0.36399999999999999},"min":{"1min":0.28100000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.48499999999999999}]},{"osd":2,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58299999999999996,"15min":0.56100000000000005},"min":{"1min":0.41099999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56599999999999995,"15min":0.53800000000000003},"min":{"1min":0.41399999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.64300000000000002}]},{"osd":3,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.59399999999999997,"15min":0.56499999999999995},"min":{"1min":0.41799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59399999999999997,"15min":0.57699999999999996},"min":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.68400000000000005}]},{"osd":5,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60299999999999998,"15min":0.58599999999999997},"min":{"1min":0.45100000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90300000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.39700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.75800000000000001}]},{"osd":6,"last update":"Wed Sep 13 04:38:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.68000000000000005,"15min":0.68400000000000005},"min":{"1min":0.47099999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81599999999999995,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.69799999999999995,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.84599999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52400000000000002,"15min":0.51900000000000002},"min":{"1min":0.36199999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73599999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.70099999999999996}]},{"osd":8,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.70699999999999996,"15min":0.69199999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76700000000000002,"15min":0.76300000000000001},"min":{"1min":0.53000000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.77600000000000002}]},{"osd":9,"last update":"Wed Sep 13 04:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.86199999999999999}]},{"osd":10,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.82899999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1622016,"data_stored":1081886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53000000000000003,"15min":0.52400000000000002},"min":{"1min":0.34399999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72099999999999997,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.317,"5min":0.315,"15min":0.315},"max":{"1min":0.70299999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.84799999999999998}]},{"osd":1,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.40300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78500000000000003,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.54300000000000004,"15min":0.52400000000000002},"min":{"1min":0.34799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89600000000000002,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.60399999999999998}]},{"osd":2,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.38,"15min":0.36099999999999999},"min":{"1min":0.26900000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.37},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.39600000000000002,"15min":0.378},"min":{"1min":0.28299999999999997,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.41199999999999998}]},{"osd":3,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.372,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92600000000000005,"5min":1.036,"15min":1.036},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.78700000000000003,"15min":0.86899999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.1259999999999999,"5min":8.0009999999999994,"15min":8.0009999999999994},"last":0.77700000000000002}]},{"osd":4,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.58299999999999996,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82299999999999995,"5min":1.599,"15min":1.599},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.57699999999999996,"15min":0.55500000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.86599999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.623}]},{"osd":6,"last update":"Wed Sep 13 04:38:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83499999999999996,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81899999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.72999999999999998}]},{"osd":7,"last update":"Wed Sep 13 04:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.878,"5min":1.488,"15min":1.488},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94899999999999995,"5min":1.51,"15min":1.51},"last":0.63900000000000001}]},{"osd":8,"last update":"Wed Sep 13 04:38:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.38900000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.60599999999999998,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.56999999999999995}]},{"osd":9,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.70099999999999996}]},{"osd":10,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.68500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-13T04:38:30.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:30 smithi149 ceph-mon[133315]: from='client.24844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:38:30.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:30 smithi149 ceph-mon[133315]: pgmap v293: 129 pgs: 129 active+clean; 583 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 1.9 KiB/s wr, 8 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-09-13T04:38:30.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:30 smithi156 ceph-mon[133675]: from='client.24844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:38:30.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:30 smithi156 ceph-mon[133675]: pgmap v293: 129 pgs: 129 active+clean; 583 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 1.9 KiB/s wr, 8 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-09-13T04:38:31.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:30 smithi002 ceph-mon[97651]: from='client.24844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:38:31.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:30 smithi002 ceph-mon[97651]: pgmap v293: 129 pgs: 129 active+clean; 583 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 1.9 KiB/s wr, 8 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-09-13T04:38:31.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:31 smithi149 ceph-mon[133315]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:38:31.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:31 smithi156 ceph-mon[133675]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:38:32.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:31 smithi002 ceph-mon[97651]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:38:32.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:32 smithi149 ceph-mon[133315]: pgmap v294: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.1 KiB/s wr, 15 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-09-13T04:38:32.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:32 smithi156 ceph-mon[133675]: pgmap v294: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.1 KiB/s wr, 15 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-09-13T04:38:33.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:32 smithi002 ceph-mon[97651]: pgmap v294: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.1 KiB/s wr, 15 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-09-13T04:38:33.104 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-13T04:38:33.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.319+0000 7fc4409ff700 1 -- 172.21.15.2:0/1008059799 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc43c102880 msgr2=0x7fc43c109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:33.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.319+0000 7fc4409ff700 1 --2- 172.21.15.2:0/1008059799 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc43c102880 0x7fc43c109130 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc43400b0d0 tx=0x7fc43400b3e0 comp rx=0 tx=0).stop 2023-09-13T04:38:33.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.320+0000 7fc4409ff700 1 -- 172.21.15.2:0/1008059799 shutdown_connections 2023-09-13T04:38:33.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.320+0000 7fc4409ff700 1 --2- 172.21.15.2:0/1008059799 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc43c102880 0x7fc43c109130 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:33.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.320+0000 7fc4409ff700 1 --2- 172.21.15.2:0/1008059799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc43c100cf0 0x7fc43c101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:33.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.320+0000 7fc4409ff700 1 --2- 172.21.15.2:0/1008059799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc43c101ee0 0x7fc43c102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:33.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.320+0000 7fc4409ff700 1 -- 172.21.15.2:0/1008059799 >> 172.21.15.2:0/1008059799 conn(0x7fc43c0fc220 msgr2=0x7fc43c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:33.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.320+0000 7fc4409ff700 1 -- 172.21.15.2:0/1008059799 shutdown_connections 2023-09-13T04:38:33.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.320+0000 7fc4409ff700 1 -- 172.21.15.2:0/1008059799 wait complete. 2023-09-13T04:38:33.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.321+0000 7fc4409ff700 1 Processor -- start 2023-09-13T04:38:33.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.321+0000 7fc4409ff700 1 -- start start 2023-09-13T04:38:33.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.322+0000 7fc4409ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc43c100cf0 0x7fc43c198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:33.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.322+0000 7fc4409ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc43c101ee0 0x7fc43c199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:33.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.322+0000 7fc4409ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc43c102880 0x7fc43c1a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:33.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.322+0000 7fc4409ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c1a0830 con 0x7fc43c100cf0 2023-09-13T04:38:33.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.322+0000 7fc4409ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c1a09a0 con 0x7fc43c102880 2023-09-13T04:38:33.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.322+0000 7fc4409ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c1a0b60 con 0x7fc43c101ee0 2023-09-13T04:38:33.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.322+0000 7fc43a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc43c100cf0 0x7fc43c198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:33.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.322+0000 7fc43a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc43c100cf0 0x7fc43c198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38190/0 (socket says 172.21.15.2:38190) 2023-09-13T04:38:33.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.322+0000 7fc43a59c700 1 -- 172.21.15.2:0/3563109629 learned_addr learned my addr 172.21.15.2:0/3563109629 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:33.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.322+0000 7fc439d9b700 1 --2- 172.21.15.2:0/3563109629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc43c101ee0 0x7fc43c199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:33.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.322+0000 7fc43ad9d700 1 --2- 172.21.15.2:0/3563109629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc43c102880 0x7fc43c1a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:33.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.323+0000 7fc43a59c700 1 -- 172.21.15.2:0/3563109629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc43c101ee0 msgr2=0x7fc43c199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:33.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.323+0000 7fc43a59c700 1 --2- 172.21.15.2:0/3563109629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc43c101ee0 0x7fc43c199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:33.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.323+0000 7fc43a59c700 1 -- 172.21.15.2:0/3563109629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc43c102880 msgr2=0x7fc43c1a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:33.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.323+0000 7fc43a59c700 1 --2- 172.21.15.2:0/3563109629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc43c102880 0x7fc43c1a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:33.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.323+0000 7fc43a59c700 1 -- 172.21.15.2:0/3563109629 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc434009d00 con 0x7fc43c100cf0 2023-09-13T04:38:33.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.323+0000 7fc43a59c700 1 --2- 172.21.15.2:0/3563109629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc43c100cf0 0x7fc43c198cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00e970 tx=0x7fc42c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:33.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.323+0000 7fc42b7fe700 1 -- 172.21.15.2:0/3563109629 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c009790 con 0x7fc43c100cf0 2023-09-13T04:38:33.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.323+0000 7fc42b7fe700 1 -- 172.21.15.2:0/3563109629 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc42c00c7e0 con 0x7fc43c100cf0 2023-09-13T04:38:33.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.323+0000 7fc42b7fe700 1 -- 172.21.15.2:0/3563109629 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c003c70 con 0x7fc43c100cf0 2023-09-13T04:38:33.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.323+0000 7fc4409ff700 1 -- 172.21.15.2:0/3563109629 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc43c1a0e40 con 0x7fc43c100cf0 2023-09-13T04:38:33.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.323+0000 7fc4409ff700 1 -- 172.21.15.2:0/3563109629 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc43c1a1210 con 0x7fc43c100cf0 2023-09-13T04:38:33.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.324+0000 7fc4409ff700 1 -- 172.21.15.2:0/3563109629 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc43c10c560 con 0x7fc43c100cf0 2023-09-13T04:38:33.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.326+0000 7fc42b7fe700 1 -- 172.21.15.2:0/3563109629 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fc42c00cc50 con 0x7fc43c100cf0 2023-09-13T04:38:33.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.328+0000 7fc42b7fe700 1 --2- 172.21.15.2:0/3563109629 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc4240a0710 0x7fc4240a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:33.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.328+0000 7fc42b7fe700 1 -- 172.21.15.2:0/3563109629 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc42c014070 con 0x7fc43c100cf0 2023-09-13T04:38:33.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.328+0000 7fc439d9b700 1 --2- 172.21.15.2:0/3563109629 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc4240a0710 0x7fc4240a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:33.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.328+0000 7fc439d9b700 1 --2- 172.21.15.2:0/3563109629 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc4240a0710 0x7fc4240a2bd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc43c19a1c0 tx=0x7fc430009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:33.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.331+0000 7fc42b7fe700 1 -- 172.21.15.2:0/3563109629 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc42c095820 con 0x7fc43c100cf0 2023-09-13T04:38:33.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.541+0000 7fc4409ff700 1 -- 172.21.15.2:0/3563109629 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc43c108e50 con 0x7fc4240a0710 2023-09-13T04:38:33.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.554+0000 7fc42b7fe700 1 -- 172.21.15.2:0/3563109629 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422610 (secure 0 0 0) 0x7fc43c108e50 con 0x7fc4240a0710 2023-09-13T04:38:33.562 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:38:33.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.559+0000 7fc4409ff700 1 -- 172.21.15.2:0/3563109629 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc4240a0710 msgr2=0x7fc4240a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:33.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.559+0000 7fc4409ff700 1 --2- 172.21.15.2:0/3563109629 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc4240a0710 0x7fc4240a2bd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc43c19a1c0 tx=0x7fc430009260 comp rx=0 tx=0).stop 2023-09-13T04:38:33.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.559+0000 7fc4409ff700 1 -- 172.21.15.2:0/3563109629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc43c100cf0 msgr2=0x7fc43c198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:33.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.559+0000 7fc4409ff700 1 --2- 172.21.15.2:0/3563109629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc43c100cf0 0x7fc43c198cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00e970 tx=0x7fc42c00ec80 comp rx=0 tx=0).stop 2023-09-13T04:38:33.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.560+0000 7fc4409ff700 1 -- 172.21.15.2:0/3563109629 shutdown_connections 2023-09-13T04:38:33.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.560+0000 7fc4409ff700 1 --2- 172.21.15.2:0/3563109629 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fc4240a0710 0x7fc4240a2bd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:33.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.560+0000 7fc4409ff700 1 --2- 172.21.15.2:0/3563109629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc43c102880 0x7fc43c1a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:33.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.560+0000 7fc4409ff700 1 --2- 172.21.15.2:0/3563109629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc43c100cf0 0x7fc43c198cc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:33.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.560+0000 7fc4409ff700 1 --2- 172.21.15.2:0/3563109629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc43c101ee0 0x7fc43c199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:33.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.560+0000 7fc4409ff700 1 -- 172.21.15.2:0/3563109629 >> 172.21.15.2:0/3563109629 conn(0x7fc43c0fc220 msgr2=0x7fc43c107730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:33.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.560+0000 7fc4409ff700 1 -- 172.21.15.2:0/3563109629 shutdown_connections 2023-09-13T04:38:33.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:33.560+0000 7fc4409ff700 1 -- 172.21.15.2:0/3563109629 wait complete. 2023-09-13T04:38:33.568 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-09-13T04:38:33.580 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2023-09-13T04:38:33.507706+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597820,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":182,"num_read_kb":155,"num_write":310,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":250,"ondisk_log_size":250,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337928,"kb_used_data":13684,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603896,"statfs":{"total":1151940427776,"available":1151594389504,"internally_reserved":0,"allocated":14012416,"data_stored":7673352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2953,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":46,"num_read_kb":36,"num_write":81,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.922155"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592832+0000","last_change":"2023-09-13T04:37:59.244279+0000","last_active":"2023-09-13T04:38:23.592832+0000","last_peered":"2023-09-13T04:38:23.592832+0000","last_clean":"2023-09-13T04:38:23.592832+0000","last_became_active":"2023-09-13T04:37:59.243976+0000","last_became_peered":"2023-09-13T04:37:59.243976+0000","last_unstale":"2023-09-13T04:38:23.592832+0000","last_undegraded":"2023-09-13T04:38:23.592832+0000","last_fullsized":"2023-09-13T04:38:23.592832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:21:52.809411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592579+0000","last_change":"2023-09-13T04:37:59.243391+0000","last_active":"2023-09-13T04:38:23.592579+0000","last_peered":"2023-09-13T04:38:23.592579+0000","last_clean":"2023-09-13T04:38:23.592579+0000","last_became_active":"2023-09-13T04:37:59.243044+0000","last_became_peered":"2023-09-13T04:37:59.243044+0000","last_unstale":"2023-09-13T04:38:23.592579+0000","last_undegraded":"2023-09-13T04:38:23.592579+0000","last_fullsized":"2023-09-13T04:38:23.592579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:02:17.806638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592756+0000","last_change":"2023-09-13T04:37:59.242826+0000","last_active":"2023-09-13T04:38:23.592756+0000","last_peered":"2023-09-13T04:38:23.592756+0000","last_clean":"2023-09-13T04:38:23.592756+0000","last_became_active":"2023-09-13T04:37:59.242125+0000","last_became_peered":"2023-09-13T04:37:59.242125+0000","last_unstale":"2023-09-13T04:38:23.592756+0000","last_undegraded":"2023-09-13T04:38:23.592756+0000","last_fullsized":"2023-09-13T04:38:23.592756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:08:29.216093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590536+0000","last_change":"2023-09-13T04:37:59.253322+0000","last_active":"2023-09-13T04:38:23.590536+0000","last_peered":"2023-09-13T04:38:23.590536+0000","last_clean":"2023-09-13T04:38:23.590536+0000","last_became_active":"2023-09-13T04:37:59.253050+0000","last_became_peered":"2023-09-13T04:37:59.253050+0000","last_unstale":"2023-09-13T04:38:23.590536+0000","last_undegraded":"2023-09-13T04:38:23.590536+0000","last_fullsized":"2023-09-13T04:38:23.590536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:45:22.357879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593487+0000","last_change":"2023-09-13T04:37:59.250089+0000","last_active":"2023-09-13T04:38:23.593487+0000","last_peered":"2023-09-13T04:38:23.593487+0000","last_clean":"2023-09-13T04:38:23.593487+0000","last_became_active":"2023-09-13T04:37:59.249771+0000","last_became_peered":"2023-09-13T04:37:59.249771+0000","last_unstale":"2023-09-13T04:38:23.593487+0000","last_undegraded":"2023-09-13T04:38:23.593487+0000","last_fullsized":"2023-09-13T04:38:23.593487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:46:59.917756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595139+0000","last_change":"2023-09-13T04:37:59.247259+0000","last_active":"2023-09-13T04:38:23.595139+0000","last_peered":"2023-09-13T04:38:23.595139+0000","last_clean":"2023-09-13T04:38:23.595139+0000","last_became_active":"2023-09-13T04:37:59.246911+0000","last_became_peered":"2023-09-13T04:37:59.246911+0000","last_unstale":"2023-09-13T04:38:23.595139+0000","last_undegraded":"2023-09-13T04:38:23.595139+0000","last_fullsized":"2023-09-13T04:38:23.595139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:00:39.742392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587857+0000","last_change":"2023-09-13T04:37:59.244083+0000","last_active":"2023-09-13T04:38:23.587857+0000","last_peered":"2023-09-13T04:38:23.587857+0000","last_clean":"2023-09-13T04:38:23.587857+0000","last_became_active":"2023-09-13T04:37:59.243789+0000","last_became_peered":"2023-09-13T04:37:59.243789+0000","last_unstale":"2023-09-13T04:38:23.587857+0000","last_undegraded":"2023-09-13T04:38:23.587857+0000","last_fullsized":"2023-09-13T04:38:23.587857+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:24:00.622790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.602097+0000","last_change":"2023-09-13T04:38:24.601265+0000","last_active":"2023-09-13T04:38:24.602097+0000","last_peered":"2023-09-13T04:38:24.602097+0000","last_clean":"2023-09-13T04:38:24.602097+0000","last_became_active":"2023-09-13T04:38:24.600939+0000","last_became_peered":"2023-09-13T04:38:24.600939+0000","last_unstale":"2023-09-13T04:38:24.602097+0000","last_undegraded":"2023-09-13T04:38:24.602097+0000","last_fullsized":"2023-09-13T04:38:24.602097+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:18:23.642333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594759+0000","last_change":"2023-09-13T04:37:59.238053+0000","last_active":"2023-09-13T04:38:23.594759+0000","last_peered":"2023-09-13T04:38:23.594759+0000","last_clean":"2023-09-13T04:38:23.594759+0000","last_became_active":"2023-09-13T04:37:59.237777+0000","last_became_peered":"2023-09-13T04:37:59.237777+0000","last_unstale":"2023-09-13T04:38:23.594759+0000","last_undegraded":"2023-09-13T04:38:23.594759+0000","last_fullsized":"2023-09-13T04:38:23.594759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:45:39.835659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594929+0000","last_change":"2023-09-13T04:37:59.252605+0000","last_active":"2023-09-13T04:38:23.594929+0000","last_peered":"2023-09-13T04:38:23.594929+0000","last_clean":"2023-09-13T04:38:23.594929+0000","last_became_active":"2023-09-13T04:37:59.252309+0000","last_became_peered":"2023-09-13T04:37:59.252309+0000","last_unstale":"2023-09-13T04:38:23.594929+0000","last_undegraded":"2023-09-13T04:38:23.594929+0000","last_fullsized":"2023-09-13T04:38:23.594929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:41:45.064230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588277+0000","last_change":"2023-09-13T04:37:59.243345+0000","last_active":"2023-09-13T04:38:23.588277+0000","last_peered":"2023-09-13T04:38:23.588277+0000","last_clean":"2023-09-13T04:38:23.588277+0000","last_became_active":"2023-09-13T04:37:59.242726+0000","last_became_peered":"2023-09-13T04:37:59.242726+0000","last_unstale":"2023-09-13T04:38:23.588277+0000","last_undegraded":"2023-09-13T04:38:23.588277+0000","last_fullsized":"2023-09-13T04:38:23.588277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:39:55.115281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590601+0000","last_change":"2023-09-13T04:37:59.247415+0000","last_active":"2023-09-13T04:38:23.590601+0000","last_peered":"2023-09-13T04:38:23.590601+0000","last_clean":"2023-09-13T04:38:23.590601+0000","last_became_active":"2023-09-13T04:37:59.247050+0000","last_became_peered":"2023-09-13T04:37:59.247050+0000","last_unstale":"2023-09-13T04:38:23.590601+0000","last_undegraded":"2023-09-13T04:38:23.590601+0000","last_fullsized":"2023-09-13T04:38:23.590601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:03:44.793628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597902+0000","last_change":"2023-09-13T04:37:59.248935+0000","last_active":"2023-09-13T04:38:23.597902+0000","last_peered":"2023-09-13T04:38:23.597902+0000","last_clean":"2023-09-13T04:38:23.597902+0000","last_became_active":"2023-09-13T04:37:59.248554+0000","last_became_peered":"2023-09-13T04:37:59.248554+0000","last_unstale":"2023-09-13T04:38:23.597902+0000","last_undegraded":"2023-09-13T04:38:23.597902+0000","last_fullsized":"2023-09-13T04:38:23.597902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:53:31.882567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592020+0000","last_change":"2023-09-13T04:37:59.242851+0000","last_active":"2023-09-13T04:38:23.592020+0000","last_peered":"2023-09-13T04:38:23.592020+0000","last_clean":"2023-09-13T04:38:23.592020+0000","last_became_active":"2023-09-13T04:37:59.242384+0000","last_became_peered":"2023-09-13T04:37:59.242384+0000","last_unstale":"2023-09-13T04:38:23.592020+0000","last_undegraded":"2023-09-13T04:38:23.592020+0000","last_fullsized":"2023-09-13T04:38:23.592020+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:59:36.508808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.599392+0000","last_change":"2023-09-13T04:38:24.599392+0000","last_active":"2023-09-13T04:38:24.599392+0000","last_peered":"2023-09-13T04:38:24.599392+0000","last_clean":"2023-09-13T04:38:24.599392+0000","last_became_active":"2023-09-13T04:38:24.599029+0000","last_became_peered":"2023-09-13T04:38:24.599029+0000","last_unstale":"2023-09-13T04:38:24.599392+0000","last_undegraded":"2023-09-13T04:38:24.599392+0000","last_fullsized":"2023-09-13T04:38:24.599392+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:23:03.703387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592657+0000","last_change":"2023-09-13T04:37:59.249107+0000","last_active":"2023-09-13T04:38:23.592657+0000","last_peered":"2023-09-13T04:38:23.592657+0000","last_clean":"2023-09-13T04:38:23.592657+0000","last_became_active":"2023-09-13T04:37:59.248803+0000","last_became_peered":"2023-09-13T04:37:59.248803+0000","last_unstale":"2023-09-13T04:38:23.592657+0000","last_undegraded":"2023-09-13T04:38:23.592657+0000","last_fullsized":"2023-09-13T04:38:23.592657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:50:52.481194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592743+0000","last_change":"2023-09-13T04:37:59.243724+0000","last_active":"2023-09-13T04:38:23.592743+0000","last_peered":"2023-09-13T04:38:23.592743+0000","last_clean":"2023-09-13T04:38:23.592743+0000","last_became_active":"2023-09-13T04:37:59.243385+0000","last_became_peered":"2023-09-13T04:37:59.243385+0000","last_unstale":"2023-09-13T04:38:23.592743+0000","last_undegraded":"2023-09-13T04:38:23.592743+0000","last_fullsized":"2023-09-13T04:38:23.592743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:55:55.203940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588550+0000","last_change":"2023-09-13T04:37:59.238986+0000","last_active":"2023-09-13T04:38:23.588550+0000","last_peered":"2023-09-13T04:38:23.588550+0000","last_clean":"2023-09-13T04:38:23.588550+0000","last_became_active":"2023-09-13T04:37:59.238668+0000","last_became_peered":"2023-09-13T04:37:59.238668+0000","last_unstale":"2023-09-13T04:38:23.588550+0000","last_undegraded":"2023-09-13T04:38:23.588550+0000","last_fullsized":"2023-09-13T04:38:23.588550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:21:10.225223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588585+0000","last_change":"2023-09-13T04:37:59.245598+0000","last_active":"2023-09-13T04:38:23.588585+0000","last_peered":"2023-09-13T04:38:23.588585+0000","last_clean":"2023-09-13T04:38:23.588585+0000","last_became_active":"2023-09-13T04:37:59.244652+0000","last_became_peered":"2023-09-13T04:37:59.244652+0000","last_unstale":"2023-09-13T04:38:23.588585+0000","last_undegraded":"2023-09-13T04:38:23.588585+0000","last_fullsized":"2023-09-13T04:38:23.588585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:34:43.671168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588238+0000","last_change":"2023-09-13T04:38:03.617708+0000","last_active":"2023-09-13T04:38:23.588238+0000","last_peered":"2023-09-13T04:38:23.588238+0000","last_clean":"2023-09-13T04:38:23.588238+0000","last_became_active":"2023-09-13T04:38:03.617277+0000","last_became_peered":"2023-09-13T04:38:03.617277+0000","last_unstale":"2023-09-13T04:38:23.588238+0000","last_undegraded":"2023-09-13T04:38:23.588238+0000","last_fullsized":"2023-09-13T04:38:23.588238+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:26:44.318030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.621267+0000","last_change":"2023-09-13T04:37:58.595156+0000","last_active":"2023-09-13T04:38:25.621267+0000","last_peered":"2023-09-13T04:38:25.621267+0000","last_clean":"2023-09-13T04:38:25.621267+0000","last_became_active":"2023-09-13T04:37:58.594863+0000","last_became_peered":"2023-09-13T04:37:58.594863+0000","last_unstale":"2023-09-13T04:38:25.621267+0000","last_undegraded":"2023-09-13T04:38:25.621267+0000","last_fullsized":"2023-09-13T04:38:25.621267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:27:14.420495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595813+0000","last_change":"2023-09-13T04:37:59.250252+0000","last_active":"2023-09-13T04:38:23.595813+0000","last_peered":"2023-09-13T04:38:23.595813+0000","last_clean":"2023-09-13T04:38:23.595813+0000","last_became_active":"2023-09-13T04:37:59.249927+0000","last_became_peered":"2023-09-13T04:37:59.249927+0000","last_unstale":"2023-09-13T04:38:23.595813+0000","last_undegraded":"2023-09-13T04:38:23.595813+0000","last_fullsized":"2023-09-13T04:38:23.595813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:18:06.757040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.599683+0000","last_change":"2023-09-13T04:38:24.599683+0000","last_active":"2023-09-13T04:38:24.599683+0000","last_peered":"2023-09-13T04:38:24.599683+0000","last_clean":"2023-09-13T04:38:24.599683+0000","last_became_active":"2023-09-13T04:38:24.599230+0000","last_became_peered":"2023-09-13T04:38:24.599230+0000","last_unstale":"2023-09-13T04:38:24.599683+0000","last_undegraded":"2023-09-13T04:38:24.599683+0000","last_fullsized":"2023-09-13T04:38:24.599683+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:48:22.584735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.596874+0000","last_change":"2023-09-13T04:37:58.597785+0000","last_active":"2023-09-13T04:38:24.596874+0000","last_peered":"2023-09-13T04:38:24.596874+0000","last_clean":"2023-09-13T04:38:24.596874+0000","last_became_active":"2023-09-13T04:37:58.597485+0000","last_became_peered":"2023-09-13T04:37:58.597485+0000","last_unstale":"2023-09-13T04:38:24.596874+0000","last_undegraded":"2023-09-13T04:38:24.596874+0000","last_fullsized":"2023-09-13T04:38:24.596874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:39:57.156038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592830+0000","last_change":"2023-09-13T04:37:59.241840+0000","last_active":"2023-09-13T04:38:23.592830+0000","last_peered":"2023-09-13T04:38:23.592830+0000","last_clean":"2023-09-13T04:38:23.592830+0000","last_became_active":"2023-09-13T04:37:59.241545+0000","last_became_peered":"2023-09-13T04:37:59.241545+0000","last_unstale":"2023-09-13T04:38:23.592830+0000","last_undegraded":"2023-09-13T04:38:23.592830+0000","last_fullsized":"2023-09-13T04:38:23.592830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:04:17.439520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591101+0000","last_change":"2023-09-13T04:38:03.621792+0000","last_active":"2023-09-13T04:38:23.591101+0000","last_peered":"2023-09-13T04:38:23.591101+0000","last_clean":"2023-09-13T04:38:23.591101+0000","last_became_active":"2023-09-13T04:38:03.621433+0000","last_became_peered":"2023-09-13T04:38:03.621433+0000","last_unstale":"2023-09-13T04:38:23.591101+0000","last_undegraded":"2023-09-13T04:38:23.591101+0000","last_fullsized":"2023-09-13T04:38:23.591101+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:52:05.926733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.618200+0000","last_change":"2023-09-13T04:37:58.590817+0000","last_active":"2023-09-13T04:38:25.618200+0000","last_peered":"2023-09-13T04:38:25.618200+0000","last_clean":"2023-09-13T04:38:25.618200+0000","last_became_active":"2023-09-13T04:37:58.590348+0000","last_became_peered":"2023-09-13T04:37:58.590348+0000","last_unstale":"2023-09-13T04:38:25.618200+0000","last_undegraded":"2023-09-13T04:38:25.618200+0000","last_fullsized":"2023-09-13T04:38:25.618200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:38:35.680905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594252+0000","last_change":"2023-09-13T04:37:59.237304+0000","last_active":"2023-09-13T04:38:23.594252+0000","last_peered":"2023-09-13T04:38:23.594252+0000","last_clean":"2023-09-13T04:38:23.594252+0000","last_became_active":"2023-09-13T04:37:59.237008+0000","last_became_peered":"2023-09-13T04:37:59.237008+0000","last_unstale":"2023-09-13T04:38:23.594252+0000","last_undegraded":"2023-09-13T04:38:23.594252+0000","last_fullsized":"2023-09-13T04:38:23.594252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:28:14.254219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590193+0000","last_change":"2023-09-13T04:38:03.606749+0000","last_active":"2023-09-13T04:38:23.590193+0000","last_peered":"2023-09-13T04:38:23.590193+0000","last_clean":"2023-09-13T04:38:23.590193+0000","last_became_active":"2023-09-13T04:38:03.606276+0000","last_became_peered":"2023-09-13T04:38:03.606276+0000","last_unstale":"2023-09-13T04:38:23.590193+0000","last_undegraded":"2023-09-13T04:38:23.590193+0000","last_fullsized":"2023-09-13T04:38:23.590193+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:30:53.572543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593692+0000","last_change":"2023-09-13T04:37:58.590937+0000","last_active":"2023-09-13T04:38:23.593692+0000","last_peered":"2023-09-13T04:38:23.593692+0000","last_clean":"2023-09-13T04:38:23.593692+0000","last_became_active":"2023-09-13T04:37:58.590606+0000","last_became_peered":"2023-09-13T04:37:58.590606+0000","last_unstale":"2023-09-13T04:38:23.593692+0000","last_undegraded":"2023-09-13T04:38:23.593692+0000","last_fullsized":"2023-09-13T04:38:23.593692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:30:03.354670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591224+0000","last_change":"2023-09-13T04:37:59.251899+0000","last_active":"2023-09-13T04:38:23.591224+0000","last_peered":"2023-09-13T04:38:23.591224+0000","last_clean":"2023-09-13T04:38:23.591224+0000","last_became_active":"2023-09-13T04:37:59.251557+0000","last_became_peered":"2023-09-13T04:37:59.251557+0000","last_unstale":"2023-09-13T04:38:23.591224+0000","last_undegraded":"2023-09-13T04:38:23.591224+0000","last_fullsized":"2023-09-13T04:38:23.591224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:57:45.241188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588439+0000","last_change":"2023-09-13T04:38:03.606076+0000","last_active":"2023-09-13T04:38:23.588439+0000","last_peered":"2023-09-13T04:38:23.588439+0000","last_clean":"2023-09-13T04:38:23.588439+0000","last_became_active":"2023-09-13T04:38:03.605556+0000","last_became_peered":"2023-09-13T04:38:03.605556+0000","last_unstale":"2023-09-13T04:38:23.588439+0000","last_undegraded":"2023-09-13T04:38:23.588439+0000","last_fullsized":"2023-09-13T04:38:23.588439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:03:32.344625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:29.622950+0000","last_change":"2023-09-13T04:37:58.588170+0000","last_active":"2023-09-13T04:38:29.622950+0000","last_peered":"2023-09-13T04:38:29.622950+0000","last_clean":"2023-09-13T04:38:29.622950+0000","last_became_active":"2023-09-13T04:37:58.587860+0000","last_became_peered":"2023-09-13T04:37:58.587860+0000","last_unstale":"2023-09-13T04:38:29.622950+0000","last_undegraded":"2023-09-13T04:38:29.622950+0000","last_fullsized":"2023-09-13T04:38:29.622950+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:38:40.048150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595472+0000","last_change":"2023-09-13T04:37:59.240248+0000","last_active":"2023-09-13T04:38:23.595472+0000","last_peered":"2023-09-13T04:38:23.595472+0000","last_clean":"2023-09-13T04:38:23.595472+0000","last_became_active":"2023-09-13T04:37:59.240008+0000","last_became_peered":"2023-09-13T04:37:59.240008+0000","last_unstale":"2023-09-13T04:38:23.595472+0000","last_undegraded":"2023-09-13T04:38:23.595472+0000","last_fullsized":"2023-09-13T04:38:23.595472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:15:13.275110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587821+0000","last_change":"2023-09-13T04:38:03.616607+0000","last_active":"2023-09-13T04:38:23.587821+0000","last_peered":"2023-09-13T04:38:23.587821+0000","last_clean":"2023-09-13T04:38:23.587821+0000","last_became_active":"2023-09-13T04:38:03.616209+0000","last_became_peered":"2023-09-13T04:38:03.616209+0000","last_unstale":"2023-09-13T04:38:23.587821+0000","last_undegraded":"2023-09-13T04:38:23.587821+0000","last_fullsized":"2023-09-13T04:38:23.587821+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:24:21.934053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.617180+0000","last_change":"2023-09-13T04:37:58.589104+0000","last_active":"2023-09-13T04:38:25.617180+0000","last_peered":"2023-09-13T04:38:25.617180+0000","last_clean":"2023-09-13T04:38:25.617180+0000","last_became_active":"2023-09-13T04:37:58.588780+0000","last_became_peered":"2023-09-13T04:37:58.588780+0000","last_unstale":"2023-09-13T04:38:25.617180+0000","last_undegraded":"2023-09-13T04:38:25.617180+0000","last_fullsized":"2023-09-13T04:38:25.617180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:01:32.839729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.600030+0000","last_change":"2023-09-13T04:38:24.598663+0000","last_active":"2023-09-13T04:38:24.600030+0000","last_peered":"2023-09-13T04:38:24.600030+0000","last_clean":"2023-09-13T04:38:24.600030+0000","last_became_active":"2023-09-13T04:38:24.598258+0000","last_became_peered":"2023-09-13T04:38:24.598258+0000","last_unstale":"2023-09-13T04:38:24.600030+0000","last_undegraded":"2023-09-13T04:38:24.600030+0000","last_fullsized":"2023-09-13T04:38:24.600030+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:50:28.157135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592633+0000","last_change":"2023-09-13T04:38:03.619930+0000","last_active":"2023-09-13T04:38:23.592633+0000","last_peered":"2023-09-13T04:38:23.592633+0000","last_clean":"2023-09-13T04:38:23.592633+0000","last_became_active":"2023-09-13T04:38:03.619560+0000","last_became_peered":"2023-09-13T04:38:03.619560+0000","last_unstale":"2023-09-13T04:38:23.592633+0000","last_undegraded":"2023-09-13T04:38:23.592633+0000","last_fullsized":"2023-09-13T04:38:23.592633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:48:06.867602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.619713+0000","last_change":"2023-09-13T04:37:58.590396+0000","last_active":"2023-09-13T04:38:25.619713+0000","last_peered":"2023-09-13T04:38:25.619713+0000","last_clean":"2023-09-13T04:38:25.619713+0000","last_became_active":"2023-09-13T04:37:58.589929+0000","last_became_peered":"2023-09-13T04:37:58.589929+0000","last_unstale":"2023-09-13T04:38:25.619713+0000","last_undegraded":"2023-09-13T04:38:25.619713+0000","last_fullsized":"2023-09-13T04:38:25.619713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:28:36.322096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597620+0000","last_change":"2023-09-13T04:37:59.246033+0000","last_active":"2023-09-13T04:38:23.597620+0000","last_peered":"2023-09-13T04:38:23.597620+0000","last_clean":"2023-09-13T04:38:23.597620+0000","last_became_active":"2023-09-13T04:37:59.245664+0000","last_became_peered":"2023-09-13T04:37:59.245664+0000","last_unstale":"2023-09-13T04:38:23.597620+0000","last_undegraded":"2023-09-13T04:38:23.597620+0000","last_fullsized":"2023-09-13T04:38:23.597620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:24:17.145055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592344+0000","last_change":"2023-09-13T04:38:03.608504+0000","last_active":"2023-09-13T04:38:23.592344+0000","last_peered":"2023-09-13T04:38:23.592344+0000","last_clean":"2023-09-13T04:38:23.592344+0000","last_became_active":"2023-09-13T04:38:03.608058+0000","last_became_peered":"2023-09-13T04:38:03.608058+0000","last_unstale":"2023-09-13T04:38:23.592344+0000","last_undegraded":"2023-09-13T04:38:23.592344+0000","last_fullsized":"2023-09-13T04:38:23.592344+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:42:43.485453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244691+0000","last_change":"2023-09-13T04:37:58.588208+0000","last_active":"2023-09-13T04:38:24.244691+0000","last_peered":"2023-09-13T04:38:24.244691+0000","last_clean":"2023-09-13T04:38:24.244691+0000","last_became_active":"2023-09-13T04:37:58.587840+0000","last_became_peered":"2023-09-13T04:37:58.587840+0000","last_unstale":"2023-09-13T04:38:24.244691+0000","last_undegraded":"2023-09-13T04:38:24.244691+0000","last_fullsized":"2023-09-13T04:38:24.244691+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:44:58.012233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587683+0000","last_change":"2023-09-13T04:37:59.245102+0000","last_active":"2023-09-13T04:38:23.587683+0000","last_peered":"2023-09-13T04:38:23.587683+0000","last_clean":"2023-09-13T04:38:23.587683+0000","last_became_active":"2023-09-13T04:37:59.244799+0000","last_became_peered":"2023-09-13T04:37:59.244799+0000","last_unstale":"2023-09-13T04:38:23.587683+0000","last_undegraded":"2023-09-13T04:38:23.587683+0000","last_fullsized":"2023-09-13T04:38:23.587683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:57:16.295623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.596645+0000","last_change":"2023-09-13T04:38:03.607021+0000","last_active":"2023-09-13T04:38:23.596645+0000","last_peered":"2023-09-13T04:38:23.596645+0000","last_clean":"2023-09-13T04:38:23.596645+0000","last_became_active":"2023-09-13T04:38:03.606611+0000","last_became_peered":"2023-09-13T04:38:03.606611+0000","last_unstale":"2023-09-13T04:38:23.596645+0000","last_undegraded":"2023-09-13T04:38:23.596645+0000","last_fullsized":"2023-09-13T04:38:23.596645+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:33:20.551479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:28.621134+0000","last_change":"2023-09-13T04:37:58.588731+0000","last_active":"2023-09-13T04:38:28.621134+0000","last_peered":"2023-09-13T04:38:28.621134+0000","last_clean":"2023-09-13T04:38:28.621134+0000","last_became_active":"2023-09-13T04:37:58.588398+0000","last_became_peered":"2023-09-13T04:37:58.588398+0000","last_unstale":"2023-09-13T04:38:28.621134+0000","last_undegraded":"2023-09-13T04:38:28.621134+0000","last_fullsized":"2023-09-13T04:38:28.621134+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:13:10.724085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594790+0000","last_change":"2023-09-13T04:37:59.244757+0000","last_active":"2023-09-13T04:38:23.594790+0000","last_peered":"2023-09-13T04:38:23.594790+0000","last_clean":"2023-09-13T04:38:23.594790+0000","last_became_active":"2023-09-13T04:37:59.244423+0000","last_became_peered":"2023-09-13T04:37:59.244423+0000","last_unstale":"2023-09-13T04:38:23.594790+0000","last_undegraded":"2023-09-13T04:38:23.594790+0000","last_fullsized":"2023-09-13T04:38:23.594790+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:14:37.973058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587303+0000","last_change":"2023-09-13T04:38:03.618358+0000","last_active":"2023-09-13T04:38:23.587303+0000","last_peered":"2023-09-13T04:38:23.587303+0000","last_clean":"2023-09-13T04:38:23.587303+0000","last_became_active":"2023-09-13T04:38:03.617291+0000","last_became_peered":"2023-09-13T04:38:03.617291+0000","last_unstale":"2023-09-13T04:38:23.587303+0000","last_undegraded":"2023-09-13T04:38:23.587303+0000","last_fullsized":"2023-09-13T04:38:23.587303+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:33:13.955353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244519+0000","last_change":"2023-09-13T04:37:58.590978+0000","last_active":"2023-09-13T04:38:24.244519+0000","last_peered":"2023-09-13T04:38:24.244519+0000","last_clean":"2023-09-13T04:38:24.244519+0000","last_became_active":"2023-09-13T04:37:58.590673+0000","last_became_peered":"2023-09-13T04:37:58.590673+0000","last_unstale":"2023-09-13T04:38:24.244519+0000","last_undegraded":"2023-09-13T04:38:24.244519+0000","last_fullsized":"2023-09-13T04:38:24.244519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:10:32.570029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592737+0000","last_change":"2023-09-13T04:37:59.251283+0000","last_active":"2023-09-13T04:38:23.592737+0000","last_peered":"2023-09-13T04:38:23.592737+0000","last_clean":"2023-09-13T04:38:23.592737+0000","last_became_active":"2023-09-13T04:37:59.250119+0000","last_became_peered":"2023-09-13T04:37:59.250119+0000","last_unstale":"2023-09-13T04:38:23.592737+0000","last_undegraded":"2023-09-13T04:38:23.592737+0000","last_fullsized":"2023-09-13T04:38:23.592737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:48:54.389970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.596107+0000","last_change":"2023-09-13T04:38:24.596107+0000","last_active":"2023-09-13T04:38:24.596107+0000","last_peered":"2023-09-13T04:38:24.596107+0000","last_clean":"2023-09-13T04:38:24.596107+0000","last_became_active":"2023-09-13T04:38:24.595653+0000","last_became_peered":"2023-09-13T04:38:24.595653+0000","last_unstale":"2023-09-13T04:38:24.596107+0000","last_undegraded":"2023-09-13T04:38:24.596107+0000","last_fullsized":"2023-09-13T04:38:24.596107+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:02:53.708658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.589390+0000","last_change":"2023-09-13T04:37:58.584909+0000","last_active":"2023-09-13T04:38:24.589390+0000","last_peered":"2023-09-13T04:38:24.589390+0000","last_clean":"2023-09-13T04:38:24.589390+0000","last_became_active":"2023-09-13T04:37:58.584630+0000","last_became_peered":"2023-09-13T04:37:58.584630+0000","last_unstale":"2023-09-13T04:38:24.589390+0000","last_undegraded":"2023-09-13T04:38:24.589390+0000","last_fullsized":"2023-09-13T04:38:24.589390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:57:56.567499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594349+0000","last_change":"2023-09-13T04:37:59.254448+0000","last_active":"2023-09-13T04:38:23.594349+0000","last_peered":"2023-09-13T04:38:23.594349+0000","last_clean":"2023-09-13T04:38:23.594349+0000","last_became_active":"2023-09-13T04:37:59.254138+0000","last_became_peered":"2023-09-13T04:37:59.254138+0000","last_unstale":"2023-09-13T04:38:23.594349+0000","last_undegraded":"2023-09-13T04:38:23.594349+0000","last_fullsized":"2023-09-13T04:38:23.594349+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:36:14.939881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588357+0000","last_change":"2023-09-13T04:38:03.613746+0000","last_active":"2023-09-13T04:38:23.588357+0000","last_peered":"2023-09-13T04:38:23.588357+0000","last_clean":"2023-09-13T04:38:23.588357+0000","last_became_active":"2023-09-13T04:38:03.613354+0000","last_became_peered":"2023-09-13T04:38:03.613354+0000","last_unstale":"2023-09-13T04:38:23.588357+0000","last_undegraded":"2023-09-13T04:38:23.588357+0000","last_fullsized":"2023-09-13T04:38:23.588357+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:41:01.294912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:29.625469+0000","last_change":"2023-09-13T04:37:58.591196+0000","last_active":"2023-09-13T04:38:29.625469+0000","last_peered":"2023-09-13T04:38:29.625469+0000","last_clean":"2023-09-13T04:38:29.625469+0000","last_became_active":"2023-09-13T04:37:58.590883+0000","last_became_peered":"2023-09-13T04:37:58.590883+0000","last_unstale":"2023-09-13T04:38:29.625469+0000","last_undegraded":"2023-09-13T04:38:29.625469+0000","last_fullsized":"2023-09-13T04:38:29.625469+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:46:58.530270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595499+0000","last_change":"2023-09-13T04:37:59.243483+0000","last_active":"2023-09-13T04:38:23.595499+0000","last_peered":"2023-09-13T04:38:23.595499+0000","last_clean":"2023-09-13T04:38:23.595499+0000","last_became_active":"2023-09-13T04:37:59.243088+0000","last_became_peered":"2023-09-13T04:37:59.243088+0000","last_unstale":"2023-09-13T04:38:23.595499+0000","last_undegraded":"2023-09-13T04:38:23.595499+0000","last_fullsized":"2023-09-13T04:38:23.595499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:56:14.093819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591697+0000","last_change":"2023-09-13T04:38:03.608820+0000","last_active":"2023-09-13T04:38:23.591697+0000","last_peered":"2023-09-13T04:38:23.591697+0000","last_clean":"2023-09-13T04:38:23.591697+0000","last_became_active":"2023-09-13T04:38:03.608200+0000","last_became_peered":"2023-09-13T04:38:03.608200+0000","last_unstale":"2023-09-13T04:38:23.591697+0000","last_undegraded":"2023-09-13T04:38:23.591697+0000","last_fullsized":"2023-09-13T04:38:23.591697+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:54:52.416501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.620980+0000","last_change":"2023-09-13T04:38:24.606820+0000","last_active":"2023-09-13T04:38:25.620980+0000","last_peered":"2023-09-13T04:38:25.620980+0000","last_clean":"2023-09-13T04:38:25.620980+0000","last_became_active":"2023-09-13T04:38:24.599059+0000","last_became_peered":"2023-09-13T04:38:24.599059+0000","last_unstale":"2023-09-13T04:38:25.620980+0000","last_undegraded":"2023-09-13T04:38:25.620980+0000","last_fullsized":"2023-09-13T04:38:25.620980+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:08:12.634357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597568+0000","last_change":"2023-09-13T04:37:59.250666+0000","last_active":"2023-09-13T04:38:23.597568+0000","last_peered":"2023-09-13T04:38:23.597568+0000","last_clean":"2023-09-13T04:38:23.597568+0000","last_became_active":"2023-09-13T04:37:59.250240+0000","last_became_peered":"2023-09-13T04:37:59.250240+0000","last_unstale":"2023-09-13T04:38:23.597568+0000","last_undegraded":"2023-09-13T04:38:23.597568+0000","last_fullsized":"2023-09-13T04:38:23.597568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:44:31.324222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592271+0000","last_change":"2023-09-13T04:38:03.616296+0000","last_active":"2023-09-13T04:38:23.592271+0000","last_peered":"2023-09-13T04:38:23.592271+0000","last_clean":"2023-09-13T04:38:23.592271+0000","last_became_active":"2023-09-13T04:38:03.615934+0000","last_became_peered":"2023-09-13T04:38:03.615934+0000","last_unstale":"2023-09-13T04:38:23.592271+0000","last_undegraded":"2023-09-13T04:38:23.592271+0000","last_fullsized":"2023-09-13T04:38:23.592271+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:51:18.994472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244805+0000","last_change":"2023-09-13T04:37:58.594782+0000","last_active":"2023-09-13T04:38:24.244805+0000","last_peered":"2023-09-13T04:38:24.244805+0000","last_clean":"2023-09-13T04:38:24.244805+0000","last_became_active":"2023-09-13T04:37:58.594480+0000","last_became_peered":"2023-09-13T04:37:58.594480+0000","last_unstale":"2023-09-13T04:38:24.244805+0000","last_undegraded":"2023-09-13T04:38:24.244805+0000","last_fullsized":"2023-09-13T04:38:24.244805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:40:19.635672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588001+0000","last_change":"2023-09-13T04:37:59.252575+0000","last_active":"2023-09-13T04:38:23.588001+0000","last_peered":"2023-09-13T04:38:23.588001+0000","last_clean":"2023-09-13T04:38:23.588001+0000","last_became_active":"2023-09-13T04:37:59.252268+0000","last_became_peered":"2023-09-13T04:37:59.252268+0000","last_unstale":"2023-09-13T04:38:23.588001+0000","last_undegraded":"2023-09-13T04:38:23.588001+0000","last_fullsized":"2023-09-13T04:38:23.588001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:27:10.814930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587908+0000","last_change":"2023-09-13T04:38:03.618835+0000","last_active":"2023-09-13T04:38:23.587908+0000","last_peered":"2023-09-13T04:38:23.587908+0000","last_clean":"2023-09-13T04:38:23.587908+0000","last_became_active":"2023-09-13T04:38:03.618226+0000","last_became_peered":"2023-09-13T04:38:03.618226+0000","last_unstale":"2023-09-13T04:38:23.587908+0000","last_undegraded":"2023-09-13T04:38:23.587908+0000","last_fullsized":"2023-09-13T04:38:23.587908+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:01:28.157490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:28.623926+0000","last_change":"2023-09-13T04:37:58.588545+0000","last_active":"2023-09-13T04:38:28.623926+0000","last_peered":"2023-09-13T04:38:28.623926+0000","last_clean":"2023-09-13T04:38:28.623926+0000","last_became_active":"2023-09-13T04:37:58.587995+0000","last_became_peered":"2023-09-13T04:37:58.587995+0000","last_unstale":"2023-09-13T04:38:28.623926+0000","last_undegraded":"2023-09-13T04:38:28.623926+0000","last_fullsized":"2023-09-13T04:38:28.623926+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:40:08.797724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587793+0000","last_change":"2023-09-13T04:37:59.249100+0000","last_active":"2023-09-13T04:38:23.587793+0000","last_peered":"2023-09-13T04:38:23.587793+0000","last_clean":"2023-09-13T04:38:23.587793+0000","last_became_active":"2023-09-13T04:37:59.248771+0000","last_became_peered":"2023-09-13T04:37:59.248771+0000","last_unstale":"2023-09-13T04:38:23.587793+0000","last_undegraded":"2023-09-13T04:38:23.587793+0000","last_fullsized":"2023-09-13T04:38:23.587793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:27:37.964595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597364+0000","last_change":"2023-09-13T04:38:03.616988+0000","last_active":"2023-09-13T04:38:23.597364+0000","last_peered":"2023-09-13T04:38:23.597364+0000","last_clean":"2023-09-13T04:38:23.597364+0000","last_became_active":"2023-09-13T04:38:03.616524+0000","last_became_peered":"2023-09-13T04:38:03.616524+0000","last_unstale":"2023-09-13T04:38:23.597364+0000","last_undegraded":"2023-09-13T04:38:23.597364+0000","last_fullsized":"2023-09-13T04:38:23.597364+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:42:41.431115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.622279+0000","last_change":"2023-09-13T04:38:24.613384+0000","last_active":"2023-09-13T04:38:25.622279+0000","last_peered":"2023-09-13T04:38:25.622279+0000","last_clean":"2023-09-13T04:38:25.622279+0000","last_became_active":"2023-09-13T04:38:24.599349+0000","last_became_peered":"2023-09-13T04:38:24.599349+0000","last_unstale":"2023-09-13T04:38:25.622279+0000","last_undegraded":"2023-09-13T04:38:25.622279+0000","last_fullsized":"2023-09-13T04:38:25.622279+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:23:23.479715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593329+0000","last_change":"2023-09-13T04:37:59.252237+0000","last_active":"2023-09-13T04:38:23.593329+0000","last_peered":"2023-09-13T04:38:23.593329+0000","last_clean":"2023-09-13T04:38:23.593329+0000","last_became_active":"2023-09-13T04:37:59.251718+0000","last_became_peered":"2023-09-13T04:37:59.251718+0000","last_unstale":"2023-09-13T04:38:23.593329+0000","last_undegraded":"2023-09-13T04:38:23.593329+0000","last_fullsized":"2023-09-13T04:38:23.593329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:45:48.405226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593276+0000","last_change":"2023-09-13T04:38:03.607096+0000","last_active":"2023-09-13T04:38:23.593276+0000","last_peered":"2023-09-13T04:38:23.593276+0000","last_clean":"2023-09-13T04:38:23.593276+0000","last_became_active":"2023-09-13T04:38:03.606656+0000","last_became_peered":"2023-09-13T04:38:03.606656+0000","last_unstale":"2023-09-13T04:38:23.593276+0000","last_undegraded":"2023-09-13T04:38:23.593276+0000","last_fullsized":"2023-09-13T04:38:23.593276+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:59:26.140360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:26.617580+0000","last_change":"2023-09-13T04:37:58.591047+0000","last_active":"2023-09-13T04:38:26.617580+0000","last_peered":"2023-09-13T04:38:26.617580+0000","last_clean":"2023-09-13T04:38:26.617580+0000","last_became_active":"2023-09-13T04:37:58.590740+0000","last_became_peered":"2023-09-13T04:37:58.590740+0000","last_unstale":"2023-09-13T04:38:26.617580+0000","last_undegraded":"2023-09-13T04:38:26.617580+0000","last_fullsized":"2023-09-13T04:38:26.617580+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:14:42.280416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592406+0000","last_change":"2023-09-13T04:37:59.249362+0000","last_active":"2023-09-13T04:38:23.592406+0000","last_peered":"2023-09-13T04:38:23.592406+0000","last_clean":"2023-09-13T04:38:23.592406+0000","last_became_active":"2023-09-13T04:37:59.249041+0000","last_became_peered":"2023-09-13T04:37:59.249041+0000","last_unstale":"2023-09-13T04:38:23.592406+0000","last_undegraded":"2023-09-13T04:38:23.592406+0000","last_fullsized":"2023-09-13T04:38:23.592406+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:06:33.822157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.598370+0000","last_change":"2023-09-13T04:38:24.598370+0000","last_active":"2023-09-13T04:38:24.598370+0000","last_peered":"2023-09-13T04:38:24.598370+0000","last_clean":"2023-09-13T04:38:24.598370+0000","last_became_active":"2023-09-13T04:38:24.597931+0000","last_became_peered":"2023-09-13T04:38:24.597931+0000","last_unstale":"2023-09-13T04:38:24.598370+0000","last_undegraded":"2023-09-13T04:38:24.598370+0000","last_fullsized":"2023-09-13T04:38:24.598370+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:14:28.911329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588100+0000","last_change":"2023-09-13T04:37:58.592496+0000","last_active":"2023-09-13T04:38:23.588100+0000","last_peered":"2023-09-13T04:38:23.588100+0000","last_clean":"2023-09-13T04:38:23.588100+0000","last_became_active":"2023-09-13T04:37:58.592106+0000","last_became_peered":"2023-09-13T04:37:58.592106+0000","last_unstale":"2023-09-13T04:38:23.588100+0000","last_undegraded":"2023-09-13T04:38:23.588100+0000","last_fullsized":"2023-09-13T04:38:23.588100+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:28:55.618563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594212+0000","last_change":"2023-09-13T04:37:59.250019+0000","last_active":"2023-09-13T04:38:23.594212+0000","last_peered":"2023-09-13T04:38:23.594212+0000","last_clean":"2023-09-13T04:38:23.594212+0000","last_became_active":"2023-09-13T04:37:59.249278+0000","last_became_peered":"2023-09-13T04:37:59.249278+0000","last_unstale":"2023-09-13T04:38:23.594212+0000","last_undegraded":"2023-09-13T04:38:23.594212+0000","last_fullsized":"2023-09-13T04:38:23.594212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:15:31.217236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597640+0000","last_change":"2023-09-13T04:38:03.614542+0000","last_active":"2023-09-13T04:38:23.597640+0000","last_peered":"2023-09-13T04:38:23.597640+0000","last_clean":"2023-09-13T04:38:23.597640+0000","last_became_active":"2023-09-13T04:38:03.614103+0000","last_became_peered":"2023-09-13T04:38:03.614103+0000","last_unstale":"2023-09-13T04:38:23.597640+0000","last_undegraded":"2023-09-13T04:38:23.597640+0000","last_fullsized":"2023-09-13T04:38:23.597640+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:06:48.426843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.592720+0000","last_change":"2023-09-13T04:37:58.591864+0000","last_active":"2023-09-13T04:38:24.592720+0000","last_peered":"2023-09-13T04:38:24.592720+0000","last_clean":"2023-09-13T04:38:24.592720+0000","last_became_active":"2023-09-13T04:37:58.591562+0000","last_became_peered":"2023-09-13T04:37:58.591562+0000","last_unstale":"2023-09-13T04:38:24.592720+0000","last_undegraded":"2023-09-13T04:38:24.592720+0000","last_fullsized":"2023-09-13T04:38:24.592720+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:31:27.280489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594567+0000","last_change":"2023-09-13T04:37:59.252600+0000","last_active":"2023-09-13T04:38:23.594567+0000","last_peered":"2023-09-13T04:38:23.594567+0000","last_clean":"2023-09-13T04:38:23.594567+0000","last_became_active":"2023-09-13T04:37:59.252306+0000","last_became_peered":"2023-09-13T04:37:59.252306+0000","last_unstale":"2023-09-13T04:38:23.594567+0000","last_undegraded":"2023-09-13T04:38:23.594567+0000","last_fullsized":"2023-09-13T04:38:23.594567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:18:55.686847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.596920+0000","last_change":"2023-09-13T04:38:03.742502+0000","last_active":"2023-09-13T04:38:23.596920+0000","last_peered":"2023-09-13T04:38:23.596920+0000","last_clean":"2023-09-13T04:38:23.596920+0000","last_became_active":"2023-09-13T04:38:03.742054+0000","last_became_peered":"2023-09-13T04:38:03.742054+0000","last_unstale":"2023-09-13T04:38:23.596920+0000","last_undegraded":"2023-09-13T04:38:23.596920+0000","last_fullsized":"2023-09-13T04:38:23.596920+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:55:54.601145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593527+0000","last_change":"2023-09-13T04:37:59.245644+0000","last_active":"2023-09-13T04:38:23.593527+0000","last_peered":"2023-09-13T04:38:23.593527+0000","last_clean":"2023-09-13T04:38:23.593527+0000","last_became_active":"2023-09-13T04:37:59.245284+0000","last_became_peered":"2023-09-13T04:37:59.245284+0000","last_unstale":"2023-09-13T04:38:23.593527+0000","last_undegraded":"2023-09-13T04:38:23.593527+0000","last_fullsized":"2023-09-13T04:38:23.593527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:19:04.002414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:29.624843+0000","last_change":"2023-09-13T04:37:58.590249+0000","last_active":"2023-09-13T04:38:29.624843+0000","last_peered":"2023-09-13T04:38:29.624843+0000","last_clean":"2023-09-13T04:38:29.624843+0000","last_became_active":"2023-09-13T04:37:58.589521+0000","last_became_peered":"2023-09-13T04:37:58.589521+0000","last_unstale":"2023-09-13T04:38:29.624843+0000","last_undegraded":"2023-09-13T04:38:29.624843+0000","last_fullsized":"2023-09-13T04:38:29.624843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:11:57.542149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.598613+0000","last_change":"2023-09-13T04:38:24.598613+0000","last_active":"2023-09-13T04:38:24.598613+0000","last_peered":"2023-09-13T04:38:24.598613+0000","last_clean":"2023-09-13T04:38:24.598613+0000","last_became_active":"2023-09-13T04:38:24.598150+0000","last_became_peered":"2023-09-13T04:38:24.598150+0000","last_unstale":"2023-09-13T04:38:24.598613+0000","last_undegraded":"2023-09-13T04:38:24.598613+0000","last_fullsized":"2023-09-13T04:38:24.598613+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:02:29.894446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595221+0000","last_change":"2023-09-13T04:37:59.243794+0000","last_active":"2023-09-13T04:38:23.595221+0000","last_peered":"2023-09-13T04:38:23.595221+0000","last_clean":"2023-09-13T04:38:23.595221+0000","last_became_active":"2023-09-13T04:37:59.243368+0000","last_became_peered":"2023-09-13T04:37:59.243368+0000","last_unstale":"2023-09-13T04:38:23.595221+0000","last_undegraded":"2023-09-13T04:38:23.595221+0000","last_fullsized":"2023-09-13T04:38:23.595221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:06:45.318737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:26.615677+0000","last_change":"2023-09-13T04:37:58.589195+0000","last_active":"2023-09-13T04:38:26.615677+0000","last_peered":"2023-09-13T04:38:26.615677+0000","last_clean":"2023-09-13T04:38:26.615677+0000","last_became_active":"2023-09-13T04:37:58.588905+0000","last_became_peered":"2023-09-13T04:37:58.588905+0000","last_unstale":"2023-09-13T04:38:26.615677+0000","last_undegraded":"2023-09-13T04:38:26.615677+0000","last_fullsized":"2023-09-13T04:38:26.615677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:13:15.978891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":326,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587638+0000","last_change":"2023-09-13T04:36:52.227264+0000","last_active":"2023-09-13T04:38:23.587638+0000","last_peered":"2023-09-13T04:38:23.587638+0000","last_clean":"2023-09-13T04:38:23.587638+0000","last_became_active":"2023-09-13T04:36:52.226773+0000","last_became_peered":"2023-09-13T04:36:52.226773+0000","last_unstale":"2023-09-13T04:38:23.587638+0000","last_undegraded":"2023-09-13T04:38:23.587638+0000","last_fullsized":"2023-09-13T04:38:23.587638+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:33:42.026499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:33:42.026499+0000","last_clean_scrub_stamp":"2023-09-13T04:33:42.026499+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:28:55.806587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592890+0000","last_change":"2023-09-13T04:38:03.742790+0000","last_active":"2023-09-13T04:38:23.592890+0000","last_peered":"2023-09-13T04:38:23.592890+0000","last_clean":"2023-09-13T04:38:23.592890+0000","last_became_active":"2023-09-13T04:38:03.742338+0000","last_became_peered":"2023-09-13T04:38:03.742338+0000","last_unstale":"2023-09-13T04:38:23.592890+0000","last_undegraded":"2023-09-13T04:38:23.592890+0000","last_fullsized":"2023-09-13T04:38:23.592890+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:01:38.910623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597412+0000","last_change":"2023-09-13T04:37:59.252669+0000","last_active":"2023-09-13T04:38:23.597412+0000","last_peered":"2023-09-13T04:38:23.597412+0000","last_clean":"2023-09-13T04:38:23.597412+0000","last_became_active":"2023-09-13T04:37:59.252223+0000","last_became_peered":"2023-09-13T04:37:59.252223+0000","last_unstale":"2023-09-13T04:38:23.597412+0000","last_undegraded":"2023-09-13T04:38:23.597412+0000","last_fullsized":"2023-09-13T04:38:23.597412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:57:21.090083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.615697+0000","last_change":"2023-09-13T04:37:58.590540+0000","last_active":"2023-09-13T04:38:25.615697+0000","last_peered":"2023-09-13T04:38:25.615697+0000","last_clean":"2023-09-13T04:38:25.615697+0000","last_became_active":"2023-09-13T04:37:58.590061+0000","last_became_peered":"2023-09-13T04:37:58.590061+0000","last_unstale":"2023-09-13T04:38:25.615697+0000","last_undegraded":"2023-09-13T04:38:25.615697+0000","last_fullsized":"2023-09-13T04:38:25.615697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:57:17.133461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592719+0000","last_change":"2023-09-13T04:38:03.614615+0000","last_active":"2023-09-13T04:38:23.592719+0000","last_peered":"2023-09-13T04:38:23.592719+0000","last_clean":"2023-09-13T04:38:23.592719+0000","last_became_active":"2023-09-13T04:38:03.614086+0000","last_became_peered":"2023-09-13T04:38:03.614086+0000","last_unstale":"2023-09-13T04:38:23.592719+0000","last_undegraded":"2023-09-13T04:38:23.592719+0000","last_fullsized":"2023-09-13T04:38:23.592719+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:25:35.525513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.595581+0000","last_change":"2023-09-13T04:37:58.592096+0000","last_active":"2023-09-13T04:38:24.595581+0000","last_peered":"2023-09-13T04:38:24.595581+0000","last_clean":"2023-09-13T04:38:24.595581+0000","last_became_active":"2023-09-13T04:37:58.591872+0000","last_became_peered":"2023-09-13T04:37:58.591872+0000","last_unstale":"2023-09-13T04:38:24.595581+0000","last_undegraded":"2023-09-13T04:38:24.595581+0000","last_fullsized":"2023-09-13T04:38:24.595581+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:05:23.936105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597816+0000","last_change":"2023-09-13T04:37:59.240759+0000","last_active":"2023-09-13T04:38:23.597816+0000","last_peered":"2023-09-13T04:38:23.597816+0000","last_clean":"2023-09-13T04:38:23.597816+0000","last_became_active":"2023-09-13T04:37:59.240414+0000","last_became_peered":"2023-09-13T04:37:59.240414+0000","last_unstale":"2023-09-13T04:38:23.597816+0000","last_undegraded":"2023-09-13T04:38:23.597816+0000","last_fullsized":"2023-09-13T04:38:23.597816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:37:20.301938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597533+0000","last_change":"2023-09-13T04:38:03.742519+0000","last_active":"2023-09-13T04:38:23.597533+0000","last_peered":"2023-09-13T04:38:23.597533+0000","last_clean":"2023-09-13T04:38:23.597533+0000","last_became_active":"2023-09-13T04:38:03.742075+0000","last_became_peered":"2023-09-13T04:38:03.742075+0000","last_unstale":"2023-09-13T04:38:23.597533+0000","last_undegraded":"2023-09-13T04:38:23.597533+0000","last_fullsized":"2023-09-13T04:38:23.597533+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:14:33.155837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.619899+0000","last_change":"2023-09-13T04:37:58.593891+0000","last_active":"2023-09-13T04:38:25.619899+0000","last_peered":"2023-09-13T04:38:25.619899+0000","last_clean":"2023-09-13T04:38:25.619899+0000","last_became_active":"2023-09-13T04:37:58.593497+0000","last_became_peered":"2023-09-13T04:37:58.593497+0000","last_unstale":"2023-09-13T04:38:25.619899+0000","last_undegraded":"2023-09-13T04:38:25.619899+0000","last_fullsized":"2023-09-13T04:38:25.619899+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:25:53.652601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595975+0000","last_change":"2023-09-13T04:37:59.247151+0000","last_active":"2023-09-13T04:38:23.595975+0000","last_peered":"2023-09-13T04:38:23.595975+0000","last_clean":"2023-09-13T04:38:23.595975+0000","last_became_active":"2023-09-13T04:37:59.246853+0000","last_became_peered":"2023-09-13T04:37:59.246853+0000","last_unstale":"2023-09-13T04:38:23.595975+0000","last_undegraded":"2023-09-13T04:38:23.595975+0000","last_fullsized":"2023-09-13T04:38:23.595975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:15:22.659365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587493+0000","last_change":"2023-09-13T04:38:03.606594+0000","last_active":"2023-09-13T04:38:23.587493+0000","last_peered":"2023-09-13T04:38:23.587493+0000","last_clean":"2023-09-13T04:38:23.587493+0000","last_became_active":"2023-09-13T04:38:03.606058+0000","last_became_peered":"2023-09-13T04:38:03.606058+0000","last_unstale":"2023-09-13T04:38:23.587493+0000","last_undegraded":"2023-09-13T04:38:23.587493+0000","last_fullsized":"2023-09-13T04:38:23.587493+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:57:39.237520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.246057+0000","last_change":"2023-09-13T04:37:58.588362+0000","last_active":"2023-09-13T04:38:24.246057+0000","last_peered":"2023-09-13T04:38:24.246057+0000","last_clean":"2023-09-13T04:38:24.246057+0000","last_became_active":"2023-09-13T04:37:58.588065+0000","last_became_peered":"2023-09-13T04:37:58.588065+0000","last_unstale":"2023-09-13T04:38:24.246057+0000","last_undegraded":"2023-09-13T04:38:24.246057+0000","last_fullsized":"2023-09-13T04:38:24.246057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:32:04.255654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592468+0000","last_change":"2023-09-13T04:37:59.251883+0000","last_active":"2023-09-13T04:38:23.592468+0000","last_peered":"2023-09-13T04:38:23.592468+0000","last_clean":"2023-09-13T04:38:23.592468+0000","last_became_active":"2023-09-13T04:37:59.251522+0000","last_became_peered":"2023-09-13T04:37:59.251522+0000","last_unstale":"2023-09-13T04:38:23.592468+0000","last_undegraded":"2023-09-13T04:38:23.592468+0000","last_fullsized":"2023-09-13T04:38:23.592468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:05:06.420799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590729+0000","last_change":"2023-09-13T04:38:03.742726+0000","last_active":"2023-09-13T04:38:23.590729+0000","last_peered":"2023-09-13T04:38:23.590729+0000","last_clean":"2023-09-13T04:38:23.590729+0000","last_became_active":"2023-09-13T04:38:03.742293+0000","last_became_peered":"2023-09-13T04:38:03.742293+0000","last_unstale":"2023-09-13T04:38:23.590729+0000","last_undegraded":"2023-09-13T04:38:23.590729+0000","last_fullsized":"2023-09-13T04:38:23.590729+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:52:15.089764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.243314+0000","last_change":"2023-09-13T04:37:58.584923+0000","last_active":"2023-09-13T04:38:24.243314+0000","last_peered":"2023-09-13T04:38:24.243314+0000","last_clean":"2023-09-13T04:38:24.243314+0000","last_became_active":"2023-09-13T04:37:58.584320+0000","last_became_peered":"2023-09-13T04:37:58.584320+0000","last_unstale":"2023-09-13T04:38:24.243314+0000","last_undegraded":"2023-09-13T04:38:24.243314+0000","last_fullsized":"2023-09-13T04:38:24.243314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:16:53.670534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592586+0000","last_change":"2023-09-13T04:37:59.243029+0000","last_active":"2023-09-13T04:38:23.592586+0000","last_peered":"2023-09-13T04:38:23.592586+0000","last_clean":"2023-09-13T04:38:23.592586+0000","last_became_active":"2023-09-13T04:37:59.242733+0000","last_became_peered":"2023-09-13T04:37:59.242733+0000","last_unstale":"2023-09-13T04:38:23.592586+0000","last_undegraded":"2023-09-13T04:38:23.592586+0000","last_fullsized":"2023-09-13T04:38:23.592586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:48:21.430026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588037+0000","last_change":"2023-09-13T04:38:03.616735+0000","last_active":"2023-09-13T04:38:23.588037+0000","last_peered":"2023-09-13T04:38:23.588037+0000","last_clean":"2023-09-13T04:38:23.588037+0000","last_became_active":"2023-09-13T04:38:03.616275+0000","last_became_peered":"2023-09-13T04:38:03.616275+0000","last_unstale":"2023-09-13T04:38:23.588037+0000","last_undegraded":"2023-09-13T04:38:23.588037+0000","last_fullsized":"2023-09-13T04:38:23.588037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:12:27.267918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244170+0000","last_change":"2023-09-13T04:37:58.595040+0000","last_active":"2023-09-13T04:38:24.244170+0000","last_peered":"2023-09-13T04:38:24.244170+0000","last_clean":"2023-09-13T04:38:24.244170+0000","last_became_active":"2023-09-13T04:37:58.591403+0000","last_became_peered":"2023-09-13T04:37:58.591403+0000","last_unstale":"2023-09-13T04:38:24.244170+0000","last_undegraded":"2023-09-13T04:38:24.244170+0000","last_fullsized":"2023-09-13T04:38:24.244170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:42:06.561066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592175+0000","last_change":"2023-09-13T04:37:59.248837+0000","last_active":"2023-09-13T04:38:23.592175+0000","last_peered":"2023-09-13T04:38:23.592175+0000","last_clean":"2023-09-13T04:38:23.592175+0000","last_became_active":"2023-09-13T04:37:59.248573+0000","last_became_peered":"2023-09-13T04:37:59.248573+0000","last_unstale":"2023-09-13T04:38:23.592175+0000","last_undegraded":"2023-09-13T04:38:23.592175+0000","last_fullsized":"2023-09-13T04:38:23.592175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:56:22.980484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590405+0000","last_change":"2023-09-13T04:38:03.616370+0000","last_active":"2023-09-13T04:38:23.590405+0000","last_peered":"2023-09-13T04:38:23.590405+0000","last_clean":"2023-09-13T04:38:23.590405+0000","last_became_active":"2023-09-13T04:38:03.615996+0000","last_became_peered":"2023-09-13T04:38:03.615996+0000","last_unstale":"2023-09-13T04:38:23.590405+0000","last_undegraded":"2023-09-13T04:38:23.590405+0000","last_fullsized":"2023-09-13T04:38:23.590405+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:56:59.924205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.590955+0000","last_change":"2023-09-13T04:37:58.597902+0000","last_active":"2023-09-13T04:38:24.590955+0000","last_peered":"2023-09-13T04:38:24.590955+0000","last_clean":"2023-09-13T04:38:24.590955+0000","last_became_active":"2023-09-13T04:37:58.597578+0000","last_became_peered":"2023-09-13T04:37:58.597578+0000","last_unstale":"2023-09-13T04:38:24.590955+0000","last_undegraded":"2023-09-13T04:38:24.590955+0000","last_fullsized":"2023-09-13T04:38:24.590955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:10:09.931949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597024+0000","last_change":"2023-09-13T04:37:59.248924+0000","last_active":"2023-09-13T04:38:23.597024+0000","last_peered":"2023-09-13T04:38:23.597024+0000","last_clean":"2023-09-13T04:38:23.597024+0000","last_became_active":"2023-09-13T04:37:59.248483+0000","last_became_peered":"2023-09-13T04:37:59.248483+0000","last_unstale":"2023-09-13T04:38:23.597024+0000","last_undegraded":"2023-09-13T04:38:23.597024+0000","last_fullsized":"2023-09-13T04:38:23.597024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:49:56.078544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590322+0000","last_change":"2023-09-13T04:38:03.742260+0000","last_active":"2023-09-13T04:38:23.590322+0000","last_peered":"2023-09-13T04:38:23.590322+0000","last_clean":"2023-09-13T04:38:23.590322+0000","last_became_active":"2023-09-13T04:38:03.741893+0000","last_became_peered":"2023-09-13T04:38:03.741893+0000","last_unstale":"2023-09-13T04:38:23.590322+0000","last_undegraded":"2023-09-13T04:38:23.590322+0000","last_fullsized":"2023-09-13T04:38:23.590322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:59:00.653545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.594861+0000","last_change":"2023-09-13T04:37:58.586898+0000","last_active":"2023-09-13T04:38:24.594861+0000","last_peered":"2023-09-13T04:38:24.594861+0000","last_clean":"2023-09-13T04:38:24.594861+0000","last_became_active":"2023-09-13T04:37:58.586594+0000","last_became_peered":"2023-09-13T04:37:58.586594+0000","last_unstale":"2023-09-13T04:38:24.594861+0000","last_undegraded":"2023-09-13T04:38:24.594861+0000","last_fullsized":"2023-09-13T04:38:24.594861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:29:43.718522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587969+0000","last_change":"2023-09-13T04:37:59.244749+0000","last_active":"2023-09-13T04:38:23.587969+0000","last_peered":"2023-09-13T04:38:23.587969+0000","last_clean":"2023-09-13T04:38:23.587969+0000","last_became_active":"2023-09-13T04:37:59.244391+0000","last_became_peered":"2023-09-13T04:37:59.244391+0000","last_unstale":"2023-09-13T04:38:23.587969+0000","last_undegraded":"2023-09-13T04:38:23.587969+0000","last_fullsized":"2023-09-13T04:38:23.587969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:33:02.195729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592524+0000","last_change":"2023-09-13T04:38:03.609766+0000","last_active":"2023-09-13T04:38:23.592524+0000","last_peered":"2023-09-13T04:38:23.592524+0000","last_clean":"2023-09-13T04:38:23.592524+0000","last_became_active":"2023-09-13T04:38:03.609432+0000","last_became_peered":"2023-09-13T04:38:03.609432+0000","last_unstale":"2023-09-13T04:38:23.592524+0000","last_undegraded":"2023-09-13T04:38:23.592524+0000","last_fullsized":"2023-09-13T04:38:23.592524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:30:56.115483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.620943+0000","last_change":"2023-09-13T04:37:58.589119+0000","last_active":"2023-09-13T04:38:25.620943+0000","last_peered":"2023-09-13T04:38:25.620943+0000","last_clean":"2023-09-13T04:38:25.620943+0000","last_became_active":"2023-09-13T04:37:58.588803+0000","last_became_peered":"2023-09-13T04:37:58.588803+0000","last_unstale":"2023-09-13T04:38:25.620943+0000","last_undegraded":"2023-09-13T04:38:25.620943+0000","last_fullsized":"2023-09-13T04:38:25.620943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:51:20.888666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588392+0000","last_change":"2023-09-13T04:37:59.243506+0000","last_active":"2023-09-13T04:38:23.588392+0000","last_peered":"2023-09-13T04:38:23.588392+0000","last_clean":"2023-09-13T04:38:23.588392+0000","last_became_active":"2023-09-13T04:37:59.243082+0000","last_became_peered":"2023-09-13T04:37:59.243082+0000","last_unstale":"2023-09-13T04:38:23.588392+0000","last_undegraded":"2023-09-13T04:38:23.588392+0000","last_fullsized":"2023-09-13T04:38:23.588392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:01:41.179286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592019+0000","last_change":"2023-09-13T04:38:03.609042+0000","last_active":"2023-09-13T04:38:23.592019+0000","last_peered":"2023-09-13T04:38:23.592019+0000","last_clean":"2023-09-13T04:38:23.592019+0000","last_became_active":"2023-09-13T04:38:03.608553+0000","last_became_peered":"2023-09-13T04:38:03.608553+0000","last_unstale":"2023-09-13T04:38:23.592019+0000","last_undegraded":"2023-09-13T04:38:23.592019+0000","last_fullsized":"2023-09-13T04:38:23.592019+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:05:56.648475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.589585+0000","last_change":"2023-09-13T04:37:58.598056+0000","last_active":"2023-09-13T04:38:24.589585+0000","last_peered":"2023-09-13T04:38:24.589585+0000","last_clean":"2023-09-13T04:38:24.589585+0000","last_became_active":"2023-09-13T04:37:58.597724+0000","last_became_peered":"2023-09-13T04:37:58.597724+0000","last_unstale":"2023-09-13T04:38:24.589585+0000","last_undegraded":"2023-09-13T04:38:24.589585+0000","last_fullsized":"2023-09-13T04:38:24.589585+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:43:08.296866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588462+0000","last_change":"2023-09-13T04:37:59.246505+0000","last_active":"2023-09-13T04:38:23.588462+0000","last_peered":"2023-09-13T04:38:23.588462+0000","last_clean":"2023-09-13T04:38:23.588462+0000","last_became_active":"2023-09-13T04:37:59.246024+0000","last_became_peered":"2023-09-13T04:37:59.246024+0000","last_unstale":"2023-09-13T04:38:23.588462+0000","last_undegraded":"2023-09-13T04:38:23.588462+0000","last_fullsized":"2023-09-13T04:38:23.588462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:36:58.008066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590871+0000","last_change":"2023-09-13T04:38:03.605354+0000","last_active":"2023-09-13T04:38:23.590871+0000","last_peered":"2023-09-13T04:38:23.590871+0000","last_clean":"2023-09-13T04:38:23.590871+0000","last_became_active":"2023-09-13T04:38:03.604840+0000","last_became_peered":"2023-09-13T04:38:03.604840+0000","last_unstale":"2023-09-13T04:38:23.590871+0000","last_undegraded":"2023-09-13T04:38:23.590871+0000","last_fullsized":"2023-09-13T04:38:23.590871+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:46:29.867914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597380+0000","last_change":"2023-09-13T04:37:58.589052+0000","last_active":"2023-09-13T04:38:23.597380+0000","last_peered":"2023-09-13T04:38:23.597380+0000","last_clean":"2023-09-13T04:38:23.597380+0000","last_became_active":"2023-09-13T04:37:58.588707+0000","last_became_peered":"2023-09-13T04:37:58.588707+0000","last_unstale":"2023-09-13T04:38:23.597380+0000","last_undegraded":"2023-09-13T04:38:23.597380+0000","last_fullsized":"2023-09-13T04:38:23.597380+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:25:23.966046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588356+0000","last_change":"2023-09-13T04:37:59.241157+0000","last_active":"2023-09-13T04:38:23.588356+0000","last_peered":"2023-09-13T04:38:23.588356+0000","last_clean":"2023-09-13T04:38:23.588356+0000","last_became_active":"2023-09-13T04:37:59.240857+0000","last_became_peered":"2023-09-13T04:37:59.240857+0000","last_unstale":"2023-09-13T04:38:23.588356+0000","last_undegraded":"2023-09-13T04:38:23.588356+0000","last_fullsized":"2023-09-13T04:38:23.588356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:04:44.986908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592376+0000","last_change":"2023-09-13T04:37:59.242447+0000","last_active":"2023-09-13T04:38:23.592376+0000","last_peered":"2023-09-13T04:38:23.592376+0000","last_clean":"2023-09-13T04:38:23.592376+0000","last_became_active":"2023-09-13T04:37:59.242130+0000","last_became_peered":"2023-09-13T04:37:59.242130+0000","last_unstale":"2023-09-13T04:38:23.592376+0000","last_undegraded":"2023-09-13T04:38:23.592376+0000","last_fullsized":"2023-09-13T04:38:23.592376+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:12:12.844888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.595646+0000","last_change":"2023-09-13T04:38:24.595646+0000","last_active":"2023-09-13T04:38:24.595646+0000","last_peered":"2023-09-13T04:38:24.595646+0000","last_clean":"2023-09-13T04:38:24.595646+0000","last_became_active":"2023-09-13T04:38:24.595295+0000","last_became_peered":"2023-09-13T04:38:24.595295+0000","last_unstale":"2023-09-13T04:38:24.595646+0000","last_undegraded":"2023-09-13T04:38:24.595646+0000","last_fullsized":"2023-09-13T04:38:24.595646+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:24:27.174979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587574+0000","last_change":"2023-09-13T04:37:59.245358+0000","last_active":"2023-09-13T04:38:23.587574+0000","last_peered":"2023-09-13T04:38:23.587574+0000","last_clean":"2023-09-13T04:38:23.587574+0000","last_became_active":"2023-09-13T04:37:59.245052+0000","last_became_peered":"2023-09-13T04:37:59.245052+0000","last_unstale":"2023-09-13T04:38:23.587574+0000","last_undegraded":"2023-09-13T04:38:23.587574+0000","last_fullsized":"2023-09-13T04:38:23.587574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:05:42.561716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592358+0000","last_change":"2023-09-13T04:37:59.254053+0000","last_active":"2023-09-13T04:38:23.592358+0000","last_peered":"2023-09-13T04:38:23.592358+0000","last_clean":"2023-09-13T04:38:23.592358+0000","last_became_active":"2023-09-13T04:37:59.253544+0000","last_became_peered":"2023-09-13T04:37:59.253544+0000","last_unstale":"2023-09-13T04:38:23.592358+0000","last_undegraded":"2023-09-13T04:38:23.592358+0000","last_fullsized":"2023-09-13T04:38:23.592358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:18:54.689208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597920+0000","last_change":"2023-09-13T04:37:59.253403+0000","last_active":"2023-09-13T04:38:23.597920+0000","last_peered":"2023-09-13T04:38:23.597920+0000","last_clean":"2023-09-13T04:38:23.597920+0000","last_became_active":"2023-09-13T04:37:59.253062+0000","last_became_peered":"2023-09-13T04:37:59.253062+0000","last_unstale":"2023-09-13T04:38:23.597920+0000","last_undegraded":"2023-09-13T04:38:23.597920+0000","last_fullsized":"2023-09-13T04:38:23.597920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:31:09.898886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592570+0000","last_change":"2023-09-13T04:37:59.240916+0000","last_active":"2023-09-13T04:38:23.592570+0000","last_peered":"2023-09-13T04:38:23.592570+0000","last_clean":"2023-09-13T04:38:23.592570+0000","last_became_active":"2023-09-13T04:37:59.240605+0000","last_became_peered":"2023-09-13T04:37:59.240605+0000","last_unstale":"2023-09-13T04:38:23.592570+0000","last_undegraded":"2023-09-13T04:38:23.592570+0000","last_fullsized":"2023-09-13T04:38:23.592570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:15:48.241587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588778+0000","last_change":"2023-09-13T04:37:59.238569+0000","last_active":"2023-09-13T04:38:23.588778+0000","last_peered":"2023-09-13T04:38:23.588778+0000","last_clean":"2023-09-13T04:38:23.588778+0000","last_became_active":"2023-09-13T04:37:59.238255+0000","last_became_peered":"2023-09-13T04:37:59.238255+0000","last_unstale":"2023-09-13T04:38:23.588778+0000","last_undegraded":"2023-09-13T04:38:23.588778+0000","last_fullsized":"2023-09-13T04:38:23.588778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:19:03.433404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595875+0000","last_change":"2023-09-13T04:37:59.252243+0000","last_active":"2023-09-13T04:38:23.595875+0000","last_peered":"2023-09-13T04:38:23.595875+0000","last_clean":"2023-09-13T04:38:23.595875+0000","last_became_active":"2023-09-13T04:37:59.251902+0000","last_became_peered":"2023-09-13T04:37:59.251902+0000","last_unstale":"2023-09-13T04:38:23.595875+0000","last_undegraded":"2023-09-13T04:38:23.595875+0000","last_fullsized":"2023-09-13T04:38:23.595875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:35:57.395627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593414+0000","last_change":"2023-09-13T04:37:59.245847+0000","last_active":"2023-09-13T04:38:23.593414+0000","last_peered":"2023-09-13T04:38:23.593414+0000","last_clean":"2023-09-13T04:38:23.593414+0000","last_became_active":"2023-09-13T04:37:59.245542+0000","last_became_peered":"2023-09-13T04:37:59.245542+0000","last_unstale":"2023-09-13T04:38:23.593414+0000","last_undegraded":"2023-09-13T04:38:23.593414+0000","last_fullsized":"2023-09-13T04:38:23.593414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:24:16.813333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.596961+0000","last_change":"2023-09-13T04:37:59.246963+0000","last_active":"2023-09-13T04:38:23.596961+0000","last_peered":"2023-09-13T04:38:23.596961+0000","last_clean":"2023-09-13T04:38:23.596961+0000","last_became_active":"2023-09-13T04:37:59.246448+0000","last_became_peered":"2023-09-13T04:37:59.246448+0000","last_unstale":"2023-09-13T04:38:23.596961+0000","last_undegraded":"2023-09-13T04:38:23.596961+0000","last_fullsized":"2023-09-13T04:38:23.596961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:25:45.418778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594086+0000","last_change":"2023-09-13T04:37:59.245399+0000","last_active":"2023-09-13T04:38:23.594086+0000","last_peered":"2023-09-13T04:38:23.594086+0000","last_clean":"2023-09-13T04:38:23.594086+0000","last_became_active":"2023-09-13T04:37:59.245060+0000","last_became_peered":"2023-09-13T04:37:59.245060+0000","last_unstale":"2023-09-13T04:38:23.594086+0000","last_undegraded":"2023-09-13T04:38:23.594086+0000","last_fullsized":"2023-09-13T04:38:23.594086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:25:41.078238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597807+0000","last_change":"2023-09-13T04:37:59.244776+0000","last_active":"2023-09-13T04:38:23.597807+0000","last_peered":"2023-09-13T04:38:23.597807+0000","last_clean":"2023-09-13T04:38:23.597807+0000","last_became_active":"2023-09-13T04:37:59.244409+0000","last_became_peered":"2023-09-13T04:37:59.244409+0000","last_unstale":"2023-09-13T04:38:23.597807+0000","last_undegraded":"2023-09-13T04:38:23.597807+0000","last_fullsized":"2023-09-13T04:38:23.597807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:04:02.561414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592391+0000","last_change":"2023-09-13T04:37:59.248153+0000","last_active":"2023-09-13T04:38:23.592391+0000","last_peered":"2023-09-13T04:38:23.592391+0000","last_clean":"2023-09-13T04:38:23.592391+0000","last_became_active":"2023-09-13T04:37:59.247840+0000","last_became_peered":"2023-09-13T04:37:59.247840+0000","last_unstale":"2023-09-13T04:38:23.592391+0000","last_undegraded":"2023-09-13T04:38:23.592391+0000","last_fullsized":"2023-09-13T04:38:23.592391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:52:53.021219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7452,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":46,"num_read_kb":36,"num_write":129,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":22356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416402,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":490770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.51000000000000001}]},{"osd":1,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.60199999999999998}]},{"osd":2,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.67600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":4,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.629}]},{"osd":5,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.89300000000000002}]},{"osd":6,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.84099999999999997}]},{"osd":7,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.70999999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":9,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.996}]},{"osd":10,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.754},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.90500000000000003}]}]},{"osd":10,"up_from":71,"seq":304942678038,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1085549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":1,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.70899999999999996}]},{"osd":2,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.53300000000000003}]},{"osd":3,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.80300000000000005}]},{"osd":4,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.67100000000000004}]},{"osd":5,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.74299999999999999}]},{"osd":6,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":3.4129999999999998,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.89300000000000002}]},{"osd":7,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.64500000000000002}]},{"osd":8,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.83899999999999997}]},{"osd":9,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.86899999999999999}]},{"osd":11,"last update":"Wed Sep 13 04:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.90900000000000003}]}]},{"osd":9,"up_from":66,"seq":283467841563,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":490860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58099999999999996,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.316,"15min":0.316},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.498}]},{"osd":1,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.44,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.71999999999999997,"15min":0.73199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82899999999999996,"5min":0.995,"15min":0.995},"last":0.54400000000000004}]},{"osd":2,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.747,"15min":0.75600000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.042,"5min":1.083,"15min":1.083},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.746,"15min":0.754},"min":{"1min":0.57099999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.83499999999999996,"5min":1.109,"15min":1.109},"last":0.63100000000000001}]},{"osd":3,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.42899999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.35999999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":0.47099999999999997}]},{"osd":4,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.78800000000000003,"15min":0.80500000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.82999999999999996,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.80400000000000005,"15min":0.81200000000000006},"min":{"1min":0.55600000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2,"5min":1.278,"15min":1.278},"last":0.75900000000000001}]},{"osd":5,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.85399999999999998,"15min":0.86699999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2689999999999999,"5min":1.339,"15min":1.339},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.81000000000000005,"15min":0.81599999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.119,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.59499999999999997}]},{"osd":6,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.72099999999999997}]},{"osd":7,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.90200000000000002,"15min":0.91400000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.3009999999999999,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.90500000000000003,"15min":0.92600000000000005},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1439999999999999,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.72099999999999997}]},{"osd":8,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.91700000000000004,"15min":0.93999999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0249999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.95499999999999996,"15min":0.97099999999999997},"min":{"1min":0.60299999999999998,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.2350000000000001,"5min":1.857,"15min":1.857},"last":0.65200000000000002}]},{"osd":10,"last update":"Wed Sep 13 04:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.68700000000000006}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]}]},{"osd":8,"up_from":58,"seq":249108103199,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":490337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.33200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56999999999999995,"15min":0.56499999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.54900000000000004}]},{"osd":1,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.54800000000000004,"15min":0.53600000000000003},"min":{"1min":0.38400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.58199999999999996},"min":{"1min":0.374,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.52400000000000002}]},{"osd":2,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.47499999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.504},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.48899999999999999,"15min":0.48599999999999999},"min":{"1min":0.29799999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.48399999999999999}]},{"osd":3,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.62},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64600000000000002,"15min":0.64300000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.71299999999999997}]},{"osd":4,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58899999999999997,"15min":0.58399999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.41099999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.76900000000000002}]},{"osd":5,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.34499999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80700000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.79400000000000004}]},{"osd":6,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.73099999999999998}]},{"osd":7,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71199999999999997,"15min":0.70999999999999996},"min":{"1min":0.48399999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69099999999999995,"15min":0.68700000000000006},"min":{"1min":0.503,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.86699999999999999}]},{"osd":9,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.95399999999999996}]},{"osd":10,"last update":"Wed Sep 13 04:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.88100000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]}]},{"osd":7,"up_from":51,"seq":219043332132,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1609728,"data_stored":1082202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.46500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.746,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60499999999999998,"15min":0.60799999999999998},"min":{"1min":0.41599999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.40200000000000002}]},{"osd":1,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.40899999999999997,"15min":0.40300000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42099999999999999,"15min":0.41999999999999998},"min":{"1min":0.30199999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65800000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.43099999999999999}]},{"osd":2,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.42599999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83899999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.57799999999999996,"15min":0.57099999999999995},"min":{"1min":0.443,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.70599999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.80200000000000005}]},{"osd":3,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66600000000000004,"15min":0.66900000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82799999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.64900000000000002}]},{"osd":4,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76600000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.502},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51200000000000001,"15min":0.51500000000000001},"min":{"1min":0.34499999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.47599999999999998}]},{"osd":5,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.628,"15min":0.626},"min":{"1min":0.47799999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64600000000000002,"15min":0.64200000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.85399999999999998}]},{"osd":6,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72199999999999998,"15min":0.72099999999999997},"min":{"1min":0.50600000000000001,"5min":0.496,"15min":0.496},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.58199999999999996}]},{"osd":8,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70299999999999996,"15min":0.70399999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.752},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.51700000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.88100000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.88800000000000001}]},{"osd":9,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.77900000000000003}]},{"osd":10,"last update":"Wed Sep 13 04:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.63}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]}]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":490132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46100000000000002,"15min":0.45100000000000001},"min":{"1min":0.34699999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.503,"15min":0.48999999999999999},"min":{"1min":0.38100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.50800000000000001}]},{"osd":1,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.47099999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82499999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.70999999999999996,"15min":0.67900000000000005},"min":{"1min":0.60399999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.753}]},{"osd":2,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.52400000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0129999999999999,"5min":1.054,"15min":1.054},"last":0.70099999999999996}]},{"osd":3,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.66100000000000003,"15min":0.64100000000000001},"min":{"1min":0.47799999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83599999999999997,"5min":1.113,"15min":1.113},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.626,"15min":0.61799999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89900000000000002,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.61299999999999999}]},{"osd":4,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76400000000000001,"15min":0.747},"min":{"1min":0.67900000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0509999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.75900000000000001,"15min":0.77100000000000002},"min":{"1min":0.52600000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92900000000000005,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.85399999999999998}]},{"osd":5,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.80800000000000005,"15min":0.79200000000000004},"min":{"1min":0.56100000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1950000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.877},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.79200000000000004,"15min":0.80800000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.90100000000000002}]},{"osd":7,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.83999999999999997,"15min":0.82399999999999995},"min":{"1min":0.63500000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.79800000000000004,"15min":0.78800000000000003},"min":{"1min":0.58999999999999997,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.95999999999999996}]},{"osd":8,"last update":"Wed Sep 13 04:38:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.81599999999999995,"15min":0.82399999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96599999999999997,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.91900000000000004,"15min":0.92400000000000004},"min":{"1min":0.61199999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2629999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.94399999999999995}]},{"osd":9,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.86499999999999999,"15min":0.88300000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.98099999999999998,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.88900000000000001,"5min":1.0069999999999999,"15min":1.0269999999999999},"min":{"1min":0.72399999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.0089999999999999,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.97799999999999998}]},{"osd":10,"last update":"Wed Sep 13 04:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":1.002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":493884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55600000000000005,"15min":0.53700000000000003},"min":{"1min":0.33700000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.503},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56999999999999995,"15min":0.55200000000000005},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.64200000000000002}]},{"osd":2,"last update":"Wed Sep 13 04:37:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.55900000000000005,"15min":0.53800000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.54800000000000004,"15min":0.51800000000000002},"min":{"1min":0.42299999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.72099999999999997}]},{"osd":3,"last update":"Wed Sep 13 04:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83099999999999996,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.622},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.623,"15min":0.60899999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91600000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.55400000000000005}]},{"osd":4,"last update":"Wed Sep 13 04:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.46200000000000002,"15min":0.45300000000000001},"min":{"1min":0.29599999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.45800000000000002,"15min":0.44800000000000001},"min":{"1min":0.29199999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.53100000000000003}]},{"osd":5,"last update":"Wed Sep 13 04:37:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.44800000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89000000000000001,"5min":1.3,"15min":1.3},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61499999999999999,"15min":0.58299999999999996},"min":{"1min":0.435,"5min":0.37,"15min":0.37},"max":{"1min":1,"5min":1.264,"15min":1.264},"last":0.85299999999999998}]},{"osd":6,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.99199999999999999,"5min":1.482,"15min":1.482},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95399999999999996,"5min":1.375,"15min":1.375},"last":0.89800000000000002}]},{"osd":7,"last update":"Wed Sep 13 04:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.60699999999999998,"15min":0.58699999999999997},"min":{"1min":0.39900000000000002,"5min":0.372,"15min":0.372},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.65900000000000003,"15min":0.64300000000000002},"min":{"1min":0.47399999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.83999999999999997}]},{"osd":8,"last update":"Wed Sep 13 04:38:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.48899999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2010000000000001,"5min":1.393,"15min":1.393},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.093,"5min":1.224,"15min":1.224},"last":0.95399999999999996}]},{"osd":9,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.94299999999999995}]},{"osd":10,"last update":"Wed Sep 13 04:37:53 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.97299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":491520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 13 04:37:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":0.66000000000000003,"15min":0.58299999999999996},"min":{"1min":0.42199999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.2359999999999998,"5min":6.2359999999999998,"15min":6.2359999999999998},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.58699999999999997,"15min":0.55200000000000005},"min":{"1min":0.47899999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.67000000000000004}]},{"osd":2,"last update":"Wed Sep 13 04:37:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.624,"15min":0.59899999999999998},"min":{"1min":0.42899999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93899999999999995,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63700000000000001,"15min":0.60299999999999998},"min":{"1min":0.439,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.96899999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.89900000000000002}]},{"osd":3,"last update":"Wed Sep 13 04:36:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.57099999999999995,"15min":0.55500000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.45500000000000002}]},{"osd":4,"last update":"Wed Sep 13 04:37:30 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.65900000000000003,"15min":0.61299999999999999},"min":{"1min":0.58699999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.64900000000000002,"15min":0.621},"min":{"1min":0.51400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.75900000000000001}]},{"osd":5,"last update":"Wed Sep 13 04:38:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68300000000000005,"15min":0.67000000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.70699999999999996,"15min":0.68300000000000005},"min":{"1min":0.55100000000000005,"5min":0.435,"15min":0.435},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.85199999999999998}]},{"osd":6,"last update":"Wed Sep 13 04:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.44600000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94899999999999995,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88700000000000001,"5min":1.165,"15min":1.165},"last":0.56999999999999995}]},{"osd":7,"last update":"Wed Sep 13 04:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.79500000000000004,"15min":0.78500000000000003},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.81599999999999995,"15min":0.80200000000000005},"min":{"1min":0.60099999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":1.0109999999999999}]},{"osd":8,"last update":"Wed Sep 13 04:38:00 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":0.90600000000000003,"15min":0.85999999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":6.2089999999999996,"5min":6.2089999999999996,"15min":6.2089999999999996},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.171,"5min":0.87,"15min":0.81999999999999995},"min":{"1min":0.52400000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":6.1699999999999999,"5min":6.1699999999999999,"15min":6.1699999999999999},"last":0.98899999999999999}]},{"osd":9,"last update":"Wed Sep 13 04:37:30 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.874},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.96699999999999997}]},{"osd":10,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":1.048},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":6.1859999999999999,"5min":6.1859999999999999,"15min":6.1859999999999999},"last":0.83499999999999996}]},{"osd":11,"last update":"Wed Sep 13 04:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.79900000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":495200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:55 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.34599999999999997,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79100000000000004,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51900000000000002,"15min":0.51800000000000002},"min":{"1min":0.371,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78300000000000003,"5min":1.429,"15min":1.429},"last":0.58499999999999996}]},{"osd":1,"last update":"Wed Sep 13 04:37:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.53500000000000003,"15min":0.496},"min":{"1min":0.39000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80700000000000005,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54300000000000004,"15min":0.50700000000000001},"min":{"1min":0.39100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70799999999999996,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.46100000000000002}]},{"osd":3,"last update":"Wed Sep 13 04:38:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.36499999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.747,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.441,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81399999999999995,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.623}]},{"osd":4,"last update":"Wed Sep 13 04:37:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.52700000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78000000000000003,"5min":1.244,"15min":1.244},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55800000000000005,"15min":0.53000000000000003},"min":{"1min":0.436,"5min":0.374,"15min":0.374},"max":{"1min":0.85499999999999998,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.73099999999999998}]},{"osd":5,"last update":"Wed Sep 13 04:37:57 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.28699999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.70899999999999996,"5min":1.796,"15min":1.796},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.30599999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.99299999999999999,"5min":1.546,"15min":1.546},"last":0.60299999999999998}]},{"osd":6,"last update":"Wed Sep 13 04:38:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.69199999999999995,"15min":0.69899999999999995},"min":{"1min":0.48499999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81599999999999995,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66700000000000004,"15min":0.67200000000000004},"min":{"1min":0.48199999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85899999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.56299999999999994}]},{"osd":7,"last update":"Wed Sep 13 04:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65400000000000003,"15min":0.65800000000000003},"min":{"1min":0.42499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94699999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.497,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92500000000000004,"5min":1.147,"15min":1.147},"last":0.70899999999999996}]},{"osd":8,"last update":"Wed Sep 13 04:38:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61699999999999999,"15min":0.61899999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.96599999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.38400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.76300000000000001}]},{"osd":9,"last update":"Wed Sep 13 04:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72099999999999997,"15min":0.71899999999999997},"min":{"1min":0.57299999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.93100000000000005,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.69099999999999995}]},{"osd":10,"last update":"Wed Sep 13 04:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.86699999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":3,"up_from":25,"seq":107374182456,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":490326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.42699999999999999,"15min":0.39000000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.46500000000000002,"15min":0.435},"min":{"1min":0.375,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78500000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46100000000000002}]},{"osd":1,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.57699999999999996},"min":{"1min":0.45700000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64200000000000002,"15min":0.57699999999999996},"min":{"1min":0.436,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92200000000000004,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.76300000000000001}]},{"osd":2,"last update":"Wed Sep 13 04:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.67400000000000004,"15min":0.629},"min":{"1min":0.53000000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64500000000000002,"15min":0.621},"min":{"1min":0.45800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.59799999999999998}]},{"osd":4,"last update":"Wed Sep 13 04:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.64300000000000002,"15min":0.60099999999999998},"min":{"1min":0.52900000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.64100000000000001,"15min":0.60699999999999998},"min":{"1min":0.44500000000000001,"5min":0.439,"15min":0.439},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.70399999999999996}]},{"osd":5,"last update":"Wed Sep 13 04:38:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.68999999999999995,"15min":0.65000000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.024,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.873,"5min":0.72399999999999998,"15min":0.66600000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":0.82099999999999995}]},{"osd":6,"last update":"Wed Sep 13 04:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.74099999999999999,"15min":0.70299999999999996},"min":{"1min":0.47399999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1060000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.76800000000000002,"15min":0.745},"min":{"1min":0.504,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0529999999999999,"5min":1.109,"15min":1.109},"last":0.879}]},{"osd":7,"last update":"Wed Sep 13 04:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.86299999999999999,"15min":0.86399999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.85399999999999998,"15min":0.84999999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.90800000000000003}]},{"osd":8,"last update":"Wed Sep 13 04:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.875}]},{"osd":9,"last update":"Wed Sep 13 04:38:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.94699999999999995,"15min":0.96399999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.2030000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.84099999999999997,"15min":0.84899999999999998},"min":{"1min":0.52100000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.01,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.65900000000000003}]},{"osd":10,"last update":"Wed Sep 13 04:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.94599999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":4,"up_from":33,"seq":141733920819,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":490686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.56599999999999995,"15min":0.54400000000000004},"min":{"1min":0.43099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.753},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76500000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.68999999999999995}]},{"osd":1,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.373,"15min":0.34699999999999998},"min":{"1min":0.28399999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.501},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.38700000000000001,"15min":0.36399999999999999},"min":{"1min":0.28100000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.45600000000000002}]},{"osd":2,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58299999999999996,"15min":0.56100000000000005},"min":{"1min":0.41099999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56599999999999995,"15min":0.53800000000000003},"min":{"1min":0.41399999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.73699999999999999}]},{"osd":3,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.59399999999999997,"15min":0.56499999999999995},"min":{"1min":0.41799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.627},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59399999999999997,"15min":0.57699999999999996},"min":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.68100000000000005}]},{"osd":5,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60299999999999998,"15min":0.58599999999999997},"min":{"1min":0.45100000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90300000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.39700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.53800000000000003}]},{"osd":6,"last update":"Wed Sep 13 04:38:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.68000000000000005,"15min":0.68400000000000005},"min":{"1min":0.47099999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81599999999999995,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.69799999999999995,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.84599999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.70399999999999996}]},{"osd":7,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52400000000000002,"15min":0.51900000000000002},"min":{"1min":0.36199999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73599999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.85199999999999998}]},{"osd":8,"last update":"Wed Sep 13 04:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.70699999999999996,"15min":0.69199999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76700000000000002,"15min":0.76300000000000001},"min":{"1min":0.53000000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.89400000000000002}]},{"osd":9,"last update":"Wed Sep 13 04:38:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.80700000000000005,"15min":0.81200000000000006},"min":{"1min":0.59299999999999997,"5min":0.495,"15min":0.495},"max":{"1min":0.93000000000000005,"5min":1.141,"15min":1.141},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.77100000000000002,"15min":0.77300000000000002},"min":{"1min":0.60799999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.879,"5min":1.056,"15min":1.056},"last":0.879}]},{"osd":10,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.83799999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1622016,"data_stored":1081886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53000000000000003,"15min":0.52400000000000002},"min":{"1min":0.34399999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72099999999999997,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.317,"5min":0.315,"15min":0.315},"max":{"1min":0.70299999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.50900000000000001}]},{"osd":1,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.40300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78500000000000003,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.54300000000000004,"15min":0.52400000000000002},"min":{"1min":0.34799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89600000000000002,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.66300000000000003}]},{"osd":2,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.38,"15min":0.36099999999999999},"min":{"1min":0.26900000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.39600000000000002,"15min":0.378},"min":{"1min":0.28299999999999997,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.71799999999999997}]},{"osd":3,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.372,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92600000000000005,"5min":1.036,"15min":1.036},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.78700000000000003,"15min":0.86899999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.1259999999999999,"5min":8.0009999999999994,"15min":8.0009999999999994},"last":0.73599999999999999}]},{"osd":4,"last update":"Wed Sep 13 04:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.58299999999999996,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82299999999999995,"5min":1.599,"15min":1.599},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.57699999999999996,"15min":0.55500000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.86599999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.75600000000000001}]},{"osd":6,"last update":"Wed Sep 13 04:38:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83499999999999996,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81899999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.79100000000000004}]},{"osd":7,"last update":"Wed Sep 13 04:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.878,"5min":1.488,"15min":1.488},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94899999999999995,"5min":1.51,"15min":1.51},"last":0.85499999999999998}]},{"osd":8,"last update":"Wed Sep 13 04:38:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.38900000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.60599999999999998,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.877}]},{"osd":9,"last update":"Wed Sep 13 04:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.91000000000000003}]},{"osd":10,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.92300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-13T04:38:33.590 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-13T04:38:33.591 INFO:tasks.cephadm:Setting up client nodes... 2023-09-13T04:38:33.591 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-09-13T04:38:34.356 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:38:34.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:34 smithi149 ceph-mon[133315]: pgmap v295: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.5 KiB/s wr, 12 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2023-09-13T04:38:34.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:34 smithi149 ceph-mon[133315]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:38:34.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:34 smithi156 ceph-mon[133675]: pgmap v295: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.5 KiB/s wr, 12 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2023-09-13T04:38:34.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:34 smithi156 ceph-mon[133675]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:38:34.974 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:34 smithi002 ceph-mon[97651]: pgmap v295: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.5 KiB/s wr, 12 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2023-09-13T04:38:34.974 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:34 smithi002 ceph-mon[97651]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:38:35.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.880+0000 7f6dc0006700 1 -- 172.21.15.2:0/1418096109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6db8108c80 msgr2=0x7f6db810b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:35.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.880+0000 7f6dc0006700 1 --2- 172.21.15.2:0/1418096109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6db8108c80 0x7f6db810b110 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6db0009f90 tx=0x7f6db000b330 comp rx=0 tx=0).stop 2023-09-13T04:38:35.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.881+0000 7f6dc0006700 1 -- 172.21.15.2:0/1418096109 shutdown_connections 2023-09-13T04:38:35.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.881+0000 7f6dc0006700 1 --2- 172.21.15.2:0/1418096109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6db8108c80 0x7f6db810b110 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:35.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.881+0000 7f6dc0006700 1 --2- 172.21.15.2:0/1418096109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6db8074ce0 0x7f6db8073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:35.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.881+0000 7f6dc0006700 1 --2- 172.21.15.2:0/1418096109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6db8073710 0x7f6db8073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:35.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.881+0000 7f6dc0006700 1 -- 172.21.15.2:0/1418096109 >> 172.21.15.2:0/1418096109 conn(0x7f6db80fc370 msgr2=0x7f6db80fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:35.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.881+0000 7f6dc0006700 1 -- 172.21.15.2:0/1418096109 shutdown_connections 2023-09-13T04:38:35.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.881+0000 7f6dc0006700 1 -- 172.21.15.2:0/1418096109 wait complete. 2023-09-13T04:38:35.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.882+0000 7f6dc0006700 1 Processor -- start 2023-09-13T04:38:35.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.882+0000 7f6dc0006700 1 -- start start 2023-09-13T04:38:35.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.882+0000 7f6dc0006700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6db8073710 0x7f6db8198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:35.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.882+0000 7f6dc0006700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6db8074ce0 0x7f6db81992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:35.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.882+0000 7f6dc0006700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6db8108c80 0x7f6db81a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:35.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.883+0000 7f6dc0006700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db81a08e0 con 0x7f6db8073710 2023-09-13T04:38:35.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.883+0000 7f6dc0006700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db81a0a20 con 0x7f6db8108c80 2023-09-13T04:38:35.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.883+0000 7f6dc0006700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db81a0be0 con 0x7f6db8074ce0 2023-09-13T04:38:35.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.883+0000 7f6dbdda2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6db8073710 0x7f6db8198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:35.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.883+0000 7f6dbdda2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6db8073710 0x7f6db8198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47174/0 (socket says 172.21.15.2:47174) 2023-09-13T04:38:35.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.883+0000 7f6dbdda2700 1 -- 172.21.15.2:0/3840591411 learned_addr learned my addr 172.21.15.2:0/3840591411 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:35.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.883+0000 7f6dbd5a1700 1 --2- 172.21.15.2:0/3840591411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6db8074ce0 0x7f6db81992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:35.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.883+0000 7f6dbe5a3700 1 --2- 172.21.15.2:0/3840591411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6db8108c80 0x7f6db81a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:35.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.883+0000 7f6dbdda2700 1 -- 172.21.15.2:0/3840591411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6db8074ce0 msgr2=0x7f6db81992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:35.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.883+0000 7f6dbdda2700 1 --2- 172.21.15.2:0/3840591411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6db8074ce0 0x7f6db81992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:35.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.883+0000 7f6dbdda2700 1 -- 172.21.15.2:0/3840591411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6db8108c80 msgr2=0x7f6db81a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:35.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.883+0000 7f6dbdda2700 1 --2- 172.21.15.2:0/3840591411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6db8108c80 0x7f6db81a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:35.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.883+0000 7f6dbdda2700 1 -- 172.21.15.2:0/3840591411 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db0009c40 con 0x7f6db8073710 2023-09-13T04:38:35.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.883+0000 7f6dbe5a3700 1 --2- 172.21.15.2:0/3840591411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6db8108c80 0x7f6db81a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:38:35.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.884+0000 7f6dbdda2700 1 --2- 172.21.15.2:0/3840591411 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6db8073710 0x7f6db8198d90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6db400e970 tx=0x7f6db400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:35.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.884+0000 7f6daeffd700 1 -- 172.21.15.2:0/3840591411 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db4009790 con 0x7f6db8073710 2023-09-13T04:38:35.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.884+0000 7f6daeffd700 1 -- 172.21.15.2:0/3840591411 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6db400eeb0 con 0x7f6db8073710 2023-09-13T04:38:35.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.884+0000 7f6dc0006700 1 -- 172.21.15.2:0/3840591411 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db81a0ec0 con 0x7f6db8073710 2023-09-13T04:38:35.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.884+0000 7f6daeffd700 1 -- 172.21.15.2:0/3840591411 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db40055c0 con 0x7f6db8073710 2023-09-13T04:38:35.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.884+0000 7f6dc0006700 1 -- 172.21.15.2:0/3840591411 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db81a12e0 con 0x7f6db8073710 2023-09-13T04:38:35.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.885+0000 7f6dc0006700 1 -- 172.21.15.2:0/3840591411 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db8107330 con 0x7f6db8073710 2023-09-13T04:38:35.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.887+0000 7f6daeffd700 1 -- 172.21.15.2:0/3840591411 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f6db4004750 con 0x7f6db8073710 2023-09-13T04:38:35.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.888+0000 7f6daeffd700 1 --2- 172.21.15.2:0/3840591411 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f6da40a06c0 0x7f6da40a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:35.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.888+0000 7f6daeffd700 1 -- 172.21.15.2:0/3840591411 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6db4014070 con 0x7f6db8073710 2023-09-13T04:38:35.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.889+0000 7f6dbd5a1700 1 --2- 172.21.15.2:0/3840591411 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f6da40a06c0 0x7f6da40a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:35.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.889+0000 7f6dbd5a1700 1 --2- 172.21.15.2:0/3840591411 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f6da40a06c0 0x7f6da40a2b80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6db8074a10 tx=0x7f6da8009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:35.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:35.893+0000 7f6daeffd700 1 -- 172.21.15.2:0/3840591411 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6db4095900 con 0x7f6db8073710 2023-09-13T04:38:36.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:36.190+0000 7f6dc0006700 1 -- 172.21.15.2:0/3840591411 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f6db810e7d0 con 0x7f6db8073710 2023-09-13T04:38:36.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:36.195+0000 7f6daeffd700 1 -- 172.21.15.2:0/3840591411 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f6db4098f20 con 0x7f6db8073710 2023-09-13T04:38:36.196 INFO:teuthology.orchestra.run.smithi002.stdout:[client.0] 2023-09-13T04:38:36.196 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQDMPAFlc513CxAAZywmJoO54wwOlQnjoUWgVw== 2023-09-13T04:38:36.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:36.197+0000 7f6dc0006700 1 -- 172.21.15.2:0/3840591411 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f6da40a06c0 msgr2=0x7f6da40a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:36.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:36.197+0000 7f6dc0006700 1 --2- 172.21.15.2:0/3840591411 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f6da40a06c0 0x7f6da40a2b80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6db8074a10 tx=0x7f6da8009310 comp rx=0 tx=0).stop 2023-09-13T04:38:36.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:36.197+0000 7f6dc0006700 1 -- 172.21.15.2:0/3840591411 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6db8073710 msgr2=0x7f6db8198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:36.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:36.197+0000 7f6dc0006700 1 --2- 172.21.15.2:0/3840591411 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6db8073710 0x7f6db8198d90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6db400e970 tx=0x7f6db400ed30 comp rx=0 tx=0).stop 2023-09-13T04:38:36.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:36.197+0000 7f6dc0006700 1 -- 172.21.15.2:0/3840591411 shutdown_connections 2023-09-13T04:38:36.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:36.197+0000 7f6dc0006700 1 --2- 172.21.15.2:0/3840591411 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f6da40a06c0 0x7f6da40a2b80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:36.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:36.197+0000 7f6dc0006700 1 --2- 172.21.15.2:0/3840591411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6db8108c80 0x7f6db81a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:36.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:36.197+0000 7f6dc0006700 1 --2- 172.21.15.2:0/3840591411 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6db8073710 0x7f6db8198d90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:36.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:36.197+0000 7f6dc0006700 1 --2- 172.21.15.2:0/3840591411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6db8074ce0 0x7f6db81992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:36.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:36.197+0000 7f6dc0006700 1 -- 172.21.15.2:0/3840591411 >> 172.21.15.2:0/3840591411 conn(0x7f6db80fc370 msgr2=0x7f6db80fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:36.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:36.198+0000 7f6dc0006700 1 -- 172.21.15.2:0/3840591411 shutdown_connections 2023-09-13T04:38:36.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:36.198+0000 7f6dc0006700 1 -- 172.21.15.2:0/3840591411 wait complete. 2023-09-13T04:38:36.657 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-09-13T04:38:36.658 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-09-13T04:38:36.658 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-09-13T04:38:36.762 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-13T04:38:36.764 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-09-13T04:38:36.764 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph mgr dump --format=json 2023-09-13T04:38:36.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:36 smithi149 ceph-mon[133315]: pgmap v296: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.3 KiB/s wr, 13 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2023-09-13T04:38:36.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:36 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3840591411' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-13T04:38:36.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:36 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3840591411' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-13T04:38:36.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:36 smithi156 ceph-mon[133675]: pgmap v296: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.3 KiB/s wr, 13 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2023-09-13T04:38:36.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:36 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3840591411' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-13T04:38:36.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:36 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3840591411' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-13T04:38:37.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:36 smithi002 ceph-mon[97651]: pgmap v296: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.3 KiB/s wr, 13 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2023-09-13T04:38:37.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:36 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3840591411' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-13T04:38:37.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:36 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3840591411' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-13T04:38:37.531 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:38:38.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:38 smithi002 ceph-mon[97651]: pgmap v297: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 11 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2023-09-13T04:38:38.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:38 smithi149 ceph-mon[133315]: pgmap v297: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 11 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2023-09-13T04:38:38.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:38 smithi156 ceph-mon[133675]: pgmap v297: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 11 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2023-09-13T04:38:39.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.085+0000 7f83191df700 1 -- 172.21.15.2:0/4114567442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8314106d90 msgr2=0x7f8314107170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:39.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.085+0000 7f83191df700 1 --2- 172.21.15.2:0/4114567442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8314106d90 0x7f8314107170 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f83040099c0 tx=0x7f8304009cd0 comp rx=0 tx=0).stop 2023-09-13T04:38:39.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.085+0000 7f83191df700 1 -- 172.21.15.2:0/4114567442 shutdown_connections 2023-09-13T04:38:39.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.085+0000 7f83191df700 1 --2- 172.21.15.2:0/4114567442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8314101740 0x7f831410dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:39.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.085+0000 7f83191df700 1 --2- 172.21.15.2:0/4114567442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8314106d90 0x7f8314107170 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:39.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.085+0000 7f83191df700 1 --2- 172.21.15.2:0/4114567442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8314100d80 0x7f8314101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:39.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.085+0000 7f83191df700 1 -- 172.21.15.2:0/4114567442 >> 172.21.15.2:0/4114567442 conn(0x7f8314077ea0 msgr2=0x7f83140782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:39.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.085+0000 7f83191df700 1 -- 172.21.15.2:0/4114567442 shutdown_connections 2023-09-13T04:38:39.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.085+0000 7f83191df700 1 -- 172.21.15.2:0/4114567442 wait complete. 2023-09-13T04:38:39.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.086+0000 7f83191df700 1 Processor -- start 2023-09-13T04:38:39.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.086+0000 7f83191df700 1 -- start start 2023-09-13T04:38:39.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.087+0000 7f83191df700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8314100d80 0x7f8314196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:39.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.087+0000 7f83191df700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8314101740 0x7f8314197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:39.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.087+0000 7f83191df700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8314106d90 0x7f831419b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:39.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.087+0000 7f83191df700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831419bda0 con 0x7f8314100d80 2023-09-13T04:38:39.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.087+0000 7f83191df700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831419bf10 con 0x7f8314106d90 2023-09-13T04:38:39.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.087+0000 7f83191df700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831419c0d0 con 0x7f8314101740 2023-09-13T04:38:39.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.087+0000 7f8312d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8314100d80 0x7f8314196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:39.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.087+0000 7f8312d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8314100d80 0x7f8314196e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47188/0 (socket says 172.21.15.2:47188) 2023-09-13T04:38:39.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.087+0000 7f8312d9d700 1 -- 172.21.15.2:0/773000063 learned_addr learned my addr 172.21.15.2:0/773000063 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:39.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.087+0000 7f831259c700 1 --2- 172.21.15.2:0/773000063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8314101740 0x7f8314197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:39.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.087+0000 7f831359e700 1 --2- 172.21.15.2:0/773000063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8314106d90 0x7f831419b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:39.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.088+0000 7f8312d9d700 1 -- 172.21.15.2:0/773000063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8314101740 msgr2=0x7f8314197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:39.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.088+0000 7f8312d9d700 1 --2- 172.21.15.2:0/773000063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8314101740 0x7f8314197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:39.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.088+0000 7f8312d9d700 1 -- 172.21.15.2:0/773000063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8314106d90 msgr2=0x7f831419b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:39.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.088+0000 7f8312d9d700 1 --2- 172.21.15.2:0/773000063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8314106d90 0x7f831419b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:39.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.088+0000 7f8312d9d700 1 -- 172.21.15.2:0/773000063 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83040095f0 con 0x7f8314100d80 2023-09-13T04:38:39.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.088+0000 7f8312d9d700 1 --2- 172.21.15.2:0/773000063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8314100d80 0x7f8314196e30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f83040098e0 tx=0x7f830400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:39.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.088+0000 7f82fbfff700 1 -- 172.21.15.2:0/773000063 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830401d070 con 0x7f8314100d80 2023-09-13T04:38:39.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.089+0000 7f82fbfff700 1 -- 172.21.15.2:0/773000063 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8304022470 con 0x7f8314100d80 2023-09-13T04:38:39.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.089+0000 7f83191df700 1 -- 172.21.15.2:0/773000063 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f831419c350 con 0x7f8314100d80 2023-09-13T04:38:39.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.089+0000 7f82fbfff700 1 -- 172.21.15.2:0/773000063 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8304017650 con 0x7f8314100d80 2023-09-13T04:38:39.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.089+0000 7f83191df700 1 -- 172.21.15.2:0/773000063 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83141a3c10 con 0x7f8314100d80 2023-09-13T04:38:39.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.090+0000 7f83191df700 1 -- 172.21.15.2:0/773000063 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f831404e9b0 con 0x7f8314100d80 2023-09-13T04:38:39.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.092+0000 7f82fbfff700 1 -- 172.21.15.2:0/773000063 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f8304022850 con 0x7f8314100d80 2023-09-13T04:38:39.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.093+0000 7f82fbfff700 1 --2- 172.21.15.2:0/773000063 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f82fc0a0640 0x7f82fc0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:39.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.093+0000 7f82fbfff700 1 -- 172.21.15.2:0/773000063 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f83040ccfd0 con 0x7f8314100d80 2023-09-13T04:38:39.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.093+0000 7f831259c700 1 --2- 172.21.15.2:0/773000063 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f82fc0a0640 0x7f82fc0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:39.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.094+0000 7f831259c700 1 --2- 172.21.15.2:0/773000063 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f82fc0a0640 0x7f82fc0a2b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8314198400 tx=0x7f8308009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:39.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.095+0000 7f82fbfff700 1 -- 172.21.15.2:0/773000063 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8304096820 con 0x7f8314100d80 2023-09-13T04:38:39.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.369+0000 7f83191df700 1 -- 172.21.15.2:0/773000063 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f8314110ca0 con 0x7f8314100d80 2023-09-13T04:38:39.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.375+0000 7f82fbfff700 1 -- 172.21.15.2:0/773000063 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v16) v1 ==== 74+0+258922 (secure 0 0 0) 0x7f8304099e40 con 0x7f8314100d80 2023-09-13T04:38:39.380 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:38:39.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.379+0000 7f83191df700 1 -- 172.21.15.2:0/773000063 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f82fc0a0640 msgr2=0x7f82fc0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:39.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.379+0000 7f83191df700 1 --2- 172.21.15.2:0/773000063 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f82fc0a0640 0x7f82fc0a2b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8314198400 tx=0x7f8308009260 comp rx=0 tx=0).stop 2023-09-13T04:38:39.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.379+0000 7f83191df700 1 -- 172.21.15.2:0/773000063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8314100d80 msgr2=0x7f8314196e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:39.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.379+0000 7f83191df700 1 --2- 172.21.15.2:0/773000063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8314100d80 0x7f8314196e30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f83040098e0 tx=0x7f830400f690 comp rx=0 tx=0).stop 2023-09-13T04:38:39.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.379+0000 7f83191df700 1 -- 172.21.15.2:0/773000063 shutdown_connections 2023-09-13T04:38:39.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.379+0000 7f83191df700 1 --2- 172.21.15.2:0/773000063 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f82fc0a0640 0x7f82fc0a2b00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:39.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.379+0000 7f83191df700 1 --2- 172.21.15.2:0/773000063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8314106d90 0x7f831419b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:39.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.379+0000 7f83191df700 1 --2- 172.21.15.2:0/773000063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8314100d80 0x7f8314196e30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:39.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.379+0000 7f83191df700 1 --2- 172.21.15.2:0/773000063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8314101740 0x7f8314197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:39.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.379+0000 7f83191df700 1 -- 172.21.15.2:0/773000063 >> 172.21.15.2:0/773000063 conn(0x7f8314077ea0 msgr2=0x7f83140ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:39.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.379+0000 7f83191df700 1 -- 172.21.15.2:0/773000063 shutdown_connections 2023-09-13T04:38:39.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:39.380+0000 7f83191df700 1 -- 172.21.15.2:0/773000063 wait complete. 2023-09-13T04:38:39.827 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":16,"active_gid":14148,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":1461251021},{"type":"v1","addr":"172.21.15.2:6801","nonce":1461251021}]},"active_addr":"172.21.15.2:6801/1461251021","active_change":"2023-09-13T04:31:23.417843+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.2:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":227933483}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":2528007246}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":2955382590}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":3568881483}]}]} 2023-09-13T04:38:39.833 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-09-13T04:38:39.833 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-09-13T04:38:39.834 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph osd dump --format=json 2023-09-13T04:38:40.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:40 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/773000063' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-13T04:38:40.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:40 smithi002 ceph-mon[97651]: pgmap v298: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 8 op/s 2023-09-13T04:38:40.568 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:38:40.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:40 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/773000063' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-13T04:38:40.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:40 smithi149 ceph-mon[133315]: pgmap v298: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 8 op/s 2023-09-13T04:38:40.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:40 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/773000063' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-13T04:38:40.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:40 smithi156 ceph-mon[133675]: pgmap v298: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 8 op/s 2023-09-13T04:38:42.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.128+0000 7f4ebfd3c700 1 -- 172.21.15.2:0/3091756736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb8073800 msgr2=0x7f4eb810d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:42.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.128+0000 7f4ebfd3c700 1 --2- 172.21.15.2:0/3091756736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb8073800 0x7f4eb810d1f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009990 tx=0x7f4ea8009ca0 comp rx=0 tx=0).stop 2023-09-13T04:38:42.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.128+0000 7f4ebfd3c700 1 -- 172.21.15.2:0/3091756736 shutdown_connections 2023-09-13T04:38:42.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.128+0000 7f4ebfd3c700 1 --2- 172.21.15.2:0/3091756736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4eb810d730 0x7f4eb810fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:42.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.128+0000 7f4ebfd3c700 1 --2- 172.21.15.2:0/3091756736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4eb8072ee0 0x7f4eb80732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:42.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.128+0000 7f4ebfd3c700 1 --2- 172.21.15.2:0/3091756736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb8073800 0x7f4eb810d1f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:42.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.128+0000 7f4ebfd3c700 1 -- 172.21.15.2:0/3091756736 >> 172.21.15.2:0/3091756736 conn(0x7f4eb8077ea0 msgr2=0x7f4eb80782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:42.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.128+0000 7f4ebfd3c700 1 -- 172.21.15.2:0/3091756736 shutdown_connections 2023-09-13T04:38:42.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.128+0000 7f4ebfd3c700 1 -- 172.21.15.2:0/3091756736 wait complete. 2023-09-13T04:38:42.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.129+0000 7f4ebfd3c700 1 Processor -- start 2023-09-13T04:38:42.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.129+0000 7f4ebfd3c700 1 -- start start 2023-09-13T04:38:42.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.129+0000 7f4ebfd3c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4eb8072ee0 0x7f4eb8198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:42.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.129+0000 7f4ebfd3c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb8073800 0x7f4eb8199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.129+0000 7f4ebfd3c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4eb810d730 0x7f4eb819d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.129+0000 7f4ebfd3c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb819dfa0 con 0x7f4eb8073800 2023-09-13T04:38:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.129+0000 7f4ebfd3c700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb819e110 con 0x7f4eb8072ee0 2023-09-13T04:38:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.129+0000 7f4ebfd3c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb819e2d0 con 0x7f4eb810d730 2023-09-13T04:38:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.130+0000 7f4ebd2d7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb8073800 0x7f4eb8199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:42.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.130+0000 7f4ebdad8700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4eb8072ee0 0x7f4eb8198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:42.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.130+0000 7f4ebd2d7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb8073800 0x7f4eb8199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47206/0 (socket says 172.21.15.2:47206) 2023-09-13T04:38:42.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.130+0000 7f4ebd2d7700 1 -- 172.21.15.2:0/1211737258 learned_addr learned my addr 172.21.15.2:0/1211737258 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:42.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.130+0000 7f4ebe2d9700 1 --2- 172.21.15.2:0/1211737258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4eb810d730 0x7f4eb819d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:42.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.130+0000 7f4ebd2d7700 1 -- 172.21.15.2:0/1211737258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4eb810d730 msgr2=0x7f4eb819d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:42.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.130+0000 7f4ebd2d7700 1 --2- 172.21.15.2:0/1211737258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4eb810d730 0x7f4eb819d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:42.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.130+0000 7f4ebd2d7700 1 -- 172.21.15.2:0/1211737258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4eb8072ee0 msgr2=0x7f4eb8198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:42.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.130+0000 7f4ebd2d7700 1 --2- 172.21.15.2:0/1211737258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4eb8072ee0 0x7f4eb8198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:42.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.130+0000 7f4ebd2d7700 1 -- 172.21.15.2:0/1211737258 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea80095f0 con 0x7f4eb8073800 2023-09-13T04:38:42.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.130+0000 7f4ebe2d9700 1 --2- 172.21.15.2:0/1211737258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4eb810d730 0x7f4eb819d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:38:42.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.130+0000 7f4ebdad8700 1 --2- 172.21.15.2:0/1211737258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4eb8072ee0 0x7f4eb8198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:38:42.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.130+0000 7f4ebd2d7700 1 --2- 172.21.15.2:0/1211737258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb8073800 0x7f4eb8199520 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8000c00 tx=0x7f4ea800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:42.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.131+0000 7f4eaeffd700 1 -- 172.21.15.2:0/1211737258 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea801d070 con 0x7f4eb8073800 2023-09-13T04:38:42.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.131+0000 7f4eaeffd700 1 -- 172.21.15.2:0/1211737258 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ea800be40 con 0x7f4eb8073800 2023-09-13T04:38:42.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.131+0000 7f4eaeffd700 1 -- 172.21.15.2:0/1211737258 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8017650 con 0x7f4eb8073800 2023-09-13T04:38:42.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.131+0000 7f4ebfd3c700 1 -- 172.21.15.2:0/1211737258 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb819e550 con 0x7f4eb8073800 2023-09-13T04:38:42.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.131+0000 7f4ebfd3c700 1 -- 172.21.15.2:0/1211737258 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb81a5c40 con 0x7f4eb8073800 2023-09-13T04:38:42.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.132+0000 7f4ebfd3c700 1 -- 172.21.15.2:0/1211737258 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb804e9b0 con 0x7f4eb8073800 2023-09-13T04:38:42.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.134+0000 7f4eaeffd700 1 -- 172.21.15.2:0/1211737258 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f4ea8022470 con 0x7f4eb8073800 2023-09-13T04:38:42.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.135+0000 7f4eaeffd700 1 --2- 172.21.15.2:0/1211737258 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4ea40a0640 0x7f4ea40a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:42.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.135+0000 7f4eaeffd700 1 -- 172.21.15.2:0/1211737258 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4ea80cd420 con 0x7f4eb8073800 2023-09-13T04:38:42.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.136+0000 7f4ebdad8700 1 --2- 172.21.15.2:0/1211737258 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4ea40a0640 0x7f4ea40a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:42.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.136+0000 7f4ebdad8700 1 --2- 172.21.15.2:0/1211737258 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4ea40a0640 0x7f4ea40a2b00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4007fd0 tx=0x7f4eb40073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:42.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.140+0000 7f4eaeffd700 1 -- 172.21.15.2:0/1211737258 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4ea8096d90 con 0x7f4eb8073800 2023-09-13T04:38:42.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.349+0000 7f4ebfd3c700 1 -- 172.21.15.2:0/1211737258 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4eb819a340 con 0x7f4eb8073800 2023-09-13T04:38:42.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.350+0000 7f4eaeffd700 1 -- 172.21.15.2:0/1211737258 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24456 (secure 0 0 0) 0x7f4ea809a3b0 con 0x7f4eb8073800 2023-09-13T04:38:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:38:42.353 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":90,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:38:24.584729+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T04:37:57.094802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T04:37:58.564487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-13T04:38:02.096895+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:38:42.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.353+0000 7f4ebfd3c700 1 -- 172.21.15.2:0/1211737258 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4ea40a0640 msgr2=0x7f4ea40a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:42.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.353+0000 7f4ebfd3c700 1 --2- 172.21.15.2:0/1211737258 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4ea40a0640 0x7f4ea40a2b00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4007fd0 tx=0x7f4eb40073d0 comp rx=0 tx=0).stop 2023-09-13T04:38:42.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.353+0000 7f4ebfd3c700 1 -- 172.21.15.2:0/1211737258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb8073800 msgr2=0x7f4eb8199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:42.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.353+0000 7f4ebfd3c700 1 --2- 172.21.15.2:0/1211737258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb8073800 0x7f4eb8199520 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8000c00 tx=0x7f4ea800f690 comp rx=0 tx=0).stop 2023-09-13T04:38:42.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.353+0000 7f4ebfd3c700 1 -- 172.21.15.2:0/1211737258 shutdown_connections 2023-09-13T04:38:42.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.353+0000 7f4ebfd3c700 1 --2- 172.21.15.2:0/1211737258 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4ea40a0640 0x7f4ea40a2b00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:42.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.353+0000 7f4ebfd3c700 1 --2- 172.21.15.2:0/1211737258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4eb810d730 0x7f4eb819d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:42.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.353+0000 7f4ebfd3c700 1 --2- 172.21.15.2:0/1211737258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4eb8072ee0 0x7f4eb8198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:42.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.353+0000 7f4ebfd3c700 1 --2- 172.21.15.2:0/1211737258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb8073800 0x7f4eb8199520 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:42.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.353+0000 7f4ebfd3c700 1 -- 172.21.15.2:0/1211737258 >> 172.21.15.2:0/1211737258 conn(0x7f4eb8077ea0 msgr2=0x7f4eb80fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:42.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.353+0000 7f4ebfd3c700 1 -- 172.21.15.2:0/1211737258 shutdown_connections 2023-09-13T04:38:42.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:42.353+0000 7f4ebfd3c700 1 -- 172.21.15.2:0/1211737258 wait complete. 2023-09-13T04:38:42.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:42 smithi002 ceph-mon[97651]: pgmap v299: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 10 op/s 2023-09-13T04:38:42.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:42 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1211737258' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:42.866 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-09-13T04:38:42.866 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph osd dump --format=json 2023-09-13T04:38:42.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:42 smithi149 ceph-mon[133315]: pgmap v299: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 10 op/s 2023-09-13T04:38:42.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:42 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1211737258' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:42.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:42 smithi156 ceph-mon[133675]: pgmap v299: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 10 op/s 2023-09-13T04:38:42.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:42 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1211737258' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:43.605 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:38:44.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:44 smithi002 ceph-mon[97651]: pgmap v300: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:38:44.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:44 smithi149 ceph-mon[133315]: pgmap v300: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:38:44.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:44 smithi156 ceph-mon[133675]: pgmap v300: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:38:45.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.260+0000 7feb15fa8700 1 -- 172.21.15.2:0/1995821297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb10106d90 msgr2=0x7feb10107170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:45.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.260+0000 7feb15fa8700 1 --2- 172.21.15.2:0/1995821297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb10106d90 0x7feb10107170 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7feb00009920 tx=0x7feb00009c30 comp rx=0 tx=0).stop 2023-09-13T04:38:45.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.260+0000 7feb15fa8700 1 -- 172.21.15.2:0/1995821297 shutdown_connections 2023-09-13T04:38:45.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.260+0000 7feb15fa8700 1 --2- 172.21.15.2:0/1995821297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb10101740 0x7feb1010dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:45.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.260+0000 7feb15fa8700 1 --2- 172.21.15.2:0/1995821297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb10106d90 0x7feb10107170 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:45.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.260+0000 7feb15fa8700 1 --2- 172.21.15.2:0/1995821297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb10100d80 0x7feb10101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:45.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.260+0000 7feb15fa8700 1 -- 172.21.15.2:0/1995821297 >> 172.21.15.2:0/1995821297 conn(0x7feb10077ea0 msgr2=0x7feb100782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:45.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.260+0000 7feb15fa8700 1 -- 172.21.15.2:0/1995821297 shutdown_connections 2023-09-13T04:38:45.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.260+0000 7feb15fa8700 1 -- 172.21.15.2:0/1995821297 wait complete. 2023-09-13T04:38:45.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.261+0000 7feb15fa8700 1 Processor -- start 2023-09-13T04:38:45.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.261+0000 7feb15fa8700 1 -- start start 2023-09-13T04:38:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.262+0000 7feb15fa8700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb10100d80 0x7feb10196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.262+0000 7feb15fa8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb10101740 0x7feb101973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.262+0000 7feb15fa8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb10106d90 0x7feb1019b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.262+0000 7feb15fa8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1019be50 con 0x7feb10101740 2023-09-13T04:38:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.262+0000 7feb15fa8700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1019bfc0 con 0x7feb10100d80 2023-09-13T04:38:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.262+0000 7feb15fa8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1019c180 con 0x7feb10106d90 2023-09-13T04:38:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.262+0000 7feb0f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb10100d80 0x7feb10196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.262+0000 7feb0effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb10101740 0x7feb101973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.262+0000 7feb0f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb10100d80 0x7feb10196e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:46412/0 (socket says 172.21.15.2:46412) 2023-09-13T04:38:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.262+0000 7feb0f7fe700 1 -- 172.21.15.2:0/3499887610 learned_addr learned my addr 172.21.15.2:0/3499887610 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.262+0000 7feb0ffff700 1 --2- 172.21.15.2:0/3499887610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb10106d90 0x7feb1019b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.262+0000 7feb0f7fe700 1 -- 172.21.15.2:0/3499887610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb10106d90 msgr2=0x7feb1019b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.262+0000 7feb0f7fe700 1 --2- 172.21.15.2:0/3499887610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb10106d90 0x7feb1019b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.262+0000 7feb0f7fe700 1 -- 172.21.15.2:0/3499887610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb10101740 msgr2=0x7feb101973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.262+0000 7feb0f7fe700 1 --2- 172.21.15.2:0/3499887610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb10101740 0x7feb101973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.262+0000 7feb0f7fe700 1 -- 172.21.15.2:0/3499887610 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb00009520 con 0x7feb10100d80 2023-09-13T04:38:45.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.262+0000 7feb0effd700 1 --2- 172.21.15.2:0/3499887610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb10101740 0x7feb101973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:38:45.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.263+0000 7feb0f7fe700 1 --2- 172.21.15.2:0/3499887610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb10100d80 0x7feb10196e90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7feb00009230 tx=0x7feb0000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:45.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.263+0000 7feb0cff9700 1 -- 172.21.15.2:0/3499887610 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0001c070 con 0x7feb10100d80 2023-09-13T04:38:45.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.263+0000 7feb0cff9700 1 -- 172.21.15.2:0/3499887610 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb0000bbd0 con 0x7feb10100d80 2023-09-13T04:38:45.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.263+0000 7feb15fa8700 1 -- 172.21.15.2:0/3499887610 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb1019c400 con 0x7feb10100d80 2023-09-13T04:38:45.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.263+0000 7feb0cff9700 1 -- 172.21.15.2:0/3499887610 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb000039b0 con 0x7feb10100d80 2023-09-13T04:38:45.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.263+0000 7feb15fa8700 1 -- 172.21.15.2:0/3499887610 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb101a3cc0 con 0x7feb10100d80 2023-09-13T04:38:45.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.264+0000 7feb15fa8700 1 -- 172.21.15.2:0/3499887610 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb1004e9b0 con 0x7feb10100d80 2023-09-13T04:38:45.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.266+0000 7feb0cff9700 1 -- 172.21.15.2:0/3499887610 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7feb00020030 con 0x7feb10100d80 2023-09-13T04:38:45.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.268+0000 7feb0cff9700 1 --2- 172.21.15.2:0/3499887610 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7feaf80a06c0 0x7feaf80a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:45.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.268+0000 7feb0cff9700 1 -- 172.21.15.2:0/3499887610 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7feb000ccfe0 con 0x7feb10100d80 2023-09-13T04:38:45.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.268+0000 7feb0effd700 1 --2- 172.21.15.2:0/3499887610 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7feaf80a06c0 0x7feaf80a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:45.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.269+0000 7feb0effd700 1 --2- 172.21.15.2:0/3499887610 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7feaf80a06c0 0x7feaf80a2b80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7feb101984b0 tx=0x7feb04009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:45.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.270+0000 7feb0cff9700 1 -- 172.21.15.2:0/3499887610 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7feb00096850 con 0x7feb10100d80 2023-09-13T04:38:45.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.478+0000 7feb15fa8700 1 -- 172.21.15.2:0/3499887610 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb101981f0 con 0x7feb10100d80 2023-09-13T04:38:45.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.480+0000 7feb0cff9700 1 -- 172.21.15.2:0/3499887610 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24456 (secure 0 0 0) 0x7feb00099e70 con 0x7feb10100d80 2023-09-13T04:38:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:38:45.482 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":90,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:38:24.584729+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T04:37:57.094802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T04:37:58.564487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-13T04:38:02.096895+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:38:45.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.482+0000 7feb15fa8700 1 -- 172.21.15.2:0/3499887610 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7feaf80a06c0 msgr2=0x7feaf80a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:45.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.482+0000 7feb15fa8700 1 --2- 172.21.15.2:0/3499887610 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7feaf80a06c0 0x7feaf80a2b80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7feb101984b0 tx=0x7feb04009310 comp rx=0 tx=0).stop 2023-09-13T04:38:45.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.482+0000 7feb15fa8700 1 -- 172.21.15.2:0/3499887610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb10100d80 msgr2=0x7feb10196e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:45.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.482+0000 7feb15fa8700 1 --2- 172.21.15.2:0/3499887610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb10100d80 0x7feb10196e90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7feb00009230 tx=0x7feb0000fa00 comp rx=0 tx=0).stop 2023-09-13T04:38:45.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.482+0000 7feb15fa8700 1 -- 172.21.15.2:0/3499887610 shutdown_connections 2023-09-13T04:38:45.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.482+0000 7feb15fa8700 1 --2- 172.21.15.2:0/3499887610 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7feaf80a06c0 0x7feaf80a2b80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:45.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.482+0000 7feb15fa8700 1 --2- 172.21.15.2:0/3499887610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb10106d90 0x7feb1019b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:45.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.482+0000 7feb15fa8700 1 --2- 172.21.15.2:0/3499887610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb10100d80 0x7feb10196e90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:45.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.482+0000 7feb15fa8700 1 --2- 172.21.15.2:0/3499887610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb10101740 0x7feb101973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:45.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.482+0000 7feb15fa8700 1 -- 172.21.15.2:0/3499887610 >> 172.21.15.2:0/3499887610 conn(0x7feb10077ea0 msgr2=0x7feb100ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:45.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.483+0000 7feb15fa8700 1 -- 172.21.15.2:0/3499887610 shutdown_connections 2023-09-13T04:38:45.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:45.483+0000 7feb15fa8700 1 -- 172.21.15.2:0/3499887610 wait complete. 2023-09-13T04:38:45.872 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:45 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3499887610' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:45.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:45 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3499887610' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:45.901 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph tell osd.0 flush_pg_stats 2023-09-13T04:38:45.902 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph tell osd.1 flush_pg_stats 2023-09-13T04:38:45.902 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph tell osd.2 flush_pg_stats 2023-09-13T04:38:45.903 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph tell osd.3 flush_pg_stats 2023-09-13T04:38:45.904 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph tell osd.4 flush_pg_stats 2023-09-13T04:38:45.904 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph tell osd.5 flush_pg_stats 2023-09-13T04:38:45.905 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph tell osd.6 flush_pg_stats 2023-09-13T04:38:45.905 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph tell osd.7 flush_pg_stats 2023-09-13T04:38:45.906 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph tell osd.8 flush_pg_stats 2023-09-13T04:38:45.907 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph tell osd.9 flush_pg_stats 2023-09-13T04:38:45.907 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph tell osd.10 flush_pg_stats 2023-09-13T04:38:45.908 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph tell osd.11 flush_pg_stats 2023-09-13T04:38:45.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:45 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3499887610' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:38:46.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:46 smithi149 ceph-mon[133315]: pgmap v301: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:38:46.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:46 smithi156 ceph-mon[133675]: pgmap v301: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:38:47.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:46 smithi002 ceph-mon[97651]: pgmap v301: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:38:47.641 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:38:47.852 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:38:47.856 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:38:47.988 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:38:48.005 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:38:48.095 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:38:48.258 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:38:48.259 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:38:48.312 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:38:48.323 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:38:48.558 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:38:48.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:48 smithi149 ceph-mon[133315]: pgmap v302: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:38:48.905 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:48 smithi002 ceph-mon[97651]: pgmap v302: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:38:48.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:48 smithi156 ceph-mon[133675]: pgmap v302: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:38:50.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:50 smithi002 ceph-mon[97651]: pgmap v303: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:38:50.513 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:38:50.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:50 smithi149 ceph-mon[133315]: pgmap v303: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:38:50.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:50 smithi156 ceph-mon[133675]: pgmap v303: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:38:52.623 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:52 smithi002 ceph-mon[97651]: pgmap v304: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-13T04:38:52.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:52 smithi149 ceph-mon[133315]: pgmap v304: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-13T04:38:52.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:52 smithi156 ceph-mon[133675]: pgmap v304: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-13T04:38:53.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.829+0000 7f50cb539700 1 -- 172.21.15.2:0/115024804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50c4106d90 msgr2=0x7f50c4107170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:53.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.829+0000 7f50cb539700 1 --2- 172.21.15.2:0/115024804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50c4106d90 0x7f50c4107170 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f50c00099c0 tx=0x7f50c0009cd0 comp rx=0 tx=0).stop 2023-09-13T04:38:53.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.829+0000 7f50cb539700 1 -- 172.21.15.2:0/115024804 shutdown_connections 2023-09-13T04:38:53.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.829+0000 7f50cb539700 1 --2- 172.21.15.2:0/115024804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50c4101740 0x7f50c410dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:53.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.829+0000 7f50cb539700 1 --2- 172.21.15.2:0/115024804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50c4106d90 0x7f50c4107170 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:53.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.829+0000 7f50cb539700 1 --2- 172.21.15.2:0/115024804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50c4100d80 0x7f50c4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:53.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.829+0000 7f50cb539700 1 -- 172.21.15.2:0/115024804 >> 172.21.15.2:0/115024804 conn(0x7f50c4077ea0 msgr2=0x7f50c40782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:53.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.829+0000 7f50cb539700 1 -- 172.21.15.2:0/115024804 shutdown_connections 2023-09-13T04:38:53.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.829+0000 7f50cb539700 1 -- 172.21.15.2:0/115024804 wait complete. 2023-09-13T04:38:53.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.830+0000 7f50cb539700 1 Processor -- start 2023-09-13T04:38:53.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.830+0000 7f50cb539700 1 -- start start 2023-09-13T04:38:53.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.830+0000 7f50cb539700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50c4100d80 0x7f50c4198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:53.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.831+0000 7f50cb539700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50c4101740 0x7f50c4199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:53.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.831+0000 7f50cb539700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50c4106d90 0x7f50c419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:53.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.831+0000 7f50cb539700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c419dfa0 con 0x7f50c4101740 2023-09-13T04:38:53.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.831+0000 7f50cb539700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c419e110 con 0x7f50c4106d90 2023-09-13T04:38:53.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.831+0000 7f50cb539700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c419e2d0 con 0x7f50c4100d80 2023-09-13T04:38:53.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.831+0000 7f50c92d5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50c4100d80 0x7f50c4198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:53.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.831+0000 7f50c8ad4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50c4101740 0x7f50c4199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:53.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.831+0000 7f50c92d5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50c4100d80 0x7f50c4198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:59388/0 (socket says 172.21.15.2:59388) 2023-09-13T04:38:53.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.831+0000 7f50c92d5700 1 -- 172.21.15.2:0/2858805878 learned_addr learned my addr 172.21.15.2:0/2858805878 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:53.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.831+0000 7f50c9ad6700 1 --2- 172.21.15.2:0/2858805878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50c4106d90 0x7f50c419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:53.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.831+0000 7f50c8ad4700 1 -- 172.21.15.2:0/2858805878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50c4100d80 msgr2=0x7f50c4198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:53.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.831+0000 7f50c8ad4700 1 --2- 172.21.15.2:0/2858805878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50c4100d80 0x7f50c4198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:53.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.831+0000 7f50c8ad4700 1 -- 172.21.15.2:0/2858805878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50c4106d90 msgr2=0x7f50c419d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:53.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.831+0000 7f50c8ad4700 1 --2- 172.21.15.2:0/2858805878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50c4106d90 0x7f50c419d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:53.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.831+0000 7f50c8ad4700 1 -- 172.21.15.2:0/2858805878 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c00095f0 con 0x7f50c4101740 2023-09-13T04:38:53.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.831+0000 7f50c92d5700 1 --2- 172.21.15.2:0/2858805878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50c4100d80 0x7f50c4198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:38:53.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.832+0000 7f50c8ad4700 1 --2- 172.21.15.2:0/2858805878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50c4101740 0x7f50c4199520 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f50b400e9f0 tx=0x7f50b400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:53.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.832+0000 7f50ba7fc700 1 -- 172.21.15.2:0/2858805878 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b400eee0 con 0x7f50c4101740 2023-09-13T04:38:53.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.832+0000 7f50ba7fc700 1 -- 172.21.15.2:0/2858805878 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50b400cbd0 con 0x7f50c4101740 2023-09-13T04:38:53.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.832+0000 7f50cb539700 1 -- 172.21.15.2:0/2858805878 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c419e5b0 con 0x7f50c4101740 2023-09-13T04:38:53.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.832+0000 7f50ba7fc700 1 -- 172.21.15.2:0/2858805878 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b4018680 con 0x7f50c4101740 2023-09-13T04:38:53.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.832+0000 7f50cb539700 1 -- 172.21.15.2:0/2858805878 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c41a5e10 con 0x7f50c4101740 2023-09-13T04:38:53.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.833+0000 7f50cb539700 1 -- 172.21.15.2:0/2858805878 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f50c4110f60 con 0x7f50c4101740 2023-09-13T04:38:53.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.835+0000 7f50ba7fc700 1 -- 172.21.15.2:0/2858805878 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f50b400cd40 con 0x7f50c4101740 2023-09-13T04:38:53.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.837+0000 7f50ba7fc700 1 --2- 172.21.15.2:0/2858805878 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f50b00a0640 0x7f50b00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:53.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.837+0000 7f50ba7fc700 1 -- 172.21.15.2:0/2858805878 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f50b4014070 con 0x7f50c4101740 2023-09-13T04:38:53.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.837+0000 7f50c92d5700 1 --2- 172.21.15.2:0/2858805878 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f50b00a0640 0x7f50b00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:53.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.837+0000 7f50ba7fc700 1 --2- 172.21.15.2:0/2858805878 >> [v2:172.21.15.2:6802/1800239506,v1:172.21.15.2:6803/1800239506] conn(0x7f50b00ab660 0x7f50b00adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:53.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.837+0000 7f50ba7fc700 1 -- 172.21.15.2:0/2858805878 --> [v2:172.21.15.2:6802/1800239506,v1:172.21.15.2:6803/1800239506] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f50b00ae1d0 con 0x7f50b00ab660 2023-09-13T04:38:53.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.837+0000 7f50ba7fc700 1 -- 172.21.15.2:0/2858805878 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50b40cc820 con 0x7f50c4101740 2023-09-13T04:38:53.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.837+0000 7f50c9ad6700 1 --2- 172.21.15.2:0/2858805878 >> [v2:172.21.15.2:6802/1800239506,v1:172.21.15.2:6803/1800239506] conn(0x7f50b00ab660 0x7f50b00adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:53.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.838+0000 7f50c92d5700 1 --2- 172.21.15.2:0/2858805878 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f50b00a0640 0x7f50b00a2b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f50c0000c00 tx=0x7f50c0005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:53.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.838+0000 7f50c9ad6700 1 --2- 172.21.15.2:0/2858805878 >> [v2:172.21.15.2:6802/1800239506,v1:172.21.15.2:6803/1800239506] conn(0x7f50b00ab660 0x7f50b00adb00 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:53.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.839+0000 7f50ba7fc700 1 -- 172.21.15.2:0/2858805878 <== osd.0 v2:172.21.15.2:6802/1800239506 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f50b00ae1d0 con 0x7f50b00ab660 2023-09-13T04:38:53.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.856+0000 7f50cb539700 1 -- 172.21.15.2:0/2858805878 --> [v2:172.21.15.2:6802/1800239506,v1:172.21.15.2:6803/1800239506] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f50c404e9b0 con 0x7f50b00ab660 2023-09-13T04:38:53.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.857+0000 7f50ba7fc700 1 -- 172.21.15.2:0/2858805878 <== osd.0 v2:172.21.15.2:6802/1800239506 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f50c404e9b0 con 0x7f50b00ab660 2023-09-13T04:38:53.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.857+0000 7f50cb539700 1 -- 172.21.15.2:0/2858805878 >> [v2:172.21.15.2:6802/1800239506,v1:172.21.15.2:6803/1800239506] conn(0x7f50b00ab660 msgr2=0x7f50b00adb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:53.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.857+0000 7f50cb539700 1 --2- 172.21.15.2:0/2858805878 >> [v2:172.21.15.2:6802/1800239506,v1:172.21.15.2:6803/1800239506] conn(0x7f50b00ab660 0x7f50b00adb00 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:53.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.857+0000 7f50cb539700 1 -- 172.21.15.2:0/2858805878 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f50b00a0640 msgr2=0x7f50b00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:53.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.857+0000 7f50cb539700 1 --2- 172.21.15.2:0/2858805878 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f50b00a0640 0x7f50b00a2b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f50c0000c00 tx=0x7f50c0005fd0 comp rx=0 tx=0).stop 2023-09-13T04:38:53.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.857+0000 7f50cb539700 1 -- 172.21.15.2:0/2858805878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50c4101740 msgr2=0x7f50c4199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:53.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.857+0000 7f50cb539700 1 --2- 172.21.15.2:0/2858805878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50c4101740 0x7f50c4199520 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f50b400e9f0 tx=0x7f50b400ed00 comp rx=0 tx=0).stop 2023-09-13T04:38:53.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.857+0000 7f50c9ad6700 1 -- 172.21.15.2:0/2858805878 reap_dead start 2023-09-13T04:38:53.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.857+0000 7f50cb539700 1 -- 172.21.15.2:0/2858805878 shutdown_connections 2023-09-13T04:38:53.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.857+0000 7f50cb539700 1 -- 172.21.15.2:0/2858805878 >> 172.21.15.2:0/2858805878 conn(0x7f50c4077ea0 msgr2=0x7f50c4073030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:53.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.857+0000 7f50cb539700 1 -- 172.21.15.2:0/2858805878 shutdown_connections 2023-09-13T04:38:53.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:52.857+0000 7f50cb539700 1 -- 172.21.15.2:0/2858805878 wait complete. 2023-09-13T04:38:54.578 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:54 smithi002 ceph-mon[97651]: pgmap v305: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:38:54.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:54 smithi149 ceph-mon[133315]: pgmap v305: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:38:54.917 INFO:teuthology.orchestra.run.smithi002.stdout:34359738441 2023-09-13T04:38:54.918 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph osd last-stat-seq osd.0 2023-09-13T04:38:54.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:54 smithi156 ceph-mon[133675]: pgmap v305: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:38:55.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.997+0000 7f1d8edbf700 1 -- 172.21.15.2:0/2777545107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d881060b0 msgr2=0x7f1d8810dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:55.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.997+0000 7f1d8edbf700 1 --2- 172.21.15.2:0/2777545107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d881060b0 0x7f1d8810dc30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000b0a0 tx=0x7f1d8000b3b0 comp rx=0 tx=0).stop 2023-09-13T04:38:55.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.997+0000 7f1d8edbf700 1 -- 172.21.15.2:0/2777545107 shutdown_connections 2023-09-13T04:38:55.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.997+0000 7f1d8edbf700 1 --2- 172.21.15.2:0/2777545107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d881060b0 0x7f1d8810dc30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:55.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.997+0000 7f1d8edbf700 1 --2- 172.21.15.2:0/2777545107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d88068a00 0x7f1d88068de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:55.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.997+0000 7f1d8edbf700 1 --2- 172.21.15.2:0/2777545107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d880693b0 0x7f1d88105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:55.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.997+0000 7f1d8edbf700 1 -- 172.21.15.2:0/2777545107 >> 172.21.15.2:0/2777545107 conn(0x7f1d88075c00 msgr2=0x7f1d88076010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:56.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.998+0000 7f1d8edbf700 1 -- 172.21.15.2:0/2777545107 shutdown_connections 2023-09-13T04:38:56.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.998+0000 7f1d8edbf700 1 -- 172.21.15.2:0/2777545107 wait complete. 2023-09-13T04:38:56.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.999+0000 7f1d8edbf700 1 Processor -- start 2023-09-13T04:38:56.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.999+0000 7f1d8edbf700 1 -- start start 2023-09-13T04:38:56.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.000+0000 7f1d8edbf700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d88068a00 0x7f1d88196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:56.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.000+0000 7f1d8edbf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d880693b0 0x7f1d88197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:56.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.000+0000 7f1d8edbf700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d881060b0 0x7f1d8819b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:56.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.000+0000 7f1d8edbf700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8819bfa0 con 0x7f1d880693b0 2023-09-13T04:38:56.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.000+0000 7f1d8edbf700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8819c110 con 0x7f1d88068a00 2023-09-13T04:38:56.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.000+0000 7f1d8edbf700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8819c2d0 con 0x7f1d881060b0 2023-09-13T04:38:56.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.001+0000 7f1d87fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d880693b0 0x7f1d88197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:56.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.001+0000 7f1d8cb5b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d88068a00 0x7f1d88196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:56.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.001+0000 7f1d87fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d880693b0 0x7f1d88197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35712/0 (socket says 172.21.15.2:35712) 2023-09-13T04:38:56.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.001+0000 7f1d87fff700 1 -- 172.21.15.2:0/326188188 learned_addr learned my addr 172.21.15.2:0/326188188 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:56.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.001+0000 7f1d8d35c700 1 --2- 172.21.15.2:0/326188188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d881060b0 0x7f1d8819b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:56.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.001+0000 7f1d8cb5b700 1 -- 172.21.15.2:0/326188188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d881060b0 msgr2=0x7f1d8819b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:56.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.001+0000 7f1d8cb5b700 1 --2- 172.21.15.2:0/326188188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d881060b0 0x7f1d8819b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:56.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.001+0000 7f1d8cb5b700 1 -- 172.21.15.2:0/326188188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d880693b0 msgr2=0x7f1d88197520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:56.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.001+0000 7f1d8cb5b700 1 --2- 172.21.15.2:0/326188188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d880693b0 0x7f1d88197520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:56.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.001+0000 7f1d8cb5b700 1 -- 172.21.15.2:0/326188188 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d80009d00 con 0x7f1d88068a00 2023-09-13T04:38:56.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.002+0000 7f1d8cb5b700 1 --2- 172.21.15.2:0/326188188 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d88068a00 0x7f1d88196fe0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c00d950 tx=0x7f1d7c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:56.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.002+0000 7f1d85ffb700 1 -- 172.21.15.2:0/326188188 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c00f890 con 0x7f1d88068a00 2023-09-13T04:38:56.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.002+0000 7f1d8edbf700 1 -- 172.21.15.2:0/326188188 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d8819c5b0 con 0x7f1d88068a00 2023-09-13T04:38:56.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.002+0000 7f1d85ffb700 1 -- 172.21.15.2:0/326188188 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d7c010460 con 0x7f1d88068a00 2023-09-13T04:38:56.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.002+0000 7f1d85ffb700 1 -- 172.21.15.2:0/326188188 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c00e610 con 0x7f1d88068a00 2023-09-13T04:38:56.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.002+0000 7f1d8edbf700 1 -- 172.21.15.2:0/326188188 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d881a3e10 con 0x7f1d88068a00 2023-09-13T04:38:56.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.003+0000 7f1d8edbf700 1 -- 172.21.15.2:0/326188188 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1d88110f20 con 0x7f1d88068a00 2023-09-13T04:38:56.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.005+0000 7f1d85ffb700 1 -- 172.21.15.2:0/326188188 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f1d7c0105d0 con 0x7f1d88068a00 2023-09-13T04:38:56.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.008+0000 7f1d85ffb700 1 --2- 172.21.15.2:0/326188188 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1d780a06c0 0x7f1d780a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:56.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.008+0000 7f1d85ffb700 1 -- 172.21.15.2:0/326188188 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1d7c0cbf30 con 0x7f1d88068a00 2023-09-13T04:38:56.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.008+0000 7f1d87fff700 1 --2- 172.21.15.2:0/326188188 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1d780a06c0 0x7f1d780a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:56.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.008+0000 7f1d85ffb700 1 --2- 172.21.15.2:0/326188188 >> [v2:172.21.15.149:6824/3663849505,v1:172.21.15.149:6825/3663849505] conn(0x7f1d780ab6e0 0x7f1d780adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:56.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.008+0000 7f1d85ffb700 1 -- 172.21.15.2:0/326188188 --> [v2:172.21.15.149:6824/3663849505,v1:172.21.15.149:6825/3663849505] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d780ae250 con 0x7f1d780ab6e0 2023-09-13T04:38:56.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.008+0000 7f1d85ffb700 1 -- 172.21.15.2:0/326188188 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d7c0cc2a0 con 0x7f1d88068a00 2023-09-13T04:38:56.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.009+0000 7f1d8d35c700 1 --2- 172.21.15.2:0/326188188 >> [v2:172.21.15.149:6824/3663849505,v1:172.21.15.149:6825/3663849505] conn(0x7f1d780ab6e0 0x7f1d780adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:56.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.009+0000 7f1d87fff700 1 --2- 172.21.15.2:0/326188188 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1d780a06c0 0x7f1d780a2b80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1d88198600 tx=0x7f1d7400b460 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:56.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.010+0000 7f1d8d35c700 1 --2- 172.21.15.2:0/326188188 >> [v2:172.21.15.149:6824/3663849505,v1:172.21.15.149:6825/3663849505] conn(0x7f1d780ab6e0 0x7f1d780adb80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:56.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.010+0000 7f1d85ffb700 1 -- 172.21.15.2:0/326188188 <== osd.10 v2:172.21.15.149:6824/3663849505 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f1d780ae250 con 0x7f1d780ab6e0 2023-09-13T04:38:56.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.028+0000 7f1d8edbf700 1 -- 172.21.15.2:0/326188188 --> [v2:172.21.15.149:6824/3663849505,v1:172.21.15.149:6825/3663849505] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1d8804e9b0 con 0x7f1d780ab6e0 2023-09-13T04:38:56.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.028+0000 7f1d85ffb700 1 -- 172.21.15.2:0/326188188 <== osd.10 v2:172.21.15.149:6824/3663849505 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1d8804e9b0 con 0x7f1d780ab6e0 2023-09-13T04:38:56.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.029+0000 7f1d8edbf700 1 -- 172.21.15.2:0/326188188 >> [v2:172.21.15.149:6824/3663849505,v1:172.21.15.149:6825/3663849505] conn(0x7f1d780ab6e0 msgr2=0x7f1d780adb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:56.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.029+0000 7f1d8edbf700 1 --2- 172.21.15.2:0/326188188 >> [v2:172.21.15.149:6824/3663849505,v1:172.21.15.149:6825/3663849505] conn(0x7f1d780ab6e0 0x7f1d780adb80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:56.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.029+0000 7f1d8edbf700 1 -- 172.21.15.2:0/326188188 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1d780a06c0 msgr2=0x7f1d780a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:56.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.029+0000 7f1d8edbf700 1 --2- 172.21.15.2:0/326188188 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1d780a06c0 0x7f1d780a2b80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1d88198600 tx=0x7f1d7400b460 comp rx=0 tx=0).stop 2023-09-13T04:38:56.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.029+0000 7f1d8edbf700 1 -- 172.21.15.2:0/326188188 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d88068a00 msgr2=0x7f1d88196fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:56.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.029+0000 7f1d8edbf700 1 --2- 172.21.15.2:0/326188188 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d88068a00 0x7f1d88196fe0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c00d950 tx=0x7f1d7c00dc60 comp rx=0 tx=0).stop 2023-09-13T04:38:56.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.029+0000 7f1d8d35c700 1 -- 172.21.15.2:0/326188188 reap_dead start 2023-09-13T04:38:56.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.029+0000 7f1d8edbf700 1 -- 172.21.15.2:0/326188188 shutdown_connections 2023-09-13T04:38:56.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.029+0000 7f1d8edbf700 1 -- 172.21.15.2:0/326188188 >> 172.21.15.2:0/326188188 conn(0x7f1d88075c00 msgr2=0x7f1d880fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:56.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.029+0000 7f1d8edbf700 1 -- 172.21.15.2:0/326188188 shutdown_connections 2023-09-13T04:38:56.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.029+0000 7f1d8edbf700 1 -- 172.21.15.2:0/326188188 wait complete. 2023-09-13T04:38:56.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.490+0000 7fb5dcf7f700 1 -- 172.21.15.2:0/2074066432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5d8100620 msgr2=0x7fb5d810ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:56.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.490+0000 7fb5dcf7f700 1 --2- 172.21.15.2:0/2074066432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5d8100620 0x7fb5d810ca90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000b0d0 tx=0x7fb5d000b3e0 comp rx=0 tx=0).stop 2023-09-13T04:38:56.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.490+0000 7fb5dcf7f700 1 -- 172.21.15.2:0/2074066432 shutdown_connections 2023-09-13T04:38:56.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.490+0000 7fb5dcf7f700 1 --2- 172.21.15.2:0/2074066432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5d8100620 0x7fb5d810ca90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:56.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.490+0000 7fb5dcf7f700 1 --2- 172.21.15.2:0/2074066432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5d8105c70 0x7fb5d8106050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:56.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.490+0000 7fb5dcf7f700 1 --2- 172.21.15.2:0/2074066432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5d80ffc60 0x7fb5d81000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:56.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.490+0000 7fb5dcf7f700 1 -- 172.21.15.2:0/2074066432 >> 172.21.15.2:0/2074066432 conn(0x7fb5d8075c00 msgr2=0x7fb5d8076010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:56.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.490+0000 7fb5dcf7f700 1 -- 172.21.15.2:0/2074066432 shutdown_connections 2023-09-13T04:38:56.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.490+0000 7fb5dcf7f700 1 -- 172.21.15.2:0/2074066432 wait complete. 2023-09-13T04:38:56.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.491+0000 7fb5dcf7f700 1 Processor -- start 2023-09-13T04:38:56.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.491+0000 7fb5dcf7f700 1 -- start start 2023-09-13T04:38:56.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.492+0000 7fb5dcf7f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5d80ffc60 0x7fb5d8199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:56.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.492+0000 7fb5dcf7f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5d8100620 0x7fb5d8199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:56.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.492+0000 7fb5dcf7f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5d8105c70 0x7fb5d819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:56.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.492+0000 7fb5dcf7f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d819e0c0 con 0x7fb5d80ffc60 2023-09-13T04:38:56.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.492+0000 7fb5dcf7f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d819e230 con 0x7fb5d8105c70 2023-09-13T04:38:56.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.492+0000 7fb5dcf7f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d819e3f0 con 0x7fb5d8100620 2023-09-13T04:38:56.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.492+0000 7fb5d659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5d80ffc60 0x7fb5d8199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:56.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.492+0000 7fb5d659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5d80ffc60 0x7fb5d8199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35732/0 (socket says 172.21.15.2:35732) 2023-09-13T04:38:56.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.492+0000 7fb5d659c700 1 -- 172.21.15.2:0/258600539 learned_addr learned my addr 172.21.15.2:0/258600539 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:56.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.492+0000 7fb5d5d9b700 1 --2- 172.21.15.2:0/258600539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5d8100620 0x7fb5d8199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:56.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.492+0000 7fb5d6d9d700 1 --2- 172.21.15.2:0/258600539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5d8105c70 0x7fb5d819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:56.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.492+0000 7fb5d659c700 1 -- 172.21.15.2:0/258600539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5d8100620 msgr2=0x7fb5d8199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:56.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.492+0000 7fb5d659c700 1 --2- 172.21.15.2:0/258600539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5d8100620 0x7fb5d8199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:56.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.492+0000 7fb5d659c700 1 -- 172.21.15.2:0/258600539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5d8105c70 msgr2=0x7fb5d819dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:56.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.492+0000 7fb5d659c700 1 --2- 172.21.15.2:0/258600539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5d8105c70 0x7fb5d819dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:56.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.492+0000 7fb5d659c700 1 -- 172.21.15.2:0/258600539 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d0009d00 con 0x7fb5d80ffc60 2023-09-13T04:38:56.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.492+0000 7fb5d6d9d700 1 --2- 172.21.15.2:0/258600539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5d8105c70 0x7fb5d819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:38:56.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.493+0000 7fb5d659c700 1 --2- 172.21.15.2:0/258600539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5d80ffc60 0x7fb5d8199100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800e9a0 tx=0x7fb5c800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:56.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.493+0000 7fb5c77fe700 1 -- 172.21.15.2:0/258600539 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8009790 con 0x7fb5d80ffc60 2023-09-13T04:38:56.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.493+0000 7fb5c77fe700 1 -- 172.21.15.2:0/258600539 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5c8010430 con 0x7fb5d80ffc60 2023-09-13T04:38:56.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.493+0000 7fb5dcf7f700 1 -- 172.21.15.2:0/258600539 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d819e6d0 con 0x7fb5d80ffc60 2023-09-13T04:38:56.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.493+0000 7fb5c77fe700 1 -- 172.21.15.2:0/258600539 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8018680 con 0x7fb5d80ffc60 2023-09-13T04:38:56.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.493+0000 7fb5dcf7f700 1 -- 172.21.15.2:0/258600539 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d81a5f30 con 0x7fb5d80ffc60 2023-09-13T04:38:56.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.494+0000 7fb5dcf7f700 1 -- 172.21.15.2:0/258600539 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb5d810fe40 con 0x7fb5d80ffc60 2023-09-13T04:38:56.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.496+0000 7fb5c77fe700 1 -- 172.21.15.2:0/258600539 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fb5c80105a0 con 0x7fb5d80ffc60 2023-09-13T04:38:56.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.498+0000 7fb5c77fe700 1 --2- 172.21.15.2:0/258600539 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb5c00a0640 0x7fb5c00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:56.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.498+0000 7fb5c77fe700 1 -- 172.21.15.2:0/258600539 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb5c8014070 con 0x7fb5d80ffc60 2023-09-13T04:38:56.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.498+0000 7fb5d5d9b700 1 --2- 172.21.15.2:0/258600539 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb5c00a0640 0x7fb5c00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:56.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.498+0000 7fb5c77fe700 1 --2- 172.21.15.2:0/258600539 >> [v2:172.21.15.156:6816/1139967383,v1:172.21.15.156:6817/1139967383] conn(0x7fb5c00ab660 0x7fb5c00adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:56.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.498+0000 7fb5c77fe700 1 -- 172.21.15.2:0/258600539 --> [v2:172.21.15.156:6816/1139967383,v1:172.21.15.156:6817/1139967383] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb5c00ae1d0 con 0x7fb5c00ab660 2023-09-13T04:38:56.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.498+0000 7fb5c77fe700 1 -- 172.21.15.2:0/258600539 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5c80cc670 con 0x7fb5d80ffc60 2023-09-13T04:38:56.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.499+0000 7fb5d5d9b700 1 --2- 172.21.15.2:0/258600539 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb5c00a0640 0x7fb5c00a2b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb5d819a720 tx=0x7fb5cc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:56.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.499+0000 7fb5d6d9d700 1 --2- 172.21.15.2:0/258600539 >> [v2:172.21.15.156:6816/1139967383,v1:172.21.15.156:6817/1139967383] conn(0x7fb5c00ab660 0x7fb5c00adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:56.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.499+0000 7fb5d6d9d700 1 --2- 172.21.15.2:0/258600539 >> [v2:172.21.15.156:6816/1139967383,v1:172.21.15.156:6817/1139967383] conn(0x7fb5c00ab660 0x7fb5c00adb00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:56.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.500+0000 7fb5c77fe700 1 -- 172.21.15.2:0/258600539 <== osd.8 v2:172.21.15.156:6816/1139967383 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fb5c00ae1d0 con 0x7fb5c00ab660 2023-09-13T04:38:56.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.517+0000 7fb5dcf7f700 1 -- 172.21.15.2:0/258600539 --> [v2:172.21.15.156:6816/1139967383,v1:172.21.15.156:6817/1139967383] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb5d804e9b0 con 0x7fb5c00ab660 2023-09-13T04:38:56.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.518+0000 7fb5c77fe700 1 -- 172.21.15.2:0/258600539 <== osd.8 v2:172.21.15.156:6816/1139967383 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb5d804e9b0 con 0x7fb5c00ab660 2023-09-13T04:38:56.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.518+0000 7fb5dcf7f700 1 -- 172.21.15.2:0/258600539 >> [v2:172.21.15.156:6816/1139967383,v1:172.21.15.156:6817/1139967383] conn(0x7fb5c00ab660 msgr2=0x7fb5c00adb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:56.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.518+0000 7fb5dcf7f700 1 --2- 172.21.15.2:0/258600539 >> [v2:172.21.15.156:6816/1139967383,v1:172.21.15.156:6817/1139967383] conn(0x7fb5c00ab660 0x7fb5c00adb00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:56.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.518+0000 7fb5dcf7f700 1 -- 172.21.15.2:0/258600539 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb5c00a0640 msgr2=0x7fb5c00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:56.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.518+0000 7fb5dcf7f700 1 --2- 172.21.15.2:0/258600539 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb5c00a0640 0x7fb5c00a2b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb5d819a720 tx=0x7fb5cc009260 comp rx=0 tx=0).stop 2023-09-13T04:38:56.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.518+0000 7fb5dcf7f700 1 -- 172.21.15.2:0/258600539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5d80ffc60 msgr2=0x7fb5d8199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:56.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.518+0000 7fb5dcf7f700 1 --2- 172.21.15.2:0/258600539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5d80ffc60 0x7fb5d8199100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800e9a0 tx=0x7fb5c800ecb0 comp rx=0 tx=0).stop 2023-09-13T04:38:56.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.518+0000 7fb5d6d9d700 1 -- 172.21.15.2:0/258600539 reap_dead start 2023-09-13T04:38:56.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.518+0000 7fb5dcf7f700 1 -- 172.21.15.2:0/258600539 shutdown_connections 2023-09-13T04:38:56.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.518+0000 7fb5dcf7f700 1 -- 172.21.15.2:0/258600539 >> 172.21.15.2:0/258600539 conn(0x7fb5d8075c00 msgr2=0x7fb5d80687a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:56.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.519+0000 7fb5dcf7f700 1 -- 172.21.15.2:0/258600539 shutdown_connections 2023-09-13T04:38:56.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:56.519+0000 7fb5dcf7f700 1 -- 172.21.15.2:0/258600539 wait complete. 2023-09-13T04:38:56.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:56 smithi002 ceph-mon[97651]: pgmap v306: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:38:56.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:56 smithi149 ceph-mon[133315]: pgmap v306: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:38:56.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:56 smithi156 ceph-mon[133675]: pgmap v306: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:38:57.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.939+0000 7fd2d6ef7700 1 -- 172.21.15.2:0/3395342481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2d0073800 msgr2=0x7fd2d010d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:57.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.939+0000 7fd2d6ef7700 1 --2- 172.21.15.2:0/3395342481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2d0073800 0x7fd2d010d1f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009990 tx=0x7fd2bc009ca0 comp rx=0 tx=0).stop 2023-09-13T04:38:57.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.939+0000 7fd2d6ef7700 1 -- 172.21.15.2:0/3395342481 shutdown_connections 2023-09-13T04:38:57.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.939+0000 7fd2d6ef7700 1 --2- 172.21.15.2:0/3395342481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2d010d730 0x7fd2d010fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:57.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.939+0000 7fd2d6ef7700 1 --2- 172.21.15.2:0/3395342481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2d0072ee0 0x7fd2d00732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:57.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.939+0000 7fd2d6ef7700 1 --2- 172.21.15.2:0/3395342481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2d0073800 0x7fd2d010d1f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:57.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.939+0000 7fd2d6ef7700 1 -- 172.21.15.2:0/3395342481 >> 172.21.15.2:0/3395342481 conn(0x7fd2d0077ea0 msgr2=0x7fd2d00782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:57.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.939+0000 7fd2d6ef7700 1 -- 172.21.15.2:0/3395342481 shutdown_connections 2023-09-13T04:38:57.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.939+0000 7fd2d6ef7700 1 -- 172.21.15.2:0/3395342481 wait complete. 2023-09-13T04:38:57.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.941+0000 7fd2d6ef7700 1 Processor -- start 2023-09-13T04:38:57.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.941+0000 7fd2d6ef7700 1 -- start start 2023-09-13T04:38:57.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.941+0000 7fd2d6ef7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2d0072ee0 0x7fd2d01990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:57.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.941+0000 7fd2d6ef7700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2d0073800 0x7fd2d0199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:57.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.941+0000 7fd2d6ef7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2d010d730 0x7fd2d019da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:57.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.942+0000 7fd2d6ef7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d019e090 con 0x7fd2d0072ee0 2023-09-13T04:38:57.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.942+0000 7fd2d6ef7700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d019e200 con 0x7fd2d0073800 2023-09-13T04:38:57.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.942+0000 7fd2d6ef7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d019e3c0 con 0x7fd2d010d730 2023-09-13T04:38:57.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.942+0000 7fd2d4c93700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2d0072ee0 0x7fd2d01990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:57.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.942+0000 7fd2d4c93700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2d0072ee0 0x7fd2d01990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35690/0 (socket says 172.21.15.2:35690) 2023-09-13T04:38:57.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.942+0000 7fd2d4c93700 1 -- 172.21.15.2:0/587650114 learned_addr learned my addr 172.21.15.2:0/587650114 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:57.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.942+0000 7fd2d5494700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2d010d730 0x7fd2d019da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:57.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.942+0000 7fd2cffff700 1 --2- 172.21.15.2:0/587650114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2d0073800 0x7fd2d0199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:57.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.943+0000 7fd2d4c93700 1 -- 172.21.15.2:0/587650114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2d010d730 msgr2=0x7fd2d019da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:57.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.943+0000 7fd2d4c93700 1 --2- 172.21.15.2:0/587650114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2d010d730 0x7fd2d019da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:57.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.943+0000 7fd2d4c93700 1 -- 172.21.15.2:0/587650114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2d0073800 msgr2=0x7fd2d0199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:57.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.943+0000 7fd2d4c93700 1 --2- 172.21.15.2:0/587650114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2d0073800 0x7fd2d0199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:57.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.943+0000 7fd2d4c93700 1 -- 172.21.15.2:0/587650114 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2bc0095f0 con 0x7fd2d0072ee0 2023-09-13T04:38:57.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.943+0000 7fd2d5494700 1 --2- 172.21.15.2:0/587650114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2d010d730 0x7fd2d019da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:38:57.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.943+0000 7fd2cffff700 1 --2- 172.21.15.2:0/587650114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2d0073800 0x7fd2d0199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:38:57.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.943+0000 7fd2d4c93700 1 --2- 172.21.15.2:0/587650114 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2d0072ee0 0x7fd2d01990d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400e9a0 tx=0x7fd2c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:57.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.943+0000 7fd2cdffb700 1 -- 172.21.15.2:0/587650114 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4009790 con 0x7fd2d0072ee0 2023-09-13T04:38:57.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.943+0000 7fd2d6ef7700 1 -- 172.21.15.2:0/587650114 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d019e6a0 con 0x7fd2d0072ee0 2023-09-13T04:38:57.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.943+0000 7fd2cdffb700 1 -- 172.21.15.2:0/587650114 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2c4010430 con 0x7fd2d0072ee0 2023-09-13T04:38:57.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.943+0000 7fd2cdffb700 1 -- 172.21.15.2:0/587650114 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4018680 con 0x7fd2d0072ee0 2023-09-13T04:38:57.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.944+0000 7fd2d6ef7700 1 -- 172.21.15.2:0/587650114 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d01a5f00 con 0x7fd2d0072ee0 2023-09-13T04:38:57.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.945+0000 7fd2d6ef7700 1 -- 172.21.15.2:0/587650114 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd2d0113130 con 0x7fd2d0072ee0 2023-09-13T04:38:57.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.949+0000 7fd2cdffb700 1 -- 172.21.15.2:0/587650114 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fd2c40105a0 con 0x7fd2d0072ee0 2023-09-13T04:38:57.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.951+0000 7fd2cdffb700 1 --2- 172.21.15.2:0/587650114 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd2c00a0640 0x7fd2c00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:57.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.952+0000 7fd2cdffb700 1 -- 172.21.15.2:0/587650114 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd2c4014070 con 0x7fd2d0072ee0 2023-09-13T04:38:57.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.952+0000 7fd2cdffb700 1 --2- 172.21.15.2:0/587650114 >> [v2:172.21.15.149:6800/1737673954,v1:172.21.15.149:6801/1737673954] conn(0x7fd2c00ab660 0x7fd2c00adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:57.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.952+0000 7fd2cdffb700 1 -- 172.21.15.2:0/587650114 --> [v2:172.21.15.149:6800/1737673954,v1:172.21.15.149:6801/1737673954] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd2c00ae1d0 con 0x7fd2c00ab660 2023-09-13T04:38:57.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.952+0000 7fd2cdffb700 1 -- 172.21.15.2:0/587650114 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2c40d1500 con 0x7fd2d0072ee0 2023-09-13T04:38:57.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.952+0000 7fd2cffff700 1 --2- 172.21.15.2:0/587650114 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd2c00a0640 0x7fd2c00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:57.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.953+0000 7fd2d5494700 1 --2- 172.21.15.2:0/587650114 >> [v2:172.21.15.149:6800/1737673954,v1:172.21.15.149:6801/1737673954] conn(0x7fd2c00ab660 0x7fd2c00adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:57.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.953+0000 7fd2cffff700 1 --2- 172.21.15.2:0/587650114 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd2c00a0640 0x7fd2c00a2b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00b2e0 tx=0x7fd2bc0058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:57.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.953+0000 7fd2d5494700 1 --2- 172.21.15.2:0/587650114 >> [v2:172.21.15.149:6800/1737673954,v1:172.21.15.149:6801/1737673954] conn(0x7fd2c00ab660 0x7fd2c00adb00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:57.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.954+0000 7fd2cdffb700 1 -- 172.21.15.2:0/587650114 <== osd.1 v2:172.21.15.149:6800/1737673954 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fd2c00ae1d0 con 0x7fd2c00ab660 2023-09-13T04:38:57.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.984+0000 7fd2d6ef7700 1 -- 172.21.15.2:0/587650114 --> [v2:172.21.15.149:6800/1737673954,v1:172.21.15.149:6801/1737673954] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd2d004e9b0 con 0x7fd2c00ab660 2023-09-13T04:38:57.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.985+0000 7fd2cdffb700 1 -- 172.21.15.2:0/587650114 <== osd.1 v2:172.21.15.149:6800/1737673954 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd2d004e9b0 con 0x7fd2c00ab660 2023-09-13T04:38:57.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.985+0000 7fd2d6ef7700 1 -- 172.21.15.2:0/587650114 >> [v2:172.21.15.149:6800/1737673954,v1:172.21.15.149:6801/1737673954] conn(0x7fd2c00ab660 msgr2=0x7fd2c00adb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:57.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.985+0000 7fd2d6ef7700 1 --2- 172.21.15.2:0/587650114 >> [v2:172.21.15.149:6800/1737673954,v1:172.21.15.149:6801/1737673954] conn(0x7fd2c00ab660 0x7fd2c00adb00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:57.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.985+0000 7fd2d6ef7700 1 -- 172.21.15.2:0/587650114 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd2c00a0640 msgr2=0x7fd2c00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:57.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.985+0000 7fd2d6ef7700 1 --2- 172.21.15.2:0/587650114 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd2c00a0640 0x7fd2c00a2b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00b2e0 tx=0x7fd2bc0058e0 comp rx=0 tx=0).stop 2023-09-13T04:38:57.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.985+0000 7fd2d6ef7700 1 -- 172.21.15.2:0/587650114 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2d0072ee0 msgr2=0x7fd2d01990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:57.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.985+0000 7fd2d6ef7700 1 --2- 172.21.15.2:0/587650114 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2d0072ee0 0x7fd2d01990d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400e9a0 tx=0x7fd2c400ecb0 comp rx=0 tx=0).stop 2023-09-13T04:38:57.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.986+0000 7fd2d5494700 1 -- 172.21.15.2:0/587650114 reap_dead start 2023-09-13T04:38:57.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.986+0000 7fd2d6ef7700 1 -- 172.21.15.2:0/587650114 shutdown_connections 2023-09-13T04:38:57.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.986+0000 7fd2d6ef7700 1 -- 172.21.15.2:0/587650114 >> 172.21.15.2:0/587650114 conn(0x7fd2d0077ea0 msgr2=0x7fd2d00fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:57.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.986+0000 7fd2d6ef7700 1 -- 172.21.15.2:0/587650114 shutdown_connections 2023-09-13T04:38:57.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:55.986+0000 7fd2d6ef7700 1 -- 172.21.15.2:0/587650114 wait complete. 2023-09-13T04:38:57.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.167+0000 7fae3c937700 1 -- 172.21.15.2:0/761474442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae380738f0 msgr2=0x7fae3810d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:57.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.167+0000 7fae3c937700 1 --2- 172.21.15.2:0/761474442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae380738f0 0x7fae3810d090 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0098f0 tx=0x7fae2c009c00 comp rx=0 tx=0).stop 2023-09-13T04:38:57.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.168+0000 7fae3c937700 1 -- 172.21.15.2:0/761474442 shutdown_connections 2023-09-13T04:38:57.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.168+0000 7fae3c937700 1 --2- 172.21.15.2:0/761474442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae3810d5d0 0x7fae3810f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:57.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.168+0000 7fae3c937700 1 --2- 172.21.15.2:0/761474442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae38072fd0 0x7fae380733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:57.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.168+0000 7fae3c937700 1 --2- 172.21.15.2:0/761474442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae380738f0 0x7fae3810d090 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:57.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.168+0000 7fae3c937700 1 -- 172.21.15.2:0/761474442 >> 172.21.15.2:0/761474442 conn(0x7fae380fc820 msgr2=0x7fae380fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:57.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.168+0000 7fae3c937700 1 -- 172.21.15.2:0/761474442 shutdown_connections 2023-09-13T04:38:57.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.168+0000 7fae3c937700 1 -- 172.21.15.2:0/761474442 wait complete. 2023-09-13T04:38:57.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.169+0000 7fae3c937700 1 Processor -- start 2023-09-13T04:38:57.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.169+0000 7fae3c937700 1 -- start start 2023-09-13T04:38:57.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.170+0000 7fae3c937700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae38072fd0 0x7fae38199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.170+0000 7fae3c937700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae380738f0 0x7fae381995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.170+0000 7fae3c937700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae3810d5d0 0x7fae3819da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.170+0000 7fae3c937700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3819e040 con 0x7fae3810d5d0 2023-09-13T04:38:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.170+0000 7fae3c937700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3819e1b0 con 0x7fae38072fd0 2023-09-13T04:38:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.170+0000 7fae3c937700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3819e370 con 0x7fae380738f0 2023-09-13T04:38:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.170+0000 7fae35d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae380738f0 0x7fae381995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.170+0000 7fae3659c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae38072fd0 0x7fae38199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.170+0000 7fae3659c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae38072fd0 0x7fae38199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:55406/0 (socket says 172.21.15.2:55406) 2023-09-13T04:38:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.170+0000 7fae3659c700 1 -- 172.21.15.2:0/1196279850 learned_addr learned my addr 172.21.15.2:0/1196279850 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.170+0000 7fae35d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae380738f0 0x7fae381995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:48724/0 (socket says 172.21.15.2:48724) 2023-09-13T04:38:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.171+0000 7fae36d9d700 1 --2- 172.21.15.2:0/1196279850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae3810d5d0 0x7fae3819da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.171+0000 7fae35d9b700 1 -- 172.21.15.2:0/1196279850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae38072fd0 msgr2=0x7fae38199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.171+0000 7fae35d9b700 1 --2- 172.21.15.2:0/1196279850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae38072fd0 0x7fae38199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.171+0000 7fae35d9b700 1 -- 172.21.15.2:0/1196279850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae3810d5d0 msgr2=0x7fae3819da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.171+0000 7fae35d9b700 1 --2- 172.21.15.2:0/1196279850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae3810d5d0 0x7fae3819da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:57.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.171+0000 7fae35d9b700 1 -- 172.21.15.2:0/1196279850 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2c009520 con 0x7fae380738f0 2023-09-13T04:38:57.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.171+0000 7fae35d9b700 1 --2- 172.21.15.2:0/1196279850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae380738f0 0x7fae381995c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fae2c009810 tx=0x7fae2c00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:57.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.172+0000 7fae277fe700 1 -- 172.21.15.2:0/1196279850 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c01c070 con 0x7fae380738f0 2023-09-13T04:38:57.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.172+0000 7fae277fe700 1 -- 172.21.15.2:0/1196279850 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae2c005040 con 0x7fae380738f0 2023-09-13T04:38:57.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.172+0000 7fae277fe700 1 -- 172.21.15.2:0/1196279850 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c003970 con 0x7fae380738f0 2023-09-13T04:38:57.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.172+0000 7fae3c937700 1 -- 172.21.15.2:0/1196279850 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3819e5f0 con 0x7fae380738f0 2023-09-13T04:38:57.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.172+0000 7fae3c937700 1 -- 172.21.15.2:0/1196279850 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae381a5ce0 con 0x7fae380738f0 2023-09-13T04:38:57.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.173+0000 7fae3c937700 1 -- 172.21.15.2:0/1196279850 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fae3804e9b0 con 0x7fae380738f0 2023-09-13T04:38:57.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.177+0000 7fae277fe700 1 -- 172.21.15.2:0/1196279850 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fae2c020030 con 0x7fae380738f0 2023-09-13T04:38:57.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.179+0000 7fae277fe700 1 --2- 172.21.15.2:0/1196279850 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fae200a06c0 0x7fae200a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:57.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.179+0000 7fae277fe700 1 -- 172.21.15.2:0/1196279850 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fae2c0cd240 con 0x7fae380738f0 2023-09-13T04:38:57.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.179+0000 7fae277fe700 1 --2- 172.21.15.2:0/1196279850 >> [v2:172.21.15.2:6810/3764424070,v1:172.21.15.2:6811/3764424070] conn(0x7fae200ab6e0 0x7fae200adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:57.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.180+0000 7fae277fe700 1 -- 172.21.15.2:0/1196279850 --> [v2:172.21.15.2:6810/3764424070,v1:172.21.15.2:6811/3764424070] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae200ae250 con 0x7fae200ab6e0 2023-09-13T04:38:57.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.180+0000 7fae277fe700 1 -- 172.21.15.2:0/1196279850 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae2c0cd5b0 con 0x7fae380738f0 2023-09-13T04:38:57.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.180+0000 7fae3659c700 1 --2- 172.21.15.2:0/1196279850 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fae200a06c0 0x7fae200a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:57.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.181+0000 7fae36d9d700 1 --2- 172.21.15.2:0/1196279850 >> [v2:172.21.15.2:6810/3764424070,v1:172.21.15.2:6811/3764424070] conn(0x7fae200ab6e0 0x7fae200adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:57.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.181+0000 7fae3659c700 1 --2- 172.21.15.2:0/1196279850 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fae200a06c0 0x7fae200a2b80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fae28009d20 tx=0x7fae28009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:57.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.182+0000 7fae36d9d700 1 --2- 172.21.15.2:0/1196279850 >> [v2:172.21.15.2:6810/3764424070,v1:172.21.15.2:6811/3764424070] conn(0x7fae200ab6e0 0x7fae200adb80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:57.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.183+0000 7fae277fe700 1 -- 172.21.15.2:0/1196279850 <== osd.3 v2:172.21.15.2:6810/3764424070 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fae200ae250 con 0x7fae200ab6e0 2023-09-13T04:38:57.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.205+0000 7fae3c937700 1 -- 172.21.15.2:0/1196279850 --> [v2:172.21.15.2:6810/3764424070,v1:172.21.15.2:6811/3764424070] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fae381131b0 con 0x7fae200ab6e0 2023-09-13T04:38:57.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.206+0000 7fae277fe700 1 -- 172.21.15.2:0/1196279850 <== osd.3 v2:172.21.15.2:6810/3764424070 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fae3000f040 con 0x7fae200ab6e0 2023-09-13T04:38:57.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.206+0000 7fae3c937700 1 -- 172.21.15.2:0/1196279850 >> [v2:172.21.15.2:6810/3764424070,v1:172.21.15.2:6811/3764424070] conn(0x7fae200ab6e0 msgr2=0x7fae200adb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:57.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.206+0000 7fae3c937700 1 --2- 172.21.15.2:0/1196279850 >> [v2:172.21.15.2:6810/3764424070,v1:172.21.15.2:6811/3764424070] conn(0x7fae200ab6e0 0x7fae200adb80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:57.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.206+0000 7fae3c937700 1 -- 172.21.15.2:0/1196279850 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fae200a06c0 msgr2=0x7fae200a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:57.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.206+0000 7fae3c937700 1 --2- 172.21.15.2:0/1196279850 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fae200a06c0 0x7fae200a2b80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fae28009d20 tx=0x7fae28009260 comp rx=0 tx=0).stop 2023-09-13T04:38:57.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.206+0000 7fae3c937700 1 -- 172.21.15.2:0/1196279850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae380738f0 msgr2=0x7fae381995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:57.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.206+0000 7fae3c937700 1 --2- 172.21.15.2:0/1196279850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae380738f0 0x7fae381995c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fae2c009810 tx=0x7fae2c00fa00 comp rx=0 tx=0).stop 2023-09-13T04:38:57.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.206+0000 7fae36d9d700 1 -- 172.21.15.2:0/1196279850 reap_dead start 2023-09-13T04:38:57.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.207+0000 7fae3c937700 1 -- 172.21.15.2:0/1196279850 shutdown_connections 2023-09-13T04:38:57.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.207+0000 7fae3c937700 1 -- 172.21.15.2:0/1196279850 >> 172.21.15.2:0/1196279850 conn(0x7fae380fc820 msgr2=0x7fae3810f710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:57.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.207+0000 7fae3c937700 1 -- 172.21.15.2:0/1196279850 shutdown_connections 2023-09-13T04:38:57.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.207+0000 7fae3c937700 1 -- 172.21.15.2:0/1196279850 wait complete. 2023-09-13T04:38:57.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.020+0000 7fdfe7e1b700 1 -- 172.21.15.2:0/3333834847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfe0100d80 msgr2=0x7fdfe0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:57.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.020+0000 7fdfe7e1b700 1 --2- 172.21.15.2:0/3333834847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfe0100d80 0x7fdfe0101200 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdfd00099c0 tx=0x7fdfd0009cd0 comp rx=0 tx=0).stop 2023-09-13T04:38:57.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.020+0000 7fdfe7e1b700 1 -- 172.21.15.2:0/3333834847 shutdown_connections 2023-09-13T04:38:57.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.020+0000 7fdfe7e1b700 1 --2- 172.21.15.2:0/3333834847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfe0101740 0x7fdfe010dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:57.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.020+0000 7fdfe7e1b700 1 --2- 172.21.15.2:0/3333834847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfe0106d90 0x7fdfe0107170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:57.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.020+0000 7fdfe7e1b700 1 --2- 172.21.15.2:0/3333834847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfe0100d80 0x7fdfe0101200 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:57.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.020+0000 7fdfe7e1b700 1 -- 172.21.15.2:0/3333834847 >> 172.21.15.2:0/3333834847 conn(0x7fdfe0077ea0 msgr2=0x7fdfe00782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:57.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.021+0000 7fdfe7e1b700 1 -- 172.21.15.2:0/3333834847 shutdown_connections 2023-09-13T04:38:57.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.021+0000 7fdfe7e1b700 1 -- 172.21.15.2:0/3333834847 wait complete. 2023-09-13T04:38:57.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.022+0000 7fdfe7e1b700 1 Processor -- start 2023-09-13T04:38:57.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.022+0000 7fdfe7e1b700 1 -- start start 2023-09-13T04:38:57.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.022+0000 7fdfe7e1b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfe0100d80 0x7fdfe0196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:57.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.022+0000 7fdfe7e1b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfe0101740 0x7fdfe0197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:57.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.022+0000 7fdfe7e1b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfe0106d90 0x7fdfe019b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:57.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.022+0000 7fdfe7e1b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe019bd00 con 0x7fdfe0106d90 2023-09-13T04:38:57.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.022+0000 7fdfe7e1b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe019be70 con 0x7fdfe0100d80 2023-09-13T04:38:57.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.022+0000 7fdfe7e1b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe019c030 con 0x7fdfe0101740 2023-09-13T04:38:57.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.023+0000 7fdfe5bb7700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfe0100d80 0x7fdfe0196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:57.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.023+0000 7fdfe5bb7700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfe0100d80 0x7fdfe0196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:55378/0 (socket says 172.21.15.2:55378) 2023-09-13T04:38:57.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.023+0000 7fdfe5bb7700 1 -- 172.21.15.2:0/3104506178 learned_addr learned my addr 172.21.15.2:0/3104506178 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:57.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.023+0000 7fdfe63b8700 1 --2- 172.21.15.2:0/3104506178 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfe0106d90 0x7fdfe019b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:57.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.023+0000 7fdfe53b6700 1 --2- 172.21.15.2:0/3104506178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfe0101740 0x7fdfe0197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:57.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.023+0000 7fdfe5bb7700 1 -- 172.21.15.2:0/3104506178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfe0101740 msgr2=0x7fdfe0197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:57.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.023+0000 7fdfe5bb7700 1 --2- 172.21.15.2:0/3104506178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfe0101740 0x7fdfe0197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:57.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.023+0000 7fdfe5bb7700 1 -- 172.21.15.2:0/3104506178 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfe0106d90 msgr2=0x7fdfe019b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:57.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.023+0000 7fdfe5bb7700 1 --2- 172.21.15.2:0/3104506178 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfe0106d90 0x7fdfe019b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:57.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.023+0000 7fdfe5bb7700 1 -- 172.21.15.2:0/3104506178 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd8009c40 con 0x7fdfe0100d80 2023-09-13T04:38:57.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.023+0000 7fdfe5bb7700 1 --2- 172.21.15.2:0/3104506178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfe0100d80 0x7fdfe0196d40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00e9a0 tx=0x7fdfdc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:57.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.024+0000 7fdfd6ffd700 1 -- 172.21.15.2:0/3104506178 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc009790 con 0x7fdfe0100d80 2023-09-13T04:38:57.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.024+0000 7fdfd6ffd700 1 -- 172.21.15.2:0/3104506178 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfdc00cbd0 con 0x7fdfe0100d80 2023-09-13T04:38:57.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.024+0000 7fdfe7e1b700 1 -- 172.21.15.2:0/3104506178 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd00095f0 con 0x7fdfe0100d80 2023-09-13T04:38:57.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.024+0000 7fdfd6ffd700 1 -- 172.21.15.2:0/3104506178 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc018680 con 0x7fdfe0100d80 2023-09-13T04:38:57.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.024+0000 7fdfe7e1b700 1 -- 172.21.15.2:0/3104506178 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe01a39a0 con 0x7fdfe0100d80 2023-09-13T04:38:57.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.025+0000 7fdfe7e1b700 1 -- 172.21.15.2:0/3104506178 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdfe004e9b0 con 0x7fdfe0100d80 2023-09-13T04:38:57.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.027+0000 7fdfd6ffd700 1 -- 172.21.15.2:0/3104506178 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fdfdc00cd40 con 0x7fdfe0100d80 2023-09-13T04:38:57.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.028+0000 7fdfd6ffd700 1 --2- 172.21.15.2:0/3104506178 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fdfcc0a06c0 0x7fdfcc0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:57.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.029+0000 7fdfd6ffd700 1 -- 172.21.15.2:0/3104506178 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdfdc014070 con 0x7fdfe0100d80 2023-09-13T04:38:57.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.029+0000 7fdfe53b6700 1 --2- 172.21.15.2:0/3104506178 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fdfcc0a06c0 0x7fdfcc0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:57.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.029+0000 7fdfd6ffd700 1 --2- 172.21.15.2:0/3104506178 >> [v2:172.21.15.149:6816/3932319846,v1:172.21.15.149:6817/3932319846] conn(0x7fdfcc0ab6e0 0x7fdfcc0adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:57.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.029+0000 7fdfd6ffd700 1 -- 172.21.15.2:0/3104506178 --> [v2:172.21.15.149:6816/3932319846,v1:172.21.15.149:6817/3932319846] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdfcc0ae250 con 0x7fdfcc0ab6e0 2023-09-13T04:38:57.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.029+0000 7fdfd6ffd700 1 -- 172.21.15.2:0/3104506178 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfdc0cc830 con 0x7fdfe0100d80 2023-09-13T04:38:57.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.029+0000 7fdfe63b8700 1 --2- 172.21.15.2:0/3104506178 >> [v2:172.21.15.149:6816/3932319846,v1:172.21.15.149:6817/3932319846] conn(0x7fdfcc0ab6e0 0x7fdfcc0adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:57.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.029+0000 7fdfe53b6700 1 --2- 172.21.15.2:0/3104506178 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fdfcc0a06c0 0x7fdfcc0a2b80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0009fd0 tx=0x7fdfd001a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:57.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.030+0000 7fdfe63b8700 1 --2- 172.21.15.2:0/3104506178 >> [v2:172.21.15.149:6816/3932319846,v1:172.21.15.149:6817/3932319846] conn(0x7fdfcc0ab6e0 0x7fdfcc0adb80 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:57.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.031+0000 7fdfd6ffd700 1 -- 172.21.15.2:0/3104506178 <== osd.7 v2:172.21.15.149:6816/3932319846 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fdfcc0ae250 con 0x7fdfcc0ab6e0 2023-09-13T04:38:57.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.059+0000 7fdfe7e1b700 1 -- 172.21.15.2:0/3104506178 --> [v2:172.21.15.149:6816/3932319846,v1:172.21.15.149:6817/3932319846] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdfe0110f50 con 0x7fdfcc0ab6e0 2023-09-13T04:38:57.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.059+0000 7fdfd6ffd700 1 -- 172.21.15.2:0/3104506178 <== osd.7 v2:172.21.15.149:6816/3932319846 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fdfd800f040 con 0x7fdfcc0ab6e0 2023-09-13T04:38:57.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.060+0000 7fdfe7e1b700 1 -- 172.21.15.2:0/3104506178 >> [v2:172.21.15.149:6816/3932319846,v1:172.21.15.149:6817/3932319846] conn(0x7fdfcc0ab6e0 msgr2=0x7fdfcc0adb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:57.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.060+0000 7fdfe7e1b700 1 --2- 172.21.15.2:0/3104506178 >> [v2:172.21.15.149:6816/3932319846,v1:172.21.15.149:6817/3932319846] conn(0x7fdfcc0ab6e0 0x7fdfcc0adb80 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:57.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.060+0000 7fdfe7e1b700 1 -- 172.21.15.2:0/3104506178 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fdfcc0a06c0 msgr2=0x7fdfcc0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:57.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.060+0000 7fdfe7e1b700 1 --2- 172.21.15.2:0/3104506178 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fdfcc0a06c0 0x7fdfcc0a2b80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0009fd0 tx=0x7fdfd001a040 comp rx=0 tx=0).stop 2023-09-13T04:38:57.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.060+0000 7fdfe7e1b700 1 -- 172.21.15.2:0/3104506178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfe0100d80 msgr2=0x7fdfe0196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:57.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.060+0000 7fdfe7e1b700 1 --2- 172.21.15.2:0/3104506178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfe0100d80 0x7fdfe0196d40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00e9a0 tx=0x7fdfdc00ecb0 comp rx=0 tx=0).stop 2023-09-13T04:38:57.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.060+0000 7fdfe63b8700 1 -- 172.21.15.2:0/3104506178 reap_dead start 2023-09-13T04:38:57.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.060+0000 7fdfe7e1b700 1 -- 172.21.15.2:0/3104506178 shutdown_connections 2023-09-13T04:38:57.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.060+0000 7fdfe7e1b700 1 -- 172.21.15.2:0/3104506178 >> 172.21.15.2:0/3104506178 conn(0x7fdfe0077ea0 msgr2=0x7fdfe00ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:57.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.060+0000 7fdfe7e1b700 1 -- 172.21.15.2:0/3104506178 shutdown_connections 2023-09-13T04:38:57.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:57.060+0000 7fdfe7e1b700 1 -- 172.21.15.2:0/3104506178 wait complete. 2023-09-13T04:38:58.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.610+0000 7fae220d0700 1 -- 172.21.15.2:0/3743579822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae1c101cf0 msgr2=0x7fae1c1020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:58.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.610+0000 7fae220d0700 1 --2- 172.21.15.2:0/3743579822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae1c101cf0 0x7fae1c1020d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fae18009990 tx=0x7fae18009ca0 comp rx=0 tx=0).stop 2023-09-13T04:38:58.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.610+0000 7fae220d0700 1 -- 172.21.15.2:0/3743579822 shutdown_connections 2023-09-13T04:38:58.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.610+0000 7fae220d0700 1 --2- 172.21.15.2:0/3743579822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae1c10b120 0x7fae1c10d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:58.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.610+0000 7fae220d0700 1 --2- 172.21.15.2:0/3743579822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae1c101cf0 0x7fae1c1020d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:58.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.610+0000 7fae220d0700 1 --2- 172.21.15.2:0/3743579822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae1c102610 0x7fae1c10ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:58.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.610+0000 7fae220d0700 1 -- 172.21.15.2:0/3743579822 >> 172.21.15.2:0/3743579822 conn(0x7fae1c0fb560 msgr2=0x7fae1c0fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:58.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.610+0000 7fae220d0700 1 -- 172.21.15.2:0/3743579822 shutdown_connections 2023-09-13T04:38:58.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.610+0000 7fae220d0700 1 -- 172.21.15.2:0/3743579822 wait complete. 2023-09-13T04:38:58.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.612+0000 7fae220d0700 1 Processor -- start 2023-09-13T04:38:58.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.612+0000 7fae220d0700 1 -- start start 2023-09-13T04:38:58.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.613+0000 7fae220d0700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae1c101cf0 0x7fae1c107210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:58.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.613+0000 7fae220d0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae1c102610 0x7fae1c107750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:58.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.613+0000 7fae220d0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae1c10b120 0x7fae1c1a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:58.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.613+0000 7fae220d0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c107cc0 con 0x7fae1c10b120 2023-09-13T04:38:58.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.613+0000 7fae220d0700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c107e30 con 0x7fae1c101cf0 2023-09-13T04:38:58.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.613+0000 7fae220d0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c1b2160 con 0x7fae1c102610 2023-09-13T04:38:58.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.613+0000 7fae210ce700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae1c101cf0 0x7fae1c107210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:58.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.613+0000 7fae210ce700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae1c101cf0 0x7fae1c107210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:55434/0 (socket says 172.21.15.2:55434) 2023-09-13T04:38:58.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.613+0000 7fae210ce700 1 -- 172.21.15.2:0/2788119204 learned_addr learned my addr 172.21.15.2:0/2788119204 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:58.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.613+0000 7fae208cd700 1 --2- 172.21.15.2:0/2788119204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae1c102610 0x7fae1c107750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:58.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.614+0000 7fae210ce700 1 -- 172.21.15.2:0/2788119204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae1c102610 msgr2=0x7fae1c107750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:58.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.614+0000 7fae210ce700 1 --2- 172.21.15.2:0/2788119204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae1c102610 0x7fae1c107750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:58.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.614+0000 7fae218cf700 1 --2- 172.21.15.2:0/2788119204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae1c10b120 0x7fae1c1a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:58.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.614+0000 7fae210ce700 1 -- 172.21.15.2:0/2788119204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae1c10b120 msgr2=0x7fae1c1a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:58.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.614+0000 7fae210ce700 1 --2- 172.21.15.2:0/2788119204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae1c10b120 0x7fae1c1a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:58.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.614+0000 7fae210ce700 1 -- 172.21.15.2:0/2788119204 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae180095f0 con 0x7fae1c101cf0 2023-09-13T04:38:58.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.614+0000 7fae208cd700 1 --2- 172.21.15.2:0/2788119204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae1c102610 0x7fae1c107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:38:58.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.614+0000 7fae218cf700 1 --2- 172.21.15.2:0/2788119204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae1c10b120 0x7fae1c1a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:38:58.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.614+0000 7fae210ce700 1 --2- 172.21.15.2:0/2788119204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae1c101cf0 0x7fae1c107210 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fae18000c00 tx=0x7fae18009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:58.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.615+0000 7fae127fc700 1 -- 172.21.15.2:0/2788119204 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1801d070 con 0x7fae1c101cf0 2023-09-13T04:38:58.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.615+0000 7fae127fc700 1 -- 172.21.15.2:0/2788119204 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae18021410 con 0x7fae1c101cf0 2023-09-13T04:38:58.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.615+0000 7fae127fc700 1 -- 172.21.15.2:0/2788119204 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae18017650 con 0x7fae1c101cf0 2023-09-13T04:38:58.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.615+0000 7fae220d0700 1 -- 172.21.15.2:0/2788119204 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1c1b2390 con 0x7fae1c101cf0 2023-09-13T04:38:58.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.615+0000 7fae220d0700 1 -- 172.21.15.2:0/2788119204 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae1c1b2780 con 0x7fae1c101cf0 2023-09-13T04:38:58.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.616+0000 7fae220d0700 1 -- 172.21.15.2:0/2788119204 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fae1c100e30 con 0x7fae1c101cf0 2023-09-13T04:38:58.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.619+0000 7fae127fc700 1 -- 172.21.15.2:0/2788119204 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fae1800fa80 con 0x7fae1c101cf0 2023-09-13T04:38:58.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.622+0000 7fae127fc700 1 --2- 172.21.15.2:0/2788119204 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fae080a0690 0x7fae080a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:58.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.622+0000 7fae127fc700 1 -- 172.21.15.2:0/2788119204 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fae180cdd30 con 0x7fae1c101cf0 2023-09-13T04:38:58.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.622+0000 7fae127fc700 1 --2- 172.21.15.2:0/2788119204 >> [v2:172.21.15.156:6808/1709817635,v1:172.21.15.156:6809/1709817635] conn(0x7fae080ab6b0 0x7fae080adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:58.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.622+0000 7fae208cd700 1 --2- 172.21.15.2:0/2788119204 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fae080a0690 0x7fae080a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:58.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.622+0000 7fae127fc700 1 -- 172.21.15.2:0/2788119204 --> [v2:172.21.15.156:6808/1709817635,v1:172.21.15.156:6809/1709817635] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae080ae220 con 0x7fae080ab6b0 2023-09-13T04:38:58.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.622+0000 7fae127fc700 1 -- 172.21.15.2:0/2788119204 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae180ce0a0 con 0x7fae1c101cf0 2023-09-13T04:38:58.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.623+0000 7fae218cf700 1 --2- 172.21.15.2:0/2788119204 >> [v2:172.21.15.156:6808/1709817635,v1:172.21.15.156:6809/1709817635] conn(0x7fae080ab6b0 0x7fae080adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:58.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.623+0000 7fae208cd700 1 --2- 172.21.15.2:0/2788119204 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fae080a0690 0x7fae080a2b50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fae1c1088e0 tx=0x7fae0c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:58.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.624+0000 7fae218cf700 1 --2- 172.21.15.2:0/2788119204 >> [v2:172.21.15.156:6808/1709817635,v1:172.21.15.156:6809/1709817635] conn(0x7fae080ab6b0 0x7fae080adb50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:58.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.625+0000 7fae127fc700 1 -- 172.21.15.2:0/2788119204 <== osd.5 v2:172.21.15.156:6808/1709817635 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fae080ae220 con 0x7fae080ab6b0 2023-09-13T04:38:58.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.644+0000 7fae220d0700 1 -- 172.21.15.2:0/2788119204 --> [v2:172.21.15.156:6808/1709817635,v1:172.21.15.156:6809/1709817635] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fae1c04f200 con 0x7fae080ab6b0 2023-09-13T04:38:58.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.645+0000 7fae127fc700 1 -- 172.21.15.2:0/2788119204 <== osd.5 v2:172.21.15.156:6808/1709817635 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fae1c04f200 con 0x7fae080ab6b0 2023-09-13T04:38:58.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.646+0000 7fae220d0700 1 -- 172.21.15.2:0/2788119204 >> [v2:172.21.15.156:6808/1709817635,v1:172.21.15.156:6809/1709817635] conn(0x7fae080ab6b0 msgr2=0x7fae080adb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:58.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.646+0000 7fae220d0700 1 --2- 172.21.15.2:0/2788119204 >> [v2:172.21.15.156:6808/1709817635,v1:172.21.15.156:6809/1709817635] conn(0x7fae080ab6b0 0x7fae080adb50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:58.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.646+0000 7fae220d0700 1 -- 172.21.15.2:0/2788119204 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fae080a0690 msgr2=0x7fae080a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:58.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.646+0000 7fae220d0700 1 --2- 172.21.15.2:0/2788119204 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fae080a0690 0x7fae080a2b50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fae1c1088e0 tx=0x7fae0c009260 comp rx=0 tx=0).stop 2023-09-13T04:38:58.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.646+0000 7fae220d0700 1 -- 172.21.15.2:0/2788119204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae1c101cf0 msgr2=0x7fae1c107210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:58.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.646+0000 7fae220d0700 1 --2- 172.21.15.2:0/2788119204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae1c101cf0 0x7fae1c107210 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fae18000c00 tx=0x7fae18009580 comp rx=0 tx=0).stop 2023-09-13T04:38:58.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.646+0000 7fae218cf700 1 -- 172.21.15.2:0/2788119204 reap_dead start 2023-09-13T04:38:58.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.646+0000 7fae220d0700 1 -- 172.21.15.2:0/2788119204 shutdown_connections 2023-09-13T04:38:58.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.646+0000 7fae220d0700 1 -- 172.21.15.2:0/2788119204 >> 172.21.15.2:0/2788119204 conn(0x7fae1c0fb560 msgr2=0x7fae1c10f820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:58.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.646+0000 7fae220d0700 1 -- 172.21.15.2:0/2788119204 shutdown_connections 2023-09-13T04:38:58.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.646+0000 7fae220d0700 1 -- 172.21.15.2:0/2788119204 wait complete. 2023-09-13T04:38:58.670 INFO:teuthology.orchestra.run.smithi002.stdout:219043332138 2023-09-13T04:38:58.670 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph osd last-stat-seq osd.7 2023-09-13T04:38:58.853 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:38:58 smithi002 ceph-mon[97651]: pgmap v307: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2023-09-13T04:38:58.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:38:58 smithi149 ceph-mon[133315]: pgmap v307: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2023-09-13T04:38:58.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:38:58 smithi156 ceph-mon[133675]: pgmap v307: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2023-09-13T04:38:59.403 INFO:teuthology.orchestra.run.smithi002.stdout:304942678044 2023-09-13T04:38:59.403 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph osd last-stat-seq osd.10 2023-09-13T04:38:59.446 INFO:teuthology.orchestra.run.smithi002.stdout:107374182461 2023-09-13T04:38:59.446 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph osd last-stat-seq osd.3 2023-09-13T04:38:59.523 INFO:teuthology.orchestra.run.smithi002.stdout:163208757300 2023-09-13T04:38:59.523 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph osd last-stat-seq osd.5 2023-09-13T04:38:59.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.988+0000 7fd2cb302700 1 -- 172.21.15.2:0/452272529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2c410b460 msgr2=0x7fd2c4107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:59.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.988+0000 7fd2cb302700 1 --2- 172.21.15.2:0/452272529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2c410b460 0x7fd2c4107cb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0009990 tx=0x7fd2c0009ca0 comp rx=0 tx=0).stop 2023-09-13T04:38:59.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.988+0000 7fd2cb302700 1 -- 172.21.15.2:0/452272529 shutdown_connections 2023-09-13T04:38:59.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.988+0000 7fd2cb302700 1 --2- 172.21.15.2:0/452272529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c4115de0 0x7fd2c41181d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:59.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.988+0000 7fd2cb302700 1 --2- 172.21.15.2:0/452272529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2c410b460 0x7fd2c4107cb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:59.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.988+0000 7fd2cb302700 1 --2- 172.21.15.2:0/452272529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2c4108280 0x7fd2c4108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:59.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.988+0000 7fd2cb302700 1 -- 172.21.15.2:0/452272529 >> 172.21.15.2:0/452272529 conn(0x7fd2c406d260 msgr2=0x7fd2c406d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:59.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.988+0000 7fd2cb302700 1 -- 172.21.15.2:0/452272529 shutdown_connections 2023-09-13T04:38:59.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.988+0000 7fd2cb302700 1 -- 172.21.15.2:0/452272529 wait complete. 2023-09-13T04:38:59.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.989+0000 7fd2cb302700 1 Processor -- start 2023-09-13T04:38:59.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.989+0000 7fd2cb302700 1 -- start start 2023-09-13T04:38:59.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.990+0000 7fd2cb302700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c4108280 0x7fd2c41a7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:59.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.990+0000 7fd2cb302700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2c410b460 0x7fd2c41a7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:59.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.990+0000 7fd2cb302700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2c4115de0 0x7fd2c41a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:59.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.990+0000 7fd2cb302700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c41a1cf0 con 0x7fd2c4115de0 2023-09-13T04:38:59.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.990+0000 7fd2cb302700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c41a1e60 con 0x7fd2c410b460 2023-09-13T04:38:59.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.990+0000 7fd2cb302700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c41a2020 con 0x7fd2c4108280 2023-09-13T04:38:59.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.990+0000 7fd2c989f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2c4115de0 0x7fd2c41a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:59.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.990+0000 7fd2c989f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2c4115de0 0x7fd2c41a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35800/0 (socket says 172.21.15.2:35800) 2023-09-13T04:38:59.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.990+0000 7fd2c989f700 1 -- 172.21.15.2:0/2565297236 learned_addr learned my addr 172.21.15.2:0/2565297236 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:38:59.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.991+0000 7fd2c889d700 1 --2- 172.21.15.2:0/2565297236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2c410b460 0x7fd2c41a7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:59.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.991+0000 7fd2c989f700 1 -- 172.21.15.2:0/2565297236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c4108280 msgr2=0x7fd2c41a7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:59.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.991+0000 7fd2c989f700 1 --2- 172.21.15.2:0/2565297236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c4108280 0x7fd2c41a7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:59.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.991+0000 7fd2c989f700 1 -- 172.21.15.2:0/2565297236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2c410b460 msgr2=0x7fd2c41a7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:59.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.991+0000 7fd2c989f700 1 --2- 172.21.15.2:0/2565297236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2c410b460 0x7fd2c41a7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:59.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.991+0000 7fd2c989f700 1 -- 172.21.15.2:0/2565297236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c00095f0 con 0x7fd2c4115de0 2023-09-13T04:38:59.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.991+0000 7fd2c889d700 1 --2- 172.21.15.2:0/2565297236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2c410b460 0x7fd2c41a7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:38:59.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.991+0000 7fd2c989f700 1 --2- 172.21.15.2:0/2565297236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2c4115de0 0x7fd2c41a1680 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00ef30 tx=0x7fd2bc00c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:59.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.991+0000 7fd2ba7fc700 1 -- 172.21.15.2:0/2565297236 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc018070 con 0x7fd2c4115de0 2023-09-13T04:38:59.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.992+0000 7fd2cb302700 1 -- 172.21.15.2:0/2565297236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c41a22a0 con 0x7fd2c4115de0 2023-09-13T04:38:59.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.992+0000 7fd2ba7fc700 1 -- 172.21.15.2:0/2565297236 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2bc00f040 con 0x7fd2c4115de0 2023-09-13T04:38:59.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.992+0000 7fd2ba7fc700 1 -- 172.21.15.2:0/2565297236 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc003dc0 con 0x7fd2c4115de0 2023-09-13T04:38:59.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.992+0000 7fd2cb302700 1 -- 172.21.15.2:0/2565297236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c410ba30 con 0x7fd2c4115de0 2023-09-13T04:38:59.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.993+0000 7fd2cb302700 1 -- 172.21.15.2:0/2565297236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd2c404e9b0 con 0x7fd2c4115de0 2023-09-13T04:38:59.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.996+0000 7fd2ba7fc700 1 -- 172.21.15.2:0/2565297236 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fd2bc00caa0 con 0x7fd2c4115de0 2023-09-13T04:38:59.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.997+0000 7fd2ba7fc700 1 --2- 172.21.15.2:0/2565297236 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd2b00a06c0 0x7fd2b00a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:59.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.997+0000 7fd2ba7fc700 1 -- 172.21.15.2:0/2565297236 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd2bc0cc3e0 con 0x7fd2c4115de0 2023-09-13T04:38:59.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.998+0000 7fd2ba7fc700 1 --2- 172.21.15.2:0/2565297236 >> [v2:172.21.15.156:6800/3090983787,v1:172.21.15.156:6801/3090983787] conn(0x7fd2b00ab6e0 0x7fd2b00adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:38:59.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.998+0000 7fd2ba7fc700 1 -- 172.21.15.2:0/2565297236 --> [v2:172.21.15.156:6800/3090983787,v1:172.21.15.156:6801/3090983787] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd2b00ae250 con 0x7fd2b00ab6e0 2023-09-13T04:38:59.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.998+0000 7fd2ba7fc700 1 -- 172.21.15.2:0/2565297236 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2bc0cc750 con 0x7fd2c4115de0 2023-09-13T04:38:59.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.998+0000 7fd2c909e700 1 --2- 172.21.15.2:0/2565297236 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd2b00a06c0 0x7fd2b00a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:59.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.998+0000 7fd2c889d700 1 --2- 172.21.15.2:0/2565297236 >> [v2:172.21.15.156:6800/3090983787,v1:172.21.15.156:6801/3090983787] conn(0x7fd2b00ab6e0 0x7fd2b00adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:38:59.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.999+0000 7fd2c909e700 1 --2- 172.21.15.2:0/2565297236 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd2b00a06c0 0x7fd2b00a2b80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00098e0 tx=0x7fd2c0005e80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:59.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:58.999+0000 7fd2c889d700 1 --2- 172.21.15.2:0/2565297236 >> [v2:172.21.15.156:6800/3090983787,v1:172.21.15.156:6801/3090983787] conn(0x7fd2b00ab6e0 0x7fd2b00adb80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:38:59.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.000+0000 7fd2ba7fc700 1 -- 172.21.15.2:0/2565297236 <== osd.2 v2:172.21.15.156:6800/3090983787 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fd2b00ae250 con 0x7fd2b00ab6e0 2023-09-13T04:38:59.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.017+0000 7fd2cb302700 1 -- 172.21.15.2:0/2565297236 --> [v2:172.21.15.156:6800/3090983787,v1:172.21.15.156:6801/3090983787] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd2c411b990 con 0x7fd2b00ab6e0 2023-09-13T04:38:59.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.018+0000 7fd2ba7fc700 1 -- 172.21.15.2:0/2565297236 <== osd.2 v2:172.21.15.156:6800/3090983787 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd2b40099e0 con 0x7fd2b00ab6e0 2023-09-13T04:38:59.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.018+0000 7fd2cb302700 1 -- 172.21.15.2:0/2565297236 >> [v2:172.21.15.156:6800/3090983787,v1:172.21.15.156:6801/3090983787] conn(0x7fd2b00ab6e0 msgr2=0x7fd2b00adb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:59.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.018+0000 7fd2cb302700 1 --2- 172.21.15.2:0/2565297236 >> [v2:172.21.15.156:6800/3090983787,v1:172.21.15.156:6801/3090983787] conn(0x7fd2b00ab6e0 0x7fd2b00adb80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:38:59.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.019+0000 7fd2cb302700 1 -- 172.21.15.2:0/2565297236 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd2b00a06c0 msgr2=0x7fd2b00a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:59.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.019+0000 7fd2cb302700 1 --2- 172.21.15.2:0/2565297236 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd2b00a06c0 0x7fd2b00a2b80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00098e0 tx=0x7fd2c0005e80 comp rx=0 tx=0).stop 2023-09-13T04:38:59.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.019+0000 7fd2cb302700 1 -- 172.21.15.2:0/2565297236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2c4115de0 msgr2=0x7fd2c41a1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:38:59.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.019+0000 7fd2cb302700 1 --2- 172.21.15.2:0/2565297236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2c4115de0 0x7fd2c41a1680 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00ef30 tx=0x7fd2bc00c450 comp rx=0 tx=0).stop 2023-09-13T04:38:59.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.019+0000 7fd2c989f700 1 -- 172.21.15.2:0/2565297236 reap_dead start 2023-09-13T04:38:59.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.019+0000 7fd2cb302700 1 -- 172.21.15.2:0/2565297236 shutdown_connections 2023-09-13T04:38:59.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.019+0000 7fd2cb302700 1 -- 172.21.15.2:0/2565297236 >> 172.21.15.2:0/2565297236 conn(0x7fd2c406d260 msgr2=0x7fd2c40717d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:38:59.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.019+0000 7fd2cb302700 1 -- 172.21.15.2:0/2565297236 shutdown_connections 2023-09-13T04:38:59.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.019+0000 7fd2cb302700 1 -- 172.21.15.2:0/2565297236 wait complete. 2023-09-13T04:39:00.284 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:00 smithi002 ceph-mon[97651]: pgmap v308: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-09-13T04:39:00.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.138+0000 7f95f8e3e700 1 -- 172.21.15.2:0/2851658853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95f4101740 msgr2=0x7f95f410dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:00.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.138+0000 7f95f8e3e700 1 --2- 172.21.15.2:0/2851658853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95f4101740 0x7f95f410dbb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f95ec009fc0 tx=0x7f95ec00b330 comp rx=0 tx=0).stop 2023-09-13T04:39:00.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.138+0000 7f95f8e3e700 1 -- 172.21.15.2:0/2851658853 shutdown_connections 2023-09-13T04:39:00.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.138+0000 7f95f8e3e700 1 --2- 172.21.15.2:0/2851658853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95f4101740 0x7f95f410dbb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:00.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.138+0000 7f95f8e3e700 1 --2- 172.21.15.2:0/2851658853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95f4106d90 0x7f95f4107170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:00.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.138+0000 7f95f8e3e700 1 --2- 172.21.15.2:0/2851658853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95f4100d80 0x7f95f4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:00.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.138+0000 7f95f8e3e700 1 -- 172.21.15.2:0/2851658853 >> 172.21.15.2:0/2851658853 conn(0x7f95f4077ea0 msgr2=0x7f95f40782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:00.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.138+0000 7f95f8e3e700 1 -- 172.21.15.2:0/2851658853 shutdown_connections 2023-09-13T04:39:00.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.138+0000 7f95f8e3e700 1 -- 172.21.15.2:0/2851658853 wait complete. 2023-09-13T04:39:00.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.139+0000 7f95f8e3e700 1 Processor -- start 2023-09-13T04:39:00.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.139+0000 7f95f8e3e700 1 -- start start 2023-09-13T04:39:00.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.140+0000 7f95f8e3e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95f4100d80 0x7f95f4199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:00.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.140+0000 7f95f8e3e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95f4101740 0x7f95f4199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:00.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.140+0000 7f95f8e3e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95f4106d90 0x7f95f419d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:00.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.140+0000 7f95f8e3e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f419dff0 con 0x7f95f4101740 2023-09-13T04:39:00.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.140+0000 7f95f8e3e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f419e160 con 0x7f95f4106d90 2023-09-13T04:39:00.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.140+0000 7f95f8e3e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f419e320 con 0x7f95f4100d80 2023-09-13T04:39:00.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.140+0000 7f95f259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95f4100d80 0x7f95f4199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:00.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.140+0000 7f95f1d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95f4101740 0x7f95f4199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:00.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.140+0000 7f95f259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95f4100d80 0x7f95f4199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:48750/0 (socket says 172.21.15.2:48750) 2023-09-13T04:39:00.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.140+0000 7f95f259c700 1 -- 172.21.15.2:0/310457275 learned_addr learned my addr 172.21.15.2:0/310457275 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:00.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.141+0000 7f95f2d9d700 1 --2- 172.21.15.2:0/310457275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95f4106d90 0x7f95f419d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:00.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.141+0000 7f95f1d9b700 1 -- 172.21.15.2:0/310457275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95f4100d80 msgr2=0x7f95f4199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:00.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.141+0000 7f95f1d9b700 1 --2- 172.21.15.2:0/310457275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95f4100d80 0x7f95f4199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:00.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.141+0000 7f95f1d9b700 1 -- 172.21.15.2:0/310457275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95f4106d90 msgr2=0x7f95f419d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:00.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.141+0000 7f95f1d9b700 1 --2- 172.21.15.2:0/310457275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95f4106d90 0x7f95f419d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:00.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.141+0000 7f95f1d9b700 1 -- 172.21.15.2:0/310457275 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e8009520 con 0x7f95f4101740 2023-09-13T04:39:00.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.141+0000 7f95f2d9d700 1 --2- 172.21.15.2:0/310457275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95f4106d90 0x7f95f419d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:39:00.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.141+0000 7f95f1d9b700 1 --2- 172.21.15.2:0/310457275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95f4101740 0x7f95f4199570 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f95e800e9f0 tx=0x7f95e800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:00.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.141+0000 7f95e37fe700 1 -- 172.21.15.2:0/310457275 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e800cb70 con 0x7f95f4101740 2023-09-13T04:39:00.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.141+0000 7f95e37fe700 1 -- 172.21.15.2:0/310457275 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95e80044e0 con 0x7f95f4101740 2023-09-13T04:39:00.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.141+0000 7f95f8e3e700 1 -- 172.21.15.2:0/310457275 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e40095f0 con 0x7f95f4101740 2023-09-13T04:39:00.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.141+0000 7f95e37fe700 1 -- 172.21.15.2:0/310457275 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e8003d50 con 0x7f95f4101740 2023-09-13T04:39:00.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.142+0000 7f95f8e3e700 1 -- 172.21.15.2:0/310457275 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ec009c40 con 0x7f95f4101740 2023-09-13T04:39:00.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.142+0000 7f95f8e3e700 1 -- 172.21.15.2:0/310457275 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f95f404e9b0 con 0x7f95f4101740 2023-09-13T04:39:00.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.145+0000 7f95e37fe700 1 -- 172.21.15.2:0/310457275 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f95e8003eb0 con 0x7f95f4101740 2023-09-13T04:39:00.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.147+0000 7f95e37fe700 1 --2- 172.21.15.2:0/310457275 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f95dc0a0640 0x7f95dc0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:00.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.147+0000 7f95e37fe700 1 -- 172.21.15.2:0/310457275 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f95e8014070 con 0x7f95f4101740 2023-09-13T04:39:00.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.147+0000 7f95f259c700 1 --2- 172.21.15.2:0/310457275 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f95dc0a0640 0x7f95dc0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:00.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.148+0000 7f95e37fe700 1 --2- 172.21.15.2:0/310457275 >> [v2:172.21.15.2:6826/3097876180,v1:172.21.15.2:6827/3097876180] conn(0x7f95dc0ab660 0x7f95dc0adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:00.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.148+0000 7f95e37fe700 1 -- 172.21.15.2:0/310457275 --> [v2:172.21.15.2:6826/3097876180,v1:172.21.15.2:6827/3097876180] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f95dc0ae1d0 con 0x7f95dc0ab660 2023-09-13T04:39:00.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.148+0000 7f95e37fe700 1 -- 172.21.15.2:0/310457275 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95e80cc750 con 0x7f95f4101740 2023-09-13T04:39:00.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.148+0000 7f95f259c700 1 --2- 172.21.15.2:0/310457275 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f95dc0a0640 0x7f95dc0a2b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f95e4005d10 tx=0x7f95e4009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:00.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.148+0000 7f95f2d9d700 1 --2- 172.21.15.2:0/310457275 >> [v2:172.21.15.2:6826/3097876180,v1:172.21.15.2:6827/3097876180] conn(0x7f95dc0ab660 0x7f95dc0adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:00.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.149+0000 7f95f2d9d700 1 --2- 172.21.15.2:0/310457275 >> [v2:172.21.15.2:6826/3097876180,v1:172.21.15.2:6827/3097876180] conn(0x7f95dc0ab660 0x7f95dc0adb00 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:00.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.149+0000 7f95e37fe700 1 -- 172.21.15.2:0/310457275 <== osd.9 v2:172.21.15.2:6826/3097876180 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f95dc0ae1d0 con 0x7f95dc0ab660 2023-09-13T04:39:00.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.168+0000 7f95f8e3e700 1 -- 172.21.15.2:0/310457275 --> [v2:172.21.15.2:6826/3097876180,v1:172.21.15.2:6827/3097876180] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f95f4110f50 con 0x7f95dc0ab660 2023-09-13T04:39:00.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.168+0000 7f95e37fe700 1 -- 172.21.15.2:0/310457275 <== osd.9 v2:172.21.15.2:6826/3097876180 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f95ec007db0 con 0x7f95dc0ab660 2023-09-13T04:39:00.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.169+0000 7f95f8e3e700 1 -- 172.21.15.2:0/310457275 >> [v2:172.21.15.2:6826/3097876180,v1:172.21.15.2:6827/3097876180] conn(0x7f95dc0ab660 msgr2=0x7f95dc0adb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:00.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.169+0000 7f95f8e3e700 1 --2- 172.21.15.2:0/310457275 >> [v2:172.21.15.2:6826/3097876180,v1:172.21.15.2:6827/3097876180] conn(0x7f95dc0ab660 0x7f95dc0adb00 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:00.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.169+0000 7f95f8e3e700 1 -- 172.21.15.2:0/310457275 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f95dc0a0640 msgr2=0x7f95dc0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:00.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.169+0000 7f95f8e3e700 1 --2- 172.21.15.2:0/310457275 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f95dc0a0640 0x7f95dc0a2b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f95e4005d10 tx=0x7f95e4009310 comp rx=0 tx=0).stop 2023-09-13T04:39:00.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.169+0000 7f95f8e3e700 1 -- 172.21.15.2:0/310457275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95f4101740 msgr2=0x7f95f4199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:00.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.169+0000 7f95f8e3e700 1 --2- 172.21.15.2:0/310457275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95f4101740 0x7f95f4199570 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f95e800e9f0 tx=0x7f95e800ed00 comp rx=0 tx=0).stop 2023-09-13T04:39:00.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.169+0000 7f95f2d9d700 1 -- 172.21.15.2:0/310457275 reap_dead start 2023-09-13T04:39:00.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.169+0000 7f95f8e3e700 1 -- 172.21.15.2:0/310457275 shutdown_connections 2023-09-13T04:39:00.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.169+0000 7f95f8e3e700 1 -- 172.21.15.2:0/310457275 >> 172.21.15.2:0/310457275 conn(0x7f95f4077ea0 msgr2=0x7f95f40ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:00.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.169+0000 7f95f8e3e700 1 -- 172.21.15.2:0/310457275 shutdown_connections 2023-09-13T04:39:00.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:38:59.169+0000 7f95f8e3e700 1 -- 172.21.15.2:0/310457275 wait complete. 2023-09-13T04:39:00.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:00 smithi149 ceph-mon[133315]: pgmap v308: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-09-13T04:39:00.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.642+0000 7f87ae800700 1 -- 172.21.15.2:0/1612957249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87a8100d80 msgr2=0x7f87a8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:00.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.642+0000 7f87ae800700 1 --2- 172.21.15.2:0/1612957249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87a8100d80 0x7f87a8101200 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f879c0099c0 tx=0x7f879c009cd0 comp rx=0 tx=0).stop 2023-09-13T04:39:00.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.642+0000 7f87ae800700 1 -- 172.21.15.2:0/1612957249 shutdown_connections 2023-09-13T04:39:00.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.642+0000 7f87ae800700 1 --2- 172.21.15.2:0/1612957249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87a8101740 0x7f87a810dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:00.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.642+0000 7f87ae800700 1 --2- 172.21.15.2:0/1612957249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87a8106d90 0x7f87a8107170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:00.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.642+0000 7f87ae800700 1 --2- 172.21.15.2:0/1612957249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87a8100d80 0x7f87a8101200 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:00.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.642+0000 7f87ae800700 1 -- 172.21.15.2:0/1612957249 >> 172.21.15.2:0/1612957249 conn(0x7f87a8077ea0 msgr2=0x7f87a80782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:00.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.642+0000 7f87ae800700 1 -- 172.21.15.2:0/1612957249 shutdown_connections 2023-09-13T04:39:00.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.642+0000 7f87ae800700 1 -- 172.21.15.2:0/1612957249 wait complete. 2023-09-13T04:39:00.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.644+0000 7f87ae800700 1 Processor -- start 2023-09-13T04:39:00.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.644+0000 7f87ae800700 1 -- start start 2023-09-13T04:39:00.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.645+0000 7f87ae800700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87a8100d80 0x7f87a8196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:00.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.645+0000 7f87ae800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87a8101740 0x7f87a8197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:00.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.645+0000 7f87ae800700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87a8106d90 0x7f87a819b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:00.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.645+0000 7f87ae800700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a819bd80 con 0x7f87a8106d90 2023-09-13T04:39:00.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.645+0000 7f87ae800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a819bef0 con 0x7f87a8101740 2023-09-13T04:39:00.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.645+0000 7f87ae800700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a819c0b0 con 0x7f87a8100d80 2023-09-13T04:39:00.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.645+0000 7f87a7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87a8100d80 0x7f87a8196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:00.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.645+0000 7f87a7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87a8100d80 0x7f87a8196e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:48790/0 (socket says 172.21.15.2:48790) 2023-09-13T04:39:00.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.645+0000 7f87acd9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87a8106d90 0x7f87a819b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:00.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.645+0000 7f87a7fff700 1 -- 172.21.15.2:0/1038049806 learned_addr learned my addr 172.21.15.2:0/1038049806 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:00.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.645+0000 7f87a77fe700 1 --2- 172.21.15.2:0/1038049806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87a8101740 0x7f87a8197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:00.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.645+0000 7f87a7fff700 1 -- 172.21.15.2:0/1038049806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87a8101740 msgr2=0x7f87a8197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:00.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.645+0000 7f87a7fff700 1 --2- 172.21.15.2:0/1038049806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87a8101740 0x7f87a8197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:00.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.645+0000 7f87a7fff700 1 -- 172.21.15.2:0/1038049806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87a8106d90 msgr2=0x7f87a819b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:00.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.645+0000 7f87a7fff700 1 --2- 172.21.15.2:0/1038049806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87a8106d90 0x7f87a819b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:00.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.645+0000 7f87a7fff700 1 -- 172.21.15.2:0/1038049806 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879c0095f0 con 0x7f87a8100d80 2023-09-13T04:39:00.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.646+0000 7f87a7fff700 1 --2- 172.21.15.2:0/1038049806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87a8100d80 0x7f87a8196e10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f879800e9a0 tx=0x7f879800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:00.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.646+0000 7f87a57fa700 1 -- 172.21.15.2:0/1038049806 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8798009790 con 0x7f87a8100d80 2023-09-13T04:39:00.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.646+0000 7f87ae800700 1 -- 172.21.15.2:0/1038049806 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a819c390 con 0x7f87a8100d80 2023-09-13T04:39:00.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.646+0000 7f87a57fa700 1 -- 172.21.15.2:0/1038049806 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8798010430 con 0x7f87a8100d80 2023-09-13T04:39:00.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.646+0000 7f87a57fa700 1 -- 172.21.15.2:0/1038049806 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8798018680 con 0x7f87a8100d80 2023-09-13T04:39:00.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.646+0000 7f87ae800700 1 -- 172.21.15.2:0/1038049806 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a81a3bf0 con 0x7f87a8100d80 2023-09-13T04:39:00.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.647+0000 7f87ae800700 1 -- 172.21.15.2:0/1038049806 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f87a804e9b0 con 0x7f87a8100d80 2023-09-13T04:39:00.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.649+0000 7f87a57fa700 1 -- 172.21.15.2:0/1038049806 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f87980105a0 con 0x7f87a8100d80 2023-09-13T04:39:00.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.651+0000 7f87a57fa700 1 --2- 172.21.15.2:0/1038049806 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f87900a0640 0x7f87900a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:00.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.651+0000 7f87a57fa700 1 -- 172.21.15.2:0/1038049806 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8798014070 con 0x7f87a8100d80 2023-09-13T04:39:00.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.651+0000 7f87a77fe700 1 --2- 172.21.15.2:0/1038049806 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f87900a0640 0x7f87900a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:00.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.651+0000 7f87a57fa700 1 --2- 172.21.15.2:0/1038049806 >> [v2:172.21.15.2:6818/3548993414,v1:172.21.15.2:6819/3548993414] conn(0x7f87900ab660 0x7f87900adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:00.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.651+0000 7f87a57fa700 1 -- 172.21.15.2:0/1038049806 --> [v2:172.21.15.2:6818/3548993414,v1:172.21.15.2:6819/3548993414] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f87900ae1d0 con 0x7f87900ab660 2023-09-13T04:39:00.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.651+0000 7f87a57fa700 1 -- 172.21.15.2:0/1038049806 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87980cc6c0 con 0x7f87a8100d80 2023-09-13T04:39:00.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.651+0000 7f87acd9d700 1 --2- 172.21.15.2:0/1038049806 >> [v2:172.21.15.2:6818/3548993414,v1:172.21.15.2:6819/3548993414] conn(0x7f87900ab660 0x7f87900adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:00.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.652+0000 7f87a77fe700 1 --2- 172.21.15.2:0/1038049806 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f87900a0640 0x7f87900a2b00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f879c009fd0 tx=0x7f879c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:00.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.652+0000 7f87acd9d700 1 --2- 172.21.15.2:0/1038049806 >> [v2:172.21.15.2:6818/3548993414,v1:172.21.15.2:6819/3548993414] conn(0x7f87900ab660 0x7f87900adb00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:00.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.653+0000 7f87a57fa700 1 -- 172.21.15.2:0/1038049806 <== osd.6 v2:172.21.15.2:6818/3548993414 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f87900ae1d0 con 0x7f87900ab660 2023-09-13T04:39:00.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.670+0000 7f87ae800700 1 -- 172.21.15.2:0/1038049806 --> [v2:172.21.15.2:6818/3548993414,v1:172.21.15.2:6819/3548993414] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f87a8110f50 con 0x7f87900ab660 2023-09-13T04:39:00.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.670+0000 7f87a57fa700 1 -- 172.21.15.2:0/1038049806 <== osd.6 v2:172.21.15.2:6818/3548993414 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f87a000f040 con 0x7f87900ab660 2023-09-13T04:39:00.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.670+0000 7f87ae800700 1 -- 172.21.15.2:0/1038049806 >> [v2:172.21.15.2:6818/3548993414,v1:172.21.15.2:6819/3548993414] conn(0x7f87900ab660 msgr2=0x7f87900adb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:00.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.670+0000 7f87ae800700 1 --2- 172.21.15.2:0/1038049806 >> [v2:172.21.15.2:6818/3548993414,v1:172.21.15.2:6819/3548993414] conn(0x7f87900ab660 0x7f87900adb00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:00.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.670+0000 7f87ae800700 1 -- 172.21.15.2:0/1038049806 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f87900a0640 msgr2=0x7f87900a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:00.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.670+0000 7f87ae800700 1 --2- 172.21.15.2:0/1038049806 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f87900a0640 0x7f87900a2b00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f879c009fd0 tx=0x7f879c005fb0 comp rx=0 tx=0).stop 2023-09-13T04:39:00.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.671+0000 7f87ae800700 1 -- 172.21.15.2:0/1038049806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87a8100d80 msgr2=0x7f87a8196e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:00.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.671+0000 7f87ae800700 1 --2- 172.21.15.2:0/1038049806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87a8100d80 0x7f87a8196e10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f879800e9a0 tx=0x7f879800ecb0 comp rx=0 tx=0).stop 2023-09-13T04:39:00.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.671+0000 7f87acd9d700 1 -- 172.21.15.2:0/1038049806 reap_dead start 2023-09-13T04:39:00.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.671+0000 7f87ae800700 1 -- 172.21.15.2:0/1038049806 shutdown_connections 2023-09-13T04:39:00.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.671+0000 7f87ae800700 1 -- 172.21.15.2:0/1038049806 >> 172.21.15.2:0/1038049806 conn(0x7f87a8077ea0 msgr2=0x7f87a80ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:00.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.671+0000 7f87ae800700 1 -- 172.21.15.2:0/1038049806 shutdown_connections 2023-09-13T04:39:00.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.671+0000 7f87ae800700 1 -- 172.21.15.2:0/1038049806 wait complete. 2023-09-13T04:39:00.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:00 smithi156 ceph-mon[133675]: pgmap v308: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-09-13T04:39:00.888 INFO:teuthology.orchestra.run.smithi002.stdout:249108103205 2023-09-13T04:39:00.889 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph osd last-stat-seq osd.8 2023-09-13T04:39:00.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.230+0000 7f8cd6798700 1 -- 172.21.15.2:0/3845759967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8cd0073420 msgr2=0x7f8cd0073880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:00.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.230+0000 7f8cd6798700 1 --2- 172.21.15.2:0/3845759967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8cd0073420 0x7f8cd0073880 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009990 tx=0x7f8cc4009ca0 comp rx=0 tx=0).stop 2023-09-13T04:39:00.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.230+0000 7f8cd6798700 1 -- 172.21.15.2:0/3845759967 shutdown_connections 2023-09-13T04:39:00.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.230+0000 7f8cd6798700 1 --2- 172.21.15.2:0/3845759967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cd01119c0 0x7f8cd0113db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:00.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.230+0000 7f8cd6798700 1 --2- 172.21.15.2:0/3845759967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cd0074cf0 0x7f8cd0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:00.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.230+0000 7f8cd6798700 1 --2- 172.21.15.2:0/3845759967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8cd0073420 0x7f8cd0073880 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:00.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.230+0000 7f8cd6798700 1 -- 172.21.15.2:0/3845759967 >> 172.21.15.2:0/3845759967 conn(0x7f8cd0077ea0 msgr2=0x7f8cd00782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:00.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.230+0000 7f8cd6798700 1 -- 172.21.15.2:0/3845759967 shutdown_connections 2023-09-13T04:39:00.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.230+0000 7f8cd6798700 1 -- 172.21.15.2:0/3845759967 wait complete. 2023-09-13T04:39:00.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.231+0000 7f8cd6798700 1 Processor -- start 2023-09-13T04:39:00.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.231+0000 7f8cd6798700 1 -- start start 2023-09-13T04:39:00.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.231+0000 7f8cd6798700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cd0073420 0x7f8cd019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:00.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.231+0000 7f8cd6798700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cd0074cf0 0x7f8cd019d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:00.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.231+0000 7f8cd6798700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8cd01119c0 0x7f8cd01a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:00.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.231+0000 7f8cd6798700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd01a23e0 con 0x7f8cd01119c0 2023-09-13T04:39:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.231+0000 7f8cd6798700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd01a2550 con 0x7f8cd0073420 2023-09-13T04:39:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.231+0000 7f8cd6798700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd01a2710 con 0x7f8cd0074cf0 2023-09-13T04:39:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.232+0000 7f8ccffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cd0073420 0x7f8cd019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.232+0000 7f8ccffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cd0073420 0x7f8cd019d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:55510/0 (socket says 172.21.15.2:55510) 2023-09-13T04:39:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.232+0000 7f8cd4d35700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8cd01119c0 0x7f8cd01a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:00.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.232+0000 7f8ccffff700 1 -- 172.21.15.2:0/1895221935 learned_addr learned my addr 172.21.15.2:0/1895221935 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:00.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.232+0000 7f8ccf7fe700 1 --2- 172.21.15.2:0/1895221935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cd0074cf0 0x7f8cd019d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:00.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.232+0000 7f8ccffff700 1 -- 172.21.15.2:0/1895221935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cd0074cf0 msgr2=0x7f8cd019d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:00.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.232+0000 7f8ccffff700 1 --2- 172.21.15.2:0/1895221935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cd0074cf0 0x7f8cd019d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:00.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.232+0000 7f8ccffff700 1 -- 172.21.15.2:0/1895221935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8cd01119c0 msgr2=0x7f8cd01a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:00.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.232+0000 7f8ccffff700 1 --2- 172.21.15.2:0/1895221935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8cd01119c0 0x7f8cd01a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:00.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.232+0000 7f8ccffff700 1 -- 172.21.15.2:0/1895221935 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc40095f0 con 0x7f8cd0073420 2023-09-13T04:39:00.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.232+0000 7f8cd4d35700 1 --2- 172.21.15.2:0/1895221935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8cd01119c0 0x7f8cd01a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:39:00.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.232+0000 7f8ccffff700 1 --2- 172.21.15.2:0/1895221935 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cd0073420 0x7f8cd019d470 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000e9a0 tx=0x7f8cc000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:00.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.233+0000 7f8ccd7fa700 1 -- 172.21.15.2:0/1895221935 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0009790 con 0x7f8cd0073420 2023-09-13T04:39:00.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.233+0000 7f8cd6798700 1 -- 172.21.15.2:0/1895221935 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd01a29f0 con 0x7f8cd0073420 2023-09-13T04:39:00.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.233+0000 7f8ccd7fa700 1 -- 172.21.15.2:0/1895221935 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cc0010430 con 0x7f8cd0073420 2023-09-13T04:39:00.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.233+0000 7f8ccd7fa700 1 -- 172.21.15.2:0/1895221935 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0018680 con 0x7f8cd0073420 2023-09-13T04:39:00.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.233+0000 7f8cd6798700 1 -- 172.21.15.2:0/1895221935 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd01aa250 con 0x7f8cd0073420 2023-09-13T04:39:00.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.233+0000 7f8cd6798700 1 -- 172.21.15.2:0/1895221935 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8cd004e9b0 con 0x7f8cd0073420 2023-09-13T04:39:00.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.236+0000 7f8ccd7fa700 1 -- 172.21.15.2:0/1895221935 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f8cc00105a0 con 0x7f8cd0073420 2023-09-13T04:39:00.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.237+0000 7f8ccd7fa700 1 --2- 172.21.15.2:0/1895221935 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8cb80a0640 0x7f8cb80a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:00.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.238+0000 7f8ccd7fa700 1 -- 172.21.15.2:0/1895221935 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8cc0014070 con 0x7f8cd0073420 2023-09-13T04:39:00.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.238+0000 7f8ccf7fe700 1 --2- 172.21.15.2:0/1895221935 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8cb80a0640 0x7f8cb80a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:00.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.238+0000 7f8ccd7fa700 1 --2- 172.21.15.2:0/1895221935 >> [v2:172.21.15.156:6824/215467467,v1:172.21.15.156:6825/215467467] conn(0x7f8cb80ab660 0x7f8cb80adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:00.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.238+0000 7f8ccd7fa700 1 -- 172.21.15.2:0/1895221935 --> [v2:172.21.15.156:6824/215467467,v1:172.21.15.156:6825/215467467] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8cb80ae1d0 con 0x7f8cb80ab660 2023-09-13T04:39:00.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.238+0000 7f8ccd7fa700 1 -- 172.21.15.2:0/1895221935 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cc00cc720 con 0x7f8cd0073420 2023-09-13T04:39:00.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.238+0000 7f8ccf7fe700 1 --2- 172.21.15.2:0/1895221935 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8cb80a0640 0x7f8cb80a2b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400b2e0 tx=0x7f8cc4005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:00.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.238+0000 7f8cd4d35700 1 --2- 172.21.15.2:0/1895221935 >> [v2:172.21.15.156:6824/215467467,v1:172.21.15.156:6825/215467467] conn(0x7f8cb80ab660 0x7f8cb80adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:00.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.239+0000 7f8cd4d35700 1 --2- 172.21.15.2:0/1895221935 >> [v2:172.21.15.156:6824/215467467,v1:172.21.15.156:6825/215467467] conn(0x7f8cb80ab660 0x7f8cb80adb00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:00.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.240+0000 7f8ccd7fa700 1 -- 172.21.15.2:0/1895221935 <== osd.11 v2:172.21.15.156:6824/215467467 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f8cb80ae1d0 con 0x7f8cb80ab660 2023-09-13T04:39:00.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.260+0000 7f8cd6798700 1 -- 172.21.15.2:0/1895221935 --> [v2:172.21.15.156:6824/215467467,v1:172.21.15.156:6825/215467467] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8cd01175a0 con 0x7f8cb80ab660 2023-09-13T04:39:00.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.261+0000 7f8ccd7fa700 1 -- 172.21.15.2:0/1895221935 <== osd.11 v2:172.21.15.156:6824/215467467 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8cc800f040 con 0x7f8cb80ab660 2023-09-13T04:39:00.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.262+0000 7f8cd6798700 1 -- 172.21.15.2:0/1895221935 >> [v2:172.21.15.156:6824/215467467,v1:172.21.15.156:6825/215467467] conn(0x7f8cb80ab660 msgr2=0x7f8cb80adb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:00.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.262+0000 7f8cd6798700 1 --2- 172.21.15.2:0/1895221935 >> [v2:172.21.15.156:6824/215467467,v1:172.21.15.156:6825/215467467] conn(0x7f8cb80ab660 0x7f8cb80adb00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:00.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.262+0000 7f8cd6798700 1 -- 172.21.15.2:0/1895221935 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8cb80a0640 msgr2=0x7f8cb80a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:00.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.262+0000 7f8cd6798700 1 --2- 172.21.15.2:0/1895221935 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8cb80a0640 0x7f8cb80a2b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400b2e0 tx=0x7f8cc4005960 comp rx=0 tx=0).stop 2023-09-13T04:39:00.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.262+0000 7f8cd6798700 1 -- 172.21.15.2:0/1895221935 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cd0073420 msgr2=0x7f8cd019d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:00.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.262+0000 7f8cd6798700 1 --2- 172.21.15.2:0/1895221935 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cd0073420 0x7f8cd019d470 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000e9a0 tx=0x7f8cc000ecb0 comp rx=0 tx=0).stop 2023-09-13T04:39:00.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.262+0000 7f8cd4d35700 1 -- 172.21.15.2:0/1895221935 reap_dead start 2023-09-13T04:39:00.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.262+0000 7f8cd6798700 1 -- 172.21.15.2:0/1895221935 shutdown_connections 2023-09-13T04:39:00.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.262+0000 7f8cd6798700 1 -- 172.21.15.2:0/1895221935 >> 172.21.15.2:0/1895221935 conn(0x7f8cd0077ea0 msgr2=0x7f8cd0110520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:00.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.263+0000 7f8cd6798700 1 -- 172.21.15.2:0/1895221935 shutdown_connections 2023-09-13T04:39:00.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.263+0000 7f8cd6798700 1 -- 172.21.15.2:0/1895221935 wait complete. 2023-09-13T04:39:01.140 INFO:teuthology.orchestra.run.smithi002.stdout:77309411395 2023-09-13T04:39:01.141 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph osd last-stat-seq osd.2 2023-09-13T04:39:01.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.118+0000 7f255602d700 1 -- 172.21.15.2:0/3495839657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2550068a00 msgr2=0x7f2550068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:01.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.118+0000 7f255602d700 1 --2- 172.21.15.2:0/3495839657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2550068a00 0x7f2550068de0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2540009990 tx=0x7f2540009ca0 comp rx=0 tx=0).stop 2023-09-13T04:39:01.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.118+0000 7f255602d700 1 -- 172.21.15.2:0/3495839657 shutdown_connections 2023-09-13T04:39:01.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.118+0000 7f255602d700 1 --2- 172.21.15.2:0/3495839657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25501060b0 0x7f255010dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:01.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.118+0000 7f255602d700 1 --2- 172.21.15.2:0/3495839657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2550068a00 0x7f2550068de0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:01.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.118+0000 7f255602d700 1 --2- 172.21.15.2:0/3495839657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25500693b0 0x7f2550105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:01.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.118+0000 7f255602d700 1 -- 172.21.15.2:0/3495839657 >> 172.21.15.2:0/3495839657 conn(0x7f2550075c00 msgr2=0x7f2550076010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:01.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.118+0000 7f255602d700 1 -- 172.21.15.2:0/3495839657 shutdown_connections 2023-09-13T04:39:01.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.118+0000 7f255602d700 1 -- 172.21.15.2:0/3495839657 wait complete. 2023-09-13T04:39:01.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.119+0000 7f255602d700 1 Processor -- start 2023-09-13T04:39:01.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.120+0000 7f255602d700 1 -- start start 2023-09-13T04:39:01.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.120+0000 7f255602d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2550068a00 0x7f2550199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:01.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.120+0000 7f255602d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25500693b0 0x7f2550199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:01.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.120+0000 7f255602d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25501060b0 0x7f255019d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:01.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.120+0000 7f255602d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255019dff0 con 0x7f25500693b0 2023-09-13T04:39:01.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.120+0000 7f255602d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255019e160 con 0x7f2550068a00 2023-09-13T04:39:01.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.120+0000 7f255602d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255019e320 con 0x7f25501060b0 2023-09-13T04:39:01.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.120+0000 7f254effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25500693b0 0x7f2550199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:01.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.120+0000 7f254effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25500693b0 0x7f2550199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35836/0 (socket says 172.21.15.2:35836) 2023-09-13T04:39:01.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.120+0000 7f254effd700 1 -- 172.21.15.2:0/172705336 learned_addr learned my addr 172.21.15.2:0/172705336 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:01.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.120+0000 7f254f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2550068a00 0x7f2550199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:01.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.121+0000 7f254ffff700 1 --2- 172.21.15.2:0/172705336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25501060b0 0x7f255019d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:01.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.121+0000 7f254effd700 1 -- 172.21.15.2:0/172705336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25501060b0 msgr2=0x7f255019d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:01.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.121+0000 7f254effd700 1 --2- 172.21.15.2:0/172705336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25501060b0 0x7f255019d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:01.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.121+0000 7f254effd700 1 -- 172.21.15.2:0/172705336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2550068a00 msgr2=0x7f2550199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:01.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.121+0000 7f254effd700 1 --2- 172.21.15.2:0/172705336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2550068a00 0x7f2550199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:01.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.121+0000 7f254effd700 1 -- 172.21.15.2:0/172705336 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25400095f0 con 0x7f25500693b0 2023-09-13T04:39:01.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.121+0000 7f254f7fe700 1 --2- 172.21.15.2:0/172705336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2550068a00 0x7f2550199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:39:01.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.121+0000 7f254effd700 1 --2- 172.21.15.2:0/172705336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25500693b0 0x7f2550199570 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f254400e940 tx=0x7f254400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:01.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.121+0000 7f254cff9700 1 -- 172.21.15.2:0/172705336 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2544009730 con 0x7f25500693b0 2023-09-13T04:39:01.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.122+0000 7f255602d700 1 -- 172.21.15.2:0/172705336 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f255019e600 con 0x7f25500693b0 2023-09-13T04:39:01.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.121+0000 7f254cff9700 1 -- 172.21.15.2:0/172705336 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2544010430 con 0x7f25500693b0 2023-09-13T04:39:01.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.122+0000 7f254cff9700 1 -- 172.21.15.2:0/172705336 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254400cea0 con 0x7f25500693b0 2023-09-13T04:39:01.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.122+0000 7f255602d700 1 -- 172.21.15.2:0/172705336 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25501a5e60 con 0x7f25500693b0 2023-09-13T04:39:01.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.122+0000 7f255602d700 1 -- 172.21.15.2:0/172705336 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f255004e9b0 con 0x7f25500693b0 2023-09-13T04:39:01.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.125+0000 7f254cff9700 1 -- 172.21.15.2:0/172705336 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f2544010810 con 0x7f25500693b0 2023-09-13T04:39:01.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.126+0000 7f254cff9700 1 --2- 172.21.15.2:0/172705336 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f25380a0640 0x7f25380a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:01.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.126+0000 7f254cff9700 1 -- 172.21.15.2:0/172705336 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2544014070 con 0x7f25500693b0 2023-09-13T04:39:01.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.126+0000 7f254f7fe700 1 --2- 172.21.15.2:0/172705336 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f25380a0640 0x7f25380a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:01.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.126+0000 7f254cff9700 1 --2- 172.21.15.2:0/172705336 >> [v2:172.21.15.149:6808/3095161596,v1:172.21.15.149:6809/3095161596] conn(0x7f25380ab660 0x7f25380adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:01.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.126+0000 7f254cff9700 1 -- 172.21.15.2:0/172705336 --> [v2:172.21.15.149:6808/3095161596,v1:172.21.15.149:6809/3095161596] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f25380ae1d0 con 0x7f25380ab660 2023-09-13T04:39:01.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.126+0000 7f254cff9700 1 -- 172.21.15.2:0/172705336 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25440cc680 con 0x7f25500693b0 2023-09-13T04:39:01.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.127+0000 7f254ffff700 1 --2- 172.21.15.2:0/172705336 >> [v2:172.21.15.149:6808/3095161596,v1:172.21.15.149:6809/3095161596] conn(0x7f25380ab660 0x7f25380adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:01.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.127+0000 7f254f7fe700 1 --2- 172.21.15.2:0/172705336 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f25380a0640 0x7f25380a2b00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f254000b2e0 tx=0x7f2540005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:01.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.127+0000 7f254ffff700 1 --2- 172.21.15.2:0/172705336 >> [v2:172.21.15.149:6808/3095161596,v1:172.21.15.149:6809/3095161596] conn(0x7f25380ab660 0x7f25380adb00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:01.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.128+0000 7f254cff9700 1 -- 172.21.15.2:0/172705336 <== osd.4 v2:172.21.15.149:6808/3095161596 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f25380ae1d0 con 0x7f25380ab660 2023-09-13T04:39:01.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.147+0000 7f255602d700 1 -- 172.21.15.2:0/172705336 --> [v2:172.21.15.149:6808/3095161596,v1:172.21.15.149:6809/3095161596] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2550110f70 con 0x7f25380ab660 2023-09-13T04:39:01.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.148+0000 7f254cff9700 1 -- 172.21.15.2:0/172705336 <== osd.4 v2:172.21.15.149:6808/3095161596 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f254800f040 con 0x7f25380ab660 2023-09-13T04:39:01.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.148+0000 7f255602d700 1 -- 172.21.15.2:0/172705336 >> [v2:172.21.15.149:6808/3095161596,v1:172.21.15.149:6809/3095161596] conn(0x7f25380ab660 msgr2=0x7f25380adb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:01.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.148+0000 7f255602d700 1 --2- 172.21.15.2:0/172705336 >> [v2:172.21.15.149:6808/3095161596,v1:172.21.15.149:6809/3095161596] conn(0x7f25380ab660 0x7f25380adb00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:01.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.148+0000 7f255602d700 1 -- 172.21.15.2:0/172705336 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f25380a0640 msgr2=0x7f25380a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:01.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.149+0000 7f255602d700 1 --2- 172.21.15.2:0/172705336 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f25380a0640 0x7f25380a2b00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f254000b2e0 tx=0x7f2540005fd0 comp rx=0 tx=0).stop 2023-09-13T04:39:01.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.149+0000 7f255602d700 1 -- 172.21.15.2:0/172705336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25500693b0 msgr2=0x7f2550199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:01.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.149+0000 7f255602d700 1 --2- 172.21.15.2:0/172705336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25500693b0 0x7f2550199570 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f254400e940 tx=0x7f254400ec50 comp rx=0 tx=0).stop 2023-09-13T04:39:01.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.149+0000 7f254ffff700 1 -- 172.21.15.2:0/172705336 reap_dead start 2023-09-13T04:39:01.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.149+0000 7f255602d700 1 -- 172.21.15.2:0/172705336 shutdown_connections 2023-09-13T04:39:01.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.149+0000 7f255602d700 1 -- 172.21.15.2:0/172705336 >> 172.21.15.2:0/172705336 conn(0x7f2550075c00 msgr2=0x7f255010bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:01.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.149+0000 7f255602d700 1 -- 172.21.15.2:0/172705336 shutdown_connections 2023-09-13T04:39:01.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:00.149+0000 7f255602d700 1 -- 172.21.15.2:0/172705336 wait complete. 2023-09-13T04:39:01.201 INFO:teuthology.orchestra.run.smithi002.stdout:55834574916 2023-09-13T04:39:01.202 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph osd last-stat-seq osd.1 2023-09-13T04:39:01.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:01 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:39:01.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:01 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:39:01.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:01 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:39:02.050 INFO:teuthology.orchestra.run.smithi002.stdout:197568495662 2023-09-13T04:39:02.051 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph osd last-stat-seq osd.6 2023-09-13T04:39:02.144 INFO:teuthology.orchestra.run.smithi002.stdout:283467841569 2023-09-13T04:39:02.144 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph osd last-stat-seq osd.9 2023-09-13T04:39:02.207 INFO:teuthology.orchestra.run.smithi002.stdout:339302416408 2023-09-13T04:39:02.208 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph osd last-stat-seq osd.11 2023-09-13T04:39:02.352 INFO:teuthology.orchestra.run.smithi002.stdout:141733920826 2023-09-13T04:39:02.353 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph osd last-stat-seq osd.4 2023-09-13T04:39:02.431 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:39:02.566 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:39:02.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:02 smithi149 ceph-mon[133315]: pgmap v309: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-13T04:39:02.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:02 smithi156 ceph-mon[133675]: pgmap v309: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-13T04:39:02.966 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:39:03.000 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:02 smithi002 ceph-mon[97651]: pgmap v309: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-13T04:39:03.219 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:39:03.219 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:39:03.225 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:39:03.377 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:39:04.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:04 smithi002 ceph-mon[97651]: pgmap v310: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:39:04.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:04 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:39:04.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:04 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:39:04.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:04 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:39:04.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:04 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:39:04.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:04 smithi149 ceph-mon[133315]: pgmap v310: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:39:04.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:04 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:39:04.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:04 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:39:04.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:04 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:39:04.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:04 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:39:04.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:04 smithi156 ceph-mon[133675]: pgmap v310: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:39:04.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:04 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:39:04.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:04 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:39:04.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:04 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:39:04.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:04 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:39:06.422 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:39:06.612 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:06 smithi002 ceph-mon[97651]: pgmap v311: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:39:06.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:06 smithi149 ceph-mon[133315]: pgmap v311: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:39:06.923 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:39:06.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:06 smithi156 ceph-mon[133675]: pgmap v311: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:39:06.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.779+0000 7fa7c53ca700 1 -- 172.21.15.2:0/31664977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7c0106d00 msgr2=0x7fa7c00ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:06.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.779+0000 7fa7c53ca700 1 --2- 172.21.15.2:0/31664977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7c0106d00 0x7fa7c00ff140 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009990 tx=0x7fa7b0009ca0 comp rx=0 tx=0).stop 2023-09-13T04:39:06.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.780+0000 7fa7c53ca700 1 -- 172.21.15.2:0/31664977 shutdown_connections 2023-09-13T04:39:06.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.780+0000 7fa7c53ca700 1 --2- 172.21.15.2:0/31664977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7c0111a50 0x7fa7c0113e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:06.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.780+0000 7fa7c53ca700 1 --2- 172.21.15.2:0/31664977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7c0106d00 0x7fa7c00ff140 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:06.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.780+0000 7fa7c53ca700 1 --2- 172.21.15.2:0/31664977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7c00ff710 0x7fa7c00ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:06.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.780+0000 7fa7c53ca700 1 -- 172.21.15.2:0/31664977 >> 172.21.15.2:0/31664977 conn(0x7fa7c0075a90 msgr2=0x7fa7c0075ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:06.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.782+0000 7fa7c53ca700 1 -- 172.21.15.2:0/31664977 shutdown_connections 2023-09-13T04:39:06.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.787+0000 7fa7c53ca700 1 -- 172.21.15.2:0/31664977 wait complete. 2023-09-13T04:39:06.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.791+0000 7fa7c53ca700 1 Processor -- start 2023-09-13T04:39:06.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.792+0000 7fa7c53ca700 1 -- start start 2023-09-13T04:39:06.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.792+0000 7fa7c53ca700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7c00ff710 0x7fa7c010c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:06.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.792+0000 7fa7c53ca700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7c0106d00 0x7fa7c010cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:06.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.792+0000 7fa7c53ca700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7c0111a50 0x7fa7c0107a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:06.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.792+0000 7fa7c53ca700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c0107ff0 con 0x7fa7c0106d00 2023-09-13T04:39:06.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.792+0000 7fa7c53ca700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c0108130 con 0x7fa7c0111a50 2023-09-13T04:39:06.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.792+0000 7fa7c53ca700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c01082f0 con 0x7fa7c00ff710 2023-09-13T04:39:06.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.793+0000 7fa7be7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7c0106d00 0x7fa7c010cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:06.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.793+0000 7fa7be7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7c0106d00 0x7fa7c010cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36460/0 (socket says 172.21.15.2:36460) 2023-09-13T04:39:06.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.793+0000 7fa7be7fc700 1 -- 172.21.15.2:0/4276364741 learned_addr learned my addr 172.21.15.2:0/4276364741 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:06.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.793+0000 7fa7bf7fe700 1 --2- 172.21.15.2:0/4276364741 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7c0111a50 0x7fa7c0107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:06.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.793+0000 7fa7beffd700 1 --2- 172.21.15.2:0/4276364741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7c00ff710 0x7fa7c010c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:06.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.794+0000 7fa7bf7fe700 1 -- 172.21.15.2:0/4276364741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7c00ff710 msgr2=0x7fa7c010c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:06.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.794+0000 7fa7bf7fe700 1 --2- 172.21.15.2:0/4276364741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7c00ff710 0x7fa7c010c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:06.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.794+0000 7fa7bf7fe700 1 -- 172.21.15.2:0/4276364741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7c0106d00 msgr2=0x7fa7c010cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:06.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.794+0000 7fa7bf7fe700 1 --2- 172.21.15.2:0/4276364741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7c0106d00 0x7fa7c010cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:06.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.794+0000 7fa7bf7fe700 1 -- 172.21.15.2:0/4276364741 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b00095f0 con 0x7fa7c0111a50 2023-09-13T04:39:06.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.794+0000 7fa7bf7fe700 1 --2- 172.21.15.2:0/4276364741 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7c0111a50 0x7fa7c0107a20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800be20 tx=0x7fa7b8009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:06.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.794+0000 7fa7a7fff700 1 -- 172.21.15.2:0/4276364741 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b800cc10 con 0x7fa7c0111a50 2023-09-13T04:39:06.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.794+0000 7fa7a7fff700 1 -- 172.21.15.2:0/4276364741 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7b800cd70 con 0x7fa7c0111a50 2023-09-13T04:39:06.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.794+0000 7fa7a7fff700 1 -- 172.21.15.2:0/4276364741 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8012730 con 0x7fa7c0111a50 2023-09-13T04:39:06.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.795+0000 7fa7c53ca700 1 -- 172.21.15.2:0/4276364741 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c01085d0 con 0x7fa7c0111a50 2023-09-13T04:39:06.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.795+0000 7fa7c53ca700 1 -- 172.21.15.2:0/4276364741 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c010b760 con 0x7fa7c0111a50 2023-09-13T04:39:06.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.797+0000 7fa7c53ca700 1 -- 172.21.15.2:0/4276364741 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c004e9b0 con 0x7fa7c0111a50 2023-09-13T04:39:06.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.800+0000 7fa7a7fff700 1 -- 172.21.15.2:0/4276364741 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fa7b8014a90 con 0x7fa7c0111a50 2023-09-13T04:39:06.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.802+0000 7fa7a7fff700 1 --2- 172.21.15.2:0/4276364741 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fa7a80a0790 0x7fa7a80a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:06.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.802+0000 7fa7a7fff700 1 -- 172.21.15.2:0/4276364741 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa7b8006e40 con 0x7fa7c0111a50 2023-09-13T04:39:06.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.803+0000 7fa7beffd700 1 --2- 172.21.15.2:0/4276364741 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fa7a80a0790 0x7fa7a80a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:06.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.803+0000 7fa7a7fff700 1 -- 172.21.15.2:0/4276364741 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa7b80954b0 con 0x7fa7c0111a50 2023-09-13T04:39:06.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:06.804+0000 7fa7beffd700 1 --2- 172.21.15.2:0/4276364741 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fa7a80a0790 0x7fa7a80a2c50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000b5c0 tx=0x7fa7b00058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:07.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:07.069+0000 7fa7c53ca700 1 -- 172.21.15.2:0/4276364741 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fa7c0108760 con 0x7fa7c0111a50 2023-09-13T04:39:07.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:07.070+0000 7fa7a7fff700 1 -- 172.21.15.2:0/4276364741 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fa7b8019090 con 0x7fa7c0111a50 2023-09-13T04:39:07.071 INFO:teuthology.orchestra.run.smithi002.stdout:304942678046 2023-09-13T04:39:07.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:07.072+0000 7fa7c53ca700 1 -- 172.21.15.2:0/4276364741 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fa7a80a0790 msgr2=0x7fa7a80a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:07.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:07.072+0000 7fa7c53ca700 1 --2- 172.21.15.2:0/4276364741 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fa7a80a0790 0x7fa7a80a2c50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000b5c0 tx=0x7fa7b00058e0 comp rx=0 tx=0).stop 2023-09-13T04:39:07.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:07.072+0000 7fa7c53ca700 1 -- 172.21.15.2:0/4276364741 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7c0111a50 msgr2=0x7fa7c0107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:07.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:07.072+0000 7fa7c53ca700 1 --2- 172.21.15.2:0/4276364741 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7c0111a50 0x7fa7c0107a20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800be20 tx=0x7fa7b8009ed0 comp rx=0 tx=0).stop 2023-09-13T04:39:07.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:07.072+0000 7fa7c53ca700 1 -- 172.21.15.2:0/4276364741 shutdown_connections 2023-09-13T04:39:07.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:07.072+0000 7fa7c53ca700 1 --2- 172.21.15.2:0/4276364741 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fa7a80a0790 0x7fa7a80a2c50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:07.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:07.072+0000 7fa7c53ca700 1 --2- 172.21.15.2:0/4276364741 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7c0111a50 0x7fa7c0107a20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:07.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:07.072+0000 7fa7c53ca700 1 --2- 172.21.15.2:0/4276364741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7c00ff710 0x7fa7c010c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:07.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:07.072+0000 7fa7c53ca700 1 --2- 172.21.15.2:0/4276364741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7c0106d00 0x7fa7c010cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:07.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:07.072+0000 7fa7c53ca700 1 -- 172.21.15.2:0/4276364741 >> 172.21.15.2:0/4276364741 conn(0x7fa7c0075a90 msgr2=0x7fa7c00fc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:07.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:07.072+0000 7fa7c53ca700 1 -- 172.21.15.2:0/4276364741 shutdown_connections 2023-09-13T04:39:07.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:07.072+0000 7fa7c53ca700 1 -- 172.21.15.2:0/4276364741 wait complete. 2023-09-13T04:39:07.152 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:39:07.369 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:39:07.625 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:07 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/4276364741' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-13T04:39:07.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:07 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/4276364741' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-13T04:39:07.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:07 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/4276364741' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-13T04:39:08.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.416+0000 7fdae4d67700 1 -- 172.21.15.2:0/3065258728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdae0072ee0 msgr2=0x7fdae00732c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:08.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.416+0000 7fdae4d67700 1 --2- 172.21.15.2:0/3065258728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdae0072ee0 0x7fdae00732c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009990 tx=0x7fdad0009ca0 comp rx=0 tx=0).stop 2023-09-13T04:39:08.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.416+0000 7fdae4d67700 1 -- 172.21.15.2:0/3065258728 shutdown_connections 2023-09-13T04:39:08.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.416+0000 7fdae4d67700 1 --2- 172.21.15.2:0/3065258728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdae010d730 0x7fdae010fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:08.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.416+0000 7fdae4d67700 1 --2- 172.21.15.2:0/3065258728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdae0072ee0 0x7fdae00732c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:08.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.416+0000 7fdae4d67700 1 --2- 172.21.15.2:0/3065258728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdae0073800 0x7fdae010d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:08.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.416+0000 7fdae4d67700 1 -- 172.21.15.2:0/3065258728 >> 172.21.15.2:0/3065258728 conn(0x7fdae0077ea0 msgr2=0x7fdae00782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:08.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.416+0000 7fdae4d67700 1 -- 172.21.15.2:0/3065258728 shutdown_connections 2023-09-13T04:39:08.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.416+0000 7fdae4d67700 1 -- 172.21.15.2:0/3065258728 wait complete. 2023-09-13T04:39:08.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.418+0000 7fdae4d67700 1 Processor -- start 2023-09-13T04:39:08.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.418+0000 7fdae4d67700 1 -- start start 2023-09-13T04:39:08.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.419+0000 7fdae4d67700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdae0072ee0 0x7fdae01990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:08.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.419+0000 7fdae4d67700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdae0073800 0x7fdae01995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:08.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.419+0000 7fdae4d67700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdae010d730 0x7fdae019da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:08.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.419+0000 7fdae4d67700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae019e070 con 0x7fdae010d730 2023-09-13T04:39:08.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.419+0000 7fdae4d67700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae019e1e0 con 0x7fdae0073800 2023-09-13T04:39:08.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.419+0000 7fdae4d67700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae019e3a0 con 0x7fdae0072ee0 2023-09-13T04:39:08.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.419+0000 7fdade59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdae0072ee0 0x7fdae01990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:08.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.419+0000 7fdade59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdae0072ee0 0x7fdae01990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:50740/0 (socket says 172.21.15.2:50740) 2023-09-13T04:39:08.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.419+0000 7fdade59c700 1 -- 172.21.15.2:0/3252926105 learned_addr learned my addr 172.21.15.2:0/3252926105 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:08.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.419+0000 7fdaded9d700 1 --2- 172.21.15.2:0/3252926105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdae010d730 0x7fdae019da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:08.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.419+0000 7fdaddd9b700 1 --2- 172.21.15.2:0/3252926105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdae0073800 0x7fdae01995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:08.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.420+0000 7fdade59c700 1 -- 172.21.15.2:0/3252926105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdae0073800 msgr2=0x7fdae01995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:08.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.420+0000 7fdade59c700 1 --2- 172.21.15.2:0/3252926105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdae0073800 0x7fdae01995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:08.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.420+0000 7fdade59c700 1 -- 172.21.15.2:0/3252926105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdae010d730 msgr2=0x7fdae019da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:08.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.420+0000 7fdade59c700 1 --2- 172.21.15.2:0/3252926105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdae010d730 0x7fdae019da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:08.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.420+0000 7fdade59c700 1 -- 172.21.15.2:0/3252926105 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad00095f0 con 0x7fdae0072ee0 2023-09-13T04:39:08.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.420+0000 7fdaddd9b700 1 --2- 172.21.15.2:0/3252926105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdae0073800 0x7fdae01995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:39:08.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.420+0000 7fdaded9d700 1 --2- 172.21.15.2:0/3252926105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdae010d730 0x7fdae019da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:39:08.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.420+0000 7fdade59c700 1 --2- 172.21.15.2:0/3252926105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdae0072ee0 0x7fdae01990b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdad0005910 tx=0x7fdad0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:08.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.420+0000 7fdacf7fe700 1 -- 172.21.15.2:0/3252926105 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad001d070 con 0x7fdae0072ee0 2023-09-13T04:39:08.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.420+0000 7fdae4d67700 1 -- 172.21.15.2:0/3252926105 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae019e620 con 0x7fdae0072ee0 2023-09-13T04:39:08.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.420+0000 7fdacf7fe700 1 -- 172.21.15.2:0/3252926105 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdad0022470 con 0x7fdae0072ee0 2023-09-13T04:39:08.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.420+0000 7fdacf7fe700 1 -- 172.21.15.2:0/3252926105 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad0017650 con 0x7fdae0072ee0 2023-09-13T04:39:08.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.421+0000 7fdae4d67700 1 -- 172.21.15.2:0/3252926105 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae01a5ee0 con 0x7fdae0072ee0 2023-09-13T04:39:08.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.422+0000 7fdae4d67700 1 -- 172.21.15.2:0/3252926105 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdae004e9b0 con 0x7fdae0072ee0 2023-09-13T04:39:08.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.424+0000 7fdacf7fe700 1 -- 172.21.15.2:0/3252926105 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fdad000fad0 con 0x7fdae0072ee0 2023-09-13T04:39:08.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.426+0000 7fdacf7fe700 1 --2- 172.21.15.2:0/3252926105 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fdac80a0640 0x7fdac80a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:08.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.426+0000 7fdacf7fe700 1 -- 172.21.15.2:0/3252926105 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdad00cdc20 con 0x7fdae0072ee0 2023-09-13T04:39:08.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.426+0000 7fdaddd9b700 1 --2- 172.21.15.2:0/3252926105 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fdac80a0640 0x7fdac80a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:08.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.427+0000 7fdaddd9b700 1 --2- 172.21.15.2:0/3252926105 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fdac80a0640 0x7fdac80a2b00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdae019a6d0 tx=0x7fdad40073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:08.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.427+0000 7fdacf7fe700 1 -- 172.21.15.2:0/3252926105 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fdad00cde60 con 0x7fdae0072ee0 2023-09-13T04:39:08.597 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:08 smithi002 ceph-mon[97651]: pgmap v312: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:39:08.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.684+0000 7fdae4d67700 1 -- 172.21.15.2:0/3252926105 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fdae0112ee0 con 0x7fdae0072ee0 2023-09-13T04:39:08.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.684+0000 7fdacf7fe700 1 -- 172.21.15.2:0/3252926105 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdad0097470 con 0x7fdae0072ee0 2023-09-13T04:39:08.686 INFO:teuthology.orchestra.run.smithi002.stdout:163208757302 2023-09-13T04:39:08.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.687+0000 7fdae4d67700 1 -- 172.21.15.2:0/3252926105 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fdac80a0640 msgr2=0x7fdac80a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:08.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.687+0000 7fdae4d67700 1 --2- 172.21.15.2:0/3252926105 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fdac80a0640 0x7fdac80a2b00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdae019a6d0 tx=0x7fdad40073d0 comp rx=0 tx=0).stop 2023-09-13T04:39:08.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.687+0000 7fdae4d67700 1 -- 172.21.15.2:0/3252926105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdae0072ee0 msgr2=0x7fdae01990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:08.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.687+0000 7fdae4d67700 1 --2- 172.21.15.2:0/3252926105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdae0072ee0 0x7fdae01990b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdad0005910 tx=0x7fdad0005740 comp rx=0 tx=0).stop 2023-09-13T04:39:08.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.687+0000 7fdae4d67700 1 -- 172.21.15.2:0/3252926105 shutdown_connections 2023-09-13T04:39:08.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.687+0000 7fdae4d67700 1 --2- 172.21.15.2:0/3252926105 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fdac80a0640 0x7fdac80a2b00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:08.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.687+0000 7fdae4d67700 1 --2- 172.21.15.2:0/3252926105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdae010d730 0x7fdae019da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:08.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.687+0000 7fdae4d67700 1 --2- 172.21.15.2:0/3252926105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdae0072ee0 0x7fdae01990b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:08.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.687+0000 7fdae4d67700 1 --2- 172.21.15.2:0/3252926105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdae0073800 0x7fdae01995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:08.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.687+0000 7fdae4d67700 1 -- 172.21.15.2:0/3252926105 >> 172.21.15.2:0/3252926105 conn(0x7fdae0077ea0 msgr2=0x7fdae00fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:08.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.687+0000 7fdae4d67700 1 -- 172.21.15.2:0/3252926105 shutdown_connections 2023-09-13T04:39:08.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.687+0000 7fdae4d67700 1 -- 172.21.15.2:0/3252926105 wait complete. 2023-09-13T04:39:08.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.755+0000 7fd7d8288700 1 -- 172.21.15.2:0/3656768656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd7d0106d90 msgr2=0x7fd7d0107170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:08.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.755+0000 7fd7d8288700 1 --2- 172.21.15.2:0/3656768656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd7d0106d90 0x7fd7d0107170 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0099c0 tx=0x7fd7cc009cd0 comp rx=0 tx=0).stop 2023-09-13T04:39:08.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.755+0000 7fd7d8288700 1 -- 172.21.15.2:0/3656768656 shutdown_connections 2023-09-13T04:39:08.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.755+0000 7fd7d8288700 1 --2- 172.21.15.2:0/3656768656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7d0101740 0x7fd7d010dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:08.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.755+0000 7fd7d8288700 1 --2- 172.21.15.2:0/3656768656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd7d0106d90 0x7fd7d0107170 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:08.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.755+0000 7fd7d8288700 1 --2- 172.21.15.2:0/3656768656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d0100d80 0x7fd7d0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:08.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.755+0000 7fd7d8288700 1 -- 172.21.15.2:0/3656768656 >> 172.21.15.2:0/3656768656 conn(0x7fd7d0077ea0 msgr2=0x7fd7d00782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:08.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.755+0000 7fd7d8288700 1 -- 172.21.15.2:0/3656768656 shutdown_connections 2023-09-13T04:39:08.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.755+0000 7fd7d8288700 1 -- 172.21.15.2:0/3656768656 wait complete. 2023-09-13T04:39:08.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.756+0000 7fd7d8288700 1 Processor -- start 2023-09-13T04:39:08.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.756+0000 7fd7d8288700 1 -- start start 2023-09-13T04:39:08.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.757+0000 7fd7d8288700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd7d0100d80 0x7fd7d0072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:08.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.757+0000 7fd7d8288700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7d0101740 0x7fd7d00734f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:08.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.757+0000 7fd7d8288700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d0106d90 0x7fd7d0073a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:08.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.757+0000 7fd7d8288700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d01a51e0 con 0x7fd7d0100d80 2023-09-13T04:39:08.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.757+0000 7fd7d8288700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d01a5350 con 0x7fd7d0101740 2023-09-13T04:39:08.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.757+0000 7fd7d8288700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d01a5510 con 0x7fd7d0106d90 2023-09-13T04:39:08.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.757+0000 7fd7d6024700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd7d0100d80 0x7fd7d0072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:08.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.757+0000 7fd7d6024700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd7d0100d80 0x7fd7d0072fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36510/0 (socket says 172.21.15.2:36510) 2023-09-13T04:39:08.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.757+0000 7fd7d6024700 1 -- 172.21.15.2:0/1548383818 learned_addr learned my addr 172.21.15.2:0/1548383818 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:08.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.757+0000 7fd7d5823700 1 --2- 172.21.15.2:0/1548383818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7d0101740 0x7fd7d00734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:08.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.757+0000 7fd7d6825700 1 --2- 172.21.15.2:0/1548383818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d0106d90 0x7fd7d0073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:08.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.757+0000 7fd7d6024700 1 -- 172.21.15.2:0/1548383818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d0106d90 msgr2=0x7fd7d0073a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:08.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.757+0000 7fd7d6024700 1 --2- 172.21.15.2:0/1548383818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d0106d90 0x7fd7d0073a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:08.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.757+0000 7fd7d6024700 1 -- 172.21.15.2:0/1548383818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7d0101740 msgr2=0x7fd7d00734f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:08.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.757+0000 7fd7d6024700 1 --2- 172.21.15.2:0/1548383818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7d0101740 0x7fd7d00734f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:08.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.758+0000 7fd7d6024700 1 -- 172.21.15.2:0/1548383818 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc0095f0 con 0x7fd7d0100d80 2023-09-13T04:39:08.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.758+0000 7fd7d6024700 1 --2- 172.21.15.2:0/1548383818 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd7d0100d80 0x7fd7d0072fb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0098e0 tx=0x7fd7cc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:08.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.758+0000 7fd7c77fe700 1 -- 172.21.15.2:0/1548383818 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc01d070 con 0x7fd7d0100d80 2023-09-13T04:39:08.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.758+0000 7fd7c77fe700 1 -- 172.21.15.2:0/1548383818 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7cc022470 con 0x7fd7d0100d80 2023-09-13T04:39:08.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.758+0000 7fd7d8288700 1 -- 172.21.15.2:0/1548383818 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d01a5790 con 0x7fd7d0100d80 2023-09-13T04:39:08.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.758+0000 7fd7c77fe700 1 -- 172.21.15.2:0/1548383818 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc017650 con 0x7fd7d0100d80 2023-09-13T04:39:08.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.758+0000 7fd7d8288700 1 -- 172.21.15.2:0/1548383818 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d01a5ba0 con 0x7fd7d0100d80 2023-09-13T04:39:08.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.759+0000 7fd7d8288700 1 -- 172.21.15.2:0/1548383818 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d004e9b0 con 0x7fd7d0100d80 2023-09-13T04:39:08.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.761+0000 7fd7c77fe700 1 -- 172.21.15.2:0/1548383818 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fd7cc022850 con 0x7fd7d0100d80 2023-09-13T04:39:08.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.762+0000 7fd7c77fe700 1 --2- 172.21.15.2:0/1548383818 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd7bc0a0640 0x7fd7bc0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:08.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.763+0000 7fd7c77fe700 1 -- 172.21.15.2:0/1548383818 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd7cc0ccac0 con 0x7fd7d0100d80 2023-09-13T04:39:08.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.763+0000 7fd7d5823700 1 --2- 172.21.15.2:0/1548383818 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd7bc0a0640 0x7fd7bc0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:08.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.763+0000 7fd7d5823700 1 --2- 172.21.15.2:0/1548383818 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd7bc0a0640 0x7fd7bc0a2b00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00747a0 tx=0x7fd7c0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:08.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.766+0000 7fd7c77fe700 1 -- 172.21.15.2:0/1548383818 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd7cc096310 con 0x7fd7d0100d80 2023-09-13T04:39:08.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:08 smithi149 ceph-mon[133315]: pgmap v312: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:39:08.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:08 smithi156 ceph-mon[133675]: pgmap v312: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:39:08.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.995+0000 7fd7d8288700 1 -- 172.21.15.2:0/1548383818 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fd7d0110ed0 con 0x7fd7d0100d80 2023-09-13T04:39:08.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:08.997+0000 7fd7c77fe700 1 -- 172.21.15.2:0/1548383818 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd7cc099930 con 0x7fd7d0100d80 2023-09-13T04:39:08.999 INFO:teuthology.orchestra.run.smithi002.stdout:249108103207 2023-09-13T04:39:09.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.002+0000 7fd7d8288700 1 -- 172.21.15.2:0/1548383818 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd7bc0a0640 msgr2=0x7fd7bc0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:09.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.002+0000 7fd7d8288700 1 --2- 172.21.15.2:0/1548383818 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd7bc0a0640 0x7fd7bc0a2b00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00747a0 tx=0x7fd7c0009260 comp rx=0 tx=0).stop 2023-09-13T04:39:09.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.003+0000 7fd7d8288700 1 -- 172.21.15.2:0/1548383818 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd7d0100d80 msgr2=0x7fd7d0072fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:09.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.003+0000 7fd7d8288700 1 --2- 172.21.15.2:0/1548383818 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd7d0100d80 0x7fd7d0072fb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0098e0 tx=0x7fd7cc00f690 comp rx=0 tx=0).stop 2023-09-13T04:39:09.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.004+0000 7fd7d8288700 1 -- 172.21.15.2:0/1548383818 shutdown_connections 2023-09-13T04:39:09.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.004+0000 7fd7d8288700 1 --2- 172.21.15.2:0/1548383818 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd7bc0a0640 0x7fd7bc0a2b00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:09.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.004+0000 7fd7d8288700 1 --2- 172.21.15.2:0/1548383818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d0106d90 0x7fd7d0073a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:09.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.004+0000 7fd7d8288700 1 --2- 172.21.15.2:0/1548383818 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd7d0100d80 0x7fd7d0072fb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:09.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.004+0000 7fd7d8288700 1 --2- 172.21.15.2:0/1548383818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7d0101740 0x7fd7d00734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:09.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.004+0000 7fd7d8288700 1 -- 172.21.15.2:0/1548383818 >> 172.21.15.2:0/1548383818 conn(0x7fd7d0077ea0 msgr2=0x7fd7d00ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:09.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.004+0000 7fd7d8288700 1 -- 172.21.15.2:0/1548383818 shutdown_connections 2023-09-13T04:39:09.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.004+0000 7fd7d8288700 1 -- 172.21.15.2:0/1548383818 wait complete. 2023-09-13T04:39:09.041 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678044 got 304942678046 for osd.10 2023-09-13T04:39:09.041 DEBUG:teuthology.parallel:result is None 2023-09-13T04:39:09.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:09 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3252926105' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-13T04:39:09.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:09 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1548383818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-13T04:39:09.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:09 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3252926105' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-13T04:39:09.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:09 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1548383818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-13T04:39:10.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:09 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3252926105' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-13T04:39:10.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:09 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1548383818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-13T04:39:10.102 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:39:10.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.313+0000 7f1c3a212700 1 -- 172.21.15.2:0/2611712511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c34100d80 msgr2=0x7f1c34101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:10.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.313+0000 7f1c3a212700 1 --2- 172.21.15.2:0/2611712511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c34100d80 0x7f1c34101200 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1c280099c0 tx=0x7f1c28009cd0 comp rx=0 tx=0).stop 2023-09-13T04:39:10.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.313+0000 7f1c3a212700 1 -- 172.21.15.2:0/2611712511 shutdown_connections 2023-09-13T04:39:10.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.313+0000 7f1c3a212700 1 --2- 172.21.15.2:0/2611712511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c34101740 0x7f1c3410dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:10.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.313+0000 7f1c3a212700 1 --2- 172.21.15.2:0/2611712511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c34106d90 0x7f1c34107170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:10.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.313+0000 7f1c3a212700 1 --2- 172.21.15.2:0/2611712511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c34100d80 0x7f1c34101200 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:10.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.313+0000 7f1c3a212700 1 -- 172.21.15.2:0/2611712511 >> 172.21.15.2:0/2611712511 conn(0x7f1c34077ea0 msgr2=0x7f1c340782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:10.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.313+0000 7f1c3a212700 1 -- 172.21.15.2:0/2611712511 shutdown_connections 2023-09-13T04:39:10.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.313+0000 7f1c3a212700 1 -- 172.21.15.2:0/2611712511 wait complete. 2023-09-13T04:39:10.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.314+0000 7f1c3a212700 1 Processor -- start 2023-09-13T04:39:10.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.314+0000 7f1c3a212700 1 -- start start 2023-09-13T04:39:10.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.315+0000 7f1c3a212700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c34100d80 0x7f1c34196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:10.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.315+0000 7f1c3a212700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c34101740 0x7f1c341973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:10.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.315+0000 7f1c3a212700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c34106d90 0x7f1c3419b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:10.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.315+0000 7f1c3a212700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3419be40 con 0x7f1c34100d80 2023-09-13T04:39:10.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.315+0000 7f1c3a212700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3419bfb0 con 0x7f1c34101740 2023-09-13T04:39:10.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.315+0000 7f1c3a212700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3419c170 con 0x7f1c34106d90 2023-09-13T04:39:10.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.315+0000 7f1c337fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c34100d80 0x7f1c34196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:10.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.315+0000 7f1c337fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c34100d80 0x7f1c34196e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36528/0 (socket says 172.21.15.2:36528) 2023-09-13T04:39:10.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.315+0000 7f1c337fe700 1 -- 172.21.15.2:0/2192311155 learned_addr learned my addr 172.21.15.2:0/2192311155 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:10.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.315+0000 7f1c32ffd700 1 --2- 172.21.15.2:0/2192311155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c34101740 0x7f1c341973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:10.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.315+0000 7f1c33fff700 1 --2- 172.21.15.2:0/2192311155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c34106d90 0x7f1c3419b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:10.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.315+0000 7f1c337fe700 1 -- 172.21.15.2:0/2192311155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c34106d90 msgr2=0x7f1c3419b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:10.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.315+0000 7f1c337fe700 1 --2- 172.21.15.2:0/2192311155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c34106d90 0x7f1c3419b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:10.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.315+0000 7f1c337fe700 1 -- 172.21.15.2:0/2192311155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c34101740 msgr2=0x7f1c341973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:10.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.315+0000 7f1c337fe700 1 --2- 172.21.15.2:0/2192311155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c34101740 0x7f1c341973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:10.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.315+0000 7f1c337fe700 1 -- 172.21.15.2:0/2192311155 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2c009c40 con 0x7f1c34100d80 2023-09-13T04:39:10.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.316+0000 7f1c33fff700 1 --2- 172.21.15.2:0/2192311155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c34106d90 0x7f1c3419b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:39:10.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.316+0000 7f1c32ffd700 1 --2- 172.21.15.2:0/2192311155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c34101740 0x7f1c341973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:39:10.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.316+0000 7f1c337fe700 1 --2- 172.21.15.2:0/2192311155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c34100d80 0x7f1c34196e80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400e9a0 tx=0x7f1c2400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:10.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.316+0000 7f1c30ff9700 1 -- 172.21.15.2:0/2192311155 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c24009790 con 0x7f1c34100d80 2023-09-13T04:39:10.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.316+0000 7f1c30ff9700 1 -- 172.21.15.2:0/2192311155 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c2400cbd0 con 0x7f1c34100d80 2023-09-13T04:39:10.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.316+0000 7f1c30ff9700 1 -- 172.21.15.2:0/2192311155 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c24018680 con 0x7f1c34100d80 2023-09-13T04:39:10.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.316+0000 7f1c3a212700 1 -- 172.21.15.2:0/2192311155 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c280095f0 con 0x7f1c34100d80 2023-09-13T04:39:10.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.316+0000 7f1c3a212700 1 -- 172.21.15.2:0/2192311155 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c341a3ae0 con 0x7f1c34100d80 2023-09-13T04:39:10.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.317+0000 7f1c3a212700 1 -- 172.21.15.2:0/2192311155 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c3404e9b0 con 0x7f1c34100d80 2023-09-13T04:39:10.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.319+0000 7f1c30ff9700 1 -- 172.21.15.2:0/2192311155 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f1c2400cd40 con 0x7f1c34100d80 2023-09-13T04:39:10.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.321+0000 7f1c30ff9700 1 --2- 172.21.15.2:0/2192311155 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1c1c0a06c0 0x7f1c1c0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:10.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.321+0000 7f1c30ff9700 1 -- 172.21.15.2:0/2192311155 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1c24014070 con 0x7f1c34100d80 2023-09-13T04:39:10.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.321+0000 7f1c32ffd700 1 --2- 172.21.15.2:0/2192311155 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1c1c0a06c0 0x7f1c1c0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:10.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.322+0000 7f1c32ffd700 1 --2- 172.21.15.2:0/2192311155 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1c1c0a06c0 0x7f1c1c0a2b80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1c28009990 tx=0x7f1c2801a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:10.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.325+0000 7f1c30ff9700 1 -- 172.21.15.2:0/2192311155 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1c24051e10 con 0x7f1c34100d80 2023-09-13T04:39:10.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.547+0000 7f1c3a212700 1 -- 172.21.15.2:0/2192311155 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f1c34110db0 con 0x7f1c34100d80 2023-09-13T04:39:10.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.550+0000 7f1c30ff9700 1 -- 172.21.15.2:0/2192311155 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1c24095960 con 0x7f1c34100d80 2023-09-13T04:39:10.417 INFO:teuthology.orchestra.run.smithi002.stdout:197568495664 2023-09-13T04:39:10.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.552+0000 7f1c3a212700 1 -- 172.21.15.2:0/2192311155 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1c1c0a06c0 msgr2=0x7f1c1c0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:10.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.552+0000 7f1c3a212700 1 --2- 172.21.15.2:0/2192311155 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1c1c0a06c0 0x7f1c1c0a2b80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1c28009990 tx=0x7f1c2801a040 comp rx=0 tx=0).stop 2023-09-13T04:39:10.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.552+0000 7f1c3a212700 1 -- 172.21.15.2:0/2192311155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c34100d80 msgr2=0x7f1c34196e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:10.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.552+0000 7f1c3a212700 1 --2- 172.21.15.2:0/2192311155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c34100d80 0x7f1c34196e80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400e9a0 tx=0x7f1c2400ed60 comp rx=0 tx=0).stop 2023-09-13T04:39:10.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.552+0000 7f1c3a212700 1 -- 172.21.15.2:0/2192311155 shutdown_connections 2023-09-13T04:39:10.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.553+0000 7f1c3a212700 1 --2- 172.21.15.2:0/2192311155 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1c1c0a06c0 0x7f1c1c0a2b80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:10.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.553+0000 7f1c3a212700 1 --2- 172.21.15.2:0/2192311155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c34106d90 0x7f1c3419b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:10.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.553+0000 7f1c3a212700 1 --2- 172.21.15.2:0/2192311155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c34100d80 0x7f1c34196e80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:10.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.553+0000 7f1c3a212700 1 --2- 172.21.15.2:0/2192311155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c34101740 0x7f1c341973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:10.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.553+0000 7f1c3a212700 1 -- 172.21.15.2:0/2192311155 >> 172.21.15.2:0/2192311155 conn(0x7f1c34077ea0 msgr2=0x7f1c340ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:10.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.553+0000 7f1c3a212700 1 -- 172.21.15.2:0/2192311155 shutdown_connections 2023-09-13T04:39:10.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:09.553+0000 7f1c3a212700 1 -- 172.21.15.2:0/2192311155 wait complete. 2023-09-13T04:39:10.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:10 smithi002 ceph-mon[97651]: pgmap v313: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2023-09-13T04:39:10.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:10 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2192311155' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-13T04:39:10.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:10 smithi149 ceph-mon[133315]: pgmap v313: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2023-09-13T04:39:10.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:10 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2192311155' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-13T04:39:10.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:10 smithi156 ceph-mon[133675]: pgmap v313: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2023-09-13T04:39:10.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:10 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2192311155' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-13T04:39:11.231 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103205 got 249108103207 for osd.8 2023-09-13T04:39:11.231 DEBUG:teuthology.parallel:result is None 2023-09-13T04:39:11.493 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495662 got 197568495664 for osd.6 2023-09-13T04:39:11.493 DEBUG:teuthology.parallel:result is None 2023-09-13T04:39:11.720 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757300 got 163208757302 for osd.5 2023-09-13T04:39:11.720 DEBUG:teuthology.parallel:result is None 2023-09-13T04:39:12.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.091+0000 7f51496bf700 1 -- 172.21.15.2:0/3545380384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5144073420 msgr2=0x7f5144073880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:12.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.091+0000 7f51496bf700 1 --2- 172.21.15.2:0/3545380384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5144073420 0x7f5144073880 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5138009990 tx=0x7f5138009ca0 comp rx=0 tx=0).stop 2023-09-13T04:39:12.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.091+0000 7f51496bf700 1 -- 172.21.15.2:0/3545380384 shutdown_connections 2023-09-13T04:39:12.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.091+0000 7f51496bf700 1 --2- 172.21.15.2:0/3545380384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51441119c0 0x7f5144113db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:12.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.091+0000 7f51496bf700 1 --2- 172.21.15.2:0/3545380384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144074cf0 0x7f5144072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:12.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.091+0000 7f51496bf700 1 --2- 172.21.15.2:0/3545380384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5144073420 0x7f5144073880 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:12.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.091+0000 7f51496bf700 1 -- 172.21.15.2:0/3545380384 >> 172.21.15.2:0/3545380384 conn(0x7f5144077ea0 msgr2=0x7f51440782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:12.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.091+0000 7f51496bf700 1 -- 172.21.15.2:0/3545380384 shutdown_connections 2023-09-13T04:39:12.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.091+0000 7f51496bf700 1 -- 172.21.15.2:0/3545380384 wait complete. 2023-09-13T04:39:12.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.092+0000 7f51496bf700 1 Processor -- start 2023-09-13T04:39:12.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.093+0000 7f51496bf700 1 -- start start 2023-09-13T04:39:12.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.093+0000 7f51496bf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5144073420 0x7f514419d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:12.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.093+0000 7f51496bf700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144074cf0 0x7f514419d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:12.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.093+0000 7f51496bf700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51441119c0 0x7f51441a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:12.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.093+0000 7f51496bf700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51441a2430 con 0x7f5144073420 2023-09-13T04:39:12.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.093+0000 7f51496bf700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51441a25a0 con 0x7f5144074cf0 2023-09-13T04:39:12.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.093+0000 7f51496bf700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51441a2760 con 0x7f51441119c0 2023-09-13T04:39:12.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.093+0000 7f5142ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5144073420 0x7f514419d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:12.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.093+0000 7f5142ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5144073420 0x7f514419d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36536/0 (socket says 172.21.15.2:36536) 2023-09-13T04:39:12.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.093+0000 7f5142ffd700 1 -- 172.21.15.2:0/1766603147 learned_addr learned my addr 172.21.15.2:0/1766603147 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:12.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.093+0000 7f51437fe700 1 --2- 172.21.15.2:0/1766603147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51441119c0 0x7f51441a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:12.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.093+0000 7f51427fc700 1 --2- 172.21.15.2:0/1766603147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144074cf0 0x7f514419d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:12.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.094+0000 7f5142ffd700 1 -- 172.21.15.2:0/1766603147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51441119c0 msgr2=0x7f51441a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:12.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.094+0000 7f5142ffd700 1 --2- 172.21.15.2:0/1766603147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51441119c0 0x7f51441a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:12.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.094+0000 7f5142ffd700 1 -- 172.21.15.2:0/1766603147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144074cf0 msgr2=0x7f514419d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:12.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.094+0000 7f5142ffd700 1 --2- 172.21.15.2:0/1766603147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144074cf0 0x7f514419d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:12.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.094+0000 7f5142ffd700 1 -- 172.21.15.2:0/1766603147 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51380095f0 con 0x7f5144073420 2023-09-13T04:39:12.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.094+0000 7f5142ffd700 1 --2- 172.21.15.2:0/1766603147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5144073420 0x7f514419d470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f513400d950 tx=0x7f513400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:12.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.094+0000 7f512bfff700 1 -- 172.21.15.2:0/1766603147 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513400f890 con 0x7f5144073420 2023-09-13T04:39:12.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.094+0000 7f512bfff700 1 -- 172.21.15.2:0/1766603147 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5134010460 con 0x7f5144073420 2023-09-13T04:39:12.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.094+0000 7f51496bf700 1 -- 172.21.15.2:0/1766603147 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51441a2a40 con 0x7f5144073420 2023-09-13T04:39:12.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.094+0000 7f512bfff700 1 -- 172.21.15.2:0/1766603147 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513400e610 con 0x7f5144073420 2023-09-13T04:39:12.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.095+0000 7f51496bf700 1 -- 172.21.15.2:0/1766603147 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51441aa2a0 con 0x7f5144073420 2023-09-13T04:39:12.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.096+0000 7f51496bf700 1 -- 172.21.15.2:0/1766603147 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514404e9b0 con 0x7f5144073420 2023-09-13T04:39:12.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.098+0000 7f512bfff700 1 -- 172.21.15.2:0/1766603147 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f51340108d0 con 0x7f5144073420 2023-09-13T04:39:12.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.100+0000 7f512bfff700 1 --2- 172.21.15.2:0/1766603147 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f512c0a06c0 0x7f512c0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:12.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.100+0000 7f512bfff700 1 -- 172.21.15.2:0/1766603147 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f51340cbfe0 con 0x7f5144073420 2023-09-13T04:39:12.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.100+0000 7f51427fc700 1 --2- 172.21.15.2:0/1766603147 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f512c0a06c0 0x7f512c0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:12.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.101+0000 7f51427fc700 1 --2- 172.21.15.2:0/1766603147 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f512c0a06c0 0x7f512c0a2b80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f514419ea90 tx=0x7f5138005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:12.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.103+0000 7f512bfff700 1 -- 172.21.15.2:0/1766603147 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f51340957c0 con 0x7f5144073420 2023-09-13T04:39:12.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.357+0000 7f51496bf700 1 -- 172.21.15.2:0/1766603147 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f51441173d0 con 0x7f5144073420 2023-09-13T04:39:12.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.358+0000 7f512bfff700 1 -- 172.21.15.2:0/1766603147 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5134020020 con 0x7f5144073420 2023-09-13T04:39:12.360 INFO:teuthology.orchestra.run.smithi002.stdout:34359738445 2023-09-13T04:39:12.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.360+0000 7f51496bf700 1 -- 172.21.15.2:0/1766603147 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f512c0a06c0 msgr2=0x7f512c0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:12.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.360+0000 7f51496bf700 1 --2- 172.21.15.2:0/1766603147 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f512c0a06c0 0x7f512c0a2b80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f514419ea90 tx=0x7f5138005fd0 comp rx=0 tx=0).stop 2023-09-13T04:39:12.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.360+0000 7f51496bf700 1 -- 172.21.15.2:0/1766603147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5144073420 msgr2=0x7f514419d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:12.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.360+0000 7f51496bf700 1 --2- 172.21.15.2:0/1766603147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5144073420 0x7f514419d470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f513400d950 tx=0x7f513400dc60 comp rx=0 tx=0).stop 2023-09-13T04:39:12.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.361+0000 7f51496bf700 1 -- 172.21.15.2:0/1766603147 shutdown_connections 2023-09-13T04:39:12.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.361+0000 7f51496bf700 1 --2- 172.21.15.2:0/1766603147 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f512c0a06c0 0x7f512c0a2b80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:12.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.361+0000 7f51496bf700 1 --2- 172.21.15.2:0/1766603147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51441119c0 0x7f51441a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:12.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.361+0000 7f51496bf700 1 --2- 172.21.15.2:0/1766603147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5144073420 0x7f514419d470 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:12.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.361+0000 7f51496bf700 1 --2- 172.21.15.2:0/1766603147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144074cf0 0x7f514419d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:12.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.361+0000 7f51496bf700 1 -- 172.21.15.2:0/1766603147 >> 172.21.15.2:0/1766603147 conn(0x7f5144077ea0 msgr2=0x7f51441104c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:12.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.361+0000 7f51496bf700 1 -- 172.21.15.2:0/1766603147 shutdown_connections 2023-09-13T04:39:12.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.361+0000 7f51496bf700 1 -- 172.21.15.2:0/1766603147 wait complete. 2023-09-13T04:39:12.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:12 smithi002 ceph-mon[97651]: pgmap v314: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2023-09-13T04:39:12.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:12 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1766603147' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-13T04:39:12.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:12 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/463347913' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-13T04:39:12.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:12 smithi149 ceph-mon[133315]: pgmap v314: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2023-09-13T04:39:12.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:12 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1766603147' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-13T04:39:12.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:12 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/463347913' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-13T04:39:12.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:12 smithi156 ceph-mon[133675]: pgmap v314: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2023-09-13T04:39:12.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:12 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1766603147' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-13T04:39:12.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:12 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/463347913' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-13T04:39:12.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.153+0000 7f2cc84d9700 1 -- 172.21.15.2:0/4102836677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2cc01060b0 msgr2=0x7f2cc010dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:12.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.153+0000 7f2cc84d9700 1 --2- 172.21.15.2:0/4102836677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2cc01060b0 0x7f2cc010dc30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8009f90 tx=0x7f2cb800b330 comp rx=0 tx=0).stop 2023-09-13T04:39:13.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.153+0000 7f2cc84d9700 1 -- 172.21.15.2:0/4102836677 shutdown_connections 2023-09-13T04:39:13.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.153+0000 7f2cc84d9700 1 --2- 172.21.15.2:0/4102836677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2cc01060b0 0x7f2cc010dc30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.153+0000 7f2cc84d9700 1 --2- 172.21.15.2:0/4102836677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cc0068a00 0x7f2cc0068de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.153+0000 7f2cc84d9700 1 --2- 172.21.15.2:0/4102836677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cc00693b0 0x7f2cc0105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.153+0000 7f2cc84d9700 1 -- 172.21.15.2:0/4102836677 >> 172.21.15.2:0/4102836677 conn(0x7f2cc0075c00 msgr2=0x7f2cc0076010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:13.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.153+0000 7f2cc84d9700 1 -- 172.21.15.2:0/4102836677 shutdown_connections 2023-09-13T04:39:13.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.154+0000 7f2cc84d9700 1 -- 172.21.15.2:0/4102836677 wait complete. 2023-09-13T04:39:13.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.155+0000 7f2cc84d9700 1 Processor -- start 2023-09-13T04:39:13.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.155+0000 7f2cc84d9700 1 -- start start 2023-09-13T04:39:13.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.155+0000 7f2cc84d9700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cc0068a00 0x7f2cc019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:13.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.155+0000 7f2cc84d9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2cc00693b0 0x7f2cc019aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:13.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.155+0000 7f2cc84d9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cc01060b0 0x7f2cc0194a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:13.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.155+0000 7f2cc84d9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc0195090 con 0x7f2cc00693b0 2023-09-13T04:39:13.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.155+0000 7f2cc84d9700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc0195200 con 0x7f2cc0068a00 2023-09-13T04:39:13.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.155+0000 7f2cc84d9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc01953c0 con 0x7f2cc01060b0 2023-09-13T04:39:13.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.156+0000 7f2cc6275700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cc0068a00 0x7f2cc019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:13.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.156+0000 7f2cc6275700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cc0068a00 0x7f2cc019a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:52950/0 (socket says 172.21.15.2:52950) 2023-09-13T04:39:13.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.156+0000 7f2cc6275700 1 -- 172.21.15.2:0/463347913 learned_addr learned my addr 172.21.15.2:0/463347913 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:13.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.156+0000 7f2cc6a76700 1 --2- 172.21.15.2:0/463347913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cc01060b0 0x7f2cc0194a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:13.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.156+0000 7f2cc6275700 1 -- 172.21.15.2:0/463347913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cc01060b0 msgr2=0x7f2cc0194a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:13.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.156+0000 7f2cc6275700 1 --2- 172.21.15.2:0/463347913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cc01060b0 0x7f2cc0194a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.156+0000 7f2cc6275700 1 -- 172.21.15.2:0/463347913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2cc00693b0 msgr2=0x7f2cc019aee0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-13T04:39:13.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.156+0000 7f2cc6275700 1 --2- 172.21.15.2:0/463347913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2cc00693b0 0x7f2cc019aee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.156+0000 7f2cc6275700 1 -- 172.21.15.2:0/463347913 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb8009c40 con 0x7f2cc0068a00 2023-09-13T04:39:13.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.156+0000 7f2cc6275700 1 --2- 172.21.15.2:0/463347913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cc0068a00 0x7f2cc019a9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00d950 tx=0x7f2cbc00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:13.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.157+0000 7f2cb77fe700 1 -- 172.21.15.2:0/463347913 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc009940 con 0x7f2cc0068a00 2023-09-13T04:39:13.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.157+0000 7f2cc84d9700 1 -- 172.21.15.2:0/463347913 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc01956a0 con 0x7f2cc0068a00 2023-09-13T04:39:13.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.157+0000 7f2cb77fe700 1 -- 172.21.15.2:0/463347913 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cbc010460 con 0x7f2cc0068a00 2023-09-13T04:39:13.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.157+0000 7f2cc84d9700 1 -- 172.21.15.2:0/463347913 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc01aa1a0 con 0x7f2cc0068a00 2023-09-13T04:39:13.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.158+0000 7f2cb77fe700 1 -- 172.21.15.2:0/463347913 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc00f620 con 0x7f2cc0068a00 2023-09-13T04:39:13.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.158+0000 7f2cc84d9700 1 -- 172.21.15.2:0/463347913 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ca4005320 con 0x7f2cc0068a00 2023-09-13T04:39:13.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.160+0000 7f2cb77fe700 1 -- 172.21.15.2:0/463347913 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f2cbc009aa0 con 0x7f2cc0068a00 2023-09-13T04:39:13.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.162+0000 7f2cb77fe700 1 --2- 172.21.15.2:0/463347913 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2cac0a0620 0x7f2cac0a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:13.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.162+0000 7f2cb77fe700 1 -- 172.21.15.2:0/463347913 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2cbc0cc0c0 con 0x7f2cc0068a00 2023-09-13T04:39:13.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.162+0000 7f2cc5a74700 1 --2- 172.21.15.2:0/463347913 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2cac0a0620 0x7f2cac0a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:13.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.162+0000 7f2cc5a74700 1 --2- 172.21.15.2:0/463347913 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2cac0a0620 0x7f2cac0a2ae0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2cc01077e0 tx=0x7f2cb0009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:13.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.166+0000 7f2cb77fe700 1 -- 172.21.15.2:0/463347913 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2cbc095880 con 0x7f2cc0068a00 2023-09-13T04:39:13.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.414+0000 7f2cc84d9700 1 -- 172.21.15.2:0/463347913 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2ca4005190 con 0x7f2cc0068a00 2023-09-13T04:39:13.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.415+0000 7f2cb77fe700 1 -- 172.21.15.2:0/463347913 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2cbc098ea0 con 0x7f2cc0068a00 2023-09-13T04:39:13.008 INFO:teuthology.orchestra.run.smithi002.stdout:107374182464 2023-09-13T04:39:13.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.417+0000 7f2cc84d9700 1 -- 172.21.15.2:0/463347913 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2cac0a0620 msgr2=0x7f2cac0a2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:13.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.417+0000 7f2cc84d9700 1 --2- 172.21.15.2:0/463347913 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2cac0a0620 0x7f2cac0a2ae0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2cc01077e0 tx=0x7f2cb0009310 comp rx=0 tx=0).stop 2023-09-13T04:39:13.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.417+0000 7f2cc84d9700 1 -- 172.21.15.2:0/463347913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cc0068a00 msgr2=0x7f2cc019a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:13.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.417+0000 7f2cc84d9700 1 --2- 172.21.15.2:0/463347913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cc0068a00 0x7f2cc019a9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00d950 tx=0x7f2cbc00dc60 comp rx=0 tx=0).stop 2023-09-13T04:39:13.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.417+0000 7f2cc84d9700 1 -- 172.21.15.2:0/463347913 shutdown_connections 2023-09-13T04:39:13.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.417+0000 7f2cc84d9700 1 --2- 172.21.15.2:0/463347913 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2cac0a0620 0x7f2cac0a2ae0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.417+0000 7f2cc84d9700 1 --2- 172.21.15.2:0/463347913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cc01060b0 0x7f2cc0194a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.417+0000 7f2cc84d9700 1 --2- 172.21.15.2:0/463347913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cc0068a00 0x7f2cc019a9a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.417+0000 7f2cc84d9700 1 --2- 172.21.15.2:0/463347913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2cc00693b0 0x7f2cc019aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.417+0000 7f2cc84d9700 1 -- 172.21.15.2:0/463347913 >> 172.21.15.2:0/463347913 conn(0x7f2cc0075c00 msgr2=0x7f2cc00fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:13.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.418+0000 7f2cc84d9700 1 -- 172.21.15.2:0/463347913 shutdown_connections 2023-09-13T04:39:13.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:12.418+0000 7f2cc84d9700 1 -- 172.21.15.2:0/463347913 wait complete. 2023-09-13T04:39:13.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.361+0000 7f657facf700 1 -- 172.21.15.2:0/1641007693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6578106d90 msgr2=0x7f6578107170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:13.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.361+0000 7f657facf700 1 --2- 172.21.15.2:0/1641007693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6578106d90 0x7f6578107170 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f65740099c0 tx=0x7f6574009cd0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.362+0000 7f657facf700 1 -- 172.21.15.2:0/1641007693 shutdown_connections 2023-09-13T04:39:13.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.362+0000 7f657facf700 1 --2- 172.21.15.2:0/1641007693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6578101740 0x7f657810dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.362+0000 7f657facf700 1 --2- 172.21.15.2:0/1641007693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6578106d90 0x7f6578107170 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.362+0000 7f657facf700 1 --2- 172.21.15.2:0/1641007693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6578100d80 0x7f6578101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.362+0000 7f657facf700 1 -- 172.21.15.2:0/1641007693 >> 172.21.15.2:0/1641007693 conn(0x7f6578077ea0 msgr2=0x7f65780782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:13.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.362+0000 7f657facf700 1 -- 172.21.15.2:0/1641007693 shutdown_connections 2023-09-13T04:39:13.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.362+0000 7f657facf700 1 -- 172.21.15.2:0/1641007693 wait complete. 2023-09-13T04:39:13.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.363+0000 7f657facf700 1 Processor -- start 2023-09-13T04:39:13.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.363+0000 7f657facf700 1 -- start start 2023-09-13T04:39:13.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.363+0000 7f657facf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6578100d80 0x7f6578198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:13.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.363+0000 7f657facf700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6578101740 0x7f65781994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:13.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.363+0000 7f657facf700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6578106d90 0x7f657819d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:13.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.363+0000 7f657facf700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657819df20 con 0x7f6578100d80 2023-09-13T04:39:13.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.363+0000 7f657facf700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657819e090 con 0x7f6578106d90 2023-09-13T04:39:13.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.363+0000 7f657facf700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657819e250 con 0x7f6578101740 2023-09-13T04:39:13.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.364+0000 7f657d06a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6578101740 0x7f65781994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:13.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.364+0000 7f657d86b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6578100d80 0x7f6578198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:13.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.364+0000 7f657d86b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6578100d80 0x7f6578198f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36568/0 (socket says 172.21.15.2:36568) 2023-09-13T04:39:13.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.364+0000 7f657d86b700 1 -- 172.21.15.2:0/2849661851 learned_addr learned my addr 172.21.15.2:0/2849661851 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:13.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.364+0000 7f657d06a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6578101740 0x7f65781994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:50804/0 (socket says 172.21.15.2:50804) 2023-09-13T04:39:13.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.364+0000 7f657e06c700 1 --2- 172.21.15.2:0/2849661851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6578106d90 0x7f657819d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:13.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.364+0000 7f657d86b700 1 -- 172.21.15.2:0/2849661851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6578101740 msgr2=0x7f65781994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:13.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.364+0000 7f657d86b700 1 --2- 172.21.15.2:0/2849661851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6578101740 0x7f65781994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.364+0000 7f657d86b700 1 -- 172.21.15.2:0/2849661851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6578106d90 msgr2=0x7f657819d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:13.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.364+0000 7f657d86b700 1 --2- 172.21.15.2:0/2849661851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6578106d90 0x7f657819d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.364+0000 7f657d86b700 1 -- 172.21.15.2:0/2849661851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6568009520 con 0x7f6578100d80 2023-09-13T04:39:13.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.364+0000 7f657e06c700 1 --2- 172.21.15.2:0/2849661851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6578106d90 0x7f657819d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:39:13.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.365+0000 7f657d86b700 1 --2- 172.21.15.2:0/2849661851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6578100d80 0x7f6578198f60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f657400b580 tx=0x7f657400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:13.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.365+0000 7f656effd700 1 -- 172.21.15.2:0/2849661851 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657401d070 con 0x7f6578100d80 2023-09-13T04:39:13.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.365+0000 7f656effd700 1 -- 172.21.15.2:0/2849661851 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6574005050 con 0x7f6578100d80 2023-09-13T04:39:13.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.365+0000 7f656effd700 1 -- 172.21.15.2:0/2849661851 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6574004a40 con 0x7f6578100d80 2023-09-13T04:39:13.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.365+0000 7f657facf700 1 -- 172.21.15.2:0/2849661851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65740095f0 con 0x7f6578100d80 2023-09-13T04:39:13.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.365+0000 7f657facf700 1 -- 172.21.15.2:0/2849661851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657819e750 con 0x7f6578100d80 2023-09-13T04:39:13.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.366+0000 7f657facf700 1 -- 172.21.15.2:0/2849661851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657804e9b0 con 0x7f6578100d80 2023-09-13T04:39:13.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.370+0000 7f656effd700 1 -- 172.21.15.2:0/2849661851 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f65740051c0 con 0x7f6578100d80 2023-09-13T04:39:13.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.371+0000 7f656effd700 1 --2- 172.21.15.2:0/2849661851 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f65640a06c0 0x7f65640a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:13.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.371+0000 7f656effd700 1 -- 172.21.15.2:0/2849661851 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f65740ccc70 con 0x7f6578100d80 2023-09-13T04:39:13.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.372+0000 7f657d06a700 1 --2- 172.21.15.2:0/2849661851 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f65640a06c0 0x7f65640a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:13.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.372+0000 7f657d06a700 1 --2- 172.21.15.2:0/2849661851 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f65640a06c0 0x7f65640a2b80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f657819a580 tx=0x7f6568009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:13.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.379+0000 7f656effd700 1 -- 172.21.15.2:0/2849661851 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6574096520 con 0x7f6578100d80 2023-09-13T04:39:13.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.385+0000 7f02a6f5f700 1 -- 172.21.15.2:0/3336379317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02a01060b0 msgr2=0x7f02a010dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:13.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.385+0000 7f02a6f5f700 1 --2- 172.21.15.2:0/3336379317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02a01060b0 0x7f02a010dc30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f029800b0a0 tx=0x7f029800b3b0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.385+0000 7f02a6f5f700 1 -- 172.21.15.2:0/3336379317 shutdown_connections 2023-09-13T04:39:13.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.385+0000 7f02a6f5f700 1 --2- 172.21.15.2:0/3336379317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02a01060b0 0x7f02a010dc30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.385+0000 7f02a6f5f700 1 --2- 172.21.15.2:0/3336379317 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02a0068a00 0x7f02a0068de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.385+0000 7f02a6f5f700 1 --2- 172.21.15.2:0/3336379317 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02a00693b0 0x7f02a0105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.385+0000 7f02a6f5f700 1 -- 172.21.15.2:0/3336379317 >> 172.21.15.2:0/3336379317 conn(0x7f02a0075c00 msgr2=0x7f02a0076010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:13.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.386+0000 7f02a6f5f700 1 -- 172.21.15.2:0/3336379317 shutdown_connections 2023-09-13T04:39:13.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.386+0000 7f02a6f5f700 1 -- 172.21.15.2:0/3336379317 wait complete. 2023-09-13T04:39:13.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.387+0000 7f02a6f5f700 1 Processor -- start 2023-09-13T04:39:13.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.387+0000 7f02a6f5f700 1 -- start start 2023-09-13T04:39:13.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.388+0000 7f02a6f5f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02a0068a00 0x7f02a00ffd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:13.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.388+0000 7f02a6f5f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02a00693b0 0x7f02a0100270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:13.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.388+0000 7f02a6f5f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02a01060b0 0x7f02a01007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:13.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.388+0000 7f02a6f5f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a01a5350 con 0x7f02a00693b0 2023-09-13T04:39:13.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.388+0000 7f02a6f5f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a01a54c0 con 0x7f02a01060b0 2023-09-13T04:39:13.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.388+0000 7f02a6f5f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a01a5680 con 0x7f02a0068a00 2023-09-13T04:39:13.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.388+0000 7f02a4cfb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02a0068a00 0x7f02a00ffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:13.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.388+0000 7f02a4cfb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02a0068a00 0x7f02a00ffd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:50818/0 (socket says 172.21.15.2:50818) 2023-09-13T04:39:13.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.388+0000 7f02a4cfb700 1 -- 172.21.15.2:0/3740459805 learned_addr learned my addr 172.21.15.2:0/3740459805 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:13.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.388+0000 7f029ffff700 1 --2- 172.21.15.2:0/3740459805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02a00693b0 0x7f02a0100270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:13.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.389+0000 7f02a54fc700 1 --2- 172.21.15.2:0/3740459805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02a01060b0 0x7f02a01007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:13.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.389+0000 7f02a4cfb700 1 -- 172.21.15.2:0/3740459805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02a01060b0 msgr2=0x7f02a01007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:13.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.389+0000 7f02a4cfb700 1 --2- 172.21.15.2:0/3740459805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02a01060b0 0x7f02a01007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.389+0000 7f02a4cfb700 1 -- 172.21.15.2:0/3740459805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02a00693b0 msgr2=0x7f02a0100270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:13.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.389+0000 7f02a4cfb700 1 --2- 172.21.15.2:0/3740459805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02a00693b0 0x7f02a0100270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.389+0000 7f02a4cfb700 1 -- 172.21.15.2:0/3740459805 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0298009d00 con 0x7f02a0068a00 2023-09-13T04:39:13.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.389+0000 7f02a4cfb700 1 --2- 172.21.15.2:0/3740459805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02a0068a00 0x7f02a00ffd30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f029400d950 tx=0x7f029400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:13.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.389+0000 7f029dffb700 1 -- 172.21.15.2:0/3740459805 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029400f890 con 0x7f02a0068a00 2023-09-13T04:39:13.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.389+0000 7f029dffb700 1 -- 172.21.15.2:0/3740459805 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0294010460 con 0x7f02a0068a00 2023-09-13T04:39:13.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.390+0000 7f02a6f5f700 1 -- 172.21.15.2:0/3740459805 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a01a5960 con 0x7f02a0068a00 2023-09-13T04:39:13.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.390+0000 7f029dffb700 1 -- 172.21.15.2:0/3740459805 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029400e610 con 0x7f02a0068a00 2023-09-13T04:39:13.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.390+0000 7f02a6f5f700 1 -- 172.21.15.2:0/3740459805 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a01a5e60 con 0x7f02a0068a00 2023-09-13T04:39:13.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.391+0000 7f02a6f5f700 1 -- 172.21.15.2:0/3740459805 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02a004e9b0 con 0x7f02a0068a00 2023-09-13T04:39:13.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.393+0000 7f029dffb700 1 -- 172.21.15.2:0/3740459805 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f02940108d0 con 0x7f02a0068a00 2023-09-13T04:39:13.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.394+0000 7f029dffb700 1 --2- 172.21.15.2:0/3740459805 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f02900a0640 0x7f02900a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:13.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.394+0000 7f029dffb700 1 -- 172.21.15.2:0/3740459805 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f02940cbc90 con 0x7f02a0068a00 2023-09-13T04:39:13.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.395+0000 7f029ffff700 1 --2- 172.21.15.2:0/3740459805 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f02900a0640 0x7f02900a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:13.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.395+0000 7f029ffff700 1 --2- 172.21.15.2:0/3740459805 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f02900a0640 0x7f02900a2b00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f02a0101520 tx=0x7f028c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:13.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.396+0000 7f029dffb700 1 -- 172.21.15.2:0/3740459805 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0294095450 con 0x7f02a0068a00 2023-09-13T04:39:13.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.599+0000 7f657facf700 1 -- 172.21.15.2:0/2849661851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f657819a2c0 con 0x7f6578100d80 2023-09-13T04:39:13.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.599+0000 7f656effd700 1 -- 172.21.15.2:0/2849661851 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6574099b40 con 0x7f6578100d80 2023-09-13T04:39:13.601 INFO:teuthology.orchestra.run.smithi002.stdout:141733920828 2023-09-13T04:39:13.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.602+0000 7f657facf700 1 -- 172.21.15.2:0/2849661851 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f65640a06c0 msgr2=0x7f65640a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:13.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.602+0000 7f657facf700 1 --2- 172.21.15.2:0/2849661851 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f65640a06c0 0x7f65640a2b80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f657819a580 tx=0x7f6568009260 comp rx=0 tx=0).stop 2023-09-13T04:39:13.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.602+0000 7f657facf700 1 -- 172.21.15.2:0/2849661851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6578100d80 msgr2=0x7f6578198f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:13.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.602+0000 7f657facf700 1 --2- 172.21.15.2:0/2849661851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6578100d80 0x7f6578198f60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f657400b580 tx=0x7f657400bfd0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.602+0000 7f657facf700 1 -- 172.21.15.2:0/2849661851 shutdown_connections 2023-09-13T04:39:13.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.602+0000 7f657facf700 1 --2- 172.21.15.2:0/2849661851 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f65640a06c0 0x7f65640a2b80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.602+0000 7f657facf700 1 --2- 172.21.15.2:0/2849661851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6578106d90 0x7f657819d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.602+0000 7f657facf700 1 --2- 172.21.15.2:0/2849661851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6578100d80 0x7f6578198f60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.602+0000 7f657facf700 1 --2- 172.21.15.2:0/2849661851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6578101740 0x7f65781994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.602+0000 7f657facf700 1 -- 172.21.15.2:0/2849661851 >> 172.21.15.2:0/2849661851 conn(0x7f6578077ea0 msgr2=0x7f65780ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:13.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.602+0000 7f657facf700 1 -- 172.21.15.2:0/2849661851 shutdown_connections 2023-09-13T04:39:13.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.602+0000 7f657facf700 1 -- 172.21.15.2:0/2849661851 wait complete. 2023-09-13T04:39:13.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.626+0000 7f02a6f5f700 1 -- 172.21.15.2:0/3740459805 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f02a0110e70 con 0x7f02a0068a00 2023-09-13T04:39:13.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.626+0000 7f029dffb700 1 -- 172.21.15.2:0/3740459805 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0294020090 con 0x7f02a0068a00 2023-09-13T04:39:13.628 INFO:teuthology.orchestra.run.smithi002.stdout:55834574920 2023-09-13T04:39:13.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.628+0000 7f02a6f5f700 1 -- 172.21.15.2:0/3740459805 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f02900a0640 msgr2=0x7f02900a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:13.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.628+0000 7f02a6f5f700 1 --2- 172.21.15.2:0/3740459805 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f02900a0640 0x7f02900a2b00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f02a0101520 tx=0x7f028c009260 comp rx=0 tx=0).stop 2023-09-13T04:39:13.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.628+0000 7f02a6f5f700 1 -- 172.21.15.2:0/3740459805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02a0068a00 msgr2=0x7f02a00ffd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:13.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.628+0000 7f02a6f5f700 1 --2- 172.21.15.2:0/3740459805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02a0068a00 0x7f02a00ffd30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f029400d950 tx=0x7f029400dc60 comp rx=0 tx=0).stop 2023-09-13T04:39:13.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.629+0000 7f02a6f5f700 1 -- 172.21.15.2:0/3740459805 shutdown_connections 2023-09-13T04:39:13.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.629+0000 7f02a6f5f700 1 --2- 172.21.15.2:0/3740459805 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f02900a0640 0x7f02900a2b00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.629+0000 7f02a6f5f700 1 --2- 172.21.15.2:0/3740459805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02a01060b0 0x7f02a01007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.629+0000 7f02a6f5f700 1 --2- 172.21.15.2:0/3740459805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02a0068a00 0x7f02a00ffd30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.629+0000 7f02a6f5f700 1 --2- 172.21.15.2:0/3740459805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02a00693b0 0x7f02a0100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:13.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.629+0000 7f02a6f5f700 1 -- 172.21.15.2:0/3740459805 >> 172.21.15.2:0/3740459805 conn(0x7f02a0075c00 msgr2=0x7f02a00fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:13.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.629+0000 7f02a6f5f700 1 -- 172.21.15.2:0/3740459805 shutdown_connections 2023-09-13T04:39:13.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:13.629+0000 7f02a6f5f700 1 -- 172.21.15.2:0/3740459805 wait complete. 2023-09-13T04:39:14.118 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738441 got 34359738445 for osd.0 2023-09-13T04:39:14.119 DEBUG:teuthology.parallel:result is None 2023-09-13T04:39:14.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.475+0000 7f1a95eb9700 1 -- 172.21.15.2:0/1844941686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a90106d90 msgr2=0x7f1a90107170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:14.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.475+0000 7f1a95eb9700 1 --2- 172.21.15.2:0/1844941686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a90106d90 0x7f1a90107170 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1a800099c0 tx=0x7f1a80009cd0 comp rx=0 tx=0).stop 2023-09-13T04:39:14.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.478+0000 7f1a95eb9700 1 -- 172.21.15.2:0/1844941686 shutdown_connections 2023-09-13T04:39:14.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.478+0000 7f1a95eb9700 1 --2- 172.21.15.2:0/1844941686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a90101740 0x7f1a9010dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:14.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.478+0000 7f1a95eb9700 1 --2- 172.21.15.2:0/1844941686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a90106d90 0x7f1a90107170 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:14.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.478+0000 7f1a95eb9700 1 --2- 172.21.15.2:0/1844941686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a90100d80 0x7f1a90101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:14.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.478+0000 7f1a95eb9700 1 -- 172.21.15.2:0/1844941686 >> 172.21.15.2:0/1844941686 conn(0x7f1a90077ea0 msgr2=0x7f1a900782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:14.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.478+0000 7f1a95eb9700 1 -- 172.21.15.2:0/1844941686 shutdown_connections 2023-09-13T04:39:14.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.478+0000 7f1a95eb9700 1 -- 172.21.15.2:0/1844941686 wait complete. 2023-09-13T04:39:14.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.479+0000 7f1a95eb9700 1 Processor -- start 2023-09-13T04:39:14.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.480+0000 7f1a95eb9700 1 -- start start 2023-09-13T04:39:14.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.480+0000 7f1a95eb9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a90100d80 0x7f1a9010a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:14.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.480+0000 7f1a95eb9700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a90101740 0x7f1a9010a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:14.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.480+0000 7f1a95eb9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a90106d90 0x7f1a901a9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:14.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.480+0000 7f1a95eb9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a901a9740 con 0x7f1a90106d90 2023-09-13T04:39:14.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.480+0000 7f1a95eb9700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a901a9880 con 0x7f1a90101740 2023-09-13T04:39:14.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.480+0000 7f1a95eb9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a901a99c0 con 0x7f1a90100d80 2023-09-13T04:39:14.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.481+0000 7f1a8effd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a90101740 0x7f1a9010a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:14.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.481+0000 7f1a8f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a90100d80 0x7f1a9010a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:14.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.481+0000 7f1a8effd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a90101740 0x7f1a9010a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:44446/0 (socket says 172.21.15.2:44446) 2023-09-13T04:39:14.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.481+0000 7f1a8f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a90100d80 0x7f1a9010a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:35560/0 (socket says 172.21.15.2:35560) 2023-09-13T04:39:14.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.481+0000 7f1a8ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a90106d90 0x7f1a901a9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:14.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.481+0000 7f1a8effd700 1 -- 172.21.15.2:0/1127481755 learned_addr learned my addr 172.21.15.2:0/1127481755 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:14.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.481+0000 7f1a8effd700 1 -- 172.21.15.2:0/1127481755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a90100d80 msgr2=0x7f1a9010a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:14.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.481+0000 7f1a8effd700 1 --2- 172.21.15.2:0/1127481755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a90100d80 0x7f1a9010a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:14.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.481+0000 7f1a8effd700 1 -- 172.21.15.2:0/1127481755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a90106d90 msgr2=0x7f1a901a9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:14.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.481+0000 7f1a8effd700 1 --2- 172.21.15.2:0/1127481755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a90106d90 0x7f1a901a9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:14.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.481+0000 7f1a8effd700 1 -- 172.21.15.2:0/1127481755 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a800095f0 con 0x7f1a90101740 2023-09-13T04:39:14.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.482+0000 7f1a8effd700 1 --2- 172.21.15.2:0/1127481755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a90101740 0x7f1a9010a640 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1a8400e9d0 tx=0x7f1a8400ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:14.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.482+0000 7f1a8cff9700 1 -- 172.21.15.2:0/1127481755 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a84009790 con 0x7f1a90101740 2023-09-13T04:39:14.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.482+0000 7f1a8cff9700 1 -- 172.21.15.2:0/1127481755 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a8400ee70 con 0x7f1a90101740 2023-09-13T04:39:14.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.482+0000 7f1a8cff9700 1 -- 172.21.15.2:0/1127481755 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a84010ea0 con 0x7f1a90101740 2023-09-13T04:39:14.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.482+0000 7f1a95eb9700 1 -- 172.21.15.2:0/1127481755 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a901a9c20 con 0x7f1a90101740 2023-09-13T04:39:14.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.482+0000 7f1a95eb9700 1 -- 172.21.15.2:0/1127481755 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a901aa0d0 con 0x7f1a90101740 2023-09-13T04:39:14.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.483+0000 7f1a95eb9700 1 -- 172.21.15.2:0/1127481755 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a9004e9b0 con 0x7f1a90101740 2023-09-13T04:39:14.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.485+0000 7f1a8cff9700 1 -- 172.21.15.2:0/1127481755 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f1a8400cbd0 con 0x7f1a90101740 2023-09-13T04:39:14.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.486+0000 7f1a8cff9700 1 --2- 172.21.15.2:0/1127481755 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1a780a06c0 0x7f1a780a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:14.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.487+0000 7f1a8cff9700 1 -- 172.21.15.2:0/1127481755 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1a84014070 con 0x7f1a90101740 2023-09-13T04:39:14.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.487+0000 7f1a8f7fe700 1 --2- 172.21.15.2:0/1127481755 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1a780a06c0 0x7f1a780a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:14.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.487+0000 7f1a8f7fe700 1 --2- 172.21.15.2:0/1127481755 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1a780a06c0 0x7f1a780a2b80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009990 tx=0x7f1a80005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:14.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.489+0000 7f1a8cff9700 1 -- 172.21.15.2:0/1127481755 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1a84095d60 con 0x7f1a90101740 2023-09-13T04:39:14.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.726+0000 7f1a95eb9700 1 -- 172.21.15.2:0/1127481755 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1a9010b630 con 0x7f1a90101740 2023-09-13T04:39:14.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.727+0000 7f1a8cff9700 1 -- 172.21.15.2:0/1127481755 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1a84099380 con 0x7f1a90101740 2023-09-13T04:39:14.729 INFO:teuthology.orchestra.run.smithi002.stdout:77309411398 2023-09-13T04:39:14.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.731+0000 7f1a95eb9700 1 -- 172.21.15.2:0/1127481755 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1a780a06c0 msgr2=0x7f1a780a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:14.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.731+0000 7f1a95eb9700 1 --2- 172.21.15.2:0/1127481755 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1a780a06c0 0x7f1a780a2b80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009990 tx=0x7f1a80005fb0 comp rx=0 tx=0).stop 2023-09-13T04:39:14.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.731+0000 7f1a95eb9700 1 -- 172.21.15.2:0/1127481755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a90101740 msgr2=0x7f1a9010a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:14.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.731+0000 7f1a95eb9700 1 --2- 172.21.15.2:0/1127481755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a90101740 0x7f1a9010a640 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1a8400e9d0 tx=0x7f1a8400ece0 comp rx=0 tx=0).stop 2023-09-13T04:39:14.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.731+0000 7f1a95eb9700 1 -- 172.21.15.2:0/1127481755 shutdown_connections 2023-09-13T04:39:14.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.731+0000 7f1a95eb9700 1 --2- 172.21.15.2:0/1127481755 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1a780a06c0 0x7f1a780a2b80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:14.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.731+0000 7f1a95eb9700 1 --2- 172.21.15.2:0/1127481755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a90106d90 0x7f1a901a9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:14.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.731+0000 7f1a95eb9700 1 --2- 172.21.15.2:0/1127481755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a90100d80 0x7f1a9010a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:14.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.731+0000 7f1a95eb9700 1 --2- 172.21.15.2:0/1127481755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a90101740 0x7f1a9010a640 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:14.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.731+0000 7f1a95eb9700 1 -- 172.21.15.2:0/1127481755 >> 172.21.15.2:0/1127481755 conn(0x7f1a90077ea0 msgr2=0x7f1a90104940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:14.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.731+0000 7f1a95eb9700 1 -- 172.21.15.2:0/1127481755 shutdown_connections 2023-09-13T04:39:14.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:14.731+0000 7f1a95eb9700 1 -- 172.21.15.2:0/1127481755 wait complete. 2023-09-13T04:39:14.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:14 smithi002 ceph-mon[97651]: pgmap v315: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:39:14.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:14 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2849661851' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-13T04:39:14.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:14 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3740459805' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-13T04:39:14.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:14 smithi149 ceph-mon[133315]: pgmap v315: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:39:14.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:14 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2849661851' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-13T04:39:14.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:14 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3740459805' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-13T04:39:14.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:14 smithi156 ceph-mon[133675]: pgmap v315: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:39:14.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:14 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2849661851' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-13T04:39:14.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:14 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3740459805' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-13T04:39:14.966 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920826 got 141733920828 for osd.4 2023-09-13T04:39:14.967 DEBUG:teuthology.parallel:result is None 2023-09-13T04:39:15.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.297+0000 7f60c7205700 1 -- 172.21.15.2:0/2971066637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60c0072ee0 msgr2=0x7f60c00732c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:15.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.298+0000 7f60c7205700 1 --2- 172.21.15.2:0/2971066637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60c0072ee0 0x7f60c00732c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009990 tx=0x7f60b4009ca0 comp rx=0 tx=0).stop 2023-09-13T04:39:15.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.298+0000 7f60c7205700 1 -- 172.21.15.2:0/2971066637 shutdown_connections 2023-09-13T04:39:15.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.298+0000 7f60c7205700 1 --2- 172.21.15.2:0/2971066637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60c010d730 0x7f60c010fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:15.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.298+0000 7f60c7205700 1 --2- 172.21.15.2:0/2971066637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60c0072ee0 0x7f60c00732c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:15.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.298+0000 7f60c7205700 1 --2- 172.21.15.2:0/2971066637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60c0073800 0x7f60c010d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:15.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.298+0000 7f60c7205700 1 -- 172.21.15.2:0/2971066637 >> 172.21.15.2:0/2971066637 conn(0x7f60c0077ea0 msgr2=0x7f60c00782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:15.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.298+0000 7f60c7205700 1 -- 172.21.15.2:0/2971066637 shutdown_connections 2023-09-13T04:39:15.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.298+0000 7f60c7205700 1 -- 172.21.15.2:0/2971066637 wait complete. 2023-09-13T04:39:15.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.299+0000 7f60c7205700 1 Processor -- start 2023-09-13T04:39:15.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.300+0000 7f60c7205700 1 -- start start 2023-09-13T04:39:15.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.300+0000 7f60c7205700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60c0072ee0 0x7f60c01990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:15.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.300+0000 7f60c7205700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60c0073800 0x7f60c01995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:15.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.300+0000 7f60c7205700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60c010d730 0x7f60c019da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:15.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.300+0000 7f60c7205700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c019e070 con 0x7f60c0073800 2023-09-13T04:39:15.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.300+0000 7f60c7205700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c019e1e0 con 0x7f60c010d730 2023-09-13T04:39:15.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.300+0000 7f60c7205700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c019e3a0 con 0x7f60c0072ee0 2023-09-13T04:39:15.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.301+0000 7f60bffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60c0073800 0x7f60c01995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:15.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.301+0000 7f60c57a2700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60c010d730 0x7f60c019da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:15.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.301+0000 7f60c4fa1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60c0072ee0 0x7f60c01990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:15.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.301+0000 7f60c57a2700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60c010d730 0x7f60c019da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:44466/0 (socket says 172.21.15.2:44466) 2023-09-13T04:39:15.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.301+0000 7f60bffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60c0073800 0x7f60c01995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53440/0 (socket says 172.21.15.2:53440) 2023-09-13T04:39:15.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.301+0000 7f60c57a2700 1 -- 172.21.15.2:0/2676801568 learned_addr learned my addr 172.21.15.2:0/2676801568 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:15.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.301+0000 7f60c57a2700 1 -- 172.21.15.2:0/2676801568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60c0072ee0 msgr2=0x7f60c01990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:15.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.301+0000 7f60c57a2700 1 --2- 172.21.15.2:0/2676801568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60c0072ee0 0x7f60c01990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:15.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.301+0000 7f60c57a2700 1 -- 172.21.15.2:0/2676801568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60c0073800 msgr2=0x7f60c01995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:15.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.301+0000 7f60c57a2700 1 --2- 172.21.15.2:0/2676801568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60c0073800 0x7f60c01995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:15.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.301+0000 7f60c57a2700 1 -- 172.21.15.2:0/2676801568 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b40095f0 con 0x7f60c010d730 2023-09-13T04:39:15.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.301+0000 7f60c57a2700 1 --2- 172.21.15.2:0/2676801568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60c010d730 0x7f60c019da60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f60b800ef90 tx=0x7f60b800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:15.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.302+0000 7f60bdffb700 1 -- 172.21.15.2:0/2676801568 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b8018070 con 0x7f60c010d730 2023-09-13T04:39:15.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.302+0000 7f60bdffb700 1 -- 172.21.15.2:0/2676801568 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60b800f040 con 0x7f60c010d730 2023-09-13T04:39:15.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.302+0000 7f60bdffb700 1 -- 172.21.15.2:0/2676801568 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b800ccc0 con 0x7f60c010d730 2023-09-13T04:39:15.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.302+0000 7f60c7205700 1 -- 172.21.15.2:0/2676801568 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c019e620 con 0x7f60c010d730 2023-09-13T04:39:15.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.302+0000 7f60c7205700 1 -- 172.21.15.2:0/2676801568 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c01a5ee0 con 0x7f60c010d730 2023-09-13T04:39:15.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.303+0000 7f60c7205700 1 -- 172.21.15.2:0/2676801568 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c004e9b0 con 0x7f60c010d730 2023-09-13T04:39:15.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.305+0000 7f60bdffb700 1 -- 172.21.15.2:0/2676801568 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f60b8003a00 con 0x7f60c010d730 2023-09-13T04:39:15.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.307+0000 7f60bdffb700 1 --2- 172.21.15.2:0/2676801568 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f60b00a06c0 0x7f60b00a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:15.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.307+0000 7f60bdffb700 1 -- 172.21.15.2:0/2676801568 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f60b80cbea0 con 0x7f60c010d730 2023-09-13T04:39:15.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.307+0000 7f60c4fa1700 1 --2- 172.21.15.2:0/2676801568 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f60b00a06c0 0x7f60b00a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:15.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.308+0000 7f60c4fa1700 1 --2- 172.21.15.2:0/2676801568 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f60b00a06c0 0x7f60b00a2b80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f60b400b2e0 tx=0x7f60b4005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:15.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.309+0000 7f60bdffb700 1 -- 172.21.15.2:0/2676801568 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f60b80956f0 con 0x7f60c010d730 2023-09-13T04:39:15.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.406+0000 7f514ab3e700 1 -- 172.21.15.2:0/4212187728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5144100d80 msgr2=0x7f5144101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:15.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.406+0000 7f514ab3e700 1 --2- 172.21.15.2:0/4212187728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5144100d80 0x7f5144101200 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f51300099c0 tx=0x7f5130009cd0 comp rx=0 tx=0).stop 2023-09-13T04:39:15.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.406+0000 7f514ab3e700 1 -- 172.21.15.2:0/4212187728 shutdown_connections 2023-09-13T04:39:15.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.406+0000 7f514ab3e700 1 --2- 172.21.15.2:0/4212187728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5144101740 0x7f514410dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:15.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.406+0000 7f514ab3e700 1 --2- 172.21.15.2:0/4212187728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144106d90 0x7f5144107170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:15.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.406+0000 7f514ab3e700 1 --2- 172.21.15.2:0/4212187728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5144100d80 0x7f5144101200 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:15.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.406+0000 7f514ab3e700 1 -- 172.21.15.2:0/4212187728 >> 172.21.15.2:0/4212187728 conn(0x7f5144077ea0 msgr2=0x7f51440782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:15.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.406+0000 7f514ab3e700 1 -- 172.21.15.2:0/4212187728 shutdown_connections 2023-09-13T04:39:15.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.406+0000 7f514ab3e700 1 -- 172.21.15.2:0/4212187728 wait complete. 2023-09-13T04:39:15.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.407+0000 7f514ab3e700 1 Processor -- start 2023-09-13T04:39:15.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.408+0000 7f514ab3e700 1 -- start start 2023-09-13T04:39:15.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.408+0000 7f514ab3e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5144100d80 0x7f5144072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:15.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.408+0000 7f514ab3e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144101740 0x7f51440734f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:15.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.408+0000 7f514ab3e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5144106d90 0x7f5144073a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:15.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.408+0000 7f514ab3e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51441a51f0 con 0x7f5144100d80 2023-09-13T04:39:15.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.408+0000 7f514ab3e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51441a5360 con 0x7f5144101740 2023-09-13T04:39:15.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.408+0000 7f514ab3e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51441a5520 con 0x7f5144106d90 2023-09-13T04:39:15.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.408+0000 7f51488da700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5144100d80 0x7f5144072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:15.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.408+0000 7f51488da700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5144100d80 0x7f5144072fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53448/0 (socket says 172.21.15.2:53448) 2023-09-13T04:39:15.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.408+0000 7f51488da700 1 -- 172.21.15.2:0/2730997546 learned_addr learned my addr 172.21.15.2:0/2730997546 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:15.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.408+0000 7f5143fff700 1 --2- 172.21.15.2:0/2730997546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144101740 0x7f51440734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:15.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.409+0000 7f51490db700 1 --2- 172.21.15.2:0/2730997546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5144106d90 0x7f5144073a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:15.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.409+0000 7f51488da700 1 -- 172.21.15.2:0/2730997546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5144106d90 msgr2=0x7f5144073a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:15.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.409+0000 7f51488da700 1 --2- 172.21.15.2:0/2730997546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5144106d90 0x7f5144073a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:15.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.409+0000 7f51488da700 1 -- 172.21.15.2:0/2730997546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144101740 msgr2=0x7f51440734f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:15.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.409+0000 7f51488da700 1 --2- 172.21.15.2:0/2730997546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144101740 0x7f51440734f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:15.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.409+0000 7f51488da700 1 -- 172.21.15.2:0/2730997546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51300095f0 con 0x7f5144100d80 2023-09-13T04:39:15.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.409+0000 7f51488da700 1 --2- 172.21.15.2:0/2730997546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5144100d80 0x7f5144072fb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f513800e9a0 tx=0x7f513800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:15.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.409+0000 7f5141ffb700 1 -- 172.21.15.2:0/2730997546 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5138009790 con 0x7f5144100d80 2023-09-13T04:39:15.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.409+0000 7f514ab3e700 1 -- 172.21.15.2:0/2730997546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51441a5800 con 0x7f5144100d80 2023-09-13T04:39:15.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.409+0000 7f5141ffb700 1 -- 172.21.15.2:0/2730997546 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f513800cbd0 con 0x7f5144100d80 2023-09-13T04:39:15.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.409+0000 7f5141ffb700 1 -- 172.21.15.2:0/2730997546 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5138018680 con 0x7f5144100d80 2023-09-13T04:39:15.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.410+0000 7f514ab3e700 1 -- 172.21.15.2:0/2730997546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51441a5cd0 con 0x7f5144100d80 2023-09-13T04:39:15.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.411+0000 7f514ab3e700 1 -- 172.21.15.2:0/2730997546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514404e9b0 con 0x7f5144100d80 2023-09-13T04:39:15.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.414+0000 7f5141ffb700 1 -- 172.21.15.2:0/2730997546 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f513800cd40 con 0x7f5144100d80 2023-09-13T04:39:15.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.416+0000 7f5141ffb700 1 --2- 172.21.15.2:0/2730997546 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f51340a0640 0x7f51340a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:15.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.416+0000 7f5141ffb700 1 -- 172.21.15.2:0/2730997546 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5138014070 con 0x7f5144100d80 2023-09-13T04:39:15.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.416+0000 7f5143fff700 1 --2- 172.21.15.2:0/2730997546 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f51340a0640 0x7f51340a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:15.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.417+0000 7f5143fff700 1 --2- 172.21.15.2:0/2730997546 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f51340a0640 0x7f51340a2b00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5130009990 tx=0x7f5130005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:15.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.417+0000 7f5141ffb700 1 -- 172.21.15.2:0/2730997546 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5138095bf0 con 0x7f5144100d80 2023-09-13T04:39:15.528 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574916 got 55834574920 for osd.1 2023-09-13T04:39:15.529 DEBUG:teuthology.parallel:result is None 2023-09-13T04:39:15.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.555+0000 7f60c7205700 1 -- 172.21.15.2:0/2676801568 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f60c0112f90 con 0x7f60c010d730 2023-09-13T04:39:15.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.555+0000 7f60bdffb700 1 -- 172.21.15.2:0/2676801568 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f60b8098d10 con 0x7f60c010d730 2023-09-13T04:39:15.557 INFO:teuthology.orchestra.run.smithi002.stdout:219043332141 2023-09-13T04:39:15.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.558+0000 7f60c7205700 1 -- 172.21.15.2:0/2676801568 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f60b00a06c0 msgr2=0x7f60b00a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:15.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.558+0000 7f60c7205700 1 --2- 172.21.15.2:0/2676801568 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f60b00a06c0 0x7f60b00a2b80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f60b400b2e0 tx=0x7f60b4005fd0 comp rx=0 tx=0).stop 2023-09-13T04:39:15.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.558+0000 7f60c7205700 1 -- 172.21.15.2:0/2676801568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60c010d730 msgr2=0x7f60c019da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:15.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.558+0000 7f60c7205700 1 --2- 172.21.15.2:0/2676801568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60c010d730 0x7f60c019da60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f60b800ef90 tx=0x7f60b800c3a0 comp rx=0 tx=0).stop 2023-09-13T04:39:15.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.558+0000 7f60c7205700 1 -- 172.21.15.2:0/2676801568 shutdown_connections 2023-09-13T04:39:15.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.558+0000 7f60c7205700 1 --2- 172.21.15.2:0/2676801568 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f60b00a06c0 0x7f60b00a2b80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:15.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.558+0000 7f60c7205700 1 --2- 172.21.15.2:0/2676801568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60c010d730 0x7f60c019da60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:15.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.558+0000 7f60c7205700 1 --2- 172.21.15.2:0/2676801568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60c0072ee0 0x7f60c01990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:15.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.558+0000 7f60c7205700 1 --2- 172.21.15.2:0/2676801568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60c0073800 0x7f60c01995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:15.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.558+0000 7f60c7205700 1 -- 172.21.15.2:0/2676801568 >> 172.21.15.2:0/2676801568 conn(0x7f60c0077ea0 msgr2=0x7f60c00fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:15.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.558+0000 7f60c7205700 1 -- 172.21.15.2:0/2676801568 shutdown_connections 2023-09-13T04:39:15.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.558+0000 7f60c7205700 1 -- 172.21.15.2:0/2676801568 wait complete. 2023-09-13T04:39:15.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.661+0000 7f514ab3e700 1 -- 172.21.15.2:0/2730997546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f5144110ec0 con 0x7f5144100d80 2023-09-13T04:39:15.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.662+0000 7f5141ffb700 1 -- 172.21.15.2:0/2730997546 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5138099210 con 0x7f5144100d80 2023-09-13T04:39:15.663 INFO:teuthology.orchestra.run.smithi002.stdout:283467841572 2023-09-13T04:39:15.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.665+0000 7f514ab3e700 1 -- 172.21.15.2:0/2730997546 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f51340a0640 msgr2=0x7f51340a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:15.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.665+0000 7f514ab3e700 1 --2- 172.21.15.2:0/2730997546 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f51340a0640 0x7f51340a2b00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5130009990 tx=0x7f5130005fb0 comp rx=0 tx=0).stop 2023-09-13T04:39:15.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.665+0000 7f514ab3e700 1 -- 172.21.15.2:0/2730997546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5144100d80 msgr2=0x7f5144072fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:15.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.665+0000 7f514ab3e700 1 --2- 172.21.15.2:0/2730997546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5144100d80 0x7f5144072fb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f513800e9a0 tx=0x7f513800ecb0 comp rx=0 tx=0).stop 2023-09-13T04:39:15.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.665+0000 7f514ab3e700 1 -- 172.21.15.2:0/2730997546 shutdown_connections 2023-09-13T04:39:15.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.665+0000 7f514ab3e700 1 --2- 172.21.15.2:0/2730997546 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f51340a0640 0x7f51340a2b00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:15.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.665+0000 7f514ab3e700 1 --2- 172.21.15.2:0/2730997546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5144106d90 0x7f5144073a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:15.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.665+0000 7f514ab3e700 1 --2- 172.21.15.2:0/2730997546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5144100d80 0x7f5144072fb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:15.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.665+0000 7f514ab3e700 1 --2- 172.21.15.2:0/2730997546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144101740 0x7f51440734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:15.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.665+0000 7f514ab3e700 1 -- 172.21.15.2:0/2730997546 >> 172.21.15.2:0/2730997546 conn(0x7f5144077ea0 msgr2=0x7f51440ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:15.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.666+0000 7f514ab3e700 1 -- 172.21.15.2:0/2730997546 shutdown_connections 2023-09-13T04:39:15.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:15.666+0000 7f514ab3e700 1 -- 172.21.15.2:0/2730997546 wait complete. 2023-09-13T04:39:15.696 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:15 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1127481755' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-13T04:39:15.697 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:15 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2676801568' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-13T04:39:15.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:15 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1127481755' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-13T04:39:15.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:15 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2676801568' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-13T04:39:15.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:15 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1127481755' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-13T04:39:15.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:15 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2676801568' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-13T04:39:16.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.129+0000 7fda6ce0c700 1 -- 172.21.15.2:0/892785924 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda68101740 msgr2=0x7fda6810dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:16.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.129+0000 7fda6ce0c700 1 --2- 172.21.15.2:0/892785924 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda68101740 0x7fda6810dbb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fda6000b0d0 tx=0x7fda6000b3e0 comp rx=0 tx=0).stop 2023-09-13T04:39:16.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.129+0000 7fda6ce0c700 1 -- 172.21.15.2:0/892785924 shutdown_connections 2023-09-13T04:39:16.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.129+0000 7fda6ce0c700 1 --2- 172.21.15.2:0/892785924 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda68101740 0x7fda6810dbb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:16.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.129+0000 7fda6ce0c700 1 --2- 172.21.15.2:0/892785924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda68106d90 0x7fda68107170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:16.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.129+0000 7fda6ce0c700 1 --2- 172.21.15.2:0/892785924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda68100d80 0x7fda68101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:16.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.129+0000 7fda6ce0c700 1 -- 172.21.15.2:0/892785924 >> 172.21.15.2:0/892785924 conn(0x7fda68077ea0 msgr2=0x7fda680782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:16.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.129+0000 7fda6ce0c700 1 -- 172.21.15.2:0/892785924 shutdown_connections 2023-09-13T04:39:16.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.130+0000 7fda6ce0c700 1 -- 172.21.15.2:0/892785924 wait complete. 2023-09-13T04:39:16.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.130+0000 7fda6ce0c700 1 Processor -- start 2023-09-13T04:39:16.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.131+0000 7fda6ce0c700 1 -- start start 2023-09-13T04:39:16.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.131+0000 7fda6ce0c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda68100d80 0x7fda681990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:16.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.131+0000 7fda6ce0c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda68101740 0x7fda681995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:16.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.131+0000 7fda6ce0c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda68106d90 0x7fda6819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:16.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.131+0000 7fda6ce0c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6819e060 con 0x7fda68101740 2023-09-13T04:39:16.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.131+0000 7fda6ce0c700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6819e1d0 con 0x7fda68106d90 2023-09-13T04:39:16.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.131+0000 7fda6ce0c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6819e390 con 0x7fda68100d80 2023-09-13T04:39:16.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.131+0000 7fda65d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda68101740 0x7fda681995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:16.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.131+0000 7fda65d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda68101740 0x7fda681995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53452/0 (socket says 172.21.15.2:53452) 2023-09-13T04:39:16.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.131+0000 7fda65d9b700 1 -- 172.21.15.2:0/1430470394 learned_addr learned my addr 172.21.15.2:0/1430470394 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:16.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.131+0000 7fda66d9d700 1 --2- 172.21.15.2:0/1430470394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda68106d90 0x7fda6819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:16.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.132+0000 7fda6659c700 1 --2- 172.21.15.2:0/1430470394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda68100d80 0x7fda681990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:16.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.132+0000 7fda65d9b700 1 -- 172.21.15.2:0/1430470394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda68100d80 msgr2=0x7fda681990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:16.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.132+0000 7fda65d9b700 1 --2- 172.21.15.2:0/1430470394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda68100d80 0x7fda681990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:16.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.132+0000 7fda65d9b700 1 -- 172.21.15.2:0/1430470394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda68106d90 msgr2=0x7fda6819da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:16.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.132+0000 7fda65d9b700 1 --2- 172.21.15.2:0/1430470394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda68106d90 0x7fda6819da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:16.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.132+0000 7fda65d9b700 1 -- 172.21.15.2:0/1430470394 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda58009520 con 0x7fda68101740 2023-09-13T04:39:16.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.132+0000 7fda65d9b700 1 --2- 172.21.15.2:0/1430470394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda68101740 0x7fda681995e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00e9c0 tx=0x7fda5c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:16.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.132+0000 7fda577fe700 1 -- 172.21.15.2:0/1430470394 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c0097b0 con 0x7fda68101740 2023-09-13T04:39:16.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.132+0000 7fda577fe700 1 -- 172.21.15.2:0/1430470394 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda5c00ee60 con 0x7fda68101740 2023-09-13T04:39:16.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.132+0000 7fda6ce0c700 1 -- 172.21.15.2:0/1430470394 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda60009d00 con 0x7fda68101740 2023-09-13T04:39:16.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.132+0000 7fda577fe700 1 -- 172.21.15.2:0/1430470394 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c0054f0 con 0x7fda68101740 2023-09-13T04:39:16.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.133+0000 7fda6ce0c700 1 -- 172.21.15.2:0/1430470394 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda681a5d00 con 0x7fda68101740 2023-09-13T04:39:16.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.134+0000 7fda6ce0c700 1 -- 172.21.15.2:0/1430470394 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda6804e9b0 con 0x7fda68101740 2023-09-13T04:39:16.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.136+0000 7fda577fe700 1 -- 172.21.15.2:0/1430470394 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fda5c004750 con 0x7fda68101740 2023-09-13T04:39:16.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.137+0000 7fda577fe700 1 --2- 172.21.15.2:0/1430470394 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fda500a06c0 0x7fda500a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:16.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.137+0000 7fda577fe700 1 -- 172.21.15.2:0/1430470394 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fda5c014070 con 0x7fda68101740 2023-09-13T04:39:16.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.137+0000 7fda6659c700 1 --2- 172.21.15.2:0/1430470394 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fda500a06c0 0x7fda500a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:16.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.138+0000 7fda6659c700 1 --2- 172.21.15.2:0/1430470394 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fda500a06c0 0x7fda500a2b80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fda58009df0 tx=0x7fda58009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:16.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.141+0000 7fda577fe700 1 -- 172.21.15.2:0/1430470394 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fda5c095880 con 0x7fda68101740 2023-09-13T04:39:16.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.360+0000 7fda6ce0c700 1 -- 172.21.15.2:0/1430470394 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fda68110e10 con 0x7fda68101740 2023-09-13T04:39:16.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.361+0000 7fda577fe700 1 -- 172.21.15.2:0/1430470394 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fda5c098ea0 con 0x7fda68101740 2023-09-13T04:39:16.363 INFO:teuthology.orchestra.run.smithi002.stdout:339302416411 2023-09-13T04:39:16.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.364+0000 7fda6ce0c700 1 -- 172.21.15.2:0/1430470394 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fda500a06c0 msgr2=0x7fda500a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:16.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.364+0000 7fda6ce0c700 1 --2- 172.21.15.2:0/1430470394 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fda500a06c0 0x7fda500a2b80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fda58009df0 tx=0x7fda58009260 comp rx=0 tx=0).stop 2023-09-13T04:39:16.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.364+0000 7fda6ce0c700 1 -- 172.21.15.2:0/1430470394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda68101740 msgr2=0x7fda681995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:16.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.364+0000 7fda6ce0c700 1 --2- 172.21.15.2:0/1430470394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda68101740 0x7fda681995e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00e9c0 tx=0x7fda5c00ecd0 comp rx=0 tx=0).stop 2023-09-13T04:39:16.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.364+0000 7fda6ce0c700 1 -- 172.21.15.2:0/1430470394 shutdown_connections 2023-09-13T04:39:16.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.364+0000 7fda6ce0c700 1 --2- 172.21.15.2:0/1430470394 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fda500a06c0 0x7fda500a2b80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:16.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.364+0000 7fda6ce0c700 1 --2- 172.21.15.2:0/1430470394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda68106d90 0x7fda6819da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:16.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.364+0000 7fda6ce0c700 1 --2- 172.21.15.2:0/1430470394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda68100d80 0x7fda681990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:16.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.364+0000 7fda6ce0c700 1 --2- 172.21.15.2:0/1430470394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda68101740 0x7fda681995e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:16.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.365+0000 7fda6ce0c700 1 -- 172.21.15.2:0/1430470394 >> 172.21.15.2:0/1430470394 conn(0x7fda68077ea0 msgr2=0x7fda680ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:16.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.365+0000 7fda6ce0c700 1 -- 172.21.15.2:0/1430470394 shutdown_connections 2023-09-13T04:39:16.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:16.365+0000 7fda6ce0c700 1 -- 172.21.15.2:0/1430470394 wait complete. 2023-09-13T04:39:16.640 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:16 smithi002 ceph-mon[97651]: pgmap v316: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:39:16.641 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:16 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:39:16.641 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:16 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:39:16.641 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:16 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2730997546' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-13T04:39:16.642 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:16 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1430470394' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-13T04:39:16.643 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411395 got 77309411398 for osd.2 2023-09-13T04:39:16.643 DEBUG:teuthology.parallel:result is None 2023-09-13T04:39:16.883 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182461 got 107374182464 for osd.3 2023-09-13T04:39:16.883 DEBUG:teuthology.parallel:result is None 2023-09-13T04:39:16.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:16 smithi149 ceph-mon[133315]: pgmap v316: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:39:16.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:16 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:39:16.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:16 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:39:16.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:16 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2730997546' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-13T04:39:16.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:16 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1430470394' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-13T04:39:16.918 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841569 got 283467841572 for osd.9 2023-09-13T04:39:16.919 DEBUG:teuthology.parallel:result is None 2023-09-13T04:39:16.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:16 smithi156 ceph-mon[133675]: pgmap v316: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:39:16.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:16 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:39:16.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:16 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:39:16.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:16 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2730997546' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-13T04:39:16.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:16 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1430470394' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-13T04:39:17.051 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332138 got 219043332141 for osd.7 2023-09-13T04:39:17.051 DEBUG:teuthology.parallel:result is None 2023-09-13T04:39:17.358 INFO:tasks.cephadm.ceph_manager.ceph:need seq 339302416408 got 339302416411 for osd.11 2023-09-13T04:39:17.358 DEBUG:teuthology.parallel:result is None 2023-09-13T04:39:17.358 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-09-13T04:39:17.359 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph pg dump --format=json 2023-09-13T04:39:17.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:17 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:39:17.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:17 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:39:17.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:17 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:39:17.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:17 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:39:17.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:17 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:39:17.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:17 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:39:18.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:17 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:39:18.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:17 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:39:18.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:17 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' 2023-09-13T04:39:18.091 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:39:18.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:18 smithi156 ceph-mon[133675]: pgmap v317: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:39:19.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:18 smithi002 ceph-mon[97651]: pgmap v317: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:39:19.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:18 smithi149 ceph-mon[133315]: pgmap v317: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:39:19.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.713+0000 7f83ec5b7700 1 -- 172.21.15.2:0/951910694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83e4068a00 msgr2=0x7f83e4068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:19.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.713+0000 7f83ec5b7700 1 --2- 172.21.15.2:0/951910694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83e4068a00 0x7f83e4068de0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009990 tx=0x7f83e0009ca0 comp rx=0 tx=0).stop 2023-09-13T04:39:19.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.713+0000 7f83ec5b7700 1 -- 172.21.15.2:0/951910694 shutdown_connections 2023-09-13T04:39:19.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.713+0000 7f83ec5b7700 1 --2- 172.21.15.2:0/951910694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83e41060b0 0x7f83e410dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:19.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.713+0000 7f83ec5b7700 1 --2- 172.21.15.2:0/951910694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83e4068a00 0x7f83e4068de0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:19.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.713+0000 7f83ec5b7700 1 --2- 172.21.15.2:0/951910694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83e40693b0 0x7f83e4105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:19.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.713+0000 7f83ec5b7700 1 -- 172.21.15.2:0/951910694 >> 172.21.15.2:0/951910694 conn(0x7f83e4075c00 msgr2=0x7f83e4076010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:19.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.713+0000 7f83ec5b7700 1 -- 172.21.15.2:0/951910694 shutdown_connections 2023-09-13T04:39:19.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.713+0000 7f83ec5b7700 1 -- 172.21.15.2:0/951910694 wait complete. 2023-09-13T04:39:19.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.715+0000 7f83ec5b7700 1 Processor -- start 2023-09-13T04:39:19.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.715+0000 7f83ec5b7700 1 -- start start 2023-09-13T04:39:19.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.716+0000 7f83ec5b7700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83e4068a00 0x7f83e4196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:19.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.716+0000 7f83ec5b7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83e40693b0 0x7f83e4197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:19.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.716+0000 7f83ec5b7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83e41060b0 0x7f83e419b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:19.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.716+0000 7f83ec5b7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e419bea0 con 0x7f83e41060b0 2023-09-13T04:39:19.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.716+0000 7f83ec5b7700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e419c010 con 0x7f83e4068a00 2023-09-13T04:39:19.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.716+0000 7f83ec5b7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e419c1d0 con 0x7f83e40693b0 2023-09-13T04:39:19.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.716+0000 7f83eab54700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83e41060b0 0x7f83e419b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:19.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.716+0000 7f83eab54700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83e41060b0 0x7f83e419b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53476/0 (socket says 172.21.15.2:53476) 2023-09-13T04:39:19.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.716+0000 7f83eab54700 1 -- 172.21.15.2:0/278221622 learned_addr learned my addr 172.21.15.2:0/278221622 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:19.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.716+0000 7f83e9b52700 1 --2- 172.21.15.2:0/278221622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83e40693b0 0x7f83e4197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:19.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.716+0000 7f83ea353700 1 --2- 172.21.15.2:0/278221622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83e4068a00 0x7f83e4196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:19.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.716+0000 7f83eab54700 1 -- 172.21.15.2:0/278221622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83e40693b0 msgr2=0x7f83e4197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:19.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.716+0000 7f83eab54700 1 --2- 172.21.15.2:0/278221622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83e40693b0 0x7f83e4197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:19.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.716+0000 7f83eab54700 1 -- 172.21.15.2:0/278221622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83e4068a00 msgr2=0x7f83e4196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:19.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.716+0000 7f83eab54700 1 --2- 172.21.15.2:0/278221622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83e4068a00 0x7f83e4196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:19.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.716+0000 7f83eab54700 1 -- 172.21.15.2:0/278221622 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e00095f0 con 0x7f83e41060b0 2023-09-13T04:39:19.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.717+0000 7f83eab54700 1 --2- 172.21.15.2:0/278221622 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83e41060b0 0x7f83e419b890 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f83dc00ada0 tx=0x7f83dc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:19.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.717+0000 7f83db7fe700 1 -- 172.21.15.2:0/278221622 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc00f070 con 0x7f83e41060b0 2023-09-13T04:39:19.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.717+0000 7f83ec5b7700 1 -- 172.21.15.2:0/278221622 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e419c4b0 con 0x7f83e41060b0 2023-09-13T04:39:19.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.717+0000 7f83db7fe700 1 -- 172.21.15.2:0/278221622 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83dc007b30 con 0x7f83e41060b0 2023-09-13T04:39:19.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.717+0000 7f83db7fe700 1 -- 172.21.15.2:0/278221622 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc005280 con 0x7f83e41060b0 2023-09-13T04:39:19.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.717+0000 7f83ec5b7700 1 -- 172.21.15.2:0/278221622 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e41a3d10 con 0x7f83e41060b0 2023-09-13T04:39:19.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.718+0000 7f83ec5b7700 1 -- 172.21.15.2:0/278221622 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e4109280 con 0x7f83e41060b0 2023-09-13T04:39:19.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.720+0000 7f83db7fe700 1 -- 172.21.15.2:0/278221622 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f83dc00c8d0 con 0x7f83e41060b0 2023-09-13T04:39:19.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.722+0000 7f83db7fe700 1 --2- 172.21.15.2:0/278221622 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f83d00a0640 0x7f83d00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:19.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.722+0000 7f83db7fe700 1 -- 172.21.15.2:0/278221622 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f83dc0cb370 con 0x7f83e41060b0 2023-09-13T04:39:19.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.722+0000 7f83ea353700 1 --2- 172.21.15.2:0/278221622 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f83d00a0640 0x7f83d00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:19.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.723+0000 7f83ea353700 1 --2- 172.21.15.2:0/278221622 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f83d00a0640 0x7f83d00a2b00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f83e000b5c0 tx=0x7f83e0005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:19.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.726+0000 7f83db7fe700 1 -- 172.21.15.2:0/278221622 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f83dc094c20 con 0x7f83e41060b0 2023-09-13T04:39:19.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.930+0000 7f83ec5b7700 1 -- 172.21.15.2:0/278221622 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f83e4110cc0 con 0x7f83d00a0640 2023-09-13T04:39:19.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.944+0000 7f83db7fe700 1 -- 172.21.15.2:0/278221622 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426821 (secure 0 0 0) 0x7f83e4110cc0 con 0x7f83d00a0640 2023-09-13T04:39:19.954 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:39:19.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.951+0000 7f83ec5b7700 1 -- 172.21.15.2:0/278221622 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f83d00a0640 msgr2=0x7f83d00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:19.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.951+0000 7f83ec5b7700 1 --2- 172.21.15.2:0/278221622 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f83d00a0640 0x7f83d00a2b00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f83e000b5c0 tx=0x7f83e0005fd0 comp rx=0 tx=0).stop 2023-09-13T04:39:19.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.951+0000 7f83ec5b7700 1 -- 172.21.15.2:0/278221622 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83e41060b0 msgr2=0x7f83e419b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:19.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.951+0000 7f83ec5b7700 1 --2- 172.21.15.2:0/278221622 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83e41060b0 0x7f83e419b890 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f83dc00ada0 tx=0x7f83dc00c330 comp rx=0 tx=0).stop 2023-09-13T04:39:19.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.951+0000 7f83ec5b7700 1 -- 172.21.15.2:0/278221622 shutdown_connections 2023-09-13T04:39:19.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.951+0000 7f83ec5b7700 1 --2- 172.21.15.2:0/278221622 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f83d00a0640 0x7f83d00a2b00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:19.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.951+0000 7f83ec5b7700 1 --2- 172.21.15.2:0/278221622 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83e41060b0 0x7f83e419b890 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:19.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.951+0000 7f83ec5b7700 1 --2- 172.21.15.2:0/278221622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83e4068a00 0x7f83e4196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:19.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.951+0000 7f83ec5b7700 1 --2- 172.21.15.2:0/278221622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83e40693b0 0x7f83e4197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:19.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.951+0000 7f83ec5b7700 1 -- 172.21.15.2:0/278221622 >> 172.21.15.2:0/278221622 conn(0x7f83e4075c00 msgr2=0x7f83e40ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:19.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.951+0000 7f83ec5b7700 1 -- 172.21.15.2:0/278221622 shutdown_connections 2023-09-13T04:39:19.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:19.951+0000 7f83ec5b7700 1 -- 172.21.15.2:0/278221622 wait complete. 2023-09-13T04:39:19.961 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-09-13T04:39:20.484 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2023-09-13T04:39:19.520979+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597820,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":452,"num_read_kb":290,"num_write":310,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":250,"ondisk_log_size":250,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337928,"kb_used_data":13684,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603896,"statfs":{"total":1151940427776,"available":1151594389504,"internally_reserved":0,"allocated":14012416,"data_stored":7673352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003395"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592832+0000","last_change":"2023-09-13T04:37:59.244279+0000","last_active":"2023-09-13T04:38:23.592832+0000","last_peered":"2023-09-13T04:38:23.592832+0000","last_clean":"2023-09-13T04:38:23.592832+0000","last_became_active":"2023-09-13T04:37:59.243976+0000","last_became_peered":"2023-09-13T04:37:59.243976+0000","last_unstale":"2023-09-13T04:38:23.592832+0000","last_undegraded":"2023-09-13T04:38:23.592832+0000","last_fullsized":"2023-09-13T04:38:23.592832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:21:52.809411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592579+0000","last_change":"2023-09-13T04:37:59.243391+0000","last_active":"2023-09-13T04:38:23.592579+0000","last_peered":"2023-09-13T04:38:23.592579+0000","last_clean":"2023-09-13T04:38:23.592579+0000","last_became_active":"2023-09-13T04:37:59.243044+0000","last_became_peered":"2023-09-13T04:37:59.243044+0000","last_unstale":"2023-09-13T04:38:23.592579+0000","last_undegraded":"2023-09-13T04:38:23.592579+0000","last_fullsized":"2023-09-13T04:38:23.592579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:02:17.806638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592756+0000","last_change":"2023-09-13T04:37:59.242826+0000","last_active":"2023-09-13T04:38:23.592756+0000","last_peered":"2023-09-13T04:38:23.592756+0000","last_clean":"2023-09-13T04:38:23.592756+0000","last_became_active":"2023-09-13T04:37:59.242125+0000","last_became_peered":"2023-09-13T04:37:59.242125+0000","last_unstale":"2023-09-13T04:38:23.592756+0000","last_undegraded":"2023-09-13T04:38:23.592756+0000","last_fullsized":"2023-09-13T04:38:23.592756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:08:29.216093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590536+0000","last_change":"2023-09-13T04:37:59.253322+0000","last_active":"2023-09-13T04:38:23.590536+0000","last_peered":"2023-09-13T04:38:23.590536+0000","last_clean":"2023-09-13T04:38:23.590536+0000","last_became_active":"2023-09-13T04:37:59.253050+0000","last_became_peered":"2023-09-13T04:37:59.253050+0000","last_unstale":"2023-09-13T04:38:23.590536+0000","last_undegraded":"2023-09-13T04:38:23.590536+0000","last_fullsized":"2023-09-13T04:38:23.590536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:45:22.357879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593487+0000","last_change":"2023-09-13T04:37:59.250089+0000","last_active":"2023-09-13T04:38:23.593487+0000","last_peered":"2023-09-13T04:38:23.593487+0000","last_clean":"2023-09-13T04:38:23.593487+0000","last_became_active":"2023-09-13T04:37:59.249771+0000","last_became_peered":"2023-09-13T04:37:59.249771+0000","last_unstale":"2023-09-13T04:38:23.593487+0000","last_undegraded":"2023-09-13T04:38:23.593487+0000","last_fullsized":"2023-09-13T04:38:23.593487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:46:59.917756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595139+0000","last_change":"2023-09-13T04:37:59.247259+0000","last_active":"2023-09-13T04:38:23.595139+0000","last_peered":"2023-09-13T04:38:23.595139+0000","last_clean":"2023-09-13T04:38:23.595139+0000","last_became_active":"2023-09-13T04:37:59.246911+0000","last_became_peered":"2023-09-13T04:37:59.246911+0000","last_unstale":"2023-09-13T04:38:23.595139+0000","last_undegraded":"2023-09-13T04:38:23.595139+0000","last_fullsized":"2023-09-13T04:38:23.595139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:00:39.742392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587857+0000","last_change":"2023-09-13T04:37:59.244083+0000","last_active":"2023-09-13T04:38:23.587857+0000","last_peered":"2023-09-13T04:38:23.587857+0000","last_clean":"2023-09-13T04:38:23.587857+0000","last_became_active":"2023-09-13T04:37:59.243789+0000","last_became_peered":"2023-09-13T04:37:59.243789+0000","last_unstale":"2023-09-13T04:38:23.587857+0000","last_undegraded":"2023-09-13T04:38:23.587857+0000","last_fullsized":"2023-09-13T04:38:23.587857+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:24:00.622790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.602097+0000","last_change":"2023-09-13T04:38:24.601265+0000","last_active":"2023-09-13T04:38:24.602097+0000","last_peered":"2023-09-13T04:38:24.602097+0000","last_clean":"2023-09-13T04:38:24.602097+0000","last_became_active":"2023-09-13T04:38:24.600939+0000","last_became_peered":"2023-09-13T04:38:24.600939+0000","last_unstale":"2023-09-13T04:38:24.602097+0000","last_undegraded":"2023-09-13T04:38:24.602097+0000","last_fullsized":"2023-09-13T04:38:24.602097+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:18:23.642333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594759+0000","last_change":"2023-09-13T04:37:59.238053+0000","last_active":"2023-09-13T04:38:23.594759+0000","last_peered":"2023-09-13T04:38:23.594759+0000","last_clean":"2023-09-13T04:38:23.594759+0000","last_became_active":"2023-09-13T04:37:59.237777+0000","last_became_peered":"2023-09-13T04:37:59.237777+0000","last_unstale":"2023-09-13T04:38:23.594759+0000","last_undegraded":"2023-09-13T04:38:23.594759+0000","last_fullsized":"2023-09-13T04:38:23.594759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:45:39.835659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594929+0000","last_change":"2023-09-13T04:37:59.252605+0000","last_active":"2023-09-13T04:38:23.594929+0000","last_peered":"2023-09-13T04:38:23.594929+0000","last_clean":"2023-09-13T04:38:23.594929+0000","last_became_active":"2023-09-13T04:37:59.252309+0000","last_became_peered":"2023-09-13T04:37:59.252309+0000","last_unstale":"2023-09-13T04:38:23.594929+0000","last_undegraded":"2023-09-13T04:38:23.594929+0000","last_fullsized":"2023-09-13T04:38:23.594929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:41:45.064230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588277+0000","last_change":"2023-09-13T04:37:59.243345+0000","last_active":"2023-09-13T04:38:23.588277+0000","last_peered":"2023-09-13T04:38:23.588277+0000","last_clean":"2023-09-13T04:38:23.588277+0000","last_became_active":"2023-09-13T04:37:59.242726+0000","last_became_peered":"2023-09-13T04:37:59.242726+0000","last_unstale":"2023-09-13T04:38:23.588277+0000","last_undegraded":"2023-09-13T04:38:23.588277+0000","last_fullsized":"2023-09-13T04:38:23.588277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:39:55.115281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590601+0000","last_change":"2023-09-13T04:37:59.247415+0000","last_active":"2023-09-13T04:38:23.590601+0000","last_peered":"2023-09-13T04:38:23.590601+0000","last_clean":"2023-09-13T04:38:23.590601+0000","last_became_active":"2023-09-13T04:37:59.247050+0000","last_became_peered":"2023-09-13T04:37:59.247050+0000","last_unstale":"2023-09-13T04:38:23.590601+0000","last_undegraded":"2023-09-13T04:38:23.590601+0000","last_fullsized":"2023-09-13T04:38:23.590601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:03:44.793628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597902+0000","last_change":"2023-09-13T04:37:59.248935+0000","last_active":"2023-09-13T04:38:23.597902+0000","last_peered":"2023-09-13T04:38:23.597902+0000","last_clean":"2023-09-13T04:38:23.597902+0000","last_became_active":"2023-09-13T04:37:59.248554+0000","last_became_peered":"2023-09-13T04:37:59.248554+0000","last_unstale":"2023-09-13T04:38:23.597902+0000","last_undegraded":"2023-09-13T04:38:23.597902+0000","last_fullsized":"2023-09-13T04:38:23.597902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:53:31.882567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592020+0000","last_change":"2023-09-13T04:37:59.242851+0000","last_active":"2023-09-13T04:38:23.592020+0000","last_peered":"2023-09-13T04:38:23.592020+0000","last_clean":"2023-09-13T04:38:23.592020+0000","last_became_active":"2023-09-13T04:37:59.242384+0000","last_became_peered":"2023-09-13T04:37:59.242384+0000","last_unstale":"2023-09-13T04:38:23.592020+0000","last_undegraded":"2023-09-13T04:38:23.592020+0000","last_fullsized":"2023-09-13T04:38:23.592020+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:59:36.508808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.599392+0000","last_change":"2023-09-13T04:38:24.599392+0000","last_active":"2023-09-13T04:38:24.599392+0000","last_peered":"2023-09-13T04:38:24.599392+0000","last_clean":"2023-09-13T04:38:24.599392+0000","last_became_active":"2023-09-13T04:38:24.599029+0000","last_became_peered":"2023-09-13T04:38:24.599029+0000","last_unstale":"2023-09-13T04:38:24.599392+0000","last_undegraded":"2023-09-13T04:38:24.599392+0000","last_fullsized":"2023-09-13T04:38:24.599392+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:23:03.703387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592657+0000","last_change":"2023-09-13T04:37:59.249107+0000","last_active":"2023-09-13T04:38:23.592657+0000","last_peered":"2023-09-13T04:38:23.592657+0000","last_clean":"2023-09-13T04:38:23.592657+0000","last_became_active":"2023-09-13T04:37:59.248803+0000","last_became_peered":"2023-09-13T04:37:59.248803+0000","last_unstale":"2023-09-13T04:38:23.592657+0000","last_undegraded":"2023-09-13T04:38:23.592657+0000","last_fullsized":"2023-09-13T04:38:23.592657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:50:52.481194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592743+0000","last_change":"2023-09-13T04:37:59.243724+0000","last_active":"2023-09-13T04:38:23.592743+0000","last_peered":"2023-09-13T04:38:23.592743+0000","last_clean":"2023-09-13T04:38:23.592743+0000","last_became_active":"2023-09-13T04:37:59.243385+0000","last_became_peered":"2023-09-13T04:37:59.243385+0000","last_unstale":"2023-09-13T04:38:23.592743+0000","last_undegraded":"2023-09-13T04:38:23.592743+0000","last_fullsized":"2023-09-13T04:38:23.592743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:55:55.203940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588550+0000","last_change":"2023-09-13T04:37:59.238986+0000","last_active":"2023-09-13T04:38:23.588550+0000","last_peered":"2023-09-13T04:38:23.588550+0000","last_clean":"2023-09-13T04:38:23.588550+0000","last_became_active":"2023-09-13T04:37:59.238668+0000","last_became_peered":"2023-09-13T04:37:59.238668+0000","last_unstale":"2023-09-13T04:38:23.588550+0000","last_undegraded":"2023-09-13T04:38:23.588550+0000","last_fullsized":"2023-09-13T04:38:23.588550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:21:10.225223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588585+0000","last_change":"2023-09-13T04:37:59.245598+0000","last_active":"2023-09-13T04:38:23.588585+0000","last_peered":"2023-09-13T04:38:23.588585+0000","last_clean":"2023-09-13T04:38:23.588585+0000","last_became_active":"2023-09-13T04:37:59.244652+0000","last_became_peered":"2023-09-13T04:37:59.244652+0000","last_unstale":"2023-09-13T04:38:23.588585+0000","last_undegraded":"2023-09-13T04:38:23.588585+0000","last_fullsized":"2023-09-13T04:38:23.588585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:34:43.671168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588238+0000","last_change":"2023-09-13T04:38:03.617708+0000","last_active":"2023-09-13T04:38:23.588238+0000","last_peered":"2023-09-13T04:38:23.588238+0000","last_clean":"2023-09-13T04:38:23.588238+0000","last_became_active":"2023-09-13T04:38:03.617277+0000","last_became_peered":"2023-09-13T04:38:03.617277+0000","last_unstale":"2023-09-13T04:38:23.588238+0000","last_undegraded":"2023-09-13T04:38:23.588238+0000","last_fullsized":"2023-09-13T04:38:23.588238+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:26:44.318030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.621267+0000","last_change":"2023-09-13T04:37:58.595156+0000","last_active":"2023-09-13T04:38:25.621267+0000","last_peered":"2023-09-13T04:38:25.621267+0000","last_clean":"2023-09-13T04:38:25.621267+0000","last_became_active":"2023-09-13T04:37:58.594863+0000","last_became_peered":"2023-09-13T04:37:58.594863+0000","last_unstale":"2023-09-13T04:38:25.621267+0000","last_undegraded":"2023-09-13T04:38:25.621267+0000","last_fullsized":"2023-09-13T04:38:25.621267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:27:14.420495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595813+0000","last_change":"2023-09-13T04:37:59.250252+0000","last_active":"2023-09-13T04:38:23.595813+0000","last_peered":"2023-09-13T04:38:23.595813+0000","last_clean":"2023-09-13T04:38:23.595813+0000","last_became_active":"2023-09-13T04:37:59.249927+0000","last_became_peered":"2023-09-13T04:37:59.249927+0000","last_unstale":"2023-09-13T04:38:23.595813+0000","last_undegraded":"2023-09-13T04:38:23.595813+0000","last_fullsized":"2023-09-13T04:38:23.595813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:18:06.757040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.599683+0000","last_change":"2023-09-13T04:38:24.599683+0000","last_active":"2023-09-13T04:38:24.599683+0000","last_peered":"2023-09-13T04:38:24.599683+0000","last_clean":"2023-09-13T04:38:24.599683+0000","last_became_active":"2023-09-13T04:38:24.599230+0000","last_became_peered":"2023-09-13T04:38:24.599230+0000","last_unstale":"2023-09-13T04:38:24.599683+0000","last_undegraded":"2023-09-13T04:38:24.599683+0000","last_fullsized":"2023-09-13T04:38:24.599683+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:48:22.584735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.596874+0000","last_change":"2023-09-13T04:37:58.597785+0000","last_active":"2023-09-13T04:38:24.596874+0000","last_peered":"2023-09-13T04:38:24.596874+0000","last_clean":"2023-09-13T04:38:24.596874+0000","last_became_active":"2023-09-13T04:37:58.597485+0000","last_became_peered":"2023-09-13T04:37:58.597485+0000","last_unstale":"2023-09-13T04:38:24.596874+0000","last_undegraded":"2023-09-13T04:38:24.596874+0000","last_fullsized":"2023-09-13T04:38:24.596874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:39:57.156038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592830+0000","last_change":"2023-09-13T04:37:59.241840+0000","last_active":"2023-09-13T04:38:23.592830+0000","last_peered":"2023-09-13T04:38:23.592830+0000","last_clean":"2023-09-13T04:38:23.592830+0000","last_became_active":"2023-09-13T04:37:59.241545+0000","last_became_peered":"2023-09-13T04:37:59.241545+0000","last_unstale":"2023-09-13T04:38:23.592830+0000","last_undegraded":"2023-09-13T04:38:23.592830+0000","last_fullsized":"2023-09-13T04:38:23.592830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:04:17.439520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591101+0000","last_change":"2023-09-13T04:38:03.621792+0000","last_active":"2023-09-13T04:38:23.591101+0000","last_peered":"2023-09-13T04:38:23.591101+0000","last_clean":"2023-09-13T04:38:23.591101+0000","last_became_active":"2023-09-13T04:38:03.621433+0000","last_became_peered":"2023-09-13T04:38:03.621433+0000","last_unstale":"2023-09-13T04:38:23.591101+0000","last_undegraded":"2023-09-13T04:38:23.591101+0000","last_fullsized":"2023-09-13T04:38:23.591101+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:52:05.926733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.618200+0000","last_change":"2023-09-13T04:37:58.590817+0000","last_active":"2023-09-13T04:38:25.618200+0000","last_peered":"2023-09-13T04:38:25.618200+0000","last_clean":"2023-09-13T04:38:25.618200+0000","last_became_active":"2023-09-13T04:37:58.590348+0000","last_became_peered":"2023-09-13T04:37:58.590348+0000","last_unstale":"2023-09-13T04:38:25.618200+0000","last_undegraded":"2023-09-13T04:38:25.618200+0000","last_fullsized":"2023-09-13T04:38:25.618200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:38:35.680905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594252+0000","last_change":"2023-09-13T04:37:59.237304+0000","last_active":"2023-09-13T04:38:23.594252+0000","last_peered":"2023-09-13T04:38:23.594252+0000","last_clean":"2023-09-13T04:38:23.594252+0000","last_became_active":"2023-09-13T04:37:59.237008+0000","last_became_peered":"2023-09-13T04:37:59.237008+0000","last_unstale":"2023-09-13T04:38:23.594252+0000","last_undegraded":"2023-09-13T04:38:23.594252+0000","last_fullsized":"2023-09-13T04:38:23.594252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:28:14.254219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590193+0000","last_change":"2023-09-13T04:38:03.606749+0000","last_active":"2023-09-13T04:38:23.590193+0000","last_peered":"2023-09-13T04:38:23.590193+0000","last_clean":"2023-09-13T04:38:23.590193+0000","last_became_active":"2023-09-13T04:38:03.606276+0000","last_became_peered":"2023-09-13T04:38:03.606276+0000","last_unstale":"2023-09-13T04:38:23.590193+0000","last_undegraded":"2023-09-13T04:38:23.590193+0000","last_fullsized":"2023-09-13T04:38:23.590193+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:30:53.572543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593692+0000","last_change":"2023-09-13T04:37:58.590937+0000","last_active":"2023-09-13T04:38:23.593692+0000","last_peered":"2023-09-13T04:38:23.593692+0000","last_clean":"2023-09-13T04:38:23.593692+0000","last_became_active":"2023-09-13T04:37:58.590606+0000","last_became_peered":"2023-09-13T04:37:58.590606+0000","last_unstale":"2023-09-13T04:38:23.593692+0000","last_undegraded":"2023-09-13T04:38:23.593692+0000","last_fullsized":"2023-09-13T04:38:23.593692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:30:03.354670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591224+0000","last_change":"2023-09-13T04:37:59.251899+0000","last_active":"2023-09-13T04:38:23.591224+0000","last_peered":"2023-09-13T04:38:23.591224+0000","last_clean":"2023-09-13T04:38:23.591224+0000","last_became_active":"2023-09-13T04:37:59.251557+0000","last_became_peered":"2023-09-13T04:37:59.251557+0000","last_unstale":"2023-09-13T04:38:23.591224+0000","last_undegraded":"2023-09-13T04:38:23.591224+0000","last_fullsized":"2023-09-13T04:38:23.591224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:57:45.241188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588439+0000","last_change":"2023-09-13T04:38:03.606076+0000","last_active":"2023-09-13T04:38:23.588439+0000","last_peered":"2023-09-13T04:38:23.588439+0000","last_clean":"2023-09-13T04:38:23.588439+0000","last_became_active":"2023-09-13T04:38:03.605556+0000","last_became_peered":"2023-09-13T04:38:03.605556+0000","last_unstale":"2023-09-13T04:38:23.588439+0000","last_undegraded":"2023-09-13T04:38:23.588439+0000","last_fullsized":"2023-09-13T04:38:23.588439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:03:32.344625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":76,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:39:14.688703+0000","last_change":"2023-09-13T04:37:58.588170+0000","last_active":"2023-09-13T04:39:14.688703+0000","last_peered":"2023-09-13T04:39:14.688703+0000","last_clean":"2023-09-13T04:39:14.688703+0000","last_became_active":"2023-09-13T04:37:58.587860+0000","last_became_peered":"2023-09-13T04:37:58.587860+0000","last_unstale":"2023-09-13T04:39:14.688703+0000","last_undegraded":"2023-09-13T04:39:14.688703+0000","last_fullsized":"2023-09-13T04:39:14.688703+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:38:40.048150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595472+0000","last_change":"2023-09-13T04:37:59.240248+0000","last_active":"2023-09-13T04:38:23.595472+0000","last_peered":"2023-09-13T04:38:23.595472+0000","last_clean":"2023-09-13T04:38:23.595472+0000","last_became_active":"2023-09-13T04:37:59.240008+0000","last_became_peered":"2023-09-13T04:37:59.240008+0000","last_unstale":"2023-09-13T04:38:23.595472+0000","last_undegraded":"2023-09-13T04:38:23.595472+0000","last_fullsized":"2023-09-13T04:38:23.595472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:15:13.275110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587821+0000","last_change":"2023-09-13T04:38:03.616607+0000","last_active":"2023-09-13T04:38:23.587821+0000","last_peered":"2023-09-13T04:38:23.587821+0000","last_clean":"2023-09-13T04:38:23.587821+0000","last_became_active":"2023-09-13T04:38:03.616209+0000","last_became_peered":"2023-09-13T04:38:03.616209+0000","last_unstale":"2023-09-13T04:38:23.587821+0000","last_undegraded":"2023-09-13T04:38:23.587821+0000","last_fullsized":"2023-09-13T04:38:23.587821+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:24:21.934053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.617180+0000","last_change":"2023-09-13T04:37:58.589104+0000","last_active":"2023-09-13T04:38:25.617180+0000","last_peered":"2023-09-13T04:38:25.617180+0000","last_clean":"2023-09-13T04:38:25.617180+0000","last_became_active":"2023-09-13T04:37:58.588780+0000","last_became_peered":"2023-09-13T04:37:58.588780+0000","last_unstale":"2023-09-13T04:38:25.617180+0000","last_undegraded":"2023-09-13T04:38:25.617180+0000","last_fullsized":"2023-09-13T04:38:25.617180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:01:32.839729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.600030+0000","last_change":"2023-09-13T04:38:24.598663+0000","last_active":"2023-09-13T04:38:24.600030+0000","last_peered":"2023-09-13T04:38:24.600030+0000","last_clean":"2023-09-13T04:38:24.600030+0000","last_became_active":"2023-09-13T04:38:24.598258+0000","last_became_peered":"2023-09-13T04:38:24.598258+0000","last_unstale":"2023-09-13T04:38:24.600030+0000","last_undegraded":"2023-09-13T04:38:24.600030+0000","last_fullsized":"2023-09-13T04:38:24.600030+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:50:28.157135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592633+0000","last_change":"2023-09-13T04:38:03.619930+0000","last_active":"2023-09-13T04:38:23.592633+0000","last_peered":"2023-09-13T04:38:23.592633+0000","last_clean":"2023-09-13T04:38:23.592633+0000","last_became_active":"2023-09-13T04:38:03.619560+0000","last_became_peered":"2023-09-13T04:38:03.619560+0000","last_unstale":"2023-09-13T04:38:23.592633+0000","last_undegraded":"2023-09-13T04:38:23.592633+0000","last_fullsized":"2023-09-13T04:38:23.592633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:48:06.867602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.619713+0000","last_change":"2023-09-13T04:37:58.590396+0000","last_active":"2023-09-13T04:38:25.619713+0000","last_peered":"2023-09-13T04:38:25.619713+0000","last_clean":"2023-09-13T04:38:25.619713+0000","last_became_active":"2023-09-13T04:37:58.589929+0000","last_became_peered":"2023-09-13T04:37:58.589929+0000","last_unstale":"2023-09-13T04:38:25.619713+0000","last_undegraded":"2023-09-13T04:38:25.619713+0000","last_fullsized":"2023-09-13T04:38:25.619713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:28:36.322096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597620+0000","last_change":"2023-09-13T04:37:59.246033+0000","last_active":"2023-09-13T04:38:23.597620+0000","last_peered":"2023-09-13T04:38:23.597620+0000","last_clean":"2023-09-13T04:38:23.597620+0000","last_became_active":"2023-09-13T04:37:59.245664+0000","last_became_peered":"2023-09-13T04:37:59.245664+0000","last_unstale":"2023-09-13T04:38:23.597620+0000","last_undegraded":"2023-09-13T04:38:23.597620+0000","last_fullsized":"2023-09-13T04:38:23.597620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:24:17.145055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592344+0000","last_change":"2023-09-13T04:38:03.608504+0000","last_active":"2023-09-13T04:38:23.592344+0000","last_peered":"2023-09-13T04:38:23.592344+0000","last_clean":"2023-09-13T04:38:23.592344+0000","last_became_active":"2023-09-13T04:38:03.608058+0000","last_became_peered":"2023-09-13T04:38:03.608058+0000","last_unstale":"2023-09-13T04:38:23.592344+0000","last_undegraded":"2023-09-13T04:38:23.592344+0000","last_fullsized":"2023-09-13T04:38:23.592344+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:42:43.485453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244691+0000","last_change":"2023-09-13T04:37:58.588208+0000","last_active":"2023-09-13T04:38:24.244691+0000","last_peered":"2023-09-13T04:38:24.244691+0000","last_clean":"2023-09-13T04:38:24.244691+0000","last_became_active":"2023-09-13T04:37:58.587840+0000","last_became_peered":"2023-09-13T04:37:58.587840+0000","last_unstale":"2023-09-13T04:38:24.244691+0000","last_undegraded":"2023-09-13T04:38:24.244691+0000","last_fullsized":"2023-09-13T04:38:24.244691+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:44:58.012233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587683+0000","last_change":"2023-09-13T04:37:59.245102+0000","last_active":"2023-09-13T04:38:23.587683+0000","last_peered":"2023-09-13T04:38:23.587683+0000","last_clean":"2023-09-13T04:38:23.587683+0000","last_became_active":"2023-09-13T04:37:59.244799+0000","last_became_peered":"2023-09-13T04:37:59.244799+0000","last_unstale":"2023-09-13T04:38:23.587683+0000","last_undegraded":"2023-09-13T04:38:23.587683+0000","last_fullsized":"2023-09-13T04:38:23.587683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:57:16.295623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.596645+0000","last_change":"2023-09-13T04:38:03.607021+0000","last_active":"2023-09-13T04:38:23.596645+0000","last_peered":"2023-09-13T04:38:23.596645+0000","last_clean":"2023-09-13T04:38:23.596645+0000","last_became_active":"2023-09-13T04:38:03.606611+0000","last_became_peered":"2023-09-13T04:38:03.606611+0000","last_unstale":"2023-09-13T04:38:23.596645+0000","last_undegraded":"2023-09-13T04:38:23.596645+0000","last_fullsized":"2023-09-13T04:38:23.596645+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:33:20.551479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":70,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:39:13.686637+0000","last_change":"2023-09-13T04:37:58.588731+0000","last_active":"2023-09-13T04:39:13.686637+0000","last_peered":"2023-09-13T04:39:13.686637+0000","last_clean":"2023-09-13T04:39:13.686637+0000","last_became_active":"2023-09-13T04:37:58.588398+0000","last_became_peered":"2023-09-13T04:37:58.588398+0000","last_unstale":"2023-09-13T04:39:13.686637+0000","last_undegraded":"2023-09-13T04:39:13.686637+0000","last_fullsized":"2023-09-13T04:39:13.686637+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:13:10.724085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594790+0000","last_change":"2023-09-13T04:37:59.244757+0000","last_active":"2023-09-13T04:38:23.594790+0000","last_peered":"2023-09-13T04:38:23.594790+0000","last_clean":"2023-09-13T04:38:23.594790+0000","last_became_active":"2023-09-13T04:37:59.244423+0000","last_became_peered":"2023-09-13T04:37:59.244423+0000","last_unstale":"2023-09-13T04:38:23.594790+0000","last_undegraded":"2023-09-13T04:38:23.594790+0000","last_fullsized":"2023-09-13T04:38:23.594790+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:14:37.973058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587303+0000","last_change":"2023-09-13T04:38:03.618358+0000","last_active":"2023-09-13T04:38:23.587303+0000","last_peered":"2023-09-13T04:38:23.587303+0000","last_clean":"2023-09-13T04:38:23.587303+0000","last_became_active":"2023-09-13T04:38:03.617291+0000","last_became_peered":"2023-09-13T04:38:03.617291+0000","last_unstale":"2023-09-13T04:38:23.587303+0000","last_undegraded":"2023-09-13T04:38:23.587303+0000","last_fullsized":"2023-09-13T04:38:23.587303+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:33:13.955353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244519+0000","last_change":"2023-09-13T04:37:58.590978+0000","last_active":"2023-09-13T04:38:24.244519+0000","last_peered":"2023-09-13T04:38:24.244519+0000","last_clean":"2023-09-13T04:38:24.244519+0000","last_became_active":"2023-09-13T04:37:58.590673+0000","last_became_peered":"2023-09-13T04:37:58.590673+0000","last_unstale":"2023-09-13T04:38:24.244519+0000","last_undegraded":"2023-09-13T04:38:24.244519+0000","last_fullsized":"2023-09-13T04:38:24.244519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:10:32.570029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592737+0000","last_change":"2023-09-13T04:37:59.251283+0000","last_active":"2023-09-13T04:38:23.592737+0000","last_peered":"2023-09-13T04:38:23.592737+0000","last_clean":"2023-09-13T04:38:23.592737+0000","last_became_active":"2023-09-13T04:37:59.250119+0000","last_became_peered":"2023-09-13T04:37:59.250119+0000","last_unstale":"2023-09-13T04:38:23.592737+0000","last_undegraded":"2023-09-13T04:38:23.592737+0000","last_fullsized":"2023-09-13T04:38:23.592737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:48:54.389970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.596107+0000","last_change":"2023-09-13T04:38:24.596107+0000","last_active":"2023-09-13T04:38:24.596107+0000","last_peered":"2023-09-13T04:38:24.596107+0000","last_clean":"2023-09-13T04:38:24.596107+0000","last_became_active":"2023-09-13T04:38:24.595653+0000","last_became_peered":"2023-09-13T04:38:24.595653+0000","last_unstale":"2023-09-13T04:38:24.596107+0000","last_undegraded":"2023-09-13T04:38:24.596107+0000","last_fullsized":"2023-09-13T04:38:24.596107+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:02:53.708658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.589390+0000","last_change":"2023-09-13T04:37:58.584909+0000","last_active":"2023-09-13T04:38:24.589390+0000","last_peered":"2023-09-13T04:38:24.589390+0000","last_clean":"2023-09-13T04:38:24.589390+0000","last_became_active":"2023-09-13T04:37:58.584630+0000","last_became_peered":"2023-09-13T04:37:58.584630+0000","last_unstale":"2023-09-13T04:38:24.589390+0000","last_undegraded":"2023-09-13T04:38:24.589390+0000","last_fullsized":"2023-09-13T04:38:24.589390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:57:56.567499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594349+0000","last_change":"2023-09-13T04:37:59.254448+0000","last_active":"2023-09-13T04:38:23.594349+0000","last_peered":"2023-09-13T04:38:23.594349+0000","last_clean":"2023-09-13T04:38:23.594349+0000","last_became_active":"2023-09-13T04:37:59.254138+0000","last_became_peered":"2023-09-13T04:37:59.254138+0000","last_unstale":"2023-09-13T04:38:23.594349+0000","last_undegraded":"2023-09-13T04:38:23.594349+0000","last_fullsized":"2023-09-13T04:38:23.594349+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:36:14.939881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588357+0000","last_change":"2023-09-13T04:38:03.613746+0000","last_active":"2023-09-13T04:38:23.588357+0000","last_peered":"2023-09-13T04:38:23.588357+0000","last_clean":"2023-09-13T04:38:23.588357+0000","last_became_active":"2023-09-13T04:38:03.613354+0000","last_became_peered":"2023-09-13T04:38:03.613354+0000","last_unstale":"2023-09-13T04:38:23.588357+0000","last_undegraded":"2023-09-13T04:38:23.588357+0000","last_fullsized":"2023-09-13T04:38:23.588357+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:41:01.294912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":130,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:39:14.688141+0000","last_change":"2023-09-13T04:37:58.591196+0000","last_active":"2023-09-13T04:39:14.688141+0000","last_peered":"2023-09-13T04:39:14.688141+0000","last_clean":"2023-09-13T04:39:14.688141+0000","last_became_active":"2023-09-13T04:37:58.590883+0000","last_became_peered":"2023-09-13T04:37:58.590883+0000","last_unstale":"2023-09-13T04:39:14.688141+0000","last_undegraded":"2023-09-13T04:39:14.688141+0000","last_fullsized":"2023-09-13T04:39:14.688141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:46:58.530270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":100,"num_read_kb":54,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595499+0000","last_change":"2023-09-13T04:37:59.243483+0000","last_active":"2023-09-13T04:38:23.595499+0000","last_peered":"2023-09-13T04:38:23.595499+0000","last_clean":"2023-09-13T04:38:23.595499+0000","last_became_active":"2023-09-13T04:37:59.243088+0000","last_became_peered":"2023-09-13T04:37:59.243088+0000","last_unstale":"2023-09-13T04:38:23.595499+0000","last_undegraded":"2023-09-13T04:38:23.595499+0000","last_fullsized":"2023-09-13T04:38:23.595499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:56:14.093819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591697+0000","last_change":"2023-09-13T04:38:03.608820+0000","last_active":"2023-09-13T04:38:23.591697+0000","last_peered":"2023-09-13T04:38:23.591697+0000","last_clean":"2023-09-13T04:38:23.591697+0000","last_became_active":"2023-09-13T04:38:03.608200+0000","last_became_peered":"2023-09-13T04:38:03.608200+0000","last_unstale":"2023-09-13T04:38:23.591697+0000","last_undegraded":"2023-09-13T04:38:23.591697+0000","last_fullsized":"2023-09-13T04:38:23.591697+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:54:52.416501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.620980+0000","last_change":"2023-09-13T04:38:24.606820+0000","last_active":"2023-09-13T04:38:25.620980+0000","last_peered":"2023-09-13T04:38:25.620980+0000","last_clean":"2023-09-13T04:38:25.620980+0000","last_became_active":"2023-09-13T04:38:24.599059+0000","last_became_peered":"2023-09-13T04:38:24.599059+0000","last_unstale":"2023-09-13T04:38:25.620980+0000","last_undegraded":"2023-09-13T04:38:25.620980+0000","last_fullsized":"2023-09-13T04:38:25.620980+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:08:12.634357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597568+0000","last_change":"2023-09-13T04:37:59.250666+0000","last_active":"2023-09-13T04:38:23.597568+0000","last_peered":"2023-09-13T04:38:23.597568+0000","last_clean":"2023-09-13T04:38:23.597568+0000","last_became_active":"2023-09-13T04:37:59.250240+0000","last_became_peered":"2023-09-13T04:37:59.250240+0000","last_unstale":"2023-09-13T04:38:23.597568+0000","last_undegraded":"2023-09-13T04:38:23.597568+0000","last_fullsized":"2023-09-13T04:38:23.597568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:44:31.324222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592271+0000","last_change":"2023-09-13T04:38:03.616296+0000","last_active":"2023-09-13T04:38:23.592271+0000","last_peered":"2023-09-13T04:38:23.592271+0000","last_clean":"2023-09-13T04:38:23.592271+0000","last_became_active":"2023-09-13T04:38:03.615934+0000","last_became_peered":"2023-09-13T04:38:03.615934+0000","last_unstale":"2023-09-13T04:38:23.592271+0000","last_undegraded":"2023-09-13T04:38:23.592271+0000","last_fullsized":"2023-09-13T04:38:23.592271+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:51:18.994472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244805+0000","last_change":"2023-09-13T04:37:58.594782+0000","last_active":"2023-09-13T04:38:24.244805+0000","last_peered":"2023-09-13T04:38:24.244805+0000","last_clean":"2023-09-13T04:38:24.244805+0000","last_became_active":"2023-09-13T04:37:58.594480+0000","last_became_peered":"2023-09-13T04:37:58.594480+0000","last_unstale":"2023-09-13T04:38:24.244805+0000","last_undegraded":"2023-09-13T04:38:24.244805+0000","last_fullsized":"2023-09-13T04:38:24.244805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:40:19.635672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588001+0000","last_change":"2023-09-13T04:37:59.252575+0000","last_active":"2023-09-13T04:38:23.588001+0000","last_peered":"2023-09-13T04:38:23.588001+0000","last_clean":"2023-09-13T04:38:23.588001+0000","last_became_active":"2023-09-13T04:37:59.252268+0000","last_became_peered":"2023-09-13T04:37:59.252268+0000","last_unstale":"2023-09-13T04:38:23.588001+0000","last_undegraded":"2023-09-13T04:38:23.588001+0000","last_fullsized":"2023-09-13T04:38:23.588001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:27:10.814930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587908+0000","last_change":"2023-09-13T04:38:03.618835+0000","last_active":"2023-09-13T04:38:23.587908+0000","last_peered":"2023-09-13T04:38:23.587908+0000","last_clean":"2023-09-13T04:38:23.587908+0000","last_became_active":"2023-09-13T04:38:03.618226+0000","last_became_peered":"2023-09-13T04:38:03.618226+0000","last_unstale":"2023-09-13T04:38:23.587908+0000","last_undegraded":"2023-09-13T04:38:23.587908+0000","last_fullsized":"2023-09-13T04:38:23.587908+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:01:28.157490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":72,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:39:13.688826+0000","last_change":"2023-09-13T04:37:58.588545+0000","last_active":"2023-09-13T04:39:13.688826+0000","last_peered":"2023-09-13T04:39:13.688826+0000","last_clean":"2023-09-13T04:39:13.688826+0000","last_became_active":"2023-09-13T04:37:58.587995+0000","last_became_peered":"2023-09-13T04:37:58.587995+0000","last_unstale":"2023-09-13T04:39:13.688826+0000","last_undegraded":"2023-09-13T04:39:13.688826+0000","last_fullsized":"2023-09-13T04:39:13.688826+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:40:08.797724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587793+0000","last_change":"2023-09-13T04:37:59.249100+0000","last_active":"2023-09-13T04:38:23.587793+0000","last_peered":"2023-09-13T04:38:23.587793+0000","last_clean":"2023-09-13T04:38:23.587793+0000","last_became_active":"2023-09-13T04:37:59.248771+0000","last_became_peered":"2023-09-13T04:37:59.248771+0000","last_unstale":"2023-09-13T04:38:23.587793+0000","last_undegraded":"2023-09-13T04:38:23.587793+0000","last_fullsized":"2023-09-13T04:38:23.587793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:27:37.964595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597364+0000","last_change":"2023-09-13T04:38:03.616988+0000","last_active":"2023-09-13T04:38:23.597364+0000","last_peered":"2023-09-13T04:38:23.597364+0000","last_clean":"2023-09-13T04:38:23.597364+0000","last_became_active":"2023-09-13T04:38:03.616524+0000","last_became_peered":"2023-09-13T04:38:03.616524+0000","last_unstale":"2023-09-13T04:38:23.597364+0000","last_undegraded":"2023-09-13T04:38:23.597364+0000","last_fullsized":"2023-09-13T04:38:23.597364+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:42:41.431115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.622279+0000","last_change":"2023-09-13T04:38:24.613384+0000","last_active":"2023-09-13T04:38:25.622279+0000","last_peered":"2023-09-13T04:38:25.622279+0000","last_clean":"2023-09-13T04:38:25.622279+0000","last_became_active":"2023-09-13T04:38:24.599349+0000","last_became_peered":"2023-09-13T04:38:24.599349+0000","last_unstale":"2023-09-13T04:38:25.622279+0000","last_undegraded":"2023-09-13T04:38:25.622279+0000","last_fullsized":"2023-09-13T04:38:25.622279+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:23:23.479715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593329+0000","last_change":"2023-09-13T04:37:59.252237+0000","last_active":"2023-09-13T04:38:23.593329+0000","last_peered":"2023-09-13T04:38:23.593329+0000","last_clean":"2023-09-13T04:38:23.593329+0000","last_became_active":"2023-09-13T04:37:59.251718+0000","last_became_peered":"2023-09-13T04:37:59.251718+0000","last_unstale":"2023-09-13T04:38:23.593329+0000","last_undegraded":"2023-09-13T04:38:23.593329+0000","last_fullsized":"2023-09-13T04:38:23.593329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:45:48.405226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593276+0000","last_change":"2023-09-13T04:38:03.607096+0000","last_active":"2023-09-13T04:38:23.593276+0000","last_peered":"2023-09-13T04:38:23.593276+0000","last_clean":"2023-09-13T04:38:23.593276+0000","last_became_active":"2023-09-13T04:38:03.606656+0000","last_became_peered":"2023-09-13T04:38:03.606656+0000","last_unstale":"2023-09-13T04:38:23.593276+0000","last_undegraded":"2023-09-13T04:38:23.593276+0000","last_fullsized":"2023-09-13T04:38:23.593276+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:59:26.140360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:26.617580+0000","last_change":"2023-09-13T04:37:58.591047+0000","last_active":"2023-09-13T04:38:26.617580+0000","last_peered":"2023-09-13T04:38:26.617580+0000","last_clean":"2023-09-13T04:38:26.617580+0000","last_became_active":"2023-09-13T04:37:58.590740+0000","last_became_peered":"2023-09-13T04:37:58.590740+0000","last_unstale":"2023-09-13T04:38:26.617580+0000","last_undegraded":"2023-09-13T04:38:26.617580+0000","last_fullsized":"2023-09-13T04:38:26.617580+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:14:42.280416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592406+0000","last_change":"2023-09-13T04:37:59.249362+0000","last_active":"2023-09-13T04:38:23.592406+0000","last_peered":"2023-09-13T04:38:23.592406+0000","last_clean":"2023-09-13T04:38:23.592406+0000","last_became_active":"2023-09-13T04:37:59.249041+0000","last_became_peered":"2023-09-13T04:37:59.249041+0000","last_unstale":"2023-09-13T04:38:23.592406+0000","last_undegraded":"2023-09-13T04:38:23.592406+0000","last_fullsized":"2023-09-13T04:38:23.592406+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:06:33.822157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.598370+0000","last_change":"2023-09-13T04:38:24.598370+0000","last_active":"2023-09-13T04:38:24.598370+0000","last_peered":"2023-09-13T04:38:24.598370+0000","last_clean":"2023-09-13T04:38:24.598370+0000","last_became_active":"2023-09-13T04:38:24.597931+0000","last_became_peered":"2023-09-13T04:38:24.597931+0000","last_unstale":"2023-09-13T04:38:24.598370+0000","last_undegraded":"2023-09-13T04:38:24.598370+0000","last_fullsized":"2023-09-13T04:38:24.598370+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:14:28.911329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588100+0000","last_change":"2023-09-13T04:37:58.592496+0000","last_active":"2023-09-13T04:38:23.588100+0000","last_peered":"2023-09-13T04:38:23.588100+0000","last_clean":"2023-09-13T04:38:23.588100+0000","last_became_active":"2023-09-13T04:37:58.592106+0000","last_became_peered":"2023-09-13T04:37:58.592106+0000","last_unstale":"2023-09-13T04:38:23.588100+0000","last_undegraded":"2023-09-13T04:38:23.588100+0000","last_fullsized":"2023-09-13T04:38:23.588100+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:28:55.618563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594212+0000","last_change":"2023-09-13T04:37:59.250019+0000","last_active":"2023-09-13T04:38:23.594212+0000","last_peered":"2023-09-13T04:38:23.594212+0000","last_clean":"2023-09-13T04:38:23.594212+0000","last_became_active":"2023-09-13T04:37:59.249278+0000","last_became_peered":"2023-09-13T04:37:59.249278+0000","last_unstale":"2023-09-13T04:38:23.594212+0000","last_undegraded":"2023-09-13T04:38:23.594212+0000","last_fullsized":"2023-09-13T04:38:23.594212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:15:31.217236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597640+0000","last_change":"2023-09-13T04:38:03.614542+0000","last_active":"2023-09-13T04:38:23.597640+0000","last_peered":"2023-09-13T04:38:23.597640+0000","last_clean":"2023-09-13T04:38:23.597640+0000","last_became_active":"2023-09-13T04:38:03.614103+0000","last_became_peered":"2023-09-13T04:38:03.614103+0000","last_unstale":"2023-09-13T04:38:23.597640+0000","last_undegraded":"2023-09-13T04:38:23.597640+0000","last_fullsized":"2023-09-13T04:38:23.597640+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:06:48.426843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.592720+0000","last_change":"2023-09-13T04:37:58.591864+0000","last_active":"2023-09-13T04:38:24.592720+0000","last_peered":"2023-09-13T04:38:24.592720+0000","last_clean":"2023-09-13T04:38:24.592720+0000","last_became_active":"2023-09-13T04:37:58.591562+0000","last_became_peered":"2023-09-13T04:37:58.591562+0000","last_unstale":"2023-09-13T04:38:24.592720+0000","last_undegraded":"2023-09-13T04:38:24.592720+0000","last_fullsized":"2023-09-13T04:38:24.592720+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:31:27.280489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594567+0000","last_change":"2023-09-13T04:37:59.252600+0000","last_active":"2023-09-13T04:38:23.594567+0000","last_peered":"2023-09-13T04:38:23.594567+0000","last_clean":"2023-09-13T04:38:23.594567+0000","last_became_active":"2023-09-13T04:37:59.252306+0000","last_became_peered":"2023-09-13T04:37:59.252306+0000","last_unstale":"2023-09-13T04:38:23.594567+0000","last_undegraded":"2023-09-13T04:38:23.594567+0000","last_fullsized":"2023-09-13T04:38:23.594567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:18:55.686847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.596920+0000","last_change":"2023-09-13T04:38:03.742502+0000","last_active":"2023-09-13T04:38:23.596920+0000","last_peered":"2023-09-13T04:38:23.596920+0000","last_clean":"2023-09-13T04:38:23.596920+0000","last_became_active":"2023-09-13T04:38:03.742054+0000","last_became_peered":"2023-09-13T04:38:03.742054+0000","last_unstale":"2023-09-13T04:38:23.596920+0000","last_undegraded":"2023-09-13T04:38:23.596920+0000","last_fullsized":"2023-09-13T04:38:23.596920+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:55:54.601145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593527+0000","last_change":"2023-09-13T04:37:59.245644+0000","last_active":"2023-09-13T04:38:23.593527+0000","last_peered":"2023-09-13T04:38:23.593527+0000","last_clean":"2023-09-13T04:38:23.593527+0000","last_became_active":"2023-09-13T04:37:59.245284+0000","last_became_peered":"2023-09-13T04:37:59.245284+0000","last_unstale":"2023-09-13T04:38:23.593527+0000","last_undegraded":"2023-09-13T04:38:23.593527+0000","last_fullsized":"2023-09-13T04:38:23.593527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:19:04.002414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":73,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:39:14.689539+0000","last_change":"2023-09-13T04:37:58.590249+0000","last_active":"2023-09-13T04:39:14.689539+0000","last_peered":"2023-09-13T04:39:14.689539+0000","last_clean":"2023-09-13T04:39:14.689539+0000","last_became_active":"2023-09-13T04:37:58.589521+0000","last_became_peered":"2023-09-13T04:37:58.589521+0000","last_unstale":"2023-09-13T04:39:14.689539+0000","last_undegraded":"2023-09-13T04:39:14.689539+0000","last_fullsized":"2023-09-13T04:39:14.689539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:11:57.542149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.598613+0000","last_change":"2023-09-13T04:38:24.598613+0000","last_active":"2023-09-13T04:38:24.598613+0000","last_peered":"2023-09-13T04:38:24.598613+0000","last_clean":"2023-09-13T04:38:24.598613+0000","last_became_active":"2023-09-13T04:38:24.598150+0000","last_became_peered":"2023-09-13T04:38:24.598150+0000","last_unstale":"2023-09-13T04:38:24.598613+0000","last_undegraded":"2023-09-13T04:38:24.598613+0000","last_fullsized":"2023-09-13T04:38:24.598613+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:02:29.894446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595221+0000","last_change":"2023-09-13T04:37:59.243794+0000","last_active":"2023-09-13T04:38:23.595221+0000","last_peered":"2023-09-13T04:38:23.595221+0000","last_clean":"2023-09-13T04:38:23.595221+0000","last_became_active":"2023-09-13T04:37:59.243368+0000","last_became_peered":"2023-09-13T04:37:59.243368+0000","last_unstale":"2023-09-13T04:38:23.595221+0000","last_undegraded":"2023-09-13T04:38:23.595221+0000","last_fullsized":"2023-09-13T04:38:23.595221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:06:45.318737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:26.615677+0000","last_change":"2023-09-13T04:37:58.589195+0000","last_active":"2023-09-13T04:38:26.615677+0000","last_peered":"2023-09-13T04:38:26.615677+0000","last_clean":"2023-09-13T04:38:26.615677+0000","last_became_active":"2023-09-13T04:37:58.588905+0000","last_became_peered":"2023-09-13T04:37:58.588905+0000","last_unstale":"2023-09-13T04:38:26.615677+0000","last_undegraded":"2023-09-13T04:38:26.615677+0000","last_fullsized":"2023-09-13T04:38:26.615677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:13:15.978891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":326,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587638+0000","last_change":"2023-09-13T04:36:52.227264+0000","last_active":"2023-09-13T04:38:23.587638+0000","last_peered":"2023-09-13T04:38:23.587638+0000","last_clean":"2023-09-13T04:38:23.587638+0000","last_became_active":"2023-09-13T04:36:52.226773+0000","last_became_peered":"2023-09-13T04:36:52.226773+0000","last_unstale":"2023-09-13T04:38:23.587638+0000","last_undegraded":"2023-09-13T04:38:23.587638+0000","last_fullsized":"2023-09-13T04:38:23.587638+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:33:42.026499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:33:42.026499+0000","last_clean_scrub_stamp":"2023-09-13T04:33:42.026499+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:28:55.806587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592890+0000","last_change":"2023-09-13T04:38:03.742790+0000","last_active":"2023-09-13T04:38:23.592890+0000","last_peered":"2023-09-13T04:38:23.592890+0000","last_clean":"2023-09-13T04:38:23.592890+0000","last_became_active":"2023-09-13T04:38:03.742338+0000","last_became_peered":"2023-09-13T04:38:03.742338+0000","last_unstale":"2023-09-13T04:38:23.592890+0000","last_undegraded":"2023-09-13T04:38:23.592890+0000","last_fullsized":"2023-09-13T04:38:23.592890+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:01:38.910623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597412+0000","last_change":"2023-09-13T04:37:59.252669+0000","last_active":"2023-09-13T04:38:23.597412+0000","last_peered":"2023-09-13T04:38:23.597412+0000","last_clean":"2023-09-13T04:38:23.597412+0000","last_became_active":"2023-09-13T04:37:59.252223+0000","last_became_peered":"2023-09-13T04:37:59.252223+0000","last_unstale":"2023-09-13T04:38:23.597412+0000","last_undegraded":"2023-09-13T04:38:23.597412+0000","last_fullsized":"2023-09-13T04:38:23.597412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:57:21.090083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.615697+0000","last_change":"2023-09-13T04:37:58.590540+0000","last_active":"2023-09-13T04:38:25.615697+0000","last_peered":"2023-09-13T04:38:25.615697+0000","last_clean":"2023-09-13T04:38:25.615697+0000","last_became_active":"2023-09-13T04:37:58.590061+0000","last_became_peered":"2023-09-13T04:37:58.590061+0000","last_unstale":"2023-09-13T04:38:25.615697+0000","last_undegraded":"2023-09-13T04:38:25.615697+0000","last_fullsized":"2023-09-13T04:38:25.615697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:57:17.133461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592719+0000","last_change":"2023-09-13T04:38:03.614615+0000","last_active":"2023-09-13T04:38:23.592719+0000","last_peered":"2023-09-13T04:38:23.592719+0000","last_clean":"2023-09-13T04:38:23.592719+0000","last_became_active":"2023-09-13T04:38:03.614086+0000","last_became_peered":"2023-09-13T04:38:03.614086+0000","last_unstale":"2023-09-13T04:38:23.592719+0000","last_undegraded":"2023-09-13T04:38:23.592719+0000","last_fullsized":"2023-09-13T04:38:23.592719+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:25:35.525513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.595581+0000","last_change":"2023-09-13T04:37:58.592096+0000","last_active":"2023-09-13T04:38:24.595581+0000","last_peered":"2023-09-13T04:38:24.595581+0000","last_clean":"2023-09-13T04:38:24.595581+0000","last_became_active":"2023-09-13T04:37:58.591872+0000","last_became_peered":"2023-09-13T04:37:58.591872+0000","last_unstale":"2023-09-13T04:38:24.595581+0000","last_undegraded":"2023-09-13T04:38:24.595581+0000","last_fullsized":"2023-09-13T04:38:24.595581+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:05:23.936105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597816+0000","last_change":"2023-09-13T04:37:59.240759+0000","last_active":"2023-09-13T04:38:23.597816+0000","last_peered":"2023-09-13T04:38:23.597816+0000","last_clean":"2023-09-13T04:38:23.597816+0000","last_became_active":"2023-09-13T04:37:59.240414+0000","last_became_peered":"2023-09-13T04:37:59.240414+0000","last_unstale":"2023-09-13T04:38:23.597816+0000","last_undegraded":"2023-09-13T04:38:23.597816+0000","last_fullsized":"2023-09-13T04:38:23.597816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:37:20.301938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597533+0000","last_change":"2023-09-13T04:38:03.742519+0000","last_active":"2023-09-13T04:38:23.597533+0000","last_peered":"2023-09-13T04:38:23.597533+0000","last_clean":"2023-09-13T04:38:23.597533+0000","last_became_active":"2023-09-13T04:38:03.742075+0000","last_became_peered":"2023-09-13T04:38:03.742075+0000","last_unstale":"2023-09-13T04:38:23.597533+0000","last_undegraded":"2023-09-13T04:38:23.597533+0000","last_fullsized":"2023-09-13T04:38:23.597533+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:14:33.155837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.619899+0000","last_change":"2023-09-13T04:37:58.593891+0000","last_active":"2023-09-13T04:38:25.619899+0000","last_peered":"2023-09-13T04:38:25.619899+0000","last_clean":"2023-09-13T04:38:25.619899+0000","last_became_active":"2023-09-13T04:37:58.593497+0000","last_became_peered":"2023-09-13T04:37:58.593497+0000","last_unstale":"2023-09-13T04:38:25.619899+0000","last_undegraded":"2023-09-13T04:38:25.619899+0000","last_fullsized":"2023-09-13T04:38:25.619899+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:25:53.652601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595975+0000","last_change":"2023-09-13T04:37:59.247151+0000","last_active":"2023-09-13T04:38:23.595975+0000","last_peered":"2023-09-13T04:38:23.595975+0000","last_clean":"2023-09-13T04:38:23.595975+0000","last_became_active":"2023-09-13T04:37:59.246853+0000","last_became_peered":"2023-09-13T04:37:59.246853+0000","last_unstale":"2023-09-13T04:38:23.595975+0000","last_undegraded":"2023-09-13T04:38:23.595975+0000","last_fullsized":"2023-09-13T04:38:23.595975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:15:22.659365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587493+0000","last_change":"2023-09-13T04:38:03.606594+0000","last_active":"2023-09-13T04:38:23.587493+0000","last_peered":"2023-09-13T04:38:23.587493+0000","last_clean":"2023-09-13T04:38:23.587493+0000","last_became_active":"2023-09-13T04:38:03.606058+0000","last_became_peered":"2023-09-13T04:38:03.606058+0000","last_unstale":"2023-09-13T04:38:23.587493+0000","last_undegraded":"2023-09-13T04:38:23.587493+0000","last_fullsized":"2023-09-13T04:38:23.587493+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:57:39.237520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.246057+0000","last_change":"2023-09-13T04:37:58.588362+0000","last_active":"2023-09-13T04:38:24.246057+0000","last_peered":"2023-09-13T04:38:24.246057+0000","last_clean":"2023-09-13T04:38:24.246057+0000","last_became_active":"2023-09-13T04:37:58.588065+0000","last_became_peered":"2023-09-13T04:37:58.588065+0000","last_unstale":"2023-09-13T04:38:24.246057+0000","last_undegraded":"2023-09-13T04:38:24.246057+0000","last_fullsized":"2023-09-13T04:38:24.246057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:32:04.255654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592468+0000","last_change":"2023-09-13T04:37:59.251883+0000","last_active":"2023-09-13T04:38:23.592468+0000","last_peered":"2023-09-13T04:38:23.592468+0000","last_clean":"2023-09-13T04:38:23.592468+0000","last_became_active":"2023-09-13T04:37:59.251522+0000","last_became_peered":"2023-09-13T04:37:59.251522+0000","last_unstale":"2023-09-13T04:38:23.592468+0000","last_undegraded":"2023-09-13T04:38:23.592468+0000","last_fullsized":"2023-09-13T04:38:23.592468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:05:06.420799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590729+0000","last_change":"2023-09-13T04:38:03.742726+0000","last_active":"2023-09-13T04:38:23.590729+0000","last_peered":"2023-09-13T04:38:23.590729+0000","last_clean":"2023-09-13T04:38:23.590729+0000","last_became_active":"2023-09-13T04:38:03.742293+0000","last_became_peered":"2023-09-13T04:38:03.742293+0000","last_unstale":"2023-09-13T04:38:23.590729+0000","last_undegraded":"2023-09-13T04:38:23.590729+0000","last_fullsized":"2023-09-13T04:38:23.590729+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:52:15.089764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.243314+0000","last_change":"2023-09-13T04:37:58.584923+0000","last_active":"2023-09-13T04:38:24.243314+0000","last_peered":"2023-09-13T04:38:24.243314+0000","last_clean":"2023-09-13T04:38:24.243314+0000","last_became_active":"2023-09-13T04:37:58.584320+0000","last_became_peered":"2023-09-13T04:37:58.584320+0000","last_unstale":"2023-09-13T04:38:24.243314+0000","last_undegraded":"2023-09-13T04:38:24.243314+0000","last_fullsized":"2023-09-13T04:38:24.243314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:16:53.670534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592586+0000","last_change":"2023-09-13T04:37:59.243029+0000","last_active":"2023-09-13T04:38:23.592586+0000","last_peered":"2023-09-13T04:38:23.592586+0000","last_clean":"2023-09-13T04:38:23.592586+0000","last_became_active":"2023-09-13T04:37:59.242733+0000","last_became_peered":"2023-09-13T04:37:59.242733+0000","last_unstale":"2023-09-13T04:38:23.592586+0000","last_undegraded":"2023-09-13T04:38:23.592586+0000","last_fullsized":"2023-09-13T04:38:23.592586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:48:21.430026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588037+0000","last_change":"2023-09-13T04:38:03.616735+0000","last_active":"2023-09-13T04:38:23.588037+0000","last_peered":"2023-09-13T04:38:23.588037+0000","last_clean":"2023-09-13T04:38:23.588037+0000","last_became_active":"2023-09-13T04:38:03.616275+0000","last_became_peered":"2023-09-13T04:38:03.616275+0000","last_unstale":"2023-09-13T04:38:23.588037+0000","last_undegraded":"2023-09-13T04:38:23.588037+0000","last_fullsized":"2023-09-13T04:38:23.588037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:12:27.267918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244170+0000","last_change":"2023-09-13T04:37:58.595040+0000","last_active":"2023-09-13T04:38:24.244170+0000","last_peered":"2023-09-13T04:38:24.244170+0000","last_clean":"2023-09-13T04:38:24.244170+0000","last_became_active":"2023-09-13T04:37:58.591403+0000","last_became_peered":"2023-09-13T04:37:58.591403+0000","last_unstale":"2023-09-13T04:38:24.244170+0000","last_undegraded":"2023-09-13T04:38:24.244170+0000","last_fullsized":"2023-09-13T04:38:24.244170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:42:06.561066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592175+0000","last_change":"2023-09-13T04:37:59.248837+0000","last_active":"2023-09-13T04:38:23.592175+0000","last_peered":"2023-09-13T04:38:23.592175+0000","last_clean":"2023-09-13T04:38:23.592175+0000","last_became_active":"2023-09-13T04:37:59.248573+0000","last_became_peered":"2023-09-13T04:37:59.248573+0000","last_unstale":"2023-09-13T04:38:23.592175+0000","last_undegraded":"2023-09-13T04:38:23.592175+0000","last_fullsized":"2023-09-13T04:38:23.592175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:56:22.980484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590405+0000","last_change":"2023-09-13T04:38:03.616370+0000","last_active":"2023-09-13T04:38:23.590405+0000","last_peered":"2023-09-13T04:38:23.590405+0000","last_clean":"2023-09-13T04:38:23.590405+0000","last_became_active":"2023-09-13T04:38:03.615996+0000","last_became_peered":"2023-09-13T04:38:03.615996+0000","last_unstale":"2023-09-13T04:38:23.590405+0000","last_undegraded":"2023-09-13T04:38:23.590405+0000","last_fullsized":"2023-09-13T04:38:23.590405+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:56:59.924205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.590955+0000","last_change":"2023-09-13T04:37:58.597902+0000","last_active":"2023-09-13T04:38:24.590955+0000","last_peered":"2023-09-13T04:38:24.590955+0000","last_clean":"2023-09-13T04:38:24.590955+0000","last_became_active":"2023-09-13T04:37:58.597578+0000","last_became_peered":"2023-09-13T04:37:58.597578+0000","last_unstale":"2023-09-13T04:38:24.590955+0000","last_undegraded":"2023-09-13T04:38:24.590955+0000","last_fullsized":"2023-09-13T04:38:24.590955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:10:09.931949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597024+0000","last_change":"2023-09-13T04:37:59.248924+0000","last_active":"2023-09-13T04:38:23.597024+0000","last_peered":"2023-09-13T04:38:23.597024+0000","last_clean":"2023-09-13T04:38:23.597024+0000","last_became_active":"2023-09-13T04:37:59.248483+0000","last_became_peered":"2023-09-13T04:37:59.248483+0000","last_unstale":"2023-09-13T04:38:23.597024+0000","last_undegraded":"2023-09-13T04:38:23.597024+0000","last_fullsized":"2023-09-13T04:38:23.597024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:49:56.078544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590322+0000","last_change":"2023-09-13T04:38:03.742260+0000","last_active":"2023-09-13T04:38:23.590322+0000","last_peered":"2023-09-13T04:38:23.590322+0000","last_clean":"2023-09-13T04:38:23.590322+0000","last_became_active":"2023-09-13T04:38:03.741893+0000","last_became_peered":"2023-09-13T04:38:03.741893+0000","last_unstale":"2023-09-13T04:38:23.590322+0000","last_undegraded":"2023-09-13T04:38:23.590322+0000","last_fullsized":"2023-09-13T04:38:23.590322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:59:00.653545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.594861+0000","last_change":"2023-09-13T04:37:58.586898+0000","last_active":"2023-09-13T04:38:24.594861+0000","last_peered":"2023-09-13T04:38:24.594861+0000","last_clean":"2023-09-13T04:38:24.594861+0000","last_became_active":"2023-09-13T04:37:58.586594+0000","last_became_peered":"2023-09-13T04:37:58.586594+0000","last_unstale":"2023-09-13T04:38:24.594861+0000","last_undegraded":"2023-09-13T04:38:24.594861+0000","last_fullsized":"2023-09-13T04:38:24.594861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:29:43.718522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587969+0000","last_change":"2023-09-13T04:37:59.244749+0000","last_active":"2023-09-13T04:38:23.587969+0000","last_peered":"2023-09-13T04:38:23.587969+0000","last_clean":"2023-09-13T04:38:23.587969+0000","last_became_active":"2023-09-13T04:37:59.244391+0000","last_became_peered":"2023-09-13T04:37:59.244391+0000","last_unstale":"2023-09-13T04:38:23.587969+0000","last_undegraded":"2023-09-13T04:38:23.587969+0000","last_fullsized":"2023-09-13T04:38:23.587969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:33:02.195729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592524+0000","last_change":"2023-09-13T04:38:03.609766+0000","last_active":"2023-09-13T04:38:23.592524+0000","last_peered":"2023-09-13T04:38:23.592524+0000","last_clean":"2023-09-13T04:38:23.592524+0000","last_became_active":"2023-09-13T04:38:03.609432+0000","last_became_peered":"2023-09-13T04:38:03.609432+0000","last_unstale":"2023-09-13T04:38:23.592524+0000","last_undegraded":"2023-09-13T04:38:23.592524+0000","last_fullsized":"2023-09-13T04:38:23.592524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:30:56.115483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.620943+0000","last_change":"2023-09-13T04:37:58.589119+0000","last_active":"2023-09-13T04:38:25.620943+0000","last_peered":"2023-09-13T04:38:25.620943+0000","last_clean":"2023-09-13T04:38:25.620943+0000","last_became_active":"2023-09-13T04:37:58.588803+0000","last_became_peered":"2023-09-13T04:37:58.588803+0000","last_unstale":"2023-09-13T04:38:25.620943+0000","last_undegraded":"2023-09-13T04:38:25.620943+0000","last_fullsized":"2023-09-13T04:38:25.620943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:51:20.888666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588392+0000","last_change":"2023-09-13T04:37:59.243506+0000","last_active":"2023-09-13T04:38:23.588392+0000","last_peered":"2023-09-13T04:38:23.588392+0000","last_clean":"2023-09-13T04:38:23.588392+0000","last_became_active":"2023-09-13T04:37:59.243082+0000","last_became_peered":"2023-09-13T04:37:59.243082+0000","last_unstale":"2023-09-13T04:38:23.588392+0000","last_undegraded":"2023-09-13T04:38:23.588392+0000","last_fullsized":"2023-09-13T04:38:23.588392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:01:41.179286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592019+0000","last_change":"2023-09-13T04:38:03.609042+0000","last_active":"2023-09-13T04:38:23.592019+0000","last_peered":"2023-09-13T04:38:23.592019+0000","last_clean":"2023-09-13T04:38:23.592019+0000","last_became_active":"2023-09-13T04:38:03.608553+0000","last_became_peered":"2023-09-13T04:38:03.608553+0000","last_unstale":"2023-09-13T04:38:23.592019+0000","last_undegraded":"2023-09-13T04:38:23.592019+0000","last_fullsized":"2023-09-13T04:38:23.592019+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:05:56.648475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.589585+0000","last_change":"2023-09-13T04:37:58.598056+0000","last_active":"2023-09-13T04:38:24.589585+0000","last_peered":"2023-09-13T04:38:24.589585+0000","last_clean":"2023-09-13T04:38:24.589585+0000","last_became_active":"2023-09-13T04:37:58.597724+0000","last_became_peered":"2023-09-13T04:37:58.597724+0000","last_unstale":"2023-09-13T04:38:24.589585+0000","last_undegraded":"2023-09-13T04:38:24.589585+0000","last_fullsized":"2023-09-13T04:38:24.589585+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:43:08.296866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588462+0000","last_change":"2023-09-13T04:37:59.246505+0000","last_active":"2023-09-13T04:38:23.588462+0000","last_peered":"2023-09-13T04:38:23.588462+0000","last_clean":"2023-09-13T04:38:23.588462+0000","last_became_active":"2023-09-13T04:37:59.246024+0000","last_became_peered":"2023-09-13T04:37:59.246024+0000","last_unstale":"2023-09-13T04:38:23.588462+0000","last_undegraded":"2023-09-13T04:38:23.588462+0000","last_fullsized":"2023-09-13T04:38:23.588462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:36:58.008066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590871+0000","last_change":"2023-09-13T04:38:03.605354+0000","last_active":"2023-09-13T04:38:23.590871+0000","last_peered":"2023-09-13T04:38:23.590871+0000","last_clean":"2023-09-13T04:38:23.590871+0000","last_became_active":"2023-09-13T04:38:03.604840+0000","last_became_peered":"2023-09-13T04:38:03.604840+0000","last_unstale":"2023-09-13T04:38:23.590871+0000","last_undegraded":"2023-09-13T04:38:23.590871+0000","last_fullsized":"2023-09-13T04:38:23.590871+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:46:29.867914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597380+0000","last_change":"2023-09-13T04:37:58.589052+0000","last_active":"2023-09-13T04:38:23.597380+0000","last_peered":"2023-09-13T04:38:23.597380+0000","last_clean":"2023-09-13T04:38:23.597380+0000","last_became_active":"2023-09-13T04:37:58.588707+0000","last_became_peered":"2023-09-13T04:37:58.588707+0000","last_unstale":"2023-09-13T04:38:23.597380+0000","last_undegraded":"2023-09-13T04:38:23.597380+0000","last_fullsized":"2023-09-13T04:38:23.597380+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:25:23.966046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588356+0000","last_change":"2023-09-13T04:37:59.241157+0000","last_active":"2023-09-13T04:38:23.588356+0000","last_peered":"2023-09-13T04:38:23.588356+0000","last_clean":"2023-09-13T04:38:23.588356+0000","last_became_active":"2023-09-13T04:37:59.240857+0000","last_became_peered":"2023-09-13T04:37:59.240857+0000","last_unstale":"2023-09-13T04:38:23.588356+0000","last_undegraded":"2023-09-13T04:38:23.588356+0000","last_fullsized":"2023-09-13T04:38:23.588356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:04:44.986908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592376+0000","last_change":"2023-09-13T04:37:59.242447+0000","last_active":"2023-09-13T04:38:23.592376+0000","last_peered":"2023-09-13T04:38:23.592376+0000","last_clean":"2023-09-13T04:38:23.592376+0000","last_became_active":"2023-09-13T04:37:59.242130+0000","last_became_peered":"2023-09-13T04:37:59.242130+0000","last_unstale":"2023-09-13T04:38:23.592376+0000","last_undegraded":"2023-09-13T04:38:23.592376+0000","last_fullsized":"2023-09-13T04:38:23.592376+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:12:12.844888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.595646+0000","last_change":"2023-09-13T04:38:24.595646+0000","last_active":"2023-09-13T04:38:24.595646+0000","last_peered":"2023-09-13T04:38:24.595646+0000","last_clean":"2023-09-13T04:38:24.595646+0000","last_became_active":"2023-09-13T04:38:24.595295+0000","last_became_peered":"2023-09-13T04:38:24.595295+0000","last_unstale":"2023-09-13T04:38:24.595646+0000","last_undegraded":"2023-09-13T04:38:24.595646+0000","last_fullsized":"2023-09-13T04:38:24.595646+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:24:27.174979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587574+0000","last_change":"2023-09-13T04:37:59.245358+0000","last_active":"2023-09-13T04:38:23.587574+0000","last_peered":"2023-09-13T04:38:23.587574+0000","last_clean":"2023-09-13T04:38:23.587574+0000","last_became_active":"2023-09-13T04:37:59.245052+0000","last_became_peered":"2023-09-13T04:37:59.245052+0000","last_unstale":"2023-09-13T04:38:23.587574+0000","last_undegraded":"2023-09-13T04:38:23.587574+0000","last_fullsized":"2023-09-13T04:38:23.587574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:05:42.561716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592358+0000","last_change":"2023-09-13T04:37:59.254053+0000","last_active":"2023-09-13T04:38:23.592358+0000","last_peered":"2023-09-13T04:38:23.592358+0000","last_clean":"2023-09-13T04:38:23.592358+0000","last_became_active":"2023-09-13T04:37:59.253544+0000","last_became_peered":"2023-09-13T04:37:59.253544+0000","last_unstale":"2023-09-13T04:38:23.592358+0000","last_undegraded":"2023-09-13T04:38:23.592358+0000","last_fullsized":"2023-09-13T04:38:23.592358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:18:54.689208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597920+0000","last_change":"2023-09-13T04:37:59.253403+0000","last_active":"2023-09-13T04:38:23.597920+0000","last_peered":"2023-09-13T04:38:23.597920+0000","last_clean":"2023-09-13T04:38:23.597920+0000","last_became_active":"2023-09-13T04:37:59.253062+0000","last_became_peered":"2023-09-13T04:37:59.253062+0000","last_unstale":"2023-09-13T04:38:23.597920+0000","last_undegraded":"2023-09-13T04:38:23.597920+0000","last_fullsized":"2023-09-13T04:38:23.597920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:31:09.898886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592570+0000","last_change":"2023-09-13T04:37:59.240916+0000","last_active":"2023-09-13T04:38:23.592570+0000","last_peered":"2023-09-13T04:38:23.592570+0000","last_clean":"2023-09-13T04:38:23.592570+0000","last_became_active":"2023-09-13T04:37:59.240605+0000","last_became_peered":"2023-09-13T04:37:59.240605+0000","last_unstale":"2023-09-13T04:38:23.592570+0000","last_undegraded":"2023-09-13T04:38:23.592570+0000","last_fullsized":"2023-09-13T04:38:23.592570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:15:48.241587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588778+0000","last_change":"2023-09-13T04:37:59.238569+0000","last_active":"2023-09-13T04:38:23.588778+0000","last_peered":"2023-09-13T04:38:23.588778+0000","last_clean":"2023-09-13T04:38:23.588778+0000","last_became_active":"2023-09-13T04:37:59.238255+0000","last_became_peered":"2023-09-13T04:37:59.238255+0000","last_unstale":"2023-09-13T04:38:23.588778+0000","last_undegraded":"2023-09-13T04:38:23.588778+0000","last_fullsized":"2023-09-13T04:38:23.588778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:19:03.433404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595875+0000","last_change":"2023-09-13T04:37:59.252243+0000","last_active":"2023-09-13T04:38:23.595875+0000","last_peered":"2023-09-13T04:38:23.595875+0000","last_clean":"2023-09-13T04:38:23.595875+0000","last_became_active":"2023-09-13T04:37:59.251902+0000","last_became_peered":"2023-09-13T04:37:59.251902+0000","last_unstale":"2023-09-13T04:38:23.595875+0000","last_undegraded":"2023-09-13T04:38:23.595875+0000","last_fullsized":"2023-09-13T04:38:23.595875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:35:57.395627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593414+0000","last_change":"2023-09-13T04:37:59.245847+0000","last_active":"2023-09-13T04:38:23.593414+0000","last_peered":"2023-09-13T04:38:23.593414+0000","last_clean":"2023-09-13T04:38:23.593414+0000","last_became_active":"2023-09-13T04:37:59.245542+0000","last_became_peered":"2023-09-13T04:37:59.245542+0000","last_unstale":"2023-09-13T04:38:23.593414+0000","last_undegraded":"2023-09-13T04:38:23.593414+0000","last_fullsized":"2023-09-13T04:38:23.593414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:24:16.813333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.596961+0000","last_change":"2023-09-13T04:37:59.246963+0000","last_active":"2023-09-13T04:38:23.596961+0000","last_peered":"2023-09-13T04:38:23.596961+0000","last_clean":"2023-09-13T04:38:23.596961+0000","last_became_active":"2023-09-13T04:37:59.246448+0000","last_became_peered":"2023-09-13T04:37:59.246448+0000","last_unstale":"2023-09-13T04:38:23.596961+0000","last_undegraded":"2023-09-13T04:38:23.596961+0000","last_fullsized":"2023-09-13T04:38:23.596961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:25:45.418778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594086+0000","last_change":"2023-09-13T04:37:59.245399+0000","last_active":"2023-09-13T04:38:23.594086+0000","last_peered":"2023-09-13T04:38:23.594086+0000","last_clean":"2023-09-13T04:38:23.594086+0000","last_became_active":"2023-09-13T04:37:59.245060+0000","last_became_peered":"2023-09-13T04:37:59.245060+0000","last_unstale":"2023-09-13T04:38:23.594086+0000","last_undegraded":"2023-09-13T04:38:23.594086+0000","last_fullsized":"2023-09-13T04:38:23.594086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:25:41.078238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597807+0000","last_change":"2023-09-13T04:37:59.244776+0000","last_active":"2023-09-13T04:38:23.597807+0000","last_peered":"2023-09-13T04:38:23.597807+0000","last_clean":"2023-09-13T04:38:23.597807+0000","last_became_active":"2023-09-13T04:37:59.244409+0000","last_became_peered":"2023-09-13T04:37:59.244409+0000","last_unstale":"2023-09-13T04:38:23.597807+0000","last_undegraded":"2023-09-13T04:38:23.597807+0000","last_fullsized":"2023-09-13T04:38:23.597807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:04:02.561414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592391+0000","last_change":"2023-09-13T04:37:59.248153+0000","last_active":"2023-09-13T04:38:23.592391+0000","last_peered":"2023-09-13T04:38:23.592391+0000","last_clean":"2023-09-13T04:38:23.592391+0000","last_became_active":"2023-09-13T04:37:59.247840+0000","last_became_peered":"2023-09-13T04:37:59.247840+0000","last_unstale":"2023-09-13T04:38:23.592391+0000","last_undegraded":"2023-09-13T04:38:23.592391+0000","last_fullsized":"2023-09-13T04:38:23.592391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:52:53.021219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7452,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":316,"num_read_kb":171,"num_write":129,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":22356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416412,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":490770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.625,"15min":0.61599999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":2.8420000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":0.624},{"interface":"front","average":{"1min":0.622,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.66500000000000004}]},{"osd":1,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.66800000000000004,"15min":0.66100000000000003},"min":{"1min":0.46999999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92700000000000005,"5min":1.024,"15min":1.024},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.42099999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80600000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.80200000000000005}]},{"osd":2,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.47299999999999998,"15min":0.46899999999999997},"min":{"1min":0.34799999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.34799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":4,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.68600000000000005,"15min":0.67800000000000005},"min":{"1min":0.53100000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.42899999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86699999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.60699999999999998}]},{"osd":5,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.64700000000000002,"15min":0.63100000000000001},"min":{"1min":0.45300000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.45500000000000002}]},{"osd":6,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70599999999999996,"15min":0.70399999999999996},"min":{"1min":0.46999999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.70799999999999996,"15min":0.69799999999999995},"min":{"1min":0.55900000000000005,"5min":0.498,"15min":0.498},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.54800000000000004}]},{"osd":7,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.77500000000000002,"15min":0.76500000000000001},"min":{"1min":0.59599999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0229999999999999,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.50800000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94099999999999995,"5min":1.262,"15min":1.262},"last":0.75}]},{"osd":8,"last update":"Wed Sep 13 04:38:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.69899999999999995}]},{"osd":9,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.77400000000000002,"15min":0.76200000000000001},"min":{"1min":0.61499999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.121,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.82499999999999996,"15min":0.81100000000000005},"min":{"1min":0.66200000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1830000000000001,"5min":1.361,"15min":1.361},"last":0.81299999999999994}]},{"osd":10,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.78900000000000003,"15min":0.78600000000000003},"min":{"1min":0.56000000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.002,"5min":1.155,"15min":1.155},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.77300000000000002,"15min":0.76700000000000002},"min":{"1min":0.59599999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95699999999999996,"5min":1.282,"15min":1.282},"last":0.77700000000000002}]}]},{"osd":10,"up_from":71,"seq":304942678048,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1085549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:00 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":4.0510000000000002,"5min":4.0510000000000002,"15min":4.0510000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":4.1120000000000001,"5min":4.1120000000000001,"15min":4.1120000000000001},"last":0.505}]},{"osd":1,"last update":"Wed Sep 13 04:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.46300000000000002,"15min":0.45700000000000002},"min":{"1min":0.32900000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.47899999999999998,"15min":0.47099999999999997},"min":{"1min":0.312,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.57099999999999995}]},{"osd":2,"last update":"Wed Sep 13 04:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.63300000000000001,"15min":0.63100000000000001},"min":{"1min":0.47499999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.83999999999999997,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.63900000000000001,"15min":0.64100000000000001},"min":{"1min":0.51300000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.78200000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.75800000000000001}]},{"osd":3,"last update":"Wed Sep 13 04:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.70799999999999996,"15min":0.67200000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.5570000000000004,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69299999999999995,"15min":0.69099999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84799999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.63}]},{"osd":4,"last update":"Wed Sep 13 04:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60599999999999998,"15min":0.60299999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.372},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55200000000000005,"15min":0.54800000000000004},"min":{"1min":0.36199999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78000000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.432}]},{"osd":5,"last update":"Wed Sep 13 04:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69699999999999995,"15min":0.69299999999999995},"min":{"1min":0.51700000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.87,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.68600000000000005}]},{"osd":6,"last update":"Wed Sep 13 04:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.86599999999999999,"15min":0.88400000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.476,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":1.1200000000000001,"5min":0.872,"15min":0.82999999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":7.7839999999999998,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":0.73899999999999999}]},{"osd":7,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.64500000000000002}]},{"osd":8,"last update":"Wed Sep 13 04:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.77300000000000002,"15min":0.76900000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.79700000000000004,"15min":0.80500000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.92600000000000005,"5min":1.03,"15min":1.03},"last":0.71899999999999997}]},{"osd":9,"last update":"Wed Sep 13 04:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.878,"15min":0.85499999999999998},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":4.2679999999999998,"5min":4.2679999999999998,"15min":4.2679999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.874,"15min":0.85899999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":4.3600000000000003,"5min":4.3600000000000003,"15min":4.3600000000000003},"last":0.66200000000000003}]},{"osd":11,"last update":"Wed Sep 13 04:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.83299999999999996}]}]},{"osd":9,"up_from":66,"seq":283467841573,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":490860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58099999999999996,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.316,"15min":0.316},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.60099999999999998}]},{"osd":1,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.44,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":1.036},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.71999999999999997,"15min":0.73199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82899999999999996,"5min":0.995,"15min":0.995},"last":0.68000000000000005}]},{"osd":2,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.747,"15min":0.75600000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.042,"5min":1.083,"15min":1.083},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.746,"15min":0.754},"min":{"1min":0.57099999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.83499999999999996,"5min":1.109,"15min":1.109},"last":0.64200000000000002}]},{"osd":3,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.42899999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.35999999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":0.999}]},{"osd":4,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.78800000000000003,"15min":0.80500000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.82999999999999996,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.80400000000000005,"15min":0.81200000000000006},"min":{"1min":0.55600000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2,"5min":1.278,"15min":1.278},"last":0.78200000000000003}]},{"osd":5,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.85399999999999998,"15min":0.86699999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2689999999999999,"5min":1.339,"15min":1.339},"last":0.748},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.81000000000000005,"15min":0.81599999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.119,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":1.071}]},{"osd":6,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.72099999999999997}]},{"osd":7,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.90200000000000002,"15min":0.91400000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.3009999999999999,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.90500000000000003,"15min":0.92600000000000005},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1439999999999999,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":1.1060000000000001}]},{"osd":8,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.91700000000000004,"15min":0.93999999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0249999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.95499999999999996,"15min":0.97099999999999997},"min":{"1min":0.60299999999999998,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.2350000000000001,"5min":1.857,"15min":1.857},"last":0.71099999999999997}]},{"osd":10,"last update":"Wed Sep 13 04:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.89300000000000002,"15min":0.89900000000000002},"min":{"1min":0.67200000000000004,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.206,"5min":1.333,"15min":1.333},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.90100000000000002,"15min":0.91300000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.044,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.1639999999999999}]},{"osd":11,"last update":"Wed Sep 13 04:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":1.137}]}]},{"osd":8,"up_from":58,"seq":249108103209,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":490337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58699999999999997,"15min":0.58799999999999997},"min":{"1min":0.38900000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.72799999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.41199999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80400000000000005,"5min":1.016,"15min":1.016},"last":0.496}]},{"osd":1,"last update":"Wed Sep 13 04:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.57499999999999996,"15min":0.54500000000000004},"min":{"1min":0.45700000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.87,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.46100000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82799999999999996,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.629}]},{"osd":2,"last update":"Wed Sep 13 04:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47899999999999998,"15min":0.47499999999999998},"min":{"1min":0.32100000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67800000000000005,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.49199999999999999,"15min":0.48699999999999999},"min":{"1min":0.309,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82299999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.45800000000000002}]},{"osd":3,"last update":"Wed Sep 13 04:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.434,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90600000000000003,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.64400000000000002,"15min":0.64200000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.80800000000000005,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.58199999999999996}]},{"osd":4,"last update":"Wed Sep 13 04:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.61099999999999999,"15min":0.59099999999999997},"min":{"1min":0.53800000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91200000000000003,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.64200000000000002,"15min":0.621},"min":{"1min":0.51000000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89600000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.67300000000000004}]},{"osd":5,"last update":"Wed Sep 13 04:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.35999999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.90000000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59099999999999997,"15min":0.58799999999999997},"min":{"1min":0.36099999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.871,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.65700000000000003}]},{"osd":6,"last update":"Wed Sep 13 04:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.90000000000000002,"5min":1.01,"15min":1.01},"last":0.749},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54000000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.97199999999999998,"5min":1.077,"15min":1.077},"last":0.77700000000000002}]},{"osd":7,"last update":"Wed Sep 13 04:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.71999999999999997,"15min":0.71299999999999997},"min":{"1min":0.54100000000000004,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94599999999999995,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.71199999999999997,"15min":0.69399999999999995},"min":{"1min":0.57199999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.71999999999999997}]},{"osd":9,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.77000000000000002,"15min":0.76200000000000001},"min":{"1min":0.59099999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.76800000000000002,"15min":0.75600000000000001},"min":{"1min":0.58199999999999996,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.86599999999999999}]},{"osd":10,"last update":"Wed Sep 13 04:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.751,"15min":0.73699999999999999},"min":{"1min":0.57399999999999995,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.77100000000000002,"15min":0.75900000000000001},"min":{"1min":0.56399999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.83399999999999996}]},{"osd":11,"last update":"Wed Sep 13 04:39:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.85199999999999998}]}]},{"osd":7,"up_from":51,"seq":219043332143,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1609728,"data_stored":1082202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.46500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.746,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60499999999999998,"15min":0.60799999999999998},"min":{"1min":0.41599999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.40200000000000002}]},{"osd":1,"last update":"Wed Sep 13 04:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.42299999999999999,"15min":0.40799999999999997},"min":{"1min":0.30599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65600000000000003,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.496},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.432,"15min":0.42399999999999999},"min":{"1min":0.32900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61899999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.64500000000000002}]},{"osd":2,"last update":"Wed Sep 13 04:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60399999999999998,"15min":0.59199999999999997},"min":{"1min":0.40600000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83399999999999996,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.59899999999999998,"15min":0.57799999999999996},"min":{"1min":0.49099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.749}]},{"osd":3,"last update":"Wed Sep 13 04:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.69699999999999995,"15min":0.65300000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":7.0789999999999997,"5min":7.0789999999999997,"15min":7.0789999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.45000000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":0.76300000000000001}]},{"osd":4,"last update":"Wed Sep 13 04:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51400000000000001,"15min":0.51100000000000001},"min":{"1min":0.379,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.50800000000000001,"15min":0.51400000000000001},"min":{"1min":0.35799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.68400000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53500000000000003}]},{"osd":5,"last update":"Wed Sep 13 04:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64100000000000001,"15min":0.63},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.66300000000000003,"15min":0.64700000000000002},"min":{"1min":0.56599999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88100000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.81599999999999995}]},{"osd":6,"last update":"Wed Sep 13 04:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.60299999999999998}]},{"osd":8,"last update":"Wed Sep 13 04:39:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.71699999999999997,"15min":0.70899999999999996},"min":{"1min":0.59399999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72099999999999997,"15min":0.71699999999999997},"min":{"1min":0.56599999999999995,"5min":0.437,"15min":0.437},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.91000000000000003}]},{"osd":9,"last update":"Wed Sep 13 04:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.76800000000000002,"15min":0.76600000000000001},"min":{"1min":0.60299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.76100000000000001,"15min":0.75900000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.877}]},{"osd":10,"last update":"Wed Sep 13 04:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69499999999999995,"15min":0.69699999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.56499999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.85799999999999998}]},{"osd":11,"last update":"Wed Sep 13 04:38:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.94399999999999995}]}]},{"osd":6,"up_from":46,"seq":197568495666,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":490132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46100000000000002,"15min":0.45100000000000001},"min":{"1min":0.34699999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.503,"15min":0.48999999999999999},"min":{"1min":0.38100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.436}]},{"osd":1,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.47099999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82499999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.70999999999999996,"15min":0.67900000000000005},"min":{"1min":0.60399999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.58799999999999997}]},{"osd":2,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.52400000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0129999999999999,"5min":1.054,"15min":1.054},"last":0.80900000000000005}]},{"osd":3,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.66100000000000003,"15min":0.64100000000000001},"min":{"1min":0.47799999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83599999999999997,"5min":1.113,"15min":1.113},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.626,"15min":0.61799999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89900000000000002,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.46100000000000002}]},{"osd":4,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76400000000000001,"15min":0.747},"min":{"1min":0.67900000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0509999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.62},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.75900000000000001,"15min":0.77100000000000002},"min":{"1min":0.52600000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92900000000000005,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.69099999999999995}]},{"osd":5,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.80800000000000005,"15min":0.79200000000000004},"min":{"1min":0.56100000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1950000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.872},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.79200000000000004,"15min":0.80800000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.79100000000000004}]},{"osd":7,"last update":"Wed Sep 13 04:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.85699999999999998,"15min":0.82899999999999996},"min":{"1min":0.69299999999999995,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.361,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.1819999999999999,"5min":0.878,"15min":0.81499999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":8.3970000000000002,"5min":8.3970000000000002,"15min":8.3970000000000002},"last":0.60199999999999998}]},{"osd":8,"last update":"Wed Sep 13 04:39:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.82099999999999995,"15min":0.82499999999999996},"min":{"1min":0.65200000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.93000000000000005,"15min":0.92800000000000005},"min":{"1min":0.57999999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.84599999999999997}]},{"osd":9,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.86499999999999999,"15min":0.88300000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.98099999999999998,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.88900000000000001,"5min":1.0069999999999999,"15min":1.0269999999999999},"min":{"1min":0.72399999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.0089999999999999,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.77000000000000002}]},{"osd":10,"last update":"Wed Sep 13 04:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.995,"15min":0.99299999999999999},"min":{"1min":0.81399999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.262,"5min":0.94499999999999995,"15min":0.89200000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":8.4380000000000006,"5min":8.4380000000000006,"15min":8.4380000000000006},"last":0.82299999999999995}]},{"osd":11,"last update":"Wed Sep 13 04:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":8.5410000000000004,"5min":8.5410000000000004,"15min":8.5410000000000004},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.78200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":493884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57099999999999995,"15min":0.54200000000000004},"min":{"1min":0.432,"5min":0.307,"15min":0.307},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.59099999999999997,"15min":0.55900000000000005},"min":{"1min":0.42399999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.94099999999999995,"5min":0.999,"15min":0.999},"last":0.73999999999999999}]},{"osd":2,"last update":"Wed Sep 13 04:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58099999999999996,"15min":0.54500000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.58699999999999997,"15min":0.53100000000000003},"min":{"1min":0.42399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.57599999999999996}]},{"osd":3,"last update":"Wed Sep 13 04:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83099999999999996,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.623,"15min":0.60899999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91600000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.66700000000000004}]},{"osd":4,"last update":"Wed Sep 13 04:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46800000000000003,"15min":0.45500000000000002},"min":{"1min":0.30299999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79500000000000004,"5min":0.878,"15min":0.878},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.47499999999999998,"15min":0.45400000000000001},"min":{"1min":0.29899999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.747,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.439}]},{"osd":5,"last update":"Wed Sep 13 04:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.60499999999999998,"15min":0.57299999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96899999999999997,"5min":1.3,"15min":1.3},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.65100000000000002,"15min":0.59499999999999997},"min":{"1min":0.47299999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.96999999999999997,"5min":1.264,"15min":1.264},"last":0.629}]},{"osd":6,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.99199999999999999,"5min":1.482,"15min":1.482},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95399999999999996,"5min":1.375,"15min":1.375},"last":0.72199999999999998}]},{"osd":7,"last update":"Wed Sep 13 04:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.63300000000000001,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.372,"15min":0.372},"max":{"1min":1.137,"5min":1.347,"15min":1.347},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.67800000000000005,"15min":0.65000000000000002},"min":{"1min":0.38900000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0209999999999999,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.872}]},{"osd":8,"last update":"Wed Sep 13 04:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.75700000000000001,"15min":0.74199999999999999},"min":{"1min":0.48099999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0980000000000001,"5min":1.393,"15min":1.393},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.73999999999999999,"15min":0.72399999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.032,"5min":1.224,"15min":1.224},"last":0.79900000000000004}]},{"osd":9,"last update":"Wed Sep 13 04:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.85399999999999998,"15min":0.85899999999999999},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1819999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.85399999999999998,"15min":0.86299999999999999},"min":{"1min":0.58199999999999996,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.02,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.93000000000000005}]},{"osd":10,"last update":"Wed Sep 13 04:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81599999999999995,"15min":0.82099999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0740000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.56399999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.071,"5min":1.504,"15min":1.504},"last":0.77800000000000002}]},{"osd":11,"last update":"Wed Sep 13 04:38:59 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.83299999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":491520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 13 04:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.69999999999999996,"15min":0.59599999999999997},"min":{"1min":0.38700000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94499999999999995,"5min":6.2359999999999998,"15min":6.2359999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.61799999999999999,"15min":0.56200000000000006},"min":{"1min":0.42699999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.92000000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.40600000000000003}]},{"osd":2,"last update":"Wed Sep 13 04:38:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64200000000000002,"15min":0.60499999999999998},"min":{"1min":0.40000000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.66300000000000003,"15min":0.61199999999999999},"min":{"1min":0.47199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.89500000000000002}]},{"osd":3,"last update":"Wed Sep 13 04:39:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.433}]},{"osd":4,"last update":"Wed Sep 13 04:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.69399999999999995,"15min":0.625},"min":{"1min":0.47499999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.079,"5min":1.135,"15min":1.135},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.67700000000000005,"15min":0.63},"min":{"1min":0.51500000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93400000000000005,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.57599999999999996}]},{"osd":5,"last update":"Wed Sep 13 04:39:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.68300000000000005,"15min":0.67000000000000004},"min":{"1min":0.40999999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.73899999999999999,"15min":0.69299999999999995},"min":{"1min":0.52300000000000002,"5min":0.435,"15min":0.435},"max":{"1min":1.329,"5min":1.444,"15min":1.444},"last":0.82799999999999996}]},{"osd":6,"last update":"Wed Sep 13 04:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.44600000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94899999999999995,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88700000000000001,"5min":1.165,"15min":1.165},"last":0.55300000000000005}]},{"osd":7,"last update":"Wed Sep 13 04:39:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.70199999999999996}]},{"osd":8,"last update":"Wed Sep 13 04:39:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.90800000000000003,"15min":0.86099999999999999},"min":{"1min":0.57699999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3759999999999999,"5min":6.2089999999999996,"15min":6.2089999999999996},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.90300000000000002,"15min":0.83099999999999996},"min":{"1min":0.58999999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.591,"5min":6.1699999999999999,"15min":6.1699999999999999},"last":0.79300000000000004}]},{"osd":9,"last update":"Wed Sep 13 04:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.92200000000000004,"15min":0.94599999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97399999999999998,"5min":1.357,"15min":1.357},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.171,"5min":1.484,"15min":1.484},"last":0.75}]},{"osd":10,"last update":"Wed Sep 13 04:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.98899999999999999,"15min":0.98499999999999999},"min":{"1min":0.80800000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.522,"5min":1.591,"15min":1.591},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.1040000000000001,"15min":1.139},"min":{"1min":0.66300000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.3859999999999999,"5min":6.1859999999999999,"15min":6.1859999999999999},"last":0.623}]},{"osd":11,"last update":"Wed Sep 13 04:39:16 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":0.99099999999999999,"15min":0.98399999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.86099999999999999,"15min":0.85299999999999998},"min":{"1min":0.71499999999999997,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.72399999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":495200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.54900000000000004,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.54900000000000004,"15min":0.52800000000000002},"min":{"1min":0.39900000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.436}]},{"osd":1,"last update":"Wed Sep 13 04:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.57399999999999995,"15min":0.50900000000000001},"min":{"1min":0.438,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.86599999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.57799999999999996,"15min":0.51900000000000002},"min":{"1min":0.41399999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.877,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.499}]},{"osd":3,"last update":"Wed Sep 13 04:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.61899999999999999,"15min":0.59199999999999997},"min":{"1min":0.45700000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.623,"15min":0.60999999999999999},"min":{"1min":0.48899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.60799999999999998}]},{"osd":4,"last update":"Wed Sep 13 04:39:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.58599999999999997}]},{"osd":5,"last update":"Wed Sep 13 04:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.47299999999999998,"15min":0.49399999999999999},"min":{"1min":0.378,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.65200000000000002,"5min":1.796,"15min":1.796},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52800000000000002,"15min":0.53200000000000003},"min":{"1min":0.38300000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73899999999999999,"5min":1.546,"15min":1.546},"last":0.68500000000000005}]},{"osd":6,"last update":"Wed Sep 13 04:38:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.69199999999999995,"15min":0.69899999999999995},"min":{"1min":0.48499999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81599999999999995,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.752},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66700000000000004,"15min":0.67200000000000004},"min":{"1min":0.48199999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85899999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.72699999999999998}]},{"osd":7,"last update":"Wed Sep 13 04:39:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.74299999999999999}]},{"osd":8,"last update":"Wed Sep 13 04:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61799999999999999,"15min":0.62},"min":{"1min":0.40300000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.82399999999999995,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.66700000000000004,"15min":0.65300000000000002},"min":{"1min":0.51200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.90400000000000003,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.87}]},{"osd":9,"last update":"Wed Sep 13 04:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72099999999999997,"15min":0.71899999999999997},"min":{"1min":0.57299999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.93100000000000005,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.77800000000000002}]},{"osd":10,"last update":"Wed Sep 13 04:38:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.70199999999999996,"15min":0.68999999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.92600000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.75800000000000001,"15min":0.73599999999999999},"min":{"1min":0.66900000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.012,"5min":1.032,"15min":1.032},"last":0.91200000000000003}]},{"osd":11,"last update":"Wed Sep 13 04:39:00 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.94799999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":490326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.45500000000000002,"15min":0.39900000000000002},"min":{"1min":0.30399999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.52900000000000003,"15min":0.45600000000000002},"min":{"1min":0.34399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.5940000000000003,"5min":4.5940000000000003,"15min":4.5940000000000003},"last":0.55800000000000005}]},{"osd":1,"last update":"Wed Sep 13 04:39:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.64000000000000001,"15min":0.58699999999999997},"min":{"1min":0.42399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.68000000000000005,"15min":0.58999999999999997},"min":{"1min":0.50600000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":1.105}]},{"osd":2,"last update":"Wed Sep 13 04:39:13 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.72799999999999998,"15min":0.64700000000000002},"min":{"1min":0.47699999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":4.3920000000000003,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.65900000000000003,"15min":0.626},"min":{"1min":0.46100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97999999999999998,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.50800000000000001}]},{"osd":4,"last update":"Wed Sep 13 04:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66700000000000004,"15min":0.60899999999999999},"min":{"1min":0.53100000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.66700000000000004,"15min":0.61599999999999999},"min":{"1min":0.46500000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.88900000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.67900000000000005}]},{"osd":5,"last update":"Wed Sep 13 04:39:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.65400000000000003},"min":{"1min":0.432,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89600000000000002,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.77600000000000002,"15min":0.68400000000000005},"min":{"1min":0.48299999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":4.3339999999999996,"5min":4.3339999999999996,"15min":4.3339999999999996},"last":0.85899999999999999}]},{"osd":6,"last update":"Wed Sep 13 04:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.74099999999999999,"15min":0.70299999999999996},"min":{"1min":0.47399999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1060000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.76800000000000002,"15min":0.745},"min":{"1min":0.504,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0529999999999999,"5min":1.109,"15min":1.109},"last":1.1870000000000001}]},{"osd":7,"last update":"Wed Sep 13 04:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.85699999999999998,"15min":0.86199999999999999},"min":{"1min":0.56299999999999994,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.014,"5min":0.88800000000000001,"15min":0.86099999999999999},"min":{"1min":0.59299999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":4.4699999999999998,"5min":4.4699999999999998,"15min":4.4699999999999998},"last":0.92500000000000004}]},{"osd":8,"last update":"Wed Sep 13 04:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.875}]},{"osd":9,"last update":"Wed Sep 13 04:38:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.94699999999999995,"15min":0.96399999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.2030000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.84099999999999997,"15min":0.84899999999999998},"min":{"1min":0.52100000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.01,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.71299999999999997}]},{"osd":10,"last update":"Wed Sep 13 04:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.58899999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.091,"5min":0.98199999999999998,"15min":0.96399999999999997},"min":{"1min":0.747,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":4.5099999999999998,"5min":4.5099999999999998,"15min":4.5099999999999998},"last":0.96199999999999997}]},{"osd":11,"last update":"Wed Sep 13 04:38:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.83599999999999997}]}]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":490686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.56599999999999995,"15min":0.54400000000000004},"min":{"1min":0.43099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.753},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76500000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.68999999999999995}]},{"osd":1,"last update":"Wed Sep 13 04:38:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.39600000000000002,"15min":0.35399999999999998},"min":{"1min":0.29499999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.623,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.40699999999999997,"15min":0.371},"min":{"1min":0.33600000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.65900000000000003,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.495}]},{"osd":2,"last update":"Wed Sep 13 04:38:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60099999999999998,"15min":0.56699999999999995},"min":{"1min":0.35799999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88800000000000001,"5min":1.631,"15min":1.631},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.58999999999999997,"15min":0.54600000000000004},"min":{"1min":0.42299999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.878,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.65600000000000003}]},{"osd":3,"last update":"Wed Sep 13 04:38:41 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60899999999999999,"15min":0.56999999999999995},"min":{"1min":0.51900000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.81799999999999995,"5min":0.995,"15min":0.995},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60099999999999998,"15min":0.57899999999999996},"min":{"1min":0.44500000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78700000000000003,"5min":1.115,"15min":1.115},"last":0.68500000000000005}]},{"osd":5,"last update":"Wed Sep 13 04:38:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.622,"15min":0.59199999999999997},"min":{"1min":0.54700000000000004,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89300000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.624,"15min":0.61099999999999999},"min":{"1min":0.49299999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86899999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.77900000000000003}]},{"osd":6,"last update":"Wed Sep 13 04:38:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.68000000000000005,"15min":0.68400000000000005},"min":{"1min":0.47099999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81599999999999995,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.69799999999999995,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.84599999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.88200000000000001}]},{"osd":7,"last update":"Wed Sep 13 04:38:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.61799999999999999,"15min":0.59899999999999998},"min":{"1min":0.46500000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.85399999999999998,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.753}]},{"osd":8,"last update":"Wed Sep 13 04:39:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.72399999999999998,"15min":0.69799999999999995},"min":{"1min":0.59599999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.97599999999999998,"5min":1.042,"15min":1.042},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76700000000000002,"15min":0.76300000000000001},"min":{"1min":0.57099999999999995,"5min":0.441,"15min":0.441},"max":{"1min":0.98799999999999999,"5min":1.095,"15min":1.095},"last":0.92600000000000005}]},{"osd":9,"last update":"Wed Sep 13 04:38:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.80700000000000005,"15min":0.81200000000000006},"min":{"1min":0.59299999999999997,"5min":0.495,"15min":0.495},"max":{"1min":0.93000000000000005,"5min":1.141,"15min":1.141},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.77100000000000002,"15min":0.77300000000000002},"min":{"1min":0.60799999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.879,"5min":1.056,"15min":1.056},"last":0.82799999999999996}]},{"osd":10,"last update":"Wed Sep 13 04:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.75900000000000001,"15min":0.76000000000000001},"min":{"1min":0.54600000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.93600000000000005,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.78900000000000003,"15min":0.79200000000000004},"min":{"1min":0.60499999999999998,"5min":0.498,"15min":0.498},"max":{"1min":0.97099999999999997,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.80100000000000005}]},{"osd":11,"last update":"Wed Sep 13 04:38:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.91500000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1622016,"data_stored":1081886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.35199999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47099999999999997}]},{"osd":1,"last update":"Wed Sep 13 04:39:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.55900000000000005,"15min":0.53000000000000003},"min":{"1min":0.46600000000000003,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.83999999999999997,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.57299999999999995,"15min":0.53400000000000003},"min":{"1min":0.51600000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84899999999999998,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.65600000000000003}]},{"osd":2,"last update":"Wed Sep 13 04:39:00 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.40699999999999997,"15min":0.37},"min":{"1min":0.307,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.42299999999999999,"15min":0.38700000000000001},"min":{"1min":0.312,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.75900000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.495}]},{"osd":3,"last update":"Wed Sep 13 04:39:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.60299999999999998,"15min":0.58699999999999997},"min":{"1min":0.42999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.73399999999999999,"15min":0.85099999999999998},"min":{"1min":0.39700000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77700000000000002,"5min":8.0009999999999994,"15min":8.0009999999999994},"last":0.745}]},{"osd":4,"last update":"Wed Sep 13 04:39:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.60599999999999998,"15min":0.57099999999999995},"min":{"1min":0.45800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.877,"5min":1.599,"15min":1.599},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.60599999999999998,"15min":0.56499999999999995},"min":{"1min":0.52100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.93899999999999995,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.61499999999999999}]},{"osd":6,"last update":"Wed Sep 13 04:38:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83499999999999996,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81899999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.63700000000000001}]},{"osd":7,"last update":"Wed Sep 13 04:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.65900000000000003,"15min":0.64900000000000002},"min":{"1min":0.51800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90400000000000003,"5min":1.488,"15min":1.488},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.68100000000000005,"15min":0.66900000000000004},"min":{"1min":0.53400000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90000000000000002,"5min":1.51,"15min":1.51},"last":0.67700000000000005}]},{"osd":8,"last update":"Wed Sep 13 04:39:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59599999999999997,"15min":0.58799999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.83699999999999997,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.877,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.53400000000000003}]},{"osd":9,"last update":"Wed Sep 13 04:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.72799999999999998,"15min":0.72499999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.60599999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.71299999999999997}]},{"osd":10,"last update":"Wed Sep 13 04:38:54 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.73299999999999998,"15min":0.72199999999999998},"min":{"1min":0.59699999999999998,"5min":0.496,"15min":0.496},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.754,"15min":0.748},"min":{"1min":0.60699999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.70099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-13T04:39:20.494 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph pg dump --format=json 2023-09-13T04:39:20.558 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:20 smithi002 ceph-mon[97651]: pgmap v318: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:39:20.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:20 smithi149 ceph-mon[133315]: pgmap v318: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:39:20.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:20 smithi156 ceph-mon[133675]: pgmap v318: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:39:21.229 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:39:21.541 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:21 smithi002 ceph-mon[97651]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:39:21.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:21 smithi149 ceph-mon[133315]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:39:21.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:21 smithi156 ceph-mon[133675]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:39:22.524 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:22 smithi002 ceph-mon[97651]: pgmap v319: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-13T04:39:22.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:22 smithi149 ceph-mon[133315]: pgmap v319: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-13T04:39:22.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:22 smithi156 ceph-mon[133675]: pgmap v319: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-13T04:39:22.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.780+0000 7f9c54a08700 1 -- 172.21.15.2:0/899332376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c50073420 msgr2=0x7f9c50073880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:22.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.780+0000 7f9c54a08700 1 --2- 172.21.15.2:0/899332376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c50073420 0x7f9c50073880 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9c440098f0 tx=0x7f9c44009c00 comp rx=0 tx=0).stop 2023-09-13T04:39:22.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.780+0000 7f9c54a08700 1 -- 172.21.15.2:0/899332376 shutdown_connections 2023-09-13T04:39:22.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.780+0000 7f9c54a08700 1 --2- 172.21.15.2:0/899332376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c501119c0 0x7f9c50113db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:22.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.780+0000 7f9c54a08700 1 --2- 172.21.15.2:0/899332376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c50074cf0 0x7f9c50072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:22.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.780+0000 7f9c54a08700 1 --2- 172.21.15.2:0/899332376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c50073420 0x7f9c50073880 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:22.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.780+0000 7f9c54a08700 1 -- 172.21.15.2:0/899332376 >> 172.21.15.2:0/899332376 conn(0x7f9c50077ea0 msgr2=0x7f9c500782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:22.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.781+0000 7f9c54a08700 1 -- 172.21.15.2:0/899332376 shutdown_connections 2023-09-13T04:39:22.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.781+0000 7f9c54a08700 1 -- 172.21.15.2:0/899332376 wait complete. 2023-09-13T04:39:22.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.782+0000 7f9c54a08700 1 Processor -- start 2023-09-13T04:39:22.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.782+0000 7f9c54a08700 1 -- start start 2023-09-13T04:39:22.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.782+0000 7f9c54a08700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c50073420 0x7f9c5019d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:22.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.782+0000 7f9c54a08700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c50074cf0 0x7f9c5019d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:22.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.782+0000 7f9c54a08700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c501119c0 0x7f9c501a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:22.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.782+0000 7f9c54a08700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c501a2440 con 0x7f9c50073420 2023-09-13T04:39:22.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.782+0000 7f9c54a08700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c501a25b0 con 0x7f9c501119c0 2023-09-13T04:39:22.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.782+0000 7f9c54a08700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c501a2770 con 0x7f9c50074cf0 2023-09-13T04:39:22.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.782+0000 7f9c4e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c50073420 0x7f9c5019d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:22.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.783+0000 7f9c4e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c50073420 0x7f9c5019d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53500/0 (socket says 172.21.15.2:53500) 2023-09-13T04:39:22.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.783+0000 7f9c4e59c700 1 -- 172.21.15.2:0/2564681393 learned_addr learned my addr 172.21.15.2:0/2564681393 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:22.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.783+0000 7f9c4ed9d700 1 --2- 172.21.15.2:0/2564681393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c501119c0 0x7f9c501a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:22.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.783+0000 7f9c4dd9b700 1 --2- 172.21.15.2:0/2564681393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c50074cf0 0x7f9c5019d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:22.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.783+0000 7f9c4e59c700 1 -- 172.21.15.2:0/2564681393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c50074cf0 msgr2=0x7f9c5019d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:22.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.783+0000 7f9c4e59c700 1 --2- 172.21.15.2:0/2564681393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c50074cf0 0x7f9c5019d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:22.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.783+0000 7f9c4e59c700 1 -- 172.21.15.2:0/2564681393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c501119c0 msgr2=0x7f9c501a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:22.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.783+0000 7f9c4e59c700 1 --2- 172.21.15.2:0/2564681393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c501119c0 0x7f9c501a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:22.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.783+0000 7f9c4e59c700 1 -- 172.21.15.2:0/2564681393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c44009520 con 0x7f9c50073420 2023-09-13T04:39:22.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.783+0000 7f9c4e59c700 1 --2- 172.21.15.2:0/2564681393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c50073420 0x7f9c5019d480 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000e9c0 tx=0x7f9c4000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:22.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.783+0000 7f9c3f7fe700 1 -- 172.21.15.2:0/2564681393 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c40004a30 con 0x7f9c50073420 2023-09-13T04:39:22.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.784+0000 7f9c54a08700 1 -- 172.21.15.2:0/2564681393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c501a2a50 con 0x7f9c50073420 2023-09-13T04:39:22.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.784+0000 7f9c3f7fe700 1 -- 172.21.15.2:0/2564681393 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c4000cc80 con 0x7f9c50073420 2023-09-13T04:39:22.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.784+0000 7f9c3f7fe700 1 -- 172.21.15.2:0/2564681393 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c400054e0 con 0x7f9c50073420 2023-09-13T04:39:22.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.784+0000 7f9c54a08700 1 -- 172.21.15.2:0/2564681393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c501aa2b0 con 0x7f9c50073420 2023-09-13T04:39:22.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.785+0000 7f9c54a08700 1 -- 172.21.15.2:0/2564681393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c5004e9b0 con 0x7f9c50073420 2023-09-13T04:39:22.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.787+0000 7f9c3f7fe700 1 -- 172.21.15.2:0/2564681393 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f9c40004b90 con 0x7f9c50073420 2023-09-13T04:39:22.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.788+0000 7f9c3f7fe700 1 --2- 172.21.15.2:0/2564681393 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f9c380a0640 0x7f9c380a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:22.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.789+0000 7f9c3f7fe700 1 -- 172.21.15.2:0/2564681393 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9c40014070 con 0x7f9c50073420 2023-09-13T04:39:22.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.789+0000 7f9c4dd9b700 1 --2- 172.21.15.2:0/2564681393 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f9c380a0640 0x7f9c380a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:22.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.789+0000 7f9c4dd9b700 1 --2- 172.21.15.2:0/2564681393 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f9c380a0640 0x7f9c380a2b00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9c44005e90 tx=0x7f9c44019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:22.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.792+0000 7f9c3f7fe700 1 -- 172.21.15.2:0/2564681393 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9c400cf030 con 0x7f9c50073420 2023-09-13T04:39:22.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:22.996+0000 7f9c54a08700 1 -- 172.21.15.2:0/2564681393 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9c50117470 con 0x7f9c380a0640 2023-09-13T04:39:23.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:23.010+0000 7f9c3f7fe700 1 -- 172.21.15.2:0/2564681393 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426714 (secure 0 0 0) 0x7f9c50117470 con 0x7f9c380a0640 2023-09-13T04:39:23.012 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:39:23.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:23.015+0000 7f9c54a08700 1 -- 172.21.15.2:0/2564681393 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f9c380a0640 msgr2=0x7f9c380a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:23.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:23.015+0000 7f9c54a08700 1 --2- 172.21.15.2:0/2564681393 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f9c380a0640 0x7f9c380a2b00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9c44005e90 tx=0x7f9c44019040 comp rx=0 tx=0).stop 2023-09-13T04:39:23.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:23.015+0000 7f9c54a08700 1 -- 172.21.15.2:0/2564681393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c50073420 msgr2=0x7f9c5019d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:23.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:23.015+0000 7f9c54a08700 1 --2- 172.21.15.2:0/2564681393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c50073420 0x7f9c5019d480 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000e9c0 tx=0x7f9c4000ecd0 comp rx=0 tx=0).stop 2023-09-13T04:39:23.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:23.015+0000 7f9c54a08700 1 -- 172.21.15.2:0/2564681393 shutdown_connections 2023-09-13T04:39:23.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:23.015+0000 7f9c54a08700 1 --2- 172.21.15.2:0/2564681393 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f9c380a0640 0x7f9c380a2b00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:23.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:23.015+0000 7f9c54a08700 1 --2- 172.21.15.2:0/2564681393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c501119c0 0x7f9c501a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:23.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:23.015+0000 7f9c54a08700 1 --2- 172.21.15.2:0/2564681393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c50073420 0x7f9c5019d480 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:23.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:23.015+0000 7f9c54a08700 1 --2- 172.21.15.2:0/2564681393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c50074cf0 0x7f9c5019d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:23.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:23.015+0000 7f9c54a08700 1 -- 172.21.15.2:0/2564681393 >> 172.21.15.2:0/2564681393 conn(0x7f9c50077ea0 msgr2=0x7f9c501104e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:23.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:23.015+0000 7f9c54a08700 1 -- 172.21.15.2:0/2564681393 shutdown_connections 2023-09-13T04:39:23.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:23.015+0000 7f9c54a08700 1 -- 172.21.15.2:0/2564681393 wait complete. 2023-09-13T04:39:23.037 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-09-13T04:39:23.428 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2023-09-13T04:39:21.521416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597820,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":482,"num_read_kb":305,"num_write":310,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":250,"ondisk_log_size":250,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337928,"kb_used_data":13684,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603896,"statfs":{"total":1151940427776,"available":1151594389504,"internally_reserved":0,"allocated":14012416,"data_stored":7673352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003317"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592832+0000","last_change":"2023-09-13T04:37:59.244279+0000","last_active":"2023-09-13T04:38:23.592832+0000","last_peered":"2023-09-13T04:38:23.592832+0000","last_clean":"2023-09-13T04:38:23.592832+0000","last_became_active":"2023-09-13T04:37:59.243976+0000","last_became_peered":"2023-09-13T04:37:59.243976+0000","last_unstale":"2023-09-13T04:38:23.592832+0000","last_undegraded":"2023-09-13T04:38:23.592832+0000","last_fullsized":"2023-09-13T04:38:23.592832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:21:52.809411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592579+0000","last_change":"2023-09-13T04:37:59.243391+0000","last_active":"2023-09-13T04:38:23.592579+0000","last_peered":"2023-09-13T04:38:23.592579+0000","last_clean":"2023-09-13T04:38:23.592579+0000","last_became_active":"2023-09-13T04:37:59.243044+0000","last_became_peered":"2023-09-13T04:37:59.243044+0000","last_unstale":"2023-09-13T04:38:23.592579+0000","last_undegraded":"2023-09-13T04:38:23.592579+0000","last_fullsized":"2023-09-13T04:38:23.592579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:02:17.806638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592756+0000","last_change":"2023-09-13T04:37:59.242826+0000","last_active":"2023-09-13T04:38:23.592756+0000","last_peered":"2023-09-13T04:38:23.592756+0000","last_clean":"2023-09-13T04:38:23.592756+0000","last_became_active":"2023-09-13T04:37:59.242125+0000","last_became_peered":"2023-09-13T04:37:59.242125+0000","last_unstale":"2023-09-13T04:38:23.592756+0000","last_undegraded":"2023-09-13T04:38:23.592756+0000","last_fullsized":"2023-09-13T04:38:23.592756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:08:29.216093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590536+0000","last_change":"2023-09-13T04:37:59.253322+0000","last_active":"2023-09-13T04:38:23.590536+0000","last_peered":"2023-09-13T04:38:23.590536+0000","last_clean":"2023-09-13T04:38:23.590536+0000","last_became_active":"2023-09-13T04:37:59.253050+0000","last_became_peered":"2023-09-13T04:37:59.253050+0000","last_unstale":"2023-09-13T04:38:23.590536+0000","last_undegraded":"2023-09-13T04:38:23.590536+0000","last_fullsized":"2023-09-13T04:38:23.590536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:45:22.357879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593487+0000","last_change":"2023-09-13T04:37:59.250089+0000","last_active":"2023-09-13T04:38:23.593487+0000","last_peered":"2023-09-13T04:38:23.593487+0000","last_clean":"2023-09-13T04:38:23.593487+0000","last_became_active":"2023-09-13T04:37:59.249771+0000","last_became_peered":"2023-09-13T04:37:59.249771+0000","last_unstale":"2023-09-13T04:38:23.593487+0000","last_undegraded":"2023-09-13T04:38:23.593487+0000","last_fullsized":"2023-09-13T04:38:23.593487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:46:59.917756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595139+0000","last_change":"2023-09-13T04:37:59.247259+0000","last_active":"2023-09-13T04:38:23.595139+0000","last_peered":"2023-09-13T04:38:23.595139+0000","last_clean":"2023-09-13T04:38:23.595139+0000","last_became_active":"2023-09-13T04:37:59.246911+0000","last_became_peered":"2023-09-13T04:37:59.246911+0000","last_unstale":"2023-09-13T04:38:23.595139+0000","last_undegraded":"2023-09-13T04:38:23.595139+0000","last_fullsized":"2023-09-13T04:38:23.595139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:00:39.742392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587857+0000","last_change":"2023-09-13T04:37:59.244083+0000","last_active":"2023-09-13T04:38:23.587857+0000","last_peered":"2023-09-13T04:38:23.587857+0000","last_clean":"2023-09-13T04:38:23.587857+0000","last_became_active":"2023-09-13T04:37:59.243789+0000","last_became_peered":"2023-09-13T04:37:59.243789+0000","last_unstale":"2023-09-13T04:38:23.587857+0000","last_undegraded":"2023-09-13T04:38:23.587857+0000","last_fullsized":"2023-09-13T04:38:23.587857+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:24:00.622790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.602097+0000","last_change":"2023-09-13T04:38:24.601265+0000","last_active":"2023-09-13T04:38:24.602097+0000","last_peered":"2023-09-13T04:38:24.602097+0000","last_clean":"2023-09-13T04:38:24.602097+0000","last_became_active":"2023-09-13T04:38:24.600939+0000","last_became_peered":"2023-09-13T04:38:24.600939+0000","last_unstale":"2023-09-13T04:38:24.602097+0000","last_undegraded":"2023-09-13T04:38:24.602097+0000","last_fullsized":"2023-09-13T04:38:24.602097+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:18:23.642333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594759+0000","last_change":"2023-09-13T04:37:59.238053+0000","last_active":"2023-09-13T04:38:23.594759+0000","last_peered":"2023-09-13T04:38:23.594759+0000","last_clean":"2023-09-13T04:38:23.594759+0000","last_became_active":"2023-09-13T04:37:59.237777+0000","last_became_peered":"2023-09-13T04:37:59.237777+0000","last_unstale":"2023-09-13T04:38:23.594759+0000","last_undegraded":"2023-09-13T04:38:23.594759+0000","last_fullsized":"2023-09-13T04:38:23.594759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:45:39.835659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594929+0000","last_change":"2023-09-13T04:37:59.252605+0000","last_active":"2023-09-13T04:38:23.594929+0000","last_peered":"2023-09-13T04:38:23.594929+0000","last_clean":"2023-09-13T04:38:23.594929+0000","last_became_active":"2023-09-13T04:37:59.252309+0000","last_became_peered":"2023-09-13T04:37:59.252309+0000","last_unstale":"2023-09-13T04:38:23.594929+0000","last_undegraded":"2023-09-13T04:38:23.594929+0000","last_fullsized":"2023-09-13T04:38:23.594929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:41:45.064230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588277+0000","last_change":"2023-09-13T04:37:59.243345+0000","last_active":"2023-09-13T04:38:23.588277+0000","last_peered":"2023-09-13T04:38:23.588277+0000","last_clean":"2023-09-13T04:38:23.588277+0000","last_became_active":"2023-09-13T04:37:59.242726+0000","last_became_peered":"2023-09-13T04:37:59.242726+0000","last_unstale":"2023-09-13T04:38:23.588277+0000","last_undegraded":"2023-09-13T04:38:23.588277+0000","last_fullsized":"2023-09-13T04:38:23.588277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:39:55.115281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590601+0000","last_change":"2023-09-13T04:37:59.247415+0000","last_active":"2023-09-13T04:38:23.590601+0000","last_peered":"2023-09-13T04:38:23.590601+0000","last_clean":"2023-09-13T04:38:23.590601+0000","last_became_active":"2023-09-13T04:37:59.247050+0000","last_became_peered":"2023-09-13T04:37:59.247050+0000","last_unstale":"2023-09-13T04:38:23.590601+0000","last_undegraded":"2023-09-13T04:38:23.590601+0000","last_fullsized":"2023-09-13T04:38:23.590601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:03:44.793628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597902+0000","last_change":"2023-09-13T04:37:59.248935+0000","last_active":"2023-09-13T04:38:23.597902+0000","last_peered":"2023-09-13T04:38:23.597902+0000","last_clean":"2023-09-13T04:38:23.597902+0000","last_became_active":"2023-09-13T04:37:59.248554+0000","last_became_peered":"2023-09-13T04:37:59.248554+0000","last_unstale":"2023-09-13T04:38:23.597902+0000","last_undegraded":"2023-09-13T04:38:23.597902+0000","last_fullsized":"2023-09-13T04:38:23.597902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:53:31.882567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592020+0000","last_change":"2023-09-13T04:37:59.242851+0000","last_active":"2023-09-13T04:38:23.592020+0000","last_peered":"2023-09-13T04:38:23.592020+0000","last_clean":"2023-09-13T04:38:23.592020+0000","last_became_active":"2023-09-13T04:37:59.242384+0000","last_became_peered":"2023-09-13T04:37:59.242384+0000","last_unstale":"2023-09-13T04:38:23.592020+0000","last_undegraded":"2023-09-13T04:38:23.592020+0000","last_fullsized":"2023-09-13T04:38:23.592020+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:59:36.508808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.599392+0000","last_change":"2023-09-13T04:38:24.599392+0000","last_active":"2023-09-13T04:38:24.599392+0000","last_peered":"2023-09-13T04:38:24.599392+0000","last_clean":"2023-09-13T04:38:24.599392+0000","last_became_active":"2023-09-13T04:38:24.599029+0000","last_became_peered":"2023-09-13T04:38:24.599029+0000","last_unstale":"2023-09-13T04:38:24.599392+0000","last_undegraded":"2023-09-13T04:38:24.599392+0000","last_fullsized":"2023-09-13T04:38:24.599392+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:23:03.703387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592657+0000","last_change":"2023-09-13T04:37:59.249107+0000","last_active":"2023-09-13T04:38:23.592657+0000","last_peered":"2023-09-13T04:38:23.592657+0000","last_clean":"2023-09-13T04:38:23.592657+0000","last_became_active":"2023-09-13T04:37:59.248803+0000","last_became_peered":"2023-09-13T04:37:59.248803+0000","last_unstale":"2023-09-13T04:38:23.592657+0000","last_undegraded":"2023-09-13T04:38:23.592657+0000","last_fullsized":"2023-09-13T04:38:23.592657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:50:52.481194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592743+0000","last_change":"2023-09-13T04:37:59.243724+0000","last_active":"2023-09-13T04:38:23.592743+0000","last_peered":"2023-09-13T04:38:23.592743+0000","last_clean":"2023-09-13T04:38:23.592743+0000","last_became_active":"2023-09-13T04:37:59.243385+0000","last_became_peered":"2023-09-13T04:37:59.243385+0000","last_unstale":"2023-09-13T04:38:23.592743+0000","last_undegraded":"2023-09-13T04:38:23.592743+0000","last_fullsized":"2023-09-13T04:38:23.592743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:55:55.203940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588550+0000","last_change":"2023-09-13T04:37:59.238986+0000","last_active":"2023-09-13T04:38:23.588550+0000","last_peered":"2023-09-13T04:38:23.588550+0000","last_clean":"2023-09-13T04:38:23.588550+0000","last_became_active":"2023-09-13T04:37:59.238668+0000","last_became_peered":"2023-09-13T04:37:59.238668+0000","last_unstale":"2023-09-13T04:38:23.588550+0000","last_undegraded":"2023-09-13T04:38:23.588550+0000","last_fullsized":"2023-09-13T04:38:23.588550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:21:10.225223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588585+0000","last_change":"2023-09-13T04:37:59.245598+0000","last_active":"2023-09-13T04:38:23.588585+0000","last_peered":"2023-09-13T04:38:23.588585+0000","last_clean":"2023-09-13T04:38:23.588585+0000","last_became_active":"2023-09-13T04:37:59.244652+0000","last_became_peered":"2023-09-13T04:37:59.244652+0000","last_unstale":"2023-09-13T04:38:23.588585+0000","last_undegraded":"2023-09-13T04:38:23.588585+0000","last_fullsized":"2023-09-13T04:38:23.588585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:34:43.671168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588238+0000","last_change":"2023-09-13T04:38:03.617708+0000","last_active":"2023-09-13T04:38:23.588238+0000","last_peered":"2023-09-13T04:38:23.588238+0000","last_clean":"2023-09-13T04:38:23.588238+0000","last_became_active":"2023-09-13T04:38:03.617277+0000","last_became_peered":"2023-09-13T04:38:03.617277+0000","last_unstale":"2023-09-13T04:38:23.588238+0000","last_undegraded":"2023-09-13T04:38:23.588238+0000","last_fullsized":"2023-09-13T04:38:23.588238+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:26:44.318030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.621267+0000","last_change":"2023-09-13T04:37:58.595156+0000","last_active":"2023-09-13T04:38:25.621267+0000","last_peered":"2023-09-13T04:38:25.621267+0000","last_clean":"2023-09-13T04:38:25.621267+0000","last_became_active":"2023-09-13T04:37:58.594863+0000","last_became_peered":"2023-09-13T04:37:58.594863+0000","last_unstale":"2023-09-13T04:38:25.621267+0000","last_undegraded":"2023-09-13T04:38:25.621267+0000","last_fullsized":"2023-09-13T04:38:25.621267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:27:14.420495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595813+0000","last_change":"2023-09-13T04:37:59.250252+0000","last_active":"2023-09-13T04:38:23.595813+0000","last_peered":"2023-09-13T04:38:23.595813+0000","last_clean":"2023-09-13T04:38:23.595813+0000","last_became_active":"2023-09-13T04:37:59.249927+0000","last_became_peered":"2023-09-13T04:37:59.249927+0000","last_unstale":"2023-09-13T04:38:23.595813+0000","last_undegraded":"2023-09-13T04:38:23.595813+0000","last_fullsized":"2023-09-13T04:38:23.595813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:18:06.757040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.599683+0000","last_change":"2023-09-13T04:38:24.599683+0000","last_active":"2023-09-13T04:38:24.599683+0000","last_peered":"2023-09-13T04:38:24.599683+0000","last_clean":"2023-09-13T04:38:24.599683+0000","last_became_active":"2023-09-13T04:38:24.599230+0000","last_became_peered":"2023-09-13T04:38:24.599230+0000","last_unstale":"2023-09-13T04:38:24.599683+0000","last_undegraded":"2023-09-13T04:38:24.599683+0000","last_fullsized":"2023-09-13T04:38:24.599683+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:48:22.584735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.596874+0000","last_change":"2023-09-13T04:37:58.597785+0000","last_active":"2023-09-13T04:38:24.596874+0000","last_peered":"2023-09-13T04:38:24.596874+0000","last_clean":"2023-09-13T04:38:24.596874+0000","last_became_active":"2023-09-13T04:37:58.597485+0000","last_became_peered":"2023-09-13T04:37:58.597485+0000","last_unstale":"2023-09-13T04:38:24.596874+0000","last_undegraded":"2023-09-13T04:38:24.596874+0000","last_fullsized":"2023-09-13T04:38:24.596874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:39:57.156038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592830+0000","last_change":"2023-09-13T04:37:59.241840+0000","last_active":"2023-09-13T04:38:23.592830+0000","last_peered":"2023-09-13T04:38:23.592830+0000","last_clean":"2023-09-13T04:38:23.592830+0000","last_became_active":"2023-09-13T04:37:59.241545+0000","last_became_peered":"2023-09-13T04:37:59.241545+0000","last_unstale":"2023-09-13T04:38:23.592830+0000","last_undegraded":"2023-09-13T04:38:23.592830+0000","last_fullsized":"2023-09-13T04:38:23.592830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:04:17.439520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591101+0000","last_change":"2023-09-13T04:38:03.621792+0000","last_active":"2023-09-13T04:38:23.591101+0000","last_peered":"2023-09-13T04:38:23.591101+0000","last_clean":"2023-09-13T04:38:23.591101+0000","last_became_active":"2023-09-13T04:38:03.621433+0000","last_became_peered":"2023-09-13T04:38:03.621433+0000","last_unstale":"2023-09-13T04:38:23.591101+0000","last_undegraded":"2023-09-13T04:38:23.591101+0000","last_fullsized":"2023-09-13T04:38:23.591101+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:52:05.926733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.618200+0000","last_change":"2023-09-13T04:37:58.590817+0000","last_active":"2023-09-13T04:38:25.618200+0000","last_peered":"2023-09-13T04:38:25.618200+0000","last_clean":"2023-09-13T04:38:25.618200+0000","last_became_active":"2023-09-13T04:37:58.590348+0000","last_became_peered":"2023-09-13T04:37:58.590348+0000","last_unstale":"2023-09-13T04:38:25.618200+0000","last_undegraded":"2023-09-13T04:38:25.618200+0000","last_fullsized":"2023-09-13T04:38:25.618200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:38:35.680905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594252+0000","last_change":"2023-09-13T04:37:59.237304+0000","last_active":"2023-09-13T04:38:23.594252+0000","last_peered":"2023-09-13T04:38:23.594252+0000","last_clean":"2023-09-13T04:38:23.594252+0000","last_became_active":"2023-09-13T04:37:59.237008+0000","last_became_peered":"2023-09-13T04:37:59.237008+0000","last_unstale":"2023-09-13T04:38:23.594252+0000","last_undegraded":"2023-09-13T04:38:23.594252+0000","last_fullsized":"2023-09-13T04:38:23.594252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:28:14.254219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590193+0000","last_change":"2023-09-13T04:38:03.606749+0000","last_active":"2023-09-13T04:38:23.590193+0000","last_peered":"2023-09-13T04:38:23.590193+0000","last_clean":"2023-09-13T04:38:23.590193+0000","last_became_active":"2023-09-13T04:38:03.606276+0000","last_became_peered":"2023-09-13T04:38:03.606276+0000","last_unstale":"2023-09-13T04:38:23.590193+0000","last_undegraded":"2023-09-13T04:38:23.590193+0000","last_fullsized":"2023-09-13T04:38:23.590193+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:30:53.572543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593692+0000","last_change":"2023-09-13T04:37:58.590937+0000","last_active":"2023-09-13T04:38:23.593692+0000","last_peered":"2023-09-13T04:38:23.593692+0000","last_clean":"2023-09-13T04:38:23.593692+0000","last_became_active":"2023-09-13T04:37:58.590606+0000","last_became_peered":"2023-09-13T04:37:58.590606+0000","last_unstale":"2023-09-13T04:38:23.593692+0000","last_undegraded":"2023-09-13T04:38:23.593692+0000","last_fullsized":"2023-09-13T04:38:23.593692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:30:03.354670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591224+0000","last_change":"2023-09-13T04:37:59.251899+0000","last_active":"2023-09-13T04:38:23.591224+0000","last_peered":"2023-09-13T04:38:23.591224+0000","last_clean":"2023-09-13T04:38:23.591224+0000","last_became_active":"2023-09-13T04:37:59.251557+0000","last_became_peered":"2023-09-13T04:37:59.251557+0000","last_unstale":"2023-09-13T04:38:23.591224+0000","last_undegraded":"2023-09-13T04:38:23.591224+0000","last_fullsized":"2023-09-13T04:38:23.591224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:57:45.241188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588439+0000","last_change":"2023-09-13T04:38:03.606076+0000","last_active":"2023-09-13T04:38:23.588439+0000","last_peered":"2023-09-13T04:38:23.588439+0000","last_clean":"2023-09-13T04:38:23.588439+0000","last_became_active":"2023-09-13T04:38:03.605556+0000","last_became_peered":"2023-09-13T04:38:03.605556+0000","last_unstale":"2023-09-13T04:38:23.588439+0000","last_undegraded":"2023-09-13T04:38:23.588439+0000","last_fullsized":"2023-09-13T04:38:23.588439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:03:32.344625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:39:19.695817+0000","last_change":"2023-09-13T04:37:58.588170+0000","last_active":"2023-09-13T04:39:19.695817+0000","last_peered":"2023-09-13T04:39:19.695817+0000","last_clean":"2023-09-13T04:39:19.695817+0000","last_became_active":"2023-09-13T04:37:58.587860+0000","last_became_peered":"2023-09-13T04:37:58.587860+0000","last_unstale":"2023-09-13T04:39:19.695817+0000","last_undegraded":"2023-09-13T04:39:19.695817+0000","last_fullsized":"2023-09-13T04:39:19.695817+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:38:40.048150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595472+0000","last_change":"2023-09-13T04:37:59.240248+0000","last_active":"2023-09-13T04:38:23.595472+0000","last_peered":"2023-09-13T04:38:23.595472+0000","last_clean":"2023-09-13T04:38:23.595472+0000","last_became_active":"2023-09-13T04:37:59.240008+0000","last_became_peered":"2023-09-13T04:37:59.240008+0000","last_unstale":"2023-09-13T04:38:23.595472+0000","last_undegraded":"2023-09-13T04:38:23.595472+0000","last_fullsized":"2023-09-13T04:38:23.595472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:15:13.275110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587821+0000","last_change":"2023-09-13T04:38:03.616607+0000","last_active":"2023-09-13T04:38:23.587821+0000","last_peered":"2023-09-13T04:38:23.587821+0000","last_clean":"2023-09-13T04:38:23.587821+0000","last_became_active":"2023-09-13T04:38:03.616209+0000","last_became_peered":"2023-09-13T04:38:03.616209+0000","last_unstale":"2023-09-13T04:38:23.587821+0000","last_undegraded":"2023-09-13T04:38:23.587821+0000","last_fullsized":"2023-09-13T04:38:23.587821+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:24:21.934053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.617180+0000","last_change":"2023-09-13T04:37:58.589104+0000","last_active":"2023-09-13T04:38:25.617180+0000","last_peered":"2023-09-13T04:38:25.617180+0000","last_clean":"2023-09-13T04:38:25.617180+0000","last_became_active":"2023-09-13T04:37:58.588780+0000","last_became_peered":"2023-09-13T04:37:58.588780+0000","last_unstale":"2023-09-13T04:38:25.617180+0000","last_undegraded":"2023-09-13T04:38:25.617180+0000","last_fullsized":"2023-09-13T04:38:25.617180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:01:32.839729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.600030+0000","last_change":"2023-09-13T04:38:24.598663+0000","last_active":"2023-09-13T04:38:24.600030+0000","last_peered":"2023-09-13T04:38:24.600030+0000","last_clean":"2023-09-13T04:38:24.600030+0000","last_became_active":"2023-09-13T04:38:24.598258+0000","last_became_peered":"2023-09-13T04:38:24.598258+0000","last_unstale":"2023-09-13T04:38:24.600030+0000","last_undegraded":"2023-09-13T04:38:24.600030+0000","last_fullsized":"2023-09-13T04:38:24.600030+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:50:28.157135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592633+0000","last_change":"2023-09-13T04:38:03.619930+0000","last_active":"2023-09-13T04:38:23.592633+0000","last_peered":"2023-09-13T04:38:23.592633+0000","last_clean":"2023-09-13T04:38:23.592633+0000","last_became_active":"2023-09-13T04:38:03.619560+0000","last_became_peered":"2023-09-13T04:38:03.619560+0000","last_unstale":"2023-09-13T04:38:23.592633+0000","last_undegraded":"2023-09-13T04:38:23.592633+0000","last_fullsized":"2023-09-13T04:38:23.592633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:48:06.867602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.619713+0000","last_change":"2023-09-13T04:37:58.590396+0000","last_active":"2023-09-13T04:38:25.619713+0000","last_peered":"2023-09-13T04:38:25.619713+0000","last_clean":"2023-09-13T04:38:25.619713+0000","last_became_active":"2023-09-13T04:37:58.589929+0000","last_became_peered":"2023-09-13T04:37:58.589929+0000","last_unstale":"2023-09-13T04:38:25.619713+0000","last_undegraded":"2023-09-13T04:38:25.619713+0000","last_fullsized":"2023-09-13T04:38:25.619713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:28:36.322096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597620+0000","last_change":"2023-09-13T04:37:59.246033+0000","last_active":"2023-09-13T04:38:23.597620+0000","last_peered":"2023-09-13T04:38:23.597620+0000","last_clean":"2023-09-13T04:38:23.597620+0000","last_became_active":"2023-09-13T04:37:59.245664+0000","last_became_peered":"2023-09-13T04:37:59.245664+0000","last_unstale":"2023-09-13T04:38:23.597620+0000","last_undegraded":"2023-09-13T04:38:23.597620+0000","last_fullsized":"2023-09-13T04:38:23.597620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:24:17.145055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592344+0000","last_change":"2023-09-13T04:38:03.608504+0000","last_active":"2023-09-13T04:38:23.592344+0000","last_peered":"2023-09-13T04:38:23.592344+0000","last_clean":"2023-09-13T04:38:23.592344+0000","last_became_active":"2023-09-13T04:38:03.608058+0000","last_became_peered":"2023-09-13T04:38:03.608058+0000","last_unstale":"2023-09-13T04:38:23.592344+0000","last_undegraded":"2023-09-13T04:38:23.592344+0000","last_fullsized":"2023-09-13T04:38:23.592344+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:42:43.485453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244691+0000","last_change":"2023-09-13T04:37:58.588208+0000","last_active":"2023-09-13T04:38:24.244691+0000","last_peered":"2023-09-13T04:38:24.244691+0000","last_clean":"2023-09-13T04:38:24.244691+0000","last_became_active":"2023-09-13T04:37:58.587840+0000","last_became_peered":"2023-09-13T04:37:58.587840+0000","last_unstale":"2023-09-13T04:38:24.244691+0000","last_undegraded":"2023-09-13T04:38:24.244691+0000","last_fullsized":"2023-09-13T04:38:24.244691+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:44:58.012233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587683+0000","last_change":"2023-09-13T04:37:59.245102+0000","last_active":"2023-09-13T04:38:23.587683+0000","last_peered":"2023-09-13T04:38:23.587683+0000","last_clean":"2023-09-13T04:38:23.587683+0000","last_became_active":"2023-09-13T04:37:59.244799+0000","last_became_peered":"2023-09-13T04:37:59.244799+0000","last_unstale":"2023-09-13T04:38:23.587683+0000","last_undegraded":"2023-09-13T04:38:23.587683+0000","last_fullsized":"2023-09-13T04:38:23.587683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:57:16.295623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.596645+0000","last_change":"2023-09-13T04:38:03.607021+0000","last_active":"2023-09-13T04:38:23.596645+0000","last_peered":"2023-09-13T04:38:23.596645+0000","last_clean":"2023-09-13T04:38:23.596645+0000","last_became_active":"2023-09-13T04:38:03.606611+0000","last_became_peered":"2023-09-13T04:38:03.606611+0000","last_unstale":"2023-09-13T04:38:23.596645+0000","last_undegraded":"2023-09-13T04:38:23.596645+0000","last_fullsized":"2023-09-13T04:38:23.596645+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:33:20.551479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":75,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:39:18.693681+0000","last_change":"2023-09-13T04:37:58.588731+0000","last_active":"2023-09-13T04:39:18.693681+0000","last_peered":"2023-09-13T04:39:18.693681+0000","last_clean":"2023-09-13T04:39:18.693681+0000","last_became_active":"2023-09-13T04:37:58.588398+0000","last_became_peered":"2023-09-13T04:37:58.588398+0000","last_unstale":"2023-09-13T04:39:18.693681+0000","last_undegraded":"2023-09-13T04:39:18.693681+0000","last_fullsized":"2023-09-13T04:39:18.693681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:13:10.724085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594790+0000","last_change":"2023-09-13T04:37:59.244757+0000","last_active":"2023-09-13T04:38:23.594790+0000","last_peered":"2023-09-13T04:38:23.594790+0000","last_clean":"2023-09-13T04:38:23.594790+0000","last_became_active":"2023-09-13T04:37:59.244423+0000","last_became_peered":"2023-09-13T04:37:59.244423+0000","last_unstale":"2023-09-13T04:38:23.594790+0000","last_undegraded":"2023-09-13T04:38:23.594790+0000","last_fullsized":"2023-09-13T04:38:23.594790+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:14:37.973058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587303+0000","last_change":"2023-09-13T04:38:03.618358+0000","last_active":"2023-09-13T04:38:23.587303+0000","last_peered":"2023-09-13T04:38:23.587303+0000","last_clean":"2023-09-13T04:38:23.587303+0000","last_became_active":"2023-09-13T04:38:03.617291+0000","last_became_peered":"2023-09-13T04:38:03.617291+0000","last_unstale":"2023-09-13T04:38:23.587303+0000","last_undegraded":"2023-09-13T04:38:23.587303+0000","last_fullsized":"2023-09-13T04:38:23.587303+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:33:13.955353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244519+0000","last_change":"2023-09-13T04:37:58.590978+0000","last_active":"2023-09-13T04:38:24.244519+0000","last_peered":"2023-09-13T04:38:24.244519+0000","last_clean":"2023-09-13T04:38:24.244519+0000","last_became_active":"2023-09-13T04:37:58.590673+0000","last_became_peered":"2023-09-13T04:37:58.590673+0000","last_unstale":"2023-09-13T04:38:24.244519+0000","last_undegraded":"2023-09-13T04:38:24.244519+0000","last_fullsized":"2023-09-13T04:38:24.244519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:10:32.570029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592737+0000","last_change":"2023-09-13T04:37:59.251283+0000","last_active":"2023-09-13T04:38:23.592737+0000","last_peered":"2023-09-13T04:38:23.592737+0000","last_clean":"2023-09-13T04:38:23.592737+0000","last_became_active":"2023-09-13T04:37:59.250119+0000","last_became_peered":"2023-09-13T04:37:59.250119+0000","last_unstale":"2023-09-13T04:38:23.592737+0000","last_undegraded":"2023-09-13T04:38:23.592737+0000","last_fullsized":"2023-09-13T04:38:23.592737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:48:54.389970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.596107+0000","last_change":"2023-09-13T04:38:24.596107+0000","last_active":"2023-09-13T04:38:24.596107+0000","last_peered":"2023-09-13T04:38:24.596107+0000","last_clean":"2023-09-13T04:38:24.596107+0000","last_became_active":"2023-09-13T04:38:24.595653+0000","last_became_peered":"2023-09-13T04:38:24.595653+0000","last_unstale":"2023-09-13T04:38:24.596107+0000","last_undegraded":"2023-09-13T04:38:24.596107+0000","last_fullsized":"2023-09-13T04:38:24.596107+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:02:53.708658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.589390+0000","last_change":"2023-09-13T04:37:58.584909+0000","last_active":"2023-09-13T04:38:24.589390+0000","last_peered":"2023-09-13T04:38:24.589390+0000","last_clean":"2023-09-13T04:38:24.589390+0000","last_became_active":"2023-09-13T04:37:58.584630+0000","last_became_peered":"2023-09-13T04:37:58.584630+0000","last_unstale":"2023-09-13T04:38:24.589390+0000","last_undegraded":"2023-09-13T04:38:24.589390+0000","last_fullsized":"2023-09-13T04:38:24.589390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:57:56.567499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594349+0000","last_change":"2023-09-13T04:37:59.254448+0000","last_active":"2023-09-13T04:38:23.594349+0000","last_peered":"2023-09-13T04:38:23.594349+0000","last_clean":"2023-09-13T04:38:23.594349+0000","last_became_active":"2023-09-13T04:37:59.254138+0000","last_became_peered":"2023-09-13T04:37:59.254138+0000","last_unstale":"2023-09-13T04:38:23.594349+0000","last_undegraded":"2023-09-13T04:38:23.594349+0000","last_fullsized":"2023-09-13T04:38:23.594349+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:36:14.939881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588357+0000","last_change":"2023-09-13T04:38:03.613746+0000","last_active":"2023-09-13T04:38:23.588357+0000","last_peered":"2023-09-13T04:38:23.588357+0000","last_clean":"2023-09-13T04:38:23.588357+0000","last_became_active":"2023-09-13T04:38:03.613354+0000","last_became_peered":"2023-09-13T04:38:03.613354+0000","last_unstale":"2023-09-13T04:38:23.588357+0000","last_undegraded":"2023-09-13T04:38:23.588357+0000","last_fullsized":"2023-09-13T04:38:23.588357+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:41:01.294912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":140,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:39:19.695138+0000","last_change":"2023-09-13T04:37:58.591196+0000","last_active":"2023-09-13T04:39:19.695138+0000","last_peered":"2023-09-13T04:39:19.695138+0000","last_clean":"2023-09-13T04:39:19.695138+0000","last_became_active":"2023-09-13T04:37:58.590883+0000","last_became_peered":"2023-09-13T04:37:58.590883+0000","last_unstale":"2023-09-13T04:39:19.695138+0000","last_undegraded":"2023-09-13T04:39:19.695138+0000","last_fullsized":"2023-09-13T04:39:19.695138+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:46:58.530270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":110,"num_read_kb":59,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595499+0000","last_change":"2023-09-13T04:37:59.243483+0000","last_active":"2023-09-13T04:38:23.595499+0000","last_peered":"2023-09-13T04:38:23.595499+0000","last_clean":"2023-09-13T04:38:23.595499+0000","last_became_active":"2023-09-13T04:37:59.243088+0000","last_became_peered":"2023-09-13T04:37:59.243088+0000","last_unstale":"2023-09-13T04:38:23.595499+0000","last_undegraded":"2023-09-13T04:38:23.595499+0000","last_fullsized":"2023-09-13T04:38:23.595499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:56:14.093819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.591697+0000","last_change":"2023-09-13T04:38:03.608820+0000","last_active":"2023-09-13T04:38:23.591697+0000","last_peered":"2023-09-13T04:38:23.591697+0000","last_clean":"2023-09-13T04:38:23.591697+0000","last_became_active":"2023-09-13T04:38:03.608200+0000","last_became_peered":"2023-09-13T04:38:03.608200+0000","last_unstale":"2023-09-13T04:38:23.591697+0000","last_undegraded":"2023-09-13T04:38:23.591697+0000","last_fullsized":"2023-09-13T04:38:23.591697+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:54:52.416501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.620980+0000","last_change":"2023-09-13T04:38:24.606820+0000","last_active":"2023-09-13T04:38:25.620980+0000","last_peered":"2023-09-13T04:38:25.620980+0000","last_clean":"2023-09-13T04:38:25.620980+0000","last_became_active":"2023-09-13T04:38:24.599059+0000","last_became_peered":"2023-09-13T04:38:24.599059+0000","last_unstale":"2023-09-13T04:38:25.620980+0000","last_undegraded":"2023-09-13T04:38:25.620980+0000","last_fullsized":"2023-09-13T04:38:25.620980+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:08:12.634357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597568+0000","last_change":"2023-09-13T04:37:59.250666+0000","last_active":"2023-09-13T04:38:23.597568+0000","last_peered":"2023-09-13T04:38:23.597568+0000","last_clean":"2023-09-13T04:38:23.597568+0000","last_became_active":"2023-09-13T04:37:59.250240+0000","last_became_peered":"2023-09-13T04:37:59.250240+0000","last_unstale":"2023-09-13T04:38:23.597568+0000","last_undegraded":"2023-09-13T04:38:23.597568+0000","last_fullsized":"2023-09-13T04:38:23.597568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:44:31.324222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592271+0000","last_change":"2023-09-13T04:38:03.616296+0000","last_active":"2023-09-13T04:38:23.592271+0000","last_peered":"2023-09-13T04:38:23.592271+0000","last_clean":"2023-09-13T04:38:23.592271+0000","last_became_active":"2023-09-13T04:38:03.615934+0000","last_became_peered":"2023-09-13T04:38:03.615934+0000","last_unstale":"2023-09-13T04:38:23.592271+0000","last_undegraded":"2023-09-13T04:38:23.592271+0000","last_fullsized":"2023-09-13T04:38:23.592271+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:51:18.994472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244805+0000","last_change":"2023-09-13T04:37:58.594782+0000","last_active":"2023-09-13T04:38:24.244805+0000","last_peered":"2023-09-13T04:38:24.244805+0000","last_clean":"2023-09-13T04:38:24.244805+0000","last_became_active":"2023-09-13T04:37:58.594480+0000","last_became_peered":"2023-09-13T04:37:58.594480+0000","last_unstale":"2023-09-13T04:38:24.244805+0000","last_undegraded":"2023-09-13T04:38:24.244805+0000","last_fullsized":"2023-09-13T04:38:24.244805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:40:19.635672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588001+0000","last_change":"2023-09-13T04:37:59.252575+0000","last_active":"2023-09-13T04:38:23.588001+0000","last_peered":"2023-09-13T04:38:23.588001+0000","last_clean":"2023-09-13T04:38:23.588001+0000","last_became_active":"2023-09-13T04:37:59.252268+0000","last_became_peered":"2023-09-13T04:37:59.252268+0000","last_unstale":"2023-09-13T04:38:23.588001+0000","last_undegraded":"2023-09-13T04:38:23.588001+0000","last_fullsized":"2023-09-13T04:38:23.588001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:27:10.814930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587908+0000","last_change":"2023-09-13T04:38:03.618835+0000","last_active":"2023-09-13T04:38:23.587908+0000","last_peered":"2023-09-13T04:38:23.587908+0000","last_clean":"2023-09-13T04:38:23.587908+0000","last_became_active":"2023-09-13T04:38:03.618226+0000","last_became_peered":"2023-09-13T04:38:03.618226+0000","last_unstale":"2023-09-13T04:38:23.587908+0000","last_undegraded":"2023-09-13T04:38:23.587908+0000","last_fullsized":"2023-09-13T04:38:23.587908+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:01:28.157490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":77,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:39:18.695465+0000","last_change":"2023-09-13T04:37:58.588545+0000","last_active":"2023-09-13T04:39:18.695465+0000","last_peered":"2023-09-13T04:39:18.695465+0000","last_clean":"2023-09-13T04:39:18.695465+0000","last_became_active":"2023-09-13T04:37:58.587995+0000","last_became_peered":"2023-09-13T04:37:58.587995+0000","last_unstale":"2023-09-13T04:39:18.695465+0000","last_undegraded":"2023-09-13T04:39:18.695465+0000","last_fullsized":"2023-09-13T04:39:18.695465+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:40:08.797724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587793+0000","last_change":"2023-09-13T04:37:59.249100+0000","last_active":"2023-09-13T04:38:23.587793+0000","last_peered":"2023-09-13T04:38:23.587793+0000","last_clean":"2023-09-13T04:38:23.587793+0000","last_became_active":"2023-09-13T04:37:59.248771+0000","last_became_peered":"2023-09-13T04:37:59.248771+0000","last_unstale":"2023-09-13T04:38:23.587793+0000","last_undegraded":"2023-09-13T04:38:23.587793+0000","last_fullsized":"2023-09-13T04:38:23.587793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:27:37.964595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597364+0000","last_change":"2023-09-13T04:38:03.616988+0000","last_active":"2023-09-13T04:38:23.597364+0000","last_peered":"2023-09-13T04:38:23.597364+0000","last_clean":"2023-09-13T04:38:23.597364+0000","last_became_active":"2023-09-13T04:38:03.616524+0000","last_became_peered":"2023-09-13T04:38:03.616524+0000","last_unstale":"2023-09-13T04:38:23.597364+0000","last_undegraded":"2023-09-13T04:38:23.597364+0000","last_fullsized":"2023-09-13T04:38:23.597364+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:42:41.431115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.622279+0000","last_change":"2023-09-13T04:38:24.613384+0000","last_active":"2023-09-13T04:38:25.622279+0000","last_peered":"2023-09-13T04:38:25.622279+0000","last_clean":"2023-09-13T04:38:25.622279+0000","last_became_active":"2023-09-13T04:38:24.599349+0000","last_became_peered":"2023-09-13T04:38:24.599349+0000","last_unstale":"2023-09-13T04:38:25.622279+0000","last_undegraded":"2023-09-13T04:38:25.622279+0000","last_fullsized":"2023-09-13T04:38:25.622279+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:23:23.479715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593329+0000","last_change":"2023-09-13T04:37:59.252237+0000","last_active":"2023-09-13T04:38:23.593329+0000","last_peered":"2023-09-13T04:38:23.593329+0000","last_clean":"2023-09-13T04:38:23.593329+0000","last_became_active":"2023-09-13T04:37:59.251718+0000","last_became_peered":"2023-09-13T04:37:59.251718+0000","last_unstale":"2023-09-13T04:38:23.593329+0000","last_undegraded":"2023-09-13T04:38:23.593329+0000","last_fullsized":"2023-09-13T04:38:23.593329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:45:48.405226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593276+0000","last_change":"2023-09-13T04:38:03.607096+0000","last_active":"2023-09-13T04:38:23.593276+0000","last_peered":"2023-09-13T04:38:23.593276+0000","last_clean":"2023-09-13T04:38:23.593276+0000","last_became_active":"2023-09-13T04:38:03.606656+0000","last_became_peered":"2023-09-13T04:38:03.606656+0000","last_unstale":"2023-09-13T04:38:23.593276+0000","last_undegraded":"2023-09-13T04:38:23.593276+0000","last_fullsized":"2023-09-13T04:38:23.593276+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:59:26.140360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:26.617580+0000","last_change":"2023-09-13T04:37:58.591047+0000","last_active":"2023-09-13T04:38:26.617580+0000","last_peered":"2023-09-13T04:38:26.617580+0000","last_clean":"2023-09-13T04:38:26.617580+0000","last_became_active":"2023-09-13T04:37:58.590740+0000","last_became_peered":"2023-09-13T04:37:58.590740+0000","last_unstale":"2023-09-13T04:38:26.617580+0000","last_undegraded":"2023-09-13T04:38:26.617580+0000","last_fullsized":"2023-09-13T04:38:26.617580+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:14:42.280416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592406+0000","last_change":"2023-09-13T04:37:59.249362+0000","last_active":"2023-09-13T04:38:23.592406+0000","last_peered":"2023-09-13T04:38:23.592406+0000","last_clean":"2023-09-13T04:38:23.592406+0000","last_became_active":"2023-09-13T04:37:59.249041+0000","last_became_peered":"2023-09-13T04:37:59.249041+0000","last_unstale":"2023-09-13T04:38:23.592406+0000","last_undegraded":"2023-09-13T04:38:23.592406+0000","last_fullsized":"2023-09-13T04:38:23.592406+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:06:33.822157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.598370+0000","last_change":"2023-09-13T04:38:24.598370+0000","last_active":"2023-09-13T04:38:24.598370+0000","last_peered":"2023-09-13T04:38:24.598370+0000","last_clean":"2023-09-13T04:38:24.598370+0000","last_became_active":"2023-09-13T04:38:24.597931+0000","last_became_peered":"2023-09-13T04:38:24.597931+0000","last_unstale":"2023-09-13T04:38:24.598370+0000","last_undegraded":"2023-09-13T04:38:24.598370+0000","last_fullsized":"2023-09-13T04:38:24.598370+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:14:28.911329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588100+0000","last_change":"2023-09-13T04:37:58.592496+0000","last_active":"2023-09-13T04:38:23.588100+0000","last_peered":"2023-09-13T04:38:23.588100+0000","last_clean":"2023-09-13T04:38:23.588100+0000","last_became_active":"2023-09-13T04:37:58.592106+0000","last_became_peered":"2023-09-13T04:37:58.592106+0000","last_unstale":"2023-09-13T04:38:23.588100+0000","last_undegraded":"2023-09-13T04:38:23.588100+0000","last_fullsized":"2023-09-13T04:38:23.588100+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:28:55.618563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594212+0000","last_change":"2023-09-13T04:37:59.250019+0000","last_active":"2023-09-13T04:38:23.594212+0000","last_peered":"2023-09-13T04:38:23.594212+0000","last_clean":"2023-09-13T04:38:23.594212+0000","last_became_active":"2023-09-13T04:37:59.249278+0000","last_became_peered":"2023-09-13T04:37:59.249278+0000","last_unstale":"2023-09-13T04:38:23.594212+0000","last_undegraded":"2023-09-13T04:38:23.594212+0000","last_fullsized":"2023-09-13T04:38:23.594212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:15:31.217236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597640+0000","last_change":"2023-09-13T04:38:03.614542+0000","last_active":"2023-09-13T04:38:23.597640+0000","last_peered":"2023-09-13T04:38:23.597640+0000","last_clean":"2023-09-13T04:38:23.597640+0000","last_became_active":"2023-09-13T04:38:03.614103+0000","last_became_peered":"2023-09-13T04:38:03.614103+0000","last_unstale":"2023-09-13T04:38:23.597640+0000","last_undegraded":"2023-09-13T04:38:23.597640+0000","last_fullsized":"2023-09-13T04:38:23.597640+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:06:48.426843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.592720+0000","last_change":"2023-09-13T04:37:58.591864+0000","last_active":"2023-09-13T04:38:24.592720+0000","last_peered":"2023-09-13T04:38:24.592720+0000","last_clean":"2023-09-13T04:38:24.592720+0000","last_became_active":"2023-09-13T04:37:58.591562+0000","last_became_peered":"2023-09-13T04:37:58.591562+0000","last_unstale":"2023-09-13T04:38:24.592720+0000","last_undegraded":"2023-09-13T04:38:24.592720+0000","last_fullsized":"2023-09-13T04:38:24.592720+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:31:27.280489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594567+0000","last_change":"2023-09-13T04:37:59.252600+0000","last_active":"2023-09-13T04:38:23.594567+0000","last_peered":"2023-09-13T04:38:23.594567+0000","last_clean":"2023-09-13T04:38:23.594567+0000","last_became_active":"2023-09-13T04:37:59.252306+0000","last_became_peered":"2023-09-13T04:37:59.252306+0000","last_unstale":"2023-09-13T04:38:23.594567+0000","last_undegraded":"2023-09-13T04:38:23.594567+0000","last_fullsized":"2023-09-13T04:38:23.594567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:18:55.686847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.596920+0000","last_change":"2023-09-13T04:38:03.742502+0000","last_active":"2023-09-13T04:38:23.596920+0000","last_peered":"2023-09-13T04:38:23.596920+0000","last_clean":"2023-09-13T04:38:23.596920+0000","last_became_active":"2023-09-13T04:38:03.742054+0000","last_became_peered":"2023-09-13T04:38:03.742054+0000","last_unstale":"2023-09-13T04:38:23.596920+0000","last_undegraded":"2023-09-13T04:38:23.596920+0000","last_fullsized":"2023-09-13T04:38:23.596920+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:55:54.601145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593527+0000","last_change":"2023-09-13T04:37:59.245644+0000","last_active":"2023-09-13T04:38:23.593527+0000","last_peered":"2023-09-13T04:38:23.593527+0000","last_clean":"2023-09-13T04:38:23.593527+0000","last_became_active":"2023-09-13T04:37:59.245284+0000","last_became_peered":"2023-09-13T04:37:59.245284+0000","last_unstale":"2023-09-13T04:38:23.593527+0000","last_undegraded":"2023-09-13T04:38:23.593527+0000","last_fullsized":"2023-09-13T04:38:23.593527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:19:04.002414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":78,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:39:19.696175+0000","last_change":"2023-09-13T04:37:58.590249+0000","last_active":"2023-09-13T04:39:19.696175+0000","last_peered":"2023-09-13T04:39:19.696175+0000","last_clean":"2023-09-13T04:39:19.696175+0000","last_became_active":"2023-09-13T04:37:58.589521+0000","last_became_peered":"2023-09-13T04:37:58.589521+0000","last_unstale":"2023-09-13T04:39:19.696175+0000","last_undegraded":"2023-09-13T04:39:19.696175+0000","last_fullsized":"2023-09-13T04:39:19.696175+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:11:57.542149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.598613+0000","last_change":"2023-09-13T04:38:24.598613+0000","last_active":"2023-09-13T04:38:24.598613+0000","last_peered":"2023-09-13T04:38:24.598613+0000","last_clean":"2023-09-13T04:38:24.598613+0000","last_became_active":"2023-09-13T04:38:24.598150+0000","last_became_peered":"2023-09-13T04:38:24.598150+0000","last_unstale":"2023-09-13T04:38:24.598613+0000","last_undegraded":"2023-09-13T04:38:24.598613+0000","last_fullsized":"2023-09-13T04:38:24.598613+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:02:29.894446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595221+0000","last_change":"2023-09-13T04:37:59.243794+0000","last_active":"2023-09-13T04:38:23.595221+0000","last_peered":"2023-09-13T04:38:23.595221+0000","last_clean":"2023-09-13T04:38:23.595221+0000","last_became_active":"2023-09-13T04:37:59.243368+0000","last_became_peered":"2023-09-13T04:37:59.243368+0000","last_unstale":"2023-09-13T04:38:23.595221+0000","last_undegraded":"2023-09-13T04:38:23.595221+0000","last_fullsized":"2023-09-13T04:38:23.595221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:06:45.318737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:26.615677+0000","last_change":"2023-09-13T04:37:58.589195+0000","last_active":"2023-09-13T04:38:26.615677+0000","last_peered":"2023-09-13T04:38:26.615677+0000","last_clean":"2023-09-13T04:38:26.615677+0000","last_became_active":"2023-09-13T04:37:58.588905+0000","last_became_peered":"2023-09-13T04:37:58.588905+0000","last_unstale":"2023-09-13T04:38:26.615677+0000","last_undegraded":"2023-09-13T04:38:26.615677+0000","last_fullsized":"2023-09-13T04:38:26.615677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:13:15.978891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":326,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587638+0000","last_change":"2023-09-13T04:36:52.227264+0000","last_active":"2023-09-13T04:38:23.587638+0000","last_peered":"2023-09-13T04:38:23.587638+0000","last_clean":"2023-09-13T04:38:23.587638+0000","last_became_active":"2023-09-13T04:36:52.226773+0000","last_became_peered":"2023-09-13T04:36:52.226773+0000","last_unstale":"2023-09-13T04:38:23.587638+0000","last_undegraded":"2023-09-13T04:38:23.587638+0000","last_fullsized":"2023-09-13T04:38:23.587638+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:33:42.026499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:33:42.026499+0000","last_clean_scrub_stamp":"2023-09-13T04:33:42.026499+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:28:55.806587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592890+0000","last_change":"2023-09-13T04:38:03.742790+0000","last_active":"2023-09-13T04:38:23.592890+0000","last_peered":"2023-09-13T04:38:23.592890+0000","last_clean":"2023-09-13T04:38:23.592890+0000","last_became_active":"2023-09-13T04:38:03.742338+0000","last_became_peered":"2023-09-13T04:38:03.742338+0000","last_unstale":"2023-09-13T04:38:23.592890+0000","last_undegraded":"2023-09-13T04:38:23.592890+0000","last_fullsized":"2023-09-13T04:38:23.592890+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:01:38.910623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597412+0000","last_change":"2023-09-13T04:37:59.252669+0000","last_active":"2023-09-13T04:38:23.597412+0000","last_peered":"2023-09-13T04:38:23.597412+0000","last_clean":"2023-09-13T04:38:23.597412+0000","last_became_active":"2023-09-13T04:37:59.252223+0000","last_became_peered":"2023-09-13T04:37:59.252223+0000","last_unstale":"2023-09-13T04:38:23.597412+0000","last_undegraded":"2023-09-13T04:38:23.597412+0000","last_fullsized":"2023-09-13T04:38:23.597412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:57:21.090083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.615697+0000","last_change":"2023-09-13T04:37:58.590540+0000","last_active":"2023-09-13T04:38:25.615697+0000","last_peered":"2023-09-13T04:38:25.615697+0000","last_clean":"2023-09-13T04:38:25.615697+0000","last_became_active":"2023-09-13T04:37:58.590061+0000","last_became_peered":"2023-09-13T04:37:58.590061+0000","last_unstale":"2023-09-13T04:38:25.615697+0000","last_undegraded":"2023-09-13T04:38:25.615697+0000","last_fullsized":"2023-09-13T04:38:25.615697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:57:17.133461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592719+0000","last_change":"2023-09-13T04:38:03.614615+0000","last_active":"2023-09-13T04:38:23.592719+0000","last_peered":"2023-09-13T04:38:23.592719+0000","last_clean":"2023-09-13T04:38:23.592719+0000","last_became_active":"2023-09-13T04:38:03.614086+0000","last_became_peered":"2023-09-13T04:38:03.614086+0000","last_unstale":"2023-09-13T04:38:23.592719+0000","last_undegraded":"2023-09-13T04:38:23.592719+0000","last_fullsized":"2023-09-13T04:38:23.592719+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:25:35.525513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.595581+0000","last_change":"2023-09-13T04:37:58.592096+0000","last_active":"2023-09-13T04:38:24.595581+0000","last_peered":"2023-09-13T04:38:24.595581+0000","last_clean":"2023-09-13T04:38:24.595581+0000","last_became_active":"2023-09-13T04:37:58.591872+0000","last_became_peered":"2023-09-13T04:37:58.591872+0000","last_unstale":"2023-09-13T04:38:24.595581+0000","last_undegraded":"2023-09-13T04:38:24.595581+0000","last_fullsized":"2023-09-13T04:38:24.595581+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:05:23.936105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597816+0000","last_change":"2023-09-13T04:37:59.240759+0000","last_active":"2023-09-13T04:38:23.597816+0000","last_peered":"2023-09-13T04:38:23.597816+0000","last_clean":"2023-09-13T04:38:23.597816+0000","last_became_active":"2023-09-13T04:37:59.240414+0000","last_became_peered":"2023-09-13T04:37:59.240414+0000","last_unstale":"2023-09-13T04:38:23.597816+0000","last_undegraded":"2023-09-13T04:38:23.597816+0000","last_fullsized":"2023-09-13T04:38:23.597816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:37:20.301938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597533+0000","last_change":"2023-09-13T04:38:03.742519+0000","last_active":"2023-09-13T04:38:23.597533+0000","last_peered":"2023-09-13T04:38:23.597533+0000","last_clean":"2023-09-13T04:38:23.597533+0000","last_became_active":"2023-09-13T04:38:03.742075+0000","last_became_peered":"2023-09-13T04:38:03.742075+0000","last_unstale":"2023-09-13T04:38:23.597533+0000","last_undegraded":"2023-09-13T04:38:23.597533+0000","last_fullsized":"2023-09-13T04:38:23.597533+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:14:33.155837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.619899+0000","last_change":"2023-09-13T04:37:58.593891+0000","last_active":"2023-09-13T04:38:25.619899+0000","last_peered":"2023-09-13T04:38:25.619899+0000","last_clean":"2023-09-13T04:38:25.619899+0000","last_became_active":"2023-09-13T04:37:58.593497+0000","last_became_peered":"2023-09-13T04:37:58.593497+0000","last_unstale":"2023-09-13T04:38:25.619899+0000","last_undegraded":"2023-09-13T04:38:25.619899+0000","last_fullsized":"2023-09-13T04:38:25.619899+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:25:53.652601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595975+0000","last_change":"2023-09-13T04:37:59.247151+0000","last_active":"2023-09-13T04:38:23.595975+0000","last_peered":"2023-09-13T04:38:23.595975+0000","last_clean":"2023-09-13T04:38:23.595975+0000","last_became_active":"2023-09-13T04:37:59.246853+0000","last_became_peered":"2023-09-13T04:37:59.246853+0000","last_unstale":"2023-09-13T04:38:23.595975+0000","last_undegraded":"2023-09-13T04:38:23.595975+0000","last_fullsized":"2023-09-13T04:38:23.595975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:15:22.659365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587493+0000","last_change":"2023-09-13T04:38:03.606594+0000","last_active":"2023-09-13T04:38:23.587493+0000","last_peered":"2023-09-13T04:38:23.587493+0000","last_clean":"2023-09-13T04:38:23.587493+0000","last_became_active":"2023-09-13T04:38:03.606058+0000","last_became_peered":"2023-09-13T04:38:03.606058+0000","last_unstale":"2023-09-13T04:38:23.587493+0000","last_undegraded":"2023-09-13T04:38:23.587493+0000","last_fullsized":"2023-09-13T04:38:23.587493+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:57:39.237520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.246057+0000","last_change":"2023-09-13T04:37:58.588362+0000","last_active":"2023-09-13T04:38:24.246057+0000","last_peered":"2023-09-13T04:38:24.246057+0000","last_clean":"2023-09-13T04:38:24.246057+0000","last_became_active":"2023-09-13T04:37:58.588065+0000","last_became_peered":"2023-09-13T04:37:58.588065+0000","last_unstale":"2023-09-13T04:38:24.246057+0000","last_undegraded":"2023-09-13T04:38:24.246057+0000","last_fullsized":"2023-09-13T04:38:24.246057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:32:04.255654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592468+0000","last_change":"2023-09-13T04:37:59.251883+0000","last_active":"2023-09-13T04:38:23.592468+0000","last_peered":"2023-09-13T04:38:23.592468+0000","last_clean":"2023-09-13T04:38:23.592468+0000","last_became_active":"2023-09-13T04:37:59.251522+0000","last_became_peered":"2023-09-13T04:37:59.251522+0000","last_unstale":"2023-09-13T04:38:23.592468+0000","last_undegraded":"2023-09-13T04:38:23.592468+0000","last_fullsized":"2023-09-13T04:38:23.592468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:05:06.420799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590729+0000","last_change":"2023-09-13T04:38:03.742726+0000","last_active":"2023-09-13T04:38:23.590729+0000","last_peered":"2023-09-13T04:38:23.590729+0000","last_clean":"2023-09-13T04:38:23.590729+0000","last_became_active":"2023-09-13T04:38:03.742293+0000","last_became_peered":"2023-09-13T04:38:03.742293+0000","last_unstale":"2023-09-13T04:38:23.590729+0000","last_undegraded":"2023-09-13T04:38:23.590729+0000","last_fullsized":"2023-09-13T04:38:23.590729+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:52:15.089764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.243314+0000","last_change":"2023-09-13T04:37:58.584923+0000","last_active":"2023-09-13T04:38:24.243314+0000","last_peered":"2023-09-13T04:38:24.243314+0000","last_clean":"2023-09-13T04:38:24.243314+0000","last_became_active":"2023-09-13T04:37:58.584320+0000","last_became_peered":"2023-09-13T04:37:58.584320+0000","last_unstale":"2023-09-13T04:38:24.243314+0000","last_undegraded":"2023-09-13T04:38:24.243314+0000","last_fullsized":"2023-09-13T04:38:24.243314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:16:53.670534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592586+0000","last_change":"2023-09-13T04:37:59.243029+0000","last_active":"2023-09-13T04:38:23.592586+0000","last_peered":"2023-09-13T04:38:23.592586+0000","last_clean":"2023-09-13T04:38:23.592586+0000","last_became_active":"2023-09-13T04:37:59.242733+0000","last_became_peered":"2023-09-13T04:37:59.242733+0000","last_unstale":"2023-09-13T04:38:23.592586+0000","last_undegraded":"2023-09-13T04:38:23.592586+0000","last_fullsized":"2023-09-13T04:38:23.592586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:48:21.430026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588037+0000","last_change":"2023-09-13T04:38:03.616735+0000","last_active":"2023-09-13T04:38:23.588037+0000","last_peered":"2023-09-13T04:38:23.588037+0000","last_clean":"2023-09-13T04:38:23.588037+0000","last_became_active":"2023-09-13T04:38:03.616275+0000","last_became_peered":"2023-09-13T04:38:03.616275+0000","last_unstale":"2023-09-13T04:38:23.588037+0000","last_undegraded":"2023-09-13T04:38:23.588037+0000","last_fullsized":"2023-09-13T04:38:23.588037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:12:27.267918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.244170+0000","last_change":"2023-09-13T04:37:58.595040+0000","last_active":"2023-09-13T04:38:24.244170+0000","last_peered":"2023-09-13T04:38:24.244170+0000","last_clean":"2023-09-13T04:38:24.244170+0000","last_became_active":"2023-09-13T04:37:58.591403+0000","last_became_peered":"2023-09-13T04:37:58.591403+0000","last_unstale":"2023-09-13T04:38:24.244170+0000","last_undegraded":"2023-09-13T04:38:24.244170+0000","last_fullsized":"2023-09-13T04:38:24.244170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:42:06.561066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592175+0000","last_change":"2023-09-13T04:37:59.248837+0000","last_active":"2023-09-13T04:38:23.592175+0000","last_peered":"2023-09-13T04:38:23.592175+0000","last_clean":"2023-09-13T04:38:23.592175+0000","last_became_active":"2023-09-13T04:37:59.248573+0000","last_became_peered":"2023-09-13T04:37:59.248573+0000","last_unstale":"2023-09-13T04:38:23.592175+0000","last_undegraded":"2023-09-13T04:38:23.592175+0000","last_fullsized":"2023-09-13T04:38:23.592175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:56:22.980484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590405+0000","last_change":"2023-09-13T04:38:03.616370+0000","last_active":"2023-09-13T04:38:23.590405+0000","last_peered":"2023-09-13T04:38:23.590405+0000","last_clean":"2023-09-13T04:38:23.590405+0000","last_became_active":"2023-09-13T04:38:03.615996+0000","last_became_peered":"2023-09-13T04:38:03.615996+0000","last_unstale":"2023-09-13T04:38:23.590405+0000","last_undegraded":"2023-09-13T04:38:23.590405+0000","last_fullsized":"2023-09-13T04:38:23.590405+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:56:59.924205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.590955+0000","last_change":"2023-09-13T04:37:58.597902+0000","last_active":"2023-09-13T04:38:24.590955+0000","last_peered":"2023-09-13T04:38:24.590955+0000","last_clean":"2023-09-13T04:38:24.590955+0000","last_became_active":"2023-09-13T04:37:58.597578+0000","last_became_peered":"2023-09-13T04:37:58.597578+0000","last_unstale":"2023-09-13T04:38:24.590955+0000","last_undegraded":"2023-09-13T04:38:24.590955+0000","last_fullsized":"2023-09-13T04:38:24.590955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:10:09.931949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597024+0000","last_change":"2023-09-13T04:37:59.248924+0000","last_active":"2023-09-13T04:38:23.597024+0000","last_peered":"2023-09-13T04:38:23.597024+0000","last_clean":"2023-09-13T04:38:23.597024+0000","last_became_active":"2023-09-13T04:37:59.248483+0000","last_became_peered":"2023-09-13T04:37:59.248483+0000","last_unstale":"2023-09-13T04:38:23.597024+0000","last_undegraded":"2023-09-13T04:38:23.597024+0000","last_fullsized":"2023-09-13T04:38:23.597024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:49:56.078544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590322+0000","last_change":"2023-09-13T04:38:03.742260+0000","last_active":"2023-09-13T04:38:23.590322+0000","last_peered":"2023-09-13T04:38:23.590322+0000","last_clean":"2023-09-13T04:38:23.590322+0000","last_became_active":"2023-09-13T04:38:03.741893+0000","last_became_peered":"2023-09-13T04:38:03.741893+0000","last_unstale":"2023-09-13T04:38:23.590322+0000","last_undegraded":"2023-09-13T04:38:23.590322+0000","last_fullsized":"2023-09-13T04:38:23.590322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:59:00.653545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.594861+0000","last_change":"2023-09-13T04:37:58.586898+0000","last_active":"2023-09-13T04:38:24.594861+0000","last_peered":"2023-09-13T04:38:24.594861+0000","last_clean":"2023-09-13T04:38:24.594861+0000","last_became_active":"2023-09-13T04:37:58.586594+0000","last_became_peered":"2023-09-13T04:37:58.586594+0000","last_unstale":"2023-09-13T04:38:24.594861+0000","last_undegraded":"2023-09-13T04:38:24.594861+0000","last_fullsized":"2023-09-13T04:38:24.594861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:29:43.718522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587969+0000","last_change":"2023-09-13T04:37:59.244749+0000","last_active":"2023-09-13T04:38:23.587969+0000","last_peered":"2023-09-13T04:38:23.587969+0000","last_clean":"2023-09-13T04:38:23.587969+0000","last_became_active":"2023-09-13T04:37:59.244391+0000","last_became_peered":"2023-09-13T04:37:59.244391+0000","last_unstale":"2023-09-13T04:38:23.587969+0000","last_undegraded":"2023-09-13T04:38:23.587969+0000","last_fullsized":"2023-09-13T04:38:23.587969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:33:02.195729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592524+0000","last_change":"2023-09-13T04:38:03.609766+0000","last_active":"2023-09-13T04:38:23.592524+0000","last_peered":"2023-09-13T04:38:23.592524+0000","last_clean":"2023-09-13T04:38:23.592524+0000","last_became_active":"2023-09-13T04:38:03.609432+0000","last_became_peered":"2023-09-13T04:38:03.609432+0000","last_unstale":"2023-09-13T04:38:23.592524+0000","last_undegraded":"2023-09-13T04:38:23.592524+0000","last_fullsized":"2023-09-13T04:38:23.592524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:30:56.115483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:25.620943+0000","last_change":"2023-09-13T04:37:58.589119+0000","last_active":"2023-09-13T04:38:25.620943+0000","last_peered":"2023-09-13T04:38:25.620943+0000","last_clean":"2023-09-13T04:38:25.620943+0000","last_became_active":"2023-09-13T04:37:58.588803+0000","last_became_peered":"2023-09-13T04:37:58.588803+0000","last_unstale":"2023-09-13T04:38:25.620943+0000","last_undegraded":"2023-09-13T04:38:25.620943+0000","last_fullsized":"2023-09-13T04:38:25.620943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:51:20.888666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588392+0000","last_change":"2023-09-13T04:37:59.243506+0000","last_active":"2023-09-13T04:38:23.588392+0000","last_peered":"2023-09-13T04:38:23.588392+0000","last_clean":"2023-09-13T04:38:23.588392+0000","last_became_active":"2023-09-13T04:37:59.243082+0000","last_became_peered":"2023-09-13T04:37:59.243082+0000","last_unstale":"2023-09-13T04:38:23.588392+0000","last_undegraded":"2023-09-13T04:38:23.588392+0000","last_fullsized":"2023-09-13T04:38:23.588392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:01:41.179286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592019+0000","last_change":"2023-09-13T04:38:03.609042+0000","last_active":"2023-09-13T04:38:23.592019+0000","last_peered":"2023-09-13T04:38:23.592019+0000","last_clean":"2023-09-13T04:38:23.592019+0000","last_became_active":"2023-09-13T04:38:03.608553+0000","last_became_peered":"2023-09-13T04:38:03.608553+0000","last_unstale":"2023-09-13T04:38:23.592019+0000","last_undegraded":"2023-09-13T04:38:23.592019+0000","last_fullsized":"2023-09-13T04:38:23.592019+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:05:56.648475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.589585+0000","last_change":"2023-09-13T04:37:58.598056+0000","last_active":"2023-09-13T04:38:24.589585+0000","last_peered":"2023-09-13T04:38:24.589585+0000","last_clean":"2023-09-13T04:38:24.589585+0000","last_became_active":"2023-09-13T04:37:58.597724+0000","last_became_peered":"2023-09-13T04:37:58.597724+0000","last_unstale":"2023-09-13T04:38:24.589585+0000","last_undegraded":"2023-09-13T04:38:24.589585+0000","last_fullsized":"2023-09-13T04:38:24.589585+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:43:08.296866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588462+0000","last_change":"2023-09-13T04:37:59.246505+0000","last_active":"2023-09-13T04:38:23.588462+0000","last_peered":"2023-09-13T04:38:23.588462+0000","last_clean":"2023-09-13T04:38:23.588462+0000","last_became_active":"2023-09-13T04:37:59.246024+0000","last_became_peered":"2023-09-13T04:37:59.246024+0000","last_unstale":"2023-09-13T04:38:23.588462+0000","last_undegraded":"2023-09-13T04:38:23.588462+0000","last_fullsized":"2023-09-13T04:38:23.588462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:36:58.008066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.590871+0000","last_change":"2023-09-13T04:38:03.605354+0000","last_active":"2023-09-13T04:38:23.590871+0000","last_peered":"2023-09-13T04:38:23.590871+0000","last_clean":"2023-09-13T04:38:23.590871+0000","last_became_active":"2023-09-13T04:38:03.604840+0000","last_became_peered":"2023-09-13T04:38:03.604840+0000","last_unstale":"2023-09-13T04:38:23.590871+0000","last_undegraded":"2023-09-13T04:38:23.590871+0000","last_fullsized":"2023-09-13T04:38:23.590871+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:46:29.867914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597380+0000","last_change":"2023-09-13T04:37:58.589052+0000","last_active":"2023-09-13T04:38:23.597380+0000","last_peered":"2023-09-13T04:38:23.597380+0000","last_clean":"2023-09-13T04:38:23.597380+0000","last_became_active":"2023-09-13T04:37:58.588707+0000","last_became_peered":"2023-09-13T04:37:58.588707+0000","last_unstale":"2023-09-13T04:38:23.597380+0000","last_undegraded":"2023-09-13T04:38:23.597380+0000","last_fullsized":"2023-09-13T04:38:23.597380+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:25:23.966046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588356+0000","last_change":"2023-09-13T04:37:59.241157+0000","last_active":"2023-09-13T04:38:23.588356+0000","last_peered":"2023-09-13T04:38:23.588356+0000","last_clean":"2023-09-13T04:38:23.588356+0000","last_became_active":"2023-09-13T04:37:59.240857+0000","last_became_peered":"2023-09-13T04:37:59.240857+0000","last_unstale":"2023-09-13T04:38:23.588356+0000","last_undegraded":"2023-09-13T04:38:23.588356+0000","last_fullsized":"2023-09-13T04:38:23.588356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:04:44.986908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592376+0000","last_change":"2023-09-13T04:37:59.242447+0000","last_active":"2023-09-13T04:38:23.592376+0000","last_peered":"2023-09-13T04:38:23.592376+0000","last_clean":"2023-09-13T04:38:23.592376+0000","last_became_active":"2023-09-13T04:37:59.242130+0000","last_became_peered":"2023-09-13T04:37:59.242130+0000","last_unstale":"2023-09-13T04:38:23.592376+0000","last_undegraded":"2023-09-13T04:38:23.592376+0000","last_fullsized":"2023-09-13T04:38:23.592376+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:12:12.844888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-13T04:38:24.595646+0000","last_change":"2023-09-13T04:38:24.595646+0000","last_active":"2023-09-13T04:38:24.595646+0000","last_peered":"2023-09-13T04:38:24.595646+0000","last_clean":"2023-09-13T04:38:24.595646+0000","last_became_active":"2023-09-13T04:38:24.595295+0000","last_became_peered":"2023-09-13T04:38:24.595295+0000","last_unstale":"2023-09-13T04:38:24.595646+0000","last_undegraded":"2023-09-13T04:38:24.595646+0000","last_fullsized":"2023-09-13T04:38:24.595646+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:24:27.174979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.587574+0000","last_change":"2023-09-13T04:37:59.245358+0000","last_active":"2023-09-13T04:38:23.587574+0000","last_peered":"2023-09-13T04:38:23.587574+0000","last_clean":"2023-09-13T04:38:23.587574+0000","last_became_active":"2023-09-13T04:37:59.245052+0000","last_became_peered":"2023-09-13T04:37:59.245052+0000","last_unstale":"2023-09-13T04:38:23.587574+0000","last_undegraded":"2023-09-13T04:38:23.587574+0000","last_fullsized":"2023-09-13T04:38:23.587574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:05:42.561716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592358+0000","last_change":"2023-09-13T04:37:59.254053+0000","last_active":"2023-09-13T04:38:23.592358+0000","last_peered":"2023-09-13T04:38:23.592358+0000","last_clean":"2023-09-13T04:38:23.592358+0000","last_became_active":"2023-09-13T04:37:59.253544+0000","last_became_peered":"2023-09-13T04:37:59.253544+0000","last_unstale":"2023-09-13T04:38:23.592358+0000","last_undegraded":"2023-09-13T04:38:23.592358+0000","last_fullsized":"2023-09-13T04:38:23.592358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:18:54.689208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597920+0000","last_change":"2023-09-13T04:37:59.253403+0000","last_active":"2023-09-13T04:38:23.597920+0000","last_peered":"2023-09-13T04:38:23.597920+0000","last_clean":"2023-09-13T04:38:23.597920+0000","last_became_active":"2023-09-13T04:37:59.253062+0000","last_became_peered":"2023-09-13T04:37:59.253062+0000","last_unstale":"2023-09-13T04:38:23.597920+0000","last_undegraded":"2023-09-13T04:38:23.597920+0000","last_fullsized":"2023-09-13T04:38:23.597920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:31:09.898886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592570+0000","last_change":"2023-09-13T04:37:59.240916+0000","last_active":"2023-09-13T04:38:23.592570+0000","last_peered":"2023-09-13T04:38:23.592570+0000","last_clean":"2023-09-13T04:38:23.592570+0000","last_became_active":"2023-09-13T04:37:59.240605+0000","last_became_peered":"2023-09-13T04:37:59.240605+0000","last_unstale":"2023-09-13T04:38:23.592570+0000","last_undegraded":"2023-09-13T04:38:23.592570+0000","last_fullsized":"2023-09-13T04:38:23.592570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:15:48.241587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.588778+0000","last_change":"2023-09-13T04:37:59.238569+0000","last_active":"2023-09-13T04:38:23.588778+0000","last_peered":"2023-09-13T04:38:23.588778+0000","last_clean":"2023-09-13T04:38:23.588778+0000","last_became_active":"2023-09-13T04:37:59.238255+0000","last_became_peered":"2023-09-13T04:37:59.238255+0000","last_unstale":"2023-09-13T04:38:23.588778+0000","last_undegraded":"2023-09-13T04:38:23.588778+0000","last_fullsized":"2023-09-13T04:38:23.588778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:19:03.433404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.595875+0000","last_change":"2023-09-13T04:37:59.252243+0000","last_active":"2023-09-13T04:38:23.595875+0000","last_peered":"2023-09-13T04:38:23.595875+0000","last_clean":"2023-09-13T04:38:23.595875+0000","last_became_active":"2023-09-13T04:37:59.251902+0000","last_became_peered":"2023-09-13T04:37:59.251902+0000","last_unstale":"2023-09-13T04:38:23.595875+0000","last_undegraded":"2023-09-13T04:38:23.595875+0000","last_fullsized":"2023-09-13T04:38:23.595875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:35:57.395627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.593414+0000","last_change":"2023-09-13T04:37:59.245847+0000","last_active":"2023-09-13T04:38:23.593414+0000","last_peered":"2023-09-13T04:38:23.593414+0000","last_clean":"2023-09-13T04:38:23.593414+0000","last_became_active":"2023-09-13T04:37:59.245542+0000","last_became_peered":"2023-09-13T04:37:59.245542+0000","last_unstale":"2023-09-13T04:38:23.593414+0000","last_undegraded":"2023-09-13T04:38:23.593414+0000","last_fullsized":"2023-09-13T04:38:23.593414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:24:16.813333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.596961+0000","last_change":"2023-09-13T04:37:59.246963+0000","last_active":"2023-09-13T04:38:23.596961+0000","last_peered":"2023-09-13T04:38:23.596961+0000","last_clean":"2023-09-13T04:38:23.596961+0000","last_became_active":"2023-09-13T04:37:59.246448+0000","last_became_peered":"2023-09-13T04:37:59.246448+0000","last_unstale":"2023-09-13T04:38:23.596961+0000","last_undegraded":"2023-09-13T04:38:23.596961+0000","last_fullsized":"2023-09-13T04:38:23.596961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:25:45.418778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.594086+0000","last_change":"2023-09-13T04:37:59.245399+0000","last_active":"2023-09-13T04:38:23.594086+0000","last_peered":"2023-09-13T04:38:23.594086+0000","last_clean":"2023-09-13T04:38:23.594086+0000","last_became_active":"2023-09-13T04:37:59.245060+0000","last_became_peered":"2023-09-13T04:37:59.245060+0000","last_unstale":"2023-09-13T04:38:23.594086+0000","last_undegraded":"2023-09-13T04:38:23.594086+0000","last_fullsized":"2023-09-13T04:38:23.594086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:25:41.078238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.597807+0000","last_change":"2023-09-13T04:37:59.244776+0000","last_active":"2023-09-13T04:38:23.597807+0000","last_peered":"2023-09-13T04:38:23.597807+0000","last_clean":"2023-09-13T04:38:23.597807+0000","last_became_active":"2023-09-13T04:37:59.244409+0000","last_became_peered":"2023-09-13T04:37:59.244409+0000","last_unstale":"2023-09-13T04:38:23.597807+0000","last_undegraded":"2023-09-13T04:38:23.597807+0000","last_fullsized":"2023-09-13T04:38:23.597807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:04:02.561414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-13T04:38:23.592391+0000","last_change":"2023-09-13T04:37:59.248153+0000","last_active":"2023-09-13T04:38:23.592391+0000","last_peered":"2023-09-13T04:38:23.592391+0000","last_clean":"2023-09-13T04:38:23.592391+0000","last_became_active":"2023-09-13T04:37:59.247840+0000","last_became_peered":"2023-09-13T04:37:59.247840+0000","last_unstale":"2023-09-13T04:38:23.592391+0000","last_undegraded":"2023-09-13T04:38:23.592391+0000","last_fullsized":"2023-09-13T04:38:23.592391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:52:53.021219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7452,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":346,"num_read_kb":186,"num_write":129,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":22356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416413,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":490770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.625,"15min":0.61599999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":2.8420000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.52200000000000002}]},{"osd":1,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.66800000000000004,"15min":0.66100000000000003},"min":{"1min":0.46999999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92700000000000005,"5min":1.024,"15min":1.024},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.42099999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80600000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.91800000000000004}]},{"osd":2,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.47299999999999998,"15min":0.46899999999999997},"min":{"1min":0.34799999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":4,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.68600000000000005,"15min":0.67800000000000005},"min":{"1min":0.53100000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.88},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.42899999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86699999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.67600000000000005}]},{"osd":5,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.64700000000000002,"15min":0.63100000000000001},"min":{"1min":0.45300000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.71099999999999997}]},{"osd":6,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70599999999999996,"15min":0.70399999999999996},"min":{"1min":0.46999999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.70799999999999996,"15min":0.69799999999999995},"min":{"1min":0.55900000000000005,"5min":0.498,"15min":0.498},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.66100000000000003}]},{"osd":7,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.77500000000000002,"15min":0.76500000000000001},"min":{"1min":0.59599999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0229999999999999,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.50800000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94099999999999995,"5min":1.262,"15min":1.262},"last":0.89400000000000002}]},{"osd":8,"last update":"Wed Sep 13 04:38:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.95799999999999996}]},{"osd":9,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.77400000000000002,"15min":0.76200000000000001},"min":{"1min":0.61499999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.121,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.82499999999999996,"15min":0.81100000000000005},"min":{"1min":0.66200000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1830000000000001,"5min":1.361,"15min":1.361},"last":0.98299999999999998}]},{"osd":10,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.78900000000000003,"15min":0.78600000000000003},"min":{"1min":0.56000000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.002,"5min":1.155,"15min":1.155},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.77300000000000002,"15min":0.76700000000000002},"min":{"1min":0.59599999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95699999999999996,"5min":1.282,"15min":1.282},"last":0.86199999999999999}]}]},{"osd":10,"up_from":71,"seq":304942678049,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1085549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:00 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":4.0510000000000002,"5min":4.0510000000000002,"15min":4.0510000000000002},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":4.1120000000000001,"5min":4.1120000000000001,"15min":4.1120000000000001},"last":0.58599999999999997}]},{"osd":1,"last update":"Wed Sep 13 04:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.46300000000000002,"15min":0.45700000000000002},"min":{"1min":0.32900000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.47899999999999998,"15min":0.47099999999999997},"min":{"1min":0.312,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.63500000000000001}]},{"osd":2,"last update":"Wed Sep 13 04:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.63300000000000001,"15min":0.63100000000000001},"min":{"1min":0.47499999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.83999999999999997,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.755},{"interface":"front","average":{"1min":0.624,"5min":0.63900000000000001,"15min":0.64100000000000001},"min":{"1min":0.51300000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.78200000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.76800000000000002}]},{"osd":3,"last update":"Wed Sep 13 04:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.70799999999999996,"15min":0.67200000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.5570000000000004,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69299999999999995,"15min":0.69099999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84799999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.78900000000000003}]},{"osd":4,"last update":"Wed Sep 13 04:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60599999999999998,"15min":0.60299999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55200000000000005,"15min":0.54800000000000004},"min":{"1min":0.36199999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78000000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.71099999999999997}]},{"osd":5,"last update":"Wed Sep 13 04:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69699999999999995,"15min":0.69299999999999995},"min":{"1min":0.51700000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.87,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.85599999999999998}]},{"osd":6,"last update":"Wed Sep 13 04:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.86599999999999999,"15min":0.88400000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.476,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.1200000000000001,"5min":0.872,"15min":0.82999999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":7.7839999999999998,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":0.89200000000000002}]},{"osd":7,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.64500000000000002}]},{"osd":8,"last update":"Wed Sep 13 04:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.77300000000000002,"15min":0.76900000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.79700000000000004,"15min":0.80500000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.92600000000000005,"5min":1.03,"15min":1.03},"last":0.95899999999999996}]},{"osd":9,"last update":"Wed Sep 13 04:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.878,"15min":0.85499999999999998},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":4.2679999999999998,"5min":4.2679999999999998,"15min":4.2679999999999998},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.874,"15min":0.85899999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":4.3600000000000003,"5min":4.3600000000000003,"15min":4.3600000000000003},"last":0.878}]},{"osd":11,"last update":"Wed Sep 13 04:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.82799999999999996,"15min":0.82999999999999996},"min":{"1min":0.64100000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.80700000000000005,"15min":0.80600000000000005},"min":{"1min":0.65300000000000002,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.97099999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.93000000000000005}]}]},{"osd":9,"up_from":66,"seq":283467841573,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":490860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58099999999999996,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.316,"15min":0.316},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.60099999999999998}]},{"osd":1,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.44,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":1.036},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.71999999999999997,"15min":0.73199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82899999999999996,"5min":0.995,"15min":0.995},"last":0.68000000000000005}]},{"osd":2,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.747,"15min":0.75600000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.042,"5min":1.083,"15min":1.083},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.746,"15min":0.754},"min":{"1min":0.57099999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.83499999999999996,"5min":1.109,"15min":1.109},"last":0.64200000000000002}]},{"osd":3,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.42899999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.35999999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":0.999}]},{"osd":4,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.78800000000000003,"15min":0.80500000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.82999999999999996,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.80400000000000005,"15min":0.81200000000000006},"min":{"1min":0.55600000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2,"5min":1.278,"15min":1.278},"last":0.78200000000000003}]},{"osd":5,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.85399999999999998,"15min":0.86699999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2689999999999999,"5min":1.339,"15min":1.339},"last":0.748},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.81000000000000005,"15min":0.81599999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.119,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":1.071}]},{"osd":6,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.72099999999999997}]},{"osd":7,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.90200000000000002,"15min":0.91400000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.3009999999999999,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.90500000000000003,"15min":0.92600000000000005},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1439999999999999,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":1.1060000000000001}]},{"osd":8,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.91700000000000004,"15min":0.93999999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0249999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.95499999999999996,"15min":0.97099999999999997},"min":{"1min":0.60299999999999998,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.2350000000000001,"5min":1.857,"15min":1.857},"last":0.71099999999999997}]},{"osd":10,"last update":"Wed Sep 13 04:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.89300000000000002,"15min":0.89900000000000002},"min":{"1min":0.67200000000000004,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.206,"5min":1.333,"15min":1.333},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.90100000000000002,"15min":0.91300000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.044,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.1639999999999999}]},{"osd":11,"last update":"Wed Sep 13 04:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":1.137}]}]},{"osd":8,"up_from":58,"seq":249108103210,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":490337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58699999999999997,"15min":0.58799999999999997},"min":{"1min":0.38900000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.72799999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.41199999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80400000000000005,"5min":1.016,"15min":1.016},"last":0.72999999999999998}]},{"osd":1,"last update":"Wed Sep 13 04:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.57499999999999996,"15min":0.54500000000000004},"min":{"1min":0.45700000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.87,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.46100000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82799999999999996,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.59499999999999997}]},{"osd":2,"last update":"Wed Sep 13 04:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47899999999999998,"15min":0.47499999999999998},"min":{"1min":0.32100000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67800000000000005,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.49199999999999999,"15min":0.48699999999999999},"min":{"1min":0.309,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82299999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.39800000000000002}]},{"osd":3,"last update":"Wed Sep 13 04:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.434,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90600000000000003,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.64400000000000002,"15min":0.64200000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.80800000000000005,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.53600000000000003}]},{"osd":4,"last update":"Wed Sep 13 04:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.61099999999999999,"15min":0.59099999999999997},"min":{"1min":0.53800000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91200000000000003,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.63},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.64200000000000002,"15min":0.621},"min":{"1min":0.51000000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89600000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.70499999999999996}]},{"osd":5,"last update":"Wed Sep 13 04:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.35999999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.90000000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.501},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59099999999999997,"15min":0.58799999999999997},"min":{"1min":0.36099999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.871,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.51500000000000001}]},{"osd":6,"last update":"Wed Sep 13 04:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.90000000000000002,"5min":1.01,"15min":1.01},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54000000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.97199999999999998,"5min":1.077,"15min":1.077},"last":0.64900000000000002}]},{"osd":7,"last update":"Wed Sep 13 04:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.71999999999999997,"15min":0.71299999999999997},"min":{"1min":0.54100000000000004,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94599999999999995,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.71199999999999997,"15min":0.69399999999999995},"min":{"1min":0.57199999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.83499999999999996}]},{"osd":9,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.77000000000000002,"15min":0.76200000000000001},"min":{"1min":0.59099999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.76800000000000002,"15min":0.75600000000000001},"min":{"1min":0.58199999999999996,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.77700000000000002}]},{"osd":10,"last update":"Wed Sep 13 04:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.751,"15min":0.73699999999999999},"min":{"1min":0.57399999999999995,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.77100000000000002,"15min":0.75900000000000001},"min":{"1min":0.56399999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.81100000000000005}]},{"osd":11,"last update":"Wed Sep 13 04:39:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.752},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.873}]}]},{"osd":7,"up_from":51,"seq":219043332143,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1609728,"data_stored":1082202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.46500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.746,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60499999999999998,"15min":0.60799999999999998},"min":{"1min":0.41599999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.40200000000000002}]},{"osd":1,"last update":"Wed Sep 13 04:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.42299999999999999,"15min":0.40799999999999997},"min":{"1min":0.30599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65600000000000003,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.496},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.432,"15min":0.42399999999999999},"min":{"1min":0.32900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61899999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.64500000000000002}]},{"osd":2,"last update":"Wed Sep 13 04:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60399999999999998,"15min":0.59199999999999997},"min":{"1min":0.40600000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83399999999999996,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.59899999999999998,"15min":0.57799999999999996},"min":{"1min":0.49099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.749}]},{"osd":3,"last update":"Wed Sep 13 04:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.69699999999999995,"15min":0.65300000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":7.0789999999999997,"5min":7.0789999999999997,"15min":7.0789999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.45000000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":0.76300000000000001}]},{"osd":4,"last update":"Wed Sep 13 04:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51400000000000001,"15min":0.51100000000000001},"min":{"1min":0.379,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.50800000000000001,"15min":0.51400000000000001},"min":{"1min":0.35799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.68400000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53500000000000003}]},{"osd":5,"last update":"Wed Sep 13 04:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64100000000000001,"15min":0.63},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.66300000000000003,"15min":0.64700000000000002},"min":{"1min":0.56599999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88100000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.81599999999999995}]},{"osd":6,"last update":"Wed Sep 13 04:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.60299999999999998}]},{"osd":8,"last update":"Wed Sep 13 04:39:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.71699999999999997,"15min":0.70899999999999996},"min":{"1min":0.59399999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72099999999999997,"15min":0.71699999999999997},"min":{"1min":0.56599999999999995,"5min":0.437,"15min":0.437},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.91000000000000003}]},{"osd":9,"last update":"Wed Sep 13 04:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.76800000000000002,"15min":0.76600000000000001},"min":{"1min":0.60299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.76100000000000001,"15min":0.75900000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.877}]},{"osd":10,"last update":"Wed Sep 13 04:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69499999999999995,"15min":0.69699999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.56499999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.85799999999999998}]},{"osd":11,"last update":"Wed Sep 13 04:38:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.94399999999999995}]}]},{"osd":6,"up_from":46,"seq":197568495667,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":490132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.51200000000000001,"15min":0.46800000000000003},"min":{"1min":0.33600000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":2.738,"5min":2.738,"15min":2.738},"last":0.39000000000000001},{"interface":"front","average":{"1min":1.0920000000000001,"5min":0.625,"15min":0.53100000000000003},"min":{"1min":0.41599999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":8.5749999999999993,"5min":8.5749999999999993,"15min":8.5749999999999993},"last":0.41599999999999998}]},{"osd":1,"last update":"Wed Sep 13 04:39:20 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":0.74099999999999999,"15min":0.64500000000000002},"min":{"1min":0.52700000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":10.374000000000001,"5min":10.374000000000001,"15min":10.374000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.72999999999999998,"15min":0.68600000000000005},"min":{"1min":0.51500000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.66600000000000004}]},{"osd":2,"last update":"Wed Sep 13 04:39:20 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":0.78100000000000003,"15min":0.71699999999999997},"min":{"1min":0.505,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":8.4740000000000002,"5min":8.4740000000000002,"15min":8.4740000000000002},"last":0.505},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.69799999999999995,"15min":0.68400000000000005},"min":{"1min":0.55900000000000005,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.64600000000000002}]},{"osd":3,"last update":"Wed Sep 13 04:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.69899999999999995,"15min":0.65400000000000003},"min":{"1min":0.46000000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.331,"5min":2.331,"15min":2.331},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.874,"5min":0.67800000000000005,"15min":0.63500000000000001},"min":{"1min":0.45000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.585,"5min":2.585,"15min":2.585},"last":0.55300000000000005}]},{"osd":4,"last update":"Wed Sep 13 04:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.78000000000000003,"15min":0.752},"min":{"1min":0.60699999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.2809999999999999,"5min":0.85999999999999999,"15min":0.80400000000000005},"min":{"1min":0.58199999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":8.3350000000000009,"5min":8.3350000000000009,"15min":8.3350000000000009},"last":0.67900000000000005}]},{"osd":5,"last update":"Wed Sep 13 04:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.82699999999999996,"15min":0.79900000000000004},"min":{"1min":0.52500000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":1.288,"5min":0.88600000000000001,"15min":0.83999999999999997},"min":{"1min":0.54100000000000004,"5min":0.379,"15min":0.379},"max":{"1min":8.5060000000000002,"5min":8.5060000000000002,"15min":8.5060000000000002},"last":0.54100000000000004}]},{"osd":7,"last update":"Wed Sep 13 04:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.85699999999999998,"15min":0.82899999999999996},"min":{"1min":0.69299999999999995,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.361,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.1819999999999999,"5min":0.878,"15min":0.81499999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":8.3970000000000002,"5min":8.3970000000000002,"15min":8.3970000000000002},"last":0.627}]},{"osd":8,"last update":"Wed Sep 13 04:39:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.82099999999999995,"15min":0.82499999999999996},"min":{"1min":0.65200000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.873},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.93000000000000005,"15min":0.92800000000000005},"min":{"1min":0.57999999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.752}]},{"osd":9,"last update":"Wed Sep 13 04:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.86499999999999999,"15min":0.88300000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.98099999999999998,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.88900000000000001,"5min":1.0069999999999999,"15min":1.0269999999999999},"min":{"1min":0.72399999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.0089999999999999,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.72399999999999998}]},{"osd":10,"last update":"Wed Sep 13 04:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.995,"15min":0.99299999999999999},"min":{"1min":0.81399999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.262,"5min":0.94499999999999995,"15min":0.89200000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":8.4380000000000006,"5min":8.4380000000000006,"15min":8.4380000000000006},"last":0.79700000000000004}]},{"osd":11,"last update":"Wed Sep 13 04:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":8.5410000000000004,"5min":8.5410000000000004,"15min":8.5410000000000004},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.90100000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":493884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57099999999999995,"15min":0.54200000000000004},"min":{"1min":0.432,"5min":0.307,"15min":0.307},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.59099999999999997,"15min":0.55900000000000005},"min":{"1min":0.42399999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.94099999999999995,"5min":0.999,"15min":0.999},"last":0.73999999999999999}]},{"osd":2,"last update":"Wed Sep 13 04:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58099999999999996,"15min":0.54500000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.58699999999999997,"15min":0.53100000000000003},"min":{"1min":0.42399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.57599999999999996}]},{"osd":3,"last update":"Wed Sep 13 04:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83099999999999996,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.623,"15min":0.60899999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91600000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.66700000000000004}]},{"osd":4,"last update":"Wed Sep 13 04:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46800000000000003,"15min":0.45500000000000002},"min":{"1min":0.30299999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79500000000000004,"5min":0.878,"15min":0.878},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.47499999999999998,"15min":0.45400000000000001},"min":{"1min":0.29899999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.747,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.439}]},{"osd":5,"last update":"Wed Sep 13 04:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.60499999999999998,"15min":0.57299999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96899999999999997,"5min":1.3,"15min":1.3},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.65100000000000002,"15min":0.59499999999999997},"min":{"1min":0.47299999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.96999999999999997,"5min":1.264,"15min":1.264},"last":0.629}]},{"osd":6,"last update":"Wed Sep 13 04:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.99199999999999999,"5min":1.482,"15min":1.482},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95399999999999996,"5min":1.375,"15min":1.375},"last":0.72199999999999998}]},{"osd":7,"last update":"Wed Sep 13 04:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.63300000000000001,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.372,"15min":0.372},"max":{"1min":1.137,"5min":1.347,"15min":1.347},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.67800000000000005,"15min":0.65000000000000002},"min":{"1min":0.38900000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0209999999999999,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.872}]},{"osd":8,"last update":"Wed Sep 13 04:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.75700000000000001,"15min":0.74199999999999999},"min":{"1min":0.48099999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0980000000000001,"5min":1.393,"15min":1.393},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.73999999999999999,"15min":0.72399999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.032,"5min":1.224,"15min":1.224},"last":0.79900000000000004}]},{"osd":9,"last update":"Wed Sep 13 04:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.85399999999999998,"15min":0.85899999999999999},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1819999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.85399999999999998,"15min":0.86299999999999999},"min":{"1min":0.58199999999999996,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.02,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.93000000000000005}]},{"osd":10,"last update":"Wed Sep 13 04:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81599999999999995,"15min":0.82099999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0740000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.56399999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.071,"5min":1.504,"15min":1.504},"last":0.77800000000000002}]},{"osd":11,"last update":"Wed Sep 13 04:38:59 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.83299999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":491520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 13 04:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.69999999999999996,"15min":0.59599999999999997},"min":{"1min":0.38700000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94499999999999995,"5min":6.2359999999999998,"15min":6.2359999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.61799999999999999,"15min":0.56200000000000006},"min":{"1min":0.42699999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.92000000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.40600000000000003}]},{"osd":2,"last update":"Wed Sep 13 04:38:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64200000000000002,"15min":0.60499999999999998},"min":{"1min":0.40000000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.66300000000000003,"15min":0.61199999999999999},"min":{"1min":0.47199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.89500000000000002}]},{"osd":3,"last update":"Wed Sep 13 04:39:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.433}]},{"osd":4,"last update":"Wed Sep 13 04:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.69399999999999995,"15min":0.625},"min":{"1min":0.47499999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.079,"5min":1.135,"15min":1.135},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.67700000000000005,"15min":0.63},"min":{"1min":0.51500000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93400000000000005,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.57599999999999996}]},{"osd":5,"last update":"Wed Sep 13 04:39:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.68300000000000005,"15min":0.67000000000000004},"min":{"1min":0.40999999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.73899999999999999,"15min":0.69299999999999995},"min":{"1min":0.52300000000000002,"5min":0.435,"15min":0.435},"max":{"1min":1.329,"5min":1.444,"15min":1.444},"last":0.82799999999999996}]},{"osd":6,"last update":"Wed Sep 13 04:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.44600000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94899999999999995,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88700000000000001,"5min":1.165,"15min":1.165},"last":0.55300000000000005}]},{"osd":7,"last update":"Wed Sep 13 04:39:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.70199999999999996}]},{"osd":8,"last update":"Wed Sep 13 04:39:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.90800000000000003,"15min":0.86099999999999999},"min":{"1min":0.57699999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3759999999999999,"5min":6.2089999999999996,"15min":6.2089999999999996},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.90300000000000002,"15min":0.83099999999999996},"min":{"1min":0.58999999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.591,"5min":6.1699999999999999,"15min":6.1699999999999999},"last":0.79300000000000004}]},{"osd":9,"last update":"Wed Sep 13 04:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.92200000000000004,"15min":0.94599999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97399999999999998,"5min":1.357,"15min":1.357},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.171,"5min":1.484,"15min":1.484},"last":0.75}]},{"osd":10,"last update":"Wed Sep 13 04:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.98899999999999999,"15min":0.98499999999999999},"min":{"1min":0.80800000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.522,"5min":1.591,"15min":1.591},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.1040000000000001,"15min":1.139},"min":{"1min":0.66300000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.3859999999999999,"5min":6.1859999999999999,"15min":6.1859999999999999},"last":0.623}]},{"osd":11,"last update":"Wed Sep 13 04:39:16 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":0.99099999999999999,"15min":0.98399999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.86099999999999999,"15min":0.85299999999999998},"min":{"1min":0.71499999999999997,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.72399999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":495200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.54900000000000004,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.54900000000000004,"15min":0.52800000000000002},"min":{"1min":0.39900000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.436}]},{"osd":1,"last update":"Wed Sep 13 04:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.57399999999999995,"15min":0.50900000000000001},"min":{"1min":0.438,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.86599999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.57799999999999996,"15min":0.51900000000000002},"min":{"1min":0.41399999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.877,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.499}]},{"osd":3,"last update":"Wed Sep 13 04:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.61899999999999999,"15min":0.59199999999999997},"min":{"1min":0.45700000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.623,"15min":0.60999999999999999},"min":{"1min":0.48899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.60799999999999998}]},{"osd":4,"last update":"Wed Sep 13 04:39:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.58599999999999997}]},{"osd":5,"last update":"Wed Sep 13 04:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.47299999999999998,"15min":0.49399999999999999},"min":{"1min":0.378,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.65200000000000002,"5min":1.796,"15min":1.796},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52800000000000002,"15min":0.53200000000000003},"min":{"1min":0.38300000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73899999999999999,"5min":1.546,"15min":1.546},"last":0.68500000000000005}]},{"osd":6,"last update":"Wed Sep 13 04:38:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.69199999999999995,"15min":0.69899999999999995},"min":{"1min":0.48499999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81599999999999995,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.752},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66700000000000004,"15min":0.67200000000000004},"min":{"1min":0.48199999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85899999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.72699999999999998}]},{"osd":7,"last update":"Wed Sep 13 04:39:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.74299999999999999}]},{"osd":8,"last update":"Wed Sep 13 04:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61799999999999999,"15min":0.62},"min":{"1min":0.40300000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.82399999999999995,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.66700000000000004,"15min":0.65300000000000002},"min":{"1min":0.51200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.90400000000000003,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.87}]},{"osd":9,"last update":"Wed Sep 13 04:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72099999999999997,"15min":0.71899999999999997},"min":{"1min":0.57299999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.93100000000000005,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.77800000000000002}]},{"osd":10,"last update":"Wed Sep 13 04:38:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.70199999999999996,"15min":0.68999999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.92600000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.75800000000000001,"15min":0.73599999999999999},"min":{"1min":0.66900000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.012,"5min":1.032,"15min":1.032},"last":0.91200000000000003}]},{"osd":11,"last update":"Wed Sep 13 04:39:00 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.94799999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":490326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.45500000000000002,"15min":0.39900000000000002},"min":{"1min":0.30399999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.52900000000000003,"15min":0.45600000000000002},"min":{"1min":0.34399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.5940000000000003,"5min":4.5940000000000003,"15min":4.5940000000000003},"last":0.55800000000000005}]},{"osd":1,"last update":"Wed Sep 13 04:39:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.64000000000000001,"15min":0.58699999999999997},"min":{"1min":0.42399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.68000000000000005,"15min":0.58999999999999997},"min":{"1min":0.50600000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":1.105}]},{"osd":2,"last update":"Wed Sep 13 04:39:13 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.72799999999999998,"15min":0.64700000000000002},"min":{"1min":0.47699999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":4.3920000000000003,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.65900000000000003,"15min":0.626},"min":{"1min":0.46100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97999999999999998,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.50800000000000001}]},{"osd":4,"last update":"Wed Sep 13 04:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66700000000000004,"15min":0.60899999999999999},"min":{"1min":0.53100000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.66700000000000004,"15min":0.61599999999999999},"min":{"1min":0.46500000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.88900000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.67900000000000005}]},{"osd":5,"last update":"Wed Sep 13 04:39:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.65400000000000003},"min":{"1min":0.432,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89600000000000002,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.77600000000000002,"15min":0.68400000000000005},"min":{"1min":0.48299999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":4.3339999999999996,"5min":4.3339999999999996,"15min":4.3339999999999996},"last":0.85899999999999999}]},{"osd":6,"last update":"Wed Sep 13 04:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.74099999999999999,"15min":0.70299999999999996},"min":{"1min":0.47399999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1060000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.76800000000000002,"15min":0.745},"min":{"1min":0.504,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0529999999999999,"5min":1.109,"15min":1.109},"last":1.1870000000000001}]},{"osd":7,"last update":"Wed Sep 13 04:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.85699999999999998,"15min":0.86199999999999999},"min":{"1min":0.56299999999999994,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.014,"5min":0.88800000000000001,"15min":0.86099999999999999},"min":{"1min":0.59299999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":4.4699999999999998,"5min":4.4699999999999998,"15min":4.4699999999999998},"last":0.92500000000000004}]},{"osd":8,"last update":"Wed Sep 13 04:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.875}]},{"osd":9,"last update":"Wed Sep 13 04:38:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.94699999999999995,"15min":0.96399999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.2030000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.84099999999999997,"15min":0.84899999999999998},"min":{"1min":0.52100000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.01,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.71299999999999997}]},{"osd":10,"last update":"Wed Sep 13 04:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.58899999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.091,"5min":0.98199999999999998,"15min":0.96399999999999997},"min":{"1min":0.747,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":4.5099999999999998,"5min":4.5099999999999998,"15min":4.5099999999999998},"last":0.96199999999999997}]},{"osd":11,"last update":"Wed Sep 13 04:38:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.83599999999999997}]}]},{"osd":4,"up_from":33,"seq":141733920830,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":490686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.56599999999999995,"15min":0.54400000000000004},"min":{"1min":0.43099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.753},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76500000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.68999999999999995}]},{"osd":1,"last update":"Wed Sep 13 04:38:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.39600000000000002,"15min":0.35399999999999998},"min":{"1min":0.29499999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.623,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.40699999999999997,"15min":0.371},"min":{"1min":0.33600000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.65900000000000003,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.44600000000000001}]},{"osd":2,"last update":"Wed Sep 13 04:38:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60099999999999998,"15min":0.56699999999999995},"min":{"1min":0.35799999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88800000000000001,"5min":1.631,"15min":1.631},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.58999999999999997,"15min":0.54600000000000004},"min":{"1min":0.42299999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.878,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.55100000000000005}]},{"osd":3,"last update":"Wed Sep 13 04:38:41 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60899999999999999,"15min":0.56999999999999995},"min":{"1min":0.51900000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.81799999999999995,"5min":0.995,"15min":0.995},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60099999999999998,"15min":0.57899999999999996},"min":{"1min":0.44500000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78700000000000003,"5min":1.115,"15min":1.115},"last":0.76600000000000001}]},{"osd":5,"last update":"Wed Sep 13 04:38:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.622,"15min":0.59199999999999997},"min":{"1min":0.54700000000000004,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89300000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.624,"15min":0.61099999999999999},"min":{"1min":0.49299999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86899999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.53400000000000003}]},{"osd":6,"last update":"Wed Sep 13 04:38:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.68000000000000005,"15min":0.68400000000000005},"min":{"1min":0.47099999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81599999999999995,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.69799999999999995,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.84599999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.78000000000000003}]},{"osd":7,"last update":"Wed Sep 13 04:38:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.61799999999999999,"15min":0.59899999999999998},"min":{"1min":0.46500000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.85399999999999998,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.68300000000000005}]},{"osd":8,"last update":"Wed Sep 13 04:39:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.72399999999999998,"15min":0.69799999999999995},"min":{"1min":0.59599999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.97599999999999998,"5min":1.042,"15min":1.042},"last":0.873},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76700000000000002,"15min":0.76300000000000001},"min":{"1min":0.57099999999999995,"5min":0.441,"15min":0.441},"max":{"1min":0.98799999999999999,"5min":1.095,"15min":1.095},"last":0.745}]},{"osd":9,"last update":"Wed Sep 13 04:38:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.80700000000000005,"15min":0.81200000000000006},"min":{"1min":0.59299999999999997,"5min":0.495,"15min":0.495},"max":{"1min":0.93000000000000005,"5min":1.141,"15min":1.141},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.77100000000000002,"15min":0.77300000000000002},"min":{"1min":0.60799999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.879,"5min":1.056,"15min":1.056},"last":0.80700000000000005}]},{"osd":10,"last update":"Wed Sep 13 04:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.75900000000000001,"15min":0.76000000000000001},"min":{"1min":0.54600000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.93600000000000005,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.78900000000000003,"15min":0.79200000000000004},"min":{"1min":0.60499999999999998,"5min":0.498,"15min":0.498},"max":{"1min":0.97099999999999997,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.70299999999999996}]},{"osd":11,"last update":"Wed Sep 13 04:38:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.90800000000000003}]}]},{"osd":5,"up_from":38,"seq":163208757305,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1622016,"data_stored":1081886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.35199999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.51000000000000001}]},{"osd":1,"last update":"Wed Sep 13 04:39:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.55900000000000005,"15min":0.53000000000000003},"min":{"1min":0.46600000000000003,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.83999999999999997,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.57299999999999995,"15min":0.53400000000000003},"min":{"1min":0.51600000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84899999999999998,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.752}]},{"osd":2,"last update":"Wed Sep 13 04:39:00 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.40699999999999997,"15min":0.37},"min":{"1min":0.307,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.42299999999999999,"15min":0.38700000000000001},"min":{"1min":0.312,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.75900000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.48099999999999998}]},{"osd":3,"last update":"Wed Sep 13 04:39:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.60299999999999998,"15min":0.58699999999999997},"min":{"1min":0.42999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.73399999999999999,"15min":0.85099999999999998},"min":{"1min":0.39700000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77700000000000002,"5min":8.0009999999999994,"15min":8.0009999999999994},"last":0.57499999999999996}]},{"osd":4,"last update":"Wed Sep 13 04:39:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.60599999999999998,"15min":0.57099999999999995},"min":{"1min":0.45800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.877,"5min":1.599,"15min":1.599},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.60599999999999998,"15min":0.56499999999999995},"min":{"1min":0.52100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.93899999999999995,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.82499999999999996}]},{"osd":6,"last update":"Wed Sep 13 04:38:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83499999999999996,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81899999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.85899999999999999}]},{"osd":7,"last update":"Wed Sep 13 04:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.65900000000000003,"15min":0.64900000000000002},"min":{"1min":0.51800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90400000000000003,"5min":1.488,"15min":1.488},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.68100000000000005,"15min":0.66900000000000004},"min":{"1min":0.53400000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90000000000000002,"5min":1.51,"15min":1.51},"last":0.80400000000000005}]},{"osd":8,"last update":"Wed Sep 13 04:39:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59599999999999997,"15min":0.58799999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.83699999999999997,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.877},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.877,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.89000000000000001}]},{"osd":9,"last update":"Wed Sep 13 04:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.72799999999999998,"15min":0.72499999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.60599999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.92900000000000005}]},{"osd":10,"last update":"Wed Sep 13 04:38:54 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.73299999999999998,"15min":0.72199999999999998},"min":{"1min":0.59699999999999998,"5min":0.496,"15min":0.496},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.754,"15min":0.748},"min":{"1min":0.60699999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.73199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-13T04:39:23.439 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-09-13T04:39:23.439 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-09-13T04:39:23.440 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-09-13T04:39:23.440 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph health --format=json 2023-09-13T04:39:24.184 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:39:24.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:24 smithi002 ceph-mon[97651]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:39:24.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:24 smithi002 ceph-mon[97651]: pgmap v320: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:39:24.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:24 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-09-13T04:39:24.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:24 smithi149 ceph-mon[133315]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:39:24.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:24 smithi149 ceph-mon[133315]: pgmap v320: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:39:24.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:24 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-09-13T04:39:24.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:24 smithi156 ceph-mon[133675]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:39:24.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:24 smithi156 ceph-mon[133675]: pgmap v320: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:39:24.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:24 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-09-13T04:39:25.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.812+0000 7f360a9af700 1 -- 172.21.15.2:0/1652543638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3604100d80 msgr2=0x7f3604101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:25.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.812+0000 7f360a9af700 1 --2- 172.21.15.2:0/1652543638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3604100d80 0x7f3604101200 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f35f80099c0 tx=0x7f35f8009cd0 comp rx=0 tx=0).stop 2023-09-13T04:39:25.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.812+0000 7f360a9af700 1 -- 172.21.15.2:0/1652543638 shutdown_connections 2023-09-13T04:39:25.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.812+0000 7f360a9af700 1 --2- 172.21.15.2:0/1652543638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3604101740 0x7f360410dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:25.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.812+0000 7f360a9af700 1 --2- 172.21.15.2:0/1652543638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3604106d90 0x7f3604107170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:25.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.812+0000 7f360a9af700 1 --2- 172.21.15.2:0/1652543638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3604100d80 0x7f3604101200 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:25.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.812+0000 7f360a9af700 1 -- 172.21.15.2:0/1652543638 >> 172.21.15.2:0/1652543638 conn(0x7f3604077ea0 msgr2=0x7f36040782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:25.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.812+0000 7f360a9af700 1 -- 172.21.15.2:0/1652543638 shutdown_connections 2023-09-13T04:39:25.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.812+0000 7f360a9af700 1 -- 172.21.15.2:0/1652543638 wait complete. 2023-09-13T04:39:25.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.814+0000 7f360a9af700 1 Processor -- start 2023-09-13T04:39:25.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.814+0000 7f360a9af700 1 -- start start 2023-09-13T04:39:25.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.815+0000 7f360a9af700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3604100d80 0x7f3604072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:25.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.815+0000 7f360a9af700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3604101740 0x7f36040734f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:25.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.815+0000 7f360a9af700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3604106d90 0x7f3604073a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:25.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.815+0000 7f360a9af700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36041a5200 con 0x7f3604101740 2023-09-13T04:39:25.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.815+0000 7f360a9af700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36041a5370 con 0x7f3604100d80 2023-09-13T04:39:25.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.815+0000 7f360a9af700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36041a5530 con 0x7f3604106d90 2023-09-13T04:39:25.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.815+0000 7f36037fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3604101740 0x7f36040734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:25.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.815+0000 7f36037fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3604101740 0x7f36040734f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51556/0 (socket says 172.21.15.2:51556) 2023-09-13T04:39:25.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.815+0000 7f36037fe700 1 -- 172.21.15.2:0/1643955413 learned_addr learned my addr 172.21.15.2:0/1643955413 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:25.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.815+0000 7f3608f4c700 1 --2- 172.21.15.2:0/1643955413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3604106d90 0x7f3604073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:25.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.815+0000 7f3603fff700 1 --2- 172.21.15.2:0/1643955413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3604100d80 0x7f3604072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:25.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.816+0000 7f36037fe700 1 -- 172.21.15.2:0/1643955413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3604106d90 msgr2=0x7f3604073a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:25.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.816+0000 7f36037fe700 1 --2- 172.21.15.2:0/1643955413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3604106d90 0x7f3604073a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:25.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.816+0000 7f36037fe700 1 -- 172.21.15.2:0/1643955413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3604100d80 msgr2=0x7f3604072fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:25.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.816+0000 7f36037fe700 1 --2- 172.21.15.2:0/1643955413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3604100d80 0x7f3604072fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:25.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.816+0000 7f36037fe700 1 -- 172.21.15.2:0/1643955413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f80095f0 con 0x7f3604101740 2023-09-13T04:39:25.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.816+0000 7f36037fe700 1 --2- 172.21.15.2:0/1643955413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3604101740 0x7f36040734f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009990 tx=0x7f35f8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:25.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.816+0000 7f36017fa700 1 -- 172.21.15.2:0/1643955413 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f801d070 con 0x7f3604101740 2023-09-13T04:39:25.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.816+0000 7f36017fa700 1 -- 172.21.15.2:0/1643955413 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35f8003f90 con 0x7f3604101740 2023-09-13T04:39:25.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.816+0000 7f360a9af700 1 -- 172.21.15.2:0/1643955413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36041a57b0 con 0x7f3604101740 2023-09-13T04:39:25.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.816+0000 7f36017fa700 1 -- 172.21.15.2:0/1643955413 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f8022ba0 con 0x7f3604101740 2023-09-13T04:39:25.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.816+0000 7f360a9af700 1 -- 172.21.15.2:0/1643955413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36041a5bc0 con 0x7f3604101740 2023-09-13T04:39:25.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.817+0000 7f360a9af700 1 -- 172.21.15.2:0/1643955413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f360404e9b0 con 0x7f3604101740 2023-09-13T04:39:25.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.820+0000 7f36017fa700 1 -- 172.21.15.2:0/1643955413 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f35f8003ad0 con 0x7f3604101740 2023-09-13T04:39:25.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.821+0000 7f36017fa700 1 --2- 172.21.15.2:0/1643955413 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f35ec0a0640 0x7f35ec0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:25.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.821+0000 7f36017fa700 1 -- 172.21.15.2:0/1643955413 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f35f80cdb30 con 0x7f3604101740 2023-09-13T04:39:25.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.821+0000 7f3603fff700 1 --2- 172.21.15.2:0/1643955413 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f35ec0a0640 0x7f35ec0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:25.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.822+0000 7f3603fff700 1 --2- 172.21.15.2:0/1643955413 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f35ec0a0640 0x7f35ec0a2b00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f35f4005950 tx=0x7f35f400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:25.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:25.823+0000 7f36017fa700 1 -- 172.21.15.2:0/1643955413 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f35f8097360 con 0x7f3604101740 2023-09-13T04:39:25.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:25 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-09-13T04:39:25.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:25 smithi149 ceph-mon[133315]: osdmap e91: 12 total, 12 up, 12 in 2023-09-13T04:39:25.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:25 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-09-13T04:39:25.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:25 smithi156 ceph-mon[133675]: osdmap e91: 12 total, 12 up, 12 in 2023-09-13T04:39:26.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:25 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-09-13T04:39:26.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:25 smithi002 ceph-mon[97651]: osdmap e91: 12 total, 12 up, 12 in 2023-09-13T04:39:26.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.081+0000 7f360a9af700 1 -- 172.21.15.2:0/1643955413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f36040744e0 con 0x7f3604101740 2023-09-13T04:39:26.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.081+0000 7f36017fa700 1 -- 172.21.15.2:0/1643955413 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f35f809a980 con 0x7f3604101740 2023-09-13T04:39:26.083 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:39:26.083 INFO:teuthology.orchestra.run.smithi002.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-09-13T04:39:26.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.084+0000 7f360a9af700 1 -- 172.21.15.2:0/1643955413 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f35ec0a0640 msgr2=0x7f35ec0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:26.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.084+0000 7f360a9af700 1 --2- 172.21.15.2:0/1643955413 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f35ec0a0640 0x7f35ec0a2b00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f35f4005950 tx=0x7f35f400b410 comp rx=0 tx=0).stop 2023-09-13T04:39:26.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.084+0000 7f360a9af700 1 -- 172.21.15.2:0/1643955413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3604101740 msgr2=0x7f36040734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:26.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.084+0000 7f360a9af700 1 --2- 172.21.15.2:0/1643955413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3604101740 0x7f36040734f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009990 tx=0x7f35f8005740 comp rx=0 tx=0).stop 2023-09-13T04:39:26.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.084+0000 7f360a9af700 1 -- 172.21.15.2:0/1643955413 shutdown_connections 2023-09-13T04:39:26.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.084+0000 7f360a9af700 1 --2- 172.21.15.2:0/1643955413 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f35ec0a0640 0x7f35ec0a2b00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:26.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.084+0000 7f360a9af700 1 --2- 172.21.15.2:0/1643955413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3604106d90 0x7f3604073a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:26.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.084+0000 7f360a9af700 1 --2- 172.21.15.2:0/1643955413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3604100d80 0x7f3604072fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:26.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.084+0000 7f360a9af700 1 --2- 172.21.15.2:0/1643955413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3604101740 0x7f36040734f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:26.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.084+0000 7f360a9af700 1 -- 172.21.15.2:0/1643955413 >> 172.21.15.2:0/1643955413 conn(0x7f3604077ea0 msgr2=0x7f36040ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:26.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.084+0000 7f360a9af700 1 -- 172.21.15.2:0/1643955413 shutdown_connections 2023-09-13T04:39:26.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.084+0000 7f360a9af700 1 -- 172.21.15.2:0/1643955413 wait complete. 2023-09-13T04:39:26.677 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-09-13T04:39:26.678 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T04:39:26.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:26 smithi149 ceph-mon[133315]: pgmap v322: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-09-13T04:39:26.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:26 smithi149 ceph-mon[133315]: osdmap e92: 12 total, 12 up, 12 in 2023-09-13T04:39:26.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:26 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1643955413' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-13T04:39:26.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.896+0000 7fd9529ff700 1 -- 172.21.15.2:0/1075928665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd94c108bb0 msgr2=0x7fd94c10b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:26.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.896+0000 7fd9529ff700 1 --2- 172.21.15.2:0/1075928665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd94c108bb0 0x7fd94c10b040 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd94400b0a0 tx=0x7fd94400b3b0 comp rx=0 tx=0).stop 2023-09-13T04:39:26.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.896+0000 7fd9529ff700 1 -- 172.21.15.2:0/1075928665 shutdown_connections 2023-09-13T04:39:26.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.896+0000 7fd9529ff700 1 --2- 172.21.15.2:0/1075928665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd94c108bb0 0x7fd94c10b040 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:26.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.896+0000 7fd9529ff700 1 --2- 172.21.15.2:0/1075928665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd94c074cf0 0x7fd94c073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:26.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.896+0000 7fd9529ff700 1 --2- 172.21.15.2:0/1075928665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd94c073710 0x7fd94c073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:26.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.896+0000 7fd9529ff700 1 -- 172.21.15.2:0/1075928665 >> 172.21.15.2:0/1075928665 conn(0x7fd94c0fc220 msgr2=0x7fd94c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:26.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.896+0000 7fd9529ff700 1 -- 172.21.15.2:0/1075928665 shutdown_connections 2023-09-13T04:39:26.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.896+0000 7fd9529ff700 1 -- 172.21.15.2:0/1075928665 wait complete. 2023-09-13T04:39:26.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.897+0000 7fd9529ff700 1 Processor -- start 2023-09-13T04:39:26.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.897+0000 7fd9529ff700 1 -- start start 2023-09-13T04:39:26.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.897+0000 7fd9529ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd94c073710 0x7fd94c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:26.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.897+0000 7fd9529ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd94c074cf0 0x7fd94c1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:26.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.897+0000 7fd9529ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd94c108bb0 0x7fd94c1a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:26.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.897+0000 7fd9529ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c1a0800 con 0x7fd94c073710 2023-09-13T04:39:26.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.897+0000 7fd9529ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c1a0970 con 0x7fd94c074cf0 2023-09-13T04:39:26.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.897+0000 7fd9529ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c1a0b30 con 0x7fd94c108bb0 2023-09-13T04:39:26.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.898+0000 7fd94bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd94c073710 0x7fd94c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:26.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.898+0000 7fd94b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd94c074cf0 0x7fd94c1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:26.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.898+0000 7fd94bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd94c073710 0x7fd94c198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51576/0 (socket says 172.21.15.2:51576) 2023-09-13T04:39:26.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.898+0000 7fd94bfff700 1 -- 172.21.15.2:0/4290076327 learned_addr learned my addr 172.21.15.2:0/4290076327 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:26.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.898+0000 7fd94b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd94c074cf0 0x7fd94c1991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:48086/0 (socket says 172.21.15.2:48086) 2023-09-13T04:39:26.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.898+0000 7fd950f9c700 1 --2- 172.21.15.2:0/4290076327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd94c108bb0 0x7fd94c1a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.898+0000 7fd94b7fe700 1 -- 172.21.15.2:0/4290076327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd94c108bb0 msgr2=0x7fd94c1a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.898+0000 7fd94b7fe700 1 --2- 172.21.15.2:0/4290076327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd94c108bb0 0x7fd94c1a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.898+0000 7fd94b7fe700 1 -- 172.21.15.2:0/4290076327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd94c073710 msgr2=0x7fd94c198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.898+0000 7fd94b7fe700 1 --2- 172.21.15.2:0/4290076327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd94c073710 0x7fd94c198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.898+0000 7fd94b7fe700 1 -- 172.21.15.2:0/4290076327 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93c009520 con 0x7fd94c074cf0 2023-09-13T04:39:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.898+0000 7fd94bfff700 1 --2- 172.21.15.2:0/4290076327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd94c073710 0x7fd94c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:39:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.898+0000 7fd94b7fe700 1 --2- 172.21.15.2:0/4290076327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd94c074cf0 0x7fd94c1991d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd94000e9c0 tx=0x7fd94000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.899+0000 7fd9497fa700 1 -- 172.21.15.2:0/4290076327 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd940004220 con 0x7fd94c074cf0 2023-09-13T04:39:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.899+0000 7fd9529ff700 1 -- 172.21.15.2:0/4290076327 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd944009d00 con 0x7fd94c074cf0 2023-09-13T04:39:26.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.899+0000 7fd9497fa700 1 -- 172.21.15.2:0/4290076327 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd940004380 con 0x7fd94c074cf0 2023-09-13T04:39:26.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.899+0000 7fd9497fa700 1 -- 172.21.15.2:0/4290076327 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd940003be0 con 0x7fd94c074cf0 2023-09-13T04:39:26.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.899+0000 7fd9529ff700 1 -- 172.21.15.2:0/4290076327 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94c1a1190 con 0x7fd94c074cf0 2023-09-13T04:39:26.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.900+0000 7fd9529ff700 1 -- 172.21.15.2:0/4290076327 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd94c107260 con 0x7fd94c074cf0 2023-09-13T04:39:26.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.902+0000 7fd9497fa700 1 -- 172.21.15.2:0/4290076327 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fd940003d40 con 0x7fd94c074cf0 2023-09-13T04:39:26.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.903+0000 7fd9497fa700 1 --2- 172.21.15.2:0/4290076327 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd9340a0690 0x7fd9340a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:26.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.903+0000 7fd9497fa700 1 -- 172.21.15.2:0/4290076327 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd940014070 con 0x7fd94c074cf0 2023-09-13T04:39:26.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.903+0000 7fd94bfff700 1 --2- 172.21.15.2:0/4290076327 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd9340a0690 0x7fd9340a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:26.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.904+0000 7fd94bfff700 1 --2- 172.21.15.2:0/4290076327 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd9340a0690 0x7fd9340a2b50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd94c199b60 tx=0x7fd93c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:26.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:26.905+0000 7fd9497fa700 1 -- 172.21.15.2:0/4290076327 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd940095ae0 con 0x7fd94c074cf0 2023-09-13T04:39:26.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:26 smithi156 ceph-mon[133675]: pgmap v322: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-09-13T04:39:26.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:26 smithi156 ceph-mon[133675]: osdmap e92: 12 total, 12 up, 12 in 2023-09-13T04:39:26.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:26 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1643955413' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-13T04:39:27.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:26 smithi002 ceph-mon[97651]: pgmap v322: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-09-13T04:39:27.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:26 smithi002 ceph-mon[97651]: osdmap e92: 12 total, 12 up, 12 in 2023-09-13T04:39:27.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:26 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1643955413' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-13T04:39:27.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.118+0000 7fd9529ff700 1 -- 172.21.15.2:0/4290076327 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd94c10e730 con 0x7fd94c074cf0 2023-09-13T04:39:27.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.121+0000 7fd9497fa700 1 -- 172.21.15.2:0/4290076327 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24503 (secure 0 0 0) 0x7fd940099100 con 0x7fd94c074cf0 2023-09-13T04:39:27.122 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:39:27.122 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":92,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:39:25.580735+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T04:37:57.094802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T04:37:58.564487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-13T04:38:02.096895+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:39:27.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.123+0000 7fd9529ff700 1 -- 172.21.15.2:0/4290076327 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd9340a0690 msgr2=0x7fd9340a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:27.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.123+0000 7fd9529ff700 1 --2- 172.21.15.2:0/4290076327 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd9340a0690 0x7fd9340a2b50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd94c199b60 tx=0x7fd93c009260 comp rx=0 tx=0).stop 2023-09-13T04:39:27.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.123+0000 7fd9529ff700 1 -- 172.21.15.2:0/4290076327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd94c074cf0 msgr2=0x7fd94c1991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:27.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.123+0000 7fd9529ff700 1 --2- 172.21.15.2:0/4290076327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd94c074cf0 0x7fd94c1991d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd94000e9c0 tx=0x7fd94000ecd0 comp rx=0 tx=0).stop 2023-09-13T04:39:27.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.123+0000 7fd9529ff700 1 -- 172.21.15.2:0/4290076327 shutdown_connections 2023-09-13T04:39:27.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.123+0000 7fd9529ff700 1 --2- 172.21.15.2:0/4290076327 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd9340a0690 0x7fd9340a2b50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:27.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.123+0000 7fd9529ff700 1 --2- 172.21.15.2:0/4290076327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd94c108bb0 0x7fd94c1a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:27.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.123+0000 7fd9529ff700 1 --2- 172.21.15.2:0/4290076327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd94c073710 0x7fd94c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:27.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.123+0000 7fd9529ff700 1 --2- 172.21.15.2:0/4290076327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd94c074cf0 0x7fd94c1991d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:27.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.123+0000 7fd9529ff700 1 -- 172.21.15.2:0/4290076327 >> 172.21.15.2:0/4290076327 conn(0x7fd94c0fc220 msgr2=0x7fd94c0fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:27.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.123+0000 7fd9529ff700 1 -- 172.21.15.2:0/4290076327 shutdown_connections 2023-09-13T04:39:27.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.123+0000 7fd9529ff700 1 -- 172.21.15.2:0/4290076327 wait complete. 2023-09-13T04:39:27.143 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T04:33:42.025714+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-13T04:37:57.094802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-13T04:37:58.564487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-13T04:38:02.096895+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-13T04:39:27.144 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-13T04:39:27.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.354+0000 7fa64b9ff700 1 -- 172.21.15.2:0/3913867325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa644102870 msgr2=0x7fa644109120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:27.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.354+0000 7fa64b9ff700 1 --2- 172.21.15.2:0/3913867325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa644102870 0x7fa644109120 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00b0d0 tx=0x7fa63c00b3e0 comp rx=0 tx=0).stop 2023-09-13T04:39:27.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.354+0000 7fa64b9ff700 1 -- 172.21.15.2:0/3913867325 shutdown_connections 2023-09-13T04:39:27.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.354+0000 7fa64b9ff700 1 --2- 172.21.15.2:0/3913867325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa644102870 0x7fa644109120 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:27.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.354+0000 7fa64b9ff700 1 --2- 172.21.15.2:0/3913867325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa644100ce0 0x7fa644101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:27.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.354+0000 7fa64b9ff700 1 --2- 172.21.15.2:0/3913867325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa644101ed0 0x7fa644102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:27.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.354+0000 7fa64b9ff700 1 -- 172.21.15.2:0/3913867325 >> 172.21.15.2:0/3913867325 conn(0x7fa6440fc230 msgr2=0x7fa6440fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:27.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.354+0000 7fa64b9ff700 1 -- 172.21.15.2:0/3913867325 shutdown_connections 2023-09-13T04:39:27.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.354+0000 7fa64b9ff700 1 -- 172.21.15.2:0/3913867325 wait complete. 2023-09-13T04:39:27.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.355+0000 7fa64b9ff700 1 Processor -- start 2023-09-13T04:39:27.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.356+0000 7fa64b9ff700 1 -- start start 2023-09-13T04:39:27.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.356+0000 7fa64b9ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa644100ce0 0x7fa644196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:27.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.356+0000 7fa64b9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa644101ed0 0x7fa644197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:27.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.356+0000 7fa64b9ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa644102870 0x7fa64419e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:27.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.356+0000 7fa64b9ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64419e680 con 0x7fa644102870 2023-09-13T04:39:27.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.356+0000 7fa64b9ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64419e7c0 con 0x7fa644100ce0 2023-09-13T04:39:27.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.356+0000 7fa64b9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64419e950 con 0x7fa644101ed0 2023-09-13T04:39:27.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.356+0000 7fa649f9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa644102870 0x7fa64419e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:27.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.356+0000 7fa649f9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa644102870 0x7fa64419e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51592/0 (socket says 172.21.15.2:51592) 2023-09-13T04:39:27.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.356+0000 7fa649f9c700 1 -- 172.21.15.2:0/1816772391 learned_addr learned my addr 172.21.15.2:0/1816772391 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:27.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.357+0000 7fa64979b700 1 --2- 172.21.15.2:0/1816772391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa644100ce0 0x7fa644196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:27.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.357+0000 7fa648f9a700 1 --2- 172.21.15.2:0/1816772391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa644101ed0 0x7fa644197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:27.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.357+0000 7fa649f9c700 1 -- 172.21.15.2:0/1816772391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa644101ed0 msgr2=0x7fa644197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:27.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.357+0000 7fa649f9c700 1 --2- 172.21.15.2:0/1816772391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa644101ed0 0x7fa644197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:27.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.357+0000 7fa649f9c700 1 -- 172.21.15.2:0/1816772391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa644100ce0 msgr2=0x7fa644196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:27.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.357+0000 7fa649f9c700 1 --2- 172.21.15.2:0/1816772391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa644100ce0 0x7fa644196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:27.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.357+0000 7fa649f9c700 1 -- 172.21.15.2:0/1816772391 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c009d00 con 0x7fa644102870 2023-09-13T04:39:27.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.357+0000 7fa648f9a700 1 --2- 172.21.15.2:0/1816772391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa644101ed0 0x7fa644197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:39:27.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.357+0000 7fa649f9c700 1 --2- 172.21.15.2:0/1816772391 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa644102870 0x7fa64419e0b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa63c008eb0 tx=0x7fa63c008ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:27.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.357+0000 7fa63a7fc700 1 -- 172.21.15.2:0/1816772391 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c00e040 con 0x7fa644102870 2023-09-13T04:39:27.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.357+0000 7fa63a7fc700 1 -- 172.21.15.2:0/1816772391 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa63c00be30 con 0x7fa644102870 2023-09-13T04:39:27.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.357+0000 7fa63a7fc700 1 -- 172.21.15.2:0/1816772391 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c01d470 con 0x7fa644102870 2023-09-13T04:39:27.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.357+0000 7fa64b9ff700 1 -- 172.21.15.2:0/1816772391 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64419ebd0 con 0x7fa644102870 2023-09-13T04:39:27.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.358+0000 7fa64b9ff700 1 -- 172.21.15.2:0/1816772391 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64419f000 con 0x7fa644102870 2023-09-13T04:39:27.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.358+0000 7fa64b9ff700 1 -- 172.21.15.2:0/1816772391 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa644107c10 con 0x7fa644102870 2023-09-13T04:39:27.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.362+0000 7fa63a7fc700 1 -- 172.21.15.2:0/1816772391 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fa63c005190 con 0x7fa644102870 2023-09-13T04:39:27.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.363+0000 7fa63a7fc700 1 --2- 172.21.15.2:0/1816772391 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fa6300a0640 0x7fa6300a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:27.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.364+0000 7fa63a7fc700 1 -- 172.21.15.2:0/1816772391 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa63c0cd520 con 0x7fa644102870 2023-09-13T04:39:27.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.364+0000 7fa64979b700 1 --2- 172.21.15.2:0/1816772391 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fa6300a0640 0x7fa6300a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:27.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.364+0000 7fa64979b700 1 --2- 172.21.15.2:0/1816772391 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fa6300a0640 0x7fa6300a2b00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa644101d30 tx=0x7fa640009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:27.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.366+0000 7fa63a7fc700 1 -- 172.21.15.2:0/1816772391 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa63c096d50 con 0x7fa644102870 2023-09-13T04:39:27.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.579+0000 7fa64b9ff700 1 -- 172.21.15.2:0/1816772391 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa64410c760 con 0x7fa644102870 2023-09-13T04:39:27.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.579+0000 7fa63a7fc700 1 -- 172.21.15.2:0/1816772391 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa63c09a370 con 0x7fa644102870 2023-09-13T04:39:27.581 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-09-13T04:39:27.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.581+0000 7fa64b9ff700 1 -- 172.21.15.2:0/1816772391 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fa6300a0640 msgr2=0x7fa6300a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:27.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.581+0000 7fa64b9ff700 1 --2- 172.21.15.2:0/1816772391 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fa6300a0640 0x7fa6300a2b00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa644101d30 tx=0x7fa640009260 comp rx=0 tx=0).stop 2023-09-13T04:39:27.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.581+0000 7fa64b9ff700 1 -- 172.21.15.2:0/1816772391 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa644102870 msgr2=0x7fa64419e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:27.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.581+0000 7fa64b9ff700 1 --2- 172.21.15.2:0/1816772391 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa644102870 0x7fa64419e0b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa63c008eb0 tx=0x7fa63c008ee0 comp rx=0 tx=0).stop 2023-09-13T04:39:27.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.582+0000 7fa64b9ff700 1 -- 172.21.15.2:0/1816772391 shutdown_connections 2023-09-13T04:39:27.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.582+0000 7fa64b9ff700 1 --2- 172.21.15.2:0/1816772391 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fa6300a0640 0x7fa6300a2b00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:27.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.582+0000 7fa64b9ff700 1 --2- 172.21.15.2:0/1816772391 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa644102870 0x7fa64419e0b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:27.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.582+0000 7fa64b9ff700 1 --2- 172.21.15.2:0/1816772391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa644100ce0 0x7fa644196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:27.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.582+0000 7fa64b9ff700 1 --2- 172.21.15.2:0/1816772391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa644101ed0 0x7fa644197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:27.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.582+0000 7fa64b9ff700 1 -- 172.21.15.2:0/1816772391 >> 172.21.15.2:0/1816772391 conn(0x7fa6440fc230 msgr2=0x7fa6440fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:27.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.582+0000 7fa64b9ff700 1 -- 172.21.15.2:0/1816772391 shutdown_connections 2023-09-13T04:39:27.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.582+0000 7fa64b9ff700 1 -- 172.21.15.2:0/1816772391 wait complete. 2023-09-13T04:39:27.603 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-13T04:39:27.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.809+0000 7f821a1ff700 1 -- 172.21.15.2:0/661932365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8214074cf0 msgr2=0x7f8214073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:27.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.809+0000 7f821a1ff700 1 --2- 172.21.15.2:0/661932365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8214074cf0 0x7f8214073140 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8204009990 tx=0x7f8204009ca0 comp rx=0 tx=0).stop 2023-09-13T04:39:27.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.809+0000 7f821a1ff700 1 -- 172.21.15.2:0/661932365 shutdown_connections 2023-09-13T04:39:27.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.809+0000 7f821a1ff700 1 --2- 172.21.15.2:0/661932365 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8214108ba0 0x7f821410b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:27.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.809+0000 7f821a1ff700 1 --2- 172.21.15.2:0/661932365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8214074cf0 0x7f8214073140 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:27.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.809+0000 7f821a1ff700 1 --2- 172.21.15.2:0/661932365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8214073710 0x7f8214073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:27.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.809+0000 7f821a1ff700 1 -- 172.21.15.2:0/661932365 >> 172.21.15.2:0/661932365 conn(0x7f82140fc230 msgr2=0x7f82140fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:27.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.809+0000 7f821a1ff700 1 -- 172.21.15.2:0/661932365 shutdown_connections 2023-09-13T04:39:27.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.809+0000 7f821a1ff700 1 -- 172.21.15.2:0/661932365 wait complete. 2023-09-13T04:39:27.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.810+0000 7f821a1ff700 1 Processor -- start 2023-09-13T04:39:27.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.810+0000 7f821a1ff700 1 -- start start 2023-09-13T04:39:27.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.810+0000 7f821a1ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8214073710 0x7f8214198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:27.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.810+0000 7f821a1ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8214074cf0 0x7f8214199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:27.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.810+0000 7f821a1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8214108ba0 0x7f82141a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:27.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.811+0000 7f821a1ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82141a0770 con 0x7f8214073710 2023-09-13T04:39:27.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.811+0000 7f821a1ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82141a08b0 con 0x7f8214074cf0 2023-09-13T04:39:27.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.811+0000 7f821a1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82141a0a40 con 0x7f8214108ba0 2023-09-13T04:39:27.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.811+0000 7f82137fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8214073710 0x7f8214198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:27.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.811+0000 7f8212ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8214074cf0 0x7f8214199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:27.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.811+0000 7f82137fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8214073710 0x7f8214198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51614/0 (socket says 172.21.15.2:51614) 2023-09-13T04:39:27.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.811+0000 7f82137fe700 1 -- 172.21.15.2:0/748764899 learned_addr learned my addr 172.21.15.2:0/748764899 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:27.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.811+0000 7f8213fff700 1 --2- 172.21.15.2:0/748764899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8214108ba0 0x7f82141a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:27.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.811+0000 7f82137fe700 1 -- 172.21.15.2:0/748764899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8214108ba0 msgr2=0x7f82141a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:27.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.811+0000 7f82137fe700 1 --2- 172.21.15.2:0/748764899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8214108ba0 0x7f82141a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:27.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.811+0000 7f82137fe700 1 -- 172.21.15.2:0/748764899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8214074cf0 msgr2=0x7f8214199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:27.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.811+0000 7f82137fe700 1 --2- 172.21.15.2:0/748764899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8214074cf0 0x7f8214199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:27.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.811+0000 7f82137fe700 1 -- 172.21.15.2:0/748764899 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82040095f0 con 0x7f8214073710 2023-09-13T04:39:27.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.811+0000 7f8212ffd700 1 --2- 172.21.15.2:0/748764899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8214074cf0 0x7f8214199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:39:27.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.811+0000 7f82137fe700 1 --2- 172.21.15.2:0/748764899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8214073710 0x7f8214198bf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8204004730 tx=0x7f8204004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:27.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.812+0000 7f8210ff9700 1 -- 172.21.15.2:0/748764899 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820401d070 con 0x7f8214073710 2023-09-13T04:39:27.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.812+0000 7f821a1ff700 1 -- 172.21.15.2:0/748764899 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82141a0cc0 con 0x7f8214073710 2023-09-13T04:39:27.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.812+0000 7f8210ff9700 1 -- 172.21.15.2:0/748764899 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8204004c90 con 0x7f8214073710 2023-09-13T04:39:27.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.812+0000 7f8210ff9700 1 -- 172.21.15.2:0/748764899 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82040054b0 con 0x7f8214073710 2023-09-13T04:39:27.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.812+0000 7f821a1ff700 1 -- 172.21.15.2:0/748764899 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82141a10e0 con 0x7f8214073710 2023-09-13T04:39:27.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.813+0000 7f821a1ff700 1 -- 172.21.15.2:0/748764899 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8214107250 con 0x7f8214073710 2023-09-13T04:39:27.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.815+0000 7f8210ff9700 1 -- 172.21.15.2:0/748764899 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f820400fbc0 con 0x7f8214073710 2023-09-13T04:39:27.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.816+0000 7f8210ff9700 1 --2- 172.21.15.2:0/748764899 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f81fc0a06c0 0x7f81fc0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:27.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.816+0000 7f8210ff9700 1 -- 172.21.15.2:0/748764899 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f82040cd120 con 0x7f8214073710 2023-09-13T04:39:27.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.817+0000 7f8212ffd700 1 --2- 172.21.15.2:0/748764899 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f81fc0a06c0 0x7f81fc0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:27.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.817+0000 7f8212ffd700 1 --2- 172.21.15.2:0/748764899 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f81fc0a06c0 0x7f81fc0a2b80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8214074a10 tx=0x7f8208009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:27.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:27.820+0000 7f8210ff9700 1 -- 172.21.15.2:0/748764899 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f820405ae10 con 0x7f8214073710 2023-09-13T04:39:27.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:27 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/4290076327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:39:27.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:27 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1816772391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:39:27.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:27 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/4290076327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:39:27.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:27 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1816772391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:39:28.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:27 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/4290076327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:39:28.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:27 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1816772391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:39:28.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.032+0000 7f821a1ff700 1 -- 172.21.15.2:0/748764899 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f821410e720 con 0x7f8214073710 2023-09-13T04:39:28.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.032+0000 7f8210ff9700 1 -- 172.21.15.2:0/748764899 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8204096950 con 0x7f8214073710 2023-09-13T04:39:28.034 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-09-13T04:39:28.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.035+0000 7f821a1ff700 1 -- 172.21.15.2:0/748764899 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f81fc0a06c0 msgr2=0x7f81fc0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:28.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.035+0000 7f821a1ff700 1 --2- 172.21.15.2:0/748764899 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f81fc0a06c0 0x7f81fc0a2b80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8214074a10 tx=0x7f8208009260 comp rx=0 tx=0).stop 2023-09-13T04:39:28.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.035+0000 7f821a1ff700 1 -- 172.21.15.2:0/748764899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8214073710 msgr2=0x7f8214198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:28.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.035+0000 7f821a1ff700 1 --2- 172.21.15.2:0/748764899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8214073710 0x7f8214198bf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8204004730 tx=0x7f8204004760 comp rx=0 tx=0).stop 2023-09-13T04:39:28.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.035+0000 7f821a1ff700 1 -- 172.21.15.2:0/748764899 shutdown_connections 2023-09-13T04:39:28.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.035+0000 7f821a1ff700 1 --2- 172.21.15.2:0/748764899 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f81fc0a06c0 0x7f81fc0a2b80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.035+0000 7f821a1ff700 1 --2- 172.21.15.2:0/748764899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8214108ba0 0x7f82141a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.035+0000 7f821a1ff700 1 --2- 172.21.15.2:0/748764899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8214073710 0x7f8214198bf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.035+0000 7f821a1ff700 1 --2- 172.21.15.2:0/748764899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8214074cf0 0x7f8214199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.035+0000 7f821a1ff700 1 -- 172.21.15.2:0/748764899 >> 172.21.15.2:0/748764899 conn(0x7f82140fc230 msgr2=0x7f82140fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:28.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.035+0000 7f821a1ff700 1 -- 172.21.15.2:0/748764899 shutdown_connections 2023-09-13T04:39:28.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.035+0000 7f821a1ff700 1 -- 172.21.15.2:0/748764899 wait complete. 2023-09-13T04:39:28.055 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-13T04:39:28.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.262+0000 7fbbbd5ff700 1 -- 172.21.15.2:0/2902029775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbb8101740 msgr2=0x7fbbb8103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:28.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.262+0000 7fbbbd5ff700 1 --2- 172.21.15.2:0/2902029775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbb8101740 0x7fbbb8103b30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbba8009990 tx=0x7fbba8009ca0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.262+0000 7fbbbd5ff700 1 -- 172.21.15.2:0/2902029775 shutdown_connections 2023-09-13T04:39:28.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.262+0000 7fbbbd5ff700 1 --2- 172.21.15.2:0/2902029775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbb81069a0 0x7fbbb8108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.262+0000 7fbbbd5ff700 1 --2- 172.21.15.2:0/2902029775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbb8101740 0x7fbbb8103b30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.262+0000 7fbbbd5ff700 1 --2- 172.21.15.2:0/2902029775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb8104070 0x7fbbb8106460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.262+0000 7fbbbd5ff700 1 -- 172.21.15.2:0/2902029775 >> 172.21.15.2:0/2902029775 conn(0x7fbbb80fb130 msgr2=0x7fbbb80fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:28.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.262+0000 7fbbbd5ff700 1 -- 172.21.15.2:0/2902029775 shutdown_connections 2023-09-13T04:39:28.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.262+0000 7fbbbd5ff700 1 -- 172.21.15.2:0/2902029775 wait complete. 2023-09-13T04:39:28.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.263+0000 7fbbbd5ff700 1 Processor -- start 2023-09-13T04:39:28.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.263+0000 7fbbbd5ff700 1 -- start start 2023-09-13T04:39:28.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.263+0000 7fbbbd5ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb8101740 0x7fbbb8198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:28.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.264+0000 7fbbbd5ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbb8104070 0x7fbbb8199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:28.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.264+0000 7fbbbd5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbb81069a0 0x7fbbb81a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:28.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.264+0000 7fbbbd5ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb81a0770 con 0x7fbbb8104070 2023-09-13T04:39:28.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.264+0000 7fbbbd5ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb81a08e0 con 0x7fbbb8101740 2023-09-13T04:39:28.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.264+0000 7fbbbd5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb81a0aa0 con 0x7fbbb81069a0 2023-09-13T04:39:28.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.264+0000 7fbbb67fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbb8104070 0x7fbbb8199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:28.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.264+0000 7fbbb67fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbb8104070 0x7fbbb8199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51632/0 (socket says 172.21.15.2:51632) 2023-09-13T04:39:28.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.264+0000 7fbbb67fc700 1 -- 172.21.15.2:0/3537429860 learned_addr learned my addr 172.21.15.2:0/3537429860 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:28.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.264+0000 7fbbb6ffd700 1 --2- 172.21.15.2:0/3537429860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb8101740 0x7fbbb8198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:28.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.264+0000 7fbbb77fe700 1 --2- 172.21.15.2:0/3537429860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbb81069a0 0x7fbbb81a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:28.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.264+0000 7fbbb67fc700 1 -- 172.21.15.2:0/3537429860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbb81069a0 msgr2=0x7fbbb81a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:28.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.264+0000 7fbbb67fc700 1 --2- 172.21.15.2:0/3537429860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbb81069a0 0x7fbbb81a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.264+0000 7fbbb67fc700 1 -- 172.21.15.2:0/3537429860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb8101740 msgr2=0x7fbbb8198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:28.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.264+0000 7fbbb67fc700 1 --2- 172.21.15.2:0/3537429860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb8101740 0x7fbbb8198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.264+0000 7fbbb67fc700 1 -- 172.21.15.2:0/3537429860 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba80095f0 con 0x7fbbb8104070 2023-09-13T04:39:28.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.265+0000 7fbbb67fc700 1 --2- 172.21.15.2:0/3537429860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbb8104070 0x7fbbb8199170 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbbac00e9a0 tx=0x7fbbac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:28.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.265+0000 7fbb9ffff700 1 -- 172.21.15.2:0/3537429860 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac009790 con 0x7fbbb8104070 2023-09-13T04:39:28.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.265+0000 7fbb9ffff700 1 -- 172.21.15.2:0/3537429860 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbac010430 con 0x7fbbb8104070 2023-09-13T04:39:28.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.265+0000 7fbbbd5ff700 1 -- 172.21.15.2:0/3537429860 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb81a0d80 con 0x7fbbb8104070 2023-09-13T04:39:28.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.265+0000 7fbb9ffff700 1 -- 172.21.15.2:0/3537429860 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac018680 con 0x7fbbb8104070 2023-09-13T04:39:28.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.265+0000 7fbbbd5ff700 1 -- 172.21.15.2:0/3537429860 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb81a1220 con 0x7fbbb8104070 2023-09-13T04:39:28.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.266+0000 7fbbbd5ff700 1 -- 172.21.15.2:0/3537429860 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb8108570 con 0x7fbbb8104070 2023-09-13T04:39:28.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.268+0000 7fbb9ffff700 1 -- 172.21.15.2:0/3537429860 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fbbac0105a0 con 0x7fbbb8104070 2023-09-13T04:39:28.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.269+0000 7fbb9ffff700 1 --2- 172.21.15.2:0/3537429860 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fbba00a06c0 0x7fbba00a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:28.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.269+0000 7fbb9ffff700 1 -- 172.21.15.2:0/3537429860 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbbac014070 con 0x7fbbb8104070 2023-09-13T04:39:28.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.270+0000 7fbbb6ffd700 1 --2- 172.21.15.2:0/3537429860 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fbba00a06c0 0x7fbba00a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:28.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.270+0000 7fbbb6ffd700 1 --2- 172.21.15.2:0/3537429860 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fbba00a06c0 0x7fbba00a2b80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbba800b2e0 tx=0x7fbba8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:28.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.273+0000 7fbb9ffff700 1 -- 172.21.15.2:0/3537429860 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fbbac051e10 con 0x7fbbb8104070 2023-09-13T04:39:28.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.485+0000 7fbbbd5ff700 1 -- 172.21.15.2:0/3537429860 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbbb802cd70 con 0x7fbbb8104070 2023-09-13T04:39:28.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.485+0000 7fbb9ffff700 1 -- 172.21.15.2:0/3537429860 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbbac095810 con 0x7fbbb8104070 2023-09-13T04:39:28.487 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-09-13T04:39:28.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.488+0000 7fbbbd5ff700 1 -- 172.21.15.2:0/3537429860 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fbba00a06c0 msgr2=0x7fbba00a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:28.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.488+0000 7fbbbd5ff700 1 --2- 172.21.15.2:0/3537429860 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fbba00a06c0 0x7fbba00a2b80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbba800b2e0 tx=0x7fbba8005fb0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.488+0000 7fbbbd5ff700 1 -- 172.21.15.2:0/3537429860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbb8104070 msgr2=0x7fbbb8199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:28.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.488+0000 7fbbbd5ff700 1 --2- 172.21.15.2:0/3537429860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbb8104070 0x7fbbb8199170 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbbac00e9a0 tx=0x7fbbac00ecb0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.488+0000 7fbbbd5ff700 1 -- 172.21.15.2:0/3537429860 shutdown_connections 2023-09-13T04:39:28.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.488+0000 7fbbbd5ff700 1 --2- 172.21.15.2:0/3537429860 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fbba00a06c0 0x7fbba00a2b80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.488+0000 7fbbbd5ff700 1 --2- 172.21.15.2:0/3537429860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbb81069a0 0x7fbbb81a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.488+0000 7fbbbd5ff700 1 --2- 172.21.15.2:0/3537429860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb8101740 0x7fbbb8198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.488+0000 7fbbbd5ff700 1 --2- 172.21.15.2:0/3537429860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbb8104070 0x7fbbb8199170 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.488+0000 7fbbbd5ff700 1 -- 172.21.15.2:0/3537429860 >> 172.21.15.2:0/3537429860 conn(0x7fbbb80fb130 msgr2=0x7fbbb80ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:28.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.488+0000 7fbbbd5ff700 1 -- 172.21.15.2:0/3537429860 shutdown_connections 2023-09-13T04:39:28.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.488+0000 7fbbbd5ff700 1 -- 172.21.15.2:0/3537429860 wait complete. 2023-09-13T04:39:28.508 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-13T04:39:28.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.716+0000 7efd153ff700 1 -- 172.21.15.2:0/3075999132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd10100cf0 msgr2=0x7efd10101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:28.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.716+0000 7efd153ff700 1 --2- 172.21.15.2:0/3075999132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd10100cf0 0x7efd10101110 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efd000099c0 tx=0x7efd00009cd0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.716+0000 7efd153ff700 1 -- 172.21.15.2:0/3075999132 shutdown_connections 2023-09-13T04:39:28.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.716+0000 7efd153ff700 1 --2- 172.21.15.2:0/3075999132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd10102880 0x7efd10109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.716+0000 7efd153ff700 1 --2- 172.21.15.2:0/3075999132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd10100cf0 0x7efd10101110 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.716+0000 7efd153ff700 1 --2- 172.21.15.2:0/3075999132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd10101ee0 0x7efd10102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.716+0000 7efd153ff700 1 -- 172.21.15.2:0/3075999132 >> 172.21.15.2:0/3075999132 conn(0x7efd100fc260 msgr2=0x7efd100fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:28.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.716+0000 7efd153ff700 1 -- 172.21.15.2:0/3075999132 shutdown_connections 2023-09-13T04:39:28.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.716+0000 7efd153ff700 1 -- 172.21.15.2:0/3075999132 wait complete. 2023-09-13T04:39:28.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.718+0000 7efd153ff700 1 Processor -- start 2023-09-13T04:39:28.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.718+0000 7efd153ff700 1 -- start start 2023-09-13T04:39:28.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.718+0000 7efd153ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd10100cf0 0x7efd10198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:28.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.719+0000 7efd153ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd10101ee0 0x7efd10199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:28.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.719+0000 7efd153ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd10102880 0x7efd101a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:28.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.719+0000 7efd153ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd101a0790 con 0x7efd10102880 2023-09-13T04:39:28.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.719+0000 7efd153ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd101a0900 con 0x7efd10101ee0 2023-09-13T04:39:28.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.719+0000 7efd153ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd101a0ac0 con 0x7efd10100cf0 2023-09-13T04:39:28.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.719+0000 7efd0effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd10100cf0 0x7efd10198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:28.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.719+0000 7efd0e7fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd10101ee0 0x7efd10199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:28.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.719+0000 7efd0f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd10102880 0x7efd101a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:28.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.719+0000 7efd0e7fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd10101ee0 0x7efd10199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:48148/0 (socket says 172.21.15.2:48148) 2023-09-13T04:39:28.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.719+0000 7efd0e7fc700 1 -- 172.21.15.2:0/2494645786 learned_addr learned my addr 172.21.15.2:0/2494645786 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:28.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.719+0000 7efd0e7fc700 1 -- 172.21.15.2:0/2494645786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd10100cf0 msgr2=0x7efd10198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:28.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.719+0000 7efd0e7fc700 1 --2- 172.21.15.2:0/2494645786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd10100cf0 0x7efd10198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.719+0000 7efd0e7fc700 1 -- 172.21.15.2:0/2494645786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd10102880 msgr2=0x7efd101a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:28.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.719+0000 7efd0e7fc700 1 --2- 172.21.15.2:0/2494645786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd10102880 0x7efd101a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.719+0000 7efd0e7fc700 1 -- 172.21.15.2:0/2494645786 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd000095f0 con 0x7efd10101ee0 2023-09-13T04:39:28.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.720+0000 7efd0e7fc700 1 --2- 172.21.15.2:0/2494645786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd10101ee0 0x7efd10199190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efd0400e9a0 tx=0x7efd0400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:28.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.720+0000 7efcf7fff700 1 -- 172.21.15.2:0/2494645786 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd04009790 con 0x7efd10101ee0 2023-09-13T04:39:28.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.720+0000 7efd153ff700 1 -- 172.21.15.2:0/2494645786 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd101a0da0 con 0x7efd10101ee0 2023-09-13T04:39:28.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.720+0000 7efcf7fff700 1 -- 172.21.15.2:0/2494645786 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd04010430 con 0x7efd10101ee0 2023-09-13T04:39:28.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.720+0000 7efcf7fff700 1 -- 172.21.15.2:0/2494645786 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd04018680 con 0x7efd10101ee0 2023-09-13T04:39:28.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.720+0000 7efd153ff700 1 -- 172.21.15.2:0/2494645786 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd101a1210 con 0x7efd10101ee0 2023-09-13T04:39:28.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.721+0000 7efd153ff700 1 -- 172.21.15.2:0/2494645786 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd10193340 con 0x7efd10101ee0 2023-09-13T04:39:28.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.723+0000 7efcf7fff700 1 -- 172.21.15.2:0/2494645786 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7efd040105a0 con 0x7efd10101ee0 2023-09-13T04:39:28.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.724+0000 7efcf7fff700 1 --2- 172.21.15.2:0/2494645786 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7efcf80a06c0 0x7efcf80a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:28.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.725+0000 7efcf7fff700 1 -- 172.21.15.2:0/2494645786 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7efd04014070 con 0x7efd10101ee0 2023-09-13T04:39:28.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.725+0000 7efd0effd700 1 --2- 172.21.15.2:0/2494645786 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7efcf80a06c0 0x7efcf80a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:28.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.725+0000 7efd0effd700 1 --2- 172.21.15.2:0/2494645786 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7efcf80a06c0 0x7efcf80a2b80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efd00009990 tx=0x7efd0001a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:28.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.727+0000 7efcf7fff700 1 -- 172.21.15.2:0/2494645786 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7efd04095a80 con 0x7efd10101ee0 2023-09-13T04:39:28.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:28 smithi149 ceph-mon[133315]: pgmap v324: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-13T04:39:28.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:28 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/748764899' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:39:28.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:28 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3537429860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:39:28.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:28 smithi156 ceph-mon[133675]: pgmap v324: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-13T04:39:28.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:28 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/748764899' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:39:28.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:28 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3537429860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:39:28.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.939+0000 7efd153ff700 1 -- 172.21.15.2:0/2494645786 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7efd1002cd70 con 0x7efd10101ee0 2023-09-13T04:39:28.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.940+0000 7efcf7fff700 1 -- 172.21.15.2:0/2494645786 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7efd04026020 con 0x7efd10101ee0 2023-09-13T04:39:28.941 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-09-13T04:39:28.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.942+0000 7efd153ff700 1 -- 172.21.15.2:0/2494645786 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7efcf80a06c0 msgr2=0x7efcf80a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:28.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.942+0000 7efd153ff700 1 --2- 172.21.15.2:0/2494645786 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7efcf80a06c0 0x7efcf80a2b80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efd00009990 tx=0x7efd0001a040 comp rx=0 tx=0).stop 2023-09-13T04:39:28.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.942+0000 7efd153ff700 1 -- 172.21.15.2:0/2494645786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd10101ee0 msgr2=0x7efd10199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:28.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.942+0000 7efd153ff700 1 --2- 172.21.15.2:0/2494645786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd10101ee0 0x7efd10199190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efd0400e9a0 tx=0x7efd0400ecb0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.942+0000 7efd153ff700 1 -- 172.21.15.2:0/2494645786 shutdown_connections 2023-09-13T04:39:28.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.942+0000 7efd153ff700 1 --2- 172.21.15.2:0/2494645786 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7efcf80a06c0 0x7efcf80a2b80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.942+0000 7efd153ff700 1 --2- 172.21.15.2:0/2494645786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd10102880 0x7efd101a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.942+0000 7efd153ff700 1 --2- 172.21.15.2:0/2494645786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd10100cf0 0x7efd10198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.942+0000 7efd153ff700 1 --2- 172.21.15.2:0/2494645786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd10101ee0 0x7efd10199190 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:28.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.942+0000 7efd153ff700 1 -- 172.21.15.2:0/2494645786 >> 172.21.15.2:0/2494645786 conn(0x7efd100fc260 msgr2=0x7efd10107820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:28.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.942+0000 7efd153ff700 1 -- 172.21.15.2:0/2494645786 shutdown_connections 2023-09-13T04:39:28.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:28.942+0000 7efd153ff700 1 -- 172.21.15.2:0/2494645786 wait complete. 2023-09-13T04:39:28.960 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:28 smithi002 ceph-mon[97651]: pgmap v324: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-13T04:39:28.960 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:28 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/748764899' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:39:28.960 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:28 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3537429860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:39:28.962 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:39:29.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.166+0000 7f1e807ff700 1 -- 172.21.15.2:0/2882111246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e78073710 msgr2=0x7f1e78073b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:29.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.166+0000 7f1e807ff700 1 --2- 172.21.15.2:0/2882111246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e78073710 0x7f1e78073b90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1e680098f0 tx=0x7f1e68009c00 comp rx=0 tx=0).stop 2023-09-13T04:39:29.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.166+0000 7f1e807ff700 1 -- 172.21.15.2:0/2882111246 shutdown_connections 2023-09-13T04:39:29.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.166+0000 7f1e807ff700 1 --2- 172.21.15.2:0/2882111246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e78108bb0 0x7f1e7810b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:29.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.166+0000 7f1e807ff700 1 --2- 172.21.15.2:0/2882111246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e78074cf0 0x7f1e78073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:29.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.166+0000 7f1e807ff700 1 --2- 172.21.15.2:0/2882111246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e78073710 0x7f1e78073b90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:29.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.166+0000 7f1e807ff700 1 -- 172.21.15.2:0/2882111246 >> 172.21.15.2:0/2882111246 conn(0x7f1e780fc220 msgr2=0x7f1e780fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:29.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.166+0000 7f1e807ff700 1 -- 172.21.15.2:0/2882111246 shutdown_connections 2023-09-13T04:39:29.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.166+0000 7f1e807ff700 1 -- 172.21.15.2:0/2882111246 wait complete. 2023-09-13T04:39:29.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.167+0000 7f1e807ff700 1 Processor -- start 2023-09-13T04:39:29.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.167+0000 7f1e807ff700 1 -- start start 2023-09-13T04:39:29.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.168+0000 7f1e807ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e78073710 0x7f1e78198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:29.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.168+0000 7f1e807ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e78074cf0 0x7f1e781991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:29.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.168+0000 7f1e807ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e78108bb0 0x7f1e781a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:29.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.168+0000 7f1e807ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e781a07b0 con 0x7f1e78108bb0 2023-09-13T04:39:29.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.168+0000 7f1e807ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e781a0920 con 0x7f1e78074cf0 2023-09-13T04:39:29.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.168+0000 7f1e807ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e781a0ae0 con 0x7f1e78073710 2023-09-13T04:39:29.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.168+0000 7f1e7e59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e78073710 0x7f1e78198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:29.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.168+0000 7f1e7e59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e78073710 0x7f1e78198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:54054/0 (socket says 172.21.15.2:54054) 2023-09-13T04:39:29.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.168+0000 7f1e7ed9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e78108bb0 0x7f1e781a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:29.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.168+0000 7f1e7e59b700 1 -- 172.21.15.2:0/2891632731 learned_addr learned my addr 172.21.15.2:0/2891632731 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:29.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.168+0000 7f1e7dd9a700 1 --2- 172.21.15.2:0/2891632731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e78074cf0 0x7f1e781991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:29.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.168+0000 7f1e7e59b700 1 -- 172.21.15.2:0/2891632731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e78074cf0 msgr2=0x7f1e781991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:29.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.168+0000 7f1e7e59b700 1 --2- 172.21.15.2:0/2891632731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e78074cf0 0x7f1e781991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:29.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.168+0000 7f1e7e59b700 1 -- 172.21.15.2:0/2891632731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e78108bb0 msgr2=0x7f1e781a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:29.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.168+0000 7f1e7e59b700 1 --2- 172.21.15.2:0/2891632731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e78108bb0 0x7f1e781a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:29.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.168+0000 7f1e7e59b700 1 -- 172.21.15.2:0/2891632731 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e68009520 con 0x7f1e78073710 2023-09-13T04:39:29.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.169+0000 7f1e7e59b700 1 --2- 172.21.15.2:0/2891632731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e78073710 0x7f1e78198c70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400e9a0 tx=0x7f1e7400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:29.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.169+0000 7f1e6f7fe700 1 -- 172.21.15.2:0/2891632731 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e74009790 con 0x7f1e78073710 2023-09-13T04:39:29.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.169+0000 7f1e6f7fe700 1 -- 172.21.15.2:0/2891632731 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e74010430 con 0x7f1e78073710 2023-09-13T04:39:29.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.169+0000 7f1e6f7fe700 1 -- 172.21.15.2:0/2891632731 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e74018680 con 0x7f1e78073710 2023-09-13T04:39:29.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.169+0000 7f1e807ff700 1 -- 172.21.15.2:0/2891632731 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e781a0dc0 con 0x7f1e78073710 2023-09-13T04:39:29.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.169+0000 7f1e807ff700 1 -- 172.21.15.2:0/2891632731 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e781a11c0 con 0x7f1e78073710 2023-09-13T04:39:29.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.170+0000 7f1e807ff700 1 -- 172.21.15.2:0/2891632731 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e78107260 con 0x7f1e78073710 2023-09-13T04:39:29.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.173+0000 7f1e6f7fe700 1 -- 172.21.15.2:0/2891632731 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f1e740105a0 con 0x7f1e78073710 2023-09-13T04:39:29.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.174+0000 7f1e6f7fe700 1 --2- 172.21.15.2:0/2891632731 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1e640a0710 0x7f1e640a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:29.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.174+0000 7f1e6f7fe700 1 -- 172.21.15.2:0/2891632731 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1e74014070 con 0x7f1e78073710 2023-09-13T04:39:29.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.174+0000 7f1e7dd9a700 1 --2- 172.21.15.2:0/2891632731 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1e640a0710 0x7f1e640a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:29.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.175+0000 7f1e7dd9a700 1 --2- 172.21.15.2:0/2891632731 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1e640a0710 0x7f1e640a2bd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009fd0 tx=0x7f1e68019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:29.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.176+0000 7f1e6f7fe700 1 -- 172.21.15.2:0/2891632731 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1e74095a80 con 0x7f1e78073710 2023-09-13T04:39:29.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.431+0000 7f1e807ff700 1 -- 172.21.15.2:0/2891632731 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e7810e810 con 0x7f1e78073710 2023-09-13T04:39:29.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.433+0000 7f1e6f7fe700 1 -- 172.21.15.2:0/2891632731 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11303 (secure 0 0 0) 0x7f1e74026020 con 0x7f1e78073710 2023-09-13T04:39:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:39:29.434 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2469307307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2469307307},{"type":"v1","addr":"172.21.15.2:6839","nonce":2469307307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T04:37:59.110701+0000","modified":"2023-09-13T04:38:26.610260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24554,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/846579077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":846579077},{"type":"v1","addr":"172.21.15.156:6835","nonce":846579077}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1896833690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1896833690},{"type":"v1","addr":"172.21.15.149:6837","nonce":1896833690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/524563090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":524563090},{"type":"v1","addr":"172.21.15.156:6837","nonce":524563090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6839/1785833227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1785833227},{"type":"v1","addr":"172.21.15.156:6839","nonce":1785833227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.2:6841/482545391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":482545391},{"type":"v1","addr":"172.21.15.2:6841","nonce":482545391}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/389167841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":389167841},{"type":"v1","addr":"172.21.15.149:6839","nonce":389167841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T04:39:29.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.435+0000 7f1e807ff700 1 -- 172.21.15.2:0/2891632731 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1e640a0710 msgr2=0x7f1e640a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:29.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.435+0000 7f1e807ff700 1 --2- 172.21.15.2:0/2891632731 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1e640a0710 0x7f1e640a2bd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009fd0 tx=0x7f1e68019040 comp rx=0 tx=0).stop 2023-09-13T04:39:29.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.435+0000 7f1e807ff700 1 -- 172.21.15.2:0/2891632731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e78073710 msgr2=0x7f1e78198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:29.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.435+0000 7f1e807ff700 1 --2- 172.21.15.2:0/2891632731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e78073710 0x7f1e78198c70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400e9a0 tx=0x7f1e7400ecb0 comp rx=0 tx=0).stop 2023-09-13T04:39:29.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.435+0000 7f1e807ff700 1 -- 172.21.15.2:0/2891632731 shutdown_connections 2023-09-13T04:39:29.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.435+0000 7f1e807ff700 1 --2- 172.21.15.2:0/2891632731 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1e640a0710 0x7f1e640a2bd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:29.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.435+0000 7f1e807ff700 1 --2- 172.21.15.2:0/2891632731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e78108bb0 0x7f1e781a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:29.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.435+0000 7f1e807ff700 1 --2- 172.21.15.2:0/2891632731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e78073710 0x7f1e78198c70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:29.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.435+0000 7f1e807ff700 1 --2- 172.21.15.2:0/2891632731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e78074cf0 0x7f1e781991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:29.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.435+0000 7f1e807ff700 1 -- 172.21.15.2:0/2891632731 >> 172.21.15.2:0/2891632731 conn(0x7f1e780fc220 msgr2=0x7f1e780fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:29.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.435+0000 7f1e807ff700 1 -- 172.21.15.2:0/2891632731 shutdown_connections 2023-09-13T04:39:29.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.435+0000 7f1e807ff700 1 -- 172.21.15.2:0/2891632731 wait complete. 2023-09-13T04:39:29.440 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 23 2023-09-13T04:39:29.455 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T04:39:29.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.658+0000 7f69b7fff700 1 -- 172.21.15.2:0/1392017750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69b0100cd0 msgr2=0x7f69b01010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:29.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.658+0000 7f69b7fff700 1 --2- 172.21.15.2:0/1392017750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69b0100cd0 0x7f69b01010f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009990 tx=0x7f69ac009ca0 comp rx=0 tx=0).stop 2023-09-13T04:39:29.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.658+0000 7f69b7fff700 1 -- 172.21.15.2:0/1392017750 shutdown_connections 2023-09-13T04:39:29.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.658+0000 7f69b7fff700 1 --2- 172.21.15.2:0/1392017750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69b0102860 0x7f69b0109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:29.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.658+0000 7f69b7fff700 1 --2- 172.21.15.2:0/1392017750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69b0100cd0 0x7f69b01010f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:29.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.658+0000 7f69b7fff700 1 --2- 172.21.15.2:0/1392017750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69b0101ec0 0x7f69b0102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:29.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.658+0000 7f69b7fff700 1 -- 172.21.15.2:0/1392017750 >> 172.21.15.2:0/1392017750 conn(0x7f69b00fc220 msgr2=0x7f69b00fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:29.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.658+0000 7f69b7fff700 1 -- 172.21.15.2:0/1392017750 shutdown_connections 2023-09-13T04:39:29.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.659+0000 7f69b7fff700 1 -- 172.21.15.2:0/1392017750 wait complete. 2023-09-13T04:39:29.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.659+0000 7f69b7fff700 1 Processor -- start 2023-09-13T04:39:29.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.660+0000 7f69b7fff700 1 -- start start 2023-09-13T04:39:29.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.660+0000 7f69b7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69b0100cd0 0x7f69b0198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:29.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.660+0000 7f69b7fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69b0101ec0 0x7f69b0199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:29.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.660+0000 7f69b7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69b0102860 0x7f69b01a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:29.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.660+0000 7f69b7fff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b01a0760 con 0x7f69b0102860 2023-09-13T04:39:29.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.660+0000 7f69b7fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b01a08a0 con 0x7f69b0101ec0 2023-09-13T04:39:29.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.660+0000 7f69b7fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b01a0a30 con 0x7f69b0100cd0 2023-09-13T04:39:29.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.660+0000 7f69b659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69b0102860 0x7f69b01a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:29.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.660+0000 7f69b659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69b0102860 0x7f69b01a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51678/0 (socket says 172.21.15.2:51678) 2023-09-13T04:39:29.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.660+0000 7f69b659c700 1 -- 172.21.15.2:0/1069236217 learned_addr learned my addr 172.21.15.2:0/1069236217 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:29.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.660+0000 7f69b5d9b700 1 --2- 172.21.15.2:0/1069236217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69b0100cd0 0x7f69b0198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:29.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.660+0000 7f69b559a700 1 --2- 172.21.15.2:0/1069236217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69b0101ec0 0x7f69b0199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:29.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.661+0000 7f69b659c700 1 -- 172.21.15.2:0/1069236217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69b0100cd0 msgr2=0x7f69b0198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:29.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.661+0000 7f69b659c700 1 --2- 172.21.15.2:0/1069236217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69b0100cd0 0x7f69b0198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:29.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.661+0000 7f69b659c700 1 -- 172.21.15.2:0/1069236217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69b0101ec0 msgr2=0x7f69b0199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:29.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.661+0000 7f69b659c700 1 --2- 172.21.15.2:0/1069236217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69b0101ec0 0x7f69b0199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:29.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.661+0000 7f69b659c700 1 -- 172.21.15.2:0/1069236217 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ac0095f0 con 0x7f69b0102860 2023-09-13T04:39:29.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.661+0000 7f69b659c700 1 --2- 172.21.15.2:0/1069236217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69b0102860 0x7f69b01a0190 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f69a8007ee0 tx=0x7f69a800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:29.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.661+0000 7f69a6ffd700 1 -- 172.21.15.2:0/1069236217 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a8017070 con 0x7f69b0102860 2023-09-13T04:39:29.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.661+0000 7f69a6ffd700 1 -- 172.21.15.2:0/1069236217 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69a800f040 con 0x7f69b0102860 2023-09-13T04:39:29.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.661+0000 7f69b7fff700 1 -- 172.21.15.2:0/1069236217 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b01a0d10 con 0x7f69b0102860 2023-09-13T04:39:29.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.661+0000 7f69a6ffd700 1 -- 172.21.15.2:0/1069236217 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a8005580 con 0x7f69b0102860 2023-09-13T04:39:29.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.662+0000 7f69b7fff700 1 -- 172.21.15.2:0/1069236217 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b01a11d0 con 0x7f69b0102860 2023-09-13T04:39:29.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.663+0000 7f69b7fff700 1 -- 172.21.15.2:0/1069236217 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b010c530 con 0x7f69b0102860 2023-09-13T04:39:29.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.664+0000 7f69a6ffd700 1 -- 172.21.15.2:0/1069236217 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f69a800db20 con 0x7f69b0102860 2023-09-13T04:39:29.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.666+0000 7f69a6ffd700 1 --2- 172.21.15.2:0/1069236217 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f699c0a0710 0x7f699c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:29.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.666+0000 7f69a6ffd700 1 -- 172.21.15.2:0/1069236217 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f69a80cb630 con 0x7f69b0102860 2023-09-13T04:39:29.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.666+0000 7f69b5d9b700 1 --2- 172.21.15.2:0/1069236217 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f699c0a0710 0x7f699c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:29.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.667+0000 7f69b5d9b700 1 --2- 172.21.15.2:0/1069236217 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f699c0a0710 0x7f699c0a2bd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f69ac00b2e0 tx=0x7f69ac005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:29.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.670+0000 7f69a6ffd700 1 -- 172.21.15.2:0/1069236217 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f69a8094dd0 con 0x7f69b0102860 2023-09-13T04:39:29.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:29 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2494645786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:39:29.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:29 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2891632731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:39:29.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.883+0000 7f69b7fff700 1 -- 172.21.15.2:0/1069236217 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69b01a13c0 con 0x7f69b0102860 2023-09-13T04:39:29.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.885+0000 7f69a6ffd700 1 -- 172.21.15.2:0/1069236217 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24503 (secure 0 0 0) 0x7f69a80983f0 con 0x7f69b0102860 2023-09-13T04:39:29.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:29 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2494645786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:39:29.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:29 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2891632731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:39:29.888 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:39:29.888 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":92,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:39:25.580735+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T04:37:57.094802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T04:37:58.564487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-13T04:38:02.096895+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:39:29.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.887+0000 7f69b7fff700 1 -- 172.21.15.2:0/1069236217 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f699c0a0710 msgr2=0x7f699c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:29.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.887+0000 7f69b7fff700 1 --2- 172.21.15.2:0/1069236217 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f699c0a0710 0x7f699c0a2bd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f69ac00b2e0 tx=0x7f69ac005fb0 comp rx=0 tx=0).stop 2023-09-13T04:39:29.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.887+0000 7f69b7fff700 1 -- 172.21.15.2:0/1069236217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69b0102860 msgr2=0x7f69b01a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:29.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.887+0000 7f69b7fff700 1 --2- 172.21.15.2:0/1069236217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69b0102860 0x7f69b01a0190 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f69a8007ee0 tx=0x7f69a800d3b0 comp rx=0 tx=0).stop 2023-09-13T04:39:29.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.887+0000 7f69b7fff700 1 -- 172.21.15.2:0/1069236217 shutdown_connections 2023-09-13T04:39:29.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.887+0000 7f69b7fff700 1 --2- 172.21.15.2:0/1069236217 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f699c0a0710 0x7f699c0a2bd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:29.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.887+0000 7f69b7fff700 1 --2- 172.21.15.2:0/1069236217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69b0102860 0x7f69b01a0190 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:29.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.887+0000 7f69b7fff700 1 --2- 172.21.15.2:0/1069236217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69b0100cd0 0x7f69b0198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:29.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.887+0000 7f69b7fff700 1 --2- 172.21.15.2:0/1069236217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69b0101ec0 0x7f69b0199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:29.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.887+0000 7f69b7fff700 1 -- 172.21.15.2:0/1069236217 >> 172.21.15.2:0/1069236217 conn(0x7f69b00fc220 msgr2=0x7f69b00fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:29.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.887+0000 7f69b7fff700 1 -- 172.21.15.2:0/1069236217 shutdown_connections 2023-09-13T04:39:29.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:29.887+0000 7f69b7fff700 1 -- 172.21.15.2:0/1069236217 wait complete. 2023-09-13T04:39:29.908 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T04:33:42.025714+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-13T04:37:57.094802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-13T04:37:58.564487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-13T04:38:02.096895+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-13T04:39:29.909 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-13T04:39:29.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:29 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2494645786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:39:29.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:29 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2891632731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:39:30.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.115+0000 7fd87b3ff700 1 -- 172.21.15.2:0/3512449739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd874102880 msgr2=0x7fd874109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:30.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.115+0000 7fd87b3ff700 1 --2- 172.21.15.2:0/3512449739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd874102880 0x7fd874109130 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00b0d0 tx=0x7fd86c00b3e0 comp rx=0 tx=0).stop 2023-09-13T04:39:30.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.115+0000 7fd87b3ff700 1 -- 172.21.15.2:0/3512449739 shutdown_connections 2023-09-13T04:39:30.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.115+0000 7fd87b3ff700 1 --2- 172.21.15.2:0/3512449739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd874102880 0x7fd874109130 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:30.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.115+0000 7fd87b3ff700 1 --2- 172.21.15.2:0/3512449739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd874100cf0 0x7fd874101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:30.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.115+0000 7fd87b3ff700 1 --2- 172.21.15.2:0/3512449739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd874101ee0 0x7fd874102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:30.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.115+0000 7fd87b3ff700 1 -- 172.21.15.2:0/3512449739 >> 172.21.15.2:0/3512449739 conn(0x7fd8740fc260 msgr2=0x7fd8740fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:30.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.115+0000 7fd87b3ff700 1 -- 172.21.15.2:0/3512449739 shutdown_connections 2023-09-13T04:39:30.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.115+0000 7fd87b3ff700 1 -- 172.21.15.2:0/3512449739 wait complete. 2023-09-13T04:39:30.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.116+0000 7fd87b3ff700 1 Processor -- start 2023-09-13T04:39:30.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.117+0000 7fd87b3ff700 1 -- start start 2023-09-13T04:39:30.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.117+0000 7fd87b3ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd874100cf0 0x7fd874198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:30.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.117+0000 7fd87b3ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd874101ee0 0x7fd874199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:30.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.117+0000 7fd87b3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd874102880 0x7fd8741a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:30.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.117+0000 7fd87b3ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8741a0830 con 0x7fd874100cf0 2023-09-13T04:39:30.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.117+0000 7fd87b3ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8741a09a0 con 0x7fd874101ee0 2023-09-13T04:39:30.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.117+0000 7fd87b3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8741a0b60 con 0x7fd874102880 2023-09-13T04:39:30.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.117+0000 7fd87919b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd874100cf0 0x7fd874198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:30.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.117+0000 7fd87919b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd874100cf0 0x7fd874198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51684/0 (socket says 172.21.15.2:51684) 2023-09-13T04:39:30.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.118+0000 7fd87919b700 1 -- 172.21.15.2:0/3096066070 learned_addr learned my addr 172.21.15.2:0/3096066070 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:30.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.118+0000 7fd87899a700 1 --2- 172.21.15.2:0/3096066070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd874101ee0 0x7fd874199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:30.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.118+0000 7fd87999c700 1 --2- 172.21.15.2:0/3096066070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd874102880 0x7fd8741a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:30.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.118+0000 7fd87919b700 1 -- 172.21.15.2:0/3096066070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd874102880 msgr2=0x7fd8741a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:30.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.118+0000 7fd87919b700 1 --2- 172.21.15.2:0/3096066070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd874102880 0x7fd8741a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:30.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.118+0000 7fd87919b700 1 -- 172.21.15.2:0/3096066070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd874101ee0 msgr2=0x7fd874199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:30.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.118+0000 7fd87919b700 1 --2- 172.21.15.2:0/3096066070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd874101ee0 0x7fd874199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:30.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.118+0000 7fd87919b700 1 -- 172.21.15.2:0/3096066070 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86c009d00 con 0x7fd874100cf0 2023-09-13T04:39:30.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.118+0000 7fd87899a700 1 --2- 172.21.15.2:0/3096066070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd874101ee0 0x7fd874199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:39:30.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.118+0000 7fd87919b700 1 --2- 172.21.15.2:0/3096066070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd874100cf0 0x7fd874198cc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd87000e970 tx=0x7fd87000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:30.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.118+0000 7fd86a7fc700 1 -- 172.21.15.2:0/3096066070 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd870009790 con 0x7fd874100cf0 2023-09-13T04:39:30.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.118+0000 7fd86a7fc700 1 -- 172.21.15.2:0/3096066070 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8700044e0 con 0x7fd874100cf0 2023-09-13T04:39:30.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.118+0000 7fd87b3ff700 1 -- 172.21.15.2:0/3096066070 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8741a0e40 con 0x7fd874100cf0 2023-09-13T04:39:30.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.118+0000 7fd86a7fc700 1 -- 172.21.15.2:0/3096066070 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd870003e30 con 0x7fd874100cf0 2023-09-13T04:39:30.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.119+0000 7fd87b3ff700 1 -- 172.21.15.2:0/3096066070 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8741a1260 con 0x7fd874100cf0 2023-09-13T04:39:30.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.120+0000 7fd87b3ff700 1 -- 172.21.15.2:0/3096066070 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd87410c560 con 0x7fd874100cf0 2023-09-13T04:39:30.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.122+0000 7fd86a7fc700 1 -- 172.21.15.2:0/3096066070 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fd870004000 con 0x7fd874100cf0 2023-09-13T04:39:30.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.123+0000 7fd86a7fc700 1 --2- 172.21.15.2:0/3096066070 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd8600a0640 0x7fd8600a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:30.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.123+0000 7fd86a7fc700 1 -- 172.21.15.2:0/3096066070 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd870014070 con 0x7fd874100cf0 2023-09-13T04:39:30.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.123+0000 7fd87899a700 1 --2- 172.21.15.2:0/3096066070 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd8600a0640 0x7fd8600a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:30.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.124+0000 7fd87899a700 1 --2- 172.21.15.2:0/3096066070 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd8600a0640 0x7fd8600a2b00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd87419a1c0 tx=0x7fd864009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:30.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.125+0000 7fd86a7fc700 1 -- 172.21.15.2:0/3096066070 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd870095a90 con 0x7fd874100cf0 2023-09-13T04:39:30.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.336+0000 7fd87b3ff700 1 -- 172.21.15.2:0/3096066070 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd8741a14e0 con 0x7fd874100cf0 2023-09-13T04:39:30.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.336+0000 7fd86a7fc700 1 -- 172.21.15.2:0/3096066070 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd8700990b0 con 0x7fd874100cf0 2023-09-13T04:39:30.338 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-09-13T04:39:30.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.339+0000 7fd87b3ff700 1 -- 172.21.15.2:0/3096066070 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd8600a0640 msgr2=0x7fd8600a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:30.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.339+0000 7fd87b3ff700 1 --2- 172.21.15.2:0/3096066070 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd8600a0640 0x7fd8600a2b00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd87419a1c0 tx=0x7fd864009260 comp rx=0 tx=0).stop 2023-09-13T04:39:30.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.339+0000 7fd87b3ff700 1 -- 172.21.15.2:0/3096066070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd874100cf0 msgr2=0x7fd874198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:30.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.339+0000 7fd87b3ff700 1 --2- 172.21.15.2:0/3096066070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd874100cf0 0x7fd874198cc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd87000e970 tx=0x7fd87000ed30 comp rx=0 tx=0).stop 2023-09-13T04:39:30.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.339+0000 7fd87b3ff700 1 -- 172.21.15.2:0/3096066070 shutdown_connections 2023-09-13T04:39:30.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.339+0000 7fd87b3ff700 1 --2- 172.21.15.2:0/3096066070 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd8600a0640 0x7fd8600a2b00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:30.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.339+0000 7fd87b3ff700 1 --2- 172.21.15.2:0/3096066070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd874102880 0x7fd8741a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:30.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.339+0000 7fd87b3ff700 1 --2- 172.21.15.2:0/3096066070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd874100cf0 0x7fd874198cc0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:30.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.339+0000 7fd87b3ff700 1 --2- 172.21.15.2:0/3096066070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd874101ee0 0x7fd874199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:30.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.339+0000 7fd87b3ff700 1 -- 172.21.15.2:0/3096066070 >> 172.21.15.2:0/3096066070 conn(0x7fd8740fc260 msgr2=0x7fd874107710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:30.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.339+0000 7fd87b3ff700 1 -- 172.21.15.2:0/3096066070 shutdown_connections 2023-09-13T04:39:30.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.339+0000 7fd87b3ff700 1 -- 172.21.15.2:0/3096066070 wait complete. 2023-09-13T04:39:30.359 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-13T04:39:30.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.565+0000 7fe3f5fff700 1 -- 172.21.15.2:0/3143611040 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3f0102870 msgr2=0x7fe3f0109120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:30.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.565+0000 7fe3f5fff700 1 --2- 172.21.15.2:0/3143611040 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3f0102870 0x7fe3f0109120 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800b0d0 tx=0x7fe3e800b3e0 comp rx=0 tx=0).stop 2023-09-13T04:39:30.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.565+0000 7fe3f5fff700 1 -- 172.21.15.2:0/3143611040 shutdown_connections 2023-09-13T04:39:30.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.565+0000 7fe3f5fff700 1 --2- 172.21.15.2:0/3143611040 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3f0102870 0x7fe3f0109120 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:30.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.565+0000 7fe3f5fff700 1 --2- 172.21.15.2:0/3143611040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3f0100ce0 0x7fe3f0101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:30.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.565+0000 7fe3f5fff700 1 --2- 172.21.15.2:0/3143611040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3f0101ed0 0x7fe3f0102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:30.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.565+0000 7fe3f5fff700 1 -- 172.21.15.2:0/3143611040 >> 172.21.15.2:0/3143611040 conn(0x7fe3f00fc230 msgr2=0x7fe3f00fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:30.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.565+0000 7fe3f5fff700 1 -- 172.21.15.2:0/3143611040 shutdown_connections 2023-09-13T04:39:30.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.565+0000 7fe3f5fff700 1 -- 172.21.15.2:0/3143611040 wait complete. 2023-09-13T04:39:30.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.566+0000 7fe3f5fff700 1 Processor -- start 2023-09-13T04:39:30.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.567+0000 7fe3f5fff700 1 -- start start 2023-09-13T04:39:30.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.567+0000 7fe3f5fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3f0100ce0 0x7fe3f0196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:30.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.567+0000 7fe3f5fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3f0101ed0 0x7fe3f0197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:30.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.567+0000 7fe3f5fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3f0102870 0x7fe3f019e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:30.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.567+0000 7fe3f5fff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f019e680 con 0x7fe3f0100ce0 2023-09-13T04:39:30.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.567+0000 7fe3f5fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f019e7c0 con 0x7fe3f0102870 2023-09-13T04:39:30.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.567+0000 7fe3f5fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f019e950 con 0x7fe3f0101ed0 2023-09-13T04:39:30.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.567+0000 7fe3ef7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3f0100ce0 0x7fe3f0196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:30.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.567+0000 7fe3ef7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3f0100ce0 0x7fe3f0196b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51712/0 (socket says 172.21.15.2:51712) 2023-09-13T04:39:30.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.567+0000 7fe3ef7fe700 1 -- 172.21.15.2:0/729900055 learned_addr learned my addr 172.21.15.2:0/729900055 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:30.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.567+0000 7fe3effff700 1 --2- 172.21.15.2:0/729900055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3f0102870 0x7fe3f019e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:30.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.567+0000 7fe3eeffd700 1 --2- 172.21.15.2:0/729900055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3f0101ed0 0x7fe3f0197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:30.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.568+0000 7fe3ef7fe700 1 -- 172.21.15.2:0/729900055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3f0101ed0 msgr2=0x7fe3f0197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:30.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.568+0000 7fe3ef7fe700 1 --2- 172.21.15.2:0/729900055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3f0101ed0 0x7fe3f0197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:30.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.568+0000 7fe3ef7fe700 1 -- 172.21.15.2:0/729900055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3f0102870 msgr2=0x7fe3f019e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:30.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.568+0000 7fe3ef7fe700 1 --2- 172.21.15.2:0/729900055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3f0102870 0x7fe3f019e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:30.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.568+0000 7fe3ef7fe700 1 -- 172.21.15.2:0/729900055 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e8009d00 con 0x7fe3f0100ce0 2023-09-13T04:39:30.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.568+0000 7fe3ef7fe700 1 --2- 172.21.15.2:0/729900055 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3f0100ce0 0x7fe3f0196b00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe3e000e970 tx=0x7fe3e000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:30.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.568+0000 7fe3ecff9700 1 -- 172.21.15.2:0/729900055 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0009790 con 0x7fe3f0100ce0 2023-09-13T04:39:30.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.568+0000 7fe3ecff9700 1 -- 172.21.15.2:0/729900055 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3e000c7e0 con 0x7fe3f0100ce0 2023-09-13T04:39:30.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.568+0000 7fe3ecff9700 1 -- 172.21.15.2:0/729900055 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0003c70 con 0x7fe3f0100ce0 2023-09-13T04:39:30.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.568+0000 7fe3f5fff700 1 -- 172.21.15.2:0/729900055 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f019ec30 con 0x7fe3f0100ce0 2023-09-13T04:39:30.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.568+0000 7fe3f5fff700 1 -- 172.21.15.2:0/729900055 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f019f0f0 con 0x7fe3f0100ce0 2023-09-13T04:39:30.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.569+0000 7fe3f5fff700 1 -- 172.21.15.2:0/729900055 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f0107c10 con 0x7fe3f0100ce0 2023-09-13T04:39:30.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.571+0000 7fe3ecff9700 1 -- 172.21.15.2:0/729900055 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fe3e000cc50 con 0x7fe3f0100ce0 2023-09-13T04:39:30.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.572+0000 7fe3ecff9700 1 --2- 172.21.15.2:0/729900055 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe3d80a06c0 0x7fe3d80a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:30.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.573+0000 7fe3ecff9700 1 -- 172.21.15.2:0/729900055 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe3e0014070 con 0x7fe3f0100ce0 2023-09-13T04:39:30.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.573+0000 7fe3eeffd700 1 --2- 172.21.15.2:0/729900055 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe3d80a06c0 0x7fe3d80a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:30.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.573+0000 7fe3eeffd700 1 --2- 172.21.15.2:0/729900055 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe3d80a06c0 0x7fe3d80a2b80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4009c60 tx=0x7fe3e4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:30.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.576+0000 7fe3ecff9700 1 -- 172.21.15.2:0/729900055 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe3e0095820 con 0x7fe3f0100ce0 2023-09-13T04:39:30.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:30 smithi002 ceph-mon[97651]: pgmap v325: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2023-09-13T04:39:30.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:30 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1069236217' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:39:30.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:30 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3096066070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:39:30.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.791+0000 7fe3f5fff700 1 -- 172.21.15.2:0/729900055 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe3f002cd70 con 0x7fe3f0100ce0 2023-09-13T04:39:30.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.791+0000 7fe3ecff9700 1 -- 172.21.15.2:0/729900055 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe3e0003dd0 con 0x7fe3f0100ce0 2023-09-13T04:39:30.793 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-09-13T04:39:30.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.794+0000 7fe3f5fff700 1 -- 172.21.15.2:0/729900055 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe3d80a06c0 msgr2=0x7fe3d80a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:30.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.794+0000 7fe3f5fff700 1 --2- 172.21.15.2:0/729900055 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe3d80a06c0 0x7fe3d80a2b80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4009c60 tx=0x7fe3e4009260 comp rx=0 tx=0).stop 2023-09-13T04:39:30.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.794+0000 7fe3f5fff700 1 -- 172.21.15.2:0/729900055 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3f0100ce0 msgr2=0x7fe3f0196b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:30.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.794+0000 7fe3f5fff700 1 --2- 172.21.15.2:0/729900055 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3f0100ce0 0x7fe3f0196b00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe3e000e970 tx=0x7fe3e000ec80 comp rx=0 tx=0).stop 2023-09-13T04:39:30.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.794+0000 7fe3f5fff700 1 -- 172.21.15.2:0/729900055 shutdown_connections 2023-09-13T04:39:30.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.794+0000 7fe3f5fff700 1 --2- 172.21.15.2:0/729900055 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe3d80a06c0 0x7fe3d80a2b80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:30.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.794+0000 7fe3f5fff700 1 --2- 172.21.15.2:0/729900055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3f0102870 0x7fe3f019e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:30.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.794+0000 7fe3f5fff700 1 --2- 172.21.15.2:0/729900055 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3f0100ce0 0x7fe3f0196b00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:30.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.794+0000 7fe3f5fff700 1 --2- 172.21.15.2:0/729900055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3f0101ed0 0x7fe3f0197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:30.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.794+0000 7fe3f5fff700 1 -- 172.21.15.2:0/729900055 >> 172.21.15.2:0/729900055 conn(0x7fe3f00fc230 msgr2=0x7fe3f00fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:30.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.794+0000 7fe3f5fff700 1 -- 172.21.15.2:0/729900055 shutdown_connections 2023-09-13T04:39:30.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:30.794+0000 7fe3f5fff700 1 -- 172.21.15.2:0/729900055 wait complete. 2023-09-13T04:39:30.814 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-13T04:39:30.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:30 smithi149 ceph-mon[133315]: pgmap v325: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2023-09-13T04:39:30.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:30 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1069236217' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:39:30.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:30 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3096066070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:39:30.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:30 smithi156 ceph-mon[133675]: pgmap v325: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2023-09-13T04:39:30.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:30 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1069236217' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:39:30.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:30 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3096066070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:39:31.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.020+0000 7f794a9ff700 1 -- 172.21.15.2:0/593693230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7944101f30 msgr2=0x7f7944102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:31.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.020+0000 7f794a9ff700 1 --2- 172.21.15.2:0/593693230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7944101f30 0x7f7944102390 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f79380099c0 tx=0x7f7938009cd0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.020+0000 7f794a9ff700 1 -- 172.21.15.2:0/593693230 shutdown_connections 2023-09-13T04:39:31.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.020+0000 7f794a9ff700 1 --2- 172.21.15.2:0/593693230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79441028d0 0x7f7944109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.020+0000 7f794a9ff700 1 --2- 172.21.15.2:0/593693230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7944100d40 0x7f7944101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.020+0000 7f794a9ff700 1 --2- 172.21.15.2:0/593693230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7944101f30 0x7f7944102390 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.020+0000 7f794a9ff700 1 -- 172.21.15.2:0/593693230 >> 172.21.15.2:0/593693230 conn(0x7f79440fc2d0 msgr2=0x7f79440fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:31.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.020+0000 7f794a9ff700 1 -- 172.21.15.2:0/593693230 shutdown_connections 2023-09-13T04:39:31.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.020+0000 7f794a9ff700 1 -- 172.21.15.2:0/593693230 wait complete. 2023-09-13T04:39:31.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.021+0000 7f794a9ff700 1 Processor -- start 2023-09-13T04:39:31.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.022+0000 7f794a9ff700 1 -- start start 2023-09-13T04:39:31.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.022+0000 7f794a9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7944100d40 0x7f7944198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:31.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.022+0000 7f794a9ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7944101f30 0x7f7944199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:31.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.022+0000 7f794a9ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79441028d0 0x7f79441a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:31.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.022+0000 7f794a9ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79441a0760 con 0x7f79441028d0 2023-09-13T04:39:31.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.022+0000 7f794a9ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79441a08d0 con 0x7f7944101f30 2023-09-13T04:39:31.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.022+0000 7f794a9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79441a0a90 con 0x7f7944100d40 2023-09-13T04:39:31.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.022+0000 7f7948f9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79441028d0 0x7f79441a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:31.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.022+0000 7f79437fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7944101f30 0x7f7944199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:31.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.022+0000 7f79437fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7944101f30 0x7f7944199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:48230/0 (socket says 172.21.15.2:48230) 2023-09-13T04:39:31.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.022+0000 7f79437fe700 1 -- 172.21.15.2:0/1681708929 learned_addr learned my addr 172.21.15.2:0/1681708929 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:31.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.022+0000 7f7943fff700 1 --2- 172.21.15.2:0/1681708929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7944100d40 0x7f7944198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:31.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.023+0000 7f79437fe700 1 -- 172.21.15.2:0/1681708929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7944100d40 msgr2=0x7f7944198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:31.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.023+0000 7f79437fe700 1 --2- 172.21.15.2:0/1681708929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7944100d40 0x7f7944198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.023+0000 7f79437fe700 1 -- 172.21.15.2:0/1681708929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79441028d0 msgr2=0x7f79441a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:31.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.023+0000 7f79437fe700 1 --2- 172.21.15.2:0/1681708929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79441028d0 0x7f79441a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.023+0000 7f79437fe700 1 -- 172.21.15.2:0/1681708929 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79380095f0 con 0x7f7944101f30 2023-09-13T04:39:31.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.023+0000 7f7948f9c700 1 --2- 172.21.15.2:0/1681708929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79441028d0 0x7f79441a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:39:31.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.023+0000 7f79437fe700 1 --2- 172.21.15.2:0/1681708929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7944101f30 0x7f7944199160 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7938009990 tx=0x7f793800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:31.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.023+0000 7f79417fa700 1 -- 172.21.15.2:0/1681708929 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793801d070 con 0x7f7944101f30 2023-09-13T04:39:31.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.023+0000 7f794a9ff700 1 -- 172.21.15.2:0/1681708929 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79441a0d10 con 0x7f7944101f30 2023-09-13T04:39:31.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.023+0000 7f79417fa700 1 -- 172.21.15.2:0/1681708929 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7938005030 con 0x7f7944101f30 2023-09-13T04:39:31.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.023+0000 7f79417fa700 1 -- 172.21.15.2:0/1681708929 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938004a70 con 0x7f7944101f30 2023-09-13T04:39:31.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.023+0000 7f794a9ff700 1 -- 172.21.15.2:0/1681708929 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79441a1140 con 0x7f7944101f30 2023-09-13T04:39:31.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.024+0000 7f794a9ff700 1 -- 172.21.15.2:0/1681708929 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f794410c530 con 0x7f7944101f30 2023-09-13T04:39:31.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.026+0000 7f79417fa700 1 -- 172.21.15.2:0/1681708929 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f7938005410 con 0x7f7944101f30 2023-09-13T04:39:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.028+0000 7f79417fa700 1 --2- 172.21.15.2:0/1681708929 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f792c0a0640 0x7f792c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.028+0000 7f79417fa700 1 -- 172.21.15.2:0/1681708929 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f79380cd170 con 0x7f7944101f30 2023-09-13T04:39:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.028+0000 7f7943fff700 1 --2- 172.21.15.2:0/1681708929 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f792c0a0640 0x7f792c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.029+0000 7f7943fff700 1 --2- 172.21.15.2:0/1681708929 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f792c0a0640 0x7f792c0a2b00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f794419a0f0 tx=0x7f7934009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.030+0000 7f79417fa700 1 -- 172.21.15.2:0/1681708929 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f79380969a0 con 0x7f7944101f30 2023-09-13T04:39:31.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.241+0000 7f794a9ff700 1 -- 172.21.15.2:0/1681708929 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f79441a12d0 con 0x7f7944101f30 2023-09-13T04:39:31.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.241+0000 7f79417fa700 1 -- 172.21.15.2:0/1681708929 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7938099fc0 con 0x7f7944101f30 2023-09-13T04:39:31.243 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-09-13T04:39:31.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.244+0000 7f794a9ff700 1 -- 172.21.15.2:0/1681708929 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f792c0a0640 msgr2=0x7f792c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:31.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.244+0000 7f794a9ff700 1 --2- 172.21.15.2:0/1681708929 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f792c0a0640 0x7f792c0a2b00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f794419a0f0 tx=0x7f7934009260 comp rx=0 tx=0).stop 2023-09-13T04:39:31.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.244+0000 7f794a9ff700 1 -- 172.21.15.2:0/1681708929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7944101f30 msgr2=0x7f7944199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:31.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.244+0000 7f794a9ff700 1 --2- 172.21.15.2:0/1681708929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7944101f30 0x7f7944199160 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7938009990 tx=0x7f793800f690 comp rx=0 tx=0).stop 2023-09-13T04:39:31.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.244+0000 7f794a9ff700 1 -- 172.21.15.2:0/1681708929 shutdown_connections 2023-09-13T04:39:31.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.244+0000 7f794a9ff700 1 --2- 172.21.15.2:0/1681708929 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f792c0a0640 0x7f792c0a2b00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.244+0000 7f794a9ff700 1 --2- 172.21.15.2:0/1681708929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79441028d0 0x7f79441a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.244+0000 7f794a9ff700 1 --2- 172.21.15.2:0/1681708929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7944100d40 0x7f7944198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.244+0000 7f794a9ff700 1 --2- 172.21.15.2:0/1681708929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7944101f30 0x7f7944199160 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.244+0000 7f794a9ff700 1 -- 172.21.15.2:0/1681708929 >> 172.21.15.2:0/1681708929 conn(0x7f79440fc2d0 msgr2=0x7f79440fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:31.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.244+0000 7f794a9ff700 1 -- 172.21.15.2:0/1681708929 shutdown_connections 2023-09-13T04:39:31.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.244+0000 7f794a9ff700 1 -- 172.21.15.2:0/1681708929 wait complete. 2023-09-13T04:39:31.264 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-13T04:39:31.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.471+0000 7f7c367ff700 1 -- 172.21.15.2:0/1457238675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c30101ee0 msgr2=0x7f7c30102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:31.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.471+0000 7f7c367ff700 1 --2- 172.21.15.2:0/1457238675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c30101ee0 0x7f7c30102340 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7c240099c0 tx=0x7f7c24009cd0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.471+0000 7f7c367ff700 1 -- 172.21.15.2:0/1457238675 shutdown_connections 2023-09-13T04:39:31.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.471+0000 7f7c367ff700 1 --2- 172.21.15.2:0/1457238675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c30102880 0x7f7c30109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.471+0000 7f7c367ff700 1 --2- 172.21.15.2:0/1457238675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c30100cf0 0x7f7c30101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.471+0000 7f7c367ff700 1 --2- 172.21.15.2:0/1457238675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c30101ee0 0x7f7c30102340 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.471+0000 7f7c367ff700 1 -- 172.21.15.2:0/1457238675 >> 172.21.15.2:0/1457238675 conn(0x7f7c300fc260 msgr2=0x7f7c300fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.471+0000 7f7c367ff700 1 -- 172.21.15.2:0/1457238675 shutdown_connections 2023-09-13T04:39:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.471+0000 7f7c367ff700 1 -- 172.21.15.2:0/1457238675 wait complete. 2023-09-13T04:39:31.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.472+0000 7f7c367ff700 1 Processor -- start 2023-09-13T04:39:31.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.472+0000 7f7c367ff700 1 -- start start 2023-09-13T04:39:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.473+0000 7f7c367ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c30100cf0 0x7f7c30196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.473+0000 7f7c367ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c30101ee0 0x7f7c30196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.473+0000 7f7c367ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c30102880 0x7f7c3019dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.473+0000 7f7c367ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3019e590 con 0x7f7c30100cf0 2023-09-13T04:39:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.473+0000 7f7c367ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3019e6d0 con 0x7f7c30101ee0 2023-09-13T04:39:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.473+0000 7f7c367ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3019e860 con 0x7f7c30102880 2023-09-13T04:39:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.473+0000 7f7c2ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c30100cf0 0x7f7c30196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.473+0000 7f7c2ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c30100cf0 0x7f7c30196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51754/0 (socket says 172.21.15.2:51754) 2023-09-13T04:39:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.473+0000 7f7c2ffff700 1 -- 172.21.15.2:0/297236101 learned_addr learned my addr 172.21.15.2:0/297236101 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.473+0000 7f7c2f7fe700 1 --2- 172.21.15.2:0/297236101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c30101ee0 0x7f7c30196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.473+0000 7f7c34d9c700 1 --2- 172.21.15.2:0/297236101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c30102880 0x7f7c3019dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.474+0000 7f7c2ffff700 1 -- 172.21.15.2:0/297236101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c30102880 msgr2=0x7f7c3019dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.474+0000 7f7c2ffff700 1 --2- 172.21.15.2:0/297236101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c30102880 0x7f7c3019dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.474+0000 7f7c2ffff700 1 -- 172.21.15.2:0/297236101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c30101ee0 msgr2=0x7f7c30196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.474+0000 7f7c2ffff700 1 --2- 172.21.15.2:0/297236101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c30101ee0 0x7f7c30196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.474+0000 7f7c2ffff700 1 -- 172.21.15.2:0/297236101 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c240095f0 con 0x7f7c30100cf0 2023-09-13T04:39:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.474+0000 7f7c34d9c700 1 --2- 172.21.15.2:0/297236101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c30102880 0x7f7c3019dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:39:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.474+0000 7f7c2ffff700 1 --2- 172.21.15.2:0/297236101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c30100cf0 0x7f7c30196a10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000e970 tx=0x7f7c2000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.474+0000 7f7c2d7fa700 1 -- 172.21.15.2:0/297236101 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c20009790 con 0x7f7c30100cf0 2023-09-13T04:39:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.474+0000 7f7c2d7fa700 1 -- 172.21.15.2:0/297236101 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c2000eec0 con 0x7f7c30100cf0 2023-09-13T04:39:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.474+0000 7f7c367ff700 1 -- 172.21.15.2:0/297236101 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c3019eb40 con 0x7f7c30100cf0 2023-09-13T04:39:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.474+0000 7f7c2d7fa700 1 -- 172.21.15.2:0/297236101 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c20003cf0 con 0x7f7c30100cf0 2023-09-13T04:39:31.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.474+0000 7f7c367ff700 1 -- 172.21.15.2:0/297236101 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c3019f0a0 con 0x7f7c30100cf0 2023-09-13T04:39:31.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.476+0000 7f7c367ff700 1 -- 172.21.15.2:0/297236101 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c30107c70 con 0x7f7c30100cf0 2023-09-13T04:39:31.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.478+0000 7f7c2d7fa700 1 -- 172.21.15.2:0/297236101 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f7c2000ccd0 con 0x7f7c30100cf0 2023-09-13T04:39:31.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.479+0000 7f7c2d7fa700 1 --2- 172.21.15.2:0/297236101 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f7c180a06c0 0x7f7c180a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:31.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.479+0000 7f7c2d7fa700 1 -- 172.21.15.2:0/297236101 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f7c20014070 con 0x7f7c30100cf0 2023-09-13T04:39:31.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.479+0000 7f7c2f7fe700 1 --2- 172.21.15.2:0/297236101 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f7c180a06c0 0x7f7c180a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:31.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.480+0000 7f7c2f7fe700 1 --2- 172.21.15.2:0/297236101 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f7c180a06c0 0x7f7c180a2b80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400b580 tx=0x7f7c24005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:31.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.483+0000 7f7c2d7fa700 1 -- 172.21.15.2:0/297236101 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7c200958b0 con 0x7f7c30100cf0 2023-09-13T04:39:31.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.702+0000 7f7c367ff700 1 -- 172.21.15.2:0/297236101 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7c3002d070 con 0x7f7c30100cf0 2023-09-13T04:39:31.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.702+0000 7f7c2d7fa700 1 -- 172.21.15.2:0/297236101 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7c20098ed0 con 0x7f7c30100cf0 2023-09-13T04:39:31.704 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-09-13T04:39:31.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.705+0000 7f7c367ff700 1 -- 172.21.15.2:0/297236101 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f7c180a06c0 msgr2=0x7f7c180a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:31.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.705+0000 7f7c367ff700 1 --2- 172.21.15.2:0/297236101 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f7c180a06c0 0x7f7c180a2b80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400b580 tx=0x7f7c24005fb0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.705+0000 7f7c367ff700 1 -- 172.21.15.2:0/297236101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c30100cf0 msgr2=0x7f7c30196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:31.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.705+0000 7f7c367ff700 1 --2- 172.21.15.2:0/297236101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c30100cf0 0x7f7c30196a10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000e970 tx=0x7f7c2000ec80 comp rx=0 tx=0).stop 2023-09-13T04:39:31.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.705+0000 7f7c367ff700 1 -- 172.21.15.2:0/297236101 shutdown_connections 2023-09-13T04:39:31.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.705+0000 7f7c367ff700 1 --2- 172.21.15.2:0/297236101 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f7c180a06c0 0x7f7c180a2b80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.705+0000 7f7c367ff700 1 --2- 172.21.15.2:0/297236101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c30102880 0x7f7c3019dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.705+0000 7f7c367ff700 1 --2- 172.21.15.2:0/297236101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c30100cf0 0x7f7c30196a10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.705+0000 7f7c367ff700 1 --2- 172.21.15.2:0/297236101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c30101ee0 0x7f7c30196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.705+0000 7f7c367ff700 1 -- 172.21.15.2:0/297236101 >> 172.21.15.2:0/297236101 conn(0x7f7c300fc260 msgr2=0x7f7c300fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:31.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.705+0000 7f7c367ff700 1 -- 172.21.15.2:0/297236101 shutdown_connections 2023-09-13T04:39:31.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.705+0000 7f7c367ff700 1 -- 172.21.15.2:0/297236101 wait complete. 2023-09-13T04:39:31.725 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:39:31.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:31 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/729900055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:39:31.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:31 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1681708929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:39:31.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.929+0000 7fe0ecbff700 1 -- 172.21.15.2:0/2868953242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0e8102880 msgr2=0x7fe0e8109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:31.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.929+0000 7fe0ecbff700 1 --2- 172.21.15.2:0/2868953242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0e8102880 0x7fe0e8109130 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000b0d0 tx=0x7fe0e000b3e0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.929+0000 7fe0ecbff700 1 -- 172.21.15.2:0/2868953242 shutdown_connections 2023-09-13T04:39:31.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.929+0000 7fe0ecbff700 1 --2- 172.21.15.2:0/2868953242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0e8102880 0x7fe0e8109130 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.929+0000 7fe0ecbff700 1 --2- 172.21.15.2:0/2868953242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0e8100cf0 0x7fe0e8101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.929+0000 7fe0ecbff700 1 --2- 172.21.15.2:0/2868953242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0e8101ee0 0x7fe0e8102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.929+0000 7fe0ecbff700 1 -- 172.21.15.2:0/2868953242 >> 172.21.15.2:0/2868953242 conn(0x7fe0e80fc220 msgr2=0x7fe0e80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:31.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.929+0000 7fe0ecbff700 1 -- 172.21.15.2:0/2868953242 shutdown_connections 2023-09-13T04:39:31.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.929+0000 7fe0ecbff700 1 -- 172.21.15.2:0/2868953242 wait complete. 2023-09-13T04:39:31.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.930+0000 7fe0ecbff700 1 Processor -- start 2023-09-13T04:39:31.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.931+0000 7fe0ecbff700 1 -- start start 2023-09-13T04:39:31.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.931+0000 7fe0ecbff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0e8100cf0 0x7fe0e8196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:31.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.931+0000 7fe0ecbff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0e8101ee0 0x7fe0e8196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:31.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.931+0000 7fe0ecbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0e8102880 0x7fe0e819e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:31.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.931+0000 7fe0ecbff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e819e620 con 0x7fe0e8100cf0 2023-09-13T04:39:31.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.931+0000 7fe0ecbff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e819e760 con 0x7fe0e8101ee0 2023-09-13T04:39:31.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.931+0000 7fe0ecbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e819e8f0 con 0x7fe0e8102880 2023-09-13T04:39:31.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.931+0000 7fe0e659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0e8100cf0 0x7fe0e8196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:31.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.931+0000 7fe0e659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0e8100cf0 0x7fe0e8196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51776/0 (socket says 172.21.15.2:51776) 2023-09-13T04:39:31.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.931+0000 7fe0e659c700 1 -- 172.21.15.2:0/3257902713 learned_addr learned my addr 172.21.15.2:0/3257902713 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:31.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.931+0000 7fe0e6d9d700 1 --2- 172.21.15.2:0/3257902713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0e8102880 0x7fe0e819e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:31.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.931+0000 7fe0e5d9b700 1 --2- 172.21.15.2:0/3257902713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0e8101ee0 0x7fe0e8196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:31.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.931+0000 7fe0e659c700 1 -- 172.21.15.2:0/3257902713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0e8102880 msgr2=0x7fe0e819e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:31.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.931+0000 7fe0e659c700 1 --2- 172.21.15.2:0/3257902713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0e8102880 0x7fe0e819e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.932+0000 7fe0e659c700 1 -- 172.21.15.2:0/3257902713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0e8101ee0 msgr2=0x7fe0e8196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:31.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.932+0000 7fe0e659c700 1 --2- 172.21.15.2:0/3257902713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0e8101ee0 0x7fe0e8196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:31.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.932+0000 7fe0e659c700 1 -- 172.21.15.2:0/3257902713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e0009d00 con 0x7fe0e8100cf0 2023-09-13T04:39:31.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.932+0000 7fe0e659c700 1 --2- 172.21.15.2:0/3257902713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0e8100cf0 0x7fe0e8196aa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800e970 tx=0x7fe0d800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:31.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.932+0000 7fe0d77fe700 1 -- 172.21.15.2:0/3257902713 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8009790 con 0x7fe0e8100cf0 2023-09-13T04:39:31.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.932+0000 7fe0d77fe700 1 -- 172.21.15.2:0/3257902713 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0d800c7e0 con 0x7fe0e8100cf0 2023-09-13T04:39:31.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.932+0000 7fe0d77fe700 1 -- 172.21.15.2:0/3257902713 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8003c70 con 0x7fe0e8100cf0 2023-09-13T04:39:31.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.932+0000 7fe0ecbff700 1 -- 172.21.15.2:0/3257902713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e819ebd0 con 0x7fe0e8100cf0 2023-09-13T04:39:31.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.932+0000 7fe0ecbff700 1 -- 172.21.15.2:0/3257902713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e819f090 con 0x7fe0e8100cf0 2023-09-13T04:39:31.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.933+0000 7fe0ecbff700 1 -- 172.21.15.2:0/3257902713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0e810c560 con 0x7fe0e8100cf0 2023-09-13T04:39:31.941 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:31 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/729900055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:39:31.941 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:31 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1681708929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:39:31.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.935+0000 7fe0d77fe700 1 -- 172.21.15.2:0/3257902713 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fe0d800cc50 con 0x7fe0e8100cf0 2023-09-13T04:39:31.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.937+0000 7fe0d77fe700 1 --2- 172.21.15.2:0/3257902713 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe0d00a0690 0x7fe0d00a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:31.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.938+0000 7fe0d77fe700 1 -- 172.21.15.2:0/3257902713 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe0d8014070 con 0x7fe0e8100cf0 2023-09-13T04:39:31.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.938+0000 7fe0e5d9b700 1 --2- 172.21.15.2:0/3257902713 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe0d00a0690 0x7fe0d00a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:31.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.938+0000 7fe0e5d9b700 1 --2- 172.21.15.2:0/3257902713 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe0d00a0690 0x7fe0d00a2b50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc009c60 tx=0x7fe0dc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:31.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:31.942+0000 7fe0d77fe700 1 -- 172.21.15.2:0/3257902713 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe0d8095890 con 0x7fe0e8100cf0 2023-09-13T04:39:32.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:31 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/729900055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:39:32.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:31 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1681708929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:39:32.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.203+0000 7fe0ecbff700 1 -- 172.21.15.2:0/3257902713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0e8107b00 con 0x7fe0e8100cf0 2023-09-13T04:39:32.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.204+0000 7fe0d77fe700 1 -- 172.21.15.2:0/3257902713 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11303 (secure 0 0 0) 0x7fe0d8003dd0 con 0x7fe0e8100cf0 2023-09-13T04:39:32.205 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:39:32.206 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2469307307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2469307307},{"type":"v1","addr":"172.21.15.2:6839","nonce":2469307307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T04:37:59.110701+0000","modified":"2023-09-13T04:38:26.610260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24554,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/846579077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":846579077},{"type":"v1","addr":"172.21.15.156:6835","nonce":846579077}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1896833690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1896833690},{"type":"v1","addr":"172.21.15.149:6837","nonce":1896833690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/524563090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":524563090},{"type":"v1","addr":"172.21.15.156:6837","nonce":524563090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6839/1785833227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1785833227},{"type":"v1","addr":"172.21.15.156:6839","nonce":1785833227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.2:6841/482545391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":482545391},{"type":"v1","addr":"172.21.15.2:6841","nonce":482545391}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/389167841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":389167841},{"type":"v1","addr":"172.21.15.149:6839","nonce":389167841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T04:39:32.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.206+0000 7fe0ecbff700 1 -- 172.21.15.2:0/3257902713 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe0d00a0690 msgr2=0x7fe0d00a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:32.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.206+0000 7fe0ecbff700 1 --2- 172.21.15.2:0/3257902713 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe0d00a0690 0x7fe0d00a2b50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc009c60 tx=0x7fe0dc009260 comp rx=0 tx=0).stop 2023-09-13T04:39:32.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.206+0000 7fe0ecbff700 1 -- 172.21.15.2:0/3257902713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0e8100cf0 msgr2=0x7fe0e8196aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:32.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.206+0000 7fe0ecbff700 1 --2- 172.21.15.2:0/3257902713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0e8100cf0 0x7fe0e8196aa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800e970 tx=0x7fe0d800ec80 comp rx=0 tx=0).stop 2023-09-13T04:39:32.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.206+0000 7fe0ecbff700 1 -- 172.21.15.2:0/3257902713 shutdown_connections 2023-09-13T04:39:32.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.206+0000 7fe0ecbff700 1 --2- 172.21.15.2:0/3257902713 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe0d00a0690 0x7fe0d00a2b50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:32.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.206+0000 7fe0ecbff700 1 --2- 172.21.15.2:0/3257902713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0e8102880 0x7fe0e819e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:32.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.206+0000 7fe0ecbff700 1 --2- 172.21.15.2:0/3257902713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0e8100cf0 0x7fe0e8196aa0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:32.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.206+0000 7fe0ecbff700 1 --2- 172.21.15.2:0/3257902713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0e8101ee0 0x7fe0e8196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:32.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.206+0000 7fe0ecbff700 1 -- 172.21.15.2:0/3257902713 >> 172.21.15.2:0/3257902713 conn(0x7fe0e80fc220 msgr2=0x7fe0e80fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:32.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.206+0000 7fe0ecbff700 1 -- 172.21.15.2:0/3257902713 shutdown_connections 2023-09-13T04:39:32.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.206+0000 7fe0ecbff700 1 -- 172.21.15.2:0/3257902713 wait complete. 2023-09-13T04:39:32.210 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 23 2023-09-13T04:39:32.226 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T04:39:32.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.429+0000 7f6f3bfff700 1 -- 172.21.15.2:0/2461451413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f34101ee0 msgr2=0x7f6f34102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:32.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.429+0000 7f6f3bfff700 1 --2- 172.21.15.2:0/2461451413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f34101ee0 0x7f6f34102340 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6f24009920 tx=0x7f6f24009c30 comp rx=0 tx=0).stop 2023-09-13T04:39:32.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.429+0000 7f6f3bfff700 1 -- 172.21.15.2:0/2461451413 shutdown_connections 2023-09-13T04:39:32.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.429+0000 7f6f3bfff700 1 --2- 172.21.15.2:0/2461451413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f34102880 0x7f6f34109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:32.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.429+0000 7f6f3bfff700 1 --2- 172.21.15.2:0/2461451413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f34100cf0 0x7f6f34101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:32.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.429+0000 7f6f3bfff700 1 --2- 172.21.15.2:0/2461451413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f34101ee0 0x7f6f34102340 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:32.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.429+0000 7f6f3bfff700 1 -- 172.21.15.2:0/2461451413 >> 172.21.15.2:0/2461451413 conn(0x7f6f340fc220 msgr2=0x7f6f340fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:32.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.429+0000 7f6f3bfff700 1 -- 172.21.15.2:0/2461451413 shutdown_connections 2023-09-13T04:39:32.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.429+0000 7f6f3bfff700 1 -- 172.21.15.2:0/2461451413 wait complete. 2023-09-13T04:39:32.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.430+0000 7f6f3bfff700 1 Processor -- start 2023-09-13T04:39:32.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.430+0000 7f6f3bfff700 1 -- start start 2023-09-13T04:39:32.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.431+0000 7f6f3bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f34100cf0 0x7f6f34198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:32.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.431+0000 7f6f3bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f34101ee0 0x7f6f341991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:32.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.431+0000 7f6f3bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f34102880 0x7f6f341a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:32.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.431+0000 7f6f3bfff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f341a07e0 con 0x7f6f34101ee0 2023-09-13T04:39:32.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.431+0000 7f6f3bfff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f341a0950 con 0x7f6f34100cf0 2023-09-13T04:39:32.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.431+0000 7f6f3bfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f341a0b10 con 0x7f6f34102880 2023-09-13T04:39:32.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.431+0000 7f6f3959a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f34101ee0 0x7f6f341991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:32.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.431+0000 7f6f3959a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f34101ee0 0x7f6f341991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51802/0 (socket says 172.21.15.2:51802) 2023-09-13T04:39:32.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.431+0000 7f6f3959a700 1 -- 172.21.15.2:0/3121695761 learned_addr learned my addr 172.21.15.2:0/3121695761 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:32.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.431+0000 7f6f39d9b700 1 --2- 172.21.15.2:0/3121695761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f34100cf0 0x7f6f34198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:32.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.431+0000 7f6f3a59c700 1 --2- 172.21.15.2:0/3121695761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f34102880 0x7f6f341a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:32.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.431+0000 7f6f3959a700 1 -- 172.21.15.2:0/3121695761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f34102880 msgr2=0x7f6f341a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:32.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.431+0000 7f6f3959a700 1 --2- 172.21.15.2:0/3121695761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f34102880 0x7f6f341a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:32.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.431+0000 7f6f3959a700 1 -- 172.21.15.2:0/3121695761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f34100cf0 msgr2=0x7f6f34198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:32.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.431+0000 7f6f3959a700 1 --2- 172.21.15.2:0/3121695761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f34100cf0 0x7f6f34198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:32.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.431+0000 7f6f3959a700 1 -- 172.21.15.2:0/3121695761 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2c009d00 con 0x7f6f34101ee0 2023-09-13T04:39:32.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.431+0000 7f6f3a59c700 1 --2- 172.21.15.2:0/3121695761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f34102880 0x7f6f341a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:39:32.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.432+0000 7f6f3959a700 1 --2- 172.21.15.2:0/3121695761 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f34101ee0 0x7f6f341991b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6f24009230 tx=0x7f6f2400bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:32.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.432+0000 7f6f2affd700 1 -- 172.21.15.2:0/3121695761 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2401c070 con 0x7f6f34101ee0 2023-09-13T04:39:32.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.432+0000 7f6f2affd700 1 -- 172.21.15.2:0/3121695761 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f24005040 con 0x7f6f34101ee0 2023-09-13T04:39:32.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.432+0000 7f6f3bfff700 1 -- 172.21.15.2:0/3121695761 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f24009520 con 0x7f6f34101ee0 2023-09-13T04:39:32.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.432+0000 7f6f2affd700 1 -- 172.21.15.2:0/3121695761 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f24003ae0 con 0x7f6f34101ee0 2023-09-13T04:39:32.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.432+0000 7f6f3bfff700 1 -- 172.21.15.2:0/3121695761 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f341a1020 con 0x7f6f34101ee0 2023-09-13T04:39:32.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.433+0000 7f6f3bfff700 1 -- 172.21.15.2:0/3121695761 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f3410c550 con 0x7f6f34101ee0 2023-09-13T04:39:32.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.435+0000 7f6f2affd700 1 -- 172.21.15.2:0/3121695761 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f6f24020030 con 0x7f6f34101ee0 2023-09-13T04:39:32.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.436+0000 7f6f2affd700 1 --2- 172.21.15.2:0/3121695761 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f6f200a0710 0x7f6f200a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:32.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.436+0000 7f6f2affd700 1 -- 172.21.15.2:0/3121695761 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6f240cce70 con 0x7f6f34101ee0 2023-09-13T04:39:32.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.437+0000 7f6f39d9b700 1 --2- 172.21.15.2:0/3121695761 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f6f200a0710 0x7f6f200a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:32.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.437+0000 7f6f39d9b700 1 --2- 172.21.15.2:0/3121695761 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f6f200a0710 0x7f6f200a2bd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6f34199b40 tx=0x7f6f30009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:32.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.440+0000 7f6f2affd700 1 -- 172.21.15.2:0/3121695761 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6f240966a0 con 0x7f6f34101ee0 2023-09-13T04:39:32.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.656+0000 7f6f3bfff700 1 -- 172.21.15.2:0/3121695761 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f341a11b0 con 0x7f6f34101ee0 2023-09-13T04:39:32.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.658+0000 7f6f2affd700 1 -- 172.21.15.2:0/3121695761 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24503 (secure 0 0 0) 0x7f6f24099cc0 con 0x7f6f34101ee0 2023-09-13T04:39:32.659 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:39:32.659 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":92,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:39:25.580735+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T04:37:57.094802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T04:37:58.564487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-13T04:38:02.096895+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:39:32.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.660+0000 7f6f3bfff700 1 -- 172.21.15.2:0/3121695761 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f6f200a0710 msgr2=0x7f6f200a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:32.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.660+0000 7f6f3bfff700 1 --2- 172.21.15.2:0/3121695761 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f6f200a0710 0x7f6f200a2bd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6f34199b40 tx=0x7f6f30009260 comp rx=0 tx=0).stop 2023-09-13T04:39:32.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.660+0000 7f6f3bfff700 1 -- 172.21.15.2:0/3121695761 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f34101ee0 msgr2=0x7f6f341991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:32.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.660+0000 7f6f3bfff700 1 --2- 172.21.15.2:0/3121695761 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f34101ee0 0x7f6f341991b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6f24009230 tx=0x7f6f2400bf80 comp rx=0 tx=0).stop 2023-09-13T04:39:32.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.660+0000 7f6f3bfff700 1 -- 172.21.15.2:0/3121695761 shutdown_connections 2023-09-13T04:39:32.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.660+0000 7f6f3bfff700 1 --2- 172.21.15.2:0/3121695761 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f6f200a0710 0x7f6f200a2bd0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:32.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.660+0000 7f6f3bfff700 1 --2- 172.21.15.2:0/3121695761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f34102880 0x7f6f341a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:32.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.660+0000 7f6f3bfff700 1 --2- 172.21.15.2:0/3121695761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f34100cf0 0x7f6f34198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:32.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.660+0000 7f6f3bfff700 1 --2- 172.21.15.2:0/3121695761 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f34101ee0 0x7f6f341991b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:32.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.660+0000 7f6f3bfff700 1 -- 172.21.15.2:0/3121695761 >> 172.21.15.2:0/3121695761 conn(0x7f6f340fc220 msgr2=0x7f6f340fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:32.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.660+0000 7f6f3bfff700 1 -- 172.21.15.2:0/3121695761 shutdown_connections 2023-09-13T04:39:32.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.660+0000 7f6f3bfff700 1 -- 172.21.15.2:0/3121695761 wait complete. 2023-09-13T04:39:32.681 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:39:32.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.883+0000 7fea5b5ff700 1 -- 172.21.15.2:0/1081176303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea54102880 msgr2=0x7fea54109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:32.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.883+0000 7fea5b5ff700 1 --2- 172.21.15.2:0/1081176303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea54102880 0x7fea54109130 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009920 tx=0x7fea4c009c30 comp rx=0 tx=0).stop 2023-09-13T04:39:32.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.883+0000 7fea5b5ff700 1 -- 172.21.15.2:0/1081176303 shutdown_connections 2023-09-13T04:39:32.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.883+0000 7fea5b5ff700 1 --2- 172.21.15.2:0/1081176303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea54102880 0x7fea54109130 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:32.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.883+0000 7fea5b5ff700 1 --2- 172.21.15.2:0/1081176303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea54100cf0 0x7fea54101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:32.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.883+0000 7fea5b5ff700 1 --2- 172.21.15.2:0/1081176303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea54101ee0 0x7fea54102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:32.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.883+0000 7fea5b5ff700 1 -- 172.21.15.2:0/1081176303 >> 172.21.15.2:0/1081176303 conn(0x7fea540fc220 msgr2=0x7fea540fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:32.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.883+0000 7fea5b5ff700 1 -- 172.21.15.2:0/1081176303 shutdown_connections 2023-09-13T04:39:32.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.884+0000 7fea5b5ff700 1 -- 172.21.15.2:0/1081176303 wait complete. 2023-09-13T04:39:32.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:32 smithi149 ceph-mon[133315]: pgmap v326: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-13T04:39:32.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:32 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/297236101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:39:32.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:32 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3257902713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:39:32.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.885+0000 7fea5b5ff700 1 Processor -- start 2023-09-13T04:39:32.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.885+0000 7fea5b5ff700 1 -- start start 2023-09-13T04:39:32.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.885+0000 7fea5b5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea54100cf0 0x7fea54198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:32.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.885+0000 7fea5b5ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea54101ee0 0x7fea54199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:32.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.885+0000 7fea5b5ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea54102880 0x7fea541a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:32.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.885+0000 7fea5b5ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea541a0770 con 0x7fea54101ee0 2023-09-13T04:39:32.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.885+0000 7fea5b5ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea541a08e0 con 0x7fea54102880 2023-09-13T04:39:32.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.885+0000 7fea5b5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea541a0aa0 con 0x7fea54100cf0 2023-09-13T04:39:32.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.885+0000 7fea58b9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea54101ee0 0x7fea54199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:32.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.885+0000 7fea58b9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea54101ee0 0x7fea54199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51820/0 (socket says 172.21.15.2:51820) 2023-09-13T04:39:32.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.885+0000 7fea58b9a700 1 -- 172.21.15.2:0/953203655 learned_addr learned my addr 172.21.15.2:0/953203655 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:32.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.885+0000 7fea5939b700 1 --2- 172.21.15.2:0/953203655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea54100cf0 0x7fea54198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:32.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.885+0000 7fea59b9c700 1 --2- 172.21.15.2:0/953203655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea54102880 0x7fea541a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:32.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.886+0000 7fea58b9a700 1 -- 172.21.15.2:0/953203655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea54100cf0 msgr2=0x7fea54198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:32.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.886+0000 7fea58b9a700 1 --2- 172.21.15.2:0/953203655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea54100cf0 0x7fea54198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:32.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.886+0000 7fea58b9a700 1 -- 172.21.15.2:0/953203655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea54102880 msgr2=0x7fea541a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:32.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.886+0000 7fea58b9a700 1 --2- 172.21.15.2:0/953203655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea54102880 0x7fea541a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:32.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.886+0000 7fea58b9a700 1 -- 172.21.15.2:0/953203655 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4c009520 con 0x7fea54101ee0 2023-09-13T04:39:32.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.886+0000 7fea58b9a700 1 --2- 172.21.15.2:0/953203655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea54101ee0 0x7fea54199170 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fea4400e9c0 tx=0x7fea4400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:32.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.886+0000 7fea4a7fc700 1 -- 172.21.15.2:0/953203655 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44004a30 con 0x7fea54101ee0 2023-09-13T04:39:32.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.886+0000 7fea4a7fc700 1 -- 172.21.15.2:0/953203655 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea4400cc80 con 0x7fea54101ee0 2023-09-13T04:39:32.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.886+0000 7fea4a7fc700 1 -- 172.21.15.2:0/953203655 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea440054e0 con 0x7fea54101ee0 2023-09-13T04:39:32.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.886+0000 7fea5b5ff700 1 -- 172.21.15.2:0/953203655 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea541a0d80 con 0x7fea54101ee0 2023-09-13T04:39:32.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.887+0000 7fea5b5ff700 1 -- 172.21.15.2:0/953203655 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea541a1180 con 0x7fea54101ee0 2023-09-13T04:39:32.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.887+0000 7fea5b5ff700 1 -- 172.21.15.2:0/953203655 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea5410c560 con 0x7fea54101ee0 2023-09-13T04:39:32.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.889+0000 7fea4a7fc700 1 -- 172.21.15.2:0/953203655 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fea44004b90 con 0x7fea54101ee0 2023-09-13T04:39:32.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.891+0000 7fea4a7fc700 1 --2- 172.21.15.2:0/953203655 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fea400a0710 0x7fea400a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:32.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.891+0000 7fea4a7fc700 1 -- 172.21.15.2:0/953203655 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fea44014070 con 0x7fea54101ee0 2023-09-13T04:39:32.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.891+0000 7fea5939b700 1 --2- 172.21.15.2:0/953203655 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fea400a0710 0x7fea400a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:32.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.892+0000 7fea5939b700 1 --2- 172.21.15.2:0/953203655 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fea400a0710 0x7fea400a2bd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fea5419a100 tx=0x7fea5000b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:32.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:32.895+0000 7fea4a7fc700 1 -- 172.21.15.2:0/953203655 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fea440d0070 con 0x7fea54101ee0 2023-09-13T04:39:32.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:32 smithi156 ceph-mon[133675]: pgmap v326: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-13T04:39:32.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:32 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/297236101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:39:32.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:32 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3257902713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:39:33.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:32 smithi002 ceph-mon[97651]: pgmap v326: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-13T04:39:33.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:32 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/297236101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:39:33.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:32 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3257902713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:39:33.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.150+0000 7fea5b5ff700 1 -- 172.21.15.2:0/953203655 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea541a1650 con 0x7fea54101ee0 2023-09-13T04:39:33.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.150+0000 7fea4a7fc700 1 -- 172.21.15.2:0/953203655 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11303 (secure 0 0 0) 0x7fea44095710 con 0x7fea54101ee0 2023-09-13T04:39:33.152 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:39:33.152 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2469307307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2469307307},{"type":"v1","addr":"172.21.15.2:6839","nonce":2469307307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T04:37:59.110701+0000","modified":"2023-09-13T04:38:26.610260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24554,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/846579077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":846579077},{"type":"v1","addr":"172.21.15.156:6835","nonce":846579077}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1896833690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1896833690},{"type":"v1","addr":"172.21.15.149:6837","nonce":1896833690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/524563090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":524563090},{"type":"v1","addr":"172.21.15.156:6837","nonce":524563090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6839/1785833227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1785833227},{"type":"v1","addr":"172.21.15.156:6839","nonce":1785833227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.2:6841/482545391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":482545391},{"type":"v1","addr":"172.21.15.2:6841","nonce":482545391}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/389167841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":389167841},{"type":"v1","addr":"172.21.15.149:6839","nonce":389167841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T04:39:33.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.154+0000 7fea5b5ff700 1 -- 172.21.15.2:0/953203655 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fea400a0710 msgr2=0x7fea400a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.154+0000 7fea5b5ff700 1 --2- 172.21.15.2:0/953203655 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fea400a0710 0x7fea400a2bd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fea5419a100 tx=0x7fea5000b040 comp rx=0 tx=0).stop 2023-09-13T04:39:33.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.154+0000 7fea5b5ff700 1 -- 172.21.15.2:0/953203655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea54101ee0 msgr2=0x7fea54199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.154+0000 7fea5b5ff700 1 --2- 172.21.15.2:0/953203655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea54101ee0 0x7fea54199170 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fea4400e9c0 tx=0x7fea4400ecd0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.154+0000 7fea5b5ff700 1 -- 172.21.15.2:0/953203655 shutdown_connections 2023-09-13T04:39:33.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.154+0000 7fea5b5ff700 1 --2- 172.21.15.2:0/953203655 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fea400a0710 0x7fea400a2bd0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.154+0000 7fea5b5ff700 1 --2- 172.21.15.2:0/953203655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea54102880 0x7fea541a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.154+0000 7fea5b5ff700 1 --2- 172.21.15.2:0/953203655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea54100cf0 0x7fea54198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.154+0000 7fea5b5ff700 1 --2- 172.21.15.2:0/953203655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea54101ee0 0x7fea54199170 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.154+0000 7fea5b5ff700 1 -- 172.21.15.2:0/953203655 >> 172.21.15.2:0/953203655 conn(0x7fea540fc220 msgr2=0x7fea540fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:33.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.154+0000 7fea5b5ff700 1 -- 172.21.15.2:0/953203655 shutdown_connections 2023-09-13T04:39:33.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.154+0000 7fea5b5ff700 1 -- 172.21.15.2:0/953203655 wait complete. 2023-09-13T04:39:33.157 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 23 2023-09-13T04:39:33.173 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-09-13T04:37:59.110701+0000', 'modified': '2023-09-13T04:38:26.610260+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24566, 'mds_1': 24554, 'mds_2': 24541}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24511': {'gid': 24511, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6835/846579077', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6834', 'nonce': 846579077}, {'type': 'v1', 'addr': '172.21.15.156:6835', 'nonce': 846579077}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24521': {'gid': 24521, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6837/1896833690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 1896833690}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 1896833690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24523': {'gid': 24523, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6837/524563090', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6836', 'nonce': 524563090}, {'type': 'v1', 'addr': '172.21.15.156:6837', 'nonce': 524563090}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24541': {'gid': 24541, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.156:6839/1785833227', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6838', 'nonce': 1785833227}, {'type': 'v1', 'addr': '172.21.15.156:6839', 'nonce': 1785833227}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24554': {'gid': 24554, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.2:6841/482545391', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6840', 'nonce': 482545391}, {'type': 'v1', 'addr': '172.21.15.2:6841', 'nonce': 482545391}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24566': {'gid': 24566, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.149:6839/389167841', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 389167841}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 389167841}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-13T04:39:33.174 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-09-13T04:39:33.175 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-09-13T04:39:33.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.379+0000 7f4fc8fff700 1 -- 172.21.15.2:0/966106900 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fc4102820 msgr2=0x7f4fc41090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.379+0000 7f4fc8fff700 1 --2- 172.21.15.2:0/966106900 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fc4102820 0x7f4fc41090d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00b0d0 tx=0x7f4fbc00b3e0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.379+0000 7f4fc8fff700 1 -- 172.21.15.2:0/966106900 shutdown_connections 2023-09-13T04:39:33.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.379+0000 7f4fc8fff700 1 --2- 172.21.15.2:0/966106900 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fc4102820 0x7f4fc41090d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.379+0000 7f4fc8fff700 1 --2- 172.21.15.2:0/966106900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fc4100c90 0x7f4fc41010b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.379+0000 7f4fc8fff700 1 --2- 172.21.15.2:0/966106900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc4101e80 0x7f4fc41022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.379+0000 7f4fc8fff700 1 -- 172.21.15.2:0/966106900 >> 172.21.15.2:0/966106900 conn(0x7f4fc40fc200 msgr2=0x7f4fc40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:33.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.379+0000 7f4fc8fff700 1 -- 172.21.15.2:0/966106900 shutdown_connections 2023-09-13T04:39:33.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.379+0000 7f4fc8fff700 1 -- 172.21.15.2:0/966106900 wait complete. 2023-09-13T04:39:33.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.380+0000 7f4fc8fff700 1 Processor -- start 2023-09-13T04:39:33.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.381+0000 7f4fc8fff700 1 -- start start 2023-09-13T04:39:33.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.381+0000 7f4fc8fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fc4100c90 0x7f4fc4196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.381+0000 7f4fc8fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc4101e80 0x7f4fc4196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.381+0000 7f4fc8fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fc4102820 0x7f4fc419e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.381+0000 7f4fc8fff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc419e620 con 0x7f4fc4100c90 2023-09-13T04:39:33.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.381+0000 7f4fc8fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc419e760 con 0x7f4fc4102820 2023-09-13T04:39:33.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.381+0000 7f4fc8fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc419e8f0 con 0x7f4fc4101e80 2023-09-13T04:39:33.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.381+0000 7f4fc259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fc4100c90 0x7f4fc4196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.381+0000 7f4fc259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fc4100c90 0x7f4fc4196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51824/0 (socket says 172.21.15.2:51824) 2023-09-13T04:39:33.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.381+0000 7f4fc259c700 1 -- 172.21.15.2:0/1918328944 learned_addr learned my addr 172.21.15.2:0/1918328944 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:33.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.381+0000 7f4fc1d9b700 1 --2- 172.21.15.2:0/1918328944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc4101e80 0x7f4fc4196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.382+0000 7f4fc2d9d700 1 --2- 172.21.15.2:0/1918328944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fc4102820 0x7f4fc419e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.382+0000 7f4fc259c700 1 -- 172.21.15.2:0/1918328944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc4101e80 msgr2=0x7f4fc4196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.382+0000 7f4fc259c700 1 --2- 172.21.15.2:0/1918328944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc4101e80 0x7f4fc4196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.382+0000 7f4fc259c700 1 -- 172.21.15.2:0/1918328944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fc4102820 msgr2=0x7f4fc419e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.382+0000 7f4fc259c700 1 --2- 172.21.15.2:0/1918328944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fc4102820 0x7f4fc419e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.382+0000 7f4fc259c700 1 -- 172.21.15.2:0/1918328944 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc009d00 con 0x7f4fc4100c90 2023-09-13T04:39:33.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.382+0000 7f4fc1d9b700 1 --2- 172.21.15.2:0/1918328944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc4101e80 0x7f4fc4196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:39:33.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.382+0000 7f4fc2d9d700 1 --2- 172.21.15.2:0/1918328944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fc4102820 0x7f4fc419e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:39:33.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.382+0000 7f4fc259c700 1 --2- 172.21.15.2:0/1918328944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fc4100c90 0x7f4fc4196aa0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400e970 tx=0x7f4fb400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:33.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.382+0000 7f4fb37fe700 1 -- 172.21.15.2:0/1918328944 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4009790 con 0x7f4fc4100c90 2023-09-13T04:39:33.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.382+0000 7f4fc8fff700 1 -- 172.21.15.2:0/1918328944 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc419ebd0 con 0x7f4fc4100c90 2023-09-13T04:39:33.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.382+0000 7f4fb37fe700 1 -- 172.21.15.2:0/1918328944 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fb40044e0 con 0x7f4fc4100c90 2023-09-13T04:39:33.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.382+0000 7f4fb37fe700 1 -- 172.21.15.2:0/1918328944 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4003e30 con 0x7f4fc4100c90 2023-09-13T04:39:33.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.382+0000 7f4fc8fff700 1 -- 172.21.15.2:0/1918328944 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc419f0e0 con 0x7f4fc4100c90 2023-09-13T04:39:33.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.386+0000 7f4fb37fe700 1 -- 172.21.15.2:0/1918328944 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f4fb4004000 con 0x7f4fc4100c90 2023-09-13T04:39:33.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.387+0000 7f4fdca04b80 1 Processor -- start 2023-09-13T04:39:33.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.388+0000 7f4fdca04b80 1 -- start start 2023-09-13T04:39:33.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.388+0000 7f4fdca04b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c1cbae6b0 0x563c1cbaead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.388+0000 7f4fdca04b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbaf010 0x563c1cbad470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.388+0000 7f4fdca04b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c1cbad9b0 0x563c1cbade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.388+0000 7f4fdca04b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c1cb06730 con 0x563c1cbaf010 2023-09-13T04:39:33.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.388+0000 7f4fdca04b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c1cb04050 con 0x563c1cbae6b0 2023-09-13T04:39:33.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.388+0000 7f4fdca04b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c1caff940 con 0x563c1cbad9b0 2023-09-13T04:39:33.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.388+0000 7f4fb37fe700 1 --2- 172.21.15.2:0/1918328944 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4fac0a0640 0x7f4fac0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.388+0000 7f4fc259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbaf010 0x563c1cbad470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.388+0000 7f4fc259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbaf010 0x563c1cbad470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51828/0 (socket says 172.21.15.2:51828) 2023-09-13T04:39:33.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.388+0000 7f4fc259c700 1 -- 172.21.15.2:0/1261432889 learned_addr learned my addr 172.21.15.2:0/1261432889 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:33.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.388+0000 7f4fc2d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c1cbae6b0 0x563c1cbaead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.389+0000 7f4fc2d9d700 1 --2- 172.21.15.2:0/1918328944 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4fac0a0640 0x7f4fac0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.389+0000 7f4fc1d9b700 1 --2- 172.21.15.2:0/1261432889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c1cbad9b0 0x563c1cbade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.389+0000 7f4fb37fe700 1 -- 172.21.15.2:0/1918328944 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4fb4014070 con 0x7f4fc4100c90 2023-09-13T04:39:33.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.389+0000 7f4fc259c700 1 -- 172.21.15.2:0/1261432889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c1cbad9b0 msgr2=0x563c1cbade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:39:33.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.389+0000 7f4fc259c700 1 --2- 172.21.15.2:0/1261432889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c1cbad9b0 0x563c1cbade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.389+0000 7f4fc259c700 1 -- 172.21.15.2:0/1261432889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c1cbae6b0 msgr2=0x563c1cbaead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:39:33.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.389+0000 7f4fc259c700 1 --2- 172.21.15.2:0/1261432889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c1cbae6b0 0x563c1cbaead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.389+0000 7f4fc259c700 1 -- 172.21.15.2:0/1261432889 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c1c97f5e0 con 0x563c1cbaf010 2023-09-13T04:39:33.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.389+0000 7f4fc1d9b700 1 --2- 172.21.15.2:0/1261432889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c1cbad9b0 0x563c1cbade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:39:33.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.389+0000 7f4fc259c700 1 --2- 172.21.15.2:0/1261432889 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbaf010 0x563c1cbad470 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4064f00 tx=0x7f4fb406c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb1d5e1c9dd3abd server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:33.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.389+0000 7f4fc2d9d700 1 --2- 172.21.15.2:0/1918328944 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4fac0a0640 0x7f4fac0a2b00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00b770 tx=0x7f4fbc00b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:33.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.389+0000 7f4fb0ff9700 1 -- 172.21.15.2:0/1261432889 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4099010 con 0x563c1cbaf010 2023-09-13T04:39:33.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.389+0000 7f4fb0ff9700 1 -- 172.21.15.2:0/1261432889 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fb4067070 con 0x563c1cbaf010 2023-09-13T04:39:33.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.389+0000 7f4fb0ff9700 1 -- 172.21.15.2:0/1261432889 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb40996b0 con 0x563c1cbaf010 2023-09-13T04:39:33.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.390+0000 7f4fdca04b80 1 -- 172.21.15.2:0/1261432889 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbaf010 msgr2=0x563c1cbad470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.390+0000 7f4fdca04b80 1 --2- 172.21.15.2:0/1261432889 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbaf010 0x563c1cbad470 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4064f00 tx=0x7f4fb406c820 comp rx=0 tx=0).stop 2023-09-13T04:39:33.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.390+0000 7f4fdca04b80 1 -- 172.21.15.2:0/1261432889 shutdown_connections 2023-09-13T04:39:33.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.390+0000 7f4fdca04b80 1 --2- 172.21.15.2:0/1261432889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c1cbad9b0 0x563c1cbade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.390+0000 7f4fdca04b80 1 --2- 172.21.15.2:0/1261432889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c1cbae6b0 0x563c1cbaead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.390+0000 7f4fdca04b80 1 --2- 172.21.15.2:0/1261432889 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbaf010 0x563c1cbad470 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.390+0000 7f4fdca04b80 1 -- 172.21.15.2:0/1261432889 >> 172.21.15.2:0/1261432889 conn(0x563c1cb0bc30 msgr2=0x563c1cba7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:33.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.390+0000 7f4fdca04b80 1 -- 172.21.15.2:0/1261432889 shutdown_connections 2023-09-13T04:39:33.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.390+0000 7f4fdca04b80 1 -- 172.21.15.2:0/1261432889 wait complete. 2023-09-13T04:39:33.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.391+0000 7f4fdca04b80 1 -- start start 2023-09-13T04:39:33.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.391+0000 7f4fdca04b80 1 Processor -- start 2023-09-13T04:39:33.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fdca04b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbad9b0 0x563c1cbb6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fdca04b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c1cbae6b0 0x563c1cbb7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fdca04b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c1cbaf010 0x563c1cbbe3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fdca04b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c1ca9db60 con 0x563c1cbad9b0 2023-09-13T04:39:33.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fdca04b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c1ca9d860 con 0x563c1cbae6b0 2023-09-13T04:39:33.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fdca04b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c1c9818d0 con 0x563c1cbaf010 2023-09-13T04:39:33.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fdca04b80 10 client.0 _ll_register_callbacks cb 0x563c1ca9b630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:39:33.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fc259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbad9b0 0x563c1cbb6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fc259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbad9b0 0x563c1cbb6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51838/0 (socket says 172.21.15.2:51838) 2023-09-13T04:39:33.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fc259c700 1 -- 172.21.15.2:0/2643593890 learned_addr learned my addr 172.21.15.2:0/2643593890 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:33.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fc1d9b700 1 --2- 172.21.15.2:0/2643593890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c1cbae6b0 0x563c1cbb7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fc2d9d700 1 --2- 172.21.15.2:0/2643593890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c1cbaf010 0x563c1cbbe3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fc259c700 1 -- 172.21.15.2:0/2643593890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c1cbaf010 msgr2=0x563c1cbbe3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:39:33.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fc259c700 1 --2- 172.21.15.2:0/2643593890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c1cbaf010 0x563c1cbbe3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fc259c700 1 -- 172.21.15.2:0/2643593890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c1cbae6b0 msgr2=0x563c1cbb7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:39:33.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fc259c700 1 --2- 172.21.15.2:0/2643593890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c1cbae6b0 0x563c1cbb7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fc259c700 1 -- 172.21.15.2:0/2643593890 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc419f0e0 con 0x563c1cbad9b0 2023-09-13T04:39:33.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fc1d9b700 1 --2- 172.21.15.2:0/2643593890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c1cbae6b0 0x563c1cbb7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:39:33.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fc2d9d700 1 --2- 172.21.15.2:0/2643593890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c1cbaf010 0x563c1cbbe3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:39:33.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fc259c700 1 --2- 172.21.15.2:0/2643593890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbad9b0 0x563c1cbb6e20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40a11f0 tx=0x7f4fb4003790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56564c4d1e3bdcc server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:33.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fa9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:39:33.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fa9ffb700 1 -- 172.21.15.2:0/2643593890 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb40996b0 con 0x563c1cbad9b0 2023-09-13T04:39:33.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fdca04b80 1 -- 172.21.15.2:0/2643593890 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c1cba0620 con 0x563c1cbad9b0 2023-09-13T04:39:33.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fa9ffb700 1 -- 172.21.15.2:0/2643593890 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4fb40977f0 con 0x563c1cbad9b0 2023-09-13T04:39:33.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fa9ffb700 1 -- 172.21.15.2:0/2643593890 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fb4097a50 con 0x563c1cbad9b0 2023-09-13T04:39:33.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fdca04b80 10 client.15090 fetch_fsmap learned FSMap version 23 2023-09-13T04:39:33.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.393+0000 7f4fdca04b80 1 -- 172.21.15.2:0/2643593890 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563c1cb17750 con 0x563c1cbad9b0 2023-09-13T04:39:33.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.394+0000 7f4fa9ffb700 1 -- 172.21.15.2:0/2643593890 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563c1cba0620 con 0x563c1cbad9b0 2023-09-13T04:39:33.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.394+0000 7f4fa9ffb700 1 -- 172.21.15.2:0/2643593890 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4fb406c400 con 0x563c1cbad9b0 2023-09-13T04:39:33.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.394+0000 7f4fdca04b80 10 client.15090 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:39:33.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.394+0000 7f4fdca04b80 10 client.15090 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-13T04:39:33.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.394+0000 7f4fdca04b80 20 client.15090 populate_metadata read hostname 'smithi002' 2023-09-13T04:39:33.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.394+0000 7f4fdca04b80 1 --2- 172.21.15.2:0/2643593890 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x563c1cbc26b0 0x563c1cbc12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.394+0000 7f4fdca04b80 4 client.15090 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:39:33.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.394+0000 7f4fdca04b80 1 -- 172.21.15.2:0/2643593890 --> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c1caf8920 con 0x563c1cbc26b0 2023-09-13T04:39:33.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.394+0000 7f4fc1d9b700 1 --2- 172.21.15.2:0/2643593890 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x563c1cbc26b0 0x563c1cbc12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.395+0000 7f4fc1d9b700 1 --2- 172.21.15.2:0/2643593890 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x563c1cbc26b0 0x563c1cbc12d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ede08928a91b2558 server_cookie=ea1bd685ead65702 in_seq=0 out_seq=0 2023-09-13T04:39:33.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.395+0000 7f4fa9ffb700 10 client.15090 ms_handle_connect on v2:172.21.15.156:6838/1785833227 2023-09-13T04:39:33.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.396+0000 7f4fa9ffb700 1 -- 172.21.15.2:0/2643593890 <== mds.2 v2:172.21.15.156:6838/1785833227 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563c1caf8920 con 0x563c1cbc26b0 2023-09-13T04:39:33.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.396+0000 7f4fa9ffb700 10 client.15090 handle_command_reply: tid=0 2023-09-13T04:39:33.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.396+0000 7f4fdca04b80 1 client.15090 shutdown 2023-09-13T04:39:33.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.396+0000 7f4fdca04b80 1 -- 172.21.15.2:0/2643593890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbad9b0 msgr2=0x563c1cbb6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.396+0000 7f4fdca04b80 1 --2- 172.21.15.2:0/2643593890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbad9b0 0x563c1cbb6e20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40a11f0 tx=0x7f4fb4003790 comp rx=0 tx=0).stop 2023-09-13T04:39:33.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.396+0000 7f4fdca04b80 1 -- 172.21.15.2:0/2643593890 shutdown_connections 2023-09-13T04:39:33.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.396+0000 7f4fdca04b80 1 --2- 172.21.15.2:0/2643593890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c1cbaf010 0x563c1cbbe3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.396+0000 7f4fdca04b80 1 --2- 172.21.15.2:0/2643593890 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x563c1cbc26b0 0x563c1cbc12d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.396+0000 7f4fdca04b80 1 --2- 172.21.15.2:0/2643593890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbad9b0 0x563c1cbb6e20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.396+0000 7f4fdca04b80 1 --2- 172.21.15.2:0/2643593890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c1cbae6b0 0x563c1cbb7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.396+0000 7f4fdca04b80 1 -- 172.21.15.2:0/2643593890 >> 172.21.15.2:0/2643593890 conn(0x563c1cb0bc30 msgr2=0x563c1cba7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:33.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.396+0000 7f4fa9ffb700 0 client.15090 ms_handle_reset on v2:172.21.15.156:6838/1785833227 2023-09-13T04:39:33.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.398+0000 7f4fdca04b80 1 -- 172.21.15.2:0/2643593890 shutdown_connections 2023-09-13T04:39:33.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.398+0000 7f4fdca04b80 1 -- 172.21.15.2:0/2643593890 wait complete. 2023-09-13T04:39:33.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.398+0000 7f4fdca04b80 20 client.15090 trim_cache size 0 max 16384 2023-09-13T04:39:33.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.415+0000 7f4fdca04b80 1 Processor -- start 2023-09-13T04:39:33.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.416+0000 7f4fdca04b80 1 -- start start 2023-09-13T04:39:33.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.416+0000 7f4fdca04b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbad9b0 0x563c1cbad470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.416+0000 7f4fdca04b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c1cbae6b0 0x563c1cbbe360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.416+0000 7f4fdca04b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c1cbaf010 0x563c1cbb4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.416+0000 7f4fdca04b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c1caf8920 con 0x563c1cbad9b0 2023-09-13T04:39:33.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.416+0000 7f4fdca04b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c1cba0620 con 0x563c1cbae6b0 2023-09-13T04:39:33.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.416+0000 7f4fdca04b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c1caa7470 con 0x563c1cbaf010 2023-09-13T04:39:33.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.416+0000 7f4fc259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbad9b0 0x563c1cbad470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.416+0000 7f4fc259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbad9b0 0x563c1cbad470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51854/0 (socket says 172.21.15.2:51854) 2023-09-13T04:39:33.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.416+0000 7f4fc259c700 1 -- 172.21.15.2:0/2764493473 learned_addr learned my addr 172.21.15.2:0/2764493473 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:33.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.416+0000 7f4fc2d9d700 1 --2- 172.21.15.2:0/2764493473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c1cbaf010 0x563c1cbb4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.416+0000 7f4fc1d9b700 1 --2- 172.21.15.2:0/2764493473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c1cbae6b0 0x563c1cbbe360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.417+0000 7f4fc259c700 1 -- 172.21.15.2:0/2764493473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c1cbaf010 msgr2=0x563c1cbb4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:39:33.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.417+0000 7f4fc259c700 1 --2- 172.21.15.2:0/2764493473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c1cbaf010 0x563c1cbb4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.417+0000 7f4fc259c700 1 -- 172.21.15.2:0/2764493473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c1cbae6b0 msgr2=0x563c1cbbe360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:39:33.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.417+0000 7f4fc259c700 1 --2- 172.21.15.2:0/2764493473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c1cbae6b0 0x563c1cbbe360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.417+0000 7f4fc259c700 1 -- 172.21.15.2:0/2764493473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c1c92bd90 con 0x563c1cbad9b0 2023-09-13T04:39:33.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.417+0000 7f4fc259c700 1 --2- 172.21.15.2:0/2764493473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbad9b0 0x563c1cbad470 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400c5b0 tx=0x7f4fb4004650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c785816afdd48714 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:33.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.417+0000 7f4fb17fa700 1 -- 172.21.15.2:0/2764493473 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4099010 con 0x563c1cbad9b0 2023-09-13T04:39:33.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.417+0000 7f4fb17fa700 1 -- 172.21.15.2:0/2764493473 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fb40996b0 con 0x563c1cbad9b0 2023-09-13T04:39:33.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.417+0000 7f4fb17fa700 1 -- 172.21.15.2:0/2764493473 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb405f950 con 0x563c1cbad9b0 2023-09-13T04:39:33.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.418+0000 7f4fdca04b80 1 -- 172.21.15.2:0/2764493473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbad9b0 msgr2=0x563c1cbad470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.418+0000 7f4fdca04b80 1 --2- 172.21.15.2:0/2764493473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbad9b0 0x563c1cbad470 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400c5b0 tx=0x7f4fb4004650 comp rx=0 tx=0).stop 2023-09-13T04:39:33.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.418+0000 7f4fdca04b80 1 -- 172.21.15.2:0/2764493473 shutdown_connections 2023-09-13T04:39:33.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.418+0000 7f4fdca04b80 1 --2- 172.21.15.2:0/2764493473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c1cbaf010 0x563c1cbb4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.418+0000 7f4fdca04b80 1 --2- 172.21.15.2:0/2764493473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbad9b0 0x563c1cbad470 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.418+0000 7f4fdca04b80 1 --2- 172.21.15.2:0/2764493473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c1cbae6b0 0x563c1cbbe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.418+0000 7f4fdca04b80 1 -- 172.21.15.2:0/2764493473 >> 172.21.15.2:0/2764493473 conn(0x563c1cb0bc30 msgr2=0x563c1cba7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:33.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.418+0000 7f4fdca04b80 1 -- 172.21.15.2:0/2764493473 shutdown_connections 2023-09-13T04:39:33.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.418+0000 7f4fdca04b80 1 -- 172.21.15.2:0/2764493473 wait complete. 2023-09-13T04:39:33.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.418+0000 7f4fdca04b80 1 -- start start 2023-09-13T04:39:33.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.419+0000 7f4fdca04b80 1 Processor -- start 2023-09-13T04:39:33.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.419+0000 7f4fdca04b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbad9b0 0x563c1cbb84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.419+0000 7f4fdca04b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c1cbae6b0 0x563c1cbb8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.419+0000 7f4fdca04b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c1cbaf010 0x563c1cbbd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.419+0000 7f4fdca04b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c1cbbde00 con 0x563c1cbad9b0 2023-09-13T04:39:33.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.419+0000 7f4fdca04b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c1cbbdf70 con 0x563c1cbae6b0 2023-09-13T04:39:33.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.419+0000 7f4fdca04b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c1cbbe130 con 0x563c1cbaf010 2023-09-13T04:39:33.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.419+0000 7f4fdca04b80 10 client.0 _ll_register_callbacks cb 0x563c1ca9b630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:39:33.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.420+0000 7f4fc259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbad9b0 0x563c1cbb84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.420+0000 7f4fc1d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c1cbae6b0 0x563c1cbb8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.420+0000 7f4fc1d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c1cbae6b0 0x563c1cbb8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:48378/0 (socket says 172.21.15.2:48378) 2023-09-13T04:39:33.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.420+0000 7f4fc2d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c1cbaf010 0x563c1cbbd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.420+0000 7f4fc1d9b700 1 -- 172.21.15.2:0/1849027977 learned_addr learned my addr 172.21.15.2:0/1849027977 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:33.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.420+0000 7f4fc1d9b700 1 -- 172.21.15.2:0/1849027977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c1cbaf010 msgr2=0x563c1cbbd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:39:33.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.420+0000 7f4fc1d9b700 1 --2- 172.21.15.2:0/1849027977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c1cbaf010 0x563c1cbbd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.420+0000 7f4fc1d9b700 1 -- 172.21.15.2:0/1849027977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbad9b0 msgr2=0x563c1cbb84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:39:33.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.420+0000 7f4fc1d9b700 1 --2- 172.21.15.2:0/1849027977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbad9b0 0x563c1cbb84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.420+0000 7f4fc1d9b700 1 -- 172.21.15.2:0/1849027977 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c1cb17750 con 0x563c1cbae6b0 2023-09-13T04:39:33.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.420+0000 7f4fc1d9b700 1 --2- 172.21.15.2:0/1849027977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c1cbae6b0 0x563c1cbb8a20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x563c1cbc33e0 tx=0x7f4fb801dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1f2b5fac34b2157 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:33.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.420+0000 7f4fa9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:39:33.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.421+0000 7f4fa9ffb700 1 -- 172.21.15.2:0/1849027977 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8018040 con 0x563c1cbae6b0 2023-09-13T04:39:33.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.421+0000 7f4fdca04b80 1 -- 172.21.15.2:0/1849027977 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4fb4067070 con 0x563c1cbae6b0 2023-09-13T04:39:33.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.421+0000 7f4fa9ffb700 1 -- 172.21.15.2:0/1849027977 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4fb8004050 con 0x563c1cbae6b0 2023-09-13T04:39:33.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.421+0000 7f4fa9ffb700 1 -- 172.21.15.2:0/1849027977 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fb80142d0 con 0x563c1cbae6b0 2023-09-13T04:39:33.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.421+0000 7f4fdca04b80 10 client.25130 fetch_fsmap learned FSMap version 23 2023-09-13T04:39:33.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.421+0000 7f4fdca04b80 1 -- 172.21.15.2:0/1849027977 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563c1c914c10 con 0x563c1cbae6b0 2023-09-13T04:39:33.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.421+0000 7f4fa9ffb700 1 -- 172.21.15.2:0/1849027977 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fb4067070 con 0x563c1cbae6b0 2023-09-13T04:39:33.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.421+0000 7f4fa9ffb700 1 -- 172.21.15.2:0/1849027977 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4fb80144d0 con 0x563c1cbae6b0 2023-09-13T04:39:33.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.421+0000 7f4fdca04b80 10 client.25130 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:39:33.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.421+0000 7f4fdca04b80 10 client.25130 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-13T04:39:33.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.421+0000 7f4fdca04b80 20 client.25130 populate_metadata read hostname 'smithi002' 2023-09-13T04:39:33.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.421+0000 7f4fdca04b80 1 --2- 172.21.15.2:0/1849027977 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x563c1cc1ac70 0x563c1cc170d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.421+0000 7f4fdca04b80 4 client.25130 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "status"}] 2023-09-13T04:39:33.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.421+0000 7f4fdca04b80 1 -- 172.21.15.2:0/1849027977 --> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563c1cab9000 con 0x563c1cc1ac70 2023-09-13T04:39:33.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.422+0000 7f4fc259c700 1 --2- 172.21.15.2:0/1849027977 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x563c1cc1ac70 0x563c1cc170d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.422+0000 7f4fc259c700 1 --2- 172.21.15.2:0/1849027977 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x563c1cc1ac70 0x563c1cc170d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c4d663139aaef888 server_cookie=845be71449d25b17 in_seq=0 out_seq=0 2023-09-13T04:39:33.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.422+0000 7f4fa9ffb700 10 client.25130 ms_handle_connect on v2:172.21.15.156:6838/1785833227 2023-09-13T04:39:33.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.422+0000 7f4fa9ffb700 1 -- 172.21.15.2:0/1849027977 <== mds.2 v2:172.21.15.156:6838/1785833227 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x563c1cab9000 con 0x563c1cc1ac70 2023-09-13T04:39:33.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.422+0000 7f4fa9ffb700 10 client.25130 handle_command_reply: tid=0 2023-09-13T04:39:33.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.422+0000 7f4fdca04b80 1 client.25130 shutdown 2023-09-13T04:39:33.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.423+0000 7f4fdca04b80 1 -- 172.21.15.2:0/1849027977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c1cbae6b0 msgr2=0x563c1cbb8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.423+0000 7f4fdca04b80 1 --2- 172.21.15.2:0/1849027977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c1cbae6b0 0x563c1cbb8a20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x563c1cbc33e0 tx=0x7f4fb801dbe0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.423+0000 7f4fdca04b80 1 -- 172.21.15.2:0/1849027977 shutdown_connections 2023-09-13T04:39:33.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.423+0000 7f4fdca04b80 1 --2- 172.21.15.2:0/1849027977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c1cbaf010 0x563c1cbbd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.423+0000 7f4fdca04b80 1 --2- 172.21.15.2:0/1849027977 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x563c1cc1ac70 0x563c1cc170d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.423+0000 7f4fdca04b80 1 --2- 172.21.15.2:0/1849027977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c1cbad9b0 0x563c1cbb84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.423+0000 7f4fdca04b80 1 --2- 172.21.15.2:0/1849027977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c1cbae6b0 0x563c1cbb8a20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.423+0000 7f4fa9ffb700 0 client.25130 ms_handle_reset on v2:172.21.15.156:6838/1785833227 2023-09-13T04:39:33.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.423+0000 7f4fdca04b80 1 -- 172.21.15.2:0/1849027977 >> 172.21.15.2:0/1849027977 conn(0x563c1cb0bc30 msgr2=0x563c1cbb4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:33.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.423+0000 7f4fdca04b80 1 -- 172.21.15.2:0/1849027977 shutdown_connections 2023-09-13T04:39:33.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.423+0000 7f4fdca04b80 1 -- 172.21.15.2:0/1849027977 wait complete. 2023-09-13T04:39:33.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.423+0000 7f4fdca04b80 20 client.25130 trim_cache size 0 max 16384 2023-09-13T04:39:33.447 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:39:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da", 2023-09-13T04:39:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 2, 2023-09-13T04:39:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24541, 2023-09-13T04:39:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2023-09-13T04:39:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2023-09-13T04:39:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2023-09-13T04:39:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 67.814970479999999, 2023-09-13T04:39:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 23, 2023-09-13T04:39:33.449 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 92, 2023-09-13T04:39:33.449 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 90, 2023-09-13T04:39:33.449 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 109.212668646 2023-09-13T04:39:33.449 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:39:33.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.423+0000 7f4fc8fff700 1 -- 172.21.15.2:0/1918328944 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4fac0a0640 msgr2=0x7f4fac0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.423+0000 7f4fc8fff700 1 --2- 172.21.15.2:0/1918328944 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4fac0a0640 0x7f4fac0a2b00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00b770 tx=0x7f4fbc00b040 comp rx=0 tx=0).stop 2023-09-13T04:39:33.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.423+0000 7f4fc8fff700 1 -- 172.21.15.2:0/1918328944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fc4100c90 msgr2=0x7f4fc4196aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.423+0000 7f4fc8fff700 1 --2- 172.21.15.2:0/1918328944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fc4100c90 0x7f4fc4196aa0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400e970 tx=0x7f4fb400ed30 comp rx=0 tx=0).stop 2023-09-13T04:39:33.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.424+0000 7f4fc8fff700 1 -- 172.21.15.2:0/1918328944 shutdown_connections 2023-09-13T04:39:33.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.424+0000 7f4fc8fff700 1 --2- 172.21.15.2:0/1918328944 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4fac0a0640 0x7f4fac0a2b00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.424+0000 7f4fc8fff700 1 --2- 172.21.15.2:0/1918328944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fc4102820 0x7f4fc419e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.424+0000 7f4fc8fff700 1 --2- 172.21.15.2:0/1918328944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fc4100c90 0x7f4fc4196aa0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.424+0000 7f4fc8fff700 1 --2- 172.21.15.2:0/1918328944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc4101e80 0x7f4fc4196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.424+0000 7f4fc8fff700 1 -- 172.21.15.2:0/1918328944 >> 172.21.15.2:0/1918328944 conn(0x7f4fc40fc200 msgr2=0x7f4fc4105340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:33.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.424+0000 7f4fc8fff700 1 -- 172.21.15.2:0/1918328944 shutdown_connections 2023-09-13T04:39:33.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.424+0000 7f4fc8fff700 1 -- 172.21.15.2:0/1918328944 wait complete. 2023-09-13T04:39:33.454 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-09-13T04:39:33.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.661+0000 7f86cf9ff700 1 -- 172.21.15.2:0/2208105848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86c8102870 msgr2=0x7f86c8109120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.661+0000 7f86cf9ff700 1 --2- 172.21.15.2:0/2208105848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86c8102870 0x7f86c8109120 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f86c000b0d0 tx=0x7f86c000b3e0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.661+0000 7f86cf9ff700 1 -- 172.21.15.2:0/2208105848 shutdown_connections 2023-09-13T04:39:33.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.661+0000 7f86cf9ff700 1 --2- 172.21.15.2:0/2208105848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86c8102870 0x7f86c8109120 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.661+0000 7f86cf9ff700 1 --2- 172.21.15.2:0/2208105848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c8100ce0 0x7f86c8101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.661+0000 7f86cf9ff700 1 --2- 172.21.15.2:0/2208105848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86c8101ed0 0x7f86c8102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.661+0000 7f86cf9ff700 1 -- 172.21.15.2:0/2208105848 >> 172.21.15.2:0/2208105848 conn(0x7f86c80fc230 msgr2=0x7f86c80fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:33.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.661+0000 7f86cf9ff700 1 -- 172.21.15.2:0/2208105848 shutdown_connections 2023-09-13T04:39:33.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.661+0000 7f86cf9ff700 1 -- 172.21.15.2:0/2208105848 wait complete. 2023-09-13T04:39:33.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.662+0000 7f86cf9ff700 1 Processor -- start 2023-09-13T04:39:33.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.663+0000 7f86cf9ff700 1 -- start start 2023-09-13T04:39:33.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.663+0000 7f86cf9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c8100ce0 0x7f86c8198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.663+0000 7f86cf9ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86c8101ed0 0x7f86c81991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.663+0000 7f86cf9ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86c8102870 0x7f86c81a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.663+0000 7f86cf9ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c81a07f0 con 0x7f86c8102870 2023-09-13T04:39:33.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.663+0000 7f86cf9ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c81a0960 con 0x7f86c8101ed0 2023-09-13T04:39:33.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.663+0000 7f86cf9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c81a0b20 con 0x7f86c8100ce0 2023-09-13T04:39:33.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.663+0000 7f86cdf9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86c8102870 0x7f86c81a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.664+0000 7f86cdf9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86c8102870 0x7f86c81a0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51878/0 (socket says 172.21.15.2:51878) 2023-09-13T04:39:33.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.664+0000 7f86cdf9c700 1 -- 172.21.15.2:0/2448781096 learned_addr learned my addr 172.21.15.2:0/2448781096 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:33.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.664+0000 7f86cd79b700 1 --2- 172.21.15.2:0/2448781096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c8100ce0 0x7f86c8198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.664+0000 7f86ccf9a700 1 --2- 172.21.15.2:0/2448781096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86c8101ed0 0x7f86c81991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.664+0000 7f86cdf9c700 1 -- 172.21.15.2:0/2448781096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c8100ce0 msgr2=0x7f86c8198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.664+0000 7f86cdf9c700 1 --2- 172.21.15.2:0/2448781096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c8100ce0 0x7f86c8198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.664+0000 7f86cdf9c700 1 -- 172.21.15.2:0/2448781096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86c8101ed0 msgr2=0x7f86c81991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.664+0000 7f86cdf9c700 1 --2- 172.21.15.2:0/2448781096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86c8101ed0 0x7f86c81991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.664+0000 7f86cdf9c700 1 -- 172.21.15.2:0/2448781096 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c0009d00 con 0x7f86c8102870 2023-09-13T04:39:33.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.664+0000 7f86cdf9c700 1 --2- 172.21.15.2:0/2448781096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86c8102870 0x7f86c81a0220 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f86c0008f10 tx=0x7f86c0008f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:33.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.664+0000 7f86be7fc700 1 -- 172.21.15.2:0/2448781096 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c000e040 con 0x7f86c8102870 2023-09-13T04:39:33.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.664+0000 7f86be7fc700 1 -- 172.21.15.2:0/2448781096 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86c0005070 con 0x7f86c8102870 2023-09-13T04:39:33.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.664+0000 7f86cf9ff700 1 -- 172.21.15.2:0/2448781096 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c81a0da0 con 0x7f86c8102870 2023-09-13T04:39:33.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.664+0000 7f86be7fc700 1 -- 172.21.15.2:0/2448781096 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c001b6a0 con 0x7f86c8102870 2023-09-13T04:39:33.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.665+0000 7f86cf9ff700 1 -- 172.21.15.2:0/2448781096 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c81a11d0 con 0x7f86c8102870 2023-09-13T04:39:33.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.668+0000 7f86be7fc700 1 -- 172.21.15.2:0/2448781096 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f86c00051e0 con 0x7f86c8102870 2023-09-13T04:39:33.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.669+0000 7f86be7fc700 1 --2- 172.21.15.2:0/2448781096 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f86b40a06c0 0x7f86b40a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.669+0000 7f86e335bb80 1 Processor -- start 2023-09-13T04:39:33.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.669+0000 7f86be7fc700 1 -- 172.21.15.2:0/2448781096 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f86c00ccfe0 con 0x7f86c8102870 2023-09-13T04:39:33.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.669+0000 7f86cd79b700 1 --2- 172.21.15.2:0/2448781096 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f86b40a06c0 0x7f86b40a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.669+0000 7f86e335bb80 1 -- start start 2023-09-13T04:39:33.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.670+0000 7f86e335bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55693d7b16b0 0x55693d7b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.670+0000 7f86e335bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b2010 0x55693d7b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.670+0000 7f86e335bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b09b0 0x55693d7b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.670+0000 7f86e335bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55693d709730 con 0x55693d7b09b0 2023-09-13T04:39:33.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.670+0000 7f86e335bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55693d707050 con 0x55693d7b2010 2023-09-13T04:39:33.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.670+0000 7f86e335bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55693d702940 con 0x55693d7b16b0 2023-09-13T04:39:33.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.670+0000 7f86cd79b700 1 --2- 172.21.15.2:0/2448781096 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f86b40a06c0 0x7f86b40a2b80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f86c8199b50 tx=0x7f86c4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:33.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.670+0000 7f86cdf9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b09b0 0x55693d7b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.670+0000 7f86cdf9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b09b0 0x55693d7b0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51882/0 (socket says 172.21.15.2:51882) 2023-09-13T04:39:33.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.670+0000 7f86cdf9c700 1 -- 172.21.15.2:0/3229823475 learned_addr learned my addr 172.21.15.2:0/3229823475 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:33.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.670+0000 7f86ccf9a700 1 --2- 172.21.15.2:0/3229823475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b2010 0x55693d7b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.670+0000 7f86cd79b700 1 --2- 172.21.15.2:0/3229823475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55693d7b16b0 0x55693d7b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.671+0000 7f86cdf9c700 1 -- 172.21.15.2:0/3229823475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55693d7b16b0 msgr2=0x55693d7b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:39:33.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.671+0000 7f86cdf9c700 1 --2- 172.21.15.2:0/3229823475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55693d7b16b0 0x55693d7b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.671+0000 7f86cdf9c700 1 -- 172.21.15.2:0/3229823475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b2010 msgr2=0x55693d7b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:39:33.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.671+0000 7f86cdf9c700 1 --2- 172.21.15.2:0/3229823475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b2010 0x55693d7b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.671+0000 7f86cdf9c700 1 -- 172.21.15.2:0/3229823475 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55693d5825e0 con 0x55693d7b09b0 2023-09-13T04:39:33.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.671+0000 7f86ccf9a700 1 --2- 172.21.15.2:0/3229823475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b2010 0x55693d7b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:39:33.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.671+0000 7f86cdf9c700 1 --2- 172.21.15.2:0/3229823475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b09b0 0x55693d7b0e30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f86c0007990 tx=0x7f86c0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28e026f284e3504 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:33.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.671+0000 7f86b37fe700 1 -- 172.21.15.2:0/3229823475 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c009e120 con 0x55693d7b09b0 2023-09-13T04:39:33.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.671+0000 7f86b37fe700 1 -- 172.21.15.2:0/3229823475 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86c0099e30 con 0x55693d7b09b0 2023-09-13T04:39:33.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.671+0000 7f86b37fe700 1 -- 172.21.15.2:0/3229823475 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c00850d0 con 0x55693d7b09b0 2023-09-13T04:39:33.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.672+0000 7f86e335bb80 1 -- 172.21.15.2:0/3229823475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b09b0 msgr2=0x55693d7b0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.672+0000 7f86e335bb80 1 --2- 172.21.15.2:0/3229823475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b09b0 0x55693d7b0e30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f86c0007990 tx=0x7f86c0005780 comp rx=0 tx=0).stop 2023-09-13T04:39:33.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.672+0000 7f86e335bb80 1 -- 172.21.15.2:0/3229823475 shutdown_connections 2023-09-13T04:39:33.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.672+0000 7f86e335bb80 1 --2- 172.21.15.2:0/3229823475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b09b0 0x55693d7b0e30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.672+0000 7f86e335bb80 1 --2- 172.21.15.2:0/3229823475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55693d7b16b0 0x55693d7b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.672+0000 7f86e335bb80 1 --2- 172.21.15.2:0/3229823475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b2010 0x55693d7b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.672+0000 7f86e335bb80 1 -- 172.21.15.2:0/3229823475 >> 172.21.15.2:0/3229823475 conn(0x55693d70ec30 msgr2=0x55693d7aaec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:33.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.672+0000 7f86e335bb80 1 -- 172.21.15.2:0/3229823475 shutdown_connections 2023-09-13T04:39:33.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.672+0000 7f86e335bb80 1 -- 172.21.15.2:0/3229823475 wait complete. 2023-09-13T04:39:33.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.673+0000 7f86e335bb80 1 -- start start 2023-09-13T04:39:33.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.673+0000 7f86e335bb80 1 Processor -- start 2023-09-13T04:39:33.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.673+0000 7f86e335bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b09b0 0x55693d7b9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.673+0000 7f86e335bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b16b0 0x55693d7ba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.673+0000 7f86e335bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55693d7b2010 0x55693d7c1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.673+0000 7f86e335bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55693d6a0b60 con 0x55693d7b16b0 2023-09-13T04:39:33.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.673+0000 7f86e335bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55693d6a0860 con 0x55693d7b09b0 2023-09-13T04:39:33.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.673+0000 7f86e335bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55693d5848d0 con 0x55693d7b2010 2023-09-13T04:39:33.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.673+0000 7f86e335bb80 10 client.0 _ll_register_callbacks cb 0x55693d69e630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:39:33.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.674+0000 7f86ccf9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b16b0 0x55693d7ba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.674+0000 7f86cd79b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b09b0 0x55693d7b9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.674+0000 7f86cd79b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b09b0 0x55693d7b9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:48424/0 (socket says 172.21.15.2:48424) 2023-09-13T04:39:33.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.674+0000 7f86cd79b700 1 -- 172.21.15.2:0/2418876217 learned_addr learned my addr 172.21.15.2:0/2418876217 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:33.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.674+0000 7f86cdf9c700 1 --2- 172.21.15.2:0/2418876217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55693d7b2010 0x55693d7c1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.674+0000 7f86cd79b700 1 -- 172.21.15.2:0/2418876217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55693d7b2010 msgr2=0x55693d7c1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:39:33.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.674+0000 7f86cd79b700 1 --2- 172.21.15.2:0/2418876217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55693d7b2010 0x55693d7c1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.674+0000 7f86cd79b700 1 -- 172.21.15.2:0/2418876217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b16b0 msgr2=0x55693d7ba140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:39:33.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.674+0000 7f86cd79b700 1 --2- 172.21.15.2:0/2418876217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b16b0 0x55693d7ba140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.674+0000 7f86cd79b700 1 -- 172.21.15.2:0/2418876217 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c81a11d0 con 0x55693d7b09b0 2023-09-13T04:39:33.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.674+0000 7f86cd79b700 1 --2- 172.21.15.2:0/2418876217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b09b0 0x55693d7b9c00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f86c4004520 tx=0x7f86c40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af554304cff3af7c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:33.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.674+0000 7f86b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:39:33.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.675+0000 7f86b0ff9700 1 -- 172.21.15.2:0/2418876217 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4022070 con 0x55693d7b09b0 2023-09-13T04:39:33.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.675+0000 7f86e335bb80 1 -- 172.21.15.2:0/2418876217 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55693d7a3620 con 0x55693d7b09b0 2023-09-13T04:39:33.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.675+0000 7f86b0ff9700 1 -- 172.21.15.2:0/2418876217 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f86c4004830 con 0x55693d7b09b0 2023-09-13T04:39:33.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.675+0000 7f86b0ff9700 1 -- 172.21.15.2:0/2418876217 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86c4004550 con 0x55693d7b09b0 2023-09-13T04:39:33.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.675+0000 7f86e335bb80 10 client.25142 fetch_fsmap learned FSMap version 23 2023-09-13T04:39:33.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.675+0000 7f86e335bb80 1 -- 172.21.15.2:0/2418876217 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55693d71a750 con 0x55693d7b09b0 2023-09-13T04:39:33.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.675+0000 7f86b0ff9700 1 -- 172.21.15.2:0/2418876217 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55693d7a3620 con 0x55693d7b09b0 2023-09-13T04:39:33.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.675+0000 7f86b0ff9700 1 -- 172.21.15.2:0/2418876217 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f86c401d410 con 0x55693d7b09b0 2023-09-13T04:39:33.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.675+0000 7f86e335bb80 10 client.25142 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:39:33.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.675+0000 7f86e335bb80 10 client.25142 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-13T04:39:33.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.675+0000 7f86e335bb80 20 client.25142 populate_metadata read hostname 'smithi002' 2023-09-13T04:39:33.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.675+0000 7f86e335bb80 1 --2- 172.21.15.2:0/2418876217 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x55693d7c57f0 0x55693d7c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.676+0000 7f86e335bb80 4 client.25142 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:39:33.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.676+0000 7f86e335bb80 1 -- 172.21.15.2:0/2418876217 --> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55693d6fb920 con 0x55693d7c57f0 2023-09-13T04:39:33.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.676+0000 7f86ccf9a700 1 --2- 172.21.15.2:0/2418876217 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x55693d7c57f0 0x55693d7c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.676+0000 7f86ccf9a700 1 --2- 172.21.15.2:0/2418876217 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x55693d7c57f0 0x55693d7c42e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=64a9965e60f940ab server_cookie=8884b81c60962551 in_seq=0 out_seq=0 2023-09-13T04:39:33.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.676+0000 7f86b0ff9700 10 client.25142 ms_handle_connect on v2:172.21.15.2:6840/482545391 2023-09-13T04:39:33.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.677+0000 7f86b0ff9700 1 -- 172.21.15.2:0/2418876217 <== mds.1 v2:172.21.15.2:6840/482545391 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55693d6fb920 con 0x55693d7c57f0 2023-09-13T04:39:33.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.677+0000 7f86b0ff9700 10 client.25142 handle_command_reply: tid=0 2023-09-13T04:39:33.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.677+0000 7f86e335bb80 1 client.25142 shutdown 2023-09-13T04:39:33.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.677+0000 7f86e335bb80 1 -- 172.21.15.2:0/2418876217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b09b0 msgr2=0x55693d7b9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.677+0000 7f86e335bb80 1 --2- 172.21.15.2:0/2418876217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b09b0 0x55693d7b9c00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f86c4004520 tx=0x7f86c40047c0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.679+0000 7f86e335bb80 1 -- 172.21.15.2:0/2418876217 shutdown_connections 2023-09-13T04:39:33.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.679+0000 7f86e335bb80 1 --2- 172.21.15.2:0/2418876217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55693d7b2010 0x55693d7c1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.679+0000 7f86e335bb80 1 --2- 172.21.15.2:0/2418876217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b09b0 0x55693d7b9c00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.679+0000 7f86e335bb80 1 --2- 172.21.15.2:0/2418876217 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x55693d7c57f0 0x55693d7c42e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.679+0000 7f86e335bb80 1 --2- 172.21.15.2:0/2418876217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b16b0 0x55693d7ba140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.679+0000 7f86e335bb80 1 -- 172.21.15.2:0/2418876217 >> 172.21.15.2:0/2418876217 conn(0x55693d70ec30 msgr2=0x55693d7a57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:33.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.679+0000 7f86b0ff9700 0 client.25142 ms_handle_reset on v2:172.21.15.2:6840/482545391 2023-09-13T04:39:33.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.679+0000 7f86e335bb80 1 -- 172.21.15.2:0/2418876217 shutdown_connections 2023-09-13T04:39:33.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.679+0000 7f86e335bb80 1 -- 172.21.15.2:0/2418876217 wait complete. 2023-09-13T04:39:33.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.679+0000 7f86e335bb80 20 client.25142 trim_cache size 0 max 16384 2023-09-13T04:39:33.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.696+0000 7f86e335bb80 1 Processor -- start 2023-09-13T04:39:33.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.696+0000 7f86e335bb80 1 -- start start 2023-09-13T04:39:33.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.696+0000 7f86e335bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55693d7b09b0 0x55693d7b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.696+0000 7f86e335bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b16b0 0x55693d7c1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.696+0000 7f86e335bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b2010 0x55693d7c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.696+0000 7f86e335bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55693d7ab490 con 0x55693d7b2010 2023-09-13T04:39:33.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.696+0000 7f86e335bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55693d6fb950 con 0x55693d7b16b0 2023-09-13T04:39:33.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.696+0000 7f86e335bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55693d6c2df0 con 0x55693d7b09b0 2023-09-13T04:39:33.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.697+0000 7f86cdf9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b2010 0x55693d7c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.697+0000 7f86cd79b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55693d7b09b0 0x55693d7b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.697+0000 7f86ccf9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b16b0 0x55693d7c1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.697+0000 7f86cdf9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b2010 0x55693d7c42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51896/0 (socket says 172.21.15.2:51896) 2023-09-13T04:39:33.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.697+0000 7f86cdf9c700 1 -- 172.21.15.2:0/1685726270 learned_addr learned my addr 172.21.15.2:0/1685726270 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:33.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.697+0000 7f86cd79b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55693d7b09b0 0x55693d7b0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:54308/0 (socket says 172.21.15.2:54308) 2023-09-13T04:39:33.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.697+0000 7f86ccf9a700 1 -- 172.21.15.2:0/1685726270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55693d7b09b0 msgr2=0x55693d7b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:39:33.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.697+0000 7f86ccf9a700 1 --2- 172.21.15.2:0/1685726270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55693d7b09b0 0x55693d7b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.697+0000 7f86ccf9a700 1 -- 172.21.15.2:0/1685726270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b2010 msgr2=0x55693d7c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:39:33.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.697+0000 7f86ccf9a700 1 --2- 172.21.15.2:0/1685726270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b2010 0x55693d7c42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.697+0000 7f86ccf9a700 1 -- 172.21.15.2:0/1685726270 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55693d52ed90 con 0x55693d7b16b0 2023-09-13T04:39:33.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.697+0000 7f86cdf9c700 1 --2- 172.21.15.2:0/1685726270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b2010 0x55693d7c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:39:33.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.697+0000 7f86ccf9a700 1 --2- 172.21.15.2:0/1685726270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b16b0 0x55693d7c1140 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55693d7c6e80 tx=0x7f86b80073d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcc463778faefe1a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:33.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.698+0000 7f86b3fff700 1 -- 172.21.15.2:0/1685726270 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b8016070 con 0x55693d7b16b0 2023-09-13T04:39:33.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.698+0000 7f86b3fff700 1 -- 172.21.15.2:0/1685726270 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86b8017040 con 0x55693d7b16b0 2023-09-13T04:39:33.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.698+0000 7f86b3fff700 1 -- 172.21.15.2:0/1685726270 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b8011b70 con 0x55693d7b16b0 2023-09-13T04:39:33.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.698+0000 7f86e335bb80 1 -- 172.21.15.2:0/1685726270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b16b0 msgr2=0x55693d7c1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.698+0000 7f86e335bb80 1 --2- 172.21.15.2:0/1685726270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b16b0 0x55693d7c1140 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55693d7c6e80 tx=0x7f86b80073d0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.698+0000 7f86e335bb80 1 -- 172.21.15.2:0/1685726270 shutdown_connections 2023-09-13T04:39:33.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.698+0000 7f86e335bb80 1 --2- 172.21.15.2:0/1685726270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b2010 0x55693d7c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.698+0000 7f86e335bb80 1 --2- 172.21.15.2:0/1685726270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55693d7b09b0 0x55693d7b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.698+0000 7f86e335bb80 1 --2- 172.21.15.2:0/1685726270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b16b0 0x55693d7c1140 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.698+0000 7f86e335bb80 1 -- 172.21.15.2:0/1685726270 >> 172.21.15.2:0/1685726270 conn(0x55693d70ec30 msgr2=0x55693d7b48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:33.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.698+0000 7f86e335bb80 1 -- 172.21.15.2:0/1685726270 shutdown_connections 2023-09-13T04:39:33.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.698+0000 7f86e335bb80 1 -- 172.21.15.2:0/1685726270 wait complete. 2023-09-13T04:39:33.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.699+0000 7f86e335bb80 1 -- start start 2023-09-13T04:39:33.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.699+0000 7f86e335bb80 1 Processor -- start 2023-09-13T04:39:33.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.699+0000 7f86e335bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55693d7b09b0 0x55693d7bbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.699+0000 7f86e335bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b16b0 0x55693d7bc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.699+0000 7f86e335bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b2010 0x55693d8191e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.699+0000 7f86e335bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55693d7bc8c0 con 0x55693d7b16b0 2023-09-13T04:39:33.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.699+0000 7f86e335bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55693d7bca30 con 0x55693d7b2010 2023-09-13T04:39:33.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.699+0000 7f86e335bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55693d819720 con 0x55693d7b09b0 2023-09-13T04:39:33.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.699+0000 7f86e335bb80 10 client.0 _ll_register_callbacks cb 0x55693d69e630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:39:33.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.700+0000 7f86ccf9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b16b0 0x55693d7bc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.700+0000 7f86cdf9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b2010 0x55693d8191e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.700+0000 7f86cd79b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55693d7b09b0 0x55693d7bbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.700+0000 7f86cdf9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b2010 0x55693d8191e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:48434/0 (socket says 172.21.15.2:48434) 2023-09-13T04:39:33.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.700+0000 7f86cdf9c700 1 -- 172.21.15.2:0/1857081442 learned_addr learned my addr 172.21.15.2:0/1857081442 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:33.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.700+0000 7f86ccf9a700 1 -- 172.21.15.2:0/1857081442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55693d7b09b0 msgr2=0x55693d7bbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:39:33.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.700+0000 7f86ccf9a700 1 --2- 172.21.15.2:0/1857081442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55693d7b09b0 0x55693d7bbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.700+0000 7f86ccf9a700 1 -- 172.21.15.2:0/1857081442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b2010 msgr2=0x55693d8191e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:39:33.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.700+0000 7f86ccf9a700 1 --2- 172.21.15.2:0/1857081442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b2010 0x55693d8191e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.700+0000 7f86ccf9a700 1 -- 172.21.15.2:0/1857081442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55693d5825e0 con 0x55693d7b16b0 2023-09-13T04:39:33.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.700+0000 7f86cdf9c700 1 --2- 172.21.15.2:0/1857081442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b2010 0x55693d8191e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:39:33.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.700+0000 7f86cd79b700 1 --2- 172.21.15.2:0/1857081442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55693d7b09b0 0x55693d7bbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:39:33.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.700+0000 7f86ccf9a700 1 --2- 172.21.15.2:0/1857081442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b16b0 0x55693d7bc380 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f86b80059d0 tx=0x7f86b801f0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc70180654b61a74 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:33.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.700+0000 7f86b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:39:33.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.701+0000 7f86b0ff9700 1 -- 172.21.15.2:0/1857081442 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b8016070 con 0x55693d7b16b0 2023-09-13T04:39:33.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.701+0000 7f86e335bb80 1 -- 172.21.15.2:0/1857081442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86c0099e30 con 0x55693d7b16b0 2023-09-13T04:39:33.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.701+0000 7f86b0ff9700 1 -- 172.21.15.2:0/1857081442 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f86b8004230 con 0x55693d7b16b0 2023-09-13T04:39:33.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.701+0000 7f86b0ff9700 1 -- 172.21.15.2:0/1857081442 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86b80043f0 con 0x55693d7b16b0 2023-09-13T04:39:33.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.701+0000 7f86e335bb80 10 client.15114 fetch_fsmap learned FSMap version 23 2023-09-13T04:39:33.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.701+0000 7f86e335bb80 1 -- 172.21.15.2:0/1857081442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f86b8007db0 con 0x55693d7b16b0 2023-09-13T04:39:33.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.701+0000 7f86b0ff9700 1 -- 172.21.15.2:0/1857081442 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86b8032020 con 0x55693d7b16b0 2023-09-13T04:39:33.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.701+0000 7f86b0ff9700 1 -- 172.21.15.2:0/1857081442 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f86b8027070 con 0x55693d7b16b0 2023-09-13T04:39:33.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.701+0000 7f86e335bb80 10 client.15114 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:39:33.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.701+0000 7f86e335bb80 10 client.15114 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-13T04:39:33.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.701+0000 7f86e335bb80 20 client.15114 populate_metadata read hostname 'smithi002' 2023-09-13T04:39:33.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.701+0000 7f86e335bb80 1 --2- 172.21.15.2:0/1857081442 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x55693d81e9c0 0x55693d81ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.701+0000 7f86e335bb80 4 client.15114 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "status"}] 2023-09-13T04:39:33.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.701+0000 7f86e335bb80 1 -- 172.21.15.2:0/1857081442 --> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55693d6bc000 con 0x55693d81e9c0 2023-09-13T04:39:33.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.702+0000 7f86cd79b700 1 --2- 172.21.15.2:0/1857081442 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x55693d81e9c0 0x55693d81ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.702+0000 7f86cd79b700 1 --2- 172.21.15.2:0/1857081442 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x55693d81e9c0 0x55693d81ae40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e94497f9079c1a63 server_cookie=6a3d034e6e55d569 in_seq=0 out_seq=0 2023-09-13T04:39:33.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.702+0000 7f86b0ff9700 10 client.15114 ms_handle_connect on v2:172.21.15.2:6840/482545391 2023-09-13T04:39:33.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.702+0000 7f86b0ff9700 1 -- 172.21.15.2:0/1857081442 <== mds.1 v2:172.21.15.2:6840/482545391 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55693d6bc000 con 0x55693d81e9c0 2023-09-13T04:39:33.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.702+0000 7f86b0ff9700 10 client.15114 handle_command_reply: tid=0 2023-09-13T04:39:33.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.702+0000 7f86e335bb80 1 client.15114 shutdown 2023-09-13T04:39:33.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.703+0000 7f86e335bb80 1 -- 172.21.15.2:0/1857081442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b16b0 msgr2=0x55693d7bc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.703+0000 7f86e335bb80 1 --2- 172.21.15.2:0/1857081442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b16b0 0x55693d7bc380 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f86b80059d0 tx=0x7f86b801f0f0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.703+0000 7f86e335bb80 1 -- 172.21.15.2:0/1857081442 shutdown_connections 2023-09-13T04:39:33.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.703+0000 7f86e335bb80 1 --2- 172.21.15.2:0/1857081442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55693d7b2010 0x55693d8191e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.703+0000 7f86e335bb80 1 --2- 172.21.15.2:0/1857081442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55693d7b09b0 0x55693d7bbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.703+0000 7f86e335bb80 1 --2- 172.21.15.2:0/1857081442 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x55693d81e9c0 0x55693d81ae40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.703+0000 7f86e335bb80 1 --2- 172.21.15.2:0/1857081442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55693d7b16b0 0x55693d7bc380 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.703+0000 7f86b0ff9700 0 client.15114 ms_handle_reset on v2:172.21.15.2:6840/482545391 2023-09-13T04:39:33.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.703+0000 7f86e335bb80 1 -- 172.21.15.2:0/1857081442 >> 172.21.15.2:0/1857081442 conn(0x55693d70ec30 msgr2=0x55693d7c42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:33.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.703+0000 7f86e335bb80 1 -- 172.21.15.2:0/1857081442 shutdown_connections 2023-09-13T04:39:33.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.703+0000 7f86e335bb80 1 -- 172.21.15.2:0/1857081442 wait complete. 2023-09-13T04:39:33.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.703+0000 7f86e335bb80 20 client.15114 trim_cache size 0 max 16384 2023-09-13T04:39:33.723 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:39:33.723 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da", 2023-09-13T04:39:33.723 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 1, 2023-09-13T04:39:33.723 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24554, 2023-09-13T04:39:33.723 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2023-09-13T04:39:33.724 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2023-09-13T04:39:33.724 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2023-09-13T04:39:33.724 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 69.470798169000005, 2023-09-13T04:39:33.724 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 23, 2023-09-13T04:39:33.725 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 92, 2023-09-13T04:39:33.725 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 90, 2023-09-13T04:39:33.725 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 103.100956672 2023-09-13T04:39:33.725 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:39:33.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.703+0000 7f86cf9ff700 1 -- 172.21.15.2:0/2448781096 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f86b40a06c0 msgr2=0x7f86b40a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.703+0000 7f86cf9ff700 1 --2- 172.21.15.2:0/2448781096 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f86b40a06c0 0x7f86b40a2b80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f86c8199b50 tx=0x7f86c4009260 comp rx=0 tx=0).stop 2023-09-13T04:39:33.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.703+0000 7f86cf9ff700 1 -- 172.21.15.2:0/2448781096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86c8102870 msgr2=0x7f86c81a0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.703+0000 7f86cf9ff700 1 --2- 172.21.15.2:0/2448781096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86c8102870 0x7f86c81a0220 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f86c0008f10 tx=0x7f86c0008f40 comp rx=0 tx=0).stop 2023-09-13T04:39:33.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.704+0000 7f86cf9ff700 1 -- 172.21.15.2:0/2448781096 shutdown_connections 2023-09-13T04:39:33.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.704+0000 7f86cf9ff700 1 --2- 172.21.15.2:0/2448781096 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f86b40a06c0 0x7f86b40a2b80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.704+0000 7f86cf9ff700 1 --2- 172.21.15.2:0/2448781096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86c8102870 0x7f86c81a0220 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.704+0000 7f86cf9ff700 1 --2- 172.21.15.2:0/2448781096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c8100ce0 0x7f86c8198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.704+0000 7f86cf9ff700 1 --2- 172.21.15.2:0/2448781096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86c8101ed0 0x7f86c81991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.704+0000 7f86cf9ff700 1 -- 172.21.15.2:0/2448781096 >> 172.21.15.2:0/2448781096 conn(0x7f86c80fc230 msgr2=0x7f86c8107770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:33.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.704+0000 7f86cf9ff700 1 -- 172.21.15.2:0/2448781096 shutdown_connections 2023-09-13T04:39:33.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.704+0000 7f86cf9ff700 1 -- 172.21.15.2:0/2448781096 wait complete. 2023-09-13T04:39:33.728 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:33 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3121695761' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:39:33.729 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:33 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/953203655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:39:33.730 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-13T04:39:33.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:33 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3121695761' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:39:33.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:33 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/953203655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:39:33.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.935+0000 7f5f431ff700 1 -- 172.21.15.2:0/4066424184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f3c101ed0 msgr2=0x7f5f3c102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.935+0000 7f5f431ff700 1 --2- 172.21.15.2:0/4066424184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f3c101ed0 0x7f5f3c102330 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c0099c0 tx=0x7f5f2c009cd0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.935+0000 7f5f431ff700 1 -- 172.21.15.2:0/4066424184 shutdown_connections 2023-09-13T04:39:33.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.935+0000 7f5f431ff700 1 --2- 172.21.15.2:0/4066424184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f3c102870 0x7f5f3c109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.935+0000 7f5f431ff700 1 --2- 172.21.15.2:0/4066424184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f3c100ce0 0x7f5f3c101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.935+0000 7f5f431ff700 1 --2- 172.21.15.2:0/4066424184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f3c101ed0 0x7f5f3c102330 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.935+0000 7f5f431ff700 1 -- 172.21.15.2:0/4066424184 >> 172.21.15.2:0/4066424184 conn(0x7f5f3c0fc230 msgr2=0x7f5f3c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:33.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.935+0000 7f5f431ff700 1 -- 172.21.15.2:0/4066424184 shutdown_connections 2023-09-13T04:39:33.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.935+0000 7f5f431ff700 1 -- 172.21.15.2:0/4066424184 wait complete. 2023-09-13T04:39:33.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:33 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3121695761' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:39:33.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:33 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/953203655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:39:33.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.936+0000 7f5f431ff700 1 Processor -- start 2023-09-13T04:39:33.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.937+0000 7f5f431ff700 1 -- start start 2023-09-13T04:39:33.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.937+0000 7f5f431ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f3c100ce0 0x7f5f3c1969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.937+0000 7f5f431ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f3c101ed0 0x7f5f3c196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.937+0000 7f5f431ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f3c102870 0x7f5f3c19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.937+0000 7f5f431ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c19e560 con 0x7f5f3c102870 2023-09-13T04:39:33.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.937+0000 7f5f431ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c19e6a0 con 0x7f5f3c100ce0 2023-09-13T04:39:33.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.937+0000 7f5f431ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c19e830 con 0x7f5f3c101ed0 2023-09-13T04:39:33.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.937+0000 7f5f4179c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f3c102870 0x7f5f3c19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.937+0000 7f5f4179c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f3c102870 0x7f5f3c19df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51922/0 (socket says 172.21.15.2:51922) 2023-09-13T04:39:33.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.937+0000 7f5f4179c700 1 -- 172.21.15.2:0/4136791895 learned_addr learned my addr 172.21.15.2:0/4136791895 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:33.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.937+0000 7f5f33fff700 1 --2- 172.21.15.2:0/4136791895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f3c101ed0 0x7f5f3c196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.937+0000 7f5f40f9b700 1 --2- 172.21.15.2:0/4136791895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f3c100ce0 0x7f5f3c1969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.938+0000 7f5f4179c700 1 -- 172.21.15.2:0/4136791895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f3c101ed0 msgr2=0x7f5f3c196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.938+0000 7f5f4179c700 1 --2- 172.21.15.2:0/4136791895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f3c101ed0 0x7f5f3c196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.938+0000 7f5f4179c700 1 -- 172.21.15.2:0/4136791895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f3c100ce0 msgr2=0x7f5f3c1969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.938+0000 7f5f4179c700 1 --2- 172.21.15.2:0/4136791895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f3c100ce0 0x7f5f3c1969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.938+0000 7f5f4179c700 1 -- 172.21.15.2:0/4136791895 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c0095f0 con 0x7f5f3c102870 2023-09-13T04:39:33.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.938+0000 7f5f4179c700 1 --2- 172.21.15.2:0/4136791895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f3c102870 0x7f5f3c19df90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5f3400d980 tx=0x7f5f3400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:33.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.938+0000 7f5f31ffb700 1 -- 172.21.15.2:0/4136791895 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f34004a30 con 0x7f5f3c102870 2023-09-13T04:39:33.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.938+0000 7f5f431ff700 1 -- 172.21.15.2:0/4136791895 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f3c19eb10 con 0x7f5f3c102870 2023-09-13T04:39:33.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.938+0000 7f5f31ffb700 1 -- 172.21.15.2:0/4136791895 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f340054e0 con 0x7f5f3c102870 2023-09-13T04:39:33.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.938+0000 7f5f31ffb700 1 -- 172.21.15.2:0/4136791895 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3400fbc0 con 0x7f5f3c102870 2023-09-13T04:39:33.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.938+0000 7f5f431ff700 1 -- 172.21.15.2:0/4136791895 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f3c19f020 con 0x7f5f3c102870 2023-09-13T04:39:33.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.942+0000 7f5f31ffb700 1 -- 172.21.15.2:0/4136791895 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f5f34004c20 con 0x7f5f3c102870 2023-09-13T04:39:33.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.943+0000 7f5f31ffb700 1 --2- 172.21.15.2:0/4136791895 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f5f240a0640 0x7f5f240a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.943+0000 7f5f56bfcb80 1 Processor -- start 2023-09-13T04:39:33.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.943+0000 7f5f31ffb700 1 -- 172.21.15.2:0/4136791895 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5f340cbc60 con 0x7f5f3c102870 2023-09-13T04:39:33.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.943+0000 7f5f56bfcb80 1 -- start start 2023-09-13T04:39:33.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.943+0000 7f5f40f9b700 1 --2- 172.21.15.2:0/4136791895 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f5f240a0640 0x7f5f240a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.944+0000 7f5f56bfcb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df222ac0 0x5563df222ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.944+0000 7f5f56bfcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563df223420 0x5563df221880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.944+0000 7f5f56bfcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563df221dc0 0x5563df222220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.944+0000 7f5f56bfcb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563df176730 con 0x5563df222ac0 2023-09-13T04:39:33.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.944+0000 7f5f56bfcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563df174050 con 0x5563df223420 2023-09-13T04:39:33.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.944+0000 7f5f56bfcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563df16f940 con 0x5563df221dc0 2023-09-13T04:39:33.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.944+0000 7f5f40f9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df222ac0 0x5563df222ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.944+0000 7f5f40f9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df222ac0 0x5563df222ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51932/0 (socket says 172.21.15.2:51932) 2023-09-13T04:39:33.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.944+0000 7f5f33fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563df223420 0x5563df221880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.944+0000 7f5f40f9b700 1 -- 172.21.15.2:0/2345921173 learned_addr learned my addr 172.21.15.2:0/2345921173 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:33.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.944+0000 7f5f4179c700 1 --2- 172.21.15.2:0/2345921173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563df221dc0 0x5563df222220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.945+0000 7f5f40f9b700 1 --2- 172.21.15.2:0/4136791895 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f5f240a0640 0x7f5f240a2b00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009550 tx=0x7f5f380077e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:33.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.945+0000 7f5f33fff700 1 -- 172.21.15.2:0/2345921173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563df221dc0 msgr2=0x5563df222220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:39:33.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.945+0000 7f5f33fff700 1 --2- 172.21.15.2:0/2345921173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563df221dc0 0x5563df222220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.945+0000 7f5f33fff700 1 -- 172.21.15.2:0/2345921173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df222ac0 msgr2=0x5563df222ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:39:33.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.945+0000 7f5f33fff700 1 --2- 172.21.15.2:0/2345921173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df222ac0 0x5563df222ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.945+0000 7f5f33fff700 1 -- 172.21.15.2:0/2345921173 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563defef5e0 con 0x5563df223420 2023-09-13T04:39:33.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.945+0000 7f5f33fff700 1 --2- 172.21.15.2:0/2345921173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563df223420 0x5563df221880 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c009990 tx=0x7f5f2c005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f476c52c713e9142 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:33.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.945+0000 7f5f22ffd700 1 -- 172.21.15.2:0/2345921173 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c01d070 con 0x5563df223420 2023-09-13T04:39:33.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.945+0000 7f5f22ffd700 1 -- 172.21.15.2:0/2345921173 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f2c01e040 con 0x5563df223420 2023-09-13T04:39:33.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.945+0000 7f5f22ffd700 1 -- 172.21.15.2:0/2345921173 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c0038c0 con 0x5563df223420 2023-09-13T04:39:33.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.946+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/2345921173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563df223420 msgr2=0x5563df221880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.946+0000 7f5f56bfcb80 1 --2- 172.21.15.2:0/2345921173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563df223420 0x5563df221880 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c009990 tx=0x7f5f2c005be0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.946+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/2345921173 shutdown_connections 2023-09-13T04:39:33.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.946+0000 7f5f56bfcb80 1 --2- 172.21.15.2:0/2345921173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563df221dc0 0x5563df222220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.946+0000 7f5f56bfcb80 1 --2- 172.21.15.2:0/2345921173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df222ac0 0x5563df222ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.946+0000 7f5f56bfcb80 1 --2- 172.21.15.2:0/2345921173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563df223420 0x5563df221880 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.946+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/2345921173 >> 172.21.15.2:0/2345921173 conn(0x5563df17bc30 msgr2=0x5563df217ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:33.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.946+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/2345921173 shutdown_connections 2023-09-13T04:39:33.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.946+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/2345921173 wait complete. 2023-09-13T04:39:33.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.947+0000 7f5f56bfcb80 1 -- start start 2023-09-13T04:39:33.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.947+0000 7f5f56bfcb80 1 Processor -- start 2023-09-13T04:39:33.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.947+0000 7f5f56bfcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563df221dc0 0x5563df22b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.947+0000 7f5f56bfcb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df222ac0 0x5563df22b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.947+0000 7f5f56bfcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563df223420 0x5563df2327b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.947+0000 7f5f56bfcb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563df10db60 con 0x5563df222ac0 2023-09-13T04:39:33.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.947+0000 7f5f56bfcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563df10d860 con 0x5563df223420 2023-09-13T04:39:33.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.947+0000 7f5f56bfcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563deff18d0 con 0x5563df221dc0 2023-09-13T04:39:33.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.947+0000 7f5f56bfcb80 10 client.0 _ll_register_callbacks cb 0x5563df10b630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:39:33.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.948+0000 7f5f4179c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563df223420 0x5563df2327b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.948+0000 7f5f33fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df222ac0 0x5563df22b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.948+0000 7f5f4179c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563df223420 0x5563df2327b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:48476/0 (socket says 172.21.15.2:48476) 2023-09-13T04:39:33.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.948+0000 7f5f4179c700 1 -- 172.21.15.2:0/3699360165 learned_addr learned my addr 172.21.15.2:0/3699360165 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:33.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.948+0000 7f5f40f9b700 1 --2- 172.21.15.2:0/3699360165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563df221dc0 0x5563df22b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.948+0000 7f5f4179c700 1 -- 172.21.15.2:0/3699360165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563df221dc0 msgr2=0x5563df22b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:39:33.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.948+0000 7f5f4179c700 1 --2- 172.21.15.2:0/3699360165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563df221dc0 0x5563df22b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.948+0000 7f5f4179c700 1 -- 172.21.15.2:0/3699360165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df222ac0 msgr2=0x5563df22b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:39:33.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.948+0000 7f5f4179c700 1 --2- 172.21.15.2:0/3699360165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df222ac0 0x5563df22b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.948+0000 7f5f4179c700 1 -- 172.21.15.2:0/3699360165 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f3c109a70 con 0x5563df223420 2023-09-13T04:39:33.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.948+0000 7f5f33fff700 1 --2- 172.21.15.2:0/3699360165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df222ac0 0x5563df22b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:39:33.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.948+0000 7f5f4179c700 1 --2- 172.21.15.2:0/3699360165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563df223420 0x5563df2327b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5f340c97a0 tx=0x7f5f340998a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4bef1daf4f7c0e5 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:33.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.948+0000 7f5f1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:39:33.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.949+0000 7f5f1bfff700 1 -- 172.21.15.2:0/3699360165 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f34014020 con 0x5563df223420 2023-09-13T04:39:33.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.949+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/3699360165 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5563df210620 con 0x5563df223420 2023-09-13T04:39:33.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.949+0000 7f5f1bfff700 1 -- 172.21.15.2:0/3699360165 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5f340a1190 con 0x5563df223420 2023-09-13T04:39:33.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.949+0000 7f5f1bfff700 1 -- 172.21.15.2:0/3699360165 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f3409b070 con 0x5563df223420 2023-09-13T04:39:33.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.949+0000 7f5f56bfcb80 10 client.25160 fetch_fsmap learned FSMap version 23 2023-09-13T04:39:33.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.949+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/3699360165 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5563df187750 con 0x5563df223420 2023-09-13T04:39:33.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.949+0000 7f5f1bfff700 1 -- 172.21.15.2:0/3699360165 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5563df210620 con 0x5563df223420 2023-09-13T04:39:33.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.949+0000 7f5f1bfff700 1 -- 172.21.15.2:0/3699360165 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5f34099380 con 0x5563df223420 2023-09-13T04:39:33.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.949+0000 7f5f56bfcb80 10 client.25160 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:39:33.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.949+0000 7f5f56bfcb80 10 client.25160 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-13T04:39:33.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.949+0000 7f5f56bfcb80 20 client.25160 populate_metadata read hostname 'smithi002' 2023-09-13T04:39:33.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.949+0000 7f5f56bfcb80 1 --2- 172.21.15.2:0/3699360165 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5563df236e20 0x5563df2358c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.949+0000 7f5f56bfcb80 4 client.25160 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:39:33.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.949+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/3699360165 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5563df168920 con 0x5563df236e20 2023-09-13T04:39:33.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.950+0000 7f5f40f9b700 1 --2- 172.21.15.2:0/3699360165 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5563df236e20 0x5563df2358c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.950+0000 7f5f40f9b700 1 --2- 172.21.15.2:0/3699360165 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5563df236e20 0x5563df2358c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=417395bd110a495e server_cookie=55a609ac38f36fd1 in_seq=0 out_seq=0 2023-09-13T04:39:33.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.950+0000 7f5f1bfff700 10 client.25160 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:39:33.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.951+0000 7f5f1bfff700 1 -- 172.21.15.2:0/3699360165 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5563df168920 con 0x5563df236e20 2023-09-13T04:39:33.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.951+0000 7f5f1bfff700 10 client.25160 handle_command_reply: tid=0 2023-09-13T04:39:33.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.951+0000 7f5f56bfcb80 1 client.25160 shutdown 2023-09-13T04:39:33.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.951+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/3699360165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563df223420 msgr2=0x5563df2327b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.951+0000 7f5f56bfcb80 1 --2- 172.21.15.2:0/3699360165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563df223420 0x5563df2327b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5f340c97a0 tx=0x7f5f340998a0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.951+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/3699360165 shutdown_connections 2023-09-13T04:39:33.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.951+0000 7f5f56bfcb80 1 --2- 172.21.15.2:0/3699360165 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5563df236e20 0x5563df2358c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.951+0000 7f5f56bfcb80 1 --2- 172.21.15.2:0/3699360165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563df223420 0x5563df2327b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.951+0000 7f5f56bfcb80 1 --2- 172.21.15.2:0/3699360165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563df221dc0 0x5563df22b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.951+0000 7f5f56bfcb80 1 --2- 172.21.15.2:0/3699360165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df222ac0 0x5563df22b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.951+0000 7f5f1bfff700 0 client.25160 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:39:33.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.951+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/3699360165 >> 172.21.15.2:0/3699360165 conn(0x5563df17bc30 msgr2=0x5563df2127a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:33.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.952+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/3699360165 shutdown_connections 2023-09-13T04:39:33.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.952+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/3699360165 wait complete. 2023-09-13T04:39:33.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.952+0000 7f5f56bfcb80 20 client.25160 trim_cache size 0 max 16384 2023-09-13T04:39:33.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.970+0000 7f5f56bfcb80 1 Processor -- start 2023-09-13T04:39:33.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.970+0000 7f5f56bfcb80 1 -- start start 2023-09-13T04:39:33.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.970+0000 7f5f56bfcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563df221dc0 0x5563df225d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.970+0000 7f5f56bfcb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df222ac0 0x5563df226280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.970+0000 7f5f56bfcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563df223420 0x5563df232770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.970+0000 7f5f56bfcb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563df210620 con 0x5563df222ac0 2023-09-13T04:39:33.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.970+0000 7f5f56bfcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563df22c420 con 0x5563df221dc0 2023-09-13T04:39:33.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.970+0000 7f5f56bfcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563df12fdf0 con 0x5563df223420 2023-09-13T04:39:33.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.971+0000 7f5f33fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df222ac0 0x5563df226280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.971+0000 7f5f33fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df222ac0 0x5563df226280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51950/0 (socket says 172.21.15.2:51950) 2023-09-13T04:39:33.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.971+0000 7f5f33fff700 1 -- 172.21.15.2:0/265171958 learned_addr learned my addr 172.21.15.2:0/265171958 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:33.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.971+0000 7f5f40f9b700 1 --2- 172.21.15.2:0/265171958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563df221dc0 0x5563df225d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.971+0000 7f5f4179c700 1 --2- 172.21.15.2:0/265171958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563df223420 0x5563df232770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.971+0000 7f5f33fff700 1 -- 172.21.15.2:0/265171958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563df223420 msgr2=0x5563df232770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:39:33.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.971+0000 7f5f33fff700 1 --2- 172.21.15.2:0/265171958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563df223420 0x5563df232770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.971+0000 7f5f33fff700 1 -- 172.21.15.2:0/265171958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563df221dc0 msgr2=0x5563df225d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:39:33.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.971+0000 7f5f33fff700 1 --2- 172.21.15.2:0/265171958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563df221dc0 0x5563df225d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.971+0000 7f5f33fff700 1 -- 172.21.15.2:0/265171958 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563def9bd90 con 0x5563df222ac0 2023-09-13T04:39:33.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.971+0000 7f5f40f9b700 1 --2- 172.21.15.2:0/265171958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563df221dc0 0x5563df225d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:39:33.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.971+0000 7f5f33fff700 1 --2- 172.21.15.2:0/265171958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df222ac0 0x5563df226280 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c005cd0 tx=0x7f5f2c0058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d00a4f8b6b5e43 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:33.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.972+0000 7f5f237fe700 1 -- 172.21.15.2:0/265171958 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c01d040 con 0x5563df222ac0 2023-09-13T04:39:33.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.972+0000 7f5f237fe700 1 -- 172.21.15.2:0/265171958 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f2c031070 con 0x5563df222ac0 2023-09-13T04:39:33.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.972+0000 7f5f237fe700 1 -- 172.21.15.2:0/265171958 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c003fe0 con 0x5563df222ac0 2023-09-13T04:39:33.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.972+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/265171958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df222ac0 msgr2=0x5563df226280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:33.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.972+0000 7f5f56bfcb80 1 --2- 172.21.15.2:0/265171958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df222ac0 0x5563df226280 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c005cd0 tx=0x7f5f2c0058e0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.972+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/265171958 shutdown_connections 2023-09-13T04:39:33.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.972+0000 7f5f56bfcb80 1 --2- 172.21.15.2:0/265171958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563df223420 0x5563df232770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.972+0000 7f5f56bfcb80 1 --2- 172.21.15.2:0/265171958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563df221dc0 0x5563df225d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.972+0000 7f5f56bfcb80 1 --2- 172.21.15.2:0/265171958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df222ac0 0x5563df226280 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.972+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/265171958 >> 172.21.15.2:0/265171958 conn(0x5563df17bc30 msgr2=0x5563df229f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:33.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.972+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/265171958 shutdown_connections 2023-09-13T04:39:33.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.973+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/265171958 wait complete. 2023-09-13T04:39:33.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.973+0000 7f5f56bfcb80 1 -- start start 2023-09-13T04:39:33.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.973+0000 7f5f56bfcb80 1 Processor -- start 2023-09-13T04:39:33.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.974+0000 7f5f56bfcb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df221dc0 0x5563df228690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.974+0000 7f5f56bfcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563df222ac0 0x5563df2326d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.974+0000 7f5f56bfcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563df223420 0x5563df226cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.974+0000 7f5f56bfcb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563df227300 con 0x5563df221dc0 2023-09-13T04:39:33.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.974+0000 7f5f56bfcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563df227470 con 0x5563df222ac0 2023-09-13T04:39:33.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.974+0000 7f5f56bfcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563df227630 con 0x5563df223420 2023-09-13T04:39:33.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.974+0000 7f5f56bfcb80 10 client.0 _ll_register_callbacks cb 0x5563df10b630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:39:33.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.974+0000 7f5f40f9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df221dc0 0x5563df228690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.974+0000 7f5f40f9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df221dc0 0x5563df228690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51964/0 (socket says 172.21.15.2:51964) 2023-09-13T04:39:33.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.974+0000 7f5f40f9b700 1 -- 172.21.15.2:0/2160524237 learned_addr learned my addr 172.21.15.2:0/2160524237 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:33.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.974+0000 7f5f33fff700 1 --2- 172.21.15.2:0/2160524237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563df222ac0 0x5563df2326d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.974+0000 7f5f4179c700 1 --2- 172.21.15.2:0/2160524237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563df223420 0x5563df226cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.974+0000 7f5f40f9b700 1 -- 172.21.15.2:0/2160524237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563df223420 msgr2=0x5563df226cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:39:33.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.974+0000 7f5f40f9b700 1 --2- 172.21.15.2:0/2160524237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563df223420 0x5563df226cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.974+0000 7f5f40f9b700 1 -- 172.21.15.2:0/2160524237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563df222ac0 msgr2=0x5563df2326d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:39:33.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.974+0000 7f5f40f9b700 1 --2- 172.21.15.2:0/2160524237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563df222ac0 0x5563df2326d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:33.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.974+0000 7f5f40f9b700 1 -- 172.21.15.2:0/2160524237 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563defef5e0 con 0x5563df221dc0 2023-09-13T04:39:33.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.975+0000 7f5f40f9b700 1 --2- 172.21.15.2:0/2160524237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df221dc0 0x5563df228690 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x5563df237b50 tx=0x7f5f38004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c1103fc124455 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:33.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.975+0000 7f5f1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:39:33.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.975+0000 7f5f1bfff700 1 -- 172.21.15.2:0/2160524237 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f38003c00 con 0x5563df221dc0 2023-09-13T04:39:33.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.975+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/2160524237 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f2c01e040 con 0x5563df221dc0 2023-09-13T04:39:33.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.975+0000 7f5f1bfff700 1 -- 172.21.15.2:0/2160524237 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5f38003d60 con 0x5563df221dc0 2023-09-13T04:39:33.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.975+0000 7f5f1bfff700 1 -- 172.21.15.2:0/2160524237 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f38024640 con 0x5563df221dc0 2023-09-13T04:39:33.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.975+0000 7f5f56bfcb80 10 client.15132 fetch_fsmap learned FSMap version 23 2023-09-13T04:39:33.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.975+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/2160524237 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5563def84c10 con 0x5563df221dc0 2023-09-13T04:39:33.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.975+0000 7f5f1bfff700 1 -- 172.21.15.2:0/2160524237 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f2c01e040 con 0x5563df221dc0 2023-09-13T04:39:33.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.976+0000 7f5f1bfff700 1 -- 172.21.15.2:0/2160524237 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5f38004420 con 0x5563df221dc0 2023-09-13T04:39:33.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.976+0000 7f5f56bfcb80 10 client.15132 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:39:33.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.976+0000 7f5f56bfcb80 10 client.15132 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-13T04:39:33.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.976+0000 7f5f56bfcb80 20 client.15132 populate_metadata read hostname 'smithi002' 2023-09-13T04:39:33.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.976+0000 7f5f56bfcb80 1 --2- 172.21.15.2:0/2160524237 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5563df28f4c0 0x5563df28b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:33.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.976+0000 7f5f56bfcb80 4 client.15132 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "status"}] 2023-09-13T04:39:33.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.976+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/2160524237 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5563df129000 con 0x5563df28f4c0 2023-09-13T04:39:33.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.976+0000 7f5f33fff700 1 --2- 172.21.15.2:0/2160524237 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5563df28f4c0 0x5563df28b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:33.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.977+0000 7f5f1bfff700 10 client.15132 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:39:33.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.977+0000 7f5f33fff700 1 --2- 172.21.15.2:0/2160524237 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5563df28f4c0 0x5563df28b8e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e06910e72208d019 server_cookie=4b46137a9e3c705 in_seq=0 out_seq=0 2023-09-13T04:39:33.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.977+0000 7f5f1bfff700 1 -- 172.21.15.2:0/2160524237 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x5563df129000 con 0x5563df28f4c0 2023-09-13T04:39:34.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.977+0000 7f5f1bfff700 10 client.15132 handle_command_reply: tid=0 2023-09-13T04:39:34.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.977+0000 7f5f56bfcb80 1 client.15132 shutdown 2023-09-13T04:39:34.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.977+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/2160524237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df221dc0 msgr2=0x5563df228690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:34.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.977+0000 7f5f56bfcb80 1 --2- 172.21.15.2:0/2160524237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df221dc0 0x5563df228690 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x5563df237b50 tx=0x7f5f38004b40 comp rx=0 tx=0).stop 2023-09-13T04:39:34.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.977+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/2160524237 shutdown_connections 2023-09-13T04:39:34.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.978+0000 7f5f56bfcb80 1 --2- 172.21.15.2:0/2160524237 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5563df28f4c0 0x5563df28b8e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:34.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.978+0000 7f5f56bfcb80 1 --2- 172.21.15.2:0/2160524237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563df223420 0x5563df226cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:34.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.978+0000 7f5f1bfff700 0 client.15132 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:39:34.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.978+0000 7f5f56bfcb80 1 --2- 172.21.15.2:0/2160524237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563df221dc0 0x5563df228690 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:34.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.978+0000 7f5f56bfcb80 1 --2- 172.21.15.2:0/2160524237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563df222ac0 0x5563df2326d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:34.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.978+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/2160524237 >> 172.21.15.2:0/2160524237 conn(0x5563df17bc30 msgr2=0x5563df22b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:34.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.978+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/2160524237 shutdown_connections 2023-09-13T04:39:34.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.978+0000 7f5f56bfcb80 1 -- 172.21.15.2:0/2160524237 wait complete. 2023-09-13T04:39:34.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.978+0000 7f5f56bfcb80 20 client.15132 trim_cache size 0 max 16384 2023-09-13T04:39:34.005 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:39:34.006 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da", 2023-09-13T04:39:34.006 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 0, 2023-09-13T04:39:34.007 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24566, 2023-09-13T04:39:34.007 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2023-09-13T04:39:34.007 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2023-09-13T04:39:34.008 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2023-09-13T04:39:34.008 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 94.402430318, 2023-09-13T04:39:34.008 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 23, 2023-09-13T04:39:34.009 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 92, 2023-09-13T04:39:34.009 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 90, 2023-09-13T04:39:34.009 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 101.120938649 2023-09-13T04:39:34.010 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:39:34.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.978+0000 7f5f431ff700 1 -- 172.21.15.2:0/4136791895 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f5f240a0640 msgr2=0x7f5f240a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:34.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.978+0000 7f5f431ff700 1 --2- 172.21.15.2:0/4136791895 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f5f240a0640 0x7f5f240a2b00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009550 tx=0x7f5f380077e0 comp rx=0 tx=0).stop 2023-09-13T04:39:34.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.978+0000 7f5f431ff700 1 -- 172.21.15.2:0/4136791895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f3c102870 msgr2=0x7f5f3c19df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:34.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.978+0000 7f5f431ff700 1 --2- 172.21.15.2:0/4136791895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f3c102870 0x7f5f3c19df90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5f3400d980 tx=0x7f5f3400dc90 comp rx=0 tx=0).stop 2023-09-13T04:39:34.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.979+0000 7f5f431ff700 1 -- 172.21.15.2:0/4136791895 shutdown_connections 2023-09-13T04:39:34.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.979+0000 7f5f431ff700 1 --2- 172.21.15.2:0/4136791895 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f5f240a0640 0x7f5f240a2b00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:34.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.979+0000 7f5f431ff700 1 --2- 172.21.15.2:0/4136791895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f3c102870 0x7f5f3c19df90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:34.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.979+0000 7f5f431ff700 1 --2- 172.21.15.2:0/4136791895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f3c100ce0 0x7f5f3c1969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:34.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.979+0000 7f5f431ff700 1 --2- 172.21.15.2:0/4136791895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f3c101ed0 0x7f5f3c196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:34.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.979+0000 7f5f431ff700 1 -- 172.21.15.2:0/4136791895 >> 172.21.15.2:0/4136791895 conn(0x7f5f3c0fc230 msgr2=0x7f5f3c0fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:34.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.979+0000 7f5f431ff700 1 -- 172.21.15.2:0/4136791895 shutdown_connections 2023-09-13T04:39:34.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:33.979+0000 7f5f431ff700 1 -- 172.21.15.2:0/4136791895 wait complete. 2023-09-13T04:39:34.017 INFO:tasks.cephadm:Setup complete, yielding 2023-09-13T04:39:34.018 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-09-13T04:39:34.029 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2023-09-13T04:39:34.030 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- bash -c 'ceph orch status' 2023-09-13T04:39:34.769 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:39:34.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:34 smithi149 ceph-mon[133315]: pgmap v327: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2023-09-13T04:39:34.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:34 smithi156 ceph-mon[133675]: pgmap v327: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2023-09-13T04:39:35.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:34 smithi002 ceph-mon[97651]: pgmap v327: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2023-09-13T04:39:36.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.321+0000 7f948d5ec700 1 -- 172.21.15.2:0/2267277776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9488100d70 msgr2=0x7f9488101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:36.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.321+0000 7f948d5ec700 1 --2- 172.21.15.2:0/2267277776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9488100d70 0x7f9488101190 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f94780099c0 tx=0x7f9478009cd0 comp rx=0 tx=0).stop 2023-09-13T04:39:36.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.322+0000 7f948d5ec700 1 -- 172.21.15.2:0/2267277776 shutdown_connections 2023-09-13T04:39:36.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.322+0000 7f948d5ec700 1 --2- 172.21.15.2:0/2267277776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9488102910 0x7f94881091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:36.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.322+0000 7f948d5ec700 1 --2- 172.21.15.2:0/2267277776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9488100d70 0x7f9488101190 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:36.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.322+0000 7f948d5ec700 1 --2- 172.21.15.2:0/2267277776 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9488101f70 0x7f94881023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:36.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.322+0000 7f948d5ec700 1 -- 172.21.15.2:0/2267277776 >> 172.21.15.2:0/2267277776 conn(0x7f94880fc2d0 msgr2=0x7f94880fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:36.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.322+0000 7f948d5ec700 1 -- 172.21.15.2:0/2267277776 shutdown_connections 2023-09-13T04:39:36.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.322+0000 7f948d5ec700 1 -- 172.21.15.2:0/2267277776 wait complete. 2023-09-13T04:39:36.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.323+0000 7f948d5ec700 1 Processor -- start 2023-09-13T04:39:36.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.323+0000 7f948d5ec700 1 -- start start 2023-09-13T04:39:36.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.323+0000 7f948d5ec700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9488100d70 0x7f9488198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:36.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.324+0000 7f948d5ec700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9488101f70 0x7f94881991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:36.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.324+0000 7f948d5ec700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9488102910 0x7f94881a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:36.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.324+0000 7f948d5ec700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94881a07b0 con 0x7f9488102910 2023-09-13T04:39:36.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.324+0000 7f948d5ec700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94881a08f0 con 0x7f9488101f70 2023-09-13T04:39:36.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.324+0000 7f948d5ec700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94881a0ab0 con 0x7f9488100d70 2023-09-13T04:39:36.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.324+0000 7f94877fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9488102910 0x7f94881a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:36.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.324+0000 7f9486ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9488100d70 0x7f9488198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:36.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.324+0000 7f94877fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9488102910 0x7f94881a01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43892/0 (socket says 172.21.15.2:43892) 2023-09-13T04:39:36.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.324+0000 7f94877fe700 1 -- 172.21.15.2:0/372164094 learned_addr learned my addr 172.21.15.2:0/372164094 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:36.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.324+0000 7f94867fc700 1 --2- 172.21.15.2:0/372164094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9488101f70 0x7f94881991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:36.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.324+0000 7f94877fe700 1 -- 172.21.15.2:0/372164094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9488100d70 msgr2=0x7f9488198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:36.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.324+0000 7f94877fe700 1 --2- 172.21.15.2:0/372164094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9488100d70 0x7f9488198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:36.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.324+0000 7f94877fe700 1 -- 172.21.15.2:0/372164094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9488101f70 msgr2=0x7f94881991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:36.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.324+0000 7f94877fe700 1 --2- 172.21.15.2:0/372164094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9488101f70 0x7f94881991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:36.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.324+0000 7f94877fe700 1 -- 172.21.15.2:0/372164094 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94780095f0 con 0x7f9488102910 2023-09-13T04:39:36.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.325+0000 7f94877fe700 1 --2- 172.21.15.2:0/372164094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9488102910 0x7f94881a01e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9480007ee0 tx=0x7f948000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:36.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.325+0000 7f946ffff700 1 -- 172.21.15.2:0/372164094 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480017070 con 0x7f9488102910 2023-09-13T04:39:36.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.325+0000 7f946ffff700 1 -- 172.21.15.2:0/372164094 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f948000f040 con 0x7f9488102910 2023-09-13T04:39:36.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.325+0000 7f946ffff700 1 -- 172.21.15.2:0/372164094 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480005580 con 0x7f9488102910 2023-09-13T04:39:36.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.325+0000 7f948d5ec700 1 -- 172.21.15.2:0/372164094 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94881a0d90 con 0x7f9488102910 2023-09-13T04:39:36.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.325+0000 7f948d5ec700 1 -- 172.21.15.2:0/372164094 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94881a1180 con 0x7f9488102910 2023-09-13T04:39:36.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.326+0000 7f948d5ec700 1 -- 172.21.15.2:0/372164094 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948810c5e0 con 0x7f9488102910 2023-09-13T04:39:36.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.328+0000 7f946ffff700 1 -- 172.21.15.2:0/372164094 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f948000db20 con 0x7f9488102910 2023-09-13T04:39:36.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.329+0000 7f946ffff700 1 --2- 172.21.15.2:0/372164094 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f94700a06c0 0x7f94700a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:36.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.330+0000 7f946ffff700 1 -- 172.21.15.2:0/372164094 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f94800cb5e0 con 0x7f9488102910 2023-09-13T04:39:36.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.330+0000 7f9486ffd700 1 --2- 172.21.15.2:0/372164094 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f94700a06c0 0x7f94700a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:36.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.330+0000 7f9486ffd700 1 --2- 172.21.15.2:0/372164094 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f94700a06c0 0x7f94700a2b80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f948819a140 tx=0x7f947800b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:36.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.334+0000 7f946ffff700 1 -- 172.21.15.2:0/372164094 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9480094d80 con 0x7f9488102910 2023-09-13T04:39:36.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.559+0000 7f948d5ec700 1 -- 172.21.15.2:0/372164094 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9488108910 con 0x7f94700a06c0 2023-09-13T04:39:36.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.560+0000 7f946ffff700 1 -- 172.21.15.2:0/372164094 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f9488108910 con 0x7f94700a06c0 2023-09-13T04:39:36.561 INFO:teuthology.orchestra.run.smithi002.stdout:Backend: cephadm 2023-09-13T04:39:36.561 INFO:teuthology.orchestra.run.smithi002.stdout:Available: Yes 2023-09-13T04:39:36.561 INFO:teuthology.orchestra.run.smithi002.stdout:Paused: No 2023-09-13T04:39:36.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.563+0000 7f948d5ec700 1 -- 172.21.15.2:0/372164094 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f94700a06c0 msgr2=0x7f94700a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:36.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.563+0000 7f948d5ec700 1 --2- 172.21.15.2:0/372164094 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f94700a06c0 0x7f94700a2b80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f948819a140 tx=0x7f947800b540 comp rx=0 tx=0).stop 2023-09-13T04:39:36.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.563+0000 7f948d5ec700 1 -- 172.21.15.2:0/372164094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9488102910 msgr2=0x7f94881a01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:36.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.563+0000 7f948d5ec700 1 --2- 172.21.15.2:0/372164094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9488102910 0x7f94881a01e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9480007ee0 tx=0x7f948000d3b0 comp rx=0 tx=0).stop 2023-09-13T04:39:36.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.563+0000 7f948d5ec700 1 -- 172.21.15.2:0/372164094 shutdown_connections 2023-09-13T04:39:36.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.563+0000 7f948d5ec700 1 --2- 172.21.15.2:0/372164094 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f94700a06c0 0x7f94700a2b80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:36.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.563+0000 7f948d5ec700 1 --2- 172.21.15.2:0/372164094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9488102910 0x7f94881a01e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:36.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.563+0000 7f948d5ec700 1 --2- 172.21.15.2:0/372164094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9488100d70 0x7f9488198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:36.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.563+0000 7f948d5ec700 1 --2- 172.21.15.2:0/372164094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9488101f70 0x7f94881991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:36.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.563+0000 7f948d5ec700 1 -- 172.21.15.2:0/372164094 >> 172.21.15.2:0/372164094 conn(0x7f94880fc2d0 msgr2=0x7f94880fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:36.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.563+0000 7f948d5ec700 1 -- 172.21.15.2:0/372164094 shutdown_connections 2023-09-13T04:39:36.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:36.563+0000 7f948d5ec700 1 -- 172.21.15.2:0/372164094 wait complete. 2023-09-13T04:39:36.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:36 smithi149 ceph-mon[133315]: pgmap v328: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-09-13T04:39:36.936 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:36 smithi002 ceph-mon[97651]: pgmap v328: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-09-13T04:39:36.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:36 smithi156 ceph-mon[133675]: pgmap v328: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-09-13T04:39:36.964 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- bash -c 'ceph orch ps' 2023-09-13T04:39:37.702 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:39:37.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:37 smithi149 ceph-mon[133315]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:39:37.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:37 smithi156 ceph-mon[133675]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:39:38.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:37 smithi002 ceph-mon[97651]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:39:38.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:38 smithi149 ceph-mon[133315]: pgmap v329: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:39:38.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:38 smithi156 ceph-mon[133675]: pgmap v329: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:39:38.962 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:38 smithi002 ceph-mon[97651]: pgmap v329: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:39:39.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.238+0000 7fda0f647700 1 -- 172.21.15.2:0/2206789363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda08100d70 msgr2=0x7fda08101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:39.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.238+0000 7fda0f647700 1 --2- 172.21.15.2:0/2206789363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda08100d70 0x7fda08101190 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fda040099c0 tx=0x7fda04009cd0 comp rx=0 tx=0).stop 2023-09-13T04:39:39.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.238+0000 7fda0f647700 1 -- 172.21.15.2:0/2206789363 shutdown_connections 2023-09-13T04:39:39.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.238+0000 7fda0f647700 1 --2- 172.21.15.2:0/2206789363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda08102910 0x7fda081091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:39.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.238+0000 7fda0f647700 1 --2- 172.21.15.2:0/2206789363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda08100d70 0x7fda08101190 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:39.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.238+0000 7fda0f647700 1 --2- 172.21.15.2:0/2206789363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda08101f70 0x7fda081023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:39.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.238+0000 7fda0f647700 1 -- 172.21.15.2:0/2206789363 >> 172.21.15.2:0/2206789363 conn(0x7fda080fc2d0 msgr2=0x7fda080fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:39.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.238+0000 7fda0f647700 1 -- 172.21.15.2:0/2206789363 shutdown_connections 2023-09-13T04:39:39.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.238+0000 7fda0f647700 1 -- 172.21.15.2:0/2206789363 wait complete. 2023-09-13T04:39:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.239+0000 7fda0f647700 1 Processor -- start 2023-09-13T04:39:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.239+0000 7fda0f647700 1 -- start start 2023-09-13T04:39:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.240+0000 7fda0f647700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda08100d70 0x7fda08074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:39.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.240+0000 7fda0f647700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda08101f70 0x7fda080730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:39.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.240+0000 7fda0f647700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda08102910 0x7fda080735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:39.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.240+0000 7fda0f647700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda08073d60 con 0x7fda08100d70 2023-09-13T04:39:39.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.240+0000 7fda0f647700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda08073ed0 con 0x7fda08101f70 2023-09-13T04:39:39.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.240+0000 7fda0f647700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda081a0af0 con 0x7fda08102910 2023-09-13T04:39:39.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.240+0000 7fda0d3e3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda08100d70 0x7fda08074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:39.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.240+0000 7fda0d3e3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda08100d70 0x7fda08074a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43908/0 (socket says 172.21.15.2:43908) 2023-09-13T04:39:39.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.240+0000 7fda0d3e3700 1 -- 172.21.15.2:0/289859813 learned_addr learned my addr 172.21.15.2:0/289859813 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:39.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.240+0000 7fda0cbe2700 1 --2- 172.21.15.2:0/289859813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda08101f70 0x7fda080730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:39.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.240+0000 7fda0dbe4700 1 --2- 172.21.15.2:0/289859813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda08102910 0x7fda080735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:39.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.241+0000 7fda0d3e3700 1 -- 172.21.15.2:0/289859813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda08102910 msgr2=0x7fda080735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:39.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.241+0000 7fda0d3e3700 1 --2- 172.21.15.2:0/289859813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda08102910 0x7fda080735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:39.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.241+0000 7fda0d3e3700 1 -- 172.21.15.2:0/289859813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda08101f70 msgr2=0x7fda080730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:39.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.241+0000 7fda0d3e3700 1 --2- 172.21.15.2:0/289859813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda08101f70 0x7fda080730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:39.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.241+0000 7fda0d3e3700 1 -- 172.21.15.2:0/289859813 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda040095f0 con 0x7fda08100d70 2023-09-13T04:39:39.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.241+0000 7fda0dbe4700 1 --2- 172.21.15.2:0/289859813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda08102910 0x7fda080735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:39:39.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.241+0000 7fda0cbe2700 1 --2- 172.21.15.2:0/289859813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda08101f70 0x7fda080730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:39:39.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.241+0000 7fda0d3e3700 1 --2- 172.21.15.2:0/289859813 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda08100d70 0x7fda08074a50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fda040036f0 tx=0x7fda04009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:39.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.241+0000 7fd9fe7fc700 1 -- 172.21.15.2:0/289859813 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0401d070 con 0x7fda08100d70 2023-09-13T04:39:39.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.241+0000 7fd9fe7fc700 1 -- 172.21.15.2:0/289859813 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda04004110 con 0x7fda08100d70 2023-09-13T04:39:39.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.241+0000 7fda0f647700 1 -- 172.21.15.2:0/289859813 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda081a0d70 con 0x7fda08100d70 2023-09-13T04:39:39.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.241+0000 7fd9fe7fc700 1 -- 172.21.15.2:0/289859813 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda04021410 con 0x7fda08100d70 2023-09-13T04:39:39.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.241+0000 7fda0f647700 1 -- 172.21.15.2:0/289859813 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda081a1190 con 0x7fda08100d70 2023-09-13T04:39:39.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.242+0000 7fda0f647700 1 -- 172.21.15.2:0/289859813 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda08108360 con 0x7fda08100d70 2023-09-13T04:39:39.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.244+0000 7fd9fe7fc700 1 -- 172.21.15.2:0/289859813 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fda0400f660 con 0x7fda08100d70 2023-09-13T04:39:39.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.246+0000 7fd9fe7fc700 1 --2- 172.21.15.2:0/289859813 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd9f40a06c0 0x7fd9f40a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:39.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.246+0000 7fd9fe7fc700 1 -- 172.21.15.2:0/289859813 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fda040ccea0 con 0x7fda08100d70 2023-09-13T04:39:39.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.246+0000 7fda0cbe2700 1 --2- 172.21.15.2:0/289859813 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd9f40a06c0 0x7fd9f40a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:39.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.247+0000 7fda0cbe2700 1 --2- 172.21.15.2:0/289859813 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd9f40a06c0 0x7fd9f40a2b80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8005950 tx=0x7fd9f80058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:39.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.250+0000 7fd9fe7fc700 1 -- 172.21.15.2:0/289859813 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fda040966d0 con 0x7fda08100d70 2023-09-13T04:39:39.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.470+0000 7fda0f647700 1 -- 172.21.15.2:0/289859813 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fda0802d060 con 0x7fd9f40a06c0 2023-09-13T04:39:39.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.483+0000 7fd9fe7fc700 1 -- 172.21.15.2:0/289859813 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7fda0802d060 con 0x7fd9f40a06c0 2023-09-13T04:39:39.484 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-09-13T04:39:39.484 INFO:teuthology.orchestra.run.smithi002.stdout:mds.a smithi002 running (109s) - 108s 17.0M - 18.0.0-6083-g4a754ac9 ac82d04f119c 9244ceb730c1 2023-09-13T04:39:39.484 INFO:teuthology.orchestra.run.smithi002.stdout:mds.b smithi149 running (107s) - 106s 25.0M - 18.0.0-6083-g4a754ac9 ac82d04f119c 5cb269cd0136 2023-09-13T04:39:39.485 INFO:teuthology.orchestra.run.smithi002.stdout:mds.c smithi156 running (119s) - 119s 19.8M - 18.0.0-6083-g4a754ac9 ac82d04f119c ddc6fedb9be4 2023-09-13T04:39:39.485 INFO:teuthology.orchestra.run.smithi002.stdout:mds.d smithi002 running (2m) - 2m 13.4M - 18.0.0-6083-g4a754ac9 ac82d04f119c 4ff4715e5eed 2023-09-13T04:39:39.485 INFO:teuthology.orchestra.run.smithi002.stdout:mds.e smithi149 running (2m) - 2m 13.8M - 18.0.0-6083-g4a754ac9 ac82d04f119c 0618f1c6f313 2023-09-13T04:39:39.485 INFO:teuthology.orchestra.run.smithi002.stdout:mds.f smithi156 running (2m) - 2m 18.5M - 18.0.0-6083-g4a754ac9 ac82d04f119c f215d1637903 2023-09-13T04:39:39.485 INFO:teuthology.orchestra.run.smithi002.stdout:mds.g smithi002 running (112s) - 112s 13.4M - 18.0.0-6083-g4a754ac9 ac82d04f119c 2d8fb06de536 2023-09-13T04:39:39.486 INFO:teuthology.orchestra.run.smithi002.stdout:mds.h smithi149 running (117s) - 117s 19.5M - 18.0.0-6083-g4a754ac9 ac82d04f119c ee34900a5f4a 2023-09-13T04:39:39.486 INFO:teuthology.orchestra.run.smithi002.stdout:mds.i smithi156 running (115s) - 115s 20.3M - 18.0.0-6083-g4a754ac9 ac82d04f119c cbdd722ea508 2023-09-13T04:39:39.486 INFO:teuthology.orchestra.run.smithi002.stdout:mds.j smithi002 running (2m) - 2m 13.5M - 18.0.0-6083-g4a754ac9 ac82d04f119c 703b451296b9 2023-09-13T04:39:39.486 INFO:teuthology.orchestra.run.smithi002.stdout:mds.k smithi149 running (2m) - 2m 13.9M - 18.0.0-6083-g4a754ac9 ac82d04f119c 4532367ab42a 2023-09-13T04:39:39.486 INFO:teuthology.orchestra.run.smithi002.stdout:mds.l smithi156 running (2m) - 2m 16.2M - 18.0.0-6083-g4a754ac9 ac82d04f119c 8620f51fa936 2023-09-13T04:39:39.487 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.x smithi002 *:9283,8765 running (9m) - 9m 524M - 18.0.0-6083-g4a754ac9 ac82d04f119c b25c57d8eb9f 2023-09-13T04:39:39.487 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.y smithi149 *:8443,8765 running (7m) - 7m 449M - 18.0.0-6083-g4a754ac9 ac82d04f119c 23d3c1fec40b 2023-09-13T04:39:39.487 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.z smithi156 *:8443,8765 running (7m) - 7m 448M - 18.0.0-6083-g4a754ac9 ac82d04f119c 6b87df66ee6d 2023-09-13T04:39:39.487 INFO:teuthology.orchestra.run.smithi002.stdout:mon.a smithi002 running (9m) - 9m 92.0M 2048M 18.0.0-6083-g4a754ac9 ac82d04f119c 0b778730e2c8 2023-09-13T04:39:39.487 INFO:teuthology.orchestra.run.smithi002.stdout:mon.b smithi149 running (7m) - 7m 65.2M 2048M 18.0.0-6083-g4a754ac9 ac82d04f119c 4dffc1de4c8a 2023-09-13T04:39:39.488 INFO:teuthology.orchestra.run.smithi002.stdout:mon.c smithi156 running (7m) - 7m 60.9M 2048M 18.0.0-6083-g4a754ac9 ac82d04f119c 321adaa12f7d 2023-09-13T04:39:39.488 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (6m) - 6m 71.4M 4318M 18.0.0-6083-g4a754ac9 ac82d04f119c c4f439aadcad 2023-09-13T04:39:39.488 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi149 running (6m) - 6m 75.8M 1469M 18.0.0-6083-g4a754ac9 ac82d04f119c cf784ca9089a 2023-09-13T04:39:39.488 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi156 running (6m) - 6m 74.8M 4292M 18.0.0-6083-g4a754ac9 ac82d04f119c 298931c5aeb5 2023-09-13T04:39:39.488 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi002 running (5m) - 5m 65.9M 4318M 18.0.0-6083-g4a754ac9 ac82d04f119c 03a0cea0c9c5 2023-09-13T04:39:39.489 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi149 running (5m) - 5m 63.2M 1469M 18.0.0-6083-g4a754ac9 ac82d04f119c 75cf45c5ef08 2023-09-13T04:39:39.489 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi156 running (4m) - 4m 68.4M 4292M 18.0.0-6083-g4a754ac9 ac82d04f119c 1c0b3698ee6a 2023-09-13T04:39:39.489 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6 smithi002 running (4m) - 4m 63.5M 4318M 18.0.0-6083-g4a754ac9 ac82d04f119c 26fc7b72a5bd 2023-09-13T04:39:39.489 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7 smithi149 running (4m) - 4m 67.1M 1469M 18.0.0-6083-g4a754ac9 ac82d04f119c b9c9faa73ece 2023-09-13T04:39:39.489 INFO:teuthology.orchestra.run.smithi002.stdout:osd.8 smithi156 running (3m) - 3m 61.0M 4292M 18.0.0-6083-g4a754ac9 ac82d04f119c a13551465c23 2023-09-13T04:39:39.490 INFO:teuthology.orchestra.run.smithi002.stdout:osd.9 smithi002 running (3m) - 3m 60.1M 4318M 18.0.0-6083-g4a754ac9 ac82d04f119c 69ddd3357ece 2023-09-13T04:39:39.490 INFO:teuthology.orchestra.run.smithi002.stdout:osd.10 smithi149 running (2m) - 2m 61.8M 1469M 18.0.0-6083-g4a754ac9 ac82d04f119c 22bc3dffd692 2023-09-13T04:39:39.490 INFO:teuthology.orchestra.run.smithi002.stdout:osd.11 smithi156 running (2m) - 2m 57.6M 4292M 18.0.0-6083-g4a754ac9 ac82d04f119c 31ac57e558a0 2023-09-13T04:39:39.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.486+0000 7fda0f647700 1 -- 172.21.15.2:0/289859813 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd9f40a06c0 msgr2=0x7fd9f40a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:39.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.486+0000 7fda0f647700 1 --2- 172.21.15.2:0/289859813 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd9f40a06c0 0x7fd9f40a2b80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8005950 tx=0x7fd9f80058e0 comp rx=0 tx=0).stop 2023-09-13T04:39:39.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.486+0000 7fda0f647700 1 -- 172.21.15.2:0/289859813 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda08100d70 msgr2=0x7fda08074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:39.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.486+0000 7fda0f647700 1 --2- 172.21.15.2:0/289859813 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda08100d70 0x7fda08074a50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fda040036f0 tx=0x7fda04009580 comp rx=0 tx=0).stop 2023-09-13T04:39:39.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.486+0000 7fda0f647700 1 -- 172.21.15.2:0/289859813 shutdown_connections 2023-09-13T04:39:39.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.486+0000 7fda0f647700 1 --2- 172.21.15.2:0/289859813 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd9f40a06c0 0x7fd9f40a2b80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:39.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.486+0000 7fda0f647700 1 --2- 172.21.15.2:0/289859813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda08102910 0x7fda080735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:39.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.486+0000 7fda0f647700 1 --2- 172.21.15.2:0/289859813 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda08100d70 0x7fda08074a50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:39.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.486+0000 7fda0f647700 1 --2- 172.21.15.2:0/289859813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda08101f70 0x7fda080730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:39.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.486+0000 7fda0f647700 1 -- 172.21.15.2:0/289859813 >> 172.21.15.2:0/289859813 conn(0x7fda080fc2d0 msgr2=0x7fda081054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:39.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.486+0000 7fda0f647700 1 -- 172.21.15.2:0/289859813 shutdown_connections 2023-09-13T04:39:39.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:39.486+0000 7fda0f647700 1 -- 172.21.15.2:0/289859813 wait complete. 2023-09-13T04:39:39.895 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- bash -c 'ceph orch ls' 2023-09-13T04:39:40.628 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:39:40.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:40 smithi149 ceph-mon[133315]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:39:40.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:40 smithi149 ceph-mon[133315]: pgmap v330: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:39:40.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:40 smithi156 ceph-mon[133675]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:39:40.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:40 smithi156 ceph-mon[133675]: pgmap v330: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:39:40.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:40 smithi002 ceph-mon[97651]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:39:40.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:40 smithi002 ceph-mon[97651]: pgmap v330: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:39:42.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.146+0000 7f0fcd740700 1 -- 172.21.15.2:0/2796720076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fc8074c80 msgr2=0x7f0fc80730e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:42.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.146+0000 7f0fcd740700 1 --2- 172.21.15.2:0/2796720076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fc8074c80 0x7f0fc80730e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8009990 tx=0x7f0fb8009ca0 comp rx=0 tx=0).stop 2023-09-13T04:39:42.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.146+0000 7f0fcd740700 1 -- 172.21.15.2:0/2796720076 shutdown_connections 2023-09-13T04:39:42.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.146+0000 7f0fcd740700 1 --2- 172.21.15.2:0/2796720076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fc8108c50 0x7f0fc810b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:42.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.146+0000 7f0fcd740700 1 --2- 172.21.15.2:0/2796720076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fc8074c80 0x7f0fc80730e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:42.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.146+0000 7f0fcd740700 1 --2- 172.21.15.2:0/2796720076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fc80736b0 0x7f0fc8073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:42.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.146+0000 7f0fcd740700 1 -- 172.21.15.2:0/2796720076 >> 172.21.15.2:0/2796720076 conn(0x7f0fc80fc2d0 msgr2=0x7f0fc80fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:42.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.147+0000 7f0fcd740700 1 -- 172.21.15.2:0/2796720076 shutdown_connections 2023-09-13T04:39:42.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.147+0000 7f0fcd740700 1 -- 172.21.15.2:0/2796720076 wait complete. 2023-09-13T04:39:42.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.148+0000 7f0fcd740700 1 Processor -- start 2023-09-13T04:39:42.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.148+0000 7f0fcd740700 1 -- start start 2023-09-13T04:39:42.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.148+0000 7f0fcd740700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fc80736b0 0x7f0fc8198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:42.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.148+0000 7f0fcd740700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fc8074c80 0x7f0fc81991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:42.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.148+0000 7f0fcd740700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fc8108c50 0x7f0fc81a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:42.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.148+0000 7f0fcd740700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc81a07e0 con 0x7f0fc80736b0 2023-09-13T04:39:42.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.148+0000 7f0fcd740700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc81a0920 con 0x7f0fc8074c80 2023-09-13T04:39:42.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.148+0000 7f0fcd740700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc81a0ae0 con 0x7f0fc8108c50 2023-09-13T04:39:42.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.148+0000 7f0fc6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fc80736b0 0x7f0fc8198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:42.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.148+0000 7f0fc6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fc80736b0 0x7f0fc8198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43928/0 (socket says 172.21.15.2:43928) 2023-09-13T04:39:42.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.148+0000 7f0fc6ffd700 1 -- 172.21.15.2:0/1374684752 learned_addr learned my addr 172.21.15.2:0/1374684752 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:42.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.148+0000 7f0fc67fc700 1 --2- 172.21.15.2:0/1374684752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fc8074c80 0x7f0fc81991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:42.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.149+0000 7f0fc77fe700 1 --2- 172.21.15.2:0/1374684752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fc8108c50 0x7f0fc81a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:42.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.149+0000 7f0fc6ffd700 1 -- 172.21.15.2:0/1374684752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fc8108c50 msgr2=0x7f0fc81a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.149+0000 7f0fc6ffd700 1 --2- 172.21.15.2:0/1374684752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fc8108c50 0x7f0fc81a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.149+0000 7f0fc6ffd700 1 -- 172.21.15.2:0/1374684752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fc8074c80 msgr2=0x7f0fc81991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.149+0000 7f0fc6ffd700 1 --2- 172.21.15.2:0/1374684752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fc8074c80 0x7f0fc81991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.149+0000 7f0fc6ffd700 1 -- 172.21.15.2:0/1374684752 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb80095f0 con 0x7f0fc80736b0 2023-09-13T04:39:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.149+0000 7f0fc67fc700 1 --2- 172.21.15.2:0/1374684752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fc8074c80 0x7f0fc81991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:39:42.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.149+0000 7f0fc6ffd700 1 --2- 172.21.15.2:0/1374684752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fc80736b0 0x7f0fc8198c60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8004740 tx=0x7f0fb8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:42.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.149+0000 7f0faffff700 1 -- 172.21.15.2:0/1374684752 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb801d070 con 0x7f0fc80736b0 2023-09-13T04:39:42.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.150+0000 7f0fcd740700 1 -- 172.21.15.2:0/1374684752 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc81a0d60 con 0x7f0fc80736b0 2023-09-13T04:39:42.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.150+0000 7f0faffff700 1 -- 172.21.15.2:0/1374684752 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fb8004830 con 0x7f0fc80736b0 2023-09-13T04:39:42.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.150+0000 7f0faffff700 1 -- 172.21.15.2:0/1374684752 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb80053c0 con 0x7f0fc80736b0 2023-09-13T04:39:42.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.150+0000 7f0fcd740700 1 -- 172.21.15.2:0/1374684752 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc81a1110 con 0x7f0fc80736b0 2023-09-13T04:39:42.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.151+0000 7f0fcd740700 1 -- 172.21.15.2:0/1374684752 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc8107350 con 0x7f0fc80736b0 2023-09-13T04:39:42.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.153+0000 7f0faffff700 1 -- 172.21.15.2:0/1374684752 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f0fb8005560 con 0x7f0fc80736b0 2023-09-13T04:39:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.155+0000 7f0faffff700 1 --2- 172.21.15.2:0/1374684752 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f0fb00a0640 0x7f0fb00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.155+0000 7f0faffff700 1 -- 172.21.15.2:0/1374684752 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0fb80cd090 con 0x7f0fc80736b0 2023-09-13T04:39:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.155+0000 7f0fc67fc700 1 --2- 172.21.15.2:0/1374684752 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f0fb00a0640 0x7f0fb00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.156+0000 7f0fc67fc700 1 --2- 172.21.15.2:0/1374684752 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f0fb00a0640 0x7f0fb00a2b00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0fc819a170 tx=0x7f0fbc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:42.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.160+0000 7f0faffff700 1 -- 172.21.15.2:0/1374684752 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0fb805ae10 con 0x7f0fc80736b0 2023-09-13T04:39:42.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.379+0000 7f0fcd740700 1 -- 172.21.15.2:0/1374684752 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fc802d000 con 0x7f0fb00a0640 2023-09-13T04:39:42.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.384+0000 7f0faffff700 1 -- 172.21.15.2:0/1374684752 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f0fc802d000 con 0x7f0fb00a0640 2023-09-13T04:39:42.385 INFO:teuthology.orchestra.run.smithi002.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-09-13T04:39:42.385 INFO:teuthology.orchestra.run.smithi002.stdout:mds.all 12/12 - 2m smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l;count:12 2023-09-13T04:39:42.385 INFO:teuthology.orchestra.run.smithi002.stdout:mgr 3/3 - 7m smithi002=x;smithi149=y;smithi156=z;count:3 2023-09-13T04:39:42.386 INFO:teuthology.orchestra.run.smithi002.stdout:mon 3/3 - 7m smithi002:172.21.15.2=a;smithi149:172.21.15.149=b;smithi156:172.21.15.156=c;count:3 2023-09-13T04:39:42.386 INFO:teuthology.orchestra.run.smithi002.stdout:osd 12 - - 2023-09-13T04:39:42.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.387+0000 7f0fcd740700 1 -- 172.21.15.2:0/1374684752 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f0fb00a0640 msgr2=0x7f0fb00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:42.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.387+0000 7f0fcd740700 1 --2- 172.21.15.2:0/1374684752 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f0fb00a0640 0x7f0fb00a2b00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0fc819a170 tx=0x7f0fbc009260 comp rx=0 tx=0).stop 2023-09-13T04:39:42.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.387+0000 7f0fcd740700 1 -- 172.21.15.2:0/1374684752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fc80736b0 msgr2=0x7f0fc8198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:42.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.387+0000 7f0fcd740700 1 --2- 172.21.15.2:0/1374684752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fc80736b0 0x7f0fc8198c60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8004740 tx=0x7f0fb8009580 comp rx=0 tx=0).stop 2023-09-13T04:39:42.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.387+0000 7f0fcd740700 1 -- 172.21.15.2:0/1374684752 shutdown_connections 2023-09-13T04:39:42.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.387+0000 7f0fcd740700 1 --2- 172.21.15.2:0/1374684752 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f0fb00a0640 0x7f0fb00a2b00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:42.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.387+0000 7f0fcd740700 1 --2- 172.21.15.2:0/1374684752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fc8108c50 0x7f0fc81a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:42.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.387+0000 7f0fcd740700 1 --2- 172.21.15.2:0/1374684752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fc80736b0 0x7f0fc8198c60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:42.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.387+0000 7f0fcd740700 1 --2- 172.21.15.2:0/1374684752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fc8074c80 0x7f0fc81991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:42.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.387+0000 7f0fcd740700 1 -- 172.21.15.2:0/1374684752 >> 172.21.15.2:0/1374684752 conn(0x7f0fc80fc2d0 msgr2=0x7f0fc80fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:42.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.387+0000 7f0fcd740700 1 -- 172.21.15.2:0/1374684752 shutdown_connections 2023-09-13T04:39:42.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:42.387+0000 7f0fcd740700 1 -- 172.21.15.2:0/1374684752 wait complete. 2023-09-13T04:39:42.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:42 smithi002 ceph-mon[97651]: pgmap v331: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-13T04:39:42.823 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- bash -c 'ceph orch host ls' 2023-09-13T04:39:42.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:42 smithi149 ceph-mon[133315]: pgmap v331: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-13T04:39:42.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:42 smithi156 ceph-mon[133675]: pgmap v331: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-13T04:39:43.566 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:39:43.842 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:43 smithi002 ceph-mon[97651]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:39:43.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:43 smithi149 ceph-mon[133315]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:39:43.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:43 smithi156 ceph-mon[133675]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:39:44.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:44 smithi002 ceph-mon[97651]: pgmap v332: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:39:44.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:44 smithi149 ceph-mon[133315]: pgmap v332: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:39:44.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:44 smithi156 ceph-mon[133675]: pgmap v332: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:39:45.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.122+0000 7f6e81f33700 1 -- 172.21.15.2:0/2715271841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e7c101f70 msgr2=0x7f6e7c1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:45.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.122+0000 7f6e81f33700 1 --2- 172.21.15.2:0/2715271841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e7c101f70 0x7f6e7c1023d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6e700099c0 tx=0x7f6e70009cd0 comp rx=0 tx=0).stop 2023-09-13T04:39:45.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.122+0000 7f6e81f33700 1 -- 172.21.15.2:0/2715271841 shutdown_connections 2023-09-13T04:39:45.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.122+0000 7f6e81f33700 1 --2- 172.21.15.2:0/2715271841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e7c102910 0x7f6e7c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:45.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.122+0000 7f6e81f33700 1 --2- 172.21.15.2:0/2715271841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e7c100d70 0x7f6e7c101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:45.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.122+0000 7f6e81f33700 1 --2- 172.21.15.2:0/2715271841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e7c101f70 0x7f6e7c1023d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:45.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.122+0000 7f6e81f33700 1 -- 172.21.15.2:0/2715271841 >> 172.21.15.2:0/2715271841 conn(0x7f6e7c0fc2d0 msgr2=0x7f6e7c0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:45.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.122+0000 7f6e81f33700 1 -- 172.21.15.2:0/2715271841 shutdown_connections 2023-09-13T04:39:45.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.122+0000 7f6e81f33700 1 -- 172.21.15.2:0/2715271841 wait complete. 2023-09-13T04:39:45.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.123+0000 7f6e81f33700 1 Processor -- start 2023-09-13T04:39:45.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.123+0000 7f6e81f33700 1 -- start start 2023-09-13T04:39:45.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.123+0000 7f6e81f33700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e7c100d70 0x7f6e7c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:45.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.124+0000 7f6e81f33700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e7c101f70 0x7f6e7c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:45.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.124+0000 7f6e81f33700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e7c102910 0x7f6e7c1a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:45.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.124+0000 7f6e81f33700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c1a07f0 con 0x7f6e7c102910 2023-09-13T04:39:45.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.124+0000 7f6e81f33700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c1a0930 con 0x7f6e7c100d70 2023-09-13T04:39:45.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.124+0000 7f6e81f33700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c1a0af0 con 0x7f6e7c101f70 2023-09-13T04:39:45.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.124+0000 7f6e7bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e7c102910 0x7f6e7c1a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:45.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.124+0000 7f6e7bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e7c102910 0x7f6e7c1a0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37126/0 (socket says 172.21.15.2:37126) 2023-09-13T04:39:45.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.124+0000 7f6e7bfff700 1 -- 172.21.15.2:0/2049230889 learned_addr learned my addr 172.21.15.2:0/2049230889 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:45.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.124+0000 7f6e7affd700 1 --2- 172.21.15.2:0/2049230889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e7c101f70 0x7f6e7c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:45.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.124+0000 7f6e7b7fe700 1 --2- 172.21.15.2:0/2049230889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e7c100d70 0x7f6e7c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:45.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.124+0000 7f6e7bfff700 1 -- 172.21.15.2:0/2049230889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e7c101f70 msgr2=0x7f6e7c1991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:45.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.124+0000 7f6e7bfff700 1 --2- 172.21.15.2:0/2049230889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e7c101f70 0x7f6e7c1991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:45.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.124+0000 7f6e7bfff700 1 -- 172.21.15.2:0/2049230889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e7c100d70 msgr2=0x7f6e7c198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:45.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.124+0000 7f6e7bfff700 1 --2- 172.21.15.2:0/2049230889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e7c100d70 0x7f6e7c198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:45.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.124+0000 7f6e7bfff700 1 -- 172.21.15.2:0/2049230889 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e700095f0 con 0x7f6e7c102910 2023-09-13T04:39:45.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.125+0000 7f6e7bfff700 1 --2- 172.21.15.2:0/2049230889 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e7c102910 0x7f6e7c1a0220 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400ef30 tx=0x7f6e7400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:45.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.125+0000 7f6e78ff9700 1 -- 172.21.15.2:0/2049230889 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74018070 con 0x7f6e7c102910 2023-09-13T04:39:45.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.125+0000 7f6e78ff9700 1 -- 172.21.15.2:0/2049230889 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e7400f040 con 0x7f6e7c102910 2023-09-13T04:39:45.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.125+0000 7f6e81f33700 1 -- 172.21.15.2:0/2049230889 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7c1a0d70 con 0x7f6e7c102910 2023-09-13T04:39:45.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.125+0000 7f6e78ff9700 1 -- 172.21.15.2:0/2049230889 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7400cd00 con 0x7f6e7c102910 2023-09-13T04:39:45.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.125+0000 7f6e81f33700 1 -- 172.21.15.2:0/2049230889 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7c1a1230 con 0x7f6e7c102910 2023-09-13T04:39:45.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.126+0000 7f6e81f33700 1 -- 172.21.15.2:0/2049230889 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e7c1933b0 con 0x7f6e7c102910 2023-09-13T04:39:45.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.128+0000 7f6e78ff9700 1 -- 172.21.15.2:0/2049230889 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f6e74003a00 con 0x7f6e7c102910 2023-09-13T04:39:45.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.129+0000 7f6e78ff9700 1 --2- 172.21.15.2:0/2049230889 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f6e640a06c0 0x7f6e640a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:45.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.130+0000 7f6e78ff9700 1 -- 172.21.15.2:0/2049230889 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6e740cb9a0 con 0x7f6e7c102910 2023-09-13T04:39:45.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.130+0000 7f6e7b7fe700 1 --2- 172.21.15.2:0/2049230889 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f6e640a06c0 0x7f6e640a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:45.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.130+0000 7f6e7b7fe700 1 --2- 172.21.15.2:0/2049230889 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f6e640a06c0 0x7f6e640a2b80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c199b50 tx=0x7f6e6c00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:45.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.134+0000 7f6e78ff9700 1 -- 172.21.15.2:0/2049230889 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6e740951d0 con 0x7f6e7c102910 2023-09-13T04:39:45.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.353+0000 7f6e81f33700 1 -- 172.21.15.2:0/2049230889 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e7c02d0e0 con 0x7f6e640a06c0 2023-09-13T04:39:45.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.354+0000 7f6e78ff9700 1 -- 172.21.15.2:0/2049230889 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f6e7c02d0e0 con 0x7f6e640a06c0 2023-09-13T04:39:45.355 INFO:teuthology.orchestra.run.smithi002.stdout:HOST ADDR LABELS STATUS 2023-09-13T04:39:45.355 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002 172.21.15.2 2023-09-13T04:39:45.355 INFO:teuthology.orchestra.run.smithi002.stdout:smithi149 172.21.15.149 2023-09-13T04:39:45.356 INFO:teuthology.orchestra.run.smithi002.stdout:smithi156 172.21.15.156 2023-09-13T04:39:45.356 INFO:teuthology.orchestra.run.smithi002.stdout:3 hosts in cluster 2023-09-13T04:39:45.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.357+0000 7f6e81f33700 1 -- 172.21.15.2:0/2049230889 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f6e640a06c0 msgr2=0x7f6e640a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:45.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.357+0000 7f6e81f33700 1 --2- 172.21.15.2:0/2049230889 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f6e640a06c0 0x7f6e640a2b80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c199b50 tx=0x7f6e6c00b410 comp rx=0 tx=0).stop 2023-09-13T04:39:45.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.357+0000 7f6e81f33700 1 -- 172.21.15.2:0/2049230889 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e7c102910 msgr2=0x7f6e7c1a0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:45.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.357+0000 7f6e81f33700 1 --2- 172.21.15.2:0/2049230889 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e7c102910 0x7f6e7c1a0220 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400ef30 tx=0x7f6e7400c3a0 comp rx=0 tx=0).stop 2023-09-13T04:39:45.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.357+0000 7f6e81f33700 1 -- 172.21.15.2:0/2049230889 shutdown_connections 2023-09-13T04:39:45.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.357+0000 7f6e81f33700 1 --2- 172.21.15.2:0/2049230889 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f6e640a06c0 0x7f6e640a2b80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:45.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.357+0000 7f6e81f33700 1 --2- 172.21.15.2:0/2049230889 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e7c102910 0x7f6e7c1a0220 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:45.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.357+0000 7f6e81f33700 1 --2- 172.21.15.2:0/2049230889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e7c100d70 0x7f6e7c198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:45.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.357+0000 7f6e81f33700 1 --2- 172.21.15.2:0/2049230889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e7c101f70 0x7f6e7c1991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:45.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.357+0000 7f6e81f33700 1 -- 172.21.15.2:0/2049230889 >> 172.21.15.2:0/2049230889 conn(0x7f6e7c0fc2d0 msgr2=0x7f6e7c107890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:45.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.357+0000 7f6e81f33700 1 -- 172.21.15.2:0/2049230889 shutdown_connections 2023-09-13T04:39:45.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:45.357+0000 7f6e81f33700 1 -- 172.21.15.2:0/2049230889 wait complete. 2023-09-13T04:39:45.761 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- bash -c 'ceph orch device ls' 2023-09-13T04:39:46.498 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:39:46.860 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:46 smithi002 ceph-mon[97651]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:39:46.860 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:46 smithi002 ceph-mon[97651]: pgmap v333: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:39:46.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:46 smithi149 ceph-mon[133315]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:39:46.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:46 smithi149 ceph-mon[133315]: pgmap v333: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:39:46.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:46 smithi156 ceph-mon[133675]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:39:46.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:46 smithi156 ceph-mon[133675]: pgmap v333: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:39:48.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.014+0000 7ffb88465700 1 -- 172.21.15.2:0/3220006377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb80100d70 msgr2=0x7ffb80101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:48.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.014+0000 7ffb88465700 1 --2- 172.21.15.2:0/3220006377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb80100d70 0x7ffb80101190 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0099c0 tx=0x7ffb7c009cd0 comp rx=0 tx=0).stop 2023-09-13T04:39:48.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.014+0000 7ffb88465700 1 -- 172.21.15.2:0/3220006377 shutdown_connections 2023-09-13T04:39:48.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.014+0000 7ffb88465700 1 --2- 172.21.15.2:0/3220006377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb80102910 0x7ffb801091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:48.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.014+0000 7ffb88465700 1 --2- 172.21.15.2:0/3220006377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb80100d70 0x7ffb80101190 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:48.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.014+0000 7ffb88465700 1 --2- 172.21.15.2:0/3220006377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb80101f70 0x7ffb801023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:48.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.014+0000 7ffb88465700 1 -- 172.21.15.2:0/3220006377 >> 172.21.15.2:0/3220006377 conn(0x7ffb800fc2d0 msgr2=0x7ffb800fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:48.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.014+0000 7ffb88465700 1 -- 172.21.15.2:0/3220006377 shutdown_connections 2023-09-13T04:39:48.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.014+0000 7ffb88465700 1 -- 172.21.15.2:0/3220006377 wait complete. 2023-09-13T04:39:48.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.016+0000 7ffb88465700 1 Processor -- start 2023-09-13T04:39:48.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.016+0000 7ffb88465700 1 -- start start 2023-09-13T04:39:48.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.016+0000 7ffb88465700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb80100d70 0x7ffb80196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:48.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.016+0000 7ffb88465700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb80101f70 0x7ffb80196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:48.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.016+0000 7ffb88465700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb80102910 0x7ffb8019e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:48.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.016+0000 7ffb88465700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8019e610 con 0x7ffb80102910 2023-09-13T04:39:48.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.016+0000 7ffb88465700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8019e750 con 0x7ffb80101f70 2023-09-13T04:39:48.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.016+0000 7ffb88465700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8019e8e0 con 0x7ffb80100d70 2023-09-13T04:39:48.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.017+0000 7ffb86a02700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb80102910 0x7ffb8019e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:48.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.017+0000 7ffb85a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb80101f70 0x7ffb80196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:48.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.017+0000 7ffb86201700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb80100d70 0x7ffb80196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:48.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.017+0000 7ffb85a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb80101f70 0x7ffb80196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:58842/0 (socket says 172.21.15.2:58842) 2023-09-13T04:39:48.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.017+0000 7ffb86a02700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb80102910 0x7ffb8019e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37150/0 (socket says 172.21.15.2:37150) 2023-09-13T04:39:48.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.017+0000 7ffb85a00700 1 -- 172.21.15.2:0/3546543659 learned_addr learned my addr 172.21.15.2:0/3546543659 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:48.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.017+0000 7ffb86a02700 1 -- 172.21.15.2:0/3546543659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb80100d70 msgr2=0x7ffb80196a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:48.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.017+0000 7ffb86a02700 1 --2- 172.21.15.2:0/3546543659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb80100d70 0x7ffb80196a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:48.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.017+0000 7ffb86a02700 1 -- 172.21.15.2:0/3546543659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb80101f70 msgr2=0x7ffb80196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:48.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.017+0000 7ffb86a02700 1 --2- 172.21.15.2:0/3546543659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb80101f70 0x7ffb80196fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:48.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.017+0000 7ffb86a02700 1 -- 172.21.15.2:0/3546543659 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb7c0095f0 con 0x7ffb80102910 2023-09-13T04:39:48.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.017+0000 7ffb86a02700 1 --2- 172.21.15.2:0/3546543659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb80102910 0x7ffb8019e040 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ffb78009f60 tx=0x7ffb78009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:48.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.018+0000 7ffb777fe700 1 -- 172.21.15.2:0/3546543659 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78018070 con 0x7ffb80102910 2023-09-13T04:39:48.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.018+0000 7ffb777fe700 1 -- 172.21.15.2:0/3546543659 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb7800f040 con 0x7ffb80102910 2023-09-13T04:39:48.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.018+0000 7ffb777fe700 1 -- 172.21.15.2:0/3546543659 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7800ccc0 con 0x7ffb80102910 2023-09-13T04:39:48.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.018+0000 7ffb88465700 1 -- 172.21.15.2:0/3546543659 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8019eb60 con 0x7ffb80102910 2023-09-13T04:39:48.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.018+0000 7ffb88465700 1 -- 172.21.15.2:0/3546543659 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8019f010 con 0x7ffb80102910 2023-09-13T04:39:48.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.019+0000 7ffb88465700 1 -- 172.21.15.2:0/3546543659 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb80108360 con 0x7ffb80102910 2023-09-13T04:39:48.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.021+0000 7ffb777fe700 1 -- 172.21.15.2:0/3546543659 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7ffb78003a00 con 0x7ffb80102910 2023-09-13T04:39:48.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.023+0000 7ffb777fe700 1 --2- 172.21.15.2:0/3546543659 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ffb6c0a06c0 0x7ffb6c0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:48.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.023+0000 7ffb777fe700 1 -- 172.21.15.2:0/3546543659 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ffb780cb950 con 0x7ffb80102910 2023-09-13T04:39:48.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.023+0000 7ffb86201700 1 --2- 172.21.15.2:0/3546543659 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ffb6c0a06c0 0x7ffb6c0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:48.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.024+0000 7ffb86201700 1 --2- 172.21.15.2:0/3546543659 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ffb6c0a06c0 0x7ffb6c0a2b80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009990 tx=0x7ffb7c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:48.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.027+0000 7ffb777fe700 1 -- 172.21.15.2:0/3546543659 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ffb78095180 con 0x7ffb80102910 2023-09-13T04:39:48.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.247+0000 7ffb88465700 1 -- 172.21.15.2:0/3546543659 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb8002d150 con 0x7ffb6c0a06c0 2023-09-13T04:39:48.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.249+0000 7ffb777fe700 1 -- 172.21.15.2:0/3546543659 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7ffb8002d150 con 0x7ffb6c0a06c0 2023-09-13T04:39:48.249 INFO:teuthology.orchestra.run.smithi002.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-09-13T04:39:48.249 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300EV480DGN 447G No 3m ago Has a FileSystem, LVM detected 2023-09-13T04:39:48.250 INFO:teuthology.orchestra.run.smithi002.stdout:smithi149 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CK400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-09-13T04:39:48.250 INFO:teuthology.orchestra.run.smithi002.stdout:smithi156 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB74760040480DGN 447G No 2m ago Has a FileSystem, LVM detected 2023-09-13T04:39:48.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.251+0000 7ffb88465700 1 -- 172.21.15.2:0/3546543659 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ffb6c0a06c0 msgr2=0x7ffb6c0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:48.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.251+0000 7ffb88465700 1 --2- 172.21.15.2:0/3546543659 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ffb6c0a06c0 0x7ffb6c0a2b80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009990 tx=0x7ffb7c005fb0 comp rx=0 tx=0).stop 2023-09-13T04:39:48.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.251+0000 7ffb88465700 1 -- 172.21.15.2:0/3546543659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb80102910 msgr2=0x7ffb8019e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:48.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.251+0000 7ffb88465700 1 --2- 172.21.15.2:0/3546543659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb80102910 0x7ffb8019e040 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ffb78009f60 tx=0x7ffb78009f90 comp rx=0 tx=0).stop 2023-09-13T04:39:48.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.251+0000 7ffb88465700 1 -- 172.21.15.2:0/3546543659 shutdown_connections 2023-09-13T04:39:48.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.251+0000 7ffb88465700 1 --2- 172.21.15.2:0/3546543659 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ffb6c0a06c0 0x7ffb6c0a2b80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:48.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.251+0000 7ffb88465700 1 --2- 172.21.15.2:0/3546543659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb80102910 0x7ffb8019e040 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:48.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.251+0000 7ffb88465700 1 --2- 172.21.15.2:0/3546543659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb80100d70 0x7ffb80196a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:48.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.251+0000 7ffb88465700 1 --2- 172.21.15.2:0/3546543659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb80101f70 0x7ffb80196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:48.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.251+0000 7ffb88465700 1 -- 172.21.15.2:0/3546543659 >> 172.21.15.2:0/3546543659 conn(0x7ffb800fc2d0 msgr2=0x7ffb801054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:48.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.251+0000 7ffb88465700 1 -- 172.21.15.2:0/3546543659 shutdown_connections 2023-09-13T04:39:48.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:48.252+0000 7ffb88465700 1 -- 172.21.15.2:0/3546543659 wait complete. 2023-09-13T04:39:48.692 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-09-13T04:39:48.701 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2023-09-13T04:39:48.702 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- bash -c 'ceph fs dump' 2023-09-13T04:39:48.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:48 smithi149 ceph-mon[133315]: pgmap v334: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:39:48.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:48 smithi156 ceph-mon[133675]: pgmap v334: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:39:49.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:48 smithi002 ceph-mon[97651]: pgmap v334: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:39:49.453 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:39:49.852 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:49 smithi002 ceph-mon[97651]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:39:49.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:49 smithi149 ceph-mon[133315]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:39:49.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:49 smithi156 ceph-mon[133675]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:39:50.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:50 smithi149 ceph-mon[133315]: pgmap v335: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:39:50.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:50 smithi156 ceph-mon[133675]: pgmap v335: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:39:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.954+0000 7f851d4d2700 1 -- 172.21.15.2:0/1886971720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85180ffc70 msgr2=0x7f8518100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.954+0000 7f851d4d2700 1 --2- 172.21.15.2:0/1886971720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85180ffc70 0x7f8518100090 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f85080099c0 tx=0x7f8508009cd0 comp rx=0 tx=0).stop 2023-09-13T04:39:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.954+0000 7f851d4d2700 1 -- 172.21.15.2:0/1886971720 shutdown_connections 2023-09-13T04:39:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.954+0000 7f851d4d2700 1 --2- 172.21.15.2:0/1886971720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8518101810 0x7f85181080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.954+0000 7f851d4d2700 1 --2- 172.21.15.2:0/1886971720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85180ffc70 0x7f8518100090 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.954+0000 7f851d4d2700 1 --2- 172.21.15.2:0/1886971720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8518100e70 0x7f85181012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.954+0000 7f851d4d2700 1 -- 172.21.15.2:0/1886971720 >> 172.21.15.2:0/1886971720 conn(0x7f85180fb1f0 msgr2=0x7f85180fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.954+0000 7f851d4d2700 1 -- 172.21.15.2:0/1886971720 shutdown_connections 2023-09-13T04:39:50.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.954+0000 7f851d4d2700 1 -- 172.21.15.2:0/1886971720 wait complete. 2023-09-13T04:39:50.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.955+0000 7f851d4d2700 1 Processor -- start 2023-09-13T04:39:50.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.955+0000 7f851d4d2700 1 -- start start 2023-09-13T04:39:50.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.956+0000 7f851d4d2700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85180ffc70 0x7f8518198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:50.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.956+0000 7f851d4d2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8518100e70 0x7f85181991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:50.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.956+0000 7f851d4d2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8518101810 0x7f85181a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:50.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.956+0000 7f851d4d2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85181a0800 con 0x7f8518100e70 2023-09-13T04:39:50.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.956+0000 7f851d4d2700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85181a0940 con 0x7f85180ffc70 2023-09-13T04:39:50.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.956+0000 7f851d4d2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85181a0b00 con 0x7f8518101810 2023-09-13T04:39:50.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.956+0000 7f85167fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8518100e70 0x7f85181991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:50.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.956+0000 7f85167fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8518100e70 0x7f85181991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37170/0 (socket says 172.21.15.2:37170) 2023-09-13T04:39:50.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.956+0000 7f85167fc700 1 -- 172.21.15.2:0/1996910886 learned_addr learned my addr 172.21.15.2:0/1996910886 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:50.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.956+0000 7f85177fe700 1 --2- 172.21.15.2:0/1996910886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8518101810 0x7f85181a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:50.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.956+0000 7f8516ffd700 1 --2- 172.21.15.2:0/1996910886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85180ffc70 0x7f8518198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:50.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.956+0000 7f85167fc700 1 -- 172.21.15.2:0/1996910886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8518101810 msgr2=0x7f85181a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:50.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.956+0000 7f85167fc700 1 --2- 172.21.15.2:0/1996910886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8518101810 0x7f85181a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:50.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.956+0000 7f85167fc700 1 -- 172.21.15.2:0/1996910886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85180ffc70 msgr2=0x7f8518198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:50.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.956+0000 7f85167fc700 1 --2- 172.21.15.2:0/1996910886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85180ffc70 0x7f8518198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:50.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.956+0000 7f85167fc700 1 -- 172.21.15.2:0/1996910886 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85080095f0 con 0x7f8518100e70 2023-09-13T04:39:50.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.957+0000 7f85167fc700 1 --2- 172.21.15.2:0/1996910886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8518100e70 0x7f85181991c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f850c00e9a0 tx=0x7f850c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:50.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.957+0000 7f84fffff700 1 -- 172.21.15.2:0/1996910886 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c009790 con 0x7f8518100e70 2023-09-13T04:39:50.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.957+0000 7f84fffff700 1 -- 172.21.15.2:0/1996910886 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f850c010430 con 0x7f8518100e70 2023-09-13T04:39:50.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.957+0000 7f84fffff700 1 -- 172.21.15.2:0/1996910886 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c018680 con 0x7f8518100e70 2023-09-13T04:39:50.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.957+0000 7f851d4d2700 1 -- 172.21.15.2:0/1996910886 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85181a0de0 con 0x7f8518100e70 2023-09-13T04:39:50.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.957+0000 7f851d4d2700 1 -- 172.21.15.2:0/1996910886 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85181a1200 con 0x7f8518100e70 2023-09-13T04:39:50.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.958+0000 7f851d4d2700 1 -- 172.21.15.2:0/1996910886 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f851810b530 con 0x7f8518100e70 2023-09-13T04:39:50.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.960+0000 7f84fffff700 1 -- 172.21.15.2:0/1996910886 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f850c0105a0 con 0x7f8518100e70 2023-09-13T04:39:50.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.963+0000 7f84fffff700 1 --2- 172.21.15.2:0/1996910886 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f85000a06c0 0x7f85000a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:50.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.963+0000 7f84fffff700 1 -- 172.21.15.2:0/1996910886 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f850c014070 con 0x7f8518100e70 2023-09-13T04:39:50.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.963+0000 7f8516ffd700 1 --2- 172.21.15.2:0/1996910886 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f85000a06c0 0x7f85000a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:50.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.963+0000 7f8516ffd700 1 --2- 172.21.15.2:0/1996910886 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f85000a06c0 0x7f85000a2b80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8518199b60 tx=0x7f8508005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:50.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:50.967+0000 7f84fffff700 1 -- 172.21.15.2:0/1996910886 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f850c051e10 con 0x7f8518100e70 2023-09-13T04:39:51.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:50 smithi002 ceph-mon[97651]: pgmap v335: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:39:51.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:51.221+0000 7f851d4d2700 1 -- 172.21.15.2:0/1996910886 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f851810b770 con 0x7f8518100e70 2023-09-13T04:39:51.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:51.222+0000 7f84fffff700 1 -- 172.21.15.2:0/1996910886 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+2933 (secure 0 0 0) 0x7f850c095810 con 0x7f8518100e70 2023-09-13T04:39:51.222 INFO:teuthology.orchestra.run.smithi002.stdout:e23 2023-09-13T04:39:51.223 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-09-13T04:39:51.223 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-13T04:39:51.223 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2023-09-13T04:39:51.223 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:39:51.224 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2023-09-13T04:39:51.224 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2023-09-13T04:39:51.224 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 23 2023-09-13T04:39:51.224 INFO:teuthology.orchestra.run.smithi002.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-09-13T04:39:51.225 INFO:teuthology.orchestra.run.smithi002.stdout:created 2023-09-13T04:37:59.110701+0000 2023-09-13T04:39:51.225 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2023-09-13T04:38:26.610260+0000 2023-09-13T04:39:51.225 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2023-09-13T04:39:51.225 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2023-09-13T04:39:51.225 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 300 2023-09-13T04:39:51.226 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2023-09-13T04:39:51.226 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2023-09-13T04:39:51.226 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2023-09-13T04:39:51.226 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2023-09-13T04:39:51.226 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2023-09-13T04:39:51.227 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 0 2023-09-13T04:39:51.227 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-13T04:39:51.227 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 3 2023-09-13T04:39:51.227 INFO:teuthology.orchestra.run.smithi002.stdout:in 0,1,2 2023-09-13T04:39:51.228 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24566,1=24554,2=24541} 2023-09-13T04:39:51.228 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2023-09-13T04:39:51.228 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2023-09-13T04:39:51.228 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2023-09-13T04:39:51.228 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3,4] 2023-09-13T04:39:51.228 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2023-09-13T04:39:51.229 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data disabled 2023-09-13T04:39:51.229 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2023-09-13T04:39:51.229 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2023-09-13T04:39:51.229 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 0 2023-09-13T04:39:51.230 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.b{0:24566} state up:active seq 3 addr [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] compat {c=[1],r=[1],i=[fff]}] 2023-09-13T04:39:51.230 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.c{0:24523} state up:standby-replay seq 1 addr [v2:172.21.15.156:6836/524563090,v1:172.21.15.156:6837/524563090] compat {c=[1],r=[1],i=[fff]}] 2023-09-13T04:39:51.230 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.a{1:24554} state up:active seq 10 addr [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] compat {c=[1],r=[1],i=[fff]}] 2023-09-13T04:39:51.230 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.h{1:24521} state up:standby-replay seq 1 addr [v2:172.21.15.149:6836/1896833690,v1:172.21.15.149:6837/1896833690] compat {c=[1],r=[1],i=[fff]}] 2023-09-13T04:39:51.230 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.i{2:24541} state up:active seq 12 addr [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] compat {c=[1],r=[1],i=[fff]}] 2023-09-13T04:39:51.231 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.f{2:24511} state up:standby-replay seq 1 addr [v2:172.21.15.156:6834/846579077,v1:172.21.15.156:6835/846579077] compat {c=[1],r=[1],i=[fff]}] 2023-09-13T04:39:51.231 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:39:51.231 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:39:51.231 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2023-09-13T04:39:51.231 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:39:51.232 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.j{-1:14451} state up:standby seq 1 addr [v2:172.21.15.2:6834/2478119463,v1:172.21.15.2:6835/2478119463] compat {c=[1],r=[1],i=[fff]}] 2023-09-13T04:39:51.232 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.g{-1:14514} state up:standby seq 1 addr [v2:172.21.15.2:6838/2469307307,v1:172.21.15.2:6839/2469307307] compat {c=[1],r=[1],i=[fff]}] 2023-09-13T04:39:51.232 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.k{-1:24458} state up:standby seq 1 addr [v2:172.21.15.149:6832/3368344346,v1:172.21.15.149:6833/3368344346] compat {c=[1],r=[1],i=[fff]}] 2023-09-13T04:39:51.232 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.l{-1:24460} state up:standby seq 1 addr [v2:172.21.15.156:6832/4209382763,v1:172.21.15.156:6833/4209382763] compat {c=[1],r=[1],i=[fff]}] 2023-09-13T04:39:51.232 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.d{-1:24479} state up:standby seq 1 addr [v2:172.21.15.2:6836/258142559,v1:172.21.15.2:6837/258142559] compat {c=[1],r=[1],i=[fff]}] 2023-09-13T04:39:51.233 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.e{-1:24491} state up:standby seq 1 addr [v2:172.21.15.149:6834/725547291,v1:172.21.15.149:6835/725547291] compat {c=[1],r=[1],i=[fff]}] 2023-09-13T04:39:51.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:51.224+0000 7f851d4d2700 1 -- 172.21.15.2:0/1996910886 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f85000a06c0 msgr2=0x7f85000a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:51.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:51.224+0000 7f851d4d2700 1 --2- 172.21.15.2:0/1996910886 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f85000a06c0 0x7f85000a2b80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8518199b60 tx=0x7f8508005fb0 comp rx=0 tx=0).stop 2023-09-13T04:39:51.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:51.224+0000 7f851d4d2700 1 -- 172.21.15.2:0/1996910886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8518100e70 msgr2=0x7f85181991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:51.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:51.224+0000 7f851d4d2700 1 --2- 172.21.15.2:0/1996910886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8518100e70 0x7f85181991c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f850c00e9a0 tx=0x7f850c00ecb0 comp rx=0 tx=0).stop 2023-09-13T04:39:51.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:51.224+0000 7f851d4d2700 1 -- 172.21.15.2:0/1996910886 shutdown_connections 2023-09-13T04:39:51.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:51.224+0000 7f851d4d2700 1 --2- 172.21.15.2:0/1996910886 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f85000a06c0 0x7f85000a2b80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:51.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:51.224+0000 7f851d4d2700 1 --2- 172.21.15.2:0/1996910886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8518101810 0x7f85181a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:51.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:51.224+0000 7f851d4d2700 1 --2- 172.21.15.2:0/1996910886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85180ffc70 0x7f8518198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:51.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:51.224+0000 7f851d4d2700 1 --2- 172.21.15.2:0/1996910886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8518100e70 0x7f85181991c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:51.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:51.224+0000 7f851d4d2700 1 -- 172.21.15.2:0/1996910886 >> 172.21.15.2:0/1996910886 conn(0x7f85180fb1f0 msgr2=0x7f8518068860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:51.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:51.224+0000 7f851d4d2700 1 -- 172.21.15.2:0/1996910886 shutdown_connections 2023-09-13T04:39:51.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:51.225+0000 7f851d4d2700 1 -- 172.21.15.2:0/1996910886 wait complete. 2023-09-13T04:39:51.236 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 23 2023-09-13T04:39:51.651 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- bash -c 'ceph osd dump' 2023-09-13T04:39:51.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:51 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1996910886' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-13T04:39:51.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:51 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1996910886' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-13T04:39:52.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:51 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1996910886' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-13T04:39:52.387 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:39:52.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:52 smithi149 ceph-mon[133315]: pgmap v336: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-13T04:39:52.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:52 smithi156 ceph-mon[133675]: pgmap v336: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-13T04:39:52.953 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:52 smithi002 ceph-mon[97651]: pgmap v336: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-13T04:39:53.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.944+0000 7fcb6941d700 1 -- 172.21.15.2:0/401351703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb64100d70 msgr2=0x7fcb64101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:53.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.944+0000 7fcb6941d700 1 --2- 172.21.15.2:0/401351703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb64100d70 0x7fcb64101190 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcb540099c0 tx=0x7fcb54009cd0 comp rx=0 tx=0).stop 2023-09-13T04:39:53.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.944+0000 7fcb6941d700 1 -- 172.21.15.2:0/401351703 shutdown_connections 2023-09-13T04:39:53.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.944+0000 7fcb6941d700 1 --2- 172.21.15.2:0/401351703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb64102910 0x7fcb641091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:53.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.944+0000 7fcb6941d700 1 --2- 172.21.15.2:0/401351703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb64100d70 0x7fcb64101190 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:53.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.944+0000 7fcb6941d700 1 --2- 172.21.15.2:0/401351703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb64101f70 0x7fcb641023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:53.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.944+0000 7fcb6941d700 1 -- 172.21.15.2:0/401351703 >> 172.21.15.2:0/401351703 conn(0x7fcb640fc2d0 msgr2=0x7fcb640fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:53.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.944+0000 7fcb6941d700 1 -- 172.21.15.2:0/401351703 shutdown_connections 2023-09-13T04:39:53.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.944+0000 7fcb6941d700 1 -- 172.21.15.2:0/401351703 wait complete. 2023-09-13T04:39:53.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.945+0000 7fcb6941d700 1 Processor -- start 2023-09-13T04:39:53.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.945+0000 7fcb6941d700 1 -- start start 2023-09-13T04:39:53.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.945+0000 7fcb6941d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb64100d70 0x7fcb64198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:53.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.945+0000 7fcb6941d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb64101f70 0x7fcb641991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:53.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.945+0000 7fcb6941d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb64102910 0x7fcb641a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:53.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.945+0000 7fcb6941d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb641a0810 con 0x7fcb64101f70 2023-09-13T04:39:53.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.945+0000 7fcb6941d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb641a0950 con 0x7fcb64100d70 2023-09-13T04:39:53.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.945+0000 7fcb6941d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb641a0b10 con 0x7fcb64102910 2023-09-13T04:39:53.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.946+0000 7fcb627fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb64101f70 0x7fcb641991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:53.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.946+0000 7fcb637fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb64102910 0x7fcb641a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:53.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.946+0000 7fcb627fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb64101f70 0x7fcb641991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37182/0 (socket says 172.21.15.2:37182) 2023-09-13T04:39:53.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.946+0000 7fcb627fc700 1 -- 172.21.15.2:0/125396696 learned_addr learned my addr 172.21.15.2:0/125396696 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:53.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.946+0000 7fcb62ffd700 1 --2- 172.21.15.2:0/125396696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb64100d70 0x7fcb64198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:53.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.946+0000 7fcb637fe700 1 -- 172.21.15.2:0/125396696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb64100d70 msgr2=0x7fcb64198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:53.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.946+0000 7fcb637fe700 1 --2- 172.21.15.2:0/125396696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb64100d70 0x7fcb64198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:53.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.946+0000 7fcb637fe700 1 -- 172.21.15.2:0/125396696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb64101f70 msgr2=0x7fcb641991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:53.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.946+0000 7fcb637fe700 1 --2- 172.21.15.2:0/125396696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb64101f70 0x7fcb641991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:53.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.946+0000 7fcb637fe700 1 -- 172.21.15.2:0/125396696 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb540095f0 con 0x7fcb64102910 2023-09-13T04:39:53.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.946+0000 7fcb637fe700 1 --2- 172.21.15.2:0/125396696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb64102910 0x7fcb641a0240 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00efc0 tx=0x7fcb5c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:53.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.947+0000 7fcb4bfff700 1 -- 172.21.15.2:0/125396696 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c018070 con 0x7fcb64102910 2023-09-13T04:39:53.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.947+0000 7fcb6941d700 1 -- 172.21.15.2:0/125396696 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb641a0d90 con 0x7fcb64102910 2023-09-13T04:39:53.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.947+0000 7fcb4bfff700 1 -- 172.21.15.2:0/125396696 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb5c00f040 con 0x7fcb64102910 2023-09-13T04:39:53.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.947+0000 7fcb4bfff700 1 -- 172.21.15.2:0/125396696 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c00cd40 con 0x7fcb64102910 2023-09-13T04:39:53.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.947+0000 7fcb6941d700 1 -- 172.21.15.2:0/125396696 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb641a11e0 con 0x7fcb64102910 2023-09-13T04:39:53.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.948+0000 7fcb6941d700 1 -- 172.21.15.2:0/125396696 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb641933b0 con 0x7fcb64102910 2023-09-13T04:39:53.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.950+0000 7fcb4bfff700 1 -- 172.21.15.2:0/125396696 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fcb5c003a00 con 0x7fcb64102910 2023-09-13T04:39:53.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.952+0000 7fcb4bfff700 1 --2- 172.21.15.2:0/125396696 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fcb4c0a0640 0x7fcb4c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:53.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.952+0000 7fcb4bfff700 1 -- 172.21.15.2:0/125396696 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fcb5c0cc4a0 con 0x7fcb64102910 2023-09-13T04:39:53.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.952+0000 7fcb62ffd700 1 --2- 172.21.15.2:0/125396696 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fcb4c0a0640 0x7fcb4c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:53.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.953+0000 7fcb62ffd700 1 --2- 172.21.15.2:0/125396696 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fcb4c0a0640 0x7fcb4c0a2b00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcb64199b70 tx=0x7fcb54005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:53.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:53.954+0000 7fcb4bfff700 1 -- 172.21.15.2:0/125396696 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcb5c095cd0 con 0x7fcb64102910 2023-09-13T04:39:54.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.164+0000 7fcb6941d700 1 -- 172.21.15.2:0/125396696 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fcb6402d070 con 0x7fcb64102910 2023-09-13T04:39:54.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.165+0000 7fcb4bfff700 1 -- 172.21.15.2:0/125396696 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) v1 ==== 56+0+5612 (secure 0 0 0) 0x7fcb5c0992f0 con 0x7fcb64102910 2023-09-13T04:39:54.167 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 92 2023-09-13T04:39:54.167 INFO:teuthology.orchestra.run.smithi002.stdout:fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:39:54.167 INFO:teuthology.orchestra.run.smithi002.stdout:created 2023-09-13T04:30:22.164674+0000 2023-09-13T04:39:54.167 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2023-09-13T04:39:25.580735+0000 2023-09-13T04:39:54.168 INFO:teuthology.orchestra.run.smithi002.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-09-13T04:39:54.168 INFO:teuthology.orchestra.run.smithi002.stdout:crush_version 27 2023-09-13T04:39:54.168 INFO:teuthology.orchestra.run.smithi002.stdout:full_ratio 0.95 2023-09-13T04:39:54.168 INFO:teuthology.orchestra.run.smithi002.stdout:backfillfull_ratio 0.9 2023-09-13T04:39:54.168 INFO:teuthology.orchestra.run.smithi002.stdout:nearfull_ratio 0.85 2023-09-13T04:39:54.169 INFO:teuthology.orchestra.run.smithi002.stdout:require_min_compat_client luminous 2023-09-13T04:39:54.169 INFO:teuthology.orchestra.run.smithi002.stdout:min_compat_client luminous 2023-09-13T04:39:54.169 INFO:teuthology.orchestra.run.smithi002.stdout:require_osd_release reef 2023-09-13T04:39:54.169 INFO:teuthology.orchestra.run.smithi002.stdout:stretch_mode_enabled false 2023-09-13T04:39:54.169 INFO:teuthology.orchestra.run.smithi002.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-09-13T04:39:54.170 INFO:teuthology.orchestra.run.smithi002.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-09-13T04:39:54.170 INFO:teuthology.orchestra.run.smithi002.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-09-13T04:39:54.170 INFO:teuthology.orchestra.run.smithi002.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-09-13T04:39:54.170 INFO:teuthology.orchestra.run.smithi002.stdout:max_osd 12 2023-09-13T04:39:54.170 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 up in weight 1 up_from 8 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6802/1800239506,v1:172.21.15.2:6803/1800239506] [v2:172.21.15.2:6804/1800239506,v1:172.21.15.2:6805/1800239506] exists,up c9a443bc-d0b1-41ea-8409-e0cccb0116ef 2023-09-13T04:39:54.171 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 up in weight 1 up_from 13 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6800/1737673954,v1:172.21.15.149:6801/1737673954] [v2:172.21.15.149:6802/1737673954,v1:172.21.15.149:6803/1737673954] exists,up 620d0f1a-56e1-4edd-afa4-100d96afd043 2023-09-13T04:39:54.171 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 up in weight 1 up_from 18 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6800/3090983787,v1:172.21.15.156:6801/3090983787] [v2:172.21.15.156:6802/3090983787,v1:172.21.15.156:6803/3090983787] exists,up 9ee5587a-dd0b-42b0-ae9a-a717ae6afad9 2023-09-13T04:39:54.171 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 up in weight 1 up_from 25 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6810/3764424070,v1:172.21.15.2:6811/3764424070] [v2:172.21.15.2:6812/3764424070,v1:172.21.15.2:6813/3764424070] exists,up 93397693-f57f-4e1a-a1f8-10e3651e5e75 2023-09-13T04:39:54.171 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 up in weight 1 up_from 33 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6808/3095161596,v1:172.21.15.149:6809/3095161596] [v2:172.21.15.149:6810/3095161596,v1:172.21.15.149:6811/3095161596] exists,up 09ed1cc4-cce9-454b-bced-dbe54d622f97 2023-09-13T04:39:54.171 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 up in weight 1 up_from 38 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6808/1709817635,v1:172.21.15.156:6809/1709817635] [v2:172.21.15.156:6810/1709817635,v1:172.21.15.156:6811/1709817635] exists,up 2ede570f-f574-4d3f-934d-245afd175128 2023-09-13T04:39:54.172 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6 up in weight 1 up_from 46 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6818/3548993414,v1:172.21.15.2:6819/3548993414] [v2:172.21.15.2:6820/3548993414,v1:172.21.15.2:6821/3548993414] exists,up 46e6e453-ced8-44b7-8ec7-e38497f8d4ab 2023-09-13T04:39:54.172 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7 up in weight 1 up_from 51 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6816/3932319846,v1:172.21.15.149:6817/3932319846] [v2:172.21.15.149:6818/3932319846,v1:172.21.15.149:6819/3932319846] exists,up 8d82c256-1078-4815-83d9-6fcdb98746ff 2023-09-13T04:39:54.172 INFO:teuthology.orchestra.run.smithi002.stdout:osd.8 up in weight 1 up_from 58 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6816/1139967383,v1:172.21.15.156:6817/1139967383] [v2:172.21.15.156:6818/1139967383,v1:172.21.15.156:6819/1139967383] exists,up fbb7a301-2a19-4239-a27d-d07410dd84c7 2023-09-13T04:39:54.172 INFO:teuthology.orchestra.run.smithi002.stdout:osd.9 up in weight 1 up_from 66 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6826/3097876180,v1:172.21.15.2:6827/3097876180] [v2:172.21.15.2:6828/3097876180,v1:172.21.15.2:6829/3097876180] exists,up eb50721f-5036-47d4-89f8-0fa09538266d 2023-09-13T04:39:54.172 INFO:teuthology.orchestra.run.smithi002.stdout:osd.10 up in weight 1 up_from 71 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6824/3663849505,v1:172.21.15.149:6825/3663849505] [v2:172.21.15.149:6826/3663849505,v1:172.21.15.149:6827/3663849505] exists,up 23ee78a4-63ef-48b5-9383-046730f45bf6 2023-09-13T04:39:54.173 INFO:teuthology.orchestra.run.smithi002.stdout:osd.11 up in weight 1 up_from 79 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6824/215467467,v1:172.21.15.156:6825/215467467] [v2:172.21.15.156:6826/215467467,v1:172.21.15.156:6827/215467467] exists,up 41b43893-e5cc-422c-9efd-0f25edbf5d2a 2023-09-13T04:39:54.173 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 2.10 [8,11] 2023-09-13T04:39:54.173 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 2.13 [2,11] 2023-09-13T04:39:54.173 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.13 [4,9] 2023-09-13T04:39:54.173 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.1b [5,9] 2023-09-13T04:39:54.173 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.21 [4,8] 2023-09-13T04:39:54.174 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.31 [4,6] 2023-09-13T04:39:54.174 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.38 [3,9] 2023-09-13T04:39:54.174 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 4.2 [9,1] 2023-09-13T04:39:54.174 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 4.5 [5,8] 2023-09-13T04:39:54.174 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 4.13 [9,8] 2023-09-13T04:39:54.175 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 4.18 [9,10] 2023-09-13T04:39:54.175 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:6801/1393377368 expires 2023-09-14T04:31:23.417691+0000 2023-09-13T04:39:54.175 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:6801/2772434518 expires 2023-09-14T04:30:54.963309+0000 2023-09-13T04:39:54.175 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/3810974822 expires 2023-09-14T04:31:23.417691+0000 2023-09-13T04:39:54.175 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:6800/2772434518 expires 2023-09-14T04:30:54.963309+0000 2023-09-13T04:39:54.176 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/1907805835 expires 2023-09-14T04:30:54.963309+0000 2023-09-13T04:39:54.176 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/2811866029 expires 2023-09-14T04:30:54.963309+0000 2023-09-13T04:39:54.176 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/2538614772 expires 2023-09-14T04:31:23.417691+0000 2023-09-13T04:39:54.176 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:6800/1393377368 expires 2023-09-14T04:31:23.417691+0000 2023-09-13T04:39:54.176 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/3722465120 expires 2023-09-14T04:31:23.417691+0000 2023-09-13T04:39:54.177 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/516951274 expires 2023-09-14T04:30:54.963309+0000 2023-09-13T04:39:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.168+0000 7fcb6941d700 1 -- 172.21.15.2:0/125396696 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fcb4c0a0640 msgr2=0x7fcb4c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:54.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.168+0000 7fcb6941d700 1 --2- 172.21.15.2:0/125396696 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fcb4c0a0640 0x7fcb4c0a2b00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcb64199b70 tx=0x7fcb54005fb0 comp rx=0 tx=0).stop 2023-09-13T04:39:54.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.168+0000 7fcb6941d700 1 -- 172.21.15.2:0/125396696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb64102910 msgr2=0x7fcb641a0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:54.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.168+0000 7fcb6941d700 1 --2- 172.21.15.2:0/125396696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb64102910 0x7fcb641a0240 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00efc0 tx=0x7fcb5c00c3a0 comp rx=0 tx=0).stop 2023-09-13T04:39:54.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.168+0000 7fcb6941d700 1 -- 172.21.15.2:0/125396696 shutdown_connections 2023-09-13T04:39:54.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.168+0000 7fcb6941d700 1 --2- 172.21.15.2:0/125396696 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fcb4c0a0640 0x7fcb4c0a2b00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:54.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.168+0000 7fcb6941d700 1 --2- 172.21.15.2:0/125396696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb64102910 0x7fcb641a0240 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:54.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.168+0000 7fcb6941d700 1 --2- 172.21.15.2:0/125396696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb64100d70 0x7fcb64198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:54.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.168+0000 7fcb6941d700 1 --2- 172.21.15.2:0/125396696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb64101f70 0x7fcb641991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:54.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.168+0000 7fcb6941d700 1 -- 172.21.15.2:0/125396696 >> 172.21.15.2:0/125396696 conn(0x7fcb640fc2d0 msgr2=0x7fcb641078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:54.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.168+0000 7fcb6941d700 1 -- 172.21.15.2:0/125396696 shutdown_connections 2023-09-13T04:39:54.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.168+0000 7fcb6941d700 1 -- 172.21.15.2:0/125396696 wait complete. 2023-09-13T04:39:54.684 INFO:teuthology.run_tasks:Running task fs.ready... 2023-09-13T04:39:54.696 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T04:39:54.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:54 smithi149 ceph-mon[133315]: pgmap v337: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:39:54.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:54 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/125396696' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-13T04:39:54.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.904+0000 7fb7211ff700 1 -- 172.21.15.2:0/614555987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb71c101ee0 msgr2=0x7fb71c102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:54.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.904+0000 7fb7211ff700 1 --2- 172.21.15.2:0/614555987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb71c101ee0 0x7fb71c102340 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb7100099c0 tx=0x7fb710009cd0 comp rx=0 tx=0).stop 2023-09-13T04:39:54.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.904+0000 7fb7211ff700 1 -- 172.21.15.2:0/614555987 shutdown_connections 2023-09-13T04:39:54.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.904+0000 7fb7211ff700 1 --2- 172.21.15.2:0/614555987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb71c102880 0x7fb71c109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:54.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.904+0000 7fb7211ff700 1 --2- 172.21.15.2:0/614555987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb71c100cf0 0x7fb71c101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:54.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.904+0000 7fb7211ff700 1 --2- 172.21.15.2:0/614555987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb71c101ee0 0x7fb71c102340 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:54.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.904+0000 7fb7211ff700 1 -- 172.21.15.2:0/614555987 >> 172.21.15.2:0/614555987 conn(0x7fb71c0fc220 msgr2=0x7fb71c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:54.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.904+0000 7fb7211ff700 1 -- 172.21.15.2:0/614555987 shutdown_connections 2023-09-13T04:39:54.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.904+0000 7fb7211ff700 1 -- 172.21.15.2:0/614555987 wait complete. 2023-09-13T04:39:54.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.905+0000 7fb7211ff700 1 Processor -- start 2023-09-13T04:39:54.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.905+0000 7fb7211ff700 1 -- start start 2023-09-13T04:39:54.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.905+0000 7fb7211ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb71c100cf0 0x7fb71c1969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:54.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.905+0000 7fb7211ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb71c101ee0 0x7fb71c196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:54.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.906+0000 7fb7211ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb71c102880 0x7fb71c19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:54.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.906+0000 7fb7211ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c19e560 con 0x7fb71c100cf0 2023-09-13T04:39:54.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.906+0000 7fb7211ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c19e6a0 con 0x7fb71c101ee0 2023-09-13T04:39:54.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.906+0000 7fb7211ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c19e830 con 0x7fb71c102880 2023-09-13T04:39:54.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.906+0000 7fb71ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb71c100cf0 0x7fb71c1969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:54.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.906+0000 7fb71ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb71c100cf0 0x7fb71c1969e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57878/0 (socket says 172.21.15.2:57878) 2023-09-13T04:39:54.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.906+0000 7fb71ad9d700 1 -- 172.21.15.2:0/113410787 learned_addr learned my addr 172.21.15.2:0/113410787 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:54.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.906+0000 7fb71b59e700 1 --2- 172.21.15.2:0/113410787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb71c102880 0x7fb71c19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:54.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.906+0000 7fb71a59c700 1 --2- 172.21.15.2:0/113410787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb71c101ee0 0x7fb71c196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:54.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.906+0000 7fb71ad9d700 1 -- 172.21.15.2:0/113410787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb71c102880 msgr2=0x7fb71c19df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:54.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.906+0000 7fb71ad9d700 1 --2- 172.21.15.2:0/113410787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb71c102880 0x7fb71c19df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:54.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.906+0000 7fb71ad9d700 1 -- 172.21.15.2:0/113410787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb71c101ee0 msgr2=0x7fb71c196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:54.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.906+0000 7fb71ad9d700 1 --2- 172.21.15.2:0/113410787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb71c101ee0 0x7fb71c196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:54.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.906+0000 7fb71ad9d700 1 -- 172.21.15.2:0/113410787 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7100095f0 con 0x7fb71c100cf0 2023-09-13T04:39:54.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.906+0000 7fb71ad9d700 1 --2- 172.21.15.2:0/113410787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb71c100cf0 0x7fb71c1969e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb70c00e970 tx=0x7fb70c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:54.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.907+0000 7fb703fff700 1 -- 172.21.15.2:0/113410787 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70c009790 con 0x7fb71c100cf0 2023-09-13T04:39:54.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.907+0000 7fb703fff700 1 -- 172.21.15.2:0/113410787 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb70c00c7e0 con 0x7fb71c100cf0 2023-09-13T04:39:54.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.907+0000 7fb7211ff700 1 -- 172.21.15.2:0/113410787 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb71c19eb10 con 0x7fb71c100cf0 2023-09-13T04:39:54.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.907+0000 7fb703fff700 1 -- 172.21.15.2:0/113410787 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70c003c70 con 0x7fb71c100cf0 2023-09-13T04:39:54.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.907+0000 7fb7211ff700 1 -- 172.21.15.2:0/113410787 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb71c19f020 con 0x7fb71c100cf0 2023-09-13T04:39:54.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.908+0000 7fb7211ff700 1 -- 172.21.15.2:0/113410787 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb71c107c20 con 0x7fb71c100cf0 2023-09-13T04:39:54.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.910+0000 7fb703fff700 1 -- 172.21.15.2:0/113410787 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fb70c00cc50 con 0x7fb71c100cf0 2023-09-13T04:39:54.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.912+0000 7fb703fff700 1 --2- 172.21.15.2:0/113410787 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb7040a0690 0x7fb7040a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:54.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.913+0000 7fb703fff700 1 -- 172.21.15.2:0/113410787 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb70c014070 con 0x7fb71c100cf0 2023-09-13T04:39:54.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.913+0000 7fb71a59c700 1 --2- 172.21.15.2:0/113410787 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb7040a0690 0x7fb7040a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:54.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.913+0000 7fb71a59c700 1 --2- 172.21.15.2:0/113410787 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb7040a0690 0x7fb7040a2b50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb71000b580 tx=0x7fb710005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:54.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:54.917+0000 7fb703fff700 1 -- 172.21.15.2:0/113410787 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb70c095880 con 0x7fb71c100cf0 2023-09-13T04:39:54.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:54 smithi156 ceph-mon[133675]: pgmap v337: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:39:54.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:54 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/125396696' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-13T04:39:55.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:54 smithi002 ceph-mon[97651]: pgmap v337: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:39:55.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:54 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/125396696' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-13T04:39:55.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.130+0000 7fb7211ff700 1 -- 172.21.15.2:0/113410787 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb71c10c550 con 0x7fb71c100cf0 2023-09-13T04:39:55.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.132+0000 7fb703fff700 1 -- 172.21.15.2:0/113410787 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24503 (secure 0 0 0) 0x7fb70c003dd0 con 0x7fb71c100cf0 2023-09-13T04:39:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:39:55.134 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":92,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:39:25.580735+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T04:37:57.094802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T04:37:58.564487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-13T04:38:02.096895+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:39:55.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.135+0000 7fb7211ff700 1 -- 172.21.15.2:0/113410787 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb7040a0690 msgr2=0x7fb7040a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:55.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.135+0000 7fb7211ff700 1 --2- 172.21.15.2:0/113410787 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb7040a0690 0x7fb7040a2b50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb71000b580 tx=0x7fb710005fb0 comp rx=0 tx=0).stop 2023-09-13T04:39:55.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.135+0000 7fb7211ff700 1 -- 172.21.15.2:0/113410787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb71c100cf0 msgr2=0x7fb71c1969e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:55.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.135+0000 7fb7211ff700 1 --2- 172.21.15.2:0/113410787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb71c100cf0 0x7fb71c1969e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb70c00e970 tx=0x7fb70c00ec80 comp rx=0 tx=0).stop 2023-09-13T04:39:55.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.135+0000 7fb7211ff700 1 -- 172.21.15.2:0/113410787 shutdown_connections 2023-09-13T04:39:55.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.135+0000 7fb7211ff700 1 --2- 172.21.15.2:0/113410787 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb7040a0690 0x7fb7040a2b50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:55.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.135+0000 7fb7211ff700 1 --2- 172.21.15.2:0/113410787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb71c102880 0x7fb71c19df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:55.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.135+0000 7fb7211ff700 1 --2- 172.21.15.2:0/113410787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb71c100cf0 0x7fb71c1969e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:55.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.135+0000 7fb7211ff700 1 --2- 172.21.15.2:0/113410787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb71c101ee0 0x7fb71c196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:55.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.135+0000 7fb7211ff700 1 -- 172.21.15.2:0/113410787 >> 172.21.15.2:0/113410787 conn(0x7fb71c0fc220 msgr2=0x7fb71c0fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:55.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.135+0000 7fb7211ff700 1 -- 172.21.15.2:0/113410787 shutdown_connections 2023-09-13T04:39:55.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.135+0000 7fb7211ff700 1 -- 172.21.15.2:0/113410787 wait complete. 2023-09-13T04:39:55.156 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T04:33:42.025714+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-13T04:37:57.094802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-13T04:37:58.564487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-13T04:38:02.096895+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-13T04:39:55.156 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-13T04:39:55.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.363+0000 7fb6877ff700 1 -- 172.21.15.2:0/3509457461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb680100cf0 msgr2=0x7fb680101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:55.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.363+0000 7fb6877ff700 1 --2- 172.21.15.2:0/3509457461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb680100cf0 0x7fb680101110 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb67c0099c0 tx=0x7fb67c009cd0 comp rx=0 tx=0).stop 2023-09-13T04:39:55.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.363+0000 7fb6877ff700 1 -- 172.21.15.2:0/3509457461 shutdown_connections 2023-09-13T04:39:55.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.363+0000 7fb6877ff700 1 --2- 172.21.15.2:0/3509457461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb680102880 0x7fb680109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:55.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.363+0000 7fb6877ff700 1 --2- 172.21.15.2:0/3509457461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb680100cf0 0x7fb680101110 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:55.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.363+0000 7fb6877ff700 1 --2- 172.21.15.2:0/3509457461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb680101ee0 0x7fb680102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:55.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.363+0000 7fb6877ff700 1 -- 172.21.15.2:0/3509457461 >> 172.21.15.2:0/3509457461 conn(0x7fb6800fc260 msgr2=0x7fb6800fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:55.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.363+0000 7fb6877ff700 1 -- 172.21.15.2:0/3509457461 shutdown_connections 2023-09-13T04:39:55.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.363+0000 7fb6877ff700 1 -- 172.21.15.2:0/3509457461 wait complete. 2023-09-13T04:39:55.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.364+0000 7fb6877ff700 1 Processor -- start 2023-09-13T04:39:55.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.364+0000 7fb6877ff700 1 -- start start 2023-09-13T04:39:55.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.364+0000 7fb6877ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb680100cf0 0x7fb680196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:55.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.364+0000 7fb6877ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb680101ee0 0x7fb680196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.364+0000 7fb6877ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb680102880 0x7fb68019dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.364+0000 7fb6877ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68019e590 con 0x7fb680102880 2023-09-13T04:39:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.364+0000 7fb6877ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68019e6d0 con 0x7fb680100cf0 2023-09-13T04:39:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.364+0000 7fb6877ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68019e860 con 0x7fb680101ee0 2023-09-13T04:39:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.365+0000 7fb685d9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb680102880 0x7fb68019dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:55.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.365+0000 7fb685d9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb680102880 0x7fb68019dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57896/0 (socket says 172.21.15.2:57896) 2023-09-13T04:39:55.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.365+0000 7fb684d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb680101ee0 0x7fb680196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:55.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.365+0000 7fb685d9c700 1 -- 172.21.15.2:0/3277590415 learned_addr learned my addr 172.21.15.2:0/3277590415 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:55.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.365+0000 7fb68559b700 1 --2- 172.21.15.2:0/3277590415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb680100cf0 0x7fb680196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:55.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.365+0000 7fb685d9c700 1 -- 172.21.15.2:0/3277590415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb680101ee0 msgr2=0x7fb680196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:55.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.365+0000 7fb685d9c700 1 --2- 172.21.15.2:0/3277590415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb680101ee0 0x7fb680196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:55.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.365+0000 7fb685d9c700 1 -- 172.21.15.2:0/3277590415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb680100cf0 msgr2=0x7fb680196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:55.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.365+0000 7fb685d9c700 1 --2- 172.21.15.2:0/3277590415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb680100cf0 0x7fb680196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:55.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.365+0000 7fb685d9c700 1 -- 172.21.15.2:0/3277590415 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67c0095f0 con 0x7fb680102880 2023-09-13T04:39:55.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.365+0000 7fb684d9a700 1 --2- 172.21.15.2:0/3277590415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb680101ee0 0x7fb680196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:39:55.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.365+0000 7fb685d9c700 1 --2- 172.21.15.2:0/3277590415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb680102880 0x7fb68019dfc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb678007ee0 tx=0x7fb67800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:55.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.365+0000 7fb6767fc700 1 -- 172.21.15.2:0/3277590415 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67800dd20 con 0x7fb680102880 2023-09-13T04:39:55.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.366+0000 7fb6767fc700 1 -- 172.21.15.2:0/3277590415 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb67800f040 con 0x7fb680102880 2023-09-13T04:39:55.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.366+0000 7fb6877ff700 1 -- 172.21.15.2:0/3277590415 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb68019eb40 con 0x7fb680102880 2023-09-13T04:39:55.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.366+0000 7fb6767fc700 1 -- 172.21.15.2:0/3277590415 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6780048a0 con 0x7fb680102880 2023-09-13T04:39:55.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.366+0000 7fb6877ff700 1 -- 172.21.15.2:0/3277590415 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb68019f050 con 0x7fb680102880 2023-09-13T04:39:55.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.367+0000 7fb6877ff700 1 -- 172.21.15.2:0/3277590415 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb680107c70 con 0x7fb680102880 2023-09-13T04:39:55.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.369+0000 7fb6767fc700 1 -- 172.21.15.2:0/3277590415 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fb678003a00 con 0x7fb680102880 2023-09-13T04:39:55.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.370+0000 7fb6767fc700 1 --2- 172.21.15.2:0/3277590415 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb66c0a06c0 0x7fb66c0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:55.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.370+0000 7fb6767fc700 1 -- 172.21.15.2:0/3277590415 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb6780cb710 con 0x7fb680102880 2023-09-13T04:39:55.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.370+0000 7fb68559b700 1 --2- 172.21.15.2:0/3277590415 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb66c0a06c0 0x7fb66c0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:55.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.371+0000 7fb68559b700 1 --2- 172.21.15.2:0/3277590415 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb66c0a06c0 0x7fb66c0a2b80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb67c009990 tx=0x7fb67c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:55.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.374+0000 7fb6767fc700 1 -- 172.21.15.2:0/3277590415 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb67804ee10 con 0x7fb680102880 2023-09-13T04:39:55.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.584+0000 7fb6877ff700 1 -- 172.21.15.2:0/3277590415 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb68002d070 con 0x7fb680102880 2023-09-13T04:39:55.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.585+0000 7fb6767fc700 1 -- 172.21.15.2:0/3277590415 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb678094eb0 con 0x7fb680102880 2023-09-13T04:39:55.586 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-09-13T04:39:55.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.587+0000 7fb6877ff700 1 -- 172.21.15.2:0/3277590415 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb66c0a06c0 msgr2=0x7fb66c0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:55.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.587+0000 7fb6877ff700 1 --2- 172.21.15.2:0/3277590415 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb66c0a06c0 0x7fb66c0a2b80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb67c009990 tx=0x7fb67c005fb0 comp rx=0 tx=0).stop 2023-09-13T04:39:55.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.587+0000 7fb6877ff700 1 -- 172.21.15.2:0/3277590415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb680102880 msgr2=0x7fb68019dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:55.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.587+0000 7fb6877ff700 1 --2- 172.21.15.2:0/3277590415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb680102880 0x7fb68019dfc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb678007ee0 tx=0x7fb67800d3b0 comp rx=0 tx=0).stop 2023-09-13T04:39:55.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.587+0000 7fb6877ff700 1 -- 172.21.15.2:0/3277590415 shutdown_connections 2023-09-13T04:39:55.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.587+0000 7fb6877ff700 1 --2- 172.21.15.2:0/3277590415 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fb66c0a06c0 0x7fb66c0a2b80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:55.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.587+0000 7fb6877ff700 1 --2- 172.21.15.2:0/3277590415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb680102880 0x7fb68019dfc0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:55.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.587+0000 7fb6877ff700 1 --2- 172.21.15.2:0/3277590415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb680100cf0 0x7fb680196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:55.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.587+0000 7fb6877ff700 1 --2- 172.21.15.2:0/3277590415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb680101ee0 0x7fb680196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:55.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.587+0000 7fb6877ff700 1 -- 172.21.15.2:0/3277590415 >> 172.21.15.2:0/3277590415 conn(0x7fb6800fc260 msgr2=0x7fb6800fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:55.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.587+0000 7fb6877ff700 1 -- 172.21.15.2:0/3277590415 shutdown_connections 2023-09-13T04:39:55.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.587+0000 7fb6877ff700 1 -- 172.21.15.2:0/3277590415 wait complete. 2023-09-13T04:39:55.608 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-13T04:39:55.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.817+0000 7f19459ff700 1 -- 172.21.15.2:0/2684068399 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19401069e0 msgr2=0x7f1940108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:55.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.817+0000 7f19459ff700 1 --2- 172.21.15.2:0/2684068399 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19401069e0 0x7f1940108e50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f193800b0a0 tx=0x7f193800b3b0 comp rx=0 tx=0).stop 2023-09-13T04:39:55.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.817+0000 7f19459ff700 1 -- 172.21.15.2:0/2684068399 shutdown_connections 2023-09-13T04:39:55.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.817+0000 7f19459ff700 1 --2- 172.21.15.2:0/2684068399 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19401069e0 0x7f1940108e50 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:55.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.817+0000 7f19459ff700 1 --2- 172.21.15.2:0/2684068399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1940101780 0x7f1940103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:55.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.817+0000 7f19459ff700 1 --2- 172.21.15.2:0/2684068399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19401040b0 0x7f19401064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:55.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.817+0000 7f19459ff700 1 -- 172.21.15.2:0/2684068399 >> 172.21.15.2:0/2684068399 conn(0x7f19400fb170 msgr2=0x7f19400fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:55.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.817+0000 7f19459ff700 1 -- 172.21.15.2:0/2684068399 shutdown_connections 2023-09-13T04:39:55.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.817+0000 7f19459ff700 1 -- 172.21.15.2:0/2684068399 wait complete. 2023-09-13T04:39:55.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.818+0000 7f19459ff700 1 Processor -- start 2023-09-13T04:39:55.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.819+0000 7f19459ff700 1 -- start start 2023-09-13T04:39:55.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.819+0000 7f19459ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1940101780 0x7f1940196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:55.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.819+0000 7f19459ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19401040b0 0x7f1940196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:55.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.819+0000 7f19459ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19401069e0 0x7f194019e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:55.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.819+0000 7f19459ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194019e5d0 con 0x7f19401040b0 2023-09-13T04:39:55.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.819+0000 7f19459ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194019e710 con 0x7f1940101780 2023-09-13T04:39:55.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.819+0000 7f19459ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194019e8a0 con 0x7f19401069e0 2023-09-13T04:39:55.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.819+0000 7f1937fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19401040b0 0x7f1940196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:55.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.819+0000 7f1937fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19401040b0 0x7f1940196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57910/0 (socket says 172.21.15.2:57910) 2023-09-13T04:39:55.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.819+0000 7f1937fff700 1 -- 172.21.15.2:0/184301702 learned_addr learned my addr 172.21.15.2:0/184301702 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:55.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.819+0000 7f193effd700 1 --2- 172.21.15.2:0/184301702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1940101780 0x7f1940196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:55.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.819+0000 7f193f7fe700 1 --2- 172.21.15.2:0/184301702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19401069e0 0x7f194019e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:55.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.820+0000 7f1937fff700 1 -- 172.21.15.2:0/184301702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19401069e0 msgr2=0x7f194019e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:55.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.820+0000 7f1937fff700 1 --2- 172.21.15.2:0/184301702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19401069e0 0x7f194019e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:55.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.820+0000 7f1937fff700 1 -- 172.21.15.2:0/184301702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1940101780 msgr2=0x7f1940196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:55.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.820+0000 7f1937fff700 1 --2- 172.21.15.2:0/184301702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1940101780 0x7f1940196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:55.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.820+0000 7f1937fff700 1 -- 172.21.15.2:0/184301702 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1928009520 con 0x7f19401040b0 2023-09-13T04:39:55.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.820+0000 7f1937fff700 1 --2- 172.21.15.2:0/184301702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19401040b0 0x7f1940196f90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f192800eb30 tx=0x7f192800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:55.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.820+0000 7f19449fd700 1 -- 172.21.15.2:0/184301702 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1928009880 con 0x7f19401040b0 2023-09-13T04:39:55.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.820+0000 7f19449fd700 1 -- 172.21.15.2:0/184301702 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19280044e0 con 0x7f19401040b0 2023-09-13T04:39:55.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.820+0000 7f19459ff700 1 -- 172.21.15.2:0/184301702 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1938009d00 con 0x7f19401040b0 2023-09-13T04:39:55.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.820+0000 7f19449fd700 1 -- 172.21.15.2:0/184301702 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1928003d80 con 0x7f19401040b0 2023-09-13T04:39:55.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.820+0000 7f19459ff700 1 -- 172.21.15.2:0/184301702 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f194019ef00 con 0x7f19401040b0 2023-09-13T04:39:55.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.821+0000 7f19459ff700 1 -- 172.21.15.2:0/184301702 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19400fccf0 con 0x7f19401040b0 2023-09-13T04:39:55.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.823+0000 7f19449fd700 1 -- 172.21.15.2:0/184301702 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f1928003f00 con 0x7f19401040b0 2023-09-13T04:39:55.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.825+0000 7f19449fd700 1 --2- 172.21.15.2:0/184301702 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f192c0a0640 0x7f192c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:55.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.825+0000 7f19449fd700 1 -- 172.21.15.2:0/184301702 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1928014070 con 0x7f19401040b0 2023-09-13T04:39:55.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.826+0000 7f193effd700 1 --2- 172.21.15.2:0/184301702 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f192c0a0640 0x7f192c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:55.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.826+0000 7f193effd700 1 --2- 172.21.15.2:0/184301702 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f192c0a0640 0x7f192c0a2b00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1930009c60 tx=0x7f1930009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:55.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:55.827+0000 7f19449fd700 1 -- 172.21.15.2:0/184301702 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1928095d50 con 0x7f19401040b0 2023-09-13T04:39:55.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:55 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/113410787' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:39:55.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:55 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3277590415' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:39:55.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:55 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/113410787' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:39:55.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:55 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3277590415' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:39:56.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:55 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/113410787' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:39:56.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:55 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3277590415' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:39:56.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.035+0000 7f19459ff700 1 -- 172.21.15.2:0/184301702 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f194010c670 con 0x7f19401040b0 2023-09-13T04:39:56.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.036+0000 7f19449fd700 1 -- 172.21.15.2:0/184301702 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f192800fe60 con 0x7f19401040b0 2023-09-13T04:39:56.038 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-09-13T04:39:56.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.038+0000 7f19459ff700 1 -- 172.21.15.2:0/184301702 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f192c0a0640 msgr2=0x7f192c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:56.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.038+0000 7f19459ff700 1 --2- 172.21.15.2:0/184301702 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f192c0a0640 0x7f192c0a2b00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1930009c60 tx=0x7f1930009260 comp rx=0 tx=0).stop 2023-09-13T04:39:56.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.038+0000 7f19459ff700 1 -- 172.21.15.2:0/184301702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19401040b0 msgr2=0x7f1940196f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:56.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.038+0000 7f19459ff700 1 --2- 172.21.15.2:0/184301702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19401040b0 0x7f1940196f90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f192800eb30 tx=0x7f192800ee40 comp rx=0 tx=0).stop 2023-09-13T04:39:56.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.038+0000 7f19459ff700 1 -- 172.21.15.2:0/184301702 shutdown_connections 2023-09-13T04:39:56.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.038+0000 7f19459ff700 1 --2- 172.21.15.2:0/184301702 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f192c0a0640 0x7f192c0a2b00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.038+0000 7f19459ff700 1 --2- 172.21.15.2:0/184301702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19401069e0 0x7f194019e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.038+0000 7f19459ff700 1 --2- 172.21.15.2:0/184301702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1940101780 0x7f1940196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.038+0000 7f19459ff700 1 --2- 172.21.15.2:0/184301702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19401040b0 0x7f1940196f90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.038+0000 7f19459ff700 1 -- 172.21.15.2:0/184301702 >> 172.21.15.2:0/184301702 conn(0x7f19400fb170 msgr2=0x7f1940102410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:56.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.038+0000 7f19459ff700 1 -- 172.21.15.2:0/184301702 shutdown_connections 2023-09-13T04:39:56.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.038+0000 7f19459ff700 1 -- 172.21.15.2:0/184301702 wait complete. 2023-09-13T04:39:56.058 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-13T04:39:56.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.265+0000 7f96737ff700 1 -- 172.21.15.2:0/646361034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f966c101ed0 msgr2=0x7f966c102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:56.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.265+0000 7f96737ff700 1 --2- 172.21.15.2:0/646361034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f966c101ed0 0x7f966c102330 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f965c0099c0 tx=0x7f965c009cd0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.265+0000 7f96737ff700 1 -- 172.21.15.2:0/646361034 shutdown_connections 2023-09-13T04:39:56.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.265+0000 7f96737ff700 1 --2- 172.21.15.2:0/646361034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f966c102870 0x7f966c109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.265+0000 7f96737ff700 1 --2- 172.21.15.2:0/646361034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f966c100ce0 0x7f966c101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.265+0000 7f96737ff700 1 --2- 172.21.15.2:0/646361034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f966c101ed0 0x7f966c102330 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.265+0000 7f96737ff700 1 -- 172.21.15.2:0/646361034 >> 172.21.15.2:0/646361034 conn(0x7f966c0fc230 msgr2=0x7f966c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:56.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.265+0000 7f96737ff700 1 -- 172.21.15.2:0/646361034 shutdown_connections 2023-09-13T04:39:56.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.265+0000 7f96737ff700 1 -- 172.21.15.2:0/646361034 wait complete. 2023-09-13T04:39:56.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.266+0000 7f96737ff700 1 Processor -- start 2023-09-13T04:39:56.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.266+0000 7f96737ff700 1 -- start start 2023-09-13T04:39:56.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.266+0000 7f96737ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f966c100ce0 0x7f966c196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:56.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.266+0000 7f96737ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f966c101ed0 0x7f966c196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:56.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.266+0000 7f96737ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f966c102870 0x7f966c19dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:56.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.266+0000 7f96737ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966c19e5a0 con 0x7f966c100ce0 2023-09-13T04:39:56.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.266+0000 7f96737ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966c19e6e0 con 0x7f966c101ed0 2023-09-13T04:39:56.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.266+0000 7f96737ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966c19e870 con 0x7f966c102870 2023-09-13T04:39:56.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.267+0000 7f967159b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f966c100ce0 0x7f966c196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:56.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.267+0000 7f967159b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f966c100ce0 0x7f966c196a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57926/0 (socket says 172.21.15.2:57926) 2023-09-13T04:39:56.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.267+0000 7f9670d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f966c101ed0 0x7f966c196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:56.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.267+0000 7f9671d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f966c102870 0x7f966c19dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:56.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.267+0000 7f967159b700 1 -- 172.21.15.2:0/1131757470 learned_addr learned my addr 172.21.15.2:0/1131757470 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:56.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.267+0000 7f967159b700 1 -- 172.21.15.2:0/1131757470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f966c102870 msgr2=0x7f966c19dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:56.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.267+0000 7f967159b700 1 --2- 172.21.15.2:0/1131757470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f966c102870 0x7f966c19dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.267+0000 7f967159b700 1 -- 172.21.15.2:0/1131757470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f966c101ed0 msgr2=0x7f966c196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:56.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.267+0000 7f967159b700 1 --2- 172.21.15.2:0/1131757470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f966c101ed0 0x7f966c196f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.267+0000 7f967159b700 1 -- 172.21.15.2:0/1131757470 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9664009c40 con 0x7f966c100ce0 2023-09-13T04:39:56.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.267+0000 7f9670d9a700 1 --2- 172.21.15.2:0/1131757470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f966c101ed0 0x7f966c196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:39:56.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.268+0000 7f967159b700 1 --2- 172.21.15.2:0/1131757470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f966c100ce0 0x7f966c196a20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f966800e970 tx=0x7f966800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:56.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.268+0000 7f96627fc700 1 -- 172.21.15.2:0/1131757470 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9668009790 con 0x7f966c100ce0 2023-09-13T04:39:56.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.268+0000 7f96627fc700 1 -- 172.21.15.2:0/1131757470 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f966800c7c0 con 0x7f966c100ce0 2023-09-13T04:39:56.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.268+0000 7f96737ff700 1 -- 172.21.15.2:0/1131757470 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f965c0095f0 con 0x7f966c100ce0 2023-09-13T04:39:56.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.268+0000 7f96627fc700 1 -- 172.21.15.2:0/1131757470 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9668003c20 con 0x7f966c100ce0 2023-09-13T04:39:56.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.268+0000 7f96737ff700 1 -- 172.21.15.2:0/1131757470 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f966c19eed0 con 0x7f966c100ce0 2023-09-13T04:39:56.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.269+0000 7f96737ff700 1 -- 172.21.15.2:0/1131757470 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f966c10c540 con 0x7f966c100ce0 2023-09-13T04:39:56.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.271+0000 7f96627fc700 1 -- 172.21.15.2:0/1131757470 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f966800c930 con 0x7f966c100ce0 2023-09-13T04:39:56.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.273+0000 7f96627fc700 1 --2- 172.21.15.2:0/1131757470 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f96580a0640 0x7f96580a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:56.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.274+0000 7f96627fc700 1 -- 172.21.15.2:0/1131757470 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9668014070 con 0x7f966c100ce0 2023-09-13T04:39:56.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.274+0000 7f9670d9a700 1 --2- 172.21.15.2:0/1131757470 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f96580a0640 0x7f96580a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:56.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.274+0000 7f9670d9a700 1 --2- 172.21.15.2:0/1131757470 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f96580a0640 0x7f96580a2b00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f965c00b580 tx=0x7f965c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:56.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.277+0000 7f96627fc700 1 -- 172.21.15.2:0/1131757470 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9668095870 con 0x7f966c100ce0 2023-09-13T04:39:56.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.490+0000 7f96737ff700 1 -- 172.21.15.2:0/1131757470 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f966c107c10 con 0x7f966c100ce0 2023-09-13T04:39:56.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.490+0000 7f96627fc700 1 -- 172.21.15.2:0/1131757470 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9668098e90 con 0x7f966c100ce0 2023-09-13T04:39:56.492 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-09-13T04:39:56.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.492+0000 7f96737ff700 1 -- 172.21.15.2:0/1131757470 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f96580a0640 msgr2=0x7f96580a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:56.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.492+0000 7f96737ff700 1 --2- 172.21.15.2:0/1131757470 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f96580a0640 0x7f96580a2b00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f965c00b580 tx=0x7f965c005fb0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.492+0000 7f96737ff700 1 -- 172.21.15.2:0/1131757470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f966c100ce0 msgr2=0x7f966c196a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:56.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.492+0000 7f96737ff700 1 --2- 172.21.15.2:0/1131757470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f966c100ce0 0x7f966c196a20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f966800e970 tx=0x7f966800ec80 comp rx=0 tx=0).stop 2023-09-13T04:39:56.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.493+0000 7f96737ff700 1 -- 172.21.15.2:0/1131757470 shutdown_connections 2023-09-13T04:39:56.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.493+0000 7f96737ff700 1 --2- 172.21.15.2:0/1131757470 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f96580a0640 0x7f96580a2b00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.493+0000 7f96737ff700 1 --2- 172.21.15.2:0/1131757470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f966c102870 0x7f966c19dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.493+0000 7f96737ff700 1 --2- 172.21.15.2:0/1131757470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f966c100ce0 0x7f966c196a20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.493+0000 7f96737ff700 1 --2- 172.21.15.2:0/1131757470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f966c101ed0 0x7f966c196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.493+0000 7f96737ff700 1 -- 172.21.15.2:0/1131757470 >> 172.21.15.2:0/1131757470 conn(0x7f966c0fc230 msgr2=0x7f966c0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:56.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.493+0000 7f96737ff700 1 -- 172.21.15.2:0/1131757470 shutdown_connections 2023-09-13T04:39:56.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.493+0000 7f96737ff700 1 -- 172.21.15.2:0/1131757470 wait complete. 2023-09-13T04:39:56.513 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-13T04:39:56.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.720+0000 7f02e85ff700 1 -- 172.21.15.2:0/1074724626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02e0101ed0 msgr2=0x7f02e0102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:56.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.720+0000 7f02e85ff700 1 --2- 172.21.15.2:0/1074724626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02e0101ed0 0x7f02e0102330 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f02d00099c0 tx=0x7f02d0009cd0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.720+0000 7f02e85ff700 1 -- 172.21.15.2:0/1074724626 shutdown_connections 2023-09-13T04:39:56.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.720+0000 7f02e85ff700 1 --2- 172.21.15.2:0/1074724626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02e0102870 0x7f02e0109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.720+0000 7f02e85ff700 1 --2- 172.21.15.2:0/1074724626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02e0100ce0 0x7f02e0101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.720+0000 7f02e85ff700 1 --2- 172.21.15.2:0/1074724626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02e0101ed0 0x7f02e0102330 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.720+0000 7f02e85ff700 1 -- 172.21.15.2:0/1074724626 >> 172.21.15.2:0/1074724626 conn(0x7f02e00fc230 msgr2=0x7f02e00fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:56.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.720+0000 7f02e85ff700 1 -- 172.21.15.2:0/1074724626 shutdown_connections 2023-09-13T04:39:56.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.720+0000 7f02e85ff700 1 -- 172.21.15.2:0/1074724626 wait complete. 2023-09-13T04:39:56.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.721+0000 7f02e85ff700 1 Processor -- start 2023-09-13T04:39:56.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.722+0000 7f02e85ff700 1 -- start start 2023-09-13T04:39:56.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.722+0000 7f02e85ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02e0100ce0 0x7f02e0074a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:56.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.722+0000 7f02e85ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02e0101ed0 0x7f02e00730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:56.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.722+0000 7f02e85ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02e0102870 0x7f02e00735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:56.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.722+0000 7f02e85ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e0073d80 con 0x7f02e0102870 2023-09-13T04:39:56.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.722+0000 7f02e85ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e0073ef0 con 0x7f02e0100ce0 2023-09-13T04:39:56.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.722+0000 7f02e85ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e01a0a40 con 0x7f02e0101ed0 2023-09-13T04:39:56.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.722+0000 7f02e6b9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02e0102870 0x7f02e00735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:56.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.723+0000 7f02e639b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02e0100ce0 0x7f02e0074a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:56.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.723+0000 7f02e6b9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02e0102870 0x7f02e00735e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57952/0 (socket says 172.21.15.2:57952) 2023-09-13T04:39:56.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.723+0000 7f02e6b9c700 1 -- 172.21.15.2:0/3792530536 learned_addr learned my addr 172.21.15.2:0/3792530536 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:56.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.723+0000 7f02e5b9a700 1 --2- 172.21.15.2:0/3792530536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02e0101ed0 0x7f02e00730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:56.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.723+0000 7f02e639b700 1 -- 172.21.15.2:0/3792530536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02e0101ed0 msgr2=0x7f02e00730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:56.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.723+0000 7f02e639b700 1 --2- 172.21.15.2:0/3792530536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02e0101ed0 0x7f02e00730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.723+0000 7f02e639b700 1 -- 172.21.15.2:0/3792530536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02e0102870 msgr2=0x7f02e00735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:56.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.723+0000 7f02e639b700 1 --2- 172.21.15.2:0/3792530536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02e0102870 0x7f02e00735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.723+0000 7f02e639b700 1 -- 172.21.15.2:0/3792530536 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d00095f0 con 0x7f02e0100ce0 2023-09-13T04:39:56.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.723+0000 7f02e6b9c700 1 --2- 172.21.15.2:0/3792530536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02e0102870 0x7f02e00735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:39:56.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.723+0000 7f02e639b700 1 --2- 172.21.15.2:0/3792530536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02e0100ce0 0x7f02e0074a60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00e9c0 tx=0x7f02dc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:56.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.723+0000 7f02d77fe700 1 -- 172.21.15.2:0/3792530536 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc004220 con 0x7f02e0100ce0 2023-09-13T04:39:56.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.724+0000 7f02d77fe700 1 -- 172.21.15.2:0/3792530536 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02dc004380 con 0x7f02e0100ce0 2023-09-13T04:39:56.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.724+0000 7f02e85ff700 1 -- 172.21.15.2:0/3792530536 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e01a0d20 con 0x7f02e0100ce0 2023-09-13T04:39:56.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.724+0000 7f02d77fe700 1 -- 172.21.15.2:0/3792530536 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc003be0 con 0x7f02e0100ce0 2023-09-13T04:39:56.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.724+0000 7f02e85ff700 1 -- 172.21.15.2:0/3792530536 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e01a11d0 con 0x7f02e0100ce0 2023-09-13T04:39:56.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.725+0000 7f02e85ff700 1 -- 172.21.15.2:0/3792530536 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02e0107c60 con 0x7f02e0100ce0 2023-09-13T04:39:56.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.727+0000 7f02d77fe700 1 -- 172.21.15.2:0/3792530536 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f02dc003d40 con 0x7f02e0100ce0 2023-09-13T04:39:56.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.728+0000 7f02d77fe700 1 --2- 172.21.15.2:0/3792530536 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f02cc0a06c0 0x7f02cc0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:56.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.728+0000 7f02d77fe700 1 -- 172.21.15.2:0/3792530536 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f02dc014070 con 0x7f02e0100ce0 2023-09-13T04:39:56.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.728+0000 7f02e5b9a700 1 --2- 172.21.15.2:0/3792530536 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f02cc0a06c0 0x7f02cc0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:56.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.729+0000 7f02e5b9a700 1 --2- 172.21.15.2:0/3792530536 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f02cc0a06c0 0x7f02cc0a2b80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f02d0009fd0 tx=0x7f02d0019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:56.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.730+0000 7f02d77fe700 1 -- 172.21.15.2:0/3792530536 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f02dc095b40 con 0x7f02e0100ce0 2023-09-13T04:39:56.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:56 smithi149 ceph-mon[133315]: pgmap v338: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:39:56.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:56 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/184301702' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:39:56.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:56 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1131757470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:39:56.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:56 smithi156 ceph-mon[133675]: pgmap v338: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:39:56.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:56 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/184301702' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:39:56.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:56 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1131757470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:39:56.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.939+0000 7f02e85ff700 1 -- 172.21.15.2:0/3792530536 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f02e010c7a0 con 0x7f02e0100ce0 2023-09-13T04:39:56.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.939+0000 7f02d77fe700 1 -- 172.21.15.2:0/3792530536 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f02dc099160 con 0x7f02e0100ce0 2023-09-13T04:39:56.940 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-09-13T04:39:56.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.942+0000 7f02e85ff700 1 -- 172.21.15.2:0/3792530536 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f02cc0a06c0 msgr2=0x7f02cc0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:56.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.942+0000 7f02e85ff700 1 --2- 172.21.15.2:0/3792530536 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f02cc0a06c0 0x7f02cc0a2b80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f02d0009fd0 tx=0x7f02d0019040 comp rx=0 tx=0).stop 2023-09-13T04:39:56.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.942+0000 7f02e85ff700 1 -- 172.21.15.2:0/3792530536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02e0100ce0 msgr2=0x7f02e0074a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:56.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.942+0000 7f02e85ff700 1 --2- 172.21.15.2:0/3792530536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02e0100ce0 0x7f02e0074a60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00e9c0 tx=0x7f02dc00ecd0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.942+0000 7f02e85ff700 1 -- 172.21.15.2:0/3792530536 shutdown_connections 2023-09-13T04:39:56.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.942+0000 7f02e85ff700 1 --2- 172.21.15.2:0/3792530536 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f02cc0a06c0 0x7f02cc0a2b80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.942+0000 7f02e85ff700 1 --2- 172.21.15.2:0/3792530536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02e0102870 0x7f02e00735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.942+0000 7f02e85ff700 1 --2- 172.21.15.2:0/3792530536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02e0100ce0 0x7f02e0074a60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.942+0000 7f02e85ff700 1 --2- 172.21.15.2:0/3792530536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02e0101ed0 0x7f02e00730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:56.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.942+0000 7f02e85ff700 1 -- 172.21.15.2:0/3792530536 >> 172.21.15.2:0/3792530536 conn(0x7f02e00fc230 msgr2=0x7f02e00fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:56.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.942+0000 7f02e85ff700 1 -- 172.21.15.2:0/3792530536 shutdown_connections 2023-09-13T04:39:56.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:56.942+0000 7f02e85ff700 1 -- 172.21.15.2:0/3792530536 wait complete. 2023-09-13T04:39:56.960 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:56 smithi002 ceph-mon[97651]: pgmap v338: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:39:56.960 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:56 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/184301702' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:39:56.960 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:56 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1131757470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:39:56.962 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:39:57.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.165+0000 7fd46bdff700 1 -- 172.21.15.2:0/3033281597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd464101ee0 msgr2=0x7fd464102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:57.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.165+0000 7fd46bdff700 1 --2- 172.21.15.2:0/3033281597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd464101ee0 0x7fd464102340 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd454009990 tx=0x7fd454009ca0 comp rx=0 tx=0).stop 2023-09-13T04:39:57.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.165+0000 7fd46bdff700 1 -- 172.21.15.2:0/3033281597 shutdown_connections 2023-09-13T04:39:57.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.165+0000 7fd46bdff700 1 --2- 172.21.15.2:0/3033281597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd464102880 0x7fd464109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:57.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.165+0000 7fd46bdff700 1 --2- 172.21.15.2:0/3033281597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd464100cf0 0x7fd464101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:57.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.165+0000 7fd46bdff700 1 --2- 172.21.15.2:0/3033281597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd464101ee0 0x7fd464102340 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:57.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.165+0000 7fd46bdff700 1 -- 172.21.15.2:0/3033281597 >> 172.21.15.2:0/3033281597 conn(0x7fd4640fc220 msgr2=0x7fd4640fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:57.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.166+0000 7fd46bdff700 1 -- 172.21.15.2:0/3033281597 shutdown_connections 2023-09-13T04:39:57.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.166+0000 7fd46bdff700 1 -- 172.21.15.2:0/3033281597 wait complete. 2023-09-13T04:39:57.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.167+0000 7fd46bdff700 1 Processor -- start 2023-09-13T04:39:57.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.167+0000 7fd46bdff700 1 -- start start 2023-09-13T04:39:57.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.167+0000 7fd46bdff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd464100cf0 0x7fd464198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:57.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.167+0000 7fd46bdff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd464101ee0 0x7fd464199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.167+0000 7fd46bdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd464102880 0x7fd4641a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.167+0000 7fd46bdff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4641a0760 con 0x7fd464101ee0 2023-09-13T04:39:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.167+0000 7fd46bdff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4641a08a0 con 0x7fd464100cf0 2023-09-13T04:39:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.167+0000 7fd46bdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4641a0a30 con 0x7fd464102880 2023-09-13T04:39:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.167+0000 7fd46939a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd464101ee0 0x7fd464199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.167+0000 7fd46a39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd464102880 0x7fd4641a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.167+0000 7fd46a39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd464102880 0x7fd4641a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:57106/0 (socket says 172.21.15.2:57106) 2023-09-13T04:39:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.167+0000 7fd46a39c700 1 -- 172.21.15.2:0/2309339716 learned_addr learned my addr 172.21.15.2:0/2309339716 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.167+0000 7fd469b9b700 1 --2- 172.21.15.2:0/2309339716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd464100cf0 0x7fd464198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.168+0000 7fd46939a700 1 -- 172.21.15.2:0/2309339716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd464102880 msgr2=0x7fd4641a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.168+0000 7fd46939a700 1 --2- 172.21.15.2:0/2309339716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd464102880 0x7fd4641a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.168+0000 7fd46939a700 1 -- 172.21.15.2:0/2309339716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd464100cf0 msgr2=0x7fd464198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.168+0000 7fd46939a700 1 --2- 172.21.15.2:0/2309339716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd464100cf0 0x7fd464198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.168+0000 7fd46939a700 1 -- 172.21.15.2:0/2309339716 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4540095f0 con 0x7fd464101ee0 2023-09-13T04:39:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.168+0000 7fd469b9b700 1 --2- 172.21.15.2:0/2309339716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd464100cf0 0x7fd464198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:39:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.168+0000 7fd46939a700 1 --2- 172.21.15.2:0/2309339716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd464101ee0 0x7fd464199120 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd45400b680 tx=0x7fd454009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:57.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.168+0000 7fd45affd700 1 -- 172.21.15.2:0/2309339716 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45401d070 con 0x7fd464101ee0 2023-09-13T04:39:57.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.168+0000 7fd45affd700 1 -- 172.21.15.2:0/2309339716 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd454021410 con 0x7fd464101ee0 2023-09-13T04:39:57.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.168+0000 7fd46bdff700 1 -- 172.21.15.2:0/2309339716 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4641a0cb0 con 0x7fd464101ee0 2023-09-13T04:39:57.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.168+0000 7fd45affd700 1 -- 172.21.15.2:0/2309339716 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd454017650 con 0x7fd464101ee0 2023-09-13T04:39:57.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.169+0000 7fd46bdff700 1 -- 172.21.15.2:0/2309339716 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4641a1130 con 0x7fd464101ee0 2023-09-13T04:39:57.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.170+0000 7fd46bdff700 1 -- 172.21.15.2:0/2309339716 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd46410c550 con 0x7fd464101ee0 2023-09-13T04:39:57.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.172+0000 7fd45affd700 1 -- 172.21.15.2:0/2309339716 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fd454021810 con 0x7fd464101ee0 2023-09-13T04:39:57.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.173+0000 7fd45affd700 1 --2- 172.21.15.2:0/2309339716 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd4500a0710 0x7fd4500a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:57.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.173+0000 7fd45affd700 1 -- 172.21.15.2:0/2309339716 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd4540cd820 con 0x7fd464101ee0 2023-09-13T04:39:57.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.173+0000 7fd469b9b700 1 --2- 172.21.15.2:0/2309339716 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd4500a0710 0x7fd4500a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:57.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.174+0000 7fd469b9b700 1 --2- 172.21.15.2:0/2309339716 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd4500a0710 0x7fd4500a2bd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd464101d40 tx=0x7fd460009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:57.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.175+0000 7fd45affd700 1 -- 172.21.15.2:0/2309339716 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd4540970b0 con 0x7fd464101ee0 2023-09-13T04:39:57.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.432+0000 7fd46bdff700 1 -- 172.21.15.2:0/2309339716 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4641a12c0 con 0x7fd464101ee0 2023-09-13T04:39:57.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.433+0000 7fd45affd700 1 -- 172.21.15.2:0/2309339716 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11303 (secure 0 0 0) 0x7fd45409a6d0 con 0x7fd464101ee0 2023-09-13T04:39:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:39:57.435 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2469307307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2469307307},{"type":"v1","addr":"172.21.15.2:6839","nonce":2469307307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T04:37:59.110701+0000","modified":"2023-09-13T04:38:26.610260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24554,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/846579077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":846579077},{"type":"v1","addr":"172.21.15.156:6835","nonce":846579077}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1896833690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1896833690},{"type":"v1","addr":"172.21.15.149:6837","nonce":1896833690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/524563090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":524563090},{"type":"v1","addr":"172.21.15.156:6837","nonce":524563090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6839/1785833227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1785833227},{"type":"v1","addr":"172.21.15.156:6839","nonce":1785833227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.2:6841/482545391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":482545391},{"type":"v1","addr":"172.21.15.2:6841","nonce":482545391}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/389167841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":389167841},{"type":"v1","addr":"172.21.15.149:6839","nonce":389167841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T04:39:57.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.435+0000 7fd46bdff700 1 -- 172.21.15.2:0/2309339716 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd4500a0710 msgr2=0x7fd4500a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:57.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.435+0000 7fd46bdff700 1 --2- 172.21.15.2:0/2309339716 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd4500a0710 0x7fd4500a2bd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd464101d40 tx=0x7fd460009260 comp rx=0 tx=0).stop 2023-09-13T04:39:57.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.435+0000 7fd46bdff700 1 -- 172.21.15.2:0/2309339716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd464101ee0 msgr2=0x7fd464199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:57.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.435+0000 7fd46bdff700 1 --2- 172.21.15.2:0/2309339716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd464101ee0 0x7fd464199120 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd45400b680 tx=0x7fd454009580 comp rx=0 tx=0).stop 2023-09-13T04:39:57.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.435+0000 7fd46bdff700 1 -- 172.21.15.2:0/2309339716 shutdown_connections 2023-09-13T04:39:57.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.435+0000 7fd46bdff700 1 --2- 172.21.15.2:0/2309339716 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fd4500a0710 0x7fd4500a2bd0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:57.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.435+0000 7fd46bdff700 1 --2- 172.21.15.2:0/2309339716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd464102880 0x7fd4641a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:57.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.435+0000 7fd46bdff700 1 --2- 172.21.15.2:0/2309339716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd464100cf0 0x7fd464198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:57.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.435+0000 7fd46bdff700 1 --2- 172.21.15.2:0/2309339716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd464101ee0 0x7fd464199120 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:57.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.435+0000 7fd46bdff700 1 -- 172.21.15.2:0/2309339716 >> 172.21.15.2:0/2309339716 conn(0x7fd4640fc220 msgr2=0x7fd4640fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:57.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.436+0000 7fd46bdff700 1 -- 172.21.15.2:0/2309339716 shutdown_connections 2023-09-13T04:39:57.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.436+0000 7fd46bdff700 1 -- 172.21.15.2:0/2309339716 wait complete. 2023-09-13T04:39:57.440 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 23 2023-09-13T04:39:57.456 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T04:39:57.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.661+0000 7ff3737ff700 1 -- 172.21.15.2:0/815614591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff36c102880 msgr2=0x7ff36c109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:57.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.661+0000 7ff3737ff700 1 --2- 172.21.15.2:0/815614591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff36c102880 0x7ff36c109130 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff36400b0d0 tx=0x7ff36400b3e0 comp rx=0 tx=0).stop 2023-09-13T04:39:57.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.661+0000 7ff3737ff700 1 -- 172.21.15.2:0/815614591 shutdown_connections 2023-09-13T04:39:57.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.661+0000 7ff3737ff700 1 --2- 172.21.15.2:0/815614591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff36c102880 0x7ff36c109130 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:57.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.661+0000 7ff3737ff700 1 --2- 172.21.15.2:0/815614591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff36c100cf0 0x7ff36c101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:57.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.661+0000 7ff3737ff700 1 --2- 172.21.15.2:0/815614591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff36c101ee0 0x7ff36c102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:57.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.661+0000 7ff3737ff700 1 -- 172.21.15.2:0/815614591 >> 172.21.15.2:0/815614591 conn(0x7ff36c0fc220 msgr2=0x7ff36c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:57.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.661+0000 7ff3737ff700 1 -- 172.21.15.2:0/815614591 shutdown_connections 2023-09-13T04:39:57.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.661+0000 7ff3737ff700 1 -- 172.21.15.2:0/815614591 wait complete. 2023-09-13T04:39:57.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.662+0000 7ff3737ff700 1 Processor -- start 2023-09-13T04:39:57.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.662+0000 7ff3737ff700 1 -- start start 2023-09-13T04:39:57.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.662+0000 7ff3737ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff36c100cf0 0x7ff36c198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:57.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.662+0000 7ff3737ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff36c101ee0 0x7ff36c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:57.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.663+0000 7ff3737ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff36c102880 0x7ff36c1a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:57.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.663+0000 7ff3737ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c1a08e0 con 0x7ff36c102880 2023-09-13T04:39:57.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.663+0000 7ff3737ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c1a0a20 con 0x7ff36c100cf0 2023-09-13T04:39:57.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.663+0000 7ff3737ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c1a0bb0 con 0x7ff36c101ee0 2023-09-13T04:39:57.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.663+0000 7ff371d9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff36c102880 0x7ff36c1a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:57.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.663+0000 7ff371d9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff36c102880 0x7ff36c1a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57986/0 (socket says 172.21.15.2:57986) 2023-09-13T04:39:57.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.663+0000 7ff371d9c700 1 -- 172.21.15.2:0/517652525 learned_addr learned my addr 172.21.15.2:0/517652525 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:57.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.663+0000 7ff37159b700 1 --2- 172.21.15.2:0/517652525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff36c100cf0 0x7ff36c198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:57.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.663+0000 7ff370d9a700 1 --2- 172.21.15.2:0/517652525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff36c101ee0 0x7ff36c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:57.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.663+0000 7ff371d9c700 1 -- 172.21.15.2:0/517652525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff36c101ee0 msgr2=0x7ff36c1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:57.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.663+0000 7ff371d9c700 1 --2- 172.21.15.2:0/517652525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff36c101ee0 0x7ff36c1992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:57.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.663+0000 7ff371d9c700 1 -- 172.21.15.2:0/517652525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff36c100cf0 msgr2=0x7ff36c198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:57.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.663+0000 7ff371d9c700 1 --2- 172.21.15.2:0/517652525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff36c100cf0 0x7ff36c198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:57.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.663+0000 7ff371d9c700 1 -- 172.21.15.2:0/517652525 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff368009520 con 0x7ff36c102880 2023-09-13T04:39:57.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.664+0000 7ff371d9c700 1 --2- 172.21.15.2:0/517652525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff36c102880 0x7ff36c1a0310 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff364008eb0 tx=0x7ff364008ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:57.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.664+0000 7ff3627fc700 1 -- 172.21.15.2:0/517652525 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36400e040 con 0x7ff36c102880 2023-09-13T04:39:57.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.664+0000 7ff3627fc700 1 -- 172.21.15.2:0/517652525 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff36400bdb0 con 0x7ff36c102880 2023-09-13T04:39:57.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.664+0000 7ff3737ff700 1 -- 172.21.15.2:0/517652525 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff364009d00 con 0x7ff36c102880 2023-09-13T04:39:57.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.664+0000 7ff3627fc700 1 -- 172.21.15.2:0/517652525 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36401d470 con 0x7ff36c102880 2023-09-13T04:39:57.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.664+0000 7ff3737ff700 1 -- 172.21.15.2:0/517652525 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff36c1a1170 con 0x7ff36c102880 2023-09-13T04:39:57.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.665+0000 7ff3737ff700 1 -- 172.21.15.2:0/517652525 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff36c10c560 con 0x7ff36c102880 2023-09-13T04:39:57.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.667+0000 7ff3627fc700 1 -- 172.21.15.2:0/517652525 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7ff364005020 con 0x7ff36c102880 2023-09-13T04:39:57.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.668+0000 7ff3627fc700 1 --2- 172.21.15.2:0/517652525 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff3580a0710 0x7ff3580a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:57.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.669+0000 7ff3627fc700 1 -- 172.21.15.2:0/517652525 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff3640ccdd0 con 0x7ff36c102880 2023-09-13T04:39:57.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.669+0000 7ff37159b700 1 --2- 172.21.15.2:0/517652525 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff3580a0710 0x7ff3580a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:57.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.669+0000 7ff37159b700 1 --2- 172.21.15.2:0/517652525 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff3580a0710 0x7ff3580a2bd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff36c101d40 tx=0x7ff368009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:57.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.672+0000 7ff3627fc700 1 -- 172.21.15.2:0/517652525 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff364096600 con 0x7ff36c102880 2023-09-13T04:39:57.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:57 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3792530536' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:39:57.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:57 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2309339716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:39:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.883+0000 7ff3737ff700 1 -- 172.21.15.2:0/517652525 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff36c10c780 con 0x7ff36c102880 2023-09-13T04:39:57.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:57 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3792530536' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:39:57.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:57 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2309339716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:39:57.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.885+0000 7ff3627fc700 1 -- 172.21.15.2:0/517652525 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24503 (secure 0 0 0) 0x7ff364099c20 con 0x7ff36c102880 2023-09-13T04:39:57.887 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:39:57.888 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":92,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:39:25.580735+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T04:37:57.094802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T04:37:58.564487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-13T04:38:02.096895+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:39:57.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.888+0000 7ff3737ff700 1 -- 172.21.15.2:0/517652525 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff3580a0710 msgr2=0x7ff3580a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:57.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.888+0000 7ff3737ff700 1 --2- 172.21.15.2:0/517652525 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff3580a0710 0x7ff3580a2bd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff36c101d40 tx=0x7ff368009260 comp rx=0 tx=0).stop 2023-09-13T04:39:57.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.888+0000 7ff3737ff700 1 -- 172.21.15.2:0/517652525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff36c102880 msgr2=0x7ff36c1a0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:57.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.888+0000 7ff3737ff700 1 --2- 172.21.15.2:0/517652525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff36c102880 0x7ff36c1a0310 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff364008eb0 tx=0x7ff364008ee0 comp rx=0 tx=0).stop 2023-09-13T04:39:57.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.888+0000 7ff3737ff700 1 -- 172.21.15.2:0/517652525 shutdown_connections 2023-09-13T04:39:57.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.888+0000 7ff3737ff700 1 --2- 172.21.15.2:0/517652525 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff3580a0710 0x7ff3580a2bd0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:57.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.888+0000 7ff3737ff700 1 --2- 172.21.15.2:0/517652525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff36c102880 0x7ff36c1a0310 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:57.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.888+0000 7ff3737ff700 1 --2- 172.21.15.2:0/517652525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff36c100cf0 0x7ff36c198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:57.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.888+0000 7ff3737ff700 1 --2- 172.21.15.2:0/517652525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff36c101ee0 0x7ff36c1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:57.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.888+0000 7ff3737ff700 1 -- 172.21.15.2:0/517652525 >> 172.21.15.2:0/517652525 conn(0x7ff36c0fc220 msgr2=0x7ff36c107770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:57.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.888+0000 7ff3737ff700 1 -- 172.21.15.2:0/517652525 shutdown_connections 2023-09-13T04:39:57.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:57.888+0000 7ff3737ff700 1 -- 172.21.15.2:0/517652525 wait complete. 2023-09-13T04:39:57.908 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T04:33:42.025714+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-13T04:37:57.094802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-13T04:37:58.564487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-13T04:38:02.096895+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-13T04:39:57.909 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-13T04:39:57.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:57 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3792530536' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:39:57.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:57 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2309339716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:39:58.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.114+0000 7f8c683ff700 1 -- 172.21.15.2:0/1680615936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c60101ed0 msgr2=0x7f8c60102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:58.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.114+0000 7f8c683ff700 1 --2- 172.21.15.2:0/1680615936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c60101ed0 0x7f8c60102330 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8c500099c0 tx=0x7f8c50009cd0 comp rx=0 tx=0).stop 2023-09-13T04:39:58.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.114+0000 7f8c683ff700 1 -- 172.21.15.2:0/1680615936 shutdown_connections 2023-09-13T04:39:58.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.114+0000 7f8c683ff700 1 --2- 172.21.15.2:0/1680615936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c60102870 0x7f8c60109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:58.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.114+0000 7f8c683ff700 1 --2- 172.21.15.2:0/1680615936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c60100ce0 0x7f8c60101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:58.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.114+0000 7f8c683ff700 1 --2- 172.21.15.2:0/1680615936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c60101ed0 0x7f8c60102330 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:58.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.114+0000 7f8c683ff700 1 -- 172.21.15.2:0/1680615936 >> 172.21.15.2:0/1680615936 conn(0x7f8c600fc230 msgr2=0x7f8c600fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:58.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.114+0000 7f8c683ff700 1 -- 172.21.15.2:0/1680615936 shutdown_connections 2023-09-13T04:39:58.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.114+0000 7f8c683ff700 1 -- 172.21.15.2:0/1680615936 wait complete. 2023-09-13T04:39:58.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.115+0000 7f8c683ff700 1 Processor -- start 2023-09-13T04:39:58.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.115+0000 7f8c683ff700 1 -- start start 2023-09-13T04:39:58.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.115+0000 7f8c683ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c60100ce0 0x7f8c601969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:58.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.115+0000 7f8c683ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c60101ed0 0x7f8c60196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:58.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.115+0000 7f8c683ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c60102870 0x7f8c6019df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:58.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.115+0000 7f8c683ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6019e550 con 0x7f8c60100ce0 2023-09-13T04:39:58.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.115+0000 7f8c683ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6019e690 con 0x7f8c60101ed0 2023-09-13T04:39:58.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.115+0000 7f8c683ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6019e820 con 0x7f8c60102870 2023-09-13T04:39:58.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.115+0000 7f8c6619b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c60100ce0 0x7f8c601969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:58.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.116+0000 7f8c6619b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c60100ce0 0x7f8c601969d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58016/0 (socket says 172.21.15.2:58016) 2023-09-13T04:39:58.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.116+0000 7f8c6619b700 1 -- 172.21.15.2:0/1112718851 learned_addr learned my addr 172.21.15.2:0/1112718851 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:58.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.116+0000 7f8c6599a700 1 --2- 172.21.15.2:0/1112718851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c60101ed0 0x7f8c60196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:58.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.116+0000 7f8c6699c700 1 --2- 172.21.15.2:0/1112718851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c60102870 0x7f8c6019df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:58.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.116+0000 7f8c6619b700 1 -- 172.21.15.2:0/1112718851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c60102870 msgr2=0x7f8c6019df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:58.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.116+0000 7f8c6619b700 1 --2- 172.21.15.2:0/1112718851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c60102870 0x7f8c6019df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:58.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.116+0000 7f8c6619b700 1 -- 172.21.15.2:0/1112718851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c60101ed0 msgr2=0x7f8c60196f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:58.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.116+0000 7f8c6619b700 1 --2- 172.21.15.2:0/1112718851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c60101ed0 0x7f8c60196f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:58.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.116+0000 7f8c6619b700 1 -- 172.21.15.2:0/1112718851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c500095f0 con 0x7f8c60100ce0 2023-09-13T04:39:58.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.116+0000 7f8c6599a700 1 --2- 172.21.15.2:0/1112718851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c60101ed0 0x7f8c60196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:39:58.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.116+0000 7f8c6699c700 1 --2- 172.21.15.2:0/1112718851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c60102870 0x7f8c6019df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:39:58.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.116+0000 7f8c6619b700 1 --2- 172.21.15.2:0/1112718851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c60100ce0 0x7f8c601969d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00e970 tx=0x7f8c5c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:58.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.116+0000 7f8c577fe700 1 -- 172.21.15.2:0/1112718851 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c009790 con 0x7f8c60100ce0 2023-09-13T04:39:58.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.116+0000 7f8c577fe700 1 -- 172.21.15.2:0/1112718851 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c5c0044e0 con 0x7f8c60100ce0 2023-09-13T04:39:58.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.116+0000 7f8c683ff700 1 -- 172.21.15.2:0/1112718851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c6019eb00 con 0x7f8c60100ce0 2023-09-13T04:39:58.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.116+0000 7f8c577fe700 1 -- 172.21.15.2:0/1112718851 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c003e30 con 0x7f8c60100ce0 2023-09-13T04:39:58.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.117+0000 7f8c683ff700 1 -- 172.21.15.2:0/1112718851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c6019f010 con 0x7f8c60100ce0 2023-09-13T04:39:58.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.118+0000 7f8c683ff700 1 -- 172.21.15.2:0/1112718851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c60107c10 con 0x7f8c60100ce0 2023-09-13T04:39:58.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.119+0000 7f8c577fe700 1 -- 172.21.15.2:0/1112718851 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f8c5c004000 con 0x7f8c60100ce0 2023-09-13T04:39:58.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.121+0000 7f8c577fe700 1 --2- 172.21.15.2:0/1112718851 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8c4c0a0640 0x7f8c4c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:58.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.121+0000 7f8c577fe700 1 -- 172.21.15.2:0/1112718851 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8c5c014070 con 0x7f8c60100ce0 2023-09-13T04:39:58.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.121+0000 7f8c6599a700 1 --2- 172.21.15.2:0/1112718851 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8c4c0a0640 0x7f8c4c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:58.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.122+0000 7f8c6599a700 1 --2- 172.21.15.2:0/1112718851 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8c4c0a0640 0x7f8c4c0a2b00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8c5000b580 tx=0x7f8c5001a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:58.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.125+0000 7f8c577fe700 1 -- 172.21.15.2:0/1112718851 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8c5c095a20 con 0x7f8c60100ce0 2023-09-13T04:39:58.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.334+0000 7f8c683ff700 1 -- 172.21.15.2:0/1112718851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8c6002d030 con 0x7f8c60100ce0 2023-09-13T04:39:58.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.334+0000 7f8c577fe700 1 -- 172.21.15.2:0/1112718851 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8c5c099040 con 0x7f8c60100ce0 2023-09-13T04:39:58.336 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-09-13T04:39:58.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.336+0000 7f8c683ff700 1 -- 172.21.15.2:0/1112718851 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8c4c0a0640 msgr2=0x7f8c4c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:58.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.336+0000 7f8c683ff700 1 --2- 172.21.15.2:0/1112718851 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8c4c0a0640 0x7f8c4c0a2b00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8c5000b580 tx=0x7f8c5001a040 comp rx=0 tx=0).stop 2023-09-13T04:39:58.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.336+0000 7f8c683ff700 1 -- 172.21.15.2:0/1112718851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c60100ce0 msgr2=0x7f8c601969d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:58.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.336+0000 7f8c683ff700 1 --2- 172.21.15.2:0/1112718851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c60100ce0 0x7f8c601969d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00e970 tx=0x7f8c5c00ed30 comp rx=0 tx=0).stop 2023-09-13T04:39:58.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.336+0000 7f8c683ff700 1 -- 172.21.15.2:0/1112718851 shutdown_connections 2023-09-13T04:39:58.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.336+0000 7f8c683ff700 1 --2- 172.21.15.2:0/1112718851 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8c4c0a0640 0x7f8c4c0a2b00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:58.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.336+0000 7f8c683ff700 1 --2- 172.21.15.2:0/1112718851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c60102870 0x7f8c6019df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:58.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.336+0000 7f8c683ff700 1 --2- 172.21.15.2:0/1112718851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c60100ce0 0x7f8c601969d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:58.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.336+0000 7f8c683ff700 1 --2- 172.21.15.2:0/1112718851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c60101ed0 0x7f8c60196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:58.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.337+0000 7f8c683ff700 1 -- 172.21.15.2:0/1112718851 >> 172.21.15.2:0/1112718851 conn(0x7f8c600fc230 msgr2=0x7f8c600fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:58.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.337+0000 7f8c683ff700 1 -- 172.21.15.2:0/1112718851 shutdown_connections 2023-09-13T04:39:58.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.337+0000 7f8c683ff700 1 -- 172.21.15.2:0/1112718851 wait complete. 2023-09-13T04:39:58.357 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-13T04:39:58.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.564+0000 7fdcbcdff700 1 -- 172.21.15.2:0/3037377676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcb8073690 msgr2=0x7fdcb8073b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:58.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.564+0000 7fdcbcdff700 1 --2- 172.21.15.2:0/3037377676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcb8073690 0x7fdcb8073b10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009990 tx=0x7fdcac009ca0 comp rx=0 tx=0).stop 2023-09-13T04:39:58.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.564+0000 7fdcbcdff700 1 -- 172.21.15.2:0/3037377676 shutdown_connections 2023-09-13T04:39:58.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.564+0000 7fdcbcdff700 1 --2- 172.21.15.2:0/3037377676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcb8108be0 0x7fdcb810b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:58.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.564+0000 7fdcbcdff700 1 --2- 172.21.15.2:0/3037377676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcb8074c70 0x7fdcb80730c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:58.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.564+0000 7fdcbcdff700 1 --2- 172.21.15.2:0/3037377676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcb8073690 0x7fdcb8073b10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:58.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.564+0000 7fdcbcdff700 1 -- 172.21.15.2:0/3037377676 >> 172.21.15.2:0/3037377676 conn(0x7fdcb80fc260 msgr2=0x7fdcb80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:58.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.564+0000 7fdcbcdff700 1 -- 172.21.15.2:0/3037377676 shutdown_connections 2023-09-13T04:39:58.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.565+0000 7fdcbcdff700 1 -- 172.21.15.2:0/3037377676 wait complete. 2023-09-13T04:39:58.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.565+0000 7fdcbcdff700 1 Processor -- start 2023-09-13T04:39:58.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.566+0000 7fdcbcdff700 1 -- start start 2023-09-13T04:39:58.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.566+0000 7fdcbcdff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcb8073690 0x7fdcb8198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:58.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.566+0000 7fdcbcdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcb8074c70 0x7fdcb8199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:58.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.566+0000 7fdcbcdff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcb8108be0 0x7fdcb81a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:58.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.566+0000 7fdcbcdff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb81a0750 con 0x7fdcb8108be0 2023-09-13T04:39:58.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.566+0000 7fdcbcdff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb81a0890 con 0x7fdcb8073690 2023-09-13T04:39:58.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.566+0000 7fdcbcdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb81a0a20 con 0x7fdcb8074c70 2023-09-13T04:39:58.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.566+0000 7fdcb6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcb8108be0 0x7fdcb81a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:58.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.566+0000 7fdcb6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcb8108be0 0x7fdcb81a0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58026/0 (socket says 172.21.15.2:58026) 2023-09-13T04:39:58.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.566+0000 7fdcb6d9d700 1 -- 172.21.15.2:0/1764400678 learned_addr learned my addr 172.21.15.2:0/1764400678 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:58.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.566+0000 7fdcb659c700 1 --2- 172.21.15.2:0/1764400678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcb8073690 0x7fdcb8198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:58.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.566+0000 7fdcb5d9b700 1 --2- 172.21.15.2:0/1764400678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcb8074c70 0x7fdcb8199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:58.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.567+0000 7fdcb6d9d700 1 -- 172.21.15.2:0/1764400678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcb8074c70 msgr2=0x7fdcb8199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:58.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.567+0000 7fdcb6d9d700 1 --2- 172.21.15.2:0/1764400678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcb8074c70 0x7fdcb8199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:58.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.567+0000 7fdcb6d9d700 1 -- 172.21.15.2:0/1764400678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcb8073690 msgr2=0x7fdcb8198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:58.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.567+0000 7fdcb6d9d700 1 --2- 172.21.15.2:0/1764400678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcb8073690 0x7fdcb8198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:58.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.567+0000 7fdcb6d9d700 1 -- 172.21.15.2:0/1764400678 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcac0095f0 con 0x7fdcb8108be0 2023-09-13T04:39:58.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.567+0000 7fdcb659c700 1 --2- 172.21.15.2:0/1764400678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcb8073690 0x7fdcb8198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:39:58.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.567+0000 7fdcb6d9d700 1 --2- 172.21.15.2:0/1764400678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcb8108be0 0x7fdcb81a0180 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000ef30 tx=0x7fdcb000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:58.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.567+0000 7fdca77fe700 1 -- 172.21.15.2:0/1764400678 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0018070 con 0x7fdcb8108be0 2023-09-13T04:39:58.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.567+0000 7fdca77fe700 1 -- 172.21.15.2:0/1764400678 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcb000f040 con 0x7fdcb8108be0 2023-09-13T04:39:58.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.567+0000 7fdcbcdff700 1 -- 172.21.15.2:0/1764400678 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb81a0ca0 con 0x7fdcb8108be0 2023-09-13T04:39:58.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.567+0000 7fdca77fe700 1 -- 172.21.15.2:0/1764400678 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb000cd00 con 0x7fdcb8108be0 2023-09-13T04:39:58.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.567+0000 7fdcbcdff700 1 -- 172.21.15.2:0/1764400678 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb81a11b0 con 0x7fdcb8108be0 2023-09-13T04:39:58.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.568+0000 7fdcbcdff700 1 -- 172.21.15.2:0/1764400678 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb81072e0 con 0x7fdcb8108be0 2023-09-13T04:39:58.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.570+0000 7fdca77fe700 1 -- 172.21.15.2:0/1764400678 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fdcb0003a00 con 0x7fdcb8108be0 2023-09-13T04:39:58.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.572+0000 7fdca77fe700 1 --2- 172.21.15.2:0/1764400678 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fdca00a06c0 0x7fdca00a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:58.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.572+0000 7fdca77fe700 1 -- 172.21.15.2:0/1764400678 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdcb00cc3d0 con 0x7fdcb8108be0 2023-09-13T04:39:58.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.572+0000 7fdcb659c700 1 --2- 172.21.15.2:0/1764400678 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fdca00a06c0 0x7fdca00a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:58.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.573+0000 7fdcb659c700 1 --2- 172.21.15.2:0/1764400678 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fdca00a06c0 0x7fdca00a2b80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdca8011490 tx=0x7fdca8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:58.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.574+0000 7fdca77fe700 1 -- 172.21.15.2:0/1764400678 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fdcb0095c00 con 0x7fdcb8108be0 2023-09-13T04:39:58.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.793+0000 7fdcbcdff700 1 -- 172.21.15.2:0/1764400678 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdcb810e8c0 con 0x7fdcb8108be0 2023-09-13T04:39:58.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.794+0000 7fdca77fe700 1 -- 172.21.15.2:0/1764400678 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdcb0099220 con 0x7fdcb8108be0 2023-09-13T04:39:58.796 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-09-13T04:39:58.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.796+0000 7fdcbcdff700 1 -- 172.21.15.2:0/1764400678 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fdca00a06c0 msgr2=0x7fdca00a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:58.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.796+0000 7fdcbcdff700 1 --2- 172.21.15.2:0/1764400678 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fdca00a06c0 0x7fdca00a2b80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdca8011490 tx=0x7fdca8009260 comp rx=0 tx=0).stop 2023-09-13T04:39:58.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.796+0000 7fdcbcdff700 1 -- 172.21.15.2:0/1764400678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcb8108be0 msgr2=0x7fdcb81a0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:58.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.796+0000 7fdcbcdff700 1 --2- 172.21.15.2:0/1764400678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcb8108be0 0x7fdcb81a0180 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000ef30 tx=0x7fdcb000c3a0 comp rx=0 tx=0).stop 2023-09-13T04:39:58.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.797+0000 7fdcbcdff700 1 -- 172.21.15.2:0/1764400678 shutdown_connections 2023-09-13T04:39:58.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.797+0000 7fdcbcdff700 1 --2- 172.21.15.2:0/1764400678 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fdca00a06c0 0x7fdca00a2b80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:58.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.797+0000 7fdcbcdff700 1 --2- 172.21.15.2:0/1764400678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcb8108be0 0x7fdcb81a0180 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:58.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.797+0000 7fdcbcdff700 1 --2- 172.21.15.2:0/1764400678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcb8073690 0x7fdcb8198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:58.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.797+0000 7fdcbcdff700 1 --2- 172.21.15.2:0/1764400678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcb8074c70 0x7fdcb8199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:58.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.797+0000 7fdcbcdff700 1 -- 172.21.15.2:0/1764400678 >> 172.21.15.2:0/1764400678 conn(0x7fdcb80fc260 msgr2=0x7fdcb80fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:58.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.797+0000 7fdcbcdff700 1 -- 172.21.15.2:0/1764400678 shutdown_connections 2023-09-13T04:39:58.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:58.797+0000 7fdcbcdff700 1 -- 172.21.15.2:0/1764400678 wait complete. 2023-09-13T04:39:58.817 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-13T04:39:58.878 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:58 smithi002 ceph-mon[97651]: pgmap v339: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:39:58.879 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:58 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/517652525' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:39:58.879 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:58 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1112718851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:39:58.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:58 smithi149 ceph-mon[133315]: pgmap v339: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:39:58.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:58 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/517652525' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:39:58.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:58 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1112718851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:39:58.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:58 smithi156 ceph-mon[133675]: pgmap v339: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:39:58.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:58 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/517652525' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:39:58.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:58 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1112718851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:39:59.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.024+0000 7f22313ff700 1 -- 172.21.15.2:0/317856562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f222c100ce0 msgr2=0x7f222c101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:59.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.024+0000 7f22313ff700 1 --2- 172.21.15.2:0/317856562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f222c100ce0 0x7f222c101100 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f221c00b0d0 tx=0x7f221c00b3e0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.024+0000 7f22313ff700 1 -- 172.21.15.2:0/317856562 shutdown_connections 2023-09-13T04:39:59.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.024+0000 7f22313ff700 1 --2- 172.21.15.2:0/317856562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f222c102870 0x7f222c109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.024+0000 7f22313ff700 1 --2- 172.21.15.2:0/317856562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f222c100ce0 0x7f222c101100 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.024+0000 7f22313ff700 1 --2- 172.21.15.2:0/317856562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f222c101ed0 0x7f222c102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.024+0000 7f22313ff700 1 -- 172.21.15.2:0/317856562 >> 172.21.15.2:0/317856562 conn(0x7f222c0fc230 msgr2=0x7f222c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:59.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.024+0000 7f22313ff700 1 -- 172.21.15.2:0/317856562 shutdown_connections 2023-09-13T04:39:59.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.024+0000 7f22313ff700 1 -- 172.21.15.2:0/317856562 wait complete. 2023-09-13T04:39:59.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.025+0000 7f22313ff700 1 Processor -- start 2023-09-13T04:39:59.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.026+0000 7f22313ff700 1 -- start start 2023-09-13T04:39:59.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.026+0000 7f22313ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f222c100ce0 0x7f222c074a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:59.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.026+0000 7f22313ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f222c101ed0 0x7f222c0730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:59.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.026+0000 7f22313ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f222c102870 0x7f222c0735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:59.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.026+0000 7f22313ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c073d80 con 0x7f222c100ce0 2023-09-13T04:39:59.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.026+0000 7f22313ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c073ef0 con 0x7f222c101ed0 2023-09-13T04:39:59.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.026+0000 7f22313ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c1a0bf0 con 0x7f222c102870 2023-09-13T04:39:59.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.026+0000 7f222affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f222c100ce0 0x7f222c074a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:59.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.026+0000 7f222affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f222c100ce0 0x7f222c074a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58056/0 (socket says 172.21.15.2:58056) 2023-09-13T04:39:59.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.026+0000 7f222affd700 1 -- 172.21.15.2:0/876567668 learned_addr learned my addr 172.21.15.2:0/876567668 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:59.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.026+0000 7f221bfff700 1 --2- 172.21.15.2:0/876567668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f222c101ed0 0x7f222c0730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:59.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.026+0000 7f222b7fe700 1 --2- 172.21.15.2:0/876567668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f222c102870 0x7f222c0735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:59.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.027+0000 7f222affd700 1 -- 172.21.15.2:0/876567668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f222c102870 msgr2=0x7f222c0735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:59.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.027+0000 7f222affd700 1 --2- 172.21.15.2:0/876567668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f222c102870 0x7f222c0735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.027+0000 7f222affd700 1 -- 172.21.15.2:0/876567668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f222c101ed0 msgr2=0x7f222c0730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:59.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.027+0000 7f222affd700 1 --2- 172.21.15.2:0/876567668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f222c101ed0 0x7f222c0730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.027+0000 7f222affd700 1 -- 172.21.15.2:0/876567668 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221c009d00 con 0x7f222c100ce0 2023-09-13T04:39:59.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.027+0000 7f221bfff700 1 --2- 172.21.15.2:0/876567668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f222c101ed0 0x7f222c0730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:39:59.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.027+0000 7f222affd700 1 --2- 172.21.15.2:0/876567668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f222c100ce0 0x7f222c074a60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f221c00b770 tx=0x7f221c008f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:59.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.027+0000 7f2228ff9700 1 -- 172.21.15.2:0/876567668 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c00e040 con 0x7f222c100ce0 2023-09-13T04:39:59.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.027+0000 7f2228ff9700 1 -- 172.21.15.2:0/876567668 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f221c003c00 con 0x7f222c100ce0 2023-09-13T04:39:59.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.027+0000 7f2228ff9700 1 -- 172.21.15.2:0/876567668 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c01d6d0 con 0x7f222c100ce0 2023-09-13T04:39:59.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.028+0000 7f22313ff700 1 -- 172.21.15.2:0/876567668 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f222c1a0e70 con 0x7f222c100ce0 2023-09-13T04:39:59.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.028+0000 7f22313ff700 1 -- 172.21.15.2:0/876567668 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222c1a11e0 con 0x7f222c100ce0 2023-09-13T04:39:59.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.029+0000 7f22313ff700 1 -- 172.21.15.2:0/876567668 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f222c107c60 con 0x7f222c100ce0 2023-09-13T04:39:59.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.031+0000 7f2228ff9700 1 -- 172.21.15.2:0/876567668 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f221c026400 con 0x7f222c100ce0 2023-09-13T04:39:59.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.032+0000 7f2228ff9700 1 --2- 172.21.15.2:0/876567668 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f22140a0790 0x7f22140a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:59.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.033+0000 7f2228ff9700 1 -- 172.21.15.2:0/876567668 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f221c0cce20 con 0x7f222c100ce0 2023-09-13T04:39:59.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.033+0000 7f221bfff700 1 --2- 172.21.15.2:0/876567668 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f22140a0790 0x7f22140a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:59.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.033+0000 7f221bfff700 1 --2- 172.21.15.2:0/876567668 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f22140a0790 0x7f22140a2c50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2220009cb0 tx=0x7f2220009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:59.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.037+0000 7f2228ff9700 1 -- 172.21.15.2:0/876567668 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f221c0966e0 con 0x7f222c100ce0 2023-09-13T04:39:59.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.250+0000 7f22313ff700 1 -- 172.21.15.2:0/876567668 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f222c02ce90 con 0x7f222c100ce0 2023-09-13T04:39:59.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.250+0000 7f2228ff9700 1 -- 172.21.15.2:0/876567668 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f221c099d00 con 0x7f222c100ce0 2023-09-13T04:39:59.252 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-09-13T04:39:59.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.253+0000 7f22313ff700 1 -- 172.21.15.2:0/876567668 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f22140a0790 msgr2=0x7f22140a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:59.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.253+0000 7f22313ff700 1 --2- 172.21.15.2:0/876567668 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f22140a0790 0x7f22140a2c50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2220009cb0 tx=0x7f2220009260 comp rx=0 tx=0).stop 2023-09-13T04:39:59.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.253+0000 7f22313ff700 1 -- 172.21.15.2:0/876567668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f222c100ce0 msgr2=0x7f222c074a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:59.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.253+0000 7f22313ff700 1 --2- 172.21.15.2:0/876567668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f222c100ce0 0x7f222c074a60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f221c00b770 tx=0x7f221c008f70 comp rx=0 tx=0).stop 2023-09-13T04:39:59.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.253+0000 7f22313ff700 1 -- 172.21.15.2:0/876567668 shutdown_connections 2023-09-13T04:39:59.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.253+0000 7f22313ff700 1 --2- 172.21.15.2:0/876567668 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f22140a0790 0x7f22140a2c50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.253+0000 7f22313ff700 1 --2- 172.21.15.2:0/876567668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f222c102870 0x7f222c0735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.253+0000 7f22313ff700 1 --2- 172.21.15.2:0/876567668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f222c100ce0 0x7f222c074a60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.253+0000 7f22313ff700 1 --2- 172.21.15.2:0/876567668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f222c101ed0 0x7f222c0730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.253+0000 7f22313ff700 1 -- 172.21.15.2:0/876567668 >> 172.21.15.2:0/876567668 conn(0x7f222c0fc230 msgr2=0x7f222c0fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:59.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.254+0000 7f22313ff700 1 -- 172.21.15.2:0/876567668 shutdown_connections 2023-09-13T04:39:59.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.254+0000 7f22313ff700 1 -- 172.21.15.2:0/876567668 wait complete. 2023-09-13T04:39:59.275 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-13T04:39:59.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.481+0000 7fbb17bff700 1 -- 172.21.15.2:0/2182459698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb10101ee0 msgr2=0x7fbb10102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:59.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.481+0000 7fbb17bff700 1 --2- 172.21.15.2:0/2182459698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb10101ee0 0x7fbb10102340 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbb000099c0 tx=0x7fbb00009cd0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.481+0000 7fbb17bff700 1 -- 172.21.15.2:0/2182459698 shutdown_connections 2023-09-13T04:39:59.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.481+0000 7fbb17bff700 1 --2- 172.21.15.2:0/2182459698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb10102880 0x7fbb10109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.481+0000 7fbb17bff700 1 --2- 172.21.15.2:0/2182459698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb10100cf0 0x7fbb10101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.481+0000 7fbb17bff700 1 --2- 172.21.15.2:0/2182459698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb10101ee0 0x7fbb10102340 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.481+0000 7fbb17bff700 1 -- 172.21.15.2:0/2182459698 >> 172.21.15.2:0/2182459698 conn(0x7fbb100fc260 msgr2=0x7fbb100fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:59.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.482+0000 7fbb17bff700 1 -- 172.21.15.2:0/2182459698 shutdown_connections 2023-09-13T04:39:59.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.482+0000 7fbb17bff700 1 -- 172.21.15.2:0/2182459698 wait complete. 2023-09-13T04:39:59.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.483+0000 7fbb17bff700 1 Processor -- start 2023-09-13T04:39:59.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.483+0000 7fbb17bff700 1 -- start start 2023-09-13T04:39:59.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.483+0000 7fbb17bff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb10100cf0 0x7fbb10198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:59.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.483+0000 7fbb17bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb10101ee0 0x7fbb10199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:59.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.483+0000 7fbb17bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb10102880 0x7fbb101a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:59.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.483+0000 7fbb17bff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb101a0760 con 0x7fbb10100cf0 2023-09-13T04:39:59.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.483+0000 7fbb17bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb101a08d0 con 0x7fbb10101ee0 2023-09-13T04:39:59.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.483+0000 7fbb17bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb101a0a90 con 0x7fbb10102880 2023-09-13T04:39:59.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.483+0000 7fbb1599b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb10100cf0 0x7fbb10198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:59.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.483+0000 7fbb1599b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb10100cf0 0x7fbb10198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58080/0 (socket says 172.21.15.2:58080) 2023-09-13T04:39:59.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.483+0000 7fbb1599b700 1 -- 172.21.15.2:0/4285792526 learned_addr learned my addr 172.21.15.2:0/4285792526 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:59.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.483+0000 7fbb1519a700 1 --2- 172.21.15.2:0/4285792526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb10101ee0 0x7fbb10199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:59.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.483+0000 7fbb1619c700 1 --2- 172.21.15.2:0/4285792526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb10102880 0x7fbb101a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:59.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.484+0000 7fbb1599b700 1 -- 172.21.15.2:0/4285792526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb10102880 msgr2=0x7fbb101a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:59.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.484+0000 7fbb1599b700 1 --2- 172.21.15.2:0/4285792526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb10102880 0x7fbb101a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.484+0000 7fbb1599b700 1 -- 172.21.15.2:0/4285792526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb10101ee0 msgr2=0x7fbb10199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:59.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.484+0000 7fbb1599b700 1 --2- 172.21.15.2:0/4285792526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb10101ee0 0x7fbb10199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.484+0000 7fbb1599b700 1 -- 172.21.15.2:0/4285792526 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb000095f0 con 0x7fbb10100cf0 2023-09-13T04:39:59.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.484+0000 7fbb1599b700 1 --2- 172.21.15.2:0/4285792526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb10100cf0 0x7fbb10198c20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00e970 tx=0x7fbb0c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:59.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.484+0000 7fbb06ffd700 1 -- 172.21.15.2:0/4285792526 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c009790 con 0x7fbb10100cf0 2023-09-13T04:39:59.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.484+0000 7fbb06ffd700 1 -- 172.21.15.2:0/4285792526 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb0c00eec0 con 0x7fbb10100cf0 2023-09-13T04:39:59.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.484+0000 7fbb17bff700 1 -- 172.21.15.2:0/4285792526 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb101a0d70 con 0x7fbb10100cf0 2023-09-13T04:39:59.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.484+0000 7fbb06ffd700 1 -- 172.21.15.2:0/4285792526 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c003cf0 con 0x7fbb10100cf0 2023-09-13T04:39:59.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.484+0000 7fbb17bff700 1 -- 172.21.15.2:0/4285792526 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb101a1210 con 0x7fbb10100cf0 2023-09-13T04:39:59.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.485+0000 7fbb17bff700 1 -- 172.21.15.2:0/4285792526 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1010c550 con 0x7fbb10100cf0 2023-09-13T04:39:59.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.487+0000 7fbb06ffd700 1 -- 172.21.15.2:0/4285792526 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fbb0c00ccd0 con 0x7fbb10100cf0 2023-09-13T04:39:59.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.489+0000 7fbb06ffd700 1 --2- 172.21.15.2:0/4285792526 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fbafc0a06c0 0x7fbafc0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:59.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.489+0000 7fbb06ffd700 1 -- 172.21.15.2:0/4285792526 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbb0c014070 con 0x7fbb10100cf0 2023-09-13T04:39:59.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.489+0000 7fbb1519a700 1 --2- 172.21.15.2:0/4285792526 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fbafc0a06c0 0x7fbafc0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:59.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.490+0000 7fbb1519a700 1 --2- 172.21.15.2:0/4285792526 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fbafc0a06c0 0x7fbafc0a2b80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000b580 tx=0x7fbb00005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:59.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.493+0000 7fbb06ffd700 1 -- 172.21.15.2:0/4285792526 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fbb0c095890 con 0x7fbb10100cf0 2023-09-13T04:39:59.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.705+0000 7fbb17bff700 1 -- 172.21.15.2:0/4285792526 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbb101a1460 con 0x7fbb10100cf0 2023-09-13T04:39:59.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.705+0000 7fbb06ffd700 1 -- 172.21.15.2:0/4285792526 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbb0c098eb0 con 0x7fbb10100cf0 2023-09-13T04:39:59.707 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-09-13T04:39:59.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.708+0000 7fbb17bff700 1 -- 172.21.15.2:0/4285792526 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fbafc0a06c0 msgr2=0x7fbafc0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:59.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.708+0000 7fbb17bff700 1 --2- 172.21.15.2:0/4285792526 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fbafc0a06c0 0x7fbafc0a2b80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000b580 tx=0x7fbb00005fb0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.708+0000 7fbb17bff700 1 -- 172.21.15.2:0/4285792526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb10100cf0 msgr2=0x7fbb10198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:59.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.708+0000 7fbb17bff700 1 --2- 172.21.15.2:0/4285792526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb10100cf0 0x7fbb10198c20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00e970 tx=0x7fbb0c00ec80 comp rx=0 tx=0).stop 2023-09-13T04:39:59.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.708+0000 7fbb17bff700 1 -- 172.21.15.2:0/4285792526 shutdown_connections 2023-09-13T04:39:59.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.708+0000 7fbb17bff700 1 --2- 172.21.15.2:0/4285792526 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fbafc0a06c0 0x7fbafc0a2b80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.708+0000 7fbb17bff700 1 --2- 172.21.15.2:0/4285792526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb10102880 0x7fbb101a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.708+0000 7fbb17bff700 1 --2- 172.21.15.2:0/4285792526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb10100cf0 0x7fbb10198c20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.708+0000 7fbb17bff700 1 --2- 172.21.15.2:0/4285792526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb10101ee0 0x7fbb10199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.708+0000 7fbb17bff700 1 -- 172.21.15.2:0/4285792526 >> 172.21.15.2:0/4285792526 conn(0x7fbb100fc260 msgr2=0x7fbb100fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:59.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.708+0000 7fbb17bff700 1 -- 172.21.15.2:0/4285792526 shutdown_connections 2023-09-13T04:39:59.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.708+0000 7fbb17bff700 1 -- 172.21.15.2:0/4285792526 wait complete. 2023-09-13T04:39:59.728 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:39:59.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:59 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1764400678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:39:59.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:39:59 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/876567668' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:39:59.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.931+0000 7eff207ff700 1 -- 172.21.15.2:0/120265608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff18100cf0 msgr2=0x7eff18101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:59.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.931+0000 7eff207ff700 1 --2- 172.21.15.2:0/120265608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff18100cf0 0x7eff18101110 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7eff14009990 tx=0x7eff14009ca0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.931+0000 7eff207ff700 1 -- 172.21.15.2:0/120265608 shutdown_connections 2023-09-13T04:39:59.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.931+0000 7eff207ff700 1 --2- 172.21.15.2:0/120265608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff18102880 0x7eff18109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.931+0000 7eff207ff700 1 --2- 172.21.15.2:0/120265608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff18100cf0 0x7eff18101110 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.931+0000 7eff207ff700 1 --2- 172.21.15.2:0/120265608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff18101ee0 0x7eff18102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.931+0000 7eff207ff700 1 -- 172.21.15.2:0/120265608 >> 172.21.15.2:0/120265608 conn(0x7eff180fc220 msgr2=0x7eff180fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:39:59.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.931+0000 7eff207ff700 1 -- 172.21.15.2:0/120265608 shutdown_connections 2023-09-13T04:39:59.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.931+0000 7eff207ff700 1 -- 172.21.15.2:0/120265608 wait complete. 2023-09-13T04:39:59.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.932+0000 7eff207ff700 1 Processor -- start 2023-09-13T04:39:59.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.932+0000 7eff207ff700 1 -- start start 2023-09-13T04:39:59.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.932+0000 7eff207ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff18100cf0 0x7eff18198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:59.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.932+0000 7eff207ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff18101ee0 0x7eff181991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:59.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.932+0000 7eff207ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff18102880 0x7eff181a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:59.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.932+0000 7eff207ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff181a07b0 con 0x7eff18101ee0 2023-09-13T04:39:59.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.932+0000 7eff207ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff181a0920 con 0x7eff18100cf0 2023-09-13T04:39:59.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.932+0000 7eff207ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff181a0ae0 con 0x7eff18102880 2023-09-13T04:39:59.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.933+0000 7eff1dd9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff18101ee0 0x7eff181991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:59.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.933+0000 7eff1dd9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff18101ee0 0x7eff181991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58098/0 (socket says 172.21.15.2:58098) 2023-09-13T04:39:59.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.933+0000 7eff1dd9a700 1 -- 172.21.15.2:0/3610722467 learned_addr learned my addr 172.21.15.2:0/3610722467 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:39:59.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.933+0000 7eff1ed9c700 1 --2- 172.21.15.2:0/3610722467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff18102880 0x7eff181a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:59.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.933+0000 7eff1e59b700 1 --2- 172.21.15.2:0/3610722467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff18100cf0 0x7eff18198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:59.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.933+0000 7eff1dd9a700 1 -- 172.21.15.2:0/3610722467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff18102880 msgr2=0x7eff181a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:59.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.933+0000 7eff1dd9a700 1 --2- 172.21.15.2:0/3610722467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff18102880 0x7eff181a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.933+0000 7eff1dd9a700 1 -- 172.21.15.2:0/3610722467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff18100cf0 msgr2=0x7eff18198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:39:59.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.933+0000 7eff1dd9a700 1 --2- 172.21.15.2:0/3610722467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff18100cf0 0x7eff18198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:39:59.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.933+0000 7eff1dd9a700 1 -- 172.21.15.2:0/3610722467 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff10009c40 con 0x7eff18101ee0 2023-09-13T04:39:59.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.933+0000 7eff1dd9a700 1 --2- 172.21.15.2:0/3610722467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff18101ee0 0x7eff181991b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7eff0800e9c0 tx=0x7eff0800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:59.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.934+0000 7eff0f7fe700 1 -- 172.21.15.2:0/3610722467 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff080097b0 con 0x7eff18101ee0 2023-09-13T04:39:59.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.934+0000 7eff207ff700 1 -- 172.21.15.2:0/3610722467 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff140095f0 con 0x7eff18101ee0 2023-09-13T04:39:59.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.934+0000 7eff0f7fe700 1 -- 172.21.15.2:0/3610722467 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff0800ee50 con 0x7eff18101ee0 2023-09-13T04:39:59.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.934+0000 7eff0f7fe700 1 -- 172.21.15.2:0/3610722467 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff080054f0 con 0x7eff18101ee0 2023-09-13T04:39:59.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.934+0000 7eff207ff700 1 -- 172.21.15.2:0/3610722467 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff181a1140 con 0x7eff18101ee0 2023-09-13T04:39:59.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.935+0000 7eff207ff700 1 -- 172.21.15.2:0/3610722467 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff18193360 con 0x7eff18101ee0 2023-09-13T04:39:59.942 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:59 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1764400678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:39:59.942 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:39:59 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/876567668' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:39:59.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.937+0000 7eff0f7fe700 1 -- 172.21.15.2:0/3610722467 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7eff08004750 con 0x7eff18101ee0 2023-09-13T04:39:59.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.938+0000 7eff0f7fe700 1 --2- 172.21.15.2:0/3610722467 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7eff040a0690 0x7eff040a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:39:59.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.938+0000 7eff0f7fe700 1 -- 172.21.15.2:0/3610722467 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7eff08014070 con 0x7eff18101ee0 2023-09-13T04:39:59.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.939+0000 7eff1e59b700 1 --2- 172.21.15.2:0/3610722467 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7eff040a0690 0x7eff040a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:39:59.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.939+0000 7eff1e59b700 1 --2- 172.21.15.2:0/3610722467 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7eff040a0690 0x7eff040a2b50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7eff14000c00 tx=0x7eff14005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:39:59.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:39:59.942+0000 7eff0f7fe700 1 -- 172.21.15.2:0/3610722467 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7eff08095940 con 0x7eff18101ee0 2023-09-13T04:40:00.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:59 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1764400678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:40:00.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:39:59 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/876567668' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:40:00.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.199+0000 7eff207ff700 1 -- 172.21.15.2:0/3610722467 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff1802cf30 con 0x7eff18101ee0 2023-09-13T04:40:00.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.200+0000 7eff0f7fe700 1 -- 172.21.15.2:0/3610722467 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11303 (secure 0 0 0) 0x7eff08098f60 con 0x7eff18101ee0 2023-09-13T04:40:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:40:00.202 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2469307307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2469307307},{"type":"v1","addr":"172.21.15.2:6839","nonce":2469307307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T04:37:59.110701+0000","modified":"2023-09-13T04:38:26.610260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24554,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/846579077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":846579077},{"type":"v1","addr":"172.21.15.156:6835","nonce":846579077}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1896833690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1896833690},{"type":"v1","addr":"172.21.15.149:6837","nonce":1896833690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/524563090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":524563090},{"type":"v1","addr":"172.21.15.156:6837","nonce":524563090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6839/1785833227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1785833227},{"type":"v1","addr":"172.21.15.156:6839","nonce":1785833227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.2:6841/482545391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":482545391},{"type":"v1","addr":"172.21.15.2:6841","nonce":482545391}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/389167841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":389167841},{"type":"v1","addr":"172.21.15.149:6839","nonce":389167841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T04:40:00.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.203+0000 7eff207ff700 1 -- 172.21.15.2:0/3610722467 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7eff040a0690 msgr2=0x7eff040a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:00.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.203+0000 7eff207ff700 1 --2- 172.21.15.2:0/3610722467 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7eff040a0690 0x7eff040a2b50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7eff14000c00 tx=0x7eff14005fb0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.203+0000 7eff207ff700 1 -- 172.21.15.2:0/3610722467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff18101ee0 msgr2=0x7eff181991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:00.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.203+0000 7eff207ff700 1 --2- 172.21.15.2:0/3610722467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff18101ee0 0x7eff181991b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7eff0800e9c0 tx=0x7eff0800ecd0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.203+0000 7eff207ff700 1 -- 172.21.15.2:0/3610722467 shutdown_connections 2023-09-13T04:40:00.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.203+0000 7eff207ff700 1 --2- 172.21.15.2:0/3610722467 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7eff040a0690 0x7eff040a2b50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.203+0000 7eff207ff700 1 --2- 172.21.15.2:0/3610722467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff18102880 0x7eff181a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.203+0000 7eff207ff700 1 --2- 172.21.15.2:0/3610722467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff18100cf0 0x7eff18198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.203+0000 7eff207ff700 1 --2- 172.21.15.2:0/3610722467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff18101ee0 0x7eff181991b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.203+0000 7eff207ff700 1 -- 172.21.15.2:0/3610722467 >> 172.21.15.2:0/3610722467 conn(0x7eff180fc220 msgr2=0x7eff18107820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:00.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.203+0000 7eff207ff700 1 -- 172.21.15.2:0/3610722467 shutdown_connections 2023-09-13T04:40:00.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.203+0000 7eff207ff700 1 -- 172.21.15.2:0/3610722467 wait complete. 2023-09-13T04:40:00.207 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 23 2023-09-13T04:40:00.223 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T04:40:00.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.434+0000 7f528ffff700 1 -- 172.21.15.2:0/3239569872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5288100cf0 msgr2=0x7f5288101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:00.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.434+0000 7f528ffff700 1 --2- 172.21.15.2:0/3239569872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5288100cf0 0x7f5288101110 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f52840099c0 tx=0x7f5284009cd0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.434+0000 7f528ffff700 1 -- 172.21.15.2:0/3239569872 shutdown_connections 2023-09-13T04:40:00.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.434+0000 7f528ffff700 1 --2- 172.21.15.2:0/3239569872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5288102880 0x7f5288109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.434+0000 7f528ffff700 1 --2- 172.21.15.2:0/3239569872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5288100cf0 0x7f5288101110 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.434+0000 7f528ffff700 1 --2- 172.21.15.2:0/3239569872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5288101ee0 0x7f5288102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.434+0000 7f528ffff700 1 -- 172.21.15.2:0/3239569872 >> 172.21.15.2:0/3239569872 conn(0x7f52880fc220 msgr2=0x7f52880fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:00.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.434+0000 7f528ffff700 1 -- 172.21.15.2:0/3239569872 shutdown_connections 2023-09-13T04:40:00.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.434+0000 7f528ffff700 1 -- 172.21.15.2:0/3239569872 wait complete. 2023-09-13T04:40:00.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.436+0000 7f528ffff700 1 Processor -- start 2023-09-13T04:40:00.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.436+0000 7f528ffff700 1 -- start start 2023-09-13T04:40:00.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.436+0000 7f528ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5288100cf0 0x7f5288198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:00.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.436+0000 7f528ffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5288101ee0 0x7f5288199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:00.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.436+0000 7f528ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5288102880 0x7f52881a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:00.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.436+0000 7f528ffff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52881a0770 con 0x7f5288102880 2023-09-13T04:40:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.436+0000 7f528ffff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52881a08b0 con 0x7f5288101ee0 2023-09-13T04:40:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.436+0000 7f528ffff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52881a0a40 con 0x7f5288100cf0 2023-09-13T04:40:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.436+0000 7f528e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5288102880 0x7f52881a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.436+0000 7f528e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5288102880 0x7f52881a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58122/0 (socket says 172.21.15.2:58122) 2023-09-13T04:40:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.436+0000 7f528e59c700 1 -- 172.21.15.2:0/1276086351 learned_addr learned my addr 172.21.15.2:0/1276086351 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:00.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.437+0000 7f528dd9b700 1 --2- 172.21.15.2:0/1276086351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5288100cf0 0x7f5288198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:00.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.437+0000 7f528d59a700 1 --2- 172.21.15.2:0/1276086351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5288101ee0 0x7f5288199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:00.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.437+0000 7f528e59c700 1 -- 172.21.15.2:0/1276086351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5288100cf0 msgr2=0x7f5288198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:00.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.437+0000 7f528e59c700 1 --2- 172.21.15.2:0/1276086351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5288100cf0 0x7f5288198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.437+0000 7f528e59c700 1 -- 172.21.15.2:0/1276086351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5288101ee0 msgr2=0x7f5288199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:00.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.437+0000 7f528e59c700 1 --2- 172.21.15.2:0/1276086351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5288101ee0 0x7f5288199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.437+0000 7f528e59c700 1 -- 172.21.15.2:0/1276086351 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52840095f0 con 0x7f5288102880 2023-09-13T04:40:00.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.437+0000 7f528e59c700 1 --2- 172.21.15.2:0/1276086351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5288102880 0x7f52881a01a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5280007ee0 tx=0x7f528000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:00.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.437+0000 7f527effd700 1 -- 172.21.15.2:0/1276086351 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528000dd20 con 0x7f5288102880 2023-09-13T04:40:00.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.437+0000 7f527effd700 1 -- 172.21.15.2:0/1276086351 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f528000f040 con 0x7f5288102880 2023-09-13T04:40:00.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.437+0000 7f528ffff700 1 -- 172.21.15.2:0/1276086351 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52881a0d20 con 0x7f5288102880 2023-09-13T04:40:00.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.438+0000 7f527effd700 1 -- 172.21.15.2:0/1276086351 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52800048a0 con 0x7f5288102880 2023-09-13T04:40:00.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.438+0000 7f528ffff700 1 -- 172.21.15.2:0/1276086351 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52881a11d0 con 0x7f5288102880 2023-09-13T04:40:00.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.439+0000 7f528ffff700 1 -- 172.21.15.2:0/1276086351 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5288193340 con 0x7f5288102880 2023-09-13T04:40:00.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.441+0000 7f527effd700 1 -- 172.21.15.2:0/1276086351 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f5280003a00 con 0x7f5288102880 2023-09-13T04:40:00.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.442+0000 7f527effd700 1 --2- 172.21.15.2:0/1276086351 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f52740a0710 0x7f52740a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:00.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.442+0000 7f527effd700 1 -- 172.21.15.2:0/1276086351 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f52800cb710 con 0x7f5288102880 2023-09-13T04:40:00.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.442+0000 7f528dd9b700 1 --2- 172.21.15.2:0/1276086351 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f52740a0710 0x7f52740a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:00.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.443+0000 7f528dd9b700 1 --2- 172.21.15.2:0/1276086351 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f52740a0710 0x7f52740a2bd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5284009990 tx=0x7f5284005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:00.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.446+0000 7f527effd700 1 -- 172.21.15.2:0/1276086351 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f528004ee10 con 0x7f5288102880 2023-09-13T04:40:00.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.662+0000 7f528ffff700 1 -- 172.21.15.2:0/1276086351 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f528810c550 con 0x7f5288102880 2023-09-13T04:40:00.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.663+0000 7f527effd700 1 -- 172.21.15.2:0/1276086351 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24503 (secure 0 0 0) 0x7f5280094eb0 con 0x7f5288102880 2023-09-13T04:40:00.665 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:40:00.665 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":92,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:39:25.580735+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T04:37:57.094802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T04:37:58.564487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-13T04:38:02.096895+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:40:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.666+0000 7f528ffff700 1 -- 172.21.15.2:0/1276086351 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f52740a0710 msgr2=0x7f52740a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.666+0000 7f528ffff700 1 --2- 172.21.15.2:0/1276086351 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f52740a0710 0x7f52740a2bd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5284009990 tx=0x7f5284005fb0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.666+0000 7f528ffff700 1 -- 172.21.15.2:0/1276086351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5288102880 msgr2=0x7f52881a01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.666+0000 7f528ffff700 1 --2- 172.21.15.2:0/1276086351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5288102880 0x7f52881a01a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5280007ee0 tx=0x7f528000d3b0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.666+0000 7f528ffff700 1 -- 172.21.15.2:0/1276086351 shutdown_connections 2023-09-13T04:40:00.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.666+0000 7f528ffff700 1 --2- 172.21.15.2:0/1276086351 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f52740a0710 0x7f52740a2bd0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.666+0000 7f528ffff700 1 --2- 172.21.15.2:0/1276086351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5288102880 0x7f52881a01a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.666+0000 7f528ffff700 1 --2- 172.21.15.2:0/1276086351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5288100cf0 0x7f5288198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.666+0000 7f528ffff700 1 --2- 172.21.15.2:0/1276086351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5288101ee0 0x7f5288199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.666+0000 7f528ffff700 1 -- 172.21.15.2:0/1276086351 >> 172.21.15.2:0/1276086351 conn(0x7f52880fc220 msgr2=0x7f52881077b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:00.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.666+0000 7f528ffff700 1 -- 172.21.15.2:0/1276086351 shutdown_connections 2023-09-13T04:40:00.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.666+0000 7f528ffff700 1 -- 172.21.15.2:0/1276086351 wait complete. 2023-09-13T04:40:00.687 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-09-13T04:37:59.110701+0000', 'modified': '2023-09-13T04:38:26.610260+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24566, 'mds_1': 24554, 'mds_2': 24541}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24511': {'gid': 24511, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6835/846579077', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6834', 'nonce': 846579077}, {'type': 'v1', 'addr': '172.21.15.156:6835', 'nonce': 846579077}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24521': {'gid': 24521, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6837/1896833690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 1896833690}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 1896833690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24523': {'gid': 24523, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6837/524563090', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6836', 'nonce': 524563090}, {'type': 'v1', 'addr': '172.21.15.156:6837', 'nonce': 524563090}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24541': {'gid': 24541, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.156:6839/1785833227', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6838', 'nonce': 1785833227}, {'type': 'v1', 'addr': '172.21.15.156:6839', 'nonce': 1785833227}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24554': {'gid': 24554, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.2:6841/482545391', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6840', 'nonce': 482545391}, {'type': 'v1', 'addr': '172.21.15.2:6841', 'nonce': 482545391}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24566': {'gid': 24566, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.149:6839/389167841', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 389167841}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 389167841}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-13T04:40:00.687 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-09-13T04:40:00.688 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-09-13T04:40:00.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:00 smithi149 ceph-mon[133315]: pgmap v340: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:40:00.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:00 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/4285792526' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:40:00.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:00 smithi149 ceph-mon[133315]: overall HEALTH_OK 2023-09-13T04:40:00.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:00 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3610722467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:40:00.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.892+0000 7f1becbff700 1 -- 172.21.15.2:0/3891027767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1be8102870 msgr2=0x7f1be8109120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:00.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.892+0000 7f1becbff700 1 --2- 172.21.15.2:0/3891027767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1be8102870 0x7f1be8109120 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1be000b0d0 tx=0x7f1be000b3e0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.892+0000 7f1becbff700 1 -- 172.21.15.2:0/3891027767 shutdown_connections 2023-09-13T04:40:00.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.892+0000 7f1becbff700 1 --2- 172.21.15.2:0/3891027767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1be8102870 0x7f1be8109120 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.892+0000 7f1becbff700 1 --2- 172.21.15.2:0/3891027767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1be8100ce0 0x7f1be8101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.892+0000 7f1becbff700 1 --2- 172.21.15.2:0/3891027767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1be8101ed0 0x7f1be8102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.892+0000 7f1becbff700 1 -- 172.21.15.2:0/3891027767 >> 172.21.15.2:0/3891027767 conn(0x7f1be80fc230 msgr2=0x7f1be80fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:00.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.892+0000 7f1becbff700 1 -- 172.21.15.2:0/3891027767 shutdown_connections 2023-09-13T04:40:00.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.892+0000 7f1becbff700 1 -- 172.21.15.2:0/3891027767 wait complete. 2023-09-13T04:40:00.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.893+0000 7f1becbff700 1 Processor -- start 2023-09-13T04:40:00.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.893+0000 7f1becbff700 1 -- start start 2023-09-13T04:40:00.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.894+0000 7f1becbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1be8100ce0 0x7f1be8198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:00.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.894+0000 7f1becbff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1be8101ed0 0x7f1be8199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:00.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.894+0000 7f1becbff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1be8102870 0x7f1be81a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:00.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.894+0000 7f1becbff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be81a0830 con 0x7f1be8101ed0 2023-09-13T04:40:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.894+0000 7f1becbff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be81a09a0 con 0x7f1be8102870 2023-09-13T04:40:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.894+0000 7f1becbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be81a0b60 con 0x7f1be8100ce0 2023-09-13T04:40:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.894+0000 7f1be5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1be8101ed0 0x7f1be8199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.894+0000 7f1be5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1be8101ed0 0x7f1be8199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58132/0 (socket says 172.21.15.2:58132) 2023-09-13T04:40:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.894+0000 7f1be5d9b700 1 -- 172.21.15.2:0/1557837928 learned_addr learned my addr 172.21.15.2:0/1557837928 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:00.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.894+0000 7f1be659c700 1 --2- 172.21.15.2:0/1557837928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1be8100ce0 0x7f1be8198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:00.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.894+0000 7f1be6d9d700 1 --2- 172.21.15.2:0/1557837928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1be8102870 0x7f1be81a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:00.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.894+0000 7f1be5d9b700 1 -- 172.21.15.2:0/1557837928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1be8100ce0 msgr2=0x7f1be8198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:00.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.894+0000 7f1be5d9b700 1 --2- 172.21.15.2:0/1557837928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1be8100ce0 0x7f1be8198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.894+0000 7f1be5d9b700 1 -- 172.21.15.2:0/1557837928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1be8102870 msgr2=0x7f1be81a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:00.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.894+0000 7f1be5d9b700 1 --2- 172.21.15.2:0/1557837928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1be8102870 0x7f1be81a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.894+0000 7f1be5d9b700 1 -- 172.21.15.2:0/1557837928 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd8009520 con 0x7f1be8101ed0 2023-09-13T04:40:00.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.895+0000 7f1be5d9b700 1 --2- 172.21.15.2:0/1557837928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1be8101ed0 0x7f1be8199230 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00e9a0 tx=0x7f1bdc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:00.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.895+0000 7f1bd77fe700 1 -- 172.21.15.2:0/1557837928 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc009790 con 0x7f1be8101ed0 2023-09-13T04:40:00.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.895+0000 7f1bd77fe700 1 -- 172.21.15.2:0/1557837928 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bdc010430 con 0x7f1be8101ed0 2023-09-13T04:40:00.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.895+0000 7f1becbff700 1 -- 172.21.15.2:0/1557837928 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be0009d00 con 0x7f1be8101ed0 2023-09-13T04:40:00.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.895+0000 7f1bd77fe700 1 -- 172.21.15.2:0/1557837928 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc018680 con 0x7f1be8101ed0 2023-09-13T04:40:00.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.895+0000 7f1becbff700 1 -- 172.21.15.2:0/1557837928 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be81a11c0 con 0x7f1be8101ed0 2023-09-13T04:40:00.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.898+0000 7f1bd77fe700 1 -- 172.21.15.2:0/1557837928 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f1bdc0105a0 con 0x7f1be8101ed0 2023-09-13T04:40:00.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.900+0000 7f1c004a0b80 1 Processor -- start 2023-09-13T04:40:00.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.901+0000 7f1c004a0b80 1 -- start start 2023-09-13T04:40:00.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.901+0000 7f1bd77fe700 1 --2- 172.21.15.2:0/1557837928 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1bd00a0640 0x7f1bd00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:00.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.901+0000 7f1c004a0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bef53b6b0 0x555bef53bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:00.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.901+0000 7f1c004a0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53c010 0x555bef53a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:00.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.901+0000 7f1c004a0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bef53a9b0 0x555bef53ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:00.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.901+0000 7f1c004a0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bef493730 con 0x555bef53c010 2023-09-13T04:40:00.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.901+0000 7f1c004a0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bef491050 con 0x555bef53b6b0 2023-09-13T04:40:00.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.901+0000 7f1c004a0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bef48c940 con 0x555bef53a9b0 2023-09-13T04:40:00.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.901+0000 7f1bd77fe700 1 -- 172.21.15.2:0/1557837928 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1bdc014070 con 0x7f1be8101ed0 2023-09-13T04:40:00.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.901+0000 7f1be5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53c010 0x555bef53a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:00.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.901+0000 7f1be6d9d700 1 --2- 172.21.15.2:0/1557837928 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1bd00a0640 0x7f1bd00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:00.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.901+0000 7f1be5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53c010 0x555bef53a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58146/0 (socket says 172.21.15.2:58146) 2023-09-13T04:40:00.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.901+0000 7f1be659c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bef53b6b0 0x555bef53bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:00.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.901+0000 7f1be5d9b700 1 -- 172.21.15.2:0/3071826568 learned_addr learned my addr 172.21.15.2:0/3071826568 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:00.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.902+0000 7f1be6d9d700 1 --2- 172.21.15.2:0/3071826568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bef53a9b0 0x555bef53ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:00.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.902+0000 7f1be659c700 1 -- 172.21.15.2:0/3071826568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bef53a9b0 msgr2=0x555bef53ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:00.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.902+0000 7f1be659c700 1 --2- 172.21.15.2:0/3071826568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bef53a9b0 0x555bef53ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.902+0000 7f1be659c700 1 -- 172.21.15.2:0/3071826568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53c010 msgr2=0x555bef53a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:00.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.902+0000 7f1be659c700 1 --2- 172.21.15.2:0/3071826568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53c010 0x555bef53a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.902+0000 7f1be659c700 1 -- 172.21.15.2:0/3071826568 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bef30c5e0 con 0x555bef53b6b0 2023-09-13T04:40:00.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.902+0000 7f1be5d9b700 1 --2- 172.21.15.2:0/3071826568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53c010 0x555bef53a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:00.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.902+0000 7f1be6d9d700 1 --2- 172.21.15.2:0/1557837928 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1bd00a0640 0x7f1bd00a2b00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1be000b770 tx=0x7f1be0009b80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:00.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.902+0000 7f1be659c700 1 --2- 172.21.15.2:0/3071826568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bef53b6b0 0x555bef53bad0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1be8101d30 tx=0x7f1bd8011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8e13088ecf1918e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:00.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.902+0000 7f1bd4ff9700 1 -- 172.21.15.2:0/3071826568 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8004230 con 0x555bef53b6b0 2023-09-13T04:40:00.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.903+0000 7f1bd4ff9700 1 -- 172.21.15.2:0/3071826568 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bd8004cf0 con 0x555bef53b6b0 2023-09-13T04:40:00.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.903+0000 7f1bd4ff9700 1 -- 172.21.15.2:0/3071826568 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd80054a0 con 0x555bef53b6b0 2023-09-13T04:40:00.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.903+0000 7f1c004a0b80 1 -- 172.21.15.2:0/3071826568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bef53b6b0 msgr2=0x555bef53bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:00.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.903+0000 7f1c004a0b80 1 --2- 172.21.15.2:0/3071826568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bef53b6b0 0x555bef53bad0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1be8101d30 tx=0x7f1bd8011720 comp rx=0 tx=0).stop 2023-09-13T04:40:00.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.903+0000 7f1c004a0b80 1 -- 172.21.15.2:0/3071826568 shutdown_connections 2023-09-13T04:40:00.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.903+0000 7f1c004a0b80 1 --2- 172.21.15.2:0/3071826568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bef53a9b0 0x555bef53ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.903+0000 7f1c004a0b80 1 --2- 172.21.15.2:0/3071826568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bef53b6b0 0x555bef53bad0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.903+0000 7f1c004a0b80 1 --2- 172.21.15.2:0/3071826568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53c010 0x555bef53a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.903+0000 7f1c004a0b80 1 -- 172.21.15.2:0/3071826568 >> 172.21.15.2:0/3071826568 conn(0x555bef498c30 msgr2=0x555bef534ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:00.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.903+0000 7f1c004a0b80 1 -- 172.21.15.2:0/3071826568 shutdown_connections 2023-09-13T04:40:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.903+0000 7f1c004a0b80 1 -- 172.21.15.2:0/3071826568 wait complete. 2023-09-13T04:40:00.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.904+0000 7f1c004a0b80 1 -- start start 2023-09-13T04:40:00.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.904+0000 7f1c004a0b80 1 Processor -- start 2023-09-13T04:40:00.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.905+0000 7f1c004a0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bef53a9b0 0x555bef543e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:00.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.905+0000 7f1c004a0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53b6b0 0x555bef5443d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:00.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.905+0000 7f1c004a0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bef53c010 0x555bef54b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:00.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.905+0000 7f1c004a0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bef42ab60 con 0x555bef53b6b0 2023-09-13T04:40:00.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.905+0000 7f1c004a0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bef42a860 con 0x555bef53c010 2023-09-13T04:40:00.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.905+0000 7f1c004a0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bef30e8d0 con 0x555bef53a9b0 2023-09-13T04:40:00.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.905+0000 7f1c004a0b80 10 client.0 _ll_register_callbacks cb 0x555bef428630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:00.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.905+0000 7f1be5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53b6b0 0x555bef5443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:00.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.905+0000 7f1be5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53b6b0 0x555bef5443d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58162/0 (socket says 172.21.15.2:58162) 2023-09-13T04:40:00.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.905+0000 7f1be5d9b700 1 -- 172.21.15.2:0/2950829752 learned_addr learned my addr 172.21.15.2:0/2950829752 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:00.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.906+0000 7f1be659c700 1 --2- 172.21.15.2:0/2950829752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bef53a9b0 0x555bef543e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:00.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.906+0000 7f1be6d9d700 1 --2- 172.21.15.2:0/2950829752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bef53c010 0x555bef54b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:00.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.906+0000 7f1be5d9b700 1 -- 172.21.15.2:0/2950829752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bef53a9b0 msgr2=0x555bef543e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:00.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.906+0000 7f1be5d9b700 1 --2- 172.21.15.2:0/2950829752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bef53a9b0 0x555bef543e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.906+0000 7f1be5d9b700 1 -- 172.21.15.2:0/2950829752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bef53c010 msgr2=0x555bef54b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:00.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.906+0000 7f1be5d9b700 1 --2- 172.21.15.2:0/2950829752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bef53c010 0x555bef54b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.906+0000 7f1be5d9b700 1 -- 172.21.15.2:0/2950829752 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bd8009a70 con 0x555bef53b6b0 2023-09-13T04:40:00.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.906+0000 7f1be6d9d700 1 --2- 172.21.15.2:0/2950829752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bef53c010 0x555bef54b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:00.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.906+0000 7f1be5d9b700 1 --2- 172.21.15.2:0/2950829752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53b6b0 0x555bef5443d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc099230 tx=0x7f1bdc099a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b659028bc20b28b4 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:00.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.906+0000 7f1bcdffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:40:00.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.906+0000 7f1bcdffb700 1 -- 172.21.15.2:0/2950829752 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc099c30 con 0x555bef53b6b0 2023-09-13T04:40:00.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.906+0000 7f1c004a0b80 1 -- 172.21.15.2:0/2950829752 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555bef52d620 con 0x555bef53b6b0 2023-09-13T04:40:00.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.906+0000 7f1bcdffb700 1 -- 172.21.15.2:0/2950829752 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1bdc099530 con 0x555bef53b6b0 2023-09-13T04:40:00.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.907+0000 7f1bcdffb700 1 -- 172.21.15.2:0/2950829752 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bdc09d360 con 0x555bef53b6b0 2023-09-13T04:40:00.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.907+0000 7f1c004a0b80 10 client.15264 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:00.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.907+0000 7f1c004a0b80 1 -- 172.21.15.2:0/2950829752 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x555bef4a4750 con 0x555bef53b6b0 2023-09-13T04:40:00.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.907+0000 7f1bcdffb700 1 -- 172.21.15.2:0/2950829752 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555bef52d620 con 0x555bef53b6b0 2023-09-13T04:40:00.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.907+0000 7f1bcdffb700 1 -- 172.21.15.2:0/2950829752 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1bdc0a1110 con 0x555bef53b6b0 2023-09-13T04:40:00.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.907+0000 7f1c004a0b80 10 client.15264 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:00.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.907+0000 7f1c004a0b80 10 client.15264 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-13T04:40:00.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.907+0000 7f1c004a0b80 20 client.15264 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:00.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.907+0000 7f1c004a0b80 1 --2- 172.21.15.2:0/2950829752 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x555bef54fab0 0x555bef54e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:00.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.907+0000 7f1c004a0b80 4 client.15264 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:40:00.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.907+0000 7f1c004a0b80 1 -- 172.21.15.2:0/2950829752 --> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555bef485920 con 0x555bef54fab0 2023-09-13T04:40:00.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.908+0000 7f1be659c700 1 --2- 172.21.15.2:0/2950829752 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x555bef54fab0 0x555bef54e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:00.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.908+0000 7f1be659c700 1 --2- 172.21.15.2:0/2950829752 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x555bef54fab0 0x555bef54e520 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e17bacdb7747e9a server_cookie=96fa0859968e9cc7 in_seq=0 out_seq=0 2023-09-13T04:40:00.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.908+0000 7f1bcdffb700 10 client.15264 ms_handle_connect on v2:172.21.15.156:6838/1785833227 2023-09-13T04:40:00.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.909+0000 7f1bcdffb700 1 -- 172.21.15.2:0/2950829752 <== mds.2 v2:172.21.15.156:6838/1785833227 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555bef485920 con 0x555bef54fab0 2023-09-13T04:40:00.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.909+0000 7f1bcdffb700 10 client.15264 handle_command_reply: tid=0 2023-09-13T04:40:00.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.910+0000 7f1c004a0b80 1 client.15264 shutdown 2023-09-13T04:40:00.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.910+0000 7f1c004a0b80 1 -- 172.21.15.2:0/2950829752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53b6b0 msgr2=0x555bef5443d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:00.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.910+0000 7f1c004a0b80 1 --2- 172.21.15.2:0/2950829752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53b6b0 0x555bef5443d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc099230 tx=0x7f1bdc099a70 comp rx=0 tx=0).stop 2023-09-13T04:40:00.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.910+0000 7f1c004a0b80 1 -- 172.21.15.2:0/2950829752 shutdown_connections 2023-09-13T04:40:00.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.910+0000 7f1c004a0b80 1 --2- 172.21.15.2:0/2950829752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bef53c010 0x555bef54b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.910+0000 7f1c004a0b80 1 --2- 172.21.15.2:0/2950829752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bef53a9b0 0x555bef543e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.910+0000 7f1c004a0b80 1 --2- 172.21.15.2:0/2950829752 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x555bef54fab0 0x555bef54e520 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.910+0000 7f1c004a0b80 1 --2- 172.21.15.2:0/2950829752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53b6b0 0x555bef5443d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.910+0000 7f1c004a0b80 1 -- 172.21.15.2:0/2950829752 >> 172.21.15.2:0/2950829752 conn(0x555bef498c30 msgr2=0x555bef52f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:00.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.910+0000 7f1bcdffb700 0 client.15264 ms_handle_reset on v2:172.21.15.156:6838/1785833227 2023-09-13T04:40:00.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.910+0000 7f1c004a0b80 1 -- 172.21.15.2:0/2950829752 shutdown_connections 2023-09-13T04:40:00.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.910+0000 7f1c004a0b80 1 -- 172.21.15.2:0/2950829752 wait complete. 2023-09-13T04:40:00.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.910+0000 7f1c004a0b80 20 client.15264 trim_cache size 0 max 16384 2023-09-13T04:40:00.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.928+0000 7f1c004a0b80 1 Processor -- start 2023-09-13T04:40:00.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.928+0000 7f1c004a0b80 1 -- start start 2023-09-13T04:40:00.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.929+0000 7f1c004a0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bef53a9b0 0x555bef53a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:00.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.929+0000 7f1c004a0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53b6b0 0x555bef54b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:00.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.929+0000 7f1c004a0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bef53c010 0x555bef54e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:00.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.929+0000 7f1c004a0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bef5450b0 con 0x555bef53b6b0 2023-09-13T04:40:00.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.929+0000 7f1c004a0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bef44ce20 con 0x555bef53c010 2023-09-13T04:40:00.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.929+0000 7f1c004a0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bef448f00 con 0x555bef53a9b0 2023-09-13T04:40:00.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.929+0000 7f1be659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bef53a9b0 0x555bef53a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:00.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.929+0000 7f1be5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53b6b0 0x555bef54b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:00.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.929+0000 7f1be659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bef53a9b0 0x555bef53a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:57280/0 (socket says 172.21.15.2:57280) 2023-09-13T04:40:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.929+0000 7f1be659c700 1 -- 172.21.15.2:0/2605784019 learned_addr learned my addr 172.21.15.2:0/2605784019 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.929+0000 7f1be6d9d700 1 --2- 172.21.15.2:0/2605784019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bef53c010 0x555bef54e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.929+0000 7f1be5d9b700 1 -- 172.21.15.2:0/2605784019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bef53a9b0 msgr2=0x555bef53a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.929+0000 7f1be5d9b700 1 --2- 172.21.15.2:0/2605784019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bef53a9b0 0x555bef53a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.929+0000 7f1be5d9b700 1 -- 172.21.15.2:0/2605784019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bef53c010 msgr2=0x555bef54e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.929+0000 7f1be5d9b700 1 --2- 172.21.15.2:0/2605784019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bef53c010 0x555bef54e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.929+0000 7f1be5d9b700 1 -- 172.21.15.2:0/2605784019 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bef2b8d90 con 0x555bef53b6b0 2023-09-13T04:40:00.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.929+0000 7f1be6d9d700 1 --2- 172.21.15.2:0/2605784019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bef53c010 0x555bef54e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:40:00.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.929+0000 7f1be659c700 1 --2- 172.21.15.2:0/2605784019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bef53a9b0 0x555bef53a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:00.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.930+0000 7f1be5d9b700 1 --2- 172.21.15.2:0/2605784019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53b6b0 0x555bef54b3d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00c2d0 tx=0x7f1bdc097ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7905dce4fbbccd94 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:00.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.930+0000 7f1bd57fa700 1 -- 172.21.15.2:0/2605784019 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc09d180 con 0x555bef53b6b0 2023-09-13T04:40:00.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.930+0000 7f1bd57fa700 1 -- 172.21.15.2:0/2605784019 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bdc0984e0 con 0x555bef53b6b0 2023-09-13T04:40:00.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.930+0000 7f1bd57fa700 1 -- 172.21.15.2:0/2605784019 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc0986d0 con 0x555bef53b6b0 2023-09-13T04:40:00.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.930+0000 7f1c004a0b80 1 -- 172.21.15.2:0/2605784019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53b6b0 msgr2=0x555bef54b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:00.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.930+0000 7f1c004a0b80 1 --2- 172.21.15.2:0/2605784019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53b6b0 0x555bef54b3d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00c2d0 tx=0x7f1bdc097ae0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.930+0000 7f1c004a0b80 1 -- 172.21.15.2:0/2605784019 shutdown_connections 2023-09-13T04:40:00.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.930+0000 7f1c004a0b80 1 --2- 172.21.15.2:0/2605784019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bef53c010 0x555bef54e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.930+0000 7f1c004a0b80 1 --2- 172.21.15.2:0/2605784019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bef53a9b0 0x555bef53a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.930+0000 7f1c004a0b80 1 --2- 172.21.15.2:0/2605784019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53b6b0 0x555bef54b3d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.931+0000 7f1c004a0b80 1 -- 172.21.15.2:0/2605784019 >> 172.21.15.2:0/2605784019 conn(0x555bef498c30 msgr2=0x555bef53e890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:00.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.931+0000 7f1c004a0b80 1 -- 172.21.15.2:0/2605784019 shutdown_connections 2023-09-13T04:40:00.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.931+0000 7f1c004a0b80 1 -- 172.21.15.2:0/2605784019 wait complete. 2023-09-13T04:40:00.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.931+0000 7f1c004a0b80 1 -- start start 2023-09-13T04:40:00.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.931+0000 7f1c004a0b80 1 Processor -- start 2023-09-13T04:40:00.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.932+0000 7f1c004a0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53a9b0 0x555bef545f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:00.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.932+0000 7f1c004a0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bef53b6b0 0x555bef546450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:00.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.932+0000 7f1c004a0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bef53c010 0x555bef5a34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:00.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.932+0000 7f1c004a0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bef54b480 con 0x555bef53a9b0 2023-09-13T04:40:00.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.932+0000 7f1c004a0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bef54b5f0 con 0x555bef53c010 2023-09-13T04:40:00.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.932+0000 7f1c004a0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bef54b7b0 con 0x555bef53b6b0 2023-09-13T04:40:00.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.932+0000 7f1c004a0b80 10 client.0 _ll_register_callbacks cb 0x555bef428630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:00.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.932+0000 7f1be659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53a9b0 0x555bef545f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:00.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.932+0000 7f1be659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53a9b0 0x555bef545f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58178/0 (socket says 172.21.15.2:58178) 2023-09-13T04:40:00.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.932+0000 7f1be659c700 1 -- 172.21.15.2:0/786062358 learned_addr learned my addr 172.21.15.2:0/786062358 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:00.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.932+0000 7f1be6d9d700 1 --2- 172.21.15.2:0/786062358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bef53c010 0x555bef5a34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:00.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.932+0000 7f1be5d9b700 1 --2- 172.21.15.2:0/786062358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bef53b6b0 0x555bef546450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:00.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.932+0000 7f1be659c700 1 -- 172.21.15.2:0/786062358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bef53b6b0 msgr2=0x555bef546450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:00.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.932+0000 7f1be659c700 1 --2- 172.21.15.2:0/786062358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bef53b6b0 0x555bef546450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.932+0000 7f1be659c700 1 -- 172.21.15.2:0/786062358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bef53c010 msgr2=0x555bef5a34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:00.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.932+0000 7f1be659c700 1 --2- 172.21.15.2:0/786062358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bef53c010 0x555bef5a34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.932+0000 7f1be659c700 1 -- 172.21.15.2:0/786062358 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555bef30c5e0 con 0x555bef53a9b0 2023-09-13T04:40:00.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.932+0000 7f1be6d9d700 1 --2- 172.21.15.2:0/786062358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bef53c010 0x555bef5a34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:00.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.933+0000 7f1be659c700 1 --2- 172.21.15.2:0/786062358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53a9b0 0x555bef545f10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8005920 tx=0x7f1bd8005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4884cc31eae1ed70 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:00.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.933+0000 7f1bcdffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:40:00.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.933+0000 7f1bcdffb700 1 -- 172.21.15.2:0/786062358 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd80049d0 con 0x555bef53a9b0 2023-09-13T04:40:00.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.933+0000 7f1c004a0b80 1 -- 172.21.15.2:0/786062358 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bd8004cf0 con 0x555bef53a9b0 2023-09-13T04:40:00.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.933+0000 7f1bcdffb700 1 -- 172.21.15.2:0/786062358 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1bd8004b30 con 0x555bef53a9b0 2023-09-13T04:40:00.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.933+0000 7f1bcdffb700 1 -- 172.21.15.2:0/786062358 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bd802ae70 con 0x555bef53a9b0 2023-09-13T04:40:00.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.933+0000 7f1c004a0b80 10 client.15270 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:00.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.933+0000 7f1c004a0b80 1 -- 172.21.15.2:0/786062358 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x555bef4a4750 con 0x555bef53a9b0 2023-09-13T04:40:00.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.933+0000 7f1bcdffb700 1 -- 172.21.15.2:0/786062358 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bd8004cf0 con 0x555bef53a9b0 2023-09-13T04:40:00.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.933+0000 7f1bcdffb700 1 -- 172.21.15.2:0/786062358 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1bd8026440 con 0x555bef53a9b0 2023-09-13T04:40:00.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.934+0000 7f1c004a0b80 10 client.15270 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:00.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.934+0000 7f1c004a0b80 10 client.15270 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-13T04:40:00.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.934+0000 7f1c004a0b80 20 client.15270 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:00.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.934+0000 7f1c004a0b80 1 --2- 172.21.15.2:0/786062358 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x555bef5a8aa0 0x555bef5a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:00.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.934+0000 7f1c004a0b80 4 client.15270 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "status"}] 2023-09-13T04:40:00.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.934+0000 7f1c004a0b80 1 -- 172.21.15.2:0/786062358 --> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555bef446000 con 0x555bef5a8aa0 2023-09-13T04:40:00.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.934+0000 7f1be5d9b700 1 --2- 172.21.15.2:0/786062358 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x555bef5a8aa0 0x555bef5a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:00.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.934+0000 7f1be5d9b700 1 --2- 172.21.15.2:0/786062358 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x555bef5a8aa0 0x555bef5a4f40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=10239694e0581847 server_cookie=c41ebe6d6c582c2a in_seq=0 out_seq=0 2023-09-13T04:40:00.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.934+0000 7f1bcdffb700 10 client.15270 ms_handle_connect on v2:172.21.15.156:6838/1785833227 2023-09-13T04:40:00.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.935+0000 7f1bcdffb700 1 -- 172.21.15.2:0/786062358 <== mds.2 v2:172.21.15.156:6838/1785833227 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x555bef446000 con 0x555bef5a8aa0 2023-09-13T04:40:00.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.935+0000 7f1bcdffb700 10 client.15270 handle_command_reply: tid=0 2023-09-13T04:40:00.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.935+0000 7f1c004a0b80 1 client.15270 shutdown 2023-09-13T04:40:00.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.935+0000 7f1c004a0b80 1 -- 172.21.15.2:0/786062358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53a9b0 msgr2=0x555bef545f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:00.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.935+0000 7f1c004a0b80 1 --2- 172.21.15.2:0/786062358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53a9b0 0x555bef545f10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8005920 tx=0x7f1bd8005e20 comp rx=0 tx=0).stop 2023-09-13T04:40:00.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.935+0000 7f1c004a0b80 1 -- 172.21.15.2:0/786062358 shutdown_connections 2023-09-13T04:40:00.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.935+0000 7f1c004a0b80 1 --2- 172.21.15.2:0/786062358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bef53c010 0x555bef5a34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.935+0000 7f1c004a0b80 1 --2- 172.21.15.2:0/786062358 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x555bef5a8aa0 0x555bef5a4f40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.935+0000 7f1c004a0b80 1 --2- 172.21.15.2:0/786062358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555bef53a9b0 0x555bef545f10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.935+0000 7f1c004a0b80 1 --2- 172.21.15.2:0/786062358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bef53b6b0 0x555bef546450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.935+0000 7f1bcdffb700 0 client.15270 ms_handle_reset on v2:172.21.15.156:6838/1785833227 2023-09-13T04:40:00.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.935+0000 7f1c004a0b80 1 -- 172.21.15.2:0/786062358 >> 172.21.15.2:0/786062358 conn(0x555bef498c30 msgr2=0x555bef54e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:00.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.935+0000 7f1c004a0b80 1 -- 172.21.15.2:0/786062358 shutdown_connections 2023-09-13T04:40:00.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.935+0000 7f1c004a0b80 1 -- 172.21.15.2:0/786062358 wait complete. 2023-09-13T04:40:00.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.935+0000 7f1c004a0b80 20 client.15270 trim_cache size 0 max 16384 2023-09-13T04:40:00.955 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:40:00.955 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da", 2023-09-13T04:40:00.955 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 2, 2023-09-13T04:40:00.955 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24541, 2023-09-13T04:40:00.956 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2023-09-13T04:40:00.956 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2023-09-13T04:40:00.956 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2023-09-13T04:40:00.956 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 95.327130878999995, 2023-09-13T04:40:00.956 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 23, 2023-09-13T04:40:00.957 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 92, 2023-09-13T04:40:00.957 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 90, 2023-09-13T04:40:00.957 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 136.72483923300001 2023-09-13T04:40:00.957 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:40:00.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.936+0000 7f1becbff700 1 -- 172.21.15.2:0/1557837928 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1bd00a0640 msgr2=0x7f1bd00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:00.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.936+0000 7f1becbff700 1 --2- 172.21.15.2:0/1557837928 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1bd00a0640 0x7f1bd00a2b00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1be000b770 tx=0x7f1be0009b80 comp rx=0 tx=0).stop 2023-09-13T04:40:00.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.936+0000 7f1becbff700 1 -- 172.21.15.2:0/1557837928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1be8101ed0 msgr2=0x7f1be8199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:00.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.936+0000 7f1becbff700 1 --2- 172.21.15.2:0/1557837928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1be8101ed0 0x7f1be8199230 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00e9a0 tx=0x7f1bdc00ecb0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.936+0000 7f1becbff700 1 -- 172.21.15.2:0/1557837928 shutdown_connections 2023-09-13T04:40:00.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.936+0000 7f1becbff700 1 --2- 172.21.15.2:0/1557837928 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1bd00a0640 0x7f1bd00a2b00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.936+0000 7f1becbff700 1 --2- 172.21.15.2:0/1557837928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1be8102870 0x7f1be81a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.936+0000 7f1becbff700 1 --2- 172.21.15.2:0/1557837928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1be8100ce0 0x7f1be8198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.936+0000 7f1becbff700 1 --2- 172.21.15.2:0/1557837928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1be8101ed0 0x7f1be8199230 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:00.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.936+0000 7f1becbff700 1 -- 172.21.15.2:0/1557837928 >> 172.21.15.2:0/1557837928 conn(0x7f1be80fc230 msgr2=0x7f1be8107760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:00.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.936+0000 7f1becbff700 1 -- 172.21.15.2:0/1557837928 shutdown_connections 2023-09-13T04:40:00.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:00.936+0000 7f1becbff700 1 -- 172.21.15.2:0/1557837928 wait complete. 2023-09-13T04:40:00.960 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:00 smithi002 ceph-mon[97651]: pgmap v340: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:40:00.961 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:00 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/4285792526' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:40:00.961 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:00 smithi002 ceph-mon[97651]: overall HEALTH_OK 2023-09-13T04:40:00.961 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:00 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3610722467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:40:00.962 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-09-13T04:40:00.963 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:00 smithi156 ceph-mon[133675]: pgmap v340: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:40:00.963 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:00 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/4285792526' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:40:00.963 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:00 smithi156 ceph-mon[133675]: overall HEALTH_OK 2023-09-13T04:40:00.963 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:00 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3610722467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:40:01.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.167+0000 7fed137ff700 1 -- 172.21.15.2:0/68225046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed0c073710 msgr2=0x7fed0c073b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:01.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.167+0000 7fed137ff700 1 --2- 172.21.15.2:0/68225046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed0c073710 0x7fed0c073b90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009990 tx=0x7fecfc009ca0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.167+0000 7fed137ff700 1 -- 172.21.15.2:0/68225046 shutdown_connections 2023-09-13T04:40:01.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.167+0000 7fed137ff700 1 --2- 172.21.15.2:0/68225046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed0c108ba0 0x7fed0c10b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.167+0000 7fed137ff700 1 --2- 172.21.15.2:0/68225046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed0c074cf0 0x7fed0c073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.167+0000 7fed137ff700 1 --2- 172.21.15.2:0/68225046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed0c073710 0x7fed0c073b90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.167+0000 7fed137ff700 1 -- 172.21.15.2:0/68225046 >> 172.21.15.2:0/68225046 conn(0x7fed0c0fc230 msgr2=0x7fed0c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:01.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.167+0000 7fed137ff700 1 -- 172.21.15.2:0/68225046 shutdown_connections 2023-09-13T04:40:01.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.167+0000 7fed137ff700 1 -- 172.21.15.2:0/68225046 wait complete. 2023-09-13T04:40:01.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.168+0000 7fed137ff700 1 Processor -- start 2023-09-13T04:40:01.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.168+0000 7fed137ff700 1 -- start start 2023-09-13T04:40:01.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.169+0000 7fed137ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed0c073710 0x7fed0c198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.169+0000 7fed137ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed0c074cf0 0x7fed0c1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.169+0000 7fed137ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed0c108ba0 0x7fed0c1a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.169+0000 7fed137ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c1a07a0 con 0x7fed0c074cf0 2023-09-13T04:40:01.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.169+0000 7fed137ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c1a0910 con 0x7fed0c108ba0 2023-09-13T04:40:01.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.169+0000 7fed137ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c1a0ad0 con 0x7fed0c073710 2023-09-13T04:40:01.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.169+0000 7fed10d9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed0c074cf0 0x7fed0c1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.169+0000 7fed10d9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed0c074cf0 0x7fed0c1991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58210/0 (socket says 172.21.15.2:58210) 2023-09-13T04:40:01.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.169+0000 7fed10d9a700 1 -- 172.21.15.2:0/2237101957 learned_addr learned my addr 172.21.15.2:0/2237101957 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:01.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.169+0000 7fed1159b700 1 --2- 172.21.15.2:0/2237101957 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed0c073710 0x7fed0c198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.169+0000 7fed11d9c700 1 --2- 172.21.15.2:0/2237101957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed0c108ba0 0x7fed0c1a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.170+0000 7fed10d9a700 1 -- 172.21.15.2:0/2237101957 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed0c073710 msgr2=0x7fed0c198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:01.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.170+0000 7fed10d9a700 1 --2- 172.21.15.2:0/2237101957 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed0c073710 0x7fed0c198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.170+0000 7fed10d9a700 1 -- 172.21.15.2:0/2237101957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed0c108ba0 msgr2=0x7fed0c1a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:01.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.170+0000 7fed10d9a700 1 --2- 172.21.15.2:0/2237101957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed0c108ba0 0x7fed0c1a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.170+0000 7fed10d9a700 1 -- 172.21.15.2:0/2237101957 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecfc0095f0 con 0x7fed0c074cf0 2023-09-13T04:40:01.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.170+0000 7fed10d9a700 1 --2- 172.21.15.2:0/2237101957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed0c074cf0 0x7fed0c1991a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00b680 tx=0x7fecfc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:01.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.170+0000 7fed027fc700 1 -- 172.21.15.2:0/2237101957 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc01d070 con 0x7fed0c074cf0 2023-09-13T04:40:01.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.170+0000 7fed137ff700 1 -- 172.21.15.2:0/2237101957 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed0c1a0d50 con 0x7fed0c074cf0 2023-09-13T04:40:01.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.170+0000 7fed027fc700 1 -- 172.21.15.2:0/2237101957 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecfc021410 con 0x7fed0c074cf0 2023-09-13T04:40:01.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.170+0000 7fed027fc700 1 -- 172.21.15.2:0/2237101957 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc017650 con 0x7fed0c074cf0 2023-09-13T04:40:01.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.170+0000 7fed137ff700 1 -- 172.21.15.2:0/2237101957 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0c1a11d0 con 0x7fed0c074cf0 2023-09-13T04:40:01.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.174+0000 7fed027fc700 1 -- 172.21.15.2:0/2237101957 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fecfc021580 con 0x7fed0c074cf0 2023-09-13T04:40:01.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.175+0000 7fed027fc700 1 --2- 172.21.15.2:0/2237101957 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fecf80a06c0 0x7fecf80a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.175+0000 7fed27052b80 1 Processor -- start 2023-09-13T04:40:01.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.175+0000 7fed027fc700 1 -- 172.21.15.2:0/2237101957 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fecfc0cd370 con 0x7fed0c074cf0 2023-09-13T04:40:01.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.175+0000 7fed1159b700 1 --2- 172.21.15.2:0/2237101957 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fecf80a06c0 0x7fecf80a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.175+0000 7fed27052b80 1 -- start start 2023-09-13T04:40:01.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.176+0000 7fed27052b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911366b0 0x560891136ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.176+0000 7fed27052b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560891137010 0x560891135470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.176+0000 7fed27052b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608911359b0 0x560891135e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.176+0000 7fed27052b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56089108e730 con 0x5608911366b0 2023-09-13T04:40:01.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.176+0000 7fed27052b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56089108c050 con 0x560891137010 2023-09-13T04:40:01.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.176+0000 7fed27052b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560891087940 con 0x5608911359b0 2023-09-13T04:40:01.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.176+0000 7fed1159b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911366b0 0x560891136ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.176+0000 7fed11d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608911359b0 0x560891135e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.176+0000 7fed11d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608911359b0 0x560891135e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:57318/0 (socket says 172.21.15.2:57318) 2023-09-13T04:40:01.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.176+0000 7fed1159b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911366b0 0x560891136ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58222/0 (socket says 172.21.15.2:58222) 2023-09-13T04:40:01.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.176+0000 7fed11d9c700 1 -- 172.21.15.2:0/662120484 learned_addr learned my addr 172.21.15.2:0/662120484 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:01.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.176+0000 7fed10d9a700 1 --2- 172.21.15.2:0/662120484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560891137010 0x560891135470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.176+0000 7fed1159b700 1 --2- 172.21.15.2:0/2237101957 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fecf80a06c0 0x7fecf80a2b80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fed0c19a130 tx=0x7fed080106b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:01.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.176+0000 7fed11d9c700 1 -- 172.21.15.2:0/662120484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560891137010 msgr2=0x560891135470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:01.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.176+0000 7fed11d9c700 1 --2- 172.21.15.2:0/662120484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560891137010 0x560891135470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.176+0000 7fed11d9c700 1 -- 172.21.15.2:0/662120484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911366b0 msgr2=0x560891136ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:01.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.176+0000 7fed11d9c700 1 --2- 172.21.15.2:0/662120484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911366b0 0x560891136ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.176+0000 7fed11d9c700 1 -- 172.21.15.2:0/662120484 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560890f075e0 con 0x5608911359b0 2023-09-13T04:40:01.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.177+0000 7fed11d9c700 1 --2- 172.21.15.2:0/662120484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608911359b0 0x560891135e30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fed04011680 tx=0x7fed04009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f2be122f6e45ebe server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:01.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.177+0000 7fecf77fe700 1 -- 172.21.15.2:0/662120484 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0400e040 con 0x5608911359b0 2023-09-13T04:40:01.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.177+0000 7fecf77fe700 1 -- 172.21.15.2:0/662120484 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed0400f040 con 0x5608911359b0 2023-09-13T04:40:01.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.177+0000 7fecf77fe700 1 -- 172.21.15.2:0/662120484 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed04004880 con 0x5608911359b0 2023-09-13T04:40:01.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.178+0000 7fed27052b80 1 -- 172.21.15.2:0/662120484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608911359b0 msgr2=0x560891135e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:01.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.178+0000 7fed27052b80 1 --2- 172.21.15.2:0/662120484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608911359b0 0x560891135e30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fed04011680 tx=0x7fed04009920 comp rx=0 tx=0).stop 2023-09-13T04:40:01.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.178+0000 7fed27052b80 1 -- 172.21.15.2:0/662120484 shutdown_connections 2023-09-13T04:40:01.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.178+0000 7fed27052b80 1 --2- 172.21.15.2:0/662120484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608911359b0 0x560891135e30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.178+0000 7fed27052b80 1 --2- 172.21.15.2:0/662120484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911366b0 0x560891136ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.178+0000 7fed27052b80 1 --2- 172.21.15.2:0/662120484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560891137010 0x560891135470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.178+0000 7fed27052b80 1 -- 172.21.15.2:0/662120484 >> 172.21.15.2:0/662120484 conn(0x560891093c30 msgr2=0x56089112fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:01.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.178+0000 7fed27052b80 1 -- 172.21.15.2:0/662120484 shutdown_connections 2023-09-13T04:40:01.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.178+0000 7fed27052b80 1 -- 172.21.15.2:0/662120484 wait complete. 2023-09-13T04:40:01.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.178+0000 7fed27052b80 1 -- start start 2023-09-13T04:40:01.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.179+0000 7fed27052b80 1 Processor -- start 2023-09-13T04:40:01.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.179+0000 7fed27052b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911359b0 0x56089113eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.179+0000 7fed27052b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608911366b0 0x56089113f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.179+0000 7fed27052b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560891137010 0x560891146440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.179+0000 7fed27052b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560891025b60 con 0x5608911359b0 2023-09-13T04:40:01.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.179+0000 7fed27052b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560891025860 con 0x560891137010 2023-09-13T04:40:01.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.179+0000 7fed27052b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560890f098d0 con 0x5608911366b0 2023-09-13T04:40:01.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.179+0000 7fed27052b80 10 client.0 _ll_register_callbacks cb 0x560891023630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:01.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.179+0000 7fed1159b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911359b0 0x56089113eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.179+0000 7fed1159b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911359b0 0x56089113eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58226/0 (socket says 172.21.15.2:58226) 2023-09-13T04:40:01.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.179+0000 7fed1159b700 1 -- 172.21.15.2:0/2289812075 learned_addr learned my addr 172.21.15.2:0/2289812075 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:01.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.179+0000 7fed10d9a700 1 --2- 172.21.15.2:0/2289812075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608911366b0 0x56089113f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.179+0000 7fed11d9c700 1 --2- 172.21.15.2:0/2289812075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560891137010 0x560891146440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.180+0000 7fed1159b700 1 -- 172.21.15.2:0/2289812075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608911366b0 msgr2=0x56089113f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:01.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.180+0000 7fed1159b700 1 --2- 172.21.15.2:0/2289812075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608911366b0 0x56089113f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.180+0000 7fed1159b700 1 -- 172.21.15.2:0/2289812075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560891137010 msgr2=0x560891146440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:01.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.180+0000 7fed1159b700 1 --2- 172.21.15.2:0/2289812075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560891137010 0x560891146440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.180+0000 7fed1159b700 1 -- 172.21.15.2:0/2289812075 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed04011330 con 0x5608911359b0 2023-09-13T04:40:01.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.180+0000 7fed1159b700 1 --2- 172.21.15.2:0/2289812075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911359b0 0x56089113eec0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fed080136e0 tx=0x7fed080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681050ddee1aad29 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:01.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.180+0000 7fecf4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:40:01.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.180+0000 7fecf4ff9700 1 -- 172.21.15.2:0/2289812075 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed08028640 con 0x5608911359b0 2023-09-13T04:40:01.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.180+0000 7fed27052b80 1 -- 172.21.15.2:0/2289812075 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560891128620 con 0x5608911359b0 2023-09-13T04:40:01.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.180+0000 7fecf4ff9700 1 -- 172.21.15.2:0/2289812075 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fed08019040 con 0x5608911359b0 2023-09-13T04:40:01.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.180+0000 7fecf4ff9700 1 -- 172.21.15.2:0/2289812075 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed080132e0 con 0x5608911359b0 2023-09-13T04:40:01.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.180+0000 7fed27052b80 10 client.15282 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:01.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.180+0000 7fed27052b80 1 -- 172.21.15.2:0/2289812075 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56089109f750 con 0x5608911359b0 2023-09-13T04:40:01.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.181+0000 7fecf4ff9700 1 -- 172.21.15.2:0/2289812075 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560891128620 con 0x5608911359b0 2023-09-13T04:40:01.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.181+0000 7fecf4ff9700 1 -- 172.21.15.2:0/2289812075 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fed08011710 con 0x5608911359b0 2023-09-13T04:40:01.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.181+0000 7fed27052b80 10 client.15282 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:01.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.181+0000 7fed27052b80 10 client.15282 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-13T04:40:01.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.181+0000 7fed27052b80 20 client.15282 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:01.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.181+0000 7fed27052b80 1 --2- 172.21.15.2:0/2289812075 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x56089114aaf0 0x5608911495a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.181+0000 7fed27052b80 4 client.15282 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:40:01.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.181+0000 7fed27052b80 1 -- 172.21.15.2:0/2289812075 --> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560891080920 con 0x56089114aaf0 2023-09-13T04:40:01.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.181+0000 7fed10d9a700 1 --2- 172.21.15.2:0/2289812075 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x56089114aaf0 0x5608911495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.182+0000 7fed10d9a700 1 --2- 172.21.15.2:0/2289812075 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x56089114aaf0 0x5608911495a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9e0789de09bd2376 server_cookie=ef754388b97826b7 in_seq=0 out_seq=0 2023-09-13T04:40:01.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.182+0000 7fecf4ff9700 10 client.15282 ms_handle_connect on v2:172.21.15.2:6840/482545391 2023-09-13T04:40:01.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.182+0000 7fecf4ff9700 1 -- 172.21.15.2:0/2289812075 <== mds.1 v2:172.21.15.2:6840/482545391 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560891080920 con 0x56089114aaf0 2023-09-13T04:40:01.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.183+0000 7fecf4ff9700 10 client.15282 handle_command_reply: tid=0 2023-09-13T04:40:01.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.183+0000 7fed27052b80 1 client.15282 shutdown 2023-09-13T04:40:01.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.183+0000 7fed27052b80 1 -- 172.21.15.2:0/2289812075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911359b0 msgr2=0x56089113eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:01.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.183+0000 7fed27052b80 1 --2- 172.21.15.2:0/2289812075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911359b0 0x56089113eec0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fed080136e0 tx=0x7fed080047c0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.183+0000 7fed27052b80 1 -- 172.21.15.2:0/2289812075 shutdown_connections 2023-09-13T04:40:01.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.183+0000 7fed27052b80 1 --2- 172.21.15.2:0/2289812075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560891137010 0x560891146440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.183+0000 7fed27052b80 1 --2- 172.21.15.2:0/2289812075 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x56089114aaf0 0x5608911495a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.183+0000 7fed27052b80 1 --2- 172.21.15.2:0/2289812075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911359b0 0x56089113eec0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.183+0000 7fecf4ff9700 0 client.15282 ms_handle_reset on v2:172.21.15.2:6840/482545391 2023-09-13T04:40:01.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.183+0000 7fed27052b80 1 --2- 172.21.15.2:0/2289812075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608911366b0 0x56089113f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.183+0000 7fed27052b80 1 -- 172.21.15.2:0/2289812075 >> 172.21.15.2:0/2289812075 conn(0x560891093c30 msgr2=0x56089112fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:01.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.183+0000 7fed27052b80 1 -- 172.21.15.2:0/2289812075 shutdown_connections 2023-09-13T04:40:01.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.183+0000 7fed27052b80 1 -- 172.21.15.2:0/2289812075 wait complete. 2023-09-13T04:40:01.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.183+0000 7fed27052b80 20 client.15282 trim_cache size 0 max 16384 2023-09-13T04:40:01.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.202+0000 7fed27052b80 1 Processor -- start 2023-09-13T04:40:01.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.202+0000 7fed27052b80 1 -- start start 2023-09-13T04:40:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.202+0000 7fed27052b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5608911359b0 0x560891135470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.202+0000 7fed27052b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911366b0 0x560891146400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.202+0000 7fed27052b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560891137010 0x5608911398f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.202+0000 7fed27052b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56089113ea90 con 0x5608911366b0 2023-09-13T04:40:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.202+0000 7fed27052b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560891043f00 con 0x5608911359b0 2023-09-13T04:40:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.202+0000 7fed27052b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560891021e00 con 0x560891137010 2023-09-13T04:40:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.202+0000 7fed10d9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911366b0 0x560891146400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.202+0000 7fed10d9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911366b0 0x560891146400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58234/0 (socket says 172.21.15.2:58234) 2023-09-13T04:40:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.202+0000 7fed10d9a700 1 -- 172.21.15.2:0/3004905735 learned_addr learned my addr 172.21.15.2:0/3004905735 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.202+0000 7fed1159b700 1 --2- 172.21.15.2:0/3004905735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5608911359b0 0x560891135470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.202+0000 7fed11d9c700 1 --2- 172.21.15.2:0/3004905735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560891137010 0x5608911398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.203+0000 7fed10d9a700 1 -- 172.21.15.2:0/3004905735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560891137010 msgr2=0x5608911398f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.203+0000 7fed10d9a700 1 --2- 172.21.15.2:0/3004905735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560891137010 0x5608911398f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.203+0000 7fed10d9a700 1 -- 172.21.15.2:0/3004905735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5608911359b0 msgr2=0x560891135470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.203+0000 7fed10d9a700 1 --2- 172.21.15.2:0/3004905735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5608911359b0 0x560891135470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.203+0000 7fed10d9a700 1 -- 172.21.15.2:0/3004905735 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560890eb3d90 con 0x5608911366b0 2023-09-13T04:40:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.203+0000 7fed10d9a700 1 --2- 172.21.15.2:0/3004905735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911366b0 0x560891146400 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fecfc09a560 tx=0x7fecfc0cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b734492a3690aee6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.203+0000 7fecf7fff700 1 -- 172.21.15.2:0/3004905735 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc0a2840 con 0x5608911366b0 2023-09-13T04:40:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.203+0000 7fecf7fff700 1 -- 172.21.15.2:0/3004905735 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecfc0cd120 con 0x5608911366b0 2023-09-13T04:40:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.203+0000 7fecf7fff700 1 -- 172.21.15.2:0/3004905735 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc099d50 con 0x5608911366b0 2023-09-13T04:40:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.204+0000 7fed27052b80 1 -- 172.21.15.2:0/3004905735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911366b0 msgr2=0x560891146400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.204+0000 7fed27052b80 1 --2- 172.21.15.2:0/3004905735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911366b0 0x560891146400 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fecfc09a560 tx=0x7fecfc0cc040 comp rx=0 tx=0).stop 2023-09-13T04:40:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.204+0000 7fed27052b80 1 -- 172.21.15.2:0/3004905735 shutdown_connections 2023-09-13T04:40:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.204+0000 7fed27052b80 1 --2- 172.21.15.2:0/3004905735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560891137010 0x5608911398f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.204+0000 7fed27052b80 1 --2- 172.21.15.2:0/3004905735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5608911359b0 0x560891135470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.204+0000 7fed27052b80 1 --2- 172.21.15.2:0/3004905735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911366b0 0x560891146400 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.204+0000 7fed27052b80 1 -- 172.21.15.2:0/3004905735 >> 172.21.15.2:0/3004905735 conn(0x560891093c30 msgr2=0x56089113cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:01.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.204+0000 7fed27052b80 1 -- 172.21.15.2:0/3004905735 shutdown_connections 2023-09-13T04:40:01.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.204+0000 7fed27052b80 1 -- 172.21.15.2:0/3004905735 wait complete. 2023-09-13T04:40:01.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.204+0000 7fed27052b80 1 -- start start 2023-09-13T04:40:01.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.205+0000 7fed27052b80 1 Processor -- start 2023-09-13T04:40:01.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.205+0000 7fed27052b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911359b0 0x56089113aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.205+0000 7fed27052b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608911366b0 0x560891146400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.205+0000 7fed27052b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560891137010 0x56089113af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.205+0000 7fed27052b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56089113c6b0 con 0x5608911359b0 2023-09-13T04:40:01.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.205+0000 7fed27052b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56089113b4c0 con 0x560891137010 2023-09-13T04:40:01.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.205+0000 7fed27052b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56089113b680 con 0x5608911366b0 2023-09-13T04:40:01.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.205+0000 7fed27052b80 10 client.0 _ll_register_callbacks cb 0x560891023630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:01.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.205+0000 7fed1159b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911359b0 0x56089113aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.205+0000 7fed1159b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911359b0 0x56089113aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58244/0 (socket says 172.21.15.2:58244) 2023-09-13T04:40:01.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.205+0000 7fed1159b700 1 -- 172.21.15.2:0/1704161778 learned_addr learned my addr 172.21.15.2:0/1704161778 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:01.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.205+0000 7fed10d9a700 1 --2- 172.21.15.2:0/1704161778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608911366b0 0x560891146400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.205+0000 7fed11d9c700 1 --2- 172.21.15.2:0/1704161778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560891137010 0x56089113af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.206+0000 7fed1159b700 1 -- 172.21.15.2:0/1704161778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608911366b0 msgr2=0x560891146400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:01.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.206+0000 7fed1159b700 1 --2- 172.21.15.2:0/1704161778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608911366b0 0x560891146400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.206+0000 7fed1159b700 1 -- 172.21.15.2:0/1704161778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560891137010 msgr2=0x56089113af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:01.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.206+0000 7fed1159b700 1 --2- 172.21.15.2:0/1704161778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560891137010 0x56089113af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.206+0000 7fed1159b700 1 -- 172.21.15.2:0/1704161778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0c1a11d0 con 0x5608911359b0 2023-09-13T04:40:01.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.206+0000 7fed11d9c700 1 --2- 172.21.15.2:0/1704161778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560891137010 0x56089113af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:40:01.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.206+0000 7fed1159b700 1 --2- 172.21.15.2:0/1704161778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911359b0 0x56089113aa10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fed08011a70 tx=0x7fed080119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e88df96de27a4d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:01.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.206+0000 7fecf4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:40:01.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.206+0000 7fecf4ff9700 1 -- 172.21.15.2:0/1704161778 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0800bed0 con 0x5608911359b0 2023-09-13T04:40:01.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.206+0000 7fecf4ff9700 1 -- 172.21.15.2:0/1704161778 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fed08019040 con 0x5608911359b0 2023-09-13T04:40:01.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.206+0000 7fed27052b80 1 -- 172.21.15.2:0/1704161778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed0400f040 con 0x5608911359b0 2023-09-13T04:40:01.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.206+0000 7fecf4ff9700 1 -- 172.21.15.2:0/1704161778 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed080132e0 con 0x5608911359b0 2023-09-13T04:40:01.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.207+0000 7fed27052b80 10 client.15288 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:01.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.207+0000 7fed27052b80 1 -- 172.21.15.2:0/1704161778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560890e9cc10 con 0x5608911359b0 2023-09-13T04:40:01.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.207+0000 7fecf4ff9700 1 -- 172.21.15.2:0/1704161778 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed0400f040 con 0x5608911359b0 2023-09-13T04:40:01.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.207+0000 7fecf4ff9700 1 -- 172.21.15.2:0/1704161778 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fed0803c060 con 0x5608911359b0 2023-09-13T04:40:01.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.207+0000 7fed27052b80 10 client.15288 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:01.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.207+0000 7fed27052b80 10 client.15288 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-13T04:40:01.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.207+0000 7fed27052b80 20 client.15288 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:01.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.207+0000 7fed27052b80 1 --2- 172.21.15.2:0/1704161778 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x5608911a3460 0x56089119f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.207+0000 7fed27052b80 4 client.15288 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "status"}] 2023-09-13T04:40:01.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.207+0000 7fed27052b80 1 -- 172.21.15.2:0/1704161778 --> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560891041000 con 0x5608911a3460 2023-09-13T04:40:01.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.207+0000 7fed10d9a700 1 --2- 172.21.15.2:0/1704161778 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x5608911a3460 0x56089119f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.208+0000 7fecf4ff9700 10 client.15288 ms_handle_connect on v2:172.21.15.2:6840/482545391 2023-09-13T04:40:01.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.208+0000 7fed10d9a700 1 --2- 172.21.15.2:0/1704161778 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x5608911a3460 0x56089119f8c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae6ab1a203bffeed server_cookie=92981e2c376be047 in_seq=0 out_seq=0 2023-09-13T04:40:01.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.208+0000 7fecf4ff9700 1 -- 172.21.15.2:0/1704161778 <== mds.1 v2:172.21.15.2:6840/482545391 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x560891041000 con 0x5608911a3460 2023-09-13T04:40:01.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.208+0000 7fecf4ff9700 10 client.15288 handle_command_reply: tid=0 2023-09-13T04:40:01.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.208+0000 7fed27052b80 1 client.15288 shutdown 2023-09-13T04:40:01.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.208+0000 7fed27052b80 1 -- 172.21.15.2:0/1704161778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911359b0 msgr2=0x56089113aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:01.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.208+0000 7fed27052b80 1 --2- 172.21.15.2:0/1704161778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911359b0 0x56089113aa10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fed08011a70 tx=0x7fed080119b0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.208+0000 7fed27052b80 1 -- 172.21.15.2:0/1704161778 shutdown_connections 2023-09-13T04:40:01.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.208+0000 7fed27052b80 1 --2- 172.21.15.2:0/1704161778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560891137010 0x56089113af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.208+0000 7fed27052b80 1 --2- 172.21.15.2:0/1704161778 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x5608911a3460 0x56089119f8c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.208+0000 7fed27052b80 1 --2- 172.21.15.2:0/1704161778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5608911359b0 0x56089113aa10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.208+0000 7fed27052b80 1 --2- 172.21.15.2:0/1704161778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608911366b0 0x560891146400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.208+0000 7fed27052b80 1 -- 172.21.15.2:0/1704161778 >> 172.21.15.2:0/1704161778 conn(0x560891093c30 msgr2=0x5608911398f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:01.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.208+0000 7fecf4ff9700 0 client.15288 ms_handle_reset on v2:172.21.15.2:6840/482545391 2023-09-13T04:40:01.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.208+0000 7fed27052b80 1 -- 172.21.15.2:0/1704161778 shutdown_connections 2023-09-13T04:40:01.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.209+0000 7fed27052b80 1 -- 172.21.15.2:0/1704161778 wait complete. 2023-09-13T04:40:01.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.209+0000 7fed27052b80 20 client.15288 trim_cache size 0 max 16384 2023-09-13T04:40:01.235 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:40:01.235 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da", 2023-09-13T04:40:01.235 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 1, 2023-09-13T04:40:01.235 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24554, 2023-09-13T04:40:01.236 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2023-09-13T04:40:01.236 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2023-09-13T04:40:01.236 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2023-09-13T04:40:01.236 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 96.976487539000004, 2023-09-13T04:40:01.236 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 23, 2023-09-13T04:40:01.236 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 92, 2023-09-13T04:40:01.237 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 90, 2023-09-13T04:40:01.237 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 130.60664535699999 2023-09-13T04:40:01.237 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:40:01.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.209+0000 7fed137ff700 1 -- 172.21.15.2:0/2237101957 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fecf80a06c0 msgr2=0x7fecf80a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:01.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.209+0000 7fed137ff700 1 --2- 172.21.15.2:0/2237101957 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fecf80a06c0 0x7fecf80a2b80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fed0c19a130 tx=0x7fed080106b0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.209+0000 7fed137ff700 1 -- 172.21.15.2:0/2237101957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed0c074cf0 msgr2=0x7fed0c1991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:01.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.209+0000 7fed137ff700 1 --2- 172.21.15.2:0/2237101957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed0c074cf0 0x7fed0c1991a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00b680 tx=0x7fecfc005740 comp rx=0 tx=0).stop 2023-09-13T04:40:01.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.209+0000 7fed137ff700 1 -- 172.21.15.2:0/2237101957 shutdown_connections 2023-09-13T04:40:01.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.209+0000 7fed137ff700 1 --2- 172.21.15.2:0/2237101957 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fecf80a06c0 0x7fecf80a2b80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.209+0000 7fed137ff700 1 --2- 172.21.15.2:0/2237101957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed0c108ba0 0x7fed0c1a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.209+0000 7fed137ff700 1 --2- 172.21.15.2:0/2237101957 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed0c073710 0x7fed0c198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.209+0000 7fed137ff700 1 --2- 172.21.15.2:0/2237101957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed0c074cf0 0x7fed0c1991a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.209+0000 7fed137ff700 1 -- 172.21.15.2:0/2237101957 >> 172.21.15.2:0/2237101957 conn(0x7fed0c0fc230 msgr2=0x7fed0c0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:01.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.210+0000 7fed137ff700 1 -- 172.21.15.2:0/2237101957 shutdown_connections 2023-09-13T04:40:01.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.210+0000 7fed137ff700 1 -- 172.21.15.2:0/2237101957 wait complete. 2023-09-13T04:40:01.241 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-13T04:40:01.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.444+0000 7fe8727ff700 1 -- 172.21.15.2:0/1351512032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe86c074cf0 msgr2=0x7fe86c073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:01.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.444+0000 7fe8727ff700 1 --2- 172.21.15.2:0/1351512032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe86c074cf0 0x7fe86c073140 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0098f0 tx=0x7fe85c009c00 comp rx=0 tx=0).stop 2023-09-13T04:40:01.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.444+0000 7fe8727ff700 1 -- 172.21.15.2:0/1351512032 shutdown_connections 2023-09-13T04:40:01.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.444+0000 7fe8727ff700 1 --2- 172.21.15.2:0/1351512032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe86c108bb0 0x7fe86c10b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.444+0000 7fe8727ff700 1 --2- 172.21.15.2:0/1351512032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe86c074cf0 0x7fe86c073140 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.444+0000 7fe8727ff700 1 --2- 172.21.15.2:0/1351512032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe86c073710 0x7fe86c073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.444+0000 7fe8727ff700 1 -- 172.21.15.2:0/1351512032 >> 172.21.15.2:0/1351512032 conn(0x7fe86c0fc260 msgr2=0x7fe86c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:01.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.444+0000 7fe8727ff700 1 -- 172.21.15.2:0/1351512032 shutdown_connections 2023-09-13T04:40:01.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.445+0000 7fe8727ff700 1 -- 172.21.15.2:0/1351512032 wait complete. 2023-09-13T04:40:01.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.445+0000 7fe8727ff700 1 Processor -- start 2023-09-13T04:40:01.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.446+0000 7fe8727ff700 1 -- start start 2023-09-13T04:40:01.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.446+0000 7fe8727ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe86c073710 0x7fe86c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.446+0000 7fe8727ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe86c074cf0 0x7fe86c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.446+0000 7fe8727ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe86c108bb0 0x7fe86c1a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.446+0000 7fe8727ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c1a07c0 con 0x7fe86c074cf0 2023-09-13T04:40:01.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.446+0000 7fe8727ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c1a0930 con 0x7fe86c073710 2023-09-13T04:40:01.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.446+0000 7fe8727ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c1a0af0 con 0x7fe86c108bb0 2023-09-13T04:40:01.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.446+0000 7fe86b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe86c074cf0 0x7fe86c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.446+0000 7fe86b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe86c074cf0 0x7fe86c1991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58260/0 (socket says 172.21.15.2:58260) 2023-09-13T04:40:01.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.446+0000 7fe86b7fe700 1 -- 172.21.15.2:0/1313825014 learned_addr learned my addr 172.21.15.2:0/1313825014 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:01.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.446+0000 7fe86bfff700 1 --2- 172.21.15.2:0/1313825014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe86c073710 0x7fe86c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.446+0000 7fe870d9c700 1 --2- 172.21.15.2:0/1313825014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe86c108bb0 0x7fe86c1a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.447+0000 7fe86b7fe700 1 -- 172.21.15.2:0/1313825014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe86c108bb0 msgr2=0x7fe86c1a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:01.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.447+0000 7fe86b7fe700 1 --2- 172.21.15.2:0/1313825014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe86c108bb0 0x7fe86c1a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.447+0000 7fe86b7fe700 1 -- 172.21.15.2:0/1313825014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe86c073710 msgr2=0x7fe86c198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:01.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.447+0000 7fe86b7fe700 1 --2- 172.21.15.2:0/1313825014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe86c073710 0x7fe86c198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.447+0000 7fe86b7fe700 1 -- 172.21.15.2:0/1313825014 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85c009520 con 0x7fe86c074cf0 2023-09-13T04:40:01.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.447+0000 7fe86b7fe700 1 --2- 172.21.15.2:0/1313825014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe86c074cf0 0x7fe86c1991c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe86000e9f0 tx=0x7fe86000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:01.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.447+0000 7fe8697fa700 1 -- 172.21.15.2:0/1313825014 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860004220 con 0x7fe86c074cf0 2023-09-13T04:40:01.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.447+0000 7fe8697fa700 1 -- 172.21.15.2:0/1313825014 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe860004380 con 0x7fe86c074cf0 2023-09-13T04:40:01.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.447+0000 7fe8697fa700 1 -- 172.21.15.2:0/1313825014 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860003be0 con 0x7fe86c074cf0 2023-09-13T04:40:01.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.447+0000 7fe8727ff700 1 -- 172.21.15.2:0/1313825014 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe86c1a0dd0 con 0x7fe86c074cf0 2023-09-13T04:40:01.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.447+0000 7fe8727ff700 1 -- 172.21.15.2:0/1313825014 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe86c1a11a0 con 0x7fe86c074cf0 2023-09-13T04:40:01.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.450+0000 7fe8697fa700 1 -- 172.21.15.2:0/1313825014 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fe860003d40 con 0x7fe86c074cf0 2023-09-13T04:40:01.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.452+0000 7fe8861cdb80 1 Processor -- start 2023-09-13T04:40:01.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.452+0000 7fe8861cdb80 1 -- start start 2023-09-13T04:40:01.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.452+0000 7fe8861cdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a29e986b0 0x557a29e98ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.452+0000 7fe8861cdb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a29e99010 0x557a29e97470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.452+0000 7fe8861cdb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29e979b0 0x557a29e97e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.452+0000 7fe8861cdb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a29df0730 con 0x557a29e979b0 2023-09-13T04:40:01.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.452+0000 7fe8861cdb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a29dee050 con 0x557a29e99010 2023-09-13T04:40:01.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.452+0000 7fe8861cdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a29de9940 con 0x557a29e986b0 2023-09-13T04:40:01.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.452+0000 7fe8697fa700 1 --2- 172.21.15.2:0/1313825014 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe8540a0640 0x7fe8540a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.452+0000 7fe86b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29e979b0 0x557a29e97e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.453+0000 7fe870d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a29e986b0 0x557a29e98ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.453+0000 7fe8697fa700 1 -- 172.21.15.2:0/1313825014 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe860014070 con 0x7fe86c074cf0 2023-09-13T04:40:01.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.453+0000 7fe86bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a29e99010 0x557a29e97470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.453+0000 7fe870d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a29e986b0 0x557a29e98ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:57364/0 (socket says 172.21.15.2:57364) 2023-09-13T04:40:01.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.453+0000 7fe870d9c700 1 -- 172.21.15.2:0/3938544031 learned_addr learned my addr 172.21.15.2:0/3938544031 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:01.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.453+0000 7fe870d9c700 1 --2- 172.21.15.2:0/1313825014 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe8540a0640 0x7fe8540a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.453+0000 7fe86b7fe700 1 -- 172.21.15.2:0/3938544031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a29e986b0 msgr2=0x557a29e98ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:01.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.453+0000 7fe86b7fe700 1 --2- 172.21.15.2:0/3938544031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a29e986b0 0x557a29e98ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.453+0000 7fe86b7fe700 1 -- 172.21.15.2:0/3938544031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a29e99010 msgr2=0x557a29e97470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:01.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.453+0000 7fe86b7fe700 1 --2- 172.21.15.2:0/3938544031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a29e99010 0x557a29e97470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.453+0000 7fe86b7fe700 1 -- 172.21.15.2:0/3938544031 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a29c695e0 con 0x557a29e979b0 2023-09-13T04:40:01.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.453+0000 7fe86b7fe700 1 --2- 172.21.15.2:0/3938544031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29e979b0 0x557a29e97e30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe8600993b0 tx=0x7fe86008ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc96b7b7a3ca5b9 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:01.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.454+0000 7fe870d9c700 1 --2- 172.21.15.2:0/1313825014 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe8540a0640 0x7fe8540a2b00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe864009fd0 tx=0x7fe86400b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:01.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.454+0000 7fe8527fc700 1 -- 172.21.15.2:0/3938544031 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860098d80 con 0x557a29e979b0 2023-09-13T04:40:01.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.454+0000 7fe8527fc700 1 -- 172.21.15.2:0/3938544031 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe860007e40 con 0x557a29e979b0 2023-09-13T04:40:01.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.454+0000 7fe8527fc700 1 -- 172.21.15.2:0/3938544031 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8600a1490 con 0x557a29e979b0 2023-09-13T04:40:01.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.454+0000 7fe8861cdb80 1 -- 172.21.15.2:0/3938544031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29e979b0 msgr2=0x557a29e97e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:01.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.454+0000 7fe8861cdb80 1 --2- 172.21.15.2:0/3938544031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29e979b0 0x557a29e97e30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe8600993b0 tx=0x7fe86008ee00 comp rx=0 tx=0).stop 2023-09-13T04:40:01.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.454+0000 7fe8861cdb80 1 -- 172.21.15.2:0/3938544031 shutdown_connections 2023-09-13T04:40:01.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.454+0000 7fe8861cdb80 1 --2- 172.21.15.2:0/3938544031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29e979b0 0x557a29e97e30 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.454+0000 7fe8861cdb80 1 --2- 172.21.15.2:0/3938544031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a29e986b0 0x557a29e98ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.454+0000 7fe8861cdb80 1 --2- 172.21.15.2:0/3938544031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a29e99010 0x557a29e97470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.454+0000 7fe8861cdb80 1 -- 172.21.15.2:0/3938544031 >> 172.21.15.2:0/3938544031 conn(0x557a29df5c30 msgr2=0x557a29e91ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:01.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.454+0000 7fe8861cdb80 1 -- 172.21.15.2:0/3938544031 shutdown_connections 2023-09-13T04:40:01.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.454+0000 7fe8861cdb80 1 -- 172.21.15.2:0/3938544031 wait complete. 2023-09-13T04:40:01.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.455+0000 7fe8861cdb80 1 -- start start 2023-09-13T04:40:01.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.455+0000 7fe8861cdb80 1 Processor -- start 2023-09-13T04:40:01.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.456+0000 7fe8861cdb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a29e979b0 0x557a29ea0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.456+0000 7fe8861cdb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29e986b0 0x557a29ea1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.456+0000 7fe8861cdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a29e99010 0x557a29ea84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.456+0000 7fe8861cdb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a29d87b60 con 0x557a29e986b0 2023-09-13T04:40:01.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.456+0000 7fe8861cdb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a29d87860 con 0x557a29e979b0 2023-09-13T04:40:01.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.456+0000 7fe8861cdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a29c6b8d0 con 0x557a29e99010 2023-09-13T04:40:01.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.456+0000 7fe8861cdb80 10 client.0 _ll_register_callbacks cb 0x557a29d85630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:01.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.456+0000 7fe86bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a29e979b0 0x557a29ea0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.456+0000 7fe86b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29e986b0 0x557a29ea1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.456+0000 7fe86b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29e986b0 0x557a29ea1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58278/0 (socket says 172.21.15.2:58278) 2023-09-13T04:40:01.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.456+0000 7fe86b7fe700 1 -- 172.21.15.2:0/2261166793 learned_addr learned my addr 172.21.15.2:0/2261166793 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:01.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.456+0000 7fe870d9c700 1 --2- 172.21.15.2:0/2261166793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a29e99010 0x557a29ea84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.456+0000 7fe86b7fe700 1 -- 172.21.15.2:0/2261166793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a29e99010 msgr2=0x557a29ea84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:01.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.456+0000 7fe86b7fe700 1 --2- 172.21.15.2:0/2261166793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a29e99010 0x557a29ea84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.456+0000 7fe86b7fe700 1 -- 172.21.15.2:0/2261166793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a29e979b0 msgr2=0x557a29ea0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:01.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.456+0000 7fe86b7fe700 1 --2- 172.21.15.2:0/2261166793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a29e979b0 0x557a29ea0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.456+0000 7fe86b7fe700 1 -- 172.21.15.2:0/2261166793 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe86c1a11a0 con 0x557a29e986b0 2023-09-13T04:40:01.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.457+0000 7fe86b7fe700 1 --2- 172.21.15.2:0/2261166793 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29e986b0 0x557a29ea1480 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe8600a0f60 tx=0x7fe860099420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65eaa87a0a0cd20d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:01.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.457+0000 7fe84b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:40:01.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.457+0000 7fe84b7fe700 1 -- 172.21.15.2:0/2261166793 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8600044f0 con 0x557a29e986b0 2023-09-13T04:40:01.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.457+0000 7fe8861cdb80 1 -- 172.21.15.2:0/2261166793 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a29e8a620 con 0x557a29e986b0 2023-09-13T04:40:01.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.457+0000 7fe84b7fe700 1 -- 172.21.15.2:0/2261166793 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe8600a1490 con 0x557a29e986b0 2023-09-13T04:40:01.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.457+0000 7fe84b7fe700 1 -- 172.21.15.2:0/2261166793 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe860098910 con 0x557a29e986b0 2023-09-13T04:40:01.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.457+0000 7fe8861cdb80 10 client.15300 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:01.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.457+0000 7fe8861cdb80 1 -- 172.21.15.2:0/2261166793 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557a29e01750 con 0x557a29e986b0 2023-09-13T04:40:01.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.457+0000 7fe84b7fe700 1 -- 172.21.15.2:0/2261166793 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557a29e8a620 con 0x557a29e986b0 2023-09-13T04:40:01.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.458+0000 7fe84b7fe700 1 -- 172.21.15.2:0/2261166793 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe8600a85f0 con 0x557a29e986b0 2023-09-13T04:40:01.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.458+0000 7fe8861cdb80 10 client.15300 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:01.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.458+0000 7fe8861cdb80 10 client.15300 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-13T04:40:01.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.458+0000 7fe8861cdb80 20 client.15300 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:01.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.458+0000 7fe8861cdb80 1 --2- 172.21.15.2:0/2261166793 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x557a29eac760 0x557a29eab3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.458+0000 7fe8861cdb80 4 client.15300 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:40:01.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.458+0000 7fe8861cdb80 1 -- 172.21.15.2:0/2261166793 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a29de2920 con 0x557a29eac760 2023-09-13T04:40:01.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.458+0000 7fe86bfff700 1 --2- 172.21.15.2:0/2261166793 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x557a29eac760 0x557a29eab3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.459+0000 7fe86bfff700 1 --2- 172.21.15.2:0/2261166793 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x557a29eac760 0x557a29eab3f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1d36692349bb382 server_cookie=70ce119c0e95a70c in_seq=0 out_seq=0 2023-09-13T04:40:01.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.459+0000 7fe84b7fe700 10 client.15300 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:01.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.459+0000 7fe84b7fe700 1 -- 172.21.15.2:0/2261166793 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557a29de2920 con 0x557a29eac760 2023-09-13T04:40:01.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.459+0000 7fe84b7fe700 10 client.15300 handle_command_reply: tid=0 2023-09-13T04:40:01.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.460+0000 7fe8861cdb80 1 client.15300 shutdown 2023-09-13T04:40:01.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.460+0000 7fe8861cdb80 1 -- 172.21.15.2:0/2261166793 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29e986b0 msgr2=0x557a29ea1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:01.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.460+0000 7fe8861cdb80 1 --2- 172.21.15.2:0/2261166793 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29e986b0 0x557a29ea1480 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe8600a0f60 tx=0x7fe860099420 comp rx=0 tx=0).stop 2023-09-13T04:40:01.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.460+0000 7fe8861cdb80 1 -- 172.21.15.2:0/2261166793 shutdown_connections 2023-09-13T04:40:01.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.460+0000 7fe8861cdb80 1 --2- 172.21.15.2:0/2261166793 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x557a29eac760 0x557a29eab3f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.460+0000 7fe8861cdb80 1 --2- 172.21.15.2:0/2261166793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a29e99010 0x557a29ea84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.460+0000 7fe8861cdb80 1 --2- 172.21.15.2:0/2261166793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a29e979b0 0x557a29ea0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.460+0000 7fe8861cdb80 1 --2- 172.21.15.2:0/2261166793 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29e986b0 0x557a29ea1480 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.460+0000 7fe84b7fe700 0 client.15300 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:01.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.460+0000 7fe8861cdb80 1 -- 172.21.15.2:0/2261166793 >> 172.21.15.2:0/2261166793 conn(0x557a29df5c30 msgr2=0x557a29e8c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:01.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.460+0000 7fe8861cdb80 1 -- 172.21.15.2:0/2261166793 shutdown_connections 2023-09-13T04:40:01.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.460+0000 7fe8861cdb80 1 -- 172.21.15.2:0/2261166793 wait complete. 2023-09-13T04:40:01.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.460+0000 7fe8861cdb80 20 client.15300 trim_cache size 0 max 16384 2023-09-13T04:40:01.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.478+0000 7fe8861cdb80 1 Processor -- start 2023-09-13T04:40:01.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.478+0000 7fe8861cdb80 1 -- start start 2023-09-13T04:40:01.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.478+0000 7fe8861cdb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29df5c30 0x557a29e97470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.478+0000 7fe8861cdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a29e979b0 0x557a29ea8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.479+0000 7fe8861cdb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a29e986b0 0x557a29ea0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.479+0000 7fe8861cdb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a29de2920 con 0x557a29df5c30 2023-09-13T04:40:01.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.479+0000 7fe8861cdb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a29e8a620 con 0x557a29e986b0 2023-09-13T04:40:01.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.479+0000 7fe8861cdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a29ea2130 con 0x557a29e979b0 2023-09-13T04:40:01.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.479+0000 7fe86bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29df5c30 0x557a29e97470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.479+0000 7fe86bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29df5c30 0x557a29e97470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58280/0 (socket says 172.21.15.2:58280) 2023-09-13T04:40:01.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.479+0000 7fe86bfff700 1 -- 172.21.15.2:0/1251984026 learned_addr learned my addr 172.21.15.2:0/1251984026 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:01.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.479+0000 7fe86b7fe700 1 --2- 172.21.15.2:0/1251984026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a29e979b0 0x557a29ea8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.479+0000 7fe870d9c700 1 --2- 172.21.15.2:0/1251984026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a29e986b0 0x557a29ea0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.479+0000 7fe86bfff700 1 -- 172.21.15.2:0/1251984026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a29e979b0 msgr2=0x557a29ea8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:01.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.479+0000 7fe86bfff700 1 --2- 172.21.15.2:0/1251984026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a29e979b0 0x557a29ea8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.479+0000 7fe86bfff700 1 -- 172.21.15.2:0/1251984026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a29e986b0 msgr2=0x557a29ea0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:01.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.479+0000 7fe86bfff700 1 --2- 172.21.15.2:0/1251984026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a29e986b0 0x557a29ea0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.479+0000 7fe86bfff700 1 -- 172.21.15.2:0/1251984026 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a29c15d90 con 0x557a29df5c30 2023-09-13T04:40:01.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.479+0000 7fe86bfff700 1 --2- 172.21.15.2:0/1251984026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29df5c30 0x557a29e97470 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0098c0 tx=0x7fe85c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87f5b96a0b1db27 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:01.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.480+0000 7fe852ffd700 1 -- 172.21.15.2:0/1251984026 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c00f800 con 0x557a29df5c30 2023-09-13T04:40:01.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.480+0000 7fe852ffd700 1 -- 172.21.15.2:0/1251984026 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe85c01d040 con 0x557a29df5c30 2023-09-13T04:40:01.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.480+0000 7fe852ffd700 1 -- 172.21.15.2:0/1251984026 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c00b980 con 0x557a29df5c30 2023-09-13T04:40:01.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.480+0000 7fe8861cdb80 1 -- 172.21.15.2:0/1251984026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29df5c30 msgr2=0x557a29e97470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:01.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.480+0000 7fe8861cdb80 1 --2- 172.21.15.2:0/1251984026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29df5c30 0x557a29e97470 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0098c0 tx=0x7fe85c005dc0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.480+0000 7fe8861cdb80 1 -- 172.21.15.2:0/1251984026 shutdown_connections 2023-09-13T04:40:01.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.480+0000 7fe8861cdb80 1 --2- 172.21.15.2:0/1251984026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a29e986b0 0x557a29ea0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.480+0000 7fe8861cdb80 1 --2- 172.21.15.2:0/1251984026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29df5c30 0x557a29e97470 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.480+0000 7fe8861cdb80 1 --2- 172.21.15.2:0/1251984026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a29e979b0 0x557a29ea8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.480+0000 7fe8861cdb80 1 -- 172.21.15.2:0/1251984026 >> 172.21.15.2:0/1251984026 conn(0x557a29e921f0 msgr2=0x557a29e9b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:01.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.480+0000 7fe8861cdb80 1 -- 172.21.15.2:0/1251984026 shutdown_connections 2023-09-13T04:40:01.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.480+0000 7fe8861cdb80 1 -- 172.21.15.2:0/1251984026 wait complete. 2023-09-13T04:40:01.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.481+0000 7fe8861cdb80 1 -- start start 2023-09-13T04:40:01.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.481+0000 7fe8861cdb80 1 Processor -- start 2023-09-13T04:40:01.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.481+0000 7fe8861cdb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29df5c30 0x557a29ea2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.481+0000 7fe8861cdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a29e979b0 0x557a29ea84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.481+0000 7fe8861cdb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a29e986b0 0x557a29ea3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.481+0000 7fe8861cdb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a29f00150 con 0x557a29df5c30 2023-09-13T04:40:01.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.481+0000 7fe8861cdb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a29f00290 con 0x557a29e986b0 2023-09-13T04:40:01.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.481+0000 7fe8861cdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a29f003d0 con 0x557a29e979b0 2023-09-13T04:40:01.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.481+0000 7fe8861cdb80 10 client.0 _ll_register_callbacks cb 0x557a29d85630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:01.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.482+0000 7fe86bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29df5c30 0x557a29ea2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.482+0000 7fe86bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29df5c30 0x557a29ea2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58282/0 (socket says 172.21.15.2:58282) 2023-09-13T04:40:01.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.482+0000 7fe86bfff700 1 -- 172.21.15.2:0/175983915 learned_addr learned my addr 172.21.15.2:0/175983915 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:01.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.482+0000 7fe86b7fe700 1 --2- 172.21.15.2:0/175983915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a29e979b0 0x557a29ea84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.482+0000 7fe870d9c700 1 --2- 172.21.15.2:0/175983915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a29e986b0 0x557a29ea3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.482+0000 7fe86bfff700 1 -- 172.21.15.2:0/175983915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a29e979b0 msgr2=0x557a29ea84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:01.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.482+0000 7fe86bfff700 1 --2- 172.21.15.2:0/175983915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a29e979b0 0x557a29ea84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.482+0000 7fe86bfff700 1 -- 172.21.15.2:0/175983915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a29e986b0 msgr2=0x557a29ea3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:01.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.482+0000 7fe86bfff700 1 --2- 172.21.15.2:0/175983915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a29e986b0 0x557a29ea3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.482+0000 7fe86bfff700 1 -- 172.21.15.2:0/175983915 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe86c10bc60 con 0x557a29df5c30 2023-09-13T04:40:01.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.482+0000 7fe870d9c700 1 --2- 172.21.15.2:0/175983915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a29e986b0 0x557a29ea3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:40:01.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.482+0000 7fe86bfff700 1 --2- 172.21.15.2:0/175983915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29df5c30 0x557a29ea2f80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe85c009230 tx=0x7fe85c027170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0d952b65451db9 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:01.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.482+0000 7fe84b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:40:01.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.482+0000 7fe84b7fe700 1 -- 172.21.15.2:0/175983915 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c027c80 con 0x557a29df5c30 2023-09-13T04:40:01.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.483+0000 7fe8861cdb80 1 -- 172.21.15.2:0/175983915 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe860007e40 con 0x557a29df5c30 2023-09-13T04:40:01.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.483+0000 7fe84b7fe700 1 -- 172.21.15.2:0/175983915 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe85c027de0 con 0x557a29df5c30 2023-09-13T04:40:01.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.483+0000 7fe84b7fe700 1 -- 172.21.15.2:0/175983915 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe85c00b700 con 0x557a29df5c30 2023-09-13T04:40:01.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.483+0000 7fe8861cdb80 10 client.15306 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:01.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.483+0000 7fe8861cdb80 1 -- 172.21.15.2:0/175983915 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557a29bfec10 con 0x557a29df5c30 2023-09-13T04:40:01.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.483+0000 7fe84b7fe700 1 -- 172.21.15.2:0/175983915 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe860007e40 con 0x557a29df5c30 2023-09-13T04:40:01.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.483+0000 7fe84b7fe700 1 -- 172.21.15.2:0/175983915 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe85c009920 con 0x557a29df5c30 2023-09-13T04:40:01.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.483+0000 7fe8861cdb80 10 client.15306 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:01.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.483+0000 7fe8861cdb80 10 client.15306 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-13T04:40:01.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.483+0000 7fe8861cdb80 20 client.15306 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:01.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.483+0000 7fe8861cdb80 1 --2- 172.21.15.2:0/175983915 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x557a29e99010 0x557a29f01650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:01.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.483+0000 7fe8861cdb80 4 client.15306 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "status"}] 2023-09-13T04:40:01.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.483+0000 7fe8861cdb80 1 -- 172.21.15.2:0/175983915 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557a29da3000 con 0x557a29e99010 2023-09-13T04:40:01.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.484+0000 7fe86b7fe700 1 --2- 172.21.15.2:0/175983915 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x557a29e99010 0x557a29f01650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:01.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.484+0000 7fe84b7fe700 10 client.15306 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:01.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.484+0000 7fe86b7fe700 1 --2- 172.21.15.2:0/175983915 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x557a29e99010 0x557a29f01650 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7c1be60626fc6c9 server_cookie=700d598038aa1a2e in_seq=0 out_seq=0 2023-09-13T04:40:01.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.485+0000 7fe84b7fe700 1 -- 172.21.15.2:0/175983915 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x557a29da3000 con 0x557a29e99010 2023-09-13T04:40:01.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.485+0000 7fe84b7fe700 10 client.15306 handle_command_reply: tid=0 2023-09-13T04:40:01.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.485+0000 7fe8861cdb80 1 client.15306 shutdown 2023-09-13T04:40:01.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.485+0000 7fe8861cdb80 1 -- 172.21.15.2:0/175983915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29df5c30 msgr2=0x557a29ea2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:01.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.485+0000 7fe8861cdb80 1 --2- 172.21.15.2:0/175983915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29df5c30 0x557a29ea2f80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe85c009230 tx=0x7fe85c027170 comp rx=0 tx=0).stop 2023-09-13T04:40:01.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.485+0000 7fe8861cdb80 1 -- 172.21.15.2:0/175983915 shutdown_connections 2023-09-13T04:40:01.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.485+0000 7fe8861cdb80 1 --2- 172.21.15.2:0/175983915 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x557a29e99010 0x557a29f01650 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.485+0000 7fe8861cdb80 1 --2- 172.21.15.2:0/175983915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a29e986b0 0x557a29ea3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.485+0000 7fe8861cdb80 1 --2- 172.21.15.2:0/175983915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a29df5c30 0x557a29ea2f80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.485+0000 7fe84b7fe700 0 client.15306 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:01.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.485+0000 7fe8861cdb80 1 --2- 172.21.15.2:0/175983915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a29e979b0 0x557a29ea84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.485+0000 7fe8861cdb80 1 -- 172.21.15.2:0/175983915 >> 172.21.15.2:0/175983915 conn(0x557a29e921f0 msgr2=0x557a29ea0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:01.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.485+0000 7fe8861cdb80 1 -- 172.21.15.2:0/175983915 shutdown_connections 2023-09-13T04:40:01.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.485+0000 7fe8861cdb80 1 -- 172.21.15.2:0/175983915 wait complete. 2023-09-13T04:40:01.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.485+0000 7fe8861cdb80 20 client.15306 trim_cache size 0 max 16384 2023-09-13T04:40:01.500 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:40:01.500 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da", 2023-09-13T04:40:01.501 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 0, 2023-09-13T04:40:01.501 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24566, 2023-09-13T04:40:01.501 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2023-09-13T04:40:01.501 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2023-09-13T04:40:01.501 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2023-09-13T04:40:01.502 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 121.910998921, 2023-09-13T04:40:01.502 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 23, 2023-09-13T04:40:01.502 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 92, 2023-09-13T04:40:01.502 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 90, 2023-09-13T04:40:01.502 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 128.629507644 2023-09-13T04:40:01.503 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:40:01.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.486+0000 7fe8727ff700 1 -- 172.21.15.2:0/1313825014 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe8540a0640 msgr2=0x7fe8540a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:01.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.486+0000 7fe8727ff700 1 --2- 172.21.15.2:0/1313825014 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe8540a0640 0x7fe8540a2b00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe864009fd0 tx=0x7fe86400b040 comp rx=0 tx=0).stop 2023-09-13T04:40:01.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.486+0000 7fe8727ff700 1 -- 172.21.15.2:0/1313825014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe86c074cf0 msgr2=0x7fe86c1991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:01.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.486+0000 7fe8727ff700 1 --2- 172.21.15.2:0/1313825014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe86c074cf0 0x7fe86c1991c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe86000e9f0 tx=0x7fe86000ed00 comp rx=0 tx=0).stop 2023-09-13T04:40:01.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.486+0000 7fe8727ff700 1 -- 172.21.15.2:0/1313825014 shutdown_connections 2023-09-13T04:40:01.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.486+0000 7fe8727ff700 1 --2- 172.21.15.2:0/1313825014 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fe8540a0640 0x7fe8540a2b00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.486+0000 7fe8727ff700 1 --2- 172.21.15.2:0/1313825014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe86c108bb0 0x7fe86c1a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.486+0000 7fe8727ff700 1 --2- 172.21.15.2:0/1313825014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe86c073710 0x7fe86c198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.486+0000 7fe8727ff700 1 --2- 172.21.15.2:0/1313825014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe86c074cf0 0x7fe86c1991c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:01.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.486+0000 7fe8727ff700 1 -- 172.21.15.2:0/1313825014 >> 172.21.15.2:0/1313825014 conn(0x7fe86c0fc260 msgr2=0x7fe86c0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:01.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.486+0000 7fe8727ff700 1 -- 172.21.15.2:0/1313825014 shutdown_connections 2023-09-13T04:40:01.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:01.486+0000 7fe8727ff700 1 -- 172.21.15.2:0/1313825014 wait complete. 2023-09-13T04:40:01.507 INFO:teuthology.run_tasks:Running task kclient... 2023-09-13T04:40:01.523 INFO:tasks.kclient:Mounting kernel clients... 2023-09-13T04:40:01.524 INFO:tasks.kclient:config is {'client.0': {}} 2023-09-13T04:40:01.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:40:01.524 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2023-09-13T04:40:01.569 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-09-13T04:40:01.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:40:01.570 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-13T04:40:01.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:40:01.682 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link delete ceph-brx 2023-09-13T04:40:01.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:01 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1276086351' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:40:01.798 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1'} 2023-09-13T04:40:01.798 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-13T04:40:01.799 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-13T04:40:01.799 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph fs ls 2023-09-13T04:40:01.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:01 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1276086351' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:40:01.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:01 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1276086351' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:40:02.543 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:40:02.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:02 smithi149 ceph-mon[133315]: pgmap v341: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-13T04:40:02.905 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:02 smithi002 ceph-mon[97651]: pgmap v341: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-13T04:40:02.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:02 smithi156 ceph-mon[133675]: pgmap v341: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-09-13T04:40:04.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.032+0000 7f97670bb700 1 -- 172.21.15.2:0/4142270213 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9760101740 msgr2=0x7f976010dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:04.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.032+0000 7f97670bb700 1 --2- 172.21.15.2:0/4142270213 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9760101740 0x7f976010dbb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f975800b0d0 tx=0x7f975800b3e0 comp rx=0 tx=0).stop 2023-09-13T04:40:04.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.032+0000 7f97670bb700 1 -- 172.21.15.2:0/4142270213 shutdown_connections 2023-09-13T04:40:04.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.032+0000 7f97670bb700 1 --2- 172.21.15.2:0/4142270213 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9760101740 0x7f976010dbb0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:04.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.032+0000 7f97670bb700 1 --2- 172.21.15.2:0/4142270213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9760106d90 0x7f9760107170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:04.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.032+0000 7f97670bb700 1 --2- 172.21.15.2:0/4142270213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9760100d80 0x7f9760101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:04.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.032+0000 7f97670bb700 1 -- 172.21.15.2:0/4142270213 >> 172.21.15.2:0/4142270213 conn(0x7f9760077ea0 msgr2=0x7f97600782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:04.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.032+0000 7f97670bb700 1 -- 172.21.15.2:0/4142270213 shutdown_connections 2023-09-13T04:40:04.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.032+0000 7f97670bb700 1 -- 172.21.15.2:0/4142270213 wait complete. 2023-09-13T04:40:04.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.033+0000 7f97670bb700 1 Processor -- start 2023-09-13T04:40:04.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.034+0000 7f97670bb700 1 -- start start 2023-09-13T04:40:04.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.034+0000 7f97670bb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9760100d80 0x7f9760199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:04.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.034+0000 7f97670bb700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9760101740 0x7f9760199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:04.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.034+0000 7f97670bb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9760106d90 0x7f976019dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:04.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.034+0000 7f97670bb700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976019e0f0 con 0x7f9760100d80 2023-09-13T04:40:04.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.034+0000 7f97670bb700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976019e260 con 0x7f9760101740 2023-09-13T04:40:04.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.034+0000 7f97670bb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976019e420 con 0x7f9760106d90 2023-09-13T04:40:04.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.034+0000 7f9764e57700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9760100d80 0x7f9760199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:04.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.034+0000 7f9764e57700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9760100d80 0x7f9760199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47558/0 (socket says 172.21.15.2:47558) 2023-09-13T04:40:04.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.034+0000 7f9764e57700 1 -- 172.21.15.2:0/1911974754 learned_addr learned my addr 172.21.15.2:0/1911974754 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:04.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.034+0000 7f975ffff700 1 --2- 172.21.15.2:0/1911974754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9760101740 0x7f9760199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:04.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.034+0000 7f9765658700 1 --2- 172.21.15.2:0/1911974754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9760106d90 0x7f976019dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:04.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.035+0000 7f9764e57700 1 -- 172.21.15.2:0/1911974754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9760106d90 msgr2=0x7f976019dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:04.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.035+0000 7f9764e57700 1 --2- 172.21.15.2:0/1911974754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9760106d90 0x7f976019dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:04.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.035+0000 7f9764e57700 1 -- 172.21.15.2:0/1911974754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9760101740 msgr2=0x7f9760199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:04.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.035+0000 7f9764e57700 1 --2- 172.21.15.2:0/1911974754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9760101740 0x7f9760199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:04.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.035+0000 7f9764e57700 1 -- 172.21.15.2:0/1911974754 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9758009d00 con 0x7f9760100d80 2023-09-13T04:40:04.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.035+0000 7f9765658700 1 --2- 172.21.15.2:0/1911974754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9760106d90 0x7f976019dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:40:04.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.035+0000 7f975ffff700 1 --2- 172.21.15.2:0/1911974754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9760101740 0x7f9760199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:04.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.035+0000 7f9764e57700 1 --2- 172.21.15.2:0/1911974754 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9760100d80 0x7f9760199130 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f975400e9a0 tx=0x7f975400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:04.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.035+0000 7f975dffb700 1 -- 172.21.15.2:0/1911974754 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9754009790 con 0x7f9760100d80 2023-09-13T04:40:04.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.035+0000 7f975dffb700 1 -- 172.21.15.2:0/1911974754 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f975400cbd0 con 0x7f9760100d80 2023-09-13T04:40:04.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.035+0000 7f97670bb700 1 -- 172.21.15.2:0/1911974754 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f976019e700 con 0x7f9760100d80 2023-09-13T04:40:04.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.035+0000 7f975dffb700 1 -- 172.21.15.2:0/1911974754 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9754018680 con 0x7f9760100d80 2023-09-13T04:40:04.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.035+0000 7f97670bb700 1 -- 172.21.15.2:0/1911974754 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97601a5f60 con 0x7f9760100d80 2023-09-13T04:40:04.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.036+0000 7f97670bb700 1 -- 172.21.15.2:0/1911974754 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f976004e9b0 con 0x7f9760100d80 2023-09-13T04:40:04.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.038+0000 7f975dffb700 1 -- 172.21.15.2:0/1911974754 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f975400cd40 con 0x7f9760100d80 2023-09-13T04:40:04.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.041+0000 7f975dffb700 1 --2- 172.21.15.2:0/1911974754 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f97500a06c0 0x7f97500a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:04.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.041+0000 7f975dffb700 1 -- 172.21.15.2:0/1911974754 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9754014070 con 0x7f9760100d80 2023-09-13T04:40:04.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.041+0000 7f975ffff700 1 --2- 172.21.15.2:0/1911974754 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f97500a06c0 0x7f97500a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:04.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.041+0000 7f975ffff700 1 --2- 172.21.15.2:0/1911974754 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f97500a06c0 0x7f97500a2b80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f976019a750 tx=0x7f974c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:04.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.045+0000 7f975dffb700 1 -- 172.21.15.2:0/1911974754 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9754051e10 con 0x7f9760100d80 2023-09-13T04:40:04.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.302+0000 7f97670bb700 1 -- 172.21.15.2:0/1911974754 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f976019a490 con 0x7f9760100d80 2023-09-13T04:40:04.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.302+0000 7f975dffb700 1 -- 172.21.15.2:0/1911974754 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+88 (secure 0 0 0) 0x7f9754095860 con 0x7f9760100d80 2023-09-13T04:40:04.304 INFO:teuthology.orchestra.run.smithi002.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-09-13T04:40:04.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.305+0000 7f97670bb700 1 -- 172.21.15.2:0/1911974754 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f97500a06c0 msgr2=0x7f97500a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:04.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.305+0000 7f97670bb700 1 --2- 172.21.15.2:0/1911974754 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f97500a06c0 0x7f97500a2b80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f976019a750 tx=0x7f974c009260 comp rx=0 tx=0).stop 2023-09-13T04:40:04.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.305+0000 7f97670bb700 1 -- 172.21.15.2:0/1911974754 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9760100d80 msgr2=0x7f9760199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:04.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.305+0000 7f97670bb700 1 --2- 172.21.15.2:0/1911974754 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9760100d80 0x7f9760199130 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f975400e9a0 tx=0x7f975400ecb0 comp rx=0 tx=0).stop 2023-09-13T04:40:04.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.305+0000 7f97670bb700 1 -- 172.21.15.2:0/1911974754 shutdown_connections 2023-09-13T04:40:04.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.305+0000 7f97670bb700 1 --2- 172.21.15.2:0/1911974754 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f97500a06c0 0x7f97500a2b80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:04.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.305+0000 7f97670bb700 1 --2- 172.21.15.2:0/1911974754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9760106d90 0x7f976019dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:04.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.305+0000 7f97670bb700 1 --2- 172.21.15.2:0/1911974754 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9760100d80 0x7f9760199130 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:04.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.305+0000 7f97670bb700 1 --2- 172.21.15.2:0/1911974754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9760101740 0x7f9760199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:04.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.305+0000 7f97670bb700 1 -- 172.21.15.2:0/1911974754 >> 172.21.15.2:0/1911974754 conn(0x7f9760077ea0 msgr2=0x7f976010bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:04.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.305+0000 7f97670bb700 1 -- 172.21.15.2:0/1911974754 shutdown_connections 2023-09-13T04:40:04.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:04.305+0000 7f97670bb700 1 -- 172.21.15.2:0/1911974754 wait complete. 2023-09-13T04:40:04.824 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-13T04:40:04.824 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-13T04:40:04.824 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi002.front.sepia.ceph.com 2023-09-13T04:40:04.825 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-13T04:40:04.825 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-13T04:40:04.825 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-13T04:40:04.825 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-13T04:40:04.826 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-13T04:40:04.826 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-13T04:40:04.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:40:04.827 DEBUG:teuthology.orchestra.run.smithi002:> ip addr 2023-09-13T04:40:04.872 INFO:teuthology.orchestra.run.smithi002.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-13T04:40:04.872 INFO:teuthology.orchestra.run.smithi002.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-13T04:40:04.872 INFO:teuthology.orchestra.run.smithi002.stdout: inet 127.0.0.1/8 scope host lo 2023-09-13T04:40:04.873 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-09-13T04:40:04.873 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 ::1/128 scope host 2023-09-13T04:40:04.873 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-09-13T04:40:04.873 INFO:teuthology.orchestra.run.smithi002.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-13T04:40:04.873 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:02 brd ff:ff:ff:ff:ff:ff 2023-09-13T04:40:04.874 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp7s0f0 2023-09-13T04:40:04.874 INFO:teuthology.orchestra.run.smithi002.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-13T04:40:04.874 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:03 brd ff:ff:ff:ff:ff:ff 2023-09-13T04:40:04.874 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp7s0f1 2023-09-13T04:40:04.874 INFO:teuthology.orchestra.run.smithi002.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-13T04:40:04.875 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:20 brd ff:ff:ff:ff:ff:ff 2023-09-13T04:40:04.875 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp1s0f0 2023-09-13T04:40:04.875 INFO:teuthology.orchestra.run.smithi002.stdout: inet 172.21.15.2/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-09-13T04:40:04.875 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft 41627sec preferred_lft 41627sec 2023-09-13T04:40:04.876 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 fe80::ec4:7aff:febd:1920/64 scope link 2023-09-13T04:40:04.876 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-09-13T04:40:04.876 INFO:teuthology.orchestra.run.smithi002.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-13T04:40:04.876 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:21 brd ff:ff:ff:ff:ff:ff 2023-09-13T04:40:04.876 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp1s0f1 2023-09-13T04:40:04.877 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-13T04:40:04.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:40:04.878 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-09-13T04:40:04.878 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-09-13T04:40:04.879 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add name ceph-brx type bridge 2023-09-13T04:40:04.879 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr flush dev ceph-brx 2023-09-13T04:40:04.879 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set ceph-brx up 2023-09-13T04:40:04.879 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-13T04:40:04.880 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-09-13T04:40:04.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:04 smithi156 ceph-mon[133675]: pgmap v342: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:40:04.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:04 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1911974754' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-13T04:40:05.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:04 smithi002 ceph-mon[97651]: pgmap v342: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:40:05.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:04 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1911974754' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-13T04:40:05.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:40:05.129 DEBUG:teuthology.orchestra.run.smithi002:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-13T04:40:05.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:04 smithi149 ceph-mon[133315]: pgmap v342: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:40:05.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:04 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1911974754' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-13T04:40:05.201 INFO:teuthology.orchestra.run.smithi002.stdout:1 2023-09-13T04:40:05.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:40:05.204 DEBUG:teuthology.orchestra.run.smithi002:> route 2023-09-13T04:40:05.251 INFO:teuthology.orchestra.run.smithi002.stdout:Kernel IP routing table 2023-09-13T04:40:05.251 INFO:teuthology.orchestra.run.smithi002.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-13T04:40:05.252 INFO:teuthology.orchestra.run.smithi002.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-09-13T04:40:05.252 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-09-13T04:40:05.252 INFO:teuthology.orchestra.run.smithi002.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-13T04:40:05.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:40:05.253 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-09-13T04:40:05.254 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-09-13T04:40:05.254 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-09-13T04:40:05.254 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-09-13T04:40:05.255 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-09-13T04:40:05.255 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-09-13T04:40:05.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:40:05.431 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2023-09-13T04:40:05.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:40:05.476 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list-id 2023-09-13T04:40:05.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:40:05.561 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-09-13T04:40:05.561 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-09-13T04:40:05.561 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-13T04:40:05.562 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-09-13T04:40:05.562 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-09-13T04:40:05.748 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-09-13T04:40:05.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:40:05.749 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-09-13T04:40:05.749 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-09-13T04:40:05.749 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-09-13T04:40:05.749 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-13T04:40:05.750 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-09-13T04:40:05.750 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-09-13T04:40:05.750 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-09-13T04:40:05.751 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-09-13T04:40:06.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:40:06.002 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-09-13T04:40:06.002 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-09-13T04:40:06.003 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set brx.0 up 2023-09-13T04:40:06.003 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set dev brx.0 master ceph-brx 2023-09-13T04:40:06.003 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-09-13T04:40:06.179 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-13T04:40:06.179 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-13T04:40:06.223 INFO:teuthology.orchestra.run.smithi002.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-13T04:40:06.223 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-13T04:40:06.224 DEBUG:teuthology.orchestra.run.smithi002:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-13T04:40:06.306 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-09-13T04:40:06.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:40:06.306 DEBUG:teuthology.orchestra.run.smithi002:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2023-09-13T04:40:06.452 INFO:teuthology.orchestra.run.smithi002.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2023-09-13T04:40:06.453 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2023-09-13T04:40:06.453 INFO:teuthology.orchestra.run.smithi002.stdout:invalid new device string format 2023-09-13T04:40:06.732 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:06 smithi002 ceph-mon[97651]: pgmap v343: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:40:06.931 INFO:teuthology.orchestra.run.smithi002.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2023-09-13T04:40:06.931 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2023-09-13T04:40:06.931 INFO:teuthology.orchestra.run.smithi002.stdout:invalid new device string format 2023-09-13T04:40:06.931 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: resolved to: "172.21.15.2:6789,172.21.15.149:6789,172.21.15.156:6789" 2023-09-13T04:40:06.932 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.2:6789,172.21.15.149:6789,172.21.15.156:6789:/ 2023-09-13T04:40:06.932 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da" will pass to kernel 2023-09-13T04:40:06.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:06 smithi156 ceph-mon[133675]: pgmap v343: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:40:06.937 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-09-13T04:40:06.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:40:06.938 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-13T04:40:07.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:40:07.010 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-09-13T04:40:07.096 INFO:teuthology.orchestra.run.smithi002.stdout:client25348 2023-09-13T04:40:07.097 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-09-13T04:40:07.097 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-13T04:40:07.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:06 smithi149 ceph-mon[133315]: pgmap v343: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:40:07.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:40:07.142 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-09-13T04:40:07.272 INFO:teuthology.orchestra.run.smithi002.stdout:client25348 2023-09-13T04:40:07.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:40:07.273 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-09-13T04:40:07.359 INFO:teuthology.orchestra.run.smithi002.stdout:client25348 2023-09-13T04:40:07.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:40:07.360 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da.client25348/status') 2023-09-13T04:40:07.469 INFO:teuthology.orchestra.run.smithi002.stdout:instance: client.25348 (3)172.21.15.2:0/3172454000 2023-09-13T04:40:07.469 INFO:teuthology.orchestra.run.smithi002.stdout:blocklisted: false 2023-09-13T04:40:07.470 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records in 2023-09-13T04:40:07.470 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records out 2023-09-13T04:40:07.470 INFO:teuthology.orchestra.run.smithi002.stderr:70 bytes copied, 5.6288e-05 s, 1.2 MB/s 2023-09-13T04:40:07.472 INFO:teuthology.run_tasks:Running task check-counter... 2023-09-13T04:40:07.486 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-09-13T04:40:07.563 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T04:40:07.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.769+0000 7f1dd0bff700 1 -- 172.21.15.2:0/2384795692 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1dcc106a00 msgr2=0x7f1dcc108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:07.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.769+0000 7f1dd0bff700 1 --2- 172.21.15.2:0/2384795692 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1dcc106a00 0x7f1dcc108e70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400b0a0 tx=0x7f1dc400b3b0 comp rx=0 tx=0).stop 2023-09-13T04:40:07.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.769+0000 7f1dd0bff700 1 -- 172.21.15.2:0/2384795692 shutdown_connections 2023-09-13T04:40:07.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.769+0000 7f1dd0bff700 1 --2- 172.21.15.2:0/2384795692 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1dcc106a00 0x7f1dcc108e70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:07.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.769+0000 7f1dd0bff700 1 --2- 172.21.15.2:0/2384795692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1dcc1017a0 0x7f1dcc103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:07.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.769+0000 7f1dd0bff700 1 --2- 172.21.15.2:0/2384795692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dcc1040d0 0x7f1dcc1064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:07.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.769+0000 7f1dd0bff700 1 -- 172.21.15.2:0/2384795692 >> 172.21.15.2:0/2384795692 conn(0x7f1dcc0fb1b0 msgr2=0x7f1dcc0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:07.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.769+0000 7f1dd0bff700 1 -- 172.21.15.2:0/2384795692 shutdown_connections 2023-09-13T04:40:07.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.769+0000 7f1dd0bff700 1 -- 172.21.15.2:0/2384795692 wait complete. 2023-09-13T04:40:07.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.770+0000 7f1dd0bff700 1 Processor -- start 2023-09-13T04:40:07.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.770+0000 7f1dd0bff700 1 -- start start 2023-09-13T04:40:07.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.771+0000 7f1dd0bff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1dcc1017a0 0x7f1dcc198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:07.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.771+0000 7f1dd0bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dcc1040d0 0x7f1dcc1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:07.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.771+0000 7f1dd0bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1dcc106a00 0x7f1dcc1a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:07.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.771+0000 7f1dd0bff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc1a07a0 con 0x7f1dcc1017a0 2023-09-13T04:40:07.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.771+0000 7f1dd0bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc1a0910 con 0x7f1dcc1040d0 2023-09-13T04:40:07.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.771+0000 7f1dd0bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc1a0ad0 con 0x7f1dcc106a00 2023-09-13T04:40:07.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.771+0000 7f1dca59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1dcc1017a0 0x7f1dcc198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:07.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.771+0000 7f1dc9d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dcc1040d0 0x7f1dcc1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:07.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.771+0000 7f1dc9d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dcc1040d0 0x7f1dcc1991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:45938/0 (socket says 172.21.15.2:45938) 2023-09-13T04:40:07.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.771+0000 7f1dc9d9b700 1 -- 172.21.15.2:0/334413375 learned_addr learned my addr 172.21.15.2:0/334413375 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:07.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.771+0000 7f1dcad9d700 1 --2- 172.21.15.2:0/334413375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1dcc106a00 0x7f1dcc1a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:07.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.772+0000 7f1dc9d9b700 1 -- 172.21.15.2:0/334413375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1dcc106a00 msgr2=0x7f1dcc1a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:07.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.772+0000 7f1dc9d9b700 1 --2- 172.21.15.2:0/334413375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1dcc106a00 0x7f1dcc1a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:07.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.772+0000 7f1dc9d9b700 1 -- 172.21.15.2:0/334413375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1dcc1017a0 msgr2=0x7f1dcc198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:07.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.772+0000 7f1dc9d9b700 1 --2- 172.21.15.2:0/334413375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1dcc1017a0 0x7f1dcc198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:07.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.772+0000 7f1dc9d9b700 1 -- 172.21.15.2:0/334413375 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc4009d00 con 0x7f1dcc1040d0 2023-09-13T04:40:07.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.772+0000 7f1dca59c700 1 --2- 172.21.15.2:0/334413375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1dcc1017a0 0x7f1dcc198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:07.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.772+0000 7f1dc9d9b700 1 --2- 172.21.15.2:0/334413375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dcc1040d0 0x7f1dcc1991a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1dc000e9a0 tx=0x7f1dc000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:07.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.772+0000 7f1dbb7fe700 1 -- 172.21.15.2:0/334413375 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0009790 con 0x7f1dcc1040d0 2023-09-13T04:40:07.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.772+0000 7f1dd0bff700 1 -- 172.21.15.2:0/334413375 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dcc1a0db0 con 0x7f1dcc1040d0 2023-09-13T04:40:07.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.772+0000 7f1dbb7fe700 1 -- 172.21.15.2:0/334413375 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dc0010430 con 0x7f1dcc1040d0 2023-09-13T04:40:07.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.772+0000 7f1dbb7fe700 1 -- 172.21.15.2:0/334413375 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0018680 con 0x7f1dcc1040d0 2023-09-13T04:40:07.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.772+0000 7f1dd0bff700 1 -- 172.21.15.2:0/334413375 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dcc1a1280 con 0x7f1dcc1040d0 2023-09-13T04:40:07.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.773+0000 7f1dd0bff700 1 -- 172.21.15.2:0/334413375 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dcc108580 con 0x7f1dcc1040d0 2023-09-13T04:40:07.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.775+0000 7f1dbb7fe700 1 -- 172.21.15.2:0/334413375 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f1dc0010810 con 0x7f1dcc1040d0 2023-09-13T04:40:07.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.777+0000 7f1dbb7fe700 1 --2- 172.21.15.2:0/334413375 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1db40a0710 0x7f1db40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:07.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.777+0000 7f1dbb7fe700 1 -- 172.21.15.2:0/334413375 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1dc0014070 con 0x7f1dcc1040d0 2023-09-13T04:40:07.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.777+0000 7f1dca59c700 1 --2- 172.21.15.2:0/334413375 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1db40a0710 0x7f1db40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:07.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.777+0000 7f1dca59c700 1 --2- 172.21.15.2:0/334413375 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1db40a0710 0x7f1db40a2bd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc19a130 tx=0x7f1dbc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:07.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.779+0000 7f1dbb7fe700 1 -- 172.21.15.2:0/334413375 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1dc0051e10 con 0x7f1dcc1040d0 2023-09-13T04:40:07.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.993+0000 7f1dd0bff700 1 -- 172.21.15.2:0/334413375 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1dcc10c750 con 0x7f1dcc1040d0 2023-09-13T04:40:07.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.995+0000 7f1dbb7fe700 1 -- 172.21.15.2:0/334413375 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24503 (secure 0 0 0) 0x7f1dc00958e0 con 0x7f1dcc1040d0 2023-09-13T04:40:07.996 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:40:07.997 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":92,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:39:25.580735+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T04:37:57.094802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T04:37:58.564487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-13T04:38:02.096895+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:40:07.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.997+0000 7f1dd0bff700 1 -- 172.21.15.2:0/334413375 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1db40a0710 msgr2=0x7f1db40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:07.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.997+0000 7f1dd0bff700 1 --2- 172.21.15.2:0/334413375 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1db40a0710 0x7f1db40a2bd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc19a130 tx=0x7f1dbc009260 comp rx=0 tx=0).stop 2023-09-13T04:40:07.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.997+0000 7f1dd0bff700 1 -- 172.21.15.2:0/334413375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dcc1040d0 msgr2=0x7f1dcc1991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:07.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.997+0000 7f1dd0bff700 1 --2- 172.21.15.2:0/334413375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dcc1040d0 0x7f1dcc1991a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1dc000e9a0 tx=0x7f1dc000ecb0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.998+0000 7f1dd0bff700 1 -- 172.21.15.2:0/334413375 shutdown_connections 2023-09-13T04:40:08.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.998+0000 7f1dd0bff700 1 --2- 172.21.15.2:0/334413375 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1db40a0710 0x7f1db40a2bd0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.998+0000 7f1dd0bff700 1 --2- 172.21.15.2:0/334413375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1dcc106a00 0x7f1dcc1a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.998+0000 7f1dd0bff700 1 --2- 172.21.15.2:0/334413375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1dcc1017a0 0x7f1dcc198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.998+0000 7f1dd0bff700 1 --2- 172.21.15.2:0/334413375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dcc1040d0 0x7f1dcc1991a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.998+0000 7f1dd0bff700 1 -- 172.21.15.2:0/334413375 >> 172.21.15.2:0/334413375 conn(0x7f1dcc0fb1b0 msgr2=0x7f1dcc0ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:08.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.998+0000 7f1dd0bff700 1 -- 172.21.15.2:0/334413375 shutdown_connections 2023-09-13T04:40:08.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:07.998+0000 7f1dd0bff700 1 -- 172.21.15.2:0/334413375 wait complete. 2023-09-13T04:40:08.018 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T04:33:42.025714+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-13T04:37:57.094802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-13T04:37:58.564487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-13T04:38:02.096895+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-13T04:40:08.018 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-13T04:40:08.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.225+0000 7f5702fff700 1 -- 172.21.15.2:0/2894989039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56fc101ed0 msgr2=0x7f56fc102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:08.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.225+0000 7f5702fff700 1 --2- 172.21.15.2:0/2894989039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56fc101ed0 0x7f56fc102330 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f56e80099c0 tx=0x7f56e8009cd0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.225+0000 7f5702fff700 1 -- 172.21.15.2:0/2894989039 shutdown_connections 2023-09-13T04:40:08.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.225+0000 7f5702fff700 1 --2- 172.21.15.2:0/2894989039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56fc102870 0x7f56fc109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.225+0000 7f5702fff700 1 --2- 172.21.15.2:0/2894989039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56fc100ce0 0x7f56fc101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.225+0000 7f5702fff700 1 --2- 172.21.15.2:0/2894989039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56fc101ed0 0x7f56fc102330 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.225+0000 7f5702fff700 1 -- 172.21.15.2:0/2894989039 >> 172.21.15.2:0/2894989039 conn(0x7f56fc0fc230 msgr2=0x7f56fc0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:08.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.225+0000 7f5702fff700 1 -- 172.21.15.2:0/2894989039 shutdown_connections 2023-09-13T04:40:08.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.225+0000 7f5702fff700 1 -- 172.21.15.2:0/2894989039 wait complete. 2023-09-13T04:40:08.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.227+0000 7f5702fff700 1 Processor -- start 2023-09-13T04:40:08.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.227+0000 7f5702fff700 1 -- start start 2023-09-13T04:40:08.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.227+0000 7f5702fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56fc100ce0 0x7f56fc1969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:08.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.227+0000 7f5702fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56fc101ed0 0x7f56fc196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:08.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.228+0000 7f5702fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56fc102870 0x7f56fc19df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:08.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.228+0000 7f5702fff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc19e530 con 0x7f56fc101ed0 2023-09-13T04:40:08.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.228+0000 7f5702fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc19e670 con 0x7f56fc100ce0 2023-09-13T04:40:08.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.228+0000 7f5702fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc19e800 con 0x7f56fc102870 2023-09-13T04:40:08.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.228+0000 7f56fbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56fc101ed0 0x7f56fc196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:08.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.228+0000 7f5700d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56fc100ce0 0x7f56fc1969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:08.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.228+0000 7f56fbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56fc101ed0 0x7f56fc196ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47586/0 (socket says 172.21.15.2:47586) 2023-09-13T04:40:08.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.228+0000 7f56fbfff700 1 -- 172.21.15.2:0/2642703655 learned_addr learned my addr 172.21.15.2:0/2642703655 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:08.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.228+0000 7f570159c700 1 --2- 172.21.15.2:0/2642703655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56fc102870 0x7f56fc19df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:08.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.228+0000 7f56fbfff700 1 -- 172.21.15.2:0/2642703655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56fc102870 msgr2=0x7f56fc19df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:08.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.228+0000 7f56fbfff700 1 --2- 172.21.15.2:0/2642703655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56fc102870 0x7f56fc19df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.228+0000 7f56fbfff700 1 -- 172.21.15.2:0/2642703655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56fc100ce0 msgr2=0x7f56fc1969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:08.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.228+0000 7f56fbfff700 1 --2- 172.21.15.2:0/2642703655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56fc100ce0 0x7f56fc1969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.228+0000 7f56fbfff700 1 -- 172.21.15.2:0/2642703655 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e80095f0 con 0x7f56fc101ed0 2023-09-13T04:40:08.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.228+0000 7f5700d9b700 1 --2- 172.21.15.2:0/2642703655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56fc100ce0 0x7f56fc1969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:08.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.229+0000 7f56fbfff700 1 --2- 172.21.15.2:0/2642703655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56fc101ed0 0x7f56fc196ef0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f56e800c010 tx=0x7f56e800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:08.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.229+0000 7f56f9ffb700 1 -- 172.21.15.2:0/2642703655 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e801d070 con 0x7f56fc101ed0 2023-09-13T04:40:08.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.229+0000 7f56f9ffb700 1 -- 172.21.15.2:0/2642703655 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56e8005050 con 0x7f56fc101ed0 2023-09-13T04:40:08.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.229+0000 7f5702fff700 1 -- 172.21.15.2:0/2642703655 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56fc19ea80 con 0x7f56fc101ed0 2023-09-13T04:40:08.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.229+0000 7f56f9ffb700 1 -- 172.21.15.2:0/2642703655 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8004ad0 con 0x7f56fc101ed0 2023-09-13T04:40:08.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.229+0000 7f5702fff700 1 -- 172.21.15.2:0/2642703655 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56fc19eea0 con 0x7f56fc101ed0 2023-09-13T04:40:08.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.230+0000 7f5702fff700 1 -- 172.21.15.2:0/2642703655 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56fc10c560 con 0x7f56fc101ed0 2023-09-13T04:40:08.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.232+0000 7f56f9ffb700 1 -- 172.21.15.2:0/2642703655 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f56e80051c0 con 0x7f56fc101ed0 2023-09-13T04:40:08.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.233+0000 7f56f9ffb700 1 --2- 172.21.15.2:0/2642703655 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f56ec0a0640 0x7f56ec0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:08.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.233+0000 7f56f9ffb700 1 -- 172.21.15.2:0/2642703655 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f56e80ccdf0 con 0x7f56fc101ed0 2023-09-13T04:40:08.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.233+0000 7f5700d9b700 1 --2- 172.21.15.2:0/2642703655 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f56ec0a0640 0x7f56ec0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:08.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.234+0000 7f5700d9b700 1 --2- 172.21.15.2:0/2642703655 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f56ec0a0640 0x7f56ec0a2b00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f56fc101d30 tx=0x7f56f0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:08.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.238+0000 7f56f9ffb700 1 -- 172.21.15.2:0/2642703655 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f56e8096620 con 0x7f56fc101ed0 2023-09-13T04:40:08.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.451+0000 7f5702fff700 1 -- 172.21.15.2:0/2642703655 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f56fc107c30 con 0x7f56fc101ed0 2023-09-13T04:40:08.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.452+0000 7f56f9ffb700 1 -- 172.21.15.2:0/2642703655 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f56e8099c40 con 0x7f56fc101ed0 2023-09-13T04:40:08.453 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-09-13T04:40:08.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.454+0000 7f5702fff700 1 -- 172.21.15.2:0/2642703655 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f56ec0a0640 msgr2=0x7f56ec0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:08.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.454+0000 7f5702fff700 1 --2- 172.21.15.2:0/2642703655 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f56ec0a0640 0x7f56ec0a2b00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f56fc101d30 tx=0x7f56f0009260 comp rx=0 tx=0).stop 2023-09-13T04:40:08.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.454+0000 7f5702fff700 1 -- 172.21.15.2:0/2642703655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56fc101ed0 msgr2=0x7f56fc196ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:08.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.454+0000 7f5702fff700 1 --2- 172.21.15.2:0/2642703655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56fc101ed0 0x7f56fc196ef0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f56e800c010 tx=0x7f56e800f690 comp rx=0 tx=0).stop 2023-09-13T04:40:08.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.454+0000 7f5702fff700 1 -- 172.21.15.2:0/2642703655 shutdown_connections 2023-09-13T04:40:08.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.454+0000 7f5702fff700 1 --2- 172.21.15.2:0/2642703655 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f56ec0a0640 0x7f56ec0a2b00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.454+0000 7f5702fff700 1 --2- 172.21.15.2:0/2642703655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56fc102870 0x7f56fc19df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.454+0000 7f5702fff700 1 --2- 172.21.15.2:0/2642703655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56fc100ce0 0x7f56fc1969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.454+0000 7f5702fff700 1 --2- 172.21.15.2:0/2642703655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56fc101ed0 0x7f56fc196ef0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.454+0000 7f5702fff700 1 -- 172.21.15.2:0/2642703655 >> 172.21.15.2:0/2642703655 conn(0x7f56fc0fc230 msgr2=0x7f56fc0fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:08.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.454+0000 7f5702fff700 1 -- 172.21.15.2:0/2642703655 shutdown_connections 2023-09-13T04:40:08.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.454+0000 7f5702fff700 1 -- 172.21.15.2:0/2642703655 wait complete. 2023-09-13T04:40:08.474 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-13T04:40:08.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.682+0000 7f8d96fff700 1 -- 172.21.15.2:0/41849596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d90101ee0 msgr2=0x7f8d90102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:08.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.682+0000 7f8d96fff700 1 --2- 172.21.15.2:0/41849596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d90101ee0 0x7f8d90102340 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0099c0 tx=0x7f8d7c009cd0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.682+0000 7f8d96fff700 1 -- 172.21.15.2:0/41849596 shutdown_connections 2023-09-13T04:40:08.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.682+0000 7f8d96fff700 1 --2- 172.21.15.2:0/41849596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d90102880 0x7f8d90109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.682+0000 7f8d96fff700 1 --2- 172.21.15.2:0/41849596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d90100cf0 0x7f8d90101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.682+0000 7f8d96fff700 1 --2- 172.21.15.2:0/41849596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d90101ee0 0x7f8d90102340 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.682+0000 7f8d96fff700 1 -- 172.21.15.2:0/41849596 >> 172.21.15.2:0/41849596 conn(0x7f8d900fc260 msgr2=0x7f8d900fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:08.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.682+0000 7f8d96fff700 1 -- 172.21.15.2:0/41849596 shutdown_connections 2023-09-13T04:40:08.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.682+0000 7f8d96fff700 1 -- 172.21.15.2:0/41849596 wait complete. 2023-09-13T04:40:08.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.683+0000 7f8d96fff700 1 Processor -- start 2023-09-13T04:40:08.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.683+0000 7f8d96fff700 1 -- start start 2023-09-13T04:40:08.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.684+0000 7f8d96fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d90100cf0 0x7f8d90198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:08.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.684+0000 7f8d96fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d90101ee0 0x7f8d901991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:08.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.684+0000 7f8d96fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d90102880 0x7f8d901a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:08.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.684+0000 7f8d96fff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d901a07e0 con 0x7f8d90101ee0 2023-09-13T04:40:08.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.684+0000 7f8d96fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d901a0950 con 0x7f8d90102880 2023-09-13T04:40:08.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.684+0000 7f8d96fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d901a0b10 con 0x7f8d90100cf0 2023-09-13T04:40:08.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.684+0000 7f8d8ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d90101ee0 0x7f8d901991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:08.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.684+0000 7f8d94d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d90100cf0 0x7f8d90198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:08.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.684+0000 7f8d94d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d90100cf0 0x7f8d90198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:54356/0 (socket says 172.21.15.2:54356) 2023-09-13T04:40:08.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.684+0000 7f8d94d9b700 1 -- 172.21.15.2:0/4060542534 learned_addr learned my addr 172.21.15.2:0/4060542534 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:08.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.684+0000 7f8d9559c700 1 --2- 172.21.15.2:0/4060542534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d90102880 0x7f8d901a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:08.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.684+0000 7f8d8ffff700 1 -- 172.21.15.2:0/4060542534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d90100cf0 msgr2=0x7f8d90198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:08.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.684+0000 7f8d8ffff700 1 --2- 172.21.15.2:0/4060542534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d90100cf0 0x7f8d90198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.684+0000 7f8d8ffff700 1 -- 172.21.15.2:0/4060542534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d90102880 msgr2=0x7f8d901a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:08.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.684+0000 7f8d8ffff700 1 --2- 172.21.15.2:0/4060542534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d90102880 0x7f8d901a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.684+0000 7f8d8ffff700 1 -- 172.21.15.2:0/4060542534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7c0095f0 con 0x7f8d90101ee0 2023-09-13T04:40:08.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.684+0000 7f8d9559c700 1 --2- 172.21.15.2:0/4060542534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d90102880 0x7f8d901a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:40:08.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.685+0000 7f8d8ffff700 1 --2- 172.21.15.2:0/4060542534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d90101ee0 0x7f8d901991b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0098e0 tx=0x7f8d7c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:08.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.685+0000 7f8d8dffb700 1 -- 172.21.15.2:0/4060542534 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c01d070 con 0x7f8d90101ee0 2023-09-13T04:40:08.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.685+0000 7f8d8dffb700 1 -- 172.21.15.2:0/4060542534 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d7c005510 con 0x7f8d90101ee0 2023-09-13T04:40:08.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.685+0000 7f8d8dffb700 1 -- 172.21.15.2:0/4060542534 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c004b70 con 0x7f8d90101ee0 2023-09-13T04:40:08.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.685+0000 7f8d96fff700 1 -- 172.21.15.2:0/4060542534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d901a0d90 con 0x7f8d90101ee0 2023-09-13T04:40:08.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.685+0000 7f8d96fff700 1 -- 172.21.15.2:0/4060542534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d901a1160 con 0x7f8d90101ee0 2023-09-13T04:40:08.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.686+0000 7f8d96fff700 1 -- 172.21.15.2:0/4060542534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d90193340 con 0x7f8d90101ee0 2023-09-13T04:40:08.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.688+0000 7f8d8dffb700 1 -- 172.21.15.2:0/4060542534 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f8d7c004d10 con 0x7f8d90101ee0 2023-09-13T04:40:08.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.690+0000 7f8d8dffb700 1 --2- 172.21.15.2:0/4060542534 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8d800a06c0 0x7f8d800a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:08.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.691+0000 7f8d8dffb700 1 -- 172.21.15.2:0/4060542534 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8d7c0cccd0 con 0x7f8d90101ee0 2023-09-13T04:40:08.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.691+0000 7f8d94d9b700 1 --2- 172.21.15.2:0/4060542534 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8d800a06c0 0x7f8d800a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:08.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.691+0000 7f8d94d9b700 1 --2- 172.21.15.2:0/4060542534 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8d800a06c0 0x7f8d800a2b80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8d90199b40 tx=0x7f8d84009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:08.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.695+0000 7f8d8dffb700 1 -- 172.21.15.2:0/4060542534 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8d7c096500 con 0x7f8d90101ee0 2023-09-13T04:40:08.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:08 smithi002 ceph-mon[97651]: pgmap v344: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:40:08.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:08 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/334413375' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:40:08.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:08 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2642703655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:40:08.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.905+0000 7f8d96fff700 1 -- 172.21.15.2:0/4060542534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8d9002d070 con 0x7f8d90101ee0 2023-09-13T04:40:08.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.905+0000 7f8d8dffb700 1 -- 172.21.15.2:0/4060542534 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8d7c099b20 con 0x7f8d90101ee0 2023-09-13T04:40:08.907 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-09-13T04:40:08.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.908+0000 7f8d96fff700 1 -- 172.21.15.2:0/4060542534 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8d800a06c0 msgr2=0x7f8d800a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:08.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.908+0000 7f8d96fff700 1 --2- 172.21.15.2:0/4060542534 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8d800a06c0 0x7f8d800a2b80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8d90199b40 tx=0x7f8d84009260 comp rx=0 tx=0).stop 2023-09-13T04:40:08.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.908+0000 7f8d96fff700 1 -- 172.21.15.2:0/4060542534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d90101ee0 msgr2=0x7f8d901991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:08.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.908+0000 7f8d96fff700 1 --2- 172.21.15.2:0/4060542534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d90101ee0 0x7f8d901991b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0098e0 tx=0x7f8d7c00f690 comp rx=0 tx=0).stop 2023-09-13T04:40:08.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.908+0000 7f8d96fff700 1 -- 172.21.15.2:0/4060542534 shutdown_connections 2023-09-13T04:40:08.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.908+0000 7f8d96fff700 1 --2- 172.21.15.2:0/4060542534 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f8d800a06c0 0x7f8d800a2b80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.908+0000 7f8d96fff700 1 --2- 172.21.15.2:0/4060542534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d90102880 0x7f8d901a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.908+0000 7f8d96fff700 1 --2- 172.21.15.2:0/4060542534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d90100cf0 0x7f8d90198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.908+0000 7f8d96fff700 1 --2- 172.21.15.2:0/4060542534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d90101ee0 0x7f8d901991b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:08.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.908+0000 7f8d96fff700 1 -- 172.21.15.2:0/4060542534 >> 172.21.15.2:0/4060542534 conn(0x7f8d900fc260 msgr2=0x7f8d90107810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:08.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.908+0000 7f8d96fff700 1 -- 172.21.15.2:0/4060542534 shutdown_connections 2023-09-13T04:40:08.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:08.908+0000 7f8d96fff700 1 -- 172.21.15.2:0/4060542534 wait complete. 2023-09-13T04:40:08.928 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-13T04:40:08.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:08 smithi156 ceph-mon[133675]: pgmap v344: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:40:08.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:08 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/334413375' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:40:08.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:08 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2642703655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:40:09.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.133+0000 7ff5239ff700 1 -- 172.21.15.2:0/2590334653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff51c101ed0 msgr2=0x7ff51c102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:09.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.133+0000 7ff5239ff700 1 --2- 172.21.15.2:0/2590334653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff51c101ed0 0x7ff51c102330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009990 tx=0x7ff50c009ca0 comp rx=0 tx=0).stop 2023-09-13T04:40:09.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.133+0000 7ff5239ff700 1 -- 172.21.15.2:0/2590334653 shutdown_connections 2023-09-13T04:40:09.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.133+0000 7ff5239ff700 1 --2- 172.21.15.2:0/2590334653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff51c102870 0x7ff51c109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:09.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.133+0000 7ff5239ff700 1 --2- 172.21.15.2:0/2590334653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff51c100ce0 0x7ff51c101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:09.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.133+0000 7ff5239ff700 1 --2- 172.21.15.2:0/2590334653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff51c101ed0 0x7ff51c102330 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:09.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.133+0000 7ff5239ff700 1 -- 172.21.15.2:0/2590334653 >> 172.21.15.2:0/2590334653 conn(0x7ff51c0fc230 msgr2=0x7ff51c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:09.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.133+0000 7ff5239ff700 1 -- 172.21.15.2:0/2590334653 shutdown_connections 2023-09-13T04:40:09.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.133+0000 7ff5239ff700 1 -- 172.21.15.2:0/2590334653 wait complete. 2023-09-13T04:40:09.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:08 smithi149 ceph-mon[133315]: pgmap v344: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:40:09.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:08 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/334413375' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:40:09.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:08 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2642703655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:40:09.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.134+0000 7ff5239ff700 1 Processor -- start 2023-09-13T04:40:09.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.134+0000 7ff5239ff700 1 -- start start 2023-09-13T04:40:09.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.134+0000 7ff5239ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff51c100ce0 0x7ff51c074a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:09.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.134+0000 7ff5239ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff51c101ed0 0x7ff51c0730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:09.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.134+0000 7ff5239ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff51c102870 0x7ff51c0735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:09.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.134+0000 7ff5239ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c073d80 con 0x7ff51c101ed0 2023-09-13T04:40:09.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.134+0000 7ff5239ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c073ef0 con 0x7ff51c102870 2023-09-13T04:40:09.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.134+0000 7ff5239ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c1a0a10 con 0x7ff51c100ce0 2023-09-13T04:40:09.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.135+0000 7ff52179b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff51c100ce0 0x7ff51c074a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:09.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.135+0000 7ff52179b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff51c100ce0 0x7ff51c074a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:54378/0 (socket says 172.21.15.2:54378) 2023-09-13T04:40:09.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.135+0000 7ff52179b700 1 -- 172.21.15.2:0/2256153418 learned_addr learned my addr 172.21.15.2:0/2256153418 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:09.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.135+0000 7ff520f9a700 1 --2- 172.21.15.2:0/2256153418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff51c101ed0 0x7ff51c0730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:09.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.135+0000 7ff521f9c700 1 --2- 172.21.15.2:0/2256153418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff51c102870 0x7ff51c0735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:09.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.135+0000 7ff52179b700 1 -- 172.21.15.2:0/2256153418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff51c102870 msgr2=0x7ff51c0735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:09.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.135+0000 7ff52179b700 1 --2- 172.21.15.2:0/2256153418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff51c102870 0x7ff51c0735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:09.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.135+0000 7ff52179b700 1 -- 172.21.15.2:0/2256153418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff51c101ed0 msgr2=0x7ff51c0730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:09.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.135+0000 7ff52179b700 1 --2- 172.21.15.2:0/2256153418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff51c101ed0 0x7ff51c0730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:09.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.135+0000 7ff52179b700 1 -- 172.21.15.2:0/2256153418 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50c0095f0 con 0x7ff51c100ce0 2023-09-13T04:40:09.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.135+0000 7ff52179b700 1 --2- 172.21.15.2:0/2256153418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff51c100ce0 0x7ff51c074a60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff51800e970 tx=0x7ff51800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:09.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.136+0000 7ff5127fc700 1 -- 172.21.15.2:0/2256153418 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff518009790 con 0x7ff51c100ce0 2023-09-13T04:40:09.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.136+0000 7ff5127fc700 1 -- 172.21.15.2:0/2256153418 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff51800eec0 con 0x7ff51c100ce0 2023-09-13T04:40:09.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.136+0000 7ff5239ff700 1 -- 172.21.15.2:0/2256153418 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff51c1a0cf0 con 0x7ff51c100ce0 2023-09-13T04:40:09.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.136+0000 7ff5127fc700 1 -- 172.21.15.2:0/2256153418 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff518003cf0 con 0x7ff51c100ce0 2023-09-13T04:40:09.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.136+0000 7ff5239ff700 1 -- 172.21.15.2:0/2256153418 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff51c1a11a0 con 0x7ff51c100ce0 2023-09-13T04:40:09.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.137+0000 7ff5239ff700 1 -- 172.21.15.2:0/2256153418 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff51c10c4f0 con 0x7ff51c100ce0 2023-09-13T04:40:09.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.139+0000 7ff5127fc700 1 -- 172.21.15.2:0/2256153418 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7ff51800ccd0 con 0x7ff51c100ce0 2023-09-13T04:40:09.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.140+0000 7ff5127fc700 1 --2- 172.21.15.2:0/2256153418 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff5080a06c0 0x7ff5080a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:09.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.141+0000 7ff5127fc700 1 -- 172.21.15.2:0/2256153418 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff518014070 con 0x7ff51c100ce0 2023-09-13T04:40:09.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.141+0000 7ff520f9a700 1 --2- 172.21.15.2:0/2256153418 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff5080a06c0 0x7ff5080a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:09.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.141+0000 7ff520f9a700 1 --2- 172.21.15.2:0/2256153418 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff5080a06c0 0x7ff5080a2b80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff50c00b680 tx=0x7ff50c005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:09.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.142+0000 7ff5127fc700 1 -- 172.21.15.2:0/2256153418 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff518095a20 con 0x7ff51c100ce0 2023-09-13T04:40:09.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.354+0000 7ff5239ff700 1 -- 172.21.15.2:0/2256153418 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff51c107c10 con 0x7ff51c100ce0 2023-09-13T04:40:09.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.355+0000 7ff5127fc700 1 -- 172.21.15.2:0/2256153418 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff518099040 con 0x7ff51c100ce0 2023-09-13T04:40:09.356 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-09-13T04:40:09.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.357+0000 7ff5239ff700 1 -- 172.21.15.2:0/2256153418 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff5080a06c0 msgr2=0x7ff5080a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:09.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.357+0000 7ff5239ff700 1 --2- 172.21.15.2:0/2256153418 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff5080a06c0 0x7ff5080a2b80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff50c00b680 tx=0x7ff50c005fd0 comp rx=0 tx=0).stop 2023-09-13T04:40:09.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.357+0000 7ff5239ff700 1 -- 172.21.15.2:0/2256153418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff51c100ce0 msgr2=0x7ff51c074a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:09.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.357+0000 7ff5239ff700 1 --2- 172.21.15.2:0/2256153418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff51c100ce0 0x7ff51c074a60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff51800e970 tx=0x7ff51800ec80 comp rx=0 tx=0).stop 2023-09-13T04:40:09.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.357+0000 7ff5239ff700 1 -- 172.21.15.2:0/2256153418 shutdown_connections 2023-09-13T04:40:09.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.357+0000 7ff5239ff700 1 --2- 172.21.15.2:0/2256153418 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff5080a06c0 0x7ff5080a2b80 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:09.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.357+0000 7ff5239ff700 1 --2- 172.21.15.2:0/2256153418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff51c102870 0x7ff51c0735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:09.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.357+0000 7ff5239ff700 1 --2- 172.21.15.2:0/2256153418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff51c100ce0 0x7ff51c074a60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:09.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.357+0000 7ff5239ff700 1 --2- 172.21.15.2:0/2256153418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff51c101ed0 0x7ff51c0730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:09.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.357+0000 7ff5239ff700 1 -- 172.21.15.2:0/2256153418 >> 172.21.15.2:0/2256153418 conn(0x7ff51c0fc230 msgr2=0x7ff51c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:09.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.358+0000 7ff5239ff700 1 -- 172.21.15.2:0/2256153418 shutdown_connections 2023-09-13T04:40:09.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.358+0000 7ff5239ff700 1 -- 172.21.15.2:0/2256153418 wait complete. 2023-09-13T04:40:09.377 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-13T04:40:09.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.584+0000 7f3857fff700 1 -- 172.21.15.2:0/211797842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3850074cf0 msgr2=0x7f3850073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:09.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.584+0000 7f3857fff700 1 --2- 172.21.15.2:0/211797842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3850074cf0 0x7f3850073140 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f384c009990 tx=0x7f384c009ca0 comp rx=0 tx=0).stop 2023-09-13T04:40:09.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.584+0000 7f3857fff700 1 -- 172.21.15.2:0/211797842 shutdown_connections 2023-09-13T04:40:09.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.584+0000 7f3857fff700 1 --2- 172.21.15.2:0/211797842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3850108ba0 0x7f385010b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:09.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.584+0000 7f3857fff700 1 --2- 172.21.15.2:0/211797842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3850074cf0 0x7f3850073140 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:09.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.584+0000 7f3857fff700 1 --2- 172.21.15.2:0/211797842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3850073710 0x7f3850073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:09.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.584+0000 7f3857fff700 1 -- 172.21.15.2:0/211797842 >> 172.21.15.2:0/211797842 conn(0x7f38500fc230 msgr2=0x7f38500fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:09.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.584+0000 7f3857fff700 1 -- 172.21.15.2:0/211797842 shutdown_connections 2023-09-13T04:40:09.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.584+0000 7f3857fff700 1 -- 172.21.15.2:0/211797842 wait complete. 2023-09-13T04:40:09.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.585+0000 7f3857fff700 1 Processor -- start 2023-09-13T04:40:09.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.585+0000 7f3857fff700 1 -- start start 2023-09-13T04:40:09.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.585+0000 7f3857fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3850073710 0x7f3850198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:09.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.585+0000 7f3857fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3850074cf0 0x7f3850199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:09.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.585+0000 7f3857fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3850108ba0 0x7f38501a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:09.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.585+0000 7f3857fff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38501a0780 con 0x7f3850074cf0 2023-09-13T04:40:09.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.585+0000 7f3857fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38501a08f0 con 0x7f3850108ba0 2023-09-13T04:40:09.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.585+0000 7f3857fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38501a0ab0 con 0x7f3850073710 2023-09-13T04:40:09.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.586+0000 7f385559a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3850074cf0 0x7f3850199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:09.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.586+0000 7f385559a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3850074cf0 0x7f3850199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47640/0 (socket says 172.21.15.2:47640) 2023-09-13T04:40:09.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.586+0000 7f385559a700 1 -- 172.21.15.2:0/3912185965 learned_addr learned my addr 172.21.15.2:0/3912185965 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:09.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.586+0000 7f3855d9b700 1 --2- 172.21.15.2:0/3912185965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3850073710 0x7f3850198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:09.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.586+0000 7f385659c700 1 --2- 172.21.15.2:0/3912185965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3850108ba0 0x7f38501a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:09.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.586+0000 7f385559a700 1 -- 172.21.15.2:0/3912185965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3850073710 msgr2=0x7f3850198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:09.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.586+0000 7f385559a700 1 --2- 172.21.15.2:0/3912185965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3850073710 0x7f3850198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:09.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.586+0000 7f385559a700 1 -- 172.21.15.2:0/3912185965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3850108ba0 msgr2=0x7f38501a01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:09.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.586+0000 7f385559a700 1 --2- 172.21.15.2:0/3912185965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3850108ba0 0x7f38501a01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:09.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.586+0000 7f385559a700 1 -- 172.21.15.2:0/3912185965 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384c0095f0 con 0x7f3850074cf0 2023-09-13T04:40:09.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.586+0000 7f385559a700 1 --2- 172.21.15.2:0/3912185965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3850074cf0 0x7f3850199180 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f384000d980 tx=0x7f384000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:09.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.587+0000 7f3846ffd700 1 -- 172.21.15.2:0/3912185965 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3840004020 con 0x7f3850074cf0 2023-09-13T04:40:09.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.587+0000 7f3846ffd700 1 -- 172.21.15.2:0/3912185965 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f384000f690 con 0x7f3850074cf0 2023-09-13T04:40:09.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.587+0000 7f3846ffd700 1 -- 172.21.15.2:0/3912185965 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3840003a60 con 0x7f3850074cf0 2023-09-13T04:40:09.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.587+0000 7f3857fff700 1 -- 172.21.15.2:0/3912185965 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38501a0d90 con 0x7f3850074cf0 2023-09-13T04:40:09.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.587+0000 7f3857fff700 1 -- 172.21.15.2:0/3912185965 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38501a1190 con 0x7f3850074cf0 2023-09-13T04:40:09.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.588+0000 7f3857fff700 1 -- 172.21.15.2:0/3912185965 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3850107250 con 0x7f3850074cf0 2023-09-13T04:40:09.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.590+0000 7f3846ffd700 1 -- 172.21.15.2:0/3912185965 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f384000fb00 con 0x7f3850074cf0 2023-09-13T04:40:09.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.591+0000 7f3846ffd700 1 --2- 172.21.15.2:0/3912185965 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f383c0a06c0 0x7f383c0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:09.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.591+0000 7f3846ffd700 1 -- 172.21.15.2:0/3912185965 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f38400cb9e0 con 0x7f3850074cf0 2023-09-13T04:40:09.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.591+0000 7f3855d9b700 1 --2- 172.21.15.2:0/3912185965 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f383c0a06c0 0x7f383c0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:09.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.592+0000 7f3855d9b700 1 --2- 172.21.15.2:0/3912185965 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f383c0a06c0 0x7f383c0a2b80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f385019a110 tx=0x7f384c005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:09.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.595+0000 7f3846ffd700 1 -- 172.21.15.2:0/3912185965 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f38400951e0 con 0x7f3850074cf0 2023-09-13T04:40:09.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.809+0000 7f3857fff700 1 -- 172.21.15.2:0/3912185965 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f385002d090 con 0x7f3850074cf0 2023-09-13T04:40:09.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.810+0000 7f3846ffd700 1 -- 172.21.15.2:0/3912185965 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3840098800 con 0x7f3850074cf0 2023-09-13T04:40:09.814 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-09-13T04:40:09.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.812+0000 7f3857fff700 1 -- 172.21.15.2:0/3912185965 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f383c0a06c0 msgr2=0x7f383c0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:09.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.812+0000 7f3857fff700 1 --2- 172.21.15.2:0/3912185965 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f383c0a06c0 0x7f383c0a2b80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f385019a110 tx=0x7f384c005fd0 comp rx=0 tx=0).stop 2023-09-13T04:40:09.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.812+0000 7f3857fff700 1 -- 172.21.15.2:0/3912185965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3850074cf0 msgr2=0x7f3850199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:09.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.812+0000 7f3857fff700 1 --2- 172.21.15.2:0/3912185965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3850074cf0 0x7f3850199180 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f384000d980 tx=0x7f384000dc90 comp rx=0 tx=0).stop 2023-09-13T04:40:09.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.813+0000 7f3857fff700 1 -- 172.21.15.2:0/3912185965 shutdown_connections 2023-09-13T04:40:09.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.813+0000 7f3857fff700 1 --2- 172.21.15.2:0/3912185965 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f383c0a06c0 0x7f383c0a2b80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:09.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.813+0000 7f3857fff700 1 --2- 172.21.15.2:0/3912185965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3850108ba0 0x7f38501a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:09.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.813+0000 7f3857fff700 1 --2- 172.21.15.2:0/3912185965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3850073710 0x7f3850198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:09.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.813+0000 7f3857fff700 1 --2- 172.21.15.2:0/3912185965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3850074cf0 0x7f3850199180 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:09.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.813+0000 7f3857fff700 1 -- 172.21.15.2:0/3912185965 >> 172.21.15.2:0/3912185965 conn(0x7f38500fc230 msgr2=0x7f38500fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:09.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.813+0000 7f3857fff700 1 -- 172.21.15.2:0/3912185965 shutdown_connections 2023-09-13T04:40:09.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:09.813+0000 7f3857fff700 1 -- 172.21.15.2:0/3912185965 wait complete. 2023-09-13T04:40:09.833 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T04:40:09.893 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:09 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/4060542534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:40:09.893 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:09 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2256153418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:40:09.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:09 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/4060542534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:40:09.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:09 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2256153418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:40:10.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.037+0000 7f45ed7ff700 1 -- 172.21.15.2:0/1825139812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45e8102eb0 msgr2=0x7f45e81032d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:10.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.037+0000 7f45ed7ff700 1 --2- 172.21.15.2:0/1825139812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45e8102eb0 0x7f45e81032d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f45d8009990 tx=0x7f45d8009ca0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.037+0000 7f45ed7ff700 1 -- 172.21.15.2:0/1825139812 shutdown_connections 2023-09-13T04:40:10.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.037+0000 7f45ed7ff700 1 --2- 172.21.15.2:0/1825139812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45e8104a50 0x7f45e810b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.037+0000 7f45ed7ff700 1 --2- 172.21.15.2:0/1825139812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45e8102eb0 0x7f45e81032d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.037+0000 7f45ed7ff700 1 --2- 172.21.15.2:0/1825139812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e8104090 0x7f45e8104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.037+0000 7f45ed7ff700 1 -- 172.21.15.2:0/1825139812 >> 172.21.15.2:0/1825139812 conn(0x7f45e80fe470 msgr2=0x7f45e81008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:10.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.037+0000 7f45ed7ff700 1 -- 172.21.15.2:0/1825139812 shutdown_connections 2023-09-13T04:40:10.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.037+0000 7f45ed7ff700 1 -- 172.21.15.2:0/1825139812 wait complete. 2023-09-13T04:40:10.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.038+0000 7f45ed7ff700 1 Processor -- start 2023-09-13T04:40:10.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.038+0000 7f45ed7ff700 1 -- start start 2023-09-13T04:40:10.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.038+0000 7f45ed7ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45e8102eb0 0x7f45e8198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:10.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.038+0000 7f45ed7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e8104090 0x7f45e8199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:10.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.038+0000 7f45ed7ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45e8104a50 0x7f45e81a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:10.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.038+0000 7f45ed7ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e81a0750 con 0x7f45e8102eb0 2023-09-13T04:40:10.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.038+0000 7f45ed7ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e81a0890 con 0x7f45e8104a50 2023-09-13T04:40:10.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.038+0000 7f45ed7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e81a0a20 con 0x7f45e8104090 2023-09-13T04:40:10.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.038+0000 7f45e6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45e8102eb0 0x7f45e8198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:10.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.039+0000 7f45e6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45e8102eb0 0x7f45e8198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47650/0 (socket says 172.21.15.2:47650) 2023-09-13T04:40:10.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.039+0000 7f45e6ffd700 1 -- 172.21.15.2:0/3071817324 learned_addr learned my addr 172.21.15.2:0/3071817324 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:10.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.039+0000 7f45e77fe700 1 --2- 172.21.15.2:0/3071817324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45e8104a50 0x7f45e81a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:10.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.039+0000 7f45e67fc700 1 --2- 172.21.15.2:0/3071817324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e8104090 0x7f45e8199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:10.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.039+0000 7f45e6ffd700 1 -- 172.21.15.2:0/3071817324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e8104090 msgr2=0x7f45e8199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:10.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.039+0000 7f45e6ffd700 1 --2- 172.21.15.2:0/3071817324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e8104090 0x7f45e8199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.039+0000 7f45e6ffd700 1 -- 172.21.15.2:0/3071817324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45e8104a50 msgr2=0x7f45e81a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:10.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.039+0000 7f45e6ffd700 1 --2- 172.21.15.2:0/3071817324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45e8104a50 0x7f45e81a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.039+0000 7f45e6ffd700 1 -- 172.21.15.2:0/3071817324 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d80095f0 con 0x7f45e8102eb0 2023-09-13T04:40:10.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.039+0000 7f45e6ffd700 1 --2- 172.21.15.2:0/3071817324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45e8102eb0 0x7f45e8198bd0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f45d8004710 tx=0x7f45d8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:10.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.039+0000 7f45cffff700 1 -- 172.21.15.2:0/3071817324 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d801d070 con 0x7f45e8102eb0 2023-09-13T04:40:10.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.039+0000 7f45cffff700 1 -- 172.21.15.2:0/3071817324 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45d8021410 con 0x7f45e8102eb0 2023-09-13T04:40:10.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.039+0000 7f45cffff700 1 -- 172.21.15.2:0/3071817324 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d8017650 con 0x7f45e8102eb0 2023-09-13T04:40:10.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.040+0000 7f45ed7ff700 1 -- 172.21.15.2:0/3071817324 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e81a0ca0 con 0x7f45e8102eb0 2023-09-13T04:40:10.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.040+0000 7f45ed7ff700 1 -- 172.21.15.2:0/3071817324 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e81a1070 con 0x7f45e8102eb0 2023-09-13T04:40:10.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.041+0000 7f45ed7ff700 1 -- 172.21.15.2:0/3071817324 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e810e770 con 0x7f45e8102eb0 2023-09-13T04:40:10.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.043+0000 7f45cffff700 1 -- 172.21.15.2:0/3071817324 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f45d8021580 con 0x7f45e8102eb0 2023-09-13T04:40:10.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.044+0000 7f45cffff700 1 --2- 172.21.15.2:0/3071817324 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f45d00a0710 0x7f45d00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:10.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.044+0000 7f45cffff700 1 -- 172.21.15.2:0/3071817324 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f45d80cd910 con 0x7f45e8102eb0 2023-09-13T04:40:10.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.044+0000 7f45e67fc700 1 --2- 172.21.15.2:0/3071817324 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f45d00a0710 0x7f45d00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:10.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.045+0000 7f45e67fc700 1 --2- 172.21.15.2:0/3071817324 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f45d00a0710 0x7f45d00a2bd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f45e8102bd0 tx=0x7f45dc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:10.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.046+0000 7f45cffff700 1 -- 172.21.15.2:0/3071817324 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f45d80971a0 con 0x7f45e8102eb0 2023-09-13T04:40:10.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:09 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/4060542534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:40:10.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:09 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2256153418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:40:10.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.261+0000 7f45ed7ff700 1 -- 172.21.15.2:0/3071817324 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45e8109de0 con 0x7f45e8102eb0 2023-09-13T04:40:10.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.263+0000 7f45cffff700 1 -- 172.21.15.2:0/3071817324 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24503 (secure 0 0 0) 0x7f45d809a7c0 con 0x7f45e8102eb0 2023-09-13T04:40:10.264 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:40:10.265 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":92,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:39:25.580735+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T04:37:57.094802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T04:37:58.564487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-13T04:38:02.096895+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:40:10.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.265+0000 7f45ed7ff700 1 -- 172.21.15.2:0/3071817324 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f45d00a0710 msgr2=0x7f45d00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:10.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.265+0000 7f45ed7ff700 1 --2- 172.21.15.2:0/3071817324 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f45d00a0710 0x7f45d00a2bd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f45e8102bd0 tx=0x7f45dc009260 comp rx=0 tx=0).stop 2023-09-13T04:40:10.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.265+0000 7f45ed7ff700 1 -- 172.21.15.2:0/3071817324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45e8102eb0 msgr2=0x7f45e8198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:10.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.265+0000 7f45ed7ff700 1 --2- 172.21.15.2:0/3071817324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45e8102eb0 0x7f45e8198bd0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f45d8004710 tx=0x7f45d8009580 comp rx=0 tx=0).stop 2023-09-13T04:40:10.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.265+0000 7f45ed7ff700 1 -- 172.21.15.2:0/3071817324 shutdown_connections 2023-09-13T04:40:10.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.265+0000 7f45ed7ff700 1 --2- 172.21.15.2:0/3071817324 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f45d00a0710 0x7f45d00a2bd0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.265+0000 7f45ed7ff700 1 --2- 172.21.15.2:0/3071817324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45e8104a50 0x7f45e81a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.265+0000 7f45ed7ff700 1 --2- 172.21.15.2:0/3071817324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45e8102eb0 0x7f45e8198bd0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.265+0000 7f45ed7ff700 1 --2- 172.21.15.2:0/3071817324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e8104090 0x7f45e8199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.265+0000 7f45ed7ff700 1 -- 172.21.15.2:0/3071817324 >> 172.21.15.2:0/3071817324 conn(0x7f45e80fe470 msgr2=0x7f45e81006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:10.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.266+0000 7f45ed7ff700 1 -- 172.21.15.2:0/3071817324 shutdown_connections 2023-09-13T04:40:10.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.266+0000 7f45ed7ff700 1 -- 172.21.15.2:0/3071817324 wait complete. 2023-09-13T04:40:10.286 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T04:33:42.025714+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-13T04:37:57.094802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-13T04:37:58.564487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-13T04:38:02.096895+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-13T04:40:10.287 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-13T04:40:10.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.493+0000 7f73003ff700 1 -- 172.21.15.2:0/3086112513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72f8102880 msgr2=0x7f72f8109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:10.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.493+0000 7f73003ff700 1 --2- 172.21.15.2:0/3086112513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72f8102880 0x7f72f8109130 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f72f000b0d0 tx=0x7f72f000b3e0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.493+0000 7f73003ff700 1 -- 172.21.15.2:0/3086112513 shutdown_connections 2023-09-13T04:40:10.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.493+0000 7f73003ff700 1 --2- 172.21.15.2:0/3086112513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72f8102880 0x7f72f8109130 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.493+0000 7f73003ff700 1 --2- 172.21.15.2:0/3086112513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72f8100cf0 0x7f72f8101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.493+0000 7f73003ff700 1 --2- 172.21.15.2:0/3086112513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72f8101ee0 0x7f72f8102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.493+0000 7f73003ff700 1 -- 172.21.15.2:0/3086112513 >> 172.21.15.2:0/3086112513 conn(0x7f72f80fc260 msgr2=0x7f72f80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:10.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.493+0000 7f73003ff700 1 -- 172.21.15.2:0/3086112513 shutdown_connections 2023-09-13T04:40:10.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.493+0000 7f73003ff700 1 -- 172.21.15.2:0/3086112513 wait complete. 2023-09-13T04:40:10.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.494+0000 7f73003ff700 1 Processor -- start 2023-09-13T04:40:10.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.494+0000 7f73003ff700 1 -- start start 2023-09-13T04:40:10.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.495+0000 7f73003ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72f8100cf0 0x7f72f8196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:10.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.495+0000 7f73003ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72f8101ee0 0x7f72f8197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:10.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.495+0000 7f73003ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72f8102880 0x7f72f819e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:10.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.495+0000 7f73003ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f819e640 con 0x7f72f8101ee0 2023-09-13T04:40:10.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.495+0000 7f73003ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f819e780 con 0x7f72f8100cf0 2023-09-13T04:40:10.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.495+0000 7f73003ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f819e910 con 0x7f72f8102880 2023-09-13T04:40:10.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.495+0000 7f72fd99a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72f8101ee0 0x7f72f8197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:10.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.495+0000 7f72fe99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72f8102880 0x7f72f819e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:10.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.495+0000 7f72fd99a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72f8101ee0 0x7f72f8197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47672/0 (socket says 172.21.15.2:47672) 2023-09-13T04:40:10.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.495+0000 7f72fd99a700 1 -- 172.21.15.2:0/819069247 learned_addr learned my addr 172.21.15.2:0/819069247 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:10.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.495+0000 7f72fe19b700 1 --2- 172.21.15.2:0/819069247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72f8100cf0 0x7f72f8196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:10.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.495+0000 7f72fd99a700 1 -- 172.21.15.2:0/819069247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72f8102880 msgr2=0x7f72f819e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:10.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.495+0000 7f72fd99a700 1 --2- 172.21.15.2:0/819069247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72f8102880 0x7f72f819e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.495+0000 7f72fd99a700 1 -- 172.21.15.2:0/819069247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72f8100cf0 msgr2=0x7f72f8196ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:10.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.495+0000 7f72fd99a700 1 --2- 172.21.15.2:0/819069247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72f8100cf0 0x7f72f8196ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.495+0000 7f72fd99a700 1 -- 172.21.15.2:0/819069247 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f0009d00 con 0x7f72f8101ee0 2023-09-13T04:40:10.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.496+0000 7f72fd99a700 1 --2- 172.21.15.2:0/819069247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72f8101ee0 0x7f72f8197000 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f72e800d980 tx=0x7f72e800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:10.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.496+0000 7f72ef7fe700 1 -- 172.21.15.2:0/819069247 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e8004220 con 0x7f72f8101ee0 2023-09-13T04:40:10.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.496+0000 7f72ef7fe700 1 -- 172.21.15.2:0/819069247 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72e8007640 con 0x7f72f8101ee0 2023-09-13T04:40:10.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.496+0000 7f72ef7fe700 1 -- 172.21.15.2:0/819069247 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e8005690 con 0x7f72f8101ee0 2023-09-13T04:40:10.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.496+0000 7f73003ff700 1 -- 172.21.15.2:0/819069247 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f819ebf0 con 0x7f72f8101ee0 2023-09-13T04:40:10.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.496+0000 7f73003ff700 1 -- 172.21.15.2:0/819069247 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f819f0b0 con 0x7f72f8101ee0 2023-09-13T04:40:10.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.497+0000 7f73003ff700 1 -- 172.21.15.2:0/819069247 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f8107bd0 con 0x7f72f8101ee0 2023-09-13T04:40:10.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.499+0000 7f72ef7fe700 1 -- 172.21.15.2:0/819069247 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f72e8004750 con 0x7f72f8101ee0 2023-09-13T04:40:10.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.502+0000 7f72ef7fe700 1 --2- 172.21.15.2:0/819069247 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f72e40a06c0 0x7f72e40a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:10.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.502+0000 7f72ef7fe700 1 -- 172.21.15.2:0/819069247 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f72e80cbc60 con 0x7f72f8101ee0 2023-09-13T04:40:10.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.502+0000 7f72fe19b700 1 --2- 172.21.15.2:0/819069247 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f72e40a06c0 0x7f72e40a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:10.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.502+0000 7f72fe19b700 1 --2- 172.21.15.2:0/819069247 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f72e40a06c0 0x7f72e40a2b80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f72f8101d40 tx=0x7f72f4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:10.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.506+0000 7f72ef7fe700 1 -- 172.21.15.2:0/819069247 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f72e8095400 con 0x7f72f8101ee0 2023-09-13T04:40:10.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.716+0000 7f73003ff700 1 -- 172.21.15.2:0/819069247 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f72f80008d0 con 0x7f72f8101ee0 2023-09-13T04:40:10.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.717+0000 7f72ef7fe700 1 -- 172.21.15.2:0/819069247 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f72e8016020 con 0x7f72f8101ee0 2023-09-13T04:40:10.719 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-09-13T04:40:10.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.719+0000 7f73003ff700 1 -- 172.21.15.2:0/819069247 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f72e40a06c0 msgr2=0x7f72e40a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:10.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.719+0000 7f73003ff700 1 --2- 172.21.15.2:0/819069247 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f72e40a06c0 0x7f72e40a2b80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f72f8101d40 tx=0x7f72f4009260 comp rx=0 tx=0).stop 2023-09-13T04:40:10.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.719+0000 7f73003ff700 1 -- 172.21.15.2:0/819069247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72f8101ee0 msgr2=0x7f72f8197000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:10.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.719+0000 7f73003ff700 1 --2- 172.21.15.2:0/819069247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72f8101ee0 0x7f72f8197000 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f72e800d980 tx=0x7f72e800dc90 comp rx=0 tx=0).stop 2023-09-13T04:40:10.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.719+0000 7f73003ff700 1 -- 172.21.15.2:0/819069247 shutdown_connections 2023-09-13T04:40:10.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.719+0000 7f73003ff700 1 --2- 172.21.15.2:0/819069247 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f72e40a06c0 0x7f72e40a2b80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.719+0000 7f73003ff700 1 --2- 172.21.15.2:0/819069247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72f8102880 0x7f72f819e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.719+0000 7f73003ff700 1 --2- 172.21.15.2:0/819069247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72f8100cf0 0x7f72f8196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.719+0000 7f73003ff700 1 --2- 172.21.15.2:0/819069247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72f8101ee0 0x7f72f8197000 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.719+0000 7f73003ff700 1 -- 172.21.15.2:0/819069247 >> 172.21.15.2:0/819069247 conn(0x7f72f80fc260 msgr2=0x7f72f80fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:10.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.719+0000 7f73003ff700 1 -- 172.21.15.2:0/819069247 shutdown_connections 2023-09-13T04:40:10.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.719+0000 7f73003ff700 1 -- 172.21.15.2:0/819069247 wait complete. 2023-09-13T04:40:10.739 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-13T04:40:10.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:10 smithi156 ceph-mon[133675]: pgmap v345: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:40:10.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:10 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3912185965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:40:10.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:10 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3071817324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:40:10.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.947+0000 7f790c5ff700 1 -- 172.21.15.2:0/2894355252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7904102870 msgr2=0x7f7904109120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:10.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.947+0000 7f790c5ff700 1 --2- 172.21.15.2:0/2894355252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7904102870 0x7f7904109120 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00b0d0 tx=0x7f78fc00b3e0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.947+0000 7f790c5ff700 1 -- 172.21.15.2:0/2894355252 shutdown_connections 2023-09-13T04:40:10.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.947+0000 7f790c5ff700 1 --2- 172.21.15.2:0/2894355252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7904102870 0x7f7904109120 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.947+0000 7f790c5ff700 1 --2- 172.21.15.2:0/2894355252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7904100ce0 0x7f7904101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.947+0000 7f790c5ff700 1 --2- 172.21.15.2:0/2894355252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7904101ed0 0x7f7904102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.947+0000 7f790c5ff700 1 -- 172.21.15.2:0/2894355252 >> 172.21.15.2:0/2894355252 conn(0x7f79040fc230 msgr2=0x7f79040fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:10.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.947+0000 7f790c5ff700 1 -- 172.21.15.2:0/2894355252 shutdown_connections 2023-09-13T04:40:10.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.947+0000 7f790c5ff700 1 -- 172.21.15.2:0/2894355252 wait complete. 2023-09-13T04:40:10.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.948+0000 7f790c5ff700 1 Processor -- start 2023-09-13T04:40:10.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.948+0000 7f790c5ff700 1 -- start start 2023-09-13T04:40:10.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.949+0000 7f790c5ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7904100ce0 0x7f7904196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:10.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.949+0000 7f790c5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7904101ed0 0x7f7904197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:10.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.949+0000 7f790c5ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7904102870 0x7f790419e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:10.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.949+0000 7f790c5ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790419e680 con 0x7f7904102870 2023-09-13T04:40:10.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.949+0000 7f790c5ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790419e7c0 con 0x7f7904100ce0 2023-09-13T04:40:10.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.949+0000 7f790c5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790419e950 con 0x7f7904101ed0 2023-09-13T04:40:10.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.949+0000 7f790a39b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7904100ce0 0x7f7904196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:10.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.949+0000 7f790ab9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7904102870 0x7f790419e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:10.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.949+0000 7f790a39b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7904100ce0 0x7f7904196b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:46038/0 (socket says 172.21.15.2:46038) 2023-09-13T04:40:10.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.949+0000 7f790a39b700 1 -- 172.21.15.2:0/2482216389 learned_addr learned my addr 172.21.15.2:0/2482216389 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:10.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.949+0000 7f7909b9a700 1 --2- 172.21.15.2:0/2482216389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7904101ed0 0x7f7904197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:10.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.950+0000 7f790a39b700 1 -- 172.21.15.2:0/2482216389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7904101ed0 msgr2=0x7f7904197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:10.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.950+0000 7f790a39b700 1 --2- 172.21.15.2:0/2482216389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7904101ed0 0x7f7904197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.950+0000 7f790a39b700 1 -- 172.21.15.2:0/2482216389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7904102870 msgr2=0x7f790419e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:10.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.950+0000 7f790a39b700 1 --2- 172.21.15.2:0/2482216389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7904102870 0x7f790419e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:10.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.950+0000 7f790a39b700 1 -- 172.21.15.2:0/2482216389 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc009d00 con 0x7f7904100ce0 2023-09-13T04:40:10.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.950+0000 7f790ab9c700 1 --2- 172.21.15.2:0/2482216389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7904102870 0x7f790419e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:10.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.950+0000 7f790a39b700 1 --2- 172.21.15.2:0/2482216389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7904100ce0 0x7f7904196b00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f790000e970 tx=0x7f790000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:10.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.950+0000 7f78fb7fe700 1 -- 172.21.15.2:0/2482216389 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7900009790 con 0x7f7904100ce0 2023-09-13T04:40:10.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.950+0000 7f790c5ff700 1 -- 172.21.15.2:0/2482216389 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f790419ec30 con 0x7f7904100ce0 2023-09-13T04:40:10.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.950+0000 7f78fb7fe700 1 -- 172.21.15.2:0/2482216389 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f790000c7e0 con 0x7f7904100ce0 2023-09-13T04:40:10.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.950+0000 7f78fb7fe700 1 -- 172.21.15.2:0/2482216389 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7900003b60 con 0x7f7904100ce0 2023-09-13T04:40:10.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.950+0000 7f790c5ff700 1 -- 172.21.15.2:0/2482216389 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f790419f190 con 0x7f7904100ce0 2023-09-13T04:40:10.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.952+0000 7f790c5ff700 1 -- 172.21.15.2:0/2482216389 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7904107c10 con 0x7f7904100ce0 2023-09-13T04:40:10.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.954+0000 7f78fb7fe700 1 -- 172.21.15.2:0/2482216389 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f7900003d40 con 0x7f7904100ce0 2023-09-13T04:40:10.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.955+0000 7f78fb7fe700 1 --2- 172.21.15.2:0/2482216389 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f78f00a06c0 0x7f78f00a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:10.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.955+0000 7f78fb7fe700 1 -- 172.21.15.2:0/2482216389 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f7900014070 con 0x7f7904100ce0 2023-09-13T04:40:10.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.955+0000 7f7909b9a700 1 --2- 172.21.15.2:0/2482216389 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f78f00a06c0 0x7f78f00a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:10.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.956+0000 7f7909b9a700 1 --2- 172.21.15.2:0/2482216389 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f78f00a06c0 0x7f78f00a2b80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009c60 tx=0x7f78f4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:10.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:10.957+0000 7f78fb7fe700 1 -- 172.21.15.2:0/2482216389 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7900095a40 con 0x7f7904100ce0 2023-09-13T04:40:11.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:10 smithi002 ceph-mon[97651]: pgmap v345: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:40:11.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:10 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3912185965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:40:11.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:10 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3071817324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:40:11.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:10 smithi149 ceph-mon[133315]: pgmap v345: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:40:11.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:10 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3912185965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:40:11.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:10 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3071817324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:40:11.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.169+0000 7f790c5ff700 1 -- 172.21.15.2:0/2482216389 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f790410c700 con 0x7f7904100ce0 2023-09-13T04:40:11.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.170+0000 7f78fb7fe700 1 -- 172.21.15.2:0/2482216389 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7900099060 con 0x7f7904100ce0 2023-09-13T04:40:11.172 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-09-13T04:40:11.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.173+0000 7f790c5ff700 1 -- 172.21.15.2:0/2482216389 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f78f00a06c0 msgr2=0x7f78f00a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:11.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.173+0000 7f790c5ff700 1 --2- 172.21.15.2:0/2482216389 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f78f00a06c0 0x7f78f00a2b80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009c60 tx=0x7f78f4009260 comp rx=0 tx=0).stop 2023-09-13T04:40:11.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.173+0000 7f790c5ff700 1 -- 172.21.15.2:0/2482216389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7904100ce0 msgr2=0x7f7904196b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:11.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.173+0000 7f790c5ff700 1 --2- 172.21.15.2:0/2482216389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7904100ce0 0x7f7904196b00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f790000e970 tx=0x7f790000ec80 comp rx=0 tx=0).stop 2023-09-13T04:40:11.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.173+0000 7f790c5ff700 1 -- 172.21.15.2:0/2482216389 shutdown_connections 2023-09-13T04:40:11.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.173+0000 7f790c5ff700 1 --2- 172.21.15.2:0/2482216389 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f78f00a06c0 0x7f78f00a2b80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:11.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.173+0000 7f790c5ff700 1 --2- 172.21.15.2:0/2482216389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7904102870 0x7f790419e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:11.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.173+0000 7f790c5ff700 1 --2- 172.21.15.2:0/2482216389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7904100ce0 0x7f7904196b00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:11.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.173+0000 7f790c5ff700 1 --2- 172.21.15.2:0/2482216389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7904101ed0 0x7f7904197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:11.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.173+0000 7f790c5ff700 1 -- 172.21.15.2:0/2482216389 >> 172.21.15.2:0/2482216389 conn(0x7f79040fc230 msgr2=0x7f79040fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:11.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.173+0000 7f790c5ff700 1 -- 172.21.15.2:0/2482216389 shutdown_connections 2023-09-13T04:40:11.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.173+0000 7f790c5ff700 1 -- 172.21.15.2:0/2482216389 wait complete. 2023-09-13T04:40:11.193 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-13T04:40:11.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.401+0000 7fef98dff700 1 -- 172.21.15.2:0/1917491240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef94100ce0 msgr2=0x7fef94101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:11.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.401+0000 7fef98dff700 1 --2- 172.21.15.2:0/1917491240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef94100ce0 0x7fef94101100 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fef840099c0 tx=0x7fef84009cd0 comp rx=0 tx=0).stop 2023-09-13T04:40:11.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.401+0000 7fef98dff700 1 -- 172.21.15.2:0/1917491240 shutdown_connections 2023-09-13T04:40:11.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.401+0000 7fef98dff700 1 --2- 172.21.15.2:0/1917491240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef94102870 0x7fef94109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:11.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.401+0000 7fef98dff700 1 --2- 172.21.15.2:0/1917491240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef94100ce0 0x7fef94101100 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:11.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.401+0000 7fef98dff700 1 --2- 172.21.15.2:0/1917491240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef94101ed0 0x7fef94102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:11.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.401+0000 7fef98dff700 1 -- 172.21.15.2:0/1917491240 >> 172.21.15.2:0/1917491240 conn(0x7fef940fc230 msgr2=0x7fef940fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:11.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.401+0000 7fef98dff700 1 -- 172.21.15.2:0/1917491240 shutdown_connections 2023-09-13T04:40:11.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.401+0000 7fef98dff700 1 -- 172.21.15.2:0/1917491240 wait complete. 2023-09-13T04:40:11.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.402+0000 7fef98dff700 1 Processor -- start 2023-09-13T04:40:11.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.402+0000 7fef98dff700 1 -- start start 2023-09-13T04:40:11.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.403+0000 7fef98dff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef94100ce0 0x7fef94196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:11.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.403+0000 7fef98dff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef94101ed0 0x7fef94196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:11.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.403+0000 7fef98dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef94102870 0x7fef9419dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:11.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.403+0000 7fef98dff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9419e5b0 con 0x7fef94101ed0 2023-09-13T04:40:11.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.403+0000 7fef98dff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9419e6f0 con 0x7fef94100ce0 2023-09-13T04:40:11.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.403+0000 7fef98dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9419e880 con 0x7fef94102870 2023-09-13T04:40:11.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.403+0000 7fef9259c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef94100ce0 0x7fef94196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:11.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.403+0000 7fef91d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef94101ed0 0x7fef94196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:11.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.403+0000 7fef9259c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef94100ce0 0x7fef94196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:46066/0 (socket says 172.21.15.2:46066) 2023-09-13T04:40:11.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.403+0000 7fef9259c700 1 -- 172.21.15.2:0/3503190667 learned_addr learned my addr 172.21.15.2:0/3503190667 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:11.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.403+0000 7fef92d9d700 1 --2- 172.21.15.2:0/3503190667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef94102870 0x7fef9419dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:11.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.403+0000 7fef9259c700 1 -- 172.21.15.2:0/3503190667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef94102870 msgr2=0x7fef9419dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:11.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.403+0000 7fef9259c700 1 --2- 172.21.15.2:0/3503190667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef94102870 0x7fef9419dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:11.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.403+0000 7fef9259c700 1 -- 172.21.15.2:0/3503190667 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef94101ed0 msgr2=0x7fef94196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:11.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.403+0000 7fef9259c700 1 --2- 172.21.15.2:0/3503190667 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef94101ed0 0x7fef94196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:11.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.403+0000 7fef9259c700 1 -- 172.21.15.2:0/3503190667 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef840095f0 con 0x7fef94100ce0 2023-09-13T04:40:11.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.403+0000 7fef91d9b700 1 --2- 172.21.15.2:0/3503190667 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef94101ed0 0x7fef94196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:11.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.404+0000 7fef9259c700 1 --2- 172.21.15.2:0/3503190667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef94100ce0 0x7fef94196a30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fef840098e0 tx=0x7fef8400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:11.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.404+0000 7fef837fe700 1 -- 172.21.15.2:0/3503190667 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8401d070 con 0x7fef94100ce0 2023-09-13T04:40:11.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.404+0000 7fef98dff700 1 -- 172.21.15.2:0/3503190667 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9419eb00 con 0x7fef94100ce0 2023-09-13T04:40:11.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.404+0000 7fef837fe700 1 -- 172.21.15.2:0/3503190667 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef84005050 con 0x7fef94100ce0 2023-09-13T04:40:11.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.404+0000 7fef837fe700 1 -- 172.21.15.2:0/3503190667 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84004b40 con 0x7fef94100ce0 2023-09-13T04:40:11.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.404+0000 7fef98dff700 1 -- 172.21.15.2:0/3503190667 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9419ef30 con 0x7fef94100ce0 2023-09-13T04:40:11.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.405+0000 7fef98dff700 1 -- 172.21.15.2:0/3503190667 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef94107c60 con 0x7fef94100ce0 2023-09-13T04:40:11.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.407+0000 7fef837fe700 1 -- 172.21.15.2:0/3503190667 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7fef84004cc0 con 0x7fef94100ce0 2023-09-13T04:40:11.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.409+0000 7fef837fe700 1 --2- 172.21.15.2:0/3503190667 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fef7c0a0640 0x7fef7c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:11.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.409+0000 7fef837fe700 1 -- 172.21.15.2:0/3503190667 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fef840cd0a0 con 0x7fef94100ce0 2023-09-13T04:40:11.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.409+0000 7fef91d9b700 1 --2- 172.21.15.2:0/3503190667 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fef7c0a0640 0x7fef7c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:11.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.410+0000 7fef91d9b700 1 --2- 172.21.15.2:0/3503190667 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fef7c0a0640 0x7fef7c0a2b00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fef88010420 tx=0x7fef880073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:11.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.411+0000 7fef837fe700 1 -- 172.21.15.2:0/3503190667 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fef840968d0 con 0x7fef94100ce0 2023-09-13T04:40:11.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.623+0000 7fef98dff700 1 -- 172.21.15.2:0/3503190667 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fef9410c6d0 con 0x7fef94100ce0 2023-09-13T04:40:11.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.624+0000 7fef837fe700 1 -- 172.21.15.2:0/3503190667 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fef84099ef0 con 0x7fef94100ce0 2023-09-13T04:40:11.625 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-09-13T04:40:11.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.626+0000 7fef98dff700 1 -- 172.21.15.2:0/3503190667 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fef7c0a0640 msgr2=0x7fef7c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:11.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.626+0000 7fef98dff700 1 --2- 172.21.15.2:0/3503190667 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fef7c0a0640 0x7fef7c0a2b00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fef88010420 tx=0x7fef880073d0 comp rx=0 tx=0).stop 2023-09-13T04:40:11.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.626+0000 7fef98dff700 1 -- 172.21.15.2:0/3503190667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef94100ce0 msgr2=0x7fef94196a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:11.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.626+0000 7fef98dff700 1 --2- 172.21.15.2:0/3503190667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef94100ce0 0x7fef94196a30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fef840098e0 tx=0x7fef8400f690 comp rx=0 tx=0).stop 2023-09-13T04:40:11.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.626+0000 7fef98dff700 1 -- 172.21.15.2:0/3503190667 shutdown_connections 2023-09-13T04:40:11.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.626+0000 7fef98dff700 1 --2- 172.21.15.2:0/3503190667 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7fef7c0a0640 0x7fef7c0a2b00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:11.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.626+0000 7fef98dff700 1 --2- 172.21.15.2:0/3503190667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef94102870 0x7fef9419dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:11.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.626+0000 7fef98dff700 1 --2- 172.21.15.2:0/3503190667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef94100ce0 0x7fef94196a30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:11.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.626+0000 7fef98dff700 1 --2- 172.21.15.2:0/3503190667 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef94101ed0 0x7fef94196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:11.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.626+0000 7fef98dff700 1 -- 172.21.15.2:0/3503190667 >> 172.21.15.2:0/3503190667 conn(0x7fef940fc230 msgr2=0x7fef940fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:11.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.626+0000 7fef98dff700 1 -- 172.21.15.2:0/3503190667 shutdown_connections 2023-09-13T04:40:11.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.626+0000 7fef98dff700 1 -- 172.21.15.2:0/3503190667 wait complete. 2023-09-13T04:40:11.646 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-13T04:40:11.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.855+0000 7f85415ff700 1 -- 172.21.15.2:0/544771098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f853c100ce0 msgr2=0x7f853c101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:11.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.855+0000 7f85415ff700 1 --2- 172.21.15.2:0/544771098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f853c100ce0 0x7f853c101100 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f852c0099c0 tx=0x7f852c009cd0 comp rx=0 tx=0).stop 2023-09-13T04:40:11.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.855+0000 7f85415ff700 1 -- 172.21.15.2:0/544771098 shutdown_connections 2023-09-13T04:40:11.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.855+0000 7f85415ff700 1 --2- 172.21.15.2:0/544771098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f853c102870 0x7f853c109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:11.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.855+0000 7f85415ff700 1 --2- 172.21.15.2:0/544771098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f853c100ce0 0x7f853c101100 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:11.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.855+0000 7f85415ff700 1 --2- 172.21.15.2:0/544771098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f853c101ed0 0x7f853c102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:11.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.855+0000 7f85415ff700 1 -- 172.21.15.2:0/544771098 >> 172.21.15.2:0/544771098 conn(0x7f853c0fc230 msgr2=0x7f853c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:11.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.855+0000 7f85415ff700 1 -- 172.21.15.2:0/544771098 shutdown_connections 2023-09-13T04:40:11.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.855+0000 7f85415ff700 1 -- 172.21.15.2:0/544771098 wait complete. 2023-09-13T04:40:11.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.856+0000 7f85415ff700 1 Processor -- start 2023-09-13T04:40:11.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.856+0000 7f85415ff700 1 -- start start 2023-09-13T04:40:11.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.856+0000 7f85415ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f853c100ce0 0x7f853c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:11.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.856+0000 7f85415ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f853c101ed0 0x7f853c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:11.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.856+0000 7f85415ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f853c102870 0x7f853c1a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:11.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.856+0000 7f85415ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853c1a07b0 con 0x7f853c101ed0 2023-09-13T04:40:11.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.856+0000 7f85415ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853c1a0920 con 0x7f853c100ce0 2023-09-13T04:40:11.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.856+0000 7f85415ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853c1a0ae0 con 0x7f853c102870 2023-09-13T04:40:11.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.856+0000 7f853a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f853c101ed0 0x7f853c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:11.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.857+0000 7f853a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f853c101ed0 0x7f853c1991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47720/0 (socket says 172.21.15.2:47720) 2023-09-13T04:40:11.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.857+0000 7f853a7fc700 1 -- 172.21.15.2:0/3779605711 learned_addr learned my addr 172.21.15.2:0/3779605711 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:11.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.857+0000 7f853affd700 1 --2- 172.21.15.2:0/3779605711 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f853c100ce0 0x7f853c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:11.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.857+0000 7f853b7fe700 1 --2- 172.21.15.2:0/3779605711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f853c102870 0x7f853c1a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:11.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.857+0000 7f853a7fc700 1 -- 172.21.15.2:0/3779605711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f853c102870 msgr2=0x7f853c1a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:11.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.857+0000 7f853a7fc700 1 --2- 172.21.15.2:0/3779605711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f853c102870 0x7f853c1a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:11.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.857+0000 7f853a7fc700 1 -- 172.21.15.2:0/3779605711 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f853c100ce0 msgr2=0x7f853c198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:11.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.857+0000 7f853a7fc700 1 --2- 172.21.15.2:0/3779605711 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f853c100ce0 0x7f853c198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:11.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.857+0000 7f853a7fc700 1 -- 172.21.15.2:0/3779605711 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c0095f0 con 0x7f853c101ed0 2023-09-13T04:40:11.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.857+0000 7f853a7fc700 1 --2- 172.21.15.2:0/3779605711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f853c101ed0 0x7f853c1991b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f853000e9a0 tx=0x7f853000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:11.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.857+0000 7f8523fff700 1 -- 172.21.15.2:0/3779605711 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8530009790 con 0x7f853c101ed0 2023-09-13T04:40:11.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.857+0000 7f8523fff700 1 -- 172.21.15.2:0/3779605711 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8530010430 con 0x7f853c101ed0 2023-09-13T04:40:11.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.858+0000 7f85415ff700 1 -- 172.21.15.2:0/3779605711 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f853c1a0dc0 con 0x7f853c101ed0 2023-09-13T04:40:11.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.858+0000 7f8523fff700 1 -- 172.21.15.2:0/3779605711 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8530018680 con 0x7f853c101ed0 2023-09-13T04:40:11.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.858+0000 7f85415ff700 1 -- 172.21.15.2:0/3779605711 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853c1a1240 con 0x7f853c101ed0 2023-09-13T04:40:11.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.859+0000 7f85415ff700 1 -- 172.21.15.2:0/3779605711 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f853c10c540 con 0x7f853c101ed0 2023-09-13T04:40:11.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.861+0000 7f8523fff700 1 -- 172.21.15.2:0/3779605711 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f85300105a0 con 0x7f853c101ed0 2023-09-13T04:40:11.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.863+0000 7f8523fff700 1 --2- 172.21.15.2:0/3779605711 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f85240a06c0 0x7f85240a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:11.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.863+0000 7f8523fff700 1 -- 172.21.15.2:0/3779605711 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8530014070 con 0x7f853c101ed0 2023-09-13T04:40:11.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.863+0000 7f853affd700 1 --2- 172.21.15.2:0/3779605711 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f85240a06c0 0x7f85240a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:11.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.864+0000 7f853affd700 1 --2- 172.21.15.2:0/3779605711 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f85240a06c0 0x7f85240a2b80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f852c009990 tx=0x7f852c01a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:11.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:11.867+0000 7f8523fff700 1 -- 172.21.15.2:0/3779605711 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8530051e10 con 0x7f853c101ed0 2023-09-13T04:40:11.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:11 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/819069247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:40:11.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:11 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2482216389' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:40:11.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:11 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3503190667' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:40:12.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:11 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/819069247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:40:12.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:11 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2482216389' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:40:12.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:11 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3503190667' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:40:12.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.080+0000 7f85415ff700 1 -- 172.21.15.2:0/3779605711 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f853c1a13d0 con 0x7f853c101ed0 2023-09-13T04:40:12.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.081+0000 7f8523fff700 1 -- 172.21.15.2:0/3779605711 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f85300957d0 con 0x7f853c101ed0 2023-09-13T04:40:12.083 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-09-13T04:40:12.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.083+0000 7f85415ff700 1 -- 172.21.15.2:0/3779605711 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f85240a06c0 msgr2=0x7f85240a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:12.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.083+0000 7f85415ff700 1 --2- 172.21.15.2:0/3779605711 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f85240a06c0 0x7f85240a2b80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f852c009990 tx=0x7f852c01a040 comp rx=0 tx=0).stop 2023-09-13T04:40:12.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.083+0000 7f85415ff700 1 -- 172.21.15.2:0/3779605711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f853c101ed0 msgr2=0x7f853c1991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:12.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.083+0000 7f85415ff700 1 --2- 172.21.15.2:0/3779605711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f853c101ed0 0x7f853c1991b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f853000e9a0 tx=0x7f853000ecb0 comp rx=0 tx=0).stop 2023-09-13T04:40:12.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.084+0000 7f85415ff700 1 -- 172.21.15.2:0/3779605711 shutdown_connections 2023-09-13T04:40:12.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.084+0000 7f85415ff700 1 --2- 172.21.15.2:0/3779605711 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f85240a06c0 0x7f85240a2b80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:12.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.084+0000 7f85415ff700 1 --2- 172.21.15.2:0/3779605711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f853c102870 0x7f853c1a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:12.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.084+0000 7f85415ff700 1 --2- 172.21.15.2:0/3779605711 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f853c100ce0 0x7f853c198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:12.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.084+0000 7f85415ff700 1 --2- 172.21.15.2:0/3779605711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f853c101ed0 0x7f853c1991b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:12.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.084+0000 7f85415ff700 1 -- 172.21.15.2:0/3779605711 >> 172.21.15.2:0/3779605711 conn(0x7f853c0fc230 msgr2=0x7f853c0fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:12.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.084+0000 7f85415ff700 1 -- 172.21.15.2:0/3779605711 shutdown_connections 2023-09-13T04:40:12.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.084+0000 7f85415ff700 1 -- 172.21.15.2:0/3779605711 wait complete. 2023-09-13T04:40:12.103 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-09-13T04:40:12.104 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:40:12.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:11 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/819069247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:40:12.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:11 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2482216389' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:40:12.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:11 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3503190667' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:40:12.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.308+0000 7f66661ff700 1 -- 172.21.15.2:0/3846165883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6660073710 msgr2=0x7f6660073b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:12.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.308+0000 7f66661ff700 1 --2- 172.21.15.2:0/3846165883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6660073710 0x7f6660073b90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6654009990 tx=0x7f6654009ca0 comp rx=0 tx=0).stop 2023-09-13T04:40:12.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.308+0000 7f66661ff700 1 -- 172.21.15.2:0/3846165883 shutdown_connections 2023-09-13T04:40:12.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.308+0000 7f66661ff700 1 --2- 172.21.15.2:0/3846165883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6660108bb0 0x7f666010b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:12.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.308+0000 7f66661ff700 1 --2- 172.21.15.2:0/3846165883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6660074cf0 0x7f6660073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:12.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.308+0000 7f66661ff700 1 --2- 172.21.15.2:0/3846165883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6660073710 0x7f6660073b90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:12.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.308+0000 7f66661ff700 1 -- 172.21.15.2:0/3846165883 >> 172.21.15.2:0/3846165883 conn(0x7f66600fc220 msgr2=0x7f66600fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:12.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.308+0000 7f66661ff700 1 -- 172.21.15.2:0/3846165883 shutdown_connections 2023-09-13T04:40:12.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.308+0000 7f66661ff700 1 -- 172.21.15.2:0/3846165883 wait complete. 2023-09-13T04:40:12.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.309+0000 7f66661ff700 1 Processor -- start 2023-09-13T04:40:12.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.310+0000 7f66661ff700 1 -- start start 2023-09-13T04:40:12.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.310+0000 7f66661ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6660073710 0x7f6660198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:12.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.310+0000 7f66661ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6660074cf0 0x7f6660199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:12.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.310+0000 7f66661ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6660108bb0 0x7f66601a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:12.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.310+0000 7f66661ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66601a0750 con 0x7f6660074cf0 2023-09-13T04:40:12.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.310+0000 7f66661ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66601a0890 con 0x7f6660073710 2023-09-13T04:40:12.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.310+0000 7f66661ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66601a0a20 con 0x7f6660108bb0 2023-09-13T04:40:12.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.310+0000 7f665effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6660074cf0 0x7f6660199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:12.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.310+0000 7f665effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6660074cf0 0x7f6660199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47732/0 (socket says 172.21.15.2:47732) 2023-09-13T04:40:12.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.310+0000 7f665effd700 1 -- 172.21.15.2:0/322282148 learned_addr learned my addr 172.21.15.2:0/322282148 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:12.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.310+0000 7f665f7fe700 1 --2- 172.21.15.2:0/322282148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6660073710 0x7f6660198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:12.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.310+0000 7f665ffff700 1 --2- 172.21.15.2:0/322282148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6660108bb0 0x7f66601a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:12.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.311+0000 7f665effd700 1 -- 172.21.15.2:0/322282148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6660108bb0 msgr2=0x7f66601a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:12.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.311+0000 7f665effd700 1 --2- 172.21.15.2:0/322282148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6660108bb0 0x7f66601a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:12.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.311+0000 7f665effd700 1 -- 172.21.15.2:0/322282148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6660073710 msgr2=0x7f6660198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:12.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.311+0000 7f665effd700 1 --2- 172.21.15.2:0/322282148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6660073710 0x7f6660198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:12.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.311+0000 7f665effd700 1 -- 172.21.15.2:0/322282148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66540095f0 con 0x7f6660074cf0 2023-09-13T04:40:12.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.311+0000 7f665effd700 1 --2- 172.21.15.2:0/322282148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6660074cf0 0x7f6660199110 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6654005720 tx=0x7f6654009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:12.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.311+0000 7f665cff9700 1 -- 172.21.15.2:0/322282148 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665401d070 con 0x7f6660074cf0 2023-09-13T04:40:12.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.311+0000 7f665cff9700 1 -- 172.21.15.2:0/322282148 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6654021410 con 0x7f6660074cf0 2023-09-13T04:40:12.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.311+0000 7f66661ff700 1 -- 172.21.15.2:0/322282148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66601a0ca0 con 0x7f6660074cf0 2023-09-13T04:40:12.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.311+0000 7f665cff9700 1 -- 172.21.15.2:0/322282148 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654017650 con 0x7f6660074cf0 2023-09-13T04:40:12.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.311+0000 7f66661ff700 1 -- 172.21.15.2:0/322282148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66601a10c0 con 0x7f6660074cf0 2023-09-13T04:40:12.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.312+0000 7f66661ff700 1 -- 172.21.15.2:0/322282148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6660107260 con 0x7f6660074cf0 2023-09-13T04:40:12.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.314+0000 7f665cff9700 1 -- 172.21.15.2:0/322282148 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f6654021580 con 0x7f6660074cf0 2023-09-13T04:40:12.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.315+0000 7f665cff9700 1 --2- 172.21.15.2:0/322282148 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f66480a0690 0x7f66480a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:12.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.316+0000 7f665cff9700 1 -- 172.21.15.2:0/322282148 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f66540cd1c0 con 0x7f6660074cf0 2023-09-13T04:40:12.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.316+0000 7f665f7fe700 1 --2- 172.21.15.2:0/322282148 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f66480a0690 0x7f66480a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:12.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.316+0000 7f665f7fe700 1 --2- 172.21.15.2:0/322282148 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f66480a0690 0x7f66480a2b50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6650009c60 tx=0x7f6650009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:12.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.319+0000 7f665cff9700 1 -- 172.21.15.2:0/322282148 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6654096b10 con 0x7f6660074cf0 2023-09-13T04:40:12.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.575+0000 7f66661ff700 1 -- 172.21.15.2:0/322282148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f666010e730 con 0x7f6660074cf0 2023-09-13T04:40:12.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.576+0000 7f665cff9700 1 -- 172.21.15.2:0/322282148 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11303 (secure 0 0 0) 0x7f665409a130 con 0x7f6660074cf0 2023-09-13T04:40:12.578 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:40:12.579 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2469307307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2469307307},{"type":"v1","addr":"172.21.15.2:6839","nonce":2469307307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T04:37:59.110701+0000","modified":"2023-09-13T04:38:26.610260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24554,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/846579077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":846579077},{"type":"v1","addr":"172.21.15.156:6835","nonce":846579077}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1896833690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1896833690},{"type":"v1","addr":"172.21.15.149:6837","nonce":1896833690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/524563090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":524563090},{"type":"v1","addr":"172.21.15.156:6837","nonce":524563090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6839/1785833227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1785833227},{"type":"v1","addr":"172.21.15.156:6839","nonce":1785833227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.2:6841/482545391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":482545391},{"type":"v1","addr":"172.21.15.2:6841","nonce":482545391}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/389167841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":389167841},{"type":"v1","addr":"172.21.15.149:6839","nonce":389167841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T04:40:12.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.578+0000 7f66661ff700 1 -- 172.21.15.2:0/322282148 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f66480a0690 msgr2=0x7f66480a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:12.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.578+0000 7f66661ff700 1 --2- 172.21.15.2:0/322282148 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f66480a0690 0x7f66480a2b50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6650009c60 tx=0x7f6650009260 comp rx=0 tx=0).stop 2023-09-13T04:40:12.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.578+0000 7f66661ff700 1 -- 172.21.15.2:0/322282148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6660074cf0 msgr2=0x7f6660199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:12.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.578+0000 7f66661ff700 1 --2- 172.21.15.2:0/322282148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6660074cf0 0x7f6660199110 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6654005720 tx=0x7f6654009580 comp rx=0 tx=0).stop 2023-09-13T04:40:12.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.579+0000 7f66661ff700 1 -- 172.21.15.2:0/322282148 shutdown_connections 2023-09-13T04:40:12.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.579+0000 7f66661ff700 1 --2- 172.21.15.2:0/322282148 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f66480a0690 0x7f66480a2b50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:12.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.579+0000 7f66661ff700 1 --2- 172.21.15.2:0/322282148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6660108bb0 0x7f66601a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:12.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.579+0000 7f66661ff700 1 --2- 172.21.15.2:0/322282148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6660073710 0x7f6660198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:12.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.579+0000 7f66661ff700 1 --2- 172.21.15.2:0/322282148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6660074cf0 0x7f6660199110 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:12.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.579+0000 7f66661ff700 1 -- 172.21.15.2:0/322282148 >> 172.21.15.2:0/322282148 conn(0x7f66600fc220 msgr2=0x7f66600fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:12.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.579+0000 7f66661ff700 1 -- 172.21.15.2:0/322282148 shutdown_connections 2023-09-13T04:40:12.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.579+0000 7f66661ff700 1 -- 172.21.15.2:0/322282148 wait complete. 2023-09-13T04:40:12.584 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 23 2023-09-13T04:40:12.599 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-09-13T04:40:12.599 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-09-13T04:40:12.600 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-13T04:40:12.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.805+0000 7f90e53ff700 1 -- 172.21.15.2:0/3049425595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90e0102880 msgr2=0x7f90e0109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:12.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.805+0000 7f90e53ff700 1 --2- 172.21.15.2:0/3049425595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90e0102880 0x7f90e0109130 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f90d800b0d0 tx=0x7f90d800b3e0 comp rx=0 tx=0).stop 2023-09-13T04:40:12.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.805+0000 7f90e53ff700 1 -- 172.21.15.2:0/3049425595 shutdown_connections 2023-09-13T04:40:12.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.805+0000 7f90e53ff700 1 --2- 172.21.15.2:0/3049425595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90e0102880 0x7f90e0109130 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:12.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.805+0000 7f90e53ff700 1 --2- 172.21.15.2:0/3049425595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90e0100cf0 0x7f90e0101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:12.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.805+0000 7f90e53ff700 1 --2- 172.21.15.2:0/3049425595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90e0101ee0 0x7f90e0102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:12.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.805+0000 7f90e53ff700 1 -- 172.21.15.2:0/3049425595 >> 172.21.15.2:0/3049425595 conn(0x7f90e00fc220 msgr2=0x7f90e00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:12.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.805+0000 7f90e53ff700 1 -- 172.21.15.2:0/3049425595 shutdown_connections 2023-09-13T04:40:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.805+0000 7f90e53ff700 1 -- 172.21.15.2:0/3049425595 wait complete. 2023-09-13T04:40:12.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.806+0000 7f90e53ff700 1 Processor -- start 2023-09-13T04:40:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.806+0000 7f90e53ff700 1 -- start start 2023-09-13T04:40:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.807+0000 7f90e53ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90e0100cf0 0x7f90e0198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.807+0000 7f90e53ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90e0101ee0 0x7f90e01991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.807+0000 7f90e53ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90e0102880 0x7f90e01a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:12.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.807+0000 7f90e53ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e01a0810 con 0x7f90e0101ee0 2023-09-13T04:40:12.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.807+0000 7f90e53ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e01a0980 con 0x7f90e0102880 2023-09-13T04:40:12.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.807+0000 7f90e53ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e01a0b40 con 0x7f90e0100cf0 2023-09-13T04:40:12.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.807+0000 7f90de7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90e0101ee0 0x7f90e01991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:12.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.807+0000 7f90de7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90e0101ee0 0x7f90e01991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47746/0 (socket says 172.21.15.2:47746) 2023-09-13T04:40:12.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.807+0000 7f90deffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90e0100cf0 0x7f90e0198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:12.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.807+0000 7f90de7fc700 1 -- 172.21.15.2:0/4097607890 learned_addr learned my addr 172.21.15.2:0/4097607890 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:12.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.807+0000 7f90df7fe700 1 --2- 172.21.15.2:0/4097607890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90e0102880 0x7f90e01a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:12.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.807+0000 7f90deffd700 1 -- 172.21.15.2:0/4097607890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90e0102880 msgr2=0x7f90e01a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:12.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.807+0000 7f90deffd700 1 --2- 172.21.15.2:0/4097607890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90e0102880 0x7f90e01a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:12.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.807+0000 7f90deffd700 1 -- 172.21.15.2:0/4097607890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90e0101ee0 msgr2=0x7f90e01991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:12.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.807+0000 7f90deffd700 1 --2- 172.21.15.2:0/4097607890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90e0101ee0 0x7f90e01991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:12.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.807+0000 7f90deffd700 1 -- 172.21.15.2:0/4097607890 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d0009520 con 0x7f90e0100cf0 2023-09-13T04:40:12.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.807+0000 7f90de7fc700 1 --2- 172.21.15.2:0/4097607890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90e0101ee0 0x7f90e01991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:12.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.808+0000 7f90deffd700 1 --2- 172.21.15.2:0/4097607890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90e0100cf0 0x7f90e0198ca0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f90d000e9c0 tx=0x7f90d000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:12.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.808+0000 7f90c7fff700 1 -- 172.21.15.2:0/4097607890 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d000cb70 con 0x7f90e0100cf0 2023-09-13T04:40:12.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.808+0000 7f90e53ff700 1 -- 172.21.15.2:0/4097607890 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d8009d00 con 0x7f90e0100cf0 2023-09-13T04:40:12.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.808+0000 7f90c7fff700 1 -- 172.21.15.2:0/4097607890 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90d000ee60 con 0x7f90e0100cf0 2023-09-13T04:40:12.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.808+0000 7f90c7fff700 1 -- 172.21.15.2:0/4097607890 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d0018680 con 0x7f90e0100cf0 2023-09-13T04:40:12.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.808+0000 7f90e53ff700 1 -- 172.21.15.2:0/4097607890 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e01a11a0 con 0x7f90e0100cf0 2023-09-13T04:40:12.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.809+0000 7f90e53ff700 1 -- 172.21.15.2:0/4097607890 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e010c560 con 0x7f90e0100cf0 2023-09-13T04:40:12.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.811+0000 7f90c7fff700 1 -- 172.21.15.2:0/4097607890 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f90d0010430 con 0x7f90e0100cf0 2023-09-13T04:40:12.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.813+0000 7f90c7fff700 1 --2- 172.21.15.2:0/4097607890 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f90c80a0690 0x7f90c80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:12.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.813+0000 7f90c7fff700 1 -- 172.21.15.2:0/4097607890 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f90d0014070 con 0x7f90e0100cf0 2023-09-13T04:40:12.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.813+0000 7f90de7fc700 1 --2- 172.21.15.2:0/4097607890 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f90c80a0690 0x7f90c80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:12.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.813+0000 7f90de7fc700 1 --2- 172.21.15.2:0/4097607890 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f90c80a0690 0x7f90c80a2b50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f90e019a1a0 tx=0x7f90d4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:12.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:12.815+0000 7f90c7fff700 1 -- 172.21.15.2:0/4097607890 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f90d0095f50 con 0x7f90e0100cf0 2023-09-13T04:40:12.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:12 smithi156 ceph-mon[133675]: pgmap v346: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2023-09-13T04:40:12.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:12 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3779605711' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:40:12.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:12 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/322282148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:40:13.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:12 smithi002 ceph-mon[97651]: pgmap v346: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2023-09-13T04:40:13.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:12 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3779605711' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:40:13.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:12 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/322282148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:40:13.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.026+0000 7f90e53ff700 1 -- 172.21.15.2:0/4097607890 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f90e0108e40 con 0x7f90c80a0690 2023-09-13T04:40:13.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.040+0000 7f90c7fff700 1 -- 172.21.15.2:0/4097607890 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427027 (secure 0 0 0) 0x7f90e0108e40 con 0x7f90c80a0690 2023-09-13T04:40:13.047 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:40:13.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.045+0000 7f90e53ff700 1 -- 172.21.15.2:0/4097607890 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f90c80a0690 msgr2=0x7f90c80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:13.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.045+0000 7f90e53ff700 1 --2- 172.21.15.2:0/4097607890 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f90c80a0690 0x7f90c80a2b50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f90e019a1a0 tx=0x7f90d4009260 comp rx=0 tx=0).stop 2023-09-13T04:40:13.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.045+0000 7f90e53ff700 1 -- 172.21.15.2:0/4097607890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90e0100cf0 msgr2=0x7f90e0198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:13.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.045+0000 7f90e53ff700 1 --2- 172.21.15.2:0/4097607890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90e0100cf0 0x7f90e0198ca0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f90d000e9c0 tx=0x7f90d000ecd0 comp rx=0 tx=0).stop 2023-09-13T04:40:13.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.045+0000 7f90e53ff700 1 -- 172.21.15.2:0/4097607890 shutdown_connections 2023-09-13T04:40:13.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.045+0000 7f90e53ff700 1 --2- 172.21.15.2:0/4097607890 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f90c80a0690 0x7f90c80a2b50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:13.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.045+0000 7f90e53ff700 1 --2- 172.21.15.2:0/4097607890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90e0102880 0x7f90e01a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:13.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.045+0000 7f90e53ff700 1 --2- 172.21.15.2:0/4097607890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90e0100cf0 0x7f90e0198ca0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:13.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.045+0000 7f90e53ff700 1 --2- 172.21.15.2:0/4097607890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90e0101ee0 0x7f90e01991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:13.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.045+0000 7f90e53ff700 1 -- 172.21.15.2:0/4097607890 >> 172.21.15.2:0/4097607890 conn(0x7f90e00fc220 msgr2=0x7f90e0107720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:13.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.045+0000 7f90e53ff700 1 -- 172.21.15.2:0/4097607890 shutdown_connections 2023-09-13T04:40:13.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.045+0000 7f90e53ff700 1 -- 172.21.15.2:0/4097607890 wait complete. 2023-09-13T04:40:13.058 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-09-13T04:40:13.065 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2023-09-13T04:40:11.535570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598078,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":782,"num_read_kb":456,"num_write":312,"num_write_kb":2983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":252,"ondisk_log_size":252,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338328,"kb_used_data":14068,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603496,"statfs":{"total":1151940427776,"available":1151593979904,"internally_reserved":0,"allocated":14405632,"data_stored":7903338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":258,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":91,"num_read_kb":46,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003086"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.624372+0000","last_change":"2023-09-13T04:37:59.244279+0000","last_active":"2023-09-13T04:39:25.624372+0000","last_peered":"2023-09-13T04:39:25.624372+0000","last_clean":"2023-09-13T04:39:25.624372+0000","last_became_active":"2023-09-13T04:37:59.243976+0000","last_became_peered":"2023-09-13T04:37:59.243976+0000","last_unstale":"2023-09-13T04:39:25.624372+0000","last_undegraded":"2023-09-13T04:39:25.624372+0000","last_fullsized":"2023-09-13T04:39:25.624372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:21:52.809411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.856077+0000","last_change":"2023-09-13T04:37:59.243391+0000","last_active":"2023-09-13T04:39:25.856077+0000","last_peered":"2023-09-13T04:39:25.856077+0000","last_clean":"2023-09-13T04:39:25.856077+0000","last_became_active":"2023-09-13T04:37:59.243044+0000","last_became_peered":"2023-09-13T04:37:59.243044+0000","last_unstale":"2023-09-13T04:39:25.856077+0000","last_undegraded":"2023-09-13T04:39:25.856077+0000","last_fullsized":"2023-09-13T04:39:25.856077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:02:17.806638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.855493+0000","last_change":"2023-09-13T04:37:59.242826+0000","last_active":"2023-09-13T04:39:25.855493+0000","last_peered":"2023-09-13T04:39:25.855493+0000","last_clean":"2023-09-13T04:39:25.855493+0000","last_became_active":"2023-09-13T04:37:59.242125+0000","last_became_peered":"2023-09-13T04:37:59.242125+0000","last_unstale":"2023-09-13T04:39:25.855493+0000","last_undegraded":"2023-09-13T04:39:25.855493+0000","last_fullsized":"2023-09-13T04:39:25.855493+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:08:29.216093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.709501+0000","last_change":"2023-09-13T04:37:59.253322+0000","last_active":"2023-09-13T04:39:25.709501+0000","last_peered":"2023-09-13T04:39:25.709501+0000","last_clean":"2023-09-13T04:39:25.709501+0000","last_became_active":"2023-09-13T04:37:59.253050+0000","last_became_peered":"2023-09-13T04:37:59.253050+0000","last_unstale":"2023-09-13T04:39:25.709501+0000","last_undegraded":"2023-09-13T04:39:25.709501+0000","last_fullsized":"2023-09-13T04:39:25.709501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:45:22.357879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755114+0000","last_change":"2023-09-13T04:37:59.250089+0000","last_active":"2023-09-13T04:39:24.755114+0000","last_peered":"2023-09-13T04:39:24.755114+0000","last_clean":"2023-09-13T04:39:24.755114+0000","last_became_active":"2023-09-13T04:37:59.249771+0000","last_became_peered":"2023-09-13T04:37:59.249771+0000","last_unstale":"2023-09-13T04:39:24.755114+0000","last_undegraded":"2023-09-13T04:39:24.755114+0000","last_fullsized":"2023-09-13T04:39:24.755114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:46:59.917756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.703747+0000","last_change":"2023-09-13T04:37:59.247259+0000","last_active":"2023-09-13T04:39:24.703747+0000","last_peered":"2023-09-13T04:39:24.703747+0000","last_clean":"2023-09-13T04:39:24.703747+0000","last_became_active":"2023-09-13T04:37:59.246911+0000","last_became_peered":"2023-09-13T04:37:59.246911+0000","last_unstale":"2023-09-13T04:39:24.703747+0000","last_undegraded":"2023-09-13T04:39:24.703747+0000","last_fullsized":"2023-09-13T04:39:24.703747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:00:39.742392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.585151+0000","last_change":"2023-09-13T04:37:59.244083+0000","last_active":"2023-09-13T04:39:24.585151+0000","last_peered":"2023-09-13T04:39:24.585151+0000","last_clean":"2023-09-13T04:39:24.585151+0000","last_became_active":"2023-09-13T04:37:59.243789+0000","last_became_peered":"2023-09-13T04:37:59.243789+0000","last_unstale":"2023-09-13T04:39:24.585151+0000","last_undegraded":"2023-09-13T04:39:24.585151+0000","last_fullsized":"2023-09-13T04:39:24.585151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:24:00.622790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.581996+0000","last_change":"2023-09-13T04:38:24.601265+0000","last_active":"2023-09-13T04:39:24.581996+0000","last_peered":"2023-09-13T04:39:24.581996+0000","last_clean":"2023-09-13T04:39:24.581996+0000","last_became_active":"2023-09-13T04:38:24.600939+0000","last_became_peered":"2023-09-13T04:38:24.600939+0000","last_unstale":"2023-09-13T04:39:24.581996+0000","last_undegraded":"2023-09-13T04:39:24.581996+0000","last_fullsized":"2023-09-13T04:39:24.581996+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:18:23.642333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754323+0000","last_change":"2023-09-13T04:37:59.238053+0000","last_active":"2023-09-13T04:39:24.754323+0000","last_peered":"2023-09-13T04:39:24.754323+0000","last_clean":"2023-09-13T04:39:24.754323+0000","last_became_active":"2023-09-13T04:37:59.237777+0000","last_became_peered":"2023-09-13T04:37:59.237777+0000","last_unstale":"2023-09-13T04:39:24.754323+0000","last_undegraded":"2023-09-13T04:39:24.754323+0000","last_fullsized":"2023-09-13T04:39:24.754323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:45:39.835659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.704187+0000","last_change":"2023-09-13T04:37:59.252605+0000","last_active":"2023-09-13T04:39:24.704187+0000","last_peered":"2023-09-13T04:39:24.704187+0000","last_clean":"2023-09-13T04:39:24.704187+0000","last_became_active":"2023-09-13T04:37:59.252309+0000","last_became_peered":"2023-09-13T04:37:59.252309+0000","last_unstale":"2023-09-13T04:39:24.704187+0000","last_undegraded":"2023-09-13T04:39:24.704187+0000","last_fullsized":"2023-09-13T04:39:24.704187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:41:45.064230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.585552+0000","last_change":"2023-09-13T04:37:59.243345+0000","last_active":"2023-09-13T04:39:24.585552+0000","last_peered":"2023-09-13T04:39:24.585552+0000","last_clean":"2023-09-13T04:39:24.585552+0000","last_became_active":"2023-09-13T04:37:59.242726+0000","last_became_peered":"2023-09-13T04:37:59.242726+0000","last_unstale":"2023-09-13T04:39:24.585552+0000","last_undegraded":"2023-09-13T04:39:24.585552+0000","last_fullsized":"2023-09-13T04:39:24.585552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:39:55.115281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583449+0000","last_change":"2023-09-13T04:37:59.247415+0000","last_active":"2023-09-13T04:39:24.583449+0000","last_peered":"2023-09-13T04:39:24.583449+0000","last_clean":"2023-09-13T04:39:24.583449+0000","last_became_active":"2023-09-13T04:37:59.247050+0000","last_became_peered":"2023-09-13T04:37:59.247050+0000","last_unstale":"2023-09-13T04:39:24.583449+0000","last_undegraded":"2023-09-13T04:39:24.583449+0000","last_fullsized":"2023-09-13T04:39:24.583449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:03:44.793628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.588078+0000","last_change":"2023-09-13T04:37:59.248935+0000","last_active":"2023-09-13T04:39:24.588078+0000","last_peered":"2023-09-13T04:39:24.588078+0000","last_clean":"2023-09-13T04:39:24.588078+0000","last_became_active":"2023-09-13T04:37:59.248554+0000","last_became_peered":"2023-09-13T04:37:59.248554+0000","last_unstale":"2023-09-13T04:39:24.588078+0000","last_undegraded":"2023-09-13T04:39:24.588078+0000","last_fullsized":"2023-09-13T04:39:24.588078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:53:31.882567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754559+0000","last_change":"2023-09-13T04:37:59.242851+0000","last_active":"2023-09-13T04:39:24.754559+0000","last_peered":"2023-09-13T04:39:24.754559+0000","last_clean":"2023-09-13T04:39:24.754559+0000","last_became_active":"2023-09-13T04:37:59.242384+0000","last_became_peered":"2023-09-13T04:37:59.242384+0000","last_unstale":"2023-09-13T04:39:24.754559+0000","last_undegraded":"2023-09-13T04:39:24.754559+0000","last_fullsized":"2023-09-13T04:39:24.754559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:59:36.508808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754461+0000","last_change":"2023-09-13T04:38:24.599392+0000","last_active":"2023-09-13T04:39:24.754461+0000","last_peered":"2023-09-13T04:39:24.754461+0000","last_clean":"2023-09-13T04:39:24.754461+0000","last_became_active":"2023-09-13T04:38:24.599029+0000","last_became_peered":"2023-09-13T04:38:24.599029+0000","last_unstale":"2023-09-13T04:39:24.754461+0000","last_undegraded":"2023-09-13T04:39:24.754461+0000","last_fullsized":"2023-09-13T04:39:24.754461+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:23:03.703387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.753727+0000","last_change":"2023-09-13T04:37:59.249107+0000","last_active":"2023-09-13T04:39:24.753727+0000","last_peered":"2023-09-13T04:39:24.753727+0000","last_clean":"2023-09-13T04:39:24.753727+0000","last_became_active":"2023-09-13T04:37:59.248803+0000","last_became_peered":"2023-09-13T04:37:59.248803+0000","last_unstale":"2023-09-13T04:39:24.753727+0000","last_undegraded":"2023-09-13T04:39:24.753727+0000","last_fullsized":"2023-09-13T04:39:24.753727+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:50:52.481194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755155+0000","last_change":"2023-09-13T04:37:59.243724+0000","last_active":"2023-09-13T04:39:24.755155+0000","last_peered":"2023-09-13T04:39:24.755155+0000","last_clean":"2023-09-13T04:39:24.755155+0000","last_became_active":"2023-09-13T04:37:59.243385+0000","last_became_peered":"2023-09-13T04:37:59.243385+0000","last_unstale":"2023-09-13T04:39:24.755155+0000","last_undegraded":"2023-09-13T04:39:24.755155+0000","last_fullsized":"2023-09-13T04:39:24.755155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:55:55.203940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582498+0000","last_change":"2023-09-13T04:37:59.238986+0000","last_active":"2023-09-13T04:39:24.582498+0000","last_peered":"2023-09-13T04:39:24.582498+0000","last_clean":"2023-09-13T04:39:24.582498+0000","last_became_active":"2023-09-13T04:37:59.238668+0000","last_became_peered":"2023-09-13T04:37:59.238668+0000","last_unstale":"2023-09-13T04:39:24.582498+0000","last_undegraded":"2023-09-13T04:39:24.582498+0000","last_fullsized":"2023-09-13T04:39:24.582498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:21:10.225223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583227+0000","last_change":"2023-09-13T04:37:59.245598+0000","last_active":"2023-09-13T04:39:24.583227+0000","last_peered":"2023-09-13T04:39:24.583227+0000","last_clean":"2023-09-13T04:39:24.583227+0000","last_became_active":"2023-09-13T04:37:59.244652+0000","last_became_peered":"2023-09-13T04:37:59.244652+0000","last_unstale":"2023-09-13T04:39:24.583227+0000","last_undegraded":"2023-09-13T04:39:24.583227+0000","last_fullsized":"2023-09-13T04:39:24.583227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:34:43.671168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583096+0000","last_change":"2023-09-13T04:38:03.617708+0000","last_active":"2023-09-13T04:39:24.583096+0000","last_peered":"2023-09-13T04:39:24.583096+0000","last_clean":"2023-09-13T04:39:24.583096+0000","last_became_active":"2023-09-13T04:38:03.617277+0000","last_became_peered":"2023-09-13T04:38:03.617277+0000","last_unstale":"2023-09-13T04:39:24.583096+0000","last_undegraded":"2023-09-13T04:39:24.583096+0000","last_fullsized":"2023-09-13T04:39:24.583096+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:26:44.318030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.588015+0000","last_change":"2023-09-13T04:37:58.595156+0000","last_active":"2023-09-13T04:39:25.588015+0000","last_peered":"2023-09-13T04:39:25.588015+0000","last_clean":"2023-09-13T04:39:25.588015+0000","last_became_active":"2023-09-13T04:37:58.594863+0000","last_became_peered":"2023-09-13T04:37:58.594863+0000","last_unstale":"2023-09-13T04:39:25.588015+0000","last_undegraded":"2023-09-13T04:39:25.588015+0000","last_fullsized":"2023-09-13T04:39:25.588015+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:27:14.420495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.854946+0000","last_change":"2023-09-13T04:37:59.250252+0000","last_active":"2023-09-13T04:39:25.854946+0000","last_peered":"2023-09-13T04:39:25.854946+0000","last_clean":"2023-09-13T04:39:25.854946+0000","last_became_active":"2023-09-13T04:37:59.249927+0000","last_became_peered":"2023-09-13T04:37:59.249927+0000","last_unstale":"2023-09-13T04:39:25.854946+0000","last_undegraded":"2023-09-13T04:39:25.854946+0000","last_fullsized":"2023-09-13T04:39:25.854946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:18:06.757040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582790+0000","last_change":"2023-09-13T04:38:24.599683+0000","last_active":"2023-09-13T04:39:24.582790+0000","last_peered":"2023-09-13T04:39:24.582790+0000","last_clean":"2023-09-13T04:39:24.582790+0000","last_became_active":"2023-09-13T04:38:24.599230+0000","last_became_peered":"2023-09-13T04:38:24.599230+0000","last_unstale":"2023-09-13T04:39:24.582790+0000","last_undegraded":"2023-09-13T04:39:24.582790+0000","last_fullsized":"2023-09-13T04:39:24.582790+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:48:22.584735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.855657+0000","last_change":"2023-09-13T04:37:58.597785+0000","last_active":"2023-09-13T04:39:25.855657+0000","last_peered":"2023-09-13T04:39:25.855657+0000","last_clean":"2023-09-13T04:39:25.855657+0000","last_became_active":"2023-09-13T04:37:58.597485+0000","last_became_peered":"2023-09-13T04:37:58.597485+0000","last_unstale":"2023-09-13T04:39:25.855657+0000","last_undegraded":"2023-09-13T04:39:25.855657+0000","last_fullsized":"2023-09-13T04:39:25.855657+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:39:57.156038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.856081+0000","last_change":"2023-09-13T04:37:59.241840+0000","last_active":"2023-09-13T04:39:25.856081+0000","last_peered":"2023-09-13T04:39:25.856081+0000","last_clean":"2023-09-13T04:39:25.856081+0000","last_became_active":"2023-09-13T04:37:59.241545+0000","last_became_peered":"2023-09-13T04:37:59.241545+0000","last_unstale":"2023-09-13T04:39:25.856081+0000","last_undegraded":"2023-09-13T04:39:25.856081+0000","last_fullsized":"2023-09-13T04:39:25.856081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:04:17.439520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582992+0000","last_change":"2023-09-13T04:38:03.621792+0000","last_active":"2023-09-13T04:39:24.582992+0000","last_peered":"2023-09-13T04:39:24.582992+0000","last_clean":"2023-09-13T04:39:24.582992+0000","last_became_active":"2023-09-13T04:38:03.621433+0000","last_became_peered":"2023-09-13T04:38:03.621433+0000","last_unstale":"2023-09-13T04:39:24.582992+0000","last_undegraded":"2023-09-13T04:39:24.582992+0000","last_fullsized":"2023-09-13T04:39:24.582992+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:52:05.926733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754925+0000","last_change":"2023-09-13T04:37:58.590817+0000","last_active":"2023-09-13T04:39:24.754925+0000","last_peered":"2023-09-13T04:39:24.754925+0000","last_clean":"2023-09-13T04:39:24.754925+0000","last_became_active":"2023-09-13T04:37:58.590348+0000","last_became_peered":"2023-09-13T04:37:58.590348+0000","last_unstale":"2023-09-13T04:39:24.754925+0000","last_undegraded":"2023-09-13T04:39:24.754925+0000","last_fullsized":"2023-09-13T04:39:24.754925+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:38:35.680905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754016+0000","last_change":"2023-09-13T04:37:59.237304+0000","last_active":"2023-09-13T04:39:24.754016+0000","last_peered":"2023-09-13T04:39:24.754016+0000","last_clean":"2023-09-13T04:39:24.754016+0000","last_became_active":"2023-09-13T04:37:59.237008+0000","last_became_peered":"2023-09-13T04:37:59.237008+0000","last_unstale":"2023-09-13T04:39:24.754016+0000","last_undegraded":"2023-09-13T04:39:24.754016+0000","last_fullsized":"2023-09-13T04:39:24.754016+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:28:14.254219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582847+0000","last_change":"2023-09-13T04:38:03.606749+0000","last_active":"2023-09-13T04:39:24.582847+0000","last_peered":"2023-09-13T04:39:24.582847+0000","last_clean":"2023-09-13T04:39:24.582847+0000","last_became_active":"2023-09-13T04:38:03.606276+0000","last_became_peered":"2023-09-13T04:38:03.606276+0000","last_unstale":"2023-09-13T04:39:24.582847+0000","last_undegraded":"2023-09-13T04:39:24.582847+0000","last_fullsized":"2023-09-13T04:39:24.582847+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:30:53.572543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754722+0000","last_change":"2023-09-13T04:37:58.590937+0000","last_active":"2023-09-13T04:39:24.754722+0000","last_peered":"2023-09-13T04:39:24.754722+0000","last_clean":"2023-09-13T04:39:24.754722+0000","last_became_active":"2023-09-13T04:37:58.590606+0000","last_became_peered":"2023-09-13T04:37:58.590606+0000","last_unstale":"2023-09-13T04:39:24.754722+0000","last_undegraded":"2023-09-13T04:39:24.754722+0000","last_fullsized":"2023-09-13T04:39:24.754722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:30:03.354670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.706476+0000","last_change":"2023-09-13T04:37:59.251899+0000","last_active":"2023-09-13T04:39:24.706476+0000","last_peered":"2023-09-13T04:39:24.706476+0000","last_clean":"2023-09-13T04:39:24.706476+0000","last_became_active":"2023-09-13T04:37:59.251557+0000","last_became_peered":"2023-09-13T04:37:59.251557+0000","last_unstale":"2023-09-13T04:39:24.706476+0000","last_undegraded":"2023-09-13T04:39:24.706476+0000","last_fullsized":"2023-09-13T04:39:24.706476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:57:45.241188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582997+0000","last_change":"2023-09-13T04:38:03.606076+0000","last_active":"2023-09-13T04:39:24.582997+0000","last_peered":"2023-09-13T04:39:24.582997+0000","last_clean":"2023-09-13T04:39:24.582997+0000","last_became_active":"2023-09-13T04:38:03.605556+0000","last_became_peered":"2023-09-13T04:38:03.605556+0000","last_unstale":"2023-09-13T04:39:24.582997+0000","last_undegraded":"2023-09-13T04:39:24.582997+0000","last_fullsized":"2023-09-13T04:39:24.582997+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:03:32.344625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":132,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:40:08.771722+0000","last_change":"2023-09-13T04:37:58.588170+0000","last_active":"2023-09-13T04:40:08.771722+0000","last_peered":"2023-09-13T04:40:08.771722+0000","last_clean":"2023-09-13T04:40:08.771722+0000","last_became_active":"2023-09-13T04:37:58.587860+0000","last_became_peered":"2023-09-13T04:37:58.587860+0000","last_unstale":"2023-09-13T04:40:08.771722+0000","last_undegraded":"2023-09-13T04:40:08.771722+0000","last_fullsized":"2023-09-13T04:40:08.771722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:38:40.048150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754818+0000","last_change":"2023-09-13T04:37:59.240248+0000","last_active":"2023-09-13T04:39:24.754818+0000","last_peered":"2023-09-13T04:39:24.754818+0000","last_clean":"2023-09-13T04:39:24.754818+0000","last_became_active":"2023-09-13T04:37:59.240008+0000","last_became_peered":"2023-09-13T04:37:59.240008+0000","last_unstale":"2023-09-13T04:39:24.754818+0000","last_undegraded":"2023-09-13T04:39:24.754818+0000","last_fullsized":"2023-09-13T04:39:24.754818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:15:13.275110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583113+0000","last_change":"2023-09-13T04:38:03.616607+0000","last_active":"2023-09-13T04:39:24.583113+0000","last_peered":"2023-09-13T04:39:24.583113+0000","last_clean":"2023-09-13T04:39:24.583113+0000","last_became_active":"2023-09-13T04:38:03.616209+0000","last_became_peered":"2023-09-13T04:38:03.616209+0000","last_unstale":"2023-09-13T04:39:24.583113+0000","last_undegraded":"2023-09-13T04:39:24.583113+0000","last_fullsized":"2023-09-13T04:39:24.583113+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:24:21.934053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.703747+0000","last_change":"2023-09-13T04:37:58.589104+0000","last_active":"2023-09-13T04:39:24.703747+0000","last_peered":"2023-09-13T04:39:24.703747+0000","last_clean":"2023-09-13T04:39:24.703747+0000","last_became_active":"2023-09-13T04:37:58.588780+0000","last_became_peered":"2023-09-13T04:37:58.588780+0000","last_unstale":"2023-09-13T04:39:24.703747+0000","last_undegraded":"2023-09-13T04:39:24.703747+0000","last_fullsized":"2023-09-13T04:39:24.703747+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:01:32.839729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582218+0000","last_change":"2023-09-13T04:38:24.598663+0000","last_active":"2023-09-13T04:39:24.582218+0000","last_peered":"2023-09-13T04:39:24.582218+0000","last_clean":"2023-09-13T04:39:24.582218+0000","last_became_active":"2023-09-13T04:38:24.598258+0000","last_became_peered":"2023-09-13T04:38:24.598258+0000","last_unstale":"2023-09-13T04:39:24.582218+0000","last_undegraded":"2023-09-13T04:39:24.582218+0000","last_fullsized":"2023-09-13T04:39:24.582218+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:50:28.157135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.624295+0000","last_change":"2023-09-13T04:38:03.619930+0000","last_active":"2023-09-13T04:39:25.624295+0000","last_peered":"2023-09-13T04:39:25.624295+0000","last_clean":"2023-09-13T04:39:25.624295+0000","last_became_active":"2023-09-13T04:38:03.619560+0000","last_became_peered":"2023-09-13T04:38:03.619560+0000","last_unstale":"2023-09-13T04:39:25.624295+0000","last_undegraded":"2023-09-13T04:39:25.624295+0000","last_fullsized":"2023-09-13T04:39:25.624295+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:48:06.867602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.587499+0000","last_change":"2023-09-13T04:37:58.590396+0000","last_active":"2023-09-13T04:39:24.587499+0000","last_peered":"2023-09-13T04:39:24.587499+0000","last_clean":"2023-09-13T04:39:24.587499+0000","last_became_active":"2023-09-13T04:37:58.589929+0000","last_became_peered":"2023-09-13T04:37:58.589929+0000","last_unstale":"2023-09-13T04:39:24.587499+0000","last_undegraded":"2023-09-13T04:39:24.587499+0000","last_fullsized":"2023-09-13T04:39:24.587499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:28:36.322096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754248+0000","last_change":"2023-09-13T04:37:59.246033+0000","last_active":"2023-09-13T04:39:24.754248+0000","last_peered":"2023-09-13T04:39:24.754248+0000","last_clean":"2023-09-13T04:39:24.754248+0000","last_became_active":"2023-09-13T04:37:59.245664+0000","last_became_peered":"2023-09-13T04:37:59.245664+0000","last_unstale":"2023-09-13T04:39:24.754248+0000","last_undegraded":"2023-09-13T04:39:24.754248+0000","last_fullsized":"2023-09-13T04:39:24.754248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:24:17.145055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.856052+0000","last_change":"2023-09-13T04:38:03.608504+0000","last_active":"2023-09-13T04:39:25.856052+0000","last_peered":"2023-09-13T04:39:25.856052+0000","last_clean":"2023-09-13T04:39:25.856052+0000","last_became_active":"2023-09-13T04:38:03.608058+0000","last_became_peered":"2023-09-13T04:38:03.608058+0000","last_unstale":"2023-09-13T04:39:25.856052+0000","last_undegraded":"2023-09-13T04:39:25.856052+0000","last_fullsized":"2023-09-13T04:39:25.856052+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:42:43.485453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.705989+0000","last_change":"2023-09-13T04:37:58.588208+0000","last_active":"2023-09-13T04:39:24.705989+0000","last_peered":"2023-09-13T04:39:24.705989+0000","last_clean":"2023-09-13T04:39:24.705989+0000","last_became_active":"2023-09-13T04:37:58.587840+0000","last_became_peered":"2023-09-13T04:37:58.587840+0000","last_unstale":"2023-09-13T04:39:24.705989+0000","last_undegraded":"2023-09-13T04:39:24.705989+0000","last_fullsized":"2023-09-13T04:39:24.705989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:44:58.012233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582291+0000","last_change":"2023-09-13T04:37:59.245102+0000","last_active":"2023-09-13T04:39:24.582291+0000","last_peered":"2023-09-13T04:39:24.582291+0000","last_clean":"2023-09-13T04:39:24.582291+0000","last_became_active":"2023-09-13T04:37:59.244799+0000","last_became_peered":"2023-09-13T04:37:59.244799+0000","last_unstale":"2023-09-13T04:39:24.582291+0000","last_undegraded":"2023-09-13T04:39:24.582291+0000","last_fullsized":"2023-09-13T04:39:24.582291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:57:16.295623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754657+0000","last_change":"2023-09-13T04:38:03.607021+0000","last_active":"2023-09-13T04:39:24.754657+0000","last_peered":"2023-09-13T04:39:24.754657+0000","last_clean":"2023-09-13T04:39:24.754657+0000","last_became_active":"2023-09-13T04:38:03.606611+0000","last_became_peered":"2023-09-13T04:38:03.606611+0000","last_unstale":"2023-09-13T04:39:24.754657+0000","last_undegraded":"2023-09-13T04:39:24.754657+0000","last_fullsized":"2023-09-13T04:39:24.754657+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:33:20.551479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":127,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:40:08.771101+0000","last_change":"2023-09-13T04:37:58.588731+0000","last_active":"2023-09-13T04:40:08.771101+0000","last_peered":"2023-09-13T04:40:08.771101+0000","last_clean":"2023-09-13T04:40:08.771101+0000","last_became_active":"2023-09-13T04:37:58.588398+0000","last_became_peered":"2023-09-13T04:37:58.588398+0000","last_unstale":"2023-09-13T04:40:08.771101+0000","last_undegraded":"2023-09-13T04:40:08.771101+0000","last_fullsized":"2023-09-13T04:40:08.771101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:13:10.724085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754323+0000","last_change":"2023-09-13T04:37:59.244757+0000","last_active":"2023-09-13T04:39:24.754323+0000","last_peered":"2023-09-13T04:39:24.754323+0000","last_clean":"2023-09-13T04:39:24.754323+0000","last_became_active":"2023-09-13T04:37:59.244423+0000","last_became_peered":"2023-09-13T04:37:59.244423+0000","last_unstale":"2023-09-13T04:39:24.754323+0000","last_undegraded":"2023-09-13T04:39:24.754323+0000","last_fullsized":"2023-09-13T04:39:24.754323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:14:37.973058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582006+0000","last_change":"2023-09-13T04:38:03.618358+0000","last_active":"2023-09-13T04:39:24.582006+0000","last_peered":"2023-09-13T04:39:24.582006+0000","last_clean":"2023-09-13T04:39:24.582006+0000","last_became_active":"2023-09-13T04:38:03.617291+0000","last_became_peered":"2023-09-13T04:38:03.617291+0000","last_unstale":"2023-09-13T04:39:24.582006+0000","last_undegraded":"2023-09-13T04:39:24.582006+0000","last_fullsized":"2023-09-13T04:39:24.582006+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:33:13.955353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.585686+0000","last_change":"2023-09-13T04:37:58.590978+0000","last_active":"2023-09-13T04:39:24.585686+0000","last_peered":"2023-09-13T04:39:24.585686+0000","last_clean":"2023-09-13T04:39:24.585686+0000","last_became_active":"2023-09-13T04:37:58.590673+0000","last_became_peered":"2023-09-13T04:37:58.590673+0000","last_unstale":"2023-09-13T04:39:24.585686+0000","last_undegraded":"2023-09-13T04:39:24.585686+0000","last_fullsized":"2023-09-13T04:39:24.585686+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:10:32.570029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755812+0000","last_change":"2023-09-13T04:37:59.251283+0000","last_active":"2023-09-13T04:39:24.755812+0000","last_peered":"2023-09-13T04:39:24.755812+0000","last_clean":"2023-09-13T04:39:24.755812+0000","last_became_active":"2023-09-13T04:37:59.250119+0000","last_became_peered":"2023-09-13T04:37:59.250119+0000","last_unstale":"2023-09-13T04:39:24.755812+0000","last_undegraded":"2023-09-13T04:39:24.755812+0000","last_fullsized":"2023-09-13T04:39:24.755812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:48:54.389970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583163+0000","last_change":"2023-09-13T04:38:24.596107+0000","last_active":"2023-09-13T04:39:24.583163+0000","last_peered":"2023-09-13T04:39:24.583163+0000","last_clean":"2023-09-13T04:39:24.583163+0000","last_became_active":"2023-09-13T04:38:24.595653+0000","last_became_peered":"2023-09-13T04:38:24.595653+0000","last_unstale":"2023-09-13T04:39:24.583163+0000","last_undegraded":"2023-09-13T04:39:24.583163+0000","last_fullsized":"2023-09-13T04:39:24.583163+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:02:53.708658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.706415+0000","last_change":"2023-09-13T04:37:58.584909+0000","last_active":"2023-09-13T04:39:24.706415+0000","last_peered":"2023-09-13T04:39:24.706415+0000","last_clean":"2023-09-13T04:39:24.706415+0000","last_became_active":"2023-09-13T04:37:58.584630+0000","last_became_peered":"2023-09-13T04:37:58.584630+0000","last_unstale":"2023-09-13T04:39:24.706415+0000","last_undegraded":"2023-09-13T04:39:24.706415+0000","last_fullsized":"2023-09-13T04:39:24.706415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:57:56.567499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754125+0000","last_change":"2023-09-13T04:37:59.254448+0000","last_active":"2023-09-13T04:39:24.754125+0000","last_peered":"2023-09-13T04:39:24.754125+0000","last_clean":"2023-09-13T04:39:24.754125+0000","last_became_active":"2023-09-13T04:37:59.254138+0000","last_became_peered":"2023-09-13T04:37:59.254138+0000","last_unstale":"2023-09-13T04:39:24.754125+0000","last_undegraded":"2023-09-13T04:39:24.754125+0000","last_fullsized":"2023-09-13T04:39:24.754125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:36:14.939881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582104+0000","last_change":"2023-09-13T04:38:03.613746+0000","last_active":"2023-09-13T04:39:24.582104+0000","last_peered":"2023-09-13T04:39:24.582104+0000","last_clean":"2023-09-13T04:39:24.582104+0000","last_became_active":"2023-09-13T04:38:03.613354+0000","last_became_peered":"2023-09-13T04:38:03.613354+0000","last_unstale":"2023-09-13T04:39:24.582104+0000","last_undegraded":"2023-09-13T04:39:24.582104+0000","last_fullsized":"2023-09-13T04:39:24.582104+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:41:01.294912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'14","reported_seq":245,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:40:09.772943+0000","last_change":"2023-09-13T04:37:58.591196+0000","last_active":"2023-09-13T04:40:09.772943+0000","last_peered":"2023-09-13T04:40:09.772943+0000","last_clean":"2023-09-13T04:40:09.772943+0000","last_became_active":"2023-09-13T04:37:58.590883+0000","last_became_peered":"2023-09-13T04:37:58.590883+0000","last_unstale":"2023-09-13T04:40:09.772943+0000","last_undegraded":"2023-09-13T04:40:09.772943+0000","last_fullsized":"2023-09-13T04:40:09.772943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:46:58.530270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4100,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":211,"num_read_kb":110,"num_write":15,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755199+0000","last_change":"2023-09-13T04:37:59.243483+0000","last_active":"2023-09-13T04:39:24.755199+0000","last_peered":"2023-09-13T04:39:24.755199+0000","last_clean":"2023-09-13T04:39:24.755199+0000","last_became_active":"2023-09-13T04:37:59.243088+0000","last_became_peered":"2023-09-13T04:37:59.243088+0000","last_unstale":"2023-09-13T04:39:24.755199+0000","last_undegraded":"2023-09-13T04:39:24.755199+0000","last_fullsized":"2023-09-13T04:39:24.755199+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:56:14.093819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.706663+0000","last_change":"2023-09-13T04:38:03.608820+0000","last_active":"2023-09-13T04:39:24.706663+0000","last_peered":"2023-09-13T04:39:24.706663+0000","last_clean":"2023-09-13T04:39:24.706663+0000","last_became_active":"2023-09-13T04:38:03.608200+0000","last_became_peered":"2023-09-13T04:38:03.608200+0000","last_unstale":"2023-09-13T04:39:24.706663+0000","last_undegraded":"2023-09-13T04:39:24.706663+0000","last_fullsized":"2023-09-13T04:39:24.706663+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:54:52.416501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754566+0000","last_change":"2023-09-13T04:38:24.606820+0000","last_active":"2023-09-13T04:39:24.754566+0000","last_peered":"2023-09-13T04:39:24.754566+0000","last_clean":"2023-09-13T04:39:24.754566+0000","last_became_active":"2023-09-13T04:38:24.599059+0000","last_became_peered":"2023-09-13T04:38:24.599059+0000","last_unstale":"2023-09-13T04:39:24.754566+0000","last_undegraded":"2023-09-13T04:39:24.754566+0000","last_fullsized":"2023-09-13T04:39:24.754566+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:08:12.634357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.587438+0000","last_change":"2023-09-13T04:37:59.250666+0000","last_active":"2023-09-13T04:39:24.587438+0000","last_peered":"2023-09-13T04:39:24.587438+0000","last_clean":"2023-09-13T04:39:24.587438+0000","last_became_active":"2023-09-13T04:37:59.250240+0000","last_became_peered":"2023-09-13T04:37:59.250240+0000","last_unstale":"2023-09-13T04:39:24.587438+0000","last_undegraded":"2023-09-13T04:39:24.587438+0000","last_fullsized":"2023-09-13T04:39:24.587438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:44:31.324222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754566+0000","last_change":"2023-09-13T04:38:03.616296+0000","last_active":"2023-09-13T04:39:24.754566+0000","last_peered":"2023-09-13T04:39:24.754566+0000","last_clean":"2023-09-13T04:39:24.754566+0000","last_became_active":"2023-09-13T04:38:03.615934+0000","last_became_peered":"2023-09-13T04:38:03.615934+0000","last_unstale":"2023-09-13T04:39:24.754566+0000","last_undegraded":"2023-09-13T04:39:24.754566+0000","last_fullsized":"2023-09-13T04:39:24.754566+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:51:18.994472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.706408+0000","last_change":"2023-09-13T04:37:58.594782+0000","last_active":"2023-09-13T04:39:24.706408+0000","last_peered":"2023-09-13T04:39:24.706408+0000","last_clean":"2023-09-13T04:39:24.706408+0000","last_became_active":"2023-09-13T04:37:58.594480+0000","last_became_peered":"2023-09-13T04:37:58.594480+0000","last_unstale":"2023-09-13T04:39:24.706408+0000","last_undegraded":"2023-09-13T04:39:24.706408+0000","last_fullsized":"2023-09-13T04:39:24.706408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:40:19.635672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.707565+0000","last_change":"2023-09-13T04:39:25.707565+0000","last_active":"2023-09-13T04:39:25.707565+0000","last_peered":"2023-09-13T04:39:25.707565+0000","last_clean":"2023-09-13T04:39:25.707565+0000","last_became_active":"2023-09-13T04:39:25.707099+0000","last_became_peered":"2023-09-13T04:39:25.707099+0000","last_unstale":"2023-09-13T04:39:25.707565+0000","last_undegraded":"2023-09-13T04:39:25.707565+0000","last_fullsized":"2023-09-13T04:39:25.707565+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:25:25.923715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582727+0000","last_change":"2023-09-13T04:38:03.618835+0000","last_active":"2023-09-13T04:39:24.582727+0000","last_peered":"2023-09-13T04:39:24.582727+0000","last_clean":"2023-09-13T04:39:24.582727+0000","last_became_active":"2023-09-13T04:38:03.618226+0000","last_became_peered":"2023-09-13T04:38:03.618226+0000","last_unstale":"2023-09-13T04:39:24.582727+0000","last_undegraded":"2023-09-13T04:39:24.582727+0000","last_fullsized":"2023-09-13T04:39:24.582727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:01:28.157490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":129,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:40:08.771978+0000","last_change":"2023-09-13T04:37:58.588545+0000","last_active":"2023-09-13T04:40:08.771978+0000","last_peered":"2023-09-13T04:40:08.771978+0000","last_clean":"2023-09-13T04:40:08.771978+0000","last_became_active":"2023-09-13T04:37:58.587995+0000","last_became_peered":"2023-09-13T04:37:58.587995+0000","last_unstale":"2023-09-13T04:40:08.771978+0000","last_undegraded":"2023-09-13T04:40:08.771978+0000","last_fullsized":"2023-09-13T04:40:08.771978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:40:08.797724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.585205+0000","last_change":"2023-09-13T04:37:59.249100+0000","last_active":"2023-09-13T04:39:24.585205+0000","last_peered":"2023-09-13T04:39:24.585205+0000","last_clean":"2023-09-13T04:39:24.585205+0000","last_became_active":"2023-09-13T04:37:59.248771+0000","last_became_peered":"2023-09-13T04:37:59.248771+0000","last_unstale":"2023-09-13T04:39:24.585205+0000","last_undegraded":"2023-09-13T04:39:24.585205+0000","last_fullsized":"2023-09-13T04:39:24.585205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:27:37.964595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.587764+0000","last_change":"2023-09-13T04:38:03.616988+0000","last_active":"2023-09-13T04:39:24.587764+0000","last_peered":"2023-09-13T04:39:24.587764+0000","last_clean":"2023-09-13T04:39:24.587764+0000","last_became_active":"2023-09-13T04:38:03.616524+0000","last_became_peered":"2023-09-13T04:38:03.616524+0000","last_unstale":"2023-09-13T04:39:24.587764+0000","last_undegraded":"2023-09-13T04:39:24.587764+0000","last_fullsized":"2023-09-13T04:39:24.587764+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:42:41.431115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754413+0000","last_change":"2023-09-13T04:38:24.613384+0000","last_active":"2023-09-13T04:39:24.754413+0000","last_peered":"2023-09-13T04:39:24.754413+0000","last_clean":"2023-09-13T04:39:24.754413+0000","last_became_active":"2023-09-13T04:38:24.599349+0000","last_became_peered":"2023-09-13T04:38:24.599349+0000","last_unstale":"2023-09-13T04:39:24.754413+0000","last_undegraded":"2023-09-13T04:39:24.754413+0000","last_fullsized":"2023-09-13T04:39:24.754413+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:23:23.479715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.706144+0000","last_change":"2023-09-13T04:37:59.252237+0000","last_active":"2023-09-13T04:39:24.706144+0000","last_peered":"2023-09-13T04:39:24.706144+0000","last_clean":"2023-09-13T04:39:24.706144+0000","last_became_active":"2023-09-13T04:37:59.251718+0000","last_became_peered":"2023-09-13T04:37:59.251718+0000","last_unstale":"2023-09-13T04:39:24.706144+0000","last_undegraded":"2023-09-13T04:39:24.706144+0000","last_fullsized":"2023-09-13T04:39:24.706144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:45:48.405226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.706124+0000","last_change":"2023-09-13T04:38:03.607096+0000","last_active":"2023-09-13T04:39:24.706124+0000","last_peered":"2023-09-13T04:39:24.706124+0000","last_clean":"2023-09-13T04:39:24.706124+0000","last_became_active":"2023-09-13T04:38:03.606656+0000","last_became_peered":"2023-09-13T04:38:03.606656+0000","last_unstale":"2023-09-13T04:39:24.706124+0000","last_undegraded":"2023-09-13T04:39:24.706124+0000","last_fullsized":"2023-09-13T04:39:24.706124+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:59:26.140360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755201+0000","last_change":"2023-09-13T04:37:58.591047+0000","last_active":"2023-09-13T04:39:24.755201+0000","last_peered":"2023-09-13T04:39:24.755201+0000","last_clean":"2023-09-13T04:39:24.755201+0000","last_became_active":"2023-09-13T04:37:58.590740+0000","last_became_peered":"2023-09-13T04:37:58.590740+0000","last_unstale":"2023-09-13T04:39:24.755201+0000","last_undegraded":"2023-09-13T04:39:24.755201+0000","last_fullsized":"2023-09-13T04:39:24.755201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:14:42.280416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755283+0000","last_change":"2023-09-13T04:37:59.249362+0000","last_active":"2023-09-13T04:39:24.755283+0000","last_peered":"2023-09-13T04:39:24.755283+0000","last_clean":"2023-09-13T04:39:24.755283+0000","last_became_active":"2023-09-13T04:37:59.249041+0000","last_became_peered":"2023-09-13T04:37:59.249041+0000","last_unstale":"2023-09-13T04:39:24.755283+0000","last_undegraded":"2023-09-13T04:39:24.755283+0000","last_fullsized":"2023-09-13T04:39:24.755283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:06:33.822157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.588524+0000","last_change":"2023-09-13T04:38:24.598370+0000","last_active":"2023-09-13T04:39:25.588524+0000","last_peered":"2023-09-13T04:39:25.588524+0000","last_clean":"2023-09-13T04:39:25.588524+0000","last_became_active":"2023-09-13T04:38:24.597931+0000","last_became_peered":"2023-09-13T04:38:24.597931+0000","last_unstale":"2023-09-13T04:39:25.588524+0000","last_undegraded":"2023-09-13T04:39:25.588524+0000","last_fullsized":"2023-09-13T04:39:25.588524+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:14:28.911329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.587714+0000","last_change":"2023-09-13T04:37:58.592496+0000","last_active":"2023-09-13T04:39:25.587714+0000","last_peered":"2023-09-13T04:39:25.587714+0000","last_clean":"2023-09-13T04:39:25.587714+0000","last_became_active":"2023-09-13T04:37:58.592106+0000","last_became_peered":"2023-09-13T04:37:58.592106+0000","last_unstale":"2023-09-13T04:39:25.587714+0000","last_undegraded":"2023-09-13T04:39:25.587714+0000","last_fullsized":"2023-09-13T04:39:25.587714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:28:55.618563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.592208+0000","last_change":"2023-09-13T04:37:59.250019+0000","last_active":"2023-09-13T04:39:25.592208+0000","last_peered":"2023-09-13T04:39:25.592208+0000","last_clean":"2023-09-13T04:39:25.592208+0000","last_became_active":"2023-09-13T04:37:59.249278+0000","last_became_peered":"2023-09-13T04:37:59.249278+0000","last_unstale":"2023-09-13T04:39:25.592208+0000","last_undegraded":"2023-09-13T04:39:25.592208+0000","last_fullsized":"2023-09-13T04:39:25.592208+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:15:31.217236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.705979+0000","last_change":"2023-09-13T04:38:03.614542+0000","last_active":"2023-09-13T04:39:25.705979+0000","last_peered":"2023-09-13T04:39:25.705979+0000","last_clean":"2023-09-13T04:39:25.705979+0000","last_became_active":"2023-09-13T04:38:03.614103+0000","last_became_peered":"2023-09-13T04:38:03.614103+0000","last_unstale":"2023-09-13T04:39:25.705979+0000","last_undegraded":"2023-09-13T04:39:25.705979+0000","last_fullsized":"2023-09-13T04:39:25.705979+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:06:48.426843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.595603+0000","last_change":"2023-09-13T04:37:58.591864+0000","last_active":"2023-09-13T04:39:25.595603+0000","last_peered":"2023-09-13T04:39:25.595603+0000","last_clean":"2023-09-13T04:39:25.595603+0000","last_became_active":"2023-09-13T04:37:58.591562+0000","last_became_peered":"2023-09-13T04:37:58.591562+0000","last_unstale":"2023-09-13T04:39:25.595603+0000","last_undegraded":"2023-09-13T04:39:25.595603+0000","last_fullsized":"2023-09-13T04:39:25.595603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:31:27.280489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.592296+0000","last_change":"2023-09-13T04:37:59.252600+0000","last_active":"2023-09-13T04:39:25.592296+0000","last_peered":"2023-09-13T04:39:25.592296+0000","last_clean":"2023-09-13T04:39:25.592296+0000","last_became_active":"2023-09-13T04:37:59.252306+0000","last_became_peered":"2023-09-13T04:37:59.252306+0000","last_unstale":"2023-09-13T04:39:25.592296+0000","last_undegraded":"2023-09-13T04:39:25.592296+0000","last_fullsized":"2023-09-13T04:39:25.592296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:18:55.686847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.587587+0000","last_change":"2023-09-13T04:38:03.742502+0000","last_active":"2023-09-13T04:39:24.587587+0000","last_peered":"2023-09-13T04:39:24.587587+0000","last_clean":"2023-09-13T04:39:24.587587+0000","last_became_active":"2023-09-13T04:38:03.742054+0000","last_became_peered":"2023-09-13T04:38:03.742054+0000","last_unstale":"2023-09-13T04:39:24.587587+0000","last_undegraded":"2023-09-13T04:39:24.587587+0000","last_fullsized":"2023-09-13T04:39:24.587587+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:55:54.601145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.709865+0000","last_change":"2023-09-13T04:37:59.245644+0000","last_active":"2023-09-13T04:39:25.709865+0000","last_peered":"2023-09-13T04:39:25.709865+0000","last_clean":"2023-09-13T04:39:25.709865+0000","last_became_active":"2023-09-13T04:37:59.245284+0000","last_became_peered":"2023-09-13T04:37:59.245284+0000","last_unstale":"2023-09-13T04:39:25.709865+0000","last_undegraded":"2023-09-13T04:39:25.709865+0000","last_fullsized":"2023-09-13T04:39:25.709865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:19:04.002414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":131,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:40:09.772897+0000","last_change":"2023-09-13T04:37:58.590249+0000","last_active":"2023-09-13T04:40:09.772897+0000","last_peered":"2023-09-13T04:40:09.772897+0000","last_clean":"2023-09-13T04:40:09.772897+0000","last_became_active":"2023-09-13T04:37:58.589521+0000","last_became_peered":"2023-09-13T04:37:58.589521+0000","last_unstale":"2023-09-13T04:40:09.772897+0000","last_undegraded":"2023-09-13T04:40:09.772897+0000","last_fullsized":"2023-09-13T04:40:09.772897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:11:57.542149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.595865+0000","last_change":"2023-09-13T04:38:24.598613+0000","last_active":"2023-09-13T04:39:25.595865+0000","last_peered":"2023-09-13T04:39:25.595865+0000","last_clean":"2023-09-13T04:39:25.595865+0000","last_became_active":"2023-09-13T04:38:24.598150+0000","last_became_peered":"2023-09-13T04:38:24.598150+0000","last_unstale":"2023-09-13T04:39:25.595865+0000","last_undegraded":"2023-09-13T04:39:25.595865+0000","last_fullsized":"2023-09-13T04:39:25.595865+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:02:29.894446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.854964+0000","last_change":"2023-09-13T04:37:59.243794+0000","last_active":"2023-09-13T04:39:25.854964+0000","last_peered":"2023-09-13T04:39:25.854964+0000","last_clean":"2023-09-13T04:39:25.854964+0000","last_became_active":"2023-09-13T04:37:59.243368+0000","last_became_peered":"2023-09-13T04:37:59.243368+0000","last_unstale":"2023-09-13T04:39:25.854964+0000","last_undegraded":"2023-09-13T04:39:25.854964+0000","last_fullsized":"2023-09-13T04:39:25.854964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:06:45.318737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.710051+0000","last_change":"2023-09-13T04:37:58.589195+0000","last_active":"2023-09-13T04:39:25.710051+0000","last_peered":"2023-09-13T04:39:25.710051+0000","last_clean":"2023-09-13T04:39:25.710051+0000","last_became_active":"2023-09-13T04:37:58.588905+0000","last_became_peered":"2023-09-13T04:37:58.588905+0000","last_unstale":"2023-09-13T04:39:25.710051+0000","last_undegraded":"2023-09-13T04:39:25.710051+0000","last_fullsized":"2023-09-13T04:39:25.710051+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:13:15.978891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":329,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.595369+0000","last_change":"2023-09-13T04:36:52.227264+0000","last_active":"2023-09-13T04:39:25.595369+0000","last_peered":"2023-09-13T04:39:25.595369+0000","last_clean":"2023-09-13T04:39:25.595369+0000","last_became_active":"2023-09-13T04:36:52.226773+0000","last_became_peered":"2023-09-13T04:36:52.226773+0000","last_unstale":"2023-09-13T04:39:25.595369+0000","last_undegraded":"2023-09-13T04:39:25.595369+0000","last_fullsized":"2023-09-13T04:39:25.595369+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:33:42.026499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:33:42.026499+0000","last_clean_scrub_stamp":"2023-09-13T04:33:42.026499+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:28:55.806587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.706548+0000","last_change":"2023-09-13T04:38:03.742790+0000","last_active":"2023-09-13T04:39:24.706548+0000","last_peered":"2023-09-13T04:39:24.706548+0000","last_clean":"2023-09-13T04:39:24.706548+0000","last_became_active":"2023-09-13T04:38:03.742338+0000","last_became_peered":"2023-09-13T04:38:03.742338+0000","last_unstale":"2023-09-13T04:39:24.706548+0000","last_undegraded":"2023-09-13T04:39:24.706548+0000","last_fullsized":"2023-09-13T04:39:24.706548+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:01:38.910623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.705623+0000","last_change":"2023-09-13T04:37:59.252669+0000","last_active":"2023-09-13T04:39:25.705623+0000","last_peered":"2023-09-13T04:39:25.705623+0000","last_clean":"2023-09-13T04:39:25.705623+0000","last_became_active":"2023-09-13T04:37:59.252223+0000","last_became_peered":"2023-09-13T04:37:59.252223+0000","last_unstale":"2023-09-13T04:39:25.705623+0000","last_undegraded":"2023-09-13T04:39:25.705623+0000","last_fullsized":"2023-09-13T04:39:25.705623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:57:21.090083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.595510+0000","last_change":"2023-09-13T04:37:58.590540+0000","last_active":"2023-09-13T04:39:25.595510+0000","last_peered":"2023-09-13T04:39:25.595510+0000","last_clean":"2023-09-13T04:39:25.595510+0000","last_became_active":"2023-09-13T04:37:58.590061+0000","last_became_peered":"2023-09-13T04:37:58.590061+0000","last_unstale":"2023-09-13T04:39:25.595510+0000","last_undegraded":"2023-09-13T04:39:25.595510+0000","last_fullsized":"2023-09-13T04:39:25.595510+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:57:17.133461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.624078+0000","last_change":"2023-09-13T04:38:03.614615+0000","last_active":"2023-09-13T04:39:25.624078+0000","last_peered":"2023-09-13T04:39:25.624078+0000","last_clean":"2023-09-13T04:39:25.624078+0000","last_became_active":"2023-09-13T04:38:03.614086+0000","last_became_peered":"2023-09-13T04:38:03.614086+0000","last_unstale":"2023-09-13T04:39:25.624078+0000","last_undegraded":"2023-09-13T04:39:25.624078+0000","last_fullsized":"2023-09-13T04:39:25.624078+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:25:35.525513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.588681+0000","last_change":"2023-09-13T04:37:58.592096+0000","last_active":"2023-09-13T04:39:25.588681+0000","last_peered":"2023-09-13T04:39:25.588681+0000","last_clean":"2023-09-13T04:39:25.588681+0000","last_became_active":"2023-09-13T04:37:58.591872+0000","last_became_peered":"2023-09-13T04:37:58.591872+0000","last_unstale":"2023-09-13T04:39:25.588681+0000","last_undegraded":"2023-09-13T04:39:25.588681+0000","last_fullsized":"2023-09-13T04:39:25.588681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:05:23.936105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.706127+0000","last_change":"2023-09-13T04:37:59.240759+0000","last_active":"2023-09-13T04:39:25.706127+0000","last_peered":"2023-09-13T04:39:25.706127+0000","last_clean":"2023-09-13T04:39:25.706127+0000","last_became_active":"2023-09-13T04:37:59.240414+0000","last_became_peered":"2023-09-13T04:37:59.240414+0000","last_unstale":"2023-09-13T04:39:25.706127+0000","last_undegraded":"2023-09-13T04:39:25.706127+0000","last_fullsized":"2023-09-13T04:39:25.706127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:37:20.301938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.854802+0000","last_change":"2023-09-13T04:38:03.742519+0000","last_active":"2023-09-13T04:39:25.854802+0000","last_peered":"2023-09-13T04:39:25.854802+0000","last_clean":"2023-09-13T04:39:25.854802+0000","last_became_active":"2023-09-13T04:38:03.742075+0000","last_became_peered":"2023-09-13T04:38:03.742075+0000","last_unstale":"2023-09-13T04:39:25.854802+0000","last_undegraded":"2023-09-13T04:39:25.854802+0000","last_fullsized":"2023-09-13T04:39:25.854802+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:14:33.155837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582560+0000","last_change":"2023-09-13T04:37:58.593891+0000","last_active":"2023-09-13T04:39:24.582560+0000","last_peered":"2023-09-13T04:39:24.582560+0000","last_clean":"2023-09-13T04:39:24.582560+0000","last_became_active":"2023-09-13T04:37:58.593497+0000","last_became_peered":"2023-09-13T04:37:58.593497+0000","last_unstale":"2023-09-13T04:39:24.582560+0000","last_undegraded":"2023-09-13T04:39:24.582560+0000","last_fullsized":"2023-09-13T04:39:24.582560+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:25:53.652601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754528+0000","last_change":"2023-09-13T04:37:59.247151+0000","last_active":"2023-09-13T04:39:24.754528+0000","last_peered":"2023-09-13T04:39:24.754528+0000","last_clean":"2023-09-13T04:39:24.754528+0000","last_became_active":"2023-09-13T04:37:59.246853+0000","last_became_peered":"2023-09-13T04:37:59.246853+0000","last_unstale":"2023-09-13T04:39:24.754528+0000","last_undegraded":"2023-09-13T04:39:24.754528+0000","last_fullsized":"2023-09-13T04:39:24.754528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:15:22.659365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582686+0000","last_change":"2023-09-13T04:38:03.606594+0000","last_active":"2023-09-13T04:39:24.582686+0000","last_peered":"2023-09-13T04:39:24.582686+0000","last_clean":"2023-09-13T04:39:24.582686+0000","last_became_active":"2023-09-13T04:38:03.606058+0000","last_became_peered":"2023-09-13T04:38:03.606058+0000","last_unstale":"2023-09-13T04:39:24.582686+0000","last_undegraded":"2023-09-13T04:39:24.582686+0000","last_fullsized":"2023-09-13T04:39:24.582686+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:57:39.237520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.709689+0000","last_change":"2023-09-13T04:37:58.588362+0000","last_active":"2023-09-13T04:39:25.709689+0000","last_peered":"2023-09-13T04:39:25.709689+0000","last_clean":"2023-09-13T04:39:25.709689+0000","last_became_active":"2023-09-13T04:37:58.588065+0000","last_became_peered":"2023-09-13T04:37:58.588065+0000","last_unstale":"2023-09-13T04:39:25.709689+0000","last_undegraded":"2023-09-13T04:39:25.709689+0000","last_fullsized":"2023-09-13T04:39:25.709689+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:32:04.255654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.856028+0000","last_change":"2023-09-13T04:37:59.251883+0000","last_active":"2023-09-13T04:39:25.856028+0000","last_peered":"2023-09-13T04:39:25.856028+0000","last_clean":"2023-09-13T04:39:25.856028+0000","last_became_active":"2023-09-13T04:37:59.251522+0000","last_became_peered":"2023-09-13T04:37:59.251522+0000","last_unstale":"2023-09-13T04:39:25.856028+0000","last_undegraded":"2023-09-13T04:39:25.856028+0000","last_fullsized":"2023-09-13T04:39:25.856028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:05:06.420799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583139+0000","last_change":"2023-09-13T04:38:03.742726+0000","last_active":"2023-09-13T04:39:24.583139+0000","last_peered":"2023-09-13T04:39:24.583139+0000","last_clean":"2023-09-13T04:39:24.583139+0000","last_became_active":"2023-09-13T04:38:03.742293+0000","last_became_peered":"2023-09-13T04:38:03.742293+0000","last_unstale":"2023-09-13T04:39:24.583139+0000","last_undegraded":"2023-09-13T04:39:24.583139+0000","last_fullsized":"2023-09-13T04:39:24.583139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:52:15.089764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.753734+0000","last_change":"2023-09-13T04:37:58.584923+0000","last_active":"2023-09-13T04:39:24.753734+0000","last_peered":"2023-09-13T04:39:24.753734+0000","last_clean":"2023-09-13T04:39:24.753734+0000","last_became_active":"2023-09-13T04:37:58.584320+0000","last_became_peered":"2023-09-13T04:37:58.584320+0000","last_unstale":"2023-09-13T04:39:24.753734+0000","last_undegraded":"2023-09-13T04:39:24.753734+0000","last_fullsized":"2023-09-13T04:39:24.753734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:16:53.670534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.588148+0000","last_change":"2023-09-13T04:37:59.243029+0000","last_active":"2023-09-13T04:39:25.588148+0000","last_peered":"2023-09-13T04:39:25.588148+0000","last_clean":"2023-09-13T04:39:25.588148+0000","last_became_active":"2023-09-13T04:37:59.242733+0000","last_became_peered":"2023-09-13T04:37:59.242733+0000","last_unstale":"2023-09-13T04:39:25.588148+0000","last_undegraded":"2023-09-13T04:39:25.588148+0000","last_fullsized":"2023-09-13T04:39:25.588148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:48:21.430026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582868+0000","last_change":"2023-09-13T04:38:03.616735+0000","last_active":"2023-09-13T04:39:24.582868+0000","last_peered":"2023-09-13T04:39:24.582868+0000","last_clean":"2023-09-13T04:39:24.582868+0000","last_became_active":"2023-09-13T04:38:03.616275+0000","last_became_peered":"2023-09-13T04:38:03.616275+0000","last_unstale":"2023-09-13T04:39:24.582868+0000","last_undegraded":"2023-09-13T04:39:24.582868+0000","last_fullsized":"2023-09-13T04:39:24.582868+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:12:27.267918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.855026+0000","last_change":"2023-09-13T04:37:58.595040+0000","last_active":"2023-09-13T04:39:25.855026+0000","last_peered":"2023-09-13T04:39:25.855026+0000","last_clean":"2023-09-13T04:39:25.855026+0000","last_became_active":"2023-09-13T04:37:58.591403+0000","last_became_peered":"2023-09-13T04:37:58.591403+0000","last_unstale":"2023-09-13T04:39:25.855026+0000","last_undegraded":"2023-09-13T04:39:25.855026+0000","last_fullsized":"2023-09-13T04:39:25.855026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:42:06.561066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.623940+0000","last_change":"2023-09-13T04:37:59.248837+0000","last_active":"2023-09-13T04:39:25.623940+0000","last_peered":"2023-09-13T04:39:25.623940+0000","last_clean":"2023-09-13T04:39:25.623940+0000","last_became_active":"2023-09-13T04:37:59.248573+0000","last_became_peered":"2023-09-13T04:37:59.248573+0000","last_unstale":"2023-09-13T04:39:25.623940+0000","last_undegraded":"2023-09-13T04:39:25.623940+0000","last_fullsized":"2023-09-13T04:39:25.623940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:56:22.980484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583094+0000","last_change":"2023-09-13T04:38:03.616370+0000","last_active":"2023-09-13T04:39:24.583094+0000","last_peered":"2023-09-13T04:39:24.583094+0000","last_clean":"2023-09-13T04:39:24.583094+0000","last_became_active":"2023-09-13T04:38:03.615996+0000","last_became_peered":"2023-09-13T04:38:03.615996+0000","last_unstale":"2023-09-13T04:39:24.583094+0000","last_undegraded":"2023-09-13T04:39:24.583094+0000","last_fullsized":"2023-09-13T04:39:24.583094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:56:59.924205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582878+0000","last_change":"2023-09-13T04:37:58.597902+0000","last_active":"2023-09-13T04:39:24.582878+0000","last_peered":"2023-09-13T04:39:24.582878+0000","last_clean":"2023-09-13T04:39:24.582878+0000","last_became_active":"2023-09-13T04:37:58.597578+0000","last_became_peered":"2023-09-13T04:37:58.597578+0000","last_unstale":"2023-09-13T04:39:24.582878+0000","last_undegraded":"2023-09-13T04:39:24.582878+0000","last_fullsized":"2023-09-13T04:39:24.582878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:10:09.931949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755492+0000","last_change":"2023-09-13T04:37:59.248924+0000","last_active":"2023-09-13T04:39:24.755492+0000","last_peered":"2023-09-13T04:39:24.755492+0000","last_clean":"2023-09-13T04:39:24.755492+0000","last_became_active":"2023-09-13T04:37:59.248483+0000","last_became_peered":"2023-09-13T04:37:59.248483+0000","last_unstale":"2023-09-13T04:39:24.755492+0000","last_undegraded":"2023-09-13T04:39:24.755492+0000","last_fullsized":"2023-09-13T04:39:24.755492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:49:56.078544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583128+0000","last_change":"2023-09-13T04:38:03.742260+0000","last_active":"2023-09-13T04:39:24.583128+0000","last_peered":"2023-09-13T04:39:24.583128+0000","last_clean":"2023-09-13T04:39:24.583128+0000","last_became_active":"2023-09-13T04:38:03.741893+0000","last_became_peered":"2023-09-13T04:38:03.741893+0000","last_unstale":"2023-09-13T04:39:24.583128+0000","last_undegraded":"2023-09-13T04:39:24.583128+0000","last_fullsized":"2023-09-13T04:39:24.583128+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:59:00.653545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755011+0000","last_change":"2023-09-13T04:37:58.586898+0000","last_active":"2023-09-13T04:39:24.755011+0000","last_peered":"2023-09-13T04:39:24.755011+0000","last_clean":"2023-09-13T04:39:24.755011+0000","last_became_active":"2023-09-13T04:37:58.586594+0000","last_became_peered":"2023-09-13T04:37:58.586594+0000","last_unstale":"2023-09-13T04:39:24.755011+0000","last_undegraded":"2023-09-13T04:39:24.755011+0000","last_fullsized":"2023-09-13T04:39:24.755011+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:29:43.718522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.595628+0000","last_change":"2023-09-13T04:37:59.244749+0000","last_active":"2023-09-13T04:39:25.595628+0000","last_peered":"2023-09-13T04:39:25.595628+0000","last_clean":"2023-09-13T04:39:25.595628+0000","last_became_active":"2023-09-13T04:37:59.244391+0000","last_became_peered":"2023-09-13T04:37:59.244391+0000","last_unstale":"2023-09-13T04:39:25.595628+0000","last_undegraded":"2023-09-13T04:39:25.595628+0000","last_fullsized":"2023-09-13T04:39:25.595628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:33:02.195729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.753833+0000","last_change":"2023-09-13T04:38:03.609766+0000","last_active":"2023-09-13T04:39:24.753833+0000","last_peered":"2023-09-13T04:39:24.753833+0000","last_clean":"2023-09-13T04:39:24.753833+0000","last_became_active":"2023-09-13T04:38:03.609432+0000","last_became_peered":"2023-09-13T04:38:03.609432+0000","last_unstale":"2023-09-13T04:39:24.753833+0000","last_undegraded":"2023-09-13T04:39:24.753833+0000","last_fullsized":"2023-09-13T04:39:24.753833+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:30:56.115483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583249+0000","last_change":"2023-09-13T04:37:58.589119+0000","last_active":"2023-09-13T04:39:24.583249+0000","last_peered":"2023-09-13T04:39:24.583249+0000","last_clean":"2023-09-13T04:39:24.583249+0000","last_became_active":"2023-09-13T04:37:58.588803+0000","last_became_peered":"2023-09-13T04:37:58.588803+0000","last_unstale":"2023-09-13T04:39:24.583249+0000","last_undegraded":"2023-09-13T04:39:24.583249+0000","last_fullsized":"2023-09-13T04:39:24.583249+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:51:20.888666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.585669+0000","last_change":"2023-09-13T04:37:59.243506+0000","last_active":"2023-09-13T04:39:24.585669+0000","last_peered":"2023-09-13T04:39:24.585669+0000","last_clean":"2023-09-13T04:39:24.585669+0000","last_became_active":"2023-09-13T04:37:59.243082+0000","last_became_peered":"2023-09-13T04:37:59.243082+0000","last_unstale":"2023-09-13T04:39:24.585669+0000","last_undegraded":"2023-09-13T04:39:24.585669+0000","last_fullsized":"2023-09-13T04:39:24.585669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:01:41.179286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754658+0000","last_change":"2023-09-13T04:38:03.609042+0000","last_active":"2023-09-13T04:39:24.754658+0000","last_peered":"2023-09-13T04:39:24.754658+0000","last_clean":"2023-09-13T04:39:24.754658+0000","last_became_active":"2023-09-13T04:38:03.608553+0000","last_became_peered":"2023-09-13T04:38:03.608553+0000","last_unstale":"2023-09-13T04:39:24.754658+0000","last_undegraded":"2023-09-13T04:39:24.754658+0000","last_fullsized":"2023-09-13T04:39:24.754658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:05:56.648475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.706558+0000","last_change":"2023-09-13T04:37:58.598056+0000","last_active":"2023-09-13T04:39:24.706558+0000","last_peered":"2023-09-13T04:39:24.706558+0000","last_clean":"2023-09-13T04:39:24.706558+0000","last_became_active":"2023-09-13T04:37:58.597724+0000","last_became_peered":"2023-09-13T04:37:58.597724+0000","last_unstale":"2023-09-13T04:39:24.706558+0000","last_undegraded":"2023-09-13T04:39:24.706558+0000","last_fullsized":"2023-09-13T04:39:24.706558+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:43:08.296866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.585419+0000","last_change":"2023-09-13T04:37:59.246505+0000","last_active":"2023-09-13T04:39:24.585419+0000","last_peered":"2023-09-13T04:39:24.585419+0000","last_clean":"2023-09-13T04:39:24.585419+0000","last_became_active":"2023-09-13T04:37:59.246024+0000","last_became_peered":"2023-09-13T04:37:59.246024+0000","last_unstale":"2023-09-13T04:39:24.585419+0000","last_undegraded":"2023-09-13T04:39:24.585419+0000","last_fullsized":"2023-09-13T04:39:24.585419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:36:58.008066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.705906+0000","last_change":"2023-09-13T04:38:03.605354+0000","last_active":"2023-09-13T04:39:24.705906+0000","last_peered":"2023-09-13T04:39:24.705906+0000","last_clean":"2023-09-13T04:39:24.705906+0000","last_became_active":"2023-09-13T04:38:03.604840+0000","last_became_peered":"2023-09-13T04:38:03.604840+0000","last_unstale":"2023-09-13T04:39:24.705906+0000","last_undegraded":"2023-09-13T04:39:24.705906+0000","last_fullsized":"2023-09-13T04:39:24.705906+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:46:29.867914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.587864+0000","last_change":"2023-09-13T04:37:58.589052+0000","last_active":"2023-09-13T04:39:24.587864+0000","last_peered":"2023-09-13T04:39:24.587864+0000","last_clean":"2023-09-13T04:39:24.587864+0000","last_became_active":"2023-09-13T04:37:58.588707+0000","last_became_peered":"2023-09-13T04:37:58.588707+0000","last_unstale":"2023-09-13T04:39:24.587864+0000","last_undegraded":"2023-09-13T04:39:24.587864+0000","last_fullsized":"2023-09-13T04:39:24.587864+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:25:23.966046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.585734+0000","last_change":"2023-09-13T04:37:59.241157+0000","last_active":"2023-09-13T04:39:24.585734+0000","last_peered":"2023-09-13T04:39:24.585734+0000","last_clean":"2023-09-13T04:39:24.585734+0000","last_became_active":"2023-09-13T04:37:59.240857+0000","last_became_peered":"2023-09-13T04:37:59.240857+0000","last_unstale":"2023-09-13T04:39:24.585734+0000","last_undegraded":"2023-09-13T04:39:24.585734+0000","last_fullsized":"2023-09-13T04:39:24.585734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:04:44.986908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.623983+0000","last_change":"2023-09-13T04:37:59.242447+0000","last_active":"2023-09-13T04:39:25.623983+0000","last_peered":"2023-09-13T04:39:25.623983+0000","last_clean":"2023-09-13T04:39:25.623983+0000","last_became_active":"2023-09-13T04:37:59.242130+0000","last_became_peered":"2023-09-13T04:37:59.242130+0000","last_unstale":"2023-09-13T04:39:25.623983+0000","last_undegraded":"2023-09-13T04:39:25.623983+0000","last_fullsized":"2023-09-13T04:39:25.623983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:12:12.844888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.854823+0000","last_change":"2023-09-13T04:38:24.595646+0000","last_active":"2023-09-13T04:39:25.854823+0000","last_peered":"2023-09-13T04:39:25.854823+0000","last_clean":"2023-09-13T04:39:25.854823+0000","last_became_active":"2023-09-13T04:38:24.595295+0000","last_became_peered":"2023-09-13T04:38:24.595295+0000","last_unstale":"2023-09-13T04:39:25.854823+0000","last_undegraded":"2023-09-13T04:39:25.854823+0000","last_fullsized":"2023-09-13T04:39:25.854823+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:24:27.174979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.854796+0000","last_change":"2023-09-13T04:37:59.245358+0000","last_active":"2023-09-13T04:39:25.854796+0000","last_peered":"2023-09-13T04:39:25.854796+0000","last_clean":"2023-09-13T04:39:25.854796+0000","last_became_active":"2023-09-13T04:37:59.245052+0000","last_became_peered":"2023-09-13T04:37:59.245052+0000","last_unstale":"2023-09-13T04:39:25.854796+0000","last_undegraded":"2023-09-13T04:39:25.854796+0000","last_fullsized":"2023-09-13T04:39:25.854796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:05:42.561716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.855825+0000","last_change":"2023-09-13T04:37:59.254053+0000","last_active":"2023-09-13T04:39:25.855825+0000","last_peered":"2023-09-13T04:39:25.855825+0000","last_clean":"2023-09-13T04:39:25.855825+0000","last_became_active":"2023-09-13T04:37:59.253544+0000","last_became_peered":"2023-09-13T04:37:59.253544+0000","last_unstale":"2023-09-13T04:39:25.855825+0000","last_undegraded":"2023-09-13T04:39:25.855825+0000","last_fullsized":"2023-09-13T04:39:25.855825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:18:54.689208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.706160+0000","last_change":"2023-09-13T04:37:59.253403+0000","last_active":"2023-09-13T04:39:25.706160+0000","last_peered":"2023-09-13T04:39:25.706160+0000","last_clean":"2023-09-13T04:39:25.706160+0000","last_became_active":"2023-09-13T04:37:59.253062+0000","last_became_peered":"2023-09-13T04:37:59.253062+0000","last_unstale":"2023-09-13T04:39:25.706160+0000","last_undegraded":"2023-09-13T04:39:25.706160+0000","last_fullsized":"2023-09-13T04:39:25.706160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:31:09.898886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.624253+0000","last_change":"2023-09-13T04:37:59.240916+0000","last_active":"2023-09-13T04:39:25.624253+0000","last_peered":"2023-09-13T04:39:25.624253+0000","last_clean":"2023-09-13T04:39:25.624253+0000","last_became_active":"2023-09-13T04:37:59.240605+0000","last_became_peered":"2023-09-13T04:37:59.240605+0000","last_unstale":"2023-09-13T04:39:25.624253+0000","last_undegraded":"2023-09-13T04:39:25.624253+0000","last_fullsized":"2023-09-13T04:39:25.624253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:15:48.241587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583359+0000","last_change":"2023-09-13T04:37:59.238569+0000","last_active":"2023-09-13T04:39:24.583359+0000","last_peered":"2023-09-13T04:39:24.583359+0000","last_clean":"2023-09-13T04:39:24.583359+0000","last_became_active":"2023-09-13T04:37:59.238255+0000","last_became_peered":"2023-09-13T04:37:59.238255+0000","last_unstale":"2023-09-13T04:39:24.583359+0000","last_undegraded":"2023-09-13T04:39:24.583359+0000","last_fullsized":"2023-09-13T04:39:24.583359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:19:03.433404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755665+0000","last_change":"2023-09-13T04:37:59.252243+0000","last_active":"2023-09-13T04:39:24.755665+0000","last_peered":"2023-09-13T04:39:24.755665+0000","last_clean":"2023-09-13T04:39:24.755665+0000","last_became_active":"2023-09-13T04:37:59.251902+0000","last_became_peered":"2023-09-13T04:37:59.251902+0000","last_unstale":"2023-09-13T04:39:24.755665+0000","last_undegraded":"2023-09-13T04:39:24.755665+0000","last_fullsized":"2023-09-13T04:39:24.755665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:35:57.395627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.709975+0000","last_change":"2023-09-13T04:37:59.245847+0000","last_active":"2023-09-13T04:39:25.709975+0000","last_peered":"2023-09-13T04:39:25.709975+0000","last_clean":"2023-09-13T04:39:25.709975+0000","last_became_active":"2023-09-13T04:37:59.245542+0000","last_became_peered":"2023-09-13T04:37:59.245542+0000","last_unstale":"2023-09-13T04:39:25.709975+0000","last_undegraded":"2023-09-13T04:39:25.709975+0000","last_fullsized":"2023-09-13T04:39:25.709975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:24:16.813333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754898+0000","last_change":"2023-09-13T04:37:59.246963+0000","last_active":"2023-09-13T04:39:24.754898+0000","last_peered":"2023-09-13T04:39:24.754898+0000","last_clean":"2023-09-13T04:39:24.754898+0000","last_became_active":"2023-09-13T04:37:59.246448+0000","last_became_peered":"2023-09-13T04:37:59.246448+0000","last_unstale":"2023-09-13T04:39:24.754898+0000","last_undegraded":"2023-09-13T04:39:24.754898+0000","last_fullsized":"2023-09-13T04:39:24.754898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:25:45.418778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.753993+0000","last_change":"2023-09-13T04:37:59.245399+0000","last_active":"2023-09-13T04:39:24.753993+0000","last_peered":"2023-09-13T04:39:24.753993+0000","last_clean":"2023-09-13T04:39:24.753993+0000","last_became_active":"2023-09-13T04:37:59.245060+0000","last_became_peered":"2023-09-13T04:37:59.245060+0000","last_unstale":"2023-09-13T04:39:24.753993+0000","last_undegraded":"2023-09-13T04:39:24.753993+0000","last_fullsized":"2023-09-13T04:39:24.753993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:25:41.078238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.587934+0000","last_change":"2023-09-13T04:37:59.244776+0000","last_active":"2023-09-13T04:39:24.587934+0000","last_peered":"2023-09-13T04:39:24.587934+0000","last_clean":"2023-09-13T04:39:24.587934+0000","last_became_active":"2023-09-13T04:37:59.244409+0000","last_became_peered":"2023-09-13T04:37:59.244409+0000","last_unstale":"2023-09-13T04:39:24.587934+0000","last_undegraded":"2023-09-13T04:39:24.587934+0000","last_fullsized":"2023-09-13T04:39:24.587934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:04:02.561414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754806+0000","last_change":"2023-09-13T04:37:59.248153+0000","last_active":"2023-09-13T04:39:24.754806+0000","last_peered":"2023-09-13T04:39:24.754806+0000","last_clean":"2023-09-13T04:39:24.754806+0000","last_became_active":"2023-09-13T04:37:59.247840+0000","last_became_peered":"2023-09-13T04:37:59.247840+0000","last_unstale":"2023-09-13T04:39:24.754806+0000","last_undegraded":"2023-09-13T04:39:24.754806+0000","last_fullsized":"2023-09-13T04:39:24.754806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:52:53.021219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7710,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":646,"num_read_kb":337,"num_write":131,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":22872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416423,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":509914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.625,"15min":0.61599999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":2.8420000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.54600000000000004}]},{"osd":1,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.66800000000000004,"15min":0.66100000000000003},"min":{"1min":0.46999999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92700000000000005,"5min":1.024,"15min":1.024},"last":0.625},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.42099999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80600000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.47299999999999998,"15min":0.46899999999999997},"min":{"1min":0.34799999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.68400000000000005}]},{"osd":3,"last update":"Wed Sep 13 04:39:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.74399999999999999}]},{"osd":4,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.68600000000000005,"15min":0.67800000000000005},"min":{"1min":0.53100000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.42899999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86699999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.71099999999999997}]},{"osd":5,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.64700000000000002,"15min":0.63100000000000001},"min":{"1min":0.45300000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.84799999999999998}]},{"osd":6,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70599999999999996,"15min":0.70399999999999996},"min":{"1min":0.46999999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.70799999999999996,"15min":0.69799999999999995},"min":{"1min":0.55900000000000005,"5min":0.498,"15min":0.498},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.878}]},{"osd":7,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.77500000000000002,"15min":0.76500000000000001},"min":{"1min":0.59599999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0229999999999999,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.50800000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94099999999999995,"5min":1.262,"15min":1.262},"last":0.83899999999999997}]},{"osd":8,"last update":"Wed Sep 13 04:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.74199999999999999,"15min":0.73899999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.88200000000000001,"15min":0.88600000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.0840000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.89400000000000002}]},{"osd":9,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.77400000000000002,"15min":0.76200000000000001},"min":{"1min":0.61499999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.121,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.995},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.82499999999999996,"15min":0.81100000000000005},"min":{"1min":0.66200000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1830000000000001,"5min":1.361,"15min":1.361},"last":0.97599999999999998}]},{"osd":10,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.78900000000000003,"15min":0.78600000000000003},"min":{"1min":0.56000000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.002,"5min":1.155,"15min":1.155},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.77300000000000002,"15min":0.76700000000000002},"min":{"1min":0.59599999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95699999999999996,"5min":1.282,"15min":1.282},"last":0.95799999999999996}]}]},{"osd":10,"up_from":71,"seq":304942678059,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1104951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.74399999999999999,"15min":0.76800000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.84499999999999997,"5min":4.0510000000000002,"15min":4.0510000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.77000000000000002,"15min":0.80300000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82899999999999996,"5min":4.1120000000000001,"15min":4.1120000000000001},"last":0.70899999999999996}]},{"osd":1,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.46000000000000002,"15min":0.45600000000000002},"min":{"1min":0.35399999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72799999999999998,"5min":0.751,"15min":0.751},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.47499999999999998,"15min":0.46999999999999997},"min":{"1min":0.34999999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63500000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.58399999999999996}]},{"osd":2,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.626,"15min":0.628},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.755,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80900000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.81200000000000006}]},{"osd":3,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.69399999999999995,"15min":0.66800000000000004},"min":{"1min":0.41799999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.754,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.73199999999999998}]},{"osd":4,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.371,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76300000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.879,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.62}]},{"osd":5,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69799999999999995,"15min":0.69299999999999995},"min":{"1min":0.505,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84999999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.45900000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.75}]},{"osd":6,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.81699999999999995,"15min":0.86799999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90100000000000002,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.85999999999999999,"15min":0.82599999999999996},"min":{"1min":0.54400000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96599999999999997,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":0.78800000000000003}]},{"osd":7,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.64500000000000002}]},{"osd":8,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77000000000000002},"min":{"1min":0.61199999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":0.998,"15min":0.998},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.78300000000000003,"15min":0.80100000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.99299999999999999,"5min":1.03,"15min":1.03},"last":0.94899999999999995}]},{"osd":9,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.86699999999999999,"15min":0.85099999999999998},"min":{"1min":0.65400000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0640000000000001,"5min":4.2679999999999998,"15min":4.2679999999999998},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.84999999999999998,"15min":0.85099999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.96799999999999997,"5min":4.3600000000000003,"15min":4.3600000000000003},"last":0.80000000000000004}]},{"osd":11,"last update":"Wed Sep 13 04:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.82799999999999996,"15min":0.82999999999999996},"min":{"1min":0.64100000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.80700000000000005,"15min":0.80600000000000005},"min":{"1min":0.65300000000000002,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.97099999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.92000000000000004}]}]},{"osd":9,"up_from":66,"seq":283467841583,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":510004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.58899999999999997,"15min":0.58699999999999997},"min":{"1min":0.38,"5min":0.316,"15min":0.316},"max":{"1min":1.5129999999999999,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.54400000000000004}]},{"osd":1,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.69499999999999995,"15min":0.70699999999999996},"min":{"1min":0.48199999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.036,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71499999999999997,"15min":0.73099999999999998},"min":{"1min":0.46100000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.70499999999999996}]},{"osd":2,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73599999999999999,"15min":0.752},"min":{"1min":0.48199999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.026,"5min":1.083,"15min":1.083},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73899999999999999,"15min":0.751},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.65500000000000003}]},{"osd":3,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68000000000000005,"15min":0.67900000000000005},"min":{"1min":0.42299999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.5549999999999999,"5min":2.258,"15min":2.258},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.5289999999999999,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":0.59399999999999997}]},{"osd":4,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.78100000000000003,"15min":0.80300000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.79600000000000004,"15min":0.80900000000000005},"min":{"1min":0.57299999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.68999999999999995}]},{"osd":5,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84399999999999997,"15min":0.86399999999999999},"min":{"1min":0.57199999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.296,"5min":1.339,"15min":1.339},"last":0.75},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79900000000000004,"15min":0.81299999999999994},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.071,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.72499999999999998}]},{"osd":6,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.72099999999999997}]},{"osd":7,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.88,"15min":0.90700000000000003},"min":{"1min":0.57099999999999995,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.876},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.88200000000000001,"15min":0.91800000000000004},"min":{"1min":0.621,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1060000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.81399999999999995}]},{"osd":8,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.90100000000000002,"15min":0.93500000000000005},"min":{"1min":0.71399999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.878,"5min":0.93500000000000005,"15min":0.96399999999999997},"min":{"1min":0.57999999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3380000000000001,"5min":1.857,"15min":1.857},"last":0.92100000000000004}]},{"osd":10,"last update":"Wed Sep 13 04:39:56 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89200000000000002,"15min":0.89800000000000002},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.879,"5min":0.89200000000000002,"15min":0.91000000000000003},"min":{"1min":0.70799999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1639999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.85399999999999998}]},{"osd":11,"last update":"Wed Sep 13 04:40:06 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.90700000000000003,"15min":0.91100000000000003},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.0249999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.86799999999999999,"15min":0.86399999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.89600000000000002}]}]},{"osd":8,"up_from":58,"seq":249108103220,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":509481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.441,"5min":0.317,"15min":0.317},"max":{"1min":0.89000000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58799999999999997,"15min":0.57099999999999995},"min":{"1min":0.39600000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98499999999999999,"5min":1.016,"15min":1.016},"last":0.42199999999999999}]},{"osd":1,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.59999999999999998,"15min":0.55400000000000005},"min":{"1min":0.53100000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.79700000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.752},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.61899999999999999,"15min":0.59199999999999997},"min":{"1min":0.53400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82799999999999996,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.63600000000000001}]},{"osd":2,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48099999999999998,"15min":0.47499999999999998},"min":{"1min":0.40200000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76300000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.439},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.49199999999999999,"15min":0.48699999999999999},"min":{"1min":0.373,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.73099999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.38}]},{"osd":3,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.90600000000000003,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.65500000000000003,"15min":0.64600000000000002},"min":{"1min":0.53600000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.57199999999999995}]},{"osd":4,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.63100000000000001,"15min":0.59799999999999998},"min":{"1min":0.497,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.90100000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.65900000000000003,"15min":0.627},"min":{"1min":0.55400000000000005,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.71499999999999997}]},{"osd":5,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58799999999999997,"15min":0.58199999999999996},"min":{"1min":0.437,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.89800000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.61199999999999999,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.54100000000000004}]},{"osd":6,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.56999999999999995,"5min":0.504,"15min":0.504},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.64900000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0229999999999999,"5min":1.077,"15min":1.077},"last":0.58999999999999997}]},{"osd":7,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72999999999999998,"15min":0.71599999999999997},"min":{"1min":0.47999999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.97199999999999998,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.73899999999999999,"15min":0.70299999999999996},"min":{"1min":0.66300000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0309999999999999,"5min":1.04,"15min":1.04},"last":0.76500000000000001}]},{"osd":9,"last update":"Wed Sep 13 04:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.78100000000000003,"15min":0.76500000000000001},"min":{"1min":0.66700000000000004,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0640000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.78500000000000003,"15min":0.76200000000000001},"min":{"1min":0.68500000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.62}]},{"osd":10,"last update":"Wed Sep 13 04:39:58 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.77200000000000002,"15min":0.74399999999999999},"min":{"1min":0.59499999999999997,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0089999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.78200000000000003,"15min":0.76300000000000001},"min":{"1min":0.64000000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.80600000000000005}]},{"osd":11,"last update":"Wed Sep 13 04:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.63800000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.87,"5min":0.83199999999999996,"15min":0.82599999999999996},"min":{"1min":0.66500000000000004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0329999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.73199999999999998}]}]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1642496,"data_stored":1101346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.46500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.746,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60499999999999998,"15min":0.60799999999999998},"min":{"1min":0.41599999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.40200000000000002}]},{"osd":1,"last update":"Wed Sep 13 04:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.433,"15min":0.41099999999999998},"min":{"1min":0.33900000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.56200000000000006,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.498,"5min":0.44800000000000001,"15min":0.42899999999999999},"min":{"1min":0.316,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.66400000000000003}]},{"osd":2,"last update":"Wed Sep 13 04:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60799999999999998,"15min":0.59299999999999997},"min":{"1min":0.46999999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78700000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61499999999999999,"15min":0.58299999999999996},"min":{"1min":0.47299999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76400000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.80300000000000005}]},{"osd":3,"last update":"Wed Sep 13 04:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71099999999999997,"15min":0.65700000000000003},"min":{"1min":0.496,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7410000000000001,"5min":7.0789999999999997,"15min":7.0789999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68000000000000005,"15min":0.67400000000000004},"min":{"1min":0.42999999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.54800000000000004}]},{"osd":4,"last update":"Wed Sep 13 04:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52100000000000002,"15min":0.51300000000000001},"min":{"1min":0.39100000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71799999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50600000000000001,"15min":0.51300000000000001},"min":{"1min":0.36099999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.78600000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.82299999999999995}]},{"osd":5,"last update":"Wed Sep 13 04:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64300000000000002,"15min":0.63100000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.80700000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.67900000000000005,"15min":0.65300000000000002},"min":{"1min":0.52200000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89600000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.89300000000000002}]},{"osd":6,"last update":"Wed Sep 13 04:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.751,"15min":0.73099999999999998},"min":{"1min":0.53300000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":3.496,"5min":3.496,"15min":3.496},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72799999999999998,"15min":0.72299999999999998},"min":{"1min":0.55600000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.7949999999999999,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.63500000000000001}]},{"osd":8,"last update":"Wed Sep 13 04:39:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.71699999999999997,"15min":0.70899999999999996},"min":{"1min":0.59399999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72099999999999997,"15min":0.71699999999999997},"min":{"1min":0.56599999999999995,"5min":0.437,"15min":0.437},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.85299999999999998}]},{"osd":9,"last update":"Wed Sep 13 04:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.79600000000000004,"15min":0.77500000000000002},"min":{"1min":0.64600000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.77900000000000003,"15min":0.76500000000000001},"min":{"1min":0.63900000000000001,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.91000000000000003}]},{"osd":10,"last update":"Wed Sep 13 04:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69899999999999995,"15min":0.69799999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.97099999999999997,"5min":0.999,"15min":0.999},"last":0.876},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.73599999999999999,"15min":0.72299999999999998},"min":{"1min":0.54300000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98399999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.95499999999999996}]},{"osd":11,"last update":"Wed Sep 13 04:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.80800000000000005,"15min":0.81000000000000005},"min":{"1min":0.58299999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92700000000000005,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.86599999999999999,"15min":0.86699999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.97699999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":509276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.51200000000000001,"15min":0.46800000000000003},"min":{"1min":0.33600000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":2.738,"5min":2.738,"15min":2.738},"last":0.45600000000000002},{"interface":"front","average":{"1min":1.0920000000000001,"5min":0.625,"15min":0.53100000000000003},"min":{"1min":0.41599999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":8.5749999999999993,"5min":8.5749999999999993,"15min":8.5749999999999993},"last":0.504}]},{"osd":1,"last update":"Wed Sep 13 04:39:20 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":0.74099999999999999,"15min":0.64500000000000002},"min":{"1min":0.52700000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":10.374000000000001,"5min":10.374000000000001,"15min":10.374000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.72999999999999998,"15min":0.68600000000000005},"min":{"1min":0.51500000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.746}]},{"osd":2,"last update":"Wed Sep 13 04:39:20 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":0.78100000000000003,"15min":0.71699999999999997},"min":{"1min":0.505,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":8.4740000000000002,"5min":8.4740000000000002,"15min":8.4740000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.69799999999999995,"15min":0.68400000000000005},"min":{"1min":0.55900000000000005,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.52600000000000002}]},{"osd":3,"last update":"Wed Sep 13 04:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.69899999999999995,"15min":0.65400000000000003},"min":{"1min":0.46000000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.331,"5min":2.331,"15min":2.331},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.67800000000000005,"15min":0.63500000000000001},"min":{"1min":0.45000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.585,"5min":2.585,"15min":2.585},"last":0.48099999999999998}]},{"osd":4,"last update":"Wed Sep 13 04:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.78000000000000003,"15min":0.752},"min":{"1min":0.60699999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.2809999999999999,"5min":0.85999999999999999,"15min":0.80400000000000005},"min":{"1min":0.58199999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":8.3350000000000009,"5min":8.3350000000000009,"15min":8.3350000000000009},"last":0.85199999999999998}]},{"osd":5,"last update":"Wed Sep 13 04:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.82699999999999996,"15min":0.79900000000000004},"min":{"1min":0.52500000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.288,"5min":0.88600000000000001,"15min":0.83999999999999997},"min":{"1min":0.54100000000000004,"5min":0.379,"15min":0.379},"max":{"1min":8.5060000000000002,"5min":8.5060000000000002,"15min":8.5060000000000002},"last":0.79300000000000004}]},{"osd":7,"last update":"Wed Sep 13 04:39:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.87,"15min":0.83399999999999996},"min":{"1min":0.61199999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90300000000000002,"15min":0.82299999999999995},"min":{"1min":0.59599999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.702,"5min":8.3970000000000002,"15min":8.3970000000000002},"last":0.89000000000000001}]},{"osd":8,"last update":"Wed Sep 13 04:40:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.80900000000000005,"15min":0.82099999999999995},"min":{"1min":0.625,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.256,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.92200000000000004,"15min":0.92500000000000004},"min":{"1min":0.64700000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3620000000000001,"5min":1.446,"15min":1.446},"last":0.871}]},{"osd":9,"last update":"Wed Sep 13 04:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.876,"15min":0.88600000000000001},"min":{"1min":0.58299999999999996,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.026,"15min":1.0329999999999999},"min":{"1min":0.72399999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.6960000000000002,"5min":2.6960000000000002,"15min":2.6960000000000002},"last":0.94399999999999995}]},{"osd":10,"last update":"Wed Sep 13 04:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.98999999999999999,"15min":0.99199999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.96699999999999997,"15min":0.89900000000000002},"min":{"1min":0.71299999999999997,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.732,"5min":8.4380000000000006,"15min":8.4380000000000006},"last":0.92600000000000005}]},{"osd":11,"last update":"Wed Sep 13 04:40:06 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":1.272,"15min":1.3380000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.4139999999999999,"5min":8.5410000000000004,"15min":8.5410000000000004},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.93200000000000005,"15min":0.94399999999999995},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.3080000000000001,"5min":1.341,"15min":1.341},"last":0.83699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":513028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.621,"15min":0.55900000000000005},"min":{"1min":0.47199999999999998,"5min":0.307,"15min":0.307},"max":{"1min":4.1150000000000002,"5min":4.1150000000000002,"15min":4.1150000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":1.165,"5min":0.71499999999999997,"15min":0.60099999999999998},"min":{"1min":0.45100000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":12.896000000000001,"5min":12.896000000000001,"15min":12.896000000000001},"last":0.42899999999999999}]},{"osd":2,"last update":"Wed Sep 13 04:39:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.60899999999999999,"15min":0.55500000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88300000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61799999999999999,"15min":0.54200000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.877,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.66200000000000003}]},{"osd":3,"last update":"Wed Sep 13 04:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.58999999999999997},"min":{"1min":0.41699999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.64500000000000002,"15min":0.61699999999999999},"min":{"1min":0.50800000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.626}]},{"osd":4,"last update":"Wed Sep 13 04:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48299999999999998,"15min":0.46000000000000002},"min":{"1min":0.378,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63700000000000001,"5min":0.878,"15min":0.878},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.48899999999999999,"15min":0.45900000000000002},"min":{"1min":0.34699999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.82299999999999995,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.51400000000000001}]},{"osd":5,"last update":"Wed Sep 13 04:39:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.625,"15min":0.57999999999999996},"min":{"1min":0.378,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85299999999999998,"5min":1.3,"15min":1.3},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.68400000000000005,"15min":0.60599999999999998},"min":{"1min":0.46400000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.0089999999999999,"5min":1.264,"15min":1.264},"last":0.70999999999999996}]},{"osd":6,"last update":"Wed Sep 13 04:39:22 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":0.86799999999999999,"15min":0.75},"min":{"1min":0.60199999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":12.640000000000001,"5min":12.640000000000001,"15min":12.640000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.3160000000000001,"5min":0.83299999999999996,"15min":0.73999999999999999},"min":{"1min":0.56000000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":12.675000000000001,"5min":12.675000000000001,"15min":12.675000000000001},"last":0.73199999999999998}]},{"osd":7,"last update":"Wed Sep 13 04:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.66000000000000003,"15min":0.60499999999999998},"min":{"1min":0.50700000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.1399999999999999,"5min":1.347,"15min":1.347},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.70999999999999996,"15min":0.66000000000000003},"min":{"1min":0.42699999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.96899999999999997,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.81499999999999995}]},{"osd":8,"last update":"Wed Sep 13 04:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.75700000000000001,"15min":0.74199999999999999},"min":{"1min":0.48099999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0980000000000001,"5min":1.393,"15min":1.393},"last":0.872},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.73999999999999999,"15min":0.72399999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.032,"5min":1.224,"15min":1.224},"last":0.78000000000000003}]},{"osd":9,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.878,"15min":0.86699999999999999},"min":{"1min":0.64700000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.8250000000000002,"5min":2.8250000000000002,"15min":2.8250000000000002},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.879,"15min":0.871},"min":{"1min":0.66600000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":2.9569999999999999,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":0.84799999999999998}]},{"osd":10,"last update":"Wed Sep 13 04:39:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.81799999999999995,"15min":0.82199999999999995},"min":{"1min":0.51500000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.038,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.85399999999999998,"15min":0.85499999999999998},"min":{"1min":0.54700000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0780000000000001,"5min":1.504,"15min":1.504},"last":0.85999999999999999}]},{"osd":11,"last update":"Wed Sep 13 04:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.88800000000000001,"15min":0.88400000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0660000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.91700000000000004,"15min":0.91900000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.125,"5min":1.129,"15min":1.129},"last":0.91900000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":510664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 13 04:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.748,"15min":0.61199999999999999},"min":{"1min":0.45700000000000002,"5min":0.33100000000000002,"15min":0.29899999999999999},"max":{"1min":1.125,"5min":6.2359999999999998,"15min":6.2359999999999998},"last":1.01},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64900000000000002,"15min":0.57299999999999995},"min":{"1min":0.40600000000000003,"5min":0.38800000000000001,"15min":0.31900000000000001},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":1.0329999999999999}]},{"osd":2,"last update":"Wed Sep 13 04:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65900000000000003,"15min":0.61099999999999999},"min":{"1min":0.52300000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96499999999999997,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.70899999999999996,"15min":0.627},"min":{"1min":0.47999999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed Sep 13 04:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.88500000000000001,"5min":1.25,"15min":1.25},"last":0.628},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.90300000000000002,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.89800000000000002}]},{"osd":4,"last update":"Wed Sep 13 04:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.751,"15min":0.64400000000000002},"min":{"1min":0.48399999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.052},{"interface":"front","average":{"1min":0.748,"5min":0.70599999999999996,"15min":0.64000000000000001},"min":{"1min":0.46899999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.85099999999999998}]},{"osd":5,"last update":"Wed Sep 13 04:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69699999999999995,"15min":0.67500000000000004},"min":{"1min":0.48799999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89100000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.878,"5min":0.78100000000000003,"15min":0.70699999999999996},"min":{"1min":0.63500000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.2130000000000001,"5min":1.444,"15min":1.444},"last":1.155}]},{"osd":6,"last update":"Wed Sep 13 04:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.70899999999999996,"15min":0.70199999999999996},"min":{"1min":0.50700000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.751,"15min":0.73299999999999998},"min":{"1min":0.45100000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.65400000000000003}]},{"osd":7,"last update":"Wed Sep 13 04:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.82999999999999996,"15min":0.83699999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1459999999999999,"5min":1.333,"15min":1.333},"last":0.873},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.93500000000000005,"15min":0.94299999999999995},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.258,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.075}]},{"osd":8,"last update":"Wed Sep 13 04:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.91000000000000003,"15min":0.86199999999999999},"min":{"1min":0.61599999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0609999999999999,"5min":6.2089999999999996,"15min":6.2089999999999996},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.93500000000000005,"15min":0.84099999999999997},"min":{"1min":0.73199999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2350000000000001,"5min":6.1699999999999999,"15min":6.1699999999999999},"last":1.135}]},{"osd":9,"last update":"Wed Sep 13 04:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.90800000000000003,"15min":0.94099999999999995},"min":{"1min":0.56799999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.94499999999999995,"15min":0.93700000000000006},"min":{"1min":0.69399999999999995,"5min":0.433,"15min":0.433},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.96399999999999997}]},{"osd":10,"last update":"Wed Sep 13 04:39:51 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.99399999999999999,"15min":0.98599999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2989999999999999,"5min":1.591,"15min":1.591},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":1.0449999999999999,"15min":1.119},"min":{"1min":0.623,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1950000000000001,"5min":6.1859999999999999,"15min":6.1859999999999999},"last":0.93500000000000005}]},{"osd":11,"last update":"Wed Sep 13 04:39:16 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":0.99099999999999999,"15min":0.98399999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.175},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.86099999999999999,"15min":0.85299999999999998},"min":{"1min":0.71499999999999997,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.99299999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":514344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55600000000000005,"15min":0.53000000000000003},"min":{"1min":0.34200000000000003,"5min":0.29399999999999998,"15min":0.28599999999999998},"max":{"1min":0.86099999999999999,"5min":1.74,"15min":1.74},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55400000000000005,"15min":0.53000000000000003},"min":{"1min":0.38700000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.77400000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.53100000000000003}]},{"osd":1,"last update":"Wed Sep 13 04:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60799999999999998,"15min":0.52000000000000002},"min":{"1min":0.42399999999999999,"5min":0.34300000000000003,"15min":0.30599999999999999},"max":{"1min":0.91300000000000003,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.61099999999999999,"15min":0.53000000000000003},"min":{"1min":0.44600000000000001,"5min":0.36599999999999999,"15min":0.34300000000000003},"max":{"1min":0.80800000000000005,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.70199999999999996}]},{"osd":3,"last update":"Wed Sep 13 04:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.61899999999999999,"15min":0.59199999999999997},"min":{"1min":0.45700000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.623,"15min":0.60999999999999999},"min":{"1min":0.48899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.74299999999999999}]},{"osd":4,"last update":"Wed Sep 13 04:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.54500000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.624},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.61099999999999999}]},{"osd":5,"last update":"Wed Sep 13 04:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46999999999999997,"15min":0.49299999999999999},"min":{"1min":0.376,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.621,"5min":1.796,"15min":1.796},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34200000000000003,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.94799999999999995,"5min":1.546,"15min":1.546},"last":0.83299999999999996}]},{"osd":6,"last update":"Wed Sep 13 04:39:19 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.753,"15min":0.71899999999999997},"min":{"1min":0.57699999999999996,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":4.0730000000000004,"5min":4.0730000000000004,"15min":4.0730000000000004},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.56299999999999994,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.65700000000000003}]},{"osd":7,"last update":"Wed Sep 13 04:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.77900000000000003,"15min":0.77100000000000002},"min":{"1min":0.56399999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76800000000000002,"15min":0.76200000000000001},"min":{"1min":0.63400000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.86599999999999999}]},{"osd":8,"last update":"Wed Sep 13 04:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61799999999999999,"15min":0.62},"min":{"1min":0.40300000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.82399999999999995,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.66700000000000004,"15min":0.65300000000000002},"min":{"1min":0.51200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.90400000000000003,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.79600000000000004}]},{"osd":9,"last update":"Wed Sep 13 04:39:31 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.755,"15min":0.73099999999999998},"min":{"1min":0.65200000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.75600000000000001,"15min":0.72599999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":0.78400000000000003}]},{"osd":10,"last update":"Wed Sep 13 04:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71999999999999997,"15min":0.69599999999999995},"min":{"1min":0.63700000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.78300000000000003,"15min":0.745},"min":{"1min":0.65600000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.93000000000000005}]},{"osd":11,"last update":"Wed Sep 13 04:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.76600000000000001,"15min":0.76800000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.94199999999999995,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.877,"15min":0.88300000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.89600000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":509470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.45500000000000002,"15min":0.39900000000000002},"min":{"1min":0.30399999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.52900000000000003,"15min":0.45600000000000002},"min":{"1min":0.34399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.5940000000000003,"5min":4.5940000000000003,"15min":4.5940000000000003},"last":0.44600000000000001}]},{"osd":1,"last update":"Wed Sep 13 04:39:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.64000000000000001,"15min":0.58699999999999997},"min":{"1min":0.42399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.623},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.68000000000000005,"15min":0.58999999999999997},"min":{"1min":0.50600000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.83299999999999996}]},{"osd":2,"last update":"Wed Sep 13 04:39:13 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.72799999999999998,"15min":0.64700000000000002},"min":{"1min":0.47699999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":4.3920000000000003,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.65900000000000003,"15min":0.626},"min":{"1min":0.46100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97999999999999998,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.58899999999999997}]},{"osd":4,"last update":"Wed Sep 13 04:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.70399999999999996,"15min":0.621},"min":{"1min":0.47699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0649999999999999,"5min":1.147,"15min":1.147},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69599999999999995,"15min":0.626},"min":{"1min":0.498,"5min":0.439,"15min":0.439},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.66600000000000004}]},{"osd":5,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72299999999999998,"15min":0.66100000000000003},"min":{"1min":0.52000000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0109999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.077,"5min":0.86399999999999999,"15min":0.71299999999999997},"min":{"1min":0.48399999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":6.7030000000000003,"5min":6.7030000000000003,"15min":6.7030000000000003},"last":0.60099999999999998}]},{"osd":6,"last update":"Wed Sep 13 04:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.79300000000000004,"15min":0.72099999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":4.5529999999999999,"5min":4.5529999999999999,"15min":4.5529999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.78000000000000003,"15min":0.749},"min":{"1min":0.52200000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.78500000000000003}]},{"osd":7,"last update":"Wed Sep 13 04:39:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.84899999999999998,"15min":0.85899999999999999},"min":{"1min":0.56899999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.876},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.88,"15min":0.85799999999999998},"min":{"1min":0.59899999999999998,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.153,"5min":4.4699999999999998,"15min":4.4699999999999998},"last":0.67700000000000005}]},{"osd":8,"last update":"Wed Sep 13 04:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.875}]},{"osd":9,"last update":"Wed Sep 13 04:39:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":0.96299999999999997,"15min":0.96899999999999997},"min":{"1min":0.58799999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":4.4329999999999998,"5min":4.4329999999999998,"15min":4.4329999999999998},"last":0.753},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.82499999999999996,"15min":0.84299999999999997},"min":{"1min":0.51200000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.51000000000000001}]},{"osd":10,"last update":"Wed Sep 13 04:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.89100000000000001,"15min":0.89600000000000002},"min":{"1min":0.71599999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1499999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.96699999999999997,"15min":0.95899999999999996},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.2370000000000001,"5min":4.5099999999999998,"15min":4.5099999999999998},"last":0.72099999999999997}]},{"osd":11,"last update":"Wed Sep 13 04:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.879,"15min":0.88900000000000001},"min":{"1min":0.71999999999999997,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.95699999999999996,"5min":1.27,"15min":1.27},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.0620000000000001,"5min":0.98199999999999998,"15min":0.96899999999999997},"min":{"1min":0.80000000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":4.2530000000000001,"5min":4.2530000000000001,"15min":4.2530000000000001},"last":0.70699999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":509830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.56599999999999995,"15min":0.54400000000000004},"min":{"1min":0.43099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.753},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76500000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.68999999999999995}]},{"osd":1,"last update":"Wed Sep 13 04:39:42 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.41899999999999998,"15min":0.36199999999999999},"min":{"1min":0.34999999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.58099999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.433},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.434,"15min":0.38},"min":{"1min":0.36899999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.71499999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.48199999999999998}]},{"osd":2,"last update":"Wed Sep 13 04:39:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.61099999999999999,"15min":0.57099999999999995},"min":{"1min":0.39700000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81299999999999994,"5min":1.631,"15min":1.631},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.61299999999999999,"15min":0.55300000000000005},"min":{"1min":0.439,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76400000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.59799999999999998}]},{"osd":3,"last update":"Wed Sep 13 04:39:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.63500000000000001,"15min":0.57899999999999996},"min":{"1min":0.47799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.61699999999999999,"15min":0.58499999999999996},"min":{"1min":0.47899999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0009999999999999,"5min":1.115,"15min":1.115},"last":0.58299999999999996}]},{"osd":5,"last update":"Wed Sep 13 04:40:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64200000000000002,"15min":0.59899999999999998},"min":{"1min":0.51900000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81499999999999995,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.753},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63800000000000001,"15min":0.61599999999999999},"min":{"1min":0.42599999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.51400000000000001}]},{"osd":6,"last update":"Wed Sep 13 04:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67700000000000005,"15min":0.68300000000000005},"min":{"1min":0.52000000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70699999999999996,"15min":0.69699999999999995},"min":{"1min":0.59499999999999997,"5min":0.443,"15min":0.443},"max":{"1min":0.89000000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.79800000000000004}]},{"osd":7,"last update":"Wed Sep 13 04:39:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54700000000000004,"15min":0.52700000000000002},"min":{"1min":0.39500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63400000000000001,"15min":0.60399999999999998},"min":{"1min":0.47799999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.77800000000000002}]},{"osd":8,"last update":"Wed Sep 13 04:40:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.753,"15min":0.70699999999999996},"min":{"1min":0.65300000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93400000000000005,"5min":1.042,"15min":1.042},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.77500000000000002,"15min":0.76600000000000001},"min":{"1min":0.59799999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.97799999999999998,"5min":1.095,"15min":1.095},"last":0.81100000000000005}]},{"osd":9,"last update":"Wed Sep 13 04:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80400000000000005,"15min":0.81100000000000005},"min":{"1min":0.65000000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.03,"5min":1.141,"15min":1.141},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.624,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.94599999999999995,"5min":1.056,"15min":1.056},"last":0.73499999999999999}]},{"osd":10,"last update":"Wed Sep 13 04:39:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.754,"15min":0.75800000000000001},"min":{"1min":0.55100000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78900000000000003,"15min":0.79200000000000004},"min":{"1min":0.63900000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.88300000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.82699999999999996}]},{"osd":11,"last update":"Wed Sep 13 04:40:00 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.61399999999999999,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.71999999999999997,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":0.97399999999999998,"5min":0.997,"15min":0.997},"last":0.85399999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1654784,"data_stored":1101030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.438,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69299999999999995,"5min":0.754,"15min":0.754},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.52300000000000002},"min":{"1min":0.45500000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.76300000000000001,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.434}]},{"osd":1,"last update":"Wed Sep 13 04:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57099999999999995,"15min":0.53400000000000003},"min":{"1min":0.373,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.745,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.59899999999999998,"15min":0.54300000000000004},"min":{"1min":0.42399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77700000000000002,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.57299999999999995}]},{"osd":2,"last update":"Wed Sep 13 04:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42399999999999999,"15min":0.376},"min":{"1min":0.27900000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.59699999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.435,"15min":0.39100000000000001},"min":{"1min":0.315,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.55300000000000005,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.47399999999999998}]},{"osd":3,"last update":"Wed Sep 13 04:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.38600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.70899999999999996,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.67100000000000004,"15min":0.82999999999999996},"min":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.745,"5min":8.0009999999999994,"15min":8.0009999999999994},"last":0.66100000000000003}]},{"osd":4,"last update":"Wed Sep 13 04:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.628,"15min":0.57799999999999996},"min":{"1min":0.44700000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83699999999999997,"5min":1.599,"15min":1.599},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.627,"15min":0.57199999999999995},"min":{"1min":0.49199999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.82499999999999996,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.58499999999999996}]},{"osd":6,"last update":"Wed Sep 13 04:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.65600000000000003,"15min":0.65000000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.65000000000000002,"15min":0.63800000000000001},"min":{"1min":0.54700000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85899999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.55600000000000005}]},{"osd":7,"last update":"Wed Sep 13 04:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.46600000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94399999999999995,"5min":1.488,"15min":1.488},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69199999999999995,"15min":0.67300000000000004},"min":{"1min":0.58799999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.876,"5min":1.51,"15min":1.51},"last":0.59999999999999998}]},{"osd":8,"last update":"Wed Sep 13 04:40:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61199999999999999,"15min":0.59399999999999997},"min":{"1min":0.46899999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.63700000000000001,"15min":0.61299999999999999},"min":{"1min":0.5,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.68200000000000005}]},{"osd":9,"last update":"Wed Sep 13 04:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.50800000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.91400000000000003,"5min":1.008,"15min":1.008},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.74299999999999999,"15min":0.73999999999999999},"min":{"1min":0.60899999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.92900000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.61599999999999999}]},{"osd":10,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.74299999999999999,"15min":0.72499999999999998},"min":{"1min":0.51100000000000001,"5min":0.496,"15min":0.496},"max":{"1min":0.97199999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76200000000000001,"15min":0.75},"min":{"1min":0.63500000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.91900000000000004,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.749}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-13T04:40:13.076 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-13T04:40:13.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:12 smithi149 ceph-mon[133315]: pgmap v346: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2023-09-13T04:40:13.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:12 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3779605711' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:40:13.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:12 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/322282148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:40:13.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.308+0000 7f1de91ff700 1 -- 172.21.15.2:0/1459489905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1de4100cf0 msgr2=0x7f1de4101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:13.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.308+0000 7f1de91ff700 1 --2- 172.21.15.2:0/1459489905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1de4100cf0 0x7f1de4101110 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4009990 tx=0x7f1dd4009ca0 comp rx=0 tx=0).stop 2023-09-13T04:40:13.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.308+0000 7f1de91ff700 1 -- 172.21.15.2:0/1459489905 shutdown_connections 2023-09-13T04:40:13.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.308+0000 7f1de91ff700 1 --2- 172.21.15.2:0/1459489905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1de4102880 0x7f1de4109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:13.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.308+0000 7f1de91ff700 1 --2- 172.21.15.2:0/1459489905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1de4100cf0 0x7f1de4101110 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:13.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.308+0000 7f1de91ff700 1 --2- 172.21.15.2:0/1459489905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1de4101ee0 0x7f1de4102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:13.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.308+0000 7f1de91ff700 1 -- 172.21.15.2:0/1459489905 >> 172.21.15.2:0/1459489905 conn(0x7f1de40fc220 msgr2=0x7f1de40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:13.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.309+0000 7f1de91ff700 1 -- 172.21.15.2:0/1459489905 shutdown_connections 2023-09-13T04:40:13.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.309+0000 7f1de91ff700 1 -- 172.21.15.2:0/1459489905 wait complete. 2023-09-13T04:40:13.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.310+0000 7f1de91ff700 1 Processor -- start 2023-09-13T04:40:13.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.310+0000 7f1de91ff700 1 -- start start 2023-09-13T04:40:13.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.310+0000 7f1de91ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1de4100cf0 0x7f1de41969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:13.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.310+0000 7f1de91ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1de4101ee0 0x7f1de4196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:13.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.310+0000 7f1de91ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1de4102880 0x7f1de419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:13.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.310+0000 7f1de91ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de419e560 con 0x7f1de4101ee0 2023-09-13T04:40:13.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.310+0000 7f1de91ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de419e6a0 con 0x7f1de4102880 2023-09-13T04:40:13.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.310+0000 7f1de91ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de419e830 con 0x7f1de4100cf0 2023-09-13T04:40:13.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.310+0000 7f1de259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1de4101ee0 0x7f1de4196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:13.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.310+0000 7f1de2d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1de4100cf0 0x7f1de41969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:13.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.310+0000 7f1de259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1de4101ee0 0x7f1de4196f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47760/0 (socket says 172.21.15.2:47760) 2023-09-13T04:40:13.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.310+0000 7f1de259c700 1 -- 172.21.15.2:0/2999417894 learned_addr learned my addr 172.21.15.2:0/2999417894 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:13.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.310+0000 7f1de2d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1de4100cf0 0x7f1de41969e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:54576/0 (socket says 172.21.15.2:54576) 2023-09-13T04:40:13.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.311+0000 7f1de359e700 1 --2- 172.21.15.2:0/2999417894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1de4102880 0x7f1de419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:13.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.311+0000 7f1de2d9d700 1 -- 172.21.15.2:0/2999417894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1de4102880 msgr2=0x7f1de419df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:13.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.311+0000 7f1de2d9d700 1 --2- 172.21.15.2:0/2999417894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1de4102880 0x7f1de419df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:13.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.311+0000 7f1de2d9d700 1 -- 172.21.15.2:0/2999417894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1de4101ee0 msgr2=0x7f1de4196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:13.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.311+0000 7f1de2d9d700 1 --2- 172.21.15.2:0/2999417894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1de4101ee0 0x7f1de4196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:13.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.311+0000 7f1de2d9d700 1 -- 172.21.15.2:0/2999417894 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd8009520 con 0x7f1de4100cf0 2023-09-13T04:40:13.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.311+0000 7f1de259c700 1 --2- 172.21.15.2:0/2999417894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1de4101ee0 0x7f1de4196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:13.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.311+0000 7f1de2d9d700 1 --2- 172.21.15.2:0/2999417894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1de4100cf0 0x7f1de41969e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1dd40098e0 tx=0x7f1dd4009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:13.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.311+0000 7f1dcbfff700 1 -- 172.21.15.2:0/2999417894 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd401d070 con 0x7f1de4100cf0 2023-09-13T04:40:13.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.311+0000 7f1dcbfff700 1 -- 172.21.15.2:0/2999417894 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dd4021410 con 0x7f1de4100cf0 2023-09-13T04:40:13.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.311+0000 7f1de91ff700 1 -- 172.21.15.2:0/2999417894 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd40095f0 con 0x7f1de4100cf0 2023-09-13T04:40:13.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.311+0000 7f1dcbfff700 1 -- 172.21.15.2:0/2999417894 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4005720 con 0x7f1de4100cf0 2023-09-13T04:40:13.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.312+0000 7f1de91ff700 1 -- 172.21.15.2:0/2999417894 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de419ed40 con 0x7f1de4100cf0 2023-09-13T04:40:13.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.312+0000 7f1de91ff700 1 -- 172.21.15.2:0/2999417894 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1de4108290 con 0x7f1de4100cf0 2023-09-13T04:40:13.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.315+0000 7f1dcbfff700 1 -- 172.21.15.2:0/2999417894 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f1dd4021580 con 0x7f1de4100cf0 2023-09-13T04:40:13.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.316+0000 7f1dcbfff700 1 --2- 172.21.15.2:0/2999417894 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1dcc0a0690 0x7f1dcc0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:13.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.316+0000 7f1dcbfff700 1 -- 172.21.15.2:0/2999417894 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1dd40cd850 con 0x7f1de4100cf0 2023-09-13T04:40:13.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.316+0000 7f1de259c700 1 --2- 172.21.15.2:0/2999417894 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1dcc0a0690 0x7f1dcc0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:13.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.317+0000 7f1de259c700 1 --2- 172.21.15.2:0/2999417894 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1dcc0a0690 0x7f1dcc0a2b50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8009fd0 tx=0x7f1dd8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:13.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.318+0000 7f1dcbfff700 1 -- 172.21.15.2:0/2999417894 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1dd4097080 con 0x7f1de4100cf0 2023-09-13T04:40:13.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.524+0000 7f1de91ff700 1 -- 172.21.15.2:0/2999417894 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1de402cf60 con 0x7f1dcc0a0690 2023-09-13T04:40:13.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.537+0000 7f1dcbfff700 1 -- 172.21.15.2:0/2999417894 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427027 (secure 0 0 0) 0x7f1de402cf60 con 0x7f1dcc0a0690 2023-09-13T04:40:13.544 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:40:13.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.542+0000 7f1de91ff700 1 -- 172.21.15.2:0/2999417894 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1dcc0a0690 msgr2=0x7f1dcc0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:13.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.542+0000 7f1de91ff700 1 --2- 172.21.15.2:0/2999417894 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1dcc0a0690 0x7f1dcc0a2b50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8009fd0 tx=0x7f1dd8009260 comp rx=0 tx=0).stop 2023-09-13T04:40:13.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.542+0000 7f1de91ff700 1 -- 172.21.15.2:0/2999417894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1de4100cf0 msgr2=0x7f1de41969e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:13.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.542+0000 7f1de91ff700 1 --2- 172.21.15.2:0/2999417894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1de4100cf0 0x7f1de41969e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1dd40098e0 tx=0x7f1dd4009580 comp rx=0 tx=0).stop 2023-09-13T04:40:13.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.542+0000 7f1de91ff700 1 -- 172.21.15.2:0/2999417894 shutdown_connections 2023-09-13T04:40:13.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.542+0000 7f1de91ff700 1 --2- 172.21.15.2:0/2999417894 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1dcc0a0690 0x7f1dcc0a2b50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:13.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.542+0000 7f1de91ff700 1 --2- 172.21.15.2:0/2999417894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1de4102880 0x7f1de419df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:13.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.542+0000 7f1de91ff700 1 --2- 172.21.15.2:0/2999417894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1de4100cf0 0x7f1de41969e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:13.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.542+0000 7f1de91ff700 1 --2- 172.21.15.2:0/2999417894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1de4101ee0 0x7f1de4196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:13.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.542+0000 7f1de91ff700 1 -- 172.21.15.2:0/2999417894 >> 172.21.15.2:0/2999417894 conn(0x7f1de40fc220 msgr2=0x7f1de41053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:13.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.542+0000 7f1de91ff700 1 -- 172.21.15.2:0/2999417894 shutdown_connections 2023-09-13T04:40:13.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.542+0000 7f1de91ff700 1 -- 172.21.15.2:0/2999417894 wait complete. 2023-09-13T04:40:13.549 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-09-13T04:40:13.563 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2023-09-13T04:40:11.535570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598078,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":782,"num_read_kb":456,"num_write":312,"num_write_kb":2983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":252,"ondisk_log_size":252,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338328,"kb_used_data":14068,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603496,"statfs":{"total":1151940427776,"available":1151593979904,"internally_reserved":0,"allocated":14405632,"data_stored":7903338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":258,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":91,"num_read_kb":46,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003086"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.624372+0000","last_change":"2023-09-13T04:37:59.244279+0000","last_active":"2023-09-13T04:39:25.624372+0000","last_peered":"2023-09-13T04:39:25.624372+0000","last_clean":"2023-09-13T04:39:25.624372+0000","last_became_active":"2023-09-13T04:37:59.243976+0000","last_became_peered":"2023-09-13T04:37:59.243976+0000","last_unstale":"2023-09-13T04:39:25.624372+0000","last_undegraded":"2023-09-13T04:39:25.624372+0000","last_fullsized":"2023-09-13T04:39:25.624372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:21:52.809411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.856077+0000","last_change":"2023-09-13T04:37:59.243391+0000","last_active":"2023-09-13T04:39:25.856077+0000","last_peered":"2023-09-13T04:39:25.856077+0000","last_clean":"2023-09-13T04:39:25.856077+0000","last_became_active":"2023-09-13T04:37:59.243044+0000","last_became_peered":"2023-09-13T04:37:59.243044+0000","last_unstale":"2023-09-13T04:39:25.856077+0000","last_undegraded":"2023-09-13T04:39:25.856077+0000","last_fullsized":"2023-09-13T04:39:25.856077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:02:17.806638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.855493+0000","last_change":"2023-09-13T04:37:59.242826+0000","last_active":"2023-09-13T04:39:25.855493+0000","last_peered":"2023-09-13T04:39:25.855493+0000","last_clean":"2023-09-13T04:39:25.855493+0000","last_became_active":"2023-09-13T04:37:59.242125+0000","last_became_peered":"2023-09-13T04:37:59.242125+0000","last_unstale":"2023-09-13T04:39:25.855493+0000","last_undegraded":"2023-09-13T04:39:25.855493+0000","last_fullsized":"2023-09-13T04:39:25.855493+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:08:29.216093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.709501+0000","last_change":"2023-09-13T04:37:59.253322+0000","last_active":"2023-09-13T04:39:25.709501+0000","last_peered":"2023-09-13T04:39:25.709501+0000","last_clean":"2023-09-13T04:39:25.709501+0000","last_became_active":"2023-09-13T04:37:59.253050+0000","last_became_peered":"2023-09-13T04:37:59.253050+0000","last_unstale":"2023-09-13T04:39:25.709501+0000","last_undegraded":"2023-09-13T04:39:25.709501+0000","last_fullsized":"2023-09-13T04:39:25.709501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:45:22.357879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755114+0000","last_change":"2023-09-13T04:37:59.250089+0000","last_active":"2023-09-13T04:39:24.755114+0000","last_peered":"2023-09-13T04:39:24.755114+0000","last_clean":"2023-09-13T04:39:24.755114+0000","last_became_active":"2023-09-13T04:37:59.249771+0000","last_became_peered":"2023-09-13T04:37:59.249771+0000","last_unstale":"2023-09-13T04:39:24.755114+0000","last_undegraded":"2023-09-13T04:39:24.755114+0000","last_fullsized":"2023-09-13T04:39:24.755114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:46:59.917756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.703747+0000","last_change":"2023-09-13T04:37:59.247259+0000","last_active":"2023-09-13T04:39:24.703747+0000","last_peered":"2023-09-13T04:39:24.703747+0000","last_clean":"2023-09-13T04:39:24.703747+0000","last_became_active":"2023-09-13T04:37:59.246911+0000","last_became_peered":"2023-09-13T04:37:59.246911+0000","last_unstale":"2023-09-13T04:39:24.703747+0000","last_undegraded":"2023-09-13T04:39:24.703747+0000","last_fullsized":"2023-09-13T04:39:24.703747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:00:39.742392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.585151+0000","last_change":"2023-09-13T04:37:59.244083+0000","last_active":"2023-09-13T04:39:24.585151+0000","last_peered":"2023-09-13T04:39:24.585151+0000","last_clean":"2023-09-13T04:39:24.585151+0000","last_became_active":"2023-09-13T04:37:59.243789+0000","last_became_peered":"2023-09-13T04:37:59.243789+0000","last_unstale":"2023-09-13T04:39:24.585151+0000","last_undegraded":"2023-09-13T04:39:24.585151+0000","last_fullsized":"2023-09-13T04:39:24.585151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:24:00.622790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.581996+0000","last_change":"2023-09-13T04:38:24.601265+0000","last_active":"2023-09-13T04:39:24.581996+0000","last_peered":"2023-09-13T04:39:24.581996+0000","last_clean":"2023-09-13T04:39:24.581996+0000","last_became_active":"2023-09-13T04:38:24.600939+0000","last_became_peered":"2023-09-13T04:38:24.600939+0000","last_unstale":"2023-09-13T04:39:24.581996+0000","last_undegraded":"2023-09-13T04:39:24.581996+0000","last_fullsized":"2023-09-13T04:39:24.581996+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:18:23.642333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754323+0000","last_change":"2023-09-13T04:37:59.238053+0000","last_active":"2023-09-13T04:39:24.754323+0000","last_peered":"2023-09-13T04:39:24.754323+0000","last_clean":"2023-09-13T04:39:24.754323+0000","last_became_active":"2023-09-13T04:37:59.237777+0000","last_became_peered":"2023-09-13T04:37:59.237777+0000","last_unstale":"2023-09-13T04:39:24.754323+0000","last_undegraded":"2023-09-13T04:39:24.754323+0000","last_fullsized":"2023-09-13T04:39:24.754323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:45:39.835659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.704187+0000","last_change":"2023-09-13T04:37:59.252605+0000","last_active":"2023-09-13T04:39:24.704187+0000","last_peered":"2023-09-13T04:39:24.704187+0000","last_clean":"2023-09-13T04:39:24.704187+0000","last_became_active":"2023-09-13T04:37:59.252309+0000","last_became_peered":"2023-09-13T04:37:59.252309+0000","last_unstale":"2023-09-13T04:39:24.704187+0000","last_undegraded":"2023-09-13T04:39:24.704187+0000","last_fullsized":"2023-09-13T04:39:24.704187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:41:45.064230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.585552+0000","last_change":"2023-09-13T04:37:59.243345+0000","last_active":"2023-09-13T04:39:24.585552+0000","last_peered":"2023-09-13T04:39:24.585552+0000","last_clean":"2023-09-13T04:39:24.585552+0000","last_became_active":"2023-09-13T04:37:59.242726+0000","last_became_peered":"2023-09-13T04:37:59.242726+0000","last_unstale":"2023-09-13T04:39:24.585552+0000","last_undegraded":"2023-09-13T04:39:24.585552+0000","last_fullsized":"2023-09-13T04:39:24.585552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:39:55.115281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583449+0000","last_change":"2023-09-13T04:37:59.247415+0000","last_active":"2023-09-13T04:39:24.583449+0000","last_peered":"2023-09-13T04:39:24.583449+0000","last_clean":"2023-09-13T04:39:24.583449+0000","last_became_active":"2023-09-13T04:37:59.247050+0000","last_became_peered":"2023-09-13T04:37:59.247050+0000","last_unstale":"2023-09-13T04:39:24.583449+0000","last_undegraded":"2023-09-13T04:39:24.583449+0000","last_fullsized":"2023-09-13T04:39:24.583449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:03:44.793628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.588078+0000","last_change":"2023-09-13T04:37:59.248935+0000","last_active":"2023-09-13T04:39:24.588078+0000","last_peered":"2023-09-13T04:39:24.588078+0000","last_clean":"2023-09-13T04:39:24.588078+0000","last_became_active":"2023-09-13T04:37:59.248554+0000","last_became_peered":"2023-09-13T04:37:59.248554+0000","last_unstale":"2023-09-13T04:39:24.588078+0000","last_undegraded":"2023-09-13T04:39:24.588078+0000","last_fullsized":"2023-09-13T04:39:24.588078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:53:31.882567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754559+0000","last_change":"2023-09-13T04:37:59.242851+0000","last_active":"2023-09-13T04:39:24.754559+0000","last_peered":"2023-09-13T04:39:24.754559+0000","last_clean":"2023-09-13T04:39:24.754559+0000","last_became_active":"2023-09-13T04:37:59.242384+0000","last_became_peered":"2023-09-13T04:37:59.242384+0000","last_unstale":"2023-09-13T04:39:24.754559+0000","last_undegraded":"2023-09-13T04:39:24.754559+0000","last_fullsized":"2023-09-13T04:39:24.754559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:59:36.508808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754461+0000","last_change":"2023-09-13T04:38:24.599392+0000","last_active":"2023-09-13T04:39:24.754461+0000","last_peered":"2023-09-13T04:39:24.754461+0000","last_clean":"2023-09-13T04:39:24.754461+0000","last_became_active":"2023-09-13T04:38:24.599029+0000","last_became_peered":"2023-09-13T04:38:24.599029+0000","last_unstale":"2023-09-13T04:39:24.754461+0000","last_undegraded":"2023-09-13T04:39:24.754461+0000","last_fullsized":"2023-09-13T04:39:24.754461+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:23:03.703387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.753727+0000","last_change":"2023-09-13T04:37:59.249107+0000","last_active":"2023-09-13T04:39:24.753727+0000","last_peered":"2023-09-13T04:39:24.753727+0000","last_clean":"2023-09-13T04:39:24.753727+0000","last_became_active":"2023-09-13T04:37:59.248803+0000","last_became_peered":"2023-09-13T04:37:59.248803+0000","last_unstale":"2023-09-13T04:39:24.753727+0000","last_undegraded":"2023-09-13T04:39:24.753727+0000","last_fullsized":"2023-09-13T04:39:24.753727+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:50:52.481194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755155+0000","last_change":"2023-09-13T04:37:59.243724+0000","last_active":"2023-09-13T04:39:24.755155+0000","last_peered":"2023-09-13T04:39:24.755155+0000","last_clean":"2023-09-13T04:39:24.755155+0000","last_became_active":"2023-09-13T04:37:59.243385+0000","last_became_peered":"2023-09-13T04:37:59.243385+0000","last_unstale":"2023-09-13T04:39:24.755155+0000","last_undegraded":"2023-09-13T04:39:24.755155+0000","last_fullsized":"2023-09-13T04:39:24.755155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:55:55.203940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582498+0000","last_change":"2023-09-13T04:37:59.238986+0000","last_active":"2023-09-13T04:39:24.582498+0000","last_peered":"2023-09-13T04:39:24.582498+0000","last_clean":"2023-09-13T04:39:24.582498+0000","last_became_active":"2023-09-13T04:37:59.238668+0000","last_became_peered":"2023-09-13T04:37:59.238668+0000","last_unstale":"2023-09-13T04:39:24.582498+0000","last_undegraded":"2023-09-13T04:39:24.582498+0000","last_fullsized":"2023-09-13T04:39:24.582498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:21:10.225223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583227+0000","last_change":"2023-09-13T04:37:59.245598+0000","last_active":"2023-09-13T04:39:24.583227+0000","last_peered":"2023-09-13T04:39:24.583227+0000","last_clean":"2023-09-13T04:39:24.583227+0000","last_became_active":"2023-09-13T04:37:59.244652+0000","last_became_peered":"2023-09-13T04:37:59.244652+0000","last_unstale":"2023-09-13T04:39:24.583227+0000","last_undegraded":"2023-09-13T04:39:24.583227+0000","last_fullsized":"2023-09-13T04:39:24.583227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:34:43.671168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583096+0000","last_change":"2023-09-13T04:38:03.617708+0000","last_active":"2023-09-13T04:39:24.583096+0000","last_peered":"2023-09-13T04:39:24.583096+0000","last_clean":"2023-09-13T04:39:24.583096+0000","last_became_active":"2023-09-13T04:38:03.617277+0000","last_became_peered":"2023-09-13T04:38:03.617277+0000","last_unstale":"2023-09-13T04:39:24.583096+0000","last_undegraded":"2023-09-13T04:39:24.583096+0000","last_fullsized":"2023-09-13T04:39:24.583096+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:26:44.318030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.588015+0000","last_change":"2023-09-13T04:37:58.595156+0000","last_active":"2023-09-13T04:39:25.588015+0000","last_peered":"2023-09-13T04:39:25.588015+0000","last_clean":"2023-09-13T04:39:25.588015+0000","last_became_active":"2023-09-13T04:37:58.594863+0000","last_became_peered":"2023-09-13T04:37:58.594863+0000","last_unstale":"2023-09-13T04:39:25.588015+0000","last_undegraded":"2023-09-13T04:39:25.588015+0000","last_fullsized":"2023-09-13T04:39:25.588015+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:27:14.420495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.854946+0000","last_change":"2023-09-13T04:37:59.250252+0000","last_active":"2023-09-13T04:39:25.854946+0000","last_peered":"2023-09-13T04:39:25.854946+0000","last_clean":"2023-09-13T04:39:25.854946+0000","last_became_active":"2023-09-13T04:37:59.249927+0000","last_became_peered":"2023-09-13T04:37:59.249927+0000","last_unstale":"2023-09-13T04:39:25.854946+0000","last_undegraded":"2023-09-13T04:39:25.854946+0000","last_fullsized":"2023-09-13T04:39:25.854946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:18:06.757040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582790+0000","last_change":"2023-09-13T04:38:24.599683+0000","last_active":"2023-09-13T04:39:24.582790+0000","last_peered":"2023-09-13T04:39:24.582790+0000","last_clean":"2023-09-13T04:39:24.582790+0000","last_became_active":"2023-09-13T04:38:24.599230+0000","last_became_peered":"2023-09-13T04:38:24.599230+0000","last_unstale":"2023-09-13T04:39:24.582790+0000","last_undegraded":"2023-09-13T04:39:24.582790+0000","last_fullsized":"2023-09-13T04:39:24.582790+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:48:22.584735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.855657+0000","last_change":"2023-09-13T04:37:58.597785+0000","last_active":"2023-09-13T04:39:25.855657+0000","last_peered":"2023-09-13T04:39:25.855657+0000","last_clean":"2023-09-13T04:39:25.855657+0000","last_became_active":"2023-09-13T04:37:58.597485+0000","last_became_peered":"2023-09-13T04:37:58.597485+0000","last_unstale":"2023-09-13T04:39:25.855657+0000","last_undegraded":"2023-09-13T04:39:25.855657+0000","last_fullsized":"2023-09-13T04:39:25.855657+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:39:57.156038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.856081+0000","last_change":"2023-09-13T04:37:59.241840+0000","last_active":"2023-09-13T04:39:25.856081+0000","last_peered":"2023-09-13T04:39:25.856081+0000","last_clean":"2023-09-13T04:39:25.856081+0000","last_became_active":"2023-09-13T04:37:59.241545+0000","last_became_peered":"2023-09-13T04:37:59.241545+0000","last_unstale":"2023-09-13T04:39:25.856081+0000","last_undegraded":"2023-09-13T04:39:25.856081+0000","last_fullsized":"2023-09-13T04:39:25.856081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:04:17.439520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582992+0000","last_change":"2023-09-13T04:38:03.621792+0000","last_active":"2023-09-13T04:39:24.582992+0000","last_peered":"2023-09-13T04:39:24.582992+0000","last_clean":"2023-09-13T04:39:24.582992+0000","last_became_active":"2023-09-13T04:38:03.621433+0000","last_became_peered":"2023-09-13T04:38:03.621433+0000","last_unstale":"2023-09-13T04:39:24.582992+0000","last_undegraded":"2023-09-13T04:39:24.582992+0000","last_fullsized":"2023-09-13T04:39:24.582992+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:52:05.926733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754925+0000","last_change":"2023-09-13T04:37:58.590817+0000","last_active":"2023-09-13T04:39:24.754925+0000","last_peered":"2023-09-13T04:39:24.754925+0000","last_clean":"2023-09-13T04:39:24.754925+0000","last_became_active":"2023-09-13T04:37:58.590348+0000","last_became_peered":"2023-09-13T04:37:58.590348+0000","last_unstale":"2023-09-13T04:39:24.754925+0000","last_undegraded":"2023-09-13T04:39:24.754925+0000","last_fullsized":"2023-09-13T04:39:24.754925+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:38:35.680905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754016+0000","last_change":"2023-09-13T04:37:59.237304+0000","last_active":"2023-09-13T04:39:24.754016+0000","last_peered":"2023-09-13T04:39:24.754016+0000","last_clean":"2023-09-13T04:39:24.754016+0000","last_became_active":"2023-09-13T04:37:59.237008+0000","last_became_peered":"2023-09-13T04:37:59.237008+0000","last_unstale":"2023-09-13T04:39:24.754016+0000","last_undegraded":"2023-09-13T04:39:24.754016+0000","last_fullsized":"2023-09-13T04:39:24.754016+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:28:14.254219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582847+0000","last_change":"2023-09-13T04:38:03.606749+0000","last_active":"2023-09-13T04:39:24.582847+0000","last_peered":"2023-09-13T04:39:24.582847+0000","last_clean":"2023-09-13T04:39:24.582847+0000","last_became_active":"2023-09-13T04:38:03.606276+0000","last_became_peered":"2023-09-13T04:38:03.606276+0000","last_unstale":"2023-09-13T04:39:24.582847+0000","last_undegraded":"2023-09-13T04:39:24.582847+0000","last_fullsized":"2023-09-13T04:39:24.582847+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:30:53.572543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754722+0000","last_change":"2023-09-13T04:37:58.590937+0000","last_active":"2023-09-13T04:39:24.754722+0000","last_peered":"2023-09-13T04:39:24.754722+0000","last_clean":"2023-09-13T04:39:24.754722+0000","last_became_active":"2023-09-13T04:37:58.590606+0000","last_became_peered":"2023-09-13T04:37:58.590606+0000","last_unstale":"2023-09-13T04:39:24.754722+0000","last_undegraded":"2023-09-13T04:39:24.754722+0000","last_fullsized":"2023-09-13T04:39:24.754722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:30:03.354670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.706476+0000","last_change":"2023-09-13T04:37:59.251899+0000","last_active":"2023-09-13T04:39:24.706476+0000","last_peered":"2023-09-13T04:39:24.706476+0000","last_clean":"2023-09-13T04:39:24.706476+0000","last_became_active":"2023-09-13T04:37:59.251557+0000","last_became_peered":"2023-09-13T04:37:59.251557+0000","last_unstale":"2023-09-13T04:39:24.706476+0000","last_undegraded":"2023-09-13T04:39:24.706476+0000","last_fullsized":"2023-09-13T04:39:24.706476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:57:45.241188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582997+0000","last_change":"2023-09-13T04:38:03.606076+0000","last_active":"2023-09-13T04:39:24.582997+0000","last_peered":"2023-09-13T04:39:24.582997+0000","last_clean":"2023-09-13T04:39:24.582997+0000","last_became_active":"2023-09-13T04:38:03.605556+0000","last_became_peered":"2023-09-13T04:38:03.605556+0000","last_unstale":"2023-09-13T04:39:24.582997+0000","last_undegraded":"2023-09-13T04:39:24.582997+0000","last_fullsized":"2023-09-13T04:39:24.582997+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:03:32.344625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":132,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:40:08.771722+0000","last_change":"2023-09-13T04:37:58.588170+0000","last_active":"2023-09-13T04:40:08.771722+0000","last_peered":"2023-09-13T04:40:08.771722+0000","last_clean":"2023-09-13T04:40:08.771722+0000","last_became_active":"2023-09-13T04:37:58.587860+0000","last_became_peered":"2023-09-13T04:37:58.587860+0000","last_unstale":"2023-09-13T04:40:08.771722+0000","last_undegraded":"2023-09-13T04:40:08.771722+0000","last_fullsized":"2023-09-13T04:40:08.771722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:38:40.048150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754818+0000","last_change":"2023-09-13T04:37:59.240248+0000","last_active":"2023-09-13T04:39:24.754818+0000","last_peered":"2023-09-13T04:39:24.754818+0000","last_clean":"2023-09-13T04:39:24.754818+0000","last_became_active":"2023-09-13T04:37:59.240008+0000","last_became_peered":"2023-09-13T04:37:59.240008+0000","last_unstale":"2023-09-13T04:39:24.754818+0000","last_undegraded":"2023-09-13T04:39:24.754818+0000","last_fullsized":"2023-09-13T04:39:24.754818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:15:13.275110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583113+0000","last_change":"2023-09-13T04:38:03.616607+0000","last_active":"2023-09-13T04:39:24.583113+0000","last_peered":"2023-09-13T04:39:24.583113+0000","last_clean":"2023-09-13T04:39:24.583113+0000","last_became_active":"2023-09-13T04:38:03.616209+0000","last_became_peered":"2023-09-13T04:38:03.616209+0000","last_unstale":"2023-09-13T04:39:24.583113+0000","last_undegraded":"2023-09-13T04:39:24.583113+0000","last_fullsized":"2023-09-13T04:39:24.583113+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:24:21.934053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.703747+0000","last_change":"2023-09-13T04:37:58.589104+0000","last_active":"2023-09-13T04:39:24.703747+0000","last_peered":"2023-09-13T04:39:24.703747+0000","last_clean":"2023-09-13T04:39:24.703747+0000","last_became_active":"2023-09-13T04:37:58.588780+0000","last_became_peered":"2023-09-13T04:37:58.588780+0000","last_unstale":"2023-09-13T04:39:24.703747+0000","last_undegraded":"2023-09-13T04:39:24.703747+0000","last_fullsized":"2023-09-13T04:39:24.703747+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:01:32.839729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582218+0000","last_change":"2023-09-13T04:38:24.598663+0000","last_active":"2023-09-13T04:39:24.582218+0000","last_peered":"2023-09-13T04:39:24.582218+0000","last_clean":"2023-09-13T04:39:24.582218+0000","last_became_active":"2023-09-13T04:38:24.598258+0000","last_became_peered":"2023-09-13T04:38:24.598258+0000","last_unstale":"2023-09-13T04:39:24.582218+0000","last_undegraded":"2023-09-13T04:39:24.582218+0000","last_fullsized":"2023-09-13T04:39:24.582218+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:50:28.157135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.624295+0000","last_change":"2023-09-13T04:38:03.619930+0000","last_active":"2023-09-13T04:39:25.624295+0000","last_peered":"2023-09-13T04:39:25.624295+0000","last_clean":"2023-09-13T04:39:25.624295+0000","last_became_active":"2023-09-13T04:38:03.619560+0000","last_became_peered":"2023-09-13T04:38:03.619560+0000","last_unstale":"2023-09-13T04:39:25.624295+0000","last_undegraded":"2023-09-13T04:39:25.624295+0000","last_fullsized":"2023-09-13T04:39:25.624295+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:48:06.867602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.587499+0000","last_change":"2023-09-13T04:37:58.590396+0000","last_active":"2023-09-13T04:39:24.587499+0000","last_peered":"2023-09-13T04:39:24.587499+0000","last_clean":"2023-09-13T04:39:24.587499+0000","last_became_active":"2023-09-13T04:37:58.589929+0000","last_became_peered":"2023-09-13T04:37:58.589929+0000","last_unstale":"2023-09-13T04:39:24.587499+0000","last_undegraded":"2023-09-13T04:39:24.587499+0000","last_fullsized":"2023-09-13T04:39:24.587499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:28:36.322096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754248+0000","last_change":"2023-09-13T04:37:59.246033+0000","last_active":"2023-09-13T04:39:24.754248+0000","last_peered":"2023-09-13T04:39:24.754248+0000","last_clean":"2023-09-13T04:39:24.754248+0000","last_became_active":"2023-09-13T04:37:59.245664+0000","last_became_peered":"2023-09-13T04:37:59.245664+0000","last_unstale":"2023-09-13T04:39:24.754248+0000","last_undegraded":"2023-09-13T04:39:24.754248+0000","last_fullsized":"2023-09-13T04:39:24.754248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:24:17.145055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.856052+0000","last_change":"2023-09-13T04:38:03.608504+0000","last_active":"2023-09-13T04:39:25.856052+0000","last_peered":"2023-09-13T04:39:25.856052+0000","last_clean":"2023-09-13T04:39:25.856052+0000","last_became_active":"2023-09-13T04:38:03.608058+0000","last_became_peered":"2023-09-13T04:38:03.608058+0000","last_unstale":"2023-09-13T04:39:25.856052+0000","last_undegraded":"2023-09-13T04:39:25.856052+0000","last_fullsized":"2023-09-13T04:39:25.856052+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:42:43.485453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.705989+0000","last_change":"2023-09-13T04:37:58.588208+0000","last_active":"2023-09-13T04:39:24.705989+0000","last_peered":"2023-09-13T04:39:24.705989+0000","last_clean":"2023-09-13T04:39:24.705989+0000","last_became_active":"2023-09-13T04:37:58.587840+0000","last_became_peered":"2023-09-13T04:37:58.587840+0000","last_unstale":"2023-09-13T04:39:24.705989+0000","last_undegraded":"2023-09-13T04:39:24.705989+0000","last_fullsized":"2023-09-13T04:39:24.705989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:44:58.012233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582291+0000","last_change":"2023-09-13T04:37:59.245102+0000","last_active":"2023-09-13T04:39:24.582291+0000","last_peered":"2023-09-13T04:39:24.582291+0000","last_clean":"2023-09-13T04:39:24.582291+0000","last_became_active":"2023-09-13T04:37:59.244799+0000","last_became_peered":"2023-09-13T04:37:59.244799+0000","last_unstale":"2023-09-13T04:39:24.582291+0000","last_undegraded":"2023-09-13T04:39:24.582291+0000","last_fullsized":"2023-09-13T04:39:24.582291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:57:16.295623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754657+0000","last_change":"2023-09-13T04:38:03.607021+0000","last_active":"2023-09-13T04:39:24.754657+0000","last_peered":"2023-09-13T04:39:24.754657+0000","last_clean":"2023-09-13T04:39:24.754657+0000","last_became_active":"2023-09-13T04:38:03.606611+0000","last_became_peered":"2023-09-13T04:38:03.606611+0000","last_unstale":"2023-09-13T04:39:24.754657+0000","last_undegraded":"2023-09-13T04:39:24.754657+0000","last_fullsized":"2023-09-13T04:39:24.754657+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:33:20.551479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":127,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:40:08.771101+0000","last_change":"2023-09-13T04:37:58.588731+0000","last_active":"2023-09-13T04:40:08.771101+0000","last_peered":"2023-09-13T04:40:08.771101+0000","last_clean":"2023-09-13T04:40:08.771101+0000","last_became_active":"2023-09-13T04:37:58.588398+0000","last_became_peered":"2023-09-13T04:37:58.588398+0000","last_unstale":"2023-09-13T04:40:08.771101+0000","last_undegraded":"2023-09-13T04:40:08.771101+0000","last_fullsized":"2023-09-13T04:40:08.771101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:13:10.724085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754323+0000","last_change":"2023-09-13T04:37:59.244757+0000","last_active":"2023-09-13T04:39:24.754323+0000","last_peered":"2023-09-13T04:39:24.754323+0000","last_clean":"2023-09-13T04:39:24.754323+0000","last_became_active":"2023-09-13T04:37:59.244423+0000","last_became_peered":"2023-09-13T04:37:59.244423+0000","last_unstale":"2023-09-13T04:39:24.754323+0000","last_undegraded":"2023-09-13T04:39:24.754323+0000","last_fullsized":"2023-09-13T04:39:24.754323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:14:37.973058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582006+0000","last_change":"2023-09-13T04:38:03.618358+0000","last_active":"2023-09-13T04:39:24.582006+0000","last_peered":"2023-09-13T04:39:24.582006+0000","last_clean":"2023-09-13T04:39:24.582006+0000","last_became_active":"2023-09-13T04:38:03.617291+0000","last_became_peered":"2023-09-13T04:38:03.617291+0000","last_unstale":"2023-09-13T04:39:24.582006+0000","last_undegraded":"2023-09-13T04:39:24.582006+0000","last_fullsized":"2023-09-13T04:39:24.582006+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:33:13.955353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.585686+0000","last_change":"2023-09-13T04:37:58.590978+0000","last_active":"2023-09-13T04:39:24.585686+0000","last_peered":"2023-09-13T04:39:24.585686+0000","last_clean":"2023-09-13T04:39:24.585686+0000","last_became_active":"2023-09-13T04:37:58.590673+0000","last_became_peered":"2023-09-13T04:37:58.590673+0000","last_unstale":"2023-09-13T04:39:24.585686+0000","last_undegraded":"2023-09-13T04:39:24.585686+0000","last_fullsized":"2023-09-13T04:39:24.585686+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:10:32.570029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755812+0000","last_change":"2023-09-13T04:37:59.251283+0000","last_active":"2023-09-13T04:39:24.755812+0000","last_peered":"2023-09-13T04:39:24.755812+0000","last_clean":"2023-09-13T04:39:24.755812+0000","last_became_active":"2023-09-13T04:37:59.250119+0000","last_became_peered":"2023-09-13T04:37:59.250119+0000","last_unstale":"2023-09-13T04:39:24.755812+0000","last_undegraded":"2023-09-13T04:39:24.755812+0000","last_fullsized":"2023-09-13T04:39:24.755812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:48:54.389970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583163+0000","last_change":"2023-09-13T04:38:24.596107+0000","last_active":"2023-09-13T04:39:24.583163+0000","last_peered":"2023-09-13T04:39:24.583163+0000","last_clean":"2023-09-13T04:39:24.583163+0000","last_became_active":"2023-09-13T04:38:24.595653+0000","last_became_peered":"2023-09-13T04:38:24.595653+0000","last_unstale":"2023-09-13T04:39:24.583163+0000","last_undegraded":"2023-09-13T04:39:24.583163+0000","last_fullsized":"2023-09-13T04:39:24.583163+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:02:53.708658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.706415+0000","last_change":"2023-09-13T04:37:58.584909+0000","last_active":"2023-09-13T04:39:24.706415+0000","last_peered":"2023-09-13T04:39:24.706415+0000","last_clean":"2023-09-13T04:39:24.706415+0000","last_became_active":"2023-09-13T04:37:58.584630+0000","last_became_peered":"2023-09-13T04:37:58.584630+0000","last_unstale":"2023-09-13T04:39:24.706415+0000","last_undegraded":"2023-09-13T04:39:24.706415+0000","last_fullsized":"2023-09-13T04:39:24.706415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:57:56.567499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754125+0000","last_change":"2023-09-13T04:37:59.254448+0000","last_active":"2023-09-13T04:39:24.754125+0000","last_peered":"2023-09-13T04:39:24.754125+0000","last_clean":"2023-09-13T04:39:24.754125+0000","last_became_active":"2023-09-13T04:37:59.254138+0000","last_became_peered":"2023-09-13T04:37:59.254138+0000","last_unstale":"2023-09-13T04:39:24.754125+0000","last_undegraded":"2023-09-13T04:39:24.754125+0000","last_fullsized":"2023-09-13T04:39:24.754125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:36:14.939881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582104+0000","last_change":"2023-09-13T04:38:03.613746+0000","last_active":"2023-09-13T04:39:24.582104+0000","last_peered":"2023-09-13T04:39:24.582104+0000","last_clean":"2023-09-13T04:39:24.582104+0000","last_became_active":"2023-09-13T04:38:03.613354+0000","last_became_peered":"2023-09-13T04:38:03.613354+0000","last_unstale":"2023-09-13T04:39:24.582104+0000","last_undegraded":"2023-09-13T04:39:24.582104+0000","last_fullsized":"2023-09-13T04:39:24.582104+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:41:01.294912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'14","reported_seq":245,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:40:09.772943+0000","last_change":"2023-09-13T04:37:58.591196+0000","last_active":"2023-09-13T04:40:09.772943+0000","last_peered":"2023-09-13T04:40:09.772943+0000","last_clean":"2023-09-13T04:40:09.772943+0000","last_became_active":"2023-09-13T04:37:58.590883+0000","last_became_peered":"2023-09-13T04:37:58.590883+0000","last_unstale":"2023-09-13T04:40:09.772943+0000","last_undegraded":"2023-09-13T04:40:09.772943+0000","last_fullsized":"2023-09-13T04:40:09.772943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:46:58.530270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4100,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":211,"num_read_kb":110,"num_write":15,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755199+0000","last_change":"2023-09-13T04:37:59.243483+0000","last_active":"2023-09-13T04:39:24.755199+0000","last_peered":"2023-09-13T04:39:24.755199+0000","last_clean":"2023-09-13T04:39:24.755199+0000","last_became_active":"2023-09-13T04:37:59.243088+0000","last_became_peered":"2023-09-13T04:37:59.243088+0000","last_unstale":"2023-09-13T04:39:24.755199+0000","last_undegraded":"2023-09-13T04:39:24.755199+0000","last_fullsized":"2023-09-13T04:39:24.755199+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:56:14.093819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.706663+0000","last_change":"2023-09-13T04:38:03.608820+0000","last_active":"2023-09-13T04:39:24.706663+0000","last_peered":"2023-09-13T04:39:24.706663+0000","last_clean":"2023-09-13T04:39:24.706663+0000","last_became_active":"2023-09-13T04:38:03.608200+0000","last_became_peered":"2023-09-13T04:38:03.608200+0000","last_unstale":"2023-09-13T04:39:24.706663+0000","last_undegraded":"2023-09-13T04:39:24.706663+0000","last_fullsized":"2023-09-13T04:39:24.706663+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:54:52.416501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754566+0000","last_change":"2023-09-13T04:38:24.606820+0000","last_active":"2023-09-13T04:39:24.754566+0000","last_peered":"2023-09-13T04:39:24.754566+0000","last_clean":"2023-09-13T04:39:24.754566+0000","last_became_active":"2023-09-13T04:38:24.599059+0000","last_became_peered":"2023-09-13T04:38:24.599059+0000","last_unstale":"2023-09-13T04:39:24.754566+0000","last_undegraded":"2023-09-13T04:39:24.754566+0000","last_fullsized":"2023-09-13T04:39:24.754566+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:08:12.634357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.587438+0000","last_change":"2023-09-13T04:37:59.250666+0000","last_active":"2023-09-13T04:39:24.587438+0000","last_peered":"2023-09-13T04:39:24.587438+0000","last_clean":"2023-09-13T04:39:24.587438+0000","last_became_active":"2023-09-13T04:37:59.250240+0000","last_became_peered":"2023-09-13T04:37:59.250240+0000","last_unstale":"2023-09-13T04:39:24.587438+0000","last_undegraded":"2023-09-13T04:39:24.587438+0000","last_fullsized":"2023-09-13T04:39:24.587438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:44:31.324222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754566+0000","last_change":"2023-09-13T04:38:03.616296+0000","last_active":"2023-09-13T04:39:24.754566+0000","last_peered":"2023-09-13T04:39:24.754566+0000","last_clean":"2023-09-13T04:39:24.754566+0000","last_became_active":"2023-09-13T04:38:03.615934+0000","last_became_peered":"2023-09-13T04:38:03.615934+0000","last_unstale":"2023-09-13T04:39:24.754566+0000","last_undegraded":"2023-09-13T04:39:24.754566+0000","last_fullsized":"2023-09-13T04:39:24.754566+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:51:18.994472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.706408+0000","last_change":"2023-09-13T04:37:58.594782+0000","last_active":"2023-09-13T04:39:24.706408+0000","last_peered":"2023-09-13T04:39:24.706408+0000","last_clean":"2023-09-13T04:39:24.706408+0000","last_became_active":"2023-09-13T04:37:58.594480+0000","last_became_peered":"2023-09-13T04:37:58.594480+0000","last_unstale":"2023-09-13T04:39:24.706408+0000","last_undegraded":"2023-09-13T04:39:24.706408+0000","last_fullsized":"2023-09-13T04:39:24.706408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:40:19.635672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.707565+0000","last_change":"2023-09-13T04:39:25.707565+0000","last_active":"2023-09-13T04:39:25.707565+0000","last_peered":"2023-09-13T04:39:25.707565+0000","last_clean":"2023-09-13T04:39:25.707565+0000","last_became_active":"2023-09-13T04:39:25.707099+0000","last_became_peered":"2023-09-13T04:39:25.707099+0000","last_unstale":"2023-09-13T04:39:25.707565+0000","last_undegraded":"2023-09-13T04:39:25.707565+0000","last_fullsized":"2023-09-13T04:39:25.707565+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:25:25.923715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582727+0000","last_change":"2023-09-13T04:38:03.618835+0000","last_active":"2023-09-13T04:39:24.582727+0000","last_peered":"2023-09-13T04:39:24.582727+0000","last_clean":"2023-09-13T04:39:24.582727+0000","last_became_active":"2023-09-13T04:38:03.618226+0000","last_became_peered":"2023-09-13T04:38:03.618226+0000","last_unstale":"2023-09-13T04:39:24.582727+0000","last_undegraded":"2023-09-13T04:39:24.582727+0000","last_fullsized":"2023-09-13T04:39:24.582727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:01:28.157490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":129,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:40:08.771978+0000","last_change":"2023-09-13T04:37:58.588545+0000","last_active":"2023-09-13T04:40:08.771978+0000","last_peered":"2023-09-13T04:40:08.771978+0000","last_clean":"2023-09-13T04:40:08.771978+0000","last_became_active":"2023-09-13T04:37:58.587995+0000","last_became_peered":"2023-09-13T04:37:58.587995+0000","last_unstale":"2023-09-13T04:40:08.771978+0000","last_undegraded":"2023-09-13T04:40:08.771978+0000","last_fullsized":"2023-09-13T04:40:08.771978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:40:08.797724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.585205+0000","last_change":"2023-09-13T04:37:59.249100+0000","last_active":"2023-09-13T04:39:24.585205+0000","last_peered":"2023-09-13T04:39:24.585205+0000","last_clean":"2023-09-13T04:39:24.585205+0000","last_became_active":"2023-09-13T04:37:59.248771+0000","last_became_peered":"2023-09-13T04:37:59.248771+0000","last_unstale":"2023-09-13T04:39:24.585205+0000","last_undegraded":"2023-09-13T04:39:24.585205+0000","last_fullsized":"2023-09-13T04:39:24.585205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:27:37.964595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.587764+0000","last_change":"2023-09-13T04:38:03.616988+0000","last_active":"2023-09-13T04:39:24.587764+0000","last_peered":"2023-09-13T04:39:24.587764+0000","last_clean":"2023-09-13T04:39:24.587764+0000","last_became_active":"2023-09-13T04:38:03.616524+0000","last_became_peered":"2023-09-13T04:38:03.616524+0000","last_unstale":"2023-09-13T04:39:24.587764+0000","last_undegraded":"2023-09-13T04:39:24.587764+0000","last_fullsized":"2023-09-13T04:39:24.587764+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:42:41.431115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754413+0000","last_change":"2023-09-13T04:38:24.613384+0000","last_active":"2023-09-13T04:39:24.754413+0000","last_peered":"2023-09-13T04:39:24.754413+0000","last_clean":"2023-09-13T04:39:24.754413+0000","last_became_active":"2023-09-13T04:38:24.599349+0000","last_became_peered":"2023-09-13T04:38:24.599349+0000","last_unstale":"2023-09-13T04:39:24.754413+0000","last_undegraded":"2023-09-13T04:39:24.754413+0000","last_fullsized":"2023-09-13T04:39:24.754413+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:23:23.479715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.706144+0000","last_change":"2023-09-13T04:37:59.252237+0000","last_active":"2023-09-13T04:39:24.706144+0000","last_peered":"2023-09-13T04:39:24.706144+0000","last_clean":"2023-09-13T04:39:24.706144+0000","last_became_active":"2023-09-13T04:37:59.251718+0000","last_became_peered":"2023-09-13T04:37:59.251718+0000","last_unstale":"2023-09-13T04:39:24.706144+0000","last_undegraded":"2023-09-13T04:39:24.706144+0000","last_fullsized":"2023-09-13T04:39:24.706144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:45:48.405226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.706124+0000","last_change":"2023-09-13T04:38:03.607096+0000","last_active":"2023-09-13T04:39:24.706124+0000","last_peered":"2023-09-13T04:39:24.706124+0000","last_clean":"2023-09-13T04:39:24.706124+0000","last_became_active":"2023-09-13T04:38:03.606656+0000","last_became_peered":"2023-09-13T04:38:03.606656+0000","last_unstale":"2023-09-13T04:39:24.706124+0000","last_undegraded":"2023-09-13T04:39:24.706124+0000","last_fullsized":"2023-09-13T04:39:24.706124+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:59:26.140360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755201+0000","last_change":"2023-09-13T04:37:58.591047+0000","last_active":"2023-09-13T04:39:24.755201+0000","last_peered":"2023-09-13T04:39:24.755201+0000","last_clean":"2023-09-13T04:39:24.755201+0000","last_became_active":"2023-09-13T04:37:58.590740+0000","last_became_peered":"2023-09-13T04:37:58.590740+0000","last_unstale":"2023-09-13T04:39:24.755201+0000","last_undegraded":"2023-09-13T04:39:24.755201+0000","last_fullsized":"2023-09-13T04:39:24.755201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:14:42.280416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755283+0000","last_change":"2023-09-13T04:37:59.249362+0000","last_active":"2023-09-13T04:39:24.755283+0000","last_peered":"2023-09-13T04:39:24.755283+0000","last_clean":"2023-09-13T04:39:24.755283+0000","last_became_active":"2023-09-13T04:37:59.249041+0000","last_became_peered":"2023-09-13T04:37:59.249041+0000","last_unstale":"2023-09-13T04:39:24.755283+0000","last_undegraded":"2023-09-13T04:39:24.755283+0000","last_fullsized":"2023-09-13T04:39:24.755283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:06:33.822157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.588524+0000","last_change":"2023-09-13T04:38:24.598370+0000","last_active":"2023-09-13T04:39:25.588524+0000","last_peered":"2023-09-13T04:39:25.588524+0000","last_clean":"2023-09-13T04:39:25.588524+0000","last_became_active":"2023-09-13T04:38:24.597931+0000","last_became_peered":"2023-09-13T04:38:24.597931+0000","last_unstale":"2023-09-13T04:39:25.588524+0000","last_undegraded":"2023-09-13T04:39:25.588524+0000","last_fullsized":"2023-09-13T04:39:25.588524+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:14:28.911329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.587714+0000","last_change":"2023-09-13T04:37:58.592496+0000","last_active":"2023-09-13T04:39:25.587714+0000","last_peered":"2023-09-13T04:39:25.587714+0000","last_clean":"2023-09-13T04:39:25.587714+0000","last_became_active":"2023-09-13T04:37:58.592106+0000","last_became_peered":"2023-09-13T04:37:58.592106+0000","last_unstale":"2023-09-13T04:39:25.587714+0000","last_undegraded":"2023-09-13T04:39:25.587714+0000","last_fullsized":"2023-09-13T04:39:25.587714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:28:55.618563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.592208+0000","last_change":"2023-09-13T04:37:59.250019+0000","last_active":"2023-09-13T04:39:25.592208+0000","last_peered":"2023-09-13T04:39:25.592208+0000","last_clean":"2023-09-13T04:39:25.592208+0000","last_became_active":"2023-09-13T04:37:59.249278+0000","last_became_peered":"2023-09-13T04:37:59.249278+0000","last_unstale":"2023-09-13T04:39:25.592208+0000","last_undegraded":"2023-09-13T04:39:25.592208+0000","last_fullsized":"2023-09-13T04:39:25.592208+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:15:31.217236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.705979+0000","last_change":"2023-09-13T04:38:03.614542+0000","last_active":"2023-09-13T04:39:25.705979+0000","last_peered":"2023-09-13T04:39:25.705979+0000","last_clean":"2023-09-13T04:39:25.705979+0000","last_became_active":"2023-09-13T04:38:03.614103+0000","last_became_peered":"2023-09-13T04:38:03.614103+0000","last_unstale":"2023-09-13T04:39:25.705979+0000","last_undegraded":"2023-09-13T04:39:25.705979+0000","last_fullsized":"2023-09-13T04:39:25.705979+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:06:48.426843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.595603+0000","last_change":"2023-09-13T04:37:58.591864+0000","last_active":"2023-09-13T04:39:25.595603+0000","last_peered":"2023-09-13T04:39:25.595603+0000","last_clean":"2023-09-13T04:39:25.595603+0000","last_became_active":"2023-09-13T04:37:58.591562+0000","last_became_peered":"2023-09-13T04:37:58.591562+0000","last_unstale":"2023-09-13T04:39:25.595603+0000","last_undegraded":"2023-09-13T04:39:25.595603+0000","last_fullsized":"2023-09-13T04:39:25.595603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:31:27.280489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.592296+0000","last_change":"2023-09-13T04:37:59.252600+0000","last_active":"2023-09-13T04:39:25.592296+0000","last_peered":"2023-09-13T04:39:25.592296+0000","last_clean":"2023-09-13T04:39:25.592296+0000","last_became_active":"2023-09-13T04:37:59.252306+0000","last_became_peered":"2023-09-13T04:37:59.252306+0000","last_unstale":"2023-09-13T04:39:25.592296+0000","last_undegraded":"2023-09-13T04:39:25.592296+0000","last_fullsized":"2023-09-13T04:39:25.592296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:18:55.686847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.587587+0000","last_change":"2023-09-13T04:38:03.742502+0000","last_active":"2023-09-13T04:39:24.587587+0000","last_peered":"2023-09-13T04:39:24.587587+0000","last_clean":"2023-09-13T04:39:24.587587+0000","last_became_active":"2023-09-13T04:38:03.742054+0000","last_became_peered":"2023-09-13T04:38:03.742054+0000","last_unstale":"2023-09-13T04:39:24.587587+0000","last_undegraded":"2023-09-13T04:39:24.587587+0000","last_fullsized":"2023-09-13T04:39:24.587587+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:55:54.601145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.709865+0000","last_change":"2023-09-13T04:37:59.245644+0000","last_active":"2023-09-13T04:39:25.709865+0000","last_peered":"2023-09-13T04:39:25.709865+0000","last_clean":"2023-09-13T04:39:25.709865+0000","last_became_active":"2023-09-13T04:37:59.245284+0000","last_became_peered":"2023-09-13T04:37:59.245284+0000","last_unstale":"2023-09-13T04:39:25.709865+0000","last_undegraded":"2023-09-13T04:39:25.709865+0000","last_fullsized":"2023-09-13T04:39:25.709865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:19:04.002414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":131,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:40:09.772897+0000","last_change":"2023-09-13T04:37:58.590249+0000","last_active":"2023-09-13T04:40:09.772897+0000","last_peered":"2023-09-13T04:40:09.772897+0000","last_clean":"2023-09-13T04:40:09.772897+0000","last_became_active":"2023-09-13T04:37:58.589521+0000","last_became_peered":"2023-09-13T04:37:58.589521+0000","last_unstale":"2023-09-13T04:40:09.772897+0000","last_undegraded":"2023-09-13T04:40:09.772897+0000","last_fullsized":"2023-09-13T04:40:09.772897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:11:57.542149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.595865+0000","last_change":"2023-09-13T04:38:24.598613+0000","last_active":"2023-09-13T04:39:25.595865+0000","last_peered":"2023-09-13T04:39:25.595865+0000","last_clean":"2023-09-13T04:39:25.595865+0000","last_became_active":"2023-09-13T04:38:24.598150+0000","last_became_peered":"2023-09-13T04:38:24.598150+0000","last_unstale":"2023-09-13T04:39:25.595865+0000","last_undegraded":"2023-09-13T04:39:25.595865+0000","last_fullsized":"2023-09-13T04:39:25.595865+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:02:29.894446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.854964+0000","last_change":"2023-09-13T04:37:59.243794+0000","last_active":"2023-09-13T04:39:25.854964+0000","last_peered":"2023-09-13T04:39:25.854964+0000","last_clean":"2023-09-13T04:39:25.854964+0000","last_became_active":"2023-09-13T04:37:59.243368+0000","last_became_peered":"2023-09-13T04:37:59.243368+0000","last_unstale":"2023-09-13T04:39:25.854964+0000","last_undegraded":"2023-09-13T04:39:25.854964+0000","last_fullsized":"2023-09-13T04:39:25.854964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:06:45.318737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.710051+0000","last_change":"2023-09-13T04:37:58.589195+0000","last_active":"2023-09-13T04:39:25.710051+0000","last_peered":"2023-09-13T04:39:25.710051+0000","last_clean":"2023-09-13T04:39:25.710051+0000","last_became_active":"2023-09-13T04:37:58.588905+0000","last_became_peered":"2023-09-13T04:37:58.588905+0000","last_unstale":"2023-09-13T04:39:25.710051+0000","last_undegraded":"2023-09-13T04:39:25.710051+0000","last_fullsized":"2023-09-13T04:39:25.710051+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:13:15.978891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":329,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.595369+0000","last_change":"2023-09-13T04:36:52.227264+0000","last_active":"2023-09-13T04:39:25.595369+0000","last_peered":"2023-09-13T04:39:25.595369+0000","last_clean":"2023-09-13T04:39:25.595369+0000","last_became_active":"2023-09-13T04:36:52.226773+0000","last_became_peered":"2023-09-13T04:36:52.226773+0000","last_unstale":"2023-09-13T04:39:25.595369+0000","last_undegraded":"2023-09-13T04:39:25.595369+0000","last_fullsized":"2023-09-13T04:39:25.595369+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:33:42.026499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:33:42.026499+0000","last_clean_scrub_stamp":"2023-09-13T04:33:42.026499+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:28:55.806587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.706548+0000","last_change":"2023-09-13T04:38:03.742790+0000","last_active":"2023-09-13T04:39:24.706548+0000","last_peered":"2023-09-13T04:39:24.706548+0000","last_clean":"2023-09-13T04:39:24.706548+0000","last_became_active":"2023-09-13T04:38:03.742338+0000","last_became_peered":"2023-09-13T04:38:03.742338+0000","last_unstale":"2023-09-13T04:39:24.706548+0000","last_undegraded":"2023-09-13T04:39:24.706548+0000","last_fullsized":"2023-09-13T04:39:24.706548+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:01:38.910623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.705623+0000","last_change":"2023-09-13T04:37:59.252669+0000","last_active":"2023-09-13T04:39:25.705623+0000","last_peered":"2023-09-13T04:39:25.705623+0000","last_clean":"2023-09-13T04:39:25.705623+0000","last_became_active":"2023-09-13T04:37:59.252223+0000","last_became_peered":"2023-09-13T04:37:59.252223+0000","last_unstale":"2023-09-13T04:39:25.705623+0000","last_undegraded":"2023-09-13T04:39:25.705623+0000","last_fullsized":"2023-09-13T04:39:25.705623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:57:21.090083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.595510+0000","last_change":"2023-09-13T04:37:58.590540+0000","last_active":"2023-09-13T04:39:25.595510+0000","last_peered":"2023-09-13T04:39:25.595510+0000","last_clean":"2023-09-13T04:39:25.595510+0000","last_became_active":"2023-09-13T04:37:58.590061+0000","last_became_peered":"2023-09-13T04:37:58.590061+0000","last_unstale":"2023-09-13T04:39:25.595510+0000","last_undegraded":"2023-09-13T04:39:25.595510+0000","last_fullsized":"2023-09-13T04:39:25.595510+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:57:17.133461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.624078+0000","last_change":"2023-09-13T04:38:03.614615+0000","last_active":"2023-09-13T04:39:25.624078+0000","last_peered":"2023-09-13T04:39:25.624078+0000","last_clean":"2023-09-13T04:39:25.624078+0000","last_became_active":"2023-09-13T04:38:03.614086+0000","last_became_peered":"2023-09-13T04:38:03.614086+0000","last_unstale":"2023-09-13T04:39:25.624078+0000","last_undegraded":"2023-09-13T04:39:25.624078+0000","last_fullsized":"2023-09-13T04:39:25.624078+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:25:35.525513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.588681+0000","last_change":"2023-09-13T04:37:58.592096+0000","last_active":"2023-09-13T04:39:25.588681+0000","last_peered":"2023-09-13T04:39:25.588681+0000","last_clean":"2023-09-13T04:39:25.588681+0000","last_became_active":"2023-09-13T04:37:58.591872+0000","last_became_peered":"2023-09-13T04:37:58.591872+0000","last_unstale":"2023-09-13T04:39:25.588681+0000","last_undegraded":"2023-09-13T04:39:25.588681+0000","last_fullsized":"2023-09-13T04:39:25.588681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:05:23.936105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.706127+0000","last_change":"2023-09-13T04:37:59.240759+0000","last_active":"2023-09-13T04:39:25.706127+0000","last_peered":"2023-09-13T04:39:25.706127+0000","last_clean":"2023-09-13T04:39:25.706127+0000","last_became_active":"2023-09-13T04:37:59.240414+0000","last_became_peered":"2023-09-13T04:37:59.240414+0000","last_unstale":"2023-09-13T04:39:25.706127+0000","last_undegraded":"2023-09-13T04:39:25.706127+0000","last_fullsized":"2023-09-13T04:39:25.706127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:37:20.301938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.854802+0000","last_change":"2023-09-13T04:38:03.742519+0000","last_active":"2023-09-13T04:39:25.854802+0000","last_peered":"2023-09-13T04:39:25.854802+0000","last_clean":"2023-09-13T04:39:25.854802+0000","last_became_active":"2023-09-13T04:38:03.742075+0000","last_became_peered":"2023-09-13T04:38:03.742075+0000","last_unstale":"2023-09-13T04:39:25.854802+0000","last_undegraded":"2023-09-13T04:39:25.854802+0000","last_fullsized":"2023-09-13T04:39:25.854802+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:14:33.155837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582560+0000","last_change":"2023-09-13T04:37:58.593891+0000","last_active":"2023-09-13T04:39:24.582560+0000","last_peered":"2023-09-13T04:39:24.582560+0000","last_clean":"2023-09-13T04:39:24.582560+0000","last_became_active":"2023-09-13T04:37:58.593497+0000","last_became_peered":"2023-09-13T04:37:58.593497+0000","last_unstale":"2023-09-13T04:39:24.582560+0000","last_undegraded":"2023-09-13T04:39:24.582560+0000","last_fullsized":"2023-09-13T04:39:24.582560+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:25:53.652601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754528+0000","last_change":"2023-09-13T04:37:59.247151+0000","last_active":"2023-09-13T04:39:24.754528+0000","last_peered":"2023-09-13T04:39:24.754528+0000","last_clean":"2023-09-13T04:39:24.754528+0000","last_became_active":"2023-09-13T04:37:59.246853+0000","last_became_peered":"2023-09-13T04:37:59.246853+0000","last_unstale":"2023-09-13T04:39:24.754528+0000","last_undegraded":"2023-09-13T04:39:24.754528+0000","last_fullsized":"2023-09-13T04:39:24.754528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:15:22.659365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582686+0000","last_change":"2023-09-13T04:38:03.606594+0000","last_active":"2023-09-13T04:39:24.582686+0000","last_peered":"2023-09-13T04:39:24.582686+0000","last_clean":"2023-09-13T04:39:24.582686+0000","last_became_active":"2023-09-13T04:38:03.606058+0000","last_became_peered":"2023-09-13T04:38:03.606058+0000","last_unstale":"2023-09-13T04:39:24.582686+0000","last_undegraded":"2023-09-13T04:39:24.582686+0000","last_fullsized":"2023-09-13T04:39:24.582686+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:57:39.237520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.709689+0000","last_change":"2023-09-13T04:37:58.588362+0000","last_active":"2023-09-13T04:39:25.709689+0000","last_peered":"2023-09-13T04:39:25.709689+0000","last_clean":"2023-09-13T04:39:25.709689+0000","last_became_active":"2023-09-13T04:37:58.588065+0000","last_became_peered":"2023-09-13T04:37:58.588065+0000","last_unstale":"2023-09-13T04:39:25.709689+0000","last_undegraded":"2023-09-13T04:39:25.709689+0000","last_fullsized":"2023-09-13T04:39:25.709689+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:32:04.255654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.856028+0000","last_change":"2023-09-13T04:37:59.251883+0000","last_active":"2023-09-13T04:39:25.856028+0000","last_peered":"2023-09-13T04:39:25.856028+0000","last_clean":"2023-09-13T04:39:25.856028+0000","last_became_active":"2023-09-13T04:37:59.251522+0000","last_became_peered":"2023-09-13T04:37:59.251522+0000","last_unstale":"2023-09-13T04:39:25.856028+0000","last_undegraded":"2023-09-13T04:39:25.856028+0000","last_fullsized":"2023-09-13T04:39:25.856028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:05:06.420799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583139+0000","last_change":"2023-09-13T04:38:03.742726+0000","last_active":"2023-09-13T04:39:24.583139+0000","last_peered":"2023-09-13T04:39:24.583139+0000","last_clean":"2023-09-13T04:39:24.583139+0000","last_became_active":"2023-09-13T04:38:03.742293+0000","last_became_peered":"2023-09-13T04:38:03.742293+0000","last_unstale":"2023-09-13T04:39:24.583139+0000","last_undegraded":"2023-09-13T04:39:24.583139+0000","last_fullsized":"2023-09-13T04:39:24.583139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:52:15.089764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.753734+0000","last_change":"2023-09-13T04:37:58.584923+0000","last_active":"2023-09-13T04:39:24.753734+0000","last_peered":"2023-09-13T04:39:24.753734+0000","last_clean":"2023-09-13T04:39:24.753734+0000","last_became_active":"2023-09-13T04:37:58.584320+0000","last_became_peered":"2023-09-13T04:37:58.584320+0000","last_unstale":"2023-09-13T04:39:24.753734+0000","last_undegraded":"2023-09-13T04:39:24.753734+0000","last_fullsized":"2023-09-13T04:39:24.753734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:16:53.670534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.588148+0000","last_change":"2023-09-13T04:37:59.243029+0000","last_active":"2023-09-13T04:39:25.588148+0000","last_peered":"2023-09-13T04:39:25.588148+0000","last_clean":"2023-09-13T04:39:25.588148+0000","last_became_active":"2023-09-13T04:37:59.242733+0000","last_became_peered":"2023-09-13T04:37:59.242733+0000","last_unstale":"2023-09-13T04:39:25.588148+0000","last_undegraded":"2023-09-13T04:39:25.588148+0000","last_fullsized":"2023-09-13T04:39:25.588148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:48:21.430026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582868+0000","last_change":"2023-09-13T04:38:03.616735+0000","last_active":"2023-09-13T04:39:24.582868+0000","last_peered":"2023-09-13T04:39:24.582868+0000","last_clean":"2023-09-13T04:39:24.582868+0000","last_became_active":"2023-09-13T04:38:03.616275+0000","last_became_peered":"2023-09-13T04:38:03.616275+0000","last_unstale":"2023-09-13T04:39:24.582868+0000","last_undegraded":"2023-09-13T04:39:24.582868+0000","last_fullsized":"2023-09-13T04:39:24.582868+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:12:27.267918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.855026+0000","last_change":"2023-09-13T04:37:58.595040+0000","last_active":"2023-09-13T04:39:25.855026+0000","last_peered":"2023-09-13T04:39:25.855026+0000","last_clean":"2023-09-13T04:39:25.855026+0000","last_became_active":"2023-09-13T04:37:58.591403+0000","last_became_peered":"2023-09-13T04:37:58.591403+0000","last_unstale":"2023-09-13T04:39:25.855026+0000","last_undegraded":"2023-09-13T04:39:25.855026+0000","last_fullsized":"2023-09-13T04:39:25.855026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:42:06.561066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.623940+0000","last_change":"2023-09-13T04:37:59.248837+0000","last_active":"2023-09-13T04:39:25.623940+0000","last_peered":"2023-09-13T04:39:25.623940+0000","last_clean":"2023-09-13T04:39:25.623940+0000","last_became_active":"2023-09-13T04:37:59.248573+0000","last_became_peered":"2023-09-13T04:37:59.248573+0000","last_unstale":"2023-09-13T04:39:25.623940+0000","last_undegraded":"2023-09-13T04:39:25.623940+0000","last_fullsized":"2023-09-13T04:39:25.623940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:56:22.980484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583094+0000","last_change":"2023-09-13T04:38:03.616370+0000","last_active":"2023-09-13T04:39:24.583094+0000","last_peered":"2023-09-13T04:39:24.583094+0000","last_clean":"2023-09-13T04:39:24.583094+0000","last_became_active":"2023-09-13T04:38:03.615996+0000","last_became_peered":"2023-09-13T04:38:03.615996+0000","last_unstale":"2023-09-13T04:39:24.583094+0000","last_undegraded":"2023-09-13T04:39:24.583094+0000","last_fullsized":"2023-09-13T04:39:24.583094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:56:59.924205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582878+0000","last_change":"2023-09-13T04:37:58.597902+0000","last_active":"2023-09-13T04:39:24.582878+0000","last_peered":"2023-09-13T04:39:24.582878+0000","last_clean":"2023-09-13T04:39:24.582878+0000","last_became_active":"2023-09-13T04:37:58.597578+0000","last_became_peered":"2023-09-13T04:37:58.597578+0000","last_unstale":"2023-09-13T04:39:24.582878+0000","last_undegraded":"2023-09-13T04:39:24.582878+0000","last_fullsized":"2023-09-13T04:39:24.582878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:10:09.931949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755492+0000","last_change":"2023-09-13T04:37:59.248924+0000","last_active":"2023-09-13T04:39:24.755492+0000","last_peered":"2023-09-13T04:39:24.755492+0000","last_clean":"2023-09-13T04:39:24.755492+0000","last_became_active":"2023-09-13T04:37:59.248483+0000","last_became_peered":"2023-09-13T04:37:59.248483+0000","last_unstale":"2023-09-13T04:39:24.755492+0000","last_undegraded":"2023-09-13T04:39:24.755492+0000","last_fullsized":"2023-09-13T04:39:24.755492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:49:56.078544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583128+0000","last_change":"2023-09-13T04:38:03.742260+0000","last_active":"2023-09-13T04:39:24.583128+0000","last_peered":"2023-09-13T04:39:24.583128+0000","last_clean":"2023-09-13T04:39:24.583128+0000","last_became_active":"2023-09-13T04:38:03.741893+0000","last_became_peered":"2023-09-13T04:38:03.741893+0000","last_unstale":"2023-09-13T04:39:24.583128+0000","last_undegraded":"2023-09-13T04:39:24.583128+0000","last_fullsized":"2023-09-13T04:39:24.583128+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:59:00.653545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755011+0000","last_change":"2023-09-13T04:37:58.586898+0000","last_active":"2023-09-13T04:39:24.755011+0000","last_peered":"2023-09-13T04:39:24.755011+0000","last_clean":"2023-09-13T04:39:24.755011+0000","last_became_active":"2023-09-13T04:37:58.586594+0000","last_became_peered":"2023-09-13T04:37:58.586594+0000","last_unstale":"2023-09-13T04:39:24.755011+0000","last_undegraded":"2023-09-13T04:39:24.755011+0000","last_fullsized":"2023-09-13T04:39:24.755011+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:29:43.718522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.595628+0000","last_change":"2023-09-13T04:37:59.244749+0000","last_active":"2023-09-13T04:39:25.595628+0000","last_peered":"2023-09-13T04:39:25.595628+0000","last_clean":"2023-09-13T04:39:25.595628+0000","last_became_active":"2023-09-13T04:37:59.244391+0000","last_became_peered":"2023-09-13T04:37:59.244391+0000","last_unstale":"2023-09-13T04:39:25.595628+0000","last_undegraded":"2023-09-13T04:39:25.595628+0000","last_fullsized":"2023-09-13T04:39:25.595628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:33:02.195729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.753833+0000","last_change":"2023-09-13T04:38:03.609766+0000","last_active":"2023-09-13T04:39:24.753833+0000","last_peered":"2023-09-13T04:39:24.753833+0000","last_clean":"2023-09-13T04:39:24.753833+0000","last_became_active":"2023-09-13T04:38:03.609432+0000","last_became_peered":"2023-09-13T04:38:03.609432+0000","last_unstale":"2023-09-13T04:39:24.753833+0000","last_undegraded":"2023-09-13T04:39:24.753833+0000","last_fullsized":"2023-09-13T04:39:24.753833+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:30:56.115483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583249+0000","last_change":"2023-09-13T04:37:58.589119+0000","last_active":"2023-09-13T04:39:24.583249+0000","last_peered":"2023-09-13T04:39:24.583249+0000","last_clean":"2023-09-13T04:39:24.583249+0000","last_became_active":"2023-09-13T04:37:58.588803+0000","last_became_peered":"2023-09-13T04:37:58.588803+0000","last_unstale":"2023-09-13T04:39:24.583249+0000","last_undegraded":"2023-09-13T04:39:24.583249+0000","last_fullsized":"2023-09-13T04:39:24.583249+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:51:20.888666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.585669+0000","last_change":"2023-09-13T04:37:59.243506+0000","last_active":"2023-09-13T04:39:24.585669+0000","last_peered":"2023-09-13T04:39:24.585669+0000","last_clean":"2023-09-13T04:39:24.585669+0000","last_became_active":"2023-09-13T04:37:59.243082+0000","last_became_peered":"2023-09-13T04:37:59.243082+0000","last_unstale":"2023-09-13T04:39:24.585669+0000","last_undegraded":"2023-09-13T04:39:24.585669+0000","last_fullsized":"2023-09-13T04:39:24.585669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:01:41.179286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754658+0000","last_change":"2023-09-13T04:38:03.609042+0000","last_active":"2023-09-13T04:39:24.754658+0000","last_peered":"2023-09-13T04:39:24.754658+0000","last_clean":"2023-09-13T04:39:24.754658+0000","last_became_active":"2023-09-13T04:38:03.608553+0000","last_became_peered":"2023-09-13T04:38:03.608553+0000","last_unstale":"2023-09-13T04:39:24.754658+0000","last_undegraded":"2023-09-13T04:39:24.754658+0000","last_fullsized":"2023-09-13T04:39:24.754658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:05:56.648475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.706558+0000","last_change":"2023-09-13T04:37:58.598056+0000","last_active":"2023-09-13T04:39:24.706558+0000","last_peered":"2023-09-13T04:39:24.706558+0000","last_clean":"2023-09-13T04:39:24.706558+0000","last_became_active":"2023-09-13T04:37:58.597724+0000","last_became_peered":"2023-09-13T04:37:58.597724+0000","last_unstale":"2023-09-13T04:39:24.706558+0000","last_undegraded":"2023-09-13T04:39:24.706558+0000","last_fullsized":"2023-09-13T04:39:24.706558+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:43:08.296866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.585419+0000","last_change":"2023-09-13T04:37:59.246505+0000","last_active":"2023-09-13T04:39:24.585419+0000","last_peered":"2023-09-13T04:39:24.585419+0000","last_clean":"2023-09-13T04:39:24.585419+0000","last_became_active":"2023-09-13T04:37:59.246024+0000","last_became_peered":"2023-09-13T04:37:59.246024+0000","last_unstale":"2023-09-13T04:39:24.585419+0000","last_undegraded":"2023-09-13T04:39:24.585419+0000","last_fullsized":"2023-09-13T04:39:24.585419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:36:58.008066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.705906+0000","last_change":"2023-09-13T04:38:03.605354+0000","last_active":"2023-09-13T04:39:24.705906+0000","last_peered":"2023-09-13T04:39:24.705906+0000","last_clean":"2023-09-13T04:39:24.705906+0000","last_became_active":"2023-09-13T04:38:03.604840+0000","last_became_peered":"2023-09-13T04:38:03.604840+0000","last_unstale":"2023-09-13T04:39:24.705906+0000","last_undegraded":"2023-09-13T04:39:24.705906+0000","last_fullsized":"2023-09-13T04:39:24.705906+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:46:29.867914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.587864+0000","last_change":"2023-09-13T04:37:58.589052+0000","last_active":"2023-09-13T04:39:24.587864+0000","last_peered":"2023-09-13T04:39:24.587864+0000","last_clean":"2023-09-13T04:39:24.587864+0000","last_became_active":"2023-09-13T04:37:58.588707+0000","last_became_peered":"2023-09-13T04:37:58.588707+0000","last_unstale":"2023-09-13T04:39:24.587864+0000","last_undegraded":"2023-09-13T04:39:24.587864+0000","last_fullsized":"2023-09-13T04:39:24.587864+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:25:23.966046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.585734+0000","last_change":"2023-09-13T04:37:59.241157+0000","last_active":"2023-09-13T04:39:24.585734+0000","last_peered":"2023-09-13T04:39:24.585734+0000","last_clean":"2023-09-13T04:39:24.585734+0000","last_became_active":"2023-09-13T04:37:59.240857+0000","last_became_peered":"2023-09-13T04:37:59.240857+0000","last_unstale":"2023-09-13T04:39:24.585734+0000","last_undegraded":"2023-09-13T04:39:24.585734+0000","last_fullsized":"2023-09-13T04:39:24.585734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:04:44.986908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.623983+0000","last_change":"2023-09-13T04:37:59.242447+0000","last_active":"2023-09-13T04:39:25.623983+0000","last_peered":"2023-09-13T04:39:25.623983+0000","last_clean":"2023-09-13T04:39:25.623983+0000","last_became_active":"2023-09-13T04:37:59.242130+0000","last_became_peered":"2023-09-13T04:37:59.242130+0000","last_unstale":"2023-09-13T04:39:25.623983+0000","last_undegraded":"2023-09-13T04:39:25.623983+0000","last_fullsized":"2023-09-13T04:39:25.623983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:12:12.844888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.854823+0000","last_change":"2023-09-13T04:38:24.595646+0000","last_active":"2023-09-13T04:39:25.854823+0000","last_peered":"2023-09-13T04:39:25.854823+0000","last_clean":"2023-09-13T04:39:25.854823+0000","last_became_active":"2023-09-13T04:38:24.595295+0000","last_became_peered":"2023-09-13T04:38:24.595295+0000","last_unstale":"2023-09-13T04:39:25.854823+0000","last_undegraded":"2023-09-13T04:39:25.854823+0000","last_fullsized":"2023-09-13T04:39:25.854823+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:24:27.174979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.854796+0000","last_change":"2023-09-13T04:37:59.245358+0000","last_active":"2023-09-13T04:39:25.854796+0000","last_peered":"2023-09-13T04:39:25.854796+0000","last_clean":"2023-09-13T04:39:25.854796+0000","last_became_active":"2023-09-13T04:37:59.245052+0000","last_became_peered":"2023-09-13T04:37:59.245052+0000","last_unstale":"2023-09-13T04:39:25.854796+0000","last_undegraded":"2023-09-13T04:39:25.854796+0000","last_fullsized":"2023-09-13T04:39:25.854796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:05:42.561716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.855825+0000","last_change":"2023-09-13T04:37:59.254053+0000","last_active":"2023-09-13T04:39:25.855825+0000","last_peered":"2023-09-13T04:39:25.855825+0000","last_clean":"2023-09-13T04:39:25.855825+0000","last_became_active":"2023-09-13T04:37:59.253544+0000","last_became_peered":"2023-09-13T04:37:59.253544+0000","last_unstale":"2023-09-13T04:39:25.855825+0000","last_undegraded":"2023-09-13T04:39:25.855825+0000","last_fullsized":"2023-09-13T04:39:25.855825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:18:54.689208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.706160+0000","last_change":"2023-09-13T04:37:59.253403+0000","last_active":"2023-09-13T04:39:25.706160+0000","last_peered":"2023-09-13T04:39:25.706160+0000","last_clean":"2023-09-13T04:39:25.706160+0000","last_became_active":"2023-09-13T04:37:59.253062+0000","last_became_peered":"2023-09-13T04:37:59.253062+0000","last_unstale":"2023-09-13T04:39:25.706160+0000","last_undegraded":"2023-09-13T04:39:25.706160+0000","last_fullsized":"2023-09-13T04:39:25.706160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:31:09.898886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.624253+0000","last_change":"2023-09-13T04:37:59.240916+0000","last_active":"2023-09-13T04:39:25.624253+0000","last_peered":"2023-09-13T04:39:25.624253+0000","last_clean":"2023-09-13T04:39:25.624253+0000","last_became_active":"2023-09-13T04:37:59.240605+0000","last_became_peered":"2023-09-13T04:37:59.240605+0000","last_unstale":"2023-09-13T04:39:25.624253+0000","last_undegraded":"2023-09-13T04:39:25.624253+0000","last_fullsized":"2023-09-13T04:39:25.624253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:15:48.241587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583359+0000","last_change":"2023-09-13T04:37:59.238569+0000","last_active":"2023-09-13T04:39:24.583359+0000","last_peered":"2023-09-13T04:39:24.583359+0000","last_clean":"2023-09-13T04:39:24.583359+0000","last_became_active":"2023-09-13T04:37:59.238255+0000","last_became_peered":"2023-09-13T04:37:59.238255+0000","last_unstale":"2023-09-13T04:39:24.583359+0000","last_undegraded":"2023-09-13T04:39:24.583359+0000","last_fullsized":"2023-09-13T04:39:24.583359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:19:03.433404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755665+0000","last_change":"2023-09-13T04:37:59.252243+0000","last_active":"2023-09-13T04:39:24.755665+0000","last_peered":"2023-09-13T04:39:24.755665+0000","last_clean":"2023-09-13T04:39:24.755665+0000","last_became_active":"2023-09-13T04:37:59.251902+0000","last_became_peered":"2023-09-13T04:37:59.251902+0000","last_unstale":"2023-09-13T04:39:24.755665+0000","last_undegraded":"2023-09-13T04:39:24.755665+0000","last_fullsized":"2023-09-13T04:39:24.755665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:35:57.395627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.709975+0000","last_change":"2023-09-13T04:37:59.245847+0000","last_active":"2023-09-13T04:39:25.709975+0000","last_peered":"2023-09-13T04:39:25.709975+0000","last_clean":"2023-09-13T04:39:25.709975+0000","last_became_active":"2023-09-13T04:37:59.245542+0000","last_became_peered":"2023-09-13T04:37:59.245542+0000","last_unstale":"2023-09-13T04:39:25.709975+0000","last_undegraded":"2023-09-13T04:39:25.709975+0000","last_fullsized":"2023-09-13T04:39:25.709975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:24:16.813333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754898+0000","last_change":"2023-09-13T04:37:59.246963+0000","last_active":"2023-09-13T04:39:24.754898+0000","last_peered":"2023-09-13T04:39:24.754898+0000","last_clean":"2023-09-13T04:39:24.754898+0000","last_became_active":"2023-09-13T04:37:59.246448+0000","last_became_peered":"2023-09-13T04:37:59.246448+0000","last_unstale":"2023-09-13T04:39:24.754898+0000","last_undegraded":"2023-09-13T04:39:24.754898+0000","last_fullsized":"2023-09-13T04:39:24.754898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:25:45.418778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.753993+0000","last_change":"2023-09-13T04:37:59.245399+0000","last_active":"2023-09-13T04:39:24.753993+0000","last_peered":"2023-09-13T04:39:24.753993+0000","last_clean":"2023-09-13T04:39:24.753993+0000","last_became_active":"2023-09-13T04:37:59.245060+0000","last_became_peered":"2023-09-13T04:37:59.245060+0000","last_unstale":"2023-09-13T04:39:24.753993+0000","last_undegraded":"2023-09-13T04:39:24.753993+0000","last_fullsized":"2023-09-13T04:39:24.753993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:25:41.078238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.587934+0000","last_change":"2023-09-13T04:37:59.244776+0000","last_active":"2023-09-13T04:39:24.587934+0000","last_peered":"2023-09-13T04:39:24.587934+0000","last_clean":"2023-09-13T04:39:24.587934+0000","last_became_active":"2023-09-13T04:37:59.244409+0000","last_became_peered":"2023-09-13T04:37:59.244409+0000","last_unstale":"2023-09-13T04:39:24.587934+0000","last_undegraded":"2023-09-13T04:39:24.587934+0000","last_fullsized":"2023-09-13T04:39:24.587934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:04:02.561414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754806+0000","last_change":"2023-09-13T04:37:59.248153+0000","last_active":"2023-09-13T04:39:24.754806+0000","last_peered":"2023-09-13T04:39:24.754806+0000","last_clean":"2023-09-13T04:39:24.754806+0000","last_became_active":"2023-09-13T04:37:59.247840+0000","last_became_peered":"2023-09-13T04:37:59.247840+0000","last_unstale":"2023-09-13T04:39:24.754806+0000","last_undegraded":"2023-09-13T04:39:24.754806+0000","last_fullsized":"2023-09-13T04:39:24.754806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:52:53.021219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7710,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":646,"num_read_kb":337,"num_write":131,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":22872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416423,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":509914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.625,"15min":0.61599999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":2.8420000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.54600000000000004}]},{"osd":1,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.66800000000000004,"15min":0.66100000000000003},"min":{"1min":0.46999999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92700000000000005,"5min":1.024,"15min":1.024},"last":0.625},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.42099999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80600000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.47299999999999998,"15min":0.46899999999999997},"min":{"1min":0.34799999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.68400000000000005}]},{"osd":3,"last update":"Wed Sep 13 04:39:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.74399999999999999}]},{"osd":4,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.68600000000000005,"15min":0.67800000000000005},"min":{"1min":0.53100000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.42899999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86699999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.71099999999999997}]},{"osd":5,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.64700000000000002,"15min":0.63100000000000001},"min":{"1min":0.45300000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.84799999999999998}]},{"osd":6,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70599999999999996,"15min":0.70399999999999996},"min":{"1min":0.46999999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.70799999999999996,"15min":0.69799999999999995},"min":{"1min":0.55900000000000005,"5min":0.498,"15min":0.498},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.878}]},{"osd":7,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.77500000000000002,"15min":0.76500000000000001},"min":{"1min":0.59599999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0229999999999999,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.50800000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94099999999999995,"5min":1.262,"15min":1.262},"last":0.83899999999999997}]},{"osd":8,"last update":"Wed Sep 13 04:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.74199999999999999,"15min":0.73899999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.88200000000000001,"15min":0.88600000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.0840000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.89400000000000002}]},{"osd":9,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.77400000000000002,"15min":0.76200000000000001},"min":{"1min":0.61499999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.121,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.995},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.82499999999999996,"15min":0.81100000000000005},"min":{"1min":0.66200000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1830000000000001,"5min":1.361,"15min":1.361},"last":0.97599999999999998}]},{"osd":10,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.78900000000000003,"15min":0.78600000000000003},"min":{"1min":0.56000000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.002,"5min":1.155,"15min":1.155},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.77300000000000002,"15min":0.76700000000000002},"min":{"1min":0.59599999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95699999999999996,"5min":1.282,"15min":1.282},"last":0.95799999999999996}]}]},{"osd":10,"up_from":71,"seq":304942678059,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1104951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.74399999999999999,"15min":0.76800000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.84499999999999997,"5min":4.0510000000000002,"15min":4.0510000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.77000000000000002,"15min":0.80300000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82899999999999996,"5min":4.1120000000000001,"15min":4.1120000000000001},"last":0.70899999999999996}]},{"osd":1,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.46000000000000002,"15min":0.45600000000000002},"min":{"1min":0.35399999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72799999999999998,"5min":0.751,"15min":0.751},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.47499999999999998,"15min":0.46999999999999997},"min":{"1min":0.34999999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63500000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.58399999999999996}]},{"osd":2,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.626,"15min":0.628},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.755,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80900000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.81200000000000006}]},{"osd":3,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.69399999999999995,"15min":0.66800000000000004},"min":{"1min":0.41799999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.754,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.73199999999999998}]},{"osd":4,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.371,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76300000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.879,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.62}]},{"osd":5,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69799999999999995,"15min":0.69299999999999995},"min":{"1min":0.505,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84999999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.45900000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.75}]},{"osd":6,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.81699999999999995,"15min":0.86799999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90100000000000002,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.85999999999999999,"15min":0.82599999999999996},"min":{"1min":0.54400000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96599999999999997,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":0.78800000000000003}]},{"osd":7,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.64500000000000002}]},{"osd":8,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77000000000000002},"min":{"1min":0.61199999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":0.998,"15min":0.998},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.78300000000000003,"15min":0.80100000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.99299999999999999,"5min":1.03,"15min":1.03},"last":0.94899999999999995}]},{"osd":9,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.86699999999999999,"15min":0.85099999999999998},"min":{"1min":0.65400000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0640000000000001,"5min":4.2679999999999998,"15min":4.2679999999999998},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.84999999999999998,"15min":0.85099999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.96799999999999997,"5min":4.3600000000000003,"15min":4.3600000000000003},"last":0.80000000000000004}]},{"osd":11,"last update":"Wed Sep 13 04:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.82799999999999996,"15min":0.82999999999999996},"min":{"1min":0.64100000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.80700000000000005,"15min":0.80600000000000005},"min":{"1min":0.65300000000000002,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.97099999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.92000000000000004}]}]},{"osd":9,"up_from":66,"seq":283467841583,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":510004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.58899999999999997,"15min":0.58699999999999997},"min":{"1min":0.38,"5min":0.316,"15min":0.316},"max":{"1min":1.5129999999999999,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.54400000000000004}]},{"osd":1,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.69499999999999995,"15min":0.70699999999999996},"min":{"1min":0.48199999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.036,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71499999999999997,"15min":0.73099999999999998},"min":{"1min":0.46100000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.70499999999999996}]},{"osd":2,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73599999999999999,"15min":0.752},"min":{"1min":0.48199999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.026,"5min":1.083,"15min":1.083},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73899999999999999,"15min":0.751},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.65500000000000003}]},{"osd":3,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68000000000000005,"15min":0.67900000000000005},"min":{"1min":0.42299999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.5549999999999999,"5min":2.258,"15min":2.258},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.5289999999999999,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":0.59399999999999997}]},{"osd":4,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.78100000000000003,"15min":0.80300000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.79600000000000004,"15min":0.80900000000000005},"min":{"1min":0.57299999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.68999999999999995}]},{"osd":5,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84399999999999997,"15min":0.86399999999999999},"min":{"1min":0.57199999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.296,"5min":1.339,"15min":1.339},"last":0.75},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79900000000000004,"15min":0.81299999999999994},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.071,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.72499999999999998}]},{"osd":6,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.72099999999999997}]},{"osd":7,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.88,"15min":0.90700000000000003},"min":{"1min":0.57099999999999995,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.876},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.88200000000000001,"15min":0.91800000000000004},"min":{"1min":0.621,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1060000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.81399999999999995}]},{"osd":8,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.90100000000000002,"15min":0.93500000000000005},"min":{"1min":0.71399999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.878,"5min":0.93500000000000005,"15min":0.96399999999999997},"min":{"1min":0.57999999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3380000000000001,"5min":1.857,"15min":1.857},"last":0.92100000000000004}]},{"osd":10,"last update":"Wed Sep 13 04:39:56 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89200000000000002,"15min":0.89800000000000002},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.879,"5min":0.89200000000000002,"15min":0.91000000000000003},"min":{"1min":0.70799999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1639999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.85399999999999998}]},{"osd":11,"last update":"Wed Sep 13 04:40:06 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.90700000000000003,"15min":0.91100000000000003},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.0249999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.86799999999999999,"15min":0.86399999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.89600000000000002}]}]},{"osd":8,"up_from":58,"seq":249108103220,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":509481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.441,"5min":0.317,"15min":0.317},"max":{"1min":0.89000000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58799999999999997,"15min":0.57099999999999995},"min":{"1min":0.39600000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98499999999999999,"5min":1.016,"15min":1.016},"last":0.42199999999999999}]},{"osd":1,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.59999999999999998,"15min":0.55400000000000005},"min":{"1min":0.53100000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.79700000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.752},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.61899999999999999,"15min":0.59199999999999997},"min":{"1min":0.53400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82799999999999996,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.63600000000000001}]},{"osd":2,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48099999999999998,"15min":0.47499999999999998},"min":{"1min":0.40200000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76300000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.439},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.49199999999999999,"15min":0.48699999999999999},"min":{"1min":0.373,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.73099999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.38}]},{"osd":3,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.90600000000000003,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.65500000000000003,"15min":0.64600000000000002},"min":{"1min":0.53600000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.57199999999999995}]},{"osd":4,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.63100000000000001,"15min":0.59799999999999998},"min":{"1min":0.497,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.90100000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.65900000000000003,"15min":0.627},"min":{"1min":0.55400000000000005,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.71499999999999997}]},{"osd":5,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58799999999999997,"15min":0.58199999999999996},"min":{"1min":0.437,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.89800000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.61199999999999999,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.54100000000000004}]},{"osd":6,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.56999999999999995,"5min":0.504,"15min":0.504},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.64900000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0229999999999999,"5min":1.077,"15min":1.077},"last":0.58999999999999997}]},{"osd":7,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72999999999999998,"15min":0.71599999999999997},"min":{"1min":0.47999999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.97199999999999998,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.73899999999999999,"15min":0.70299999999999996},"min":{"1min":0.66300000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0309999999999999,"5min":1.04,"15min":1.04},"last":0.76500000000000001}]},{"osd":9,"last update":"Wed Sep 13 04:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.78100000000000003,"15min":0.76500000000000001},"min":{"1min":0.66700000000000004,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0640000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.78500000000000003,"15min":0.76200000000000001},"min":{"1min":0.68500000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.62}]},{"osd":10,"last update":"Wed Sep 13 04:39:58 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.77200000000000002,"15min":0.74399999999999999},"min":{"1min":0.59499999999999997,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0089999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.78200000000000003,"15min":0.76300000000000001},"min":{"1min":0.64000000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.80600000000000005}]},{"osd":11,"last update":"Wed Sep 13 04:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.63800000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.87,"5min":0.83199999999999996,"15min":0.82599999999999996},"min":{"1min":0.66500000000000004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0329999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.73199999999999998}]}]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1642496,"data_stored":1101346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.46500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.746,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60499999999999998,"15min":0.60799999999999998},"min":{"1min":0.41599999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.40200000000000002}]},{"osd":1,"last update":"Wed Sep 13 04:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.433,"15min":0.41099999999999998},"min":{"1min":0.33900000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.56200000000000006,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.498,"5min":0.44800000000000001,"15min":0.42899999999999999},"min":{"1min":0.316,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.66400000000000003}]},{"osd":2,"last update":"Wed Sep 13 04:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60799999999999998,"15min":0.59299999999999997},"min":{"1min":0.46999999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78700000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61499999999999999,"15min":0.58299999999999996},"min":{"1min":0.47299999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76400000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.80300000000000005}]},{"osd":3,"last update":"Wed Sep 13 04:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71099999999999997,"15min":0.65700000000000003},"min":{"1min":0.496,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7410000000000001,"5min":7.0789999999999997,"15min":7.0789999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68000000000000005,"15min":0.67400000000000004},"min":{"1min":0.42999999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.54800000000000004}]},{"osd":4,"last update":"Wed Sep 13 04:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52100000000000002,"15min":0.51300000000000001},"min":{"1min":0.39100000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71799999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50600000000000001,"15min":0.51300000000000001},"min":{"1min":0.36099999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.78600000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.82299999999999995}]},{"osd":5,"last update":"Wed Sep 13 04:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64300000000000002,"15min":0.63100000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.80700000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.67900000000000005,"15min":0.65300000000000002},"min":{"1min":0.52200000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89600000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.89300000000000002}]},{"osd":6,"last update":"Wed Sep 13 04:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.751,"15min":0.73099999999999998},"min":{"1min":0.53300000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":3.496,"5min":3.496,"15min":3.496},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72799999999999998,"15min":0.72299999999999998},"min":{"1min":0.55600000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.7949999999999999,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.63500000000000001}]},{"osd":8,"last update":"Wed Sep 13 04:39:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.71699999999999997,"15min":0.70899999999999996},"min":{"1min":0.59399999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72099999999999997,"15min":0.71699999999999997},"min":{"1min":0.56599999999999995,"5min":0.437,"15min":0.437},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.85299999999999998}]},{"osd":9,"last update":"Wed Sep 13 04:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.79600000000000004,"15min":0.77500000000000002},"min":{"1min":0.64600000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.77900000000000003,"15min":0.76500000000000001},"min":{"1min":0.63900000000000001,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.91000000000000003}]},{"osd":10,"last update":"Wed Sep 13 04:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69899999999999995,"15min":0.69799999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.97099999999999997,"5min":0.999,"15min":0.999},"last":0.876},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.73599999999999999,"15min":0.72299999999999998},"min":{"1min":0.54300000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98399999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.95499999999999996}]},{"osd":11,"last update":"Wed Sep 13 04:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.80800000000000005,"15min":0.81000000000000005},"min":{"1min":0.58299999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92700000000000005,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.86599999999999999,"15min":0.86699999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.97699999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":509276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.51200000000000001,"15min":0.46800000000000003},"min":{"1min":0.33600000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":2.738,"5min":2.738,"15min":2.738},"last":0.45600000000000002},{"interface":"front","average":{"1min":1.0920000000000001,"5min":0.625,"15min":0.53100000000000003},"min":{"1min":0.41599999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":8.5749999999999993,"5min":8.5749999999999993,"15min":8.5749999999999993},"last":0.504}]},{"osd":1,"last update":"Wed Sep 13 04:39:20 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":0.74099999999999999,"15min":0.64500000000000002},"min":{"1min":0.52700000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":10.374000000000001,"5min":10.374000000000001,"15min":10.374000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.72999999999999998,"15min":0.68600000000000005},"min":{"1min":0.51500000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.746}]},{"osd":2,"last update":"Wed Sep 13 04:39:20 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":0.78100000000000003,"15min":0.71699999999999997},"min":{"1min":0.505,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":8.4740000000000002,"5min":8.4740000000000002,"15min":8.4740000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.69799999999999995,"15min":0.68400000000000005},"min":{"1min":0.55900000000000005,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.52600000000000002}]},{"osd":3,"last update":"Wed Sep 13 04:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.69899999999999995,"15min":0.65400000000000003},"min":{"1min":0.46000000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.331,"5min":2.331,"15min":2.331},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.67800000000000005,"15min":0.63500000000000001},"min":{"1min":0.45000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.585,"5min":2.585,"15min":2.585},"last":0.48099999999999998}]},{"osd":4,"last update":"Wed Sep 13 04:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.78000000000000003,"15min":0.752},"min":{"1min":0.60699999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.2809999999999999,"5min":0.85999999999999999,"15min":0.80400000000000005},"min":{"1min":0.58199999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":8.3350000000000009,"5min":8.3350000000000009,"15min":8.3350000000000009},"last":0.85199999999999998}]},{"osd":5,"last update":"Wed Sep 13 04:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.82699999999999996,"15min":0.79900000000000004},"min":{"1min":0.52500000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.288,"5min":0.88600000000000001,"15min":0.83999999999999997},"min":{"1min":0.54100000000000004,"5min":0.379,"15min":0.379},"max":{"1min":8.5060000000000002,"5min":8.5060000000000002,"15min":8.5060000000000002},"last":0.79300000000000004}]},{"osd":7,"last update":"Wed Sep 13 04:39:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.87,"15min":0.83399999999999996},"min":{"1min":0.61199999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90300000000000002,"15min":0.82299999999999995},"min":{"1min":0.59599999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.702,"5min":8.3970000000000002,"15min":8.3970000000000002},"last":0.89000000000000001}]},{"osd":8,"last update":"Wed Sep 13 04:40:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.80900000000000005,"15min":0.82099999999999995},"min":{"1min":0.625,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.256,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.92200000000000004,"15min":0.92500000000000004},"min":{"1min":0.64700000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3620000000000001,"5min":1.446,"15min":1.446},"last":0.871}]},{"osd":9,"last update":"Wed Sep 13 04:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.876,"15min":0.88600000000000001},"min":{"1min":0.58299999999999996,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.026,"15min":1.0329999999999999},"min":{"1min":0.72399999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.6960000000000002,"5min":2.6960000000000002,"15min":2.6960000000000002},"last":0.94399999999999995}]},{"osd":10,"last update":"Wed Sep 13 04:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.98999999999999999,"15min":0.99199999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.96699999999999997,"15min":0.89900000000000002},"min":{"1min":0.71299999999999997,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.732,"5min":8.4380000000000006,"15min":8.4380000000000006},"last":0.92600000000000005}]},{"osd":11,"last update":"Wed Sep 13 04:40:06 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":1.272,"15min":1.3380000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.4139999999999999,"5min":8.5410000000000004,"15min":8.5410000000000004},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.93200000000000005,"15min":0.94399999999999995},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.3080000000000001,"5min":1.341,"15min":1.341},"last":0.83699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":513028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.621,"15min":0.55900000000000005},"min":{"1min":0.47199999999999998,"5min":0.307,"15min":0.307},"max":{"1min":4.1150000000000002,"5min":4.1150000000000002,"15min":4.1150000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":1.165,"5min":0.71499999999999997,"15min":0.60099999999999998},"min":{"1min":0.45100000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":12.896000000000001,"5min":12.896000000000001,"15min":12.896000000000001},"last":0.42899999999999999}]},{"osd":2,"last update":"Wed Sep 13 04:39:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.60899999999999999,"15min":0.55500000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88300000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61799999999999999,"15min":0.54200000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.877,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.66200000000000003}]},{"osd":3,"last update":"Wed Sep 13 04:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.58999999999999997},"min":{"1min":0.41699999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.64500000000000002,"15min":0.61699999999999999},"min":{"1min":0.50800000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.626}]},{"osd":4,"last update":"Wed Sep 13 04:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48299999999999998,"15min":0.46000000000000002},"min":{"1min":0.378,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63700000000000001,"5min":0.878,"15min":0.878},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.48899999999999999,"15min":0.45900000000000002},"min":{"1min":0.34699999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.82299999999999995,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.51400000000000001}]},{"osd":5,"last update":"Wed Sep 13 04:39:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.625,"15min":0.57999999999999996},"min":{"1min":0.378,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85299999999999998,"5min":1.3,"15min":1.3},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.68400000000000005,"15min":0.60599999999999998},"min":{"1min":0.46400000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.0089999999999999,"5min":1.264,"15min":1.264},"last":0.70999999999999996}]},{"osd":6,"last update":"Wed Sep 13 04:39:22 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":0.86799999999999999,"15min":0.75},"min":{"1min":0.60199999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":12.640000000000001,"5min":12.640000000000001,"15min":12.640000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.3160000000000001,"5min":0.83299999999999996,"15min":0.73999999999999999},"min":{"1min":0.56000000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":12.675000000000001,"5min":12.675000000000001,"15min":12.675000000000001},"last":0.73199999999999998}]},{"osd":7,"last update":"Wed Sep 13 04:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.66000000000000003,"15min":0.60499999999999998},"min":{"1min":0.50700000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.1399999999999999,"5min":1.347,"15min":1.347},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.70999999999999996,"15min":0.66000000000000003},"min":{"1min":0.42699999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.96899999999999997,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.81499999999999995}]},{"osd":8,"last update":"Wed Sep 13 04:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.75700000000000001,"15min":0.74199999999999999},"min":{"1min":0.48099999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0980000000000001,"5min":1.393,"15min":1.393},"last":0.872},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.73999999999999999,"15min":0.72399999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.032,"5min":1.224,"15min":1.224},"last":0.78000000000000003}]},{"osd":9,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.878,"15min":0.86699999999999999},"min":{"1min":0.64700000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.8250000000000002,"5min":2.8250000000000002,"15min":2.8250000000000002},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.879,"15min":0.871},"min":{"1min":0.66600000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":2.9569999999999999,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":0.84799999999999998}]},{"osd":10,"last update":"Wed Sep 13 04:39:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.81799999999999995,"15min":0.82199999999999995},"min":{"1min":0.51500000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.038,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.85399999999999998,"15min":0.85499999999999998},"min":{"1min":0.54700000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0780000000000001,"5min":1.504,"15min":1.504},"last":0.85999999999999999}]},{"osd":11,"last update":"Wed Sep 13 04:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.88800000000000001,"15min":0.88400000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0660000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.91700000000000004,"15min":0.91900000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.125,"5min":1.129,"15min":1.129},"last":0.91900000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":510664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 13 04:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.748,"15min":0.61199999999999999},"min":{"1min":0.45700000000000002,"5min":0.33100000000000002,"15min":0.29899999999999999},"max":{"1min":1.125,"5min":6.2359999999999998,"15min":6.2359999999999998},"last":1.01},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64900000000000002,"15min":0.57299999999999995},"min":{"1min":0.40600000000000003,"5min":0.38800000000000001,"15min":0.31900000000000001},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":1.0329999999999999}]},{"osd":2,"last update":"Wed Sep 13 04:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65900000000000003,"15min":0.61099999999999999},"min":{"1min":0.52300000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96499999999999997,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.70899999999999996,"15min":0.627},"min":{"1min":0.47999999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed Sep 13 04:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.88500000000000001,"5min":1.25,"15min":1.25},"last":0.628},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.90300000000000002,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.89800000000000002}]},{"osd":4,"last update":"Wed Sep 13 04:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.751,"15min":0.64400000000000002},"min":{"1min":0.48399999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.052},{"interface":"front","average":{"1min":0.748,"5min":0.70599999999999996,"15min":0.64000000000000001},"min":{"1min":0.46899999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.85099999999999998}]},{"osd":5,"last update":"Wed Sep 13 04:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69699999999999995,"15min":0.67500000000000004},"min":{"1min":0.48799999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89100000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.878,"5min":0.78100000000000003,"15min":0.70699999999999996},"min":{"1min":0.63500000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.2130000000000001,"5min":1.444,"15min":1.444},"last":1.155}]},{"osd":6,"last update":"Wed Sep 13 04:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.70899999999999996,"15min":0.70199999999999996},"min":{"1min":0.50700000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.751,"15min":0.73299999999999998},"min":{"1min":0.45100000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.65400000000000003}]},{"osd":7,"last update":"Wed Sep 13 04:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.82999999999999996,"15min":0.83699999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1459999999999999,"5min":1.333,"15min":1.333},"last":0.873},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.93500000000000005,"15min":0.94299999999999995},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.258,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.075}]},{"osd":8,"last update":"Wed Sep 13 04:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.91000000000000003,"15min":0.86199999999999999},"min":{"1min":0.61599999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0609999999999999,"5min":6.2089999999999996,"15min":6.2089999999999996},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.93500000000000005,"15min":0.84099999999999997},"min":{"1min":0.73199999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2350000000000001,"5min":6.1699999999999999,"15min":6.1699999999999999},"last":1.135}]},{"osd":9,"last update":"Wed Sep 13 04:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.90800000000000003,"15min":0.94099999999999995},"min":{"1min":0.56799999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.94499999999999995,"15min":0.93700000000000006},"min":{"1min":0.69399999999999995,"5min":0.433,"15min":0.433},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.96399999999999997}]},{"osd":10,"last update":"Wed Sep 13 04:39:51 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.99399999999999999,"15min":0.98599999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2989999999999999,"5min":1.591,"15min":1.591},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":1.0449999999999999,"15min":1.119},"min":{"1min":0.623,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1950000000000001,"5min":6.1859999999999999,"15min":6.1859999999999999},"last":0.93500000000000005}]},{"osd":11,"last update":"Wed Sep 13 04:39:16 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":0.99099999999999999,"15min":0.98399999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.175},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.86099999999999999,"15min":0.85299999999999998},"min":{"1min":0.71499999999999997,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.99299999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":514344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55600000000000005,"15min":0.53000000000000003},"min":{"1min":0.34200000000000003,"5min":0.29399999999999998,"15min":0.28599999999999998},"max":{"1min":0.86099999999999999,"5min":1.74,"15min":1.74},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55400000000000005,"15min":0.53000000000000003},"min":{"1min":0.38700000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.77400000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.53100000000000003}]},{"osd":1,"last update":"Wed Sep 13 04:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60799999999999998,"15min":0.52000000000000002},"min":{"1min":0.42399999999999999,"5min":0.34300000000000003,"15min":0.30599999999999999},"max":{"1min":0.91300000000000003,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.61099999999999999,"15min":0.53000000000000003},"min":{"1min":0.44600000000000001,"5min":0.36599999999999999,"15min":0.34300000000000003},"max":{"1min":0.80800000000000005,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.70199999999999996}]},{"osd":3,"last update":"Wed Sep 13 04:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.61899999999999999,"15min":0.59199999999999997},"min":{"1min":0.45700000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.623,"15min":0.60999999999999999},"min":{"1min":0.48899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.74299999999999999}]},{"osd":4,"last update":"Wed Sep 13 04:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.54500000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.624},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.61099999999999999}]},{"osd":5,"last update":"Wed Sep 13 04:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46999999999999997,"15min":0.49299999999999999},"min":{"1min":0.376,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.621,"5min":1.796,"15min":1.796},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34200000000000003,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.94799999999999995,"5min":1.546,"15min":1.546},"last":0.83299999999999996}]},{"osd":6,"last update":"Wed Sep 13 04:39:19 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.753,"15min":0.71899999999999997},"min":{"1min":0.57699999999999996,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":4.0730000000000004,"5min":4.0730000000000004,"15min":4.0730000000000004},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.56299999999999994,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.65700000000000003}]},{"osd":7,"last update":"Wed Sep 13 04:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.77900000000000003,"15min":0.77100000000000002},"min":{"1min":0.56399999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76800000000000002,"15min":0.76200000000000001},"min":{"1min":0.63400000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.86599999999999999}]},{"osd":8,"last update":"Wed Sep 13 04:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61799999999999999,"15min":0.62},"min":{"1min":0.40300000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.82399999999999995,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.66700000000000004,"15min":0.65300000000000002},"min":{"1min":0.51200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.90400000000000003,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.79600000000000004}]},{"osd":9,"last update":"Wed Sep 13 04:39:31 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.755,"15min":0.73099999999999998},"min":{"1min":0.65200000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.75600000000000001,"15min":0.72599999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":0.78400000000000003}]},{"osd":10,"last update":"Wed Sep 13 04:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71999999999999997,"15min":0.69599999999999995},"min":{"1min":0.63700000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.78300000000000003,"15min":0.745},"min":{"1min":0.65600000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.93000000000000005}]},{"osd":11,"last update":"Wed Sep 13 04:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.76600000000000001,"15min":0.76800000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.94199999999999995,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.877,"15min":0.88300000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.89600000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":509470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.45500000000000002,"15min":0.39900000000000002},"min":{"1min":0.30399999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.52900000000000003,"15min":0.45600000000000002},"min":{"1min":0.34399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.5940000000000003,"5min":4.5940000000000003,"15min":4.5940000000000003},"last":0.44600000000000001}]},{"osd":1,"last update":"Wed Sep 13 04:39:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.64000000000000001,"15min":0.58699999999999997},"min":{"1min":0.42399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.623},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.68000000000000005,"15min":0.58999999999999997},"min":{"1min":0.50600000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.83299999999999996}]},{"osd":2,"last update":"Wed Sep 13 04:39:13 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.72799999999999998,"15min":0.64700000000000002},"min":{"1min":0.47699999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":4.3920000000000003,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.65900000000000003,"15min":0.626},"min":{"1min":0.46100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97999999999999998,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.58899999999999997}]},{"osd":4,"last update":"Wed Sep 13 04:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.70399999999999996,"15min":0.621},"min":{"1min":0.47699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0649999999999999,"5min":1.147,"15min":1.147},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69599999999999995,"15min":0.626},"min":{"1min":0.498,"5min":0.439,"15min":0.439},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.66600000000000004}]},{"osd":5,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72299999999999998,"15min":0.66100000000000003},"min":{"1min":0.52000000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0109999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.077,"5min":0.86399999999999999,"15min":0.71299999999999997},"min":{"1min":0.48399999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":6.7030000000000003,"5min":6.7030000000000003,"15min":6.7030000000000003},"last":0.60099999999999998}]},{"osd":6,"last update":"Wed Sep 13 04:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.79300000000000004,"15min":0.72099999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":4.5529999999999999,"5min":4.5529999999999999,"15min":4.5529999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.78000000000000003,"15min":0.749},"min":{"1min":0.52200000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.78500000000000003}]},{"osd":7,"last update":"Wed Sep 13 04:39:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.84899999999999998,"15min":0.85899999999999999},"min":{"1min":0.56899999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.876},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.88,"15min":0.85799999999999998},"min":{"1min":0.59899999999999998,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.153,"5min":4.4699999999999998,"15min":4.4699999999999998},"last":0.67700000000000005}]},{"osd":8,"last update":"Wed Sep 13 04:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.875}]},{"osd":9,"last update":"Wed Sep 13 04:39:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":0.96299999999999997,"15min":0.96899999999999997},"min":{"1min":0.58799999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":4.4329999999999998,"5min":4.4329999999999998,"15min":4.4329999999999998},"last":0.753},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.82499999999999996,"15min":0.84299999999999997},"min":{"1min":0.51200000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.51000000000000001}]},{"osd":10,"last update":"Wed Sep 13 04:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.89100000000000001,"15min":0.89600000000000002},"min":{"1min":0.71599999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1499999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.96699999999999997,"15min":0.95899999999999996},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.2370000000000001,"5min":4.5099999999999998,"15min":4.5099999999999998},"last":0.72099999999999997}]},{"osd":11,"last update":"Wed Sep 13 04:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.879,"15min":0.88900000000000001},"min":{"1min":0.71999999999999997,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.95699999999999996,"5min":1.27,"15min":1.27},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.0620000000000001,"5min":0.98199999999999998,"15min":0.96899999999999997},"min":{"1min":0.80000000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":4.2530000000000001,"5min":4.2530000000000001,"15min":4.2530000000000001},"last":0.70699999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":509830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.56599999999999995,"15min":0.54400000000000004},"min":{"1min":0.43099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.753},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76500000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.68999999999999995}]},{"osd":1,"last update":"Wed Sep 13 04:39:42 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.41899999999999998,"15min":0.36199999999999999},"min":{"1min":0.34999999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.58099999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.433},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.434,"15min":0.38},"min":{"1min":0.36899999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.71499999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.48199999999999998}]},{"osd":2,"last update":"Wed Sep 13 04:39:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.61099999999999999,"15min":0.57099999999999995},"min":{"1min":0.39700000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81299999999999994,"5min":1.631,"15min":1.631},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.61299999999999999,"15min":0.55300000000000005},"min":{"1min":0.439,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76400000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.59799999999999998}]},{"osd":3,"last update":"Wed Sep 13 04:39:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.63500000000000001,"15min":0.57899999999999996},"min":{"1min":0.47799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.61699999999999999,"15min":0.58499999999999996},"min":{"1min":0.47899999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0009999999999999,"5min":1.115,"15min":1.115},"last":0.58299999999999996}]},{"osd":5,"last update":"Wed Sep 13 04:40:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64200000000000002,"15min":0.59899999999999998},"min":{"1min":0.51900000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81499999999999995,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.753},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63800000000000001,"15min":0.61599999999999999},"min":{"1min":0.42599999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.51400000000000001}]},{"osd":6,"last update":"Wed Sep 13 04:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67700000000000005,"15min":0.68300000000000005},"min":{"1min":0.52000000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70699999999999996,"15min":0.69699999999999995},"min":{"1min":0.59499999999999997,"5min":0.443,"15min":0.443},"max":{"1min":0.89000000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.79800000000000004}]},{"osd":7,"last update":"Wed Sep 13 04:39:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54700000000000004,"15min":0.52700000000000002},"min":{"1min":0.39500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63400000000000001,"15min":0.60399999999999998},"min":{"1min":0.47799999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.77800000000000002}]},{"osd":8,"last update":"Wed Sep 13 04:40:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.753,"15min":0.70699999999999996},"min":{"1min":0.65300000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93400000000000005,"5min":1.042,"15min":1.042},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.77500000000000002,"15min":0.76600000000000001},"min":{"1min":0.59799999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.97799999999999998,"5min":1.095,"15min":1.095},"last":0.81100000000000005}]},{"osd":9,"last update":"Wed Sep 13 04:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80400000000000005,"15min":0.81100000000000005},"min":{"1min":0.65000000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.03,"5min":1.141,"15min":1.141},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.624,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.94599999999999995,"5min":1.056,"15min":1.056},"last":0.73499999999999999}]},{"osd":10,"last update":"Wed Sep 13 04:39:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.754,"15min":0.75800000000000001},"min":{"1min":0.55100000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78900000000000003,"15min":0.79200000000000004},"min":{"1min":0.63900000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.88300000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.82699999999999996}]},{"osd":11,"last update":"Wed Sep 13 04:40:00 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.61399999999999999,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.71999999999999997,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":0.97399999999999998,"5min":0.997,"15min":0.997},"last":0.85399999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1654784,"data_stored":1101030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.438,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69299999999999995,"5min":0.754,"15min":0.754},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.52300000000000002},"min":{"1min":0.45500000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.76300000000000001,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.434}]},{"osd":1,"last update":"Wed Sep 13 04:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57099999999999995,"15min":0.53400000000000003},"min":{"1min":0.373,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.745,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.59899999999999998,"15min":0.54300000000000004},"min":{"1min":0.42399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77700000000000002,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.57299999999999995}]},{"osd":2,"last update":"Wed Sep 13 04:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42399999999999999,"15min":0.376},"min":{"1min":0.27900000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.59699999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.435,"15min":0.39100000000000001},"min":{"1min":0.315,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.55300000000000005,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.47399999999999998}]},{"osd":3,"last update":"Wed Sep 13 04:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.38600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.70899999999999996,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.67100000000000004,"15min":0.82999999999999996},"min":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.745,"5min":8.0009999999999994,"15min":8.0009999999999994},"last":0.66100000000000003}]},{"osd":4,"last update":"Wed Sep 13 04:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.628,"15min":0.57799999999999996},"min":{"1min":0.44700000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83699999999999997,"5min":1.599,"15min":1.599},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.627,"15min":0.57199999999999995},"min":{"1min":0.49199999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.82499999999999996,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.58499999999999996}]},{"osd":6,"last update":"Wed Sep 13 04:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.65600000000000003,"15min":0.65000000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.65000000000000002,"15min":0.63800000000000001},"min":{"1min":0.54700000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85899999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.55600000000000005}]},{"osd":7,"last update":"Wed Sep 13 04:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.46600000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94399999999999995,"5min":1.488,"15min":1.488},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69199999999999995,"15min":0.67300000000000004},"min":{"1min":0.58799999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.876,"5min":1.51,"15min":1.51},"last":0.59999999999999998}]},{"osd":8,"last update":"Wed Sep 13 04:40:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61199999999999999,"15min":0.59399999999999997},"min":{"1min":0.46899999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.63700000000000001,"15min":0.61299999999999999},"min":{"1min":0.5,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.68200000000000005}]},{"osd":9,"last update":"Wed Sep 13 04:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.50800000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.91400000000000003,"5min":1.008,"15min":1.008},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.74299999999999999,"15min":0.73999999999999999},"min":{"1min":0.60899999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.92900000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.61599999999999999}]},{"osd":10,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.74299999999999999,"15min":0.72499999999999998},"min":{"1min":0.51100000000000001,"5min":0.496,"15min":0.496},"max":{"1min":0.97199999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76200000000000001,"15min":0.75},"min":{"1min":0.63500000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.91900000000000004,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.749}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-13T04:40:13.571 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-09-13T04:40:13.572 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-13T04:40:13.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.807+0000 7f028b7ff700 1 -- 172.21.15.2:0/3150009045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0284101ee0 msgr2=0x7f0284102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:13.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.807+0000 7f028b7ff700 1 --2- 172.21.15.2:0/3150009045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0284101ee0 0x7f0284102340 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f02740099c0 tx=0x7f0274009cd0 comp rx=0 tx=0).stop 2023-09-13T04:40:13.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.807+0000 7f028b7ff700 1 -- 172.21.15.2:0/3150009045 shutdown_connections 2023-09-13T04:40:13.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.807+0000 7f028b7ff700 1 --2- 172.21.15.2:0/3150009045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0284102880 0x7f0284109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:13.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.807+0000 7f028b7ff700 1 --2- 172.21.15.2:0/3150009045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0284100cf0 0x7f0284101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:13.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.807+0000 7f028b7ff700 1 --2- 172.21.15.2:0/3150009045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0284101ee0 0x7f0284102340 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:13.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.807+0000 7f028b7ff700 1 -- 172.21.15.2:0/3150009045 >> 172.21.15.2:0/3150009045 conn(0x7f02840fc220 msgr2=0x7f02840fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:13.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.807+0000 7f028b7ff700 1 -- 172.21.15.2:0/3150009045 shutdown_connections 2023-09-13T04:40:13.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.807+0000 7f028b7ff700 1 -- 172.21.15.2:0/3150009045 wait complete. 2023-09-13T04:40:13.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.808+0000 7f028b7ff700 1 Processor -- start 2023-09-13T04:40:13.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.808+0000 7f028b7ff700 1 -- start start 2023-09-13T04:40:13.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.808+0000 7f028b7ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0284100cf0 0x7f0284074a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:13.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.808+0000 7f028b7ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0284101ee0 0x7f02840730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:13.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.808+0000 7f028b7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0284102880 0x7f02840735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:13.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.808+0000 7f028b7ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0284073d80 con 0x7f0284101ee0 2023-09-13T04:40:13.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.808+0000 7f028b7ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0284073ef0 con 0x7f0284100cf0 2023-09-13T04:40:13.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.808+0000 7f028b7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02841a0a30 con 0x7f0284102880 2023-09-13T04:40:13.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.809+0000 7f0288d9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0284101ee0 0x7f02840730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:13.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.809+0000 7f0288d9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0284101ee0 0x7f02840730a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47772/0 (socket says 172.21.15.2:47772) 2023-09-13T04:40:13.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.809+0000 7f0288d9a700 1 -- 172.21.15.2:0/65313756 learned_addr learned my addr 172.21.15.2:0/65313756 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:13.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.809+0000 7f028959b700 1 --2- 172.21.15.2:0/65313756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0284100cf0 0x7f0284074a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:13.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.809+0000 7f0289d9c700 1 --2- 172.21.15.2:0/65313756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0284102880 0x7f02840735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:13.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.809+0000 7f0288d9a700 1 -- 172.21.15.2:0/65313756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0284102880 msgr2=0x7f02840735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:13.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.809+0000 7f0288d9a700 1 --2- 172.21.15.2:0/65313756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0284102880 0x7f02840735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:13.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.809+0000 7f0288d9a700 1 -- 172.21.15.2:0/65313756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0284100cf0 msgr2=0x7f0284074a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:13.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.809+0000 7f0288d9a700 1 --2- 172.21.15.2:0/65313756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0284100cf0 0x7f0284074a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:13.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.809+0000 7f0288d9a700 1 -- 172.21.15.2:0/65313756 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02740095f0 con 0x7f0284101ee0 2023-09-13T04:40:13.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.809+0000 7f0288d9a700 1 --2- 172.21.15.2:0/65313756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0284101ee0 0x7f02840730a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0274009990 tx=0x7f027400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:13.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.809+0000 7f027a7fc700 1 -- 172.21.15.2:0/65313756 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027401d070 con 0x7f0284101ee0 2023-09-13T04:40:13.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.810+0000 7f028b7ff700 1 -- 172.21.15.2:0/65313756 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02841a0cb0 con 0x7f0284101ee0 2023-09-13T04:40:13.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.810+0000 7f027a7fc700 1 -- 172.21.15.2:0/65313756 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0274005030 con 0x7f0284101ee0 2023-09-13T04:40:13.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.810+0000 7f027a7fc700 1 -- 172.21.15.2:0/65313756 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0274004a70 con 0x7f0284101ee0 2023-09-13T04:40:13.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.810+0000 7f028b7ff700 1 -- 172.21.15.2:0/65313756 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02841a1130 con 0x7f0284101ee0 2023-09-13T04:40:13.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.811+0000 7f028b7ff700 1 -- 172.21.15.2:0/65313756 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f028410c580 con 0x7f0284101ee0 2023-09-13T04:40:13.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.812+0000 7f027a7fc700 1 -- 172.21.15.2:0/65313756 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f0274005410 con 0x7f0284101ee0 2023-09-13T04:40:13.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.815+0000 7f027a7fc700 1 --2- 172.21.15.2:0/65313756 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f02700a0710 0x7f02700a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:13.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.815+0000 7f027a7fc700 1 -- 172.21.15.2:0/65313756 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f02740ccc30 con 0x7f0284101ee0 2023-09-13T04:40:13.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.815+0000 7f028959b700 1 --2- 172.21.15.2:0/65313756 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f02700a0710 0x7f02700a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:13.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.815+0000 7f028959b700 1 --2- 172.21.15.2:0/65313756 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f02700a0710 0x7f02700a2bd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0284101d40 tx=0x7f0280009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:13.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:13.819+0000 7f027a7fc700 1 -- 172.21.15.2:0/65313756 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0274096460 con 0x7f0284101ee0 2023-09-13T04:40:14.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.025+0000 7f028b7ff700 1 -- 172.21.15.2:0/65313756 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f02840610f0 con 0x7f02700a0710 2023-09-13T04:40:14.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.038+0000 7f027a7fc700 1 -- 172.21.15.2:0/65313756 <== mgr.14148 v2:172.21.15.2:6800/1461251021 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427014 (secure 0 0 0) 0x7f02840610f0 con 0x7f02700a0710 2023-09-13T04:40:14.045 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:40:14.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.044+0000 7f028b7ff700 1 -- 172.21.15.2:0/65313756 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f02700a0710 msgr2=0x7f02700a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:14.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.044+0000 7f028b7ff700 1 --2- 172.21.15.2:0/65313756 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f02700a0710 0x7f02700a2bd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0284101d40 tx=0x7f0280009260 comp rx=0 tx=0).stop 2023-09-13T04:40:14.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.044+0000 7f028b7ff700 1 -- 172.21.15.2:0/65313756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0284101ee0 msgr2=0x7f02840730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:14.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.044+0000 7f028b7ff700 1 --2- 172.21.15.2:0/65313756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0284101ee0 0x7f02840730a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0274009990 tx=0x7f027400f690 comp rx=0 tx=0).stop 2023-09-13T04:40:14.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.044+0000 7f028b7ff700 1 -- 172.21.15.2:0/65313756 shutdown_connections 2023-09-13T04:40:14.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.044+0000 7f028b7ff700 1 --2- 172.21.15.2:0/65313756 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f02700a0710 0x7f02700a2bd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.044+0000 7f028b7ff700 1 --2- 172.21.15.2:0/65313756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0284102880 0x7f02840735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.044+0000 7f028b7ff700 1 --2- 172.21.15.2:0/65313756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0284100cf0 0x7f0284074a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.044+0000 7f028b7ff700 1 --2- 172.21.15.2:0/65313756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0284101ee0 0x7f02840730a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.044+0000 7f028b7ff700 1 -- 172.21.15.2:0/65313756 >> 172.21.15.2:0/65313756 conn(0x7f02840fc220 msgr2=0x7f02840fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:14.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.044+0000 7f028b7ff700 1 -- 172.21.15.2:0/65313756 shutdown_connections 2023-09-13T04:40:14.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.044+0000 7f028b7ff700 1 -- 172.21.15.2:0/65313756 wait complete. 2023-09-13T04:40:14.051 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-09-13T04:40:14.064 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2023-09-13T04:40:13.537950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598078,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":782,"num_read_kb":456,"num_write":312,"num_write_kb":2983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":252,"ondisk_log_size":252,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338316,"kb_used_data":14068,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603508,"statfs":{"total":1151940427776,"available":1151593992192,"internally_reserved":0,"allocated":14405632,"data_stored":7903854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":258,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":31,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005016"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.624372+0000","last_change":"2023-09-13T04:37:59.244279+0000","last_active":"2023-09-13T04:39:25.624372+0000","last_peered":"2023-09-13T04:39:25.624372+0000","last_clean":"2023-09-13T04:39:25.624372+0000","last_became_active":"2023-09-13T04:37:59.243976+0000","last_became_peered":"2023-09-13T04:37:59.243976+0000","last_unstale":"2023-09-13T04:39:25.624372+0000","last_undegraded":"2023-09-13T04:39:25.624372+0000","last_fullsized":"2023-09-13T04:39:25.624372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:21:52.809411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.856077+0000","last_change":"2023-09-13T04:37:59.243391+0000","last_active":"2023-09-13T04:39:25.856077+0000","last_peered":"2023-09-13T04:39:25.856077+0000","last_clean":"2023-09-13T04:39:25.856077+0000","last_became_active":"2023-09-13T04:37:59.243044+0000","last_became_peered":"2023-09-13T04:37:59.243044+0000","last_unstale":"2023-09-13T04:39:25.856077+0000","last_undegraded":"2023-09-13T04:39:25.856077+0000","last_fullsized":"2023-09-13T04:39:25.856077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:02:17.806638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.855493+0000","last_change":"2023-09-13T04:37:59.242826+0000","last_active":"2023-09-13T04:39:25.855493+0000","last_peered":"2023-09-13T04:39:25.855493+0000","last_clean":"2023-09-13T04:39:25.855493+0000","last_became_active":"2023-09-13T04:37:59.242125+0000","last_became_peered":"2023-09-13T04:37:59.242125+0000","last_unstale":"2023-09-13T04:39:25.855493+0000","last_undegraded":"2023-09-13T04:39:25.855493+0000","last_fullsized":"2023-09-13T04:39:25.855493+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:08:29.216093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.709501+0000","last_change":"2023-09-13T04:37:59.253322+0000","last_active":"2023-09-13T04:39:25.709501+0000","last_peered":"2023-09-13T04:39:25.709501+0000","last_clean":"2023-09-13T04:39:25.709501+0000","last_became_active":"2023-09-13T04:37:59.253050+0000","last_became_peered":"2023-09-13T04:37:59.253050+0000","last_unstale":"2023-09-13T04:39:25.709501+0000","last_undegraded":"2023-09-13T04:39:25.709501+0000","last_fullsized":"2023-09-13T04:39:25.709501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:45:22.357879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755114+0000","last_change":"2023-09-13T04:37:59.250089+0000","last_active":"2023-09-13T04:39:24.755114+0000","last_peered":"2023-09-13T04:39:24.755114+0000","last_clean":"2023-09-13T04:39:24.755114+0000","last_became_active":"2023-09-13T04:37:59.249771+0000","last_became_peered":"2023-09-13T04:37:59.249771+0000","last_unstale":"2023-09-13T04:39:24.755114+0000","last_undegraded":"2023-09-13T04:39:24.755114+0000","last_fullsized":"2023-09-13T04:39:24.755114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:46:59.917756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.703747+0000","last_change":"2023-09-13T04:37:59.247259+0000","last_active":"2023-09-13T04:39:24.703747+0000","last_peered":"2023-09-13T04:39:24.703747+0000","last_clean":"2023-09-13T04:39:24.703747+0000","last_became_active":"2023-09-13T04:37:59.246911+0000","last_became_peered":"2023-09-13T04:37:59.246911+0000","last_unstale":"2023-09-13T04:39:24.703747+0000","last_undegraded":"2023-09-13T04:39:24.703747+0000","last_fullsized":"2023-09-13T04:39:24.703747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:00:39.742392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.585151+0000","last_change":"2023-09-13T04:37:59.244083+0000","last_active":"2023-09-13T04:39:24.585151+0000","last_peered":"2023-09-13T04:39:24.585151+0000","last_clean":"2023-09-13T04:39:24.585151+0000","last_became_active":"2023-09-13T04:37:59.243789+0000","last_became_peered":"2023-09-13T04:37:59.243789+0000","last_unstale":"2023-09-13T04:39:24.585151+0000","last_undegraded":"2023-09-13T04:39:24.585151+0000","last_fullsized":"2023-09-13T04:39:24.585151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:24:00.622790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.581996+0000","last_change":"2023-09-13T04:38:24.601265+0000","last_active":"2023-09-13T04:39:24.581996+0000","last_peered":"2023-09-13T04:39:24.581996+0000","last_clean":"2023-09-13T04:39:24.581996+0000","last_became_active":"2023-09-13T04:38:24.600939+0000","last_became_peered":"2023-09-13T04:38:24.600939+0000","last_unstale":"2023-09-13T04:39:24.581996+0000","last_undegraded":"2023-09-13T04:39:24.581996+0000","last_fullsized":"2023-09-13T04:39:24.581996+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:18:23.642333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754323+0000","last_change":"2023-09-13T04:37:59.238053+0000","last_active":"2023-09-13T04:39:24.754323+0000","last_peered":"2023-09-13T04:39:24.754323+0000","last_clean":"2023-09-13T04:39:24.754323+0000","last_became_active":"2023-09-13T04:37:59.237777+0000","last_became_peered":"2023-09-13T04:37:59.237777+0000","last_unstale":"2023-09-13T04:39:24.754323+0000","last_undegraded":"2023-09-13T04:39:24.754323+0000","last_fullsized":"2023-09-13T04:39:24.754323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:45:39.835659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.704187+0000","last_change":"2023-09-13T04:37:59.252605+0000","last_active":"2023-09-13T04:39:24.704187+0000","last_peered":"2023-09-13T04:39:24.704187+0000","last_clean":"2023-09-13T04:39:24.704187+0000","last_became_active":"2023-09-13T04:37:59.252309+0000","last_became_peered":"2023-09-13T04:37:59.252309+0000","last_unstale":"2023-09-13T04:39:24.704187+0000","last_undegraded":"2023-09-13T04:39:24.704187+0000","last_fullsized":"2023-09-13T04:39:24.704187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:41:45.064230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.585552+0000","last_change":"2023-09-13T04:37:59.243345+0000","last_active":"2023-09-13T04:39:24.585552+0000","last_peered":"2023-09-13T04:39:24.585552+0000","last_clean":"2023-09-13T04:39:24.585552+0000","last_became_active":"2023-09-13T04:37:59.242726+0000","last_became_peered":"2023-09-13T04:37:59.242726+0000","last_unstale":"2023-09-13T04:39:24.585552+0000","last_undegraded":"2023-09-13T04:39:24.585552+0000","last_fullsized":"2023-09-13T04:39:24.585552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:39:55.115281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583449+0000","last_change":"2023-09-13T04:37:59.247415+0000","last_active":"2023-09-13T04:39:24.583449+0000","last_peered":"2023-09-13T04:39:24.583449+0000","last_clean":"2023-09-13T04:39:24.583449+0000","last_became_active":"2023-09-13T04:37:59.247050+0000","last_became_peered":"2023-09-13T04:37:59.247050+0000","last_unstale":"2023-09-13T04:39:24.583449+0000","last_undegraded":"2023-09-13T04:39:24.583449+0000","last_fullsized":"2023-09-13T04:39:24.583449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:03:44.793628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.588078+0000","last_change":"2023-09-13T04:37:59.248935+0000","last_active":"2023-09-13T04:39:24.588078+0000","last_peered":"2023-09-13T04:39:24.588078+0000","last_clean":"2023-09-13T04:39:24.588078+0000","last_became_active":"2023-09-13T04:37:59.248554+0000","last_became_peered":"2023-09-13T04:37:59.248554+0000","last_unstale":"2023-09-13T04:39:24.588078+0000","last_undegraded":"2023-09-13T04:39:24.588078+0000","last_fullsized":"2023-09-13T04:39:24.588078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:53:31.882567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754559+0000","last_change":"2023-09-13T04:37:59.242851+0000","last_active":"2023-09-13T04:39:24.754559+0000","last_peered":"2023-09-13T04:39:24.754559+0000","last_clean":"2023-09-13T04:39:24.754559+0000","last_became_active":"2023-09-13T04:37:59.242384+0000","last_became_peered":"2023-09-13T04:37:59.242384+0000","last_unstale":"2023-09-13T04:39:24.754559+0000","last_undegraded":"2023-09-13T04:39:24.754559+0000","last_fullsized":"2023-09-13T04:39:24.754559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:59:36.508808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754461+0000","last_change":"2023-09-13T04:38:24.599392+0000","last_active":"2023-09-13T04:39:24.754461+0000","last_peered":"2023-09-13T04:39:24.754461+0000","last_clean":"2023-09-13T04:39:24.754461+0000","last_became_active":"2023-09-13T04:38:24.599029+0000","last_became_peered":"2023-09-13T04:38:24.599029+0000","last_unstale":"2023-09-13T04:39:24.754461+0000","last_undegraded":"2023-09-13T04:39:24.754461+0000","last_fullsized":"2023-09-13T04:39:24.754461+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:23:03.703387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.753727+0000","last_change":"2023-09-13T04:37:59.249107+0000","last_active":"2023-09-13T04:39:24.753727+0000","last_peered":"2023-09-13T04:39:24.753727+0000","last_clean":"2023-09-13T04:39:24.753727+0000","last_became_active":"2023-09-13T04:37:59.248803+0000","last_became_peered":"2023-09-13T04:37:59.248803+0000","last_unstale":"2023-09-13T04:39:24.753727+0000","last_undegraded":"2023-09-13T04:39:24.753727+0000","last_fullsized":"2023-09-13T04:39:24.753727+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:50:52.481194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755155+0000","last_change":"2023-09-13T04:37:59.243724+0000","last_active":"2023-09-13T04:39:24.755155+0000","last_peered":"2023-09-13T04:39:24.755155+0000","last_clean":"2023-09-13T04:39:24.755155+0000","last_became_active":"2023-09-13T04:37:59.243385+0000","last_became_peered":"2023-09-13T04:37:59.243385+0000","last_unstale":"2023-09-13T04:39:24.755155+0000","last_undegraded":"2023-09-13T04:39:24.755155+0000","last_fullsized":"2023-09-13T04:39:24.755155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:55:55.203940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582498+0000","last_change":"2023-09-13T04:37:59.238986+0000","last_active":"2023-09-13T04:39:24.582498+0000","last_peered":"2023-09-13T04:39:24.582498+0000","last_clean":"2023-09-13T04:39:24.582498+0000","last_became_active":"2023-09-13T04:37:59.238668+0000","last_became_peered":"2023-09-13T04:37:59.238668+0000","last_unstale":"2023-09-13T04:39:24.582498+0000","last_undegraded":"2023-09-13T04:39:24.582498+0000","last_fullsized":"2023-09-13T04:39:24.582498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:21:10.225223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583227+0000","last_change":"2023-09-13T04:37:59.245598+0000","last_active":"2023-09-13T04:39:24.583227+0000","last_peered":"2023-09-13T04:39:24.583227+0000","last_clean":"2023-09-13T04:39:24.583227+0000","last_became_active":"2023-09-13T04:37:59.244652+0000","last_became_peered":"2023-09-13T04:37:59.244652+0000","last_unstale":"2023-09-13T04:39:24.583227+0000","last_undegraded":"2023-09-13T04:39:24.583227+0000","last_fullsized":"2023-09-13T04:39:24.583227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:34:43.671168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583096+0000","last_change":"2023-09-13T04:38:03.617708+0000","last_active":"2023-09-13T04:39:24.583096+0000","last_peered":"2023-09-13T04:39:24.583096+0000","last_clean":"2023-09-13T04:39:24.583096+0000","last_became_active":"2023-09-13T04:38:03.617277+0000","last_became_peered":"2023-09-13T04:38:03.617277+0000","last_unstale":"2023-09-13T04:39:24.583096+0000","last_undegraded":"2023-09-13T04:39:24.583096+0000","last_fullsized":"2023-09-13T04:39:24.583096+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:26:44.318030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.588015+0000","last_change":"2023-09-13T04:37:58.595156+0000","last_active":"2023-09-13T04:39:25.588015+0000","last_peered":"2023-09-13T04:39:25.588015+0000","last_clean":"2023-09-13T04:39:25.588015+0000","last_became_active":"2023-09-13T04:37:58.594863+0000","last_became_peered":"2023-09-13T04:37:58.594863+0000","last_unstale":"2023-09-13T04:39:25.588015+0000","last_undegraded":"2023-09-13T04:39:25.588015+0000","last_fullsized":"2023-09-13T04:39:25.588015+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:27:14.420495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.854946+0000","last_change":"2023-09-13T04:37:59.250252+0000","last_active":"2023-09-13T04:39:25.854946+0000","last_peered":"2023-09-13T04:39:25.854946+0000","last_clean":"2023-09-13T04:39:25.854946+0000","last_became_active":"2023-09-13T04:37:59.249927+0000","last_became_peered":"2023-09-13T04:37:59.249927+0000","last_unstale":"2023-09-13T04:39:25.854946+0000","last_undegraded":"2023-09-13T04:39:25.854946+0000","last_fullsized":"2023-09-13T04:39:25.854946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:18:06.757040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582790+0000","last_change":"2023-09-13T04:38:24.599683+0000","last_active":"2023-09-13T04:39:24.582790+0000","last_peered":"2023-09-13T04:39:24.582790+0000","last_clean":"2023-09-13T04:39:24.582790+0000","last_became_active":"2023-09-13T04:38:24.599230+0000","last_became_peered":"2023-09-13T04:38:24.599230+0000","last_unstale":"2023-09-13T04:39:24.582790+0000","last_undegraded":"2023-09-13T04:39:24.582790+0000","last_fullsized":"2023-09-13T04:39:24.582790+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:48:22.584735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.855657+0000","last_change":"2023-09-13T04:37:58.597785+0000","last_active":"2023-09-13T04:39:25.855657+0000","last_peered":"2023-09-13T04:39:25.855657+0000","last_clean":"2023-09-13T04:39:25.855657+0000","last_became_active":"2023-09-13T04:37:58.597485+0000","last_became_peered":"2023-09-13T04:37:58.597485+0000","last_unstale":"2023-09-13T04:39:25.855657+0000","last_undegraded":"2023-09-13T04:39:25.855657+0000","last_fullsized":"2023-09-13T04:39:25.855657+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:39:57.156038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.856081+0000","last_change":"2023-09-13T04:37:59.241840+0000","last_active":"2023-09-13T04:39:25.856081+0000","last_peered":"2023-09-13T04:39:25.856081+0000","last_clean":"2023-09-13T04:39:25.856081+0000","last_became_active":"2023-09-13T04:37:59.241545+0000","last_became_peered":"2023-09-13T04:37:59.241545+0000","last_unstale":"2023-09-13T04:39:25.856081+0000","last_undegraded":"2023-09-13T04:39:25.856081+0000","last_fullsized":"2023-09-13T04:39:25.856081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:04:17.439520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582992+0000","last_change":"2023-09-13T04:38:03.621792+0000","last_active":"2023-09-13T04:39:24.582992+0000","last_peered":"2023-09-13T04:39:24.582992+0000","last_clean":"2023-09-13T04:39:24.582992+0000","last_became_active":"2023-09-13T04:38:03.621433+0000","last_became_peered":"2023-09-13T04:38:03.621433+0000","last_unstale":"2023-09-13T04:39:24.582992+0000","last_undegraded":"2023-09-13T04:39:24.582992+0000","last_fullsized":"2023-09-13T04:39:24.582992+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:52:05.926733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754925+0000","last_change":"2023-09-13T04:37:58.590817+0000","last_active":"2023-09-13T04:39:24.754925+0000","last_peered":"2023-09-13T04:39:24.754925+0000","last_clean":"2023-09-13T04:39:24.754925+0000","last_became_active":"2023-09-13T04:37:58.590348+0000","last_became_peered":"2023-09-13T04:37:58.590348+0000","last_unstale":"2023-09-13T04:39:24.754925+0000","last_undegraded":"2023-09-13T04:39:24.754925+0000","last_fullsized":"2023-09-13T04:39:24.754925+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:38:35.680905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754016+0000","last_change":"2023-09-13T04:37:59.237304+0000","last_active":"2023-09-13T04:39:24.754016+0000","last_peered":"2023-09-13T04:39:24.754016+0000","last_clean":"2023-09-13T04:39:24.754016+0000","last_became_active":"2023-09-13T04:37:59.237008+0000","last_became_peered":"2023-09-13T04:37:59.237008+0000","last_unstale":"2023-09-13T04:39:24.754016+0000","last_undegraded":"2023-09-13T04:39:24.754016+0000","last_fullsized":"2023-09-13T04:39:24.754016+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:28:14.254219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582847+0000","last_change":"2023-09-13T04:38:03.606749+0000","last_active":"2023-09-13T04:39:24.582847+0000","last_peered":"2023-09-13T04:39:24.582847+0000","last_clean":"2023-09-13T04:39:24.582847+0000","last_became_active":"2023-09-13T04:38:03.606276+0000","last_became_peered":"2023-09-13T04:38:03.606276+0000","last_unstale":"2023-09-13T04:39:24.582847+0000","last_undegraded":"2023-09-13T04:39:24.582847+0000","last_fullsized":"2023-09-13T04:39:24.582847+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:30:53.572543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754722+0000","last_change":"2023-09-13T04:37:58.590937+0000","last_active":"2023-09-13T04:39:24.754722+0000","last_peered":"2023-09-13T04:39:24.754722+0000","last_clean":"2023-09-13T04:39:24.754722+0000","last_became_active":"2023-09-13T04:37:58.590606+0000","last_became_peered":"2023-09-13T04:37:58.590606+0000","last_unstale":"2023-09-13T04:39:24.754722+0000","last_undegraded":"2023-09-13T04:39:24.754722+0000","last_fullsized":"2023-09-13T04:39:24.754722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:30:03.354670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.706476+0000","last_change":"2023-09-13T04:37:59.251899+0000","last_active":"2023-09-13T04:39:24.706476+0000","last_peered":"2023-09-13T04:39:24.706476+0000","last_clean":"2023-09-13T04:39:24.706476+0000","last_became_active":"2023-09-13T04:37:59.251557+0000","last_became_peered":"2023-09-13T04:37:59.251557+0000","last_unstale":"2023-09-13T04:39:24.706476+0000","last_undegraded":"2023-09-13T04:39:24.706476+0000","last_fullsized":"2023-09-13T04:39:24.706476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:57:45.241188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582997+0000","last_change":"2023-09-13T04:38:03.606076+0000","last_active":"2023-09-13T04:39:24.582997+0000","last_peered":"2023-09-13T04:39:24.582997+0000","last_clean":"2023-09-13T04:39:24.582997+0000","last_became_active":"2023-09-13T04:38:03.605556+0000","last_became_peered":"2023-09-13T04:38:03.605556+0000","last_unstale":"2023-09-13T04:39:24.582997+0000","last_undegraded":"2023-09-13T04:39:24.582997+0000","last_fullsized":"2023-09-13T04:39:24.582997+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:03:32.344625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":132,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:40:08.771722+0000","last_change":"2023-09-13T04:37:58.588170+0000","last_active":"2023-09-13T04:40:08.771722+0000","last_peered":"2023-09-13T04:40:08.771722+0000","last_clean":"2023-09-13T04:40:08.771722+0000","last_became_active":"2023-09-13T04:37:58.587860+0000","last_became_peered":"2023-09-13T04:37:58.587860+0000","last_unstale":"2023-09-13T04:40:08.771722+0000","last_undegraded":"2023-09-13T04:40:08.771722+0000","last_fullsized":"2023-09-13T04:40:08.771722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:38:40.048150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754818+0000","last_change":"2023-09-13T04:37:59.240248+0000","last_active":"2023-09-13T04:39:24.754818+0000","last_peered":"2023-09-13T04:39:24.754818+0000","last_clean":"2023-09-13T04:39:24.754818+0000","last_became_active":"2023-09-13T04:37:59.240008+0000","last_became_peered":"2023-09-13T04:37:59.240008+0000","last_unstale":"2023-09-13T04:39:24.754818+0000","last_undegraded":"2023-09-13T04:39:24.754818+0000","last_fullsized":"2023-09-13T04:39:24.754818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:15:13.275110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583113+0000","last_change":"2023-09-13T04:38:03.616607+0000","last_active":"2023-09-13T04:39:24.583113+0000","last_peered":"2023-09-13T04:39:24.583113+0000","last_clean":"2023-09-13T04:39:24.583113+0000","last_became_active":"2023-09-13T04:38:03.616209+0000","last_became_peered":"2023-09-13T04:38:03.616209+0000","last_unstale":"2023-09-13T04:39:24.583113+0000","last_undegraded":"2023-09-13T04:39:24.583113+0000","last_fullsized":"2023-09-13T04:39:24.583113+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:24:21.934053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.703747+0000","last_change":"2023-09-13T04:37:58.589104+0000","last_active":"2023-09-13T04:39:24.703747+0000","last_peered":"2023-09-13T04:39:24.703747+0000","last_clean":"2023-09-13T04:39:24.703747+0000","last_became_active":"2023-09-13T04:37:58.588780+0000","last_became_peered":"2023-09-13T04:37:58.588780+0000","last_unstale":"2023-09-13T04:39:24.703747+0000","last_undegraded":"2023-09-13T04:39:24.703747+0000","last_fullsized":"2023-09-13T04:39:24.703747+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:01:32.839729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582218+0000","last_change":"2023-09-13T04:38:24.598663+0000","last_active":"2023-09-13T04:39:24.582218+0000","last_peered":"2023-09-13T04:39:24.582218+0000","last_clean":"2023-09-13T04:39:24.582218+0000","last_became_active":"2023-09-13T04:38:24.598258+0000","last_became_peered":"2023-09-13T04:38:24.598258+0000","last_unstale":"2023-09-13T04:39:24.582218+0000","last_undegraded":"2023-09-13T04:39:24.582218+0000","last_fullsized":"2023-09-13T04:39:24.582218+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:50:28.157135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.624295+0000","last_change":"2023-09-13T04:38:03.619930+0000","last_active":"2023-09-13T04:39:25.624295+0000","last_peered":"2023-09-13T04:39:25.624295+0000","last_clean":"2023-09-13T04:39:25.624295+0000","last_became_active":"2023-09-13T04:38:03.619560+0000","last_became_peered":"2023-09-13T04:38:03.619560+0000","last_unstale":"2023-09-13T04:39:25.624295+0000","last_undegraded":"2023-09-13T04:39:25.624295+0000","last_fullsized":"2023-09-13T04:39:25.624295+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:48:06.867602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.587499+0000","last_change":"2023-09-13T04:37:58.590396+0000","last_active":"2023-09-13T04:39:24.587499+0000","last_peered":"2023-09-13T04:39:24.587499+0000","last_clean":"2023-09-13T04:39:24.587499+0000","last_became_active":"2023-09-13T04:37:58.589929+0000","last_became_peered":"2023-09-13T04:37:58.589929+0000","last_unstale":"2023-09-13T04:39:24.587499+0000","last_undegraded":"2023-09-13T04:39:24.587499+0000","last_fullsized":"2023-09-13T04:39:24.587499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:28:36.322096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754248+0000","last_change":"2023-09-13T04:37:59.246033+0000","last_active":"2023-09-13T04:39:24.754248+0000","last_peered":"2023-09-13T04:39:24.754248+0000","last_clean":"2023-09-13T04:39:24.754248+0000","last_became_active":"2023-09-13T04:37:59.245664+0000","last_became_peered":"2023-09-13T04:37:59.245664+0000","last_unstale":"2023-09-13T04:39:24.754248+0000","last_undegraded":"2023-09-13T04:39:24.754248+0000","last_fullsized":"2023-09-13T04:39:24.754248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:24:17.145055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.856052+0000","last_change":"2023-09-13T04:38:03.608504+0000","last_active":"2023-09-13T04:39:25.856052+0000","last_peered":"2023-09-13T04:39:25.856052+0000","last_clean":"2023-09-13T04:39:25.856052+0000","last_became_active":"2023-09-13T04:38:03.608058+0000","last_became_peered":"2023-09-13T04:38:03.608058+0000","last_unstale":"2023-09-13T04:39:25.856052+0000","last_undegraded":"2023-09-13T04:39:25.856052+0000","last_fullsized":"2023-09-13T04:39:25.856052+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:42:43.485453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.705989+0000","last_change":"2023-09-13T04:37:58.588208+0000","last_active":"2023-09-13T04:39:24.705989+0000","last_peered":"2023-09-13T04:39:24.705989+0000","last_clean":"2023-09-13T04:39:24.705989+0000","last_became_active":"2023-09-13T04:37:58.587840+0000","last_became_peered":"2023-09-13T04:37:58.587840+0000","last_unstale":"2023-09-13T04:39:24.705989+0000","last_undegraded":"2023-09-13T04:39:24.705989+0000","last_fullsized":"2023-09-13T04:39:24.705989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:44:58.012233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582291+0000","last_change":"2023-09-13T04:37:59.245102+0000","last_active":"2023-09-13T04:39:24.582291+0000","last_peered":"2023-09-13T04:39:24.582291+0000","last_clean":"2023-09-13T04:39:24.582291+0000","last_became_active":"2023-09-13T04:37:59.244799+0000","last_became_peered":"2023-09-13T04:37:59.244799+0000","last_unstale":"2023-09-13T04:39:24.582291+0000","last_undegraded":"2023-09-13T04:39:24.582291+0000","last_fullsized":"2023-09-13T04:39:24.582291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:57:16.295623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754657+0000","last_change":"2023-09-13T04:38:03.607021+0000","last_active":"2023-09-13T04:39:24.754657+0000","last_peered":"2023-09-13T04:39:24.754657+0000","last_clean":"2023-09-13T04:39:24.754657+0000","last_became_active":"2023-09-13T04:38:03.606611+0000","last_became_peered":"2023-09-13T04:38:03.606611+0000","last_unstale":"2023-09-13T04:39:24.754657+0000","last_undegraded":"2023-09-13T04:39:24.754657+0000","last_fullsized":"2023-09-13T04:39:24.754657+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:33:20.551479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":127,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:40:08.771101+0000","last_change":"2023-09-13T04:37:58.588731+0000","last_active":"2023-09-13T04:40:08.771101+0000","last_peered":"2023-09-13T04:40:08.771101+0000","last_clean":"2023-09-13T04:40:08.771101+0000","last_became_active":"2023-09-13T04:37:58.588398+0000","last_became_peered":"2023-09-13T04:37:58.588398+0000","last_unstale":"2023-09-13T04:40:08.771101+0000","last_undegraded":"2023-09-13T04:40:08.771101+0000","last_fullsized":"2023-09-13T04:40:08.771101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:13:10.724085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754323+0000","last_change":"2023-09-13T04:37:59.244757+0000","last_active":"2023-09-13T04:39:24.754323+0000","last_peered":"2023-09-13T04:39:24.754323+0000","last_clean":"2023-09-13T04:39:24.754323+0000","last_became_active":"2023-09-13T04:37:59.244423+0000","last_became_peered":"2023-09-13T04:37:59.244423+0000","last_unstale":"2023-09-13T04:39:24.754323+0000","last_undegraded":"2023-09-13T04:39:24.754323+0000","last_fullsized":"2023-09-13T04:39:24.754323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:14:37.973058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582006+0000","last_change":"2023-09-13T04:38:03.618358+0000","last_active":"2023-09-13T04:39:24.582006+0000","last_peered":"2023-09-13T04:39:24.582006+0000","last_clean":"2023-09-13T04:39:24.582006+0000","last_became_active":"2023-09-13T04:38:03.617291+0000","last_became_peered":"2023-09-13T04:38:03.617291+0000","last_unstale":"2023-09-13T04:39:24.582006+0000","last_undegraded":"2023-09-13T04:39:24.582006+0000","last_fullsized":"2023-09-13T04:39:24.582006+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:33:13.955353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.585686+0000","last_change":"2023-09-13T04:37:58.590978+0000","last_active":"2023-09-13T04:39:24.585686+0000","last_peered":"2023-09-13T04:39:24.585686+0000","last_clean":"2023-09-13T04:39:24.585686+0000","last_became_active":"2023-09-13T04:37:58.590673+0000","last_became_peered":"2023-09-13T04:37:58.590673+0000","last_unstale":"2023-09-13T04:39:24.585686+0000","last_undegraded":"2023-09-13T04:39:24.585686+0000","last_fullsized":"2023-09-13T04:39:24.585686+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:10:32.570029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755812+0000","last_change":"2023-09-13T04:37:59.251283+0000","last_active":"2023-09-13T04:39:24.755812+0000","last_peered":"2023-09-13T04:39:24.755812+0000","last_clean":"2023-09-13T04:39:24.755812+0000","last_became_active":"2023-09-13T04:37:59.250119+0000","last_became_peered":"2023-09-13T04:37:59.250119+0000","last_unstale":"2023-09-13T04:39:24.755812+0000","last_undegraded":"2023-09-13T04:39:24.755812+0000","last_fullsized":"2023-09-13T04:39:24.755812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:48:54.389970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583163+0000","last_change":"2023-09-13T04:38:24.596107+0000","last_active":"2023-09-13T04:39:24.583163+0000","last_peered":"2023-09-13T04:39:24.583163+0000","last_clean":"2023-09-13T04:39:24.583163+0000","last_became_active":"2023-09-13T04:38:24.595653+0000","last_became_peered":"2023-09-13T04:38:24.595653+0000","last_unstale":"2023-09-13T04:39:24.583163+0000","last_undegraded":"2023-09-13T04:39:24.583163+0000","last_fullsized":"2023-09-13T04:39:24.583163+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:02:53.708658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.706415+0000","last_change":"2023-09-13T04:37:58.584909+0000","last_active":"2023-09-13T04:39:24.706415+0000","last_peered":"2023-09-13T04:39:24.706415+0000","last_clean":"2023-09-13T04:39:24.706415+0000","last_became_active":"2023-09-13T04:37:58.584630+0000","last_became_peered":"2023-09-13T04:37:58.584630+0000","last_unstale":"2023-09-13T04:39:24.706415+0000","last_undegraded":"2023-09-13T04:39:24.706415+0000","last_fullsized":"2023-09-13T04:39:24.706415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:57:56.567499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754125+0000","last_change":"2023-09-13T04:37:59.254448+0000","last_active":"2023-09-13T04:39:24.754125+0000","last_peered":"2023-09-13T04:39:24.754125+0000","last_clean":"2023-09-13T04:39:24.754125+0000","last_became_active":"2023-09-13T04:37:59.254138+0000","last_became_peered":"2023-09-13T04:37:59.254138+0000","last_unstale":"2023-09-13T04:39:24.754125+0000","last_undegraded":"2023-09-13T04:39:24.754125+0000","last_fullsized":"2023-09-13T04:39:24.754125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:36:14.939881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582104+0000","last_change":"2023-09-13T04:38:03.613746+0000","last_active":"2023-09-13T04:39:24.582104+0000","last_peered":"2023-09-13T04:39:24.582104+0000","last_clean":"2023-09-13T04:39:24.582104+0000","last_became_active":"2023-09-13T04:38:03.613354+0000","last_became_peered":"2023-09-13T04:38:03.613354+0000","last_unstale":"2023-09-13T04:39:24.582104+0000","last_undegraded":"2023-09-13T04:39:24.582104+0000","last_fullsized":"2023-09-13T04:39:24.582104+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:41:01.294912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'14","reported_seq":245,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:40:09.772943+0000","last_change":"2023-09-13T04:37:58.591196+0000","last_active":"2023-09-13T04:40:09.772943+0000","last_peered":"2023-09-13T04:40:09.772943+0000","last_clean":"2023-09-13T04:40:09.772943+0000","last_became_active":"2023-09-13T04:37:58.590883+0000","last_became_peered":"2023-09-13T04:37:58.590883+0000","last_unstale":"2023-09-13T04:40:09.772943+0000","last_undegraded":"2023-09-13T04:40:09.772943+0000","last_fullsized":"2023-09-13T04:40:09.772943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:46:58.530270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4100,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":211,"num_read_kb":110,"num_write":15,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755199+0000","last_change":"2023-09-13T04:37:59.243483+0000","last_active":"2023-09-13T04:39:24.755199+0000","last_peered":"2023-09-13T04:39:24.755199+0000","last_clean":"2023-09-13T04:39:24.755199+0000","last_became_active":"2023-09-13T04:37:59.243088+0000","last_became_peered":"2023-09-13T04:37:59.243088+0000","last_unstale":"2023-09-13T04:39:24.755199+0000","last_undegraded":"2023-09-13T04:39:24.755199+0000","last_fullsized":"2023-09-13T04:39:24.755199+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:56:14.093819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.706663+0000","last_change":"2023-09-13T04:38:03.608820+0000","last_active":"2023-09-13T04:39:24.706663+0000","last_peered":"2023-09-13T04:39:24.706663+0000","last_clean":"2023-09-13T04:39:24.706663+0000","last_became_active":"2023-09-13T04:38:03.608200+0000","last_became_peered":"2023-09-13T04:38:03.608200+0000","last_unstale":"2023-09-13T04:39:24.706663+0000","last_undegraded":"2023-09-13T04:39:24.706663+0000","last_fullsized":"2023-09-13T04:39:24.706663+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:54:52.416501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754566+0000","last_change":"2023-09-13T04:38:24.606820+0000","last_active":"2023-09-13T04:39:24.754566+0000","last_peered":"2023-09-13T04:39:24.754566+0000","last_clean":"2023-09-13T04:39:24.754566+0000","last_became_active":"2023-09-13T04:38:24.599059+0000","last_became_peered":"2023-09-13T04:38:24.599059+0000","last_unstale":"2023-09-13T04:39:24.754566+0000","last_undegraded":"2023-09-13T04:39:24.754566+0000","last_fullsized":"2023-09-13T04:39:24.754566+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:08:12.634357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.587438+0000","last_change":"2023-09-13T04:37:59.250666+0000","last_active":"2023-09-13T04:39:24.587438+0000","last_peered":"2023-09-13T04:39:24.587438+0000","last_clean":"2023-09-13T04:39:24.587438+0000","last_became_active":"2023-09-13T04:37:59.250240+0000","last_became_peered":"2023-09-13T04:37:59.250240+0000","last_unstale":"2023-09-13T04:39:24.587438+0000","last_undegraded":"2023-09-13T04:39:24.587438+0000","last_fullsized":"2023-09-13T04:39:24.587438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:44:31.324222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754566+0000","last_change":"2023-09-13T04:38:03.616296+0000","last_active":"2023-09-13T04:39:24.754566+0000","last_peered":"2023-09-13T04:39:24.754566+0000","last_clean":"2023-09-13T04:39:24.754566+0000","last_became_active":"2023-09-13T04:38:03.615934+0000","last_became_peered":"2023-09-13T04:38:03.615934+0000","last_unstale":"2023-09-13T04:39:24.754566+0000","last_undegraded":"2023-09-13T04:39:24.754566+0000","last_fullsized":"2023-09-13T04:39:24.754566+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:51:18.994472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.706408+0000","last_change":"2023-09-13T04:37:58.594782+0000","last_active":"2023-09-13T04:39:24.706408+0000","last_peered":"2023-09-13T04:39:24.706408+0000","last_clean":"2023-09-13T04:39:24.706408+0000","last_became_active":"2023-09-13T04:37:58.594480+0000","last_became_peered":"2023-09-13T04:37:58.594480+0000","last_unstale":"2023-09-13T04:39:24.706408+0000","last_undegraded":"2023-09-13T04:39:24.706408+0000","last_fullsized":"2023-09-13T04:39:24.706408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:40:19.635672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.707565+0000","last_change":"2023-09-13T04:39:25.707565+0000","last_active":"2023-09-13T04:39:25.707565+0000","last_peered":"2023-09-13T04:39:25.707565+0000","last_clean":"2023-09-13T04:39:25.707565+0000","last_became_active":"2023-09-13T04:39:25.707099+0000","last_became_peered":"2023-09-13T04:39:25.707099+0000","last_unstale":"2023-09-13T04:39:25.707565+0000","last_undegraded":"2023-09-13T04:39:25.707565+0000","last_fullsized":"2023-09-13T04:39:25.707565+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:25:25.923715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582727+0000","last_change":"2023-09-13T04:38:03.618835+0000","last_active":"2023-09-13T04:39:24.582727+0000","last_peered":"2023-09-13T04:39:24.582727+0000","last_clean":"2023-09-13T04:39:24.582727+0000","last_became_active":"2023-09-13T04:38:03.618226+0000","last_became_peered":"2023-09-13T04:38:03.618226+0000","last_unstale":"2023-09-13T04:39:24.582727+0000","last_undegraded":"2023-09-13T04:39:24.582727+0000","last_fullsized":"2023-09-13T04:39:24.582727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:01:28.157490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":129,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:40:08.771978+0000","last_change":"2023-09-13T04:37:58.588545+0000","last_active":"2023-09-13T04:40:08.771978+0000","last_peered":"2023-09-13T04:40:08.771978+0000","last_clean":"2023-09-13T04:40:08.771978+0000","last_became_active":"2023-09-13T04:37:58.587995+0000","last_became_peered":"2023-09-13T04:37:58.587995+0000","last_unstale":"2023-09-13T04:40:08.771978+0000","last_undegraded":"2023-09-13T04:40:08.771978+0000","last_fullsized":"2023-09-13T04:40:08.771978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:40:08.797724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.585205+0000","last_change":"2023-09-13T04:37:59.249100+0000","last_active":"2023-09-13T04:39:24.585205+0000","last_peered":"2023-09-13T04:39:24.585205+0000","last_clean":"2023-09-13T04:39:24.585205+0000","last_became_active":"2023-09-13T04:37:59.248771+0000","last_became_peered":"2023-09-13T04:37:59.248771+0000","last_unstale":"2023-09-13T04:39:24.585205+0000","last_undegraded":"2023-09-13T04:39:24.585205+0000","last_fullsized":"2023-09-13T04:39:24.585205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:27:37.964595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.587764+0000","last_change":"2023-09-13T04:38:03.616988+0000","last_active":"2023-09-13T04:39:24.587764+0000","last_peered":"2023-09-13T04:39:24.587764+0000","last_clean":"2023-09-13T04:39:24.587764+0000","last_became_active":"2023-09-13T04:38:03.616524+0000","last_became_peered":"2023-09-13T04:38:03.616524+0000","last_unstale":"2023-09-13T04:39:24.587764+0000","last_undegraded":"2023-09-13T04:39:24.587764+0000","last_fullsized":"2023-09-13T04:39:24.587764+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:42:41.431115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754413+0000","last_change":"2023-09-13T04:38:24.613384+0000","last_active":"2023-09-13T04:39:24.754413+0000","last_peered":"2023-09-13T04:39:24.754413+0000","last_clean":"2023-09-13T04:39:24.754413+0000","last_became_active":"2023-09-13T04:38:24.599349+0000","last_became_peered":"2023-09-13T04:38:24.599349+0000","last_unstale":"2023-09-13T04:39:24.754413+0000","last_undegraded":"2023-09-13T04:39:24.754413+0000","last_fullsized":"2023-09-13T04:39:24.754413+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:23:23.479715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.706144+0000","last_change":"2023-09-13T04:37:59.252237+0000","last_active":"2023-09-13T04:39:24.706144+0000","last_peered":"2023-09-13T04:39:24.706144+0000","last_clean":"2023-09-13T04:39:24.706144+0000","last_became_active":"2023-09-13T04:37:59.251718+0000","last_became_peered":"2023-09-13T04:37:59.251718+0000","last_unstale":"2023-09-13T04:39:24.706144+0000","last_undegraded":"2023-09-13T04:39:24.706144+0000","last_fullsized":"2023-09-13T04:39:24.706144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:45:48.405226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.706124+0000","last_change":"2023-09-13T04:38:03.607096+0000","last_active":"2023-09-13T04:39:24.706124+0000","last_peered":"2023-09-13T04:39:24.706124+0000","last_clean":"2023-09-13T04:39:24.706124+0000","last_became_active":"2023-09-13T04:38:03.606656+0000","last_became_peered":"2023-09-13T04:38:03.606656+0000","last_unstale":"2023-09-13T04:39:24.706124+0000","last_undegraded":"2023-09-13T04:39:24.706124+0000","last_fullsized":"2023-09-13T04:39:24.706124+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:59:26.140360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755201+0000","last_change":"2023-09-13T04:37:58.591047+0000","last_active":"2023-09-13T04:39:24.755201+0000","last_peered":"2023-09-13T04:39:24.755201+0000","last_clean":"2023-09-13T04:39:24.755201+0000","last_became_active":"2023-09-13T04:37:58.590740+0000","last_became_peered":"2023-09-13T04:37:58.590740+0000","last_unstale":"2023-09-13T04:39:24.755201+0000","last_undegraded":"2023-09-13T04:39:24.755201+0000","last_fullsized":"2023-09-13T04:39:24.755201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:14:42.280416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755283+0000","last_change":"2023-09-13T04:37:59.249362+0000","last_active":"2023-09-13T04:39:24.755283+0000","last_peered":"2023-09-13T04:39:24.755283+0000","last_clean":"2023-09-13T04:39:24.755283+0000","last_became_active":"2023-09-13T04:37:59.249041+0000","last_became_peered":"2023-09-13T04:37:59.249041+0000","last_unstale":"2023-09-13T04:39:24.755283+0000","last_undegraded":"2023-09-13T04:39:24.755283+0000","last_fullsized":"2023-09-13T04:39:24.755283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:06:33.822157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.588524+0000","last_change":"2023-09-13T04:38:24.598370+0000","last_active":"2023-09-13T04:39:25.588524+0000","last_peered":"2023-09-13T04:39:25.588524+0000","last_clean":"2023-09-13T04:39:25.588524+0000","last_became_active":"2023-09-13T04:38:24.597931+0000","last_became_peered":"2023-09-13T04:38:24.597931+0000","last_unstale":"2023-09-13T04:39:25.588524+0000","last_undegraded":"2023-09-13T04:39:25.588524+0000","last_fullsized":"2023-09-13T04:39:25.588524+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:14:28.911329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.587714+0000","last_change":"2023-09-13T04:37:58.592496+0000","last_active":"2023-09-13T04:39:25.587714+0000","last_peered":"2023-09-13T04:39:25.587714+0000","last_clean":"2023-09-13T04:39:25.587714+0000","last_became_active":"2023-09-13T04:37:58.592106+0000","last_became_peered":"2023-09-13T04:37:58.592106+0000","last_unstale":"2023-09-13T04:39:25.587714+0000","last_undegraded":"2023-09-13T04:39:25.587714+0000","last_fullsized":"2023-09-13T04:39:25.587714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:28:55.618563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.592208+0000","last_change":"2023-09-13T04:37:59.250019+0000","last_active":"2023-09-13T04:39:25.592208+0000","last_peered":"2023-09-13T04:39:25.592208+0000","last_clean":"2023-09-13T04:39:25.592208+0000","last_became_active":"2023-09-13T04:37:59.249278+0000","last_became_peered":"2023-09-13T04:37:59.249278+0000","last_unstale":"2023-09-13T04:39:25.592208+0000","last_undegraded":"2023-09-13T04:39:25.592208+0000","last_fullsized":"2023-09-13T04:39:25.592208+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:15:31.217236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.705979+0000","last_change":"2023-09-13T04:38:03.614542+0000","last_active":"2023-09-13T04:39:25.705979+0000","last_peered":"2023-09-13T04:39:25.705979+0000","last_clean":"2023-09-13T04:39:25.705979+0000","last_became_active":"2023-09-13T04:38:03.614103+0000","last_became_peered":"2023-09-13T04:38:03.614103+0000","last_unstale":"2023-09-13T04:39:25.705979+0000","last_undegraded":"2023-09-13T04:39:25.705979+0000","last_fullsized":"2023-09-13T04:39:25.705979+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:06:48.426843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.595603+0000","last_change":"2023-09-13T04:37:58.591864+0000","last_active":"2023-09-13T04:39:25.595603+0000","last_peered":"2023-09-13T04:39:25.595603+0000","last_clean":"2023-09-13T04:39:25.595603+0000","last_became_active":"2023-09-13T04:37:58.591562+0000","last_became_peered":"2023-09-13T04:37:58.591562+0000","last_unstale":"2023-09-13T04:39:25.595603+0000","last_undegraded":"2023-09-13T04:39:25.595603+0000","last_fullsized":"2023-09-13T04:39:25.595603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:31:27.280489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.592296+0000","last_change":"2023-09-13T04:37:59.252600+0000","last_active":"2023-09-13T04:39:25.592296+0000","last_peered":"2023-09-13T04:39:25.592296+0000","last_clean":"2023-09-13T04:39:25.592296+0000","last_became_active":"2023-09-13T04:37:59.252306+0000","last_became_peered":"2023-09-13T04:37:59.252306+0000","last_unstale":"2023-09-13T04:39:25.592296+0000","last_undegraded":"2023-09-13T04:39:25.592296+0000","last_fullsized":"2023-09-13T04:39:25.592296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:18:55.686847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.587587+0000","last_change":"2023-09-13T04:38:03.742502+0000","last_active":"2023-09-13T04:39:24.587587+0000","last_peered":"2023-09-13T04:39:24.587587+0000","last_clean":"2023-09-13T04:39:24.587587+0000","last_became_active":"2023-09-13T04:38:03.742054+0000","last_became_peered":"2023-09-13T04:38:03.742054+0000","last_unstale":"2023-09-13T04:39:24.587587+0000","last_undegraded":"2023-09-13T04:39:24.587587+0000","last_fullsized":"2023-09-13T04:39:24.587587+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:55:54.601145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.709865+0000","last_change":"2023-09-13T04:37:59.245644+0000","last_active":"2023-09-13T04:39:25.709865+0000","last_peered":"2023-09-13T04:39:25.709865+0000","last_clean":"2023-09-13T04:39:25.709865+0000","last_became_active":"2023-09-13T04:37:59.245284+0000","last_became_peered":"2023-09-13T04:37:59.245284+0000","last_unstale":"2023-09-13T04:39:25.709865+0000","last_undegraded":"2023-09-13T04:39:25.709865+0000","last_fullsized":"2023-09-13T04:39:25.709865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:19:04.002414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":131,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:40:09.772897+0000","last_change":"2023-09-13T04:37:58.590249+0000","last_active":"2023-09-13T04:40:09.772897+0000","last_peered":"2023-09-13T04:40:09.772897+0000","last_clean":"2023-09-13T04:40:09.772897+0000","last_became_active":"2023-09-13T04:37:58.589521+0000","last_became_peered":"2023-09-13T04:37:58.589521+0000","last_unstale":"2023-09-13T04:40:09.772897+0000","last_undegraded":"2023-09-13T04:40:09.772897+0000","last_fullsized":"2023-09-13T04:40:09.772897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:11:57.542149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.595865+0000","last_change":"2023-09-13T04:38:24.598613+0000","last_active":"2023-09-13T04:39:25.595865+0000","last_peered":"2023-09-13T04:39:25.595865+0000","last_clean":"2023-09-13T04:39:25.595865+0000","last_became_active":"2023-09-13T04:38:24.598150+0000","last_became_peered":"2023-09-13T04:38:24.598150+0000","last_unstale":"2023-09-13T04:39:25.595865+0000","last_undegraded":"2023-09-13T04:39:25.595865+0000","last_fullsized":"2023-09-13T04:39:25.595865+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:02:29.894446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.854964+0000","last_change":"2023-09-13T04:37:59.243794+0000","last_active":"2023-09-13T04:39:25.854964+0000","last_peered":"2023-09-13T04:39:25.854964+0000","last_clean":"2023-09-13T04:39:25.854964+0000","last_became_active":"2023-09-13T04:37:59.243368+0000","last_became_peered":"2023-09-13T04:37:59.243368+0000","last_unstale":"2023-09-13T04:39:25.854964+0000","last_undegraded":"2023-09-13T04:39:25.854964+0000","last_fullsized":"2023-09-13T04:39:25.854964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:06:45.318737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.710051+0000","last_change":"2023-09-13T04:37:58.589195+0000","last_active":"2023-09-13T04:39:25.710051+0000","last_peered":"2023-09-13T04:39:25.710051+0000","last_clean":"2023-09-13T04:39:25.710051+0000","last_became_active":"2023-09-13T04:37:58.588905+0000","last_became_peered":"2023-09-13T04:37:58.588905+0000","last_unstale":"2023-09-13T04:39:25.710051+0000","last_undegraded":"2023-09-13T04:39:25.710051+0000","last_fullsized":"2023-09-13T04:39:25.710051+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:13:15.978891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":329,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.595369+0000","last_change":"2023-09-13T04:36:52.227264+0000","last_active":"2023-09-13T04:39:25.595369+0000","last_peered":"2023-09-13T04:39:25.595369+0000","last_clean":"2023-09-13T04:39:25.595369+0000","last_became_active":"2023-09-13T04:36:52.226773+0000","last_became_peered":"2023-09-13T04:36:52.226773+0000","last_unstale":"2023-09-13T04:39:25.595369+0000","last_undegraded":"2023-09-13T04:39:25.595369+0000","last_fullsized":"2023-09-13T04:39:25.595369+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:33:42.026499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:33:42.026499+0000","last_clean_scrub_stamp":"2023-09-13T04:33:42.026499+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:28:55.806587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.706548+0000","last_change":"2023-09-13T04:38:03.742790+0000","last_active":"2023-09-13T04:39:24.706548+0000","last_peered":"2023-09-13T04:39:24.706548+0000","last_clean":"2023-09-13T04:39:24.706548+0000","last_became_active":"2023-09-13T04:38:03.742338+0000","last_became_peered":"2023-09-13T04:38:03.742338+0000","last_unstale":"2023-09-13T04:39:24.706548+0000","last_undegraded":"2023-09-13T04:39:24.706548+0000","last_fullsized":"2023-09-13T04:39:24.706548+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:01:38.910623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.705623+0000","last_change":"2023-09-13T04:37:59.252669+0000","last_active":"2023-09-13T04:39:25.705623+0000","last_peered":"2023-09-13T04:39:25.705623+0000","last_clean":"2023-09-13T04:39:25.705623+0000","last_became_active":"2023-09-13T04:37:59.252223+0000","last_became_peered":"2023-09-13T04:37:59.252223+0000","last_unstale":"2023-09-13T04:39:25.705623+0000","last_undegraded":"2023-09-13T04:39:25.705623+0000","last_fullsized":"2023-09-13T04:39:25.705623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:57:21.090083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.595510+0000","last_change":"2023-09-13T04:37:58.590540+0000","last_active":"2023-09-13T04:39:25.595510+0000","last_peered":"2023-09-13T04:39:25.595510+0000","last_clean":"2023-09-13T04:39:25.595510+0000","last_became_active":"2023-09-13T04:37:58.590061+0000","last_became_peered":"2023-09-13T04:37:58.590061+0000","last_unstale":"2023-09-13T04:39:25.595510+0000","last_undegraded":"2023-09-13T04:39:25.595510+0000","last_fullsized":"2023-09-13T04:39:25.595510+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:57:17.133461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.624078+0000","last_change":"2023-09-13T04:38:03.614615+0000","last_active":"2023-09-13T04:39:25.624078+0000","last_peered":"2023-09-13T04:39:25.624078+0000","last_clean":"2023-09-13T04:39:25.624078+0000","last_became_active":"2023-09-13T04:38:03.614086+0000","last_became_peered":"2023-09-13T04:38:03.614086+0000","last_unstale":"2023-09-13T04:39:25.624078+0000","last_undegraded":"2023-09-13T04:39:25.624078+0000","last_fullsized":"2023-09-13T04:39:25.624078+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:25:35.525513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.588681+0000","last_change":"2023-09-13T04:37:58.592096+0000","last_active":"2023-09-13T04:39:25.588681+0000","last_peered":"2023-09-13T04:39:25.588681+0000","last_clean":"2023-09-13T04:39:25.588681+0000","last_became_active":"2023-09-13T04:37:58.591872+0000","last_became_peered":"2023-09-13T04:37:58.591872+0000","last_unstale":"2023-09-13T04:39:25.588681+0000","last_undegraded":"2023-09-13T04:39:25.588681+0000","last_fullsized":"2023-09-13T04:39:25.588681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:05:23.936105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.706127+0000","last_change":"2023-09-13T04:37:59.240759+0000","last_active":"2023-09-13T04:39:25.706127+0000","last_peered":"2023-09-13T04:39:25.706127+0000","last_clean":"2023-09-13T04:39:25.706127+0000","last_became_active":"2023-09-13T04:37:59.240414+0000","last_became_peered":"2023-09-13T04:37:59.240414+0000","last_unstale":"2023-09-13T04:39:25.706127+0000","last_undegraded":"2023-09-13T04:39:25.706127+0000","last_fullsized":"2023-09-13T04:39:25.706127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:37:20.301938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.854802+0000","last_change":"2023-09-13T04:38:03.742519+0000","last_active":"2023-09-13T04:39:25.854802+0000","last_peered":"2023-09-13T04:39:25.854802+0000","last_clean":"2023-09-13T04:39:25.854802+0000","last_became_active":"2023-09-13T04:38:03.742075+0000","last_became_peered":"2023-09-13T04:38:03.742075+0000","last_unstale":"2023-09-13T04:39:25.854802+0000","last_undegraded":"2023-09-13T04:39:25.854802+0000","last_fullsized":"2023-09-13T04:39:25.854802+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:14:33.155837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.587948+0000","last_change":"2023-09-13T04:37:58.593891+0000","last_active":"2023-09-13T04:39:25.587948+0000","last_peered":"2023-09-13T04:39:25.587948+0000","last_clean":"2023-09-13T04:39:25.587948+0000","last_became_active":"2023-09-13T04:37:58.593497+0000","last_became_peered":"2023-09-13T04:37:58.593497+0000","last_unstale":"2023-09-13T04:39:25.587948+0000","last_undegraded":"2023-09-13T04:39:25.587948+0000","last_fullsized":"2023-09-13T04:39:25.587948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:25:53.652601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.855865+0000","last_change":"2023-09-13T04:37:59.247151+0000","last_active":"2023-09-13T04:39:25.855865+0000","last_peered":"2023-09-13T04:39:25.855865+0000","last_clean":"2023-09-13T04:39:25.855865+0000","last_became_active":"2023-09-13T04:37:59.246853+0000","last_became_peered":"2023-09-13T04:37:59.246853+0000","last_unstale":"2023-09-13T04:39:25.855865+0000","last_undegraded":"2023-09-13T04:39:25.855865+0000","last_fullsized":"2023-09-13T04:39:25.855865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:15:22.659365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.854582+0000","last_change":"2023-09-13T04:38:03.606594+0000","last_active":"2023-09-13T04:39:25.854582+0000","last_peered":"2023-09-13T04:39:25.854582+0000","last_clean":"2023-09-13T04:39:25.854582+0000","last_became_active":"2023-09-13T04:38:03.606058+0000","last_became_peered":"2023-09-13T04:38:03.606058+0000","last_unstale":"2023-09-13T04:39:25.854582+0000","last_undegraded":"2023-09-13T04:39:25.854582+0000","last_fullsized":"2023-09-13T04:39:25.854582+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:57:39.237520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.709689+0000","last_change":"2023-09-13T04:37:58.588362+0000","last_active":"2023-09-13T04:39:25.709689+0000","last_peered":"2023-09-13T04:39:25.709689+0000","last_clean":"2023-09-13T04:39:25.709689+0000","last_became_active":"2023-09-13T04:37:58.588065+0000","last_became_peered":"2023-09-13T04:37:58.588065+0000","last_unstale":"2023-09-13T04:39:25.709689+0000","last_undegraded":"2023-09-13T04:39:25.709689+0000","last_fullsized":"2023-09-13T04:39:25.709689+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:32:04.255654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.856028+0000","last_change":"2023-09-13T04:37:59.251883+0000","last_active":"2023-09-13T04:39:25.856028+0000","last_peered":"2023-09-13T04:39:25.856028+0000","last_clean":"2023-09-13T04:39:25.856028+0000","last_became_active":"2023-09-13T04:37:59.251522+0000","last_became_peered":"2023-09-13T04:37:59.251522+0000","last_unstale":"2023-09-13T04:39:25.856028+0000","last_undegraded":"2023-09-13T04:39:25.856028+0000","last_fullsized":"2023-09-13T04:39:25.856028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:05:06.420799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583139+0000","last_change":"2023-09-13T04:38:03.742726+0000","last_active":"2023-09-13T04:39:24.583139+0000","last_peered":"2023-09-13T04:39:24.583139+0000","last_clean":"2023-09-13T04:39:24.583139+0000","last_became_active":"2023-09-13T04:38:03.742293+0000","last_became_peered":"2023-09-13T04:38:03.742293+0000","last_unstale":"2023-09-13T04:39:24.583139+0000","last_undegraded":"2023-09-13T04:39:24.583139+0000","last_fullsized":"2023-09-13T04:39:24.583139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:52:15.089764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.672289+0000","last_change":"2023-09-13T04:37:58.584923+0000","last_active":"2023-09-13T04:39:25.672289+0000","last_peered":"2023-09-13T04:39:25.672289+0000","last_clean":"2023-09-13T04:39:25.672289+0000","last_became_active":"2023-09-13T04:37:58.584320+0000","last_became_peered":"2023-09-13T04:37:58.584320+0000","last_unstale":"2023-09-13T04:39:25.672289+0000","last_undegraded":"2023-09-13T04:39:25.672289+0000","last_fullsized":"2023-09-13T04:39:25.672289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:16:53.670534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.588148+0000","last_change":"2023-09-13T04:37:59.243029+0000","last_active":"2023-09-13T04:39:25.588148+0000","last_peered":"2023-09-13T04:39:25.588148+0000","last_clean":"2023-09-13T04:39:25.588148+0000","last_became_active":"2023-09-13T04:37:59.242733+0000","last_became_peered":"2023-09-13T04:37:59.242733+0000","last_unstale":"2023-09-13T04:39:25.588148+0000","last_undegraded":"2023-09-13T04:39:25.588148+0000","last_fullsized":"2023-09-13T04:39:25.588148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:48:21.430026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.582868+0000","last_change":"2023-09-13T04:38:03.616735+0000","last_active":"2023-09-13T04:39:24.582868+0000","last_peered":"2023-09-13T04:39:24.582868+0000","last_clean":"2023-09-13T04:39:24.582868+0000","last_became_active":"2023-09-13T04:38:03.616275+0000","last_became_peered":"2023-09-13T04:38:03.616275+0000","last_unstale":"2023-09-13T04:39:24.582868+0000","last_undegraded":"2023-09-13T04:39:24.582868+0000","last_fullsized":"2023-09-13T04:39:24.582868+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:12:27.267918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.855026+0000","last_change":"2023-09-13T04:37:58.595040+0000","last_active":"2023-09-13T04:39:25.855026+0000","last_peered":"2023-09-13T04:39:25.855026+0000","last_clean":"2023-09-13T04:39:25.855026+0000","last_became_active":"2023-09-13T04:37:58.591403+0000","last_became_peered":"2023-09-13T04:37:58.591403+0000","last_unstale":"2023-09-13T04:39:25.855026+0000","last_undegraded":"2023-09-13T04:39:25.855026+0000","last_fullsized":"2023-09-13T04:39:25.855026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:42:06.561066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.623940+0000","last_change":"2023-09-13T04:37:59.248837+0000","last_active":"2023-09-13T04:39:25.623940+0000","last_peered":"2023-09-13T04:39:25.623940+0000","last_clean":"2023-09-13T04:39:25.623940+0000","last_became_active":"2023-09-13T04:37:59.248573+0000","last_became_peered":"2023-09-13T04:37:59.248573+0000","last_unstale":"2023-09-13T04:39:25.623940+0000","last_undegraded":"2023-09-13T04:39:25.623940+0000","last_fullsized":"2023-09-13T04:39:25.623940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:56:22.980484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583094+0000","last_change":"2023-09-13T04:38:03.616370+0000","last_active":"2023-09-13T04:39:24.583094+0000","last_peered":"2023-09-13T04:39:24.583094+0000","last_clean":"2023-09-13T04:39:24.583094+0000","last_became_active":"2023-09-13T04:38:03.615996+0000","last_became_peered":"2023-09-13T04:38:03.615996+0000","last_unstale":"2023-09-13T04:39:24.583094+0000","last_undegraded":"2023-09-13T04:39:24.583094+0000","last_fullsized":"2023-09-13T04:39:24.583094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:56:59.924205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.854773+0000","last_change":"2023-09-13T04:37:58.597902+0000","last_active":"2023-09-13T04:39:25.854773+0000","last_peered":"2023-09-13T04:39:25.854773+0000","last_clean":"2023-09-13T04:39:25.854773+0000","last_became_active":"2023-09-13T04:37:58.597578+0000","last_became_peered":"2023-09-13T04:37:58.597578+0000","last_unstale":"2023-09-13T04:39:25.854773+0000","last_undegraded":"2023-09-13T04:39:25.854773+0000","last_fullsized":"2023-09-13T04:39:25.854773+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:10:09.931949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755492+0000","last_change":"2023-09-13T04:37:59.248924+0000","last_active":"2023-09-13T04:39:24.755492+0000","last_peered":"2023-09-13T04:39:24.755492+0000","last_clean":"2023-09-13T04:39:24.755492+0000","last_became_active":"2023-09-13T04:37:59.248483+0000","last_became_peered":"2023-09-13T04:37:59.248483+0000","last_unstale":"2023-09-13T04:39:24.755492+0000","last_undegraded":"2023-09-13T04:39:24.755492+0000","last_fullsized":"2023-09-13T04:39:24.755492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:49:56.078544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583128+0000","last_change":"2023-09-13T04:38:03.742260+0000","last_active":"2023-09-13T04:39:24.583128+0000","last_peered":"2023-09-13T04:39:24.583128+0000","last_clean":"2023-09-13T04:39:24.583128+0000","last_became_active":"2023-09-13T04:38:03.741893+0000","last_became_peered":"2023-09-13T04:38:03.741893+0000","last_unstale":"2023-09-13T04:39:24.583128+0000","last_undegraded":"2023-09-13T04:39:24.583128+0000","last_fullsized":"2023-09-13T04:39:24.583128+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:59:00.653545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.755011+0000","last_change":"2023-09-13T04:37:58.586898+0000","last_active":"2023-09-13T04:39:24.755011+0000","last_peered":"2023-09-13T04:39:24.755011+0000","last_clean":"2023-09-13T04:39:24.755011+0000","last_became_active":"2023-09-13T04:37:58.586594+0000","last_became_peered":"2023-09-13T04:37:58.586594+0000","last_unstale":"2023-09-13T04:39:24.755011+0000","last_undegraded":"2023-09-13T04:39:24.755011+0000","last_fullsized":"2023-09-13T04:39:24.755011+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:29:43.718522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.595628+0000","last_change":"2023-09-13T04:37:59.244749+0000","last_active":"2023-09-13T04:39:25.595628+0000","last_peered":"2023-09-13T04:39:25.595628+0000","last_clean":"2023-09-13T04:39:25.595628+0000","last_became_active":"2023-09-13T04:37:59.244391+0000","last_became_peered":"2023-09-13T04:37:59.244391+0000","last_unstale":"2023-09-13T04:39:25.595628+0000","last_undegraded":"2023-09-13T04:39:25.595628+0000","last_fullsized":"2023-09-13T04:39:25.595628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:33:02.195729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.753833+0000","last_change":"2023-09-13T04:38:03.609766+0000","last_active":"2023-09-13T04:39:24.753833+0000","last_peered":"2023-09-13T04:39:24.753833+0000","last_clean":"2023-09-13T04:39:24.753833+0000","last_became_active":"2023-09-13T04:38:03.609432+0000","last_became_peered":"2023-09-13T04:38:03.609432+0000","last_unstale":"2023-09-13T04:39:24.753833+0000","last_undegraded":"2023-09-13T04:39:24.753833+0000","last_fullsized":"2023-09-13T04:39:24.753833+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:30:56.115483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.583249+0000","last_change":"2023-09-13T04:37:58.589119+0000","last_active":"2023-09-13T04:39:24.583249+0000","last_peered":"2023-09-13T04:39:24.583249+0000","last_clean":"2023-09-13T04:39:24.583249+0000","last_became_active":"2023-09-13T04:37:58.588803+0000","last_became_peered":"2023-09-13T04:37:58.588803+0000","last_unstale":"2023-09-13T04:39:24.583249+0000","last_undegraded":"2023-09-13T04:39:24.583249+0000","last_fullsized":"2023-09-13T04:39:24.583249+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:51:20.888666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.585669+0000","last_change":"2023-09-13T04:37:59.243506+0000","last_active":"2023-09-13T04:39:24.585669+0000","last_peered":"2023-09-13T04:39:24.585669+0000","last_clean":"2023-09-13T04:39:24.585669+0000","last_became_active":"2023-09-13T04:37:59.243082+0000","last_became_peered":"2023-09-13T04:37:59.243082+0000","last_unstale":"2023-09-13T04:39:24.585669+0000","last_undegraded":"2023-09-13T04:39:24.585669+0000","last_fullsized":"2023-09-13T04:39:24.585669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T16:01:41.179286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754658+0000","last_change":"2023-09-13T04:38:03.609042+0000","last_active":"2023-09-13T04:39:24.754658+0000","last_peered":"2023-09-13T04:39:24.754658+0000","last_clean":"2023-09-13T04:39:24.754658+0000","last_became_active":"2023-09-13T04:38:03.608553+0000","last_became_peered":"2023-09-13T04:38:03.608553+0000","last_unstale":"2023-09-13T04:39:24.754658+0000","last_undegraded":"2023-09-13T04:39:24.754658+0000","last_fullsized":"2023-09-13T04:39:24.754658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:05:56.648475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.706558+0000","last_change":"2023-09-13T04:37:58.598056+0000","last_active":"2023-09-13T04:39:24.706558+0000","last_peered":"2023-09-13T04:39:24.706558+0000","last_clean":"2023-09-13T04:39:24.706558+0000","last_became_active":"2023-09-13T04:37:58.597724+0000","last_became_peered":"2023-09-13T04:37:58.597724+0000","last_unstale":"2023-09-13T04:39:24.706558+0000","last_undegraded":"2023-09-13T04:39:24.706558+0000","last_fullsized":"2023-09-13T04:39:24.706558+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:43:08.296866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.585419+0000","last_change":"2023-09-13T04:37:59.246505+0000","last_active":"2023-09-13T04:39:24.585419+0000","last_peered":"2023-09-13T04:39:24.585419+0000","last_clean":"2023-09-13T04:39:24.585419+0000","last_became_active":"2023-09-13T04:37:59.246024+0000","last_became_peered":"2023-09-13T04:37:59.246024+0000","last_unstale":"2023-09-13T04:39:24.585419+0000","last_undegraded":"2023-09-13T04:39:24.585419+0000","last_fullsized":"2023-09-13T04:39:24.585419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:36:58.008066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.705906+0000","last_change":"2023-09-13T04:38:03.605354+0000","last_active":"2023-09-13T04:39:24.705906+0000","last_peered":"2023-09-13T04:39:24.705906+0000","last_clean":"2023-09-13T04:39:24.705906+0000","last_became_active":"2023-09-13T04:38:03.604840+0000","last_became_peered":"2023-09-13T04:38:03.604840+0000","last_unstale":"2023-09-13T04:39:24.705906+0000","last_undegraded":"2023-09-13T04:39:24.705906+0000","last_fullsized":"2023-09-13T04:39:24.705906+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:38:02.587125+0000","last_clean_scrub_stamp":"2023-09-13T04:38:02.587125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:46:29.867914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.587864+0000","last_change":"2023-09-13T04:37:58.589052+0000","last_active":"2023-09-13T04:39:24.587864+0000","last_peered":"2023-09-13T04:39:24.587864+0000","last_clean":"2023-09-13T04:39:24.587864+0000","last_became_active":"2023-09-13T04:37:58.588707+0000","last_became_peered":"2023-09-13T04:37:58.588707+0000","last_unstale":"2023-09-13T04:39:24.587864+0000","last_undegraded":"2023-09-13T04:39:24.587864+0000","last_fullsized":"2023-09-13T04:39:24.587864+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:57.508595+0000","last_clean_scrub_stamp":"2023-09-13T04:37:57.508595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:25:23.966046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.585734+0000","last_change":"2023-09-13T04:37:59.241157+0000","last_active":"2023-09-13T04:39:24.585734+0000","last_peered":"2023-09-13T04:39:24.585734+0000","last_clean":"2023-09-13T04:39:24.585734+0000","last_became_active":"2023-09-13T04:37:59.240857+0000","last_became_peered":"2023-09-13T04:37:59.240857+0000","last_unstale":"2023-09-13T04:39:24.585734+0000","last_undegraded":"2023-09-13T04:39:24.585734+0000","last_fullsized":"2023-09-13T04:39:24.585734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:04:44.986908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.623983+0000","last_change":"2023-09-13T04:37:59.242447+0000","last_active":"2023-09-13T04:39:25.623983+0000","last_peered":"2023-09-13T04:39:25.623983+0000","last_clean":"2023-09-13T04:39:25.623983+0000","last_became_active":"2023-09-13T04:37:59.242130+0000","last_became_peered":"2023-09-13T04:37:59.242130+0000","last_unstale":"2023-09-13T04:39:25.623983+0000","last_undegraded":"2023-09-13T04:39:25.623983+0000","last_fullsized":"2023-09-13T04:39:25.623983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:12:12.844888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.854823+0000","last_change":"2023-09-13T04:38:24.595646+0000","last_active":"2023-09-13T04:39:25.854823+0000","last_peered":"2023-09-13T04:39:25.854823+0000","last_clean":"2023-09-13T04:39:25.854823+0000","last_became_active":"2023-09-13T04:38:24.595295+0000","last_became_peered":"2023-09-13T04:38:24.595295+0000","last_unstale":"2023-09-13T04:39:25.854823+0000","last_undegraded":"2023-09-13T04:39:25.854823+0000","last_fullsized":"2023-09-13T04:39:25.854823+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:24:27.174979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.854796+0000","last_change":"2023-09-13T04:37:59.245358+0000","last_active":"2023-09-13T04:39:25.854796+0000","last_peered":"2023-09-13T04:39:25.854796+0000","last_clean":"2023-09-13T04:39:25.854796+0000","last_became_active":"2023-09-13T04:37:59.245052+0000","last_became_peered":"2023-09-13T04:37:59.245052+0000","last_unstale":"2023-09-13T04:39:25.854796+0000","last_undegraded":"2023-09-13T04:39:25.854796+0000","last_fullsized":"2023-09-13T04:39:25.854796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:05:42.561716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.855825+0000","last_change":"2023-09-13T04:37:59.254053+0000","last_active":"2023-09-13T04:39:25.855825+0000","last_peered":"2023-09-13T04:39:25.855825+0000","last_clean":"2023-09-13T04:39:25.855825+0000","last_became_active":"2023-09-13T04:37:59.253544+0000","last_became_peered":"2023-09-13T04:37:59.253544+0000","last_unstale":"2023-09-13T04:39:25.855825+0000","last_undegraded":"2023-09-13T04:39:25.855825+0000","last_fullsized":"2023-09-13T04:39:25.855825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:18:54.689208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.706160+0000","last_change":"2023-09-13T04:37:59.253403+0000","last_active":"2023-09-13T04:39:25.706160+0000","last_peered":"2023-09-13T04:39:25.706160+0000","last_clean":"2023-09-13T04:39:25.706160+0000","last_became_active":"2023-09-13T04:37:59.253062+0000","last_became_peered":"2023-09-13T04:37:59.253062+0000","last_unstale":"2023-09-13T04:39:25.706160+0000","last_undegraded":"2023-09-13T04:39:25.706160+0000","last_fullsized":"2023-09-13T04:39:25.706160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:31:09.898886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.624253+0000","last_change":"2023-09-13T04:37:59.240916+0000","last_active":"2023-09-13T04:39:25.624253+0000","last_peered":"2023-09-13T04:39:25.624253+0000","last_clean":"2023-09-13T04:39:25.624253+0000","last_became_active":"2023-09-13T04:37:59.240605+0000","last_became_peered":"2023-09-13T04:37:59.240605+0000","last_unstale":"2023-09-13T04:39:25.624253+0000","last_undegraded":"2023-09-13T04:39:25.624253+0000","last_fullsized":"2023-09-13T04:39:25.624253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:15:48.241587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.855258+0000","last_change":"2023-09-13T04:37:59.238569+0000","last_active":"2023-09-13T04:39:25.855258+0000","last_peered":"2023-09-13T04:39:25.855258+0000","last_clean":"2023-09-13T04:39:25.855258+0000","last_became_active":"2023-09-13T04:37:59.238255+0000","last_became_peered":"2023-09-13T04:37:59.238255+0000","last_unstale":"2023-09-13T04:39:25.855258+0000","last_undegraded":"2023-09-13T04:39:25.855258+0000","last_fullsized":"2023-09-13T04:39:25.855258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:19:03.433404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.855917+0000","last_change":"2023-09-13T04:37:59.252243+0000","last_active":"2023-09-13T04:39:25.855917+0000","last_peered":"2023-09-13T04:39:25.855917+0000","last_clean":"2023-09-13T04:39:25.855917+0000","last_became_active":"2023-09-13T04:37:59.251902+0000","last_became_peered":"2023-09-13T04:37:59.251902+0000","last_unstale":"2023-09-13T04:39:25.855917+0000","last_undegraded":"2023-09-13T04:39:25.855917+0000","last_fullsized":"2023-09-13T04:39:25.855917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:35:57.395627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-13T04:39:25.709975+0000","last_change":"2023-09-13T04:37:59.245847+0000","last_active":"2023-09-13T04:39:25.709975+0000","last_peered":"2023-09-13T04:39:25.709975+0000","last_clean":"2023-09-13T04:39:25.709975+0000","last_became_active":"2023-09-13T04:37:59.245542+0000","last_became_peered":"2023-09-13T04:37:59.245542+0000","last_unstale":"2023-09-13T04:39:25.709975+0000","last_undegraded":"2023-09-13T04:39:25.709975+0000","last_fullsized":"2023-09-13T04:39:25.709975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:24:16.813333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754898+0000","last_change":"2023-09-13T04:37:59.246963+0000","last_active":"2023-09-13T04:39:24.754898+0000","last_peered":"2023-09-13T04:39:24.754898+0000","last_clean":"2023-09-13T04:39:24.754898+0000","last_became_active":"2023-09-13T04:37:59.246448+0000","last_became_peered":"2023-09-13T04:37:59.246448+0000","last_unstale":"2023-09-13T04:39:24.754898+0000","last_undegraded":"2023-09-13T04:39:24.754898+0000","last_fullsized":"2023-09-13T04:39:24.754898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:25:45.418778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.753993+0000","last_change":"2023-09-13T04:37:59.245399+0000","last_active":"2023-09-13T04:39:24.753993+0000","last_peered":"2023-09-13T04:39:24.753993+0000","last_clean":"2023-09-13T04:39:24.753993+0000","last_became_active":"2023-09-13T04:37:59.245060+0000","last_became_peered":"2023-09-13T04:37:59.245060+0000","last_unstale":"2023-09-13T04:39:24.753993+0000","last_undegraded":"2023-09-13T04:39:24.753993+0000","last_fullsized":"2023-09-13T04:39:24.753993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:25:41.078238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.587934+0000","last_change":"2023-09-13T04:37:59.244776+0000","last_active":"2023-09-13T04:39:24.587934+0000","last_peered":"2023-09-13T04:39:24.587934+0000","last_clean":"2023-09-13T04:39:24.587934+0000","last_became_active":"2023-09-13T04:37:59.244409+0000","last_became_peered":"2023-09-13T04:37:59.244409+0000","last_unstale":"2023-09-13T04:39:24.587934+0000","last_undegraded":"2023-09-13T04:39:24.587934+0000","last_fullsized":"2023-09-13T04:39:24.587934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:04:02.561414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-13T04:39:24.754806+0000","last_change":"2023-09-13T04:37:59.248153+0000","last_active":"2023-09-13T04:39:24.754806+0000","last_peered":"2023-09-13T04:39:24.754806+0000","last_clean":"2023-09-13T04:39:24.754806+0000","last_became_active":"2023-09-13T04:37:59.247840+0000","last_became_peered":"2023-09-13T04:37:59.247840+0000","last_unstale":"2023-09-13T04:39:24.754806+0000","last_undegraded":"2023-09-13T04:39:24.754806+0000","last_fullsized":"2023-09-13T04:39:24.754806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T04:37:58.565164+0000","last_clean_scrub_stamp":"2023-09-13T04:37:58.565164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:52:53.021219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7710,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":646,"num_read_kb":337,"num_write":131,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":23130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416423,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":509914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.625,"15min":0.61599999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":2.8420000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.54600000000000004}]},{"osd":1,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.66800000000000004,"15min":0.66100000000000003},"min":{"1min":0.46999999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92700000000000005,"5min":1.024,"15min":1.024},"last":0.625},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.42099999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80600000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.47299999999999998,"15min":0.46899999999999997},"min":{"1min":0.34799999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.68400000000000005}]},{"osd":3,"last update":"Wed Sep 13 04:39:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.74399999999999999}]},{"osd":4,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.68600000000000005,"15min":0.67800000000000005},"min":{"1min":0.53100000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.42899999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86699999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.71099999999999997}]},{"osd":5,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.64700000000000002,"15min":0.63100000000000001},"min":{"1min":0.45300000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.84799999999999998}]},{"osd":6,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70599999999999996,"15min":0.70399999999999996},"min":{"1min":0.46999999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.70799999999999996,"15min":0.69799999999999995},"min":{"1min":0.55900000000000005,"5min":0.498,"15min":0.498},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.878}]},{"osd":7,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.77500000000000002,"15min":0.76500000000000001},"min":{"1min":0.59599999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0229999999999999,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.50800000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94099999999999995,"5min":1.262,"15min":1.262},"last":0.83899999999999997}]},{"osd":8,"last update":"Wed Sep 13 04:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.74199999999999999,"15min":0.73899999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.88200000000000001,"15min":0.88600000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.0840000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.89400000000000002}]},{"osd":9,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.77400000000000002,"15min":0.76200000000000001},"min":{"1min":0.61499999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.121,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.995},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.82499999999999996,"15min":0.81100000000000005},"min":{"1min":0.66200000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1830000000000001,"5min":1.361,"15min":1.361},"last":0.97599999999999998}]},{"osd":10,"last update":"Wed Sep 13 04:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.78900000000000003,"15min":0.78600000000000003},"min":{"1min":0.56000000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.002,"5min":1.155,"15min":1.155},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.77300000000000002,"15min":0.76700000000000002},"min":{"1min":0.59599999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95699999999999996,"5min":1.282,"15min":1.282},"last":0.95799999999999996}]}]},{"osd":10,"up_from":71,"seq":304942678059,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1104951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.74399999999999999,"15min":0.76800000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.84499999999999997,"5min":4.0510000000000002,"15min":4.0510000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.77000000000000002,"15min":0.80300000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82899999999999996,"5min":4.1120000000000001,"15min":4.1120000000000001},"last":0.70899999999999996}]},{"osd":1,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.46000000000000002,"15min":0.45600000000000002},"min":{"1min":0.35399999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72799999999999998,"5min":0.751,"15min":0.751},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.47499999999999998,"15min":0.46999999999999997},"min":{"1min":0.34999999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63500000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.58399999999999996}]},{"osd":2,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.626,"15min":0.628},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.755,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80900000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.81200000000000006}]},{"osd":3,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.69399999999999995,"15min":0.66800000000000004},"min":{"1min":0.41799999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.754,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.73199999999999998}]},{"osd":4,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.371,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76300000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.879,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.62}]},{"osd":5,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69799999999999995,"15min":0.69299999999999995},"min":{"1min":0.505,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84999999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.45900000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.75}]},{"osd":6,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.81699999999999995,"15min":0.86799999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90100000000000002,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.85999999999999999,"15min":0.82599999999999996},"min":{"1min":0.54400000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96599999999999997,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":0.78800000000000003}]},{"osd":7,"last update":"Wed Sep 13 04:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.64500000000000002}]},{"osd":8,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77000000000000002},"min":{"1min":0.61199999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":0.998,"15min":0.998},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.78300000000000003,"15min":0.80100000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.99299999999999999,"5min":1.03,"15min":1.03},"last":0.94899999999999995}]},{"osd":9,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.86699999999999999,"15min":0.85099999999999998},"min":{"1min":0.65400000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0640000000000001,"5min":4.2679999999999998,"15min":4.2679999999999998},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.84999999999999998,"15min":0.85099999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.96799999999999997,"5min":4.3600000000000003,"15min":4.3600000000000003},"last":0.80000000000000004}]},{"osd":11,"last update":"Wed Sep 13 04:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.82799999999999996,"15min":0.82999999999999996},"min":{"1min":0.64100000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.80700000000000005,"15min":0.80600000000000005},"min":{"1min":0.65300000000000002,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.97099999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.92000000000000004}]}]},{"osd":9,"up_from":66,"seq":283467841584,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":510004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.58899999999999997,"15min":0.58699999999999997},"min":{"1min":0.38,"5min":0.316,"15min":0.316},"max":{"1min":1.5129999999999999,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.60199999999999998}]},{"osd":1,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.69499999999999995,"15min":0.70699999999999996},"min":{"1min":0.48199999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.036,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71499999999999997,"15min":0.73099999999999998},"min":{"1min":0.46100000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.78900000000000003}]},{"osd":2,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73599999999999999,"15min":0.752},"min":{"1min":0.48199999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.026,"5min":1.083,"15min":1.083},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73899999999999999,"15min":0.751},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.57899999999999996}]},{"osd":3,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68000000000000005,"15min":0.67900000000000005},"min":{"1min":0.42299999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.5549999999999999,"5min":2.258,"15min":2.258},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.5289999999999999,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":0.505}]},{"osd":4,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.78100000000000003,"15min":0.80300000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.79600000000000004,"15min":0.80900000000000005},"min":{"1min":0.57299999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.95199999999999996}]},{"osd":5,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84399999999999997,"15min":0.86399999999999999},"min":{"1min":0.57199999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.296,"5min":1.339,"15min":1.339},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79900000000000004,"15min":0.81299999999999994},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.071,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.625}]},{"osd":6,"last update":"Wed Sep 13 04:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.72099999999999997}]},{"osd":7,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.88,"15min":0.90700000000000003},"min":{"1min":0.57099999999999995,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.874},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.88200000000000001,"15min":0.91800000000000004},"min":{"1min":0.621,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1060000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.71399999999999997}]},{"osd":8,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.90100000000000002,"15min":0.93500000000000005},"min":{"1min":0.71399999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.878,"5min":0.93500000000000005,"15min":0.96399999999999997},"min":{"1min":0.57999999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3380000000000001,"5min":1.857,"15min":1.857},"last":0.82799999999999996}]},{"osd":10,"last update":"Wed Sep 13 04:39:56 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89200000000000002,"15min":0.89800000000000002},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.879,"5min":0.89200000000000002,"15min":0.91000000000000003},"min":{"1min":0.70799999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1639999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.89000000000000001}]},{"osd":11,"last update":"Wed Sep 13 04:40:06 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.90700000000000003,"15min":0.91100000000000003},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.0249999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.86799999999999999,"15min":0.86399999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.85899999999999999}]}]},{"osd":8,"up_from":58,"seq":249108103220,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":509481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.441,"5min":0.317,"15min":0.317},"max":{"1min":0.89000000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58799999999999997,"15min":0.57099999999999995},"min":{"1min":0.39600000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98499999999999999,"5min":1.016,"15min":1.016},"last":0.42199999999999999}]},{"osd":1,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.59999999999999998,"15min":0.55400000000000005},"min":{"1min":0.53100000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.79700000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.752},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.61899999999999999,"15min":0.59199999999999997},"min":{"1min":0.53400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82799999999999996,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.63600000000000001}]},{"osd":2,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48099999999999998,"15min":0.47499999999999998},"min":{"1min":0.40200000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76300000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.439},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.49199999999999999,"15min":0.48699999999999999},"min":{"1min":0.373,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.73099999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.38}]},{"osd":3,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.90600000000000003,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.65500000000000003,"15min":0.64600000000000002},"min":{"1min":0.53600000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.57199999999999995}]},{"osd":4,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.63100000000000001,"15min":0.59799999999999998},"min":{"1min":0.497,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.90100000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.65900000000000003,"15min":0.627},"min":{"1min":0.55400000000000005,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.71499999999999997}]},{"osd":5,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58799999999999997,"15min":0.58199999999999996},"min":{"1min":0.437,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.89800000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.61199999999999999,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.54100000000000004}]},{"osd":6,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.56999999999999995,"5min":0.504,"15min":0.504},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.64900000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0229999999999999,"5min":1.077,"15min":1.077},"last":0.58999999999999997}]},{"osd":7,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72999999999999998,"15min":0.71599999999999997},"min":{"1min":0.47999999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.97199999999999998,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.73899999999999999,"15min":0.70299999999999996},"min":{"1min":0.66300000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0309999999999999,"5min":1.04,"15min":1.04},"last":0.76500000000000001}]},{"osd":9,"last update":"Wed Sep 13 04:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.78100000000000003,"15min":0.76500000000000001},"min":{"1min":0.66700000000000004,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0640000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.78500000000000003,"15min":0.76200000000000001},"min":{"1min":0.68500000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.62}]},{"osd":10,"last update":"Wed Sep 13 04:39:58 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.77200000000000002,"15min":0.74399999999999999},"min":{"1min":0.59499999999999997,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0089999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.78200000000000003,"15min":0.76300000000000001},"min":{"1min":0.64000000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.80600000000000005}]},{"osd":11,"last update":"Wed Sep 13 04:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.63800000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.87,"5min":0.83199999999999996,"15min":0.82599999999999996},"min":{"1min":0.66500000000000004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0329999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.73199999999999998}]}]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1642496,"data_stored":1101346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.46500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.746,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60499999999999998,"15min":0.60799999999999998},"min":{"1min":0.41599999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.40200000000000002}]},{"osd":1,"last update":"Wed Sep 13 04:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.433,"15min":0.41099999999999998},"min":{"1min":0.33900000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.56200000000000006,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.498,"5min":0.44800000000000001,"15min":0.42899999999999999},"min":{"1min":0.316,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.66400000000000003}]},{"osd":2,"last update":"Wed Sep 13 04:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60799999999999998,"15min":0.59299999999999997},"min":{"1min":0.46999999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78700000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61499999999999999,"15min":0.58299999999999996},"min":{"1min":0.47299999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76400000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.80300000000000005}]},{"osd":3,"last update":"Wed Sep 13 04:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71099999999999997,"15min":0.65700000000000003},"min":{"1min":0.496,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7410000000000001,"5min":7.0789999999999997,"15min":7.0789999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68000000000000005,"15min":0.67400000000000004},"min":{"1min":0.42999999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.54800000000000004}]},{"osd":4,"last update":"Wed Sep 13 04:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52100000000000002,"15min":0.51300000000000001},"min":{"1min":0.39100000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71799999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50600000000000001,"15min":0.51300000000000001},"min":{"1min":0.36099999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.78600000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.82299999999999995}]},{"osd":5,"last update":"Wed Sep 13 04:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64300000000000002,"15min":0.63100000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.80700000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.67900000000000005,"15min":0.65300000000000002},"min":{"1min":0.52200000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89600000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.89300000000000002}]},{"osd":6,"last update":"Wed Sep 13 04:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.751,"15min":0.73099999999999998},"min":{"1min":0.53300000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":3.496,"5min":3.496,"15min":3.496},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72799999999999998,"15min":0.72299999999999998},"min":{"1min":0.55600000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.7949999999999999,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.63500000000000001}]},{"osd":8,"last update":"Wed Sep 13 04:39:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.71699999999999997,"15min":0.70899999999999996},"min":{"1min":0.59399999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72099999999999997,"15min":0.71699999999999997},"min":{"1min":0.56599999999999995,"5min":0.437,"15min":0.437},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.85299999999999998}]},{"osd":9,"last update":"Wed Sep 13 04:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.79600000000000004,"15min":0.77500000000000002},"min":{"1min":0.64600000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.77900000000000003,"15min":0.76500000000000001},"min":{"1min":0.63900000000000001,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.91000000000000003}]},{"osd":10,"last update":"Wed Sep 13 04:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69899999999999995,"15min":0.69799999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.97099999999999997,"5min":0.999,"15min":0.999},"last":0.876},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.73599999999999999,"15min":0.72299999999999998},"min":{"1min":0.54300000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98399999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.95499999999999996}]},{"osd":11,"last update":"Wed Sep 13 04:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.80800000000000005,"15min":0.81000000000000005},"min":{"1min":0.58299999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92700000000000005,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.86599999999999999,"15min":0.86699999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.97699999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":509276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.51200000000000001,"15min":0.46800000000000003},"min":{"1min":0.33600000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":2.738,"5min":2.738,"15min":2.738},"last":0.45600000000000002},{"interface":"front","average":{"1min":1.0920000000000001,"5min":0.625,"15min":0.53100000000000003},"min":{"1min":0.41599999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":8.5749999999999993,"5min":8.5749999999999993,"15min":8.5749999999999993},"last":0.504}]},{"osd":1,"last update":"Wed Sep 13 04:39:20 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":0.74099999999999999,"15min":0.64500000000000002},"min":{"1min":0.52700000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":10.374000000000001,"5min":10.374000000000001,"15min":10.374000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.72999999999999998,"15min":0.68600000000000005},"min":{"1min":0.51500000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.746}]},{"osd":2,"last update":"Wed Sep 13 04:39:20 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":0.78100000000000003,"15min":0.71699999999999997},"min":{"1min":0.505,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":8.4740000000000002,"5min":8.4740000000000002,"15min":8.4740000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.69799999999999995,"15min":0.68400000000000005},"min":{"1min":0.55900000000000005,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.52600000000000002}]},{"osd":3,"last update":"Wed Sep 13 04:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.69899999999999995,"15min":0.65400000000000003},"min":{"1min":0.46000000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.331,"5min":2.331,"15min":2.331},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.67800000000000005,"15min":0.63500000000000001},"min":{"1min":0.45000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.585,"5min":2.585,"15min":2.585},"last":0.48099999999999998}]},{"osd":4,"last update":"Wed Sep 13 04:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.78000000000000003,"15min":0.752},"min":{"1min":0.60699999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.2809999999999999,"5min":0.85999999999999999,"15min":0.80400000000000005},"min":{"1min":0.58199999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":8.3350000000000009,"5min":8.3350000000000009,"15min":8.3350000000000009},"last":0.85199999999999998}]},{"osd":5,"last update":"Wed Sep 13 04:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.82699999999999996,"15min":0.79900000000000004},"min":{"1min":0.52500000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.288,"5min":0.88600000000000001,"15min":0.83999999999999997},"min":{"1min":0.54100000000000004,"5min":0.379,"15min":0.379},"max":{"1min":8.5060000000000002,"5min":8.5060000000000002,"15min":8.5060000000000002},"last":0.79300000000000004}]},{"osd":7,"last update":"Wed Sep 13 04:39:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.87,"15min":0.83399999999999996},"min":{"1min":0.61199999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90300000000000002,"15min":0.82299999999999995},"min":{"1min":0.59599999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.702,"5min":8.3970000000000002,"15min":8.3970000000000002},"last":0.89000000000000001}]},{"osd":8,"last update":"Wed Sep 13 04:40:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.80900000000000005,"15min":0.82099999999999995},"min":{"1min":0.625,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.256,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.92200000000000004,"15min":0.92500000000000004},"min":{"1min":0.64700000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3620000000000001,"5min":1.446,"15min":1.446},"last":0.871}]},{"osd":9,"last update":"Wed Sep 13 04:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.876,"15min":0.88600000000000001},"min":{"1min":0.58299999999999996,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.026,"15min":1.0329999999999999},"min":{"1min":0.72399999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.6960000000000002,"5min":2.6960000000000002,"15min":2.6960000000000002},"last":0.94399999999999995}]},{"osd":10,"last update":"Wed Sep 13 04:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.98999999999999999,"15min":0.99199999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.96699999999999997,"15min":0.89900000000000002},"min":{"1min":0.71299999999999997,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.732,"5min":8.4380000000000006,"15min":8.4380000000000006},"last":0.92600000000000005}]},{"osd":11,"last update":"Wed Sep 13 04:40:06 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":1.272,"15min":1.3380000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.4139999999999999,"5min":8.5410000000000004,"15min":8.5410000000000004},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.93200000000000005,"15min":0.94399999999999995},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.3080000000000001,"5min":1.341,"15min":1.341},"last":0.83699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":513286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.621,"15min":0.55900000000000005},"min":{"1min":0.47199999999999998,"5min":0.307,"15min":0.307},"max":{"1min":4.1150000000000002,"5min":4.1150000000000002,"15min":4.1150000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":1.165,"5min":0.71499999999999997,"15min":0.60099999999999998},"min":{"1min":0.45100000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":12.896000000000001,"5min":12.896000000000001,"15min":12.896000000000001},"last":0.63400000000000001}]},{"osd":2,"last update":"Wed Sep 13 04:39:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.60899999999999999,"15min":0.55500000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88300000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61799999999999999,"15min":0.54200000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.877,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.78500000000000003}]},{"osd":3,"last update":"Wed Sep 13 04:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.58999999999999997},"min":{"1min":0.41699999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.64500000000000002,"15min":0.61699999999999999},"min":{"1min":0.50800000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.68700000000000006}]},{"osd":4,"last update":"Wed Sep 13 04:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48299999999999998,"15min":0.46000000000000002},"min":{"1min":0.378,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63700000000000001,"5min":0.878,"15min":0.878},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.48899999999999999,"15min":0.45900000000000002},"min":{"1min":0.34699999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.82299999999999995,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.501}]},{"osd":5,"last update":"Wed Sep 13 04:39:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.625,"15min":0.57999999999999996},"min":{"1min":0.378,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85299999999999998,"5min":1.3,"15min":1.3},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.68400000000000005,"15min":0.60599999999999998},"min":{"1min":0.46400000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.0089999999999999,"5min":1.264,"15min":1.264},"last":0.75900000000000001}]},{"osd":6,"last update":"Wed Sep 13 04:39:22 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":0.86799999999999999,"15min":0.75},"min":{"1min":0.60199999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":12.640000000000001,"5min":12.640000000000001,"15min":12.640000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.3160000000000001,"5min":0.83299999999999996,"15min":0.73999999999999999},"min":{"1min":0.56000000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":12.675000000000001,"5min":12.675000000000001,"15min":12.675000000000001},"last":0.79700000000000004}]},{"osd":7,"last update":"Wed Sep 13 04:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.66000000000000003,"15min":0.60499999999999998},"min":{"1min":0.50700000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.1399999999999999,"5min":1.347,"15min":1.347},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.70999999999999996,"15min":0.66000000000000003},"min":{"1min":0.42699999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.96899999999999997,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.81000000000000005}]},{"osd":8,"last update":"Wed Sep 13 04:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.75700000000000001,"15min":0.74199999999999999},"min":{"1min":0.48099999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0980000000000001,"5min":1.393,"15min":1.393},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.73999999999999999,"15min":0.72399999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.032,"5min":1.224,"15min":1.224},"last":0.745}]},{"osd":9,"last update":"Wed Sep 13 04:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.878,"15min":0.86699999999999999},"min":{"1min":0.64700000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.8250000000000002,"5min":2.8250000000000002,"15min":2.8250000000000002},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.879,"15min":0.871},"min":{"1min":0.66600000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":2.9569999999999999,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":0.85099999999999998}]},{"osd":10,"last update":"Wed Sep 13 04:39:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.81799999999999995,"15min":0.82199999999999995},"min":{"1min":0.51500000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.038,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.88},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.85399999999999998,"15min":0.85499999999999998},"min":{"1min":0.54700000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0780000000000001,"5min":1.504,"15min":1.504},"last":0.86599999999999999}]},{"osd":11,"last update":"Wed Sep 13 04:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.88800000000000001,"15min":0.88400000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0660000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.91700000000000004,"15min":0.91900000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.125,"5min":1.129,"15min":1.129},"last":0.89400000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":510664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 13 04:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.748,"15min":0.61199999999999999},"min":{"1min":0.45700000000000002,"5min":0.33100000000000002,"15min":0.29899999999999999},"max":{"1min":1.125,"5min":6.2359999999999998,"15min":6.2359999999999998},"last":1.01},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64900000000000002,"15min":0.57299999999999995},"min":{"1min":0.40600000000000003,"5min":0.38800000000000001,"15min":0.31900000000000001},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":1.0329999999999999}]},{"osd":2,"last update":"Wed Sep 13 04:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65900000000000003,"15min":0.61099999999999999},"min":{"1min":0.52300000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96499999999999997,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.70899999999999996,"15min":0.627},"min":{"1min":0.47999999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed Sep 13 04:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.88500000000000001,"5min":1.25,"15min":1.25},"last":0.628},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.90300000000000002,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.89800000000000002}]},{"osd":4,"last update":"Wed Sep 13 04:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.751,"15min":0.64400000000000002},"min":{"1min":0.48399999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.052},{"interface":"front","average":{"1min":0.748,"5min":0.70599999999999996,"15min":0.64000000000000001},"min":{"1min":0.46899999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.85099999999999998}]},{"osd":5,"last update":"Wed Sep 13 04:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69699999999999995,"15min":0.67500000000000004},"min":{"1min":0.48799999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89100000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.878,"5min":0.78100000000000003,"15min":0.70699999999999996},"min":{"1min":0.63500000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.2130000000000001,"5min":1.444,"15min":1.444},"last":1.155}]},{"osd":6,"last update":"Wed Sep 13 04:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.70899999999999996,"15min":0.70199999999999996},"min":{"1min":0.50700000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.751,"15min":0.73299999999999998},"min":{"1min":0.45100000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.65400000000000003}]},{"osd":7,"last update":"Wed Sep 13 04:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.82999999999999996,"15min":0.83699999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1459999999999999,"5min":1.333,"15min":1.333},"last":0.873},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.93500000000000005,"15min":0.94299999999999995},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.258,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.075}]},{"osd":8,"last update":"Wed Sep 13 04:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.91000000000000003,"15min":0.86199999999999999},"min":{"1min":0.61599999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0609999999999999,"5min":6.2089999999999996,"15min":6.2089999999999996},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.93500000000000005,"15min":0.84099999999999997},"min":{"1min":0.73199999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2350000000000001,"5min":6.1699999999999999,"15min":6.1699999999999999},"last":1.135}]},{"osd":9,"last update":"Wed Sep 13 04:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.90800000000000003,"15min":0.94099999999999995},"min":{"1min":0.56799999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.94499999999999995,"15min":0.93700000000000006},"min":{"1min":0.69399999999999995,"5min":0.433,"15min":0.433},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.96399999999999997}]},{"osd":10,"last update":"Wed Sep 13 04:39:51 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.99399999999999999,"15min":0.98599999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2989999999999999,"5min":1.591,"15min":1.591},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":1.0449999999999999,"15min":1.119},"min":{"1min":0.623,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1950000000000001,"5min":6.1859999999999999,"15min":6.1859999999999999},"last":0.93500000000000005}]},{"osd":11,"last update":"Wed Sep 13 04:39:16 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":0.99099999999999999,"15min":0.98399999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.175},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.86099999999999999,"15min":0.85299999999999998},"min":{"1min":0.71499999999999997,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.99299999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":514602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55600000000000005,"15min":0.53000000000000003},"min":{"1min":0.34200000000000003,"5min":0.29399999999999998,"15min":0.28599999999999998},"max":{"1min":0.86099999999999999,"5min":1.74,"15min":1.74},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55400000000000005,"15min":0.53000000000000003},"min":{"1min":0.38700000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.77400000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.372}]},{"osd":1,"last update":"Wed Sep 13 04:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60799999999999998,"15min":0.52000000000000002},"min":{"1min":0.42399999999999999,"5min":0.34300000000000003,"15min":0.30599999999999999},"max":{"1min":0.91300000000000003,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.61099999999999999,"15min":0.53000000000000003},"min":{"1min":0.44600000000000001,"5min":0.36599999999999999,"15min":0.34300000000000003},"max":{"1min":0.80800000000000005,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.48799999999999999}]},{"osd":3,"last update":"Wed Sep 13 04:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.61899999999999999,"15min":0.59199999999999997},"min":{"1min":0.45700000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.623,"15min":0.60999999999999999},"min":{"1min":0.48899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.53400000000000003}]},{"osd":4,"last update":"Wed Sep 13 04:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.54500000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55400000000000005}]},{"osd":5,"last update":"Wed Sep 13 04:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46999999999999997,"15min":0.49299999999999999},"min":{"1min":0.376,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.621,"5min":1.796,"15min":1.796},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34200000000000003,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.94799999999999995,"5min":1.546,"15min":1.546},"last":0.69899999999999995}]},{"osd":6,"last update":"Wed Sep 13 04:39:19 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.753,"15min":0.71899999999999997},"min":{"1min":0.57699999999999996,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":4.0730000000000004,"5min":4.0730000000000004,"15min":4.0730000000000004},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.56299999999999994,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.57599999999999996}]},{"osd":7,"last update":"Wed Sep 13 04:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.77900000000000003,"15min":0.77100000000000002},"min":{"1min":0.56399999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76800000000000002,"15min":0.76200000000000001},"min":{"1min":0.63400000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.51900000000000002}]},{"osd":8,"last update":"Wed Sep 13 04:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.63900000000000001,"15min":0.627},"min":{"1min":0.44900000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.96699999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.69499999999999995,"15min":0.66300000000000003},"min":{"1min":0.46999999999999997,"5min":0.377,"15min":0.377},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.78000000000000003}]},{"osd":9,"last update":"Wed Sep 13 04:39:31 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.755,"15min":0.73099999999999998},"min":{"1min":0.65200000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.623},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.75600000000000001,"15min":0.72599999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":0.67800000000000005}]},{"osd":10,"last update":"Wed Sep 13 04:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71999999999999997,"15min":0.69599999999999995},"min":{"1min":0.63700000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.78300000000000003,"15min":0.745},"min":{"1min":0.65600000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.89400000000000002}]},{"osd":11,"last update":"Wed Sep 13 04:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.76600000000000001,"15min":0.76800000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.94199999999999995,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.877,"15min":0.88300000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.85799999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":509470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:39:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.45500000000000002,"15min":0.39900000000000002},"min":{"1min":0.30399999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.52900000000000003,"15min":0.45600000000000002},"min":{"1min":0.34399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.5940000000000003,"5min":4.5940000000000003,"15min":4.5940000000000003},"last":0.52100000000000002}]},{"osd":1,"last update":"Wed Sep 13 04:39:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.64000000000000001,"15min":0.58699999999999997},"min":{"1min":0.42399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.68000000000000005,"15min":0.58999999999999997},"min":{"1min":0.50600000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.67600000000000005}]},{"osd":2,"last update":"Wed Sep 13 04:39:13 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.72799999999999998,"15min":0.64700000000000002},"min":{"1min":0.47699999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":4.3920000000000003,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.65900000000000003,"15min":0.626},"min":{"1min":0.46100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97999999999999998,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.65200000000000002}]},{"osd":4,"last update":"Wed Sep 13 04:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.70399999999999996,"15min":0.621},"min":{"1min":0.47699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0649999999999999,"5min":1.147,"15min":1.147},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69599999999999995,"15min":0.626},"min":{"1min":0.498,"5min":0.439,"15min":0.439},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.60699999999999998}]},{"osd":5,"last update":"Wed Sep 13 04:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72299999999999998,"15min":0.66100000000000003},"min":{"1min":0.52000000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0109999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.077,"5min":0.86399999999999999,"15min":0.71299999999999997},"min":{"1min":0.48399999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":6.7030000000000003,"5min":6.7030000000000003,"15min":6.7030000000000003},"last":0.90000000000000002}]},{"osd":6,"last update":"Wed Sep 13 04:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.79300000000000004,"15min":0.72099999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":4.5529999999999999,"5min":4.5529999999999999,"15min":4.5529999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.78000000000000003,"15min":0.749},"min":{"1min":0.52200000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.63200000000000001}]},{"osd":7,"last update":"Wed Sep 13 04:39:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.84899999999999998,"15min":0.85899999999999999},"min":{"1min":0.56899999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.88,"15min":0.85799999999999998},"min":{"1min":0.59899999999999998,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.153,"5min":4.4699999999999998,"15min":4.4699999999999998},"last":0.98599999999999999}]},{"osd":8,"last update":"Wed Sep 13 04:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.875}]},{"osd":9,"last update":"Wed Sep 13 04:39:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":0.96299999999999997,"15min":0.96899999999999997},"min":{"1min":0.58799999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":4.4329999999999998,"5min":4.4329999999999998,"15min":4.4329999999999998},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.82499999999999996,"15min":0.84299999999999997},"min":{"1min":0.51200000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.76900000000000002}]},{"osd":10,"last update":"Wed Sep 13 04:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.89100000000000001,"15min":0.89600000000000002},"min":{"1min":0.71599999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1499999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.96699999999999997,"15min":0.95899999999999996},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.2370000000000001,"5min":4.5099999999999998,"15min":4.5099999999999998},"last":1.0209999999999999}]},{"osd":11,"last update":"Wed Sep 13 04:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.879,"15min":0.88900000000000001},"min":{"1min":0.71999999999999997,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.95699999999999996,"5min":1.27,"15min":1.27},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.0620000000000001,"5min":0.98199999999999998,"15min":0.96899999999999997},"min":{"1min":0.80000000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":4.2530000000000001,"5min":4.2530000000000001,"15min":4.2530000000000001},"last":0.86899999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":509830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.56599999999999995,"15min":0.54400000000000004},"min":{"1min":0.43099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.753},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76500000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.68999999999999995}]},{"osd":1,"last update":"Wed Sep 13 04:39:42 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.41899999999999998,"15min":0.36199999999999999},"min":{"1min":0.34999999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.58099999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.433},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.434,"15min":0.38},"min":{"1min":0.36899999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.71499999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.48199999999999998}]},{"osd":2,"last update":"Wed Sep 13 04:39:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.61099999999999999,"15min":0.57099999999999995},"min":{"1min":0.39700000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81299999999999994,"5min":1.631,"15min":1.631},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.61299999999999999,"15min":0.55300000000000005},"min":{"1min":0.439,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76400000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.59799999999999998}]},{"osd":3,"last update":"Wed Sep 13 04:39:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.63500000000000001,"15min":0.57899999999999996},"min":{"1min":0.47799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.61699999999999999,"15min":0.58499999999999996},"min":{"1min":0.47899999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0009999999999999,"5min":1.115,"15min":1.115},"last":0.58299999999999996}]},{"osd":5,"last update":"Wed Sep 13 04:40:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64200000000000002,"15min":0.59899999999999998},"min":{"1min":0.51900000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81499999999999995,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.753},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63800000000000001,"15min":0.61599999999999999},"min":{"1min":0.42599999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.51400000000000001}]},{"osd":6,"last update":"Wed Sep 13 04:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67700000000000005,"15min":0.68300000000000005},"min":{"1min":0.52000000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70699999999999996,"15min":0.69699999999999995},"min":{"1min":0.59499999999999997,"5min":0.443,"15min":0.443},"max":{"1min":0.89000000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.79800000000000004}]},{"osd":7,"last update":"Wed Sep 13 04:39:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54700000000000004,"15min":0.52700000000000002},"min":{"1min":0.39500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63400000000000001,"15min":0.60399999999999998},"min":{"1min":0.47799999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.77800000000000002}]},{"osd":8,"last update":"Wed Sep 13 04:40:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.753,"15min":0.70699999999999996},"min":{"1min":0.65300000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93400000000000005,"5min":1.042,"15min":1.042},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.77500000000000002,"15min":0.76600000000000001},"min":{"1min":0.59799999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.97799999999999998,"5min":1.095,"15min":1.095},"last":0.81100000000000005}]},{"osd":9,"last update":"Wed Sep 13 04:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80400000000000005,"15min":0.81100000000000005},"min":{"1min":0.65000000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.03,"5min":1.141,"15min":1.141},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.624,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.94599999999999995,"5min":1.056,"15min":1.056},"last":0.73499999999999999}]},{"osd":10,"last update":"Wed Sep 13 04:39:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.754,"15min":0.75800000000000001},"min":{"1min":0.55100000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78900000000000003,"15min":0.79200000000000004},"min":{"1min":0.63900000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.88300000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.82699999999999996}]},{"osd":11,"last update":"Wed Sep 13 04:40:00 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.61399999999999999,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.71999999999999997,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":0.97399999999999998,"5min":0.997,"15min":0.997},"last":0.85399999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1654784,"data_stored":1101030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 04:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.438,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69299999999999995,"5min":0.754,"15min":0.754},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.52300000000000002},"min":{"1min":0.45500000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.76300000000000001,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.434}]},{"osd":1,"last update":"Wed Sep 13 04:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57099999999999995,"15min":0.53400000000000003},"min":{"1min":0.373,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.745,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.59899999999999998,"15min":0.54300000000000004},"min":{"1min":0.42399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77700000000000002,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.57299999999999995}]},{"osd":2,"last update":"Wed Sep 13 04:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42399999999999999,"15min":0.376},"min":{"1min":0.27900000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.59699999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.435,"15min":0.39100000000000001},"min":{"1min":0.315,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.55300000000000005,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.47399999999999998}]},{"osd":3,"last update":"Wed Sep 13 04:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.38600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.70899999999999996,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.67100000000000004,"15min":0.82999999999999996},"min":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.745,"5min":8.0009999999999994,"15min":8.0009999999999994},"last":0.66100000000000003}]},{"osd":4,"last update":"Wed Sep 13 04:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.628,"15min":0.57799999999999996},"min":{"1min":0.44700000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83699999999999997,"5min":1.599,"15min":1.599},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.627,"15min":0.57199999999999995},"min":{"1min":0.49199999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.82499999999999996,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.58499999999999996}]},{"osd":6,"last update":"Wed Sep 13 04:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.65600000000000003,"15min":0.65000000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.65000000000000002,"15min":0.63800000000000001},"min":{"1min":0.54700000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85899999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.55600000000000005}]},{"osd":7,"last update":"Wed Sep 13 04:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.46600000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94399999999999995,"5min":1.488,"15min":1.488},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69199999999999995,"15min":0.67300000000000004},"min":{"1min":0.58799999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.876,"5min":1.51,"15min":1.51},"last":0.59999999999999998}]},{"osd":8,"last update":"Wed Sep 13 04:40:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61199999999999999,"15min":0.59399999999999997},"min":{"1min":0.46899999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.63700000000000001,"15min":0.61299999999999999},"min":{"1min":0.5,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.68200000000000005}]},{"osd":9,"last update":"Wed Sep 13 04:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.50800000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.91400000000000003,"5min":1.008,"15min":1.008},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.74299999999999999,"15min":0.73999999999999999},"min":{"1min":0.60899999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.92900000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.61599999999999999}]},{"osd":10,"last update":"Wed Sep 13 04:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.74299999999999999,"15min":0.72499999999999998},"min":{"1min":0.51100000000000001,"5min":0.496,"15min":0.496},"max":{"1min":0.97199999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76200000000000001,"15min":0.75},"min":{"1min":0.63500000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.91900000000000004,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.749}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":4100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-13T04:40:14.073 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T04:40:14.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.306+0000 7f8957fff700 1 -- 172.21.15.2:0/2575127859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89501017a0 msgr2=0x7f8950103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.307+0000 7f8957fff700 1 --2- 172.21.15.2:0/2575127859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89501017a0 0x7f8950103b90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f894c009990 tx=0x7f894c009ca0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.307+0000 7f8957fff700 1 -- 172.21.15.2:0/2575127859 shutdown_connections 2023-09-13T04:40:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.307+0000 7f8957fff700 1 --2- 172.21.15.2:0/2575127859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8950106a00 0x7f8950108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.307+0000 7f8957fff700 1 --2- 172.21.15.2:0/2575127859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89501017a0 0x7f8950103b90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.307+0000 7f8957fff700 1 --2- 172.21.15.2:0/2575127859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89501040d0 0x7f89501064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.307+0000 7f8957fff700 1 -- 172.21.15.2:0/2575127859 >> 172.21.15.2:0/2575127859 conn(0x7f89500fb1b0 msgr2=0x7f89500fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.307+0000 7f8957fff700 1 -- 172.21.15.2:0/2575127859 shutdown_connections 2023-09-13T04:40:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.307+0000 7f8957fff700 1 -- 172.21.15.2:0/2575127859 wait complete. 2023-09-13T04:40:14.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.308+0000 7f8957fff700 1 Processor -- start 2023-09-13T04:40:14.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.308+0000 7f8957fff700 1 -- start start 2023-09-13T04:40:14.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.308+0000 7f8957fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89501017a0 0x7f8950101110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:14.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.308+0000 7f8957fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89501040d0 0x7f89500ff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:14.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.308+0000 7f8957fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8950106a00 0x7f89500ffc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:14.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.308+0000 7f8957fff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8950100430 con 0x7f8950106a00 2023-09-13T04:40:14.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.308+0000 7f8957fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89501005a0 con 0x7f89501017a0 2023-09-13T04:40:14.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.308+0000 7f8957fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89501a0a10 con 0x7f89501040d0 2023-09-13T04:40:14.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.309+0000 7f8955d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89501017a0 0x7f8950101110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:14.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.309+0000 7f895659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8950106a00 0x7f89500ffc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:14.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.309+0000 7f8955d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89501017a0 0x7f8950101110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39542/0 (socket says 172.21.15.2:39542) 2023-09-13T04:40:14.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.309+0000 7f8955d9b700 1 -- 172.21.15.2:0/1225525558 learned_addr learned my addr 172.21.15.2:0/1225525558 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:14.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.309+0000 7f895559a700 1 --2- 172.21.15.2:0/1225525558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89501040d0 0x7f89500ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:14.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.309+0000 7f8955d9b700 1 -- 172.21.15.2:0/1225525558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89501040d0 msgr2=0x7f89500ff750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:14.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.309+0000 7f8955d9b700 1 --2- 172.21.15.2:0/1225525558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89501040d0 0x7f89500ff750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.309+0000 7f8955d9b700 1 -- 172.21.15.2:0/1225525558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8950106a00 msgr2=0x7f89500ffc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:14.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.309+0000 7f8955d9b700 1 --2- 172.21.15.2:0/1225525558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8950106a00 0x7f89500ffc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.309+0000 7f8955d9b700 1 -- 172.21.15.2:0/1225525558 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894c0095f0 con 0x7f89501017a0 2023-09-13T04:40:14.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.309+0000 7f8955d9b700 1 --2- 172.21.15.2:0/1225525558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89501017a0 0x7f8950101110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f894c00b680 tx=0x7f894c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:14.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.310+0000 7f8946ffd700 1 -- 172.21.15.2:0/1225525558 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c01d070 con 0x7f89501017a0 2023-09-13T04:40:14.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.310+0000 7f8957fff700 1 -- 172.21.15.2:0/1225525558 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89501a0c90 con 0x7f89501017a0 2023-09-13T04:40:14.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.310+0000 7f8946ffd700 1 -- 172.21.15.2:0/1225525558 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f894c022470 con 0x7f89501017a0 2023-09-13T04:40:14.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.310+0000 7f8946ffd700 1 -- 172.21.15.2:0/1225525558 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c017650 con 0x7f89501017a0 2023-09-13T04:40:14.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.310+0000 7f8957fff700 1 -- 172.21.15.2:0/1225525558 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89501a1110 con 0x7f89501017a0 2023-09-13T04:40:14.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.311+0000 7f8957fff700 1 -- 172.21.15.2:0/1225525558 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f895010c570 con 0x7f89501017a0 2023-09-13T04:40:14.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.313+0000 7f8946ffd700 1 -- 172.21.15.2:0/1225525558 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f894c0225e0 con 0x7f89501017a0 2023-09-13T04:40:14.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.314+0000 7f8946ffd700 1 --2- 172.21.15.2:0/1225525558 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f893c0a0690 0x7f893c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:14.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.314+0000 7f8946ffd700 1 -- 172.21.15.2:0/1225525558 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f894c0cdc10 con 0x7f89501017a0 2023-09-13T04:40:14.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.314+0000 7f895559a700 1 --2- 172.21.15.2:0/1225525558 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f893c0a0690 0x7f893c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:14.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.315+0000 7f895559a700 1 --2- 172.21.15.2:0/1225525558 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f893c0a0690 0x7f893c0a2b50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8940009c60 tx=0x7f8940009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:14.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.316+0000 7f8946ffd700 1 -- 172.21.15.2:0/1225525558 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f894c097440 con 0x7f89501017a0 2023-09-13T04:40:14.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.528+0000 7f8957fff700 1 -- 172.21.15.2:0/1225525558 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89501a12a0 con 0x7f89501017a0 2023-09-13T04:40:14.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.530+0000 7f8946ffd700 1 -- 172.21.15.2:0/1225525558 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24503 (secure 0 0 0) 0x7f894c09aa60 con 0x7f89501017a0 2023-09-13T04:40:14.531 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:40:14.532 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":92,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:39:25.580735+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T04:37:57.094802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T04:37:58.564487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-13T04:38:02.096895+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:40:14.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.532+0000 7f8957fff700 1 -- 172.21.15.2:0/1225525558 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f893c0a0690 msgr2=0x7f893c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:14.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.532+0000 7f8957fff700 1 --2- 172.21.15.2:0/1225525558 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f893c0a0690 0x7f893c0a2b50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8940009c60 tx=0x7f8940009260 comp rx=0 tx=0).stop 2023-09-13T04:40:14.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.532+0000 7f8957fff700 1 -- 172.21.15.2:0/1225525558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89501017a0 msgr2=0x7f8950101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:14.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.532+0000 7f8957fff700 1 --2- 172.21.15.2:0/1225525558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89501017a0 0x7f8950101110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f894c00b680 tx=0x7f894c005740 comp rx=0 tx=0).stop 2023-09-13T04:40:14.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.532+0000 7f8957fff700 1 -- 172.21.15.2:0/1225525558 shutdown_connections 2023-09-13T04:40:14.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.532+0000 7f8957fff700 1 --2- 172.21.15.2:0/1225525558 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f893c0a0690 0x7f893c0a2b50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.532+0000 7f8957fff700 1 --2- 172.21.15.2:0/1225525558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8950106a00 0x7f89500ffc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.532+0000 7f8957fff700 1 --2- 172.21.15.2:0/1225525558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89501017a0 0x7f8950101110 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.532+0000 7f8957fff700 1 --2- 172.21.15.2:0/1225525558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89501040d0 0x7f89500ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.532+0000 7f8957fff700 1 -- 172.21.15.2:0/1225525558 >> 172.21.15.2:0/1225525558 conn(0x7f89500fb1b0 msgr2=0x7f8950102bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:14.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.532+0000 7f8957fff700 1 -- 172.21.15.2:0/1225525558 shutdown_connections 2023-09-13T04:40:14.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.532+0000 7f8957fff700 1 -- 172.21.15.2:0/1225525558 wait complete. 2023-09-13T04:40:14.553 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T04:33:42.025714+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-13T04:37:57.094802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-13T04:37:58.564487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-13T04:38:02.096895+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-13T04:40:14.553 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-13T04:40:14.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.762+0000 7f2f3abff700 1 -- 172.21.15.2:0/1892028256 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f341069e0 msgr2=0x7f2f34108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:14.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.762+0000 7f2f3abff700 1 --2- 172.21.15.2:0/1892028256 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f341069e0 0x7f2f34108e50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00b0a0 tx=0x7f2f2c00b3b0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.762+0000 7f2f3abff700 1 -- 172.21.15.2:0/1892028256 shutdown_connections 2023-09-13T04:40:14.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.762+0000 7f2f3abff700 1 --2- 172.21.15.2:0/1892028256 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f341069e0 0x7f2f34108e50 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.762+0000 7f2f3abff700 1 --2- 172.21.15.2:0/1892028256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f34101780 0x7f2f34103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.762+0000 7f2f3abff700 1 --2- 172.21.15.2:0/1892028256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f341040b0 0x7f2f341064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.762+0000 7f2f3abff700 1 -- 172.21.15.2:0/1892028256 >> 172.21.15.2:0/1892028256 conn(0x7f2f340fb170 msgr2=0x7f2f340fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:14.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.762+0000 7f2f3abff700 1 -- 172.21.15.2:0/1892028256 shutdown_connections 2023-09-13T04:40:14.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.762+0000 7f2f3abff700 1 -- 172.21.15.2:0/1892028256 wait complete. 2023-09-13T04:40:14.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.763+0000 7f2f3abff700 1 Processor -- start 2023-09-13T04:40:14.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.763+0000 7f2f3abff700 1 -- start start 2023-09-13T04:40:14.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.764+0000 7f2f3abff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f34101780 0x7f2f34198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.764+0000 7f2f3abff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f341040b0 0x7f2f34199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.764+0000 7f2f3abff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f341069e0 0x7f2f341a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.764+0000 7f2f3abff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f341a0850 con 0x7f2f341040b0 2023-09-13T04:40:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.764+0000 7f2f3abff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f341a09c0 con 0x7f2f34101780 2023-09-13T04:40:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.764+0000 7f2f3abff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f341a0b80 con 0x7f2f341069e0 2023-09-13T04:40:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.764+0000 7f2f33fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f341040b0 0x7f2f34199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:14.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.764+0000 7f2f3899b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f34101780 0x7f2f34198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:14.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.764+0000 7f2f3899b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f34101780 0x7f2f34198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39562/0 (socket says 172.21.15.2:39562) 2023-09-13T04:40:14.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.764+0000 7f2f3899b700 1 -- 172.21.15.2:0/3528115871 learned_addr learned my addr 172.21.15.2:0/3528115871 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:14.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.764+0000 7f2f3919c700 1 --2- 172.21.15.2:0/3528115871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f341069e0 0x7f2f341a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:14.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.764+0000 7f2f3899b700 1 -- 172.21.15.2:0/3528115871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f341069e0 msgr2=0x7f2f341a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:14.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.764+0000 7f2f3899b700 1 --2- 172.21.15.2:0/3528115871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f341069e0 0x7f2f341a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.764+0000 7f2f3899b700 1 -- 172.21.15.2:0/3528115871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f341040b0 msgr2=0x7f2f34199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:14.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.764+0000 7f2f3899b700 1 --2- 172.21.15.2:0/3528115871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f341040b0 0x7f2f34199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.764+0000 7f2f3899b700 1 -- 172.21.15.2:0/3528115871 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2c009d00 con 0x7f2f34101780 2023-09-13T04:40:14.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.765+0000 7f2f3899b700 1 --2- 172.21.15.2:0/3528115871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f34101780 0x7f2f34198d10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800e9a0 tx=0x7f2f2800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.765+0000 7f2f31ffb700 1 -- 172.21.15.2:0/3528115871 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28009790 con 0x7f2f34101780 2023-09-13T04:40:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.765+0000 7f2f31ffb700 1 -- 172.21.15.2:0/3528115871 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f28010430 con 0x7f2f34101780 2023-09-13T04:40:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.765+0000 7f2f31ffb700 1 -- 172.21.15.2:0/3528115871 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28018680 con 0x7f2f34101780 2023-09-13T04:40:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.765+0000 7f2f3abff700 1 -- 172.21.15.2:0/3528115871 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f341a0e60 con 0x7f2f34101780 2023-09-13T04:40:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.765+0000 7f2f3abff700 1 -- 172.21.15.2:0/3528115871 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f341a1230 con 0x7f2f34101780 2023-09-13T04:40:14.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.766+0000 7f2f3abff700 1 -- 172.21.15.2:0/3528115871 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f34108560 con 0x7f2f34101780 2023-09-13T04:40:14.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.768+0000 7f2f31ffb700 1 -- 172.21.15.2:0/3528115871 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f2f280105a0 con 0x7f2f34101780 2023-09-13T04:40:14.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.769+0000 7f2f31ffb700 1 --2- 172.21.15.2:0/3528115871 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2f240a0640 0x7f2f240a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:14.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.770+0000 7f2f31ffb700 1 -- 172.21.15.2:0/3528115871 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2f28014070 con 0x7f2f34101780 2023-09-13T04:40:14.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.770+0000 7f2f33fff700 1 --2- 172.21.15.2:0/3528115871 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2f240a0640 0x7f2f240a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:14.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.770+0000 7f2f33fff700 1 --2- 172.21.15.2:0/3528115871 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2f240a0640 0x7f2f240a2b00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009c60 tx=0x7f2f20009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:14.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.773+0000 7f2f31ffb700 1 -- 172.21.15.2:0/3528115871 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2f28051e10 con 0x7f2f34101780 2023-09-13T04:40:14.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:14 smithi156 ceph-mon[133675]: from='client.25420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:40:14.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:14 smithi156 ceph-mon[133675]: from='client.25426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:40:14.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:14 smithi156 ceph-mon[133675]: pgmap v347: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-09-13T04:40:14.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:14 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1225525558' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:40:14.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.985+0000 7f2f3abff700 1 -- 172.21.15.2:0/3528115871 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2f3402ce20 con 0x7f2f34101780 2023-09-13T04:40:14.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.986+0000 7f2f31ffb700 1 -- 172.21.15.2:0/3528115871 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2f28095840 con 0x7f2f34101780 2023-09-13T04:40:14.987 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-09-13T04:40:14.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.988+0000 7f2f3abff700 1 -- 172.21.15.2:0/3528115871 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2f240a0640 msgr2=0x7f2f240a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:14.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.988+0000 7f2f3abff700 1 --2- 172.21.15.2:0/3528115871 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2f240a0640 0x7f2f240a2b00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009c60 tx=0x7f2f20009260 comp rx=0 tx=0).stop 2023-09-13T04:40:14.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.988+0000 7f2f3abff700 1 -- 172.21.15.2:0/3528115871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f34101780 msgr2=0x7f2f34198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:14.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.988+0000 7f2f3abff700 1 --2- 172.21.15.2:0/3528115871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f34101780 0x7f2f34198d10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800e9a0 tx=0x7f2f2800ecb0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.988+0000 7f2f3abff700 1 -- 172.21.15.2:0/3528115871 shutdown_connections 2023-09-13T04:40:14.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.988+0000 7f2f3abff700 1 --2- 172.21.15.2:0/3528115871 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f2f240a0640 0x7f2f240a2b00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.988+0000 7f2f3abff700 1 --2- 172.21.15.2:0/3528115871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f341069e0 0x7f2f341a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.988+0000 7f2f3abff700 1 --2- 172.21.15.2:0/3528115871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f34101780 0x7f2f34198d10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.988+0000 7f2f3abff700 1 --2- 172.21.15.2:0/3528115871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f341040b0 0x7f2f34199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:14.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.988+0000 7f2f3abff700 1 -- 172.21.15.2:0/3528115871 >> 172.21.15.2:0/3528115871 conn(0x7f2f340fb170 msgr2=0x7f2f34102410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:14.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.988+0000 7f2f3abff700 1 -- 172.21.15.2:0/3528115871 shutdown_connections 2023-09-13T04:40:14.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:14.988+0000 7f2f3abff700 1 -- 172.21.15.2:0/3528115871 wait complete. 2023-09-13T04:40:15.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:14 smithi002 ceph-mon[97651]: from='client.25420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:40:15.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:14 smithi002 ceph-mon[97651]: from='client.25426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:40:15.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:14 smithi002 ceph-mon[97651]: pgmap v347: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-09-13T04:40:15.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:14 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1225525558' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:40:15.008 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-13T04:40:15.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:14 smithi149 ceph-mon[133315]: from='client.25420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:40:15.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:14 smithi149 ceph-mon[133315]: from='client.25426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:40:15.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:14 smithi149 ceph-mon[133315]: pgmap v347: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-09-13T04:40:15.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:14 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1225525558' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:40:15.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.216+0000 7efe379ff700 1 -- 172.21.15.2:0/59529311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe30101f30 msgr2=0x7efe30102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:15.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.216+0000 7efe379ff700 1 --2- 172.21.15.2:0/59529311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe30101f30 0x7efe30102390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efe20009990 tx=0x7efe20009ca0 comp rx=0 tx=0).stop 2023-09-13T04:40:15.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.216+0000 7efe379ff700 1 -- 172.21.15.2:0/59529311 shutdown_connections 2023-09-13T04:40:15.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.216+0000 7efe379ff700 1 --2- 172.21.15.2:0/59529311 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe301028d0 0x7efe30109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:15.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.216+0000 7efe379ff700 1 --2- 172.21.15.2:0/59529311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe30100d40 0x7efe30101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:15.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.216+0000 7efe379ff700 1 --2- 172.21.15.2:0/59529311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe30101f30 0x7efe30102390 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:15.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.216+0000 7efe379ff700 1 -- 172.21.15.2:0/59529311 >> 172.21.15.2:0/59529311 conn(0x7efe300fc2d0 msgr2=0x7efe300fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:15.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.216+0000 7efe379ff700 1 -- 172.21.15.2:0/59529311 shutdown_connections 2023-09-13T04:40:15.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.216+0000 7efe379ff700 1 -- 172.21.15.2:0/59529311 wait complete. 2023-09-13T04:40:15.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.218+0000 7efe379ff700 1 Processor -- start 2023-09-13T04:40:15.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.218+0000 7efe379ff700 1 -- start start 2023-09-13T04:40:15.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.218+0000 7efe379ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe30100d40 0x7efe30196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:15.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.218+0000 7efe379ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe30101f30 0x7efe30196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:15.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.218+0000 7efe379ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe301028d0 0x7efe3019dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:15.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.218+0000 7efe379ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3019e5a0 con 0x7efe30101f30 2023-09-13T04:40:15.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.218+0000 7efe379ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3019e6e0 con 0x7efe30100d40 2023-09-13T04:40:15.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.218+0000 7efe379ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3019e870 con 0x7efe301028d0 2023-09-13T04:40:15.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.219+0000 7efe34f9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe30101f30 0x7efe30196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:15.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.219+0000 7efe3579b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe30100d40 0x7efe30196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:15.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.219+0000 7efe35f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe301028d0 0x7efe3019dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:15.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.219+0000 7efe34f9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe30101f30 0x7efe30196f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47538/0 (socket says 172.21.15.2:47538) 2023-09-13T04:40:15.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.219+0000 7efe34f9a700 1 -- 172.21.15.2:0/231452647 learned_addr learned my addr 172.21.15.2:0/231452647 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:15.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.219+0000 7efe3579b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe30100d40 0x7efe30196a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39576/0 (socket says 172.21.15.2:39576) 2023-09-13T04:40:15.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.219+0000 7efe35f9c700 1 -- 172.21.15.2:0/231452647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe30100d40 msgr2=0x7efe30196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:15.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.219+0000 7efe35f9c700 1 --2- 172.21.15.2:0/231452647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe30100d40 0x7efe30196a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:15.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.219+0000 7efe35f9c700 1 -- 172.21.15.2:0/231452647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe30101f30 msgr2=0x7efe30196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:15.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.219+0000 7efe35f9c700 1 --2- 172.21.15.2:0/231452647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe30101f30 0x7efe30196f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:15.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.219+0000 7efe35f9c700 1 -- 172.21.15.2:0/231452647 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe200095f0 con 0x7efe301028d0 2023-09-13T04:40:15.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.219+0000 7efe34f9a700 1 --2- 172.21.15.2:0/231452647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe30101f30 0x7efe30196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:15.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.220+0000 7efe35f9c700 1 --2- 172.21.15.2:0/231452647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe301028d0 0x7efe3019dfd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efe2800b330 tx=0x7efe2800b640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:15.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.220+0000 7efe267fc700 1 -- 172.21.15.2:0/231452647 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe28015070 con 0x7efe301028d0 2023-09-13T04:40:15.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.220+0000 7efe267fc700 1 -- 172.21.15.2:0/231452647 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe2800bdd0 con 0x7efe301028d0 2023-09-13T04:40:15.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.220+0000 7efe379ff700 1 -- 172.21.15.2:0/231452647 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe30072ee0 con 0x7efe301028d0 2023-09-13T04:40:15.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.220+0000 7efe267fc700 1 -- 172.21.15.2:0/231452647 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe28007c20 con 0x7efe301028d0 2023-09-13T04:40:15.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.220+0000 7efe379ff700 1 -- 172.21.15.2:0/231452647 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe300733a0 con 0x7efe301028d0 2023-09-13T04:40:15.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.221+0000 7efe379ff700 1 -- 172.21.15.2:0/231452647 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe3010c530 con 0x7efe301028d0 2023-09-13T04:40:15.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.223+0000 7efe267fc700 1 -- 172.21.15.2:0/231452647 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7efe28007500 con 0x7efe301028d0 2023-09-13T04:40:15.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.224+0000 7efe267fc700 1 --2- 172.21.15.2:0/231452647 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7efe1c0a0640 0x7efe1c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:15.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.225+0000 7efe267fc700 1 -- 172.21.15.2:0/231452647 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7efe280cfe80 con 0x7efe301028d0 2023-09-13T04:40:15.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.225+0000 7efe3579b700 1 --2- 172.21.15.2:0/231452647 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7efe1c0a0640 0x7efe1c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:15.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.225+0000 7efe3579b700 1 --2- 172.21.15.2:0/231452647 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7efe1c0a0640 0x7efe1c0a2b00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efe30101d90 tx=0x7efe2c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:15.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.226+0000 7efe267fc700 1 -- 172.21.15.2:0/231452647 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7efe28099620 con 0x7efe301028d0 2023-09-13T04:40:15.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.439+0000 7efe379ff700 1 -- 172.21.15.2:0/231452647 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efe30073870 con 0x7efe301028d0 2023-09-13T04:40:15.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.440+0000 7efe267fc700 1 -- 172.21.15.2:0/231452647 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7efe2809cc40 con 0x7efe301028d0 2023-09-13T04:40:15.442 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-09-13T04:40:15.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.442+0000 7efe379ff700 1 -- 172.21.15.2:0/231452647 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7efe1c0a0640 msgr2=0x7efe1c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:15.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.442+0000 7efe379ff700 1 --2- 172.21.15.2:0/231452647 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7efe1c0a0640 0x7efe1c0a2b00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efe30101d90 tx=0x7efe2c009260 comp rx=0 tx=0).stop 2023-09-13T04:40:15.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.442+0000 7efe379ff700 1 -- 172.21.15.2:0/231452647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe301028d0 msgr2=0x7efe3019dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:15.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.442+0000 7efe379ff700 1 --2- 172.21.15.2:0/231452647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe301028d0 0x7efe3019dfd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efe2800b330 tx=0x7efe2800b640 comp rx=0 tx=0).stop 2023-09-13T04:40:15.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.443+0000 7efe379ff700 1 -- 172.21.15.2:0/231452647 shutdown_connections 2023-09-13T04:40:15.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.443+0000 7efe379ff700 1 --2- 172.21.15.2:0/231452647 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7efe1c0a0640 0x7efe1c0a2b00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:15.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.443+0000 7efe379ff700 1 --2- 172.21.15.2:0/231452647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe301028d0 0x7efe3019dfd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:15.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.443+0000 7efe379ff700 1 --2- 172.21.15.2:0/231452647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe30100d40 0x7efe30196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:15.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.443+0000 7efe379ff700 1 --2- 172.21.15.2:0/231452647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe30101f30 0x7efe30196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:15.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.443+0000 7efe379ff700 1 -- 172.21.15.2:0/231452647 >> 172.21.15.2:0/231452647 conn(0x7efe300fc2d0 msgr2=0x7efe300fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:15.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.443+0000 7efe379ff700 1 -- 172.21.15.2:0/231452647 shutdown_connections 2023-09-13T04:40:15.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.443+0000 7efe379ff700 1 -- 172.21.15.2:0/231452647 wait complete. 2023-09-13T04:40:15.463 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-13T04:40:15.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.671+0000 7f1f31fff700 1 -- 172.21.15.2:0/1114071682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f2c100cf0 msgr2=0x7f1f2c101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:15.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.671+0000 7f1f31fff700 1 --2- 172.21.15.2:0/1114071682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f2c100cf0 0x7f1f2c101110 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0099c0 tx=0x7f1f1c009cd0 comp rx=0 tx=0).stop 2023-09-13T04:40:15.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.671+0000 7f1f31fff700 1 -- 172.21.15.2:0/1114071682 shutdown_connections 2023-09-13T04:40:15.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.671+0000 7f1f31fff700 1 --2- 172.21.15.2:0/1114071682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f2c102880 0x7f1f2c109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:15.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.671+0000 7f1f31fff700 1 --2- 172.21.15.2:0/1114071682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f2c100cf0 0x7f1f2c101110 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:15.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.671+0000 7f1f31fff700 1 --2- 172.21.15.2:0/1114071682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f2c101ee0 0x7f1f2c102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:15.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.671+0000 7f1f31fff700 1 -- 172.21.15.2:0/1114071682 >> 172.21.15.2:0/1114071682 conn(0x7f1f2c0fc260 msgr2=0x7f1f2c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:15.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.671+0000 7f1f31fff700 1 -- 172.21.15.2:0/1114071682 shutdown_connections 2023-09-13T04:40:15.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.671+0000 7f1f31fff700 1 -- 172.21.15.2:0/1114071682 wait complete. 2023-09-13T04:40:15.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.672+0000 7f1f31fff700 1 Processor -- start 2023-09-13T04:40:15.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.672+0000 7f1f31fff700 1 -- start start 2023-09-13T04:40:15.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.672+0000 7f1f31fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f2c100cf0 0x7f1f2c196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:15.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.672+0000 7f1f31fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f2c101ee0 0x7f1f2c196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:15.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.672+0000 7f1f31fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f2c102880 0x7f1f2c19e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:15.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.672+0000 7f1f31fff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c19e600 con 0x7f1f2c100cf0 2023-09-13T04:40:15.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.672+0000 7f1f31fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c19e740 con 0x7f1f2c102880 2023-09-13T04:40:15.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.672+0000 7f1f31fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c19e8d0 con 0x7f1f2c101ee0 2023-09-13T04:40:15.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.673+0000 7f1f2b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f2c100cf0 0x7f1f2c196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:15.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.673+0000 7f1f2b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f2c100cf0 0x7f1f2c196a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47562/0 (socket says 172.21.15.2:47562) 2023-09-13T04:40:15.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.673+0000 7f1f2b7fe700 1 -- 172.21.15.2:0/4159107487 learned_addr learned my addr 172.21.15.2:0/4159107487 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:15.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.673+0000 7f1f2affd700 1 --2- 172.21.15.2:0/4159107487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f2c101ee0 0x7f1f2c196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:15.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.673+0000 7f1f2bfff700 1 --2- 172.21.15.2:0/4159107487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f2c102880 0x7f1f2c19e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:15.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.673+0000 7f1f2b7fe700 1 -- 172.21.15.2:0/4159107487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f2c101ee0 msgr2=0x7f1f2c196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:15.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.673+0000 7f1f2b7fe700 1 --2- 172.21.15.2:0/4159107487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f2c101ee0 0x7f1f2c196fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:15.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.673+0000 7f1f2b7fe700 1 -- 172.21.15.2:0/4159107487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f2c102880 msgr2=0x7f1f2c19e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:15.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.673+0000 7f1f2b7fe700 1 --2- 172.21.15.2:0/4159107487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f2c102880 0x7f1f2c19e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:15.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.673+0000 7f1f2b7fe700 1 -- 172.21.15.2:0/4159107487 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f20009520 con 0x7f1f2c100cf0 2023-09-13T04:40:15.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.673+0000 7f1f2affd700 1 --2- 172.21.15.2:0/4159107487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f2c101ee0 0x7f1f2c196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:15.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.673+0000 7f1f2b7fe700 1 --2- 172.21.15.2:0/4159107487 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f2c100cf0 0x7f1f2c196a80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c009990 tx=0x7f1f1c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:15.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.674+0000 7f1f28ff9700 1 -- 172.21.15.2:0/4159107487 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c01d070 con 0x7f1f2c100cf0 2023-09-13T04:40:15.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.674+0000 7f1f28ff9700 1 -- 172.21.15.2:0/4159107487 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f1c005030 con 0x7f1f2c100cf0 2023-09-13T04:40:15.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.674+0000 7f1f31fff700 1 -- 172.21.15.2:0/4159107487 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f1c0095f0 con 0x7f1f2c100cf0 2023-09-13T04:40:15.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.674+0000 7f1f28ff9700 1 -- 172.21.15.2:0/4159107487 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c004a70 con 0x7f1f2c100cf0 2023-09-13T04:40:15.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.674+0000 7f1f31fff700 1 -- 172.21.15.2:0/4159107487 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2c19ede0 con 0x7f1f2c100cf0 2023-09-13T04:40:15.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.675+0000 7f1f31fff700 1 -- 172.21.15.2:0/4159107487 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f2c108330 con 0x7f1f2c100cf0 2023-09-13T04:40:15.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.677+0000 7f1f28ff9700 1 -- 172.21.15.2:0/4159107487 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f1f1c005410 con 0x7f1f2c100cf0 2023-09-13T04:40:15.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.679+0000 7f1f28ff9700 1 --2- 172.21.15.2:0/4159107487 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1f140a0640 0x7f1f140a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:15.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.679+0000 7f1f28ff9700 1 -- 172.21.15.2:0/4159107487 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1f1c0cccb0 con 0x7f1f2c100cf0 2023-09-13T04:40:15.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.679+0000 7f1f2affd700 1 --2- 172.21.15.2:0/4159107487 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1f140a0640 0x7f1f140a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:15.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.680+0000 7f1f2affd700 1 --2- 172.21.15.2:0/4159107487 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1f140a0640 0x7f1f140a2b00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009fd0 tx=0x7f1f20009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:15.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.683+0000 7f1f28ff9700 1 -- 172.21.15.2:0/4159107487 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1f1c0964e0 con 0x7f1f2c100cf0 2023-09-13T04:40:15.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:15 smithi002 ceph-mon[97651]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:40:15.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:15 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3528115871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:40:15.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:15 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/231452647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:40:15.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.895+0000 7f1f31fff700 1 -- 172.21.15.2:0/4159107487 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1f2c02d070 con 0x7f1f2c100cf0 2023-09-13T04:40:15.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.895+0000 7f1f28ff9700 1 -- 172.21.15.2:0/4159107487 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1f1c099b00 con 0x7f1f2c100cf0 2023-09-13T04:40:15.897 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-09-13T04:40:15.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.897+0000 7f1f31fff700 1 -- 172.21.15.2:0/4159107487 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1f140a0640 msgr2=0x7f1f140a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:15.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.897+0000 7f1f31fff700 1 --2- 172.21.15.2:0/4159107487 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1f140a0640 0x7f1f140a2b00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009fd0 tx=0x7f1f20009260 comp rx=0 tx=0).stop 2023-09-13T04:40:15.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.897+0000 7f1f31fff700 1 -- 172.21.15.2:0/4159107487 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f2c100cf0 msgr2=0x7f1f2c196a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:15.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.897+0000 7f1f31fff700 1 --2- 172.21.15.2:0/4159107487 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f2c100cf0 0x7f1f2c196a80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c009990 tx=0x7f1f1c00f690 comp rx=0 tx=0).stop 2023-09-13T04:40:15.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.898+0000 7f1f31fff700 1 -- 172.21.15.2:0/4159107487 shutdown_connections 2023-09-13T04:40:15.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.898+0000 7f1f31fff700 1 --2- 172.21.15.2:0/4159107487 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f1f140a0640 0x7f1f140a2b00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:15.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.898+0000 7f1f31fff700 1 --2- 172.21.15.2:0/4159107487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f2c102880 0x7f1f2c19e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:15.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.898+0000 7f1f31fff700 1 --2- 172.21.15.2:0/4159107487 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f2c100cf0 0x7f1f2c196a80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:15.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.898+0000 7f1f31fff700 1 --2- 172.21.15.2:0/4159107487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f2c101ee0 0x7f1f2c196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:15.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.898+0000 7f1f31fff700 1 -- 172.21.15.2:0/4159107487 >> 172.21.15.2:0/4159107487 conn(0x7f1f2c0fc260 msgr2=0x7f1f2c105470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:15.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.898+0000 7f1f31fff700 1 -- 172.21.15.2:0/4159107487 shutdown_connections 2023-09-13T04:40:15.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:15.898+0000 7f1f31fff700 1 -- 172.21.15.2:0/4159107487 wait complete. 2023-09-13T04:40:15.921 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-13T04:40:15.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:15 smithi156 ceph-mon[133675]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:40:15.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:15 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3528115871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:40:15.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:15 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/231452647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:40:16.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.127+0000 7efd763ff700 1 -- 172.21.15.2:0/3198292923 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd70101ed0 msgr2=0x7efd70102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:16.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.127+0000 7efd763ff700 1 --2- 172.21.15.2:0/3198292923 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd70101ed0 0x7efd70102330 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7efd640099c0 tx=0x7efd64009cd0 comp rx=0 tx=0).stop 2023-09-13T04:40:16.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.127+0000 7efd763ff700 1 -- 172.21.15.2:0/3198292923 shutdown_connections 2023-09-13T04:40:16.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.127+0000 7efd763ff700 1 --2- 172.21.15.2:0/3198292923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd70102870 0x7efd70109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:16.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.127+0000 7efd763ff700 1 --2- 172.21.15.2:0/3198292923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd70100ce0 0x7efd70101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:16.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.127+0000 7efd763ff700 1 --2- 172.21.15.2:0/3198292923 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd70101ed0 0x7efd70102330 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:16.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.127+0000 7efd763ff700 1 -- 172.21.15.2:0/3198292923 >> 172.21.15.2:0/3198292923 conn(0x7efd700fc230 msgr2=0x7efd700fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:16.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.127+0000 7efd763ff700 1 -- 172.21.15.2:0/3198292923 shutdown_connections 2023-09-13T04:40:16.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.127+0000 7efd763ff700 1 -- 172.21.15.2:0/3198292923 wait complete. 2023-09-13T04:40:16.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.128+0000 7efd763ff700 1 Processor -- start 2023-09-13T04:40:16.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.128+0000 7efd763ff700 1 -- start start 2023-09-13T04:40:16.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.128+0000 7efd763ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd70100ce0 0x7efd70074a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:16.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.128+0000 7efd763ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd70101ed0 0x7efd700730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:16.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.128+0000 7efd763ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd70102870 0x7efd700735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:16.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.128+0000 7efd763ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd70073d80 con 0x7efd70100ce0 2023-09-13T04:40:16.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.128+0000 7efd763ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd70073ef0 con 0x7efd70101ed0 2023-09-13T04:40:16.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.128+0000 7efd763ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd701a0a60 con 0x7efd70102870 2023-09-13T04:40:16.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.129+0000 7efd6ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd70100ce0 0x7efd70074a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:16.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.129+0000 7efd6ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd70100ce0 0x7efd70074a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47572/0 (socket says 172.21.15.2:47572) 2023-09-13T04:40:16.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.129+0000 7efd6ffff700 1 -- 172.21.15.2:0/2675444764 learned_addr learned my addr 172.21.15.2:0/2675444764 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:16.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.129+0000 7efd6f7fe700 1 --2- 172.21.15.2:0/2675444764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd70101ed0 0x7efd700730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:16.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.129+0000 7efd7499c700 1 --2- 172.21.15.2:0/2675444764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd70102870 0x7efd700735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:16.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.129+0000 7efd6ffff700 1 -- 172.21.15.2:0/2675444764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd70102870 msgr2=0x7efd700735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:16.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.129+0000 7efd6ffff700 1 --2- 172.21.15.2:0/2675444764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd70102870 0x7efd700735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:16.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.129+0000 7efd6ffff700 1 -- 172.21.15.2:0/2675444764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd70101ed0 msgr2=0x7efd700730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:16.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.129+0000 7efd6ffff700 1 --2- 172.21.15.2:0/2675444764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd70101ed0 0x7efd700730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:16.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.129+0000 7efd6ffff700 1 -- 172.21.15.2:0/2675444764 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd640095f0 con 0x7efd70100ce0 2023-09-13T04:40:16.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.129+0000 7efd6ffff700 1 --2- 172.21.15.2:0/2675444764 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd70100ce0 0x7efd70074a60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7efd6000e970 tx=0x7efd6000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:16.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.130+0000 7efd6d7fa700 1 -- 172.21.15.2:0/2675444764 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd60009790 con 0x7efd70100ce0 2023-09-13T04:40:16.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.130+0000 7efd6d7fa700 1 -- 172.21.15.2:0/2675444764 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd6000c7e0 con 0x7efd70100ce0 2023-09-13T04:40:16.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.130+0000 7efd763ff700 1 -- 172.21.15.2:0/2675444764 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd701a0d40 con 0x7efd70100ce0 2023-09-13T04:40:16.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.130+0000 7efd6d7fa700 1 -- 172.21.15.2:0/2675444764 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd60003c70 con 0x7efd70100ce0 2023-09-13T04:40:16.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.130+0000 7efd763ff700 1 -- 172.21.15.2:0/2675444764 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd701a11a0 con 0x7efd70100ce0 2023-09-13T04:40:16.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.131+0000 7efd763ff700 1 -- 172.21.15.2:0/2675444764 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd7010c540 con 0x7efd70100ce0 2023-09-13T04:40:16.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:15 smithi149 ceph-mon[133315]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T04:40:16.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:15 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3528115871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:40:16.139 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:15 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/231452647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:40:16.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.133+0000 7efd6d7fa700 1 -- 172.21.15.2:0/2675444764 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7efd6000cc50 con 0x7efd70100ce0 2023-09-13T04:40:16.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.134+0000 7efd6d7fa700 1 --2- 172.21.15.2:0/2675444764 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7efd580a0640 0x7efd580a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:16.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.134+0000 7efd6d7fa700 1 -- 172.21.15.2:0/2675444764 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7efd60014070 con 0x7efd70100ce0 2023-09-13T04:40:16.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.135+0000 7efd6f7fe700 1 --2- 172.21.15.2:0/2675444764 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7efd580a0640 0x7efd580a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:16.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.135+0000 7efd6f7fe700 1 --2- 172.21.15.2:0/2675444764 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7efd580a0640 0x7efd580a2b00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efd6400b580 tx=0x7efd64005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:16.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.138+0000 7efd6d7fa700 1 -- 172.21.15.2:0/2675444764 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7efd60095890 con 0x7efd70100ce0 2023-09-13T04:40:16.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.353+0000 7efd763ff700 1 -- 172.21.15.2:0/2675444764 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7efd70107c30 con 0x7efd70100ce0 2023-09-13T04:40:16.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.354+0000 7efd6d7fa700 1 -- 172.21.15.2:0/2675444764 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7efd60003dd0 con 0x7efd70100ce0 2023-09-13T04:40:16.355 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-09-13T04:40:16.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.356+0000 7efd763ff700 1 -- 172.21.15.2:0/2675444764 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7efd580a0640 msgr2=0x7efd580a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:16.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.356+0000 7efd763ff700 1 --2- 172.21.15.2:0/2675444764 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7efd580a0640 0x7efd580a2b00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efd6400b580 tx=0x7efd64005fb0 comp rx=0 tx=0).stop 2023-09-13T04:40:16.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.356+0000 7efd763ff700 1 -- 172.21.15.2:0/2675444764 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd70100ce0 msgr2=0x7efd70074a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:16.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.356+0000 7efd763ff700 1 --2- 172.21.15.2:0/2675444764 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd70100ce0 0x7efd70074a60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7efd6000e970 tx=0x7efd6000ec80 comp rx=0 tx=0).stop 2023-09-13T04:40:16.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.356+0000 7efd763ff700 1 -- 172.21.15.2:0/2675444764 shutdown_connections 2023-09-13T04:40:16.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.356+0000 7efd763ff700 1 --2- 172.21.15.2:0/2675444764 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7efd580a0640 0x7efd580a2b00 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:16.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.356+0000 7efd763ff700 1 --2- 172.21.15.2:0/2675444764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd70102870 0x7efd700735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:16.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.356+0000 7efd763ff700 1 --2- 172.21.15.2:0/2675444764 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd70100ce0 0x7efd70074a60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:16.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.356+0000 7efd763ff700 1 --2- 172.21.15.2:0/2675444764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd70101ed0 0x7efd700730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:16.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.356+0000 7efd763ff700 1 -- 172.21.15.2:0/2675444764 >> 172.21.15.2:0/2675444764 conn(0x7efd700fc230 msgr2=0x7efd700fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:16.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.356+0000 7efd763ff700 1 -- 172.21.15.2:0/2675444764 shutdown_connections 2023-09-13T04:40:16.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.357+0000 7efd763ff700 1 -- 172.21.15.2:0/2675444764 wait complete. 2023-09-13T04:40:16.376 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:40:16.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.580+0000 7f43209ff700 1 -- 172.21.15.2:0/2605144032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f431c1017a0 msgr2=0x7f431c103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:16.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.580+0000 7f43209ff700 1 --2- 172.21.15.2:0/2605144032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f431c1017a0 0x7f431c103b90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f430c009990 tx=0x7f430c009ca0 comp rx=0 tx=0).stop 2023-09-13T04:40:16.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.580+0000 7f43209ff700 1 -- 172.21.15.2:0/2605144032 shutdown_connections 2023-09-13T04:40:16.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.580+0000 7f43209ff700 1 --2- 172.21.15.2:0/2605144032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c106a00 0x7f431c108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:16.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.580+0000 7f43209ff700 1 --2- 172.21.15.2:0/2605144032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f431c1017a0 0x7f431c103b90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:16.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.580+0000 7f43209ff700 1 --2- 172.21.15.2:0/2605144032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f431c1040d0 0x7f431c1064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:16.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.580+0000 7f43209ff700 1 -- 172.21.15.2:0/2605144032 >> 172.21.15.2:0/2605144032 conn(0x7f431c0fb1b0 msgr2=0x7f431c0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:16.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.580+0000 7f43209ff700 1 -- 172.21.15.2:0/2605144032 shutdown_connections 2023-09-13T04:40:16.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.580+0000 7f43209ff700 1 -- 172.21.15.2:0/2605144032 wait complete. 2023-09-13T04:40:16.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.581+0000 7f43209ff700 1 Processor -- start 2023-09-13T04:40:16.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.581+0000 7f43209ff700 1 -- start start 2023-09-13T04:40:16.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.581+0000 7f43209ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c1017a0 0x7f431c101110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:16.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.582+0000 7f43209ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f431c1040d0 0x7f431c0ff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:16.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.582+0000 7f43209ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f431c106a00 0x7f431c0ffc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:16.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.582+0000 7f43209ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c100430 con 0x7f431c1040d0 2023-09-13T04:40:16.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.582+0000 7f43209ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c1005a0 con 0x7f431c1017a0 2023-09-13T04:40:16.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.582+0000 7f43209ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c1a0a10 con 0x7f431c106a00 2023-09-13T04:40:16.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.582+0000 7f4313fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f431c1040d0 0x7f431c0ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:16.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.582+0000 7f4313fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f431c1040d0 0x7f431c0ff750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47596/0 (socket says 172.21.15.2:47596) 2023-09-13T04:40:16.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.582+0000 7f4313fff700 1 -- 172.21.15.2:0/2903761698 learned_addr learned my addr 172.21.15.2:0/2903761698 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:16.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.582+0000 7f431ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f431c106a00 0x7f431c0ffc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:16.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.582+0000 7f431a59c700 1 --2- 172.21.15.2:0/2903761698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c1017a0 0x7f431c101110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:16.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.582+0000 7f4313fff700 1 -- 172.21.15.2:0/2903761698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f431c106a00 msgr2=0x7f431c0ffc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:16.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.582+0000 7f4313fff700 1 --2- 172.21.15.2:0/2903761698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f431c106a00 0x7f431c0ffc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:16.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.582+0000 7f4313fff700 1 -- 172.21.15.2:0/2903761698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c1017a0 msgr2=0x7f431c101110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:16.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.582+0000 7f4313fff700 1 --2- 172.21.15.2:0/2903761698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c1017a0 0x7f431c101110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:16.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.582+0000 7f4313fff700 1 -- 172.21.15.2:0/2903761698 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4314009c40 con 0x7f431c1040d0 2023-09-13T04:40:16.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.583+0000 7f4313fff700 1 --2- 172.21.15.2:0/2903761698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f431c1040d0 0x7f431c0ff750 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f430400e9a0 tx=0x7f430400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:16.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.583+0000 7f43137fe700 1 -- 172.21.15.2:0/2903761698 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4304009790 con 0x7f431c1040d0 2023-09-13T04:40:16.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.583+0000 7f43137fe700 1 -- 172.21.15.2:0/2903761698 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f430400ce80 con 0x7f431c1040d0 2023-09-13T04:40:16.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.583+0000 7f43209ff700 1 -- 172.21.15.2:0/2903761698 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430c0095f0 con 0x7f431c1040d0 2023-09-13T04:40:16.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.583+0000 7f43137fe700 1 -- 172.21.15.2:0/2903761698 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4304018680 con 0x7f431c1040d0 2023-09-13T04:40:16.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.583+0000 7f43209ff700 1 -- 172.21.15.2:0/2903761698 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431c1a1070 con 0x7f431c1040d0 2023-09-13T04:40:16.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.584+0000 7f43209ff700 1 -- 172.21.15.2:0/2903761698 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431c0fcd30 con 0x7f431c1040d0 2023-09-13T04:40:16.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.586+0000 7f43137fe700 1 -- 172.21.15.2:0/2903761698 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f4304010430 con 0x7f431c1040d0 2023-09-13T04:40:16.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.587+0000 7f43137fe700 1 --2- 172.21.15.2:0/2903761698 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f43080a0690 0x7f43080a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:16.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.588+0000 7f43137fe700 1 -- 172.21.15.2:0/2903761698 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4304014070 con 0x7f431c1040d0 2023-09-13T04:40:16.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.588+0000 7f431a59c700 1 --2- 172.21.15.2:0/2903761698 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f43080a0690 0x7f43080a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:16.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.588+0000 7f431a59c700 1 --2- 172.21.15.2:0/2903761698 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f43080a0690 0x7f43080a2b50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f430c00b2e0 tx=0x7f430c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:16.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.589+0000 7f43137fe700 1 -- 172.21.15.2:0/2903761698 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4304095c70 con 0x7f431c1040d0 2023-09-13T04:40:16.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.851+0000 7f43209ff700 1 -- 172.21.15.2:0/2903761698 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f431c10c550 con 0x7f431c1040d0 2023-09-13T04:40:16.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.852+0000 7f43137fe700 1 -- 172.21.15.2:0/2903761698 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11303 (secure 0 0 0) 0x7f4304099290 con 0x7f431c1040d0 2023-09-13T04:40:16.854 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:40:16.854 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2469307307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2469307307},{"type":"v1","addr":"172.21.15.2:6839","nonce":2469307307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T04:37:59.110701+0000","modified":"2023-09-13T04:38:26.610260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24554,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/846579077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":846579077},{"type":"v1","addr":"172.21.15.156:6835","nonce":846579077}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1896833690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1896833690},{"type":"v1","addr":"172.21.15.149:6837","nonce":1896833690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/524563090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":524563090},{"type":"v1","addr":"172.21.15.156:6837","nonce":524563090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6839/1785833227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1785833227},{"type":"v1","addr":"172.21.15.156:6839","nonce":1785833227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.2:6841/482545391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":482545391},{"type":"v1","addr":"172.21.15.2:6841","nonce":482545391}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/389167841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":389167841},{"type":"v1","addr":"172.21.15.149:6839","nonce":389167841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T04:40:16.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.854+0000 7f43209ff700 1 -- 172.21.15.2:0/2903761698 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f43080a0690 msgr2=0x7f43080a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:16.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.854+0000 7f43209ff700 1 --2- 172.21.15.2:0/2903761698 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f43080a0690 0x7f43080a2b50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f430c00b2e0 tx=0x7f430c005fb0 comp rx=0 tx=0).stop 2023-09-13T04:40:16.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.855+0000 7f43209ff700 1 -- 172.21.15.2:0/2903761698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f431c1040d0 msgr2=0x7f431c0ff750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:16.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.855+0000 7f43209ff700 1 --2- 172.21.15.2:0/2903761698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f431c1040d0 0x7f431c0ff750 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f430400e9a0 tx=0x7f430400ecb0 comp rx=0 tx=0).stop 2023-09-13T04:40:16.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.855+0000 7f43209ff700 1 -- 172.21.15.2:0/2903761698 shutdown_connections 2023-09-13T04:40:16.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.855+0000 7f43209ff700 1 --2- 172.21.15.2:0/2903761698 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f43080a0690 0x7f43080a2b50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:16.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.855+0000 7f43209ff700 1 --2- 172.21.15.2:0/2903761698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f431c106a00 0x7f431c0ffc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:16.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.855+0000 7f43209ff700 1 --2- 172.21.15.2:0/2903761698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c1017a0 0x7f431c101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:16.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.855+0000 7f43209ff700 1 --2- 172.21.15.2:0/2903761698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f431c1040d0 0x7f431c0ff750 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:16.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.855+0000 7f43209ff700 1 -- 172.21.15.2:0/2903761698 >> 172.21.15.2:0/2903761698 conn(0x7f431c0fb1b0 msgr2=0x7f431c102410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:16.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.855+0000 7f43209ff700 1 -- 172.21.15.2:0/2903761698 shutdown_connections 2023-09-13T04:40:16.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:16.855+0000 7f43209ff700 1 -- 172.21.15.2:0/2903761698 wait complete. 2023-09-13T04:40:16.859 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 23 2023-09-13T04:40:16.875 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T04:40:16.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:16 smithi156 ceph-mon[133675]: pgmap v348: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2023-09-13T04:40:16.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:16 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/4159107487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:40:16.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:16 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2675444764' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:40:16.937 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:16 smithi002 ceph-mon[97651]: pgmap v348: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2023-09-13T04:40:16.938 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:16 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/4159107487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:40:16.938 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:16 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2675444764' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:40:17.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.093+0000 7f4a7c9ff700 1 -- 172.21.15.2:0/2242181574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a781017c0 msgr2=0x7f4a78103bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.093+0000 7f4a7c9ff700 1 --2- 172.21.15.2:0/2242181574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a781017c0 0x7f4a78103bb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4a680098f0 tx=0x7f4a68009c00 comp rx=0 tx=0).stop 2023-09-13T04:40:17.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.093+0000 7f4a7c9ff700 1 -- 172.21.15.2:0/2242181574 shutdown_connections 2023-09-13T04:40:17.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.093+0000 7f4a7c9ff700 1 --2- 172.21.15.2:0/2242181574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a78106a20 0x7f4a78108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.093+0000 7f4a7c9ff700 1 --2- 172.21.15.2:0/2242181574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a781017c0 0x7f4a78103bb0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.093+0000 7f4a7c9ff700 1 --2- 172.21.15.2:0/2242181574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a781040f0 0x7f4a781064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.093+0000 7f4a7c9ff700 1 -- 172.21.15.2:0/2242181574 >> 172.21.15.2:0/2242181574 conn(0x7f4a780fb110 msgr2=0x7f4a780fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:17.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.093+0000 7f4a7c9ff700 1 -- 172.21.15.2:0/2242181574 shutdown_connections 2023-09-13T04:40:17.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.093+0000 7f4a7c9ff700 1 -- 172.21.15.2:0/2242181574 wait complete. 2023-09-13T04:40:17.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.095+0000 7f4a7c9ff700 1 Processor -- start 2023-09-13T04:40:17.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.095+0000 7f4a7c9ff700 1 -- start start 2023-09-13T04:40:17.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.096+0000 7f4a7c9ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a781017c0 0x7f4a78101130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.096+0000 7f4a7c9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a781040f0 0x7f4a780ff770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.096+0000 7f4a7c9ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a78106a20 0x7f4a780ffcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.096+0000 7f4a7c9ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a78100450 con 0x7f4a781017c0 2023-09-13T04:40:17.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.096+0000 7f4a7c9ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a781005c0 con 0x7f4a78106a20 2023-09-13T04:40:17.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.096+0000 7f4a7c9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a781a09d0 con 0x7f4a781040f0 2023-09-13T04:40:17.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.096+0000 7f4a7659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a781017c0 0x7f4a78101130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.096+0000 7f4a7659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a781017c0 0x7f4a78101130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47612/0 (socket says 172.21.15.2:47612) 2023-09-13T04:40:17.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.096+0000 7f4a75d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a781040f0 0x7f4a780ff770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.096+0000 7f4a7659c700 1 -- 172.21.15.2:0/2380281191 learned_addr learned my addr 172.21.15.2:0/2380281191 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:17.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.096+0000 7f4a76d9d700 1 --2- 172.21.15.2:0/2380281191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a78106a20 0x7f4a780ffcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.096+0000 7f4a7659c700 1 -- 172.21.15.2:0/2380281191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a781040f0 msgr2=0x7f4a780ff770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.096+0000 7f4a7659c700 1 --2- 172.21.15.2:0/2380281191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a781040f0 0x7f4a780ff770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.096+0000 7f4a7659c700 1 -- 172.21.15.2:0/2380281191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a78106a20 msgr2=0x7f4a780ffcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.096+0000 7f4a7659c700 1 --2- 172.21.15.2:0/2380281191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a78106a20 0x7f4a780ffcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.096+0000 7f4a7659c700 1 -- 172.21.15.2:0/2380281191 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6c0095f0 con 0x7f4a781017c0 2023-09-13T04:40:17.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.096+0000 7f4a75d9b700 1 --2- 172.21.15.2:0/2380281191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a781040f0 0x7f4a780ff770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:17.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.097+0000 7f4a7659c700 1 --2- 172.21.15.2:0/2380281191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a781017c0 0x7f4a78101130 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4a68011040 tx=0x7f4a6800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:17.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.097+0000 7f4a677fe700 1 -- 172.21.15.2:0/2380281191 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6801c070 con 0x7f4a781017c0 2023-09-13T04:40:17.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.097+0000 7f4a677fe700 1 -- 172.21.15.2:0/2380281191 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a6800f7d0 con 0x7f4a781017c0 2023-09-13T04:40:17.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.097+0000 7f4a677fe700 1 -- 172.21.15.2:0/2380281191 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a680049c0 con 0x7f4a781017c0 2023-09-13T04:40:17.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.097+0000 7f4a7c9ff700 1 -- 172.21.15.2:0/2380281191 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a68009520 con 0x7f4a781017c0 2023-09-13T04:40:17.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.097+0000 7f4a7c9ff700 1 -- 172.21.15.2:0/2380281191 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a781a0e90 con 0x7f4a781017c0 2023-09-13T04:40:17.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.098+0000 7f4a7c9ff700 1 -- 172.21.15.2:0/2380281191 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a7810c4a0 con 0x7f4a781017c0 2023-09-13T04:40:17.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.100+0000 7f4a677fe700 1 -- 172.21.15.2:0/2380281191 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f4a68020070 con 0x7f4a781017c0 2023-09-13T04:40:17.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.103+0000 7f4a677fe700 1 --2- 172.21.15.2:0/2380281191 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4a600a0710 0x7f4a600a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.103+0000 7f4a677fe700 1 -- 172.21.15.2:0/2380281191 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4a680cd360 con 0x7f4a781017c0 2023-09-13T04:40:17.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.103+0000 7f4a75d9b700 1 --2- 172.21.15.2:0/2380281191 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4a600a0710 0x7f4a600a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.104+0000 7f4a75d9b700 1 --2- 172.21.15.2:0/2380281191 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4a600a0710 0x7f4a600a2bd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c005fd0 tx=0x7f4a6c009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:17.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.104+0000 7f4a677fe700 1 -- 172.21.15.2:0/2380281191 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4a68096b90 con 0x7f4a781017c0 2023-09-13T04:40:17.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:16 smithi149 ceph-mon[133315]: pgmap v348: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2023-09-13T04:40:17.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:16 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/4159107487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:40:17.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:16 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2675444764' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:40:17.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.320+0000 7f4a7c9ff700 1 -- 172.21.15.2:0/2380281191 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a781a1020 con 0x7f4a781017c0 2023-09-13T04:40:17.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.321+0000 7f4a677fe700 1 -- 172.21.15.2:0/2380281191 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24503 (secure 0 0 0) 0x7f4a68027090 con 0x7f4a781017c0 2023-09-13T04:40:17.323 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:40:17.323 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":92,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:39:25.580735+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T04:37:57.094802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T04:37:58.564487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-13T04:38:02.096895+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:40:17.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.324+0000 7f4a7c9ff700 1 -- 172.21.15.2:0/2380281191 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4a600a0710 msgr2=0x7f4a600a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.324+0000 7f4a7c9ff700 1 --2- 172.21.15.2:0/2380281191 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4a600a0710 0x7f4a600a2bd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c005fd0 tx=0x7f4a6c009310 comp rx=0 tx=0).stop 2023-09-13T04:40:17.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.324+0000 7f4a7c9ff700 1 -- 172.21.15.2:0/2380281191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a781017c0 msgr2=0x7f4a78101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.324+0000 7f4a7c9ff700 1 --2- 172.21.15.2:0/2380281191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a781017c0 0x7f4a78101130 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4a68011040 tx=0x7f4a6800b790 comp rx=0 tx=0).stop 2023-09-13T04:40:17.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.324+0000 7f4a7c9ff700 1 -- 172.21.15.2:0/2380281191 shutdown_connections 2023-09-13T04:40:17.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.324+0000 7f4a7c9ff700 1 --2- 172.21.15.2:0/2380281191 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f4a600a0710 0x7f4a600a2bd0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.324+0000 7f4a7c9ff700 1 --2- 172.21.15.2:0/2380281191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a78106a20 0x7f4a780ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.324+0000 7f4a7c9ff700 1 --2- 172.21.15.2:0/2380281191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a781017c0 0x7f4a78101130 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.324+0000 7f4a7c9ff700 1 --2- 172.21.15.2:0/2380281191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a781040f0 0x7f4a780ff770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.324+0000 7f4a7c9ff700 1 -- 172.21.15.2:0/2380281191 >> 172.21.15.2:0/2380281191 conn(0x7f4a780fb110 msgr2=0x7f4a78102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:17.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.324+0000 7f4a7c9ff700 1 -- 172.21.15.2:0/2380281191 shutdown_connections 2023-09-13T04:40:17.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.324+0000 7f4a7c9ff700 1 -- 172.21.15.2:0/2380281191 wait complete. 2023-09-13T04:40:17.345 DEBUG:tasks.fwd_scrub:Yielding 2023-09-13T04:40:17.346 INFO:teuthology.run_tasks:Running task exec... 2023-09-13T04:40:17.357 INFO:teuthology.task.exec:Executing custom commands... 2023-09-13T04:40:17.357 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2023-09-13T04:40:17.358 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-09-13T04:40:17.358 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-09-13T04:40:17.359 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-13T04:40:17.359 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-13T04:40:17.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.580+0000 7f7c19fff700 1 -- 172.21.15.2:0/1909536588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c14101740 msgr2=0x7f7c14103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.580+0000 7f7c19fff700 1 --2- 172.21.15.2:0/1909536588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c14101740 0x7f7c14103b30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009990 tx=0x7f7c04009ca0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.580+0000 7f7c19fff700 1 -- 172.21.15.2:0/1909536588 shutdown_connections 2023-09-13T04:40:17.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.580+0000 7f7c19fff700 1 --2- 172.21.15.2:0/1909536588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c141069a0 0x7f7c14108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.580+0000 7f7c19fff700 1 --2- 172.21.15.2:0/1909536588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c14101740 0x7f7c14103b30 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.580+0000 7f7c19fff700 1 --2- 172.21.15.2:0/1909536588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c14104070 0x7f7c14106460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.580+0000 7f7c19fff700 1 -- 172.21.15.2:0/1909536588 >> 172.21.15.2:0/1909536588 conn(0x7f7c140fb130 msgr2=0x7f7c140fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:17.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.580+0000 7f7c19fff700 1 -- 172.21.15.2:0/1909536588 shutdown_connections 2023-09-13T04:40:17.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.580+0000 7f7c19fff700 1 -- 172.21.15.2:0/1909536588 wait complete. 2023-09-13T04:40:17.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.581+0000 7f7c19fff700 1 Processor -- start 2023-09-13T04:40:17.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.582+0000 7f7c19fff700 1 -- start start 2023-09-13T04:40:17.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.582+0000 7f7c19fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c14101740 0x7f7c14196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.582+0000 7f7c19fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c14104070 0x7f7c14196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.582+0000 7f7c19fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c141069a0 0x7f7c1419dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.582+0000 7f7c19fff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1419e580 con 0x7f7c141069a0 2023-09-13T04:40:17.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.582+0000 7f7c19fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1419e6c0 con 0x7f7c14104070 2023-09-13T04:40:17.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.582+0000 7f7c19fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1419e850 con 0x7f7c14101740 2023-09-13T04:40:17.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.582+0000 7f7c137fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c14101740 0x7f7c14196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.582+0000 7f7c137fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c14101740 0x7f7c14196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:35274/0 (socket says 172.21.15.2:35274) 2023-09-13T04:40:17.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.582+0000 7f7c137fe700 1 -- 172.21.15.2:0/2663888850 learned_addr learned my addr 172.21.15.2:0/2663888850 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:17.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.582+0000 7f7c0bfff700 1 --2- 172.21.15.2:0/2663888850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c14104070 0x7f7c14196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.582+0000 7f7c13fff700 1 --2- 172.21.15.2:0/2663888850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c141069a0 0x7f7c1419dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.583+0000 7f7c137fe700 1 -- 172.21.15.2:0/2663888850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c14104070 msgr2=0x7f7c14196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.583+0000 7f7c137fe700 1 --2- 172.21.15.2:0/2663888850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c14104070 0x7f7c14196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.583+0000 7f7c137fe700 1 -- 172.21.15.2:0/2663888850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c141069a0 msgr2=0x7f7c1419dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.583+0000 7f7c137fe700 1 --2- 172.21.15.2:0/2663888850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c141069a0 0x7f7c1419dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.583+0000 7f7c137fe700 1 -- 172.21.15.2:0/2663888850 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c040095f0 con 0x7f7c14101740 2023-09-13T04:40:17.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.583+0000 7f7c137fe700 1 --2- 172.21.15.2:0/2663888850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c14101740 0x7f7c14196a00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400b680 tx=0x7f7c04005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:17.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.583+0000 7f7c117fa700 1 -- 172.21.15.2:0/2663888850 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0401d070 con 0x7f7c14101740 2023-09-13T04:40:17.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.583+0000 7f7c117fa700 1 -- 172.21.15.2:0/2663888850 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c04022470 con 0x7f7c14101740 2023-09-13T04:40:17.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.583+0000 7f7c19fff700 1 -- 172.21.15.2:0/2663888850 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c1419ead0 con 0x7f7c14101740 2023-09-13T04:40:17.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.583+0000 7f7c117fa700 1 -- 172.21.15.2:0/2663888850 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c04017650 con 0x7f7c14101740 2023-09-13T04:40:17.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.584+0000 7f7c19fff700 1 -- 172.21.15.2:0/2663888850 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c1419eef0 con 0x7f7c14101740 2023-09-13T04:40:17.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.585+0000 7f3fa8dff700 1 -- 172.21.15.2:0/1623146940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fa4074cf0 msgr2=0x7f3fa4073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.585+0000 7f3fa8dff700 1 --2- 172.21.15.2:0/1623146940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fa4074cf0 0x7f3fa4073140 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009990 tx=0x7f3f94009ca0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.585+0000 7f3fa8dff700 1 -- 172.21.15.2:0/1623146940 shutdown_connections 2023-09-13T04:40:17.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.585+0000 7f3fa8dff700 1 --2- 172.21.15.2:0/1623146940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fa410d2d0 0x7f3fa410f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.585+0000 7f3fa8dff700 1 --2- 172.21.15.2:0/1623146940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fa4074cf0 0x7f3fa4073140 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.585+0000 7f3fa8dff700 1 --2- 172.21.15.2:0/1623146940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fa4073710 0x7f3fa4073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.585+0000 7f3fa8dff700 1 -- 172.21.15.2:0/1623146940 >> 172.21.15.2:0/1623146940 conn(0x7f3fa40fc230 msgr2=0x7f3fa40fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:17.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.585+0000 7f3fa8dff700 1 -- 172.21.15.2:0/1623146940 shutdown_connections 2023-09-13T04:40:17.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.585+0000 7f3fa8dff700 1 -- 172.21.15.2:0/1623146940 wait complete. 2023-09-13T04:40:17.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.586+0000 7f3fa8dff700 1 Processor -- start 2023-09-13T04:40:17.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.586+0000 7f3fa8dff700 1 -- start start 2023-09-13T04:40:17.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.586+0000 7f3fa8dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fa4073710 0x7f3fa419d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.586+0000 7f3fa8dff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fa4074cf0 0x7f3fa419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.586+0000 7f3fa8dff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fa410d2d0 0x7f3fa419e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.587+0000 7f3fa8dff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa419eb10 con 0x7f3fa410d2d0 2023-09-13T04:40:17.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.587+0000 7f3fa8dff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa419ec80 con 0x7f3fa4074cf0 2023-09-13T04:40:17.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.587+0000 7f3fa8dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa419ee40 con 0x7f3fa4073710 2023-09-13T04:40:17.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.587+0000 7f3fa259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fa4073710 0x7f3fa419d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.587+0000 7f7c117fa700 1 -- 172.21.15.2:0/2663888850 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f7c040225e0 con 0x7f7c14101740 2023-09-13T04:40:17.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.587+0000 7f3fa259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fa4073710 0x7f3fa419d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:35278/0 (socket says 172.21.15.2:35278) 2023-09-13T04:40:17.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.587+0000 7f3fa259c700 1 -- 172.21.15.2:0/1649555992 learned_addr learned my addr 172.21.15.2:0/1649555992 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:17.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.587+0000 7f3fa1d9b700 1 --2- 172.21.15.2:0/1649555992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fa4074cf0 0x7f3fa419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.587+0000 7f3fa2d9d700 1 --2- 172.21.15.2:0/1649555992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fa410d2d0 0x7f3fa419e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.587+0000 7f3fa259c700 1 -- 172.21.15.2:0/1649555992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fa4074cf0 msgr2=0x7f3fa419d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.587+0000 7f3fa259c700 1 --2- 172.21.15.2:0/1649555992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fa4074cf0 0x7f3fa419d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.587+0000 7f3fa259c700 1 -- 172.21.15.2:0/1649555992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fa410d2d0 msgr2=0x7f3fa419e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.587+0000 7f3fa259c700 1 --2- 172.21.15.2:0/1649555992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fa410d2d0 0x7f3fa419e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.587+0000 7f3fa259c700 1 -- 172.21.15.2:0/1649555992 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f940095f0 con 0x7f3fa4073710 2023-09-13T04:40:17.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.587+0000 7f3fa2d9d700 1 --2- 172.21.15.2:0/1649555992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fa410d2d0 0x7f3fa419e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:40:17.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.587+0000 7f3fa259c700 1 --2- 172.21.15.2:0/1649555992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fa4073710 0x7f3fa419d0f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400b680 tx=0x7f3f940057b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:17.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.588+0000 7f3f937fe700 1 -- 172.21.15.2:0/1649555992 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9401d070 con 0x7f3fa4073710 2023-09-13T04:40:17.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.588+0000 7f3f937fe700 1 -- 172.21.15.2:0/1649555992 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f9400fbc0 con 0x7f3fa4073710 2023-09-13T04:40:17.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.588+0000 7f3f937fe700 1 -- 172.21.15.2:0/1649555992 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f94017650 con 0x7f3fa4073710 2023-09-13T04:40:17.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.588+0000 7f3fa8dff700 1 -- 172.21.15.2:0/1649555992 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa419f0c0 con 0x7f3fa4073710 2023-09-13T04:40:17.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.588+0000 7f3fa8dff700 1 -- 172.21.15.2:0/1649555992 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa4104ca0 con 0x7f3fa4073710 2023-09-13T04:40:17.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.589+0000 7f7c2d9acb80 1 Processor -- start 2023-09-13T04:40:17.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.589+0000 7f7c2d9acb80 1 -- start start 2023-09-13T04:40:17.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.589+0000 7f7c117fa700 1 --2- 172.21.15.2:0/2663888850 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f7c000a0640 0x7f7c000a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.589+0000 7f3fa8dff700 1 -- 172.21.15.2:0/1649555992 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa4102e20 con 0x7f3fa4073710 2023-09-13T04:40:17.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.589+0000 7f7c117fa700 1 -- 172.21.15.2:0/2663888850 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f7c040cd380 con 0x7f7c14101740 2023-09-13T04:40:17.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.589+0000 7f7c2d9acb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558223f0f780 0x558223f0fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.589+0000 7f7c2d9acb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558223f100e0 0x558223f0e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.589+0000 7f7c2d9acb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558223f0ea80 0x558223f0ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.589+0000 7f7c2d9acb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558223e67800 con 0x558223f0ea80 2023-09-13T04:40:17.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.589+0000 7f7c2d9acb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558223e65120 con 0x558223f0f780 2023-09-13T04:40:17.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.589+0000 7f7c2d9acb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558223e60a10 con 0x558223f100e0 2023-09-13T04:40:17.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.590+0000 7f7c13fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558223f0ea80 0x558223f0ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.590+0000 7f7c0bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558223f100e0 0x558223f0e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.590+0000 7f7c0bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558223f100e0 0x558223f0e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:35294/0 (socket says 172.21.15.2:35294) 2023-09-13T04:40:17.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.590+0000 7f7c0bfff700 1 -- 172.21.15.2:0/2462985444 learned_addr learned my addr 172.21.15.2:0/2462985444 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:17.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.590+0000 7f7c13fff700 1 --2- 172.21.15.2:0/2663888850 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f7c000a0640 0x7f7c000a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.590+0000 7f7c0bfff700 1 -- 172.21.15.2:0/2462985444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558223f0f780 msgr2=0x558223f0fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:17.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.590+0000 7f7c0bfff700 1 --2- 172.21.15.2:0/2462985444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558223f0f780 0x558223f0fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.590+0000 7f7c0bfff700 1 -- 172.21.15.2:0/2462985444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558223f0ea80 msgr2=0x558223f0ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:17.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.590+0000 7f7c0bfff700 1 --2- 172.21.15.2:0/2462985444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558223f0ea80 0x558223f0ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.590+0000 7f7c0bfff700 1 -- 172.21.15.2:0/2462985444 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558223ce06b0 con 0x558223f100e0 2023-09-13T04:40:17.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.590+0000 7f7c13fff700 1 --2- 172.21.15.2:0/2462985444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558223f0ea80 0x558223f0ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:40:17.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.590+0000 7f7c0bfff700 1 --2- 172.21.15.2:0/2462985444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558223f100e0 0x558223f0e540 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009d80 tx=0x7f7bfc011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccac0be733d8157 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:17.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.591+0000 7f7c13fff700 1 --2- 172.21.15.2:0/2663888850 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f7c000a0640 0x7f7c000a2b00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009ec0 tx=0x7f7c0c00b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:17.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.591+0000 7f7c09ffb700 1 -- 172.21.15.2:0/2462985444 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc004230 con 0x558223f100e0 2023-09-13T04:40:17.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.591+0000 7f7c09ffb700 1 -- 172.21.15.2:0/2462985444 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bfc004390 con 0x558223f100e0 2023-09-13T04:40:17.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.591+0000 7f7c09ffb700 1 -- 172.21.15.2:0/2462985444 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc019880 con 0x558223f100e0 2023-09-13T04:40:17.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.591+0000 7f3f937fe700 1 -- 172.21.15.2:0/1649555992 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f3f94032420 con 0x7f3fa4073710 2023-09-13T04:40:17.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.591+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/2462985444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558223f100e0 msgr2=0x558223f0e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.591+0000 7f7c2d9acb80 1 --2- 172.21.15.2:0/2462985444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558223f100e0 0x558223f0e540 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009d80 tx=0x7f7bfc011720 comp rx=0 tx=0).stop 2023-09-13T04:40:17.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.592+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/2462985444 shutdown_connections 2023-09-13T04:40:17.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.592+0000 7f7c2d9acb80 1 --2- 172.21.15.2:0/2462985444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558223f0ea80 0x558223f0ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.592+0000 7f7c2d9acb80 1 --2- 172.21.15.2:0/2462985444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558223f0f780 0x558223f0fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.592+0000 7f7c2d9acb80 1 --2- 172.21.15.2:0/2462985444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558223f100e0 0x558223f0e540 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.592+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/2462985444 >> 172.21.15.2:0/2462985444 conn(0x558223e6cd00 msgr2=0x558223f08f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:17.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.592+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/2462985444 shutdown_connections 2023-09-13T04:40:17.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.592+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/2462985444 wait complete. 2023-09-13T04:40:17.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.592+0000 7f7c2d9acb80 1 -- start start 2023-09-13T04:40:17.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.592+0000 7f7c2d9acb80 1 Processor -- start 2023-09-13T04:40:17.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.593+0000 7f3f937fe700 1 --2- 172.21.15.2:0/1649555992 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f3f8c0a0710 0x7f3f8c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.593+0000 7f7c2d9acb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558223f0ea80 0x558223f13ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.593+0000 7f7c2d9acb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558223f0f780 0x558223f14010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.593+0000 7f7c2d9acb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558223f100e0 0x558223f15020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.593+0000 7f7c2d9acb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558223dfec30 con 0x558223f0f780 2023-09-13T04:40:17.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.593+0000 7f7c2d9acb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558223dfe930 con 0x558223f100e0 2023-09-13T04:40:17.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.593+0000 7f7c2d9acb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558223ce29a0 con 0x558223f0ea80 2023-09-13T04:40:17.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.593+0000 7f7c2d9acb80 10 client.0 _ll_register_callbacks cb 0x558223dfc700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:17.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.593+0000 7f3f937fe700 1 -- 172.21.15.2:0/1649555992 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3f940de3b0 con 0x7f3fa4073710 2023-09-13T04:40:17.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.593+0000 7f3fa1d9b700 1 --2- 172.21.15.2:0/1649555992 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f3f8c0a0710 0x7f3f8c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.593+0000 7f7c13fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558223f100e0 0x558223f15020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.593+0000 7f7c0bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558223f0f780 0x558223f14010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.593+0000 7f7c13fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558223f100e0 0x558223f15020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39704/0 (socket says 172.21.15.2:39704) 2023-09-13T04:40:17.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.593+0000 7f7c13fff700 1 -- 172.21.15.2:0/948468453 learned_addr learned my addr 172.21.15.2:0/948468453 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:17.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.593+0000 7f7c137fe700 1 --2- 172.21.15.2:0/948468453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558223f0ea80 0x558223f13ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.594+0000 7f7c13fff700 1 -- 172.21.15.2:0/948468453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558223f0ea80 msgr2=0x558223f13ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:17.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.594+0000 7f7c13fff700 1 --2- 172.21.15.2:0/948468453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558223f0ea80 0x558223f13ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.594+0000 7f7c13fff700 1 -- 172.21.15.2:0/948468453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558223f0f780 msgr2=0x558223f14010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:17.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.594+0000 7f7c13fff700 1 --2- 172.21.15.2:0/948468453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558223f0f780 0x558223f14010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.594+0000 7f7c13fff700 1 -- 172.21.15.2:0/948468453 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bfc0099e0 con 0x558223f100e0 2023-09-13T04:40:17.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.594+0000 7f7c137fe700 1 --2- 172.21.15.2:0/948468453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558223f0ea80 0x558223f13ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:40:17.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.594+0000 7f7c13fff700 1 --2- 172.21.15.2:0/948468453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558223f100e0 0x558223f15020 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c008f10 tx=0x7f7c0c004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f5a66ee52ce2e41 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:17.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.594+0000 7f7bf2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:40:17.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.594+0000 7f7bf2ffd700 1 -- 172.21.15.2:0/948468453 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c00e040 con 0x558223f100e0 2023-09-13T04:40:17.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.594+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/948468453 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558223f016f0 con 0x558223f100e0 2023-09-13T04:40:17.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.594+0000 7f7bf2ffd700 1 -- 172.21.15.2:0/948468453 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f7c0c003c10 con 0x558223f100e0 2023-09-13T04:40:17.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.594+0000 7f3fa1d9b700 1 --2- 172.21.15.2:0/1649555992 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f3f8c0a0710 0x7f3f8c0a2bd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4074a10 tx=0x7f3f98007400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:17.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.595+0000 7f7bf2ffd700 1 -- 172.21.15.2:0/948468453 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c0c003dd0 con 0x558223f100e0 2023-09-13T04:40:17.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.595+0000 7f7c2d9acb80 10 client.25487 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:17.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.595+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/948468453 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x558223e78820 con 0x558223f100e0 2023-09-13T04:40:17.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.595+0000 7f7bf2ffd700 1 -- 172.21.15.2:0/948468453 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558223f016f0 con 0x558223f100e0 2023-09-13T04:40:17.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.595+0000 7f7bf2ffd700 1 -- 172.21.15.2:0/948468453 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7c0c007760 con 0x558223f100e0 2023-09-13T04:40:17.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.595+0000 7f7c2d9acb80 10 client.25487 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:17.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.595+0000 7f7c2d9acb80 10 client.25487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:40:17.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.595+0000 7f7c2d9acb80 20 client.25487 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:17.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.595+0000 7f7c2d9acb80 1 --2- 172.21.15.2:0/948468453 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x558223f2c6f0 0x558223f2eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.595+0000 7f7c2d9acb80 4 client.25487 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:40:17.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.595+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/948468453 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558223e599f0 con 0x558223f2c6f0 2023-09-13T04:40:17.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.596+0000 7f7c137fe700 1 --2- 172.21.15.2:0/948468453 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x558223f2c6f0 0x558223f2eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.596+0000 7f7c137fe700 1 --2- 172.21.15.2:0/948468453 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x558223f2c6f0 0x558223f2eb10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f821e89113d33d92 server_cookie=26b9decdcebe3aec in_seq=0 out_seq=0 2023-09-13T04:40:17.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.596+0000 7f7bf2ffd700 10 client.25487 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:17.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.597+0000 7f3f937fe700 1 -- 172.21.15.2:0/1649555992 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3f940a7be0 con 0x7f3fa4073710 2023-09-13T04:40:17.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.597+0000 7f7bf2ffd700 1 -- 172.21.15.2:0/948468453 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558223e599f0 con 0x558223f2c6f0 2023-09-13T04:40:17.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.597+0000 7f7bf2ffd700 10 client.25487 handle_command_reply: tid=0 2023-09-13T04:40:17.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.597+0000 7f7c2d9acb80 1 client.25487 shutdown 2023-09-13T04:40:17.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.597+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/948468453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558223f100e0 msgr2=0x558223f15020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.597+0000 7f7c2d9acb80 1 --2- 172.21.15.2:0/948468453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558223f100e0 0x558223f15020 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c008f10 tx=0x7f7c0c004b40 comp rx=0 tx=0).stop 2023-09-13T04:40:17.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.597+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/948468453 shutdown_connections 2023-09-13T04:40:17.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.597+0000 7f7c2d9acb80 1 --2- 172.21.15.2:0/948468453 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x558223f2c6f0 0x558223f2eb10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.597+0000 7f7c2d9acb80 1 --2- 172.21.15.2:0/948468453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558223f100e0 0x558223f15020 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.597+0000 7f7c2d9acb80 1 --2- 172.21.15.2:0/948468453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558223f0ea80 0x558223f13ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.597+0000 7f7bf2ffd700 0 client.25487 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:17.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.597+0000 7f7c2d9acb80 1 --2- 172.21.15.2:0/948468453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558223f0f780 0x558223f14010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.597+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/948468453 >> 172.21.15.2:0/948468453 conn(0x558223e6cd00 msgr2=0x558223f03870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:17.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.597+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/948468453 shutdown_connections 2023-09-13T04:40:17.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.597+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/948468453 wait complete. 2023-09-13T04:40:17.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.597+0000 7f7c2d9acb80 20 client.25487 trim_cache size 0 max 16384 2023-09-13T04:40:17.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.615+0000 7f7c2d9acb80 1 Processor -- start 2023-09-13T04:40:17.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.616+0000 7f7c2d9acb80 1 -- start start 2023-09-13T04:40:17.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.616+0000 7f7c2d9acb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558223f0ea80 0x558223f1a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.616+0000 7f7c2d9acb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558223f0f780 0x558223f10770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.616+0000 7f7c2d9acb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558223f100e0 0x558223f10cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.616+0000 7f7c2d9acb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558223f016d0 con 0x558223f0f780 2023-09-13T04:40:17.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.616+0000 7f7c2d9acb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558223f09a10 con 0x558223f0ea80 2023-09-13T04:40:17.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.616+0000 7f7c2d9acb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558223f05960 con 0x558223f100e0 2023-09-13T04:40:17.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.616+0000 7f7c0bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558223f0f780 0x558223f10770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.616+0000 7f7c0bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558223f0f780 0x558223f10770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47676/0 (socket says 172.21.15.2:47676) 2023-09-13T04:40:17.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.616+0000 7f7c0bfff700 1 -- 172.21.15.2:0/3097563097 learned_addr learned my addr 172.21.15.2:0/3097563097 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:17.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.616+0000 7f7c137fe700 1 --2- 172.21.15.2:0/3097563097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558223f0ea80 0x558223f1a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.616+0000 7f7c13fff700 1 --2- 172.21.15.2:0/3097563097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558223f100e0 0x558223f10cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.617+0000 7f7c137fe700 1 -- 172.21.15.2:0/3097563097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558223f100e0 msgr2=0x558223f10cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:17.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.617+0000 7f7c137fe700 1 --2- 172.21.15.2:0/3097563097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558223f100e0 0x558223f10cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.617+0000 7f7c137fe700 1 -- 172.21.15.2:0/3097563097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558223f0f780 msgr2=0x558223f10770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:17.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.617+0000 7f7c137fe700 1 --2- 172.21.15.2:0/3097563097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558223f0f780 0x558223f10770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.617+0000 7f7c137fe700 1 -- 172.21.15.2:0/3097563097 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558223c8ce60 con 0x558223f0ea80 2023-09-13T04:40:17.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.617+0000 7f7c137fe700 1 --2- 172.21.15.2:0/3097563097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558223f0ea80 0x558223f1a490 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7c040caaf0 tx=0x7f7c040cd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28e558212e6509c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:17.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.617+0000 7f7c0a7fc700 1 -- 172.21.15.2:0/3097563097 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c040b6070 con 0x558223f0ea80 2023-09-13T04:40:17.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.617+0000 7f7c0a7fc700 1 -- 172.21.15.2:0/3097563097 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c0409e7c0 con 0x558223f0ea80 2023-09-13T04:40:17.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.617+0000 7f7c0a7fc700 1 -- 172.21.15.2:0/3097563097 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c04098d30 con 0x558223f0ea80 2023-09-13T04:40:17.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.618+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/3097563097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558223f0ea80 msgr2=0x558223f1a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.618+0000 7f7c2d9acb80 1 --2- 172.21.15.2:0/3097563097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558223f0ea80 0x558223f1a490 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7c040caaf0 tx=0x7f7c040cd040 comp rx=0 tx=0).stop 2023-09-13T04:40:17.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.618+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/3097563097 shutdown_connections 2023-09-13T04:40:17.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.618+0000 7f7c2d9acb80 1 --2- 172.21.15.2:0/3097563097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558223f100e0 0x558223f10cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.618+0000 7f7c2d9acb80 1 --2- 172.21.15.2:0/3097563097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558223f0ea80 0x558223f1a490 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.618+0000 7f7c2d9acb80 1 --2- 172.21.15.2:0/3097563097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558223f0f780 0x558223f10770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.618+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/3097563097 >> 172.21.15.2:0/3097563097 conn(0x558223e6cd00 msgr2=0x558223f19d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:17.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.618+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/3097563097 shutdown_connections 2023-09-13T04:40:17.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.618+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/3097563097 wait complete. 2023-09-13T04:40:17.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.619+0000 7f7c2d9acb80 1 -- start start 2023-09-13T04:40:17.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.619+0000 7f7c2d9acb80 1 Processor -- start 2023-09-13T04:40:17.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.619+0000 7f7c2d9acb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558223f0ea80 0x558223f28de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.619+0000 7f7c2d9acb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558223f0f780 0x558223f25880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.619+0000 7f7c2d9acb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558223f100e0 0x558223f25dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.619+0000 7f7c2d9acb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558223f264a0 con 0x558223f0f780 2023-09-13T04:40:17.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.619+0000 7f7c2d9acb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558223f80510 con 0x558223f100e0 2023-09-13T04:40:17.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.619+0000 7f7c2d9acb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558223f80650 con 0x558223f0ea80 2023-09-13T04:40:17.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.619+0000 7f7c2d9acb80 10 client.0 _ll_register_callbacks cb 0x558223f15500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:17.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.620+0000 7f7c13fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558223f100e0 0x558223f25dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.620+0000 7f7c13fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558223f100e0 0x558223f25dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39720/0 (socket says 172.21.15.2:39720) 2023-09-13T04:40:17.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.620+0000 7f7c13fff700 1 -- 172.21.15.2:0/3567644651 learned_addr learned my addr 172.21.15.2:0/3567644651 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:17.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.620+0000 7f7c0bfff700 1 --2- 172.21.15.2:0/3567644651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558223f0f780 0x558223f25880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.620+0000 7f7c137fe700 1 --2- 172.21.15.2:0/3567644651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558223f0ea80 0x558223f28de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.620+0000 7f7c13fff700 1 -- 172.21.15.2:0/3567644651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558223f0ea80 msgr2=0x558223f28de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:17.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.620+0000 7f7c13fff700 1 --2- 172.21.15.2:0/3567644651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558223f0ea80 0x558223f28de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.620+0000 7f7c13fff700 1 -- 172.21.15.2:0/3567644651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558223f0f780 msgr2=0x558223f25880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:17.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.620+0000 7f7c13fff700 1 --2- 172.21.15.2:0/3567644651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558223f0f780 0x558223f25880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.620+0000 7f7c13fff700 1 -- 172.21.15.2:0/3567644651 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c1419eef0 con 0x558223f100e0 2023-09-13T04:40:17.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.620+0000 7f7c0bfff700 1 --2- 172.21.15.2:0/3567644651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558223f0f780 0x558223f25880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:40:17.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.620+0000 7f7c13fff700 1 --2- 172.21.15.2:0/3567644651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558223f100e0 0x558223f25dc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c008f10 tx=0x7f7c0c008860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b58dc22685569a0c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:17.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.621+0000 7f7bf2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:40:17.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.621+0000 7f7bf2ffd700 1 -- 172.21.15.2:0/3567644651 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c00e040 con 0x558223f100e0 2023-09-13T04:40:17.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.621+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/3567644651 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7bfc004390 con 0x558223f100e0 2023-09-13T04:40:17.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.621+0000 7f7bf2ffd700 1 -- 172.21.15.2:0/3567644651 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f7c0c01a470 con 0x558223f100e0 2023-09-13T04:40:17.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.621+0000 7f7bf2ffd700 1 -- 172.21.15.2:0/3567644651 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c0c01a630 con 0x558223f100e0 2023-09-13T04:40:17.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.621+0000 7f7c2d9acb80 10 client.25493 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:17.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.621+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/3567644651 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x558223c75ce0 con 0x558223f100e0 2023-09-13T04:40:17.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.621+0000 7f7bf2ffd700 1 -- 172.21.15.2:0/3567644651 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bfc004390 con 0x558223f100e0 2023-09-13T04:40:17.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.622+0000 7f7bf2ffd700 1 -- 172.21.15.2:0/3567644651 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7c0c02d070 con 0x558223f100e0 2023-09-13T04:40:17.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.622+0000 7f7c2d9acb80 10 client.25493 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:17.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.622+0000 7f7c2d9acb80 10 client.25493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:40:17.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.622+0000 7f7c2d9acb80 20 client.25493 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:17.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.622+0000 7f7c2d9acb80 1 --2- 172.21.15.2:0/3567644651 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x558223f82060 0x558223f84480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.622+0000 7f7c2d9acb80 4 client.25493 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-13T04:40:17.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.622+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/3567644651 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558223e1a0d0 con 0x558223f82060 2023-09-13T04:40:17.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.622+0000 7f7c137fe700 1 --2- 172.21.15.2:0/3567644651 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x558223f82060 0x558223f84480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.623+0000 7f7bf2ffd700 10 client.25493 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:17.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.623+0000 7f7c137fe700 1 --2- 172.21.15.2:0/3567644651 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x558223f82060 0x558223f84480 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20d33e0815ceab48 server_cookie=b9456e22957048b4 in_seq=0 out_seq=0 2023-09-13T04:40:17.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.623+0000 7f7bf2ffd700 1 -- 172.21.15.2:0/3567644651 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558223e1a0d0 con 0x558223f82060 2023-09-13T04:40:17.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.623+0000 7f7bf2ffd700 10 client.25493 handle_command_reply: tid=0 2023-09-13T04:40:17.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.623+0000 7f7c2d9acb80 1 client.25493 shutdown 2023-09-13T04:40:17.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.623+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/3567644651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558223f100e0 msgr2=0x558223f25dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.623+0000 7f7c2d9acb80 1 --2- 172.21.15.2:0/3567644651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558223f100e0 0x558223f25dc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c008f10 tx=0x7f7c0c008860 comp rx=0 tx=0).stop 2023-09-13T04:40:17.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.623+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/3567644651 shutdown_connections 2023-09-13T04:40:17.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.623+0000 7f7c2d9acb80 1 --2- 172.21.15.2:0/3567644651 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x558223f82060 0x558223f84480 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.623+0000 7f7c2d9acb80 1 --2- 172.21.15.2:0/3567644651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558223f100e0 0x558223f25dc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.623+0000 7f7c2d9acb80 1 --2- 172.21.15.2:0/3567644651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558223f0ea80 0x558223f28de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.623+0000 7f7bf2ffd700 0 client.25493 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:17.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.623+0000 7f7c2d9acb80 1 --2- 172.21.15.2:0/3567644651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558223f0f780 0x558223f25880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.623+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/3567644651 >> 172.21.15.2:0/3567644651 conn(0x558223e6cd00 msgr2=0x558223f11e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:17.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.624+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/3567644651 shutdown_connections 2023-09-13T04:40:17.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.624+0000 7f7c2d9acb80 1 -- 172.21.15.2:0/3567644651 wait complete. 2023-09-13T04:40:17.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.624+0000 7f7c2d9acb80 20 client.25493 trim_cache size 0 max 16384 2023-09-13T04:40:17.647 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:40:17.647 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2023-09-13T04:40:17.647 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "1bb91399-a885-4a35-91ee-1676b6635e3d", 2023-09-13T04:40:17.647 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2023-09-13T04:40:17.647 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:40:17.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.625+0000 7f7c19fff700 1 -- 172.21.15.2:0/2663888850 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f7c000a0640 msgr2=0x7f7c000a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.625+0000 7f7c19fff700 1 --2- 172.21.15.2:0/2663888850 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f7c000a0640 0x7f7c000a2b00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009ec0 tx=0x7f7c0c00b040 comp rx=0 tx=0).stop 2023-09-13T04:40:17.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.625+0000 7f7c19fff700 1 -- 172.21.15.2:0/2663888850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c14101740 msgr2=0x7f7c14196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.625+0000 7f7c19fff700 1 --2- 172.21.15.2:0/2663888850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c14101740 0x7f7c14196a00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400b680 tx=0x7f7c04005740 comp rx=0 tx=0).stop 2023-09-13T04:40:17.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.625+0000 7f7c19fff700 1 -- 172.21.15.2:0/2663888850 shutdown_connections 2023-09-13T04:40:17.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.625+0000 7f7c19fff700 1 --2- 172.21.15.2:0/2663888850 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f7c000a0640 0x7f7c000a2b00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.625+0000 7f7c19fff700 1 --2- 172.21.15.2:0/2663888850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c141069a0 0x7f7c1419dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.625+0000 7f7c19fff700 1 --2- 172.21.15.2:0/2663888850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c14101740 0x7f7c14196a00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.625+0000 7f7c19fff700 1 --2- 172.21.15.2:0/2663888850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c14104070 0x7f7c14196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.625+0000 7f7c19fff700 1 -- 172.21.15.2:0/2663888850 >> 172.21.15.2:0/2663888850 conn(0x7f7c140fb130 msgr2=0x7f7c141023b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:17.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.625+0000 7f7c19fff700 1 -- 172.21.15.2:0/2663888850 shutdown_connections 2023-09-13T04:40:17.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.625+0000 7f7c19fff700 1 -- 172.21.15.2:0/2663888850 wait complete. 2023-09-13T04:40:17.655 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T04:40:17.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.876+0000 7f811d9ff700 1 -- 172.21.15.2:0/1707200198 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81180ffe90 msgr2=0x7f8118100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.876+0000 7f811d9ff700 1 --2- 172.21.15.2:0/1707200198 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81180ffe90 0x7f8118100310 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f810c009990 tx=0x7f810c009ca0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.876+0000 7f811d9ff700 1 -- 172.21.15.2:0/1707200198 shutdown_connections 2023-09-13T04:40:17.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.876+0000 7f811d9ff700 1 --2- 172.21.15.2:0/1707200198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8118100850 0x7f8118100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.876+0000 7f811d9ff700 1 --2- 172.21.15.2:0/1707200198 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81180ff530 0x7f81180ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.876+0000 7f811d9ff700 1 --2- 172.21.15.2:0/1707200198 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81180ffe90 0x7f8118100310 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.876+0000 7f811d9ff700 1 -- 172.21.15.2:0/1707200198 >> 172.21.15.2:0/1707200198 conn(0x7f81180fb130 msgr2=0x7f81180fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:17.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.876+0000 7f811d9ff700 1 -- 172.21.15.2:0/1707200198 shutdown_connections 2023-09-13T04:40:17.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.876+0000 7f811d9ff700 1 -- 172.21.15.2:0/1707200198 wait complete. 2023-09-13T04:40:17.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.877+0000 7f811d9ff700 1 Processor -- start 2023-09-13T04:40:17.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.878+0000 7f811d9ff700 1 -- start start 2023-09-13T04:40:17.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.878+0000 7f811d9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81180ff530 0x7f8118198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.878+0000 7f811d9ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81180ffe90 0x7f8118199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.878+0000 7f811d9ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8118100850 0x7f81181a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.878+0000 7f811d9ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81181a0770 con 0x7f81180ffe90 2023-09-13T04:40:17.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.878+0000 7f811d9ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81181a08e0 con 0x7f8118100850 2023-09-13T04:40:17.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.878+0000 7f811d9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81181a0aa0 con 0x7f81180ff530 2023-09-13T04:40:17.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.878+0000 7f81167fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81180ffe90 0x7f8118199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.878+0000 7f81167fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81180ffe90 0x7f8118199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47706/0 (socket says 172.21.15.2:47706) 2023-09-13T04:40:17.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.878+0000 7f81167fc700 1 -- 172.21.15.2:0/750665752 learned_addr learned my addr 172.21.15.2:0/750665752 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:17.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.878+0000 7f8116ffd700 1 --2- 172.21.15.2:0/750665752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81180ff530 0x7f8118198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.879+0000 7f81177fe700 1 --2- 172.21.15.2:0/750665752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8118100850 0x7f81181a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.879+0000 7f8116ffd700 1 -- 172.21.15.2:0/750665752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8118100850 msgr2=0x7f81181a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.879+0000 7f8116ffd700 1 --2- 172.21.15.2:0/750665752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8118100850 0x7f81181a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.879+0000 7f8116ffd700 1 -- 172.21.15.2:0/750665752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81180ffe90 msgr2=0x7f8118199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.879+0000 7f8116ffd700 1 --2- 172.21.15.2:0/750665752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81180ffe90 0x7f8118199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.879+0000 7f8116ffd700 1 -- 172.21.15.2:0/750665752 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8110009c40 con 0x7f81180ff530 2023-09-13T04:40:17.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.879+0000 7f81177fe700 1 --2- 172.21.15.2:0/750665752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8118100850 0x7f81181a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:40:17.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.879+0000 7f8116ffd700 1 --2- 172.21.15.2:0/750665752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81180ff530 0x7f8118198c30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f810800e9f0 tx=0x7f810800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:17.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.879+0000 7f811c9fd700 1 -- 172.21.15.2:0/750665752 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810800eee0 con 0x7f81180ff530 2023-09-13T04:40:17.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.879+0000 7f811c9fd700 1 -- 172.21.15.2:0/750665752 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f810800cbd0 con 0x7f81180ff530 2023-09-13T04:40:17.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.879+0000 7f811d9ff700 1 -- 172.21.15.2:0/750665752 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f810c0095f0 con 0x7f81180ff530 2023-09-13T04:40:17.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.879+0000 7f811c9fd700 1 -- 172.21.15.2:0/750665752 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108018680 con 0x7f81180ff530 2023-09-13T04:40:17.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.880+0000 7f811d9ff700 1 -- 172.21.15.2:0/750665752 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81181a1150 con 0x7f81180ff530 2023-09-13T04:40:17.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.884+0000 7f81313f2b80 1 Processor -- start 2023-09-13T04:40:17.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.884+0000 7f811c9fd700 1 -- 172.21.15.2:0/750665752 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135258+0+0 (secure 0 0 0) 0x7f810800cd40 con 0x7f81180ff530 2023-09-13T04:40:17.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.885+0000 7f81313f2b80 1 -- start start 2023-09-13T04:40:17.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.885+0000 7f81313f2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edbf700 0x56435edbfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.885+0000 7f81313f2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56435edc0060 0x56435edbe4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.885+0000 7f81313f2b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56435edbea00 0x56435edbee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.885+0000 7f81313f2b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435ed17780 con 0x56435edbea00 2023-09-13T04:40:17.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.885+0000 7f81313f2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435ed150a0 con 0x56435edc0060 2023-09-13T04:40:17.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.885+0000 7f81313f2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435ed10990 con 0x56435edbf700 2023-09-13T04:40:17.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.886+0000 7f81177fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edbf700 0x56435edbfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.886+0000 7f81167fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56435edbea00 0x56435edbee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.886+0000 7f81177fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edbf700 0x56435edbfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:35348/0 (socket says 172.21.15.2:35348) 2023-09-13T04:40:17.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.886+0000 7f81177fe700 1 -- 172.21.15.2:0/3223199255 learned_addr learned my addr 172.21.15.2:0/3223199255 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:17.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.886+0000 7f8116ffd700 1 --2- 172.21.15.2:0/3223199255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56435edc0060 0x56435edbe4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.886+0000 7f81177fe700 1 -- 172.21.15.2:0/3223199255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56435edc0060 msgr2=0x56435edbe4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:17.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.886+0000 7f81177fe700 1 --2- 172.21.15.2:0/3223199255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56435edc0060 0x56435edbe4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.886+0000 7f81177fe700 1 -- 172.21.15.2:0/3223199255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56435edbea00 msgr2=0x56435edbee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:17.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.886+0000 7f81177fe700 1 --2- 172.21.15.2:0/3223199255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56435edbea00 0x56435edbee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.886+0000 7f81177fe700 1 -- 172.21.15.2:0/3223199255 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56435eb90630 con 0x56435edbf700 2023-09-13T04:40:17.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.886+0000 7f8116ffd700 1 --2- 172.21.15.2:0/3223199255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56435edc0060 0x56435edbe4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:40:17.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.886+0000 7f81177fe700 1 --2- 172.21.15.2:0/3223199255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edbf700 0x56435edbfb20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8110007630 tx=0x7f8110009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db5e6e1fbba9a6c2 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:17.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.886+0000 7f80fdffb700 1 -- 172.21.15.2:0/3223199255 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811000e070 con 0x56435edbf700 2023-09-13T04:40:17.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.887+0000 7f80fdffb700 1 -- 172.21.15.2:0/3223199255 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f811000f040 con 0x56435edbf700 2023-09-13T04:40:17.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.887+0000 7f80fdffb700 1 -- 172.21.15.2:0/3223199255 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8110015b70 con 0x56435edbf700 2023-09-13T04:40:17.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.887+0000 7f811c9fd700 1 --2- 172.21.15.2:0/750665752 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f81000a06c0 0x7f81000a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.887+0000 7f811c9fd700 1 -- 172.21.15.2:0/750665752 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8108014070 con 0x7f81180ff530 2023-09-13T04:40:17.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.887+0000 7f8116ffd700 1 --2- 172.21.15.2:0/750665752 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f81000a06c0 0x7f81000a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.887+0000 7f81313f2b80 1 -- 172.21.15.2:0/3223199255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edbf700 msgr2=0x56435edbfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.887+0000 7f81313f2b80 1 --2- 172.21.15.2:0/3223199255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edbf700 0x56435edbfb20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8110007630 tx=0x7f8110009920 comp rx=0 tx=0).stop 2023-09-13T04:40:17.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.888+0000 7f81313f2b80 1 -- 172.21.15.2:0/3223199255 shutdown_connections 2023-09-13T04:40:17.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.888+0000 7f81313f2b80 1 --2- 172.21.15.2:0/3223199255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56435edbea00 0x56435edbee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.888+0000 7f81313f2b80 1 --2- 172.21.15.2:0/3223199255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edbf700 0x56435edbfb20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.888+0000 7f81313f2b80 1 --2- 172.21.15.2:0/3223199255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56435edc0060 0x56435edbe4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.888+0000 7f81313f2b80 1 -- 172.21.15.2:0/3223199255 >> 172.21.15.2:0/3223199255 conn(0x56435ed1cc80 msgr2=0x56435edb8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:17.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.888+0000 7f81313f2b80 1 -- 172.21.15.2:0/3223199255 shutdown_connections 2023-09-13T04:40:17.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.888+0000 7f81313f2b80 1 -- 172.21.15.2:0/3223199255 wait complete. 2023-09-13T04:40:17.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.888+0000 7f8116ffd700 1 --2- 172.21.15.2:0/750665752 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f81000a06c0 0x7f81000a2b80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f81080994d0 tx=0x7f8108009790 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:17.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.888+0000 7f81313f2b80 1 -- start start 2023-09-13T04:40:17.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.889+0000 7f81313f2b80 1 Processor -- start 2023-09-13T04:40:17.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.889+0000 7f81313f2b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56435edbea00 0x56435edd8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.890+0000 7f81313f2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edbf700 0x56435edd9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.890+0000 7f81313f2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56435edc0060 0x56435ede04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.890+0000 7f81313f2b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435ecaebb0 con 0x56435edbea00 2023-09-13T04:40:17.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.890+0000 7f81313f2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435ecae8b0 con 0x56435edc0060 2023-09-13T04:40:17.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.890+0000 7f81313f2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435eb92920 con 0x56435edbf700 2023-09-13T04:40:17.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.890+0000 7f81313f2b80 10 client.0 _ll_register_callbacks cb 0x56435ecac680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:17.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.890+0000 7f81177fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56435edbea00 0x56435edd8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.890+0000 7f81177fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56435edbea00 0x56435edd8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47722/0 (socket says 172.21.15.2:47722) 2023-09-13T04:40:17.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.890+0000 7f81177fe700 1 -- 172.21.15.2:0/3329984297 learned_addr learned my addr 172.21.15.2:0/3329984297 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:17.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.890+0000 7f81167fc700 1 --2- 172.21.15.2:0/3329984297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edbf700 0x56435edd9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.890+0000 7f81177fe700 1 --2- 172.21.15.2:0/3329984297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56435edc0060 0x56435ede04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.890+0000 7f81167fc700 1 -- 172.21.15.2:0/3329984297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56435edc0060 msgr2=0x56435ede04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:17.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.890+0000 7f81167fc700 1 --2- 172.21.15.2:0/3329984297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56435edc0060 0x56435ede04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.890+0000 7f81167fc700 1 -- 172.21.15.2:0/3329984297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56435edbea00 msgr2=0x56435edd8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:17.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.890+0000 7f81167fc700 1 --2- 172.21.15.2:0/3329984297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56435edbea00 0x56435edd8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.890+0000 7f81167fc700 1 -- 172.21.15.2:0/3329984297 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81100092e0 con 0x56435edbf700 2023-09-13T04:40:17.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.890+0000 7f81177fe700 1 --2- 172.21.15.2:0/3329984297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56435edc0060 0x56435ede04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:40:17.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.891+0000 7f81167fc700 1 --2- 172.21.15.2:0/3329984297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edbf700 0x56435edd9480 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f810c000c00 tx=0x7f810c009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a1b01ec5dcaf04b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:17.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.891+0000 7f80f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:40:17.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.891+0000 7f80f6ffd700 1 -- 172.21.15.2:0/3329984297 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c01d030 con 0x56435edbf700 2023-09-13T04:40:17.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.891+0000 7f80f6ffd700 1 -- 172.21.15.2:0/3329984297 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f810c004700 con 0x56435edbf700 2023-09-13T04:40:17.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.891+0000 7f81313f2b80 1 -- 172.21.15.2:0/3329984297 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56435edb1670 con 0x56435edbf700 2023-09-13T04:40:17.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.891+0000 7f80f6ffd700 1 -- 172.21.15.2:0/3329984297 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f810c0056f0 con 0x56435edbf700 2023-09-13T04:40:17.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.891+0000 7f81313f2b80 10 client.25510 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:17.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.891+0000 7f81313f2b80 1 -- 172.21.15.2:0/3329984297 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56435ed287a0 con 0x56435edbf700 2023-09-13T04:40:17.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.892+0000 7f80f6ffd700 1 -- 172.21.15.2:0/3329984297 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56435edb1670 con 0x56435edbf700 2023-09-13T04:40:17.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.892+0000 7f80f6ffd700 1 -- 172.21.15.2:0/3329984297 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f810c00fa10 con 0x56435edbf700 2023-09-13T04:40:17.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.892+0000 7f81313f2b80 10 client.25510 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:17.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.892+0000 7f81313f2b80 10 client.25510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:40:17.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.892+0000 7f81313f2b80 20 client.25510 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:17.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.892+0000 7f81313f2b80 1 --2- 172.21.15.2:0/3329984297 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56435ede4d00 0x56435ede7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.892+0000 7f81313f2b80 4 client.25510 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:40:17.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.892+0000 7f81313f2b80 1 -- 172.21.15.2:0/3329984297 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56435ed09970 con 0x56435ede4d00 2023-09-13T04:40:17.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.892+0000 7f81177fe700 1 --2- 172.21.15.2:0/3329984297 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56435ede4d00 0x56435ede7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.893+0000 7f80f6ffd700 10 client.25510 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:17.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.893+0000 7f81177fe700 1 --2- 172.21.15.2:0/3329984297 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56435ede4d00 0x56435ede7120 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c77f6601dc0f840 server_cookie=6912cdc07ebaaf2b in_seq=0 out_seq=0 2023-09-13T04:40:17.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.894+0000 7f80f6ffd700 1 -- 172.21.15.2:0/3329984297 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56435ed09970 con 0x56435ede4d00 2023-09-13T04:40:17.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.894+0000 7f80f6ffd700 10 client.25510 handle_command_reply: tid=0 2023-09-13T04:40:17.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.894+0000 7f81313f2b80 1 client.25510 shutdown 2023-09-13T04:40:17.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.894+0000 7f81313f2b80 1 -- 172.21.15.2:0/3329984297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edbf700 msgr2=0x56435edd9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.894+0000 7f81313f2b80 1 --2- 172.21.15.2:0/3329984297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edbf700 0x56435edd9480 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f810c000c00 tx=0x7f810c009570 comp rx=0 tx=0).stop 2023-09-13T04:40:17.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.894+0000 7f81313f2b80 1 -- 172.21.15.2:0/3329984297 shutdown_connections 2023-09-13T04:40:17.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.894+0000 7f81313f2b80 1 --2- 172.21.15.2:0/3329984297 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56435ede4d00 0x56435ede7120 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.894+0000 7f81313f2b80 1 --2- 172.21.15.2:0/3329984297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56435edc0060 0x56435ede04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.894+0000 7f81313f2b80 1 --2- 172.21.15.2:0/3329984297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56435edbea00 0x56435edd8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.894+0000 7f81313f2b80 1 --2- 172.21.15.2:0/3329984297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edbf700 0x56435edd9480 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.894+0000 7f80f6ffd700 0 client.25510 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:17.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.894+0000 7f81313f2b80 1 -- 172.21.15.2:0/3329984297 >> 172.21.15.2:0/3329984297 conn(0x56435ed1cc80 msgr2=0x56435edb37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:17.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.894+0000 7f81313f2b80 1 -- 172.21.15.2:0/3329984297 shutdown_connections 2023-09-13T04:40:17.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.894+0000 7f81313f2b80 1 -- 172.21.15.2:0/3329984297 wait complete. 2023-09-13T04:40:17.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.895+0000 7f81313f2b80 20 client.25510 trim_cache size 0 max 16384 2023-09-13T04:40:17.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.921+0000 7f81313f2b80 1 Processor -- start 2023-09-13T04:40:17.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.921+0000 7f81313f2b80 1 -- start start 2023-09-13T04:40:17.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.921+0000 7f81313f2b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56435edbea00 0x56435edbe4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.921+0000 7f81313f2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56435edbf700 0x56435ede0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.921+0000 7f81313f2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edc0060 0x56435edd8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.921+0000 7f81313f2b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435edbafd0 con 0x56435edbea00 2023-09-13T04:40:17.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.921+0000 7f81313f2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435edb9480 con 0x56435edbf700 2023-09-13T04:40:17.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.921+0000 7f81313f2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435edb58e0 con 0x56435edc0060 2023-09-13T04:40:17.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.921+0000 7f81177fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56435edbea00 0x56435edbe4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.921+0000 7f81177fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56435edbea00 0x56435edbe4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47734/0 (socket says 172.21.15.2:47734) 2023-09-13T04:40:17.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.922+0000 7f81167fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56435edbf700 0x56435ede0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.922+0000 7f81177fe700 1 -- 172.21.15.2:0/2019263222 learned_addr learned my addr 172.21.15.2:0/2019263222 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:17.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.922+0000 7f81177fe700 1 --2- 172.21.15.2:0/2019263222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edc0060 0x56435edd8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.922+0000 7f81167fc700 1 -- 172.21.15.2:0/2019263222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edc0060 msgr2=0x56435edd8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:17.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.922+0000 7f81167fc700 1 --2- 172.21.15.2:0/2019263222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edc0060 0x56435edd8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.922+0000 7f81167fc700 1 -- 172.21.15.2:0/2019263222 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56435edbea00 msgr2=0x56435edbe4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:17.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.922+0000 7f81167fc700 1 --2- 172.21.15.2:0/2019263222 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56435edbea00 0x56435edbe4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.922+0000 7f81167fc700 1 -- 172.21.15.2:0/2019263222 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56435edebd50 con 0x56435edbf700 2023-09-13T04:40:17.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.922+0000 7f81177fe700 1 --2- 172.21.15.2:0/2019263222 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56435edbea00 0x56435edbe4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:17.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.922+0000 7f81167fc700 1 --2- 172.21.15.2:0/2019263222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56435edbf700 0x56435ede0480 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f810c00fd70 tx=0x7f810c005cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e202a16fad6272b2 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:17.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.923+0000 7f80fe7fc700 1 -- 172.21.15.2:0/2019263222 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c01d070 con 0x56435edbf700 2023-09-13T04:40:17.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.923+0000 7f80fe7fc700 1 -- 172.21.15.2:0/2019263222 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f810c03b070 con 0x56435edbf700 2023-09-13T04:40:17.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.923+0000 7f80fe7fc700 1 -- 172.21.15.2:0/2019263222 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c00f790 con 0x56435edbf700 2023-09-13T04:40:17.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.924+0000 7f81313f2b80 1 -- 172.21.15.2:0/2019263222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56435edbf700 msgr2=0x56435ede0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.924+0000 7f81313f2b80 1 --2- 172.21.15.2:0/2019263222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56435edbf700 0x56435ede0480 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f810c00fd70 tx=0x7f810c005cd0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.924+0000 7f81313f2b80 1 -- 172.21.15.2:0/2019263222 shutdown_connections 2023-09-13T04:40:17.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.924+0000 7f81313f2b80 1 --2- 172.21.15.2:0/2019263222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edc0060 0x56435edd8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.924+0000 7f81313f2b80 1 --2- 172.21.15.2:0/2019263222 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56435edbea00 0x56435edbe4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.924+0000 7f81313f2b80 1 --2- 172.21.15.2:0/2019263222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56435edbf700 0x56435ede0480 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.924+0000 7f81313f2b80 1 -- 172.21.15.2:0/2019263222 >> 172.21.15.2:0/2019263222 conn(0x56435ed1cc80 msgr2=0x56435edc2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:17.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.924+0000 7f81313f2b80 1 -- 172.21.15.2:0/2019263222 shutdown_connections 2023-09-13T04:40:17.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.924+0000 7f81313f2b80 1 -- 172.21.15.2:0/2019263222 wait complete. 2023-09-13T04:40:17.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.924+0000 7f81313f2b80 1 -- start start 2023-09-13T04:40:17.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.925+0000 7f81313f2b80 1 Processor -- start 2023-09-13T04:40:17.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.925+0000 7f81313f2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edbea00 0x56435eddc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.925+0000 7f81313f2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56435edbf700 0x56435eddc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.925+0000 7f81313f2b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56435edc0060 0x56435eddf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.925+0000 7f81313f2b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435eddfaf0 con 0x56435edc0060 2023-09-13T04:40:17.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.925+0000 7f81313f2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435eddfc60 con 0x56435edbf700 2023-09-13T04:40:17.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.925+0000 7f81313f2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435eddfe20 con 0x56435edbea00 2023-09-13T04:40:17.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.926+0000 7f81313f2b80 10 client.0 _ll_register_callbacks cb 0x56435ecac680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:17.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.926+0000 7f81177fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edbea00 0x56435eddc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.926+0000 7f81177fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edbea00 0x56435eddc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:35374/0 (socket says 172.21.15.2:35374) 2023-09-13T04:40:17.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.926+0000 7f81177fe700 1 -- 172.21.15.2:0/1980536074 learned_addr learned my addr 172.21.15.2:0/1980536074 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:17.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.926+0000 7f81177fe700 1 --2- 172.21.15.2:0/1980536074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56435edc0060 0x56435eddf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.927+0000 7f81177fe700 1 -- 172.21.15.2:0/1980536074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56435edbf700 msgr2=0x56435eddc590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:40:17.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.927+0000 7f81177fe700 1 --2- 172.21.15.2:0/1980536074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56435edbf700 0x56435eddc590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.927+0000 7f81177fe700 1 -- 172.21.15.2:0/1980536074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56435edc0060 msgr2=0x56435eddf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:17.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.927+0000 7f81177fe700 1 --2- 172.21.15.2:0/1980536074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56435edc0060 0x56435eddf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.927+0000 7f81177fe700 1 -- 172.21.15.2:0/1980536074 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56435eb90630 con 0x56435edbea00 2023-09-13T04:40:17.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.927+0000 7f81177fe700 1 --2- 172.21.15.2:0/1980536074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edbea00 0x56435eddc050 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x56435ede7a30 tx=0x7f8110025040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f95906d922411947 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:17.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.927+0000 7f80f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:40:17.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.928+0000 7f80f6ffd700 1 -- 172.21.15.2:0/1980536074 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811002d040 con 0x56435edbea00 2023-09-13T04:40:17.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.928+0000 7f80f6ffd700 1 -- 172.21.15.2:0/1980536074 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f81100167a0 con 0x56435edbea00 2023-09-13T04:40:17.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.928+0000 7f81313f2b80 1 -- 172.21.15.2:0/1980536074 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f811000f040 con 0x56435edbea00 2023-09-13T04:40:17.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.928+0000 7f80f6ffd700 1 -- 172.21.15.2:0/1980536074 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f811000e040 con 0x56435edbea00 2023-09-13T04:40:17.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.928+0000 7f81313f2b80 10 client.25516 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:17.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.928+0000 7f81313f2b80 1 -- 172.21.15.2:0/1980536074 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56435ed287a0 con 0x56435edbea00 2023-09-13T04:40:17.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.929+0000 7f80f6ffd700 1 -- 172.21.15.2:0/1980536074 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f811000f040 con 0x56435edbea00 2023-09-13T04:40:17.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.929+0000 7f80f6ffd700 1 -- 172.21.15.2:0/1980536074 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f811001c070 con 0x56435edbea00 2023-09-13T04:40:17.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.929+0000 7f81313f2b80 10 client.25516 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:17.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.929+0000 7f81313f2b80 10 client.25516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:40:17.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.929+0000 7f81313f2b80 20 client.25516 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:17.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.929+0000 7f81313f2b80 1 --2- 172.21.15.2:0/1980536074 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56435ee39880 0x56435ee3bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:17.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.929+0000 7f81313f2b80 4 client.25516 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-09-13T04:40:17.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.929+0000 7f81313f2b80 1 -- 172.21.15.2:0/1980536074 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56435ecca050 con 0x56435ee39880 2023-09-13T04:40:17.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.929+0000 7f81167fc700 1 --2- 172.21.15.2:0/1980536074 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56435ee39880 0x56435ee3bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:17.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.930+0000 7f81167fc700 1 --2- 172.21.15.2:0/1980536074 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56435ee39880 0x56435ee3bca0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1ec23ad248a631 server_cookie=f8f94f5c0bb0d61d in_seq=0 out_seq=0 2023-09-13T04:40:17.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.930+0000 7f80f6ffd700 10 client.25516 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:17.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.930+0000 7f80f6ffd700 1 -- 172.21.15.2:0/1980536074 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56435ecca050 con 0x56435ee39880 2023-09-13T04:40:17.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.930+0000 7f80f6ffd700 10 client.25516 handle_command_reply: tid=0 2023-09-13T04:40:17.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.930+0000 7f81313f2b80 1 client.25516 shutdown 2023-09-13T04:40:17.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.930+0000 7f81313f2b80 1 -- 172.21.15.2:0/1980536074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edbea00 msgr2=0x56435eddc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.930+0000 7f81313f2b80 1 --2- 172.21.15.2:0/1980536074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edbea00 0x56435eddc050 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x56435ede7a30 tx=0x7f8110025040 comp rx=0 tx=0).stop 2023-09-13T04:40:17.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.930+0000 7f81313f2b80 1 -- 172.21.15.2:0/1980536074 shutdown_connections 2023-09-13T04:40:17.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.930+0000 7f81313f2b80 1 --2- 172.21.15.2:0/1980536074 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56435ee39880 0x56435ee3bca0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.930+0000 7f81313f2b80 1 --2- 172.21.15.2:0/1980536074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56435edc0060 0x56435eddf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.930+0000 7f81313f2b80 1 --2- 172.21.15.2:0/1980536074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56435edbea00 0x56435eddc050 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.930+0000 7f80f6ffd700 0 client.25516 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:17.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.930+0000 7f81313f2b80 1 --2- 172.21.15.2:0/1980536074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56435edbf700 0x56435eddc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.930+0000 7f81313f2b80 1 -- 172.21.15.2:0/1980536074 >> 172.21.15.2:0/1980536074 conn(0x56435ed1cc80 msgr2=0x56435edd8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:17.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.931+0000 7f81313f2b80 1 -- 172.21.15.2:0/1980536074 shutdown_connections 2023-09-13T04:40:17.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.931+0000 7f81313f2b80 1 -- 172.21.15.2:0/1980536074 wait complete. 2023-09-13T04:40:17.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.931+0000 7f81313f2b80 20 client.25516 trim_cache size 0 max 16384 2023-09-13T04:40:17.944 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:40:17.944 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-13T04:40:17.945 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-09-13T04:40:17.945 INFO:teuthology.orchestra.run.smithi002.stdout: "1bb91399-a885-4a35-91ee-1676b6635e3d": { 2023-09-13T04:40:17.945 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-09-13T04:40:17.945 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "1bb91399-a885-4a35-91ee-1676b6635e3d", 2023-09-13T04:40:17.945 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-09-13T04:40:17.946 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:40:17.946 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:40:17.946 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:40:17.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.931+0000 7f3fa8dff700 1 -- 172.21.15.2:0/1649555992 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f3fa419f250 con 0x7f3fa4073710 2023-09-13T04:40:17.947 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:17 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2903761698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:40:17.947 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:17 smithi156 ceph-mon[133675]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:40:17.948 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:17 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2380281191' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:40:17.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.931+0000 7f811d9ff700 1 -- 172.21.15.2:0/750665752 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f81000a06c0 msgr2=0x7f81000a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.931+0000 7f811d9ff700 1 --2- 172.21.15.2:0/750665752 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f81000a06c0 0x7f81000a2b80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f81080994d0 tx=0x7f8108009790 comp rx=0 tx=0).stop 2023-09-13T04:40:17.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.931+0000 7f811d9ff700 1 -- 172.21.15.2:0/750665752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81180ff530 msgr2=0x7f8118198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:17.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.931+0000 7f811d9ff700 1 --2- 172.21.15.2:0/750665752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81180ff530 0x7f8118198c30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f810800e9f0 tx=0x7f810800ed00 comp rx=0 tx=0).stop 2023-09-13T04:40:17.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.931+0000 7f811d9ff700 1 -- 172.21.15.2:0/750665752 shutdown_connections 2023-09-13T04:40:17.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.931+0000 7f811d9ff700 1 --2- 172.21.15.2:0/750665752 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f81000a06c0 0x7f81000a2b80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.932+0000 7f811d9ff700 1 --2- 172.21.15.2:0/750665752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8118100850 0x7f81181a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.932+0000 7f811d9ff700 1 --2- 172.21.15.2:0/750665752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81180ff530 0x7f8118198c30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.932+0000 7f811d9ff700 1 --2- 172.21.15.2:0/750665752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81180ffe90 0x7f8118199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:17.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.932+0000 7f811d9ff700 1 -- 172.21.15.2:0/750665752 >> 172.21.15.2:0/750665752 conn(0x7f81180fb130 msgr2=0x7f81180fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:17.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.932+0000 7f811d9ff700 1 -- 172.21.15.2:0/750665752 shutdown_connections 2023-09-13T04:40:17.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:17.932+0000 7f811d9ff700 1 -- 172.21.15.2:0/750665752 wait complete. 2023-09-13T04:40:17.951 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:17 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2903761698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:40:17.951 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:17 smithi002 ceph-mon[97651]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:40:17.952 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:17 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2380281191' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:40:17.952 INFO:tasks.cephfs.filesystem:scrub status for tag:1bb91399-a885-4a35-91ee-1676b6635e3d - {'path': '/', 'tag': '1bb91399-a885-4a35-91ee-1676b6635e3d', 'options': 'recursive,force'} 2023-09-13T04:40:18.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:17 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2903761698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:40:18.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:17 smithi149 ceph-mon[133315]: from='mgr.14148 172.21.15.2:0/840412963' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:40:18.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:17 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2380281191' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:40:18.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.683+0000 7f3f937fe700 1 -- 172.21.15.2:0/1649555992 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mgrmap(e 17) v1 ==== 135275+0+0 (secure 0 0 0) 0x7f3f940a7fb0 con 0x7f3fa4073710 2023-09-13T04:40:18.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.685+0000 7f3f937fe700 1 -- 172.21.15.2:0/1649555992 <== mon.1 v2:172.21.15.156:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v17) v1 ==== 92+0+0 (secure 0 0 0) 0x7f3f9400fd30 con 0x7f3fa4073710 2023-09-13T04:40:18.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.689+0000 7f3fa8dff700 1 -- 172.21.15.2:0/1649555992 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f3f8c0a0710 msgr2=0x7f3f8c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:18.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.689+0000 7f3fa8dff700 1 --2- 172.21.15.2:0/1649555992 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f3f8c0a0710 0x7f3f8c0a2bd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4074a10 tx=0x7f3f98007400 comp rx=0 tx=0).stop 2023-09-13T04:40:18.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.689+0000 7f3fa8dff700 1 -- 172.21.15.2:0/1649555992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fa4073710 msgr2=0x7f3fa419d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:18.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.689+0000 7f3fa8dff700 1 --2- 172.21.15.2:0/1649555992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fa4073710 0x7f3fa419d0f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400b680 tx=0x7f3f940057b0 comp rx=0 tx=0).stop 2023-09-13T04:40:18.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.689+0000 7f3fa8dff700 1 -- 172.21.15.2:0/1649555992 shutdown_connections 2023-09-13T04:40:18.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.689+0000 7f3fa8dff700 1 --2- 172.21.15.2:0/1649555992 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f3f8c0a0710 0x7f3f8c0a2bd0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:18.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.689+0000 7f3fa8dff700 1 --2- 172.21.15.2:0/1649555992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fa410d2d0 0x7f3fa419e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:18.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.689+0000 7f3fa8dff700 1 --2- 172.21.15.2:0/1649555992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fa4073710 0x7f3fa419d0f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:18.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.689+0000 7f3fa8dff700 1 --2- 172.21.15.2:0/1649555992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fa4074cf0 0x7f3fa419d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:18.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.689+0000 7f3fa8dff700 1 -- 172.21.15.2:0/1649555992 >> 172.21.15.2:0/1649555992 conn(0x7f3fa40fc230 msgr2=0x7f3fa40fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:18.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.689+0000 7f3fa8dff700 1 -- 172.21.15.2:0/1649555992 shutdown_connections 2023-09-13T04:40:18.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.689+0000 7f3fa8dff700 1 -- 172.21.15.2:0/1649555992 wait complete. 2023-09-13T04:40:18.710 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-09-13T04:40:18.936 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:18 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: ignoring --setuser ceph since I am not root 2023-09-13T04:40:18.937 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:18 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: ignoring --setgroup ceph since I am not root 2023-09-13T04:40:18.937 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:18 smithi156 ceph-mgr[135821]: -- 172.21.15.156:0/4168819569 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558f600971e0 con 0x558f6006cc00 2023-09-13T04:40:18.938 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:18 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:18.858+0000 7f0f65953200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-13T04:40:18.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:18 smithi156 ceph-mon[133675]: pgmap v349: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-09-13T04:40:18.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:18 smithi156 ceph-mon[133675]: scrub queued for path: / 2023-09-13T04:40:18.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:18 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:40:18.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:18 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:40:18.940 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:18 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:40:18.940 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:18 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1649555992' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-13T04:40:18.940 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:18 smithi156 ceph-mon[133675]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-13T04:40:18.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.936+0000 7f94649ff700 1 -- 172.21.15.2:0/2914913494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9460100880 msgr2=0x7f9460100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:18.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.936+0000 7f94649ff700 1 --2- 172.21.15.2:0/2914913494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9460100880 0x7f9460100d00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f945800b0a0 tx=0x7f945800b3b0 comp rx=0 tx=0).stop 2023-09-13T04:40:18.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.936+0000 7f94649ff700 1 -- 172.21.15.2:0/2914913494 shutdown_connections 2023-09-13T04:40:18.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.936+0000 7f94649ff700 1 --2- 172.21.15.2:0/2914913494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9460100880 0x7f9460100d00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:18.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.936+0000 7f94649ff700 1 --2- 172.21.15.2:0/2914913494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94600ff560 0x7f94600ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:18.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.936+0000 7f94649ff700 1 --2- 172.21.15.2:0/2914913494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94600ffec0 0x7f9460100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:18.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.936+0000 7f94649ff700 1 -- 172.21.15.2:0/2914913494 >> 172.21.15.2:0/2914913494 conn(0x7f94600fb140 msgr2=0x7f94600fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:18.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.936+0000 7f94649ff700 1 -- 172.21.15.2:0/2914913494 shutdown_connections 2023-09-13T04:40:18.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.937+0000 7f94649ff700 1 -- 172.21.15.2:0/2914913494 wait complete. 2023-09-13T04:40:18.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.938+0000 7f94649ff700 1 Processor -- start 2023-09-13T04:40:18.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.938+0000 7f94649ff700 1 -- start start 2023-09-13T04:40:18.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.939+0000 7f94649ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94600ff560 0x7f9460198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:18.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.939+0000 7f94649ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94600ffec0 0x7f9460199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:18.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.939+0000 7f94649ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9460100880 0x7f946019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:18.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.939+0000 7f94649ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946019a810 con 0x7f9460100880 2023-09-13T04:40:18.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.939+0000 7f94649ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946019a980 con 0x7f94600ff560 2023-09-13T04:40:18.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.939+0000 7f94649ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946019ab40 con 0x7f94600ffec0 2023-09-13T04:40:18.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.939+0000 7f945e59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94600ff560 0x7f9460198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:18.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.939+0000 7f945dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94600ffec0 0x7f9460199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:18.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.939+0000 7f945dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94600ffec0 0x7f9460199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:35400/0 (socket says 172.21.15.2:35400) 2023-09-13T04:40:18.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.939+0000 7f945dd9b700 1 -- 172.21.15.2:0/2681062116 learned_addr learned my addr 172.21.15.2:0/2681062116 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:18.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.939+0000 7f945e59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94600ff560 0x7f9460198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39770/0 (socket says 172.21.15.2:39770) 2023-09-13T04:40:18.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.939+0000 7f945ed9d700 1 --2- 172.21.15.2:0/2681062116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9460100880 0x7f946019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:18.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.939+0000 7f945dd9b700 1 -- 172.21.15.2:0/2681062116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94600ff560 msgr2=0x7f9460198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:18.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.939+0000 7f945dd9b700 1 --2- 172.21.15.2:0/2681062116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94600ff560 0x7f9460198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:18.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.939+0000 7f945dd9b700 1 -- 172.21.15.2:0/2681062116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9460100880 msgr2=0x7f946019a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:18.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.939+0000 7f945dd9b700 1 --2- 172.21.15.2:0/2681062116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9460100880 0x7f946019a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:18.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.939+0000 7f945dd9b700 1 -- 172.21.15.2:0/2681062116 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9458009d00 con 0x7f94600ffec0 2023-09-13T04:40:18.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.939+0000 7f945e59c700 1 --2- 172.21.15.2:0/2681062116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94600ff560 0x7f9460198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:18.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.940+0000 7f945dd9b700 1 --2- 172.21.15.2:0/2681062116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94600ffec0 0x7f9460199360 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f945400e9a0 tx=0x7f945400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:18.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.940+0000 7f944f7fe700 1 -- 172.21.15.2:0/2681062116 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454009790 con 0x7f94600ffec0 2023-09-13T04:40:18.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.940+0000 7f944f7fe700 1 -- 172.21.15.2:0/2681062116 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f945400cbd0 con 0x7f94600ffec0 2023-09-13T04:40:18.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.940+0000 7f94649ff700 1 -- 172.21.15.2:0/2681062116 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f946019ae20 con 0x7f94600ffec0 2023-09-13T04:40:18.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.940+0000 7f944f7fe700 1 -- 172.21.15.2:0/2681062116 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454018680 con 0x7f94600ffec0 2023-09-13T04:40:18.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.940+0000 7f94649ff700 1 -- 172.21.15.2:0/2681062116 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94601a15e0 con 0x7f94600ffec0 2023-09-13T04:40:18.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.942+0000 7f94649ff700 1 -- 172.21.15.2:0/2681062116 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f946010a340 con 0x7f94600ffec0 2023-09-13T04:40:18.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.943+0000 7f944f7fe700 1 -- 172.21.15.2:0/2681062116 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 135275+0+0 (secure 0 0 0) 0x7f945400cd40 con 0x7f94600ffec0 2023-09-13T04:40:18.960 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:18 smithi149 ceph-mon[133315]: pgmap v349: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-09-13T04:40:18.960 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:18 smithi149 ceph-mon[133315]: scrub queued for path: / 2023-09-13T04:40:18.960 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:18 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:40:18.961 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:18 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:40:18.961 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:18 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:40:18.961 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:18 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1649555992' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-13T04:40:18.962 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:18 smithi149 ceph-mon[133315]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-13T04:40:18.962 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:18 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: ignoring --setuser ceph since I am not root 2023-09-13T04:40:18.963 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:18 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: ignoring --setgroup ceph since I am not root 2023-09-13T04:40:18.963 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:18 smithi149 ceph-mgr[134792]: -- 172.21.15.149:0/3495094337 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56020c4d91e0 con 0x56020c4aec00 2023-09-13T04:40:18.963 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:18 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:18.861+0000 7fbd0d065200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-13T04:40:18.964 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:18 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:18.958+0000 7fbd0d065200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-13T04:40:18.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.945+0000 7f944f7fe700 1 --2- 172.21.15.2:0/2681062116 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f94480a0710 0x7f94480a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:18.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.945+0000 7f945e59c700 1 -- 172.21.15.2:0/2681062116 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f94480a0710 msgr2=0x7f94480a2bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1461251021 2023-09-13T04:40:18.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.945+0000 7f944f7fe700 1 -- 172.21.15.2:0/2681062116 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9454014070 con 0x7f94600ffec0 2023-09-13T04:40:18.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.945+0000 7f945e59c700 1 --2- 172.21.15.2:0/2681062116 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f94480a0710 0x7f94480a2bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-13T04:40:18.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:18.949+0000 7f944f7fe700 1 -- 172.21.15.2:0/2681062116 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9454051e10 con 0x7f94600ffec0 2023-09-13T04:40:19.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:18 smithi002 ceph-mon[97651]: pgmap v349: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-09-13T04:40:19.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:18 smithi002 ceph-mon[97651]: scrub queued for path: / 2023-09-13T04:40:19.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:18 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:40:19.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:18 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:40:19.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:18 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:40:19.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:18 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1649555992' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-13T04:40:19.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:18 smithi002 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-13T04:40:19.008 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:18 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: ignoring --setuser ceph since I am not root 2023-09-13T04:40:19.008 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:18 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: ignoring --setgroup ceph since I am not root 2023-09-13T04:40:19.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.146+0000 7f945e59c700 1 -- 172.21.15.2:0/2681062116 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f94480a0710 msgr2=0x7f94480a2bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1461251021 2023-09-13T04:40:19.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.146+0000 7f945e59c700 1 --2- 172.21.15.2:0/2681062116 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f94480a0710 0x7f94480a2bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-13T04:40:19.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.198+0000 7f94649ff700 1 -- 172.21.15.2:0/2681062116 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f946004ea90 con 0x7f94600ffec0 2023-09-13T04:40:19.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.205+0000 7f944f7fe700 1 -- 172.21.15.2:0/2681062116 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9454095990 con 0x7f94600ffec0 2023-09-13T04:40:19.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.208+0000 7f94649ff700 1 -- 172.21.15.2:0/2681062116 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f94480a0710 msgr2=0x7f94480a2bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-13T04:40:19.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.208+0000 7f94649ff700 1 --2- 172.21.15.2:0/2681062116 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f94480a0710 0x7f94480a2bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:19.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.208+0000 7f94649ff700 1 -- 172.21.15.2:0/2681062116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94600ffec0 msgr2=0x7f9460199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:19.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.208+0000 7f94649ff700 1 --2- 172.21.15.2:0/2681062116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94600ffec0 0x7f9460199360 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f945400e9a0 tx=0x7f945400ecb0 comp rx=0 tx=0).stop 2023-09-13T04:40:19.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.208+0000 7f94649ff700 1 -- 172.21.15.2:0/2681062116 shutdown_connections 2023-09-13T04:40:19.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.208+0000 7f94649ff700 1 --2- 172.21.15.2:0/2681062116 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f94480a0710 0x7f94480a2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:19.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.208+0000 7f94649ff700 1 --2- 172.21.15.2:0/2681062116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9460100880 0x7f946019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:19.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.208+0000 7f94649ff700 1 --2- 172.21.15.2:0/2681062116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94600ff560 0x7f9460198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:19.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.208+0000 7f94649ff700 1 --2- 172.21.15.2:0/2681062116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94600ffec0 0x7f9460199360 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:19.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.208+0000 7f94649ff700 1 -- 172.21.15.2:0/2681062116 >> 172.21.15.2:0/2681062116 conn(0x7f94600fb140 msgr2=0x7f94600fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:19.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.208+0000 7f94649ff700 1 -- 172.21.15.2:0/2681062116 shutdown_connections 2023-09-13T04:40:19.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.209+0000 7f94649ff700 1 -- 172.21.15.2:0/2681062116 wait complete. 2023-09-13T04:40:19.231 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-09-13T04:40:19.294 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:19 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:19.006+0000 7f10b0f4b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-13T04:40:19.294 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:19 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:19.120+0000 7f10b0f4b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-13T04:40:19.294 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:19 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:19.251+0000 7f10b0f4b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-13T04:40:19.320 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:18 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:18.945+0000 7f0f65953200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-13T04:40:19.320 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:19 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:19.039+0000 7f0f65953200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-13T04:40:19.321 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:19 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:19.139+0000 7f0f65953200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-13T04:40:19.351 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:19 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:19.062+0000 7fbd0d065200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-13T04:40:19.352 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:19 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:19.161+0000 7fbd0d065200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-13T04:40:19.352 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:19 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:19.351+0000 7fbd0d065200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-13T04:40:19.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.444+0000 7ff9e7fff700 1 -- 172.21.15.2:0/544386272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9e81015b0 msgr2=0x7ff9e81039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:19.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.444+0000 7ff9e7fff700 1 --2- 172.21.15.2:0/544386272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9e81015b0 0x7ff9e81039a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff9d80098f0 tx=0x7ff9d8009c00 comp rx=0 tx=0).stop 2023-09-13T04:40:19.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.444+0000 7ff9e7fff700 1 -- 172.21.15.2:0/544386272 shutdown_connections 2023-09-13T04:40:19.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.444+0000 7ff9e7fff700 1 --2- 172.21.15.2:0/544386272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9e8106810 0x7ff9e8108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:19.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.444+0000 7ff9e7fff700 1 --2- 172.21.15.2:0/544386272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9e81015b0 0x7ff9e81039a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:19.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.444+0000 7ff9e7fff700 1 --2- 172.21.15.2:0/544386272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9e8103ee0 0x7ff9e81062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:19.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.444+0000 7ff9e7fff700 1 -- 172.21.15.2:0/544386272 >> 172.21.15.2:0/544386272 conn(0x7ff9e80faf80 msgr2=0x7ff9e80fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:19.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.444+0000 7ff9e7fff700 1 -- 172.21.15.2:0/544386272 shutdown_connections 2023-09-13T04:40:19.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.444+0000 7ff9e7fff700 1 -- 172.21.15.2:0/544386272 wait complete. 2023-09-13T04:40:19.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.446+0000 7ff9e7fff700 1 Processor -- start 2023-09-13T04:40:19.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.446+0000 7ff9e7fff700 1 -- start start 2023-09-13T04:40:19.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9e7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9e81015b0 0x7ff9e8196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:19.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9e7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9e8103ee0 0x7ff9e8196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:19.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9e7fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9e8106810 0x7ff9e819de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:19.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9e7fff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e819e3e0 con 0x7ff9e81015b0 2023-09-13T04:40:19.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9e7fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e819e520 con 0x7ff9e8106810 2023-09-13T04:40:19.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9e7fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e819e6b0 con 0x7ff9e8103ee0 2023-09-13T04:40:19.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9e67fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9e8103ee0 0x7ff9e8196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:19.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9e77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9e8106810 0x7ff9e819de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:19.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9e6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9e81015b0 0x7ff9e8196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:19.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9e77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9e8106810 0x7ff9e819de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39786/0 (socket says 172.21.15.2:39786) 2023-09-13T04:40:19.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9e67fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9e8103ee0 0x7ff9e8196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:35418/0 (socket says 172.21.15.2:35418) 2023-09-13T04:40:19.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9e77fe700 1 -- 172.21.15.2:0/1449736973 learned_addr learned my addr 172.21.15.2:0/1449736973 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:19.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9e77fe700 1 -- 172.21.15.2:0/1449736973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9e8103ee0 msgr2=0x7ff9e8196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:19.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9e77fe700 1 --2- 172.21.15.2:0/1449736973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9e8103ee0 0x7ff9e8196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:19.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9e77fe700 1 -- 172.21.15.2:0/1449736973 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9e81015b0 msgr2=0x7ff9e8196860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:19.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9e77fe700 1 --2- 172.21.15.2:0/1449736973 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9e81015b0 0x7ff9e8196860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:19.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9e77fe700 1 -- 172.21.15.2:0/1449736973 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9dc0095f0 con 0x7ff9e8106810 2023-09-13T04:40:19.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9e67fc700 1 --2- 172.21.15.2:0/1449736973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9e8103ee0 0x7ff9e8196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:19.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9e6ffd700 1 --2- 172.21.15.2:0/1449736973 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9e81015b0 0x7ff9e8196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:19.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9e77fe700 1 --2- 172.21.15.2:0/1449736973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9e8106810 0x7ff9e819de10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000ef30 tx=0x7ff9e000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:19.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9cffff700 1 -- 172.21.15.2:0/1449736973 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0018070 con 0x7ff9e8106810 2023-09-13T04:40:19.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9cffff700 1 -- 172.21.15.2:0/1449736973 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9e000f040 con 0x7ff9e8106810 2023-09-13T04:40:19.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9cffff700 1 -- 172.21.15.2:0/1449736973 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e000cd00 con 0x7ff9e8106810 2023-09-13T04:40:19.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9e7fff700 1 -- 172.21.15.2:0/1449736973 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d8009520 con 0x7ff9e8106810 2023-09-13T04:40:19.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.447+0000 7ff9e7fff700 1 -- 172.21.15.2:0/1449736973 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e80ff6d0 con 0x7ff9e8106810 2023-09-13T04:40:19.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.449+0000 7ff9e7fff700 1 -- 172.21.15.2:0/1449736973 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e80fcb00 con 0x7ff9e8106810 2023-09-13T04:40:19.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.453+0000 7ff9cffff700 1 -- 172.21.15.2:0/1449736973 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 135275+0+0 (secure 0 0 0) 0x7ff9e0003a00 con 0x7ff9e8106810 2023-09-13T04:40:19.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.453+0000 7ff9cffff700 1 --2- 172.21.15.2:0/1449736973 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff9d00a0690 0x7ff9d00a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:19.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.453+0000 7ff9cffff700 1 -- 172.21.15.2:0/1449736973 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff9e00cbaf0 con 0x7ff9e8106810 2023-09-13T04:40:19.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.453+0000 7ff9e6ffd700 1 -- 172.21.15.2:0/1449736973 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff9d00a0690 msgr2=0x7ff9d00a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1461251021 2023-09-13T04:40:19.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.453+0000 7ff9e6ffd700 1 --2- 172.21.15.2:0/1449736973 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff9d00a0690 0x7ff9d00a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-13T04:40:19.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.458+0000 7ff9cffff700 1 -- 172.21.15.2:0/1449736973 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff9e0095320 con 0x7ff9e8106810 2023-09-13T04:40:19.640 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:19 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:19.385+0000 7f10b0f4b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-13T04:40:19.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.655+0000 7ff9e6ffd700 1 -- 172.21.15.2:0/1449736973 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff9d00a0690 msgr2=0x7ff9d00a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1461251021 2023-09-13T04:40:19.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.655+0000 7ff9e6ffd700 1 --2- 172.21.15.2:0/1449736973 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff9d00a0690 0x7ff9d00a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-13T04:40:19.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.699+0000 7ff9e7fff700 1 -- 172.21.15.2:0/1449736973 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7ff9e810c360 con 0x7ff9e8106810 2023-09-13T04:40:19.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.704+0000 7ff9cffff700 1 -- 172.21.15.2:0/1449736973 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7ff9e0098940 con 0x7ff9e8106810 2023-09-13T04:40:19.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.707+0000 7ff9e7fff700 1 -- 172.21.15.2:0/1449736973 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff9d00a0690 msgr2=0x7ff9d00a2b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-13T04:40:19.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.707+0000 7ff9e7fff700 1 --2- 172.21.15.2:0/1449736973 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff9d00a0690 0x7ff9d00a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:19.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.707+0000 7ff9e7fff700 1 -- 172.21.15.2:0/1449736973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9e8106810 msgr2=0x7ff9e819de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:19.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.707+0000 7ff9e7fff700 1 --2- 172.21.15.2:0/1449736973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9e8106810 0x7ff9e819de10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000ef30 tx=0x7ff9e000c3a0 comp rx=0 tx=0).stop 2023-09-13T04:40:19.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.707+0000 7ff9e7fff700 1 -- 172.21.15.2:0/1449736973 shutdown_connections 2023-09-13T04:40:19.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.707+0000 7ff9e7fff700 1 --2- 172.21.15.2:0/1449736973 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7ff9d00a0690 0x7ff9d00a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:19.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.707+0000 7ff9e7fff700 1 --2- 172.21.15.2:0/1449736973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9e8106810 0x7ff9e819de10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:19.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.707+0000 7ff9e7fff700 1 --2- 172.21.15.2:0/1449736973 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9e81015b0 0x7ff9e8196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:19.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.707+0000 7ff9e7fff700 1 --2- 172.21.15.2:0/1449736973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9e8103ee0 0x7ff9e8196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:19.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.707+0000 7ff9e7fff700 1 -- 172.21.15.2:0/1449736973 >> 172.21.15.2:0/1449736973 conn(0x7ff9e80faf80 msgr2=0x7ff9e8102220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:19.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.707+0000 7ff9e7fff700 1 -- 172.21.15.2:0/1449736973 shutdown_connections 2023-09-13T04:40:19.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.707+0000 7ff9e7fff700 1 -- 172.21.15.2:0/1449736973 wait complete. 2023-09-13T04:40:19.728 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M' 2023-09-13T04:40:19.928 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:19 smithi156 ceph-mon[133675]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-09-13T04:40:19.928 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:19 smithi156 ceph-mon[133675]: mgrmap e17: x(active, since 8m), standbys: z, y 2023-09-13T04:40:19.928 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:19 smithi156 ceph-mon[133675]: from='client.? ' entity='client.admin' 2023-09-13T04:40:19.929 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:19 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:19.318+0000 7f0f65953200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-13T04:40:19.929 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:19 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:19.706+0000 7f0f65953200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-13T04:40:19.929 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:19 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:19.838+0000 7f0f65953200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-13T04:40:19.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.947+0000 7f79013ff700 1 -- 172.21.15.2:0/3978488862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78fc100d00 msgr2=0x7f78fc101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:19.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.947+0000 7f79013ff700 1 --2- 172.21.15.2:0/3978488862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78fc100d00 0x7f78fc101120 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f78ec0099c0 tx=0x7f78ec009cd0 comp rx=0 tx=0).stop 2023-09-13T04:40:19.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.947+0000 7f79013ff700 1 -- 172.21.15.2:0/3978488862 shutdown_connections 2023-09-13T04:40:19.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.947+0000 7f79013ff700 1 --2- 172.21.15.2:0/3978488862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78fc102890 0x7f78fc109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:19.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.947+0000 7f79013ff700 1 --2- 172.21.15.2:0/3978488862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78fc100d00 0x7f78fc101120 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:19.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.947+0000 7f79013ff700 1 --2- 172.21.15.2:0/3978488862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78fc101ef0 0x7f78fc102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:19.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.947+0000 7f79013ff700 1 -- 172.21.15.2:0/3978488862 >> 172.21.15.2:0/3978488862 conn(0x7f78fc0fc230 msgr2=0x7f78fc0fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:19.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.947+0000 7f79013ff700 1 -- 172.21.15.2:0/3978488862 shutdown_connections 2023-09-13T04:40:19.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.947+0000 7f79013ff700 1 -- 172.21.15.2:0/3978488862 wait complete. 2023-09-13T04:40:19.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.948+0000 7f79013ff700 1 Processor -- start 2023-09-13T04:40:19.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.949+0000 7f79013ff700 1 -- start start 2023-09-13T04:40:19.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.949+0000 7f79013ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78fc100d00 0x7f78fc0730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:19.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.949+0000 7f79013ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78fc101ef0 0x7f78fc0735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:19.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.949+0000 7f79013ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78fc102890 0x7f78fc073b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:19.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.949+0000 7f79013ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc1a0ae0 con 0x7f78fc101ef0 2023-09-13T04:40:19.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.949+0000 7f79013ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc1a0c20 con 0x7f78fc100d00 2023-09-13T04:40:19.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.949+0000 7f79013ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc1a0d60 con 0x7f78fc102890 2023-09-13T04:40:19.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.949+0000 7f78fa7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78fc101ef0 0x7f78fc0735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:19.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.949+0000 7f78faffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78fc100d00 0x7f78fc0730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:19.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.950+0000 7f78faffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78fc100d00 0x7f78fc0730a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39806/0 (socket says 172.21.15.2:39806) 2023-09-13T04:40:19.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.950+0000 7f78fb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78fc102890 0x7f78fc073b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:19.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.950+0000 7f78faffd700 1 -- 172.21.15.2:0/234510675 learned_addr learned my addr 172.21.15.2:0/234510675 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:19.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.950+0000 7f78faffd700 1 -- 172.21.15.2:0/234510675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78fc102890 msgr2=0x7f78fc073b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:19.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.950+0000 7f78faffd700 1 --2- 172.21.15.2:0/234510675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78fc102890 0x7f78fc073b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:19.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.950+0000 7f78faffd700 1 -- 172.21.15.2:0/234510675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78fc101ef0 msgr2=0x7f78fc0735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:19.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.950+0000 7f78faffd700 1 --2- 172.21.15.2:0/234510675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78fc101ef0 0x7f78fc0735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:19.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.950+0000 7f78faffd700 1 -- 172.21.15.2:0/234510675 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ec0095f0 con 0x7f78fc100d00 2023-09-13T04:40:19.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.950+0000 7f78fb7fe700 1 --2- 172.21.15.2:0/234510675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78fc102890 0x7f78fc073b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:19.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.950+0000 7f78faffd700 1 --2- 172.21.15.2:0/234510675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78fc100d00 0x7f78fc0730a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f78ec009fd0 tx=0x7f78ec00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:19.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.950+0000 7f78e3fff700 1 -- 172.21.15.2:0/234510675 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec01d070 con 0x7f78fc100d00 2023-09-13T04:40:19.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.951+0000 7f78e3fff700 1 -- 172.21.15.2:0/234510675 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78ec005050 con 0x7f78fc100d00 2023-09-13T04:40:19.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.951+0000 7f79013ff700 1 -- 172.21.15.2:0/234510675 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78fc1a0f90 con 0x7f78fc100d00 2023-09-13T04:40:19.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.951+0000 7f78e3fff700 1 -- 172.21.15.2:0/234510675 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec004a70 con 0x7f78fc100d00 2023-09-13T04:40:19.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.951+0000 7f79013ff700 1 -- 172.21.15.2:0/234510675 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78fc1a1300 con 0x7f78fc100d00 2023-09-13T04:40:19.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.952+0000 7f79013ff700 1 -- 172.21.15.2:0/234510675 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78fc04eb00 con 0x7f78fc100d00 2023-09-13T04:40:19.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.955+0000 7f78e3fff700 1 -- 172.21.15.2:0/234510675 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 135275+0+0 (secure 0 0 0) 0x7f78ec052080 con 0x7f78fc100d00 2023-09-13T04:40:19.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.957+0000 7f78e3fff700 1 --2- 172.21.15.2:0/234510675 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f78e40a0690 0x7f78e40a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:19.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.957+0000 7f78fa7fc700 1 -- 172.21.15.2:0/234510675 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f78e40a0690 msgr2=0x7f78e40a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1461251021 2023-09-13T04:40:19.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.957+0000 7f78e3fff700 1 -- 172.21.15.2:0/234510675 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f78ec0ccf90 con 0x7f78fc100d00 2023-09-13T04:40:19.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.957+0000 7f78fa7fc700 1 --2- 172.21.15.2:0/234510675 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f78e40a0690 0x7f78e40a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-13T04:40:19.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:19.957+0000 7f78e3fff700 1 -- 172.21.15.2:0/234510675 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f78ec09a250 con 0x7f78fc100d00 2023-09-13T04:40:19.980 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:19 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:19.757+0000 7fbd0d065200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-13T04:40:19.981 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:19 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:19.891+0000 7fbd0d065200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-13T04:40:19.981 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:19 smithi149 ceph-mon[133315]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-09-13T04:40:19.981 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:19 smithi149 ceph-mon[133315]: mgrmap e17: x(active, since 8m), standbys: z, y 2023-09-13T04:40:19.981 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:19 smithi149 ceph-mon[133315]: from='client.? ' entity='client.admin' 2023-09-13T04:40:20.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:19 smithi002 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-09-13T04:40:20.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:19 smithi002 ceph-mon[97651]: mgrmap e17: x(active, since 8m), standbys: z, y 2023-09-13T04:40:20.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:19 smithi002 ceph-mon[97651]: from='client.? ' entity='client.admin' 2023-09-13T04:40:20.006 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:19 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:19.639+0000 7f10b0f4b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-13T04:40:20.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:20.158+0000 7f78fa7fc700 1 -- 172.21.15.2:0/234510675 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f78e40a0690 msgr2=0x7f78e40a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1461251021 2023-09-13T04:40:20.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:20.158+0000 7f78fa7fc700 1 --2- 172.21.15.2:0/234510675 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f78e40a0690 0x7f78e40a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-13T04:40:20.186 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:19 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:19.926+0000 7f0f65953200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-13T04:40:20.186 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:20 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:20.022+0000 7f0f65953200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-13T04:40:20.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:20.203+0000 7f79013ff700 1 -- 172.21.15.2:0/234510675 --> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f78fc0610f0 con 0x7f78e40a0690 2023-09-13T04:40:20.386 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:19 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:19.980+0000 7fbd0d065200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-13T04:40:20.386 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:20 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:20.075+0000 7fbd0d065200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-13T04:40:20.479 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:20 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:20.166+0000 7f10b0f4b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-13T04:40:20.480 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:20 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:20.346+0000 7f10b0f4b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-13T04:40:20.480 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:20 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:20.479+0000 7f10b0f4b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-13T04:40:20.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:20.559+0000 7f78fa7fc700 1 -- 172.21.15.2:0/234510675 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f78e40a0690 msgr2=0x7f78e40a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1461251021 2023-09-13T04:40:20.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:20.559+0000 7f78fa7fc700 1 --2- 172.21.15.2:0/234510675 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f78e40a0690 0x7f78e40a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-13T04:40:20.703 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:20 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:20.441+0000 7f0f65953200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-13T04:40:20.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:20 smithi002 ceph-mon[97651]: from='client.? ' entity='client.admin' 2023-09-13T04:40:20.756 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:20 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:20.614+0000 7f10b0f4b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-13T04:40:20.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:20 smithi149 ceph-mon[133315]: from='client.? ' entity='client.admin' 2023-09-13T04:40:20.887 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:20 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:20.500+0000 7fbd0d065200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-13T04:40:21.101 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:20 smithi156 ceph-mon[133675]: from='client.? ' entity='client.admin' 2023-09-13T04:40:21.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:21.360+0000 7f78fa7fc700 1 -- 172.21.15.2:0/234510675 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f78e40a0690 msgr2=0x7f78e40a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1461251021 2023-09-13T04:40:21.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:21.360+0000 7f78fa7fc700 1 --2- 172.21.15.2:0/234510675 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f78e40a0690 0x7f78e40a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-13T04:40:21.436 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:21 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:21.100+0000 7f0f65953200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-13T04:40:21.436 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:21 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:21.182+0000 7f0f65953200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-13T04:40:21.505 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:21 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:21.182+0000 7f10b0f4b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-13T04:40:21.636 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:21 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:21.167+0000 7fbd0d065200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-13T04:40:21.636 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:21 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:21.249+0000 7fbd0d065200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-13T04:40:22.223 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:21 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:21.961+0000 7f0f65953200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-13T04:40:22.223 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:22 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:22.050+0000 7f0f65953200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-13T04:40:22.297 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:22 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:22.031+0000 7fbd0d065200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-13T04:40:22.297 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:22 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:22.122+0000 7fbd0d065200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-13T04:40:22.502 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:22 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:22.222+0000 7f0f65953200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-13T04:40:22.505 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:22 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:22.068+0000 7f10b0f4b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-13T04:40:22.505 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:22 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:22.179+0000 7f10b0f4b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-13T04:40:22.580 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:22 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:22.295+0000 7fbd0d065200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-13T04:40:22.883 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:22 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:22.501+0000 7f0f65953200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-13T04:40:22.883 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:22 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:22.732+0000 7f0f65953200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-13T04:40:22.886 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:22 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:22.578+0000 7fbd0d065200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-13T04:40:22.886 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:22 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:22.817+0000 7fbd0d065200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-13T04:40:22.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:22.962+0000 7f78fa7fc700 1 -- 172.21.15.2:0/234510675 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f78e40a0690 msgr2=0x7f78e40a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1461251021 2023-09-13T04:40:22.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:22.962+0000 7f78fa7fc700 1 --2- 172.21.15.2:0/234510675 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f78e40a0690 0x7f78e40a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-09-13T04:40:23.147 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:22 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:22.882+0000 7f0f65953200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-13T04:40:23.231 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:22 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:22.968+0000 7fbd0d065200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-13T04:40:23.436 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:23 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:23.146+0000 7f0f65953200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-13T04:40:23.505 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:23 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:23.180+0000 7f10b0f4b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-13T04:40:23.506 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:23 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:23.309+0000 7f10b0f4b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-13T04:40:23.926 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:23 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:23.544+0000 7f10b0f4b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-13T04:40:24.183 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:23 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:23.858+0000 7f0f65953200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-13T04:40:24.183 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:23 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:23.991+0000 7f0f65953200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-13T04:40:24.211 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:23 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:23.926+0000 7f10b0f4b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-13T04:40:24.276 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:23 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:23.230+0000 7fbd0d065200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-13T04:40:24.276 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:23 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:23.951+0000 7fbd0d065200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-13T04:40:24.277 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:24 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:24.082+0000 7fbd0d065200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-13T04:40:24.436 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:24 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:24.183+0000 7f0f65953200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-13T04:40:24.436 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:24 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:24.296+0000 7f0f65953200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-13T04:40:24.505 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:24 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:24.211+0000 7f10b0f4b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-13T04:40:24.505 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:24 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:24.414+0000 7f10b0f4b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-13T04:40:24.608 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:24 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:24.275+0000 7fbd0d065200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-13T04:40:24.609 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:24 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:24.389+0000 7fbd0d065200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-13T04:40:24.886 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:24 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:24.607+0000 7fbd0d065200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-13T04:40:24.936 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:24 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:24.512+0000 7f0f65953200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-13T04:40:25.255 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:24 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:24.773+0000 7f10b0f4b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-13T04:40:25.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:25.456+0000 7f78e3fff700 1 -- 172.21.15.2:0/234510675 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mgrmap(e 18) v1 ==== 135275+0+0 (secure 0 0 0) 0x7f78ec0c9b50 con 0x7f78fc100d00 2023-09-13T04:40:25.686 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:40:25 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z[135797]: 2023-09-13T04:40:25.379+0000 7f0f65953200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-13T04:40:25.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:25 smithi156 ceph-mon[133675]: Standby manager daemon z restarted 2023-09-13T04:40:25.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:25 smithi156 ceph-mon[133675]: Standby manager daemon z started 2023-09-13T04:40:25.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:25 smithi156 ceph-mon[133675]: from='mgr.? 172.21.15.156:0/2264331259' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-13T04:40:25.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:25 smithi156 ceph-mon[133675]: from='mgr.? 172.21.15.156:0/2264331259' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-13T04:40:25.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:25 smithi156 ceph-mon[133675]: from='mgr.? 172.21.15.156:0/2264331259' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-13T04:40:25.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:25 smithi156 ceph-mon[133675]: from='mgr.? 172.21.15.156:0/2264331259' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-13T04:40:25.750 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:25 smithi002 ceph-mon[97651]: Standby manager daemon z restarted 2023-09-13T04:40:25.750 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:25 smithi002 ceph-mon[97651]: Standby manager daemon z started 2023-09-13T04:40:25.750 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:25 smithi002 ceph-mon[97651]: from='mgr.? 172.21.15.156:0/2264331259' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-13T04:40:25.751 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:25 smithi002 ceph-mon[97651]: from='mgr.? 172.21.15.156:0/2264331259' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-13T04:40:25.751 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:25 smithi002 ceph-mon[97651]: from='mgr.? 172.21.15.156:0/2264331259' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-13T04:40:25.751 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:25 smithi002 ceph-mon[97651]: from='mgr.? 172.21.15.156:0/2264331259' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-13T04:40:25.886 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:40:25 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y[134768]: 2023-09-13T04:40:25.485+0000 7fbd0d065200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-13T04:40:25.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:25 smithi149 ceph-mon[133315]: Standby manager daemon z restarted 2023-09-13T04:40:25.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:25 smithi149 ceph-mon[133315]: Standby manager daemon z started 2023-09-13T04:40:25.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:25 smithi149 ceph-mon[133315]: from='mgr.? 172.21.15.156:0/2264331259' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-13T04:40:25.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:25 smithi149 ceph-mon[133315]: from='mgr.? 172.21.15.156:0/2264331259' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-13T04:40:25.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:25 smithi149 ceph-mon[133315]: from='mgr.? 172.21.15.156:0/2264331259' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-13T04:40:25.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:25 smithi149 ceph-mon[133315]: from='mgr.? 172.21.15.156:0/2264331259' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-13T04:40:26.005 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:25 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:25.749+0000 7f10b0f4b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-13T04:40:26.006 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:25 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:25.936+0000 7f10b0f4b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-13T04:40:26.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:26.165+0000 7f78fa7fc700 1 -- 172.21.15.2:0/234510675 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f78e40a0690 msgr2=0x7f78e40a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1461251021 2023-09-13T04:40:26.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:26.165+0000 7f78fa7fc700 1 --2- 172.21.15.2:0/234510675 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f78e40a0690 0x7f78e40a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-09-13T04:40:26.459 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:26 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:26.196+0000 7f10b0f4b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-13T04:40:26.459 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:26 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:26.352+0000 7f10b0f4b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-13T04:40:26.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:26.473+0000 7f78e3fff700 1 -- 172.21.15.2:0/234510675 <== mon.2 v2:172.21.15.149:3300/0 8 ==== mgrmap(e 19) v1 ==== 135275+0+0 (secure 0 0 0) 0x7f78ec0ca780 con 0x7f78fc100d00 2023-09-13T04:40:26.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:26 smithi002 ceph-mon[97651]: mgrmap e18: x(active, since 9m), standbys: y, z 2023-09-13T04:40:26.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:26 smithi002 ceph-mon[97651]: Standby manager daemon y restarted 2023-09-13T04:40:26.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:26 smithi002 ceph-mon[97651]: Standby manager daemon y started 2023-09-13T04:40:26.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:26 smithi002 ceph-mon[97651]: from='mgr.? 172.21.15.149:0/690810681' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-13T04:40:26.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:26 smithi002 ceph-mon[97651]: from='mgr.? 172.21.15.149:0/690810681' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-13T04:40:26.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:26 smithi002 ceph-mon[97651]: from='mgr.? 172.21.15.149:0/690810681' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-13T04:40:26.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:26 smithi002 ceph-mon[97651]: from='mgr.? 172.21.15.149:0/690810681' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-13T04:40:26.757 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:26 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:26.646+0000 7f10b0f4b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-13T04:40:26.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:26 smithi149 ceph-mon[133315]: mgrmap e18: x(active, since 9m), standbys: y, z 2023-09-13T04:40:26.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:26 smithi149 ceph-mon[133315]: Standby manager daemon y restarted 2023-09-13T04:40:26.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:26 smithi149 ceph-mon[133315]: Standby manager daemon y started 2023-09-13T04:40:26.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:26 smithi149 ceph-mon[133315]: from='mgr.? 172.21.15.149:0/690810681' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-13T04:40:26.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:26 smithi149 ceph-mon[133315]: from='mgr.? 172.21.15.149:0/690810681' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-13T04:40:26.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:26 smithi149 ceph-mon[133315]: from='mgr.? 172.21.15.149:0/690810681' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-13T04:40:26.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:26 smithi149 ceph-mon[133315]: from='mgr.? 172.21.15.149:0/690810681' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-13T04:40:26.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:26 smithi156 ceph-mon[133675]: mgrmap e18: x(active, since 9m), standbys: y, z 2023-09-13T04:40:26.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:26 smithi156 ceph-mon[133675]: Standby manager daemon y restarted 2023-09-13T04:40:26.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:26 smithi156 ceph-mon[133675]: Standby manager daemon y started 2023-09-13T04:40:26.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:26 smithi156 ceph-mon[133675]: from='mgr.? 172.21.15.149:0/690810681' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-13T04:40:26.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:26 smithi156 ceph-mon[133675]: from='mgr.? 172.21.15.149:0/690810681' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-13T04:40:26.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:26 smithi156 ceph-mon[133675]: from='mgr.? 172.21.15.149:0/690810681' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-13T04:40:26.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:26 smithi156 ceph-mon[133675]: from='mgr.? 172.21.15.149:0/690810681' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-13T04:40:27.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:27 smithi002 ceph-mon[97651]: mgrmap e19: x(active, since 9m), standbys: y, z 2023-09-13T04:40:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:27.792+0000 7f78e3fff700 1 -- 172.21.15.2:0/234510675 <== mon.2 v2:172.21.15.149:3300/0 9 ==== mgrmap(e 20) v1 ==== 134944+0+0 (secure 0 0 0) 0x7f78ec096b90 con 0x7f78fc100d00 2023-09-13T04:40:27.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:27.793+0000 7f78e3fff700 1 -- 172.21.15.2:0/234510675 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f78e40a0690 msgr2=0x7f78e40a2b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-13T04:40:27.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:27.793+0000 7f78e3fff700 1 --2- 172.21.15.2:0/234510675 >> [v2:172.21.15.2:6800/1461251021,v1:172.21.15.2:6801/1461251021] conn(0x7f78e40a0690 0x7f78e40a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:27.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:27 smithi149 ceph-mon[133315]: mgrmap e19: x(active, since 9m), standbys: y, z 2023-09-13T04:40:27.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:27 smithi156 ceph-mon[133675]: mgrmap e19: x(active, since 9m), standbys: y, z 2023-09-13T04:40:28.230 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:27 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:27.772+0000 7f10b0f4b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-13T04:40:28.231 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:28.007+0000 7f101405d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T04:40:28.231 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:28.007+0000 7f101405d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T04:40:28.231 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:28.007+0000 7f101405d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T04:40:28.231 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:28.007+0000 7f101405d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T04:40:28.231 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:28.007+0000 7f101405d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T04:40:28.232 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:28.009+0000 7f1010856700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T04:40:28.232 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:28.009+0000 7f1010856700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T04:40:28.232 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:28.009+0000 7f1010856700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T04:40:28.232 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:28.009+0000 7f1010856700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T04:40:28.232 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:40:28.009+0000 7f1010856700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T04:40:28.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: Active manager daemon x restarted 2023-09-13T04:40:28.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: Activating manager daemon x 2023-09-13T04:40:28.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: osdmap e93: 12 total, 12 up, 12 in 2023-09-13T04:40:28.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: mgrmap e20: x(active, starting, since 0.0121894s), standbys: y, z 2023-09-13T04:40:28.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T04:40:28.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:40:28.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:40:28.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-13T04:40:28.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-13T04:40:28.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-13T04:40:28.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-13T04:40:28.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-13T04:40:28.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-13T04:40:28.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-13T04:40:28.509 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-13T04:40:28.509 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-13T04:40:28.509 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-13T04:40:28.509 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-13T04:40:28.509 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-13T04:40:28.510 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-13T04:40:28.510 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-13T04:40:28.510 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-13T04:40:28.510 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T04:40:28.510 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T04:40:28.511 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T04:40:28.511 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T04:40:28.511 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T04:40:28.511 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T04:40:28.511 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T04:40:28.512 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T04:40:28.512 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T04:40:28.512 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T04:40:28.512 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T04:40:28.512 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T04:40:28.513 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-13T04:40:28.513 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-13T04:40:28.513 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-13T04:40:28.513 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: Manager daemon x is now available 2023-09-13T04:40:28.513 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:40:28.514 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:40:28.514 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: scrub summary: idle 2023-09-13T04:40:28.514 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-13T04:40:28.514 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-13T04:40:28.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:28.795+0000 7f78e3fff700 1 -- 172.21.15.2:0/234510675 <== mon.2 v2:172.21.15.149:3300/0 10 ==== mgrmap(e 21) v1 ==== 135071+0+0 (secure 0 0 0) 0x7f78ec097460 con 0x7f78fc100d00 2023-09-13T04:40:28.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:28.796+0000 7f78e3fff700 1 --2- 172.21.15.2:0/234510675 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f78e404dba0 0x7f78e404df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:28.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:28.796+0000 7f78e3fff700 1 -- 172.21.15.2:0/234510675 --> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f78fc0610f0 con 0x7f78e404dba0 2023-09-13T04:40:28.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:28.801+0000 7f78fa7fc700 1 --2- 172.21.15.2:0/234510675 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f78e404dba0 0x7f78e404df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:28.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:28.807+0000 7f78fa7fc700 1 --2- 172.21.15.2:0/234510675 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f78e404dba0 0x7f78e404df80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f78fc0743a0 tx=0x7f78f0009260 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:28.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:28.877+0000 7f78e3fff700 1 -- 172.21.15.2:0/234510675 <== mgr.15492 v2:172.21.15.2:6800/3640961355 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f78fc0610f0 con 0x7f78e404dba0 2023-09-13T04:40:28.877 INFO:teuthology.orchestra.run.smithi002.stdout:Schedule set for path / 2023-09-13T04:40:28.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:28.879+0000 7f79013ff700 1 -- 172.21.15.2:0/234510675 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f78e404dba0 msgr2=0x7f78e404df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:28.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:28.879+0000 7f79013ff700 1 --2- 172.21.15.2:0/234510675 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f78e404dba0 0x7f78e404df80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f78fc0743a0 tx=0x7f78f0009260 comp rx=0 tx=0).stop 2023-09-13T04:40:28.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:28.879+0000 7f79013ff700 1 -- 172.21.15.2:0/234510675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78fc100d00 msgr2=0x7f78fc0730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:28.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:28.879+0000 7f79013ff700 1 --2- 172.21.15.2:0/234510675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78fc100d00 0x7f78fc0730a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f78ec009fd0 tx=0x7f78ec00f690 comp rx=0 tx=0).stop 2023-09-13T04:40:28.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:28.879+0000 7f78fb7fe700 1 -- 172.21.15.2:0/234510675 reap_dead start 2023-09-13T04:40:28.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:28.879+0000 7f79013ff700 1 -- 172.21.15.2:0/234510675 shutdown_connections 2023-09-13T04:40:28.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:28.879+0000 7f79013ff700 1 -- 172.21.15.2:0/234510675 >> 172.21.15.2:0/234510675 conn(0x7f78fc0fc230 msgr2=0x7f78fc0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:28.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:28.880+0000 7f79013ff700 1 -- 172.21.15.2:0/234510675 shutdown_connections 2023-09-13T04:40:28.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:28.880+0000 7f79013ff700 1 -- 172.21.15.2:0/234510675 wait complete. 2023-09-13T04:40:28.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: Active manager daemon x restarted 2023-09-13T04:40:28.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: Activating manager daemon x 2023-09-13T04:40:28.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: osdmap e93: 12 total, 12 up, 12 in 2023-09-13T04:40:28.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: mgrmap e20: x(active, starting, since 0.0121894s), standbys: y, z 2023-09-13T04:40:28.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T04:40:28.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:40:28.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:40:28.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-13T04:40:28.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-13T04:40:28.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-13T04:40:28.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-13T04:40:28.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-13T04:40:28.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-13T04:40:28.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-13T04:40:28.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-13T04:40:28.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-13T04:40:28.890 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-13T04:40:28.890 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-13T04:40:28.890 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-13T04:40:28.890 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-13T04:40:28.890 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-13T04:40:28.891 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-13T04:40:28.891 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T04:40:28.891 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T04:40:28.891 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T04:40:28.891 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T04:40:28.892 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T04:40:28.892 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T04:40:28.892 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T04:40:28.892 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T04:40:28.892 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T04:40:28.893 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T04:40:28.893 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T04:40:28.893 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T04:40:28.893 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-13T04:40:28.893 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-13T04:40:28.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-13T04:40:28.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: Manager daemon x is now available 2023-09-13T04:40:28.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:40:28.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:40:28.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: scrub summary: idle 2023-09-13T04:40:28.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-13T04:40:28.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-13T04:40:28.899 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h' 2023-09-13T04:40:28.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: Active manager daemon x restarted 2023-09-13T04:40:28.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: Activating manager daemon x 2023-09-13T04:40:28.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: osdmap e93: 12 total, 12 up, 12 in 2023-09-13T04:40:28.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: mgrmap e20: x(active, starting, since 0.0121894s), standbys: y, z 2023-09-13T04:40:28.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T04:40:28.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T04:40:28.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T04:40:28.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-13T04:40:28.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-13T04:40:28.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-13T04:40:28.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-13T04:40:28.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-13T04:40:28.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-13T04:40:28.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-13T04:40:28.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-13T04:40:28.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-13T04:40:28.940 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-13T04:40:28.940 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-13T04:40:28.940 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-13T04:40:28.940 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-13T04:40:28.940 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-13T04:40:28.941 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-13T04:40:28.941 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T04:40:28.941 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T04:40:28.941 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T04:40:28.942 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T04:40:28.942 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T04:40:28.942 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T04:40:28.942 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T04:40:28.942 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T04:40:28.943 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T04:40:28.943 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T04:40:28.943 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T04:40:28.943 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T04:40:28.943 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-13T04:40:28.944 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-13T04:40:28.944 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-13T04:40:28.944 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: Manager daemon x is now available 2023-09-13T04:40:28.944 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:40:28.945 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:40:28.945 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: scrub summary: idle 2023-09-13T04:40:28.945 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-13T04:40:28.945 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-13T04:40:29.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.129+0000 7fb14cdff700 1 -- 172.21.15.2:0/4281881961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb148074cf0 msgr2=0x7fb148073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:29.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.129+0000 7fb14cdff700 1 --2- 172.21.15.2:0/4281881961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb148074cf0 0x7fb148073140 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb138009990 tx=0x7fb138009ca0 comp rx=0 tx=0).stop 2023-09-13T04:40:29.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.129+0000 7fb14cdff700 1 -- 172.21.15.2:0/4281881961 shutdown_connections 2023-09-13T04:40:29.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.129+0000 7fb14cdff700 1 --2- 172.21.15.2:0/4281881961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb148108bc0 0x7fb14810b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:29.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.129+0000 7fb14cdff700 1 --2- 172.21.15.2:0/4281881961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb148074cf0 0x7fb148073140 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:29.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.129+0000 7fb14cdff700 1 --2- 172.21.15.2:0/4281881961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb148073710 0x7fb148073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:29.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.129+0000 7fb14cdff700 1 -- 172.21.15.2:0/4281881961 >> 172.21.15.2:0/4281881961 conn(0x7fb1480fc230 msgr2=0x7fb1480fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:29.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.129+0000 7fb14cdff700 1 -- 172.21.15.2:0/4281881961 shutdown_connections 2023-09-13T04:40:29.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.129+0000 7fb14cdff700 1 -- 172.21.15.2:0/4281881961 wait complete. 2023-09-13T04:40:29.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.131+0000 7fb14cdff700 1 Processor -- start 2023-09-13T04:40:29.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.131+0000 7fb14cdff700 1 -- start start 2023-09-13T04:40:29.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.131+0000 7fb14cdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb148073710 0x7fb148198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:29.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.131+0000 7fb14cdff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb148074cf0 0x7fb1481992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:29.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.131+0000 7fb14cdff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb148108bc0 0x7fb14819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:29.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.131+0000 7fb14cdff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14819a7a0 con 0x7fb148108bc0 2023-09-13T04:40:29.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.131+0000 7fb14cdff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14819a910 con 0x7fb148074cf0 2023-09-13T04:40:29.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.131+0000 7fb14cdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14819aad0 con 0x7fb148073710 2023-09-13T04:40:29.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.132+0000 7fb145d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb148074cf0 0x7fb1481992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:29.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.132+0000 7fb146d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb148108bc0 0x7fb14819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:29.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.132+0000 7fb145d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb148074cf0 0x7fb1481992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:48200/0 (socket says 172.21.15.2:48200) 2023-09-13T04:40:29.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.132+0000 7fb145d9b700 1 -- 172.21.15.2:0/2580553663 learned_addr learned my addr 172.21.15.2:0/2580553663 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:29.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.132+0000 7fb14659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb148073710 0x7fb148198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:29.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.132+0000 7fb145d9b700 1 -- 172.21.15.2:0/2580553663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb148073710 msgr2=0x7fb148198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:29.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.132+0000 7fb145d9b700 1 --2- 172.21.15.2:0/2580553663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb148073710 0x7fb148198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:29.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.132+0000 7fb145d9b700 1 -- 172.21.15.2:0/2580553663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb148108bc0 msgr2=0x7fb14819a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:29.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.132+0000 7fb145d9b700 1 --2- 172.21.15.2:0/2580553663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb148108bc0 0x7fb14819a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:29.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.132+0000 7fb145d9b700 1 -- 172.21.15.2:0/2580553663 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb140009c40 con 0x7fb148074cf0 2023-09-13T04:40:29.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.132+0000 7fb145d9b700 1 --2- 172.21.15.2:0/2580553663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb148074cf0 0x7fb1481992f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00e970 tx=0x7fb13c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:29.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.132+0000 7fb1377fe700 1 -- 172.21.15.2:0/2580553663 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c009790 con 0x7fb148074cf0 2023-09-13T04:40:29.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.133+0000 7fb1377fe700 1 -- 172.21.15.2:0/2580553663 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb13c00c7c0 con 0x7fb148074cf0 2023-09-13T04:40:29.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.133+0000 7fb14cdff700 1 -- 172.21.15.2:0/2580553663 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1380095f0 con 0x7fb148074cf0 2023-09-13T04:40:29.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.133+0000 7fb1377fe700 1 -- 172.21.15.2:0/2580553663 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c003c20 con 0x7fb148074cf0 2023-09-13T04:40:29.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.133+0000 7fb14cdff700 1 -- 172.21.15.2:0/2580553663 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1481a14f0 con 0x7fb148074cf0 2023-09-13T04:40:29.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.134+0000 7fb14cdff700 1 -- 172.21.15.2:0/2580553663 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb148107270 con 0x7fb148074cf0 2023-09-13T04:40:29.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.136+0000 7fb1377fe700 1 -- 172.21.15.2:0/2580553663 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 21) v1 ==== 135071+0+0 (secure 0 0 0) 0x7fb13c00c930 con 0x7fb148074cf0 2023-09-13T04:40:29.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.137+0000 7fb1377fe700 1 --2- 172.21.15.2:0/2580553663 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fb1300a0440 0x7fb1300a2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:29.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.137+0000 7fb1377fe700 1 -- 172.21.15.2:0/2580553663 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fb13c014070 con 0x7fb148074cf0 2023-09-13T04:40:29.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.137+0000 7fb14659c700 1 --2- 172.21.15.2:0/2580553663 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fb1300a0440 0x7fb1300a2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:29.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.138+0000 7fb14659c700 1 --2- 172.21.15.2:0/2580553663 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fb1300a0440 0x7fb1300a2900 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb13800b2e0 tx=0x7fb138005fb0 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:29.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.141+0000 7fb1377fe700 1 -- 172.21.15.2:0/2580553663 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb13c095760 con 0x7fb148074cf0 2023-09-13T04:40:29.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.392+0000 7fb14cdff700 1 -- 172.21.15.2:0/2580553663 --> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1480610f0 con 0x7fb1300a0440 2023-09-13T04:40:29.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.451+0000 7fb1377fe700 1 -- 172.21.15.2:0/2580553663 <== mgr.15492 v2:172.21.15.2:6800/3640961355 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fb1480610f0 con 0x7fb1300a0440 2023-09-13T04:40:29.452 INFO:teuthology.orchestra.run.smithi002.stdout:Retention added to path / 2023-09-13T04:40:29.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.453+0000 7fb14cdff700 1 -- 172.21.15.2:0/2580553663 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fb1300a0440 msgr2=0x7fb1300a2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:29.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.453+0000 7fb14cdff700 1 --2- 172.21.15.2:0/2580553663 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fb1300a0440 0x7fb1300a2900 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb13800b2e0 tx=0x7fb138005fb0 comp rx=0 tx=0).stop 2023-09-13T04:40:29.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.453+0000 7fb14cdff700 1 -- 172.21.15.2:0/2580553663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb148074cf0 msgr2=0x7fb1481992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:29.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.453+0000 7fb14cdff700 1 --2- 172.21.15.2:0/2580553663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb148074cf0 0x7fb1481992f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00e970 tx=0x7fb13c00ec80 comp rx=0 tx=0).stop 2023-09-13T04:40:29.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.455+0000 7fb14cdff700 1 -- 172.21.15.2:0/2580553663 shutdown_connections 2023-09-13T04:40:29.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.455+0000 7fb14cdff700 1 --2- 172.21.15.2:0/2580553663 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fb1300a0440 0x7fb1300a2900 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:29.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.455+0000 7fb14cdff700 1 --2- 172.21.15.2:0/2580553663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb148108bc0 0x7fb14819a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:29.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.455+0000 7fb14cdff700 1 --2- 172.21.15.2:0/2580553663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb148073710 0x7fb148198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:29.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.455+0000 7fb14cdff700 1 --2- 172.21.15.2:0/2580553663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb148074cf0 0x7fb1481992f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:29.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.455+0000 7fb14cdff700 1 -- 172.21.15.2:0/2580553663 >> 172.21.15.2:0/2580553663 conn(0x7fb1480fc230 msgr2=0x7fb1480fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:29.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.455+0000 7fb14cdff700 1 -- 172.21.15.2:0/2580553663 shutdown_connections 2023-09-13T04:40:29.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.455+0000 7fb14cdff700 1 -- 172.21.15.2:0/2580553663 wait complete. 2023-09-13T04:40:29.474 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-09-13T04:40:29.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.684+0000 7f8b885ff700 1 -- 172.21.15.2:0/1539719247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b80074cf0 msgr2=0x7f8b80073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:29.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.684+0000 7f8b885ff700 1 --2- 172.21.15.2:0/1539719247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b80074cf0 0x7f8b80073140 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c0098f0 tx=0x7f8b7c009c00 comp rx=0 tx=0).stop 2023-09-13T04:40:29.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.684+0000 7f8b885ff700 1 -- 172.21.15.2:0/1539719247 shutdown_connections 2023-09-13T04:40:29.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.684+0000 7f8b885ff700 1 --2- 172.21.15.2:0/1539719247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b80108bc0 0x7f8b8010b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:29.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.684+0000 7f8b885ff700 1 --2- 172.21.15.2:0/1539719247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b80074cf0 0x7f8b80073140 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:29.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.684+0000 7f8b885ff700 1 --2- 172.21.15.2:0/1539719247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b80073710 0x7f8b80073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.684+0000 7f8b885ff700 1 -- 172.21.15.2:0/1539719247 >> 172.21.15.2:0/1539719247 conn(0x7f8b800fc230 msgr2=0x7f8b800fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.685+0000 7f8b885ff700 1 -- 172.21.15.2:0/1539719247 shutdown_connections 2023-09-13T04:40:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.685+0000 7f8b885ff700 1 -- 172.21.15.2:0/1539719247 wait complete. 2023-09-13T04:40:29.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.685+0000 7f8b885ff700 1 Processor -- start 2023-09-13T04:40:29.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.686+0000 7f8b885ff700 1 -- start start 2023-09-13T04:40:29.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.686+0000 7f8b885ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b80073710 0x7f8b80198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:29.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.686+0000 7f8b885ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b80074cf0 0x7f8b80199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:29.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.686+0000 7f8b885ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b80108bc0 0x7f8b8019a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:29.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.686+0000 7f8b885ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8019a7c0 con 0x7f8b80108bc0 2023-09-13T04:40:29.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.686+0000 7f8b885ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8019a930 con 0x7f8b80073710 2023-09-13T04:40:29.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.686+0000 7f8b885ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8019aaf0 con 0x7f8b80074cf0 2023-09-13T04:40:29.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.686+0000 7f8b86b9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b80108bc0 0x7f8b8019a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:29.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.686+0000 7f8b86b9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b80108bc0 0x7f8b8019a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33552/0 (socket says 172.21.15.2:33552) 2023-09-13T04:40:29.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.686+0000 7f8b86b9c700 1 -- 172.21.15.2:0/3022949183 learned_addr learned my addr 172.21.15.2:0/3022949183 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:29.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.686+0000 7f8b8639b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b80073710 0x7f8b80198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:29.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.686+0000 7f8b85b9a700 1 --2- 172.21.15.2:0/3022949183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b80074cf0 0x7f8b80199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:29.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.687+0000 7f8b86b9c700 1 -- 172.21.15.2:0/3022949183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b80074cf0 msgr2=0x7f8b80199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:29.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.687+0000 7f8b86b9c700 1 --2- 172.21.15.2:0/3022949183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b80074cf0 0x7f8b80199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:29.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.687+0000 7f8b86b9c700 1 -- 172.21.15.2:0/3022949183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b80073710 msgr2=0x7f8b80198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:29.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.687+0000 7f8b86b9c700 1 --2- 172.21.15.2:0/3022949183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b80073710 0x7f8b80198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:29.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.687+0000 7f8b86b9c700 1 -- 172.21.15.2:0/3022949183 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b700095f0 con 0x7f8b80108bc0 2023-09-13T04:40:29.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.687+0000 7f8b8639b700 1 --2- 172.21.15.2:0/3022949183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b80073710 0x7f8b80198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:29.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.687+0000 7f8b86b9c700 1 --2- 172.21.15.2:0/3022949183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b80108bc0 0x7f8b8019a0e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8b7800ef30 tx=0x7f8b7800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:29.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.687+0000 7f8b777fe700 1 -- 172.21.15.2:0/3022949183 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78018070 con 0x7f8b80108bc0 2023-09-13T04:40:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.687+0000 7f8b885ff700 1 -- 172.21.15.2:0/3022949183 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7c009520 con 0x7f8b80108bc0 2023-09-13T04:40:29.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.687+0000 7f8b777fe700 1 -- 172.21.15.2:0/3022949183 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b7800f040 con 0x7f8b80108bc0 2023-09-13T04:40:29.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.687+0000 7f8b777fe700 1 -- 172.21.15.2:0/3022949183 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7800cd00 con 0x7f8b80108bc0 2023-09-13T04:40:29.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.687+0000 7f8b885ff700 1 -- 172.21.15.2:0/3022949183 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b801a1330 con 0x7f8b80108bc0 2023-09-13T04:40:29.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.688+0000 7f8b885ff700 1 -- 172.21.15.2:0/3022949183 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b80107270 con 0x7f8b80108bc0 2023-09-13T04:40:29.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.690+0000 7f8b777fe700 1 -- 172.21.15.2:0/3022949183 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 21) v1 ==== 135071+0+0 (secure 0 0 0) 0x7f8b78003a00 con 0x7f8b80108bc0 2023-09-13T04:40:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.691+0000 7f8b777fe700 1 --2- 172.21.15.2:0/3022949183 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f8b6c0a0440 0x7f8b6c0a2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.692+0000 7f8b777fe700 1 -- 172.21.15.2:0/3022949183 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f8b780cb770 con 0x7f8b80108bc0 2023-09-13T04:40:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.692+0000 7f8b8639b700 1 --2- 172.21.15.2:0/3022949183 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f8b6c0a0440 0x7f8b6c0a2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.692+0000 7f8b8639b700 1 --2- 172.21.15.2:0/3022949183 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f8b6c0a0440 0x7f8b6c0a2900 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c005e90 tx=0x7f8b7c019040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.696+0000 7f8b777fe700 1 -- 172.21.15.2:0/3022949183 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8b780951b0 con 0x7f8b80108bc0 2023-09-13T04:40:29.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.956+0000 7f8b885ff700 1 -- 172.21.15.2:0/3022949183 --> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b8002cde0 con 0x7f8b6c0a0440 2023-09-13T04:40:29.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.965+0000 7f8b777fe700 1 -- 172.21.15.2:0/3022949183 <== mgr.15492 v2:172.21.15.2:6800/3640961355 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7f8b8002cde0 con 0x7f8b6c0a0440 2023-09-13T04:40:29.966 INFO:teuthology.orchestra.run.smithi002.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-09-13T00:00:00", "created": "2023-09-13T04:40:28", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-09-13T04:40:29.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.968+0000 7f8b885ff700 1 -- 172.21.15.2:0/3022949183 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f8b6c0a0440 msgr2=0x7f8b6c0a2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:29.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.968+0000 7f8b885ff700 1 --2- 172.21.15.2:0/3022949183 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f8b6c0a0440 0x7f8b6c0a2900 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c005e90 tx=0x7f8b7c019040 comp rx=0 tx=0).stop 2023-09-13T04:40:29.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.968+0000 7f8b885ff700 1 -- 172.21.15.2:0/3022949183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b80108bc0 msgr2=0x7f8b8019a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:29.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.968+0000 7f8b885ff700 1 --2- 172.21.15.2:0/3022949183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b80108bc0 0x7f8b8019a0e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8b7800ef30 tx=0x7f8b7800c3a0 comp rx=0 tx=0).stop 2023-09-13T04:40:29.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.968+0000 7f8b885ff700 1 -- 172.21.15.2:0/3022949183 shutdown_connections 2023-09-13T04:40:29.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.968+0000 7f8b885ff700 1 --2- 172.21.15.2:0/3022949183 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f8b6c0a0440 0x7f8b6c0a2900 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:29.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.968+0000 7f8b885ff700 1 --2- 172.21.15.2:0/3022949183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b80108bc0 0x7f8b8019a0e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:29.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.968+0000 7f8b885ff700 1 --2- 172.21.15.2:0/3022949183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b80073710 0x7f8b80198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:29.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.968+0000 7f8b885ff700 1 --2- 172.21.15.2:0/3022949183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b80074cf0 0x7f8b80199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:29.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.968+0000 7f8b885ff700 1 -- 172.21.15.2:0/3022949183 >> 172.21.15.2:0/3022949183 conn(0x7f8b800fc230 msgr2=0x7f8b800fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:29.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.968+0000 7f8b885ff700 1 -- 172.21.15.2:0/3022949183 shutdown_connections 2023-09-13T04:40:29.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:29.968+0000 7f8b885ff700 1 -- 172.21.15.2:0/3022949183 wait complete. 2023-09-13T04:40:29.988 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-09-13T04:40:30.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:29 smithi002 ceph-mon[97651]: mgrmap e21: x(active, since 1.01941s), standbys: y, z 2023-09-13T04:40:30.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:29 smithi002 ceph-mon[97651]: from='client.25538 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:40:30.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:29 smithi002 ceph-mon[97651]: pgmap v3: 129 pgs: 129 active+clean; 970 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:40:30.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:29 smithi149 ceph-mon[133315]: mgrmap e21: x(active, since 1.01941s), standbys: y, z 2023-09-13T04:40:30.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:29 smithi149 ceph-mon[133315]: from='client.25538 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:40:30.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:29 smithi149 ceph-mon[133315]: pgmap v3: 129 pgs: 129 active+clean; 970 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:40:30.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:29 smithi156 ceph-mon[133675]: mgrmap e21: x(active, since 1.01941s), standbys: y, z 2023-09-13T04:40:30.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:29 smithi156 ceph-mon[133675]: from='client.25538 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:40:30.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:29 smithi156 ceph-mon[133675]: pgmap v3: 129 pgs: 129 active+clean; 970 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:40:30.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.201+0000 7f14a6dff700 1 -- 172.21.15.2:0/3355984619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14a0102880 msgr2=0x7f14a0109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:30.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.201+0000 7f14a6dff700 1 --2- 172.21.15.2:0/3355984619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14a0102880 0x7f14a0109130 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1498009fc0 tx=0x7f149800b330 comp rx=0 tx=0).stop 2023-09-13T04:40:30.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.201+0000 7f14a6dff700 1 -- 172.21.15.2:0/3355984619 shutdown_connections 2023-09-13T04:40:30.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.201+0000 7f14a6dff700 1 --2- 172.21.15.2:0/3355984619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14a0102880 0x7f14a0109130 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:30.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.201+0000 7f14a6dff700 1 --2- 172.21.15.2:0/3355984619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14a0100cf0 0x7f14a0101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:30.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.201+0000 7f14a6dff700 1 --2- 172.21.15.2:0/3355984619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14a0101ee0 0x7f14a0102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:30.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.201+0000 7f14a6dff700 1 -- 172.21.15.2:0/3355984619 >> 172.21.15.2:0/3355984619 conn(0x7f14a00fc280 msgr2=0x7f14a00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:30.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.201+0000 7f14a6dff700 1 -- 172.21.15.2:0/3355984619 shutdown_connections 2023-09-13T04:40:30.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.201+0000 7f14a6dff700 1 -- 172.21.15.2:0/3355984619 wait complete. 2023-09-13T04:40:30.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.203+0000 7f14a6dff700 1 Processor -- start 2023-09-13T04:40:30.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.203+0000 7f14a6dff700 1 -- start start 2023-09-13T04:40:30.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.203+0000 7f14a6dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14a0100cf0 0x7f14a00730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:30.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.203+0000 7f14a6dff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14a0101ee0 0x7f14a00735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:30.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.203+0000 7f14a6dff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14a0102880 0x7f14a0073b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:30.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.204+0000 7f14a6dff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a01a0b50 con 0x7f14a0102880 2023-09-13T04:40:30.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.204+0000 7f14a6dff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a01a0c90 con 0x7f14a0101ee0 2023-09-13T04:40:30.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.204+0000 7f14a6dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a01a0dd0 con 0x7f14a0100cf0 2023-09-13T04:40:30.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.204+0000 7f14a4b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14a0100cf0 0x7f14a00730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:30.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.204+0000 7f14a4b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14a0100cf0 0x7f14a00730a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:59206/0 (socket says 172.21.15.2:59206) 2023-09-13T04:40:30.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.204+0000 7f14a4b9b700 1 -- 172.21.15.2:0/863120191 learned_addr learned my addr 172.21.15.2:0/863120191 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:30.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.204+0000 7f14a539c700 1 --2- 172.21.15.2:0/863120191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14a0102880 0x7f14a0073b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:30.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.204+0000 7f149ffff700 1 --2- 172.21.15.2:0/863120191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14a0101ee0 0x7f14a00735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:30.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.204+0000 7f14a4b9b700 1 -- 172.21.15.2:0/863120191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14a0101ee0 msgr2=0x7f14a00735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:30.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.204+0000 7f14a4b9b700 1 --2- 172.21.15.2:0/863120191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14a0101ee0 0x7f14a00735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:30.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.204+0000 7f14a4b9b700 1 -- 172.21.15.2:0/863120191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14a0102880 msgr2=0x7f14a0073b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:30.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.204+0000 7f14a4b9b700 1 --2- 172.21.15.2:0/863120191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14a0102880 0x7f14a0073b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:30.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.204+0000 7f14a4b9b700 1 -- 172.21.15.2:0/863120191 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1494009520 con 0x7f14a0100cf0 2023-09-13T04:40:30.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.204+0000 7f14a4b9b700 1 --2- 172.21.15.2:0/863120191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14a0100cf0 0x7f14a00730a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f149400ea50 tx=0x7f149400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:30.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.205+0000 7f149dffb700 1 -- 172.21.15.2:0/863120191 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149400cbd0 con 0x7f14a0100cf0 2023-09-13T04:40:30.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.205+0000 7f14a6dff700 1 -- 172.21.15.2:0/863120191 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148c0095f0 con 0x7f14a0100cf0 2023-09-13T04:40:30.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.205+0000 7f149dffb700 1 -- 172.21.15.2:0/863120191 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f149400cd30 con 0x7f14a0100cf0 2023-09-13T04:40:30.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.205+0000 7f149dffb700 1 -- 172.21.15.2:0/863120191 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1494018680 con 0x7f14a0100cf0 2023-09-13T04:40:30.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.205+0000 7f14a6dff700 1 -- 172.21.15.2:0/863120191 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1498009c40 con 0x7f14a0100cf0 2023-09-13T04:40:30.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.206+0000 7f14a6dff700 1 -- 172.21.15.2:0/863120191 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a0108230 con 0x7f14a0100cf0 2023-09-13T04:40:30.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.208+0000 7f149dffb700 1 -- 172.21.15.2:0/863120191 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 21) v1 ==== 135071+0+0 (secure 0 0 0) 0x7f1494010430 con 0x7f14a0100cf0 2023-09-13T04:40:30.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.209+0000 7f149dffb700 1 --2- 172.21.15.2:0/863120191 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f14900a0440 0x7f14900a2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:30.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.209+0000 7f149dffb700 1 -- 172.21.15.2:0/863120191 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f1494014070 con 0x7f14a0100cf0 2023-09-13T04:40:30.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.210+0000 7f149ffff700 1 --2- 172.21.15.2:0/863120191 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f14900a0440 0x7f14900a2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:30.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.210+0000 7f149ffff700 1 --2- 172.21.15.2:0/863120191 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f14900a0440 0x7f14900a2900 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f14a00743a0 tx=0x7f148c009310 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:30.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.211+0000 7f149dffb700 1 -- 172.21.15.2:0/863120191 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1494095f70 con 0x7f14a0100cf0 2023-09-13T04:40:30.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.285+0000 7f149dffb700 1 -- 172.21.15.2:0/863120191 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mgrmap(e 22) v1 ==== 135378+0+0 (secure 0 0 0) 0x7f1494096340 con 0x7f14a0100cf0 2023-09-13T04:40:30.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.474+0000 7f14a6dff700 1 -- 172.21.15.2:0/863120191 --> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f14a002ce10 con 0x7f14900a0440 2023-09-13T04:40:30.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.481+0000 7f149dffb700 1 -- 172.21.15.2:0/863120191 <== mgr.15492 v2:172.21.15.2:6800/3640961355 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f14a002ce10 con 0x7f14900a0440 2023-09-13T04:40:30.481 INFO:teuthology.orchestra.run.smithi002.stdout:/ 1M 6M3h 2023-09-13T04:40:30.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.484+0000 7f14a6dff700 1 -- 172.21.15.2:0/863120191 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f14900a0440 msgr2=0x7f14900a2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:30.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.484+0000 7f14a6dff700 1 --2- 172.21.15.2:0/863120191 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f14900a0440 0x7f14900a2900 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f14a00743a0 tx=0x7f148c009310 comp rx=0 tx=0).stop 2023-09-13T04:40:30.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.484+0000 7f14a6dff700 1 -- 172.21.15.2:0/863120191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14a0100cf0 msgr2=0x7f14a00730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:30.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.484+0000 7f14a6dff700 1 --2- 172.21.15.2:0/863120191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14a0100cf0 0x7f14a00730a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f149400ea50 tx=0x7f149400ed60 comp rx=0 tx=0).stop 2023-09-13T04:40:30.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.484+0000 7f14a6dff700 1 -- 172.21.15.2:0/863120191 shutdown_connections 2023-09-13T04:40:30.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.484+0000 7f14a6dff700 1 --2- 172.21.15.2:0/863120191 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f14900a0440 0x7f14900a2900 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:30.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.484+0000 7f14a6dff700 1 --2- 172.21.15.2:0/863120191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14a0102880 0x7f14a0073b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:30.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.484+0000 7f14a6dff700 1 --2- 172.21.15.2:0/863120191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14a0100cf0 0x7f14a00730a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:30.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.484+0000 7f14a6dff700 1 --2- 172.21.15.2:0/863120191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14a0101ee0 0x7f14a00735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:30.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.484+0000 7f14a6dff700 1 -- 172.21.15.2:0/863120191 >> 172.21.15.2:0/863120191 conn(0x7f14a00fc280 msgr2=0x7f14a0105370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:30.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.484+0000 7f14a6dff700 1 -- 172.21.15.2:0/863120191 shutdown_connections 2023-09-13T04:40:30.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:30.484+0000 7f14a6dff700 1 -- 172.21.15.2:0/863120191 wait complete. 2023-09-13T04:40:30.503 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-09-13T04:40:30.614 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-09-13T04:40:30.626 INFO:teuthology.run_tasks:Running task background_exec... 2023-09-13T04:40:30.646 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2023-09-13T04:40:30.646 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-09-13T04:40:30.651 INFO:teuthology.run_tasks:Running task workunit... 2023-09-13T04:40:30.664 INFO:tasks.workunit:Pulling workunits from ref 4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 2023-09-13T04:40:30.665 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-09-13T04:40:30.665 INFO:tasks.workunit:timeout=3h 2023-09-13T04:40:30.665 INFO:tasks.workunit:cleanup=True 2023-09-13T04:40:30.666 DEBUG:teuthology.orchestra.run.smithi002:> stat -- /home/ubuntu/cephtest/mnt.0 2023-09-13T04:40:30.715 INFO:teuthology.orchestra.run.smithi002.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-09-13T04:40:30.715 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-09-13T04:40:30.716 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 79h/121d Inode: 1 Links: 2 2023-09-13T04:40:30.716 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T04:40:30.716 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:cephfs_t:s0 2023-09-13T04:40:30.716 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2023-09-13T04:40:30.717 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2023-09-13 04:37:59.575899793 +0000 2023-09-13T04:40:30.717 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2023-09-13 04:38:15.671446421 +0000 2023-09-13T04:40:30.717 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2023-09-13 04:37:59.575899793 +0000 2023-09-13T04:40:30.717 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-09-13T04:40:30.718 DEBUG:teuthology.orchestra.run.smithi002:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-09-13T04:40:30.802 DEBUG:teuthology.orchestra.run.smithi002:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 2023-09-13T04:40:30.850 INFO:tasks.workunit.client.0.smithi002.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-09-13T04:40:31.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:30 smithi149 ceph-mon[133315]: [13/Sep/2023:04:40:29] ENGINE Bus STARTING 2023-09-13T04:40:31.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:30 smithi149 ceph-mon[133315]: from='client.25580 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:40:31.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:30 smithi149 ceph-mon[133315]: [13/Sep/2023:04:40:29] ENGINE Serving on https://172.21.15.2:7150 2023-09-13T04:40:31.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:30 smithi149 ceph-mon[133315]: [13/Sep/2023:04:40:29] ENGINE Serving on http://172.21.15.2:8765 2023-09-13T04:40:31.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:30 smithi149 ceph-mon[133315]: [13/Sep/2023:04:40:29] ENGINE Bus STARTED 2023-09-13T04:40:31.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:30 smithi149 ceph-mon[133315]: pgmap v4: 129 pgs: 129 active+clean; 970 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:40:31.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:30 smithi149 ceph-mon[133315]: mgrmap e22: x(active, since 2s), standbys: y, z 2023-09-13T04:40:31.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:30 smithi156 ceph-mon[133675]: [13/Sep/2023:04:40:29] ENGINE Bus STARTING 2023-09-13T04:40:31.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:30 smithi156 ceph-mon[133675]: from='client.25580 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:40:31.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:30 smithi156 ceph-mon[133675]: [13/Sep/2023:04:40:29] ENGINE Serving on https://172.21.15.2:7150 2023-09-13T04:40:31.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:30 smithi156 ceph-mon[133675]: [13/Sep/2023:04:40:29] ENGINE Serving on http://172.21.15.2:8765 2023-09-13T04:40:31.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:30 smithi156 ceph-mon[133675]: [13/Sep/2023:04:40:29] ENGINE Bus STARTED 2023-09-13T04:40:31.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:30 smithi156 ceph-mon[133675]: pgmap v4: 129 pgs: 129 active+clean; 970 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:40:31.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:30 smithi156 ceph-mon[133675]: mgrmap e22: x(active, since 2s), standbys: y, z 2023-09-13T04:40:31.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:30 smithi002 ceph-mon[97651]: [13/Sep/2023:04:40:29] ENGINE Bus STARTING 2023-09-13T04:40:31.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:30 smithi002 ceph-mon[97651]: from='client.25580 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:40:31.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:30 smithi002 ceph-mon[97651]: [13/Sep/2023:04:40:29] ENGINE Serving on https://172.21.15.2:7150 2023-09-13T04:40:31.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:30 smithi002 ceph-mon[97651]: [13/Sep/2023:04:40:29] ENGINE Serving on http://172.21.15.2:8765 2023-09-13T04:40:31.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:30 smithi002 ceph-mon[97651]: [13/Sep/2023:04:40:29] ENGINE Bus STARTED 2023-09-13T04:40:31.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:30 smithi002 ceph-mon[97651]: pgmap v4: 129 pgs: 129 active+clean; 970 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:40:31.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:30 smithi002 ceph-mon[97651]: mgrmap e22: x(active, since 2s), standbys: y, z 2023-09-13T04:40:32.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:31 smithi149 ceph-mon[133315]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:40:32.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:31 smithi149 ceph-mon[133315]: from='client.25603 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:40:32.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:31 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:32.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:31 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:32.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:31 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:32.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:31 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:32.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:31 smithi156 ceph-mon[133675]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:40:32.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:31 smithi156 ceph-mon[133675]: from='client.25603 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:40:32.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:31 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:32.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:31 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:32.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:31 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:32.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:31 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:32.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:31 smithi002 ceph-mon[97651]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:40:32.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:31 smithi002 ceph-mon[97651]: from='client.25603 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:40:32.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:31 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:32.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:31 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:32.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:31 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:32.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:31 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:33.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:32 smithi149 ceph-mon[133315]: pgmap v5: 129 pgs: 129 active+clean; 970 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:40:33.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:32 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:33.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:32 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:33.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:32 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:33.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:32 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:33.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:32 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:33.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:32 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:33.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:32 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2023-09-13T04:40:33.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:32 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:33.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:32 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:33.139 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:32 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:33.139 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:32 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:33.139 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:32 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:33.139 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:32 smithi149 ceph-mon[133315]: mgrmap e23: x(active, since 4s), standbys: y, z 2023-09-13T04:40:33.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:32 smithi156 ceph-mon[133675]: pgmap v5: 129 pgs: 129 active+clean; 970 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:40:33.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:32 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:33.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:32 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:33.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:32 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:33.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:32 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:33.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:32 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:33.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:32 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:33.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:32 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2023-09-13T04:40:33.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:32 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:33.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:32 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:33.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:32 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:33.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:32 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:33.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:32 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:33.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:32 smithi156 ceph-mon[133675]: mgrmap e23: x(active, since 4s), standbys: y, z 2023-09-13T04:40:33.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:32 smithi002 ceph-mon[97651]: pgmap v5: 129 pgs: 129 active+clean; 970 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:40:33.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:32 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:33.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:32 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:33.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:32 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:33.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:32 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:33.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:32 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:33.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:32 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:33.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:32 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2023-09-13T04:40:33.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:32 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:33.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:32 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:33.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:32 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:33.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:32 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:33.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:32 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:33.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:32 smithi002 ceph-mon[97651]: mgrmap e23: x(active, since 4s), standbys: y, z 2023-09-13T04:40:34.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:33 smithi149 ceph-mon[133315]: Adjusting osd_memory_target on smithi156 to 196.2M 2023-09-13T04:40:34.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:33 smithi149 ceph-mon[133315]: Unable to set osd_memory_target on smithi156 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-09-13T04:40:34.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:33 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:34.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:33 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:34.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:33 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:34.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:33 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:34.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:33 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:34.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:33 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:34.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:33 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:40:34.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:33 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:40:34.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:33 smithi156 ceph-mon[133675]: Adjusting osd_memory_target on smithi156 to 196.2M 2023-09-13T04:40:34.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:33 smithi156 ceph-mon[133675]: Unable to set osd_memory_target on smithi156 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-09-13T04:40:34.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:33 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:34.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:33 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:34.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:33 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:34.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:33 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:34.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:33 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:34.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:33 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:34.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:33 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:40:34.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:33 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:40:34.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:33 smithi002 ceph-mon[97651]: Adjusting osd_memory_target on smithi156 to 196.2M 2023-09-13T04:40:34.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:33 smithi002 ceph-mon[97651]: Unable to set osd_memory_target on smithi156 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-09-13T04:40:34.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:33 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:34.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:33 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:34.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:33 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:34.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:33 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:34.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:33 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:34.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:33 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:40:34.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:33 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:40:34.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:33 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:40:35.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:34 smithi149 ceph-mon[133315]: Adjusting osd_memory_target on smithi002 to 222.0M 2023-09-13T04:40:35.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:34 smithi149 ceph-mon[133315]: Unable to set osd_memory_target on smithi002 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-09-13T04:40:35.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:34 smithi149 ceph-mon[133315]: Updating smithi002:/etc/ceph/ceph.conf 2023-09-13T04:40:35.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:34 smithi149 ceph-mon[133315]: Updating smithi149:/etc/ceph/ceph.conf 2023-09-13T04:40:35.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:34 smithi149 ceph-mon[133315]: Updating smithi156:/etc/ceph/ceph.conf 2023-09-13T04:40:35.139 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:34 smithi149 ceph-mon[133315]: Updating smithi149:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:40:35.139 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:34 smithi149 ceph-mon[133315]: Updating smithi156:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:40:35.139 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:34 smithi149 ceph-mon[133315]: pgmap v6: 129 pgs: 129 active+clean; 970 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:40:35.140 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:34 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:35.140 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:34 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:35.141 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:34 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:35.141 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:34 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:35.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:34 smithi156 ceph-mon[133675]: Adjusting osd_memory_target on smithi002 to 222.0M 2023-09-13T04:40:35.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:34 smithi156 ceph-mon[133675]: Unable to set osd_memory_target on smithi002 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-09-13T04:40:35.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:34 smithi156 ceph-mon[133675]: Updating smithi002:/etc/ceph/ceph.conf 2023-09-13T04:40:35.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:34 smithi156 ceph-mon[133675]: Updating smithi149:/etc/ceph/ceph.conf 2023-09-13T04:40:35.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:34 smithi156 ceph-mon[133675]: Updating smithi156:/etc/ceph/ceph.conf 2023-09-13T04:40:35.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:34 smithi156 ceph-mon[133675]: Updating smithi149:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:40:35.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:34 smithi156 ceph-mon[133675]: Updating smithi156:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:40:35.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:34 smithi156 ceph-mon[133675]: pgmap v6: 129 pgs: 129 active+clean; 970 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:40:35.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:34 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:35.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:34 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:35.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:34 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:35.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:34 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:35.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:34 smithi002 ceph-mon[97651]: Adjusting osd_memory_target on smithi002 to 222.0M 2023-09-13T04:40:35.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:34 smithi002 ceph-mon[97651]: Unable to set osd_memory_target on smithi002 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-09-13T04:40:35.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:34 smithi002 ceph-mon[97651]: Updating smithi002:/etc/ceph/ceph.conf 2023-09-13T04:40:35.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:34 smithi002 ceph-mon[97651]: Updating smithi149:/etc/ceph/ceph.conf 2023-09-13T04:40:35.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:34 smithi002 ceph-mon[97651]: Updating smithi156:/etc/ceph/ceph.conf 2023-09-13T04:40:35.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:34 smithi002 ceph-mon[97651]: Updating smithi149:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:40:35.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:34 smithi002 ceph-mon[97651]: Updating smithi156:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:40:35.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:34 smithi002 ceph-mon[97651]: pgmap v6: 129 pgs: 129 active+clean; 970 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:40:35.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:34 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:35.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:34 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:35.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:34 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:35.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:34 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:36.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:35 smithi149 ceph-mon[133315]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2023-09-13T04:40:36.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:35 smithi149 ceph-mon[133315]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2023-09-13T04:40:36.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:35 smithi149 ceph-mon[133315]: Updating smithi002:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:40:36.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:35 smithi149 ceph-mon[133315]: Updating smithi156:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T04:40:36.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:35 smithi149 ceph-mon[133315]: Updating smithi149:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T04:40:36.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:35 smithi149 ceph-mon[133315]: Updating smithi002:/etc/ceph/ceph.client.admin.keyring 2023-09-13T04:40:36.139 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:35 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:36.139 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:35 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:36.139 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:35 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:36.140 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:35 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:40:36.140 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:35 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:40:36.140 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:35 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:40:36.140 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:35 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:36.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:35 smithi156 ceph-mon[133675]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2023-09-13T04:40:36.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:35 smithi156 ceph-mon[133675]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2023-09-13T04:40:36.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:35 smithi156 ceph-mon[133675]: Updating smithi002:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:40:36.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:35 smithi156 ceph-mon[133675]: Updating smithi156:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T04:40:36.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:35 smithi156 ceph-mon[133675]: Updating smithi149:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T04:40:36.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:35 smithi156 ceph-mon[133675]: Updating smithi002:/etc/ceph/ceph.client.admin.keyring 2023-09-13T04:40:36.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:35 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:36.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:35 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:36.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:35 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:36.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:35 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:40:36.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:35 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:40:36.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:35 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:40:36.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:35 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:36.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:35 smithi002 ceph-mon[97651]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2023-09-13T04:40:36.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:35 smithi002 ceph-mon[97651]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2023-09-13T04:40:36.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:35 smithi002 ceph-mon[97651]: Updating smithi002:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-13T04:40:36.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:35 smithi002 ceph-mon[97651]: Updating smithi156:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T04:40:36.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:35 smithi002 ceph-mon[97651]: Updating smithi149:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T04:40:36.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:35 smithi002 ceph-mon[97651]: Updating smithi002:/etc/ceph/ceph.client.admin.keyring 2023-09-13T04:40:36.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:35 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:36.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:35 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:36.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:35 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:36.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:35 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:40:36.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:35 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:40:36.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:35 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:40:36.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:35 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:40:37.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:36 smithi149 ceph-mon[133315]: Updating smithi002:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T04:40:37.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:36 smithi149 ceph-mon[133315]: pgmap v7: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 132 KiB/s wr, 67 op/s 2023-09-13T04:40:37.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:36 smithi156 ceph-mon[133675]: Updating smithi002:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T04:40:37.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:36 smithi156 ceph-mon[133675]: pgmap v7: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 132 KiB/s wr, 67 op/s 2023-09-13T04:40:37.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:36 smithi002 ceph-mon[97651]: Updating smithi002:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T04:40:37.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:36 smithi002 ceph-mon[97651]: pgmap v7: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 132 KiB/s wr, 67 op/s 2023-09-13T04:40:39.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:38 smithi149 ceph-mon[133315]: pgmap v8: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 102 KiB/s wr, 52 op/s 2023-09-13T04:40:39.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:38 smithi156 ceph-mon[133675]: pgmap v8: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 102 KiB/s wr, 52 op/s 2023-09-13T04:40:39.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:38 smithi002 ceph-mon[97651]: pgmap v8: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 102 KiB/s wr, 52 op/s 2023-09-13T04:40:40.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:40 smithi149 ceph-mon[133315]: pgmap v9: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 84 KiB/s wr, 45 op/s 2023-09-13T04:40:40.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:40 smithi156 ceph-mon[133675]: pgmap v9: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 84 KiB/s wr, 45 op/s 2023-09-13T04:40:40.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:40 smithi002 ceph-mon[97651]: pgmap v9: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 84 KiB/s wr, 45 op/s 2023-09-13T04:40:42.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:41 smithi149 ceph-mon[133315]: pgmap v10: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 77 KiB/s wr, 41 op/s 2023-09-13T04:40:42.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:41 smithi156 ceph-mon[133675]: pgmap v10: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 77 KiB/s wr, 41 op/s 2023-09-13T04:40:42.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:41 smithi002 ceph-mon[97651]: pgmap v10: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 77 KiB/s wr, 41 op/s 2023-09-13T04:40:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.917+0000 7f7a131ff700 1 -- 172.21.15.2:0/4038509815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a0c101880 msgr2=0x7f7a0c1058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.917+0000 7f7a131ff700 1 --2- 172.21.15.2:0/4038509815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a0c101880 0x7f7a0c1058c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f79f8009990 tx=0x7f79f8009ca0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.917+0000 7f7a131ff700 1 -- 172.21.15.2:0/4038509815 shutdown_connections 2023-09-13T04:40:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.917+0000 7f7a131ff700 1 --2- 172.21.15.2:0/4038509815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a0c105ff0 0x7f7a0c10db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.917+0000 7f7a131ff700 1 --2- 172.21.15.2:0/4038509815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a0c100ed0 0x7f7a0c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.917+0000 7f7a131ff700 1 --2- 172.21.15.2:0/4038509815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a0c101880 0x7f7a0c1058c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.917+0000 7f7a131ff700 1 -- 172.21.15.2:0/4038509815 >> 172.21.15.2:0/4038509815 conn(0x7f7a0c0fc750 msgr2=0x7f7a0c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.917+0000 7f7a131ff700 1 -- 172.21.15.2:0/4038509815 shutdown_connections 2023-09-13T04:40:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.917+0000 7f7a131ff700 1 -- 172.21.15.2:0/4038509815 wait complete. 2023-09-13T04:40:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.918+0000 7f7a131ff700 1 Processor -- start 2023-09-13T04:40:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.919+0000 7f7a131ff700 1 -- start start 2023-09-13T04:40:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.919+0000 7f7a131ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a0c100ed0 0x7f7a0c199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.919+0000 7f7a131ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a0c105ff0 0x7f7a0c199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.919+0000 7f7a131ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a0c19dc00 0x7f7a0c19e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.919+0000 7f7a131ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c19e680 con 0x7f7a0c105ff0 2023-09-13T04:40:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.919+0000 7f7a131ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c19e7f0 con 0x7f7a0c19dc00 2023-09-13T04:40:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.919+0000 7f7a131ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c19e9b0 con 0x7f7a0c100ed0 2023-09-13T04:40:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.919+0000 7f7a0bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a0c105ff0 0x7f7a0c199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.919+0000 7f7a0bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a0c105ff0 0x7f7a0c199590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42052/0 (socket says 172.21.15.2:42052) 2023-09-13T04:40:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.919+0000 7f7a0bfff700 1 -- 172.21.15.2:0/2723893278 learned_addr learned my addr 172.21.15.2:0/2723893278 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:43.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.919+0000 7f7a10f9b700 1 --2- 172.21.15.2:0/2723893278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a0c100ed0 0x7f7a0c199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:43.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.920+0000 7f7a1179c700 1 --2- 172.21.15.2:0/2723893278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a0c19dc00 0x7f7a0c19e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:43.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.920+0000 7f7a0bfff700 1 -- 172.21.15.2:0/2723893278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a0c100ed0 msgr2=0x7f7a0c199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:43.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.920+0000 7f7a0bfff700 1 --2- 172.21.15.2:0/2723893278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a0c100ed0 0x7f7a0c199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.920+0000 7f7a0bfff700 1 -- 172.21.15.2:0/2723893278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a0c19dc00 msgr2=0x7f7a0c19e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:43.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.920+0000 7f7a0bfff700 1 --2- 172.21.15.2:0/2723893278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a0c19dc00 0x7f7a0c19e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.920+0000 7f7a0bfff700 1 -- 172.21.15.2:0/2723893278 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f80095f0 con 0x7f7a0c105ff0 2023-09-13T04:40:43.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.920+0000 7f7a0bfff700 1 --2- 172.21.15.2:0/2723893278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a0c105ff0 0x7f7a0c199590 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f79f8000c00 tx=0x7f79f80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:43.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.920+0000 7f7a09ffb700 1 -- 172.21.15.2:0/2723893278 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f801d070 con 0x7f7a0c105ff0 2023-09-13T04:40:43.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.920+0000 7f7a09ffb700 1 -- 172.21.15.2:0/2723893278 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79f8021410 con 0x7f7a0c105ff0 2023-09-13T04:40:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.920+0000 7f7a131ff700 1 -- 172.21.15.2:0/2723893278 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a0c1a5eb0 con 0x7f7a0c105ff0 2023-09-13T04:40:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.920+0000 7f7a09ffb700 1 -- 172.21.15.2:0/2723893278 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8017650 con 0x7f7a0c105ff0 2023-09-13T04:40:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.921+0000 7f7a131ff700 1 -- 172.21.15.2:0/2723893278 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a0c1a62d0 con 0x7f7a0c105ff0 2023-09-13T04:40:43.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.924+0000 7f7a09ffb700 1 -- 172.21.15.2:0/2723893278 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 23) v1 ==== 135434+0+0 (secure 0 0 0) 0x7f79f8021580 con 0x7f7a0c105ff0 2023-09-13T04:40:43.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.925+0000 7f7a09ffb700 1 --2- 172.21.15.2:0/2723893278 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f79fc0a0840 0x7f79fc0a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:43.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.925+0000 7f7a26addb80 1 Processor -- start 2023-09-13T04:40:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.925+0000 7f7a09ffb700 1 -- 172.21.15.2:0/2723893278 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f79f80cd740 con 0x7f7a0c105ff0 2023-09-13T04:40:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.926+0000 7f7a26addb80 1 -- start start 2023-09-13T04:40:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.926+0000 7f7a10f9b700 1 --2- 172.21.15.2:0/2723893278 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f79fc0a0840 0x7f79fc0a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.926+0000 7f7a26addb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d939082be0 0x55d939082fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.926+0000 7f7a26addb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d939083500 0x55d93907e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.926+0000 7f7a26addb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d93907e840 0x55d93907ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.926+0000 7f7a26addb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d938fd7720 con 0x55d939082be0 2023-09-13T04:40:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.926+0000 7f7a26addb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d938fd5040 con 0x55d939083500 2023-09-13T04:40:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.926+0000 7f7a26addb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d938fd0930 con 0x55d93907e840 2023-09-13T04:40:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.926+0000 7f7a10f9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d939082be0 0x55d939082fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.926+0000 7f7a10f9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d939082be0 0x55d939082fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42060/0 (socket says 172.21.15.2:42060) 2023-09-13T04:40:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.926+0000 7f7a10f9b700 1 -- 172.21.15.2:0/1393968503 learned_addr learned my addr 172.21.15.2:0/1393968503 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.926+0000 7f7a0bfff700 1 --2- 172.21.15.2:0/1393968503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d939083500 0x55d93907e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.926+0000 7f7a1179c700 1 --2- 172.21.15.2:0/1393968503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d93907e840 0x55d93907ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.927+0000 7f7a10f9b700 1 --2- 172.21.15.2:0/2723893278 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f79fc0a0840 0x7f79fc0a2d00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7a00009cb0 tx=0x7f7a00011720 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.927+0000 7f7a10f9b700 1 -- 172.21.15.2:0/1393968503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d93907e840 msgr2=0x55d93907ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.927+0000 7f7a10f9b700 1 --2- 172.21.15.2:0/1393968503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d93907e840 0x55d93907ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.927+0000 7f7a10f9b700 1 -- 172.21.15.2:0/1393968503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d939083500 msgr2=0x55d93907e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.927+0000 7f7a10f9b700 1 --2- 172.21.15.2:0/1393968503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d939083500 0x55d93907e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.927+0000 7f7a10f9b700 1 -- 172.21.15.2:0/1393968503 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d938e505d0 con 0x55d939082be0 2023-09-13T04:40:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.927+0000 7f7a10f9b700 1 --2- 172.21.15.2:0/1393968503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d939082be0 0x55d939082fc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7a00005c50 tx=0x7f7a0001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636a892040f8b178 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.927+0000 7f79f6ffd700 1 -- 172.21.15.2:0/1393968503 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0001ddb0 con 0x55d939082be0 2023-09-13T04:40:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.927+0000 7f79f6ffd700 1 -- 172.21.15.2:0/1393968503 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a00004cf0 con 0x55d939082be0 2023-09-13T04:40:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.927+0000 7f79f6ffd700 1 -- 172.21.15.2:0/1393968503 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0000f620 con 0x55d939082be0 2023-09-13T04:40:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.928+0000 7f7a26addb80 1 -- 172.21.15.2:0/1393968503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d939082be0 msgr2=0x55d939082fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.928+0000 7f7a26addb80 1 --2- 172.21.15.2:0/1393968503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d939082be0 0x55d939082fc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7a00005c50 tx=0x7f7a0001c000 comp rx=0 tx=0).stop 2023-09-13T04:40:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.928+0000 7f7a26addb80 1 -- 172.21.15.2:0/1393968503 shutdown_connections 2023-09-13T04:40:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.928+0000 7f7a26addb80 1 --2- 172.21.15.2:0/1393968503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d93907e840 0x55d93907ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.928+0000 7f7a26addb80 1 --2- 172.21.15.2:0/1393968503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d939082be0 0x55d939082fc0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.928+0000 7f7a26addb80 1 --2- 172.21.15.2:0/1393968503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d939083500 0x55d93907e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.928+0000 7f7a26addb80 1 -- 172.21.15.2:0/1393968503 >> 172.21.15.2:0/1393968503 conn(0x55d938fdcc20 msgr2=0x55d939078b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.928+0000 7f7a26addb80 1 -- 172.21.15.2:0/1393968503 shutdown_connections 2023-09-13T04:40:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.928+0000 7f7a26addb80 1 -- 172.21.15.2:0/1393968503 wait complete. 2023-09-13T04:40:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.929+0000 7f7a26addb80 1 -- start start 2023-09-13T04:40:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.929+0000 7f7a26addb80 1 Processor -- start 2023-09-13T04:40:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.930+0000 7f7a26addb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d93907e840 0x55d939087870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.930+0000 7f7a26addb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d939082be0 0x55d939087db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.930+0000 7f7a26addb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d939083500 0x55d9390882f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.930+0000 7f7a26addb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d938f6eb50 con 0x55d939082be0 2023-09-13T04:40:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.930+0000 7f7a26addb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d938f6e850 con 0x55d939083500 2023-09-13T04:40:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.930+0000 7f7a26addb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d938e528c0 con 0x55d93907e840 2023-09-13T04:40:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.930+0000 7f7a26addb80 10 client.0 _ll_register_callbacks cb 0x55d938f6c620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.930+0000 7f7a0bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d939082be0 0x55d939087db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.930+0000 7f7a10f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d93907e840 0x55d939087870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.930+0000 7f7a0bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d939082be0 0x55d939087db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42064/0 (socket says 172.21.15.2:42064) 2023-09-13T04:40:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.930+0000 7f7a0bfff700 1 -- 172.21.15.2:0/2326665006 learned_addr learned my addr 172.21.15.2:0/2326665006 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.930+0000 7f7a1179c700 1 --2- 172.21.15.2:0/2326665006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d939083500 0x55d9390882f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.930+0000 7f7a0bfff700 1 -- 172.21.15.2:0/2326665006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d93907e840 msgr2=0x55d939087870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.930+0000 7f7a0bfff700 1 --2- 172.21.15.2:0/2326665006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d93907e840 0x55d939087870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.930+0000 7f7a0bfff700 1 -- 172.21.15.2:0/2326665006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d939083500 msgr2=0x55d9390882f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.930+0000 7f7a0bfff700 1 --2- 172.21.15.2:0/2326665006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d939083500 0x55d9390882f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.930+0000 7f7a0bfff700 1 -- 172.21.15.2:0/2326665006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a00011cd0 con 0x55d939082be0 2023-09-13T04:40:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.930+0000 7f7a1179c700 1 --2- 172.21.15.2:0/2326665006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d939083500 0x55d9390882f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.931+0000 7f7a0bfff700 1 --2- 172.21.15.2:0/2326665006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d939082be0 0x55d939087db0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f79f8096ca0 tx=0x7f79f80caff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0faa5470b0a71f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.931+0000 7f79ebfff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:40:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.931+0000 7f79ebfff700 1 -- 172.21.15.2:0/2326665006 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f80b1070 con 0x55d939082be0 2023-09-13T04:40:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.931+0000 7f7a26addb80 1 -- 172.21.15.2:0/2326665006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d939043ef0 con 0x55d939082be0 2023-09-13T04:40:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.931+0000 7f79ebfff700 1 -- 172.21.15.2:0/2326665006 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f79f809aea0 con 0x55d939082be0 2023-09-13T04:40:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.931+0000 7f79ebfff700 1 -- 172.21.15.2:0/2326665006 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79f809a7a0 con 0x55d939082be0 2023-09-13T04:40:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.931+0000 7f7a26addb80 10 client.15579 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.931+0000 7f7a26addb80 1 -- 172.21.15.2:0/2326665006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d938fe8740 con 0x55d939082be0 2023-09-13T04:40:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.931+0000 7f79ebfff700 1 -- 172.21.15.2:0/2326665006 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d939043ef0 con 0x55d939082be0 2023-09-13T04:40:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.932+0000 7f79ebfff700 1 -- 172.21.15.2:0/2326665006 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f79f8099c90 con 0x55d939082be0 2023-09-13T04:40:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.932+0000 7f7a26addb80 10 client.15579 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.932+0000 7f7a26addb80 10 client.15579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:40:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.932+0000 7f7a26addb80 20 client.15579 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.932+0000 7f7a26addb80 1 --2- 172.21.15.2:0/2326665006 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55d939097c70 0x55d93909a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.932+0000 7f7a26addb80 4 client.15579 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:40:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.932+0000 7f7a26addb80 1 -- 172.21.15.2:0/2326665006 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d938fc9910 con 0x55d939097c70 2023-09-13T04:40:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.932+0000 7f7a10f9b700 1 --2- 172.21.15.2:0/2326665006 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55d939097c70 0x55d93909a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.933+0000 7f7a10f9b700 1 --2- 172.21.15.2:0/2326665006 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55d939097c70 0x55d93909a060 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=369624dd9c44bbd1 server_cookie=d77c8bce058e4112 in_seq=0 out_seq=0 2023-09-13T04:40:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.933+0000 7f79ebfff700 10 client.15579 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.933+0000 7f79ebfff700 1 -- 172.21.15.2:0/2326665006 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d938fc9910 con 0x55d939097c70 2023-09-13T04:40:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.933+0000 7f79ebfff700 10 client.15579 handle_command_reply: tid=0 2023-09-13T04:40:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.933+0000 7f7a26addb80 1 client.15579 shutdown 2023-09-13T04:40:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.934+0000 7f7a26addb80 1 -- 172.21.15.2:0/2326665006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d939082be0 msgr2=0x55d939087db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:43.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.934+0000 7f7a26addb80 1 --2- 172.21.15.2:0/2326665006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d939082be0 0x55d939087db0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f79f8096ca0 tx=0x7f79f80caff0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.934+0000 7f7a26addb80 1 -- 172.21.15.2:0/2326665006 shutdown_connections 2023-09-13T04:40:43.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.934+0000 7f7a26addb80 1 --2- 172.21.15.2:0/2326665006 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55d939097c70 0x55d93909a060 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.934+0000 7f7a26addb80 1 --2- 172.21.15.2:0/2326665006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d939083500 0x55d9390882f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.934+0000 7f7a26addb80 1 --2- 172.21.15.2:0/2326665006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d93907e840 0x55d939087870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.934+0000 7f79ebfff700 0 client.15579 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:43.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.934+0000 7f7a26addb80 1 --2- 172.21.15.2:0/2326665006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d939082be0 0x55d939087db0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.934+0000 7f7a26addb80 1 -- 172.21.15.2:0/2326665006 >> 172.21.15.2:0/2326665006 conn(0x55d938fdcc20 msgr2=0x55d939084330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:43.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.934+0000 7f7a26addb80 1 -- 172.21.15.2:0/2326665006 shutdown_connections 2023-09-13T04:40:43.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.934+0000 7f7a26addb80 1 -- 172.21.15.2:0/2326665006 wait complete. 2023-09-13T04:40:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.934+0000 7f7a26addb80 20 client.15579 trim_cache size 0 max 16384 2023-09-13T04:40:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.949+0000 7f7a26addb80 1 Processor -- start 2023-09-13T04:40:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.949+0000 7f7a26addb80 1 -- start start 2023-09-13T04:40:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.950+0000 7f7a26addb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d93907e840 0x55d939073730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.950+0000 7f7a26addb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d939082be0 0x55d939088170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.950+0000 7f7a26addb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d939083500 0x55d9390936d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.950+0000 7f7a26addb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9390797e0 con 0x55d939083500 2023-09-13T04:40:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.950+0000 7f7a26addb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d939085e20 con 0x55d93907e840 2023-09-13T04:40:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.950+0000 7f7a26addb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9390948a0 con 0x55d939082be0 2023-09-13T04:40:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.950+0000 7f7a1179c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d939083500 0x55d9390936d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.950+0000 7f7a1179c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d939083500 0x55d9390936d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42068/0 (socket says 172.21.15.2:42068) 2023-09-13T04:40:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.950+0000 7f7a1179c700 1 -- 172.21.15.2:0/4219605630 learned_addr learned my addr 172.21.15.2:0/4219605630 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.950+0000 7f7a0bfff700 1 --2- 172.21.15.2:0/4219605630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d939082be0 0x55d939088170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.950+0000 7f7a10f9b700 1 --2- 172.21.15.2:0/4219605630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d93907e840 0x55d939073730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.950+0000 7f7a1179c700 1 -- 172.21.15.2:0/4219605630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d939082be0 msgr2=0x55d939088170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.950+0000 7f7a1179c700 1 --2- 172.21.15.2:0/4219605630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d939082be0 0x55d939088170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.951+0000 7f7a1179c700 1 -- 172.21.15.2:0/4219605630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d93907e840 msgr2=0x55d939073730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.951+0000 7f7a1179c700 1 --2- 172.21.15.2:0/4219605630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d93907e840 0x55d939073730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.951+0000 7f7a1179c700 1 -- 172.21.15.2:0/4219605630 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d938dfcd80 con 0x55d939083500 2023-09-13T04:40:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.951+0000 7f7a1179c700 1 --2- 172.21.15.2:0/4219605630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d939083500 0x55d9390936d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7a04011c70 tx=0x7f7a04009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a66749c6f8281c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.951+0000 7f79f77fe700 1 -- 172.21.15.2:0/4219605630 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0400e040 con 0x55d939083500 2023-09-13T04:40:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.951+0000 7f79f77fe700 1 -- 172.21.15.2:0/4219605630 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a0400f040 con 0x55d939083500 2023-09-13T04:40:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.951+0000 7f79f77fe700 1 -- 172.21.15.2:0/4219605630 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a04007e10 con 0x55d939083500 2023-09-13T04:40:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.952+0000 7f7a26addb80 1 -- 172.21.15.2:0/4219605630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d939083500 msgr2=0x55d9390936d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.952+0000 7f7a26addb80 1 --2- 172.21.15.2:0/4219605630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d939083500 0x55d9390936d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7a04011c70 tx=0x7f7a04009920 comp rx=0 tx=0).stop 2023-09-13T04:40:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.952+0000 7f7a26addb80 1 -- 172.21.15.2:0/4219605630 shutdown_connections 2023-09-13T04:40:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.952+0000 7f7a26addb80 1 --2- 172.21.15.2:0/4219605630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d939083500 0x55d9390936d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.952+0000 7f7a26addb80 1 --2- 172.21.15.2:0/4219605630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d93907e840 0x55d939073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.952+0000 7f7a26addb80 1 --2- 172.21.15.2:0/4219605630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d939082be0 0x55d939088170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.952+0000 7f7a26addb80 1 -- 172.21.15.2:0/4219605630 >> 172.21.15.2:0/4219605630 conn(0x55d938fdcc20 msgr2=0x55d939094190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.952+0000 7f7a26addb80 1 -- 172.21.15.2:0/4219605630 shutdown_connections 2023-09-13T04:40:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.952+0000 7f7a26addb80 1 -- 172.21.15.2:0/4219605630 wait complete. 2023-09-13T04:40:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.952+0000 7f7a26addb80 1 -- start start 2023-09-13T04:40:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.953+0000 7f7a26addb80 1 Processor -- start 2023-09-13T04:40:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.953+0000 7f7a26addb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d93907e840 0x55d93908cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.953+0000 7f7a26addb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d939082be0 0x55d93908d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.953+0000 7f7a26addb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d939083500 0x55d9390eaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.953+0000 7f7a26addb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d93908dd90 con 0x55d93907e840 2023-09-13T04:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.953+0000 7f7a26addb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9390eafb0 con 0x55d939082be0 2023-09-13T04:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.953+0000 7f7a26addb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9390eb120 con 0x55d939083500 2023-09-13T04:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.953+0000 7f7a26addb80 10 client.0 _ll_register_callbacks cb 0x55d938f6c620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.954+0000 7f7a10f9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d93907e840 0x55d93908cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.954+0000 7f7a10f9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d93907e840 0x55d93908cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42072/0 (socket says 172.21.15.2:42072) 2023-09-13T04:40:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.954+0000 7f7a10f9b700 1 -- 172.21.15.2:0/3162013590 learned_addr learned my addr 172.21.15.2:0/3162013590 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.954+0000 7f7a0bfff700 1 --2- 172.21.15.2:0/3162013590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d939082be0 0x55d93908d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.954+0000 7f7a1179c700 1 --2- 172.21.15.2:0/3162013590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d939083500 0x55d9390eaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.954+0000 7f7a10f9b700 1 -- 172.21.15.2:0/3162013590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d939083500 msgr2=0x55d9390eaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.954+0000 7f7a10f9b700 1 --2- 172.21.15.2:0/3162013590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d939083500 0x55d9390eaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.954+0000 7f7a10f9b700 1 -- 172.21.15.2:0/3162013590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d939082be0 msgr2=0x55d93908d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.954+0000 7f7a10f9b700 1 --2- 172.21.15.2:0/3162013590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d939082be0 0x55d93908d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.954+0000 7f7a10f9b700 1 -- 172.21.15.2:0/3162013590 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a04011920 con 0x55d93907e840 2023-09-13T04:40:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.954+0000 7f7a0bfff700 1 --2- 172.21.15.2:0/3162013590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d939082be0 0x55d93908d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.954+0000 7f7a10f9b700 1 --2- 172.21.15.2:0/3162013590 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d93907e840 0x55d93908cf60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7a00005c50 tx=0x7f7a00009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5895af1035dc0d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.954+0000 7f79ebfff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:40:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.955+0000 7f79ebfff700 1 -- 172.21.15.2:0/3162013590 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0002c040 con 0x55d93907e840 2023-09-13T04:40:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.955+0000 7f7a26addb80 1 -- 172.21.15.2:0/3162013590 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a00004cf0 con 0x55d93907e840 2023-09-13T04:40:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.955+0000 7f79ebfff700 1 -- 172.21.15.2:0/3162013590 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f7a000313b0 con 0x55d93907e840 2023-09-13T04:40:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.955+0000 7f79ebfff700 1 -- 172.21.15.2:0/3162013590 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a00011b50 con 0x55d93907e840 2023-09-13T04:40:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.955+0000 7f7a26addb80 10 client.15585 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.955+0000 7f7a26addb80 1 -- 172.21.15.2:0/3162013590 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d938de5c00 con 0x55d93907e840 2023-09-13T04:40:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.955+0000 7f79ebfff700 1 -- 172.21.15.2:0/3162013590 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a00004cf0 con 0x55d93907e840 2023-09-13T04:40:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.955+0000 7f79ebfff700 1 -- 172.21.15.2:0/3162013590 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7a000046f0 con 0x55d93907e840 2023-09-13T04:40:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.955+0000 7f7a26addb80 10 client.15585 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.955+0000 7f7a26addb80 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:40:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.955+0000 7f7a26addb80 20 client.15585 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.955+0000 7f7a26addb80 1 --2- 172.21.15.2:0/3162013590 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55d9390ec890 0x55d9390eecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.955+0000 7f7a26addb80 4 client.15585 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:40:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.955+0000 7f7a26addb80 1 -- 172.21.15.2:0/3162013590 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d938f89ff0 con 0x55d9390ec890 2023-09-13T04:40:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.956+0000 7f7a0bfff700 1 --2- 172.21.15.2:0/3162013590 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55d9390ec890 0x55d9390eecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.956+0000 7f7a0bfff700 1 --2- 172.21.15.2:0/3162013590 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55d9390ec890 0x55d9390eecb0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b18149857801545b server_cookie=d2be9c6c752760a in_seq=0 out_seq=0 2023-09-13T04:40:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.956+0000 7f79ebfff700 10 client.15585 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.964+0000 7f79ebfff700 1 -- 172.21.15.2:0/3162013590 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d938f89ff0 con 0x55d9390ec890 2023-09-13T04:40:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.964+0000 7f79ebfff700 10 client.15585 handle_command_reply: tid=0 2023-09-13T04:40:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.964+0000 7f7a26addb80 1 client.15585 shutdown 2023-09-13T04:40:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.965+0000 7f7a26addb80 1 -- 172.21.15.2:0/3162013590 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d93907e840 msgr2=0x55d93908cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.965+0000 7f7a26addb80 1 --2- 172.21.15.2:0/3162013590 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d93907e840 0x55d93908cf60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7a00005c50 tx=0x7f7a00009990 comp rx=0 tx=0).stop 2023-09-13T04:40:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.965+0000 7f7a26addb80 1 -- 172.21.15.2:0/3162013590 shutdown_connections 2023-09-13T04:40:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.965+0000 7f7a26addb80 1 --2- 172.21.15.2:0/3162013590 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55d9390ec890 0x55d9390eecb0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.965+0000 7f7a26addb80 1 --2- 172.21.15.2:0/3162013590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d939083500 0x55d9390eaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.965+0000 7f7a26addb80 1 --2- 172.21.15.2:0/3162013590 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d93907e840 0x55d93908cf60 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.965+0000 7f79ebfff700 0 client.15585 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.965+0000 7f7a26addb80 1 --2- 172.21.15.2:0/3162013590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d939082be0 0x55d93908d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.965+0000 7f7a26addb80 1 -- 172.21.15.2:0/3162013590 >> 172.21.15.2:0/3162013590 conn(0x55d938fdcc20 msgr2=0x55d939081ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.965+0000 7f7a26addb80 1 -- 172.21.15.2:0/3162013590 shutdown_connections 2023-09-13T04:40:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.965+0000 7f7a26addb80 1 -- 172.21.15.2:0/3162013590 wait complete. 2023-09-13T04:40:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.965+0000 7f7a26addb80 20 client.15585 trim_cache size 0 max 16384 2023-09-13T04:40:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:40:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:40:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:40:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:40:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.966+0000 7f7a131ff700 1 -- 172.21.15.2:0/2723893278 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f79fc0a0840 msgr2=0x7f79fc0a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.966+0000 7f7a131ff700 1 --2- 172.21.15.2:0/2723893278 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f79fc0a0840 0x7f79fc0a2d00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7a00009cb0 tx=0x7f7a00011720 comp rx=0 tx=0).stop 2023-09-13T04:40:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.966+0000 7f7a131ff700 1 -- 172.21.15.2:0/2723893278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a0c105ff0 msgr2=0x7f7a0c199590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.966+0000 7f7a131ff700 1 --2- 172.21.15.2:0/2723893278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a0c105ff0 0x7f7a0c199590 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f79f8000c00 tx=0x7f79f80047b0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.966+0000 7f7a131ff700 1 -- 172.21.15.2:0/2723893278 shutdown_connections 2023-09-13T04:40:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.966+0000 7f7a131ff700 1 --2- 172.21.15.2:0/2723893278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a0c19dc00 0x7f7a0c19e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.966+0000 7f7a131ff700 1 --2- 172.21.15.2:0/2723893278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a0c100ed0 0x7f7a0c199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.966+0000 7f7a131ff700 1 --2- 172.21.15.2:0/2723893278 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f79fc0a0840 0x7f79fc0a2d00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.966+0000 7f7a131ff700 1 --2- 172.21.15.2:0/2723893278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a0c105ff0 0x7f7a0c199590 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.966+0000 7f7a131ff700 1 -- 172.21.15.2:0/2723893278 >> 172.21.15.2:0/2723893278 conn(0x7f7a0c0fc750 msgr2=0x7f7a0c0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.966+0000 7f7a131ff700 1 -- 172.21.15.2:0/2723893278 shutdown_connections 2023-09-13T04:40:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:43.966+0000 7f7a131ff700 1 -- 172.21.15.2:0/2723893278 wait complete. 2023-09-13T04:40:44.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:43 smithi149 ceph-mon[133315]: pgmap v11: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 77 KiB/s wr, 41 op/s 2023-09-13T04:40:44.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:43 smithi156 ceph-mon[133675]: pgmap v11: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 77 KiB/s wr, 41 op/s 2023-09-13T04:40:44.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:43 smithi002 ceph-mon[97651]: pgmap v11: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 77 KiB/s wr, 41 op/s 2023-09-13T04:40:46.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:45 smithi149 ceph-mon[133315]: pgmap v12: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 77 KiB/s wr, 44 op/s 2023-09-13T04:40:46.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:45 smithi156 ceph-mon[133675]: pgmap v12: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 77 KiB/s wr, 44 op/s 2023-09-13T04:40:46.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:45 smithi002 ceph-mon[97651]: pgmap v12: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 77 KiB/s wr, 44 op/s 2023-09-13T04:40:47.953 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T04:40:48.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:47 smithi149 ceph-mon[133315]: pgmap v13: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2023-09-13T04:40:48.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.172+0000 7ffa111ff700 1 -- 172.21.15.2:0/2900874772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa0c100eb0 msgr2=0x7ffa0c101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:48.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.172+0000 7ffa111ff700 1 --2- 172.21.15.2:0/2900874772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa0c100eb0 0x7ffa0c101290 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009990 tx=0x7ff9fc009ca0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.172+0000 7ffa111ff700 1 -- 172.21.15.2:0/2900874772 shutdown_connections 2023-09-13T04:40:48.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.172+0000 7ffa111ff700 1 --2- 172.21.15.2:0/2900874772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa0c105fd0 0x7ffa0c10db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.172+0000 7ffa111ff700 1 --2- 172.21.15.2:0/2900874772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa0c100eb0 0x7ffa0c101290 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.172+0000 7ffa111ff700 1 --2- 172.21.15.2:0/2900874772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa0c101860 0x7ffa0c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.173+0000 7ffa111ff700 1 -- 172.21.15.2:0/2900874772 >> 172.21.15.2:0/2900874772 conn(0x7ffa0c0fc750 msgr2=0x7ffa0c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:48.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.173+0000 7ffa111ff700 1 -- 172.21.15.2:0/2900874772 shutdown_connections 2023-09-13T04:40:48.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.173+0000 7ffa111ff700 1 -- 172.21.15.2:0/2900874772 wait complete. 2023-09-13T04:40:48.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.174+0000 7ffa111ff700 1 Processor -- start 2023-09-13T04:40:48.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.174+0000 7ffa111ff700 1 -- start start 2023-09-13T04:40:48.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.174+0000 7ffa111ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa0c100eb0 0x7ffa0c072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.174+0000 7ffa111ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa0c105fd0 0x7ffa0c0734f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.174+0000 7ffa111ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa0c073a30 0x7ffa0c1a5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.174+0000 7ffa111ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c1a56a0 con 0x7ffa0c073a30 2023-09-13T04:40:48.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.174+0000 7ffa111ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c1a5810 con 0x7ffa0c105fd0 2023-09-13T04:40:48.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.174+0000 7ffa111ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c1a59d0 con 0x7ffa0c100eb0 2023-09-13T04:40:48.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.174+0000 7ffa0b59e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa0c073a30 0x7ffa0c1a5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.174+0000 7ffa0b59e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa0c073a30 0x7ffa0c1a5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59418/0 (socket says 172.21.15.2:59418) 2023-09-13T04:40:48.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.174+0000 7ffa0b59e700 1 -- 172.21.15.2:0/2504028302 learned_addr learned my addr 172.21.15.2:0/2504028302 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:48.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.174+0000 7ffa0a59c700 1 --2- 172.21.15.2:0/2504028302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa0c105fd0 0x7ffa0c0734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.175+0000 7ffa0ad9d700 1 --2- 172.21.15.2:0/2504028302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa0c100eb0 0x7ffa0c072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.175+0000 7ffa0b59e700 1 -- 172.21.15.2:0/2504028302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa0c100eb0 msgr2=0x7ffa0c072fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:48.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.175+0000 7ffa0b59e700 1 --2- 172.21.15.2:0/2504028302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa0c100eb0 0x7ffa0c072fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.175+0000 7ffa0b59e700 1 -- 172.21.15.2:0/2504028302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa0c105fd0 msgr2=0x7ffa0c0734f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:48.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.175+0000 7ffa0b59e700 1 --2- 172.21.15.2:0/2504028302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa0c105fd0 0x7ffa0c0734f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.175+0000 7ffa0b59e700 1 -- 172.21.15.2:0/2504028302 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9fc0095f0 con 0x7ffa0c073a30 2023-09-13T04:40:48.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.175+0000 7ffa0b59e700 1 --2- 172.21.15.2:0/2504028302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa0c073a30 0x7ffa0c1a5010 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400ef30 tx=0x7ffa0400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:48.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.175+0000 7ff9f3fff700 1 -- 172.21.15.2:0/2504028302 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa04018070 con 0x7ffa0c073a30 2023-09-13T04:40:48.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.175+0000 7ffa111ff700 1 -- 172.21.15.2:0/2504028302 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0c1a5c50 con 0x7ffa0c073a30 2023-09-13T04:40:48.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.175+0000 7ff9f3fff700 1 -- 172.21.15.2:0/2504028302 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa0400f040 con 0x7ffa0c073a30 2023-09-13T04:40:48.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.175+0000 7ff9f3fff700 1 -- 172.21.15.2:0/2504028302 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0400cd00 con 0x7ffa0c073a30 2023-09-13T04:40:48.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.176+0000 7ffa111ff700 1 -- 172.21.15.2:0/2504028302 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa0c1a6110 con 0x7ffa0c073a30 2023-09-13T04:40:48.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.180+0000 7ff9f3fff700 1 -- 172.21.15.2:0/2504028302 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 23) v1 ==== 135434+0+0 (secure 0 0 0) 0x7ffa04003a00 con 0x7ffa0c073a30 2023-09-13T04:40:48.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.181+0000 7ffa24a5ab80 1 Processor -- start 2023-09-13T04:40:48.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.181+0000 7ffa24a5ab80 1 -- start start 2023-09-13T04:40:48.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.182+0000 7ffa24a5ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f1a0fe0 0x55d50f1a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.182+0000 7ffa24a5ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50f1a1900 0x55d50f19b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.182+0000 7ffa24a5ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d50f19ba00 0x55d50f19be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.182+0000 7ffa24a5ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50f0f4780 con 0x55d50f1a0fe0 2023-09-13T04:40:48.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.182+0000 7ffa24a5ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50f0f20a0 con 0x55d50f19ba00 2023-09-13T04:40:48.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.182+0000 7ffa24a5ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50f0ed990 con 0x55d50f1a1900 2023-09-13T04:40:48.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.182+0000 7ff9f3fff700 1 --2- 172.21.15.2:0/2504028302 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7ff9f40a0840 0x7ff9f40a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.182+0000 7ff9f3fff700 1 -- 172.21.15.2:0/2504028302 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7ffa040cbf90 con 0x7ffa0c073a30 2023-09-13T04:40:48.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.182+0000 7ffa0ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50f1a1900 0x55d50f19b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.182+0000 7ffa0ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50f1a1900 0x55d50f19b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:38040/0 (socket says 172.21.15.2:38040) 2023-09-13T04:40:48.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.182+0000 7ffa0ad9d700 1 -- 172.21.15.2:0/1033516752 learned_addr learned my addr 172.21.15.2:0/1033516752 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:48.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.182+0000 7ffa0b59e700 1 --2- 172.21.15.2:0/2504028302 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7ff9f40a0840 0x7ff9f40a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.182+0000 7ffa0a59c700 1 --2- 172.21.15.2:0/1033516752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f1a0fe0 0x55d50f1a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.182+0000 7ffa0ad9d700 1 -- 172.21.15.2:0/1033516752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d50f19ba00 msgr2=0x55d50f19be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:48.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.182+0000 7ffa0a59c700 1 --2- 172.21.15.2:0/1033516752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d50f19ba00 0x55d50f19be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.182+0000 7ffa0ad9d700 1 --2- 172.21.15.2:0/1033516752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d50f19ba00 0x55d50f19be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.183+0000 7ffa0ad9d700 1 -- 172.21.15.2:0/1033516752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f1a0fe0 msgr2=0x55d50f1a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:48.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.183+0000 7ffa0ad9d700 1 --2- 172.21.15.2:0/1033516752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f1a0fe0 0x55d50f1a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.183+0000 7ffa0ad9d700 1 -- 172.21.15.2:0/1033516752 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d50ef6d630 con 0x55d50f1a1900 2023-09-13T04:40:48.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.183+0000 7ffa0a59c700 1 --2- 172.21.15.2:0/1033516752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d50f19ba00 0x55d50f19be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:40:48.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.183+0000 7ffa0b59e700 1 --2- 172.21.15.2:0/2504028302 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7ff9f40a0840 0x7ff9f40a2d00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffa040c9310 tx=0x7ffa04009040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:48.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.183+0000 7ffa0ad9d700 1 --2- 172.21.15.2:0/1033516752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50f1a1900 0x55d50f19b4c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc005ea0 tx=0x7ff9fc005ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf00cb831e61efa0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:48.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.183+0000 7ff9f17fa700 1 -- 172.21.15.2:0/1033516752 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc01d070 con 0x55d50f1a1900 2023-09-13T04:40:48.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.183+0000 7ff9f17fa700 1 -- 172.21.15.2:0/1033516752 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9fc01e040 con 0x55d50f1a1900 2023-09-13T04:40:48.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.183+0000 7ff9f17fa700 1 -- 172.21.15.2:0/1033516752 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0047d0 con 0x55d50f1a1900 2023-09-13T04:40:48.191 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:47 smithi156 ceph-mon[133675]: pgmap v13: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2023-09-13T04:40:48.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.184+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/1033516752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50f1a1900 msgr2=0x55d50f19b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:48.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.184+0000 7ffa24a5ab80 1 --2- 172.21.15.2:0/1033516752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50f1a1900 0x55d50f19b4c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc005ea0 tx=0x7ff9fc005ed0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.184+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/1033516752 shutdown_connections 2023-09-13T04:40:48.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.184+0000 7ffa24a5ab80 1 --2- 172.21.15.2:0/1033516752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d50f19ba00 0x55d50f19be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.184+0000 7ffa24a5ab80 1 --2- 172.21.15.2:0/1033516752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f1a0fe0 0x55d50f1a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.184+0000 7ffa24a5ab80 1 --2- 172.21.15.2:0/1033516752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50f1a1900 0x55d50f19b4c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.184+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/1033516752 >> 172.21.15.2:0/1033516752 conn(0x55d50f0f9c80 msgr2=0x55d50f195f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:48.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.184+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/1033516752 shutdown_connections 2023-09-13T04:40:48.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.184+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/1033516752 wait complete. 2023-09-13T04:40:48.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.185+0000 7ffa24a5ab80 1 -- start start 2023-09-13T04:40:48.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.185+0000 7ffa24a5ab80 1 Processor -- start 2023-09-13T04:40:48.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.185+0000 7ffa24a5ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f19ba00 0x55d50f1a7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.185+0000 7ffa24a5ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50f1a0fe0 0x55d50f1a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.185+0000 7ffa24a5ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d50f1a1900 0x55d50f1a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.185+0000 7ffa24a5ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50f08bbb0 con 0x55d50f19ba00 2023-09-13T04:40:48.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.185+0000 7ffa24a5ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50f08b8b0 con 0x55d50f1a1900 2023-09-13T04:40:48.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.185+0000 7ffa24a5ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50ef6f920 con 0x55d50f1a0fe0 2023-09-13T04:40:48.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.185+0000 7ffa24a5ab80 10 client.0 _ll_register_callbacks cb 0x55d50f089680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:48.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.186+0000 7ffa0a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f19ba00 0x55d50f1a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.186+0000 7ffa0a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f19ba00 0x55d50f1a7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59434/0 (socket says 172.21.15.2:59434) 2023-09-13T04:40:48.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.186+0000 7ffa0a59c700 1 -- 172.21.15.2:0/3936060571 learned_addr learned my addr 172.21.15.2:0/3936060571 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:48.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.186+0000 7ffa0ad9d700 1 --2- 172.21.15.2:0/3936060571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50f1a0fe0 0x55d50f1a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.186+0000 7ffa0a59c700 1 --2- 172.21.15.2:0/3936060571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d50f1a1900 0x55d50f1a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.186+0000 7ffa0a59c700 1 -- 172.21.15.2:0/3936060571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50f1a0fe0 msgr2=0x55d50f1a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:48.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.186+0000 7ffa0a59c700 1 --2- 172.21.15.2:0/3936060571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50f1a0fe0 0x55d50f1a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.186+0000 7ffa0a59c700 1 -- 172.21.15.2:0/3936060571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d50f1a1900 msgr2=0x55d50f1a52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:48.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.186+0000 7ffa0a59c700 1 --2- 172.21.15.2:0/3936060571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d50f1a1900 0x55d50f1a52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.186+0000 7ffa0a59c700 1 -- 172.21.15.2:0/3936060571 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa0c10e3c0 con 0x55d50f19ba00 2023-09-13T04:40:48.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.186+0000 7ffa0a59c700 1 --2- 172.21.15.2:0/3936060571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f19ba00 0x55d50f1a7910 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009740 tx=0x7ffa00009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc83bde5285f315 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:48.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.186+0000 7ff9eeff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:40:48.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.187+0000 7ff9eeff5700 1 -- 172.21.15.2:0/3936060571 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa000120d0 con 0x55d50f19ba00 2023-09-13T04:40:48.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.187+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/3936060571 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d50f18e670 con 0x55d50f19ba00 2023-09-13T04:40:48.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.187+0000 7ff9eeff5700 1 -- 172.21.15.2:0/3936060571 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7ffa00012230 con 0x55d50f19ba00 2023-09-13T04:40:48.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.187+0000 7ff9eeff5700 1 -- 172.21.15.2:0/3936060571 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa0000fb60 con 0x55d50f19ba00 2023-09-13T04:40:48.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.187+0000 7ffa24a5ab80 10 client.15597 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:48.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.187+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/3936060571 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d50f1057a0 con 0x55d50f19ba00 2023-09-13T04:40:48.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.187+0000 7ff9eeff5700 1 -- 172.21.15.2:0/3936060571 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d50f18e670 con 0x55d50f19ba00 2023-09-13T04:40:48.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.187+0000 7ff9eeff5700 1 -- 172.21.15.2:0/3936060571 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ffa00003980 con 0x55d50f19ba00 2023-09-13T04:40:48.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.187+0000 7ffa24a5ab80 10 client.15597 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:48.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.187+0000 7ffa24a5ab80 10 client.15597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:40:48.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.187+0000 7ffa24a5ab80 20 client.15597 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:48.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.187+0000 7ffa24a5ab80 1 --2- 172.21.15.2:0/3936060571 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55d50f1b5190 0x55d50f1b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.187+0000 7ffa24a5ab80 4 client.15597 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:40:48.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.187+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/3936060571 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d50f0e6970 con 0x55d50f1b5190 2023-09-13T04:40:48.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.188+0000 7ffa0ad9d700 1 --2- 172.21.15.2:0/3936060571 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55d50f1b5190 0x55d50f1b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.188+0000 7ffa0ad9d700 1 --2- 172.21.15.2:0/3936060571 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55d50f1b5190 0x55d50f1b75b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6efcbdbf6a6409d server_cookie=b68569ec07c72aa1 in_seq=0 out_seq=0 2023-09-13T04:40:48.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.188+0000 7ff9eeff5700 10 client.15597 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:48.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.189+0000 7ff9eeff5700 1 -- 172.21.15.2:0/3936060571 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d50f0e6970 con 0x55d50f1b5190 2023-09-13T04:40:48.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.189+0000 7ff9eeff5700 10 client.15597 handle_command_reply: tid=0 2023-09-13T04:40:48.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.189+0000 7ffa24a5ab80 1 client.15597 shutdown 2023-09-13T04:40:48.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.189+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/3936060571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f19ba00 msgr2=0x55d50f1a7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:48.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.189+0000 7ffa24a5ab80 1 --2- 172.21.15.2:0/3936060571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f19ba00 0x55d50f1a7910 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009740 tx=0x7ffa00009260 comp rx=0 tx=0).stop 2023-09-13T04:40:48.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.189+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/3936060571 shutdown_connections 2023-09-13T04:40:48.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.189+0000 7ffa24a5ab80 1 --2- 172.21.15.2:0/3936060571 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55d50f1b5190 0x55d50f1b75b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.189+0000 7ffa24a5ab80 1 --2- 172.21.15.2:0/3936060571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d50f1a1900 0x55d50f1a52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.189+0000 7ffa24a5ab80 1 --2- 172.21.15.2:0/3936060571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f19ba00 0x55d50f1a7910 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.189+0000 7ff9eeff5700 0 client.15597 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:48.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.189+0000 7ffa24a5ab80 1 --2- 172.21.15.2:0/3936060571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50f1a0fe0 0x55d50f1a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.189+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/3936060571 >> 172.21.15.2:0/3936060571 conn(0x55d50f0f9c80 msgr2=0x55d50f1907f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:48.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.190+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/3936060571 shutdown_connections 2023-09-13T04:40:48.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.190+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/3936060571 wait complete. 2023-09-13T04:40:48.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.190+0000 7ffa24a5ab80 20 client.15597 trim_cache size 0 max 16384 2023-09-13T04:40:48.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.207+0000 7ffa24a5ab80 1 Processor -- start 2023-09-13T04:40:48.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.208+0000 7ffa24a5ab80 1 -- start start 2023-09-13T04:40:48.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.208+0000 7ffa24a5ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50f19ba00 0x55d50f1ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.208+0000 7ffa24a5ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f1a0fe0 0x55d50f1908f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.208+0000 7ffa24a5ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d50f1a1900 0x55d50f1a74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.208+0000 7ffa24a5ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50f1ac6f0 con 0x55d50f1a0fe0 2023-09-13T04:40:48.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.208+0000 7ffa24a5ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50f19f8c0 con 0x55d50f1a1900 2023-09-13T04:40:48.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.208+0000 7ffa24a5ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50f19b4c0 con 0x55d50f19ba00 2023-09-13T04:40:48.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.208+0000 7ffa0ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f1a0fe0 0x55d50f1908f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.208+0000 7ffa0ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f1a0fe0 0x55d50f1908f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59446/0 (socket says 172.21.15.2:59446) 2023-09-13T04:40:48.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.208+0000 7ffa0ad9d700 1 -- 172.21.15.2:0/4047531839 learned_addr learned my addr 172.21.15.2:0/4047531839 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:48.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.208+0000 7ffa0a59c700 1 --2- 172.21.15.2:0/4047531839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50f19ba00 0x55d50f1ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.208+0000 7ffa0a59c700 1 --2- 172.21.15.2:0/4047531839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d50f1a1900 0x55d50f1a74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.208+0000 7ffa0ad9d700 1 -- 172.21.15.2:0/4047531839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50f19ba00 msgr2=0x55d50f1ac1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:48.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.208+0000 7ffa0ad9d700 1 --2- 172.21.15.2:0/4047531839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50f19ba00 0x55d50f1ac1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.208+0000 7ffa0ad9d700 1 -- 172.21.15.2:0/4047531839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d50f1a1900 msgr2=0x55d50f1a74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:48.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.208+0000 7ffa0ad9d700 1 --2- 172.21.15.2:0/4047531839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d50f1a1900 0x55d50f1a74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.208+0000 7ffa0ad9d700 1 -- 172.21.15.2:0/4047531839 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d50ef19de0 con 0x55d50f1a0fe0 2023-09-13T04:40:48.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.209+0000 7ffa0a59c700 1 --2- 172.21.15.2:0/4047531839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d50f1a1900 0x55d50f1a74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:40:48.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.209+0000 7ffa0ad9d700 1 --2- 172.21.15.2:0/4047531839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f1a0fe0 0x55d50f1908f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc005850 tx=0x7ff9fc01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42cf21ba4e2338e4 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:48.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.209+0000 7ff9f1ffb700 1 -- 172.21.15.2:0/4047531839 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc01d040 con 0x55d50f1a0fe0 2023-09-13T04:40:48.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.209+0000 7ff9f1ffb700 1 -- 172.21.15.2:0/4047531839 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9fc0055d0 con 0x55d50f1a0fe0 2023-09-13T04:40:48.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.209+0000 7ff9f1ffb700 1 -- 172.21.15.2:0/4047531839 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc00bdf0 con 0x55d50f1a0fe0 2023-09-13T04:40:48.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.209+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/4047531839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f1a0fe0 msgr2=0x55d50f1908f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:48.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.209+0000 7ffa24a5ab80 1 --2- 172.21.15.2:0/4047531839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f1a0fe0 0x55d50f1908f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc005850 tx=0x7ff9fc01a040 comp rx=0 tx=0).stop 2023-09-13T04:40:48.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.210+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/4047531839 shutdown_connections 2023-09-13T04:40:48.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.210+0000 7ffa24a5ab80 1 --2- 172.21.15.2:0/4047531839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d50f1a1900 0x55d50f1a74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.210+0000 7ffa24a5ab80 1 --2- 172.21.15.2:0/4047531839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50f19ba00 0x55d50f1ac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.210+0000 7ffa24a5ab80 1 --2- 172.21.15.2:0/4047531839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f1a0fe0 0x55d50f1908f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.210+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/4047531839 >> 172.21.15.2:0/4047531839 conn(0x55d50f0f9c80 msgr2=0x55d50f19ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:48.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.210+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/4047531839 shutdown_connections 2023-09-13T04:40:48.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.210+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/4047531839 wait complete. 2023-09-13T04:40:48.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.210+0000 7ffa24a5ab80 1 -- start start 2023-09-13T04:40:48.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.210+0000 7ffa24a5ab80 1 Processor -- start 2023-09-13T04:40:48.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.211+0000 7ffa24a5ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50f19ba00 0x55d50f1aa390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.211+0000 7ffa24a5ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f1a0fe0 0x55d50f1aa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.211+0000 7ffa24a5ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d50f1a1900 0x55d50f207fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.211+0000 7ffa24a5ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50f208500 con 0x55d50f1a0fe0 2023-09-13T04:40:48.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.211+0000 7ffa24a5ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50f208640 con 0x55d50f1a1900 2023-09-13T04:40:48.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.211+0000 7ffa24a5ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50f208780 con 0x55d50f19ba00 2023-09-13T04:40:48.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.211+0000 7ffa24a5ab80 10 client.0 _ll_register_callbacks cb 0x55d50f089680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:48.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.211+0000 7ffa0ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f1a0fe0 0x55d50f1aa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.211+0000 7ffa0a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50f19ba00 0x55d50f1aa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.211+0000 7ffa0ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f1a0fe0 0x55d50f1aa8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59462/0 (socket says 172.21.15.2:59462) 2023-09-13T04:40:48.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.211+0000 7ffa0ad9d700 1 -- 172.21.15.2:0/456480296 learned_addr learned my addr 172.21.15.2:0/456480296 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:48.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.211+0000 7ffa0a59c700 1 --2- 172.21.15.2:0/456480296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d50f1a1900 0x55d50f207fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.212+0000 7ffa0ad9d700 1 -- 172.21.15.2:0/456480296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50f19ba00 msgr2=0x55d50f1aa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:48.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.212+0000 7ffa0ad9d700 1 --2- 172.21.15.2:0/456480296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50f19ba00 0x55d50f1aa390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.212+0000 7ffa0ad9d700 1 -- 172.21.15.2:0/456480296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d50f1a1900 msgr2=0x55d50f207fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:48.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.212+0000 7ffa0ad9d700 1 --2- 172.21.15.2:0/456480296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d50f1a1900 0x55d50f207fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.212+0000 7ffa0ad9d700 1 -- 172.21.15.2:0/456480296 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d50ef6d630 con 0x55d50f1a0fe0 2023-09-13T04:40:48.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.212+0000 7ffa0ad9d700 1 --2- 172.21.15.2:0/456480296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f1a0fe0 0x55d50f1aa8d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009990 tx=0x7ff9fc00b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d95562638415d19 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:48.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.212+0000 7ff9eeff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:40:48.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.212+0000 7ff9eeff5700 1 -- 172.21.15.2:0/456480296 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc01d040 con 0x55d50f1a0fe0 2023-09-13T04:40:48.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.212+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/456480296 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9fc01e040 con 0x55d50f1a0fe0 2023-09-13T04:40:48.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.212+0000 7ff9eeff5700 1 -- 172.21.15.2:0/456480296 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7ff9fc004500 con 0x55d50f1a0fe0 2023-09-13T04:40:48.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.212+0000 7ff9eeff5700 1 -- 172.21.15.2:0/456480296 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9fc0046c0 con 0x55d50f1a0fe0 2023-09-13T04:40:48.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.212+0000 7ffa24a5ab80 10 client.15603 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:48.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.212+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/456480296 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d50ef02c60 con 0x55d50f1a0fe0 2023-09-13T04:40:48.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.213+0000 7ff9eeff5700 1 -- 172.21.15.2:0/456480296 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9fc047080 con 0x55d50f1a0fe0 2023-09-13T04:40:48.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.213+0000 7ff9eeff5700 1 -- 172.21.15.2:0/456480296 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff9fc00fa20 con 0x55d50f1a0fe0 2023-09-13T04:40:48.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.213+0000 7ffa24a5ab80 10 client.15603 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:48.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.213+0000 7ffa24a5ab80 10 client.15603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:40:48.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.213+0000 7ffa24a5ab80 20 client.15603 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:48.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.213+0000 7ffa24a5ab80 1 --2- 172.21.15.2:0/456480296 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55d50f209b10 0x55d50f20bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.213+0000 7ffa24a5ab80 4 client.15603 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-09-13T04:40:48.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.213+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/456480296 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d50f0a7050 con 0x55d50f209b10 2023-09-13T04:40:48.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.213+0000 7ffa0a59c700 1 --2- 172.21.15.2:0/456480296 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55d50f209b10 0x55d50f20bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.214+0000 7ffa0a59c700 1 --2- 172.21.15.2:0/456480296 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55d50f209b10 0x55d50f20bf30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b90400dc879be035 server_cookie=76a820e3556fae38 in_seq=0 out_seq=0 2023-09-13T04:40:48.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.214+0000 7ff9eeff5700 10 client.15603 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:48.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.214+0000 7ff9eeff5700 1 -- 172.21.15.2:0/456480296 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d50f0a7050 con 0x55d50f209b10 2023-09-13T04:40:48.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.214+0000 7ff9eeff5700 10 client.15603 handle_command_reply: tid=0 2023-09-13T04:40:48.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.214+0000 7ffa24a5ab80 1 client.15603 shutdown 2023-09-13T04:40:48.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.214+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/456480296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f1a0fe0 msgr2=0x55d50f1aa8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:48.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.214+0000 7ffa24a5ab80 1 --2- 172.21.15.2:0/456480296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f1a0fe0 0x55d50f1aa8d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009990 tx=0x7ff9fc00b9e0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.214+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/456480296 shutdown_connections 2023-09-13T04:40:48.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.214+0000 7ffa24a5ab80 1 --2- 172.21.15.2:0/456480296 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55d50f209b10 0x55d50f20bf30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.214+0000 7ffa24a5ab80 1 --2- 172.21.15.2:0/456480296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d50f1a1900 0x55d50f207fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.214+0000 7ffa24a5ab80 1 --2- 172.21.15.2:0/456480296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50f19ba00 0x55d50f1aa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.214+0000 7ff9eeff5700 0 client.15603 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:48.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.214+0000 7ffa24a5ab80 1 --2- 172.21.15.2:0/456480296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d50f1a0fe0 0x55d50f1aa8d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.214+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/456480296 >> 172.21.15.2:0/456480296 conn(0x55d50f0f9c80 msgr2=0x55d50f19fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:48.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.215+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/456480296 shutdown_connections 2023-09-13T04:40:48.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.215+0000 7ffa24a5ab80 1 -- 172.21.15.2:0/456480296 wait complete. 2023-09-13T04:40:48.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.215+0000 7ffa24a5ab80 20 client.15603 trim_cache size 0 max 16384 2023-09-13T04:40:48.230 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:40:48.231 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2023-09-13T04:40:48.231 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2023-09-13T04:40:48.231 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:40:48.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.215+0000 7ffa111ff700 1 -- 172.21.15.2:0/2504028302 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7ff9f40a0840 msgr2=0x7ff9f40a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:48.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.215+0000 7ffa111ff700 1 --2- 172.21.15.2:0/2504028302 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7ff9f40a0840 0x7ff9f40a2d00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffa040c9310 tx=0x7ffa04009040 comp rx=0 tx=0).stop 2023-09-13T04:40:48.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.215+0000 7ffa111ff700 1 -- 172.21.15.2:0/2504028302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa0c073a30 msgr2=0x7ffa0c1a5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:48.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.215+0000 7ffa111ff700 1 --2- 172.21.15.2:0/2504028302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa0c073a30 0x7ffa0c1a5010 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400ef30 tx=0x7ffa0400c3a0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.215+0000 7ffa111ff700 1 -- 172.21.15.2:0/2504028302 shutdown_connections 2023-09-13T04:40:48.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.215+0000 7ffa111ff700 1 --2- 172.21.15.2:0/2504028302 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7ff9f40a0840 0x7ff9f40a2d00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.215+0000 7ffa111ff700 1 --2- 172.21.15.2:0/2504028302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa0c073a30 0x7ffa0c1a5010 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.215+0000 7ffa111ff700 1 --2- 172.21.15.2:0/2504028302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa0c100eb0 0x7ffa0c072fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.215+0000 7ffa111ff700 1 --2- 172.21.15.2:0/2504028302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa0c105fd0 0x7ffa0c0734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.215+0000 7ffa111ff700 1 -- 172.21.15.2:0/2504028302 >> 172.21.15.2:0/2504028302 conn(0x7ffa0c0fc750 msgr2=0x7ffa0c1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:48.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.216+0000 7ffa111ff700 1 -- 172.21.15.2:0/2504028302 shutdown_connections 2023-09-13T04:40:48.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.216+0000 7ffa111ff700 1 -- 172.21.15.2:0/2504028302 wait complete. 2023-09-13T04:40:48.235 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:47 smithi002 ceph-mon[97651]: pgmap v13: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2023-09-13T04:40:48.236 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-13T04:40:48.236 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:40:48.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.440+0000 7f6df71ff700 1 -- 172.21.15.2:0/1584132946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6df0100eb0 msgr2=0x7f6df0101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:48.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.440+0000 7f6df71ff700 1 --2- 172.21.15.2:0/1584132946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6df0100eb0 0x7f6df0101290 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6de40098f0 tx=0x7f6de4009c00 comp rx=0 tx=0).stop 2023-09-13T04:40:48.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.440+0000 7f6df71ff700 1 -- 172.21.15.2:0/1584132946 shutdown_connections 2023-09-13T04:40:48.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.440+0000 7f6df71ff700 1 --2- 172.21.15.2:0/1584132946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6df0105fd0 0x7f6df010db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.440+0000 7f6df71ff700 1 --2- 172.21.15.2:0/1584132946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6df0100eb0 0x7f6df0101290 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.440+0000 7f6df71ff700 1 --2- 172.21.15.2:0/1584132946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6df0101860 0x7f6df01058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.440+0000 7f6df71ff700 1 -- 172.21.15.2:0/1584132946 >> 172.21.15.2:0/1584132946 conn(0x7f6df00fc6d0 msgr2=0x7f6df00feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:48.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.440+0000 7f6df71ff700 1 -- 172.21.15.2:0/1584132946 shutdown_connections 2023-09-13T04:40:48.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.440+0000 7f6df71ff700 1 -- 172.21.15.2:0/1584132946 wait complete. 2023-09-13T04:40:48.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.441+0000 7f6df71ff700 1 Processor -- start 2023-09-13T04:40:48.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.441+0000 7f6df71ff700 1 -- start start 2023-09-13T04:40:48.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.441+0000 7f6df71ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6df0100eb0 0x7f6df0198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.441+0000 7f6df71ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6df0105fd0 0x7f6df0199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.441+0000 7f6df71ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6df019d970 0x7f6df019ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.441+0000 7f6df71ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df019e3f0 con 0x7f6df0100eb0 2023-09-13T04:40:48.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.441+0000 7f6df71ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df019e560 con 0x7f6df019d970 2023-09-13T04:40:48.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.441+0000 7f6df71ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df019e720 con 0x7f6df0105fd0 2023-09-13T04:40:48.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.442+0000 7f6df4f9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6df0100eb0 0x7f6df0198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.442+0000 7f6df579c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6df019d970 0x7f6df019ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.442+0000 7f6deffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6df0105fd0 0x7f6df0199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.442+0000 7f6df579c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6df019d970 0x7f6df019ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:58528/0 (socket says 172.21.15.2:58528) 2023-09-13T04:40:48.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.442+0000 7f6df579c700 1 -- 172.21.15.2:0/1038571764 learned_addr learned my addr 172.21.15.2:0/1038571764 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:48.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.442+0000 7f6df579c700 1 -- 172.21.15.2:0/1038571764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6df0105fd0 msgr2=0x7f6df0199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:48.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.442+0000 7f6df579c700 1 --2- 172.21.15.2:0/1038571764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6df0105fd0 0x7f6df0199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.442+0000 7f6df579c700 1 -- 172.21.15.2:0/1038571764 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6df0100eb0 msgr2=0x7f6df0198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:48.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.442+0000 7f6df579c700 1 --2- 172.21.15.2:0/1038571764 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6df0100eb0 0x7f6df0198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.442+0000 7f6df579c700 1 -- 172.21.15.2:0/1038571764 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de4009520 con 0x7f6df019d970 2023-09-13T04:40:48.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.442+0000 7f6df4f9b700 1 --2- 172.21.15.2:0/1038571764 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6df0100eb0 0x7f6df0198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:48.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.442+0000 7f6df579c700 1 --2- 172.21.15.2:0/1038571764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6df019d970 0x7f6df019ddf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6de8007ee0 tx=0x7f6de800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:48.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.443+0000 7f6dedffb700 1 -- 172.21.15.2:0/1038571764 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de8017070 con 0x7f6df019d970 2023-09-13T04:40:48.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.443+0000 7f6df71ff700 1 -- 172.21.15.2:0/1038571764 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df01a5c80 con 0x7f6df019d970 2023-09-13T04:40:48.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.443+0000 7f6dedffb700 1 -- 172.21.15.2:0/1038571764 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6de800f040 con 0x7f6df019d970 2023-09-13T04:40:48.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.443+0000 7f6dedffb700 1 -- 172.21.15.2:0/1038571764 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de8005740 con 0x7f6df019d970 2023-09-13T04:40:48.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.443+0000 7f6df71ff700 1 -- 172.21.15.2:0/1038571764 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df01a6110 con 0x7f6df019d970 2023-09-13T04:40:48.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.444+0000 7f6df71ff700 1 -- 172.21.15.2:0/1038571764 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df0067ad0 con 0x7f6df019d970 2023-09-13T04:40:48.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.446+0000 7f6dedffb700 1 -- 172.21.15.2:0/1038571764 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 23) v1 ==== 135434+0+0 (secure 0 0 0) 0x7f6de800dbd0 con 0x7f6df019d970 2023-09-13T04:40:48.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.447+0000 7f6dedffb700 1 --2- 172.21.15.2:0/1038571764 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f6de00a0880 0x7f6de00a2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.447+0000 7f6dedffb700 1 -- 172.21.15.2:0/1038571764 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f6de80cc140 con 0x7f6df019d970 2023-09-13T04:40:48.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.448+0000 7f6df4f9b700 1 --2- 172.21.15.2:0/1038571764 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f6de00a0880 0x7f6de00a2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.448+0000 7f6df4f9b700 1 --2- 172.21.15.2:0/1038571764 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f6de00a0880 0x7f6de00a2d40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6de4005e90 tx=0x7f6de4019040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:48.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.450+0000 7f6dedffb700 1 -- 172.21.15.2:0/1038571764 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6de8095450 con 0x7f6df019d970 2023-09-13T04:40:48.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.706+0000 7f6df71ff700 1 -- 172.21.15.2:0/1038571764 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6df0101a70 con 0x7f6df019d970 2023-09-13T04:40:48.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.707+0000 7f6dedffb700 1 -- 172.21.15.2:0/1038571764 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11303 (secure 0 0 0) 0x7f6de8098a70 con 0x7f6df019d970 2023-09-13T04:40:48.708 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:40:48.709 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2469307307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2469307307},{"type":"v1","addr":"172.21.15.2:6839","nonce":2469307307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T04:37:59.110701+0000","modified":"2023-09-13T04:38:26.610260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24554,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/846579077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":846579077},{"type":"v1","addr":"172.21.15.156:6835","nonce":846579077}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1896833690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1896833690},{"type":"v1","addr":"172.21.15.149:6837","nonce":1896833690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/524563090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":524563090},{"type":"v1","addr":"172.21.15.156:6837","nonce":524563090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6839/1785833227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1785833227},{"type":"v1","addr":"172.21.15.156:6839","nonce":1785833227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.2:6841/482545391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":482545391},{"type":"v1","addr":"172.21.15.2:6841","nonce":482545391}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/389167841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":389167841},{"type":"v1","addr":"172.21.15.149:6839","nonce":389167841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T04:40:48.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.709+0000 7f6df71ff700 1 -- 172.21.15.2:0/1038571764 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f6de00a0880 msgr2=0x7f6de00a2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:48.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.709+0000 7f6df71ff700 1 --2- 172.21.15.2:0/1038571764 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f6de00a0880 0x7f6de00a2d40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6de4005e90 tx=0x7f6de4019040 comp rx=0 tx=0).stop 2023-09-13T04:40:48.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.709+0000 7f6df71ff700 1 -- 172.21.15.2:0/1038571764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6df019d970 msgr2=0x7f6df019ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:48.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.709+0000 7f6df71ff700 1 --2- 172.21.15.2:0/1038571764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6df019d970 0x7f6df019ddf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6de8007ee0 tx=0x7f6de800d3b0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.709+0000 7f6df71ff700 1 -- 172.21.15.2:0/1038571764 shutdown_connections 2023-09-13T04:40:48.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.709+0000 7f6df71ff700 1 --2- 172.21.15.2:0/1038571764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6df019d970 0x7f6df019ddf0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.709+0000 7f6df71ff700 1 --2- 172.21.15.2:0/1038571764 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f6de00a0880 0x7f6de00a2d40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.709+0000 7f6df71ff700 1 --2- 172.21.15.2:0/1038571764 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6df0100eb0 0x7f6df0198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.709+0000 7f6df71ff700 1 --2- 172.21.15.2:0/1038571764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6df0105fd0 0x7f6df0199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.709+0000 7f6df71ff700 1 -- 172.21.15.2:0/1038571764 >> 172.21.15.2:0/1038571764 conn(0x7f6df00fc6d0 msgr2=0x7f6df010ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:48.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.709+0000 7f6df71ff700 1 -- 172.21.15.2:0/1038571764 shutdown_connections 2023-09-13T04:40:48.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.709+0000 7f6df71ff700 1 -- 172.21.15.2:0/1038571764 wait complete. 2023-09-13T04:40:48.714 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 23 2023-09-13T04:40:48.730 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-13T04:40:48.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.938+0000 7fc4c8dff700 1 -- 172.21.15.2:0/4089566018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4c410d480 msgr2=0x7fc4c410f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:48.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.938+0000 7fc4c8dff700 1 --2- 172.21.15.2:0/4089566018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4c410d480 0x7fc4c410f8f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009f90 tx=0x7fc4bc00b330 comp rx=0 tx=0).stop 2023-09-13T04:40:48.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.938+0000 7fc4c8dff700 1 -- 172.21.15.2:0/4089566018 shutdown_connections 2023-09-13T04:40:48.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.938+0000 7fc4c8dff700 1 --2- 172.21.15.2:0/4089566018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4c410d480 0x7fc4c410f8f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.938+0000 7fc4c8dff700 1 --2- 172.21.15.2:0/4089566018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4c40730e0 0x7fc4c40734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.938+0000 7fc4c8dff700 1 --2- 172.21.15.2:0/4089566018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4c4073a00 0x7fc4c410cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.938+0000 7fc4c8dff700 1 -- 172.21.15.2:0/4089566018 >> 172.21.15.2:0/4089566018 conn(0x7fc4c40fc750 msgr2=0x7fc4c40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:48.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.938+0000 7fc4c8dff700 1 -- 172.21.15.2:0/4089566018 shutdown_connections 2023-09-13T04:40:48.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.938+0000 7fc4c8dff700 1 -- 172.21.15.2:0/4089566018 wait complete. 2023-09-13T04:40:48.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.939+0000 7fc4c8dff700 1 Processor -- start 2023-09-13T04:40:48.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.939+0000 7fc4c8dff700 1 -- start start 2023-09-13T04:40:48.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.939+0000 7fc4c8dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4c40730e0 0x7fc4c41990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.940+0000 7fc4c8dff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4c4073a00 0x7fc4c41995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.940+0000 7fc4c8dff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4c410d480 0x7fc4c419da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.940+0000 7fc4c8dff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c419e090 con 0x7fc4c4073a00 2023-09-13T04:40:48.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.940+0000 7fc4c8dff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c419e200 con 0x7fc4c410d480 2023-09-13T04:40:48.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.940+0000 7fc4c8dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c419e3c0 con 0x7fc4c40730e0 2023-09-13T04:40:48.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.940+0000 7fc4c259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4c40730e0 0x7fc4c41990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.940+0000 7fc4c259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4c40730e0 0x7fc4c41990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:38094/0 (socket says 172.21.15.2:38094) 2023-09-13T04:40:48.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.940+0000 7fc4bbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4c4073a00 0x7fc4c41995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.940+0000 7fc4c259c700 1 -- 172.21.15.2:0/3535461969 learned_addr learned my addr 172.21.15.2:0/3535461969 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:48.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.940+0000 7fc4c2d9d700 1 --2- 172.21.15.2:0/3535461969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4c410d480 0x7fc4c419da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.940+0000 7fc4c259c700 1 -- 172.21.15.2:0/3535461969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4c410d480 msgr2=0x7fc4c419da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:48.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.940+0000 7fc4c259c700 1 --2- 172.21.15.2:0/3535461969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4c410d480 0x7fc4c419da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.940+0000 7fc4c259c700 1 -- 172.21.15.2:0/3535461969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4c4073a00 msgr2=0x7fc4c41995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:48.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.940+0000 7fc4c259c700 1 --2- 172.21.15.2:0/3535461969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4c4073a00 0x7fc4c41995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.940+0000 7fc4c259c700 1 -- 172.21.15.2:0/3535461969 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4bc009c40 con 0x7fc4c40730e0 2023-09-13T04:40:48.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.940+0000 7fc4bbfff700 1 --2- 172.21.15.2:0/3535461969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4c4073a00 0x7fc4c41995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:48.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.941+0000 7fc4c259c700 1 --2- 172.21.15.2:0/3535461969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4c40730e0 0x7fc4c41990b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400e9c0 tx=0x7fc4b400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:48.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.941+0000 7fc4bb7fe700 1 -- 172.21.15.2:0/3535461969 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b400cbd0 con 0x7fc4c40730e0 2023-09-13T04:40:48.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.941+0000 7fc4bb7fe700 1 -- 172.21.15.2:0/3535461969 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4b400eeb0 con 0x7fc4c40730e0 2023-09-13T04:40:48.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.941+0000 7fc4c8dff700 1 -- 172.21.15.2:0/3535461969 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c419e6a0 con 0x7fc4c40730e0 2023-09-13T04:40:48.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.941+0000 7fc4bb7fe700 1 -- 172.21.15.2:0/3535461969 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4005580 con 0x7fc4c40730e0 2023-09-13T04:40:48.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.941+0000 7fc4c8dff700 1 -- 172.21.15.2:0/3535461969 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c41a5ee0 con 0x7fc4c40730e0 2023-09-13T04:40:48.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.944+0000 7fc4bb7fe700 1 -- 172.21.15.2:0/3535461969 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 23) v1 ==== 135434+0+0 (secure 0 0 0) 0x7fc4b4004750 con 0x7fc4c40730e0 2023-09-13T04:40:48.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.945+0000 7fc4dc7d3b80 1 Processor -- start 2023-09-13T04:40:48.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.946+0000 7fc4bb7fe700 1 --2- 172.21.15.2:0/3535461969 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc4b00a0840 0x7fc4b00a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.946+0000 7fc4dc7d3b80 1 -- start start 2023-09-13T04:40:48.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.946+0000 7fc4bb7fe700 1 -- 172.21.15.2:0/3535461969 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fc4b4014070 con 0x7fc4c40730e0 2023-09-13T04:40:48.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.946+0000 7fc4dc7d3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412accfe0 0x562412acd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.946+0000 7fc4dc7d3b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 0x562412ac74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.946+0000 7fc4c2d9d700 1 --2- 172.21.15.2:0/3535461969 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc4b00a0840 0x7fc4b00a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.946+0000 7fc4dc7d3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562412ac7a00 0x562412ac7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.946+0000 7fc4dc7d3b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562412a20780 con 0x562412acd900 2023-09-13T04:40:48.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.946+0000 7fc4dc7d3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562412a1e0a0 con 0x562412accfe0 2023-09-13T04:40:48.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.946+0000 7fc4dc7d3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562412a19990 con 0x562412ac7a00 2023-09-13T04:40:48.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.946+0000 7fc4bbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 0x562412ac74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.946+0000 7fc4c259c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412accfe0 0x562412acd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.946+0000 7fc4c259c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412accfe0 0x562412acd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:58560/0 (socket says 172.21.15.2:58560) 2023-09-13T04:40:48.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.946+0000 7fc4c259c700 1 -- 172.21.15.2:0/2262626080 learned_addr learned my addr 172.21.15.2:0/2262626080 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:48.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.946+0000 7fc4bbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 0x562412ac74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59502/0 (socket says 172.21.15.2:59502) 2023-09-13T04:40:48.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.946+0000 7fc4c2d9d700 1 --2- 172.21.15.2:0/2262626080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562412ac7a00 0x562412ac7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.947+0000 7fc4bbfff700 1 -- 172.21.15.2:0/2262626080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562412ac7a00 msgr2=0x562412ac7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:48.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.947+0000 7fc4bbfff700 1 --2- 172.21.15.2:0/2262626080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562412ac7a00 0x562412ac7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.947+0000 7fc4bbfff700 1 -- 172.21.15.2:0/2262626080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412accfe0 msgr2=0x562412acd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:48.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.947+0000 7fc4bbfff700 1 --2- 172.21.15.2:0/2262626080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412accfe0 0x562412acd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.947+0000 7fc4bbfff700 1 -- 172.21.15.2:0/2262626080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562412899630 con 0x562412acd900 2023-09-13T04:40:48.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.947+0000 7fc4c259c700 1 --2- 172.21.15.2:0/2262626080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412accfe0 0x562412acd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:48.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.947+0000 7fc4c2d9d700 1 --2- 172.21.15.2:0/3535461969 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc4b00a0840 0x7fc4b00a2d00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009550 tx=0x7fc4bc007ad0 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:48.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.947+0000 7fc4bbfff700 1 --2- 172.21.15.2:0/2262626080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 0x562412ac74c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009fb0 tx=0x7fc4ac011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854068b47a34f5e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:48.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.947+0000 7fc4b8ff9700 1 -- 172.21.15.2:0/2262626080 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac004230 con 0x562412acd900 2023-09-13T04:40:48.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.947+0000 7fc4b8ff9700 1 -- 172.21.15.2:0/2262626080 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4ac004cf0 con 0x562412acd900 2023-09-13T04:40:48.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.947+0000 7fc4b8ff9700 1 -- 172.21.15.2:0/2262626080 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac0056a0 con 0x562412acd900 2023-09-13T04:40:48.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.948+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/2262626080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 msgr2=0x562412ac74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:48.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.948+0000 7fc4dc7d3b80 1 --2- 172.21.15.2:0/2262626080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 0x562412ac74c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009fb0 tx=0x7fc4ac011720 comp rx=0 tx=0).stop 2023-09-13T04:40:48.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.948+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/2262626080 shutdown_connections 2023-09-13T04:40:48.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.948+0000 7fc4dc7d3b80 1 --2- 172.21.15.2:0/2262626080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562412ac7a00 0x562412ac7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.948+0000 7fc4dc7d3b80 1 --2- 172.21.15.2:0/2262626080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412accfe0 0x562412acd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.948+0000 7fc4dc7d3b80 1 --2- 172.21.15.2:0/2262626080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 0x562412ac74c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.948+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/2262626080 >> 172.21.15.2:0/2262626080 conn(0x562412a25c80 msgr2=0x562412ac1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:48.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.948+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/2262626080 shutdown_connections 2023-09-13T04:40:48.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.948+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/2262626080 wait complete. 2023-09-13T04:40:48.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.949+0000 7fc4dc7d3b80 1 -- start start 2023-09-13T04:40:48.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.949+0000 7fc4dc7d3b80 1 Processor -- start 2023-09-13T04:40:48.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.949+0000 7fc4dc7d3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412ac7a00 0x562412ad0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.949+0000 7fc4dc7d3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562412accfe0 0x562412ad10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.949+0000 7fc4dc7d3b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 0x562412ad1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.949+0000 7fc4dc7d3b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624129b7bb0 con 0x562412acd900 2023-09-13T04:40:48.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.949+0000 7fc4dc7d3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624129b78b0 con 0x562412ac7a00 2023-09-13T04:40:48.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.949+0000 7fc4dc7d3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56241289b920 con 0x562412accfe0 2023-09-13T04:40:48.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.949+0000 7fc4dc7d3b80 10 client.0 _ll_register_callbacks cb 0x5624129b5680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:48.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.950+0000 7fc4c259c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412ac7a00 0x562412ad0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.950+0000 7fc4c2d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 0x562412ad1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.950+0000 7fc4c259c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412ac7a00 0x562412ad0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:58572/0 (socket says 172.21.15.2:58572) 2023-09-13T04:40:48.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.950+0000 7fc4c259c700 1 -- 172.21.15.2:0/1924247937 learned_addr learned my addr 172.21.15.2:0/1924247937 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:48.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.950+0000 7fc4bbfff700 1 --2- 172.21.15.2:0/1924247937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562412accfe0 0x562412ad10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.950+0000 7fc4c2d9d700 1 -- 172.21.15.2:0/1924247937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562412accfe0 msgr2=0x562412ad10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:48.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.950+0000 7fc4c2d9d700 1 --2- 172.21.15.2:0/1924247937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562412accfe0 0x562412ad10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.950+0000 7fc4c2d9d700 1 -- 172.21.15.2:0/1924247937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412ac7a00 msgr2=0x562412ad0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:48.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.950+0000 7fc4c2d9d700 1 --2- 172.21.15.2:0/1924247937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412ac7a00 0x562412ad0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.950+0000 7fc4c2d9d700 1 -- 172.21.15.2:0/1924247937 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4ac009c90 con 0x562412acd900 2023-09-13T04:40:48.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.950+0000 7fc4c259c700 1 --2- 172.21.15.2:0/1924247937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412ac7a00 0x562412ad0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:48.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.950+0000 7fc4c2d9d700 1 --2- 172.21.15.2:0/1924247937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 0x562412ad1620 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009790 tx=0x7fc4bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a3b0fba6e43d28 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:48.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.950+0000 7fc4a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:40:48.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.951+0000 7fc4a9ffb700 1 -- 172.21.15.2:0/1924247937 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc00e070 con 0x562412acd900 2023-09-13T04:40:48.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.951+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/1924247937 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562412aba670 con 0x562412acd900 2023-09-13T04:40:48.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.951+0000 7fc4a9ffb700 1 -- 172.21.15.2:0/1924247937 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fc4bc021840 con 0x562412acd900 2023-09-13T04:40:48.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.951+0000 7fc4a9ffb700 1 -- 172.21.15.2:0/1924247937 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4bc021a00 con 0x562412acd900 2023-09-13T04:40:48.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.951+0000 7fc4dc7d3b80 10 client.15621 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:48.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.951+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/1924247937 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562412a317a0 con 0x562412acd900 2023-09-13T04:40:48.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.951+0000 7fc4a9ffb700 1 -- 172.21.15.2:0/1924247937 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562412aba670 con 0x562412acd900 2023-09-13T04:40:48.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.951+0000 7fc4a9ffb700 1 -- 172.21.15.2:0/1924247937 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc4bc033070 con 0x562412acd900 2023-09-13T04:40:48.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.952+0000 7fc4dc7d3b80 10 client.15621 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:48.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.952+0000 7fc4dc7d3b80 10 client.15621 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-09-13T04:40:48.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.952+0000 7fc4dc7d3b80 20 client.15621 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:48.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.952+0000 7fc4dc7d3b80 1 --2- 172.21.15.2:0/1924247937 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x562412ae1150 0x562412ae3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.952+0000 7fc4dc7d3b80 4 client.15621 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:40:48.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.952+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/1924247937 --> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562412a12970 con 0x562412ae1150 2023-09-13T04:40:48.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.952+0000 7fc4c259c700 1 --2- 172.21.15.2:0/1924247937 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x562412ae1150 0x562412ae3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.952+0000 7fc4c259c700 1 --2- 172.21.15.2:0/1924247937 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x562412ae1150 0x562412ae3570 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=49de12667d9d177f server_cookie=6e90f351226a8212 in_seq=0 out_seq=0 2023-09-13T04:40:48.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.952+0000 7fc4a9ffb700 10 client.15621 ms_handle_connect on v2:172.21.15.156:6838/1785833227 2023-09-13T04:40:48.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.953+0000 7fc4a9ffb700 1 -- 172.21.15.2:0/1924247937 <== mds.2 v2:172.21.15.156:6838/1785833227 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562412a12970 con 0x562412ae1150 2023-09-13T04:40:48.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.953+0000 7fc4a9ffb700 10 client.15621 handle_command_reply: tid=0 2023-09-13T04:40:48.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.953+0000 7fc4dc7d3b80 1 client.15621 shutdown 2023-09-13T04:40:48.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.953+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/1924247937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 msgr2=0x562412ad1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:48.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.953+0000 7fc4dc7d3b80 1 --2- 172.21.15.2:0/1924247937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 0x562412ad1620 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009790 tx=0x7fc4bc00b330 comp rx=0 tx=0).stop 2023-09-13T04:40:48.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.953+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/1924247937 shutdown_connections 2023-09-13T04:40:48.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.953+0000 7fc4dc7d3b80 1 --2- 172.21.15.2:0/1924247937 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x562412ae1150 0x562412ae3570 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.954+0000 7fc4dc7d3b80 1 --2- 172.21.15.2:0/1924247937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 0x562412ad1620 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.954+0000 7fc4dc7d3b80 1 --2- 172.21.15.2:0/1924247937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412ac7a00 0x562412ad0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.954+0000 7fc4a9ffb700 0 client.15621 ms_handle_reset on v2:172.21.15.156:6838/1785833227 2023-09-13T04:40:48.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.954+0000 7fc4dc7d3b80 1 --2- 172.21.15.2:0/1924247937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562412accfe0 0x562412ad10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.954+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/1924247937 >> 172.21.15.2:0/1924247937 conn(0x562412a25c80 msgr2=0x562412abc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:48.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.954+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/1924247937 shutdown_connections 2023-09-13T04:40:48.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.954+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/1924247937 wait complete. 2023-09-13T04:40:48.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.954+0000 7fc4dc7d3b80 20 client.15621 trim_cache size 0 max 16384 2023-09-13T04:40:48.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.967+0000 7fc4dc7d3b80 1 Processor -- start 2023-09-13T04:40:48.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.967+0000 7fc4dc7d3b80 1 -- start start 2023-09-13T04:40:48.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.968+0000 7fc4dc7d3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562412ac7a00 0x562412ac74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.968+0000 7fc4dc7d3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412accfe0 0x562412ad8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.968+0000 7fc4dc7d3b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 0x562412abc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:48.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.968+0000 7fc4dc7d3b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562412ac24f0 con 0x562412acd900 2023-09-13T04:40:48.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.968+0000 7fc4dc7d3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562412ad1ce0 con 0x562412accfe0 2023-09-13T04:40:48.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.968+0000 7fc4dc7d3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562412accbf0 con 0x562412ac7a00 2023-09-13T04:40:48.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.968+0000 7fc4c2d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 0x562412abc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.968+0000 7fc4c2d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 0x562412abc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59524/0 (socket says 172.21.15.2:59524) 2023-09-13T04:40:48.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.968+0000 7fc4c2d9d700 1 -- 172.21.15.2:0/3062357568 learned_addr learned my addr 172.21.15.2:0/3062357568 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:48.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.968+0000 7fc4c259c700 1 --2- 172.21.15.2:0/3062357568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562412ac7a00 0x562412ac74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.968+0000 7fc4bbfff700 1 --2- 172.21.15.2:0/3062357568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412accfe0 0x562412ad8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:48.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.969+0000 7fc4c2d9d700 1 -- 172.21.15.2:0/3062357568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562412ac7a00 msgr2=0x562412ac74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:48.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.969+0000 7fc4c2d9d700 1 --2- 172.21.15.2:0/3062357568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562412ac7a00 0x562412ac74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.969+0000 7fc4c2d9d700 1 -- 172.21.15.2:0/3062357568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412accfe0 msgr2=0x562412ad8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:48.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.969+0000 7fc4c2d9d700 1 --2- 172.21.15.2:0/3062357568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412accfe0 0x562412ad8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:48.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.969+0000 7fc4c2d9d700 1 -- 172.21.15.2:0/3062357568 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562412845de0 con 0x562412acd900 2023-09-13T04:40:49.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.969+0000 7fc4c259c700 1 --2- 172.21.15.2:0/3062357568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562412ac7a00 0x562412ac74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:49.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.969+0000 7fc4bbfff700 1 --2- 172.21.15.2:0/3062357568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412accfe0 0x562412ad8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:49.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.969+0000 7fc4c2d9d700 1 --2- 172.21.15.2:0/3062357568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 0x562412abc8f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc005640 tx=0x7fc4bc009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb1163b7d934e05 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:49.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.969+0000 7fc4b97fa700 1 -- 172.21.15.2:0/3062357568 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc00e040 con 0x562412acd900 2023-09-13T04:40:49.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.969+0000 7fc4b97fa700 1 -- 172.21.15.2:0/3062357568 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4bc003e90 con 0x562412acd900 2023-09-13T04:40:49.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.969+0000 7fc4b97fa700 1 -- 172.21.15.2:0/3062357568 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc03b480 con 0x562412acd900 2023-09-13T04:40:49.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.970+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/3062357568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 msgr2=0x562412abc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:49.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.970+0000 7fc4dc7d3b80 1 --2- 172.21.15.2:0/3062357568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 0x562412abc8f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc005640 tx=0x7fc4bc009380 comp rx=0 tx=0).stop 2023-09-13T04:40:49.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.970+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/3062357568 shutdown_connections 2023-09-13T04:40:49.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.970+0000 7fc4dc7d3b80 1 --2- 172.21.15.2:0/3062357568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 0x562412abc8f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.970+0000 7fc4dc7d3b80 1 --2- 172.21.15.2:0/3062357568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562412ac7a00 0x562412ac74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.970+0000 7fc4dc7d3b80 1 --2- 172.21.15.2:0/3062357568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412accfe0 0x562412ad8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.970+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/3062357568 >> 172.21.15.2:0/3062357568 conn(0x562412a25c80 msgr2=0x562412add4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:49.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.970+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/3062357568 shutdown_connections 2023-09-13T04:40:49.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.970+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/3062357568 wait complete. 2023-09-13T04:40:49.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.970+0000 7fc4dc7d3b80 1 -- start start 2023-09-13T04:40:49.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.970+0000 7fc4dc7d3b80 1 Processor -- start 2023-09-13T04:40:49.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.971+0000 7fc4dc7d3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412ac7a00 0x562412ac8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.971+0000 7fc4dc7d3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562412accfe0 0x562412ac8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.971+0000 7fc4dc7d3b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 0x562412ac94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.971+0000 7fc4dc7d3b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562412ad8600 con 0x562412acd900 2023-09-13T04:40:49.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.971+0000 7fc4dc7d3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562412ac9a50 con 0x562412ac7a00 2023-09-13T04:40:49.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.971+0000 7fc4dc7d3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562412ac9c10 con 0x562412accfe0 2023-09-13T04:40:49.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.971+0000 7fc4dc7d3b80 10 client.0 _ll_register_callbacks cb 0x5624129b5680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:49.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.971+0000 7fc4c2d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 0x562412ac94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.971+0000 7fc4c2d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 0x562412ac94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59534/0 (socket says 172.21.15.2:59534) 2023-09-13T04:40:49.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.971+0000 7fc4bbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562412accfe0 0x562412ac8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.971+0000 7fc4c2d9d700 1 -- 172.21.15.2:0/1241723180 learned_addr learned my addr 172.21.15.2:0/1241723180 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:49.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.971+0000 7fc4c259c700 1 --2- 172.21.15.2:0/1241723180 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412ac7a00 0x562412ac8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.972+0000 7fc4c2d9d700 1 -- 172.21.15.2:0/1241723180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562412accfe0 msgr2=0x562412ac8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:49.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.972+0000 7fc4c2d9d700 1 --2- 172.21.15.2:0/1241723180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562412accfe0 0x562412ac8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.972+0000 7fc4c2d9d700 1 -- 172.21.15.2:0/1241723180 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412ac7a00 msgr2=0x562412ac8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:49.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.972+0000 7fc4c2d9d700 1 --2- 172.21.15.2:0/1241723180 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412ac7a00 0x562412ac8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.972+0000 7fc4c2d9d700 1 -- 172.21.15.2:0/1241723180 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c419e6a0 con 0x562412acd900 2023-09-13T04:40:49.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.972+0000 7fc4c259c700 1 --2- 172.21.15.2:0/1241723180 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412ac7a00 0x562412ac8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:49.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.972+0000 7fc4bbfff700 1 --2- 172.21.15.2:0/1241723180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562412accfe0 0x562412ac8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:49.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.972+0000 7fc4c2d9d700 1 --2- 172.21.15.2:0/1241723180 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 0x562412ac94e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0056c0 tx=0x7fc4bc005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699239a757c6d3ce server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:49.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.972+0000 7fc4a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:40:49.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.972+0000 7fc4a9ffb700 1 -- 172.21.15.2:0/1241723180 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc00e040 con 0x562412acd900 2023-09-13T04:40:49.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.972+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/1241723180 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4ac004cf0 con 0x562412acd900 2023-09-13T04:40:49.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.972+0000 7fc4a9ffb700 1 -- 172.21.15.2:0/1241723180 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fc4bc021aa0 con 0x562412acd900 2023-09-13T04:40:49.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.972+0000 7fc4a9ffb700 1 -- 172.21.15.2:0/1241723180 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4bc021c60 con 0x562412acd900 2023-09-13T04:40:49.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.973+0000 7fc4dc7d3b80 10 client.15627 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:49.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.973+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/1241723180 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fc4ac01d040 con 0x562412acd900 2023-09-13T04:40:49.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.973+0000 7fc4a9ffb700 1 -- 172.21.15.2:0/1241723180 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4ac004cf0 con 0x562412acd900 2023-09-13T04:40:49.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.973+0000 7fc4a9ffb700 1 -- 172.21.15.2:0/1241723180 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc4bc042070 con 0x562412acd900 2023-09-13T04:40:49.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.973+0000 7fc4dc7d3b80 10 client.15627 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:49.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.973+0000 7fc4dc7d3b80 10 client.15627 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-09-13T04:40:49.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.973+0000 7fc4dc7d3b80 20 client.15627 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:49.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.973+0000 7fc4dc7d3b80 1 --2- 172.21.15.2:0/1241723180 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x562412b355c0 0x562412b379b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.973+0000 7fc4dc7d3b80 4 client.15627 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-09-13T04:40:49.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.973+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/1241723180 --> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5624129d3050 con 0x562412b355c0 2023-09-13T04:40:49.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.973+0000 7fc4c259c700 1 --2- 172.21.15.2:0/1241723180 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x562412b355c0 0x562412b379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.974+0000 7fc4c259c700 1 --2- 172.21.15.2:0/1241723180 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x562412b355c0 0x562412b379b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a0df0c1023a85ed4 server_cookie=b0963acb6b85e3a6 in_seq=0 out_seq=0 2023-09-13T04:40:49.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.974+0000 7fc4a9ffb700 10 client.15627 ms_handle_connect on v2:172.21.15.156:6838/1785833227 2023-09-13T04:40:49.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.974+0000 7fc4a9ffb700 1 -- 172.21.15.2:0/1241723180 <== mds.2 v2:172.21.15.156:6838/1785833227 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5624129d3050 con 0x562412b355c0 2023-09-13T04:40:49.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.974+0000 7fc4a9ffb700 10 client.15627 handle_command_reply: tid=0 2023-09-13T04:40:49.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.974+0000 7fc4dc7d3b80 1 client.15627 shutdown 2023-09-13T04:40:49.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.974+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/1241723180 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 msgr2=0x562412ac94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:49.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.974+0000 7fc4dc7d3b80 1 --2- 172.21.15.2:0/1241723180 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 0x562412ac94e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0056c0 tx=0x7fc4bc005b10 comp rx=0 tx=0).stop 2023-09-13T04:40:49.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.974+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/1241723180 shutdown_connections 2023-09-13T04:40:49.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.974+0000 7fc4dc7d3b80 1 --2- 172.21.15.2:0/1241723180 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x562412b355c0 0x562412b379b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.974+0000 7fc4dc7d3b80 1 --2- 172.21.15.2:0/1241723180 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562412acd900 0x562412ac94e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.974+0000 7fc4dc7d3b80 1 --2- 172.21.15.2:0/1241723180 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562412ac7a00 0x562412ac8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.974+0000 7fc4dc7d3b80 1 --2- 172.21.15.2:0/1241723180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562412accfe0 0x562412ac8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.974+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/1241723180 >> 172.21.15.2:0/1241723180 conn(0x562412a25c80 msgr2=0x562412acbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:49.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.974+0000 7fc4a9ffb700 0 client.15627 ms_handle_reset on v2:172.21.15.156:6838/1785833227 2023-09-13T04:40:49.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.974+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/1241723180 shutdown_connections 2023-09-13T04:40:49.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.974+0000 7fc4dc7d3b80 1 -- 172.21.15.2:0/1241723180 wait complete. 2023-09-13T04:40:49.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.975+0000 7fc4dc7d3b80 20 client.15627 trim_cache size 0 max 16384 2023-09-13T04:40:49.016 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-09-13T04:40:49.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.975+0000 7fc4c8dff700 1 -- 172.21.15.2:0/3535461969 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc4b00a0840 msgr2=0x7fc4b00a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:49.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.975+0000 7fc4c8dff700 1 --2- 172.21.15.2:0/3535461969 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc4b00a0840 0x7fc4b00a2d00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009550 tx=0x7fc4bc007ad0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.975+0000 7fc4c8dff700 1 -- 172.21.15.2:0/3535461969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4c40730e0 msgr2=0x7fc4c41990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:49.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.975+0000 7fc4c8dff700 1 --2- 172.21.15.2:0/3535461969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4c40730e0 0x7fc4c41990b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400e9c0 tx=0x7fc4b400ecd0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.975+0000 7fc4c8dff700 1 -- 172.21.15.2:0/3535461969 shutdown_connections 2023-09-13T04:40:49.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.975+0000 7fc4c8dff700 1 --2- 172.21.15.2:0/3535461969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4c410d480 0x7fc4c419da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.975+0000 7fc4c8dff700 1 --2- 172.21.15.2:0/3535461969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4c40730e0 0x7fc4c41990b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.975+0000 7fc4c8dff700 1 --2- 172.21.15.2:0/3535461969 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc4b00a0840 0x7fc4b00a2d00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.975+0000 7fc4c8dff700 1 --2- 172.21.15.2:0/3535461969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4c4073a00 0x7fc4c41995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.975+0000 7fc4c8dff700 1 -- 172.21.15.2:0/3535461969 >> 172.21.15.2:0/3535461969 conn(0x7fc4c40fc750 msgr2=0x7fc4c410f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:49.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.975+0000 7fc4c8dff700 1 -- 172.21.15.2:0/3535461969 shutdown_connections 2023-09-13T04:40:49.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:48.975+0000 7fc4c8dff700 1 -- 172.21.15.2:0/3535461969 wait complete. 2023-09-13T04:40:49.019 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:48 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1038571764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:40:49.020 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-13T04:40:49.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:48 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1038571764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:40:49.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:48 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1038571764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:40:49.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.226+0000 7f8a1b7ff700 1 -- 172.21.15.2:0/1691184574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a14102710 msgr2=0x7f8a1410abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:49.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.226+0000 7f8a1b7ff700 1 --2- 172.21.15.2:0/1691184574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a14102710 0x7f8a1410abf0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009990 tx=0x7f8a04009ca0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.226+0000 7f8a1b7ff700 1 -- 172.21.15.2:0/1691184574 shutdown_connections 2023-09-13T04:40:49.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.226+0000 7f8a1b7ff700 1 --2- 172.21.15.2:0/1691184574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a1410b210 0x7f8a1410d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.226+0000 7f8a1b7ff700 1 --2- 172.21.15.2:0/1691184574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a14101df0 0x7f8a141021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.226+0000 7f8a1b7ff700 1 --2- 172.21.15.2:0/1691184574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a14102710 0x7f8a1410abf0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.226+0000 7f8a1b7ff700 1 -- 172.21.15.2:0/1691184574 >> 172.21.15.2:0/1691184574 conn(0x7f8a140fb630 msgr2=0x7f8a140fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:49.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.226+0000 7f8a1b7ff700 1 -- 172.21.15.2:0/1691184574 shutdown_connections 2023-09-13T04:40:49.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.226+0000 7f8a1b7ff700 1 -- 172.21.15.2:0/1691184574 wait complete. 2023-09-13T04:40:49.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.227+0000 7f8a1b7ff700 1 Processor -- start 2023-09-13T04:40:49.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.227+0000 7f8a1b7ff700 1 -- start start 2023-09-13T04:40:49.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.228+0000 7f8a1b7ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a14101df0 0x7f8a140ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.228+0000 7f8a1b7ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a14102710 0x7f8a140fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.228+0000 7f8a1b7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a1410b210 0x7f8a141a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.228+0000 7f8a1b7ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a141007b0 con 0x7f8a14102710 2023-09-13T04:40:49.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.228+0000 7f8a1b7ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a141a54c0 con 0x7f8a14101df0 2023-09-13T04:40:49.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.228+0000 7f8a1b7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a141a5600 con 0x7f8a1410b210 2023-09-13T04:40:49.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.228+0000 7f8a18d9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a14102710 0x7f8a140fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.228+0000 7f8a18d9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a14102710 0x7f8a140fff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59558/0 (socket says 172.21.15.2:59558) 2023-09-13T04:40:49.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.228+0000 7f8a18d9a700 1 -- 172.21.15.2:0/1421041570 learned_addr learned my addr 172.21.15.2:0/1421041570 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:49.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.228+0000 7f8a1959b700 1 --2- 172.21.15.2:0/1421041570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a14101df0 0x7f8a140ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.228+0000 7f8a19d9c700 1 --2- 172.21.15.2:0/1421041570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a1410b210 0x7f8a141a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.229+0000 7f8a18d9a700 1 -- 172.21.15.2:0/1421041570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a1410b210 msgr2=0x7f8a141a4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:49.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.229+0000 7f8a18d9a700 1 --2- 172.21.15.2:0/1421041570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a1410b210 0x7f8a141a4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.229+0000 7f8a18d9a700 1 -- 172.21.15.2:0/1421041570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a14101df0 msgr2=0x7f8a140ffa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:49.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.229+0000 7f8a18d9a700 1 --2- 172.21.15.2:0/1421041570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a14101df0 0x7f8a140ffa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.229+0000 7f8a18d9a700 1 -- 172.21.15.2:0/1421041570 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a040095f0 con 0x7f8a14102710 2023-09-13T04:40:49.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.229+0000 7f8a18d9a700 1 --2- 172.21.15.2:0/1421041570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a14102710 0x7f8a140fff80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8a04000c00 tx=0x7f8a040047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:49.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.229+0000 7f8a0a7fc700 1 -- 172.21.15.2:0/1421041570 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0401d070 con 0x7f8a14102710 2023-09-13T04:40:49.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.229+0000 7f8a1b7ff700 1 -- 172.21.15.2:0/1421041570 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a141a5800 con 0x7f8a14102710 2023-09-13T04:40:49.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.229+0000 7f8a0a7fc700 1 -- 172.21.15.2:0/1421041570 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a04021410 con 0x7f8a14102710 2023-09-13T04:40:49.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.229+0000 7f8a0a7fc700 1 -- 172.21.15.2:0/1421041570 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a04017650 con 0x7f8a14102710 2023-09-13T04:40:49.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.229+0000 7f8a1b7ff700 1 -- 172.21.15.2:0/1421041570 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a141a5c80 con 0x7f8a14102710 2023-09-13T04:40:49.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.232+0000 7f8a0a7fc700 1 -- 172.21.15.2:0/1421041570 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 23) v1 ==== 135434+0+0 (secure 0 0 0) 0x7f8a04021580 con 0x7f8a14102710 2023-09-13T04:40:49.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.234+0000 7f8a0a7fc700 1 --2- 172.21.15.2:0/1421041570 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f8a000a0840 0x7f8a000a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.234+0000 7f8a2f1a8b80 1 Processor -- start 2023-09-13T04:40:49.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.234+0000 7f8a0a7fc700 1 -- 172.21.15.2:0/1421041570 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f8a040cd730 con 0x7f8a14102710 2023-09-13T04:40:49.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.234+0000 7f8a2f1a8b80 1 -- start start 2023-09-13T04:40:49.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.234+0000 7f8a1959b700 1 --2- 172.21.15.2:0/1421041570 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f8a000a0840 0x7f8a000a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.234+0000 7f8a2f1a8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563efc57efe0 0x563efc57f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.234+0000 7f8a2f1a8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563efc57f900 0x563efc5794c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.234+0000 7f8a2f1a8b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc579a00 0x563efc579e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.234+0000 7f8a2f1a8b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563efc4d2780 con 0x563efc579a00 2023-09-13T04:40:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.234+0000 7f8a2f1a8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563efc4d00a0 con 0x563efc57efe0 2023-09-13T04:40:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.234+0000 7f8a2f1a8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563efc4cb990 con 0x563efc57f900 2023-09-13T04:40:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.235+0000 7f8a19d9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc579a00 0x563efc579e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.235+0000 7f8a19d9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc579a00 0x563efc579e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59562/0 (socket says 172.21.15.2:59562) 2023-09-13T04:40:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.235+0000 7f8a1959b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563efc57efe0 0x563efc57f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.235+0000 7f8a19d9c700 1 -- 172.21.15.2:0/356466123 learned_addr learned my addr 172.21.15.2:0/356466123 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.235+0000 7f8a18d9a700 1 --2- 172.21.15.2:0/356466123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563efc57f900 0x563efc5794c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.235+0000 7f8a1959b700 1 --2- 172.21.15.2:0/1421041570 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f8a000a0840 0x7f8a000a2d00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8a10009c70 tx=0x7f8a10011720 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.235+0000 7f8a1959b700 1 -- 172.21.15.2:0/356466123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563efc57f900 msgr2=0x563efc5794c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.235+0000 7f8a1959b700 1 --2- 172.21.15.2:0/356466123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563efc57f900 0x563efc5794c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.235+0000 7f8a1959b700 1 -- 172.21.15.2:0/356466123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc579a00 msgr2=0x563efc579e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.235+0000 7f8a1959b700 1 --2- 172.21.15.2:0/356466123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc579a00 0x563efc579e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.235+0000 7f8a1959b700 1 -- 172.21.15.2:0/356466123 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563efc34b630 con 0x563efc57efe0 2023-09-13T04:40:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.235+0000 7f8a1959b700 1 --2- 172.21.15.2:0/356466123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563efc57efe0 0x563efc57f3c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8a10011b20 tx=0x7f8a1001c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e52dd966b5840858 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:49.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.236+0000 7f89ff7fe700 1 -- 172.21.15.2:0/356466123 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1001ddb0 con 0x563efc57efe0 2023-09-13T04:40:49.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.236+0000 7f89ff7fe700 1 -- 172.21.15.2:0/356466123 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a1000f880 con 0x563efc57efe0 2023-09-13T04:40:49.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.236+0000 7f89ff7fe700 1 -- 172.21.15.2:0/356466123 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1000e730 con 0x563efc57efe0 2023-09-13T04:40:49.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.236+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/356466123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563efc57efe0 msgr2=0x563efc57f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:49.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.236+0000 7f8a2f1a8b80 1 --2- 172.21.15.2:0/356466123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563efc57efe0 0x563efc57f3c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8a10011b20 tx=0x7f8a1001c000 comp rx=0 tx=0).stop 2023-09-13T04:40:49.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.236+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/356466123 shutdown_connections 2023-09-13T04:40:49.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.236+0000 7f8a2f1a8b80 1 --2- 172.21.15.2:0/356466123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc579a00 0x563efc579e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.236+0000 7f8a2f1a8b80 1 --2- 172.21.15.2:0/356466123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563efc57efe0 0x563efc57f3c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.236+0000 7f8a2f1a8b80 1 --2- 172.21.15.2:0/356466123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563efc57f900 0x563efc5794c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.236+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/356466123 >> 172.21.15.2:0/356466123 conn(0x563efc4d7c80 msgr2=0x563efc573f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:49.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.236+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/356466123 shutdown_connections 2023-09-13T04:40:49.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.237+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/356466123 wait complete. 2023-09-13T04:40:49.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.237+0000 7f8a2f1a8b80 1 -- start start 2023-09-13T04:40:49.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.237+0000 7f8a2f1a8b80 1 Processor -- start 2023-09-13T04:40:49.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.238+0000 7f8a2f1a8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563efc579a00 0x563efc585940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.238+0000 7f8a2f1a8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563efc57efe0 0x563efc582c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.238+0000 7f8a2f1a8b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc57f900 0x563efc5832d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.238+0000 7f8a2f1a8b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563efc469bb0 con 0x563efc57f900 2023-09-13T04:40:49.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.238+0000 7f8a2f1a8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563efc4698b0 con 0x563efc57efe0 2023-09-13T04:40:49.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.238+0000 7f8a2f1a8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563efc34d920 con 0x563efc579a00 2023-09-13T04:40:49.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.238+0000 7f8a2f1a8b80 10 client.0 _ll_register_callbacks cb 0x563efc467680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:49.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.238+0000 7f8a19d9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc57f900 0x563efc5832d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.238+0000 7f8a19d9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc57f900 0x563efc5832d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59566/0 (socket says 172.21.15.2:59566) 2023-09-13T04:40:49.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.238+0000 7f8a18d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563efc57efe0 0x563efc582c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.238+0000 7f8a19d9c700 1 -- 172.21.15.2:0/4220715950 learned_addr learned my addr 172.21.15.2:0/4220715950 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:49.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.238+0000 7f8a1959b700 1 --2- 172.21.15.2:0/4220715950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563efc579a00 0x563efc585940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.238+0000 7f8a18d9a700 1 -- 172.21.15.2:0/4220715950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563efc579a00 msgr2=0x563efc585940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:49.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.238+0000 7f8a18d9a700 1 --2- 172.21.15.2:0/4220715950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563efc579a00 0x563efc585940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.238+0000 7f8a18d9a700 1 -- 172.21.15.2:0/4220715950 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc57f900 msgr2=0x563efc5832d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:49.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.238+0000 7f8a18d9a700 1 --2- 172.21.15.2:0/4220715950 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc57f900 0x563efc5832d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.238+0000 7f8a18d9a700 1 -- 172.21.15.2:0/4220715950 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a10011cd0 con 0x563efc57efe0 2023-09-13T04:40:49.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.239+0000 7f8a18d9a700 1 --2- 172.21.15.2:0/4220715950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563efc57efe0 0x563efc582c30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8a040caa50 tx=0x7f8a04005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd5d9f9725bd77ba server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:49.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.239+0000 7f89fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:40:49.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.239+0000 7f89fcff9700 1 -- 172.21.15.2:0/4220715950 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0409abf0 con 0x563efc57efe0 2023-09-13T04:40:49.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.239+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/4220715950 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563efc56c670 con 0x563efc57efe0 2023-09-13T04:40:49.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.239+0000 7f89fcff9700 1 -- 172.21.15.2:0/4220715950 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f8a0409ae90 con 0x563efc57efe0 2023-09-13T04:40:49.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.239+0000 7f89fcff9700 1 -- 172.21.15.2:0/4220715950 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a0409a790 con 0x563efc57efe0 2023-09-13T04:40:49.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.239+0000 7f8a2f1a8b80 10 client.25664 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:49.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.239+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/4220715950 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563efc4e37a0 con 0x563efc57efe0 2023-09-13T04:40:49.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.239+0000 7f89fcff9700 1 -- 172.21.15.2:0/4220715950 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563efc56c670 con 0x563efc57efe0 2023-09-13T04:40:49.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.240+0000 7f89fcff9700 1 -- 172.21.15.2:0/4220715950 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8a040a22e0 con 0x563efc57efe0 2023-09-13T04:40:49.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.240+0000 7f8a2f1a8b80 10 client.25664 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:49.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.240+0000 7f8a2f1a8b80 10 client.25664 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-09-13T04:40:49.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.240+0000 7f8a2f1a8b80 20 client.25664 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:49.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.240+0000 7f8a2f1a8b80 1 --2- 172.21.15.2:0/4220715950 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x563efc592d50 0x563efc595170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.240+0000 7f8a2f1a8b80 4 client.25664 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:40:49.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.240+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/4220715950 --> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563efc4c4970 con 0x563efc592d50 2023-09-13T04:40:49.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.240+0000 7f8a1959b700 1 --2- 172.21.15.2:0/4220715950 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x563efc592d50 0x563efc595170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.240+0000 7f8a1959b700 1 --2- 172.21.15.2:0/4220715950 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x563efc592d50 0x563efc595170 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1ae21c6f7a26290 server_cookie=1ae8b09959a1f620 in_seq=0 out_seq=0 2023-09-13T04:40:49.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.240+0000 7f89fcff9700 10 client.25664 ms_handle_connect on v2:172.21.15.2:6840/482545391 2023-09-13T04:40:49.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.241+0000 7f89fcff9700 1 -- 172.21.15.2:0/4220715950 <== mds.1 v2:172.21.15.2:6840/482545391 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563efc4c4970 con 0x563efc592d50 2023-09-13T04:40:49.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.241+0000 7f89fcff9700 10 client.25664 handle_command_reply: tid=0 2023-09-13T04:40:49.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.241+0000 7f8a2f1a8b80 1 client.25664 shutdown 2023-09-13T04:40:49.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.241+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/4220715950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563efc57efe0 msgr2=0x563efc582c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:49.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.241+0000 7f8a2f1a8b80 1 --2- 172.21.15.2:0/4220715950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563efc57efe0 0x563efc582c30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8a040caa50 tx=0x7f8a04005c00 comp rx=0 tx=0).stop 2023-09-13T04:40:49.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.241+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/4220715950 shutdown_connections 2023-09-13T04:40:49.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.241+0000 7f8a2f1a8b80 1 --2- 172.21.15.2:0/4220715950 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x563efc592d50 0x563efc595170 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.241+0000 7f8a2f1a8b80 1 --2- 172.21.15.2:0/4220715950 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc57f900 0x563efc5832d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.242+0000 7f8a2f1a8b80 1 --2- 172.21.15.2:0/4220715950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563efc579a00 0x563efc585940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.242+0000 7f89fcff9700 0 client.25664 ms_handle_reset on v2:172.21.15.2:6840/482545391 2023-09-13T04:40:49.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.242+0000 7f8a2f1a8b80 1 --2- 172.21.15.2:0/4220715950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563efc57efe0 0x563efc582c30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.242+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/4220715950 >> 172.21.15.2:0/4220715950 conn(0x563efc4d7c80 msgr2=0x563efc573e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:49.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.242+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/4220715950 shutdown_connections 2023-09-13T04:40:49.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.242+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/4220715950 wait complete. 2023-09-13T04:40:49.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.242+0000 7f8a2f1a8b80 20 client.25664 trim_cache size 0 max 16384 2023-09-13T04:40:49.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.256+0000 7f8a2f1a8b80 1 Processor -- start 2023-09-13T04:40:49.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.256+0000 7f8a2f1a8b80 1 -- start start 2023-09-13T04:40:49.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.256+0000 7f8a2f1a8b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc579a00 0x563efc5794c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.256+0000 7f8a2f1a8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563efc57efe0 0x563efc58a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.256+0000 7f8a2f1a8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563efc57f900 0x563efc56e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.256+0000 7f8a2f1a8b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563efc5744f0 con 0x563efc579a00 2023-09-13T04:40:49.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.256+0000 7f8a2f1a8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563efc570910 con 0x563efc57f900 2023-09-13T04:40:49.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.256+0000 7f8a2f1a8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563efc58a7d0 con 0x563efc57efe0 2023-09-13T04:40:49.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.256+0000 7f8a1959b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc579a00 0x563efc5794c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.256+0000 7f8a1959b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc579a00 0x563efc5794c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59576/0 (socket says 172.21.15.2:59576) 2023-09-13T04:40:49.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.256+0000 7f8a1959b700 1 -- 172.21.15.2:0/2204713641 learned_addr learned my addr 172.21.15.2:0/2204713641 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:49.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.256+0000 7f8a18d9a700 1 --2- 172.21.15.2:0/2204713641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563efc57efe0 0x563efc58a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.256+0000 7f8a19d9c700 1 --2- 172.21.15.2:0/2204713641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563efc57f900 0x563efc56e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.257+0000 7f8a1959b700 1 -- 172.21.15.2:0/2204713641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563efc57efe0 msgr2=0x563efc58a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:49.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.257+0000 7f8a1959b700 1 --2- 172.21.15.2:0/2204713641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563efc57efe0 0x563efc58a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.257+0000 7f8a1959b700 1 -- 172.21.15.2:0/2204713641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563efc57f900 msgr2=0x563efc56e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:49.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.257+0000 7f8a1959b700 1 --2- 172.21.15.2:0/2204713641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563efc57f900 0x563efc56e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.257+0000 7f8a1959b700 1 -- 172.21.15.2:0/2204713641 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563efc575ad0 con 0x563efc579a00 2023-09-13T04:40:49.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.257+0000 7f8a18d9a700 1 --2- 172.21.15.2:0/2204713641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563efc57efe0 0x563efc58a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:40:49.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.257+0000 7f8a1959b700 1 --2- 172.21.15.2:0/2204713641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc579a00 0x563efc5794c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8a10005950 tx=0x7f8a10028040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf5ac82564b670d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:49.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.257+0000 7f89fffff700 1 -- 172.21.15.2:0/2204713641 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1002b040 con 0x563efc579a00 2023-09-13T04:40:49.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.257+0000 7f89fffff700 1 -- 172.21.15.2:0/2204713641 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a1001ddf0 con 0x563efc579a00 2023-09-13T04:40:49.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.257+0000 7f89fffff700 1 -- 172.21.15.2:0/2204713641 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1002e880 con 0x563efc579a00 2023-09-13T04:40:49.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.258+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/2204713641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc579a00 msgr2=0x563efc5794c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:49.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.258+0000 7f8a2f1a8b80 1 --2- 172.21.15.2:0/2204713641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc579a00 0x563efc5794c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8a10005950 tx=0x7f8a10028040 comp rx=0 tx=0).stop 2023-09-13T04:40:49.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.258+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/2204713641 shutdown_connections 2023-09-13T04:40:49.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.258+0000 7f8a2f1a8b80 1 --2- 172.21.15.2:0/2204713641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563efc57f900 0x563efc56e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.258+0000 7f8a2f1a8b80 1 --2- 172.21.15.2:0/2204713641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc579a00 0x563efc5794c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.258+0000 7f8a2f1a8b80 1 --2- 172.21.15.2:0/2204713641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563efc57efe0 0x563efc58a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.258+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/2204713641 >> 172.21.15.2:0/2204713641 conn(0x563efc4d7c80 msgr2=0x563efc57cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:49.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.258+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/2204713641 shutdown_connections 2023-09-13T04:40:49.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.258+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/2204713641 wait complete. 2023-09-13T04:40:49.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.258+0000 7f8a2f1a8b80 1 -- start start 2023-09-13T04:40:49.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.259+0000 7f8a2f1a8b80 1 Processor -- start 2023-09-13T04:40:49.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.259+0000 7f8a2f1a8b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc579a00 0x563efc588200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.259+0000 7f8a2f1a8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563efc57efe0 0x563efc588740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.259+0000 7f8a2f1a8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563efc57f900 0x563efc5e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.259+0000 7f8a2f1a8b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563efc588f40 con 0x563efc579a00 2023-09-13T04:40:49.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.259+0000 7f8a2f1a8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563efc5890b0 con 0x563efc57efe0 2023-09-13T04:40:49.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.259+0000 7f8a2f1a8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563efc5e6140 con 0x563efc57f900 2023-09-13T04:40:49.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.259+0000 7f8a2f1a8b80 10 client.0 _ll_register_callbacks cb 0x563efc467680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:49.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.259+0000 7f8a1959b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc579a00 0x563efc588200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.259+0000 7f8a1959b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc579a00 0x563efc588200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59592/0 (socket says 172.21.15.2:59592) 2023-09-13T04:40:49.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.259+0000 7f8a1959b700 1 -- 172.21.15.2:0/1024597335 learned_addr learned my addr 172.21.15.2:0/1024597335 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:49.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.259+0000 7f8a18d9a700 1 --2- 172.21.15.2:0/1024597335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563efc57efe0 0x563efc588740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.259+0000 7f8a19d9c700 1 --2- 172.21.15.2:0/1024597335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563efc57f900 0x563efc5e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.260+0000 7f8a1959b700 1 -- 172.21.15.2:0/1024597335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563efc57f900 msgr2=0x563efc5e5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:49.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.260+0000 7f8a1959b700 1 --2- 172.21.15.2:0/1024597335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563efc57f900 0x563efc5e5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.260+0000 7f8a1959b700 1 -- 172.21.15.2:0/1024597335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563efc57efe0 msgr2=0x563efc588740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:49.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.260+0000 7f8a1959b700 1 --2- 172.21.15.2:0/1024597335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563efc57efe0 0x563efc588740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.260+0000 7f8a1959b700 1 -- 172.21.15.2:0/1024597335 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563efc34b630 con 0x563efc579a00 2023-09-13T04:40:49.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.260+0000 7f8a19d9c700 1 --2- 172.21.15.2:0/1024597335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563efc57f900 0x563efc5e5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:40:49.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.260+0000 7f8a1959b700 1 --2- 172.21.15.2:0/1024597335 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc579a00 0x563efc588200 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8a1001db80 tx=0x7f8a10005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf44c32819c146a8 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:49.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.260+0000 7f89fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:40:49.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.260+0000 7f89fcff9700 1 -- 172.21.15.2:0/1024597335 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1002e3b0 con 0x563efc579a00 2023-09-13T04:40:49.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.260+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/1024597335 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a1000f880 con 0x563efc579a00 2023-09-13T04:40:49.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.260+0000 7f89fcff9700 1 -- 172.21.15.2:0/1024597335 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f8a1002e510 con 0x563efc579a00 2023-09-13T04:40:49.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.260+0000 7f89fcff9700 1 -- 172.21.15.2:0/1024597335 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a1000f3c0 con 0x563efc579a00 2023-09-13T04:40:49.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.260+0000 7f8a2f1a8b80 10 client.15645 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:49.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.260+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/1024597335 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563efc2f7de0 con 0x563efc579a00 2023-09-13T04:40:49.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.261+0000 7f89fcff9700 1 -- 172.21.15.2:0/1024597335 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a1000f880 con 0x563efc579a00 2023-09-13T04:40:49.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.261+0000 7f89fcff9700 1 -- 172.21.15.2:0/1024597335 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8a1002e050 con 0x563efc579a00 2023-09-13T04:40:49.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.261+0000 7f8a2f1a8b80 10 client.15645 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:49.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.261+0000 7f8a2f1a8b80 10 client.15645 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-09-13T04:40:49.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.261+0000 7f8a2f1a8b80 20 client.15645 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:49.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.261+0000 7f8a2f1a8b80 1 --2- 172.21.15.2:0/1024597335 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x563efc5e7500 0x563efc5e98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.261+0000 7f8a2f1a8b80 4 client.15645 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-09-13T04:40:49.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.261+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/1024597335 --> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563efc485050 con 0x563efc5e7500 2023-09-13T04:40:49.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.261+0000 7f8a18d9a700 1 --2- 172.21.15.2:0/1024597335 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x563efc5e7500 0x563efc5e98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.262+0000 7f8a18d9a700 1 --2- 172.21.15.2:0/1024597335 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x563efc5e7500 0x563efc5e98f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3128134d3776b0fa server_cookie=78504bfcde1a31ca in_seq=0 out_seq=0 2023-09-13T04:40:49.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.262+0000 7f89fcff9700 10 client.15645 ms_handle_connect on v2:172.21.15.2:6840/482545391 2023-09-13T04:40:49.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.262+0000 7f89fcff9700 1 -- 172.21.15.2:0/1024597335 <== mds.1 v2:172.21.15.2:6840/482545391 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563efc485050 con 0x563efc5e7500 2023-09-13T04:40:49.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.262+0000 7f89fcff9700 10 client.15645 handle_command_reply: tid=0 2023-09-13T04:40:49.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.262+0000 7f8a2f1a8b80 1 client.15645 shutdown 2023-09-13T04:40:49.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.262+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/1024597335 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc579a00 msgr2=0x563efc588200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:49.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.262+0000 7f8a2f1a8b80 1 --2- 172.21.15.2:0/1024597335 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc579a00 0x563efc588200 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8a1001db80 tx=0x7f8a10005df0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.262+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/1024597335 shutdown_connections 2023-09-13T04:40:49.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.262+0000 7f8a2f1a8b80 1 --2- 172.21.15.2:0/1024597335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563efc57f900 0x563efc5e5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.262+0000 7f8a2f1a8b80 1 --2- 172.21.15.2:0/1024597335 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x563efc5e7500 0x563efc5e98f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.262+0000 7f8a2f1a8b80 1 --2- 172.21.15.2:0/1024597335 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563efc579a00 0x563efc588200 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.262+0000 7f8a2f1a8b80 1 --2- 172.21.15.2:0/1024597335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563efc57efe0 0x563efc588740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.262+0000 7f89fcff9700 0 client.15645 ms_handle_reset on v2:172.21.15.2:6840/482545391 2023-09-13T04:40:49.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.262+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/1024597335 >> 172.21.15.2:0/1024597335 conn(0x563efc4d7c80 msgr2=0x563efc57fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:49.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.262+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/1024597335 shutdown_connections 2023-09-13T04:40:49.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.262+0000 7f8a2f1a8b80 1 -- 172.21.15.2:0/1024597335 wait complete. 2023-09-13T04:40:49.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.263+0000 7f8a2f1a8b80 20 client.15645 trim_cache size 0 max 16384 2023-09-13T04:40:49.291 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-09-13T04:40:49.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.263+0000 7f8a1b7ff700 1 -- 172.21.15.2:0/1421041570 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f8a000a0840 msgr2=0x7f8a000a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:49.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.263+0000 7f8a1b7ff700 1 --2- 172.21.15.2:0/1421041570 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f8a000a0840 0x7f8a000a2d00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8a10009c70 tx=0x7f8a10011720 comp rx=0 tx=0).stop 2023-09-13T04:40:49.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.263+0000 7f8a1b7ff700 1 -- 172.21.15.2:0/1421041570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a14102710 msgr2=0x7f8a140fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:49.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.263+0000 7f8a1b7ff700 1 --2- 172.21.15.2:0/1421041570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a14102710 0x7f8a140fff80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8a04000c00 tx=0x7f8a040047b0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.263+0000 7f8a1b7ff700 1 -- 172.21.15.2:0/1421041570 shutdown_connections 2023-09-13T04:40:49.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.263+0000 7f8a1b7ff700 1 --2- 172.21.15.2:0/1421041570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a1410b210 0x7f8a141a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.263+0000 7f8a1b7ff700 1 --2- 172.21.15.2:0/1421041570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a14101df0 0x7f8a140ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.263+0000 7f8a1b7ff700 1 --2- 172.21.15.2:0/1421041570 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f8a000a0840 0x7f8a000a2d00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.263+0000 7f8a1b7ff700 1 --2- 172.21.15.2:0/1421041570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a14102710 0x7f8a140fff80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.263+0000 7f8a1b7ff700 1 -- 172.21.15.2:0/1421041570 >> 172.21.15.2:0/1421041570 conn(0x7f8a140fb630 msgr2=0x7f8a1410d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:49.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.263+0000 7f8a1b7ff700 1 -- 172.21.15.2:0/1421041570 shutdown_connections 2023-09-13T04:40:49.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.263+0000 7f8a1b7ff700 1 -- 172.21.15.2:0/1421041570 wait complete. 2023-09-13T04:40:49.295 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-13T04:40:49.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.500+0000 7fb6de9ff700 1 -- 172.21.15.2:0/1599930983 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6d8101860 msgr2=0x7fb6d81058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:49.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.500+0000 7fb6de9ff700 1 --2- 172.21.15.2:0/1599930983 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6d8101860 0x7fb6d81058a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc009990 tx=0x7fb6cc009ca0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.500+0000 7fb6de9ff700 1 -- 172.21.15.2:0/1599930983 shutdown_connections 2023-09-13T04:40:49.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.500+0000 7fb6de9ff700 1 --2- 172.21.15.2:0/1599930983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6d8105fd0 0x7fb6d810db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.500+0000 7fb6de9ff700 1 --2- 172.21.15.2:0/1599930983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6d8100eb0 0x7fb6d8101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.500+0000 7fb6de9ff700 1 --2- 172.21.15.2:0/1599930983 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6d8101860 0x7fb6d81058a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.500+0000 7fb6de9ff700 1 -- 172.21.15.2:0/1599930983 >> 172.21.15.2:0/1599930983 conn(0x7fb6d80fc750 msgr2=0x7fb6d80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:49.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.500+0000 7fb6de9ff700 1 -- 172.21.15.2:0/1599930983 shutdown_connections 2023-09-13T04:40:49.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.500+0000 7fb6de9ff700 1 -- 172.21.15.2:0/1599930983 wait complete. 2023-09-13T04:40:49.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.501+0000 7fb6de9ff700 1 Processor -- start 2023-09-13T04:40:49.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.501+0000 7fb6de9ff700 1 -- start start 2023-09-13T04:40:49.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.502+0000 7fb6de9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6d8100eb0 0x7fb6d8199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.502+0000 7fb6de9ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6d8105fd0 0x7fb6d8199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.502+0000 7fb6de9ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6d819d9a0 0x7fb6d819de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.502+0000 7fb6de9ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d819e420 con 0x7fb6d819d9a0 2023-09-13T04:40:49.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.502+0000 7fb6de9ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d819e590 con 0x7fb6d8105fd0 2023-09-13T04:40:49.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.502+0000 7fb6de9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d819e750 con 0x7fb6d8100eb0 2023-09-13T04:40:49.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.502+0000 7fb6dcf9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6d819d9a0 0x7fb6d819de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.502+0000 7fb6d7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6d8100eb0 0x7fb6d8199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.502+0000 7fb6d7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6d8100eb0 0x7fb6d8199000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:38220/0 (socket says 172.21.15.2:38220) 2023-09-13T04:40:49.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.502+0000 7fb6dcf9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6d819d9a0 0x7fb6d819de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59608/0 (socket says 172.21.15.2:59608) 2023-09-13T04:40:49.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.502+0000 7fb6d7fff700 1 -- 172.21.15.2:0/3361781287 learned_addr learned my addr 172.21.15.2:0/3361781287 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:49.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.502+0000 7fb6d77fe700 1 --2- 172.21.15.2:0/3361781287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6d8105fd0 0x7fb6d8199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.502+0000 7fb6d7fff700 1 -- 172.21.15.2:0/3361781287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6d8105fd0 msgr2=0x7fb6d8199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:49.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.502+0000 7fb6d7fff700 1 --2- 172.21.15.2:0/3361781287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6d8105fd0 0x7fb6d8199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.502+0000 7fb6d7fff700 1 -- 172.21.15.2:0/3361781287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6d819d9a0 msgr2=0x7fb6d819de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:49.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.502+0000 7fb6d7fff700 1 --2- 172.21.15.2:0/3361781287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6d819d9a0 0x7fb6d819de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.502+0000 7fb6d7fff700 1 -- 172.21.15.2:0/3361781287 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c8009520 con 0x7fb6d8100eb0 2023-09-13T04:40:49.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.503+0000 7fb6d7fff700 1 --2- 172.21.15.2:0/3361781287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6d8100eb0 0x7fb6d8199000 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb6c800eb30 tx=0x7fb6c800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:49.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.503+0000 7fb6d57fa700 1 -- 172.21.15.2:0/3361781287 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8009880 con 0x7fb6d8100eb0 2023-09-13T04:40:49.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.503+0000 7fb6de9ff700 1 -- 172.21.15.2:0/3361781287 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6cc0095f0 con 0x7fb6d8100eb0 2023-09-13T04:40:49.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.503+0000 7fb6d57fa700 1 -- 172.21.15.2:0/3361781287 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6c80044e0 con 0x7fb6d8100eb0 2023-09-13T04:40:49.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.503+0000 7fb6d57fa700 1 -- 172.21.15.2:0/3361781287 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8003d80 con 0x7fb6d8100eb0 2023-09-13T04:40:49.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.503+0000 7fb6de9ff700 1 -- 172.21.15.2:0/3361781287 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d81a6050 con 0x7fb6d8100eb0 2023-09-13T04:40:49.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.506+0000 7fb6d57fa700 1 -- 172.21.15.2:0/3361781287 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 23) v1 ==== 135434+0+0 (secure 0 0 0) 0x7fb6c8003f20 con 0x7fb6d8100eb0 2023-09-13T04:40:49.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.507+0000 7fb6f234ab80 1 Processor -- start 2023-09-13T04:40:49.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.508+0000 7fb6f234ab80 1 -- start start 2023-09-13T04:40:49.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.508+0000 7fb6d57fa700 1 --2- 172.21.15.2:0/3361781287 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fb6c00a0840 0x7fb6c00a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.508+0000 7fb6f234ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a596fe0 0x55597a5973c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.508+0000 7fb6f234ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55597a597900 0x55597a5914c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.508+0000 7fb6f234ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597a591a00 0x55597a591e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.508+0000 7fb6f234ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597a4ea780 con 0x55597a596fe0 2023-09-13T04:40:49.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.508+0000 7fb6f234ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597a4e80a0 con 0x55597a597900 2023-09-13T04:40:49.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.508+0000 7fb6f234ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597a4e3990 con 0x55597a591a00 2023-09-13T04:40:49.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.508+0000 7fb6d57fa700 1 -- 172.21.15.2:0/3361781287 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fb6c8014070 con 0x7fb6d8100eb0 2023-09-13T04:40:49.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.508+0000 7fb6dcf9c700 1 --2- 172.21.15.2:0/3361781287 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fb6c00a0840 0x7fb6c00a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.508+0000 7fb6d7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a596fe0 0x55597a5973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.508+0000 7fb6d77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55597a597900 0x55597a5914c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.508+0000 7fb6d7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a596fe0 0x55597a5973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59610/0 (socket says 172.21.15.2:59610) 2023-09-13T04:40:49.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.508+0000 7fb6d7fff700 1 -- 172.21.15.2:0/2439222942 learned_addr learned my addr 172.21.15.2:0/2439222942 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:49.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.508+0000 7fb6dcf9c700 1 --2- 172.21.15.2:0/2439222942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597a591a00 0x55597a591e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.509+0000 7fb6d7fff700 1 -- 172.21.15.2:0/2439222942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597a591a00 msgr2=0x55597a591e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:49.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.509+0000 7fb6d7fff700 1 --2- 172.21.15.2:0/2439222942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597a591a00 0x55597a591e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.509+0000 7fb6d7fff700 1 -- 172.21.15.2:0/2439222942 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55597a597900 msgr2=0x55597a5914c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:49.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.509+0000 7fb6d7fff700 1 --2- 172.21.15.2:0/2439222942 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55597a597900 0x55597a5914c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.509+0000 7fb6d7fff700 1 -- 172.21.15.2:0/2439222942 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55597a363630 con 0x55597a596fe0 2023-09-13T04:40:49.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.509+0000 7fb6d7fff700 1 --2- 172.21.15.2:0/2439222942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a596fe0 0x55597a5973c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80c9da0 tx=0x7fb6c8099580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8d4534a142395d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:49.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.509+0000 7fb6dcf9c700 1 --2- 172.21.15.2:0/3361781287 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fb6c00a0840 0x7fb6c00a2d00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009550 tx=0x7fb6d0003a70 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:49.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.509+0000 7fb6be7fc700 1 -- 172.21.15.2:0/2439222942 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c80b9110 con 0x55597a596fe0 2023-09-13T04:40:49.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.509+0000 7fb6be7fc700 1 -- 172.21.15.2:0/2439222942 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6c8049130 con 0x55597a596fe0 2023-09-13T04:40:49.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.509+0000 7fb6be7fc700 1 -- 172.21.15.2:0/2439222942 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c80999c0 con 0x55597a596fe0 2023-09-13T04:40:49.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.510+0000 7fb6f234ab80 1 -- 172.21.15.2:0/2439222942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a596fe0 msgr2=0x55597a5973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:49.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.510+0000 7fb6f234ab80 1 --2- 172.21.15.2:0/2439222942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a596fe0 0x55597a5973c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80c9da0 tx=0x7fb6c8099580 comp rx=0 tx=0).stop 2023-09-13T04:40:49.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.510+0000 7fb6f234ab80 1 -- 172.21.15.2:0/2439222942 shutdown_connections 2023-09-13T04:40:49.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.510+0000 7fb6f234ab80 1 --2- 172.21.15.2:0/2439222942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597a591a00 0x55597a591e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.510+0000 7fb6f234ab80 1 --2- 172.21.15.2:0/2439222942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a596fe0 0x55597a5973c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.510+0000 7fb6f234ab80 1 --2- 172.21.15.2:0/2439222942 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55597a597900 0x55597a5914c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.510+0000 7fb6f234ab80 1 -- 172.21.15.2:0/2439222942 >> 172.21.15.2:0/2439222942 conn(0x55597a4efc80 msgr2=0x55597a58bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:49.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.510+0000 7fb6f234ab80 1 -- 172.21.15.2:0/2439222942 shutdown_connections 2023-09-13T04:40:49.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.510+0000 7fb6f234ab80 1 -- 172.21.15.2:0/2439222942 wait complete. 2023-09-13T04:40:49.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.511+0000 7fb6f234ab80 1 -- start start 2023-09-13T04:40:49.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.511+0000 7fb6f234ab80 1 Processor -- start 2023-09-13T04:40:49.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.511+0000 7fb6f234ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597a591a00 0x55597a59d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.511+0000 7fb6f234ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55597a596fe0 0x55597a59ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.511+0000 7fb6f234ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a597900 0x55597a59b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.511+0000 7fb6f234ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597a481bb0 con 0x55597a597900 2023-09-13T04:40:49.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.511+0000 7fb6f234ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597a4818b0 con 0x55597a596fe0 2023-09-13T04:40:49.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.511+0000 7fb6f234ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597a365920 con 0x55597a591a00 2023-09-13T04:40:49.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.511+0000 7fb6f234ab80 10 client.0 _ll_register_callbacks cb 0x55597a47f680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:49.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.512+0000 7fb6d77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55597a596fe0 0x55597a59ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.512+0000 7fb6d7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597a591a00 0x55597a59d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.512+0000 7fb6dcf9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a597900 0x55597a59b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.512+0000 7fb6d77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55597a596fe0 0x55597a59ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:58692/0 (socket says 172.21.15.2:58692) 2023-09-13T04:40:49.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.512+0000 7fb6d77fe700 1 -- 172.21.15.2:0/1039846377 learned_addr learned my addr 172.21.15.2:0/1039846377 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:49.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.512+0000 7fb6d7fff700 1 -- 172.21.15.2:0/1039846377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55597a596fe0 msgr2=0x55597a59ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:49.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.512+0000 7fb6d7fff700 1 --2- 172.21.15.2:0/1039846377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55597a596fe0 0x55597a59ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.512+0000 7fb6d7fff700 1 -- 172.21.15.2:0/1039846377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a597900 msgr2=0x55597a59b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:49.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.512+0000 7fb6d7fff700 1 --2- 172.21.15.2:0/1039846377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a597900 0x55597a59b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.512+0000 7fb6d7fff700 1 -- 172.21.15.2:0/1039846377 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d81a6050 con 0x55597a591a00 2023-09-13T04:40:49.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.512+0000 7fb6d7fff700 1 --2- 172.21.15.2:0/1039846377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597a591a00 0x55597a59d8c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80c9da0 tx=0x7fb6c8003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=312e2408d3c7bb59 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:49.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.512+0000 7fb6b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:40:49.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.512+0000 7fb6b77fe700 1 -- 172.21.15.2:0/1039846377 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c800cbd0 con 0x55597a591a00 2023-09-13T04:40:49.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.513+0000 7fb6b77fe700 1 -- 172.21.15.2:0/1039846377 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fb6c809a0c0 con 0x55597a591a00 2023-09-13T04:40:49.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.513+0000 7fb6f234ab80 1 -- 172.21.15.2:0/1039846377 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55597a584670 con 0x55597a591a00 2023-09-13T04:40:49.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.513+0000 7fb6b77fe700 1 -- 172.21.15.2:0/1039846377 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6c809d7f0 con 0x55597a591a00 2023-09-13T04:40:49.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.513+0000 7fb6f234ab80 10 client.25693 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:49.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.513+0000 7fb6f234ab80 1 -- 172.21.15.2:0/1039846377 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55597a4fb7a0 con 0x55597a591a00 2023-09-13T04:40:49.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.513+0000 7fb6b77fe700 1 -- 172.21.15.2:0/1039846377 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55597a584670 con 0x55597a591a00 2023-09-13T04:40:49.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.513+0000 7fb6b77fe700 1 -- 172.21.15.2:0/1039846377 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb6c8098eb0 con 0x55597a591a00 2023-09-13T04:40:49.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.513+0000 7fb6f234ab80 10 client.25693 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:49.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.513+0000 7fb6f234ab80 10 client.25693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:40:49.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.513+0000 7fb6f234ab80 20 client.25693 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:49.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.513+0000 7fb6f234ab80 1 --2- 172.21.15.2:0/1039846377 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55597a5aada0 0x55597a5ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.513+0000 7fb6f234ab80 4 client.25693 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:40:49.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.513+0000 7fb6f234ab80 1 -- 172.21.15.2:0/1039846377 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55597a4dc970 con 0x55597a5aada0 2023-09-13T04:40:49.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.514+0000 7fb6d77fe700 1 --2- 172.21.15.2:0/1039846377 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55597a5aada0 0x55597a5ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.514+0000 7fb6d77fe700 1 --2- 172.21.15.2:0/1039846377 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55597a5aada0 0x55597a5ad1c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c71b8136a11599fa server_cookie=d42286b97cef4835 in_seq=0 out_seq=0 2023-09-13T04:40:49.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.514+0000 7fb6b77fe700 10 client.25693 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:49.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.515+0000 7fb6b77fe700 1 -- 172.21.15.2:0/1039846377 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55597a4dc970 con 0x55597a5aada0 2023-09-13T04:40:49.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.515+0000 7fb6b77fe700 10 client.25693 handle_command_reply: tid=0 2023-09-13T04:40:49.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.515+0000 7fb6f234ab80 1 client.25693 shutdown 2023-09-13T04:40:49.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.516+0000 7fb6f234ab80 1 -- 172.21.15.2:0/1039846377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597a591a00 msgr2=0x55597a59d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:49.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.516+0000 7fb6f234ab80 1 --2- 172.21.15.2:0/1039846377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597a591a00 0x55597a59d8c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80c9da0 tx=0x7fb6c8003820 comp rx=0 tx=0).stop 2023-09-13T04:40:49.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.516+0000 7fb6f234ab80 1 -- 172.21.15.2:0/1039846377 shutdown_connections 2023-09-13T04:40:49.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.516+0000 7fb6f234ab80 1 --2- 172.21.15.2:0/1039846377 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55597a5aada0 0x55597a5ad1c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.516+0000 7fb6f234ab80 1 --2- 172.21.15.2:0/1039846377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a597900 0x55597a59b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.516+0000 7fb6f234ab80 1 --2- 172.21.15.2:0/1039846377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597a591a00 0x55597a59d8c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.516+0000 7fb6b77fe700 0 client.25693 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:49.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.516+0000 7fb6f234ab80 1 --2- 172.21.15.2:0/1039846377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55597a596fe0 0x55597a59ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.516+0000 7fb6f234ab80 1 -- 172.21.15.2:0/1039846377 >> 172.21.15.2:0/1039846377 conn(0x55597a4efc80 msgr2=0x55597a58be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:49.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.516+0000 7fb6f234ab80 1 -- 172.21.15.2:0/1039846377 shutdown_connections 2023-09-13T04:40:49.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.516+0000 7fb6f234ab80 1 -- 172.21.15.2:0/1039846377 wait complete. 2023-09-13T04:40:49.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.516+0000 7fb6f234ab80 20 client.25693 trim_cache size 0 max 16384 2023-09-13T04:40:49.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.529+0000 7fb6f234ab80 1 Processor -- start 2023-09-13T04:40:49.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.530+0000 7fb6f234ab80 1 -- start start 2023-09-13T04:40:49.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.530+0000 7fb6f234ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597a591a00 0x55597a5914c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.530+0000 7fb6f234ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55597a596fe0 0x55597a5a2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.530+0000 7fb6f234ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a597900 0x55597a5868f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.530+0000 7fb6f234ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597a58cfb0 con 0x55597a597900 2023-09-13T04:40:49.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.530+0000 7fb6f234ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597a59c120 con 0x55597a596fe0 2023-09-13T04:40:49.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.530+0000 7fb6f234ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597a5888e0 con 0x55597a591a00 2023-09-13T04:40:49.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.530+0000 7fb6dcf9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a597900 0x55597a5868f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.530+0000 7fb6dcf9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a597900 0x55597a5868f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59632/0 (socket says 172.21.15.2:59632) 2023-09-13T04:40:49.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.530+0000 7fb6dcf9c700 1 -- 172.21.15.2:0/1163524991 learned_addr learned my addr 172.21.15.2:0/1163524991 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:49.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.530+0000 7fb6d7fff700 1 --2- 172.21.15.2:0/1163524991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597a591a00 0x55597a5914c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.530+0000 7fb6d77fe700 1 --2- 172.21.15.2:0/1163524991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55597a596fe0 0x55597a5a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.530+0000 7fb6dcf9c700 1 -- 172.21.15.2:0/1163524991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597a591a00 msgr2=0x55597a5914c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:49.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.530+0000 7fb6dcf9c700 1 --2- 172.21.15.2:0/1163524991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597a591a00 0x55597a5914c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.531+0000 7fb6dcf9c700 1 -- 172.21.15.2:0/1163524991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55597a596fe0 msgr2=0x55597a5a2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:49.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.531+0000 7fb6dcf9c700 1 --2- 172.21.15.2:0/1163524991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55597a596fe0 0x55597a5a2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.531+0000 7fb6dcf9c700 1 -- 172.21.15.2:0/1163524991 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55597a5b1db0 con 0x55597a597900 2023-09-13T04:40:49.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.531+0000 7fb6dcf9c700 1 --2- 172.21.15.2:0/1163524991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a597900 0x55597a5868f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0003f00 tx=0x7fb6d001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58323383aee0d139 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:49.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.531+0000 7fb6beffd700 1 -- 172.21.15.2:0/1163524991 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d001adb0 con 0x55597a597900 2023-09-13T04:40:49.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.531+0000 7fb6beffd700 1 -- 172.21.15.2:0/1163524991 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6d000f040 con 0x55597a597900 2023-09-13T04:40:49.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.531+0000 7fb6beffd700 1 -- 172.21.15.2:0/1163524991 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0007770 con 0x55597a597900 2023-09-13T04:40:49.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.532+0000 7fb6f234ab80 1 -- 172.21.15.2:0/1163524991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a597900 msgr2=0x55597a5868f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:49.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.532+0000 7fb6f234ab80 1 --2- 172.21.15.2:0/1163524991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a597900 0x55597a5868f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0003f00 tx=0x7fb6d001a000 comp rx=0 tx=0).stop 2023-09-13T04:40:49.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.532+0000 7fb6f234ab80 1 -- 172.21.15.2:0/1163524991 shutdown_connections 2023-09-13T04:40:49.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.532+0000 7fb6f234ab80 1 --2- 172.21.15.2:0/1163524991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a597900 0x55597a5868f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.532+0000 7fb6f234ab80 1 --2- 172.21.15.2:0/1163524991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597a591a00 0x55597a5914c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.532+0000 7fb6f234ab80 1 --2- 172.21.15.2:0/1163524991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55597a596fe0 0x55597a5a2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.532+0000 7fb6f234ab80 1 -- 172.21.15.2:0/1163524991 >> 172.21.15.2:0/1163524991 conn(0x55597a4efc80 msgr2=0x55597a5a7470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:49.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.532+0000 7fb6f234ab80 1 -- 172.21.15.2:0/1163524991 shutdown_connections 2023-09-13T04:40:49.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.532+0000 7fb6f234ab80 1 -- 172.21.15.2:0/1163524991 wait complete. 2023-09-13T04:40:49.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.532+0000 7fb6f234ab80 1 -- start start 2023-09-13T04:40:49.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.532+0000 7fb6f234ab80 1 Processor -- start 2023-09-13T04:40:49.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.533+0000 7fb6f234ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55597a591a00 0x55597a592a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.533+0000 7fb6f234ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597a596fe0 0x55597a592fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.533+0000 7fb6f234ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a597900 0x55597a593560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.533+0000 7fb6f234ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597a593c70 con 0x55597a597900 2023-09-13T04:40:49.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.533+0000 7fb6f234ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597a593de0 con 0x55597a591a00 2023-09-13T04:40:49.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.533+0000 7fb6f234ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597a593fa0 con 0x55597a596fe0 2023-09-13T04:40:49.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.533+0000 7fb6f234ab80 10 client.0 _ll_register_callbacks cb 0x55597a47f680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:49.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.533+0000 7fb6dcf9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a597900 0x55597a593560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.533+0000 7fb6dcf9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a597900 0x55597a593560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59636/0 (socket says 172.21.15.2:59636) 2023-09-13T04:40:49.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.533+0000 7fb6dcf9c700 1 -- 172.21.15.2:0/4189574855 learned_addr learned my addr 172.21.15.2:0/4189574855 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:49.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.533+0000 7fb6d77fe700 1 --2- 172.21.15.2:0/4189574855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597a596fe0 0x55597a592fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.533+0000 7fb6d7fff700 1 --2- 172.21.15.2:0/4189574855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55597a591a00 0x55597a592a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.534+0000 7fb6dcf9c700 1 -- 172.21.15.2:0/4189574855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597a596fe0 msgr2=0x55597a592fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:49.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.534+0000 7fb6dcf9c700 1 --2- 172.21.15.2:0/4189574855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597a596fe0 0x55597a592fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.534+0000 7fb6dcf9c700 1 -- 172.21.15.2:0/4189574855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55597a591a00 msgr2=0x55597a592a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:49.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.534+0000 7fb6dcf9c700 1 --2- 172.21.15.2:0/4189574855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55597a591a00 0x55597a592a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.534+0000 7fb6dcf9c700 1 -- 172.21.15.2:0/4189574855 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d0009da0 con 0x55597a597900 2023-09-13T04:40:49.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.534+0000 7fb6dcf9c700 1 --2- 172.21.15.2:0/4189574855 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a597900 0x55597a593560 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009910 tx=0x7fb6d001ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578c110c7b37b383 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:49.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.534+0000 7fb6b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:40:49.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.534+0000 7fb6b77fe700 1 -- 172.21.15.2:0/4189574855 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0004ea0 con 0x55597a597900 2023-09-13T04:40:49.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.534+0000 7fb6f234ab80 1 -- 172.21.15.2:0/4189574855 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6c8049130 con 0x55597a597900 2023-09-13T04:40:49.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.534+0000 7fb6b77fe700 1 -- 172.21.15.2:0/4189574855 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fb6d0025400 con 0x55597a597900 2023-09-13T04:40:49.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.534+0000 7fb6b77fe700 1 -- 172.21.15.2:0/4189574855 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6d00255c0 con 0x55597a597900 2023-09-13T04:40:49.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.535+0000 7fb6f234ab80 10 client.15663 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:49.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.535+0000 7fb6f234ab80 1 -- 172.21.15.2:0/4189574855 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55597a30fde0 con 0x55597a597900 2023-09-13T04:40:49.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.535+0000 7fb6b77fe700 1 -- 172.21.15.2:0/4189574855 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6c8049130 con 0x55597a597900 2023-09-13T04:40:49.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.535+0000 7fb6b77fe700 1 -- 172.21.15.2:0/4189574855 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb6d0005770 con 0x55597a597900 2023-09-13T04:40:49.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.535+0000 7fb6f234ab80 10 client.15663 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:49.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.535+0000 7fb6f234ab80 10 client.15663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:40:49.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.535+0000 7fb6f234ab80 20 client.15663 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:49.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.535+0000 7fb6f234ab80 1 --2- 172.21.15.2:0/4189574855 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55597a5ff8c0 0x55597a601ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:49.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.535+0000 7fb6f234ab80 4 client.15663 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-09-13T04:40:49.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.535+0000 7fb6f234ab80 1 -- 172.21.15.2:0/4189574855 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55597a49d050 con 0x55597a5ff8c0 2023-09-13T04:40:49.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.536+0000 7fb6d7fff700 1 --2- 172.21.15.2:0/4189574855 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55597a5ff8c0 0x55597a601ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:49.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.536+0000 7fb6b77fe700 10 client.15663 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:49.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.536+0000 7fb6d7fff700 1 --2- 172.21.15.2:0/4189574855 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55597a5ff8c0 0x55597a601ce0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a97b491b1430043 server_cookie=e64402565cde6d30 in_seq=0 out_seq=0 2023-09-13T04:40:49.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.536+0000 7fb6b77fe700 1 -- 172.21.15.2:0/4189574855 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55597a49d050 con 0x55597a5ff8c0 2023-09-13T04:40:49.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.536+0000 7fb6b77fe700 10 client.15663 handle_command_reply: tid=0 2023-09-13T04:40:49.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.536+0000 7fb6f234ab80 1 client.15663 shutdown 2023-09-13T04:40:49.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.536+0000 7fb6f234ab80 1 -- 172.21.15.2:0/4189574855 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a597900 msgr2=0x55597a593560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:49.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.536+0000 7fb6f234ab80 1 --2- 172.21.15.2:0/4189574855 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a597900 0x55597a593560 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009910 tx=0x7fb6d001ade0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.536+0000 7fb6f234ab80 1 -- 172.21.15.2:0/4189574855 shutdown_connections 2023-09-13T04:40:49.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.536+0000 7fb6f234ab80 1 --2- 172.21.15.2:0/4189574855 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55597a5ff8c0 0x55597a601ce0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.536+0000 7fb6f234ab80 1 --2- 172.21.15.2:0/4189574855 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55597a597900 0x55597a593560 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.536+0000 7fb6b77fe700 0 client.15663 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:49.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.536+0000 7fb6f234ab80 1 --2- 172.21.15.2:0/4189574855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55597a591a00 0x55597a592a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.537+0000 7fb6f234ab80 1 --2- 172.21.15.2:0/4189574855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597a596fe0 0x55597a592fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.537+0000 7fb6f234ab80 1 -- 172.21.15.2:0/4189574855 >> 172.21.15.2:0/4189574855 conn(0x55597a4efc80 msgr2=0x55597a595be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:49.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.537+0000 7fb6f234ab80 1 -- 172.21.15.2:0/4189574855 shutdown_connections 2023-09-13T04:40:49.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.537+0000 7fb6f234ab80 1 -- 172.21.15.2:0/4189574855 wait complete. 2023-09-13T04:40:49.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.537+0000 7fb6f234ab80 20 client.15663 trim_cache size 0 max 16384 2023-09-13T04:40:49.559 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-09-13T04:40:49.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.537+0000 7fb6de9ff700 1 -- 172.21.15.2:0/3361781287 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fb6c00a0840 msgr2=0x7fb6c00a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:49.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.537+0000 7fb6de9ff700 1 --2- 172.21.15.2:0/3361781287 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fb6c00a0840 0x7fb6c00a2d00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009550 tx=0x7fb6d0003a70 comp rx=0 tx=0).stop 2023-09-13T04:40:49.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.537+0000 7fb6de9ff700 1 -- 172.21.15.2:0/3361781287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6d8100eb0 msgr2=0x7fb6d8199000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:49.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.537+0000 7fb6de9ff700 1 --2- 172.21.15.2:0/3361781287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6d8100eb0 0x7fb6d8199000 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb6c800eb30 tx=0x7fb6c800ee40 comp rx=0 tx=0).stop 2023-09-13T04:40:49.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.538+0000 7fb6de9ff700 1 -- 172.21.15.2:0/3361781287 shutdown_connections 2023-09-13T04:40:49.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.538+0000 7fb6de9ff700 1 --2- 172.21.15.2:0/3361781287 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fb6c00a0840 0x7fb6c00a2d00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.538+0000 7fb6de9ff700 1 --2- 172.21.15.2:0/3361781287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6d819d9a0 0x7fb6d819de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.538+0000 7fb6de9ff700 1 --2- 172.21.15.2:0/3361781287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6d8100eb0 0x7fb6d8199000 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.538+0000 7fb6de9ff700 1 --2- 172.21.15.2:0/3361781287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6d8105fd0 0x7fb6d8199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:49.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.538+0000 7fb6de9ff700 1 -- 172.21.15.2:0/3361781287 >> 172.21.15.2:0/3361781287 conn(0x7fb6d80fc750 msgr2=0x7fb6d80fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:49.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.538+0000 7fb6de9ff700 1 -- 172.21.15.2:0/3361781287 shutdown_connections 2023-09-13T04:40:49.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:49.538+0000 7fb6de9ff700 1 -- 172.21.15.2:0/3361781287 wait complete. 2023-09-13T04:40:50.564 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-13T04:40:50.565 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-13T04:40:50.633 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:50 smithi002 ceph-mon[97651]: pgmap v14: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 8 op/s 2023-09-13T04:40:50.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:50 smithi149 ceph-mon[133315]: pgmap v14: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 8 op/s 2023-09-13T04:40:50.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:50 smithi156 ceph-mon[133675]: pgmap v14: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 8 op/s 2023-09-13T04:40:50.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.782+0000 7fa810dff700 1 -- 172.21.15.2:0/396638045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa80c1027d0 msgr2=0x7fa80c10acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:50.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.782+0000 7fa810dff700 1 --2- 172.21.15.2:0/396638045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa80c1027d0 0x7fa80c10acb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4009990 tx=0x7fa7f4009ca0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.782+0000 7fa810dff700 1 -- 172.21.15.2:0/396638045 shutdown_connections 2023-09-13T04:40:50.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.782+0000 7fa810dff700 1 --2- 172.21.15.2:0/396638045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa80c10b280 0x7fa80c10d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.783+0000 7fa810dff700 1 --2- 172.21.15.2:0/396638045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa80c101eb0 0x7fa80c102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.783+0000 7fa810dff700 1 --2- 172.21.15.2:0/396638045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa80c1027d0 0x7fa80c10acb0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.783+0000 7fa810dff700 1 -- 172.21.15.2:0/396638045 >> 172.21.15.2:0/396638045 conn(0x7fa80c0fb630 msgr2=0x7fa80c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:50.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.783+0000 7fa810dff700 1 -- 172.21.15.2:0/396638045 shutdown_connections 2023-09-13T04:40:50.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.783+0000 7fa810dff700 1 -- 172.21.15.2:0/396638045 wait complete. 2023-09-13T04:40:50.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.784+0000 7fa810dff700 1 Processor -- start 2023-09-13T04:40:50.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.784+0000 7fa810dff700 1 -- start start 2023-09-13T04:40:50.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.784+0000 7fa810dff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa80c101eb0 0x7fa80c198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:50.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.784+0000 7fa810dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa80c1027d0 0x7fa80c199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:50.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.784+0000 7fa810dff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa80c10b280 0x7fa80c19d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:50.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.784+0000 7fa810dff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c19df30 con 0x7fa80c10b280 2023-09-13T04:40:50.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.784+0000 7fa810dff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c19e0a0 con 0x7fa80c101eb0 2023-09-13T04:40:50.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.784+0000 7fa810dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c19e260 con 0x7fa80c1027d0 2023-09-13T04:40:50.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.784+0000 7fa80ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa80c10b280 0x7fa80c19d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:50.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.784+0000 7fa80ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa80c10b280 0x7fa80c19d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59650/0 (socket says 172.21.15.2:59650) 2023-09-13T04:40:50.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.784+0000 7fa80ad9d700 1 -- 172.21.15.2:0/2214974270 learned_addr learned my addr 172.21.15.2:0/2214974270 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:50.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.784+0000 7fa80a59c700 1 --2- 172.21.15.2:0/2214974270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa80c101eb0 0x7fa80c198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:50.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.785+0000 7fa803fff700 1 --2- 172.21.15.2:0/2214974270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa80c1027d0 0x7fa80c199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:50.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.785+0000 7fa80ad9d700 1 -- 172.21.15.2:0/2214974270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa80c1027d0 msgr2=0x7fa80c199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:50.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.785+0000 7fa80ad9d700 1 --2- 172.21.15.2:0/2214974270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa80c1027d0 0x7fa80c199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.785+0000 7fa80ad9d700 1 -- 172.21.15.2:0/2214974270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa80c101eb0 msgr2=0x7fa80c198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:50.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.785+0000 7fa80ad9d700 1 --2- 172.21.15.2:0/2214974270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa80c101eb0 0x7fa80c198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.785+0000 7fa80ad9d700 1 -- 172.21.15.2:0/2214974270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f40095f0 con 0x7fa80c10b280 2023-09-13T04:40:50.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.785+0000 7fa803fff700 1 --2- 172.21.15.2:0/2214974270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa80c1027d0 0x7fa80c199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:40:50.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.785+0000 7fa80ad9d700 1 --2- 172.21.15.2:0/2214974270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa80c10b280 0x7fa80c19d8f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa80400ef30 tx=0x7fa80400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:50.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.785+0000 7fa8037fe700 1 -- 172.21.15.2:0/2214974270 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa804018070 con 0x7fa80c10b280 2023-09-13T04:40:50.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.785+0000 7fa8037fe700 1 -- 172.21.15.2:0/2214974270 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa80400f040 con 0x7fa80c10b280 2023-09-13T04:40:50.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.785+0000 7fa810dff700 1 -- 172.21.15.2:0/2214974270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa80c19e4e0 con 0x7fa80c10b280 2023-09-13T04:40:50.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.785+0000 7fa8037fe700 1 -- 172.21.15.2:0/2214974270 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80400cdb0 con 0x7fa80c10b280 2023-09-13T04:40:50.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.785+0000 7fa810dff700 1 -- 172.21.15.2:0/2214974270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa80c1a5d80 con 0x7fa80c10b280 2023-09-13T04:40:50.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.789+0000 7fa8037fe700 1 -- 172.21.15.2:0/2214974270 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 23) v1 ==== 135434+0+0 (secure 0 0 0) 0x7fa804003a00 con 0x7fa80c10b280 2023-09-13T04:40:50.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.790+0000 7fa824762b80 1 Processor -- start 2023-09-13T04:40:50.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.790+0000 7fa824762b80 1 -- start start 2023-09-13T04:40:50.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.791+0000 7fa8037fe700 1 --2- 172.21.15.2:0/2214974270 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fa7f80a08c0 0x7fa7f80a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:50.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.791+0000 7fa824762b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef82dd5060 0x55ef82dd5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:50.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.791+0000 7fa824762b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dd5980 0x55ef82dcf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:50.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.791+0000 7fa824762b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef82dcfa80 0x55ef82dcfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:50.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.791+0000 7fa824762b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef82d28800 con 0x55ef82dd5980 2023-09-13T04:40:50.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.791+0000 7fa824762b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef82d26120 con 0x55ef82dcfa80 2023-09-13T04:40:50.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.791+0000 7fa824762b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef82d21a10 con 0x55ef82dd5060 2023-09-13T04:40:50.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.791+0000 7fa8037fe700 1 -- 172.21.15.2:0/2214974270 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fa8040cbff0 con 0x7fa80c10b280 2023-09-13T04:40:50.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.791+0000 7fa803fff700 1 --2- 172.21.15.2:0/2214974270 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fa7f80a08c0 0x7fa7f80a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:50.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.791+0000 7fa803fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dd5980 0x55ef82dcf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:50.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.791+0000 7fa80a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef82dd5060 0x55ef82dd5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:50.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.791+0000 7fa80a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef82dd5060 0x55ef82dd5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:38268/0 (socket says 172.21.15.2:38268) 2023-09-13T04:40:50.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.791+0000 7fa803fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dd5980 0x55ef82dcf540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59662/0 (socket says 172.21.15.2:59662) 2023-09-13T04:40:50.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.791+0000 7fa80a59c700 1 -- 172.21.15.2:0/3037639437 learned_addr learned my addr 172.21.15.2:0/3037639437 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:50.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.791+0000 7fa80ad9d700 1 --2- 172.21.15.2:0/3037639437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef82dcfa80 0x55ef82dcfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:50.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.792+0000 7fa80ad9d700 1 -- 172.21.15.2:0/3037639437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef82dd5060 msgr2=0x55ef82dd5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:50.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.792+0000 7fa80ad9d700 1 --2- 172.21.15.2:0/3037639437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef82dd5060 0x55ef82dd5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.792+0000 7fa80ad9d700 1 -- 172.21.15.2:0/3037639437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dd5980 msgr2=0x55ef82dcf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:50.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.792+0000 7fa80ad9d700 1 --2- 172.21.15.2:0/3037639437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dd5980 0x55ef82dcf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.792+0000 7fa80ad9d700 1 -- 172.21.15.2:0/3037639437 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef82ba16b0 con 0x55ef82dcfa80 2023-09-13T04:40:50.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.792+0000 7fa80a59c700 1 --2- 172.21.15.2:0/3037639437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef82dd5060 0x55ef82dd5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:50.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.792+0000 7fa803fff700 1 --2- 172.21.15.2:0/2214974270 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fa7f80a08c0 0x7fa7f80a2d80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa80c19a560 tx=0x7fa7f4005590 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:50.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.792+0000 7fa80ad9d700 1 --2- 172.21.15.2:0/3037639437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef82dcfa80 0x55ef82dcfee0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa8040c9330 tx=0x7fa804098ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2011ce7261fe743 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:50.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.792+0000 7fa800ff9700 1 -- 172.21.15.2:0/3037639437 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80401b320 con 0x55ef82dcfa80 2023-09-13T04:40:50.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.792+0000 7fa800ff9700 1 -- 172.21.15.2:0/3037639437 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa804099770 con 0x55ef82dcfa80 2023-09-13T04:40:50.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.792+0000 7fa800ff9700 1 -- 172.21.15.2:0/3037639437 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8040c95e0 con 0x55ef82dcfa80 2023-09-13T04:40:50.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.793+0000 7fa824762b80 1 -- 172.21.15.2:0/3037639437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef82dcfa80 msgr2=0x55ef82dcfee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:50.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.793+0000 7fa824762b80 1 --2- 172.21.15.2:0/3037639437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef82dcfa80 0x55ef82dcfee0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa8040c9330 tx=0x7fa804098ec0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.793+0000 7fa824762b80 1 -- 172.21.15.2:0/3037639437 shutdown_connections 2023-09-13T04:40:50.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.793+0000 7fa824762b80 1 --2- 172.21.15.2:0/3037639437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef82dcfa80 0x55ef82dcfee0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.793+0000 7fa824762b80 1 --2- 172.21.15.2:0/3037639437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef82dd5060 0x55ef82dd5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.793+0000 7fa824762b80 1 --2- 172.21.15.2:0/3037639437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dd5980 0x55ef82dcf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.793+0000 7fa824762b80 1 -- 172.21.15.2:0/3037639437 >> 172.21.15.2:0/3037639437 conn(0x55ef82d2dd00 msgr2=0x55ef82dc9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:50.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.793+0000 7fa824762b80 1 -- 172.21.15.2:0/3037639437 shutdown_connections 2023-09-13T04:40:50.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.793+0000 7fa824762b80 1 -- 172.21.15.2:0/3037639437 wait complete. 2023-09-13T04:40:50.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.794+0000 7fa824762b80 1 -- start start 2023-09-13T04:40:50.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.794+0000 7fa824762b80 1 Processor -- start 2023-09-13T04:40:50.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.794+0000 7fa824762b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dcfa80 0x55ef82dd8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:50.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.794+0000 7fa824762b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef82dd5060 0x55ef82dd9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:50.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.794+0000 7fa824762b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef82dd5980 0x55ef82dd96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:50.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.794+0000 7fa824762b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef82cbfc30 con 0x55ef82dcfa80 2023-09-13T04:40:50.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.794+0000 7fa824762b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef82cbf930 con 0x55ef82dd5980 2023-09-13T04:40:50.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.794+0000 7fa824762b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef82ba39a0 con 0x55ef82dd5060 2023-09-13T04:40:50.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.794+0000 7fa824762b80 10 client.0 _ll_register_callbacks cb 0x55ef82cbd700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:50.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.795+0000 7fa80a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dcfa80 0x55ef82dd8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:50.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.795+0000 7fa80a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dcfa80 0x55ef82dd8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59670/0 (socket says 172.21.15.2:59670) 2023-09-13T04:40:50.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.795+0000 7fa80a59c700 1 -- 172.21.15.2:0/4237183673 learned_addr learned my addr 172.21.15.2:0/4237183673 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:50.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.795+0000 7fa803fff700 1 --2- 172.21.15.2:0/4237183673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef82dd5060 0x55ef82dd9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:50.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.795+0000 7fa80ad9d700 1 --2- 172.21.15.2:0/4237183673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef82dd5980 0x55ef82dd96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:50.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.795+0000 7fa80a59c700 1 -- 172.21.15.2:0/4237183673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef82dd5060 msgr2=0x55ef82dd9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:50.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.795+0000 7fa80a59c700 1 --2- 172.21.15.2:0/4237183673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef82dd5060 0x55ef82dd9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.795+0000 7fa80a59c700 1 -- 172.21.15.2:0/4237183673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef82dd5980 msgr2=0x55ef82dd96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:50.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.795+0000 7fa80a59c700 1 --2- 172.21.15.2:0/4237183673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef82dd5980 0x55ef82dd96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.795+0000 7fa80a59c700 1 -- 172.21.15.2:0/4237183673 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa80c1a5d80 con 0x55ef82dcfa80 2023-09-13T04:40:50.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.795+0000 7fa80a59c700 1 --2- 172.21.15.2:0/4237183673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dcfa80 0x55ef82dd8c20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc0128d0 tx=0x7fa7fc012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b43ecb483c2fa289 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:50.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.795+0000 7fa7f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:40:50.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.796+0000 7fa7f1ffb700 1 -- 172.21.15.2:0/4237183673 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc017070 con 0x55ef82dcfa80 2023-09-13T04:40:50.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.796+0000 7fa824762b80 1 -- 172.21.15.2:0/4237183673 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef82dc26f0 con 0x55ef82dcfa80 2023-09-13T04:40:50.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.796+0000 7fa7f1ffb700 1 -- 172.21.15.2:0/4237183673 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fa7fc004050 con 0x55ef82dcfa80 2023-09-13T04:40:50.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.796+0000 7fa7f1ffb700 1 -- 172.21.15.2:0/4237183673 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7fc004210 con 0x55ef82dcfa80 2023-09-13T04:40:50.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.796+0000 7fa824762b80 10 client.15675 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:50.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.796+0000 7fa824762b80 1 -- 172.21.15.2:0/4237183673 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ef82d39820 con 0x55ef82dcfa80 2023-09-13T04:40:50.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.796+0000 7fa7f1ffb700 1 -- 172.21.15.2:0/4237183673 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ef82dc26f0 con 0x55ef82dcfa80 2023-09-13T04:40:50.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.796+0000 7fa7f1ffb700 1 -- 172.21.15.2:0/4237183673 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa7fc00f610 con 0x55ef82dcfa80 2023-09-13T04:40:50.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.796+0000 7fa824762b80 10 client.15675 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:50.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.796+0000 7fa824762b80 10 client.15675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:40:50.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.796+0000 7fa824762b80 20 client.15675 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:50.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.796+0000 7fa824762b80 1 --2- 172.21.15.2:0/4237183673 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55ef82de9290 0x55ef82deb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:50.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.796+0000 7fa824762b80 4 client.15675 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:40:50.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.796+0000 7fa824762b80 1 -- 172.21.15.2:0/4237183673 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef82d1a9f0 con 0x55ef82de9290 2023-09-13T04:40:50.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.797+0000 7fa803fff700 1 --2- 172.21.15.2:0/4237183673 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55ef82de9290 0x55ef82deb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:50.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.797+0000 7fa7f1ffb700 10 client.15675 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:50.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.797+0000 7fa803fff700 1 --2- 172.21.15.2:0/4237183673 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55ef82de9290 0x55ef82deb6b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcdd97677c8b8fee server_cookie=869ac6001c13e08 in_seq=0 out_seq=0 2023-09-13T04:40:50.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.798+0000 7fa7f1ffb700 1 -- 172.21.15.2:0/4237183673 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ef82d1a9f0 con 0x55ef82de9290 2023-09-13T04:40:50.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.798+0000 7fa7f1ffb700 10 client.15675 handle_command_reply: tid=0 2023-09-13T04:40:50.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.798+0000 7fa824762b80 1 client.15675 shutdown 2023-09-13T04:40:50.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.798+0000 7fa824762b80 1 -- 172.21.15.2:0/4237183673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dcfa80 msgr2=0x55ef82dd8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:50.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.798+0000 7fa824762b80 1 --2- 172.21.15.2:0/4237183673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dcfa80 0x55ef82dd8c20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc0128d0 tx=0x7fa7fc012be0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.798+0000 7fa824762b80 1 -- 172.21.15.2:0/4237183673 shutdown_connections 2023-09-13T04:40:50.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.798+0000 7fa824762b80 1 --2- 172.21.15.2:0/4237183673 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55ef82de9290 0x55ef82deb6b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.798+0000 7fa824762b80 1 --2- 172.21.15.2:0/4237183673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef82dd5980 0x55ef82dd96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.798+0000 7fa824762b80 1 --2- 172.21.15.2:0/4237183673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dcfa80 0x55ef82dd8c20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.798+0000 7fa824762b80 1 --2- 172.21.15.2:0/4237183673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef82dd5060 0x55ef82dd9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.798+0000 7fa7f1ffb700 0 client.15675 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:50.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.798+0000 7fa824762b80 1 -- 172.21.15.2:0/4237183673 >> 172.21.15.2:0/4237183673 conn(0x55ef82d2dd00 msgr2=0x55ef82dc4870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:50.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.799+0000 7fa824762b80 1 -- 172.21.15.2:0/4237183673 shutdown_connections 2023-09-13T04:40:50.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.799+0000 7fa824762b80 1 -- 172.21.15.2:0/4237183673 wait complete. 2023-09-13T04:40:50.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.799+0000 7fa824762b80 20 client.15675 trim_cache size 0 max 16384 2023-09-13T04:40:50.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.816+0000 7fa824762b80 1 Processor -- start 2023-09-13T04:40:50.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.816+0000 7fa824762b80 1 -- start start 2023-09-13T04:40:50.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.817+0000 7fa824762b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef82dcfa80 0x55ef82dcf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:50.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.817+0000 7fa824762b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef82dd5060 0x55ef82dd4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:50.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.817+0000 7fa824762b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dd5980 0x55ef82de0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:50.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.817+0000 7fa824762b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef82dca570 con 0x55ef82dd5980 2023-09-13T04:40:50.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.817+0000 7fa824762b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef82de07d0 con 0x55ef82dcfa80 2023-09-13T04:40:50.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.817+0000 7fa824762b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef82dd7180 con 0x55ef82dd5060 2023-09-13T04:40:50.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.817+0000 7fa80ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dd5980 0x55ef82de0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:50.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.817+0000 7fa803fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef82dd5060 0x55ef82dd4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:50.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.817+0000 7fa803fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef82dd5060 0x55ef82dd4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:38290/0 (socket says 172.21.15.2:38290) 2023-09-13T04:40:50.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.817+0000 7fa803fff700 1 -- 172.21.15.2:0/364821715 learned_addr learned my addr 172.21.15.2:0/364821715 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:50.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.817+0000 7fa80a59c700 1 --2- 172.21.15.2:0/364821715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef82dcfa80 0x55ef82dcf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:50.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.817+0000 7fa80ad9d700 1 -- 172.21.15.2:0/364821715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef82dd5060 msgr2=0x55ef82dd4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:50.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.817+0000 7fa80ad9d700 1 --2- 172.21.15.2:0/364821715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef82dd5060 0x55ef82dd4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.817+0000 7fa80ad9d700 1 -- 172.21.15.2:0/364821715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef82dcfa80 msgr2=0x55ef82dcf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:50.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.817+0000 7fa80ad9d700 1 --2- 172.21.15.2:0/364821715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef82dcfa80 0x55ef82dcf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.817+0000 7fa80ad9d700 1 -- 172.21.15.2:0/364821715 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef82df02e0 con 0x55ef82dd5980 2023-09-13T04:40:50.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.817+0000 7fa803fff700 1 --2- 172.21.15.2:0/364821715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef82dd5060 0x55ef82dd4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:50.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.818+0000 7fa80ad9d700 1 --2- 172.21.15.2:0/364821715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dd5980 0x55ef82de0290 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa804095ae0 tx=0x7fa8040998e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa673aeab5011beb server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:50.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.818+0000 7fa8017fa700 1 -- 172.21.15.2:0/364821715 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80401b320 con 0x55ef82dd5980 2023-09-13T04:40:50.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.818+0000 7fa8017fa700 1 -- 172.21.15.2:0/364821715 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa804099070 con 0x55ef82dd5980 2023-09-13T04:40:50.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.818+0000 7fa8017fa700 1 -- 172.21.15.2:0/364821715 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80409dbe0 con 0x55ef82dd5980 2023-09-13T04:40:50.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.818+0000 7fa824762b80 1 -- 172.21.15.2:0/364821715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dd5980 msgr2=0x55ef82de0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:50.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.818+0000 7fa824762b80 1 --2- 172.21.15.2:0/364821715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dd5980 0x55ef82de0290 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa804095ae0 tx=0x7fa8040998e0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.818+0000 7fa824762b80 1 -- 172.21.15.2:0/364821715 shutdown_connections 2023-09-13T04:40:50.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.818+0000 7fa824762b80 1 --2- 172.21.15.2:0/364821715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dd5980 0x55ef82de0290 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.818+0000 7fa824762b80 1 --2- 172.21.15.2:0/364821715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef82dcfa80 0x55ef82dcf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.818+0000 7fa824762b80 1 --2- 172.21.15.2:0/364821715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef82dd5060 0x55ef82dd4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.818+0000 7fa824762b80 1 -- 172.21.15.2:0/364821715 >> 172.21.15.2:0/364821715 conn(0x55ef82d2dd00 msgr2=0x55ef82dd73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:50.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.818+0000 7fa824762b80 1 -- 172.21.15.2:0/364821715 shutdown_connections 2023-09-13T04:40:50.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.819+0000 7fa824762b80 1 -- 172.21.15.2:0/364821715 wait complete. 2023-09-13T04:40:50.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.819+0000 7fa824762b80 1 -- start start 2023-09-13T04:40:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.819+0000 7fa824762b80 1 Processor -- start 2023-09-13T04:40:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.819+0000 7fa824762b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dcfa80 0x55ef82dd1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.819+0000 7fa824762b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef82dd5060 0x55ef82dd1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.820+0000 7fa824762b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef82dd5980 0x55ef82dd1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.820+0000 7fa824762b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef82e3d0c0 con 0x55ef82dcfa80 2023-09-13T04:40:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.820+0000 7fa824762b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef82e3d200 con 0x55ef82dd5980 2023-09-13T04:40:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.820+0000 7fa824762b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef82e3d370 con 0x55ef82dd5060 2023-09-13T04:40:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.820+0000 7fa824762b80 10 client.0 _ll_register_callbacks cb 0x55ef82cbd700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.820+0000 7fa80a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dcfa80 0x55ef82dd1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.820+0000 7fa80a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dcfa80 0x55ef82dd1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59680/0 (socket says 172.21.15.2:59680) 2023-09-13T04:40:50.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.820+0000 7fa80a59c700 1 -- 172.21.15.2:0/1515847919 learned_addr learned my addr 172.21.15.2:0/1515847919 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:50.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.820+0000 7fa80ad9d700 1 --2- 172.21.15.2:0/1515847919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef82dd5980 0x55ef82dd1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:50.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.820+0000 7fa803fff700 1 --2- 172.21.15.2:0/1515847919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef82dd5060 0x55ef82dd1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:50.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.820+0000 7fa80a59c700 1 -- 172.21.15.2:0/1515847919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef82dd5060 msgr2=0x55ef82dd1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:50.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.820+0000 7fa80a59c700 1 --2- 172.21.15.2:0/1515847919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef82dd5060 0x55ef82dd1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.820+0000 7fa80a59c700 1 -- 172.21.15.2:0/1515847919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef82dd5980 msgr2=0x55ef82dd1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:50.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.820+0000 7fa80a59c700 1 --2- 172.21.15.2:0/1515847919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef82dd5980 0x55ef82dd1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.820+0000 7fa80a59c700 1 -- 172.21.15.2:0/1515847919 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa80c10e3e0 con 0x55ef82dcfa80 2023-09-13T04:40:50.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.820+0000 7fa80ad9d700 1 --2- 172.21.15.2:0/1515847919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef82dd5980 0x55ef82dd1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:50.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.821+0000 7fa80a59c700 1 --2- 172.21.15.2:0/1515847919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dcfa80 0x55ef82dd1100 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc001eb0 tx=0x7fa7fc00fde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a341eeb3ee5dc4b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:50.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.821+0000 7fa7f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:40:50.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.821+0000 7fa7f1ffb700 1 -- 172.21.15.2:0/1515847919 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc017040 con 0x55ef82dcfa80 2023-09-13T04:40:50.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.821+0000 7fa7f1ffb700 1 -- 172.21.15.2:0/1515847919 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fa7fc000e40 con 0x55ef82dcfa80 2023-09-13T04:40:50.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.821+0000 7fa824762b80 1 -- 172.21.15.2:0/1515847919 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa804099770 con 0x55ef82dcfa80 2023-09-13T04:40:50.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.821+0000 7fa7f1ffb700 1 -- 172.21.15.2:0/1515847919 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7fc0054c0 con 0x55ef82dcfa80 2023-09-13T04:40:50.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.821+0000 7fa824762b80 10 client.15681 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:50.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.821+0000 7fa824762b80 1 -- 172.21.15.2:0/1515847919 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ef82ba16b0 con 0x55ef82dcfa80 2023-09-13T04:40:50.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.821+0000 7fa7f1ffb700 1 -- 172.21.15.2:0/1515847919 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa804099770 con 0x55ef82dcfa80 2023-09-13T04:40:50.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.822+0000 7fa7f1ffb700 1 -- 172.21.15.2:0/1515847919 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa7fc012900 con 0x55ef82dcfa80 2023-09-13T04:40:50.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.822+0000 7fa824762b80 10 client.15681 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:50.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.822+0000 7fa824762b80 10 client.15681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:40:50.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.822+0000 7fa824762b80 20 client.15681 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:50.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.822+0000 7fa824762b80 1 --2- 172.21.15.2:0/1515847919 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55ef82e3eb60 0x55ef82e40f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:50.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.822+0000 7fa824762b80 4 client.15681 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-13T04:40:50.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.822+0000 7fa824762b80 1 -- 172.21.15.2:0/1515847919 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ef82cdb0d0 con 0x55ef82e3eb60 2023-09-13T04:40:50.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.822+0000 7fa803fff700 1 --2- 172.21.15.2:0/1515847919 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55ef82e3eb60 0x55ef82e40f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:50.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.823+0000 7fa803fff700 1 --2- 172.21.15.2:0/1515847919 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55ef82e3eb60 0x55ef82e40f80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e307f049b044ccc2 server_cookie=74d66c947f8d2084 in_seq=0 out_seq=0 2023-09-13T04:40:50.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.823+0000 7fa7f1ffb700 10 client.15681 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:50.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.823+0000 7fa7f1ffb700 1 -- 172.21.15.2:0/1515847919 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ef82cdb0d0 con 0x55ef82e3eb60 2023-09-13T04:40:50.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.823+0000 7fa7f1ffb700 10 client.15681 handle_command_reply: tid=0 2023-09-13T04:40:50.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.823+0000 7fa824762b80 1 client.15681 shutdown 2023-09-13T04:40:50.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.824+0000 7fa824762b80 1 -- 172.21.15.2:0/1515847919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dcfa80 msgr2=0x55ef82dd1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:50.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.824+0000 7fa824762b80 1 --2- 172.21.15.2:0/1515847919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dcfa80 0x55ef82dd1100 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc001eb0 tx=0x7fa7fc00fde0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.824+0000 7fa824762b80 1 -- 172.21.15.2:0/1515847919 shutdown_connections 2023-09-13T04:40:50.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.824+0000 7fa824762b80 1 --2- 172.21.15.2:0/1515847919 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55ef82e3eb60 0x55ef82e40f80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.824+0000 7fa824762b80 1 --2- 172.21.15.2:0/1515847919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef82dd5980 0x55ef82dd1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.824+0000 7fa824762b80 1 --2- 172.21.15.2:0/1515847919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef82dcfa80 0x55ef82dd1100 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.824+0000 7fa7f1ffb700 0 client.15681 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:50.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.824+0000 7fa824762b80 1 --2- 172.21.15.2:0/1515847919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef82dd5060 0x55ef82dd1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.824+0000 7fa824762b80 1 -- 172.21.15.2:0/1515847919 >> 172.21.15.2:0/1515847919 conn(0x55ef82d2dd00 msgr2=0x55ef82dd7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:50.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.824+0000 7fa824762b80 1 -- 172.21.15.2:0/1515847919 shutdown_connections 2023-09-13T04:40:50.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.824+0000 7fa824762b80 1 -- 172.21.15.2:0/1515847919 wait complete. 2023-09-13T04:40:50.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.824+0000 7fa824762b80 20 client.15681 trim_cache size 0 max 16384 2023-09-13T04:40:50.856 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:40:50.856 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2023-09-13T04:40:50.857 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "f51de0fb-b185-42d5-9909-6fdded2622eb", 2023-09-13T04:40:50.857 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2023-09-13T04:40:50.857 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:40:50.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.825+0000 7fa810dff700 1 -- 172.21.15.2:0/2214974270 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fa7f80a08c0 msgr2=0x7fa7f80a2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:50.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.825+0000 7fa810dff700 1 --2- 172.21.15.2:0/2214974270 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fa7f80a08c0 0x7fa7f80a2d80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa80c19a560 tx=0x7fa7f4005590 comp rx=0 tx=0).stop 2023-09-13T04:40:50.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.825+0000 7fa810dff700 1 -- 172.21.15.2:0/2214974270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa80c10b280 msgr2=0x7fa80c19d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:50.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.825+0000 7fa810dff700 1 --2- 172.21.15.2:0/2214974270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa80c10b280 0x7fa80c19d8f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa80400ef30 tx=0x7fa80400c3a0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.825+0000 7fa810dff700 1 -- 172.21.15.2:0/2214974270 shutdown_connections 2023-09-13T04:40:50.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.825+0000 7fa810dff700 1 --2- 172.21.15.2:0/2214974270 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fa7f80a08c0 0x7fa7f80a2d80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.825+0000 7fa810dff700 1 --2- 172.21.15.2:0/2214974270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa80c10b280 0x7fa80c19d8f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.825+0000 7fa810dff700 1 --2- 172.21.15.2:0/2214974270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa80c101eb0 0x7fa80c198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.825+0000 7fa810dff700 1 --2- 172.21.15.2:0/2214974270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa80c1027d0 0x7fa80c199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:50.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.825+0000 7fa810dff700 1 -- 172.21.15.2:0/2214974270 >> 172.21.15.2:0/2214974270 conn(0x7fa80c0fb630 msgr2=0x7fa80c10c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:50.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.825+0000 7fa810dff700 1 -- 172.21.15.2:0/2214974270 shutdown_connections 2023-09-13T04:40:50.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:50.825+0000 7fa810dff700 1 -- 172.21.15.2:0/2214974270 wait complete. 2023-09-13T04:40:50.862 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T04:40:51.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.067+0000 7fec5c7ff700 1 -- 172.21.15.2:0/2719537234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec54101df0 msgr2=0x7fec541021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:51.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.067+0000 7fec5c7ff700 1 --2- 172.21.15.2:0/2719537234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec54101df0 0x7fec541021d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fec50009990 tx=0x7fec50009ca0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.067+0000 7fec5c7ff700 1 -- 172.21.15.2:0/2719537234 shutdown_connections 2023-09-13T04:40:51.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.067+0000 7fec5c7ff700 1 --2- 172.21.15.2:0/2719537234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec5410b210 0x7fec5410d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.067+0000 7fec5c7ff700 1 --2- 172.21.15.2:0/2719537234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec54101df0 0x7fec541021d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.067+0000 7fec5c7ff700 1 --2- 172.21.15.2:0/2719537234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec54102710 0x7fec5410abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.067+0000 7fec5c7ff700 1 -- 172.21.15.2:0/2719537234 >> 172.21.15.2:0/2719537234 conn(0x7fec540fb630 msgr2=0x7fec540fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:51.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.067+0000 7fec5c7ff700 1 -- 172.21.15.2:0/2719537234 shutdown_connections 2023-09-13T04:40:51.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.068+0000 7fec5c7ff700 1 -- 172.21.15.2:0/2719537234 wait complete. 2023-09-13T04:40:51.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.069+0000 7fec5c7ff700 1 Processor -- start 2023-09-13T04:40:51.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.069+0000 7fec5c7ff700 1 -- start start 2023-09-13T04:40:51.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.069+0000 7fec5c7ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec54101df0 0x7fec54199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:51.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.069+0000 7fec5c7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec54102710 0x7fec541995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:51.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.069+0000 7fec5c7ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec5410b210 0x7fec5419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:51.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.069+0000 7fec5c7ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5419e060 con 0x7fec54101df0 2023-09-13T04:40:51.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.069+0000 7fec5c7ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5419e1d0 con 0x7fec5410b210 2023-09-13T04:40:51.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.069+0000 7fec5c7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5419e390 con 0x7fec54102710 2023-09-13T04:40:51.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.069+0000 7fec5a59b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec54101df0 0x7fec54199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:51.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.069+0000 7fec5a59b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec54101df0 0x7fec54199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59688/0 (socket says 172.21.15.2:59688) 2023-09-13T04:40:51.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.069+0000 7fec5a59b700 1 -- 172.21.15.2:0/3497654338 learned_addr learned my addr 172.21.15.2:0/3497654338 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:51.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.070+0000 7fec59d9a700 1 --2- 172.21.15.2:0/3497654338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec54102710 0x7fec541995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:51.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.070+0000 7fec5ad9c700 1 --2- 172.21.15.2:0/3497654338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec5410b210 0x7fec5419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:51.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.070+0000 7fec5a59b700 1 -- 172.21.15.2:0/3497654338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec54102710 msgr2=0x7fec541995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:51.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.070+0000 7fec5a59b700 1 --2- 172.21.15.2:0/3497654338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec54102710 0x7fec541995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.070+0000 7fec5a59b700 1 -- 172.21.15.2:0/3497654338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec5410b210 msgr2=0x7fec5419da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:51.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.070+0000 7fec5a59b700 1 --2- 172.21.15.2:0/3497654338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec5410b210 0x7fec5419da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.070+0000 7fec5a59b700 1 -- 172.21.15.2:0/3497654338 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec500095f0 con 0x7fec54101df0 2023-09-13T04:40:51.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.070+0000 7fec5a59b700 1 --2- 172.21.15.2:0/3497654338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec54101df0 0x7fec54199080 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fec500095c0 tx=0x7fec5000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:51.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.070+0000 7fec4b7fe700 1 -- 172.21.15.2:0/3497654338 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5001d070 con 0x7fec54101df0 2023-09-13T04:40:51.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.070+0000 7fec5c7ff700 1 -- 172.21.15.2:0/3497654338 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec5419e610 con 0x7fec54101df0 2023-09-13T04:40:51.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.070+0000 7fec4b7fe700 1 -- 172.21.15.2:0/3497654338 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec50005050 con 0x7fec54101df0 2023-09-13T04:40:51.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.070+0000 7fec4b7fe700 1 -- 172.21.15.2:0/3497654338 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec50004a90 con 0x7fec54101df0 2023-09-13T04:40:51.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.071+0000 7fec5c7ff700 1 -- 172.21.15.2:0/3497654338 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec541a5eb0 con 0x7fec54101df0 2023-09-13T04:40:51.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.074+0000 7fec4b7fe700 1 -- 172.21.15.2:0/3497654338 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 23) v1 ==== 135434+0+0 (secure 0 0 0) 0x7fec50005430 con 0x7fec54101df0 2023-09-13T04:40:51.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.075+0000 7fec7018eb80 1 Processor -- start 2023-09-13T04:40:51.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.075+0000 7fec7018eb80 1 -- start start 2023-09-13T04:40:51.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.075+0000 7fec4b7fe700 1 --2- 172.21.15.2:0/3497654338 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fec400a0840 0x7fec400a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:51.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.075+0000 7fec7018eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f8fe0 0x560ebe3f93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:51.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.075+0000 7fec4b7fe700 1 -- 172.21.15.2:0/3497654338 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fec500cd380 con 0x7fec54101df0 2023-09-13T04:40:51.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.075+0000 7fec7018eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ebe3f9900 0x560ebe3f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:51.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.075+0000 7fec7018eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ebe3f3a00 0x560ebe3f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:51.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.075+0000 7fec7018eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ebe34c780 con 0x560ebe3f8fe0 2023-09-13T04:40:51.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.075+0000 7fec7018eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ebe34a0a0 con 0x560ebe3f3a00 2023-09-13T04:40:51.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.075+0000 7fec7018eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ebe345990 con 0x560ebe3f9900 2023-09-13T04:40:51.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.076+0000 7fec5ad9c700 1 --2- 172.21.15.2:0/3497654338 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fec400a0840 0x7fec400a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:51.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.076+0000 7fec5a59b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f8fe0 0x560ebe3f93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:51.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.076+0000 7fec5a59b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f8fe0 0x560ebe3f93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59690/0 (socket says 172.21.15.2:59690) 2023-09-13T04:40:51.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.076+0000 7fec5a59b700 1 -- 172.21.15.2:0/3436893102 learned_addr learned my addr 172.21.15.2:0/3436893102 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:51.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.076+0000 7fec59d9a700 1 --2- 172.21.15.2:0/3436893102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ebe3f9900 0x560ebe3f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:51.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.076+0000 7fec5ad9c700 1 --2- 172.21.15.2:0/3436893102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ebe3f3a00 0x560ebe3f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:51.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.076+0000 7fec5a59b700 1 -- 172.21.15.2:0/3436893102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ebe3f9900 msgr2=0x560ebe3f34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:51.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.076+0000 7fec5a59b700 1 --2- 172.21.15.2:0/3436893102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ebe3f9900 0x560ebe3f34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.076+0000 7fec5a59b700 1 -- 172.21.15.2:0/3436893102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ebe3f3a00 msgr2=0x560ebe3f3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:51.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.076+0000 7fec5a59b700 1 --2- 172.21.15.2:0/3436893102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ebe3f3a00 0x560ebe3f3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.076+0000 7fec5a59b700 1 -- 172.21.15.2:0/3436893102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ebe1c5630 con 0x560ebe3f8fe0 2023-09-13T04:40:51.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.076+0000 7fec5ad9c700 1 --2- 172.21.15.2:0/3436893102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ebe3f3a00 0x560ebe3f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:40:51.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.076+0000 7fec5ad9c700 1 --2- 172.21.15.2:0/3497654338 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fec400a0840 0x7fec400a2d00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fec4c009550 tx=0x7fec4c004040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:51.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.077+0000 7fec5a59b700 1 --2- 172.21.15.2:0/3436893102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f8fe0 0x560ebe3f93c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fec5009a370 tx=0x7fec5009a010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309c94daad6f8bd2 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:51.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.077+0000 7fec48ff9700 1 -- 172.21.15.2:0/3436893102 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec500850d0 con 0x560ebe3f8fe0 2023-09-13T04:40:51.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.077+0000 7fec48ff9700 1 -- 172.21.15.2:0/3436893102 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec5009a3e0 con 0x560ebe3f8fe0 2023-09-13T04:40:51.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.077+0000 7fec48ff9700 1 -- 172.21.15.2:0/3436893102 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec50004bf0 con 0x560ebe3f8fe0 2023-09-13T04:40:51.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.077+0000 7fec7018eb80 1 -- 172.21.15.2:0/3436893102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f8fe0 msgr2=0x560ebe3f93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:51.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.077+0000 7fec7018eb80 1 --2- 172.21.15.2:0/3436893102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f8fe0 0x560ebe3f93c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fec5009a370 tx=0x7fec5009a010 comp rx=0 tx=0).stop 2023-09-13T04:40:51.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.078+0000 7fec7018eb80 1 -- 172.21.15.2:0/3436893102 shutdown_connections 2023-09-13T04:40:51.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.078+0000 7fec7018eb80 1 --2- 172.21.15.2:0/3436893102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ebe3f3a00 0x560ebe3f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.078+0000 7fec7018eb80 1 --2- 172.21.15.2:0/3436893102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f8fe0 0x560ebe3f93c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.078+0000 7fec7018eb80 1 --2- 172.21.15.2:0/3436893102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ebe3f9900 0x560ebe3f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.078+0000 7fec7018eb80 1 -- 172.21.15.2:0/3436893102 >> 172.21.15.2:0/3436893102 conn(0x560ebe351c80 msgr2=0x560ebe3edf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:51.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.078+0000 7fec7018eb80 1 -- 172.21.15.2:0/3436893102 shutdown_connections 2023-09-13T04:40:51.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.078+0000 7fec7018eb80 1 -- 172.21.15.2:0/3436893102 wait complete. 2023-09-13T04:40:51.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.078+0000 7fec7018eb80 1 -- start start 2023-09-13T04:40:51.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.078+0000 7fec7018eb80 1 Processor -- start 2023-09-13T04:40:51.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.079+0000 7fec7018eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f3a00 0x560ebe3ff7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:51.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.079+0000 7fec7018eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ebe3f8fe0 0x560ebe3fca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:51.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.079+0000 7fec7018eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ebe3f9900 0x560ebe3fcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:51.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.079+0000 7fec7018eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ebe2e3bb0 con 0x560ebe3f3a00 2023-09-13T04:40:51.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.079+0000 7fec7018eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ebe2e38b0 con 0x560ebe3f9900 2023-09-13T04:40:51.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.079+0000 7fec7018eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ebe1c7920 con 0x560ebe3f8fe0 2023-09-13T04:40:51.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.079+0000 7fec7018eb80 10 client.0 _ll_register_callbacks cb 0x560ebe2e1680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:51.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.079+0000 7fec5a59b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f3a00 0x560ebe3ff7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:51.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.079+0000 7fec5a59b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f3a00 0x560ebe3ff7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59694/0 (socket says 172.21.15.2:59694) 2023-09-13T04:40:51.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.079+0000 7fec5a59b700 1 -- 172.21.15.2:0/2667114918 learned_addr learned my addr 172.21.15.2:0/2667114918 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:51.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.079+0000 7fec5ad9c700 1 --2- 172.21.15.2:0/2667114918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ebe3f9900 0x560ebe3fcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:51.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.079+0000 7fec59d9a700 1 --2- 172.21.15.2:0/2667114918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ebe3f8fe0 0x560ebe3fca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:51.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.080+0000 7fec5a59b700 1 -- 172.21.15.2:0/2667114918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ebe3f8fe0 msgr2=0x560ebe3fca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:51.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.080+0000 7fec5a59b700 1 --2- 172.21.15.2:0/2667114918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ebe3f8fe0 0x560ebe3fca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.080+0000 7fec5a59b700 1 -- 172.21.15.2:0/2667114918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ebe3f9900 msgr2=0x560ebe3fcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:51.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.080+0000 7fec5a59b700 1 --2- 172.21.15.2:0/2667114918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ebe3f9900 0x560ebe3fcfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.080+0000 7fec5a59b700 1 -- 172.21.15.2:0/2667114918 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec541a5eb0 con 0x560ebe3f3a00 2023-09-13T04:40:51.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.080+0000 7fec59d9a700 1 --2- 172.21.15.2:0/2667114918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ebe3f8fe0 0x560ebe3fca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:40:51.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.080+0000 7fec5a59b700 1 --2- 172.21.15.2:0/2667114918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f3a00 0x560ebe3ff7f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fec5009a0e0 tx=0x7fec50009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae11178f1005533 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:51.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.080+0000 7fec3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:40:51.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.080+0000 7fec3dffb700 1 -- 172.21.15.2:0/2667114918 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5009e210 con 0x560ebe3f3a00 2023-09-13T04:40:51.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.080+0000 7fec7018eb80 1 -- 172.21.15.2:0/2667114918 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ebe3e6670 con 0x560ebe3f3a00 2023-09-13T04:40:51.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.080+0000 7fec3dffb700 1 -- 172.21.15.2:0/2667114918 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fec500a23c0 con 0x560ebe3f3a00 2023-09-13T04:40:51.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.080+0000 7fec3dffb700 1 -- 172.21.15.2:0/2667114918 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec500b8170 con 0x560ebe3f3a00 2023-09-13T04:40:51.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.080+0000 7fec7018eb80 10 client.15693 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:51.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.080+0000 7fec7018eb80 1 -- 172.21.15.2:0/2667114918 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560ebe35d7a0 con 0x560ebe3f3a00 2023-09-13T04:40:51.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.081+0000 7fec3dffb700 1 -- 172.21.15.2:0/2667114918 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec50064080 con 0x560ebe3f3a00 2023-09-13T04:40:51.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.081+0000 7fec3dffb700 1 -- 172.21.15.2:0/2667114918 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fec5009a640 con 0x560ebe3f3a00 2023-09-13T04:40:51.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.081+0000 7fec7018eb80 10 client.15693 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:51.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.081+0000 7fec7018eb80 10 client.15693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:40:51.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.081+0000 7fec7018eb80 20 client.15693 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:51.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.081+0000 7fec7018eb80 1 --2- 172.21.15.2:0/2667114918 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x560ebe40c930 0x560ebe40ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:51.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.081+0000 7fec7018eb80 4 client.15693 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:40:51.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.081+0000 7fec7018eb80 1 -- 172.21.15.2:0/2667114918 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ebe33e970 con 0x560ebe40c930 2023-09-13T04:40:51.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.081+0000 7fec59d9a700 1 --2- 172.21.15.2:0/2667114918 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x560ebe40c930 0x560ebe40ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:51.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.082+0000 7fec59d9a700 1 --2- 172.21.15.2:0/2667114918 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x560ebe40c930 0x560ebe40ed50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=648664e05479781b server_cookie=72f0f9c30978b983 in_seq=0 out_seq=0 2023-09-13T04:40:51.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.082+0000 7fec3dffb700 10 client.15693 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:51.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.082+0000 7fec3dffb700 1 -- 172.21.15.2:0/2667114918 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560ebe33e970 con 0x560ebe40c930 2023-09-13T04:40:51.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.082+0000 7fec3dffb700 10 client.15693 handle_command_reply: tid=0 2023-09-13T04:40:51.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.082+0000 7fec7018eb80 1 client.15693 shutdown 2023-09-13T04:40:51.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.083+0000 7fec7018eb80 1 -- 172.21.15.2:0/2667114918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f3a00 msgr2=0x560ebe3ff7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:51.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.083+0000 7fec7018eb80 1 --2- 172.21.15.2:0/2667114918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f3a00 0x560ebe3ff7f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fec5009a0e0 tx=0x7fec50009310 comp rx=0 tx=0).stop 2023-09-13T04:40:51.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.083+0000 7fec7018eb80 1 -- 172.21.15.2:0/2667114918 shutdown_connections 2023-09-13T04:40:51.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.083+0000 7fec7018eb80 1 --2- 172.21.15.2:0/2667114918 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x560ebe40c930 0x560ebe40ed50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.083+0000 7fec7018eb80 1 --2- 172.21.15.2:0/2667114918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ebe3f9900 0x560ebe3fcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.083+0000 7fec3dffb700 0 client.15693 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:51.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.083+0000 7fec7018eb80 1 --2- 172.21.15.2:0/2667114918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f3a00 0x560ebe3ff7f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.083+0000 7fec7018eb80 1 --2- 172.21.15.2:0/2667114918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ebe3f8fe0 0x560ebe3fca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.083+0000 7fec7018eb80 1 -- 172.21.15.2:0/2667114918 >> 172.21.15.2:0/2667114918 conn(0x560ebe351c80 msgr2=0x560ebe3e87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:51.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.083+0000 7fec7018eb80 1 -- 172.21.15.2:0/2667114918 shutdown_connections 2023-09-13T04:40:51.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.083+0000 7fec7018eb80 1 -- 172.21.15.2:0/2667114918 wait complete. 2023-09-13T04:40:51.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.083+0000 7fec7018eb80 20 client.15693 trim_cache size 0 max 16384 2023-09-13T04:40:51.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.100+0000 7fec7018eb80 1 Processor -- start 2023-09-13T04:40:51.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.100+0000 7fec7018eb80 1 -- start start 2023-09-13T04:40:51.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.101+0000 7fec7018eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ebe3f3a00 0x560ebe403e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:51.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.101+0000 7fec7018eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f8fe0 0x560ebe3e88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:51.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.101+0000 7fec7018eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ebe3f9900 0x560ebe3ff320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:51.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.101+0000 7fec7018eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ebe3f38c0 con 0x560ebe3f8fe0 2023-09-13T04:40:51.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.101+0000 7fec7018eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ebe33e970 con 0x560ebe3f9900 2023-09-13T04:40:51.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.101+0000 7fec7018eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ebe3eece0 con 0x560ebe3f3a00 2023-09-13T04:40:51.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.101+0000 7fec59d9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f8fe0 0x560ebe3e88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:51.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.101+0000 7fec59d9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f8fe0 0x560ebe3e88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59706/0 (socket says 172.21.15.2:59706) 2023-09-13T04:40:51.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.101+0000 7fec5a59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ebe3f3a00 0x560ebe403e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:51.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.101+0000 7fec59d9a700 1 -- 172.21.15.2:0/3034260600 learned_addr learned my addr 172.21.15.2:0/3034260600 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:51.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.101+0000 7fec5ad9c700 1 --2- 172.21.15.2:0/3034260600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ebe3f9900 0x560ebe3ff320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:51.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.101+0000 7fec5a59b700 1 -- 172.21.15.2:0/3034260600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ebe3f9900 msgr2=0x560ebe3ff320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:51.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.101+0000 7fec5a59b700 1 --2- 172.21.15.2:0/3034260600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ebe3f9900 0x560ebe3ff320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.101+0000 7fec5a59b700 1 -- 172.21.15.2:0/3034260600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f8fe0 msgr2=0x560ebe3e88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:51.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.101+0000 7fec5a59b700 1 --2- 172.21.15.2:0/3034260600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f8fe0 0x560ebe3e88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.101+0000 7fec5a59b700 1 -- 172.21.15.2:0/3034260600 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ebe413980 con 0x560ebe3f3a00 2023-09-13T04:40:51.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.101+0000 7fec59d9a700 1 --2- 172.21.15.2:0/3034260600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f8fe0 0x560ebe3e88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:51.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.102+0000 7fec5a59b700 1 --2- 172.21.15.2:0/3034260600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ebe3f3a00 0x560ebe403e90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fec5009a370 tx=0x7fec5009aa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f6dd70d4acd7ae3 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:51.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.102+0000 7fec497fa700 1 -- 172.21.15.2:0/3034260600 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5009e210 con 0x560ebe3f3a00 2023-09-13T04:40:51.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.102+0000 7fec497fa700 1 -- 172.21.15.2:0/3034260600 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560ebe3e6670 con 0x560ebe3f3a00 2023-09-13T04:40:51.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.102+0000 7fec497fa700 1 -- 172.21.15.2:0/3034260600 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5009a640 con 0x560ebe3f3a00 2023-09-13T04:40:51.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.102+0000 7fec7018eb80 1 -- 172.21.15.2:0/3034260600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ebe3f3a00 msgr2=0x560ebe403e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:51.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.102+0000 7fec7018eb80 1 --2- 172.21.15.2:0/3034260600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ebe3f3a00 0x560ebe403e90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fec5009a370 tx=0x7fec5009aa00 comp rx=0 tx=0).stop 2023-09-13T04:40:51.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.103+0000 7fec7018eb80 1 -- 172.21.15.2:0/3034260600 shutdown_connections 2023-09-13T04:40:51.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.103+0000 7fec7018eb80 1 --2- 172.21.15.2:0/3034260600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ebe3f9900 0x560ebe3ff320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.103+0000 7fec7018eb80 1 --2- 172.21.15.2:0/3034260600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ebe3f3a00 0x560ebe403e90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.103+0000 7fec7018eb80 1 --2- 172.21.15.2:0/3034260600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f8fe0 0x560ebe3e88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.103+0000 7fec7018eb80 1 -- 172.21.15.2:0/3034260600 >> 172.21.15.2:0/3034260600 conn(0x560ebe351c80 msgr2=0x560ebe409180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:51.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.103+0000 7fec7018eb80 1 -- 172.21.15.2:0/3034260600 shutdown_connections 2023-09-13T04:40:51.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.103+0000 7fec7018eb80 1 -- 172.21.15.2:0/3034260600 wait complete. 2023-09-13T04:40:51.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.103+0000 7fec7018eb80 1 -- start start 2023-09-13T04:40:51.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.103+0000 7fec7018eb80 1 Processor -- start 2023-09-13T04:40:51.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.104+0000 7fec7018eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ebe3f3a00 0x560ebe3f5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:51.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.104+0000 7fec7018eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f8fe0 0x560ebe3f6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:51.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.104+0000 7fec7018eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ebe3f9900 0x560ebe3f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:51.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.104+0000 7fec7018eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ebe3f67d0 con 0x560ebe3f8fe0 2023-09-13T04:40:51.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.104+0000 7fec7018eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ebe3f4fa0 con 0x560ebe3f9900 2023-09-13T04:40:51.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.104+0000 7fec7018eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ebe3f5160 con 0x560ebe3f3a00 2023-09-13T04:40:51.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.104+0000 7fec7018eb80 10 client.0 _ll_register_callbacks cb 0x560ebe2e1680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:51.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.104+0000 7fec5a59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ebe3f3a00 0x560ebe3f5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:51.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.104+0000 7fec59d9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f8fe0 0x560ebe3f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:51.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.104+0000 7fec5a59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ebe3f3a00 0x560ebe3f5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:38366/0 (socket says 172.21.15.2:38366) 2023-09-13T04:40:51.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.104+0000 7fec5a59b700 1 -- 172.21.15.2:0/3414824166 learned_addr learned my addr 172.21.15.2:0/3414824166 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:51.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.104+0000 7fec5ad9c700 1 --2- 172.21.15.2:0/3414824166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ebe3f9900 0x560ebe3f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:51.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.104+0000 7fec59d9a700 1 -- 172.21.15.2:0/3414824166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ebe3f3a00 msgr2=0x560ebe3f5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:51.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.104+0000 7fec59d9a700 1 --2- 172.21.15.2:0/3414824166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ebe3f3a00 0x560ebe3f5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.104+0000 7fec59d9a700 1 -- 172.21.15.2:0/3414824166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ebe3f9900 msgr2=0x560ebe3f4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:51.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.104+0000 7fec59d9a700 1 --2- 172.21.15.2:0/3414824166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ebe3f9900 0x560ebe3f4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.104+0000 7fec59d9a700 1 -- 172.21.15.2:0/3414824166 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec4401e9f0 con 0x560ebe3f8fe0 2023-09-13T04:40:51.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.105+0000 7fec5a59b700 1 --2- 172.21.15.2:0/3414824166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ebe3f3a00 0x560ebe3f5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:51.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.105+0000 7fec59d9a700 1 --2- 172.21.15.2:0/3414824166 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f8fe0 0x560ebe3f6090 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x560ebe409950 tx=0x7fec4401ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59fac4ced53433bb server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:51.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.105+0000 7fec3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:40:51.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.105+0000 7fec3dffb700 1 -- 172.21.15.2:0/3414824166 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec44003880 con 0x560ebe3f8fe0 2023-09-13T04:40:51.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.105+0000 7fec7018eb80 1 -- 172.21.15.2:0/3414824166 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec5009a3e0 con 0x560ebe3f8fe0 2023-09-13T04:40:51.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.105+0000 7fec3dffb700 1 -- 172.21.15.2:0/3414824166 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fec440039e0 con 0x560ebe3f8fe0 2023-09-13T04:40:51.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.105+0000 7fec3dffb700 1 -- 172.21.15.2:0/3414824166 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec44003ba0 con 0x560ebe3f8fe0 2023-09-13T04:40:51.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.105+0000 7fec7018eb80 10 client.15699 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:51.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.105+0000 7fec7018eb80 1 -- 172.21.15.2:0/3414824166 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560ebe35d7a0 con 0x560ebe3f8fe0 2023-09-13T04:40:51.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.106+0000 7fec3dffb700 1 -- 172.21.15.2:0/3414824166 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec5009a3e0 con 0x560ebe3f8fe0 2023-09-13T04:40:51.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.106+0000 7fec3dffb700 1 -- 172.21.15.2:0/3414824166 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fec440273f0 con 0x560ebe3f8fe0 2023-09-13T04:40:51.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.106+0000 7fec7018eb80 10 client.15699 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:51.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.106+0000 7fec7018eb80 10 client.15699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:40:51.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.106+0000 7fec7018eb80 20 client.15699 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:51.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.106+0000 7fec7018eb80 1 --2- 172.21.15.2:0/3414824166 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x560ebe461500 0x560ebe463920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:51.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.106+0000 7fec7018eb80 4 client.15699 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-09-13T04:40:51.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.106+0000 7fec7018eb80 1 -- 172.21.15.2:0/3414824166 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560ebe2ff050 con 0x560ebe461500 2023-09-13T04:40:51.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.106+0000 7fec5a59b700 1 --2- 172.21.15.2:0/3414824166 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x560ebe461500 0x560ebe463920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:51.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.107+0000 7fec5a59b700 1 --2- 172.21.15.2:0/3414824166 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x560ebe461500 0x560ebe463920 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2230fb54f2c58a4b server_cookie=6d221504fc9d5aa2 in_seq=0 out_seq=0 2023-09-13T04:40:51.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.107+0000 7fec3dffb700 10 client.15699 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:51.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.107+0000 7fec3dffb700 1 -- 172.21.15.2:0/3414824166 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560ebe2ff050 con 0x560ebe461500 2023-09-13T04:40:51.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.107+0000 7fec3dffb700 10 client.15699 handle_command_reply: tid=0 2023-09-13T04:40:51.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.107+0000 7fec7018eb80 1 client.15699 shutdown 2023-09-13T04:40:51.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.107+0000 7fec7018eb80 1 -- 172.21.15.2:0/3414824166 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f8fe0 msgr2=0x560ebe3f6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:51.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.107+0000 7fec7018eb80 1 --2- 172.21.15.2:0/3414824166 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f8fe0 0x560ebe3f6090 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x560ebe409950 tx=0x7fec4401ee60 comp rx=0 tx=0).stop 2023-09-13T04:40:51.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.107+0000 7fec7018eb80 1 -- 172.21.15.2:0/3414824166 shutdown_connections 2023-09-13T04:40:51.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.107+0000 7fec7018eb80 1 --2- 172.21.15.2:0/3414824166 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x560ebe461500 0x560ebe463920 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.107+0000 7fec7018eb80 1 --2- 172.21.15.2:0/3414824166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ebe3f9900 0x560ebe3f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.107+0000 7fec7018eb80 1 --2- 172.21.15.2:0/3414824166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ebe3f3a00 0x560ebe3f5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.107+0000 7fec7018eb80 1 --2- 172.21.15.2:0/3414824166 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ebe3f8fe0 0x560ebe3f6090 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.107+0000 7fec3dffb700 0 client.15699 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:51.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.107+0000 7fec7018eb80 1 -- 172.21.15.2:0/3414824166 >> 172.21.15.2:0/3414824166 conn(0x560ebe351c80 msgr2=0x560ebe4085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:51.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.108+0000 7fec7018eb80 1 -- 172.21.15.2:0/3414824166 shutdown_connections 2023-09-13T04:40:51.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.108+0000 7fec7018eb80 1 -- 172.21.15.2:0/3414824166 wait complete. 2023-09-13T04:40:51.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.108+0000 7fec7018eb80 20 client.15699 trim_cache size 0 max 16384 2023-09-13T04:40:51.125 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:40:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-13T04:40:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-09-13T04:40:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "f51de0fb-b185-42d5-9909-6fdded2622eb": { 2023-09-13T04:40:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-09-13T04:40:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "f51de0fb-b185-42d5-9909-6fdded2622eb", 2023-09-13T04:40:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-09-13T04:40:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:40:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:40:51.127 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:40:51.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.108+0000 7fec5c7ff700 1 -- 172.21.15.2:0/3497654338 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fec400a0840 msgr2=0x7fec400a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:51.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.108+0000 7fec5c7ff700 1 --2- 172.21.15.2:0/3497654338 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fec400a0840 0x7fec400a2d00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fec4c009550 tx=0x7fec4c004040 comp rx=0 tx=0).stop 2023-09-13T04:40:51.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.108+0000 7fec5c7ff700 1 -- 172.21.15.2:0/3497654338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec54101df0 msgr2=0x7fec54199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:51.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.108+0000 7fec5c7ff700 1 --2- 172.21.15.2:0/3497654338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec54101df0 0x7fec54199080 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fec500095c0 tx=0x7fec5000f690 comp rx=0 tx=0).stop 2023-09-13T04:40:51.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.108+0000 7fec5c7ff700 1 -- 172.21.15.2:0/3497654338 shutdown_connections 2023-09-13T04:40:51.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.108+0000 7fec5c7ff700 1 --2- 172.21.15.2:0/3497654338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec5410b210 0x7fec5419da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.108+0000 7fec5c7ff700 1 --2- 172.21.15.2:0/3497654338 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fec400a0840 0x7fec400a2d00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.108+0000 7fec5c7ff700 1 --2- 172.21.15.2:0/3497654338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec54101df0 0x7fec54199080 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.108+0000 7fec5c7ff700 1 --2- 172.21.15.2:0/3497654338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec54102710 0x7fec541995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:51.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.108+0000 7fec5c7ff700 1 -- 172.21.15.2:0/3497654338 >> 172.21.15.2:0/3497654338 conn(0x7fec540fb630 msgr2=0x7fec5410d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:51.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.109+0000 7fec5c7ff700 1 -- 172.21.15.2:0/3497654338 shutdown_connections 2023-09-13T04:40:51.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:40:51.109+0000 7fec5c7ff700 1 -- 172.21.15.2:0/3497654338 wait complete. 2023-09-13T04:40:51.131 INFO:tasks.cephfs.filesystem:scrub status for tag:f51de0fb-b185-42d5-9909-6fdded2622eb - {'path': '/', 'tag': 'f51de0fb-b185-42d5-9909-6fdded2622eb', 'options': 'recursive,force'} 2023-09-13T04:40:51.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:51 smithi149 ceph-mon[133315]: scrub queued for path: / 2023-09-13T04:40:51.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:51 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:40:51.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:51 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:40:51.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:51 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:40:51.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:51 smithi156 ceph-mon[133675]: scrub queued for path: / 2023-09-13T04:40:51.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:51 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:40:51.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:51 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:40:51.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:51 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:40:51.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:51 smithi002 ceph-mon[97651]: scrub queued for path: / 2023-09-13T04:40:51.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:51 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:40:51.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:51 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:40:51.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:51 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:40:52.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:52 smithi149 ceph-mon[133315]: pgmap v15: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 6 op/s 2023-09-13T04:40:52.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:52 smithi156 ceph-mon[133675]: pgmap v15: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 6 op/s 2023-09-13T04:40:52.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:52 smithi002 ceph-mon[97651]: pgmap v15: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 6 op/s 2023-09-13T04:40:53.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:53 smithi149 ceph-mon[133315]: scrub summary: idle 2023-09-13T04:40:53.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:53 smithi156 ceph-mon[133675]: scrub summary: idle 2023-09-13T04:40:53.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:53 smithi002 ceph-mon[97651]: scrub summary: idle 2023-09-13T04:40:54.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:54 smithi149 ceph-mon[133315]: pgmap v16: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 6 op/s 2023-09-13T04:40:54.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:54 smithi156 ceph-mon[133675]: pgmap v16: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 6 op/s 2023-09-13T04:40:54.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:54 smithi002 ceph-mon[97651]: pgmap v16: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 6 op/s 2023-09-13T04:40:56.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:55 smithi149 ceph-mon[133315]: pgmap v17: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-13T04:40:56.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:55 smithi156 ceph-mon[133675]: pgmap v17: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-13T04:40:56.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:55 smithi002 ceph-mon[97651]: pgmap v17: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-13T04:40:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.119+0000 7f3f1abff700 1 -- 172.21.15.2:0/2239031262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f14105ff0 msgr2=0x7f3f1410db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.119+0000 7f3f1abff700 1 --2- 172.21.15.2:0/2239031262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f14105ff0 0x7f3f1410db70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009f90 tx=0x7f3f0c00b330 comp rx=0 tx=0).stop 2023-09-13T04:40:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.119+0000 7f3f1abff700 1 -- 172.21.15.2:0/2239031262 shutdown_connections 2023-09-13T04:40:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.119+0000 7f3f1abff700 1 --2- 172.21.15.2:0/2239031262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f14105ff0 0x7f3f1410db70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.119+0000 7f3f1abff700 1 --2- 172.21.15.2:0/2239031262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f14100ed0 0x7f3f141012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.119+0000 7f3f1abff700 1 --2- 172.21.15.2:0/2239031262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f14101880 0x7f3f141058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.119+0000 7f3f1abff700 1 -- 172.21.15.2:0/2239031262 >> 172.21.15.2:0/2239031262 conn(0x7f3f140fc730 msgr2=0x7f3f140feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.119+0000 7f3f1abff700 1 -- 172.21.15.2:0/2239031262 shutdown_connections 2023-09-13T04:40:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.120+0000 7f3f1abff700 1 -- 172.21.15.2:0/2239031262 wait complete. 2023-09-13T04:40:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.120+0000 7f3f1abff700 1 Processor -- start 2023-09-13T04:40:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.121+0000 7f3f1abff700 1 -- start start 2023-09-13T04:40:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.121+0000 7f3f1abff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f14100ed0 0x7f3f141990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.121+0000 7f3f1abff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f14105ff0 0x7f3f14199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.121+0000 7f3f1abff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f1419dc70 0x7f3f1419e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.121+0000 7f3f1abff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1419e6f0 con 0x7f3f14100ed0 2023-09-13T04:40:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.121+0000 7f3f1abff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1419e860 con 0x7f3f14105ff0 2023-09-13T04:40:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.121+0000 7f3f1abff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1419ea20 con 0x7f3f1419dc70 2023-09-13T04:40:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.121+0000 7f3f1899b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f14100ed0 0x7f3f141990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.121+0000 7f3f1899b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f14100ed0 0x7f3f141990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46504/0 (socket says 172.21.15.2:46504) 2023-09-13T04:40:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.121+0000 7f3f1899b700 1 -- 172.21.15.2:0/3170719966 learned_addr learned my addr 172.21.15.2:0/3170719966 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.122+0000 7f3f13fff700 1 --2- 172.21.15.2:0/3170719966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f14105ff0 0x7f3f14199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.122+0000 7f3f1919c700 1 --2- 172.21.15.2:0/3170719966 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f1419dc70 0x7f3f1419e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.122+0000 7f3f1899b700 1 -- 172.21.15.2:0/3170719966 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f1419dc70 msgr2=0x7f3f1419e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.122+0000 7f3f1899b700 1 --2- 172.21.15.2:0/3170719966 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f1419dc70 0x7f3f1419e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.122+0000 7f3f1899b700 1 -- 172.21.15.2:0/3170719966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f14105ff0 msgr2=0x7f3f14199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.122+0000 7f3f1899b700 1 --2- 172.21.15.2:0/3170719966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f14105ff0 0x7f3f14199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.122+0000 7f3f1899b700 1 -- 172.21.15.2:0/3170719966 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f000095f0 con 0x7f3f14100ed0 2023-09-13T04:40:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.122+0000 7f3f1899b700 1 --2- 172.21.15.2:0/3170719966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f14100ed0 0x7f3f141990c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800e9a0 tx=0x7f3f0800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.123+0000 7f3f11ffb700 1 -- 172.21.15.2:0/3170719966 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f08009790 con 0x7f3f14100ed0 2023-09-13T04:40:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.123+0000 7f3f11ffb700 1 -- 172.21.15.2:0/3170719966 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f08010430 con 0x7f3f14100ed0 2023-09-13T04:40:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.123+0000 7f3f1abff700 1 -- 172.21.15.2:0/3170719966 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0c009c40 con 0x7f3f14100ed0 2023-09-13T04:40:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.123+0000 7f3f11ffb700 1 -- 172.21.15.2:0/3170719966 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f08018680 con 0x7f3f14100ed0 2023-09-13T04:40:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.123+0000 7f3f1abff700 1 -- 172.21.15.2:0/3170719966 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f141a6280 con 0x7f3f14100ed0 2023-09-13T04:40:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.126+0000 7f3f11ffb700 1 -- 172.21.15.2:0/3170719966 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 23) v1 ==== 135434+0+0 (secure 0 0 0) 0x7f3f080105a0 con 0x7f3f14100ed0 2023-09-13T04:40:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.128+0000 7f3f11ffb700 1 --2- 172.21.15.2:0/3170719966 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f3f040a08c0 0x7f3f040a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.128+0000 7f3f11ffb700 1 -- 172.21.15.2:0/3170719966 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f3f08014070 con 0x7f3f14100ed0 2023-09-13T04:40:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.128+0000 7f3f13fff700 1 --2- 172.21.15.2:0/3170719966 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f3f040a08c0 0x7f3f040a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.129+0000 7f3f13fff700 1 --2- 172.21.15.2:0/3170719966 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f3f040a08c0 0x7f3f040a2d80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3f1419a8e0 tx=0x7f3f00009310 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.130+0000 7f3f2e564b80 1 Processor -- start 2023-09-13T04:40:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.130+0000 7f3f2e564b80 1 -- start start 2023-09-13T04:40:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.130+0000 7f3f2e564b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e6318ebe0 0x558e6318efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.130+0000 7f3f2e564b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6318f500 0x558e6318a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.130+0000 7f3f2e564b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318a840 0x558e6318ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.130+0000 7f3f2e564b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e630e3720 con 0x558e6318a840 2023-09-13T04:40:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.130+0000 7f3f2e564b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e630e1040 con 0x558e6318f500 2023-09-13T04:40:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.130+0000 7f3f2e564b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e630dc930 con 0x558e6318ebe0 2023-09-13T04:40:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.131+0000 7f3f1919c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318a840 0x558e6318ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.131+0000 7f3f1919c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318a840 0x558e6318ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46518/0 (socket says 172.21.15.2:46518) 2023-09-13T04:40:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.131+0000 7f3f1919c700 1 -- 172.21.15.2:0/597951364 learned_addr learned my addr 172.21.15.2:0/597951364 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.131+0000 7f3f1899b700 1 --2- 172.21.15.2:0/597951364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e6318ebe0 0x558e6318efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.131+0000 7f3f13fff700 1 --2- 172.21.15.2:0/597951364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6318f500 0x558e6318a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.131+0000 7f3f1919c700 1 -- 172.21.15.2:0/597951364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e6318ebe0 msgr2=0x558e6318efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.131+0000 7f3f1919c700 1 --2- 172.21.15.2:0/597951364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e6318ebe0 0x558e6318efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.131+0000 7f3f1919c700 1 -- 172.21.15.2:0/597951364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6318f500 msgr2=0x558e6318a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.131+0000 7f3f1919c700 1 --2- 172.21.15.2:0/597951364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6318f500 0x558e6318a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.131+0000 7f3f1919c700 1 -- 172.21.15.2:0/597951364 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e62f5c5d0 con 0x558e6318a840 2023-09-13T04:40:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.131+0000 7f3f1919c700 1 --2- 172.21.15.2:0/597951364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318a840 0x558e6318ac80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009be0 tx=0x7f3f0c0093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee14c7a57469214 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.132+0000 7f3efeffd700 1 -- 172.21.15.2:0/597951364 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c00e040 con 0x558e6318a840 2023-09-13T04:40:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.132+0000 7f3efeffd700 1 -- 172.21.15.2:0/597951364 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f0c007d80 con 0x558e6318a840 2023-09-13T04:40:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.132+0000 7f3efeffd700 1 -- 172.21.15.2:0/597951364 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c00bcc0 con 0x558e6318a840 2023-09-13T04:40:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.132+0000 7f3f2e564b80 1 -- 172.21.15.2:0/597951364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318a840 msgr2=0x558e6318ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.132+0000 7f3f2e564b80 1 --2- 172.21.15.2:0/597951364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318a840 0x558e6318ac80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009be0 tx=0x7f3f0c0093c0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.133+0000 7f3f2e564b80 1 -- 172.21.15.2:0/597951364 shutdown_connections 2023-09-13T04:40:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.133+0000 7f3f2e564b80 1 --2- 172.21.15.2:0/597951364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318a840 0x558e6318ac80 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.133+0000 7f3f2e564b80 1 --2- 172.21.15.2:0/597951364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e6318ebe0 0x558e6318efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.133+0000 7f3f2e564b80 1 --2- 172.21.15.2:0/597951364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6318f500 0x558e6318a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.133+0000 7f3f2e564b80 1 -- 172.21.15.2:0/597951364 >> 172.21.15.2:0/597951364 conn(0x558e630e8c20 msgr2=0x558e63184b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.133+0000 7f3f2e564b80 1 -- 172.21.15.2:0/597951364 shutdown_connections 2023-09-13T04:40:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.133+0000 7f3f2e564b80 1 -- 172.21.15.2:0/597951364 wait complete. 2023-09-13T04:40:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.133+0000 7f3f2e564b80 1 -- start start 2023-09-13T04:40:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.133+0000 7f3f2e564b80 1 Processor -- start 2023-09-13T04:40:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.134+0000 7f3f2e564b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e6318a840 0x558e63196570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.134+0000 7f3f2e564b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6318ebe0 0x558e63193780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.134+0000 7f3f2e564b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318f500 0x558e63193d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.134+0000 7f3f2e564b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6307ab50 con 0x558e6318f500 2023-09-13T04:40:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.134+0000 7f3f2e564b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6307a850 con 0x558e6318ebe0 2023-09-13T04:40:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.134+0000 7f3f2e564b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e62f5e8c0 con 0x558e6318a840 2023-09-13T04:40:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.134+0000 7f3f2e564b80 10 client.0 _ll_register_callbacks cb 0x558e63078620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.134+0000 7f3f1919c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318f500 0x558e63193d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.134+0000 7f3f1899b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e6318a840 0x558e63196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.134+0000 7f3f1919c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318f500 0x558e63193d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46526/0 (socket says 172.21.15.2:46526) 2023-09-13T04:40:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.134+0000 7f3f1919c700 1 -- 172.21.15.2:0/540589696 learned_addr learned my addr 172.21.15.2:0/540589696 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.134+0000 7f3f13fff700 1 --2- 172.21.15.2:0/540589696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6318ebe0 0x558e63193780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.135+0000 7f3f1899b700 1 -- 172.21.15.2:0/540589696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6318ebe0 msgr2=0x558e63193780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.135+0000 7f3f1899b700 1 --2- 172.21.15.2:0/540589696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6318ebe0 0x558e63193780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.135+0000 7f3f1899b700 1 -- 172.21.15.2:0/540589696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318f500 msgr2=0x558e63193d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.135+0000 7f3f1899b700 1 --2- 172.21.15.2:0/540589696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318f500 0x558e63193d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.135+0000 7f3f1899b700 1 -- 172.21.15.2:0/540589696 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1410e3e0 con 0x558e6318a840 2023-09-13T04:40:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.135+0000 7f3f1919c700 1 --2- 172.21.15.2:0/540589696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318f500 0x558e63193d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.135+0000 7f3f1899b700 1 --2- 172.21.15.2:0/540589696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e6318a840 0x558e63196570 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3f080c9920 tx=0x7f3f08084390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3939bd9a16b75a83 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.135+0000 7f3ef3fff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:40:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.135+0000 7f3ef3fff700 1 -- 172.21.15.2:0/540589696 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f080cb070 con 0x558e6318a840 2023-09-13T04:40:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.135+0000 7f3f2e564b80 1 -- 172.21.15.2:0/540589696 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e6314fef0 con 0x558e6318a840 2023-09-13T04:40:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.135+0000 7f3ef3fff700 1 -- 172.21.15.2:0/540589696 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f3f0809d400 con 0x558e6318a840 2023-09-13T04:40:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.136+0000 7f3ef3fff700 1 -- 172.21.15.2:0/540589696 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f080a1640 con 0x558e6318a840 2023-09-13T04:40:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.136+0000 7f3f2e564b80 10 client.25747 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.136+0000 7f3f2e564b80 1 -- 172.21.15.2:0/540589696 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x558e630f4740 con 0x558e6318a840 2023-09-13T04:40:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.136+0000 7f3ef3fff700 1 -- 172.21.15.2:0/540589696 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e6314fef0 con 0x558e6318a840 2023-09-13T04:40:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.136+0000 7f3ef3fff700 1 -- 172.21.15.2:0/540589696 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3f08099830 con 0x558e6318a840 2023-09-13T04:40:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.136+0000 7f3f2e564b80 10 client.25747 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.136+0000 7f3f2e564b80 10 client.25747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:40:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.136+0000 7f3f2e564b80 20 client.25747 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.136+0000 7f3f2e564b80 1 --2- 172.21.15.2:0/540589696 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x558e631a3f30 0x558e631a6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.136+0000 7f3f2e564b80 4 client.25747 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:40:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.136+0000 7f3f2e564b80 1 -- 172.21.15.2:0/540589696 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e630d5910 con 0x558e631a3f30 2023-09-13T04:40:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.137+0000 7f3f13fff700 1 --2- 172.21.15.2:0/540589696 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x558e631a3f30 0x558e631a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.137+0000 7f3f13fff700 1 --2- 172.21.15.2:0/540589696 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x558e631a3f30 0x558e631a6350 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69000fc8e21a1f0d server_cookie=52d78737bc8019c3 in_seq=0 out_seq=0 2023-09-13T04:40:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.137+0000 7f3ef3fff700 10 client.25747 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.138+0000 7f3ef3fff700 1 -- 172.21.15.2:0/540589696 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558e630d5910 con 0x558e631a3f30 2023-09-13T04:40:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.138+0000 7f3ef3fff700 10 client.25747 handle_command_reply: tid=0 2023-09-13T04:40:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.138+0000 7f3f2e564b80 1 client.25747 shutdown 2023-09-13T04:40:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.138+0000 7f3f2e564b80 1 -- 172.21.15.2:0/540589696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e6318a840 msgr2=0x558e63196570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.138+0000 7f3f2e564b80 1 --2- 172.21.15.2:0/540589696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e6318a840 0x558e63196570 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3f080c9920 tx=0x7f3f08084390 comp rx=0 tx=0).stop 2023-09-13T04:40:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.138+0000 7f3f2e564b80 1 -- 172.21.15.2:0/540589696 shutdown_connections 2023-09-13T04:40:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.138+0000 7f3f2e564b80 1 --2- 172.21.15.2:0/540589696 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x558e631a3f30 0x558e631a6350 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.138+0000 7f3f2e564b80 1 --2- 172.21.15.2:0/540589696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318f500 0x558e63193d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.138+0000 7f3f2e564b80 1 --2- 172.21.15.2:0/540589696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e6318a840 0x558e63196570 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.138+0000 7f3f2e564b80 1 --2- 172.21.15.2:0/540589696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6318ebe0 0x558e63193780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.138+0000 7f3f2e564b80 1 -- 172.21.15.2:0/540589696 >> 172.21.15.2:0/540589696 conn(0x558e630e8c20 msgr2=0x558e63190330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.138+0000 7f3ef3fff700 0 client.25747 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.139+0000 7f3f2e564b80 1 -- 172.21.15.2:0/540589696 shutdown_connections 2023-09-13T04:40:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.139+0000 7f3f2e564b80 1 -- 172.21.15.2:0/540589696 wait complete. 2023-09-13T04:40:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.139+0000 7f3f2e564b80 20 client.25747 trim_cache size 0 max 16384 2023-09-13T04:40:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.154+0000 7f3f2e564b80 1 Processor -- start 2023-09-13T04:40:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.154+0000 7f3f2e564b80 1 -- start start 2023-09-13T04:40:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.154+0000 7f3f2e564b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e6318a840 0x558e6319af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.154+0000 7f3f2e564b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318ebe0 0x558e6317f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.154+0000 7f3f2e564b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6318f500 0x558e631960a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.154+0000 7f3f2e564b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e63194390 con 0x558e6318ebe0 2023-09-13T04:40:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.154+0000 7f3f2e564b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e63185100 con 0x558e6318f500 2023-09-13T04:40:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.154+0000 7f3f2e564b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e63196cd0 con 0x558e6318a840 2023-09-13T04:40:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.154+0000 7f3f13fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318ebe0 0x558e6317f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.155+0000 7f3f1899b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e6318a840 0x558e6319af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.155+0000 7f3f1899b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e6318a840 0x558e6319af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:42866/0 (socket says 172.21.15.2:42866) 2023-09-13T04:40:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.155+0000 7f3f13fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318ebe0 0x558e6317f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46530/0 (socket says 172.21.15.2:46530) 2023-09-13T04:40:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.155+0000 7f3f1899b700 1 -- 172.21.15.2:0/2819866520 learned_addr learned my addr 172.21.15.2:0/2819866520 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.155+0000 7f3f1919c700 1 --2- 172.21.15.2:0/2819866520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6318f500 0x558e631960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.155+0000 7f3f1899b700 1 -- 172.21.15.2:0/2819866520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6318f500 msgr2=0x558e631960a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.155+0000 7f3f1899b700 1 --2- 172.21.15.2:0/2819866520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6318f500 0x558e631960a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.155+0000 7f3f1899b700 1 -- 172.21.15.2:0/2819866520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318ebe0 msgr2=0x558e6317f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.155+0000 7f3f1899b700 1 --2- 172.21.15.2:0/2819866520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318ebe0 0x558e6317f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.155+0000 7f3f1899b700 1 -- 172.21.15.2:0/2819866520 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e62f08d80 con 0x558e6318a840 2023-09-13T04:40:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.155+0000 7f3f1899b700 1 --2- 172.21.15.2:0/2819866520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e6318a840 0x558e6319af40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800e970 tx=0x7f3f080952c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2bbd11006490746 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.155+0000 7f3eff7fe700 1 -- 172.21.15.2:0/2819866520 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f080cb070 con 0x558e6318a840 2023-09-13T04:40:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.155+0000 7f3eff7fe700 1 -- 172.21.15.2:0/2819866520 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f080c9860 con 0x558e6318a840 2023-09-13T04:40:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.155+0000 7f3eff7fe700 1 -- 172.21.15.2:0/2819866520 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f080b1bd0 con 0x558e6318a840 2023-09-13T04:40:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.156+0000 7f3f2e564b80 1 -- 172.21.15.2:0/2819866520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e6318a840 msgr2=0x558e6319af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.156+0000 7f3f2e564b80 1 --2- 172.21.15.2:0/2819866520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e6318a840 0x558e6319af40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800e970 tx=0x7f3f080952c0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.156+0000 7f3f2e564b80 1 -- 172.21.15.2:0/2819866520 shutdown_connections 2023-09-13T04:40:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.156+0000 7f3f2e564b80 1 --2- 172.21.15.2:0/2819866520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6318f500 0x558e631960a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.156+0000 7f3f2e564b80 1 --2- 172.21.15.2:0/2819866520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e6318a840 0x558e6319af40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.156+0000 7f3f2e564b80 1 --2- 172.21.15.2:0/2819866520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318ebe0 0x558e6317f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.156+0000 7f3f2e564b80 1 -- 172.21.15.2:0/2819866520 >> 172.21.15.2:0/2819866520 conn(0x558e630e8c20 msgr2=0x558e631888b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.156+0000 7f3f2e564b80 1 -- 172.21.15.2:0/2819866520 shutdown_connections 2023-09-13T04:40:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.156+0000 7f3f2e564b80 1 -- 172.21.15.2:0/2819866520 wait complete. 2023-09-13T04:40:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.157+0000 7f3f2e564b80 1 -- start start 2023-09-13T04:40:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.157+0000 7f3f2e564b80 1 Processor -- start 2023-09-13T04:40:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.158+0000 7f3f2e564b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e6318a840 0x558e63199120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.158+0000 7f3f2e564b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318ebe0 0x558e63199660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.158+0000 7f3f2e564b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6318f500 0x558e631f6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.158+0000 7f3f2e564b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e631f72d0 con 0x558e6318ebe0 2023-09-13T04:40:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.158+0000 7f3f2e564b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e631f7440 con 0x558e6318f500 2023-09-13T04:40:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.158+0000 7f3f2e564b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e631f7600 con 0x558e6318a840 2023-09-13T04:40:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.158+0000 7f3f2e564b80 10 client.0 _ll_register_callbacks cb 0x558e63185ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:40:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.158+0000 7f3f13fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318ebe0 0x558e63199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.158+0000 7f3f13fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318ebe0 0x558e63199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46538/0 (socket says 172.21.15.2:46538) 2023-09-13T04:40:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.158+0000 7f3f13fff700 1 -- 172.21.15.2:0/4038077709 learned_addr learned my addr 172.21.15.2:0/4038077709 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:40:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.158+0000 7f3f1899b700 1 --2- 172.21.15.2:0/4038077709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e6318a840 0x558e63199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.159+0000 7f3f1919c700 1 --2- 172.21.15.2:0/4038077709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6318f500 0x558e631f6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.159+0000 7f3f13fff700 1 -- 172.21.15.2:0/4038077709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e6318a840 msgr2=0x558e63199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.159+0000 7f3f13fff700 1 --2- 172.21.15.2:0/4038077709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e6318a840 0x558e63199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.159+0000 7f3f13fff700 1 -- 172.21.15.2:0/4038077709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6318f500 msgr2=0x558e631f6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:40:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.159+0000 7f3f13fff700 1 --2- 172.21.15.2:0/4038077709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6318f500 0x558e631f6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.159+0000 7f3f13fff700 1 -- 172.21.15.2:0/4038077709 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e630f4740 con 0x558e6318ebe0 2023-09-13T04:40:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.159+0000 7f3f1899b700 1 --2- 172.21.15.2:0/4038077709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e6318a840 0x558e63199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:40:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.159+0000 7f3f13fff700 1 --2- 172.21.15.2:0/4038077709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318ebe0 0x558e63199660 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3f00013010 tx=0x7f3f000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8abc40906fab457 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:40:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.159+0000 7f3ef3fff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:40:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.159+0000 7f3ef3fff700 1 -- 172.21.15.2:0/4038077709 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f00022040 con 0x558e6318ebe0 2023-09-13T04:40:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.160+0000 7f3f2e564b80 1 -- 172.21.15.2:0/4038077709 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e63095ff0 con 0x558e6318ebe0 2023-09-13T04:40:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.160+0000 7f3ef3fff700 1 -- 172.21.15.2:0/4038077709 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f3f00003680 con 0x558e6318ebe0 2023-09-13T04:40:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.160+0000 7f3ef3fff700 1 -- 172.21.15.2:0/4038077709 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f00003840 con 0x558e6318ebe0 2023-09-13T04:40:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.160+0000 7f3f2e564b80 10 client.15717 fetch_fsmap learned FSMap version 23 2023-09-13T04:40:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.160+0000 7f3f2e564b80 1 -- 172.21.15.2:0/4038077709 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x558e62ef1c00 con 0x558e6318ebe0 2023-09-13T04:40:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.160+0000 7f3ef3fff700 1 -- 172.21.15.2:0/4038077709 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e63095ff0 con 0x558e6318ebe0 2023-09-13T04:40:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.160+0000 7f3ef3fff700 1 -- 172.21.15.2:0/4038077709 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3f000288e0 con 0x558e6318ebe0 2023-09-13T04:40:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.160+0000 7f3f2e564b80 10 client.15717 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:40:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.160+0000 7f3f2e564b80 10 client.15717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:40:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.160+0000 7f3f2e564b80 20 client.15717 populate_metadata read hostname 'smithi002' 2023-09-13T04:40:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.160+0000 7f3f2e564b80 1 --2- 172.21.15.2:0/4038077709 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x558e631f8db0 0x558e631fb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:40:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.160+0000 7f3f2e564b80 4 client.15717 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:40:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.160+0000 7f3f2e564b80 1 -- 172.21.15.2:0/4038077709 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558e62f29f10 con 0x558e631f8db0 2023-09-13T04:40:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.161+0000 7f3f1899b700 1 --2- 172.21.15.2:0/4038077709 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x558e631f8db0 0x558e631fb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:40:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.161+0000 7f3f1899b700 1 --2- 172.21.15.2:0/4038077709 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x558e631f8db0 0x558e631fb1d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7369992065e3a386 server_cookie=309b1d95ecfb0484 in_seq=0 out_seq=0 2023-09-13T04:40:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.161+0000 7f3ef3fff700 10 client.15717 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.165+0000 7f3ef3fff700 1 -- 172.21.15.2:0/4038077709 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558e62f29f10 con 0x558e631f8db0 2023-09-13T04:40:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.165+0000 7f3ef3fff700 10 client.15717 handle_command_reply: tid=0 2023-09-13T04:40:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.166+0000 7f3f2e564b80 1 client.15717 shutdown 2023-09-13T04:40:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.166+0000 7f3f2e564b80 1 -- 172.21.15.2:0/4038077709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318ebe0 msgr2=0x558e63199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.166+0000 7f3f2e564b80 1 --2- 172.21.15.2:0/4038077709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318ebe0 0x558e63199660 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3f00013010 tx=0x7f3f000047c0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.166+0000 7f3f2e564b80 1 -- 172.21.15.2:0/4038077709 shutdown_connections 2023-09-13T04:40:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.166+0000 7f3f2e564b80 1 --2- 172.21.15.2:0/4038077709 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x558e631f8db0 0x558e631fb1d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.166+0000 7f3f2e564b80 1 --2- 172.21.15.2:0/4038077709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6318f500 0x558e631f6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.166+0000 7f3f2e564b80 1 --2- 172.21.15.2:0/4038077709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e6318a840 0x558e63199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.166+0000 7f3ef3fff700 0 client.15717 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:40:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.166+0000 7f3f2e564b80 1 --2- 172.21.15.2:0/4038077709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6318ebe0 0x558e63199660 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.166+0000 7f3f2e564b80 1 -- 172.21.15.2:0/4038077709 >> 172.21.15.2:0/4038077709 conn(0x558e630e8c20 msgr2=0x558e63192ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.166+0000 7f3f2e564b80 1 -- 172.21.15.2:0/4038077709 shutdown_connections 2023-09-13T04:40:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.166+0000 7f3f2e564b80 1 -- 172.21.15.2:0/4038077709 wait complete. 2023-09-13T04:40:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.166+0000 7f3f2e564b80 20 client.15717 trim_cache size 0 max 16384 2023-09-13T04:40:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:40:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:40:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:40:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:40:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.167+0000 7f3f1abff700 1 -- 172.21.15.2:0/3170719966 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f3f040a08c0 msgr2=0x7f3f040a2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.167+0000 7f3f1abff700 1 --2- 172.21.15.2:0/3170719966 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f3f040a08c0 0x7f3f040a2d80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3f1419a8e0 tx=0x7f3f00009310 comp rx=0 tx=0).stop 2023-09-13T04:40:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.167+0000 7f3f1abff700 1 -- 172.21.15.2:0/3170719966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f14100ed0 msgr2=0x7f3f141990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:40:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.167+0000 7f3f1abff700 1 --2- 172.21.15.2:0/3170719966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f14100ed0 0x7f3f141990c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800e9a0 tx=0x7f3f0800ecb0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.167+0000 7f3f1abff700 1 -- 172.21.15.2:0/3170719966 shutdown_connections 2023-09-13T04:40:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.167+0000 7f3f1abff700 1 --2- 172.21.15.2:0/3170719966 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f1419dc70 0x7f3f1419e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.167+0000 7f3f1abff700 1 --2- 172.21.15.2:0/3170719966 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f3f040a08c0 0x7f3f040a2d80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.167+0000 7f3f1abff700 1 --2- 172.21.15.2:0/3170719966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f14100ed0 0x7f3f141990c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.167+0000 7f3f1abff700 1 --2- 172.21.15.2:0/3170719966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f14105ff0 0x7f3f14199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:40:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.167+0000 7f3f1abff700 1 -- 172.21.15.2:0/3170719966 >> 172.21.15.2:0/3170719966 conn(0x7f3f140fc730 msgr2=0x7f3f140fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:40:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.167+0000 7f3f1abff700 1 -- 172.21.15.2:0/3170719966 shutdown_connections 2023-09-13T04:40:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:40:57.167+0000 7f3f1abff700 1 -- 172.21.15.2:0/3170719966 wait complete. 2023-09-13T04:40:58.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:40:57 smithi149 ceph-mon[133315]: pgmap v18: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 682 B/s wr, 6 op/s 2023-09-13T04:40:58.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:40:57 smithi156 ceph-mon[133675]: pgmap v18: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 682 B/s wr, 6 op/s 2023-09-13T04:40:58.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:40:57 smithi002 ceph-mon[97651]: pgmap v18: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 682 B/s wr, 6 op/s 2023-09-13T04:41:00.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:00 smithi149 ceph-mon[133315]: pgmap v19: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 9 op/s 2023-09-13T04:41:00.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:00 smithi156 ceph-mon[133675]: pgmap v19: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 9 op/s 2023-09-13T04:41:00.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:00 smithi002 ceph-mon[97651]: pgmap v19: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 9 op/s 2023-09-13T04:41:00.756 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:41:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:41:00.461+0000 7f0ff7464700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T04:41:00.756 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:41:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:41:00.461+0000 7f0ff7464700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T04:41:00.756 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:41:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:41:00.461+0000 7f0ff7464700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T04:41:00.756 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:41:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:41:00.461+0000 7f0ff7464700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T04:41:00.756 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:41:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:41:00.461+0000 7f0ff7464700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T04:41:02.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:01 smithi149 ceph-mon[133315]: pgmap v20: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-13T04:41:02.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:01 smithi149 ceph-mon[133315]: mgrmap e24: x(active, since 34s), standbys: y, z 2023-09-13T04:41:02.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:01 smithi156 ceph-mon[133675]: pgmap v20: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-13T04:41:02.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:01 smithi156 ceph-mon[133675]: mgrmap e24: x(active, since 34s), standbys: y, z 2023-09-13T04:41:02.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:01 smithi002 ceph-mon[97651]: pgmap v20: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-13T04:41:02.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:01 smithi002 ceph-mon[97651]: mgrmap e24: x(active, since 34s), standbys: y, z 2023-09-13T04:41:04.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:03 smithi149 ceph-mon[133315]: pgmap v21: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-13T04:41:04.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:03 smithi156 ceph-mon[133675]: pgmap v21: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-13T04:41:04.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:03 smithi002 ceph-mon[97651]: pgmap v21: 129 pgs: 129 active+clean; 1.2 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-13T04:41:06.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:05 smithi149 ceph-mon[133315]: pgmap v22: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2023-09-13T04:41:06.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:05 smithi156 ceph-mon[133675]: pgmap v22: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2023-09-13T04:41:06.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:05 smithi002 ceph-mon[97651]: pgmap v22: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2023-09-13T04:41:08.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:07 smithi149 ceph-mon[133315]: pgmap v23: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 18 op/s 2023-09-13T04:41:08.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:07 smithi156 ceph-mon[133675]: pgmap v23: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 18 op/s 2023-09-13T04:41:08.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:07 smithi002 ceph-mon[97651]: pgmap v23: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 18 op/s 2023-09-13T04:41:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.340+0000 7f3ef8b9d700 1 -- 172.21.15.2:0/2949107459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ef40680d0 msgr2=0x7f3ef40684b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.340+0000 7f3ef8b9d700 1 --2- 172.21.15.2:0/2949107459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ef40680d0 0x7f3ef40684b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3ee40098f0 tx=0x7f3ee4009c00 comp rx=0 tx=0).stop 2023-09-13T04:41:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.340+0000 7f3ef8b9d700 1 -- 172.21.15.2:0/2949107459 shutdown_connections 2023-09-13T04:41:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.340+0000 7f3ef8b9d700 1 --2- 172.21.15.2:0/2949107459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ef410d7d0 0x7f3ef410fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.340+0000 7f3ef8b9d700 1 --2- 172.21.15.2:0/2949107459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ef40680d0 0x7f3ef40684b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.340+0000 7f3ef8b9d700 1 --2- 172.21.15.2:0/2949107459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ef40689f0 0x7f3ef410d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.340+0000 7f3ef8b9d700 1 -- 172.21.15.2:0/2949107459 >> 172.21.15.2:0/2949107459 conn(0x7f3ef4075380 msgr2=0x7f3ef4075790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.340+0000 7f3ef8b9d700 1 -- 172.21.15.2:0/2949107459 shutdown_connections 2023-09-13T04:41:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.340+0000 7f3ef8b9d700 1 -- 172.21.15.2:0/2949107459 wait complete. 2023-09-13T04:41:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.342+0000 7f3ef8b9d700 1 Processor -- start 2023-09-13T04:41:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.342+0000 7f3ef8b9d700 1 -- start start 2023-09-13T04:41:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.342+0000 7f3ef8b9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ef40680d0 0x7f3ef4198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.342+0000 7f3ef8b9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ef40689f0 0x7f3ef4199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.342+0000 7f3ef8b9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ef410d7d0 0x7f3ef419d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.342+0000 7f3ef8b9d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef419de40 con 0x7f3ef40689f0 2023-09-13T04:41:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.342+0000 7f3ef8b9d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef419dfb0 con 0x7f3ef40680d0 2023-09-13T04:41:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.342+0000 7f3ef8b9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef419e170 con 0x7f3ef410d7d0 2023-09-13T04:41:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.343+0000 7f3ef37fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ef40680d0 0x7f3ef4198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.343+0000 7f3ef2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ef40689f0 0x7f3ef4199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.343+0000 7f3ef37fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ef40680d0 0x7f3ef4198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:59330/0 (socket says 172.21.15.2:59330) 2023-09-13T04:41:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.343+0000 7f3ef37fe700 1 -- 172.21.15.2:0/1027903311 learned_addr learned my addr 172.21.15.2:0/1027903311 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.343+0000 7f3ef2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ef40689f0 0x7f3ef4199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46240/0 (socket says 172.21.15.2:46240) 2023-09-13T04:41:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.343+0000 7f3ef3fff700 1 --2- 172.21.15.2:0/1027903311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ef410d7d0 0x7f3ef419d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.343+0000 7f3ef37fe700 1 -- 172.21.15.2:0/1027903311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ef410d7d0 msgr2=0x7f3ef419d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.343+0000 7f3ef37fe700 1 --2- 172.21.15.2:0/1027903311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ef410d7d0 0x7f3ef419d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.343+0000 7f3ef37fe700 1 -- 172.21.15.2:0/1027903311 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ef40689f0 msgr2=0x7f3ef4199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.343+0000 7f3ef37fe700 1 --2- 172.21.15.2:0/1027903311 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ef40689f0 0x7f3ef4199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.343+0000 7f3ef37fe700 1 -- 172.21.15.2:0/1027903311 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec009c40 con 0x7f3ef40680d0 2023-09-13T04:41:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.344+0000 7f3ef37fe700 1 --2- 172.21.15.2:0/1027903311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ef40680d0 0x7f3ef4198ed0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009fd0 tx=0x7f3ee400bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.344+0000 7f3ef0ff9700 1 -- 172.21.15.2:0/1027903311 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee401c070 con 0x7f3ef40680d0 2023-09-13T04:41:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.344+0000 7f3ef0ff9700 1 -- 172.21.15.2:0/1027903311 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ee4005040 con 0x7f3ef40680d0 2023-09-13T04:41:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.344+0000 7f3ef0ff9700 1 -- 172.21.15.2:0/1027903311 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4004930 con 0x7f3ef40680d0 2023-09-13T04:41:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.344+0000 7f3ef8b9d700 1 -- 172.21.15.2:0/1027903311 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee4009520 con 0x7f3ef40680d0 2023-09-13T04:41:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.344+0000 7f3ef8b9d700 1 -- 172.21.15.2:0/1027903311 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef419e630 con 0x7f3ef40680d0 2023-09-13T04:41:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.349+0000 7f3ef0ff9700 1 -- 172.21.15.2:0/1027903311 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 24) v1 ==== 135491+0+0 (secure 0 0 0) 0x7f3ee4020070 con 0x7f3ef40680d0 2023-09-13T04:41:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.351+0000 7f3f0d77ab80 1 Processor -- start 2023-09-13T04:41:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.351+0000 7f3ef0ff9700 1 --2- 172.21.15.2:0/1027903311 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f3edc0a08d0 0x7f3edc0a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.351+0000 7f3f0d77ab80 1 -- start start 2023-09-13T04:41:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.351+0000 7f3ef0ff9700 1 -- 172.21.15.2:0/1027903311 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f3ee40cd0c0 con 0x7f3ef40680d0 2023-09-13T04:41:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.352+0000 7f3f0d77ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56413bb35ce0 0x56413bb360c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.352+0000 7f3f0d77ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56413bb36600 0x56413bb300b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.352+0000 7f3f0d77ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb30640 0x56413bb30ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.352+0000 7f3f0d77ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413ba89720 con 0x56413bb35ce0 2023-09-13T04:41:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.352+0000 7f3f0d77ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413ba87040 con 0x56413bb36600 2023-09-13T04:41:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.352+0000 7f3f0d77ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413ba82930 con 0x56413bb30640 2023-09-13T04:41:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.352+0000 7f3ef3fff700 1 --2- 172.21.15.2:0/1027903311 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f3edc0a08d0 0x7f3edc0a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.352+0000 7f3ef2ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56413bb36600 0x56413bb300b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.352+0000 7f3ef2ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56413bb36600 0x56413bb300b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:59336/0 (socket says 172.21.15.2:59336) 2023-09-13T04:41:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.352+0000 7f3ef2ffd700 1 -- 172.21.15.2:0/866135971 learned_addr learned my addr 172.21.15.2:0/866135971 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.352+0000 7f3ef37fe700 1 --2- 172.21.15.2:0/866135971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56413bb35ce0 0x56413bb360c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.352+0000 7f3ef2ffd700 1 -- 172.21.15.2:0/866135971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb30640 msgr2=0x56413bb30ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.352+0000 7f3ef2ffd700 1 --2- 172.21.15.2:0/866135971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb30640 0x56413bb30ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.352+0000 7f3ef2ffd700 1 -- 172.21.15.2:0/866135971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56413bb35ce0 msgr2=0x56413bb360c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.352+0000 7f3ef2ffd700 1 --2- 172.21.15.2:0/866135971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56413bb35ce0 0x56413bb360c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.352+0000 7f3ef2ffd700 1 -- 172.21.15.2:0/866135971 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56413b9025d0 con 0x56413bb36600 2023-09-13T04:41:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.353+0000 7f3ef37fe700 1 --2- 172.21.15.2:0/866135971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56413bb35ce0 0x56413bb360c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.353+0000 7f3ef3fff700 1 --2- 172.21.15.2:0/1027903311 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f3edc0a08d0 0x7f3edc0a2d90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3eec009fd0 tx=0x7f3eec009920 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.353+0000 7f3ef2ffd700 1 --2- 172.21.15.2:0/866135971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56413bb36600 0x56413bb300b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009d30 tx=0x7f3ee8011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8927d876094b4c57 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.353+0000 7f3ed9ffb700 1 -- 172.21.15.2:0/866135971 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8004250 con 0x56413bb36600 2023-09-13T04:41:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.353+0000 7f3ed9ffb700 1 -- 172.21.15.2:0/866135971 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ee8004cf0 con 0x56413bb36600 2023-09-13T04:41:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.353+0000 7f3ed9ffb700 1 -- 172.21.15.2:0/866135971 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8005610 con 0x56413bb36600 2023-09-13T04:41:10.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.354+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/866135971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56413bb36600 msgr2=0x56413bb300b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:10.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.354+0000 7f3f0d77ab80 1 --2- 172.21.15.2:0/866135971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56413bb36600 0x56413bb300b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009d30 tx=0x7f3ee8011720 comp rx=0 tx=0).stop 2023-09-13T04:41:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.354+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/866135971 shutdown_connections 2023-09-13T04:41:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.354+0000 7f3f0d77ab80 1 --2- 172.21.15.2:0/866135971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb30640 0x56413bb30ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.354+0000 7f3f0d77ab80 1 --2- 172.21.15.2:0/866135971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56413bb35ce0 0x56413bb360c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.354+0000 7f3f0d77ab80 1 --2- 172.21.15.2:0/866135971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56413bb36600 0x56413bb300b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.354+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/866135971 >> 172.21.15.2:0/866135971 conn(0x56413ba8ec20 msgr2=0x56413bb2ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.354+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/866135971 shutdown_connections 2023-09-13T04:41:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.354+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/866135971 wait complete. 2023-09-13T04:41:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.355+0000 7f3f0d77ab80 1 -- start start 2023-09-13T04:41:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.355+0000 7f3f0d77ab80 1 Processor -- start 2023-09-13T04:41:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.356+0000 7f3f0d77ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56413bb30640 0x56413bb3c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.356+0000 7f3f0d77ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56413bb35ce0 0x56413bb39930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.356+0000 7f3f0d77ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb36600 0x56413bb3a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.356+0000 7f3f0d77ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413ba20b50 con 0x56413bb30640 2023-09-13T04:41:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.356+0000 7f3f0d77ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413ba20850 con 0x56413bb35ce0 2023-09-13T04:41:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.356+0000 7f3f0d77ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413b9048c0 con 0x56413bb36600 2023-09-13T04:41:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.356+0000 7f3f0d77ab80 10 client.0 _ll_register_callbacks cb 0x56413ba1e620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.356+0000 7f3ef3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb36600 0x56413bb3a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.356+0000 7f3ef3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb36600 0x56413bb3a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:32842/0 (socket says 172.21.15.2:32842) 2023-09-13T04:41:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.356+0000 7f3ef3fff700 1 -- 172.21.15.2:0/4077064332 learned_addr learned my addr 172.21.15.2:0/4077064332 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.356+0000 7f3ef2ffd700 1 --2- 172.21.15.2:0/4077064332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56413bb35ce0 0x56413bb39930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.357+0000 7f3ef37fe700 1 --2- 172.21.15.2:0/4077064332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56413bb30640 0x56413bb3c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.357+0000 7f3ef3fff700 1 -- 172.21.15.2:0/4077064332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56413bb35ce0 msgr2=0x56413bb39930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.357+0000 7f3ef3fff700 1 --2- 172.21.15.2:0/4077064332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56413bb35ce0 0x56413bb39930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.357+0000 7f3ef3fff700 1 -- 172.21.15.2:0/4077064332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56413bb30640 msgr2=0x56413bb3c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.357+0000 7f3ef3fff700 1 --2- 172.21.15.2:0/4077064332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56413bb30640 0x56413bb3c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.357+0000 7f3ef3fff700 1 -- 172.21.15.2:0/4077064332 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee8009960 con 0x56413bb36600 2023-09-13T04:41:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.357+0000 7f3ef2ffd700 1 --2- 172.21.15.2:0/4077064332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56413bb35ce0 0x56413bb39930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.357+0000 7f3ef3fff700 1 --2- 172.21.15.2:0/4077064332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb36600 0x56413bb3a070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3eec017480 tx=0x7f3eec004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ada2110a1fa0195 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.357+0000 7f3ed2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:41:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.357+0000 7f3ed2ffd700 1 -- 172.21.15.2:0/4077064332 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec00e040 con 0x56413bb36600 2023-09-13T04:41:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.358+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/4077064332 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56413bb23260 con 0x56413bb36600 2023-09-13T04:41:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.358+0000 7f3ed2ffd700 1 -- 172.21.15.2:0/4077064332 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f3eec02c450 con 0x56413bb36600 2023-09-13T04:41:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.358+0000 7f3ed2ffd700 1 -- 172.21.15.2:0/4077064332 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3eec02c630 con 0x56413bb36600 2023-09-13T04:41:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.358+0000 7f3f0d77ab80 10 client.25768 fetch_fsmap learned FSMap version 23 2023-09-13T04:41:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.358+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/4077064332 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56413ba9a740 con 0x56413bb36600 2023-09-13T04:41:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.358+0000 7f3ed2ffd700 1 -- 172.21.15.2:0/4077064332 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56413bb23260 con 0x56413bb36600 2023-09-13T04:41:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.358+0000 7f3ed2ffd700 1 -- 172.21.15.2:0/4077064332 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3eec022850 con 0x56413bb36600 2023-09-13T04:41:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.359+0000 7f3f0d77ab80 10 client.25768 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:41:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.359+0000 7f3f0d77ab80 10 client.25768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:41:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.359+0000 7f3f0d77ab80 20 client.25768 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.359+0000 7f3f0d77ab80 1 --2- 172.21.15.2:0/4077064332 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56413bb49f00 0x56413bb4c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.359+0000 7f3f0d77ab80 4 client.25768 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:41:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.359+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/4077064332 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56413ba7b910 con 0x56413bb49f00 2023-09-13T04:41:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.359+0000 7f3ef37fe700 1 --2- 172.21.15.2:0/4077064332 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56413bb49f00 0x56413bb4c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.360+0000 7f3ef37fe700 1 --2- 172.21.15.2:0/4077064332 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56413bb49f00 0x56413bb4c320 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f01d703c7919164 server_cookie=c299239b1d7ff3e7 in_seq=0 out_seq=0 2023-09-13T04:41:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.360+0000 7f3ed2ffd700 10 client.25768 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.361+0000 7f3ed2ffd700 1 -- 172.21.15.2:0/4077064332 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56413ba7b910 con 0x56413bb49f00 2023-09-13T04:41:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.361+0000 7f3ed2ffd700 10 client.25768 handle_command_reply: tid=0 2023-09-13T04:41:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.361+0000 7f3f0d77ab80 1 client.25768 shutdown 2023-09-13T04:41:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.361+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/4077064332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb36600 msgr2=0x56413bb3a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.361+0000 7f3f0d77ab80 1 --2- 172.21.15.2:0/4077064332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb36600 0x56413bb3a070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3eec017480 tx=0x7f3eec004ad0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.361+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/4077064332 shutdown_connections 2023-09-13T04:41:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.361+0000 7f3f0d77ab80 1 --2- 172.21.15.2:0/4077064332 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56413bb49f00 0x56413bb4c320 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.361+0000 7f3f0d77ab80 1 --2- 172.21.15.2:0/4077064332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb36600 0x56413bb3a070 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.361+0000 7f3f0d77ab80 1 --2- 172.21.15.2:0/4077064332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56413bb30640 0x56413bb3c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.361+0000 7f3ed2ffd700 0 client.25768 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.361+0000 7f3f0d77ab80 1 --2- 172.21.15.2:0/4077064332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56413bb35ce0 0x56413bb39930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.361+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/4077064332 >> 172.21.15.2:0/4077064332 conn(0x56413ba8ec20 msgr2=0x56413bb2f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.361+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/4077064332 shutdown_connections 2023-09-13T04:41:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.361+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/4077064332 wait complete. 2023-09-13T04:41:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.361+0000 7f3f0d77ab80 20 client.25768 trim_cache size 0 max 16384 2023-09-13T04:41:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.389+0000 7f3f0d77ab80 1 Processor -- start 2023-09-13T04:41:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.390+0000 7f3f0d77ab80 1 -- start start 2023-09-13T04:41:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.390+0000 7f3f0d77ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb30640 0x56413bb40f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.390+0000 7f3f0d77ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56413bb35ce0 0x56413bb2f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.390+0000 7f3f0d77ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56413bb36600 0x56413bb3c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.390+0000 7f3f0d77ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413bb271c0 con 0x56413bb35ce0 2023-09-13T04:41:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.390+0000 7f3f0d77ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413bb3cce0 con 0x56413bb36600 2023-09-13T04:41:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.390+0000 7f3f0d77ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413bb414a0 con 0x56413bb30640 2023-09-13T04:41:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.390+0000 7f3ef2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56413bb35ce0 0x56413bb2f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.390+0000 7f3ef37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb30640 0x56413bb40f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.390+0000 7f3ef37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb30640 0x56413bb40f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:32848/0 (socket says 172.21.15.2:32848) 2023-09-13T04:41:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.390+0000 7f3ef2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56413bb35ce0 0x56413bb2f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46266/0 (socket says 172.21.15.2:46266) 2023-09-13T04:41:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.390+0000 7f3ef37fe700 1 -- 172.21.15.2:0/313307872 learned_addr learned my addr 172.21.15.2:0/313307872 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.391+0000 7f3ef3fff700 1 --2- 172.21.15.2:0/313307872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56413bb36600 0x56413bb3c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.391+0000 7f3ef37fe700 1 -- 172.21.15.2:0/313307872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56413bb36600 msgr2=0x56413bb3c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.391+0000 7f3ef37fe700 1 --2- 172.21.15.2:0/313307872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56413bb36600 0x56413bb3c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.391+0000 7f3ef37fe700 1 -- 172.21.15.2:0/313307872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56413bb35ce0 msgr2=0x56413bb2f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.391+0000 7f3ef37fe700 1 --2- 172.21.15.2:0/313307872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56413bb35ce0 0x56413bb2f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.391+0000 7f3ef37fe700 1 -- 172.21.15.2:0/313307872 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56413b8aed80 con 0x56413bb30640 2023-09-13T04:41:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.391+0000 7f3ef2ffd700 1 --2- 172.21.15.2:0/313307872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56413bb35ce0 0x56413bb2f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.391+0000 7f3ef3fff700 1 --2- 172.21.15.2:0/313307872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56413bb36600 0x56413bb3c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.391+0000 7f3ef37fe700 1 --2- 172.21.15.2:0/313307872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb30640 0x56413bb40f60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4099fd0 tx=0x7f3ee4095c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8609706613410484 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.392+0000 7f3eda7fc700 1 -- 172.21.15.2:0/313307872 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40051b0 con 0x56413bb30640 2023-09-13T04:41:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.392+0000 7f3eda7fc700 1 -- 172.21.15.2:0/313307872 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ee409a740 con 0x56413bb30640 2023-09-13T04:41:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.392+0000 7f3eda7fc700 1 -- 172.21.15.2:0/313307872 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40b23d0 con 0x56413bb30640 2023-09-13T04:41:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.392+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/313307872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb30640 msgr2=0x56413bb40f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.392+0000 7f3f0d77ab80 1 --2- 172.21.15.2:0/313307872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb30640 0x56413bb40f60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4099fd0 tx=0x7f3ee4095c60 comp rx=0 tx=0).stop 2023-09-13T04:41:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.392+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/313307872 shutdown_connections 2023-09-13T04:41:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.392+0000 7f3f0d77ab80 1 --2- 172.21.15.2:0/313307872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56413bb36600 0x56413bb3c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.392+0000 7f3f0d77ab80 1 --2- 172.21.15.2:0/313307872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb30640 0x56413bb40f60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.392+0000 7f3f0d77ab80 1 --2- 172.21.15.2:0/313307872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56413bb35ce0 0x56413bb2f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.392+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/313307872 >> 172.21.15.2:0/313307872 conn(0x56413ba8ec20 msgr2=0x56413bb2fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.393+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/313307872 shutdown_connections 2023-09-13T04:41:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.393+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/313307872 wait complete. 2023-09-13T04:41:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.393+0000 7f3f0d77ab80 1 -- start start 2023-09-13T04:41:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.393+0000 7f3f0d77ab80 1 Processor -- start 2023-09-13T04:41:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.394+0000 7f3f0d77ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56413bb30640 0x56413bb3ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.394+0000 7f3f0d77ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb35ce0 0x56413bb3f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.394+0000 7f3f0d77ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56413bb36600 0x56413bb3f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.394+0000 7f3f0d77ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413bb9ced0 con 0x56413bb30640 2023-09-13T04:41:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.394+0000 7f3f0d77ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413bb9d040 con 0x56413bb36600 2023-09-13T04:41:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.394+0000 7f3f0d77ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413bb9d200 con 0x56413bb35ce0 2023-09-13T04:41:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.394+0000 7f3f0d77ab80 10 client.0 _ll_register_callbacks cb 0x56413ba1e620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.395+0000 7f3ef37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56413bb30640 0x56413bb3ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.395+0000 7f3ef2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb35ce0 0x56413bb3f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.395+0000 7f3ef37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56413bb30640 0x56413bb3ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46268/0 (socket says 172.21.15.2:46268) 2023-09-13T04:41:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.395+0000 7f3ef2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb35ce0 0x56413bb3f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:32864/0 (socket says 172.21.15.2:32864) 2023-09-13T04:41:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.395+0000 7f3ef37fe700 1 -- 172.21.15.2:0/2011317546 learned_addr learned my addr 172.21.15.2:0/2011317546 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.395+0000 7f3ef3fff700 1 --2- 172.21.15.2:0/2011317546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56413bb36600 0x56413bb3f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.395+0000 7f3ef2ffd700 1 -- 172.21.15.2:0/2011317546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56413bb36600 msgr2=0x56413bb3f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.395+0000 7f3ef2ffd700 1 --2- 172.21.15.2:0/2011317546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56413bb36600 0x56413bb3f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.395+0000 7f3ef2ffd700 1 -- 172.21.15.2:0/2011317546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56413bb30640 msgr2=0x56413bb3ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.395+0000 7f3ef2ffd700 1 --2- 172.21.15.2:0/2011317546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56413bb30640 0x56413bb3ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.395+0000 7f3ef2ffd700 1 -- 172.21.15.2:0/2011317546 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee801d040 con 0x56413bb35ce0 2023-09-13T04:41:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.396+0000 7f3ef2ffd700 1 --2- 172.21.15.2:0/2011317546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb35ce0 0x56413bb3f480 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8011a70 tx=0x7f3ee80045d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d60f052f90023a4a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.396+0000 7f3ed2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:41:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.396+0000 7f3ed2ffd700 1 -- 172.21.15.2:0/2011317546 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8003680 con 0x56413bb35ce0 2023-09-13T04:41:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.396+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/2011317546 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ee8004cf0 con 0x56413bb35ce0 2023-09-13T04:41:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.396+0000 7f3ed2ffd700 1 -- 172.21.15.2:0/2011317546 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f3ee80037e0 con 0x56413bb35ce0 2023-09-13T04:41:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.396+0000 7f3ed2ffd700 1 -- 172.21.15.2:0/2011317546 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ee80039a0 con 0x56413bb35ce0 2023-09-13T04:41:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.396+0000 7f3f0d77ab80 10 client.25774 fetch_fsmap learned FSMap version 23 2023-09-13T04:41:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.396+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/2011317546 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56413ba9a740 con 0x56413bb35ce0 2023-09-13T04:41:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.397+0000 7f3ed2ffd700 1 -- 172.21.15.2:0/2011317546 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ee8004cf0 con 0x56413bb35ce0 2023-09-13T04:41:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.397+0000 7f3ed2ffd700 1 -- 172.21.15.2:0/2011317546 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3ee8025430 con 0x56413bb35ce0 2023-09-13T04:41:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.397+0000 7f3f0d77ab80 10 client.25774 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:41:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.397+0000 7f3f0d77ab80 10 client.25774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:41:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.397+0000 7f3f0d77ab80 20 client.25774 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.397+0000 7f3f0d77ab80 1 --2- 172.21.15.2:0/2011317546 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56413bb9e800 0x56413bba0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.397+0000 7f3f0d77ab80 4 client.25774 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:41:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.397+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/2011317546 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56413ba3bff0 con 0x56413bb9e800 2023-09-13T04:41:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.398+0000 7f3ef37fe700 1 --2- 172.21.15.2:0/2011317546 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56413bb9e800 0x56413bba0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.398+0000 7f3ef37fe700 1 --2- 172.21.15.2:0/2011317546 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56413bb9e800 0x56413bba0c20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2caeed0ceddedde4 server_cookie=2228cb10bfb4ca9b in_seq=0 out_seq=0 2023-09-13T04:41:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.398+0000 7f3ed2ffd700 10 client.25774 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.405+0000 7f3ed2ffd700 1 -- 172.21.15.2:0/2011317546 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56413ba3bff0 con 0x56413bb9e800 2023-09-13T04:41:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.405+0000 7f3ed2ffd700 10 client.25774 handle_command_reply: tid=0 2023-09-13T04:41:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.405+0000 7f3f0d77ab80 1 client.25774 shutdown 2023-09-13T04:41:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.405+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/2011317546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb35ce0 msgr2=0x56413bb3f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.405+0000 7f3f0d77ab80 1 --2- 172.21.15.2:0/2011317546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb35ce0 0x56413bb3f480 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8011a70 tx=0x7f3ee80045d0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.405+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/2011317546 shutdown_connections 2023-09-13T04:41:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.405+0000 7f3f0d77ab80 1 --2- 172.21.15.2:0/2011317546 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56413bb9e800 0x56413bba0c20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.405+0000 7f3f0d77ab80 1 --2- 172.21.15.2:0/2011317546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56413bb36600 0x56413bb3f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.405+0000 7f3f0d77ab80 1 --2- 172.21.15.2:0/2011317546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56413bb30640 0x56413bb3ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.405+0000 7f3ed2ffd700 0 client.25774 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.405+0000 7f3f0d77ab80 1 --2- 172.21.15.2:0/2011317546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56413bb35ce0 0x56413bb3f480 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.405+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/2011317546 >> 172.21.15.2:0/2011317546 conn(0x56413ba8ec20 msgr2=0x56413bb461b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.405+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/2011317546 shutdown_connections 2023-09-13T04:41:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.406+0000 7f3f0d77ab80 1 -- 172.21.15.2:0/2011317546 wait complete. 2023-09-13T04:41:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.406+0000 7f3f0d77ab80 20 client.25774 trim_cache size 0 max 16384 2023-09-13T04:41:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:41:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:41:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:41:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:41:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.406+0000 7f3ef8b9d700 1 -- 172.21.15.2:0/1027903311 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f3edc0a08d0 msgr2=0x7f3edc0a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.406+0000 7f3ef8b9d700 1 --2- 172.21.15.2:0/1027903311 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f3edc0a08d0 0x7f3edc0a2d90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3eec009fd0 tx=0x7f3eec009920 comp rx=0 tx=0).stop 2023-09-13T04:41:10.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.406+0000 7f3ef8b9d700 1 -- 172.21.15.2:0/1027903311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ef40680d0 msgr2=0x7f3ef4198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:10.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.406+0000 7f3ef8b9d700 1 --2- 172.21.15.2:0/1027903311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ef40680d0 0x7f3ef4198ed0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009fd0 tx=0x7f3ee400bfa0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.407+0000 7f3ef8b9d700 1 -- 172.21.15.2:0/1027903311 shutdown_connections 2023-09-13T04:41:10.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.407+0000 7f3ef8b9d700 1 --2- 172.21.15.2:0/1027903311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ef410d7d0 0x7f3ef419d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.407+0000 7f3ef8b9d700 1 --2- 172.21.15.2:0/1027903311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ef40680d0 0x7f3ef4198ed0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.407+0000 7f3ef8b9d700 1 --2- 172.21.15.2:0/1027903311 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f3edc0a08d0 0x7f3edc0a2d90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.407+0000 7f3ef8b9d700 1 --2- 172.21.15.2:0/1027903311 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ef40689f0 0x7f3ef4199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.407+0000 7f3ef8b9d700 1 -- 172.21.15.2:0/1027903311 >> 172.21.15.2:0/1027903311 conn(0x7f3ef4075380 msgr2=0x7f3ef410ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.407+0000 7f3ef8b9d700 1 -- 172.21.15.2:0/1027903311 shutdown_connections 2023-09-13T04:41:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:10.407+0000 7f3ef8b9d700 1 -- 172.21.15.2:0/1027903311 wait complete. 2023-09-13T04:41:10.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:10 smithi149 ceph-mon[133315]: pgmap v24: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2023-09-13T04:41:10.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:10 smithi156 ceph-mon[133675]: pgmap v24: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2023-09-13T04:41:10.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:10 smithi002 ceph-mon[97651]: pgmap v24: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2023-09-13T04:41:12.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:11 smithi149 ceph-mon[133315]: pgmap v25: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 17 op/s 2023-09-13T04:41:12.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:11 smithi156 ceph-mon[133675]: pgmap v25: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 17 op/s 2023-09-13T04:41:12.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:11 smithi002 ceph-mon[97651]: pgmap v25: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 17 op/s 2023-09-13T04:41:14.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:13 smithi149 ceph-mon[133315]: pgmap v26: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 18 op/s 2023-09-13T04:41:14.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:13 smithi156 ceph-mon[133675]: pgmap v26: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 18 op/s 2023-09-13T04:41:14.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:13 smithi002 ceph-mon[97651]: pgmap v26: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 18 op/s 2023-09-13T04:41:16.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:15 smithi149 ceph-mon[133315]: pgmap v27: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 21 op/s 2023-09-13T04:41:16.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:15 smithi156 ceph-mon[133675]: pgmap v27: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 21 op/s 2023-09-13T04:41:16.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:15 smithi002 ceph-mon[97651]: pgmap v27: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 21 op/s 2023-09-13T04:41:18.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:17 smithi149 ceph-mon[133315]: pgmap v28: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2023-09-13T04:41:18.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:17 smithi156 ceph-mon[133675]: pgmap v28: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2023-09-13T04:41:18.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:17 smithi002 ceph-mon[97651]: pgmap v28: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2023-09-13T04:41:20.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:20 smithi149 ceph-mon[133315]: pgmap v29: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 853 B/s wr, 8 op/s 2023-09-13T04:41:20.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:20 smithi156 ceph-mon[133675]: pgmap v29: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 853 B/s wr, 8 op/s 2023-09-13T04:41:20.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:20 smithi002 ceph-mon[97651]: pgmap v29: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 853 B/s wr, 8 op/s 2023-09-13T04:41:21.133 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T04:41:21.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.485+0000 7fe955bff700 1 -- 172.21.15.2:0/825254638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe950100ec0 msgr2=0x7fe9501012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:21.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.485+0000 7fe955bff700 1 --2- 172.21.15.2:0/825254638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe950100ec0 0x7fe9501012a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe940009990 tx=0x7fe940009ca0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.485+0000 7fe955bff700 1 -- 172.21.15.2:0/825254638 shutdown_connections 2023-09-13T04:41:21.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.485+0000 7fe955bff700 1 --2- 172.21.15.2:0/825254638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe950105fe0 0x7fe95010db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.485+0000 7fe955bff700 1 --2- 172.21.15.2:0/825254638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe950100ec0 0x7fe9501012a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.485+0000 7fe955bff700 1 --2- 172.21.15.2:0/825254638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe950101870 0x7fe9501058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.485+0000 7fe955bff700 1 -- 172.21.15.2:0/825254638 >> 172.21.15.2:0/825254638 conn(0x7fe9500fc760 msgr2=0x7fe9500feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:21.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.485+0000 7fe955bff700 1 -- 172.21.15.2:0/825254638 shutdown_connections 2023-09-13T04:41:21.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.485+0000 7fe955bff700 1 -- 172.21.15.2:0/825254638 wait complete. 2023-09-13T04:41:21.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.487+0000 7fe955bff700 1 Processor -- start 2023-09-13T04:41:21.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.487+0000 7fe955bff700 1 -- start start 2023-09-13T04:41:21.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.488+0000 7fe955bff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe950100ec0 0x7fe950198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:21.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.488+0000 7fe955bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe950105fe0 0x7fe950199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:21.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.488+0000 7fe955bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe95019d980 0x7fe95019de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:21.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.488+0000 7fe955bff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95019e400 con 0x7fe950100ec0 2023-09-13T04:41:21.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.488+0000 7fe955bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95019e570 con 0x7fe950105fe0 2023-09-13T04:41:21.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.488+0000 7fe955bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95019e730 con 0x7fe95019d980 2023-09-13T04:41:21.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.488+0000 7fe94f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe950100ec0 0x7fe950198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:21.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.488+0000 7fe94f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe950100ec0 0x7fe950198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45094/0 (socket says 172.21.15.2:45094) 2023-09-13T04:41:21.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.488+0000 7fe94f7fe700 1 -- 172.21.15.2:0/2421770713 learned_addr learned my addr 172.21.15.2:0/2421770713 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:21.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.488+0000 7fe94effd700 1 --2- 172.21.15.2:0/2421770713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe950105fe0 0x7fe950199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:21.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.489+0000 7fe94ffff700 1 --2- 172.21.15.2:0/2421770713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe95019d980 0x7fe95019de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:21.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.489+0000 7fe94f7fe700 1 -- 172.21.15.2:0/2421770713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe95019d980 msgr2=0x7fe95019de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:21.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.489+0000 7fe94f7fe700 1 --2- 172.21.15.2:0/2421770713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe95019d980 0x7fe95019de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.489+0000 7fe94f7fe700 1 -- 172.21.15.2:0/2421770713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe950105fe0 msgr2=0x7fe950199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:21.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.489+0000 7fe94f7fe700 1 --2- 172.21.15.2:0/2421770713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe950105fe0 0x7fe950199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.489+0000 7fe94f7fe700 1 -- 172.21.15.2:0/2421770713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9400095f0 con 0x7fe950100ec0 2023-09-13T04:41:21.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.489+0000 7fe94f7fe700 1 --2- 172.21.15.2:0/2421770713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe950100ec0 0x7fe950198fe0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe940005910 tx=0x7fe940005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:21.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.490+0000 7fe94cff9700 1 -- 172.21.15.2:0/2421770713 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94001d070 con 0x7fe950100ec0 2023-09-13T04:41:21.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.490+0000 7fe94cff9700 1 -- 172.21.15.2:0/2421770713 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe940022470 con 0x7fe950100ec0 2023-09-13T04:41:21.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.490+0000 7fe94cff9700 1 -- 172.21.15.2:0/2421770713 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940017650 con 0x7fe950100ec0 2023-09-13T04:41:21.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.490+0000 7fe955bff700 1 -- 172.21.15.2:0/2421770713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9501a5c30 con 0x7fe950100ec0 2023-09-13T04:41:21.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.490+0000 7fe955bff700 1 -- 172.21.15.2:0/2421770713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9501a6050 con 0x7fe950100ec0 2023-09-13T04:41:21.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.496+0000 7fe94cff9700 1 -- 172.21.15.2:0/2421770713 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 24) v1 ==== 135491+0+0 (secure 0 0 0) 0x7fe9400225e0 con 0x7fe950100ec0 2023-09-13T04:41:21.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.496+0000 7fe9694e7b80 1 Processor -- start 2023-09-13T04:41:21.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.497+0000 7fe9694e7b80 1 -- start start 2023-09-13T04:41:21.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.497+0000 7fe9694e7b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc28698ec0 0x55fc286992a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:21.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.497+0000 7fe9694e7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc286997e0 0x55fc2869d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:21.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.497+0000 7fe9694e7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc2869d930 0x55fc2869dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:21.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.497+0000 7fe9694e7b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc285e9780 con 0x55fc28698ec0 2023-09-13T04:41:21.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.497+0000 7fe9694e7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc285e70a0 con 0x55fc286997e0 2023-09-13T04:41:21.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.497+0000 7fe9694e7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc285e2990 con 0x55fc2869d930 2023-09-13T04:41:21.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.498+0000 7fe94f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc286997e0 0x55fc2869d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:21.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.498+0000 7fe94f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc286997e0 0x55fc2869d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:44278/0 (socket says 172.21.15.2:44278) 2023-09-13T04:41:21.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.498+0000 7fe94f7fe700 1 -- 172.21.15.2:0/3187975171 learned_addr learned my addr 172.21.15.2:0/3187975171 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:21.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.498+0000 7fe94f7fe700 1 -- 172.21.15.2:0/3187975171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc2869d930 msgr2=0x55fc2869dd90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:41:21.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.498+0000 7fe94f7fe700 1 --2- 172.21.15.2:0/3187975171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc2869d930 0x55fc2869dd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.498+0000 7fe94f7fe700 1 -- 172.21.15.2:0/3187975171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc28698ec0 msgr2=0x55fc286992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:21.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.498+0000 7fe94f7fe700 1 --2- 172.21.15.2:0/3187975171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc28698ec0 0x55fc286992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.498+0000 7fe94f7fe700 1 -- 172.21.15.2:0/3187975171 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc28462630 con 0x55fc286997e0 2023-09-13T04:41:21.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.499+0000 7fe94cff9700 1 --2- 172.21.15.2:0/2421770713 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe9380a0950 0x7fe9380a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:21.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.499+0000 7fe94f7fe700 1 --2- 172.21.15.2:0/3187975171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc286997e0 0x55fc2869d3f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe9400caae0 tx=0x7fe9400d3d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20765b6972fdcbf2 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:21.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.499+0000 7fe94ffff700 1 --2- 172.21.15.2:0/2421770713 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe9380a0950 0x7fe9380a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:21.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.499+0000 7fe935ffb700 1 -- 172.21.15.2:0/3187975171 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940096d90 con 0x55fc286997e0 2023-09-13T04:41:21.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.499+0000 7fe935ffb700 1 -- 172.21.15.2:0/3187975171 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9400d36f0 con 0x55fc286997e0 2023-09-13T04:41:21.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.499+0000 7fe935ffb700 1 -- 172.21.15.2:0/3187975171 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94009aa50 con 0x55fc286997e0 2023-09-13T04:41:21.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.500+0000 7fe94cff9700 1 -- 172.21.15.2:0/2421770713 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fe9400cd8a0 con 0x7fe950100ec0 2023-09-13T04:41:21.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.501+0000 7fe94ffff700 1 --2- 172.21.15.2:0/2421770713 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe9380a0950 0x7fe9380a2e10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe9480093f0 tx=0x7fe948009380 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:21.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.501+0000 7fe9694e7b80 1 -- 172.21.15.2:0/3187975171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc286997e0 msgr2=0x55fc2869d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:21.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.501+0000 7fe9694e7b80 1 --2- 172.21.15.2:0/3187975171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc286997e0 0x55fc2869d3f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe9400caae0 tx=0x7fe9400d3d60 comp rx=0 tx=0).stop 2023-09-13T04:41:21.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.501+0000 7fe9694e7b80 1 -- 172.21.15.2:0/3187975171 shutdown_connections 2023-09-13T04:41:21.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.501+0000 7fe9694e7b80 1 --2- 172.21.15.2:0/3187975171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc2869d930 0x55fc2869dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.501+0000 7fe9694e7b80 1 --2- 172.21.15.2:0/3187975171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc28698ec0 0x55fc286992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.501+0000 7fe9694e7b80 1 --2- 172.21.15.2:0/3187975171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc286997e0 0x55fc2869d3f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.501+0000 7fe9694e7b80 1 -- 172.21.15.2:0/3187975171 >> 172.21.15.2:0/3187975171 conn(0x55fc285eec80 msgr2=0x55fc2868af20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:21.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.501+0000 7fe9694e7b80 1 -- 172.21.15.2:0/3187975171 shutdown_connections 2023-09-13T04:41:21.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.501+0000 7fe9694e7b80 1 -- 172.21.15.2:0/3187975171 wait complete. 2023-09-13T04:41:21.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.502+0000 7fe9694e7b80 1 -- start start 2023-09-13T04:41:21.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.502+0000 7fe9694e7b80 1 Processor -- start 2023-09-13T04:41:21.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.503+0000 7fe9694e7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc28698ec0 0x55fc286a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:21.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.503+0000 7fe9694e7b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc286997e0 0x55fc286a22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:21.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.503+0000 7fe9694e7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc2869d930 0x55fc286a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:21.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.503+0000 7fe9694e7b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc28580bb0 con 0x55fc286997e0 2023-09-13T04:41:21.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.503+0000 7fe9694e7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc285808b0 con 0x55fc2869d930 2023-09-13T04:41:21.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.503+0000 7fe9694e7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc28464920 con 0x55fc28698ec0 2023-09-13T04:41:21.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.503+0000 7fe9694e7b80 10 client.0 _ll_register_callbacks cb 0x55fc2857e680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:21.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.503+0000 7fe94effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc286997e0 0x55fc286a22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:21.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.503+0000 7fe94effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc286997e0 0x55fc286a22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45114/0 (socket says 172.21.15.2:45114) 2023-09-13T04:41:21.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.503+0000 7fe94effd700 1 -- 172.21.15.2:0/716221113 learned_addr learned my addr 172.21.15.2:0/716221113 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:21.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.504+0000 7fe94f7fe700 1 --2- 172.21.15.2:0/716221113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc28698ec0 0x55fc286a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:21.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.504+0000 7fe94ffff700 1 --2- 172.21.15.2:0/716221113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc2869d930 0x55fc286a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:21.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.504+0000 7fe94f7fe700 1 -- 172.21.15.2:0/716221113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc2869d930 msgr2=0x55fc286a28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:21.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.504+0000 7fe94f7fe700 1 --2- 172.21.15.2:0/716221113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc2869d930 0x55fc286a28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.504+0000 7fe94f7fe700 1 -- 172.21.15.2:0/716221113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc286997e0 msgr2=0x55fc286a22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:21.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.504+0000 7fe94f7fe700 1 --2- 172.21.15.2:0/716221113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc286997e0 0x55fc286a22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.504+0000 7fe94f7fe700 1 -- 172.21.15.2:0/716221113 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9501a6050 con 0x55fc28698ec0 2023-09-13T04:41:21.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.504+0000 7fe94f7fe700 1 --2- 172.21.15.2:0/716221113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc28698ec0 0x55fc286a50d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe9400caae0 tx=0x7fe940022c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69e74fd47ab885cd server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:21.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.504+0000 7fe92effd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:41:21.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.505+0000 7fe92effd700 1 -- 172.21.15.2:0/716221113 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94009a3b0 con 0x55fc28698ec0 2023-09-13T04:41:21.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.505+0000 7fe92effd700 1 -- 172.21.15.2:0/716221113 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fe94009b150 con 0x55fc28698ec0 2023-09-13T04:41:21.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.505+0000 7fe9694e7b80 1 -- 172.21.15.2:0/716221113 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc285db970 con 0x55fc28698ec0 2023-09-13T04:41:21.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.505+0000 7fe92effd700 1 -- 172.21.15.2:0/716221113 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe940022ad0 con 0x55fc28698ec0 2023-09-13T04:41:21.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.505+0000 7fe9694e7b80 10 client.25783 fetch_fsmap learned FSMap version 23 2023-09-13T04:41:21.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.505+0000 7fe9694e7b80 1 -- 172.21.15.2:0/716221113 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55fc285fa7a0 con 0x55fc28698ec0 2023-09-13T04:41:21.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.506+0000 7fe92effd700 1 -- 172.21.15.2:0/716221113 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fc285db970 con 0x55fc28698ec0 2023-09-13T04:41:21.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.507+0000 7fe92effd700 1 -- 172.21.15.2:0/716221113 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe94009acb0 con 0x55fc28698ec0 2023-09-13T04:41:21.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.507+0000 7fe9694e7b80 10 client.25783 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:41:21.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.507+0000 7fe9694e7b80 10 client.25783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:41:21.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.507+0000 7fe9694e7b80 20 client.25783 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:21.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.507+0000 7fe9694e7b80 1 --2- 172.21.15.2:0/716221113 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55fc286b22e0 0x55fc286b4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:21.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.507+0000 7fe9694e7b80 4 client.25783 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:41:21.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.507+0000 7fe9694e7b80 1 -- 172.21.15.2:0/716221113 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc2859c050 con 0x55fc286b22e0 2023-09-13T04:41:21.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.507+0000 7fe94effd700 1 --2- 172.21.15.2:0/716221113 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55fc286b22e0 0x55fc286b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:21.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.508+0000 7fe94effd700 1 --2- 172.21.15.2:0/716221113 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55fc286b22e0 0x55fc286b4700 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b35812ebe975fe server_cookie=305600947b28dd4e in_seq=0 out_seq=0 2023-09-13T04:41:21.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.508+0000 7fe92effd700 10 client.25783 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:21.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.509+0000 7fe92effd700 1 -- 172.21.15.2:0/716221113 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fc2859c050 con 0x55fc286b22e0 2023-09-13T04:41:21.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.509+0000 7fe92effd700 10 client.25783 handle_command_reply: tid=0 2023-09-13T04:41:21.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.509+0000 7fe9694e7b80 1 client.25783 shutdown 2023-09-13T04:41:21.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.509+0000 7fe9694e7b80 1 -- 172.21.15.2:0/716221113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc28698ec0 msgr2=0x55fc286a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:21.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.509+0000 7fe9694e7b80 1 --2- 172.21.15.2:0/716221113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc28698ec0 0x55fc286a50d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe9400caae0 tx=0x7fe940022c60 comp rx=0 tx=0).stop 2023-09-13T04:41:21.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.509+0000 7fe9694e7b80 1 -- 172.21.15.2:0/716221113 shutdown_connections 2023-09-13T04:41:21.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.509+0000 7fe9694e7b80 1 --2- 172.21.15.2:0/716221113 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55fc286b22e0 0x55fc286b4700 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.509+0000 7fe9694e7b80 1 --2- 172.21.15.2:0/716221113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc2869d930 0x55fc286a28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.509+0000 7fe9694e7b80 1 --2- 172.21.15.2:0/716221113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc28698ec0 0x55fc286a50d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.509+0000 7fe9694e7b80 1 --2- 172.21.15.2:0/716221113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc286997e0 0x55fc286a22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.509+0000 7fe9694e7b80 1 -- 172.21.15.2:0/716221113 >> 172.21.15.2:0/716221113 conn(0x55fc285eec80 msgr2=0x55fc2868ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:21.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.509+0000 7fe92effd700 0 client.25783 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:21.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.510+0000 7fe9694e7b80 1 -- 172.21.15.2:0/716221113 shutdown_connections 2023-09-13T04:41:21.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.510+0000 7fe9694e7b80 1 -- 172.21.15.2:0/716221113 wait complete. 2023-09-13T04:41:21.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.510+0000 7fe9694e7b80 20 client.25783 trim_cache size 0 max 16384 2023-09-13T04:41:21.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.544+0000 7fe9694e7b80 1 Processor -- start 2023-09-13T04:41:21.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.544+0000 7fe9694e7b80 1 -- start start 2023-09-13T04:41:21.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.545+0000 7fe9694e7b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc28698ec0 0x55fc286a1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:21.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.545+0000 7fe9694e7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc286997e0 0x55fc286a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:21.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.545+0000 7fe9694e7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc2869d930 0x55fc286a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:21.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.545+0000 7fe9694e7b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc286af0b0 con 0x55fc28698ec0 2023-09-13T04:41:21.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.545+0000 7fe9694e7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc28685ef0 con 0x55fc286997e0 2023-09-13T04:41:21.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.545+0000 7fe9694e7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc286a32a0 con 0x55fc2869d930 2023-09-13T04:41:21.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.545+0000 7fe94f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc28698ec0 0x55fc286a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:21.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.545+0000 7fe94f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc28698ec0 0x55fc286a1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45116/0 (socket says 172.21.15.2:45116) 2023-09-13T04:41:21.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.545+0000 7fe94ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc2869d930 0x55fc286a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:21.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.545+0000 7fe94f7fe700 1 -- 172.21.15.2:0/3615814013 learned_addr learned my addr 172.21.15.2:0/3615814013 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:21.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.545+0000 7fe94effd700 1 --2- 172.21.15.2:0/3615814013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc286997e0 0x55fc286a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:21.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.546+0000 7fe94effd700 1 -- 172.21.15.2:0/3615814013 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc2869d930 msgr2=0x55fc286a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:21.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.546+0000 7fe94effd700 1 --2- 172.21.15.2:0/3615814013 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc2869d930 0x55fc286a1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.546+0000 7fe94effd700 1 -- 172.21.15.2:0/3615814013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc28698ec0 msgr2=0x55fc286a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:21.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.546+0000 7fe94effd700 1 --2- 172.21.15.2:0/3615814013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc28698ec0 0x55fc286a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.546+0000 7fe94effd700 1 -- 172.21.15.2:0/3615814013 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc2840ede0 con 0x55fc286997e0 2023-09-13T04:41:21.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.546+0000 7fe94ffff700 1 --2- 172.21.15.2:0/3615814013 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc2869d930 0x55fc286a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:21.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.546+0000 7fe94f7fe700 1 --2- 172.21.15.2:0/3615814013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc28698ec0 0x55fc286a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:21.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.546+0000 7fe94effd700 1 --2- 172.21.15.2:0/3615814013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc286997e0 0x55fc286a4c00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55fc286b4fa0 tx=0x7fe9440057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0d1f3a1ec5ec355 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:21.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.547+0000 7fe9367fc700 1 -- 172.21.15.2:0/3615814013 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe944017040 con 0x55fc286997e0 2023-09-13T04:41:21.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.547+0000 7fe9367fc700 1 -- 172.21.15.2:0/3615814013 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe944009ed0 con 0x55fc286997e0 2023-09-13T04:41:21.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.547+0000 7fe9367fc700 1 -- 172.21.15.2:0/3615814013 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe944010430 con 0x55fc286997e0 2023-09-13T04:41:21.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.547+0000 7fe9694e7b80 1 -- 172.21.15.2:0/3615814013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc286997e0 msgr2=0x55fc286a4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:21.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.547+0000 7fe9694e7b80 1 --2- 172.21.15.2:0/3615814013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc286997e0 0x55fc286a4c00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55fc286b4fa0 tx=0x7fe9440057f0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.548+0000 7fe9694e7b80 1 -- 172.21.15.2:0/3615814013 shutdown_connections 2023-09-13T04:41:21.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.548+0000 7fe9694e7b80 1 --2- 172.21.15.2:0/3615814013 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc2869d930 0x55fc286a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.548+0000 7fe9694e7b80 1 --2- 172.21.15.2:0/3615814013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc28698ec0 0x55fc286a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.548+0000 7fe9694e7b80 1 --2- 172.21.15.2:0/3615814013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc286997e0 0x55fc286a4c00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.548+0000 7fe9694e7b80 1 -- 172.21.15.2:0/3615814013 >> 172.21.15.2:0/3615814013 conn(0x55fc285eec80 msgr2=0x55fc2869b990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:21.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.548+0000 7fe9694e7b80 1 -- 172.21.15.2:0/3615814013 shutdown_connections 2023-09-13T04:41:21.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.548+0000 7fe9694e7b80 1 -- 172.21.15.2:0/3615814013 wait complete. 2023-09-13T04:41:21.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.548+0000 7fe9694e7b80 1 -- start start 2023-09-13T04:41:21.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.549+0000 7fe9694e7b80 1 Processor -- start 2023-09-13T04:41:21.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.549+0000 7fe9694e7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc28698ec0 0x55fc286a91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:21.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.549+0000 7fe9694e7b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc286997e0 0x55fc286a9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:21.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.549+0000 7fe9694e7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc2869d930 0x55fc286a7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:21.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.549+0000 7fe9694e7b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc286a7da0 con 0x55fc286997e0 2023-09-13T04:41:21.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.549+0000 7fe9694e7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc286a7f10 con 0x55fc28698ec0 2023-09-13T04:41:21.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.549+0000 7fe9694e7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc286a80d0 con 0x55fc2869d930 2023-09-13T04:41:21.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.550+0000 7fe9694e7b80 10 client.0 _ll_register_callbacks cb 0x55fc2857e680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:21.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.550+0000 7fe94f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc28698ec0 0x55fc286a91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:21.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.550+0000 7fe94f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc28698ec0 0x55fc286a91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:44300/0 (socket says 172.21.15.2:44300) 2023-09-13T04:41:21.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.550+0000 7fe94f7fe700 1 -- 172.21.15.2:0/3724011184 learned_addr learned my addr 172.21.15.2:0/3724011184 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:21.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.550+0000 7fe94effd700 1 --2- 172.21.15.2:0/3724011184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc286997e0 0x55fc286a9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:21.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.550+0000 7fe94ffff700 1 --2- 172.21.15.2:0/3724011184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc2869d930 0x55fc286a7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:21.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.550+0000 7fe94f7fe700 1 -- 172.21.15.2:0/3724011184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc2869d930 msgr2=0x55fc286a7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:21.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.550+0000 7fe94f7fe700 1 --2- 172.21.15.2:0/3724011184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc2869d930 0x55fc286a7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.550+0000 7fe94f7fe700 1 -- 172.21.15.2:0/3724011184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc286997e0 msgr2=0x55fc286a9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:21.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.550+0000 7fe94f7fe700 1 --2- 172.21.15.2:0/3724011184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc286997e0 0x55fc286a9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.551+0000 7fe94f7fe700 1 -- 172.21.15.2:0/3724011184 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc285fa7a0 con 0x55fc28698ec0 2023-09-13T04:41:21.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.551+0000 7fe94f7fe700 1 --2- 172.21.15.2:0/3724011184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc28698ec0 0x55fc286a91b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe940098da0 tx=0x7fe9400cdc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4d64e33cffab240 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:21.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.551+0000 7fe92effd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:41:21.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.551+0000 7fe92effd700 1 -- 172.21.15.2:0/3724011184 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940022ad0 con 0x55fc28698ec0 2023-09-13T04:41:21.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.552+0000 7fe9694e7b80 1 -- 172.21.15.2:0/3724011184 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc28683670 con 0x55fc28698ec0 2023-09-13T04:41:21.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.552+0000 7fe92effd700 1 -- 172.21.15.2:0/3724011184 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fe94009b150 con 0x55fc28698ec0 2023-09-13T04:41:21.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.552+0000 7fe92effd700 1 -- 172.21.15.2:0/3724011184 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9400d36f0 con 0x55fc28698ec0 2023-09-13T04:41:21.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.552+0000 7fe9694e7b80 10 client.25784 fetch_fsmap learned FSMap version 23 2023-09-13T04:41:21.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.552+0000 7fe9694e7b80 1 -- 172.21.15.2:0/3724011184 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fe948004180 con 0x55fc28698ec0 2023-09-13T04:41:21.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.553+0000 7fe92effd700 1 -- 172.21.15.2:0/3724011184 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe94007d080 con 0x55fc28698ec0 2023-09-13T04:41:21.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.553+0000 7fe92effd700 1 -- 172.21.15.2:0/3724011184 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe9400a25a0 con 0x55fc28698ec0 2023-09-13T04:41:21.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.553+0000 7fe9694e7b80 10 client.25784 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:41:21.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.553+0000 7fe9694e7b80 10 client.25784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:41:21.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.553+0000 7fe9694e7b80 20 client.25784 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:21.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.553+0000 7fe9694e7b80 1 --2- 172.21.15.2:0/3724011184 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55fc28706f00 0x55fc287092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:21.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.553+0000 7fe9694e7b80 4 client.25784 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-09-13T04:41:21.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.553+0000 7fe9694e7b80 1 -- 172.21.15.2:0/3724011184 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fc2842ff70 con 0x55fc28706f00 2023-09-13T04:41:21.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.553+0000 7fe94effd700 1 --2- 172.21.15.2:0/3724011184 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55fc28706f00 0x55fc287092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:21.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.554+0000 7fe94effd700 1 --2- 172.21.15.2:0/3724011184 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55fc28706f00 0x55fc287092f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e098d2499fcc1e43 server_cookie=60af4d612746ac75 in_seq=0 out_seq=0 2023-09-13T04:41:21.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.554+0000 7fe92effd700 10 client.25784 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:21.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.554+0000 7fe92effd700 1 -- 172.21.15.2:0/3724011184 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fc2842ff70 con 0x55fc28706f00 2023-09-13T04:41:21.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.554+0000 7fe92effd700 10 client.25784 handle_command_reply: tid=0 2023-09-13T04:41:21.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.554+0000 7fe9694e7b80 1 client.25784 shutdown 2023-09-13T04:41:21.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.554+0000 7fe9694e7b80 1 -- 172.21.15.2:0/3724011184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc28698ec0 msgr2=0x55fc286a91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:21.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.554+0000 7fe9694e7b80 1 --2- 172.21.15.2:0/3724011184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc28698ec0 0x55fc286a91b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe940098da0 tx=0x7fe9400cdc20 comp rx=0 tx=0).stop 2023-09-13T04:41:21.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.555+0000 7fe9694e7b80 1 -- 172.21.15.2:0/3724011184 shutdown_connections 2023-09-13T04:41:21.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.555+0000 7fe9694e7b80 1 --2- 172.21.15.2:0/3724011184 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55fc28706f00 0x55fc287092f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.555+0000 7fe9694e7b80 1 --2- 172.21.15.2:0/3724011184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc2869d930 0x55fc286a7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.555+0000 7fe9694e7b80 1 --2- 172.21.15.2:0/3724011184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc28698ec0 0x55fc286a91b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.555+0000 7fe9694e7b80 1 --2- 172.21.15.2:0/3724011184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc286997e0 0x55fc286a9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.555+0000 7fe9694e7b80 1 -- 172.21.15.2:0/3724011184 >> 172.21.15.2:0/3724011184 conn(0x55fc285eec80 msgr2=0x55fc286848e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:21.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.555+0000 7fe92effd700 0 client.25784 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:21.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.555+0000 7fe9694e7b80 1 -- 172.21.15.2:0/3724011184 shutdown_connections 2023-09-13T04:41:21.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.555+0000 7fe9694e7b80 1 -- 172.21.15.2:0/3724011184 wait complete. 2023-09-13T04:41:21.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.555+0000 7fe9694e7b80 20 client.25784 trim_cache size 0 max 16384 2023-09-13T04:41:21.568 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:41:21.568 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2023-09-13T04:41:21.569 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2023-09-13T04:41:21.569 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:41:21.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.556+0000 7fe955bff700 1 -- 172.21.15.2:0/2421770713 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe9380a0950 msgr2=0x7fe9380a2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:21.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.556+0000 7fe955bff700 1 --2- 172.21.15.2:0/2421770713 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe9380a0950 0x7fe9380a2e10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe9480093f0 tx=0x7fe948009380 comp rx=0 tx=0).stop 2023-09-13T04:41:21.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.556+0000 7fe955bff700 1 -- 172.21.15.2:0/2421770713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe950100ec0 msgr2=0x7fe950198fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:21.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.556+0000 7fe955bff700 1 --2- 172.21.15.2:0/2421770713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe950100ec0 0x7fe950198fe0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe940005910 tx=0x7fe940005740 comp rx=0 tx=0).stop 2023-09-13T04:41:21.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.556+0000 7fe955bff700 1 -- 172.21.15.2:0/2421770713 shutdown_connections 2023-09-13T04:41:21.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.556+0000 7fe955bff700 1 --2- 172.21.15.2:0/2421770713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe95019d980 0x7fe95019de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.556+0000 7fe955bff700 1 --2- 172.21.15.2:0/2421770713 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe9380a0950 0x7fe9380a2e10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.556+0000 7fe955bff700 1 --2- 172.21.15.2:0/2421770713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe950100ec0 0x7fe950198fe0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.556+0000 7fe955bff700 1 --2- 172.21.15.2:0/2421770713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe950105fe0 0x7fe950199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.556+0000 7fe955bff700 1 -- 172.21.15.2:0/2421770713 >> 172.21.15.2:0/2421770713 conn(0x7fe9500fc760 msgr2=0x7fe95010ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:21.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.557+0000 7fe955bff700 1 -- 172.21.15.2:0/2421770713 shutdown_connections 2023-09-13T04:41:21.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.557+0000 7fe955bff700 1 -- 172.21.15.2:0/2421770713 wait complete. 2023-09-13T04:41:21.586 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-13T04:41:21.586 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:41:21.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.928+0000 7f07f759e700 1 -- 172.21.15.2:0/2913029486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07f8067e40 msgr2=0x7f07f80682c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:21.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.928+0000 7f07f759e700 1 --2- 172.21.15.2:0/2913029486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07f8067e40 0x7f07f80682c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0099c0 tx=0x7f07ec009cd0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.928+0000 7f07f759e700 1 -- 172.21.15.2:0/2913029486 shutdown_connections 2023-09-13T04:41:21.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.928+0000 7f07f759e700 1 --2- 172.21.15.2:0/2913029486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07f8068800 0x7f07f810d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.928+0000 7f07f759e700 1 --2- 172.21.15.2:0/2913029486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07f8106c10 0x7f07f8106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.928+0000 7f07f759e700 1 --2- 172.21.15.2:0/2913029486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07f8067e40 0x7f07f80682c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.928+0000 7f07f759e700 1 -- 172.21.15.2:0/2913029486 >> 172.21.15.2:0/2913029486 conn(0x7f07f8076780 msgr2=0x7f07f8076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:21.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.929+0000 7f07f759e700 1 -- 172.21.15.2:0/2913029486 shutdown_connections 2023-09-13T04:41:21.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.929+0000 7f07f759e700 1 -- 172.21.15.2:0/2913029486 wait complete. 2023-09-13T04:41:21.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.930+0000 7f07f759e700 1 Processor -- start 2023-09-13T04:41:21.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.931+0000 7f07f759e700 1 -- start start 2023-09-13T04:41:21.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.931+0000 7f07f759e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07f8067e40 0x7f07f80ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:21.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.931+0000 7f07f759e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07f8068800 0x7f07f8100210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:21.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.931+0000 7f07f759e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07f8106c10 0x7f07f8100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:21.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.931+0000 7f07f759e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f81a4e20 con 0x7f07f8106c10 2023-09-13T04:41:21.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.931+0000 7f07f759e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f81a4f90 con 0x7f07f8068800 2023-09-13T04:41:21.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.931+0000 7f07f759e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f81a5150 con 0x7f07f8067e40 2023-09-13T04:41:21.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.932+0000 7f07f5d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07f8068800 0x7f07f8100210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:21.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.932+0000 7f07f6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07f8106c10 0x7f07f8100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:21.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.932+0000 7f07f5d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07f8068800 0x7f07f8100210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:44322/0 (socket says 172.21.15.2:44322) 2023-09-13T04:41:21.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.932+0000 7f07f5d9b700 1 -- 172.21.15.2:0/2813720587 learned_addr learned my addr 172.21.15.2:0/2813720587 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:21.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.932+0000 7f07f659c700 1 --2- 172.21.15.2:0/2813720587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07f8067e40 0x7f07f80ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:21.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.932+0000 7f07f5d9b700 1 -- 172.21.15.2:0/2813720587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07f8067e40 msgr2=0x7f07f80ffcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:21.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.932+0000 7f07f5d9b700 1 --2- 172.21.15.2:0/2813720587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07f8067e40 0x7f07f80ffcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.932+0000 7f07f5d9b700 1 -- 172.21.15.2:0/2813720587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07f8106c10 msgr2=0x7f07f8100750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:21.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.932+0000 7f07f5d9b700 1 --2- 172.21.15.2:0/2813720587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07f8106c10 0x7f07f8100750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:21.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.932+0000 7f07f5d9b700 1 -- 172.21.15.2:0/2813720587 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e8009520 con 0x7f07f8068800 2023-09-13T04:41:21.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.933+0000 7f07f5d9b700 1 --2- 172.21.15.2:0/2813720587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07f8068800 0x7f07f8100210 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f07ec000c00 tx=0x7f07ec00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:21.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.933+0000 7f07e77fe700 1 -- 172.21.15.2:0/2813720587 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ec01d070 con 0x7f07f8068800 2023-09-13T04:41:21.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.933+0000 7f07e77fe700 1 -- 172.21.15.2:0/2813720587 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07ec005050 con 0x7f07f8068800 2023-09-13T04:41:21.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.933+0000 7f07e77fe700 1 -- 172.21.15.2:0/2813720587 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ec004b70 con 0x7f07f8068800 2023-09-13T04:41:21.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.933+0000 7f07f759e700 1 -- 172.21.15.2:0/2813720587 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07ec0095f0 con 0x7f07f8068800 2023-09-13T04:41:21.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.933+0000 7f07f759e700 1 -- 172.21.15.2:0/2813720587 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f81a5610 con 0x7f07f8068800 2023-09-13T04:41:21.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.935+0000 7f07f759e700 1 -- 172.21.15.2:0/2813720587 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f80679f0 con 0x7f07f8068800 2023-09-13T04:41:21.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.939+0000 7f07e77fe700 1 -- 172.21.15.2:0/2813720587 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 24) v1 ==== 135491+0+0 (secure 0 0 0) 0x7f07ec0051c0 con 0x7f07f8068800 2023-09-13T04:41:21.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.941+0000 7f07e77fe700 1 --2- 172.21.15.2:0/2813720587 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f07e00a09c0 0x7f07e00a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:21.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.941+0000 7f07e77fe700 1 -- 172.21.15.2:0/2813720587 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f07ec0cd7c0 con 0x7f07f8068800 2023-09-13T04:41:21.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.942+0000 7f07f659c700 1 --2- 172.21.15.2:0/2813720587 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f07e00a09c0 0x7f07e00a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:21.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.942+0000 7f07e77fe700 1 -- 172.21.15.2:0/2813720587 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f07ec0cdbc0 con 0x7f07f8068800 2023-09-13T04:41:21.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:21.943+0000 7f07f659c700 1 --2- 172.21.15.2:0/2813720587 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f07e00a09c0 0x7f07e00a2e80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f07e8009d80 tx=0x7f07e8009260 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:22.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:21 smithi149 ceph-mon[133315]: pgmap v30: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 853 B/s wr, 6 op/s 2023-09-13T04:41:22.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:21 smithi156 ceph-mon[133675]: pgmap v30: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 853 B/s wr, 6 op/s 2023-09-13T04:41:22.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:21 smithi002 ceph-mon[97651]: pgmap v30: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 853 B/s wr, 6 op/s 2023-09-13T04:41:22.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.451+0000 7f07f759e700 1 -- 172.21.15.2:0/2813720587 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07f8101160 con 0x7f07f8068800 2023-09-13T04:41:22.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.452+0000 7f07e77fe700 1 -- 172.21.15.2:0/2813720587 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11303 (secure 0 0 0) 0x7f07ec096a80 con 0x7f07f8068800 2023-09-13T04:41:22.453 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:41:22.453 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2469307307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2469307307},{"type":"v1","addr":"172.21.15.2:6839","nonce":2469307307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T04:37:59.110701+0000","modified":"2023-09-13T04:38:26.610260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24554,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/846579077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":846579077},{"type":"v1","addr":"172.21.15.156:6835","nonce":846579077}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1896833690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1896833690},{"type":"v1","addr":"172.21.15.149:6837","nonce":1896833690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/524563090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":524563090},{"type":"v1","addr":"172.21.15.156:6837","nonce":524563090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6839/1785833227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1785833227},{"type":"v1","addr":"172.21.15.156:6839","nonce":1785833227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.2:6841/482545391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":482545391},{"type":"v1","addr":"172.21.15.2:6841","nonce":482545391}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/389167841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":389167841},{"type":"v1","addr":"172.21.15.149:6839","nonce":389167841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T04:41:22.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.455+0000 7f07f759e700 1 -- 172.21.15.2:0/2813720587 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f07e00a09c0 msgr2=0x7f07e00a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:22.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.455+0000 7f07f759e700 1 --2- 172.21.15.2:0/2813720587 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f07e00a09c0 0x7f07e00a2e80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f07e8009d80 tx=0x7f07e8009260 comp rx=0 tx=0).stop 2023-09-13T04:41:22.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.455+0000 7f07f759e700 1 -- 172.21.15.2:0/2813720587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07f8068800 msgr2=0x7f07f8100210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:22.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.455+0000 7f07f759e700 1 --2- 172.21.15.2:0/2813720587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07f8068800 0x7f07f8100210 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f07ec000c00 tx=0x7f07ec00bfd0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.456+0000 7f07f759e700 1 -- 172.21.15.2:0/2813720587 shutdown_connections 2023-09-13T04:41:22.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.456+0000 7f07f759e700 1 --2- 172.21.15.2:0/2813720587 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f07e00a09c0 0x7f07e00a2e80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.456+0000 7f07f759e700 1 --2- 172.21.15.2:0/2813720587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07f8106c10 0x7f07f8100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.456+0000 7f07f759e700 1 --2- 172.21.15.2:0/2813720587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07f8067e40 0x7f07f80ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.456+0000 7f07f759e700 1 --2- 172.21.15.2:0/2813720587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07f8068800 0x7f07f8100210 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.456+0000 7f07f759e700 1 -- 172.21.15.2:0/2813720587 >> 172.21.15.2:0/2813720587 conn(0x7f07f8076780 msgr2=0x7f07f8104680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:22.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.456+0000 7f07f759e700 1 -- 172.21.15.2:0/2813720587 shutdown_connections 2023-09-13T04:41:22.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.456+0000 7f07f759e700 1 -- 172.21.15.2:0/2813720587 wait complete. 2023-09-13T04:41:22.459 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 23 2023-09-13T04:41:22.488 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-13T04:41:22.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.840+0000 7f8d8299d700 1 -- 172.21.15.2:0/1672401962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d7c100ab0 msgr2=0x7f8d7c100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:22.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.840+0000 7f8d8299d700 1 --2- 172.21.15.2:0/1672401962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d7c100ab0 0x7f8d7c100f30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009990 tx=0x7f8d6c009ca0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.841+0000 7f8d8299d700 1 -- 172.21.15.2:0/1672401962 shutdown_connections 2023-09-13T04:41:22.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.841+0000 7f8d8299d700 1 --2- 172.21.15.2:0/1672401962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d7c101470 0x7f8d7c10d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.841+0000 7f8d8299d700 1 --2- 172.21.15.2:0/1672401962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d7c106ac0 0x7f8d7c106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.841+0000 7f8d8299d700 1 --2- 172.21.15.2:0/1672401962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d7c100ab0 0x7f8d7c100f30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.841+0000 7f8d8299d700 1 -- 172.21.15.2:0/1672401962 >> 172.21.15.2:0/1672401962 conn(0x7f8d7c0fc560 msgr2=0x7f8d7c0fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:22.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.841+0000 7f8d8299d700 1 -- 172.21.15.2:0/1672401962 shutdown_connections 2023-09-13T04:41:22.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.841+0000 7f8d8299d700 1 -- 172.21.15.2:0/1672401962 wait complete. 2023-09-13T04:41:22.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.842+0000 7f8d8299d700 1 Processor -- start 2023-09-13T04:41:22.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.843+0000 7f8d8299d700 1 -- start start 2023-09-13T04:41:22.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.843+0000 7f8d8299d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d7c100ab0 0x7f8d7c072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:22.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.843+0000 7f8d8299d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d7c101470 0x7f8d7c0734f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:22.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.844+0000 7f8d8299d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d7c106ac0 0x7f8d7c1a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:22.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.844+0000 7f8d8299d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c073d20 con 0x7f8d7c100ab0 2023-09-13T04:41:22.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.844+0000 7f8d8299d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c1a5180 con 0x7f8d7c106ac0 2023-09-13T04:41:22.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.844+0000 7f8d8299d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c1a52c0 con 0x7f8d7c101470 2023-09-13T04:41:22.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.844+0000 7f8d8199b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d7c100ab0 0x7f8d7c072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:22.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.844+0000 7f8d8119a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d7c101470 0x7f8d7c0734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:22.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.844+0000 7f8d8199b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d7c100ab0 0x7f8d7c072fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45158/0 (socket says 172.21.15.2:45158) 2023-09-13T04:41:22.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.844+0000 7f8d8199b700 1 -- 172.21.15.2:0/5445780 learned_addr learned my addr 172.21.15.2:0/5445780 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:22.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.844+0000 7f8d8219c700 1 --2- 172.21.15.2:0/5445780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d7c106ac0 0x7f8d7c1a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:22.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.844+0000 7f8d8119a700 1 -- 172.21.15.2:0/5445780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d7c106ac0 msgr2=0x7f8d7c1a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:22.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.844+0000 7f8d8119a700 1 --2- 172.21.15.2:0/5445780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d7c106ac0 0x7f8d7c1a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.844+0000 7f8d8119a700 1 -- 172.21.15.2:0/5445780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d7c100ab0 msgr2=0x7f8d7c072fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:22.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.844+0000 7f8d8119a700 1 --2- 172.21.15.2:0/5445780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d7c100ab0 0x7f8d7c072fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.844+0000 7f8d8119a700 1 -- 172.21.15.2:0/5445780 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d78009520 con 0x7f8d7c101470 2023-09-13T04:41:22.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.845+0000 7f8d8119a700 1 --2- 172.21.15.2:0/5445780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d7c101470 0x7f8d7c0734f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c000c00 tx=0x7f8d6c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:22.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.845+0000 7f8d72ffd700 1 -- 172.21.15.2:0/5445780 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c01d070 con 0x7f8d7c101470 2023-09-13T04:41:22.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.845+0000 7f8d8299d700 1 -- 172.21.15.2:0/5445780 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d6c0095f0 con 0x7f8d7c101470 2023-09-13T04:41:22.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.845+0000 7f8d72ffd700 1 -- 172.21.15.2:0/5445780 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d6c021410 con 0x7f8d7c101470 2023-09-13T04:41:22.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.845+0000 7f8d72ffd700 1 -- 172.21.15.2:0/5445780 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c017650 con 0x7f8d7c101470 2023-09-13T04:41:22.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.845+0000 7f8d8299d700 1 -- 172.21.15.2:0/5445780 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d7c1a5860 con 0x7f8d7c101470 2023-09-13T04:41:22.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.851+0000 7f8d72ffd700 1 -- 172.21.15.2:0/5445780 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 24) v1 ==== 135491+0+0 (secure 0 0 0) 0x7f8d6c021580 con 0x7f8d7c101470 2023-09-13T04:41:22.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.852+0000 7f8d974fdb80 1 Processor -- start 2023-09-13T04:41:22.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.853+0000 7f8d974fdb80 1 -- start start 2023-09-13T04:41:22.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.853+0000 7f8d72ffd700 1 --2- 172.21.15.2:0/5445780 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f8d680a0970 0x7f8d680a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:22.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.853+0000 7f8d974fdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562020a0cec0 0x562020a0d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:22.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.853+0000 7f8d974fdb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562020a0d7e0 0x562020a115d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:22.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.853+0000 7f8d72ffd700 1 -- 172.21.15.2:0/5445780 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f8d6c0cd900 con 0x7f8d7c101470 2023-09-13T04:41:22.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.853+0000 7f8d974fdb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562020a11b10 0x562020a11f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:22.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.853+0000 7f8d974fdb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202095d780 con 0x562020a0d7e0 2023-09-13T04:41:22.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.853+0000 7f8d974fdb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202095b0a0 con 0x562020a11b10 2023-09-13T04:41:22.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.853+0000 7f8d974fdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562020956990 con 0x562020a0cec0 2023-09-13T04:41:22.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.854+0000 7f8d8119a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562020a0d7e0 0x562020a115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:22.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.854+0000 7f8d8119a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562020a0d7e0 0x562020a115d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45168/0 (socket says 172.21.15.2:45168) 2023-09-13T04:41:22.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.854+0000 7f8d8119a700 1 -- 172.21.15.2:0/160549333 learned_addr learned my addr 172.21.15.2:0/160549333 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:22.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.854+0000 7f8d8199b700 1 --2- 172.21.15.2:0/160549333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562020a0cec0 0x562020a0d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:22.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.855+0000 7f8d8119a700 1 -- 172.21.15.2:0/160549333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562020a0cec0 msgr2=0x562020a0d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:22.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.855+0000 7f8d8119a700 1 --2- 172.21.15.2:0/160549333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562020a0cec0 0x562020a0d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.855+0000 7f8d8119a700 1 -- 172.21.15.2:0/160549333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562020a11b10 msgr2=0x562020a11f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:41:22.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.855+0000 7f8d8119a700 1 --2- 172.21.15.2:0/160549333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562020a11b10 0x562020a11f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.855+0000 7f8d8119a700 1 -- 172.21.15.2:0/160549333 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620207d6630 con 0x562020a0d7e0 2023-09-13T04:41:22.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.855+0000 7f8d8219c700 1 --2- 172.21.15.2:0/5445780 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f8d680a0970 0x7f8d680a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:22.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.855+0000 7f8d8119a700 1 --2- 172.21.15.2:0/160549333 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562020a0d7e0 0x562020a115d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c09a810 tx=0x7f8d6c09a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2495e0794805a6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:22.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.855+0000 7f8d67fff700 1 -- 172.21.15.2:0/160549333 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c09a880 con 0x562020a0d7e0 2023-09-13T04:41:22.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.855+0000 7f8d67fff700 1 -- 172.21.15.2:0/160549333 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d6c09e6b0 con 0x562020a0d7e0 2023-09-13T04:41:22.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.855+0000 7f8d67fff700 1 -- 172.21.15.2:0/160549333 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c099fd0 con 0x562020a0d7e0 2023-09-13T04:41:22.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.856+0000 7f8d974fdb80 1 -- 172.21.15.2:0/160549333 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562020a0d7e0 msgr2=0x562020a115d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:22.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.856+0000 7f8d8219c700 1 --2- 172.21.15.2:0/5445780 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f8d680a0970 0x7f8d680a2e30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0750c0 tx=0x7f8d74009100 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:22.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.856+0000 7f8d974fdb80 1 --2- 172.21.15.2:0/160549333 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562020a0d7e0 0x562020a115d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c09a810 tx=0x7f8d6c09a780 comp rx=0 tx=0).stop 2023-09-13T04:41:22.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.857+0000 7f8d974fdb80 1 -- 172.21.15.2:0/160549333 shutdown_connections 2023-09-13T04:41:22.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.857+0000 7f8d974fdb80 1 --2- 172.21.15.2:0/160549333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562020a11b10 0x562020a11f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.857+0000 7f8d974fdb80 1 --2- 172.21.15.2:0/160549333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562020a0cec0 0x562020a0d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.857+0000 7f8d974fdb80 1 --2- 172.21.15.2:0/160549333 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562020a0d7e0 0x562020a115d0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.857+0000 7f8d974fdb80 1 -- 172.21.15.2:0/160549333 >> 172.21.15.2:0/160549333 conn(0x562020962c80 msgr2=0x5620209ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:22.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.857+0000 7f8d974fdb80 1 -- 172.21.15.2:0/160549333 shutdown_connections 2023-09-13T04:41:22.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.857+0000 7f8d974fdb80 1 -- 172.21.15.2:0/160549333 wait complete. 2023-09-13T04:41:22.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.858+0000 7f8d974fdb80 1 -- start start 2023-09-13T04:41:22.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.858+0000 7f8d974fdb80 1 Processor -- start 2023-09-13T04:41:22.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.859+0000 7f8d974fdb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562020a0cec0 0x562020a19220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:22.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.859+0000 7f8d974fdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562020a0d7e0 0x562020a164a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:22.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.859+0000 7f8d974fdb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562020a11b10 0x562020a16ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:22.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.859+0000 7f8d974fdb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620208f4bb0 con 0x562020a0cec0 2023-09-13T04:41:22.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.859+0000 7f8d974fdb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620208f48b0 con 0x562020a11b10 2023-09-13T04:41:22.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.859+0000 7f8d974fdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620207d8920 con 0x562020a0d7e0 2023-09-13T04:41:22.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.859+0000 7f8d974fdb80 10 client.0 _ll_register_callbacks cb 0x5620208f2680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:22.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.859+0000 7f8d8119a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562020a0d7e0 0x562020a164a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:22.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.859+0000 7f8d8119a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562020a0d7e0 0x562020a164a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:37234/0 (socket says 172.21.15.2:37234) 2023-09-13T04:41:22.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.859+0000 7f8d8119a700 1 -- 172.21.15.2:0/2101209757 learned_addr learned my addr 172.21.15.2:0/2101209757 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:22.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.860+0000 7f8d8219c700 1 --2- 172.21.15.2:0/2101209757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562020a11b10 0x562020a16ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:22.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.860+0000 7f8d8119a700 1 -- 172.21.15.2:0/2101209757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562020a11b10 msgr2=0x562020a16ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:22.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.860+0000 7f8d8119a700 1 --2- 172.21.15.2:0/2101209757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562020a11b10 0x562020a16ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.860+0000 7f8d8119a700 1 -- 172.21.15.2:0/2101209757 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562020a0cec0 msgr2=0x562020a19220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:41:22.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.860+0000 7f8d8119a700 1 --2- 172.21.15.2:0/2101209757 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562020a0cec0 0x562020a19220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.860+0000 7f8d8119a700 1 -- 172.21.15.2:0/2101209757 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d7c1a5860 con 0x562020a0d7e0 2023-09-13T04:41:22.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.860+0000 7f8d8119a700 1 --2- 172.21.15.2:0/2101209757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562020a0d7e0 0x562020a164a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0098e0 tx=0x7f8d6c09ad10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=276eecb92933c439 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:22.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.860+0000 7f8d657fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:41:22.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.861+0000 7f8d657fa700 1 -- 172.21.15.2:0/2101209757 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c0b8070 con 0x562020a0d7e0 2023-09-13T04:41:22.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.861+0000 7f8d974fdb80 1 -- 172.21.15.2:0/2101209757 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620209f7670 con 0x562020a0d7e0 2023-09-13T04:41:22.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.861+0000 7f8d657fa700 1 -- 172.21.15.2:0/2101209757 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f8d6c0a2860 con 0x562020a0d7e0 2023-09-13T04:41:22.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.861+0000 7f8d657fa700 1 -- 172.21.15.2:0/2101209757 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d6c09a880 con 0x562020a0d7e0 2023-09-13T04:41:22.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.861+0000 7f8d974fdb80 10 client.25807 fetch_fsmap learned FSMap version 23 2023-09-13T04:41:22.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.861+0000 7f8d974fdb80 1 -- 172.21.15.2:0/2101209757 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56202096e7a0 con 0x562020a0d7e0 2023-09-13T04:41:22.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.862+0000 7f8d657fa700 1 -- 172.21.15.2:0/2101209757 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5620209f7670 con 0x562020a0d7e0 2023-09-13T04:41:22.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.862+0000 7f8d657fa700 1 -- 172.21.15.2:0/2101209757 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8d6c09e130 con 0x562020a0d7e0 2023-09-13T04:41:22.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.862+0000 7f8d974fdb80 10 client.25807 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:41:22.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.862+0000 7f8d974fdb80 10 client.25807 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-09-13T04:41:22.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.862+0000 7f8d974fdb80 20 client.25807 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:22.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.862+0000 7f8d974fdb80 1 --2- 172.21.15.2:0/2101209757 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x562020a26320 0x562020a28710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:22.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.862+0000 7f8d974fdb80 4 client.25807 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:41:22.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.862+0000 7f8d974fdb80 1 -- 172.21.15.2:0/2101209757 --> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56202094f970 con 0x562020a26320 2023-09-13T04:41:22.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.862+0000 7f8d8199b700 1 --2- 172.21.15.2:0/2101209757 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x562020a26320 0x562020a28710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:22.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.863+0000 7f8d8199b700 1 --2- 172.21.15.2:0/2101209757 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x562020a26320 0x562020a28710 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6279751e9e48255 server_cookie=ce2017c401738e94 in_seq=0 out_seq=0 2023-09-13T04:41:22.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.863+0000 7f8d657fa700 10 client.25807 ms_handle_connect on v2:172.21.15.156:6838/1785833227 2023-09-13T04:41:22.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.864+0000 7f8d657fa700 1 -- 172.21.15.2:0/2101209757 <== mds.2 v2:172.21.15.156:6838/1785833227 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56202094f970 con 0x562020a26320 2023-09-13T04:41:22.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.864+0000 7f8d657fa700 10 client.25807 handle_command_reply: tid=0 2023-09-13T04:41:22.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.864+0000 7f8d974fdb80 1 client.25807 shutdown 2023-09-13T04:41:22.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.864+0000 7f8d974fdb80 1 -- 172.21.15.2:0/2101209757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562020a0d7e0 msgr2=0x562020a164a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:22.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.864+0000 7f8d974fdb80 1 --2- 172.21.15.2:0/2101209757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562020a0d7e0 0x562020a164a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0098e0 tx=0x7f8d6c09ad10 comp rx=0 tx=0).stop 2023-09-13T04:41:22.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.864+0000 7f8d974fdb80 1 -- 172.21.15.2:0/2101209757 shutdown_connections 2023-09-13T04:41:22.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.864+0000 7f8d974fdb80 1 --2- 172.21.15.2:0/2101209757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562020a11b10 0x562020a16ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.864+0000 7f8d974fdb80 1 --2- 172.21.15.2:0/2101209757 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x562020a26320 0x562020a28710 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.864+0000 7f8d974fdb80 1 --2- 172.21.15.2:0/2101209757 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562020a0cec0 0x562020a19220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.864+0000 7f8d974fdb80 1 --2- 172.21.15.2:0/2101209757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562020a0d7e0 0x562020a164a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.864+0000 7f8d974fdb80 1 -- 172.21.15.2:0/2101209757 >> 172.21.15.2:0/2101209757 conn(0x562020962c80 msgr2=0x562020a0bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:22.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.865+0000 7f8d657fa700 0 client.25807 ms_handle_reset on v2:172.21.15.156:6838/1785833227 2023-09-13T04:41:22.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.865+0000 7f8d974fdb80 1 -- 172.21.15.2:0/2101209757 shutdown_connections 2023-09-13T04:41:22.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.865+0000 7f8d974fdb80 1 -- 172.21.15.2:0/2101209757 wait complete. 2023-09-13T04:41:22.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.865+0000 7f8d974fdb80 20 client.25807 trim_cache size 0 max 16384 2023-09-13T04:41:22.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.891+0000 7f8d974fdb80 1 Processor -- start 2023-09-13T04:41:22.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.891+0000 7f8d974fdb80 1 -- start start 2023-09-13T04:41:22.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.892+0000 7f8d974fdb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562020a0cec0 0x562020a1da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:22.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.892+0000 7f8d974fdb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562020a0d7e0 0x562020a28330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:22.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.892+0000 7f8d974fdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562020a11b10 0x562020a18dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:22.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.892+0000 7f8d974fdb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562020a17490 con 0x562020a0cec0 2023-09-13T04:41:22.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.892+0000 7f8d974fdb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562020a232b0 con 0x562020a0d7e0 2023-09-13T04:41:22.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.892+0000 7f8d974fdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562020a009e0 con 0x562020a11b10 2023-09-13T04:41:22.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.892+0000 7f8d8119a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562020a0d7e0 0x562020a28330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:22.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.893+0000 7f8d8119a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562020a0d7e0 0x562020a28330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:44374/0 (socket says 172.21.15.2:44374) 2023-09-13T04:41:22.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.893+0000 7f8d8119a700 1 -- 172.21.15.2:0/284999056 learned_addr learned my addr 172.21.15.2:0/284999056 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:22.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.893+0000 7f8d8219c700 1 --2- 172.21.15.2:0/284999056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562020a11b10 0x562020a18dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:22.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.893+0000 7f8d8199b700 1 --2- 172.21.15.2:0/284999056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562020a0cec0 0x562020a1da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:22.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.893+0000 7f8d8219c700 1 -- 172.21.15.2:0/284999056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562020a0d7e0 msgr2=0x562020a28330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:22.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.893+0000 7f8d8219c700 1 --2- 172.21.15.2:0/284999056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562020a0d7e0 0x562020a28330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.893+0000 7f8d8219c700 1 -- 172.21.15.2:0/284999056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562020a0cec0 msgr2=0x562020a1da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:22.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.893+0000 7f8d8219c700 1 --2- 172.21.15.2:0/284999056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562020a0cec0 0x562020a1da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.893+0000 7f8d8219c700 1 -- 172.21.15.2:0/284999056 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562020782de0 con 0x562020a11b10 2023-09-13T04:41:22.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.894+0000 7f8d8219c700 1 --2- 172.21.15.2:0/284999056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562020a11b10 0x562020a18dc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8d74004040 tx=0x7f8d74013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fcbe68a47eb4ede server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:22.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.894+0000 7f8d70bff700 1 -- 172.21.15.2:0/284999056 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74004db0 con 0x562020a11b10 2023-09-13T04:41:22.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.894+0000 7f8d70bff700 1 -- 172.21.15.2:0/284999056 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d7401a070 con 0x562020a11b10 2023-09-13T04:41:22.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.894+0000 7f8d70bff700 1 -- 172.21.15.2:0/284999056 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74015da0 con 0x562020a11b10 2023-09-13T04:41:22.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.895+0000 7f8d974fdb80 1 -- 172.21.15.2:0/284999056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562020a11b10 msgr2=0x562020a18dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:22.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.895+0000 7f8d974fdb80 1 --2- 172.21.15.2:0/284999056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562020a11b10 0x562020a18dc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8d74004040 tx=0x7f8d74013000 comp rx=0 tx=0).stop 2023-09-13T04:41:22.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.895+0000 7f8d974fdb80 1 -- 172.21.15.2:0/284999056 shutdown_connections 2023-09-13T04:41:22.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.895+0000 7f8d974fdb80 1 --2- 172.21.15.2:0/284999056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562020a11b10 0x562020a18dc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.895+0000 7f8d974fdb80 1 --2- 172.21.15.2:0/284999056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562020a0cec0 0x562020a1da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.895+0000 7f8d974fdb80 1 --2- 172.21.15.2:0/284999056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562020a0d7e0 0x562020a28330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.895+0000 7f8d974fdb80 1 -- 172.21.15.2:0/284999056 >> 172.21.15.2:0/284999056 conn(0x562020962c80 msgr2=0x562020a22b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:22.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.895+0000 7f8d974fdb80 1 -- 172.21.15.2:0/284999056 shutdown_connections 2023-09-13T04:41:22.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.895+0000 7f8d974fdb80 1 -- 172.21.15.2:0/284999056 wait complete. 2023-09-13T04:41:22.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.896+0000 7f8d974fdb80 1 -- start start 2023-09-13T04:41:22.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.896+0000 7f8d974fdb80 1 Processor -- start 2023-09-13T04:41:22.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.897+0000 7f8d974fdb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562020a0cec0 0x562020a1bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:22.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.897+0000 7f8d974fdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562020a0d7e0 0x562020a1c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:22.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.897+0000 7f8d974fdb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562020a11b10 0x562020a730a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:22.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.897+0000 7f8d974fdb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562020a735e0 con 0x562020a0cec0 2023-09-13T04:41:22.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.897+0000 7f8d974fdb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562020a73720 con 0x562020a11b10 2023-09-13T04:41:22.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.897+0000 7f8d974fdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562020a738e0 con 0x562020a0d7e0 2023-09-13T04:41:22.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.897+0000 7f8d974fdb80 10 client.0 _ll_register_callbacks cb 0x5620208f2680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:22.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.897+0000 7f8d8199b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562020a0cec0 0x562020a1bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:22.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.898+0000 7f8d8199b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562020a0cec0 0x562020a1bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45194/0 (socket says 172.21.15.2:45194) 2023-09-13T04:41:22.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.898+0000 7f8d8199b700 1 -- 172.21.15.2:0/2100469008 learned_addr learned my addr 172.21.15.2:0/2100469008 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:22.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.898+0000 7f8d8219c700 1 --2- 172.21.15.2:0/2100469008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562020a11b10 0x562020a730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:22.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.898+0000 7f8d8119a700 1 --2- 172.21.15.2:0/2100469008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562020a0d7e0 0x562020a1c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:22.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.898+0000 7f8d8199b700 1 -- 172.21.15.2:0/2100469008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562020a0d7e0 msgr2=0x562020a1c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:22.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.898+0000 7f8d8199b700 1 --2- 172.21.15.2:0/2100469008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562020a0d7e0 0x562020a1c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.898+0000 7f8d8199b700 1 -- 172.21.15.2:0/2100469008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562020a11b10 msgr2=0x562020a730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:22.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.898+0000 7f8d8199b700 1 --2- 172.21.15.2:0/2100469008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562020a11b10 0x562020a730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.898+0000 7f8d8199b700 1 -- 172.21.15.2:0/2100469008 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d74004360 con 0x562020a0cec0 2023-09-13T04:41:22.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.898+0000 7f8d8119a700 1 --2- 172.21.15.2:0/2100469008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562020a0d7e0 0x562020a1c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:41:22.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.899+0000 7f8d8199b700 1 --2- 172.21.15.2:0/2100469008 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562020a0cec0 0x562020a1bbf0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8d7801a890 tx=0x7f8d7801aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656a702048267985 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.899+0000 7f8d657fa700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:41:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.899+0000 7f8d657fa700 1 -- 172.21.15.2:0/2100469008 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d78016040 con 0x562020a0cec0 2023-09-13T04:41:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.899+0000 7f8d657fa700 1 -- 172.21.15.2:0/2100469008 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f8d78004220 con 0x562020a0cec0 2023-09-13T04:41:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.899+0000 7f8d974fdb80 1 -- 172.21.15.2:0/2100469008 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d6c09e6b0 con 0x562020a0cec0 2023-09-13T04:41:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.900+0000 7f8d657fa700 1 -- 172.21.15.2:0/2100469008 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d7801a730 con 0x562020a0cec0 2023-09-13T04:41:22.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.900+0000 7f8d974fdb80 10 client.15777 fetch_fsmap learned FSMap version 23 2023-09-13T04:41:22.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.900+0000 7f8d974fdb80 1 -- 172.21.15.2:0/2100469008 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56202076bc60 con 0x562020a0cec0 2023-09-13T04:41:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.900+0000 7f8d657fa700 1 -- 172.21.15.2:0/2100469008 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d6c09e6b0 con 0x562020a0cec0 2023-09-13T04:41:22.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.900+0000 7f8d657fa700 1 -- 172.21.15.2:0/2100469008 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8d78004bc0 con 0x562020a0cec0 2023-09-13T04:41:22.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.900+0000 7f8d974fdb80 10 client.15777 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:41:22.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.900+0000 7f8d974fdb80 10 client.15777 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-09-13T04:41:22.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.901+0000 7f8d974fdb80 20 client.15777 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:22.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.901+0000 7f8d974fdb80 1 --2- 172.21.15.2:0/2100469008 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x562020a75100 0x562020a094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:22.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.901+0000 7f8d974fdb80 4 client.15777 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-09-13T04:41:22.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.901+0000 7f8d974fdb80 1 -- 172.21.15.2:0/2100469008 --> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562020910050 con 0x562020a75100 2023-09-13T04:41:22.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.901+0000 7f8d8119a700 1 --2- 172.21.15.2:0/2100469008 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x562020a75100 0x562020a094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:22.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.902+0000 7f8d8119a700 1 --2- 172.21.15.2:0/2100469008 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x562020a75100 0x562020a094e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b13db5accfd0ce01 server_cookie=859e0f6f019897b0 in_seq=0 out_seq=0 2023-09-13T04:41:22.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.902+0000 7f8d657fa700 10 client.15777 ms_handle_connect on v2:172.21.15.156:6838/1785833227 2023-09-13T04:41:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.902+0000 7f8d657fa700 1 -- 172.21.15.2:0/2100469008 <== mds.2 v2:172.21.15.156:6838/1785833227 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562020910050 con 0x562020a75100 2023-09-13T04:41:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.902+0000 7f8d657fa700 10 client.15777 handle_command_reply: tid=0 2023-09-13T04:41:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.902+0000 7f8d974fdb80 1 client.15777 shutdown 2023-09-13T04:41:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.902+0000 7f8d974fdb80 1 -- 172.21.15.2:0/2100469008 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562020a0cec0 msgr2=0x562020a1bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.902+0000 7f8d974fdb80 1 --2- 172.21.15.2:0/2100469008 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562020a0cec0 0x562020a1bbf0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8d7801a890 tx=0x7f8d7801aba0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.902+0000 7f8d974fdb80 1 -- 172.21.15.2:0/2100469008 shutdown_connections 2023-09-13T04:41:22.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.902+0000 7f8d974fdb80 1 --2- 172.21.15.2:0/2100469008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562020a11b10 0x562020a730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.902+0000 7f8d974fdb80 1 --2- 172.21.15.2:0/2100469008 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x562020a75100 0x562020a094e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.902+0000 7f8d974fdb80 1 --2- 172.21.15.2:0/2100469008 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562020a0cec0 0x562020a1bbf0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.902+0000 7f8d974fdb80 1 --2- 172.21.15.2:0/2100469008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562020a0d7e0 0x562020a1c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.902+0000 7f8d974fdb80 1 -- 172.21.15.2:0/2100469008 >> 172.21.15.2:0/2100469008 conn(0x562020962c80 msgr2=0x562020a28330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:22.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.902+0000 7f8d657fa700 0 client.15777 ms_handle_reset on v2:172.21.15.156:6838/1785833227 2023-09-13T04:41:22.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.903+0000 7f8d974fdb80 1 -- 172.21.15.2:0/2100469008 shutdown_connections 2023-09-13T04:41:22.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.903+0000 7f8d974fdb80 1 -- 172.21.15.2:0/2100469008 wait complete. 2023-09-13T04:41:22.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.903+0000 7f8d974fdb80 20 client.15777 trim_cache size 0 max 16384 2023-09-13T04:41:22.914 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-09-13T04:41:22.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.904+0000 7f8d8299d700 1 -- 172.21.15.2:0/5445780 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f8d680a0970 msgr2=0x7f8d680a2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:22.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.904+0000 7f8d8299d700 1 --2- 172.21.15.2:0/5445780 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f8d680a0970 0x7f8d680a2e30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0750c0 tx=0x7f8d74009100 comp rx=0 tx=0).stop 2023-09-13T04:41:22.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.904+0000 7f8d8299d700 1 -- 172.21.15.2:0/5445780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d7c101470 msgr2=0x7f8d7c0734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:22.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.904+0000 7f8d8299d700 1 --2- 172.21.15.2:0/5445780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d7c101470 0x7f8d7c0734f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c000c00 tx=0x7f8d6c005740 comp rx=0 tx=0).stop 2023-09-13T04:41:22.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.904+0000 7f8d8299d700 1 -- 172.21.15.2:0/5445780 shutdown_connections 2023-09-13T04:41:22.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.904+0000 7f8d8299d700 1 --2- 172.21.15.2:0/5445780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d7c106ac0 0x7f8d7c1a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.904+0000 7f8d8299d700 1 --2- 172.21.15.2:0/5445780 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f8d680a0970 0x7f8d680a2e30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.904+0000 7f8d8299d700 1 --2- 172.21.15.2:0/5445780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d7c100ab0 0x7f8d7c072fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.904+0000 7f8d8299d700 1 --2- 172.21.15.2:0/5445780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d7c101470 0x7f8d7c0734f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:22.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.904+0000 7f8d8299d700 1 -- 172.21.15.2:0/5445780 >> 172.21.15.2:0/5445780 conn(0x7f8d7c0fc560 msgr2=0x7f8d7c1079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:22.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.904+0000 7f8d8299d700 1 -- 172.21.15.2:0/5445780 shutdown_connections 2023-09-13T04:41:22.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:22.905+0000 7f8d8299d700 1 -- 172.21.15.2:0/5445780 wait complete. 2023-09-13T04:41:22.933 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-13T04:41:23.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:22 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2813720587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:41:23.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:22 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2813720587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:41:23.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:22 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2813720587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:41:23.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.279+0000 7f1bcf5ff700 1 -- 172.21.15.2:0/2027948062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1bc80739a0 msgr2=0x7f1bc810cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.279+0000 7f1bcf5ff700 1 --2- 172.21.15.2:0/2027948062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1bc80739a0 0x7f1bc810cfb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8009990 tx=0x7f1bb8009ca0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.280+0000 7f1bcf5ff700 1 -- 172.21.15.2:0/2027948062 shutdown_connections 2023-09-13T04:41:23.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.280+0000 7f1bcf5ff700 1 --2- 172.21.15.2:0/2027948062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1bc810d4f0 0x7f1bc810f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.280+0000 7f1bcf5ff700 1 --2- 172.21.15.2:0/2027948062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bc8073080 0x7f1bc8073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.280+0000 7f1bcf5ff700 1 --2- 172.21.15.2:0/2027948062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1bc80739a0 0x7f1bc810cfb0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.280+0000 7f1bcf5ff700 1 -- 172.21.15.2:0/2027948062 >> 172.21.15.2:0/2027948062 conn(0x7f1bc80fc760 msgr2=0x7f1bc80feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:23.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.280+0000 7f1bcf5ff700 1 -- 172.21.15.2:0/2027948062 shutdown_connections 2023-09-13T04:41:23.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.280+0000 7f1bcf5ff700 1 -- 172.21.15.2:0/2027948062 wait complete. 2023-09-13T04:41:23.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.282+0000 7f1bcf5ff700 1 Processor -- start 2023-09-13T04:41:23.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.282+0000 7f1bcf5ff700 1 -- start start 2023-09-13T04:41:23.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.282+0000 7f1bcf5ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1bc8073080 0x7f1bc8199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.282+0000 7f1bcf5ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1bc80739a0 0x7f1bc81995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.282+0000 7f1bcf5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bc810d4f0 0x7f1bc819d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.283+0000 7f1bcf5ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc819dfd0 con 0x7f1bc80739a0 2023-09-13T04:41:23.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.283+0000 7f1bcf5ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc819e140 con 0x7f1bc8073080 2023-09-13T04:41:23.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.283+0000 7f1bcf5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc819e300 con 0x7f1bc810d4f0 2023-09-13T04:41:23.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.283+0000 7f1bcd39b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1bc8073080 0x7f1bc8199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.283+0000 7f1bcd39b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1bc8073080 0x7f1bc8199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:44406/0 (socket says 172.21.15.2:44406) 2023-09-13T04:41:23.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.283+0000 7f1bcd39b700 1 -- 172.21.15.2:0/2466086780 learned_addr learned my addr 172.21.15.2:0/2466086780 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:23.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.283+0000 7f1bcdb9c700 1 --2- 172.21.15.2:0/2466086780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bc810d4f0 0x7f1bc819d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.283+0000 7f1bccb9a700 1 --2- 172.21.15.2:0/2466086780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1bc80739a0 0x7f1bc81995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.283+0000 7f1bcd39b700 1 -- 172.21.15.2:0/2466086780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bc810d4f0 msgr2=0x7f1bc819d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.283+0000 7f1bcd39b700 1 --2- 172.21.15.2:0/2466086780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bc810d4f0 0x7f1bc819d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.283+0000 7f1bcd39b700 1 -- 172.21.15.2:0/2466086780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1bc80739a0 msgr2=0x7f1bc81995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.283+0000 7f1bcd39b700 1 --2- 172.21.15.2:0/2466086780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1bc80739a0 0x7f1bc81995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.283+0000 7f1bcd39b700 1 -- 172.21.15.2:0/2466086780 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc4009520 con 0x7f1bc8073080 2023-09-13T04:41:23.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.284+0000 7f1bcd39b700 1 --2- 172.21.15.2:0/2466086780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1bc8073080 0x7f1bc8199060 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1bc400eb30 tx=0x7f1bc400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:23.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.284+0000 7f1bbe7fc700 1 -- 172.21.15.2:0/2466086780 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4009880 con 0x7f1bc8073080 2023-09-13T04:41:23.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.284+0000 7f1bbe7fc700 1 -- 172.21.15.2:0/2466086780 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bc40044e0 con 0x7f1bc8073080 2023-09-13T04:41:23.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.284+0000 7f1bcf5ff700 1 -- 172.21.15.2:0/2466086780 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb80095f0 con 0x7f1bc8073080 2023-09-13T04:41:23.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.284+0000 7f1bbe7fc700 1 -- 172.21.15.2:0/2466086780 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4003e90 con 0x7f1bc8073080 2023-09-13T04:41:23.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.284+0000 7f1bcf5ff700 1 -- 172.21.15.2:0/2466086780 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc819e7d0 con 0x7f1bc8073080 2023-09-13T04:41:23.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.290+0000 7f1bbe7fc700 1 -- 172.21.15.2:0/2466086780 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 24) v1 ==== 135491+0+0 (secure 0 0 0) 0x7f1bc4004000 con 0x7f1bc8073080 2023-09-13T04:41:23.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.291+0000 7f1be2e34b80 1 Processor -- start 2023-09-13T04:41:23.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.291+0000 7f1be2e34b80 1 -- start start 2023-09-13T04:41:23.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.292+0000 7f1be2e34b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f77b52dec0 0x55f77b52e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.292+0000 7f1be2e34b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f77b52e7e0 0x55f77b5323f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.292+0000 7f1be2e34b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f77b532930 0x55f77b532d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.292+0000 7f1be2e34b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77b47e780 con 0x55f77b52dec0 2023-09-13T04:41:23.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.292+0000 7f1be2e34b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77b47c0a0 con 0x55f77b52e7e0 2023-09-13T04:41:23.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.292+0000 7f1be2e34b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77b477990 con 0x55f77b532930 2023-09-13T04:41:23.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.292+0000 7f1bccb9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f77b532930 0x55f77b532d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.292+0000 7f1bccb9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f77b532930 0x55f77b532d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:37276/0 (socket says 172.21.15.2:37276) 2023-09-13T04:41:23.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.292+0000 7f1bccb9a700 1 -- 172.21.15.2:0/3216044871 learned_addr learned my addr 172.21.15.2:0/3216044871 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:23.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.292+0000 7f1bcdb9c700 1 --2- 172.21.15.2:0/3216044871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f77b52dec0 0x55f77b52e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.293+0000 7f1bbe7fc700 1 --2- 172.21.15.2:0/2466086780 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f1bb40a0950 0x7f1bb40a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.293+0000 7f1bccb9a700 1 -- 172.21.15.2:0/3216044871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f77b52e7e0 msgr2=0x55f77b5323f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:41:23.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.293+0000 7f1bccb9a700 1 --2- 172.21.15.2:0/3216044871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f77b52e7e0 0x55f77b5323f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.293+0000 7f1bccb9a700 1 -- 172.21.15.2:0/3216044871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f77b52dec0 msgr2=0x55f77b52e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:23.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.293+0000 7f1bccb9a700 1 --2- 172.21.15.2:0/3216044871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f77b52dec0 0x55f77b52e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.293+0000 7f1bccb9a700 1 -- 172.21.15.2:0/3216044871 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f77b2f7630 con 0x55f77b532930 2023-09-13T04:41:23.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.293+0000 7f1bcdb9c700 1 --2- 172.21.15.2:0/3216044871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f77b52dec0 0x55f77b52e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:41:23.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.293+0000 7f1bbe7fc700 1 -- 172.21.15.2:0/2466086780 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f1bc4014070 con 0x7f1bc8073080 2023-09-13T04:41:23.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.294+0000 7f1bccb9a700 1 --2- 172.21.15.2:0/3216044871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f77b532930 0x55f77b532d90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8015000 tx=0x7f1bb8005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bff927dffd88972 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:23.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.294+0000 7f1bcdb9c700 1 --2- 172.21.15.2:0/2466086780 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f1bb40a0950 0x7f1bb40a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.294+0000 7f1bb37fe700 1 -- 172.21.15.2:0/3216044871 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb800bee0 con 0x55f77b532930 2023-09-13T04:41:23.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.294+0000 7f1bb37fe700 1 -- 172.21.15.2:0/3216044871 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bb801e040 con 0x55f77b532930 2023-09-13T04:41:23.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.294+0000 7f1bb37fe700 1 -- 172.21.15.2:0/3216044871 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8004bd0 con 0x55f77b532930 2023-09-13T04:41:23.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.295+0000 7f1be2e34b80 1 -- 172.21.15.2:0/3216044871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f77b532930 msgr2=0x55f77b532d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.295+0000 7f1bcdb9c700 1 --2- 172.21.15.2:0/2466086780 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f1bb40a0950 0x7f1bb40a2e10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0009790 tx=0x7f1bc000b040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:23.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.295+0000 7f1be2e34b80 1 --2- 172.21.15.2:0/3216044871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f77b532930 0x55f77b532d90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8015000 tx=0x7f1bb8005e80 comp rx=0 tx=0).stop 2023-09-13T04:41:23.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.295+0000 7f1be2e34b80 1 -- 172.21.15.2:0/3216044871 shutdown_connections 2023-09-13T04:41:23.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.295+0000 7f1be2e34b80 1 --2- 172.21.15.2:0/3216044871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f77b532930 0x55f77b532d90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.295+0000 7f1be2e34b80 1 --2- 172.21.15.2:0/3216044871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f77b52dec0 0x55f77b52e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.295+0000 7f1be2e34b80 1 --2- 172.21.15.2:0/3216044871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f77b52e7e0 0x55f77b5323f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.295+0000 7f1be2e34b80 1 -- 172.21.15.2:0/3216044871 >> 172.21.15.2:0/3216044871 conn(0x55f77b483c80 msgr2=0x55f77b51ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:23.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.295+0000 7f1be2e34b80 1 -- 172.21.15.2:0/3216044871 shutdown_connections 2023-09-13T04:41:23.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.295+0000 7f1be2e34b80 1 -- 172.21.15.2:0/3216044871 wait complete. 2023-09-13T04:41:23.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.296+0000 7f1be2e34b80 1 -- start start 2023-09-13T04:41:23.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.296+0000 7f1be2e34b80 1 Processor -- start 2023-09-13T04:41:23.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.297+0000 7f1be2e34b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f77b52dec0 0x55f77b5373b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.297+0000 7f1be2e34b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f77b52e7e0 0x55f77b5378f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.297+0000 7f1be2e34b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f77b532930 0x55f77b537e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.297+0000 7f1be2e34b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77b415bb0 con 0x55f77b52e7e0 2023-09-13T04:41:23.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.297+0000 7f1be2e34b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77b4158b0 con 0x55f77b52dec0 2023-09-13T04:41:23.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.297+0000 7f1be2e34b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77b2f9920 con 0x55f77b532930 2023-09-13T04:41:23.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.297+0000 7f1be2e34b80 10 client.0 _ll_register_callbacks cb 0x55f77b413680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:23.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.298+0000 7f1bccb9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f77b52e7e0 0x55f77b5378f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.298+0000 7f1bccb9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f77b52e7e0 0x55f77b5378f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45226/0 (socket says 172.21.15.2:45226) 2023-09-13T04:41:23.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.298+0000 7f1bccb9a700 1 -- 172.21.15.2:0/4130078755 learned_addr learned my addr 172.21.15.2:0/4130078755 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:23.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.298+0000 7f1bccb9a700 1 -- 172.21.15.2:0/4130078755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f77b532930 msgr2=0x55f77b537e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:23.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.298+0000 7f1bccb9a700 1 --2- 172.21.15.2:0/4130078755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f77b532930 0x55f77b537e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.298+0000 7f1bccb9a700 1 -- 172.21.15.2:0/4130078755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f77b52dec0 msgr2=0x55f77b5373b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:23.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.298+0000 7f1bccb9a700 1 --2- 172.21.15.2:0/4130078755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f77b52dec0 0x55f77b5373b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.298+0000 7f1bccb9a700 1 -- 172.21.15.2:0/4130078755 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc8110630 con 0x55f77b52e7e0 2023-09-13T04:41:23.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.299+0000 7f1bccb9a700 1 --2- 172.21.15.2:0/4130078755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f77b52e7e0 0x55f77b5378f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8004750 tx=0x7f1bb8004b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71491fc32a9d599 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:23.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.299+0000 7f1bb0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:41:23.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.299+0000 7f1bb0ff9700 1 -- 172.21.15.2:0/4130078755 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8005450 con 0x55f77b52e7e0 2023-09-13T04:41:23.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.299+0000 7f1bb0ff9700 1 -- 172.21.15.2:0/4130078755 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f1bb8004e30 con 0x55f77b52e7e0 2023-09-13T04:41:23.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.299+0000 7f1be2e34b80 1 -- 172.21.15.2:0/4130078755 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f77b518670 con 0x55f77b52e7e0 2023-09-13T04:41:23.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.300+0000 7f1bb0ff9700 1 -- 172.21.15.2:0/4130078755 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bb8004570 con 0x55f77b52e7e0 2023-09-13T04:41:23.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.300+0000 7f1be2e34b80 10 client.15789 fetch_fsmap learned FSMap version 23 2023-09-13T04:41:23.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.300+0000 7f1be2e34b80 1 -- 172.21.15.2:0/4130078755 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f77b48f7a0 con 0x55f77b52e7e0 2023-09-13T04:41:23.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.300+0000 7f1bb0ff9700 1 -- 172.21.15.2:0/4130078755 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f77b518670 con 0x55f77b52e7e0 2023-09-13T04:41:23.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.300+0000 7f1bb0ff9700 1 -- 172.21.15.2:0/4130078755 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1bb8035070 con 0x55f77b52e7e0 2023-09-13T04:41:23.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.300+0000 7f1be2e34b80 10 client.15789 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:41:23.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.300+0000 7f1be2e34b80 10 client.15789 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-09-13T04:41:23.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.300+0000 7f1be2e34b80 20 client.15789 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:23.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.300+0000 7f1be2e34b80 1 --2- 172.21.15.2:0/4130078755 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x55f77b5475b0 0x55f77b5499a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.300+0000 7f1be2e34b80 4 client.15789 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:41:23.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.300+0000 7f1be2e34b80 1 -- 172.21.15.2:0/4130078755 --> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f77b470970 con 0x55f77b5475b0 2023-09-13T04:41:23.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.301+0000 7f1bcd39b700 1 --2- 172.21.15.2:0/4130078755 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x55f77b5475b0 0x55f77b5499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.302+0000 7f1bcd39b700 1 --2- 172.21.15.2:0/4130078755 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x55f77b5475b0 0x55f77b5499a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d4d01007fc63d69 server_cookie=2dad73d6f66b334d in_seq=0 out_seq=0 2023-09-13T04:41:23.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.302+0000 7f1bb0ff9700 10 client.15789 ms_handle_connect on v2:172.21.15.2:6840/482545391 2023-09-13T04:41:23.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.303+0000 7f1bb0ff9700 1 -- 172.21.15.2:0/4130078755 <== mds.1 v2:172.21.15.2:6840/482545391 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f77b470970 con 0x55f77b5475b0 2023-09-13T04:41:23.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.303+0000 7f1bb0ff9700 10 client.15789 handle_command_reply: tid=0 2023-09-13T04:41:23.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.303+0000 7f1be2e34b80 1 client.15789 shutdown 2023-09-13T04:41:23.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.303+0000 7f1be2e34b80 1 -- 172.21.15.2:0/4130078755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f77b52e7e0 msgr2=0x55f77b5378f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.303+0000 7f1be2e34b80 1 --2- 172.21.15.2:0/4130078755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f77b52e7e0 0x55f77b5378f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8004750 tx=0x7f1bb8004b10 comp rx=0 tx=0).stop 2023-09-13T04:41:23.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.303+0000 7f1be2e34b80 1 -- 172.21.15.2:0/4130078755 shutdown_connections 2023-09-13T04:41:23.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.303+0000 7f1be2e34b80 1 --2- 172.21.15.2:0/4130078755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f77b532930 0x55f77b537e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.303+0000 7f1be2e34b80 1 --2- 172.21.15.2:0/4130078755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f77b52dec0 0x55f77b5373b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.303+0000 7f1be2e34b80 1 --2- 172.21.15.2:0/4130078755 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x55f77b5475b0 0x55f77b5499a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.303+0000 7f1be2e34b80 1 --2- 172.21.15.2:0/4130078755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f77b52e7e0 0x55f77b5378f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.303+0000 7f1be2e34b80 1 -- 172.21.15.2:0/4130078755 >> 172.21.15.2:0/4130078755 conn(0x55f77b483c80 msgr2=0x55f77b533de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:23.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.303+0000 7f1bb0ff9700 0 client.15789 ms_handle_reset on v2:172.21.15.2:6840/482545391 2023-09-13T04:41:23.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.304+0000 7f1be2e34b80 1 -- 172.21.15.2:0/4130078755 shutdown_connections 2023-09-13T04:41:23.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.304+0000 7f1be2e34b80 1 -- 172.21.15.2:0/4130078755 wait complete. 2023-09-13T04:41:23.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.304+0000 7f1be2e34b80 20 client.15789 trim_cache size 0 max 16384 2023-09-13T04:41:23.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.330+0000 7f1be2e34b80 1 Processor -- start 2023-09-13T04:41:23.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.330+0000 7f1be2e34b80 1 -- start start 2023-09-13T04:41:23.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.331+0000 7f1be2e34b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f77b52dec0 0x55f77b530220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.331+0000 7f1be2e34b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f77b52e7e0 0x55f77b51a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.331+0000 7f1be2e34b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f77b532930 0x55f77b53ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.331+0000 7f1be2e34b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77b53a800 con 0x55f77b52e7e0 2023-09-13T04:41:23.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.331+0000 7f1be2e34b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77b52ec20 con 0x55f77b52dec0 2023-09-13T04:41:23.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.331+0000 7f1be2e34b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77b518610 con 0x55f77b532930 2023-09-13T04:41:23.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.331+0000 7f1bcd39b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f77b52dec0 0x55f77b530220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.331+0000 7f1bcdb9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f77b532930 0x55f77b53ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.331+0000 7f1bcd39b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f77b52dec0 0x55f77b530220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:44426/0 (socket says 172.21.15.2:44426) 2023-09-13T04:41:23.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.331+0000 7f1bcd39b700 1 -- 172.21.15.2:0/586158989 learned_addr learned my addr 172.21.15.2:0/586158989 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:23.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.331+0000 7f1bccb9a700 1 --2- 172.21.15.2:0/586158989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f77b52e7e0 0x55f77b51a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.332+0000 7f1bcd39b700 1 -- 172.21.15.2:0/586158989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f77b532930 msgr2=0x55f77b53ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:23.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.332+0000 7f1bcd39b700 1 --2- 172.21.15.2:0/586158989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f77b532930 0x55f77b53ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.332+0000 7f1bcd39b700 1 -- 172.21.15.2:0/586158989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f77b52e7e0 msgr2=0x55f77b51a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:23.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.332+0000 7f1bcd39b700 1 --2- 172.21.15.2:0/586158989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f77b52e7e0 0x55f77b51a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.332+0000 7f1bcd39b700 1 -- 172.21.15.2:0/586158989 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f77b520130 con 0x55f77b52dec0 2023-09-13T04:41:23.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.332+0000 7f1bccb9a700 1 --2- 172.21.15.2:0/586158989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f77b52e7e0 0x55f77b51a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:41:23.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.332+0000 7f1bcd39b700 1 --2- 172.21.15.2:0/586158989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f77b52dec0 0x55f77b530220 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4095d50 tx=0x7f1bc40954f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55faf68f46b812c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:23.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.332+0000 7f1bb3fff700 1 -- 172.21.15.2:0/586158989 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4099f50 con 0x55f77b52dec0 2023-09-13T04:41:23.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.332+0000 7f1bb3fff700 1 -- 172.21.15.2:0/586158989 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bc400cc90 con 0x55f77b52dec0 2023-09-13T04:41:23.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.332+0000 7f1bb3fff700 1 -- 172.21.15.2:0/586158989 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4097a20 con 0x55f77b52dec0 2023-09-13T04:41:23.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.333+0000 7f1be2e34b80 1 -- 172.21.15.2:0/586158989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f77b52dec0 msgr2=0x55f77b530220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.333+0000 7f1be2e34b80 1 --2- 172.21.15.2:0/586158989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f77b52dec0 0x55f77b530220 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4095d50 tx=0x7f1bc40954f0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.334+0000 7f1be2e34b80 1 -- 172.21.15.2:0/586158989 shutdown_connections 2023-09-13T04:41:23.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.334+0000 7f1be2e34b80 1 --2- 172.21.15.2:0/586158989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f77b532930 0x55f77b53ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.334+0000 7f1be2e34b80 1 --2- 172.21.15.2:0/586158989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f77b52dec0 0x55f77b530220 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.334+0000 7f1be2e34b80 1 --2- 172.21.15.2:0/586158989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f77b52e7e0 0x55f77b51a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.334+0000 7f1be2e34b80 1 -- 172.21.15.2:0/586158989 >> 172.21.15.2:0/586158989 conn(0x55f77b483c80 msgr2=0x55f77b5319d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:23.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.334+0000 7f1be2e34b80 1 -- 172.21.15.2:0/586158989 shutdown_connections 2023-09-13T04:41:23.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.334+0000 7f1be2e34b80 1 -- 172.21.15.2:0/586158989 wait complete. 2023-09-13T04:41:23.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.334+0000 7f1be2e34b80 1 -- start start 2023-09-13T04:41:23.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.335+0000 7f1be2e34b80 1 Processor -- start 2023-09-13T04:41:23.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.335+0000 7f1be2e34b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f77b52dec0 0x55f77b53e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.335+0000 7f1be2e34b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f77b52e7e0 0x55f77b53e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.335+0000 7f1be2e34b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f77b532930 0x55f77b53ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.336+0000 7f1be2e34b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77b53d030 con 0x55f77b52e7e0 2023-09-13T04:41:23.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.336+0000 7f1be2e34b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77b53d1a0 con 0x55f77b532930 2023-09-13T04:41:23.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.336+0000 7f1be2e34b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77b53d360 con 0x55f77b52dec0 2023-09-13T04:41:23.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.336+0000 7f1be2e34b80 10 client.0 _ll_register_callbacks cb 0x55f77b413680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:23.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.336+0000 7f1bcd39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f77b52dec0 0x55f77b53e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.336+0000 7f1bcd39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f77b52dec0 0x55f77b53e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:37298/0 (socket says 172.21.15.2:37298) 2023-09-13T04:41:23.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.336+0000 7f1bcd39b700 1 -- 172.21.15.2:0/348720054 learned_addr learned my addr 172.21.15.2:0/348720054 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:23.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.336+0000 7f1bccb9a700 1 --2- 172.21.15.2:0/348720054 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f77b52e7e0 0x55f77b53e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.336+0000 7f1bcdb9c700 1 --2- 172.21.15.2:0/348720054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f77b532930 0x55f77b53ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.337+0000 7f1bcdb9c700 1 -- 172.21.15.2:0/348720054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f77b52dec0 msgr2=0x55f77b53e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:23.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.337+0000 7f1bcdb9c700 1 --2- 172.21.15.2:0/348720054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f77b52dec0 0x55f77b53e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.337+0000 7f1bcdb9c700 1 -- 172.21.15.2:0/348720054 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f77b52e7e0 msgr2=0x55f77b53e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:23.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.337+0000 7f1bcdb9c700 1 --2- 172.21.15.2:0/348720054 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f77b52e7e0 0x55f77b53e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.337+0000 7f1bcdb9c700 1 -- 172.21.15.2:0/348720054 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc819e7d0 con 0x55f77b532930 2023-09-13T04:41:23.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.337+0000 7f1bcdb9c700 1 --2- 172.21.15.2:0/348720054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f77b532930 0x55f77b53ca60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55f77b5385f0 tx=0x7f1bc0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86a42f146f8f6f14 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:23.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.337+0000 7f1bb0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:41:23.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.337+0000 7f1bb0ff9700 1 -- 172.21.15.2:0/348720054 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0007630 con 0x55f77b532930 2023-09-13T04:41:23.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.337+0000 7f1be2e34b80 1 -- 172.21.15.2:0/348720054 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bb801e040 con 0x55f77b532930 2023-09-13T04:41:23.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.338+0000 7f1bb0ff9700 1 -- 172.21.15.2:0/348720054 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f1bc001c070 con 0x55f77b532930 2023-09-13T04:41:23.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.338+0000 7f1bb0ff9700 1 -- 172.21.15.2:0/348720054 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bc0007790 con 0x55f77b532930 2023-09-13T04:41:23.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.338+0000 7f1be2e34b80 10 client.25817 fetch_fsmap learned FSMap version 23 2023-09-13T04:41:23.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.338+0000 7f1be2e34b80 1 -- 172.21.15.2:0/348720054 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f77b2a3de0 con 0x55f77b532930 2023-09-13T04:41:23.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.338+0000 7f1bb0ff9700 1 -- 172.21.15.2:0/348720054 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bb801e040 con 0x55f77b532930 2023-09-13T04:41:23.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.339+0000 7f1bb0ff9700 1 -- 172.21.15.2:0/348720054 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1bc0046940 con 0x55f77b532930 2023-09-13T04:41:23.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.339+0000 7f1be2e34b80 10 client.25817 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:41:23.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.339+0000 7f1be2e34b80 10 client.25817 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-09-13T04:41:23.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.339+0000 7f1be2e34b80 20 client.25817 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:23.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.339+0000 7f1be2e34b80 1 --2- 172.21.15.2:0/348720054 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x55f77b526590 0x55f77b5289b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.339+0000 7f1be2e34b80 4 client.25817 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-09-13T04:41:23.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.339+0000 7f1be2e34b80 1 -- 172.21.15.2:0/348720054 --> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f77b431050 con 0x55f77b526590 2023-09-13T04:41:23.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.339+0000 7f1bcd39b700 1 --2- 172.21.15.2:0/348720054 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x55f77b526590 0x55f77b5289b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.340+0000 7f1bcd39b700 1 --2- 172.21.15.2:0/348720054 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x55f77b526590 0x55f77b5289b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6645095263bb58d9 server_cookie=f3c7f49cc9fcd894 in_seq=0 out_seq=0 2023-09-13T04:41:23.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.340+0000 7f1bb0ff9700 10 client.25817 ms_handle_connect on v2:172.21.15.2:6840/482545391 2023-09-13T04:41:23.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.340+0000 7f1bb0ff9700 1 -- 172.21.15.2:0/348720054 <== mds.1 v2:172.21.15.2:6840/482545391 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f77b431050 con 0x55f77b526590 2023-09-13T04:41:23.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.340+0000 7f1bb0ff9700 10 client.25817 handle_command_reply: tid=0 2023-09-13T04:41:23.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.340+0000 7f1be2e34b80 1 client.25817 shutdown 2023-09-13T04:41:23.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.340+0000 7f1be2e34b80 1 -- 172.21.15.2:0/348720054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f77b532930 msgr2=0x55f77b53ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.340+0000 7f1be2e34b80 1 --2- 172.21.15.2:0/348720054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f77b532930 0x55f77b53ca60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55f77b5385f0 tx=0x7f1bc0004ad0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.341+0000 7f1be2e34b80 1 -- 172.21.15.2:0/348720054 shutdown_connections 2023-09-13T04:41:23.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.341+0000 7f1be2e34b80 1 --2- 172.21.15.2:0/348720054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f77b532930 0x55f77b53ca60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.341+0000 7f1be2e34b80 1 --2- 172.21.15.2:0/348720054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f77b52dec0 0x55f77b53e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.341+0000 7f1be2e34b80 1 --2- 172.21.15.2:0/348720054 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x55f77b526590 0x55f77b5289b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.341+0000 7f1bb0ff9700 0 client.25817 ms_handle_reset on v2:172.21.15.2:6840/482545391 2023-09-13T04:41:23.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.341+0000 7f1be2e34b80 1 --2- 172.21.15.2:0/348720054 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f77b52e7e0 0x55f77b53e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.341+0000 7f1be2e34b80 1 -- 172.21.15.2:0/348720054 >> 172.21.15.2:0/348720054 conn(0x55f77b483c80 msgr2=0x55f77b535c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:23.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.341+0000 7f1be2e34b80 1 -- 172.21.15.2:0/348720054 shutdown_connections 2023-09-13T04:41:23.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.341+0000 7f1be2e34b80 1 -- 172.21.15.2:0/348720054 wait complete. 2023-09-13T04:41:23.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.341+0000 7f1be2e34b80 20 client.25817 trim_cache size 0 max 16384 2023-09-13T04:41:23.352 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-09-13T04:41:23.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.342+0000 7f1bcf5ff700 1 -- 172.21.15.2:0/2466086780 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f1bb40a0950 msgr2=0x7f1bb40a2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.342+0000 7f1bcf5ff700 1 --2- 172.21.15.2:0/2466086780 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f1bb40a0950 0x7f1bb40a2e10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0009790 tx=0x7f1bc000b040 comp rx=0 tx=0).stop 2023-09-13T04:41:23.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.342+0000 7f1bcf5ff700 1 -- 172.21.15.2:0/2466086780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1bc8073080 msgr2=0x7f1bc8199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.342+0000 7f1bcf5ff700 1 --2- 172.21.15.2:0/2466086780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1bc8073080 0x7f1bc8199060 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1bc400eb30 tx=0x7f1bc400ee40 comp rx=0 tx=0).stop 2023-09-13T04:41:23.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.342+0000 7f1bcf5ff700 1 -- 172.21.15.2:0/2466086780 shutdown_connections 2023-09-13T04:41:23.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.342+0000 7f1bcf5ff700 1 --2- 172.21.15.2:0/2466086780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bc810d4f0 0x7f1bc819d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.342+0000 7f1bcf5ff700 1 --2- 172.21.15.2:0/2466086780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1bc8073080 0x7f1bc8199060 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.342+0000 7f1bcf5ff700 1 --2- 172.21.15.2:0/2466086780 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f1bb40a0950 0x7f1bb40a2e10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.342+0000 7f1bcf5ff700 1 --2- 172.21.15.2:0/2466086780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1bc80739a0 0x7f1bc81995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.342+0000 7f1bcf5ff700 1 -- 172.21.15.2:0/2466086780 >> 172.21.15.2:0/2466086780 conn(0x7f1bc80fc760 msgr2=0x7f1bc810f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:23.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.342+0000 7f1bcf5ff700 1 -- 172.21.15.2:0/2466086780 shutdown_connections 2023-09-13T04:41:23.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.343+0000 7f1bcf5ff700 1 -- 172.21.15.2:0/2466086780 wait complete. 2023-09-13T04:41:23.371 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-13T04:41:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.653+0000 7f2f285ff700 1 -- 172.21.15.2:0/1706715601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f20106d60 msgr2=0x7f2f20107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.653+0000 7f2f285ff700 1 --2- 172.21.15.2:0/1706715601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f20106d60 0x7f2f20107140 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009990 tx=0x7f2f10009ca0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.653+0000 7f2f285ff700 1 -- 172.21.15.2:0/1706715601 shutdown_connections 2023-09-13T04:41:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.653+0000 7f2f285ff700 1 --2- 172.21.15.2:0/1706715601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f20068a90 0x7f2f2010db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.653+0000 7f2f285ff700 1 --2- 172.21.15.2:0/1706715601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f20106d60 0x7f2f20107140 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.653+0000 7f2f285ff700 1 --2- 172.21.15.2:0/1706715601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f200680d0 0x7f2f20068550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.653+0000 7f2f285ff700 1 -- 172.21.15.2:0/1706715601 >> 172.21.15.2:0/1706715601 conn(0x7f2f20076970 msgr2=0x7f2f20076d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.653+0000 7f2f285ff700 1 -- 172.21.15.2:0/1706715601 shutdown_connections 2023-09-13T04:41:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.653+0000 7f2f285ff700 1 -- 172.21.15.2:0/1706715601 wait complete. 2023-09-13T04:41:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.655+0000 7f2f285ff700 1 Processor -- start 2023-09-13T04:41:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.655+0000 7f2f285ff700 1 -- start start 2023-09-13T04:41:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.656+0000 7f2f285ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f200680d0 0x7f2f2010b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.656+0000 7f2f285ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f20068a90 0x7f2f2010bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.656+0000 7f2f285ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f20106d60 0x7f2f2019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.656+0000 7f2f285ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2019af30 con 0x7f2f20068a90 2023-09-13T04:41:23.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.656+0000 7f2f285ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2019b0a0 con 0x7f2f20106d60 2023-09-13T04:41:23.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.656+0000 7f2f285ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2019b260 con 0x7f2f200680d0 2023-09-13T04:41:23.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.656+0000 7f2f26b9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f20106d60 0x7f2f2019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.656+0000 7f2f26b9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f20106d60 0x7f2f2019a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:44444/0 (socket says 172.21.15.2:44444) 2023-09-13T04:41:23.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.656+0000 7f2f26b9c700 1 -- 172.21.15.2:0/547210932 learned_addr learned my addr 172.21.15.2:0/547210932 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:23.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.657+0000 7f2f2639b700 1 --2- 172.21.15.2:0/547210932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f200680d0 0x7f2f2010b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.657+0000 7f2f26b9c700 1 -- 172.21.15.2:0/547210932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f200680d0 msgr2=0x7f2f2010b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.657+0000 7f2f26b9c700 1 --2- 172.21.15.2:0/547210932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f200680d0 0x7f2f2010b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.657+0000 7f2f26b9c700 1 -- 172.21.15.2:0/547210932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f20068a90 msgr2=0x7f2f2010bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.657+0000 7f2f26b9c700 1 --2- 172.21.15.2:0/547210932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f20068a90 0x7f2f2010bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.657+0000 7f2f26b9c700 1 -- 172.21.15.2:0/547210932 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f100095f0 con 0x7f2f20106d60 2023-09-13T04:41:23.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.657+0000 7f2f26b9c700 1 --2- 172.21.15.2:0/547210932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f20106d60 0x7f2f2019a930 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800efc0 tx=0x7f2f1800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:23.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.658+0000 7f2f177fe700 1 -- 172.21.15.2:0/547210932 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f18018070 con 0x7f2f20106d60 2023-09-13T04:41:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.658+0000 7f2f177fe700 1 -- 172.21.15.2:0/547210932 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f1800f040 con 0x7f2f20106d60 2023-09-13T04:41:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.658+0000 7f2f177fe700 1 -- 172.21.15.2:0/547210932 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1800cd80 con 0x7f2f20106d60 2023-09-13T04:41:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.658+0000 7f2f285ff700 1 -- 172.21.15.2:0/547210932 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2019b4e0 con 0x7f2f20106d60 2023-09-13T04:41:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.658+0000 7f2f285ff700 1 -- 172.21.15.2:0/547210932 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f20197a40 con 0x7f2f20106d60 2023-09-13T04:41:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.664+0000 7f2f177fe700 1 -- 172.21.15.2:0/547210932 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 24) v1 ==== 135491+0+0 (secure 0 0 0) 0x7f2f18003a00 con 0x7f2f20106d60 2023-09-13T04:41:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.664+0000 7f2f3bf78b80 1 Processor -- start 2023-09-13T04:41:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.665+0000 7f2f3bf78b80 1 -- start start 2023-09-13T04:41:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.666+0000 7f2f3bf78b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6ef5fdbe0 0x55e6ef5fdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.666+0000 7f2f3bf78b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e6ef5fe500 0x55e6ef5f9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.666+0000 7f2f3bf78b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6ef5f9840 0x55e6ef5f9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.666+0000 7f2f3bf78b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ef552720 con 0x55e6ef5f9840 2023-09-13T04:41:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.666+0000 7f2f3bf78b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ef550040 con 0x55e6ef5fe500 2023-09-13T04:41:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.666+0000 7f2f3bf78b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ef54b930 con 0x55e6ef5fdbe0 2023-09-13T04:41:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.666+0000 7f2f25b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6ef5fdbe0 0x55e6ef5fdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.666+0000 7f2f25b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6ef5fdbe0 0x55e6ef5fdfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:37320/0 (socket says 172.21.15.2:37320) 2023-09-13T04:41:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.666+0000 7f2f25b9a700 1 -- 172.21.15.2:0/4294192718 learned_addr learned my addr 172.21.15.2:0/4294192718 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.667+0000 7f2f2639b700 1 --2- 172.21.15.2:0/4294192718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e6ef5fe500 0x55e6ef5f9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.667+0000 7f2f25b9a700 1 -- 172.21.15.2:0/4294192718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e6ef5fe500 msgr2=0x55e6ef5f9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.667+0000 7f2f25b9a700 1 --2- 172.21.15.2:0/4294192718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e6ef5fe500 0x55e6ef5f9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.667+0000 7f2f25b9a700 1 -- 172.21.15.2:0/4294192718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6ef5f9840 msgr2=0x55e6ef5f9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.667+0000 7f2f25b9a700 1 --2- 172.21.15.2:0/4294192718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6ef5f9840 0x55e6ef5f9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.667+0000 7f2f25b9a700 1 -- 172.21.15.2:0/4294192718 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6ef3cb5d0 con 0x55e6ef5fdbe0 2023-09-13T04:41:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.667+0000 7f2f25b9a700 1 --2- 172.21.15.2:0/4294192718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6ef5fdbe0 0x55e6ef5fdfc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c009c60 tx=0x7f2f1c009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35e7d633b7a141a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.668+0000 7f2f14bfe700 1 -- 172.21.15.2:0/4294192718 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c010c50 con 0x55e6ef5fdbe0 2023-09-13T04:41:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.668+0000 7f2f14bfe700 1 -- 172.21.15.2:0/4294192718 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f1c0044f0 con 0x55e6ef5fdbe0 2023-09-13T04:41:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.668+0000 7f2f14bfe700 1 -- 172.21.15.2:0/4294192718 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c011440 con 0x55e6ef5fdbe0 2023-09-13T04:41:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.669+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/4294192718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6ef5fdbe0 msgr2=0x55e6ef5fdfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.669+0000 7f2f3bf78b80 1 --2- 172.21.15.2:0/4294192718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6ef5fdbe0 0x55e6ef5fdfc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c009c60 tx=0x7f2f1c009f70 comp rx=0 tx=0).stop 2023-09-13T04:41:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.669+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/4294192718 shutdown_connections 2023-09-13T04:41:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.669+0000 7f2f3bf78b80 1 --2- 172.21.15.2:0/4294192718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6ef5f9840 0x55e6ef5f9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.669+0000 7f2f3bf78b80 1 --2- 172.21.15.2:0/4294192718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6ef5fdbe0 0x55e6ef5fdfc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.669+0000 7f2f3bf78b80 1 --2- 172.21.15.2:0/4294192718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e6ef5fe500 0x55e6ef5f9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.669+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/4294192718 >> 172.21.15.2:0/4294192718 conn(0x55e6ef557c20 msgr2=0x55e6ef5f3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.670+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/4294192718 shutdown_connections 2023-09-13T04:41:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.670+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/4294192718 wait complete. 2023-09-13T04:41:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.671+0000 7f2f3bf78b80 1 -- start start 2023-09-13T04:41:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.671+0000 7f2f3bf78b80 1 Processor -- start 2023-09-13T04:41:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.672+0000 7f2f3bf78b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6ef5f9840 0x55e6ef6028d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.672+0000 7f2f3bf78b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e6ef5fdbe0 0x55e6ef602e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.672+0000 7f2f3bf78b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6ef5fe500 0x55e6ef603350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.672+0000 7f2f3bf78b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ef4e9b50 con 0x55e6ef5fe500 2023-09-13T04:41:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.672+0000 7f2f3bf78b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ef4e9850 con 0x55e6ef5fdbe0 2023-09-13T04:41:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.672+0000 7f2f3bf78b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ef3cd8c0 con 0x55e6ef5f9840 2023-09-13T04:41:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.672+0000 7f2f3bf78b80 10 client.0 _ll_register_callbacks cb 0x55e6ef4e7620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.672+0000 7f2f25b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6ef5f9840 0x55e6ef6028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.672+0000 7f2f2639b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e6ef5fdbe0 0x55e6ef602e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.672+0000 7f2f2639b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e6ef5fdbe0 0x55e6ef602e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:44472/0 (socket says 172.21.15.2:44472) 2023-09-13T04:41:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.672+0000 7f2f2639b700 1 -- 172.21.15.2:0/4263054607 learned_addr learned my addr 172.21.15.2:0/4263054607 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.673+0000 7f2f2639b700 1 -- 172.21.15.2:0/4263054607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6ef5f9840 msgr2=0x55e6ef6028d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.673+0000 7f2f2639b700 1 --2- 172.21.15.2:0/4263054607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6ef5f9840 0x55e6ef6028d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.673+0000 7f2f2639b700 1 -- 172.21.15.2:0/4263054607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6ef5fe500 msgr2=0x55e6ef603350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.673+0000 7f2f2639b700 1 --2- 172.21.15.2:0/4263054607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6ef5fe500 0x55e6ef603350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.673+0000 7f2f2639b700 1 -- 172.21.15.2:0/4263054607 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1c009940 con 0x55e6ef5fdbe0 2023-09-13T04:41:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.673+0000 7f2f2639b700 1 --2- 172.21.15.2:0/4263054607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e6ef5fdbe0 0x55e6ef602e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009990 tx=0x7f2f100057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=570e5837814b4670 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.673+0000 7f2f09ffb700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:41:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.674+0000 7f2f09ffb700 1 -- 172.21.15.2:0/4263054607 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f10004500 con 0x55e6ef5fdbe0 2023-09-13T04:41:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.674+0000 7f2f09ffb700 1 -- 172.21.15.2:0/4263054607 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f2f10004660 con 0x55e6ef5fdbe0 2023-09-13T04:41:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.674+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/4263054607 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6ef5beef0 con 0x55e6ef5fdbe0 2023-09-13T04:41:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.674+0000 7f2f177fe700 1 --2- 172.21.15.2:0/547210932 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f2f0c0a08e0 0x7f2f0c0a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.675+0000 7f2f177fe700 1 -- 172.21.15.2:0/547210932 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f2f180cbc30 con 0x7f2f20106d60 2023-09-13T04:41:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.675+0000 7f2f25b9a700 1 --2- 172.21.15.2:0/547210932 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f2f0c0a08e0 0x7f2f0c0a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.676+0000 7f2f09ffb700 1 -- 172.21.15.2:0/4263054607 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f10004820 con 0x55e6ef5fdbe0 2023-09-13T04:41:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.676+0000 7f2f3bf78b80 10 client.25829 fetch_fsmap learned FSMap version 23 2023-09-13T04:41:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.676+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/4263054607 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e6ef563740 con 0x55e6ef5fdbe0 2023-09-13T04:41:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.676+0000 7f2f09ffb700 1 -- 172.21.15.2:0/4263054607 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e6ef5beef0 con 0x55e6ef5fdbe0 2023-09-13T04:41:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.677+0000 7f2f25b9a700 1 --2- 172.21.15.2:0/547210932 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f2f0c0a08e0 0x7f2f0c0a2da0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c003ab0 tx=0x7f2f1c013040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.678+0000 7f2f09ffb700 1 -- 172.21.15.2:0/4263054607 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2f1002c070 con 0x55e6ef5fdbe0 2023-09-13T04:41:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.678+0000 7f2f3bf78b80 10 client.25829 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:41:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.678+0000 7f2f3bf78b80 10 client.25829 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:41:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.678+0000 7f2f3bf78b80 20 client.25829 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.678+0000 7f2f3bf78b80 1 --2- 172.21.15.2:0/4263054607 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55e6ef612ff0 0x55e6ef615410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.678+0000 7f2f3bf78b80 4 client.25829 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:41:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.678+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/4263054607 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e6ef544910 con 0x55e6ef612ff0 2023-09-13T04:41:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.678+0000 7f2f25b9a700 1 --2- 172.21.15.2:0/4263054607 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55e6ef612ff0 0x55e6ef615410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.679+0000 7f2f25b9a700 1 --2- 172.21.15.2:0/4263054607 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55e6ef612ff0 0x55e6ef615410 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de0e69cd1052322b server_cookie=e7c62a8b67116eb4 in_seq=0 out_seq=0 2023-09-13T04:41:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.679+0000 7f2f09ffb700 10 client.25829 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.680+0000 7f2f09ffb700 1 -- 172.21.15.2:0/4263054607 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e6ef544910 con 0x55e6ef612ff0 2023-09-13T04:41:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.680+0000 7f2f09ffb700 10 client.25829 handle_command_reply: tid=0 2023-09-13T04:41:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.680+0000 7f2f3bf78b80 1 client.25829 shutdown 2023-09-13T04:41:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.681+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/4263054607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e6ef5fdbe0 msgr2=0x55e6ef602e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.681+0000 7f2f3bf78b80 1 --2- 172.21.15.2:0/4263054607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e6ef5fdbe0 0x55e6ef602e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009990 tx=0x7f2f100057c0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.681+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/4263054607 shutdown_connections 2023-09-13T04:41:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.681+0000 7f2f3bf78b80 1 --2- 172.21.15.2:0/4263054607 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55e6ef612ff0 0x55e6ef615410 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.681+0000 7f2f3bf78b80 1 --2- 172.21.15.2:0/4263054607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6ef5fe500 0x55e6ef603350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.681+0000 7f2f3bf78b80 1 --2- 172.21.15.2:0/4263054607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6ef5f9840 0x55e6ef6028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.681+0000 7f2f3bf78b80 1 --2- 172.21.15.2:0/4263054607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e6ef5fdbe0 0x55e6ef602e10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.681+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/4263054607 >> 172.21.15.2:0/4263054607 conn(0x55e6ef557c20 msgr2=0x55e6ef5ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.681+0000 7f2f09ffb700 0 client.25829 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.681+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/4263054607 shutdown_connections 2023-09-13T04:41:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.681+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/4263054607 wait complete. 2023-09-13T04:41:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.682+0000 7f2f3bf78b80 20 client.25829 trim_cache size 0 max 16384 2023-09-13T04:41:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.712+0000 7f2f3bf78b80 1 Processor -- start 2023-09-13T04:41:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.712+0000 7f2f3bf78b80 1 -- start start 2023-09-13T04:41:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.712+0000 7f2f3bf78b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e6ef5f9840 0x55e6ef609fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.712+0000 7f2f3bf78b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6ef5fdbe0 0x55e6ef5ee730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.713+0000 7f2f3bf78b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6ef5fe500 0x55e6ef5f8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.713+0000 7f2f3bf78b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ef5f4100 con 0x55e6ef5fdbe0 2023-09-13T04:41:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.713+0000 7f2f3bf78b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ef60a510 con 0x55e6ef5f9840 2023-09-13T04:41:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.713+0000 7f2f3bf78b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ef5f8470 con 0x55e6ef5fe500 2023-09-13T04:41:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.713+0000 7f2f26b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6ef5fe500 0x55e6ef5f8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.713+0000 7f2f26b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6ef5fe500 0x55e6ef5f8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:37346/0 (socket says 172.21.15.2:37346) 2023-09-13T04:41:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.713+0000 7f2f26b9c700 1 -- 172.21.15.2:0/1744644012 learned_addr learned my addr 172.21.15.2:0/1744644012 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.713+0000 7f2f26b9c700 1 -- 172.21.15.2:0/1744644012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e6ef5f9840 msgr2=0x55e6ef609fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:41:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.713+0000 7f2f26b9c700 1 --2- 172.21.15.2:0/1744644012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e6ef5f9840 0x55e6ef609fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.713+0000 7f2f26b9c700 1 -- 172.21.15.2:0/1744644012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6ef5fdbe0 msgr2=0x55e6ef5ee730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.713+0000 7f2f26b9c700 1 --2- 172.21.15.2:0/1744644012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6ef5fdbe0 0x55e6ef5ee730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.714+0000 7f2f26b9c700 1 -- 172.21.15.2:0/1744644012 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6ef377d80 con 0x55e6ef5fe500 2023-09-13T04:41:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.714+0000 7f2f26b9c700 1 --2- 172.21.15.2:0/1744644012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6ef5fe500 0x55e6ef5f8880 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2f18095620 tx=0x7f2f18098980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67c8ddcbc990f206 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.714+0000 7f2f153ff700 1 -- 172.21.15.2:0/1744644012 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f180992b0 con 0x55e6ef5fe500 2023-09-13T04:41:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.714+0000 7f2f153ff700 1 -- 172.21.15.2:0/1744644012 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f18098bb0 con 0x55e6ef5fe500 2023-09-13T04:41:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.714+0000 7f2f153ff700 1 -- 172.21.15.2:0/1744644012 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f18098e10 con 0x55e6ef5fe500 2023-09-13T04:41:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.715+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/1744644012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6ef5fe500 msgr2=0x55e6ef5f8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.715+0000 7f2f3bf78b80 1 --2- 172.21.15.2:0/1744644012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6ef5fe500 0x55e6ef5f8880 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2f18095620 tx=0x7f2f18098980 comp rx=0 tx=0).stop 2023-09-13T04:41:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.716+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/1744644012 shutdown_connections 2023-09-13T04:41:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.716+0000 7f2f3bf78b80 1 --2- 172.21.15.2:0/1744644012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6ef5fe500 0x55e6ef5f8880 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.716+0000 7f2f3bf78b80 1 --2- 172.21.15.2:0/1744644012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e6ef5f9840 0x55e6ef609fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.716+0000 7f2f3bf78b80 1 --2- 172.21.15.2:0/1744644012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6ef5fdbe0 0x55e6ef5ee730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.716+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/1744644012 >> 172.21.15.2:0/1744644012 conn(0x55e6ef557c20 msgr2=0x55e6ef5f7970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.716+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/1744644012 shutdown_connections 2023-09-13T04:41:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.716+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/1744644012 wait complete. 2023-09-13T04:41:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.716+0000 7f2f3bf78b80 1 -- start start 2023-09-13T04:41:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.717+0000 7f2f3bf78b80 1 Processor -- start 2023-09-13T04:41:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.717+0000 7f2f3bf78b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6ef5f9840 0x55e6ef6081b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.717+0000 7f2f3bf78b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6ef5fdbe0 0x55e6ef6086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.718+0000 7f2f3bf78b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e6ef5fe500 0x55e6ef66fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.718+0000 7f2f3bf78b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ef6703a0 con 0x55e6ef5f9840 2023-09-13T04:41:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.718+0000 7f2f3bf78b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ef670510 con 0x55e6ef5fe500 2023-09-13T04:41:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.718+0000 7f2f3bf78b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ef6706d0 con 0x55e6ef5fdbe0 2023-09-13T04:41:23.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.718+0000 7f2f3bf78b80 10 client.0 _ll_register_callbacks cb 0x55e6ef4e7620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:23.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.718+0000 7f2f2639b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6ef5f9840 0x55e6ef6081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.718+0000 7f2f2639b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6ef5f9840 0x55e6ef6081b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45310/0 (socket says 172.21.15.2:45310) 2023-09-13T04:41:23.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.718+0000 7f2f2639b700 1 -- 172.21.15.2:0/2725722579 learned_addr learned my addr 172.21.15.2:0/2725722579 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:23.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.719+0000 7f2f2639b700 1 -- 172.21.15.2:0/2725722579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6ef5fdbe0 msgr2=0x55e6ef6086f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:41:23.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.719+0000 7f2f2639b700 1 --2- 172.21.15.2:0/2725722579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6ef5fdbe0 0x55e6ef6086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.719+0000 7f2f2639b700 1 -- 172.21.15.2:0/2725722579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e6ef5fe500 msgr2=0x55e6ef66fdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:41:23.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.719+0000 7f2f2639b700 1 --2- 172.21.15.2:0/2725722579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e6ef5fe500 0x55e6ef66fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.719+0000 7f2f2639b700 1 -- 172.21.15.2:0/2725722579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f20197a40 con 0x55e6ef5f9840 2023-09-13T04:41:23.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.719+0000 7f2f2639b700 1 --2- 172.21.15.2:0/2725722579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6ef5f9840 0x55e6ef6081b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2f100098e0 tx=0x7f2f10005470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68beb0ae5b89cffb server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:23.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.719+0000 7f2f09ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:41:23.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.720+0000 7f2f09ffb700 1 -- 172.21.15.2:0/2725722579 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1000f9e0 con 0x55e6ef5f9840 2023-09-13T04:41:23.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.720+0000 7f2f09ffb700 1 -- 172.21.15.2:0/2725722579 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f2f10000e40 con 0x55e6ef5f9840 2023-09-13T04:41:23.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.720+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/2725722579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f1c0044f0 con 0x55e6ef5f9840 2023-09-13T04:41:23.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.720+0000 7f2f09ffb700 1 -- 172.21.15.2:0/2725722579 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f1000fb40 con 0x55e6ef5f9840 2023-09-13T04:41:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.720+0000 7f2f3bf78b80 10 client.15801 fetch_fsmap learned FSMap version 23 2023-09-13T04:41:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.720+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/2725722579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e6ef360c00 con 0x55e6ef5f9840 2023-09-13T04:41:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.721+0000 7f2f09ffb700 1 -- 172.21.15.2:0/2725722579 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f1c0044f0 con 0x55e6ef5f9840 2023-09-13T04:41:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.721+0000 7f2f09ffb700 1 -- 172.21.15.2:0/2725722579 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2f1002c070 con 0x55e6ef5f9840 2023-09-13T04:41:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.721+0000 7f2f3bf78b80 10 client.15801 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:41:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.721+0000 7f2f3bf78b80 10 client.15801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:41:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.721+0000 7f2f3bf78b80 20 client.15801 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.721+0000 7f2f3bf78b80 1 --2- 172.21.15.2:0/2725722579 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55e6ef678bf0 0x55e6ef67afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.721+0000 7f2f3bf78b80 4 client.15801 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:41:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.721+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/2725722579 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e6ef504ff0 con 0x55e6ef678bf0 2023-09-13T04:41:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.722+0000 7f2f25b9a700 1 --2- 172.21.15.2:0/2725722579 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55e6ef678bf0 0x55e6ef67afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.724+0000 7f2f25b9a700 1 --2- 172.21.15.2:0/2725722579 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55e6ef678bf0 0x55e6ef67afe0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=269415436db05ae2 server_cookie=985bb71e7939f224 in_seq=0 out_seq=0 2023-09-13T04:41:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.724+0000 7f2f09ffb700 10 client.15801 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:23.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.726+0000 7fcd16bff700 1 -- 172.21.15.2:0/3123459329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd100ffbe0 msgr2=0x7fcd100fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.726+0000 7fcd16bff700 1 --2- 172.21.15.2:0/3123459329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd100ffbe0 0x7fcd100fffc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800b0a0 tx=0x7fcd0800b3b0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.726+0000 7fcd16bff700 1 -- 172.21.15.2:0/3123459329 shutdown_connections 2023-09-13T04:41:23.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.726+0000 7fcd16bff700 1 --2- 172.21.15.2:0/3123459329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd10104fe0 0x7fcd10105490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.726+0000 7fcd16bff700 1 --2- 172.21.15.2:0/3123459329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd100ffbe0 0x7fcd100fffc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.726+0000 7fcd16bff700 1 --2- 172.21.15.2:0/3123459329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd10100500 0x7fcd101049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.726+0000 7fcd16bff700 1 -- 172.21.15.2:0/3123459329 >> 172.21.15.2:0/3123459329 conn(0x7fcd100fb630 msgr2=0x7fcd100fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:23.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.726+0000 7fcd16bff700 1 -- 172.21.15.2:0/3123459329 shutdown_connections 2023-09-13T04:41:23.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.726+0000 7fcd16bff700 1 -- 172.21.15.2:0/3123459329 wait complete. 2023-09-13T04:41:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.728+0000 7f2f09ffb700 1 -- 172.21.15.2:0/2725722579 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e6ef504ff0 con 0x55e6ef678bf0 2023-09-13T04:41:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.728+0000 7f2f09ffb700 10 client.15801 handle_command_reply: tid=0 2023-09-13T04:41:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.728+0000 7f2f3bf78b80 1 client.15801 shutdown 2023-09-13T04:41:23.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.728+0000 7fcd16bff700 1 Processor -- start 2023-09-13T04:41:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.728+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/2725722579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6ef5f9840 msgr2=0x55e6ef6081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.728+0000 7f2f3bf78b80 1 --2- 172.21.15.2:0/2725722579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6ef5f9840 0x55e6ef6081b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2f100098e0 tx=0x7f2f10005470 comp rx=0 tx=0).stop 2023-09-13T04:41:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.728+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/2725722579 shutdown_connections 2023-09-13T04:41:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.728+0000 7f2f3bf78b80 1 --2- 172.21.15.2:0/2725722579 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55e6ef678bf0 0x55e6ef67afe0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.728+0000 7f2f3bf78b80 1 --2- 172.21.15.2:0/2725722579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e6ef5fe500 0x55e6ef66fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.728+0000 7f2f3bf78b80 1 --2- 172.21.15.2:0/2725722579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6ef5f9840 0x55e6ef6081b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.728+0000 7f2f3bf78b80 1 --2- 172.21.15.2:0/2725722579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6ef5fdbe0 0x55e6ef6086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.728+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/2725722579 >> 172.21.15.2:0/2725722579 conn(0x55e6ef557c20 msgr2=0x55e6ef5f8880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:23.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.728+0000 7f2f09ffb700 0 client.15801 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:23.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.729+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/2725722579 shutdown_connections 2023-09-13T04:41:23.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.729+0000 7f2f3bf78b80 1 -- 172.21.15.2:0/2725722579 wait complete. 2023-09-13T04:41:23.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.728+0000 7fcd16bff700 1 -- start start 2023-09-13T04:41:23.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.729+0000 7fcd16bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd100ffbe0 0x7fcd100722b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.729+0000 7fcd16bff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd10100500 0x7fcd10072830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.729+0000 7fcd16bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd10104fe0 0x7fcd1006b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.729+0000 7fcd16bff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1006bbb0 con 0x7fcd10100500 2023-09-13T04:41:23.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.729+0000 7fcd16bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1006bd20 con 0x7fcd100ffbe0 2023-09-13T04:41:23.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.729+0000 7fcd16bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1006bee0 con 0x7fcd10104fe0 2023-09-13T04:41:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.729+0000 7f2f3bf78b80 20 client.15801 trim_cache size 0 max 16384 2023-09-13T04:41:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:41:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:41:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:41:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:41:23.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.729+0000 7fcd1499b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd100ffbe0 0x7fcd100722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.730+0000 7fcd1519c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd10104fe0 0x7fcd1006b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.730+0000 7fcd1499b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd100ffbe0 0x7fcd100722b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:44502/0 (socket says 172.21.15.2:44502) 2023-09-13T04:41:23.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.730+0000 7fcd1499b700 1 -- 172.21.15.2:0/1304134001 learned_addr learned my addr 172.21.15.2:0/1304134001 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:23.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.730+0000 7fcd0ffff700 1 --2- 172.21.15.2:0/1304134001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd10100500 0x7fcd10072830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.730+0000 7fcd1499b700 1 -- 172.21.15.2:0/1304134001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd10104fe0 msgr2=0x7fcd1006b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.730+0000 7fcd1499b700 1 --2- 172.21.15.2:0/1304134001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd10104fe0 0x7fcd1006b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.730+0000 7fcd1499b700 1 -- 172.21.15.2:0/1304134001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd10100500 msgr2=0x7fcd10072830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.730+0000 7fcd1499b700 1 --2- 172.21.15.2:0/1304134001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd10100500 0x7fcd10072830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.730+0000 7fcd1499b700 1 -- 172.21.15.2:0/1304134001 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd00009520 con 0x7fcd100ffbe0 2023-09-13T04:41:23.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.730+0000 7fcd1519c700 1 --2- 172.21.15.2:0/1304134001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd10104fe0 0x7fcd1006b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:41:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.731+0000 7f2f285ff700 1 -- 172.21.15.2:0/547210932 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f2f0c0a08e0 msgr2=0x7f2f0c0a2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.731+0000 7f2f285ff700 1 --2- 172.21.15.2:0/547210932 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f2f0c0a08e0 0x7f2f0c0a2da0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c003ab0 tx=0x7f2f1c013040 comp rx=0 tx=0).stop 2023-09-13T04:41:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.731+0000 7f2f285ff700 1 -- 172.21.15.2:0/547210932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f20106d60 msgr2=0x7f2f2019a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.731+0000 7f2f285ff700 1 --2- 172.21.15.2:0/547210932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f20106d60 0x7f2f2019a930 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800efc0 tx=0x7f2f1800c3a0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.731+0000 7fcd1499b700 1 --2- 172.21.15.2:0/1304134001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd100ffbe0 0x7fcd100722b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800b070 tx=0x7fcd0800bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:23.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.731+0000 7fcd0dffb700 1 -- 172.21.15.2:0/1304134001 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0800e040 con 0x7fcd100ffbe0 2023-09-13T04:41:23.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.731+0000 7fcd0dffb700 1 -- 172.21.15.2:0/1304134001 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd08009480 con 0x7fcd100ffbe0 2023-09-13T04:41:23.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.731+0000 7fcd0dffb700 1 -- 172.21.15.2:0/1304134001 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0801b6c0 con 0x7fcd100ffbe0 2023-09-13T04:41:23.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.731+0000 7fcd16bff700 1 -- 172.21.15.2:0/1304134001 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd08009d00 con 0x7fcd100ffbe0 2023-09-13T04:41:23.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.732+0000 7fcd16bff700 1 -- 172.21.15.2:0/1304134001 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd10070020 con 0x7fcd100ffbe0 2023-09-13T04:41:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.732+0000 7f2f285ff700 1 -- 172.21.15.2:0/547210932 shutdown_connections 2023-09-13T04:41:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.732+0000 7f2f285ff700 1 --2- 172.21.15.2:0/547210932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f20106d60 0x7f2f2019a930 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.732+0000 7f2f285ff700 1 --2- 172.21.15.2:0/547210932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f200680d0 0x7f2f2010b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.732+0000 7f2f285ff700 1 --2- 172.21.15.2:0/547210932 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f2f0c0a08e0 0x7f2f0c0a2da0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.732+0000 7f2f285ff700 1 --2- 172.21.15.2:0/547210932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f20068a90 0x7f2f2010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.732+0000 7f2f285ff700 1 -- 172.21.15.2:0/547210932 >> 172.21.15.2:0/547210932 conn(0x7f2f20076970 msgr2=0x7f2f20100950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.732+0000 7f2f285ff700 1 -- 172.21.15.2:0/547210932 shutdown_connections 2023-09-13T04:41:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:23.733+0000 7f2f285ff700 1 -- 172.21.15.2:0/547210932 wait complete. 2023-09-13T04:41:23.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.737+0000 7fcd0dffb700 1 -- 172.21.15.2:0/1304134001 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 24) v1 ==== 135491+0+0 (secure 0 0 0) 0x7fcd0801d470 con 0x7fcd100ffbe0 2023-09-13T04:41:23.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.739+0000 7fcd2a5d3b80 1 Processor -- start 2023-09-13T04:41:23.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.739+0000 7fcd2a5d3b80 1 -- start start 2023-09-13T04:41:23.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.740+0000 7fcd2a5d3b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e25ff31fe0 0x55e25ff323c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.740+0000 7fcd2a5d3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e25ff32900 0x55e25ff2c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.740+0000 7fcd2a5d3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e25ff2ca00 0x55e25ff2ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.740+0000 7fcd2a5d3b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25fe85780 con 0x55e25ff31fe0 2023-09-13T04:41:23.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.740+0000 7fcd2a5d3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25fe830a0 con 0x55e25ff32900 2023-09-13T04:41:23.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.740+0000 7fcd2a5d3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25fe7e990 con 0x55e25ff2ca00 2023-09-13T04:41:23.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.740+0000 7fcd0dffb700 1 --2- 172.21.15.2:0/1304134001 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fccf80a09d0 0x7fccf80a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.741+0000 7fcd0dffb700 1 -- 172.21.15.2:0/1304134001 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fcd080cd680 con 0x7fcd100ffbe0 2023-09-13T04:41:23.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.741+0000 7fcd0ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e25ff2ca00 0x55e25ff2ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.741+0000 7fcd1499b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e25ff32900 0x55e25ff2c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.741+0000 7fcd1499b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e25ff32900 0x55e25ff2c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:44512/0 (socket says 172.21.15.2:44512) 2023-09-13T04:41:23.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.741+0000 7fcd1499b700 1 -- 172.21.15.2:0/2674436604 learned_addr learned my addr 172.21.15.2:0/2674436604 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:23.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.741+0000 7fcd1519c700 1 --2- 172.21.15.2:0/2674436604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e25ff31fe0 0x55e25ff323c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.742+0000 7fcd1519c700 1 --2- 172.21.15.2:0/1304134001 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fccf80a09d0 0x7fccf80a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.742+0000 7fcd1499b700 1 -- 172.21.15.2:0/2674436604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e25ff2ca00 msgr2=0x55e25ff2ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:23.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.742+0000 7fcd1499b700 1 --2- 172.21.15.2:0/2674436604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e25ff2ca00 0x55e25ff2ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.742+0000 7fcd1499b700 1 -- 172.21.15.2:0/2674436604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e25ff31fe0 msgr2=0x55e25ff323c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:23.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.742+0000 7fcd1499b700 1 --2- 172.21.15.2:0/2674436604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e25ff31fe0 0x55e25ff323c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.742+0000 7fcd1499b700 1 -- 172.21.15.2:0/2674436604 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e25fcfe630 con 0x55e25ff32900 2023-09-13T04:41:23.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.742+0000 7fcd1519c700 1 --2- 172.21.15.2:0/2674436604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e25ff31fe0 0x55e25ff323c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:41:23.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.742+0000 7fcd0ffff700 1 --2- 172.21.15.2:0/2674436604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e25ff2ca00 0x55e25ff2ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:23.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.742+0000 7fcd1499b700 1 --2- 172.21.15.2:0/2674436604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e25ff32900 0x55e25ff2c4c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcd0809a590 tx=0x7fcd0801dbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=387ea7fea494c9f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:23.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.742+0000 7fccf6ffd700 1 -- 172.21.15.2:0/2674436604 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08099f60 con 0x55e25ff32900 2023-09-13T04:41:23.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.742+0000 7fccf6ffd700 1 -- 172.21.15.2:0/2674436604 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd0801aea0 con 0x55e25ff32900 2023-09-13T04:41:23.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.742+0000 7fccf6ffd700 1 -- 172.21.15.2:0/2674436604 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08098a30 con 0x55e25ff32900 2023-09-13T04:41:23.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.743+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/2674436604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e25ff32900 msgr2=0x55e25ff2c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.743+0000 7fcd2a5d3b80 1 --2- 172.21.15.2:0/2674436604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e25ff32900 0x55e25ff2c4c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcd0809a590 tx=0x7fcd0801dbf0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.744+0000 7fcd1519c700 1 --2- 172.21.15.2:0/1304134001 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fccf80a09d0 0x7fccf80a2e90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcd000125f0 tx=0x7fcd00009260 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:23.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.744+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/2674436604 shutdown_connections 2023-09-13T04:41:23.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.744+0000 7fcd2a5d3b80 1 --2- 172.21.15.2:0/2674436604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e25ff2ca00 0x55e25ff2ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.744+0000 7fcd2a5d3b80 1 --2- 172.21.15.2:0/2674436604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e25ff31fe0 0x55e25ff323c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.744+0000 7fcd2a5d3b80 1 --2- 172.21.15.2:0/2674436604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e25ff32900 0x55e25ff2c4c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.744+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/2674436604 >> 172.21.15.2:0/2674436604 conn(0x55e25fe8ac80 msgr2=0x55e25ff26f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:23.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.744+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/2674436604 shutdown_connections 2023-09-13T04:41:23.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.744+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/2674436604 wait complete. 2023-09-13T04:41:23.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.745+0000 7fcd2a5d3b80 1 -- start start 2023-09-13T04:41:23.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.745+0000 7fcd2a5d3b80 1 Processor -- start 2023-09-13T04:41:23.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.746+0000 7fcd2a5d3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e25ff2ca00 0x55e25ff38930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.746+0000 7fcd2a5d3b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e25ff31fe0 0x55e25ff35c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.746+0000 7fcd2a5d3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e25ff32900 0x55e25ff36360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.746+0000 7fcd2a5d3b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25fe1cbb0 con 0x55e25ff31fe0 2023-09-13T04:41:23.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.746+0000 7fcd2a5d3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25fe1c8b0 con 0x55e25ff32900 2023-09-13T04:41:23.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.746+0000 7fcd2a5d3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25fd00920 con 0x55e25ff2ca00 2023-09-13T04:41:23.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.746+0000 7fcd2a5d3b80 10 client.0 _ll_register_callbacks cb 0x55e25fe1a680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:23.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.747+0000 7fcd1499b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e25ff2ca00 0x55e25ff38930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.747+0000 7fcd1499b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e25ff2ca00 0x55e25ff38930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:37390/0 (socket says 172.21.15.2:37390) 2023-09-13T04:41:23.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.747+0000 7fcd1499b700 1 -- 172.21.15.2:0/3625644483 learned_addr learned my addr 172.21.15.2:0/3625644483 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:23.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.747+0000 7fcd1519c700 1 --2- 172.21.15.2:0/3625644483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e25ff32900 0x55e25ff36360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.747+0000 7fcd0ffff700 1 --2- 172.21.15.2:0/3625644483 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e25ff31fe0 0x55e25ff35c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.747+0000 7fcd1499b700 1 -- 172.21.15.2:0/3625644483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e25ff32900 msgr2=0x55e25ff36360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:23.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.747+0000 7fcd1499b700 1 --2- 172.21.15.2:0/3625644483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e25ff32900 0x55e25ff36360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.747+0000 7fcd1499b700 1 -- 172.21.15.2:0/3625644483 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e25ff31fe0 msgr2=0x55e25ff35c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:23.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.747+0000 7fcd1499b700 1 --2- 172.21.15.2:0/3625644483 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e25ff31fe0 0x55e25ff35c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.747+0000 7fcd1499b700 1 -- 172.21.15.2:0/3625644483 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd10070020 con 0x55e25ff2ca00 2023-09-13T04:41:23.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.748+0000 7fcd1499b700 1 --2- 172.21.15.2:0/3625644483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e25ff2ca00 0x55e25ff38930 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcd080ca9f0 tx=0x7fcd0809a010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a61fb3e83e45ac0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:23.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.748+0000 7fcceffff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:41:23.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.748+0000 7fcceffff700 1 -- 172.21.15.2:0/3625644483 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd080852c0 con 0x55e25ff2ca00 2023-09-13T04:41:23.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.748+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/3625644483 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e25ff1f670 con 0x55e25ff2ca00 2023-09-13T04:41:23.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.748+0000 7fcceffff700 1 -- 172.21.15.2:0/3625644483 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fcd0809e430 con 0x55e25ff2ca00 2023-09-13T04:41:23.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.749+0000 7fcceffff700 1 -- 172.21.15.2:0/3625644483 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd0809ad00 con 0x55e25ff2ca00 2023-09-13T04:41:23.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.749+0000 7fcd2a5d3b80 10 client.25855 fetch_fsmap learned FSMap version 23 2023-09-13T04:41:23.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.749+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/3625644483 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e25fe967a0 con 0x55e25ff2ca00 2023-09-13T04:41:23.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.750+0000 7fcceffff700 1 -- 172.21.15.2:0/3625644483 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e25ff1f670 con 0x55e25ff2ca00 2023-09-13T04:41:23.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.750+0000 7fcceffff700 1 -- 172.21.15.2:0/3625644483 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcd080a2150 con 0x55e25ff2ca00 2023-09-13T04:41:23.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.750+0000 7fcd2a5d3b80 10 client.25855 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:41:23.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.750+0000 7fcd2a5d3b80 10 client.25855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:41:23.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.750+0000 7fcd2a5d3b80 20 client.25855 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:23.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.751+0000 7fcd2a5d3b80 1 --2- 172.21.15.2:0/3625644483 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55e25ff45a40 0x55e25ff47e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.751+0000 7fcd2a5d3b80 4 client.25855 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:41:23.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.751+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/3625644483 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e25fe77970 con 0x55e25ff45a40 2023-09-13T04:41:23.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.751+0000 7fcd0ffff700 1 --2- 172.21.15.2:0/3625644483 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55e25ff45a40 0x55e25ff47e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.752+0000 7fcceffff700 10 client.25855 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:23.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.752+0000 7fcd0ffff700 1 --2- 172.21.15.2:0/3625644483 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55e25ff45a40 0x55e25ff47e30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ede80c9483ac540 server_cookie=21367e087c3f6d53 in_seq=0 out_seq=0 2023-09-13T04:41:23.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.753+0000 7fcceffff700 1 -- 172.21.15.2:0/3625644483 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e25fe77970 con 0x55e25ff45a40 2023-09-13T04:41:23.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.753+0000 7fcceffff700 10 client.25855 handle_command_reply: tid=0 2023-09-13T04:41:23.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.753+0000 7fcd2a5d3b80 1 client.25855 shutdown 2023-09-13T04:41:23.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.753+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/3625644483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e25ff2ca00 msgr2=0x55e25ff38930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.753+0000 7fcd2a5d3b80 1 --2- 172.21.15.2:0/3625644483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e25ff2ca00 0x55e25ff38930 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcd080ca9f0 tx=0x7fcd0809a010 comp rx=0 tx=0).stop 2023-09-13T04:41:23.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.754+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/3625644483 shutdown_connections 2023-09-13T04:41:23.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.754+0000 7fcd2a5d3b80 1 --2- 172.21.15.2:0/3625644483 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55e25ff45a40 0x55e25ff47e30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.754+0000 7fcd2a5d3b80 1 --2- 172.21.15.2:0/3625644483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e25ff32900 0x55e25ff36360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.754+0000 7fcceffff700 0 client.25855 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:23.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.754+0000 7fcd2a5d3b80 1 --2- 172.21.15.2:0/3625644483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e25ff2ca00 0x55e25ff38930 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.754+0000 7fcd2a5d3b80 1 --2- 172.21.15.2:0/3625644483 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e25ff31fe0 0x55e25ff35c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.754+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/3625644483 >> 172.21.15.2:0/3625644483 conn(0x55e25fe8ac80 msgr2=0x55e25ff217f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:23.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.754+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/3625644483 shutdown_connections 2023-09-13T04:41:23.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.754+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/3625644483 wait complete. 2023-09-13T04:41:23.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.754+0000 7fcd2a5d3b80 20 client.25855 trim_cache size 0 max 16384 2023-09-13T04:41:23.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.781+0000 7fcd2a5d3b80 1 Processor -- start 2023-09-13T04:41:23.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.781+0000 7fcd2a5d3b80 1 -- start start 2023-09-13T04:41:23.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.781+0000 7fcd2a5d3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e25ff2ca00 0x55e25ff3d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.781+0000 7fcd2a5d3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e25ff31fe0 0x55e25ff218f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.781+0000 7fcd2a5d3b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e25ff32900 0x55e25ff384d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.781+0000 7fcd2a5d3b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25ff274f0 con 0x55e25ff32900 2023-09-13T04:41:23.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.781+0000 7fcd2a5d3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25ff36960 con 0x55e25ff2ca00 2023-09-13T04:41:23.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.781+0000 7fcd2a5d3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25ff27fb0 con 0x55e25ff31fe0 2023-09-13T04:41:23.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.782+0000 7fcd1499b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e25ff2ca00 0x55e25ff3d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.782+0000 7fcd1499b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e25ff2ca00 0x55e25ff3d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:44530/0 (socket says 172.21.15.2:44530) 2023-09-13T04:41:23.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.782+0000 7fcd1499b700 1 -- 172.21.15.2:0/971999235 learned_addr learned my addr 172.21.15.2:0/971999235 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:23.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.782+0000 7fcd1519c700 1 --2- 172.21.15.2:0/971999235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e25ff32900 0x55e25ff384d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.782+0000 7fcd1519c700 1 -- 172.21.15.2:0/971999235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e25ff31fe0 msgr2=0x55e25ff218f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:41:23.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.782+0000 7fcd1519c700 1 --2- 172.21.15.2:0/971999235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e25ff31fe0 0x55e25ff218f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.782+0000 7fcd1519c700 1 -- 172.21.15.2:0/971999235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e25ff2ca00 msgr2=0x55e25ff3d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:23.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.782+0000 7fcd1519c700 1 --2- 172.21.15.2:0/971999235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e25ff2ca00 0x55e25ff3d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.782+0000 7fcd1519c700 1 -- 172.21.15.2:0/971999235 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e25fcaade0 con 0x55e25ff32900 2023-09-13T04:41:23.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.783+0000 7fcd1519c700 1 --2- 172.21.15.2:0/971999235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e25ff32900 0x55e25ff384d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcd00009ae0 tx=0x7fcd00009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ee71bec52728d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:23.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.783+0000 7fccf77fe700 1 -- 172.21.15.2:0/971999235 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0001ea80 con 0x55e25ff32900 2023-09-13T04:41:23.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.783+0000 7fccf77fe700 1 -- 172.21.15.2:0/971999235 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd00004cf0 con 0x55e25ff32900 2023-09-13T04:41:23.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.783+0000 7fccf77fe700 1 -- 172.21.15.2:0/971999235 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd00005690 con 0x55e25ff32900 2023-09-13T04:41:23.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.784+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/971999235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e25ff32900 msgr2=0x55e25ff384d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.784+0000 7fcd2a5d3b80 1 --2- 172.21.15.2:0/971999235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e25ff32900 0x55e25ff384d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcd00009ae0 tx=0x7fcd00009830 comp rx=0 tx=0).stop 2023-09-13T04:41:23.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.784+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/971999235 shutdown_connections 2023-09-13T04:41:23.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.784+0000 7fcd2a5d3b80 1 --2- 172.21.15.2:0/971999235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e25ff32900 0x55e25ff384d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.784+0000 7fcd2a5d3b80 1 --2- 172.21.15.2:0/971999235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e25ff2ca00 0x55e25ff3d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.784+0000 7fcd2a5d3b80 1 --2- 172.21.15.2:0/971999235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e25ff31fe0 0x55e25ff218f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.784+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/971999235 >> 172.21.15.2:0/971999235 conn(0x55e25fe8ac80 msgr2=0x55e25ff422a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:23.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.785+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/971999235 shutdown_connections 2023-09-13T04:41:23.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.785+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/971999235 wait complete. 2023-09-13T04:41:23.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.785+0000 7fcd2a5d3b80 1 -- start start 2023-09-13T04:41:23.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.786+0000 7fcd2a5d3b80 1 Processor -- start 2023-09-13T04:41:23.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.786+0000 7fcd2a5d3b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e25ff2ca00 0x55e25ff3ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.786+0000 7fcd2a5d3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e25ff31fe0 0x55e25ff3cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.786+0000 7fcd2a5d3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e25ff32900 0x55e25ff3b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.786+0000 7fcd2a5d3b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25ff3b720 con 0x55e25ff2ca00 2023-09-13T04:41:23.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.786+0000 7fcd2a5d3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25ff3b890 con 0x55e25ff31fe0 2023-09-13T04:41:23.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.786+0000 7fcd2a5d3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25ff3b9d0 con 0x55e25ff32900 2023-09-13T04:41:23.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.787+0000 7fcd2a5d3b80 10 client.0 _ll_register_callbacks cb 0x55e25fe1a680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:23.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.787+0000 7fcd1499b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e25ff2ca00 0x55e25ff3ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.787+0000 7fcd1499b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e25ff2ca00 0x55e25ff3ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45332/0 (socket says 172.21.15.2:45332) 2023-09-13T04:41:23.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.787+0000 7fcd1499b700 1 -- 172.21.15.2:0/815819378 learned_addr learned my addr 172.21.15.2:0/815819378 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:23.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.787+0000 7fcd0ffff700 1 --2- 172.21.15.2:0/815819378 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e25ff31fe0 0x55e25ff3cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.787+0000 7fcd1499b700 1 -- 172.21.15.2:0/815819378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e25ff32900 msgr2=0x55e25ff3b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:23.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.787+0000 7fcd1499b700 1 --2- 172.21.15.2:0/815819378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e25ff32900 0x55e25ff3b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.787+0000 7fcd1499b700 1 -- 172.21.15.2:0/815819378 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e25ff31fe0 msgr2=0x55e25ff3cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:23.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.787+0000 7fcd1499b700 1 --2- 172.21.15.2:0/815819378 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e25ff31fe0 0x55e25ff3cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.787+0000 7fcd1499b700 1 -- 172.21.15.2:0/815819378 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd00009bf0 con 0x55e25ff2ca00 2023-09-13T04:41:23.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.788+0000 7fcd0ffff700 1 --2- 172.21.15.2:0/815819378 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e25ff31fe0 0x55e25ff3cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:41:23.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.788+0000 7fcd1499b700 1 --2- 172.21.15.2:0/815819378 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e25ff2ca00 0x55e25ff3ca70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcd0809a300 tx=0x7fcd08003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb4aa791378446e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:23.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.788+0000 7fcceffff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:41:23.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.788+0000 7fcceffff700 1 -- 172.21.15.2:0/815819378 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd080a2360 con 0x55e25ff2ca00 2023-09-13T04:41:23.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.788+0000 7fcceffff700 1 -- 172.21.15.2:0/815819378 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fcd0809ad00 con 0x55e25ff2ca00 2023-09-13T04:41:23.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.788+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/815819378 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd0801aea0 con 0x55e25ff2ca00 2023-09-13T04:41:23.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.789+0000 7fcceffff700 1 -- 172.21.15.2:0/815819378 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd080852c0 con 0x55e25ff2ca00 2023-09-13T04:41:23.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.789+0000 7fcd2a5d3b80 10 client.15819 fetch_fsmap learned FSMap version 23 2023-09-13T04:41:23.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.789+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/815819378 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e25fc93c60 con 0x55e25ff2ca00 2023-09-13T04:41:23.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.789+0000 7fcceffff700 1 -- 172.21.15.2:0/815819378 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd0801aea0 con 0x55e25ff2ca00 2023-09-13T04:41:23.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.789+0000 7fcceffff700 1 -- 172.21.15.2:0/815819378 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcd080cace0 con 0x55e25ff2ca00 2023-09-13T04:41:23.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.789+0000 7fcd2a5d3b80 10 client.15819 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:41:23.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.789+0000 7fcd2a5d3b80 10 client.15819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:41:23.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.789+0000 7fcd2a5d3b80 20 client.15819 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:23.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.790+0000 7fcd2a5d3b80 1 --2- 172.21.15.2:0/815819378 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55e25ffab380 0x55e25ffad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:23.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.790+0000 7fcd2a5d3b80 4 client.15819 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-09-13T04:41:23.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.790+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/815819378 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e25fe38050 con 0x55e25ffab380 2023-09-13T04:41:23.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.790+0000 7fcd0ffff700 1 --2- 172.21.15.2:0/815819378 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55e25ffab380 0x55e25ffad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:23.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.791+0000 7fcd0ffff700 1 --2- 172.21.15.2:0/815819378 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55e25ffab380 0x55e25ffad770 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13bac30345c77f67 server_cookie=45c23f54108f986f in_seq=0 out_seq=0 2023-09-13T04:41:23.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.791+0000 7fcceffff700 10 client.15819 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:23.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.791+0000 7fcceffff700 1 -- 172.21.15.2:0/815819378 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e25fe38050 con 0x55e25ffab380 2023-09-13T04:41:23.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.791+0000 7fcceffff700 10 client.15819 handle_command_reply: tid=0 2023-09-13T04:41:23.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.791+0000 7fcd2a5d3b80 1 client.15819 shutdown 2023-09-13T04:41:23.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.791+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/815819378 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e25ff2ca00 msgr2=0x55e25ff3ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.791+0000 7fcd2a5d3b80 1 --2- 172.21.15.2:0/815819378 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e25ff2ca00 0x55e25ff3ca70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcd0809a300 tx=0x7fcd08003f10 comp rx=0 tx=0).stop 2023-09-13T04:41:23.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.791+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/815819378 shutdown_connections 2023-09-13T04:41:23.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.791+0000 7fcd2a5d3b80 1 --2- 172.21.15.2:0/815819378 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55e25ffab380 0x55e25ffad770 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.791+0000 7fcd2a5d3b80 1 --2- 172.21.15.2:0/815819378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e25ff32900 0x55e25ff3b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.791+0000 7fcd2a5d3b80 1 --2- 172.21.15.2:0/815819378 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e25ff2ca00 0x55e25ff3ca70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.791+0000 7fcd2a5d3b80 1 --2- 172.21.15.2:0/815819378 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e25ff31fe0 0x55e25ff3cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.791+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/815819378 >> 172.21.15.2:0/815819378 conn(0x55e25fe8ac80 msgr2=0x55e25ff32ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:23.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.791+0000 7fcceffff700 0 client.15819 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:23.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.792+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/815819378 shutdown_connections 2023-09-13T04:41:23.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.792+0000 7fcd2a5d3b80 1 -- 172.21.15.2:0/815819378 wait complete. 2023-09-13T04:41:23.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.792+0000 7fcd2a5d3b80 20 client.15819 trim_cache size 0 max 16384 2023-09-13T04:41:23.802 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-09-13T04:41:23.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.793+0000 7fcd16bff700 1 -- 172.21.15.2:0/1304134001 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fccf80a09d0 msgr2=0x7fccf80a2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.793+0000 7fcd16bff700 1 --2- 172.21.15.2:0/1304134001 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fccf80a09d0 0x7fccf80a2e90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcd000125f0 tx=0x7fcd00009260 comp rx=0 tx=0).stop 2023-09-13T04:41:23.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.793+0000 7fcd16bff700 1 -- 172.21.15.2:0/1304134001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd100ffbe0 msgr2=0x7fcd100722b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:23.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.793+0000 7fcd16bff700 1 --2- 172.21.15.2:0/1304134001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd100ffbe0 0x7fcd100722b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800b070 tx=0x7fcd0800bce0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.793+0000 7fcd16bff700 1 -- 172.21.15.2:0/1304134001 shutdown_connections 2023-09-13T04:41:23.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.793+0000 7fcd16bff700 1 --2- 172.21.15.2:0/1304134001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd10104fe0 0x7fcd1006b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.793+0000 7fcd16bff700 1 --2- 172.21.15.2:0/1304134001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd100ffbe0 0x7fcd100722b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.793+0000 7fcd16bff700 1 --2- 172.21.15.2:0/1304134001 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fccf80a09d0 0x7fccf80a2e90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.793+0000 7fcd16bff700 1 --2- 172.21.15.2:0/1304134001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd10100500 0x7fcd10072830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:23.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.793+0000 7fcd16bff700 1 -- 172.21.15.2:0/1304134001 >> 172.21.15.2:0/1304134001 conn(0x7fcd100fb630 msgr2=0x7fcd10102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:23.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.793+0000 7fcd16bff700 1 -- 172.21.15.2:0/1304134001 shutdown_connections 2023-09-13T04:41:23.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:23.793+0000 7fcd16bff700 1 -- 172.21.15.2:0/1304134001 wait complete. 2023-09-13T04:41:24.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:23 smithi149 ceph-mon[133315]: pgmap v31: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 853 B/s wr, 6 op/s 2023-09-13T04:41:24.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:23 smithi156 ceph-mon[133675]: pgmap v31: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 853 B/s wr, 6 op/s 2023-09-13T04:41:24.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:23 smithi002 ceph-mon[97651]: pgmap v31: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 853 B/s wr, 6 op/s 2023-09-13T04:41:24.825 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-13T04:41:24.825 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-13T04:41:25.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.174+0000 7fdbf59ff700 1 -- 172.21.15.2:0/3973961435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbf0105fd0 msgr2=0x7fdbf010db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:25.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.174+0000 7fdbf59ff700 1 --2- 172.21.15.2:0/3973961435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbf0105fd0 0x7fdbf010db50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009f90 tx=0x7fdbe800b330 comp rx=0 tx=0).stop 2023-09-13T04:41:25.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.175+0000 7fdbf59ff700 1 -- 172.21.15.2:0/3973961435 shutdown_connections 2023-09-13T04:41:25.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.175+0000 7fdbf59ff700 1 --2- 172.21.15.2:0/3973961435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbf0105fd0 0x7fdbf010db50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.175+0000 7fdbf59ff700 1 --2- 172.21.15.2:0/3973961435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbf0100eb0 0x7fdbf0101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.175+0000 7fdbf59ff700 1 --2- 172.21.15.2:0/3973961435 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbf0101860 0x7fdbf01058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.175+0000 7fdbf59ff700 1 -- 172.21.15.2:0/3973961435 >> 172.21.15.2:0/3973961435 conn(0x7fdbf00fc730 msgr2=0x7fdbf00feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:25.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.175+0000 7fdbf59ff700 1 -- 172.21.15.2:0/3973961435 shutdown_connections 2023-09-13T04:41:25.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.175+0000 7fdbf59ff700 1 -- 172.21.15.2:0/3973961435 wait complete. 2023-09-13T04:41:25.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.176+0000 7fdbf59ff700 1 Processor -- start 2023-09-13T04:41:25.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.177+0000 7fdbf59ff700 1 -- start start 2023-09-13T04:41:25.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.177+0000 7fdbf59ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbf0100eb0 0x7fdbf0198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.177+0000 7fdbf59ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbf0105fd0 0x7fdbf0199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.177+0000 7fdbf59ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbf019d8d0 0x7fdbf019dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.177+0000 7fdbf59ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf019e350 con 0x7fdbf0100eb0 2023-09-13T04:41:25.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.177+0000 7fdbf59ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf019e4c0 con 0x7fdbf019d8d0 2023-09-13T04:41:25.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.177+0000 7fdbf59ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf019e680 con 0x7fdbf0105fd0 2023-09-13T04:41:25.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.178+0000 7fdbef7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbf019d8d0 0x7fdbf019dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.178+0000 7fdbef7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbf019d8d0 0x7fdbf019dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:60272/0 (socket says 172.21.15.2:60272) 2023-09-13T04:41:25.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.178+0000 7fdbef7fe700 1 -- 172.21.15.2:0/894529437 learned_addr learned my addr 172.21.15.2:0/894529437 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:25.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.178+0000 7fdbeeffd700 1 --2- 172.21.15.2:0/894529437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbf0100eb0 0x7fdbf0198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.178+0000 7fdbee7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbf0105fd0 0x7fdbf0199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.178+0000 7fdbef7fe700 1 -- 172.21.15.2:0/894529437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbf0105fd0 msgr2=0x7fdbf0199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:25.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.178+0000 7fdbef7fe700 1 --2- 172.21.15.2:0/894529437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbf0105fd0 0x7fdbf0199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.178+0000 7fdbef7fe700 1 -- 172.21.15.2:0/894529437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbf0100eb0 msgr2=0x7fdbf0198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:25.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.178+0000 7fdbef7fe700 1 --2- 172.21.15.2:0/894529437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbf0100eb0 0x7fdbf0198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.178+0000 7fdbef7fe700 1 -- 172.21.15.2:0/894529437 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe0009520 con 0x7fdbf019d8d0 2023-09-13T04:41:25.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.179+0000 7fdbef7fe700 1 --2- 172.21.15.2:0/894529437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbf019d8d0 0x7fdbf019dd50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800b6c0 tx=0x7fdbe8007730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:25.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.179+0000 7fdbf49fd700 1 -- 172.21.15.2:0/894529437 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe800e040 con 0x7fdbf019d8d0 2023-09-13T04:41:25.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.179+0000 7fdbf49fd700 1 -- 172.21.15.2:0/894529437 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbe801bb20 con 0x7fdbf019d8d0 2023-09-13T04:41:25.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.179+0000 7fdbf49fd700 1 -- 172.21.15.2:0/894529437 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe801c880 con 0x7fdbf019d8d0 2023-09-13T04:41:25.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.179+0000 7fdbf59ff700 1 -- 172.21.15.2:0/894529437 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe8009c40 con 0x7fdbf019d8d0 2023-09-13T04:41:25.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.180+0000 7fdbf59ff700 1 -- 172.21.15.2:0/894529437 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf01a5e70 con 0x7fdbf019d8d0 2023-09-13T04:41:25.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.185+0000 7fdbf49fd700 1 -- 172.21.15.2:0/894529437 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 24) v1 ==== 135491+0+0 (secure 0 0 0) 0x7fdbe801bc90 con 0x7fdbf019d8d0 2023-09-13T04:41:25.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.186+0000 7fdc09262b80 1 Processor -- start 2023-09-13T04:41:25.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.187+0000 7fdc09262b80 1 -- start start 2023-09-13T04:41:25.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.187+0000 7fdc09262b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607584f9870 0x5607584f9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.187+0000 7fdc09262b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584fa190 0x5607584f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.187+0000 7fdc09262b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5607584f4290 0x5607584f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.187+0000 7fdc09262b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560758444800 con 0x5607584fa190 2023-09-13T04:41:25.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.187+0000 7fdc09262b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560758442120 con 0x5607584f9870 2023-09-13T04:41:25.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.187+0000 7fdc09262b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56075843da10 con 0x5607584f4290 2023-09-13T04:41:25.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.188+0000 7fdbeeffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584fa190 0x5607584f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.188+0000 7fdbeeffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584fa190 0x5607584f3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33954/0 (socket says 172.21.15.2:33954) 2023-09-13T04:41:25.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.188+0000 7fdbeeffd700 1 -- 172.21.15.2:0/1783059529 learned_addr learned my addr 172.21.15.2:0/1783059529 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:25.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.188+0000 7fdbf49fd700 1 --2- 172.21.15.2:0/894529437 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fdbd80a09d0 0x7fdbd80a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.189+0000 7fdbf49fd700 1 -- 172.21.15.2:0/894529437 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fdbe80cd980 con 0x7fdbf019d8d0 2023-09-13T04:41:25.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.189+0000 7fdbeeffd700 1 -- 172.21.15.2:0/1783059529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5607584f4290 msgr2=0x5607584f46f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:41:25.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.189+0000 7fdbeeffd700 1 --2- 172.21.15.2:0/1783059529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5607584f4290 0x5607584f46f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.189+0000 7fdbeeffd700 1 -- 172.21.15.2:0/1783059529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607584f9870 msgr2=0x5607584f9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:25.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.189+0000 7fdbeeffd700 1 --2- 172.21.15.2:0/1783059529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607584f9870 0x5607584f9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.189+0000 7fdbeeffd700 1 -- 172.21.15.2:0/1783059529 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607582bd6b0 con 0x5607584fa190 2023-09-13T04:41:25.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.189+0000 7fdbeeffd700 1 --2- 172.21.15.2:0/1783059529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584fa190 0x5607584f3d50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009f10 tx=0x7fdbe0011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1f86a9c10970e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:25.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.190+0000 7fdbd5ffb700 1 -- 172.21.15.2:0/1783059529 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0019630 con 0x5607584fa190 2023-09-13T04:41:25.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.190+0000 7fdbd5ffb700 1 -- 172.21.15.2:0/1783059529 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbe0019c50 con 0x5607584fa190 2023-09-13T04:41:25.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.190+0000 7fdbef7fe700 1 --2- 172.21.15.2:0/894529437 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fdbd80a09d0 0x7fdbd80a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.190+0000 7fdbd5ffb700 1 -- 172.21.15.2:0/1783059529 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe000ece0 con 0x5607584fa190 2023-09-13T04:41:25.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.191+0000 7fdc09262b80 1 -- 172.21.15.2:0/1783059529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584fa190 msgr2=0x5607584f3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:25.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.191+0000 7fdbef7fe700 1 --2- 172.21.15.2:0/894529437 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fdbd80a09d0 0x7fdbd80a2e90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80cac40 tx=0x7fdbe8009040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:25.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.191+0000 7fdc09262b80 1 --2- 172.21.15.2:0/1783059529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584fa190 0x5607584f3d50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009f10 tx=0x7fdbe0011720 comp rx=0 tx=0).stop 2023-09-13T04:41:25.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.191+0000 7fdc09262b80 1 -- 172.21.15.2:0/1783059529 shutdown_connections 2023-09-13T04:41:25.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.191+0000 7fdc09262b80 1 --2- 172.21.15.2:0/1783059529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5607584f4290 0x5607584f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.191+0000 7fdc09262b80 1 --2- 172.21.15.2:0/1783059529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607584f9870 0x5607584f9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.191+0000 7fdc09262b80 1 --2- 172.21.15.2:0/1783059529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584fa190 0x5607584f3d50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.191+0000 7fdc09262b80 1 -- 172.21.15.2:0/1783059529 >> 172.21.15.2:0/1783059529 conn(0x560758449d00 msgr2=0x5607584e5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:25.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.191+0000 7fdc09262b80 1 -- 172.21.15.2:0/1783059529 shutdown_connections 2023-09-13T04:41:25.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.191+0000 7fdc09262b80 1 -- 172.21.15.2:0/1783059529 wait complete. 2023-09-13T04:41:25.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.192+0000 7fdc09262b80 1 -- start start 2023-09-13T04:41:25.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.192+0000 7fdc09262b80 1 Processor -- start 2023-09-13T04:41:25.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.193+0000 7fdc09262b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584f4290 0x5607584fd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.193+0000 7fdc09262b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607584f9870 0x5607584fdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.193+0000 7fdc09262b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5607584fa190 0x5607584fe000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.193+0000 7fdc09262b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607583dbc30 con 0x5607584f4290 2023-09-13T04:41:25.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.193+0000 7fdc09262b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607583db930 con 0x5607584f9870 2023-09-13T04:41:25.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.193+0000 7fdc09262b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607582bf9a0 con 0x5607584fa190 2023-09-13T04:41:25.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.193+0000 7fdc09262b80 10 client.0 _ll_register_callbacks cb 0x5607583d9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:25.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.193+0000 7fdbee7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584f4290 0x5607584fd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.194+0000 7fdbee7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584f4290 0x5607584fd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33960/0 (socket says 172.21.15.2:33960) 2023-09-13T04:41:25.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.194+0000 7fdbee7fc700 1 -- 172.21.15.2:0/2988778725 learned_addr learned my addr 172.21.15.2:0/2988778725 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:25.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.194+0000 7fdbee7fc700 1 --2- 172.21.15.2:0/2988778725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5607584fa190 0x5607584fe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.194+0000 7fdbeeffd700 1 --2- 172.21.15.2:0/2988778725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607584f9870 0x5607584fdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.194+0000 7fdbee7fc700 1 -- 172.21.15.2:0/2988778725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5607584fa190 msgr2=0x5607584fe000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:25.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.194+0000 7fdbee7fc700 1 --2- 172.21.15.2:0/2988778725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5607584fa190 0x5607584fe000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.194+0000 7fdbee7fc700 1 -- 172.21.15.2:0/2988778725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607584f9870 msgr2=0x5607584fdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:25.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.194+0000 7fdbee7fc700 1 --2- 172.21.15.2:0/2988778725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607584f9870 0x5607584fdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.194+0000 7fdbee7fc700 1 -- 172.21.15.2:0/2988778725 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe0009b70 con 0x5607584f4290 2023-09-13T04:41:25.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.194+0000 7fdbeeffd700 1 --2- 172.21.15.2:0/2988778725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607584f9870 0x5607584fdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:41:25.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.195+0000 7fdbee7fc700 1 --2- 172.21.15.2:0/2988778725 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584f4290 0x5607584fd580 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4013d90 tx=0x7fdbe4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e0597814a39503 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:25.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.195+0000 7fdbceffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:41:25.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.195+0000 7fdbceffd700 1 -- 172.21.15.2:0/2988778725 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4013e80 con 0x5607584f4290 2023-09-13T04:41:25.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.195+0000 7fdbceffd700 1 -- 172.21.15.2:0/2988778725 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fdbe4013130 con 0x5607584f4290 2023-09-13T04:41:25.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.195+0000 7fdc09262b80 1 -- 172.21.15.2:0/2988778725 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607584de6f0 con 0x5607584f4290 2023-09-13T04:41:25.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.195+0000 7fdbceffd700 1 -- 172.21.15.2:0/2988778725 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbe40132f0 con 0x5607584f4290 2023-09-13T04:41:25.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.196+0000 7fdc09262b80 10 client.15831 fetch_fsmap learned FSMap version 23 2023-09-13T04:41:25.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.196+0000 7fdc09262b80 1 -- 172.21.15.2:0/2988778725 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560758455820 con 0x5607584f4290 2023-09-13T04:41:25.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.196+0000 7fdbceffd700 1 -- 172.21.15.2:0/2988778725 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5607584de6f0 con 0x5607584f4290 2023-09-13T04:41:25.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.196+0000 7fdbceffd700 1 -- 172.21.15.2:0/2988778725 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdbe4003890 con 0x5607584f4290 2023-09-13T04:41:25.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.196+0000 7fdc09262b80 10 client.15831 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:41:25.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.196+0000 7fdc09262b80 10 client.15831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:41:25.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.196+0000 7fdc09262b80 20 client.15831 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:25.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.196+0000 7fdc09262b80 1 --2- 172.21.15.2:0/2988778725 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5607584ec580 0x5607584ee9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.196+0000 7fdc09262b80 4 client.15831 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:41:25.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.196+0000 7fdc09262b80 1 -- 172.21.15.2:0/2988778725 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607584369f0 con 0x5607584ec580 2023-09-13T04:41:25.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.197+0000 7fdbeeffd700 1 --2- 172.21.15.2:0/2988778725 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5607584ec580 0x5607584ee9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.198+0000 7fdbeeffd700 1 --2- 172.21.15.2:0/2988778725 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5607584ec580 0x5607584ee9a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbfe854e15cd88d0 server_cookie=1f65414fe3b132fb in_seq=0 out_seq=0 2023-09-13T04:41:25.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.198+0000 7fdbceffd700 10 client.15831 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:25.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.198+0000 7fdbceffd700 1 -- 172.21.15.2:0/2988778725 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5607584369f0 con 0x5607584ec580 2023-09-13T04:41:25.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.198+0000 7fdbceffd700 10 client.15831 handle_command_reply: tid=0 2023-09-13T04:41:25.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.198+0000 7fdc09262b80 1 client.15831 shutdown 2023-09-13T04:41:25.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.199+0000 7fdc09262b80 1 -- 172.21.15.2:0/2988778725 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584f4290 msgr2=0x5607584fd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:25.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.199+0000 7fdc09262b80 1 --2- 172.21.15.2:0/2988778725 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584f4290 0x5607584fd580 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4013d90 tx=0x7fdbe4004750 comp rx=0 tx=0).stop 2023-09-13T04:41:25.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.199+0000 7fdc09262b80 1 -- 172.21.15.2:0/2988778725 shutdown_connections 2023-09-13T04:41:25.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.199+0000 7fdc09262b80 1 --2- 172.21.15.2:0/2988778725 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5607584ec580 0x5607584ee9a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.199+0000 7fdc09262b80 1 --2- 172.21.15.2:0/2988778725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5607584fa190 0x5607584fe000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.199+0000 7fdc09262b80 1 --2- 172.21.15.2:0/2988778725 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584f4290 0x5607584fd580 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.199+0000 7fdbceffd700 0 client.15831 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:25.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.199+0000 7fdc09262b80 1 --2- 172.21.15.2:0/2988778725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607584f9870 0x5607584fdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.199+0000 7fdc09262b80 1 -- 172.21.15.2:0/2988778725 >> 172.21.15.2:0/2988778725 conn(0x560758449d00 msgr2=0x5607584e0870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:25.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.199+0000 7fdc09262b80 1 -- 172.21.15.2:0/2988778725 shutdown_connections 2023-09-13T04:41:25.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.199+0000 7fdc09262b80 1 -- 172.21.15.2:0/2988778725 wait complete. 2023-09-13T04:41:25.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.199+0000 7fdc09262b80 20 client.15831 trim_cache size 0 max 16384 2023-09-13T04:41:25.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.233+0000 7fdc09262b80 1 Processor -- start 2023-09-13T04:41:25.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.234+0000 7fdc09262b80 1 -- start start 2023-09-13T04:41:25.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.234+0000 7fdc09262b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607584f4290 0x5607584e0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.234+0000 7fdc09262b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584f9870 0x56075850daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.234+0000 7fdc09262b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5607584fa190 0x5607584fde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.234+0000 7fdc09262b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607584f8220 con 0x5607584f9870 2023-09-13T04:41:25.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.234+0000 7fdc09262b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607584e0eb0 con 0x5607584f4290 2023-09-13T04:41:25.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.234+0000 7fdc09262b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56075850e030 con 0x5607584fa190 2023-09-13T04:41:25.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.235+0000 7fdbeeffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584f9870 0x56075850daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.235+0000 7fdbee7fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607584f4290 0x5607584e0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.235+0000 7fdbeeffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584f9870 0x56075850daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33962/0 (socket says 172.21.15.2:33962) 2023-09-13T04:41:25.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.235+0000 7fdbeeffd700 1 -- 172.21.15.2:0/2053453313 learned_addr learned my addr 172.21.15.2:0/2053453313 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:25.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.235+0000 7fdbee7fc700 1 --2- 172.21.15.2:0/2053453313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5607584fa190 0x5607584fde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.235+0000 7fdbee7fc700 1 -- 172.21.15.2:0/2053453313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5607584fa190 msgr2=0x5607584fde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:25.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.235+0000 7fdbee7fc700 1 --2- 172.21.15.2:0/2053453313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5607584fa190 0x5607584fde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.235+0000 7fdbee7fc700 1 -- 172.21.15.2:0/2053453313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584f9870 msgr2=0x56075850daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:25.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.235+0000 7fdbee7fc700 1 --2- 172.21.15.2:0/2053453313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584f9870 0x56075850daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.235+0000 7fdbee7fc700 1 -- 172.21.15.2:0/2053453313 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560758269e60 con 0x5607584f4290 2023-09-13T04:41:25.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.236+0000 7fdbee7fc700 1 --2- 172.21.15.2:0/2053453313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607584f4290 0x5607584e0970 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4005850 tx=0x7fdbe4015040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96079037bcfebda0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:25.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.236+0000 7fdbd67fc700 1 -- 172.21.15.2:0/2053453313 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4010430 con 0x5607584f4290 2023-09-13T04:41:25.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.236+0000 7fdbd67fc700 1 -- 172.21.15.2:0/2053453313 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbe4013130 con 0x5607584f4290 2023-09-13T04:41:25.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.236+0000 7fdbd67fc700 1 -- 172.21.15.2:0/2053453313 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4004070 con 0x5607584f4290 2023-09-13T04:41:25.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.237+0000 7fdc09262b80 1 -- 172.21.15.2:0/2053453313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607584f4290 msgr2=0x5607584e0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:25.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.237+0000 7fdc09262b80 1 --2- 172.21.15.2:0/2053453313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607584f4290 0x5607584e0970 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4005850 tx=0x7fdbe4015040 comp rx=0 tx=0).stop 2023-09-13T04:41:25.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.237+0000 7fdc09262b80 1 -- 172.21.15.2:0/2053453313 shutdown_connections 2023-09-13T04:41:25.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.237+0000 7fdc09262b80 1 --2- 172.21.15.2:0/2053453313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5607584fa190 0x5607584fde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.237+0000 7fdc09262b80 1 --2- 172.21.15.2:0/2053453313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607584f4290 0x5607584e0970 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.237+0000 7fdc09262b80 1 --2- 172.21.15.2:0/2053453313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584f9870 0x56075850daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.237+0000 7fdc09262b80 1 -- 172.21.15.2:0/2053453313 >> 172.21.15.2:0/2053453313 conn(0x560758449d00 msgr2=0x5607584faeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:25.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.237+0000 7fdc09262b80 1 -- 172.21.15.2:0/2053453313 shutdown_connections 2023-09-13T04:41:25.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.237+0000 7fdc09262b80 1 -- 172.21.15.2:0/2053453313 wait complete. 2023-09-13T04:41:25.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.238+0000 7fdc09262b80 1 -- start start 2023-09-13T04:41:25.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.238+0000 7fdc09262b80 1 Processor -- start 2023-09-13T04:41:25.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.239+0000 7fdc09262b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607584f4290 0x560758551fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.239+0000 7fdc09262b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5607584f9870 0x560758552520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.239+0000 7fdc09262b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584fa190 0x5607584f28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.239+0000 7fdc09262b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607584f2e80 con 0x5607584fa190 2023-09-13T04:41:25.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.239+0000 7fdc09262b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607584f2ff0 con 0x5607584f4290 2023-09-13T04:41:25.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.239+0000 7fdc09262b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607584f31b0 con 0x5607584f9870 2023-09-13T04:41:25.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.239+0000 7fdc09262b80 10 client.0 _ll_register_callbacks cb 0x5607583d9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:25.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.239+0000 7fdbee7fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607584f4290 0x560758551fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.239+0000 7fdbee7fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607584f4290 0x560758551fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:60316/0 (socket says 172.21.15.2:60316) 2023-09-13T04:41:25.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.239+0000 7fdbee7fc700 1 -- 172.21.15.2:0/4064297838 learned_addr learned my addr 172.21.15.2:0/4064297838 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:25.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.239+0000 7fdbee7fc700 1 --2- 172.21.15.2:0/4064297838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584fa190 0x5607584f28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.239+0000 7fdbeeffd700 1 --2- 172.21.15.2:0/4064297838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5607584f9870 0x560758552520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.240+0000 7fdbee7fc700 1 -- 172.21.15.2:0/4064297838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5607584f9870 msgr2=0x560758552520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:25.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.240+0000 7fdbee7fc700 1 --2- 172.21.15.2:0/4064297838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5607584f9870 0x560758552520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.240+0000 7fdbee7fc700 1 -- 172.21.15.2:0/4064297838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584fa190 msgr2=0x5607584f28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:25.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.240+0000 7fdbee7fc700 1 --2- 172.21.15.2:0/4064297838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584fa190 0x5607584f28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.240+0000 7fdbee7fc700 1 -- 172.21.15.2:0/4064297838 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe4032040 con 0x5607584f4290 2023-09-13T04:41:25.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.240+0000 7fdbeeffd700 1 --2- 172.21.15.2:0/4064297838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5607584f9870 0x560758552520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:41:25.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.240+0000 7fdbee7fc700 1 --2- 172.21.15.2:0/4064297838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607584f4290 0x560758551fe0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009970 tx=0x7fdbe402c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32da4abae7580284 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:25.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.240+0000 7fdbceffd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:41:25.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.240+0000 7fdbceffd700 1 -- 172.21.15.2:0/4064297838 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4004020 con 0x5607584f4290 2023-09-13T04:41:25.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.241+0000 7fdbceffd700 1 -- 172.21.15.2:0/4064297838 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fdbe4000e40 con 0x5607584f4290 2023-09-13T04:41:25.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.241+0000 7fdc09262b80 1 -- 172.21.15.2:0/4064297838 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdbe0019c50 con 0x5607584f4290 2023-09-13T04:41:25.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.241+0000 7fdbceffd700 1 -- 172.21.15.2:0/4064297838 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbe40052c0 con 0x5607584f4290 2023-09-13T04:41:25.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.241+0000 7fdc09262b80 10 client.25856 fetch_fsmap learned FSMap version 23 2023-09-13T04:41:25.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.241+0000 7fdc09262b80 1 -- 172.21.15.2:0/4064297838 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560758252ce0 con 0x5607584f4290 2023-09-13T04:41:25.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.241+0000 7fdbceffd700 1 -- 172.21.15.2:0/4064297838 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbe0019c50 con 0x5607584f4290 2023-09-13T04:41:25.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.242+0000 7fdbceffd700 1 -- 172.21.15.2:0/4064297838 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdbe4035030 con 0x5607584f4290 2023-09-13T04:41:25.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.242+0000 7fdc09262b80 10 client.25856 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:41:25.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.242+0000 7fdc09262b80 10 client.25856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:41:25.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.242+0000 7fdc09262b80 20 client.25856 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:25.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.242+0000 7fdc09262b80 1 --2- 172.21.15.2:0/4064297838 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x560758559fe0 0x56075855c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.242+0000 7fdc09262b80 4 client.25856 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-13T04:41:25.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.242+0000 7fdc09262b80 1 -- 172.21.15.2:0/4064297838 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5607583f70d0 con 0x560758559fe0 2023-09-13T04:41:25.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.243+0000 7fdbeeffd700 1 --2- 172.21.15.2:0/4064297838 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x560758559fe0 0x56075855c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.243+0000 7fdbeeffd700 1 --2- 172.21.15.2:0/4064297838 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x560758559fe0 0x56075855c400 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e06e27ed3e4989da server_cookie=9cfaf15cca0ec873 in_seq=0 out_seq=0 2023-09-13T04:41:25.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.243+0000 7fdbceffd700 10 client.25856 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:25.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.244+0000 7fdbceffd700 1 -- 172.21.15.2:0/4064297838 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5607583f70d0 con 0x560758559fe0 2023-09-13T04:41:25.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.244+0000 7fdbceffd700 10 client.25856 handle_command_reply: tid=0 2023-09-13T04:41:25.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.244+0000 7fdc09262b80 1 client.25856 shutdown 2023-09-13T04:41:25.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.245+0000 7fdc09262b80 1 -- 172.21.15.2:0/4064297838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607584f4290 msgr2=0x560758551fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:25.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.245+0000 7fdc09262b80 1 --2- 172.21.15.2:0/4064297838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607584f4290 0x560758551fe0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009970 tx=0x7fdbe402c040 comp rx=0 tx=0).stop 2023-09-13T04:41:25.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.245+0000 7fdc09262b80 1 -- 172.21.15.2:0/4064297838 shutdown_connections 2023-09-13T04:41:25.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.245+0000 7fdc09262b80 1 --2- 172.21.15.2:0/4064297838 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x560758559fe0 0x56075855c400 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.245+0000 7fdc09262b80 1 --2- 172.21.15.2:0/4064297838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5607584fa190 0x5607584f28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.245+0000 7fdc09262b80 1 --2- 172.21.15.2:0/4064297838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607584f4290 0x560758551fe0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.245+0000 7fdc09262b80 1 --2- 172.21.15.2:0/4064297838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5607584f9870 0x560758552520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.245+0000 7fdbceffd700 0 client.25856 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:25.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.245+0000 7fdc09262b80 1 -- 172.21.15.2:0/4064297838 >> 172.21.15.2:0/4064297838 conn(0x560758449d00 msgr2=0x5607584fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:25.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.246+0000 7fdc09262b80 1 -- 172.21.15.2:0/4064297838 shutdown_connections 2023-09-13T04:41:25.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.246+0000 7fdc09262b80 1 -- 172.21.15.2:0/4064297838 wait complete. 2023-09-13T04:41:25.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.246+0000 7fdc09262b80 20 client.25856 trim_cache size 0 max 16384 2023-09-13T04:41:25.256 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:41:25.256 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2023-09-13T04:41:25.257 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "7e2865f7-07c7-4a67-b469-10f73f1c731e", 2023-09-13T04:41:25.257 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2023-09-13T04:41:25.257 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:41:25.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.247+0000 7fdbf59ff700 1 -- 172.21.15.2:0/894529437 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fdbd80a09d0 msgr2=0x7fdbd80a2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:25.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.247+0000 7fdbf59ff700 1 --2- 172.21.15.2:0/894529437 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fdbd80a09d0 0x7fdbd80a2e90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80cac40 tx=0x7fdbe8009040 comp rx=0 tx=0).stop 2023-09-13T04:41:25.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.247+0000 7fdbf59ff700 1 -- 172.21.15.2:0/894529437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbf019d8d0 msgr2=0x7fdbf019dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:25.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.247+0000 7fdbf59ff700 1 --2- 172.21.15.2:0/894529437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbf019d8d0 0x7fdbf019dd50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800b6c0 tx=0x7fdbe8007730 comp rx=0 tx=0).stop 2023-09-13T04:41:25.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.247+0000 7fdbf59ff700 1 -- 172.21.15.2:0/894529437 shutdown_connections 2023-09-13T04:41:25.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.247+0000 7fdbf59ff700 1 --2- 172.21.15.2:0/894529437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbf019d8d0 0x7fdbf019dd50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.247+0000 7fdbf59ff700 1 --2- 172.21.15.2:0/894529437 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fdbd80a09d0 0x7fdbd80a2e90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.247+0000 7fdbf59ff700 1 --2- 172.21.15.2:0/894529437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbf0100eb0 0x7fdbf0198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.247+0000 7fdbf59ff700 1 --2- 172.21.15.2:0/894529437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbf0105fd0 0x7fdbf0199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.247+0000 7fdbf59ff700 1 -- 172.21.15.2:0/894529437 >> 172.21.15.2:0/894529437 conn(0x7fdbf00fc730 msgr2=0x7fdbf00fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:25.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.248+0000 7fdbf59ff700 1 -- 172.21.15.2:0/894529437 shutdown_connections 2023-09-13T04:41:25.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.248+0000 7fdbf59ff700 1 -- 172.21.15.2:0/894529437 wait complete. 2023-09-13T04:41:25.277 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T04:41:25.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.621+0000 7f89abfff700 1 -- 172.21.15.2:0/537324635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89a4100eb0 msgr2=0x7f89a4101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:25.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.621+0000 7f89abfff700 1 --2- 172.21.15.2:0/537324635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89a4100eb0 0x7f89a4101290 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009990 tx=0x7f89a0009ca0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.622+0000 7f89abfff700 1 -- 172.21.15.2:0/537324635 shutdown_connections 2023-09-13T04:41:25.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.622+0000 7f89abfff700 1 --2- 172.21.15.2:0/537324635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a4105fd0 0x7f89a410db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.622+0000 7f89abfff700 1 --2- 172.21.15.2:0/537324635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89a4100eb0 0x7f89a4101290 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.622+0000 7f89abfff700 1 --2- 172.21.15.2:0/537324635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a4101860 0x7f89a41058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.622+0000 7f89abfff700 1 -- 172.21.15.2:0/537324635 >> 172.21.15.2:0/537324635 conn(0x7f89a40fc750 msgr2=0x7f89a40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:25.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.622+0000 7f89abfff700 1 -- 172.21.15.2:0/537324635 shutdown_connections 2023-09-13T04:41:25.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.622+0000 7f89abfff700 1 -- 172.21.15.2:0/537324635 wait complete. 2023-09-13T04:41:25.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.623+0000 7f89abfff700 1 Processor -- start 2023-09-13T04:41:25.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.624+0000 7f89abfff700 1 -- start start 2023-09-13T04:41:25.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.624+0000 7f89abfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a4100eb0 0x7f89a4199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.624+0000 7f89abfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89a4105fd0 0x7f89a4199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.624+0000 7f89abfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a419d9e0 0x7f89a419de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.624+0000 7f89abfff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a419e460 con 0x7f89a419d9e0 2023-09-13T04:41:25.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.624+0000 7f89abfff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a419e5d0 con 0x7f89a4100eb0 2023-09-13T04:41:25.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.624+0000 7f89abfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a419e790 con 0x7f89a4105fd0 2023-09-13T04:41:25.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.624+0000 7f89a9d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a4100eb0 0x7f89a4199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.625+0000 7f89a9d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a4100eb0 0x7f89a4199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:60324/0 (socket says 172.21.15.2:60324) 2023-09-13T04:41:25.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.625+0000 7f89a9d9b700 1 -- 172.21.15.2:0/1424353702 learned_addr learned my addr 172.21.15.2:0/1424353702 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:25.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.625+0000 7f89a959a700 1 --2- 172.21.15.2:0/1424353702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89a4105fd0 0x7f89a4199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.625+0000 7f89aa59c700 1 --2- 172.21.15.2:0/1424353702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a419d9e0 0x7f89a419de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.625+0000 7f89a959a700 1 -- 172.21.15.2:0/1424353702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a4100eb0 msgr2=0x7f89a4199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:25.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.625+0000 7f89a959a700 1 --2- 172.21.15.2:0/1424353702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a4100eb0 0x7f89a4199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.625+0000 7f89a959a700 1 -- 172.21.15.2:0/1424353702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a419d9e0 msgr2=0x7f89a419de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:25.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.625+0000 7f89a959a700 1 --2- 172.21.15.2:0/1424353702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a419d9e0 0x7f89a419de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.625+0000 7f89a959a700 1 -- 172.21.15.2:0/1424353702 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a00095f0 con 0x7f89a4105fd0 2023-09-13T04:41:25.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.625+0000 7f89a959a700 1 --2- 172.21.15.2:0/1424353702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89a4105fd0 0x7f89a4199580 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f899400e9c0 tx=0x7f899400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:25.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.626+0000 7f899affd700 1 -- 172.21.15.2:0/1424353702 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899400c9c0 con 0x7f89a4105fd0 2023-09-13T04:41:25.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.626+0000 7f899affd700 1 -- 172.21.15.2:0/1424353702 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8994010430 con 0x7f89a4105fd0 2023-09-13T04:41:25.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.626+0000 7f899affd700 1 -- 172.21.15.2:0/1424353702 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8994018680 con 0x7f89a4105fd0 2023-09-13T04:41:25.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.627+0000 7f89abfff700 1 -- 172.21.15.2:0/1424353702 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a41a5cf0 con 0x7f89a4105fd0 2023-09-13T04:41:25.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.627+0000 7f89abfff700 1 -- 172.21.15.2:0/1424353702 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a41a6180 con 0x7f89a4105fd0 2023-09-13T04:41:25.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.632+0000 7f899affd700 1 -- 172.21.15.2:0/1424353702 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 24) v1 ==== 135491+0+0 (secure 0 0 0) 0x7f899400cb20 con 0x7f89a4105fd0 2023-09-13T04:41:25.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.634+0000 7f89bf834b80 1 Processor -- start 2023-09-13T04:41:25.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.634+0000 7f89bf834b80 1 -- start start 2023-09-13T04:41:25.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.634+0000 7f89bf834b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56301fc307f0 0x56301fc30bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.635+0000 7f89bf834b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc31110 0x56301fc2acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.635+0000 7f89bf834b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56301fc2b210 0x56301fc2b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.635+0000 7f89bf834b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301fb7b780 con 0x56301fc2b210 2023-09-13T04:41:25.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.635+0000 7f89bf834b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301fb790a0 con 0x56301fc307f0 2023-09-13T04:41:25.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.635+0000 7f89bf834b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301fb74990 con 0x56301fc31110 2023-09-13T04:41:25.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.635+0000 7f89aa59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56301fc307f0 0x56301fc30bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.635+0000 7f89aa59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56301fc307f0 0x56301fc30bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:60338/0 (socket says 172.21.15.2:60338) 2023-09-13T04:41:25.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.635+0000 7f89aa59c700 1 -- 172.21.15.2:0/4110574220 learned_addr learned my addr 172.21.15.2:0/4110574220 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:25.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.635+0000 7f89a959a700 1 --2- 172.21.15.2:0/4110574220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56301fc2b210 0x56301fc2b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.635+0000 7f89aa59c700 1 -- 172.21.15.2:0/4110574220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc31110 msgr2=0x56301fc2acd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:41:25.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.635+0000 7f89aa59c700 1 --2- 172.21.15.2:0/4110574220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc31110 0x56301fc2acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.636+0000 7f89aa59c700 1 -- 172.21.15.2:0/4110574220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56301fc2b210 msgr2=0x56301fc2b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:25.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.636+0000 7f89aa59c700 1 --2- 172.21.15.2:0/4110574220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56301fc2b210 0x56301fc2b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.636+0000 7f89aa59c700 1 -- 172.21.15.2:0/4110574220 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56301f9f4630 con 0x56301fc307f0 2023-09-13T04:41:25.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.636+0000 7f89aa59c700 1 --2- 172.21.15.2:0/4110574220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56301fc307f0 0x56301fc30bd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f899c011680 tx=0x7f899c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1224d3a616f3743f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:25.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.636+0000 7f898ffff700 1 -- 172.21.15.2:0/4110574220 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c00e040 con 0x56301fc307f0 2023-09-13T04:41:25.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.636+0000 7f899affd700 1 --2- 172.21.15.2:0/1424353702 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f89900a0950 0x7f89900a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.636+0000 7f898ffff700 1 -- 172.21.15.2:0/4110574220 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f899c00f040 con 0x56301fc307f0 2023-09-13T04:41:25.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.636+0000 7f898ffff700 1 -- 172.21.15.2:0/4110574220 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c0158f0 con 0x56301fc307f0 2023-09-13T04:41:25.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.636+0000 7f899affd700 1 -- 172.21.15.2:0/1424353702 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f8994014070 con 0x7f89a4105fd0 2023-09-13T04:41:25.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.637+0000 7f89bf834b80 1 -- 172.21.15.2:0/4110574220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56301fc307f0 msgr2=0x56301fc30bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:25.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.637+0000 7f89bf834b80 1 --2- 172.21.15.2:0/4110574220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56301fc307f0 0x56301fc30bd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f899c011680 tx=0x7f899c009920 comp rx=0 tx=0).stop 2023-09-13T04:41:25.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.637+0000 7f89a9d9b700 1 --2- 172.21.15.2:0/1424353702 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f89900a0950 0x7f89900a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.637+0000 7f89bf834b80 1 -- 172.21.15.2:0/4110574220 shutdown_connections 2023-09-13T04:41:25.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.637+0000 7f89bf834b80 1 --2- 172.21.15.2:0/4110574220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56301fc2b210 0x56301fc2b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.637+0000 7f89bf834b80 1 --2- 172.21.15.2:0/4110574220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56301fc307f0 0x56301fc30bd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.637+0000 7f89bf834b80 1 --2- 172.21.15.2:0/4110574220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc31110 0x56301fc2acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.637+0000 7f89bf834b80 1 -- 172.21.15.2:0/4110574220 >> 172.21.15.2:0/4110574220 conn(0x56301fb80c80 msgr2=0x56301fc1cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:25.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.637+0000 7f89bf834b80 1 -- 172.21.15.2:0/4110574220 shutdown_connections 2023-09-13T04:41:25.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.638+0000 7f89bf834b80 1 -- 172.21.15.2:0/4110574220 wait complete. 2023-09-13T04:41:25.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.638+0000 7f89a9d9b700 1 --2- 172.21.15.2:0/1424353702 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f89900a0950 0x7f89900a2e10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f89a000b690 tx=0x7f89a000b260 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:25.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.638+0000 7f89bf834b80 1 -- start start 2023-09-13T04:41:25.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.639+0000 7f89bf834b80 1 Processor -- start 2023-09-13T04:41:25.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.640+0000 7f89bf834b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc2b210 0x56301fc37120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.640+0000 7f89bf834b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56301fc307f0 0x56301fc34410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.640+0000 7f89bf834b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56301fc31110 0x56301fc34b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.640+0000 7f89bf834b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301fb12bb0 con 0x56301fc31110 2023-09-13T04:41:25.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.640+0000 7f89bf834b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301fb128b0 con 0x56301fc307f0 2023-09-13T04:41:25.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.640+0000 7f89bf834b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301f9f6920 con 0x56301fc2b210 2023-09-13T04:41:25.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.640+0000 7f89bf834b80 10 client.0 _ll_register_callbacks cb 0x56301fb10680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:25.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.640+0000 7f89a9d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc2b210 0x56301fc37120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.640+0000 7f89a9d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc2b210 0x56301fc37120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:57296/0 (socket says 172.21.15.2:57296) 2023-09-13T04:41:25.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.640+0000 7f89a9d9b700 1 -- 172.21.15.2:0/3908446531 learned_addr learned my addr 172.21.15.2:0/3908446531 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:25.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.640+0000 7f89aa59c700 1 --2- 172.21.15.2:0/3908446531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56301fc31110 0x56301fc34b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.641+0000 7f89a9d9b700 1 -- 172.21.15.2:0/3908446531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56301fc307f0 msgr2=0x56301fc34410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:41:25.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.641+0000 7f89a9d9b700 1 --2- 172.21.15.2:0/3908446531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56301fc307f0 0x56301fc34410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.641+0000 7f89a9d9b700 1 -- 172.21.15.2:0/3908446531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56301fc31110 msgr2=0x56301fc34b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:25.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.641+0000 7f89a9d9b700 1 --2- 172.21.15.2:0/3908446531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56301fc31110 0x56301fc34b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.641+0000 7f89a9d9b700 1 -- 172.21.15.2:0/3908446531 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899c011330 con 0x56301fc2b210 2023-09-13T04:41:25.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.641+0000 7f89a9d9b700 1 --2- 172.21.15.2:0/3908446531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc2b210 0x56301fc37120 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f89a000bb70 tx=0x7f89a0009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd33ef72648e3193 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:25.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.641+0000 7f898d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:41:25.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.641+0000 7f898d7fa700 1 -- 172.21.15.2:0/3908446531 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a0004da0 con 0x56301fc2b210 2023-09-13T04:41:25.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.641+0000 7f898d7fa700 1 -- 172.21.15.2:0/3908446531 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f89a000f460 con 0x56301fc2b210 2023-09-13T04:41:25.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.642+0000 7f89bf834b80 1 -- 172.21.15.2:0/3908446531 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56301fc15670 con 0x56301fc2b210 2023-09-13T04:41:25.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.642+0000 7f898d7fa700 1 -- 172.21.15.2:0/3908446531 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89a0005a00 con 0x56301fc2b210 2023-09-13T04:41:25.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.642+0000 7f89bf834b80 10 client.25879 fetch_fsmap learned FSMap version 23 2023-09-13T04:41:25.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.642+0000 7f89bf834b80 1 -- 172.21.15.2:0/3908446531 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56301fb8c7a0 con 0x56301fc2b210 2023-09-13T04:41:25.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.642+0000 7f898d7fa700 1 -- 172.21.15.2:0/3908446531 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56301fc15670 con 0x56301fc2b210 2023-09-13T04:41:25.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.643+0000 7f898d7fa700 1 -- 172.21.15.2:0/3908446531 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f89a0037070 con 0x56301fc2b210 2023-09-13T04:41:25.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.643+0000 7f89bf834b80 10 client.25879 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:41:25.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.643+0000 7f89bf834b80 10 client.25879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:41:25.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.643+0000 7f89bf834b80 20 client.25879 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:25.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.643+0000 7f89bf834b80 1 --2- 172.21.15.2:0/3908446531 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56301fc44930 0x56301fc46d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.643+0000 7f89bf834b80 4 client.25879 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:41:25.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.643+0000 7f89bf834b80 1 -- 172.21.15.2:0/3908446531 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56301fb6d970 con 0x56301fc44930 2023-09-13T04:41:25.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.643+0000 7f89a959a700 1 --2- 172.21.15.2:0/3908446531 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56301fc44930 0x56301fc46d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.644+0000 7f89a959a700 1 --2- 172.21.15.2:0/3908446531 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56301fc44930 0x56301fc46d50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e28862fec6189391 server_cookie=1d7c68952f7b1b9d in_seq=0 out_seq=0 2023-09-13T04:41:25.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.644+0000 7f898d7fa700 10 client.25879 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:25.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.645+0000 7f898d7fa700 1 -- 172.21.15.2:0/3908446531 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56301fb6d970 con 0x56301fc44930 2023-09-13T04:41:25.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.645+0000 7f898d7fa700 10 client.25879 handle_command_reply: tid=0 2023-09-13T04:41:25.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.645+0000 7f89bf834b80 1 client.25879 shutdown 2023-09-13T04:41:25.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.645+0000 7f89bf834b80 1 -- 172.21.15.2:0/3908446531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc2b210 msgr2=0x56301fc37120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:25.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.645+0000 7f89bf834b80 1 --2- 172.21.15.2:0/3908446531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc2b210 0x56301fc37120 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f89a000bb70 tx=0x7f89a0009ca0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.645+0000 7f89bf834b80 1 -- 172.21.15.2:0/3908446531 shutdown_connections 2023-09-13T04:41:25.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.645+0000 7f89bf834b80 1 --2- 172.21.15.2:0/3908446531 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56301fc44930 0x56301fc46d50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.645+0000 7f89bf834b80 1 --2- 172.21.15.2:0/3908446531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56301fc31110 0x56301fc34b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.645+0000 7f89bf834b80 1 --2- 172.21.15.2:0/3908446531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc2b210 0x56301fc37120 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.645+0000 7f89bf834b80 1 --2- 172.21.15.2:0/3908446531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56301fc307f0 0x56301fc34410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.645+0000 7f89bf834b80 1 -- 172.21.15.2:0/3908446531 >> 172.21.15.2:0/3908446531 conn(0x56301fb80c80 msgr2=0x56301fc177f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:25.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.645+0000 7f898d7fa700 0 client.25879 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:25.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.646+0000 7f89bf834b80 1 -- 172.21.15.2:0/3908446531 shutdown_connections 2023-09-13T04:41:25.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.646+0000 7f89bf834b80 1 -- 172.21.15.2:0/3908446531 wait complete. 2023-09-13T04:41:25.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.646+0000 7f89bf834b80 20 client.25879 trim_cache size 0 max 16384 2023-09-13T04:41:25.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.680+0000 7f89bf834b80 1 Processor -- start 2023-09-13T04:41:25.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.680+0000 7f89bf834b80 1 -- start start 2023-09-13T04:41:25.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.681+0000 7f89bf834b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc2b210 0x56301fc3ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.681+0000 7f89bf834b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56301fc307f0 0x56301fc178f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.681+0000 7f89bf834b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56301fc31110 0x56301fc2fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.681+0000 7f89bf834b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301fc15400 con 0x56301fc307f0 2023-09-13T04:41:25.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.681+0000 7f89bf834b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301fc2bc40 con 0x56301fc31110 2023-09-13T04:41:25.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.681+0000 7f89bf834b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301fc1e8e0 con 0x56301fc2b210 2023-09-13T04:41:25.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.681+0000 7f89a9d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc2b210 0x56301fc3ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.681+0000 7f89aa59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56301fc31110 0x56301fc2fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.681+0000 7f89aa59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56301fc31110 0x56301fc2fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:60352/0 (socket says 172.21.15.2:60352) 2023-09-13T04:41:25.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.681+0000 7f89aa59c700 1 -- 172.21.15.2:0/2252877145 learned_addr learned my addr 172.21.15.2:0/2252877145 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:25.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.681+0000 7f89a9d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc2b210 0x56301fc3ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:57312/0 (socket says 172.21.15.2:57312) 2023-09-13T04:41:25.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.682+0000 7f89aa59c700 1 -- 172.21.15.2:0/2252877145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc2b210 msgr2=0x56301fc3ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:25.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.682+0000 7f89aa59c700 1 --2- 172.21.15.2:0/2252877145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc2b210 0x56301fc3ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.682+0000 7f89aa59c700 1 -- 172.21.15.2:0/2252877145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56301fc307f0 msgr2=0x56301fc178f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:25.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.682+0000 7f89aa59c700 1 --2- 172.21.15.2:0/2252877145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56301fc307f0 0x56301fc178f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.682+0000 7f89aa59c700 1 -- 172.21.15.2:0/2252877145 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56301fc4b9c0 con 0x56301fc31110 2023-09-13T04:41:25.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.682+0000 7f89a9d9b700 1 --2- 172.21.15.2:0/2252877145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc2b210 0x56301fc3ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:25.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.682+0000 7f89aa59c700 1 --2- 172.21.15.2:0/2252877145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56301fc31110 0x56301fc2fbe0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f899c00b010 tx=0x7f899c009440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78e12a19a9a33dc1 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:25.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.683+0000 7f8998ff9700 1 -- 172.21.15.2:0/2252877145 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c00e050 con 0x56301fc31110 2023-09-13T04:41:25.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.683+0000 7f8998ff9700 1 -- 172.21.15.2:0/2252877145 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f899c004e50 con 0x56301fc31110 2023-09-13T04:41:25.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.683+0000 7f8998ff9700 1 -- 172.21.15.2:0/2252877145 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c015870 con 0x56301fc31110 2023-09-13T04:41:25.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.684+0000 7f89bf834b80 1 -- 172.21.15.2:0/2252877145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56301fc31110 msgr2=0x56301fc2fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:25.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.684+0000 7f89bf834b80 1 --2- 172.21.15.2:0/2252877145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56301fc31110 0x56301fc2fbe0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f899c00b010 tx=0x7f899c009440 comp rx=0 tx=0).stop 2023-09-13T04:41:25.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.684+0000 7f89bf834b80 1 -- 172.21.15.2:0/2252877145 shutdown_connections 2023-09-13T04:41:25.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.684+0000 7f89bf834b80 1 --2- 172.21.15.2:0/2252877145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56301fc31110 0x56301fc2fbe0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.684+0000 7f89bf834b80 1 --2- 172.21.15.2:0/2252877145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc2b210 0x56301fc3ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.684+0000 7f89bf834b80 1 --2- 172.21.15.2:0/2252877145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56301fc307f0 0x56301fc178f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.684+0000 7f89bf834b80 1 -- 172.21.15.2:0/2252877145 >> 172.21.15.2:0/2252877145 conn(0x56301fb80c80 msgr2=0x56301fc2f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:25.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.684+0000 7f89bf834b80 1 -- 172.21.15.2:0/2252877145 shutdown_connections 2023-09-13T04:41:25.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.684+0000 7f89bf834b80 1 -- 172.21.15.2:0/2252877145 wait complete. 2023-09-13T04:41:25.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.685+0000 7f89bf834b80 1 -- start start 2023-09-13T04:41:25.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.685+0000 7f89bf834b80 1 Processor -- start 2023-09-13T04:41:25.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.686+0000 7f89bf834b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56301fc2b210 0x56301fc2d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.686+0000 7f89bf834b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc307f0 0x56301fc2dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.686+0000 7f89bf834b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56301fc31110 0x56301fc2c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.686+0000 7f89bf834b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301fc2c7b0 con 0x56301fc2b210 2023-09-13T04:41:25.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.686+0000 7f89bf834b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301fc2c920 con 0x56301fc31110 2023-09-13T04:41:25.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.686+0000 7f89bf834b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301fc2cae0 con 0x56301fc307f0 2023-09-13T04:41:25.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.686+0000 7f89bf834b80 10 client.0 _ll_register_callbacks cb 0x56301fb10680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:25.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.686+0000 7f89a959a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc307f0 0x56301fc2dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.686+0000 7f89aa59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56301fc31110 0x56301fc2c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.686+0000 7f89a959a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc307f0 0x56301fc2dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:57322/0 (socket says 172.21.15.2:57322) 2023-09-13T04:41:25.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.686+0000 7f89a959a700 1 -- 172.21.15.2:0/1507303852 learned_addr learned my addr 172.21.15.2:0/1507303852 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:25.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.687+0000 7f89a959a700 1 -- 172.21.15.2:0/1507303852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56301fc31110 msgr2=0x56301fc2c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:25.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.687+0000 7f89a959a700 1 --2- 172.21.15.2:0/1507303852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56301fc31110 0x56301fc2c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.687+0000 7f89a959a700 1 -- 172.21.15.2:0/1507303852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56301fc2b210 msgr2=0x56301fc2d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:25.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.687+0000 7f89a959a700 1 --2- 172.21.15.2:0/1507303852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56301fc2b210 0x56301fc2d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.687+0000 7f89a959a700 1 -- 172.21.15.2:0/1507303852 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56301fb8c7a0 con 0x56301fc307f0 2023-09-13T04:41:25.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.687+0000 7f89aa59c700 1 --2- 172.21.15.2:0/1507303852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56301fc31110 0x56301fc2c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:25.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.687+0000 7f89a959a700 1 --2- 172.21.15.2:0/1507303852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc307f0 0x56301fc2dac0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f899400f010 tx=0x7f8994095ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1290ba5f6d3512cd server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:25.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.687+0000 7f898d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:41:25.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.687+0000 7f898d7fa700 1 -- 172.21.15.2:0/1507303852 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8994099100 con 0x56301fc307f0 2023-09-13T04:41:25.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.688+0000 7f89bf834b80 1 -- 172.21.15.2:0/1507303852 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f899c00f040 con 0x56301fc307f0 2023-09-13T04:41:25.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.688+0000 7f898d7fa700 1 -- 172.21.15.2:0/1507303852 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f8994099ea0 con 0x56301fc307f0 2023-09-13T04:41:25.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.688+0000 7f898d7fa700 1 -- 172.21.15.2:0/1507303852 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89940997a0 con 0x56301fc307f0 2023-09-13T04:41:25.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.688+0000 7f89bf834b80 10 client.25885 fetch_fsmap learned FSMap version 23 2023-09-13T04:41:25.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.688+0000 7f89bf834b80 1 -- 172.21.15.2:0/1507303852 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56301f9f4630 con 0x56301fc307f0 2023-09-13T04:41:25.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.688+0000 7f898d7fa700 1 -- 172.21.15.2:0/1507303852 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f899c00f040 con 0x56301fc307f0 2023-09-13T04:41:25.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.688+0000 7f898d7fa700 1 -- 172.21.15.2:0/1507303852 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f89940cc450 con 0x56301fc307f0 2023-09-13T04:41:25.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.689+0000 7f89bf834b80 10 client.25885 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:41:25.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.689+0000 7f89bf834b80 10 client.25885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:41:25.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.689+0000 7f89bf834b80 20 client.25885 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:25.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.689+0000 7f89bf834b80 1 --2- 172.21.15.2:0/1507303852 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56301fc99640 0x56301fc9ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:25.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.689+0000 7f89bf834b80 4 client.25885 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-09-13T04:41:25.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.689+0000 7f89bf834b80 1 -- 172.21.15.2:0/1507303852 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56301fb2e050 con 0x56301fc99640 2023-09-13T04:41:25.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.689+0000 7f89a9d9b700 1 --2- 172.21.15.2:0/1507303852 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56301fc99640 0x56301fc9ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:25.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.690+0000 7f89a9d9b700 1 --2- 172.21.15.2:0/1507303852 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56301fc99640 0x56301fc9ba60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=663a31e7d45d9485 server_cookie=2e3b0d4414dc5460 in_seq=0 out_seq=0 2023-09-13T04:41:25.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.690+0000 7f898d7fa700 10 client.25885 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:25.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.690+0000 7f898d7fa700 1 -- 172.21.15.2:0/1507303852 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56301fb2e050 con 0x56301fc99640 2023-09-13T04:41:25.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.690+0000 7f898d7fa700 10 client.25885 handle_command_reply: tid=0 2023-09-13T04:41:25.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.690+0000 7f89bf834b80 1 client.25885 shutdown 2023-09-13T04:41:25.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.690+0000 7f89bf834b80 1 -- 172.21.15.2:0/1507303852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc307f0 msgr2=0x56301fc2dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:25.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.690+0000 7f89bf834b80 1 --2- 172.21.15.2:0/1507303852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc307f0 0x56301fc2dac0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f899400f010 tx=0x7f8994095ae0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.690+0000 7f89bf834b80 1 -- 172.21.15.2:0/1507303852 shutdown_connections 2023-09-13T04:41:25.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.690+0000 7f89bf834b80 1 --2- 172.21.15.2:0/1507303852 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56301fc99640 0x56301fc9ba60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.690+0000 7f89bf834b80 1 --2- 172.21.15.2:0/1507303852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56301fc31110 0x56301fc2c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.690+0000 7f89bf834b80 1 --2- 172.21.15.2:0/1507303852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56301fc2b210 0x56301fc2d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.690+0000 7f89bf834b80 1 --2- 172.21.15.2:0/1507303852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56301fc307f0 0x56301fc2dac0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.690+0000 7f89bf834b80 1 -- 172.21.15.2:0/1507303852 >> 172.21.15.2:0/1507303852 conn(0x56301fb80c80 msgr2=0x56301fc31d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:25.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.691+0000 7f898d7fa700 0 client.25885 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:25.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.691+0000 7f89bf834b80 1 -- 172.21.15.2:0/1507303852 shutdown_connections 2023-09-13T04:41:25.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.691+0000 7f89bf834b80 1 -- 172.21.15.2:0/1507303852 wait complete. 2023-09-13T04:41:25.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.691+0000 7f89bf834b80 20 client.25885 trim_cache size 0 max 16384 2023-09-13T04:41:25.704 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:41:25.704 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-13T04:41:25.704 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-09-13T04:41:25.704 INFO:teuthology.orchestra.run.smithi002.stdout: "7e2865f7-07c7-4a67-b469-10f73f1c731e": { 2023-09-13T04:41:25.705 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-09-13T04:41:25.705 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "7e2865f7-07c7-4a67-b469-10f73f1c731e", 2023-09-13T04:41:25.705 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-09-13T04:41:25.705 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:41:25.706 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:41:25.706 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:41:25.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.692+0000 7f89abfff700 1 -- 172.21.15.2:0/1424353702 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f89900a0950 msgr2=0x7f89900a2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:25.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.692+0000 7f89abfff700 1 --2- 172.21.15.2:0/1424353702 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f89900a0950 0x7f89900a2e10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f89a000b690 tx=0x7f89a000b260 comp rx=0 tx=0).stop 2023-09-13T04:41:25.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.692+0000 7f89abfff700 1 -- 172.21.15.2:0/1424353702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89a4105fd0 msgr2=0x7f89a4199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:25.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.692+0000 7f89abfff700 1 --2- 172.21.15.2:0/1424353702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89a4105fd0 0x7f89a4199580 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f899400e9c0 tx=0x7f899400ecd0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.692+0000 7f89abfff700 1 -- 172.21.15.2:0/1424353702 shutdown_connections 2023-09-13T04:41:25.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.692+0000 7f89abfff700 1 --2- 172.21.15.2:0/1424353702 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f89900a0950 0x7f89900a2e10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.692+0000 7f89abfff700 1 --2- 172.21.15.2:0/1424353702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a419d9e0 0x7f89a419de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.692+0000 7f89abfff700 1 --2- 172.21.15.2:0/1424353702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a4100eb0 0x7f89a4199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.692+0000 7f89abfff700 1 --2- 172.21.15.2:0/1424353702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89a4105fd0 0x7f89a4199580 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:25.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.692+0000 7f89abfff700 1 -- 172.21.15.2:0/1424353702 >> 172.21.15.2:0/1424353702 conn(0x7f89a40fc750 msgr2=0x7f89a410ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:25.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.692+0000 7f89abfff700 1 -- 172.21.15.2:0/1424353702 shutdown_connections 2023-09-13T04:41:25.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:25.692+0000 7f89abfff700 1 -- 172.21.15.2:0/1424353702 wait complete. 2023-09-13T04:41:25.722 INFO:tasks.cephfs.filesystem:scrub status for tag:7e2865f7-07c7-4a67-b469-10f73f1c731e - {'path': '/', 'tag': '7e2865f7-07c7-4a67-b469-10f73f1c731e', 'options': 'recursive,force'} 2023-09-13T04:41:26.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:25 smithi002 ceph-mon[97651]: pgmap v32: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2023-09-13T04:41:26.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:25 smithi149 ceph-mon[133315]: pgmap v32: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2023-09-13T04:41:26.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:25 smithi156 ceph-mon[133675]: pgmap v32: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2023-09-13T04:41:27.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:26 smithi149 ceph-mon[133315]: scrub queued for path: / 2023-09-13T04:41:27.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:26 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:41:27.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:26 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:41:27.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:26 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:41:27.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:26 smithi156 ceph-mon[133675]: scrub queued for path: / 2023-09-13T04:41:27.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:26 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:41:27.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:26 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:41:27.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:26 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:41:27.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:26 smithi002 ceph-mon[97651]: scrub queued for path: / 2023-09-13T04:41:27.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:26 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:41:27.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:26 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:41:27.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:26 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:41:28.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:27 smithi149 ceph-mon[133315]: pgmap v33: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-13T04:41:28.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:27 smithi156 ceph-mon[133675]: pgmap v33: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-13T04:41:28.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:27 smithi002 ceph-mon[97651]: pgmap v33: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-13T04:41:29.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:28 smithi149 ceph-mon[133315]: scrub summary: idle 2023-09-13T04:41:29.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:28 smithi156 ceph-mon[133675]: scrub summary: idle 2023-09-13T04:41:29.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:28 smithi002 ceph-mon[97651]: scrub summary: idle 2023-09-13T04:41:30.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:30 smithi149 ceph-mon[133315]: pgmap v34: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-09-13T04:41:30.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:30 smithi156 ceph-mon[133675]: pgmap v34: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-09-13T04:41:30.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:30 smithi002 ceph-mon[97651]: pgmap v34: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-09-13T04:41:32.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:31 smithi149 ceph-mon[133315]: pgmap v35: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-13T04:41:32.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:31 smithi156 ceph-mon[133675]: pgmap v35: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-13T04:41:32.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:31 smithi002 ceph-mon[97651]: pgmap v35: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-13T04:41:34.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:33 smithi149 ceph-mon[133315]: pgmap v36: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-13T04:41:34.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:33 smithi156 ceph-mon[133675]: pgmap v36: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-13T04:41:34.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:33 smithi002 ceph-mon[97651]: pgmap v36: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-13T04:41:35.963 INFO:tasks.workunit.client.0.smithi002.stderr:Updating files: 62% (7470/11967) Updating files: 63% (7540/11967) Updating files: 64% (7659/11967) Updating files: 65% (7779/11967) Updating files: 66% (7899/11967) Updating files: 67% (8018/11967) Updating files: 68% (8138/11967) Updating files: 69% (8258/11967) Updating files: 70% (8377/11967) Updating files: 71% (8497/11967) Updating files: 72% (8617/11967) Updating files: 73% (8736/11967) Updating files: 74% (8856/11967) Updating files: 75% (8976/11967) Updating files: 76% (9095/11967) Updating files: 77% (9215/11967) Updating files: 78% (9335/11967) Updating files: 79% (9454/11967) Updating files: 80% (9574/11967) Updating files: 81% (9694/11967) Updating files: 82% (9813/11967) Updating files: 83% (9933/11967) Updating files: 84% (10053/11967) Updating files: 85% (10172/11967) Updating files: 86% (10292/11967) Updating files: 87% (10412/11967) Updating files: 88% (10531/11967) Updating files: 89% (10651/11967) Updating files: 90% (10771/11967) Updating files: 91% (10890/11967) Updating files: 92% (11010/11967) Updating files: 93% (11130/11967) Updating files: 94% (11249/11967) Updating files: 95% (11369/11967) Updating files: 96% (11489/11967) Updating files: 97% (11608/11967) Updating files: 98% (11728/11967) Updating files: 99% (11848/11967) Updating files: 99% (11870/11967) Updating files: 100% (11967/11967) Updating files: 100% (11967/11967), done. 2023-09-13T04:41:36.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:35 smithi149 ceph-mon[133315]: pgmap v37: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2023-09-13T04:41:36.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:35 smithi156 ceph-mon[133675]: pgmap v37: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2023-09-13T04:41:36.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:35 smithi002 ceph-mon[97651]: pgmap v37: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2023-09-13T04:41:36.682 INFO:tasks.workunit.client.0.smithi002.stderr:Note: switching to '4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5'. 2023-09-13T04:41:36.683 INFO:tasks.workunit.client.0.smithi002.stderr: 2023-09-13T04:41:36.683 INFO:tasks.workunit.client.0.smithi002.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-09-13T04:41:36.683 INFO:tasks.workunit.client.0.smithi002.stderr:changes and commit them, and you can discard any commits you make in this 2023-09-13T04:41:36.683 INFO:tasks.workunit.client.0.smithi002.stderr:state without impacting any branches by switching back to a branch. 2023-09-13T04:41:36.684 INFO:tasks.workunit.client.0.smithi002.stderr: 2023-09-13T04:41:36.684 INFO:tasks.workunit.client.0.smithi002.stderr:If you want to create a new branch to retain commits you create, you may 2023-09-13T04:41:36.684 INFO:tasks.workunit.client.0.smithi002.stderr:do so (now or later) by using -c with the switch command. Example: 2023-09-13T04:41:36.684 INFO:tasks.workunit.client.0.smithi002.stderr: 2023-09-13T04:41:36.684 INFO:tasks.workunit.client.0.smithi002.stderr: git switch -c 2023-09-13T04:41:36.685 INFO:tasks.workunit.client.0.smithi002.stderr: 2023-09-13T04:41:36.685 INFO:tasks.workunit.client.0.smithi002.stderr:Or undo this operation with: 2023-09-13T04:41:36.685 INFO:tasks.workunit.client.0.smithi002.stderr: 2023-09-13T04:41:36.685 INFO:tasks.workunit.client.0.smithi002.stderr: git switch - 2023-09-13T04:41:36.685 INFO:tasks.workunit.client.0.smithi002.stderr: 2023-09-13T04:41:36.686 INFO:tasks.workunit.client.0.smithi002.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-09-13T04:41:36.686 INFO:tasks.workunit.client.0.smithi002.stderr: 2023-09-13T04:41:36.686 INFO:tasks.workunit.client.0.smithi002.stderr:HEAD is now at 4a754ac981a Merge remote-tracking branch 'leonid/config_format' into wip-rishabh-2023sep12-b2 2023-09-13T04:41:36.690 DEBUG:teuthology.orchestra.run.smithi002:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-09-13T04:41:36.741 INFO:tasks.workunit.client.0.smithi002.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-09-13T04:41:36.747 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-09-13T04:41:36.747 INFO:tasks.workunit.client.0.smithi002.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-09-13T04:41:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.900+0000 7f62e81ff700 1 -- 172.21.15.2:0/3454299762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62e0073a00 msgr2=0x7f62e010cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.900+0000 7f62e81ff700 1 --2- 172.21.15.2:0/3454299762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62e0073a00 0x7f62e010cf60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009990 tx=0x7f62d0009ca0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.900+0000 7f62e81ff700 1 -- 172.21.15.2:0/3454299762 shutdown_connections 2023-09-13T04:41:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.900+0000 7f62e81ff700 1 --2- 172.21.15.2:0/3454299762 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62e010d4a0 0x7f62e010f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.900+0000 7f62e81ff700 1 --2- 172.21.15.2:0/3454299762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62e00730e0 0x7f62e00734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.900+0000 7f62e81ff700 1 --2- 172.21.15.2:0/3454299762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62e0073a00 0x7f62e010cf60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.900+0000 7f62e81ff700 1 -- 172.21.15.2:0/3454299762 >> 172.21.15.2:0/3454299762 conn(0x7f62e00fc750 msgr2=0x7f62e00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.900+0000 7f62e81ff700 1 -- 172.21.15.2:0/3454299762 shutdown_connections 2023-09-13T04:41:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.900+0000 7f62e81ff700 1 -- 172.21.15.2:0/3454299762 wait complete. 2023-09-13T04:41:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.902+0000 7f62e81ff700 1 Processor -- start 2023-09-13T04:41:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.902+0000 7f62e81ff700 1 -- start start 2023-09-13T04:41:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.903+0000 7f62e81ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62e00730e0 0x7f62e0198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.903+0000 7f62e81ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62e0073a00 0x7f62e0199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.903+0000 7f62e81ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62e010d4a0 0x7f62e019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.903+0000 7f62e81ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e019e1b0 con 0x7f62e0073a00 2023-09-13T04:41:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.903+0000 7f62e81ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e019e320 con 0x7f62e010d4a0 2023-09-13T04:41:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.903+0000 7f62e81ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e019e4e0 con 0x7f62e00730e0 2023-09-13T04:41:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.903+0000 7f62e5f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62e00730e0 0x7f62e0198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.903+0000 7f62e579a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62e0073a00 0x7f62e0199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.903+0000 7f62e5f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62e00730e0 0x7f62e0198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:56592/0 (socket says 172.21.15.2:56592) 2023-09-13T04:41:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.903+0000 7f62e5f9b700 1 -- 172.21.15.2:0/671256261 learned_addr learned my addr 172.21.15.2:0/671256261 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.903+0000 7f62e679c700 1 --2- 172.21.15.2:0/671256261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62e010d4a0 0x7f62e019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.903+0000 7f62e579a700 1 -- 172.21.15.2:0/671256261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62e00730e0 msgr2=0x7f62e0198fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.903+0000 7f62e579a700 1 --2- 172.21.15.2:0/671256261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62e00730e0 0x7f62e0198fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.903+0000 7f62e579a700 1 -- 172.21.15.2:0/671256261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62e010d4a0 msgr2=0x7f62e019db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.903+0000 7f62e579a700 1 --2- 172.21.15.2:0/671256261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62e010d4a0 0x7f62e019db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.903+0000 7f62e579a700 1 -- 172.21.15.2:0/671256261 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d00095f0 con 0x7f62e0073a00 2023-09-13T04:41:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.903+0000 7f62e5f9b700 1 --2- 172.21.15.2:0/671256261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62e00730e0 0x7f62e0198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.903+0000 7f62e679c700 1 --2- 172.21.15.2:0/671256261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62e010d4a0 0x7f62e019db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.904+0000 7f62e579a700 1 --2- 172.21.15.2:0/671256261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62e0073a00 0x7f62e0199500 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f62d00059d0 tx=0x7f62d0005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.904+0000 7f62d6ffd700 1 -- 172.21.15.2:0/671256261 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d001d070 con 0x7f62e0073a00 2023-09-13T04:41:36.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.904+0000 7f62e81ff700 1 -- 172.21.15.2:0/671256261 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e019e760 con 0x7f62e0073a00 2023-09-13T04:41:36.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.904+0000 7f62d6ffd700 1 -- 172.21.15.2:0/671256261 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62d0022470 con 0x7f62e0073a00 2023-09-13T04:41:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.904+0000 7f62d6ffd700 1 -- 172.21.15.2:0/671256261 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0017650 con 0x7f62e0073a00 2023-09-13T04:41:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.904+0000 7f62e81ff700 1 -- 172.21.15.2:0/671256261 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e01a6000 con 0x7f62e0073a00 2023-09-13T04:41:36.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.909+0000 7f62fbbf3b80 1 Processor -- start 2023-09-13T04:41:36.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.909+0000 7f62d6ffd700 1 -- 172.21.15.2:0/671256261 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 24) v1 ==== 135491+0+0 (secure 0 0 0) 0x7f62d0022890 con 0x7f62e0073a00 2023-09-13T04:41:36.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.909+0000 7f62fbbf3b80 1 -- start start 2023-09-13T04:41:36.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.910+0000 7f62fbbf3b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56009ca1bbe0 0x56009ca1bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.910+0000 7f62fbbf3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1c500 0x56009ca17300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.910+0000 7f62fbbf3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56009ca17840 0x56009ca17c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.910+0000 7f62fbbf3b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56009c970720 con 0x56009ca1bbe0 2023-09-13T04:41:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.910+0000 7f62fbbf3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56009c96e040 con 0x56009ca17840 2023-09-13T04:41:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.910+0000 7f62fbbf3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56009c969930 con 0x56009ca1c500 2023-09-13T04:41:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.910+0000 7f62e679c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56009ca1bbe0 0x56009ca1bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.910+0000 7f62e679c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56009ca1bbe0 0x56009ca1bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47964/0 (socket says 172.21.15.2:47964) 2023-09-13T04:41:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.910+0000 7f62e679c700 1 -- 172.21.15.2:0/3332821247 learned_addr learned my addr 172.21.15.2:0/3332821247 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.910+0000 7f62e5f9b700 1 --2- 172.21.15.2:0/3332821247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1c500 0x56009ca17300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.911+0000 7f62e579a700 1 --2- 172.21.15.2:0/3332821247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56009ca17840 0x56009ca17c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.911+0000 7f62e5f9b700 1 -- 172.21.15.2:0/3332821247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56009ca17840 msgr2=0x56009ca17c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.911+0000 7f62e5f9b700 1 --2- 172.21.15.2:0/3332821247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56009ca17840 0x56009ca17c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.911+0000 7f62e5f9b700 1 -- 172.21.15.2:0/3332821247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56009ca1bbe0 msgr2=0x56009ca1bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.911+0000 7f62e5f9b700 1 --2- 172.21.15.2:0/3332821247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56009ca1bbe0 0x56009ca1bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.911+0000 7f62e5f9b700 1 -- 172.21.15.2:0/3332821247 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56009c7e95d0 con 0x56009ca1c500 2023-09-13T04:41:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.911+0000 7f62e679c700 1 --2- 172.21.15.2:0/3332821247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56009ca1bbe0 0x56009ca1bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.911+0000 7f62d6ffd700 1 --2- 172.21.15.2:0/671256261 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f62cc0a08d0 0x7f62cc0a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.911+0000 7f62e5f9b700 1 --2- 172.21.15.2:0/3332821247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1c500 0x56009ca17300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f62dc011570 tx=0x7f62dc011880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef6db389589909fa server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.911+0000 7f62d6ffd700 1 -- 172.21.15.2:0/671256261 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f62d00cd920 con 0x7f62e0073a00 2023-09-13T04:41:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.912+0000 7f62e679c700 1 --2- 172.21.15.2:0/671256261 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f62cc0a08d0 0x7f62cc0a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.912+0000 7f62cbfff700 1 -- 172.21.15.2:0/3332821247 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc004230 con 0x56009ca1c500 2023-09-13T04:41:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.912+0000 7f62cbfff700 1 -- 172.21.15.2:0/3332821247 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62dc004390 con 0x56009ca1c500 2023-09-13T04:41:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.912+0000 7f62cbfff700 1 -- 172.21.15.2:0/3332821247 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc0198e0 con 0x56009ca1c500 2023-09-13T04:41:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.912+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/3332821247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1c500 msgr2=0x56009ca17300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.912+0000 7f62e679c700 1 --2- 172.21.15.2:0/671256261 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f62cc0a08d0 0x7f62cc0a2d90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x56009ca13a70 tx=0x7f62d800b040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.912+0000 7f62fbbf3b80 1 --2- 172.21.15.2:0/3332821247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1c500 0x56009ca17300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f62dc011570 tx=0x7f62dc011880 comp rx=0 tx=0).stop 2023-09-13T04:41:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.913+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/3332821247 shutdown_connections 2023-09-13T04:41:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.913+0000 7f62fbbf3b80 1 --2- 172.21.15.2:0/3332821247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56009ca17840 0x56009ca17c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.913+0000 7f62fbbf3b80 1 --2- 172.21.15.2:0/3332821247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56009ca1bbe0 0x56009ca1bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.913+0000 7f62fbbf3b80 1 --2- 172.21.15.2:0/3332821247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1c500 0x56009ca17300 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.913+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/3332821247 >> 172.21.15.2:0/3332821247 conn(0x56009c975c20 msgr2=0x56009ca11b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.913+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/3332821247 shutdown_connections 2023-09-13T04:41:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.913+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/3332821247 wait complete. 2023-09-13T04:41:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.913+0000 7f62fbbf3b80 1 -- start start 2023-09-13T04:41:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.914+0000 7f62fbbf3b80 1 Processor -- start 2023-09-13T04:41:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.914+0000 7f62fbbf3b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56009ca17840 0x56009ca23600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.914+0000 7f62fbbf3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1bbe0 0x56009ca20810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.914+0000 7f62fbbf3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56009ca1c500 0x56009ca20f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.914+0000 7f62fbbf3b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56009c907b50 con 0x56009ca17840 2023-09-13T04:41:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.914+0000 7f62fbbf3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56009c907850 con 0x56009ca1c500 2023-09-13T04:41:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.914+0000 7f62fbbf3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56009c7eb8c0 con 0x56009ca1bbe0 2023-09-13T04:41:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.914+0000 7f62fbbf3b80 10 client.0 _ll_register_callbacks cb 0x56009c905620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.914+0000 7f62e579a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1bbe0 0x56009ca20810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.914+0000 7f62e5f9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56009ca17840 0x56009ca23600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.914+0000 7f62e579a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1bbe0 0x56009ca20810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:56616/0 (socket says 172.21.15.2:56616) 2023-09-13T04:41:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.914+0000 7f62e579a700 1 -- 172.21.15.2:0/1244279757 learned_addr learned my addr 172.21.15.2:0/1244279757 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.914+0000 7f62e679c700 1 --2- 172.21.15.2:0/1244279757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56009ca1c500 0x56009ca20f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.915+0000 7f62e579a700 1 -- 172.21.15.2:0/1244279757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56009ca1c500 msgr2=0x56009ca20f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.915+0000 7f62e579a700 1 --2- 172.21.15.2:0/1244279757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56009ca1c500 0x56009ca20f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.915+0000 7f62e579a700 1 -- 172.21.15.2:0/1244279757 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56009ca17840 msgr2=0x56009ca23600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.915+0000 7f62e579a700 1 --2- 172.21.15.2:0/1244279757 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56009ca17840 0x56009ca23600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.915+0000 7f62e579a700 1 -- 172.21.15.2:0/1244279757 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62dc009d60 con 0x56009ca1bbe0 2023-09-13T04:41:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.915+0000 7f62e679c700 1 --2- 172.21.15.2:0/1244279757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56009ca1c500 0x56009ca20f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:41:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.915+0000 7f62e579a700 1 --2- 172.21.15.2:0/1244279757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1bbe0 0x56009ca20810 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f62d0006010 tx=0x7f62d009af60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45fa1fb8b889e335 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.915+0000 7f62c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:41:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.915+0000 7f62c97fa700 1 -- 172.21.15.2:0/1244279757 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d00a2770 con 0x56009ca1bbe0 2023-09-13T04:41:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.915+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/1244279757 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56009c9dcef0 con 0x56009ca1bbe0 2023-09-13T04:41:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.915+0000 7f62c97fa700 1 -- 172.21.15.2:0/1244279757 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f62d009b120 con 0x56009ca1bbe0 2023-09-13T04:41:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.916+0000 7f62c97fa700 1 -- 172.21.15.2:0/1244279757 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62d0009290 con 0x56009ca1bbe0 2023-09-13T04:41:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.916+0000 7f62fbbf3b80 10 client.25897 fetch_fsmap learned FSMap version 23 2023-09-13T04:41:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.916+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/1244279757 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56009c981740 con 0x56009ca1bbe0 2023-09-13T04:41:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.916+0000 7f62c97fa700 1 -- 172.21.15.2:0/1244279757 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56009c9dcef0 con 0x56009ca1bbe0 2023-09-13T04:41:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.916+0000 7f62c97fa700 1 -- 172.21.15.2:0/1244279757 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f62d00af030 con 0x56009ca1bbe0 2023-09-13T04:41:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.916+0000 7f62fbbf3b80 10 client.25897 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:41:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.916+0000 7f62fbbf3b80 10 client.25897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:41:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.916+0000 7f62fbbf3b80 20 client.25897 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.916+0000 7f62fbbf3b80 1 --2- 172.21.15.2:0/1244279757 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56009ca30b30 0x56009ca32f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.916+0000 7f62fbbf3b80 4 client.25897 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:41:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.916+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/1244279757 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56009c962910 con 0x56009ca30b30 2023-09-13T04:41:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.916+0000 7f62e5f9b700 1 --2- 172.21.15.2:0/1244279757 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56009ca30b30 0x56009ca32f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.917+0000 7f62e5f9b700 1 --2- 172.21.15.2:0/1244279757 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56009ca30b30 0x56009ca32f50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8662df5c70e33ba1 server_cookie=5c3a3c48f5b02d4d in_seq=0 out_seq=0 2023-09-13T04:41:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.917+0000 7f62c97fa700 10 client.25897 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.918+0000 7f62c97fa700 1 -- 172.21.15.2:0/1244279757 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56009c962910 con 0x56009ca30b30 2023-09-13T04:41:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.918+0000 7f62c97fa700 10 client.25897 handle_command_reply: tid=0 2023-09-13T04:41:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.918+0000 7f62fbbf3b80 1 client.25897 shutdown 2023-09-13T04:41:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.918+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/1244279757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1bbe0 msgr2=0x56009ca20810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.918+0000 7f62fbbf3b80 1 --2- 172.21.15.2:0/1244279757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1bbe0 0x56009ca20810 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f62d0006010 tx=0x7f62d009af60 comp rx=0 tx=0).stop 2023-09-13T04:41:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.918+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/1244279757 shutdown_connections 2023-09-13T04:41:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.918+0000 7f62fbbf3b80 1 --2- 172.21.15.2:0/1244279757 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56009ca30b30 0x56009ca32f50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.918+0000 7f62fbbf3b80 1 --2- 172.21.15.2:0/1244279757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56009ca1c500 0x56009ca20f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.918+0000 7f62fbbf3b80 1 --2- 172.21.15.2:0/1244279757 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56009ca17840 0x56009ca23600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.918+0000 7f62c97fa700 0 client.25897 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.918+0000 7f62fbbf3b80 1 --2- 172.21.15.2:0/1244279757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1bbe0 0x56009ca20810 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.918+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/1244279757 >> 172.21.15.2:0/1244279757 conn(0x56009c975c20 msgr2=0x56009ca0c630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.918+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/1244279757 shutdown_connections 2023-09-13T04:41:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.918+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/1244279757 wait complete. 2023-09-13T04:41:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.918+0000 7f62fbbf3b80 20 client.25897 trim_cache size 0 max 16384 2023-09-13T04:41:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.933+0000 7f62fbbf3b80 1 Processor -- start 2023-09-13T04:41:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.934+0000 7f62fbbf3b80 1 -- start start 2023-09-13T04:41:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.934+0000 7f62fbbf3b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56009ca17840 0x56009ca23130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.934+0000 7f62fbbf3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1bbe0 0x56009ca1dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.934+0000 7f62fbbf3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56009ca1c500 0x56009ca1e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.934+0000 7f62fbbf3b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56009ca21ca0 con 0x56009ca17840 2023-09-13T04:41:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.934+0000 7f62fbbf3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56009ca1b750 con 0x56009ca1c500 2023-09-13T04:41:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.934+0000 7f62fbbf3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56009ca1e720 con 0x56009ca1bbe0 2023-09-13T04:41:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.934+0000 7f62e5f9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56009ca17840 0x56009ca23130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.934+0000 7f62e5f9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56009ca17840 0x56009ca23130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47984/0 (socket says 172.21.15.2:47984) 2023-09-13T04:41:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.934+0000 7f62e679c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56009ca1c500 0x56009ca1e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.934+0000 7f62e5f9b700 1 -- 172.21.15.2:0/2909116251 learned_addr learned my addr 172.21.15.2:0/2909116251 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.934+0000 7f62e579a700 1 --2- 172.21.15.2:0/2909116251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1bbe0 0x56009ca1dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.935+0000 7f62e579a700 1 -- 172.21.15.2:0/2909116251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56009ca1c500 msgr2=0x56009ca1e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.935+0000 7f62e579a700 1 --2- 172.21.15.2:0/2909116251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56009ca1c500 0x56009ca1e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.935+0000 7f62e579a700 1 -- 172.21.15.2:0/2909116251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56009ca17840 msgr2=0x56009ca23130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.935+0000 7f62e579a700 1 --2- 172.21.15.2:0/2909116251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56009ca17840 0x56009ca23130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.935+0000 7f62e579a700 1 -- 172.21.15.2:0/2909116251 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56009c795d80 con 0x56009ca1bbe0 2023-09-13T04:41:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.935+0000 7f62e579a700 1 --2- 172.21.15.2:0/2909116251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1bbe0 0x56009ca1dca0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f62d009a9b0 tx=0x7f62d00cdc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e1f10f69aaab38b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.935+0000 7f62d4ff9700 1 -- 172.21.15.2:0/2909116251 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d00a2770 con 0x56009ca1bbe0 2023-09-13T04:41:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.935+0000 7f62d4ff9700 1 -- 172.21.15.2:0/2909116251 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62d0009290 con 0x56009ca1bbe0 2023-09-13T04:41:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.935+0000 7f62d4ff9700 1 -- 172.21.15.2:0/2909116251 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d00cb120 con 0x56009ca1bbe0 2023-09-13T04:41:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.936+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/2909116251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1bbe0 msgr2=0x56009ca1dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.936+0000 7f62fbbf3b80 1 --2- 172.21.15.2:0/2909116251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1bbe0 0x56009ca1dca0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f62d009a9b0 tx=0x7f62d00cdc70 comp rx=0 tx=0).stop 2023-09-13T04:41:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.936+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/2909116251 shutdown_connections 2023-09-13T04:41:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.936+0000 7f62fbbf3b80 1 --2- 172.21.15.2:0/2909116251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56009ca1c500 0x56009ca1e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.936+0000 7f62fbbf3b80 1 --2- 172.21.15.2:0/2909116251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56009ca17840 0x56009ca23130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.936+0000 7f62fbbf3b80 1 --2- 172.21.15.2:0/2909116251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1bbe0 0x56009ca1dca0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.936+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/2909116251 >> 172.21.15.2:0/2909116251 conn(0x56009c975c20 msgr2=0x56009ca168d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:36.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.936+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/2909116251 shutdown_connections 2023-09-13T04:41:36.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.936+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/2909116251 wait complete. 2023-09-13T04:41:36.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.937+0000 7f62fbbf3b80 1 -- start start 2023-09-13T04:41:36.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.937+0000 7f62fbbf3b80 1 Processor -- start 2023-09-13T04:41:36.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.937+0000 7f62fbbf3b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56009ca17840 0x56009ca23f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:36.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.937+0000 7f62fbbf3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1bbe0 0x56009ca244c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:36.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.937+0000 7f62fbbf3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56009ca1c500 0x56009ca24a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:36.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.937+0000 7f62fbbf3b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56009ca83ab0 con 0x56009ca17840 2023-09-13T04:41:36.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.937+0000 7f62fbbf3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56009ca83c20 con 0x56009ca1c500 2023-09-13T04:41:36.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.937+0000 7f62fbbf3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56009ca83de0 con 0x56009ca1bbe0 2023-09-13T04:41:36.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.937+0000 7f62fbbf3b80 10 client.0 _ll_register_callbacks cb 0x56009c905620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:36.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.938+0000 7f62e579a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1bbe0 0x56009ca244c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:36.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.938+0000 7f62e5f9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56009ca17840 0x56009ca23f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:36.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.938+0000 7f62e579a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1bbe0 0x56009ca244c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:56636/0 (socket says 172.21.15.2:56636) 2023-09-13T04:41:36.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.938+0000 7f62e579a700 1 -- 172.21.15.2:0/1874422922 learned_addr learned my addr 172.21.15.2:0/1874422922 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:36.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.938+0000 7f62e679c700 1 --2- 172.21.15.2:0/1874422922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56009ca1c500 0x56009ca24a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.938+0000 7f62e579a700 1 -- 172.21.15.2:0/1874422922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56009ca1c500 msgr2=0x56009ca24a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.938+0000 7f62e579a700 1 --2- 172.21.15.2:0/1874422922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56009ca1c500 0x56009ca24a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.938+0000 7f62e579a700 1 -- 172.21.15.2:0/1874422922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56009ca17840 msgr2=0x56009ca23f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.938+0000 7f62e579a700 1 --2- 172.21.15.2:0/1874422922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56009ca17840 0x56009ca23f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.938+0000 7f62e579a700 1 -- 172.21.15.2:0/1874422922 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d00d3040 con 0x56009ca1bbe0 2023-09-13T04:41:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.938+0000 7f62e5f9b700 1 --2- 172.21.15.2:0/1874422922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56009ca17840 0x56009ca23f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.938+0000 7f62e579a700 1 --2- 172.21.15.2:0/1874422922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1bbe0 0x56009ca244c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f62d009abe0 tx=0x7f62d00cb0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=732b228fdd296c01 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.938+0000 7f62c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:41:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.939+0000 7f62c97fa700 1 -- 172.21.15.2:0/1874422922 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d00d3040 con 0x56009ca1bbe0 2023-09-13T04:41:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.939+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/1874422922 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62dc004390 con 0x56009ca1bbe0 2023-09-13T04:41:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.939+0000 7f62c97fa700 1 -- 172.21.15.2:0/1874422922 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f62d009b120 con 0x56009ca1bbe0 2023-09-13T04:41:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.939+0000 7f62c97fa700 1 -- 172.21.15.2:0/1874422922 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62d009a7f0 con 0x56009ca1bbe0 2023-09-13T04:41:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.939+0000 7f62fbbf3b80 10 client.25903 fetch_fsmap learned FSMap version 23 2023-09-13T04:41:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.939+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/1874422922 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56009c77ec00 con 0x56009ca1bbe0 2023-09-13T04:41:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.939+0000 7f62c97fa700 1 -- 172.21.15.2:0/1874422922 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62dc004390 con 0x56009ca1bbe0 2023-09-13T04:41:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.939+0000 7f62c97fa700 1 -- 172.21.15.2:0/1874422922 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f62d00af070 con 0x56009ca1bbe0 2023-09-13T04:41:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.939+0000 7f62fbbf3b80 10 client.25903 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:41:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.939+0000 7f62fbbf3b80 10 client.25903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:41:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.939+0000 7f62fbbf3b80 20 client.25903 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.939+0000 7f62fbbf3b80 1 --2- 172.21.15.2:0/1874422922 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56009ca85040 0x56009ca87430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.939+0000 7f62fbbf3b80 4 client.25903 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:41:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.939+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/1874422922 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56009c922ff0 con 0x56009ca85040 2023-09-13T04:41:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.940+0000 7f62e5f9b700 1 --2- 172.21.15.2:0/1874422922 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56009ca85040 0x56009ca87430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.940+0000 7f62e5f9b700 1 --2- 172.21.15.2:0/1874422922 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56009ca85040 0x56009ca87430 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c51f33aff27b5b90 server_cookie=cd2c849216b86b26 in_seq=0 out_seq=0 2023-09-13T04:41:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.940+0000 7f62c97fa700 10 client.25903 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.944+0000 7f62c97fa700 1 -- 172.21.15.2:0/1874422922 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56009c922ff0 con 0x56009ca85040 2023-09-13T04:41:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.944+0000 7f62c97fa700 10 client.25903 handle_command_reply: tid=0 2023-09-13T04:41:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.944+0000 7f62fbbf3b80 1 client.25903 shutdown 2023-09-13T04:41:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.944+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/1874422922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1bbe0 msgr2=0x56009ca244c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.944+0000 7f62fbbf3b80 1 --2- 172.21.15.2:0/1874422922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1bbe0 0x56009ca244c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f62d009abe0 tx=0x7f62d00cb0f0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.944+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/1874422922 shutdown_connections 2023-09-13T04:41:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.944+0000 7f62fbbf3b80 1 --2- 172.21.15.2:0/1874422922 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56009ca85040 0x56009ca87430 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.944+0000 7f62fbbf3b80 1 --2- 172.21.15.2:0/1874422922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56009ca1c500 0x56009ca24a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.944+0000 7f62fbbf3b80 1 --2- 172.21.15.2:0/1874422922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56009ca17840 0x56009ca23f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.944+0000 7f62fbbf3b80 1 --2- 172.21.15.2:0/1874422922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56009ca1bbe0 0x56009ca244c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.944+0000 7f62c97fa700 0 client.25903 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.944+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/1874422922 >> 172.21.15.2:0/1874422922 conn(0x56009c975c20 msgr2=0x56009ca1a740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.944+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/1874422922 shutdown_connections 2023-09-13T04:41:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.944+0000 7f62fbbf3b80 1 -- 172.21.15.2:0/1874422922 wait complete. 2023-09-13T04:41:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.944+0000 7f62fbbf3b80 20 client.25903 trim_cache size 0 max 16384 2023-09-13T04:41:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:41:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:41:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:41:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:41:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.945+0000 7f62e81ff700 1 -- 172.21.15.2:0/671256261 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f62cc0a08d0 msgr2=0x7f62cc0a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.945+0000 7f62e81ff700 1 --2- 172.21.15.2:0/671256261 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f62cc0a08d0 0x7f62cc0a2d90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x56009ca13a70 tx=0x7f62d800b040 comp rx=0 tx=0).stop 2023-09-13T04:41:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.945+0000 7f62e81ff700 1 -- 172.21.15.2:0/671256261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62e0073a00 msgr2=0x7f62e0199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.945+0000 7f62e81ff700 1 --2- 172.21.15.2:0/671256261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62e0073a00 0x7f62e0199500 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f62d00059d0 tx=0x7f62d0005720 comp rx=0 tx=0).stop 2023-09-13T04:41:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.945+0000 7f62e81ff700 1 -- 172.21.15.2:0/671256261 shutdown_connections 2023-09-13T04:41:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.945+0000 7f62e81ff700 1 --2- 172.21.15.2:0/671256261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62e010d4a0 0x7f62e019db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.945+0000 7f62e81ff700 1 --2- 172.21.15.2:0/671256261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62e00730e0 0x7f62e0198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.945+0000 7f62e81ff700 1 --2- 172.21.15.2:0/671256261 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f62cc0a08d0 0x7f62cc0a2d90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.945+0000 7f62e81ff700 1 --2- 172.21.15.2:0/671256261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62e0073a00 0x7f62e0199500 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.945+0000 7f62e81ff700 1 -- 172.21.15.2:0/671256261 >> 172.21.15.2:0/671256261 conn(0x7f62e00fc750 msgr2=0x7f62e010f670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.945+0000 7f62e81ff700 1 -- 172.21.15.2:0/671256261 shutdown_connections 2023-09-13T04:41:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:36.945+0000 7f62e81ff700 1 -- 172.21.15.2:0/671256261 wait complete. 2023-09-13T04:41:37.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:36 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:41:37.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:36 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:41:37.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:36 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:41:37.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:36 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:41:37.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:36 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:41:37.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:36 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:41:37.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:36 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:41:37.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:36 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:41:37.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:36 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:41:37.478 INFO:tasks.workunit.client.0.smithi002.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-09-13T04:41:37.572 INFO:tasks.workunit.client.0.smithi002.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-09-13T04:41:37.637 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-09-13T04:41:37.641 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-09-13T04:41:37.641 INFO:tasks.workunit.client.0.smithi002.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-09-13T04:41:37.705 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-09-13T04:41:37.712 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-09-13T04:41:37.713 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-09-13T04:41:37.796 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2023-09-13T04:41:37.798 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2023-09-13T04:41:37.798 DEBUG:teuthology.orchestra.run.smithi002:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2023-09-13T04:41:37.896 INFO:tasks.workunit.client.0.smithi002.stdout:getting blogbench 2023-09-13T04:41:37.896 INFO:tasks.workunit.client.0.smithi002.stderr:+ echo 'getting blogbench' 2023-09-13T04:41:37.896 INFO:tasks.workunit.client.0.smithi002.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2023-09-13T04:41:37.928 INFO:tasks.workunit.client.0.smithi002.stderr:--2023-09-13 04:41:37-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2023-09-13T04:41:37.929 INFO:tasks.workunit.client.0.smithi002.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2023-09-13T04:41:37.951 INFO:tasks.workunit.client.0.smithi002.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2023-09-13T04:41:37.971 INFO:tasks.workunit.client.0.smithi002.stderr:HTTP request sent, awaiting response... 200 OK 2023-09-13T04:41:37.971 INFO:tasks.workunit.client.0.smithi002.stderr:Length: 103355 (101K) [application/octet-stream] 2023-09-13T04:41:37.973 INFO:tasks.workunit.client.0.smithi002.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2023-09-13T04:41:37.977 INFO:tasks.workunit.client.0.smithi002.stderr: 2023-09-13T04:41:38.012 INFO:tasks.workunit.client.0.smithi002.stderr: 0K .......... .......... .......... .......... .......... 49% 1.28M 0s 2023-09-13T04:41:38.033 INFO:tasks.workunit.client.0.smithi002.stderr: 50K .......... .......... .......... .......... .......... 99% 2.41M 0s 2023-09-13T04:41:38.033 INFO:tasks.workunit.client.0.smithi002.stderr: 100K 100% 1779G=0.06s 2023-09-13T04:41:38.034 INFO:tasks.workunit.client.0.smithi002.stderr: 2023-09-13T04:41:38.034 INFO:tasks.workunit.client.0.smithi002.stderr:2023-09-13 04:41:38 (1.69 MB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2023-09-13T04:41:38.034 INFO:tasks.workunit.client.0.smithi002.stderr: 2023-09-13T04:41:38.035 INFO:tasks.workunit.client.0.smithi002.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2023-09-13T04:41:38.070 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/ 2023-09-13T04:41:38.072 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/m4/ 2023-09-13T04:41:38.073 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/m4/acx_pthread.m4 2023-09-13T04:41:38.074 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/man/ 2023-09-13T04:41:38.075 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/man/blogbench.8 2023-09-13T04:41:38.076 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/man/Makefile.am 2023-09-13T04:41:38.077 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/man/Makefile.in 2023-09-13T04:41:38.077 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/src/ 2023-09-13T04:41:38.079 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/src/mysnprintf.c 2023-09-13T04:41:38.083 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/src/mysnprintf.h 2023-09-13T04:41:38.083 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/src/blogbench.c 2023-09-13T04:41:38.083 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/src/blogbench.h 2023-09-13T04:41:38.083 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/src/fakesnprintf.c 2023-09-13T04:41:38.084 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/src/fakesnprintf.h 2023-09-13T04:41:38.084 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/src/blogbench_p.h 2023-09-13T04:41:38.084 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/src/bsd-getopt_long.c 2023-09-13T04:41:38.084 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/src/bsd-getopt_long.h 2023-09-13T04:41:38.085 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/src/Makefile.am 2023-09-13T04:41:38.085 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/src/Makefile.in 2023-09-13T04:41:38.085 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/src/process.c 2023-09-13T04:41:38.085 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/src/globals.h 2023-09-13T04:41:38.085 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/src/commenter.c 2023-09-13T04:41:38.086 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/src/rewriter.c 2023-09-13T04:41:38.086 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/src/reader.c 2023-09-13T04:41:38.086 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/src/writer.c 2023-09-13T04:41:38.086 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/src/helpers.c 2023-09-13T04:41:38.087 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/NEWS 2023-09-13T04:41:38.094 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/compile 2023-09-13T04:41:38.094 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/depcomp 2023-09-13T04:41:38.094 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/aclocal.m4 2023-09-13T04:41:38.095 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/README 2023-09-13T04:41:38.095 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/configure 2023-09-13T04:41:38.099 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/configure.ac 2023-09-13T04:41:38.100 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/config.guess 2023-09-13T04:41:38.100 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/install-sh 2023-09-13T04:41:38.100 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/config.sub 2023-09-13T04:41:38.101 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/missing 2023-09-13T04:41:38.101 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/mkinstalldirs 2023-09-13T04:41:38.101 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/Makefile.am 2023-09-13T04:41:38.101 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/Makefile.in 2023-09-13T04:41:38.102 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/config.h.in 2023-09-13T04:41:38.102 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/AUTHORS 2023-09-13T04:41:38.102 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/INSTALL 2023-09-13T04:41:38.103 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/ChangeLog 2023-09-13T04:41:38.103 INFO:tasks.workunit.client.0.smithi002.stdout:blogbench-1.0/COPYING 2023-09-13T04:41:38.104 INFO:tasks.workunit.client.0.smithi002.stderr:+ cd blogbench-1.0/ 2023-09-13T04:41:38.104 INFO:tasks.workunit.client.0.smithi002.stderr:+ echo 'making blogbench' 2023-09-13T04:41:38.105 INFO:tasks.workunit.client.0.smithi002.stdout:making blogbench 2023-09-13T04:41:38.106 INFO:tasks.workunit.client.0.smithi002.stderr:+ ./configure 2023-09-13T04:41:38.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:37 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:41:38.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:37 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:41:38.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:37 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:41:38.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:37 smithi149 ceph-mon[133315]: pgmap v38: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:41:38.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:37 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:41:38.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:37 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:41:38.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:37 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:41:38.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:37 smithi156 ceph-mon[133675]: pgmap v38: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:41:38.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:37 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:41:38.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:37 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:41:38.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:37 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:41:38.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:37 smithi002 ceph-mon[97651]: pgmap v38: 129 pgs: 129 active+clean; 1.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:41:38.350 INFO:tasks.workunit.client.0.smithi002.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2023-09-13T04:41:39.362 INFO:tasks.workunit.client.0.smithi002.stdout:checking whether build environment is sane... yes 2023-09-13T04:41:39.381 INFO:tasks.workunit.client.0.smithi002.stdout:checking for gawk... gawk 2023-09-13T04:41:39.442 INFO:tasks.workunit.client.0.smithi002.stdout:checking whether make sets $(MAKE)... yes 2023-09-13T04:41:39.463 INFO:tasks.workunit.client.0.smithi002.stdout:checking for gcc... gcc 2023-09-13T04:41:39.562 INFO:tasks.workunit.client.0.smithi002.stdout:checking for C compiler default output file name... a.out 2023-09-13T04:41:39.567 INFO:tasks.workunit.client.0.smithi002.stdout:checking whether the C compiler works... yes 2023-09-13T04:41:39.571 INFO:tasks.workunit.client.0.smithi002.stdout:checking whether we are cross compiling... no 2023-09-13T04:41:39.633 INFO:tasks.workunit.client.0.smithi002.stdout:checking for suffix of executables... 2023-09-13T04:41:39.697 INFO:tasks.workunit.client.0.smithi002.stdout:checking for suffix of object files... o 2023-09-13T04:41:39.758 INFO:tasks.workunit.client.0.smithi002.stdout:checking whether we are using the GNU C compiler... yes 2023-09-13T04:41:39.819 INFO:tasks.workunit.client.0.smithi002.stdout:checking whether gcc accepts -g... yes 2023-09-13T04:41:39.895 INFO:tasks.workunit.client.0.smithi002.stdout:checking for gcc option to accept ANSI C... none needed 2023-09-13T04:41:39.957 INFO:tasks.workunit.client.0.smithi002.stdout:checking for style of include used by make... GNU 2023-09-13T04:41:40.082 INFO:tasks.workunit.client.0.smithi002.stdout:checking dependency style of gcc... gcc3 2023-09-13T04:41:40.083 INFO:tasks.workunit.client.0.smithi002.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2023-09-13T04:41:40.173 INFO:tasks.workunit.client.0.smithi002.stdout:checking how to run the C preprocessor... gcc -E 2023-09-13T04:41:40.241 INFO:tasks.workunit.client.0.smithi002.stdout:checking for egrep... grep -E 2023-09-13T04:41:40.258 INFO:tasks.workunit.client.0.smithi002.stdout:checking for AIX... no 2023-09-13T04:41:40.357 INFO:tasks.workunit.client.0.smithi002.stdout:checking for library containing strerror... none required 2023-09-13T04:41:40.601 INFO:tasks.workunit.client.0.smithi002.stdout:checking for ANSI C header files... yes 2023-09-13T04:41:40.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:40 smithi149 ceph-mon[133315]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 17 KiB/s wr, 8 op/s 2023-09-13T04:41:40.670 INFO:tasks.workunit.client.0.smithi002.stdout:checking whether time.h and sys/time.h may both be included... yes 2023-09-13T04:41:40.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:40 smithi156 ceph-mon[133675]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 17 KiB/s wr, 8 op/s 2023-09-13T04:41:40.752 INFO:tasks.workunit.client.0.smithi002.stdout:checking for sys/types.h... yes 2023-09-13T04:41:40.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:40 smithi002 ceph-mon[97651]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 17 KiB/s wr, 8 op/s 2023-09-13T04:41:40.840 INFO:tasks.workunit.client.0.smithi002.stdout:checking for sys/stat.h... yes 2023-09-13T04:41:40.927 INFO:tasks.workunit.client.0.smithi002.stdout:checking for stdlib.h... yes 2023-09-13T04:41:41.017 INFO:tasks.workunit.client.0.smithi002.stdout:checking for string.h... yes 2023-09-13T04:41:41.106 INFO:tasks.workunit.client.0.smithi002.stdout:checking for memory.h... yes 2023-09-13T04:41:41.195 INFO:tasks.workunit.client.0.smithi002.stdout:checking for strings.h... yes 2023-09-13T04:41:41.285 INFO:tasks.workunit.client.0.smithi002.stdout:checking for inttypes.h... yes 2023-09-13T04:41:41.375 INFO:tasks.workunit.client.0.smithi002.stdout:checking for stdint.h... yes 2023-09-13T04:41:41.467 INFO:tasks.workunit.client.0.smithi002.stdout:checking for unistd.h... yes 2023-09-13T04:41:41.490 INFO:tasks.workunit.client.0.smithi002.stdout:checking for memory.h... (cached) yes 2023-09-13T04:41:41.506 INFO:tasks.workunit.client.0.smithi002.stdout:checking for strings.h... (cached) yes 2023-09-13T04:41:41.523 INFO:tasks.workunit.client.0.smithi002.stdout:checking for unistd.h... (cached) yes 2023-09-13T04:41:41.606 INFO:tasks.workunit.client.0.smithi002.stdout:checking locale.h usability... yes 2023-09-13T04:41:41.638 INFO:tasks.workunit.client.0.smithi002.stdout:checking locale.h presence... yes 2023-09-13T04:41:41.639 INFO:tasks.workunit.client.0.smithi002.stdout:checking for locale.h... yes 2023-09-13T04:41:41.666 INFO:tasks.workunit.client.0.smithi002.stdout:checking for stdlib.h... (cached) yes 2023-09-13T04:41:41.683 INFO:tasks.workunit.client.0.smithi002.stdout:checking for string.h... (cached) yes 2023-09-13T04:41:41.793 INFO:tasks.workunit.client.0.smithi002.stdout:checking sys/socket.h usability... yes 2023-09-13T04:41:41.827 INFO:tasks.workunit.client.0.smithi002.stdout:checking sys/socket.h presence... yes 2023-09-13T04:41:41.829 INFO:tasks.workunit.client.0.smithi002.stdout:checking for sys/socket.h... yes 2023-09-13T04:41:41.930 INFO:tasks.workunit.client.0.smithi002.stdout:checking sys/time.h usability... yes 2023-09-13T04:41:41.963 INFO:tasks.workunit.client.0.smithi002.stdout:checking sys/time.h presence... yes 2023-09-13T04:41:41.965 INFO:tasks.workunit.client.0.smithi002.stdout:checking for sys/time.h... yes 2023-09-13T04:41:42.058 INFO:tasks.workunit.client.0.smithi002.stdout:checking fcntl.h usability... yes 2023-09-13T04:41:42.090 INFO:tasks.workunit.client.0.smithi002.stdout:checking fcntl.h presence... yes 2023-09-13T04:41:42.092 INFO:tasks.workunit.client.0.smithi002.stdout:checking for fcntl.h... yes 2023-09-13T04:41:42.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:41 smithi149 ceph-mon[133315]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 17 KiB/s wr, 5 op/s 2023-09-13T04:41:42.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:41 smithi156 ceph-mon[133675]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 17 KiB/s wr, 5 op/s 2023-09-13T04:41:42.195 INFO:tasks.workunit.client.0.smithi002.stdout:checking sys/fcntl.h usability... yes 2023-09-13T04:41:42.227 INFO:tasks.workunit.client.0.smithi002.stdout:checking sys/fcntl.h presence... yes 2023-09-13T04:41:42.229 INFO:tasks.workunit.client.0.smithi002.stdout:checking for sys/fcntl.h... yes 2023-09-13T04:41:42.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:41 smithi002 ceph-mon[97651]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 17 KiB/s wr, 5 op/s 2023-09-13T04:41:42.342 INFO:tasks.workunit.client.0.smithi002.stdout:checking sys/param.h usability... yes 2023-09-13T04:41:42.378 INFO:tasks.workunit.client.0.smithi002.stdout:checking sys/param.h presence... yes 2023-09-13T04:41:42.380 INFO:tasks.workunit.client.0.smithi002.stdout:checking for sys/param.h... yes 2023-09-13T04:41:42.458 INFO:tasks.workunit.client.0.smithi002.stdout:checking ioctl.h usability... no 2023-09-13T04:41:42.491 INFO:tasks.workunit.client.0.smithi002.stdout:checking ioctl.h presence... no 2023-09-13T04:41:42.493 INFO:tasks.workunit.client.0.smithi002.stdout:checking for ioctl.h... no 2023-09-13T04:41:42.578 INFO:tasks.workunit.client.0.smithi002.stdout:checking sys/ioctl.h usability... yes 2023-09-13T04:41:42.611 INFO:tasks.workunit.client.0.smithi002.stdout:checking sys/ioctl.h presence... yes 2023-09-13T04:41:42.613 INFO:tasks.workunit.client.0.smithi002.stdout:checking for sys/ioctl.h... yes 2023-09-13T04:41:42.702 INFO:tasks.workunit.client.0.smithi002.stdout:checking return type of signal handlers... void 2023-09-13T04:41:42.780 INFO:tasks.workunit.client.0.smithi002.stdout:checking for size_t... yes 2023-09-13T04:41:42.848 INFO:tasks.workunit.client.0.smithi002.stdout:checking for dev_t... yes 2023-09-13T04:41:42.925 INFO:tasks.workunit.client.0.smithi002.stdout:checking for short... yes 2023-09-13T04:41:43.026 INFO:tasks.workunit.client.0.smithi002.stdout:checking size of short... 2 2023-09-13T04:41:43.105 INFO:tasks.workunit.client.0.smithi002.stdout:checking for int... yes 2023-09-13T04:41:43.207 INFO:tasks.workunit.client.0.smithi002.stdout:checking size of int... 4 2023-09-13T04:41:43.288 INFO:tasks.workunit.client.0.smithi002.stdout:checking for long... yes 2023-09-13T04:41:43.391 INFO:tasks.workunit.client.0.smithi002.stdout:checking size of long... 8 2023-09-13T04:41:43.532 INFO:tasks.workunit.client.0.smithi002.stdout:checking for long long... yes 2023-09-13T04:41:43.655 INFO:tasks.workunit.client.0.smithi002.stdout:checking size of long long... 8 2023-09-13T04:41:43.752 INFO:tasks.workunit.client.0.smithi002.stdout:checking for ssize_t... yes 2023-09-13T04:41:43.823 INFO:tasks.workunit.client.0.smithi002.stdout:checking for sig_atomic_t... yes 2023-09-13T04:41:43.896 INFO:tasks.workunit.client.0.smithi002.stdout:checking whether gcc needs -traditional... no 2023-09-13T04:41:43.960 INFO:tasks.workunit.client.0.smithi002.stdout:checking for an ANSI C-conforming const... yes 2023-09-13T04:41:44.009 INFO:tasks.workunit.client.0.smithi002.stdout:checking for inline... inline 2023-09-13T04:41:44.106 INFO:tasks.workunit.client.0.smithi002.stdout:checking for working alloca.h... yes 2023-09-13T04:41:44.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:43 smithi149 ceph-mon[133315]: pgmap v41: 129 pgs: 129 active+clean; 2.0 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 76 KiB/s wr, 14 op/s 2023-09-13T04:41:44.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:43 smithi156 ceph-mon[133675]: pgmap v41: 129 pgs: 129 active+clean; 2.0 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 76 KiB/s wr, 14 op/s 2023-09-13T04:41:44.190 INFO:tasks.workunit.client.0.smithi002.stdout:checking for alloca... yes 2023-09-13T04:41:44.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:43 smithi002 ceph-mon[97651]: pgmap v41: 129 pgs: 129 active+clean; 2.0 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 76 KiB/s wr, 14 op/s 2023-09-13T04:41:44.294 INFO:tasks.workunit.client.0.smithi002.stdout:checking for working memcmp... yes 2023-09-13T04:41:44.390 INFO:tasks.workunit.client.0.smithi002.stdout:checking for strftime... yes 2023-09-13T04:41:44.506 INFO:tasks.workunit.client.0.smithi002.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2023-09-13T04:41:44.607 INFO:tasks.workunit.client.0.smithi002.stdout:checking whether stat accepts an empty string... no 2023-09-13T04:41:44.702 INFO:tasks.workunit.client.0.smithi002.stdout:checking for vprintf... yes 2023-09-13T04:41:44.798 INFO:tasks.workunit.client.0.smithi002.stdout:checking for _doprnt... no 2023-09-13T04:41:44.855 INFO:tasks.workunit.client.0.smithi002.stdout:checking build system type... x86_64-unknown-linux-gnu 2023-09-13T04:41:44.888 INFO:tasks.workunit.client.0.smithi002.stdout:checking host system type... x86_64-unknown-linux-gnu 2023-09-13T04:41:44.984 INFO:tasks.workunit.client.0.smithi002.stdout:checking for the pthreads library -lpthreads... no 2023-09-13T04:41:45.069 INFO:tasks.workunit.client.0.smithi002.stdout:checking whether pthreads work without any flags... no 2023-09-13T04:41:45.098 INFO:tasks.workunit.client.0.smithi002.stdout:checking whether pthreads work with -Kthread... no 2023-09-13T04:41:45.127 INFO:tasks.workunit.client.0.smithi002.stdout:checking whether pthreads work with -kthread... no 2023-09-13T04:41:45.200 INFO:tasks.workunit.client.0.smithi002.stdout:checking for the pthreads library -llthread... no 2023-09-13T04:41:45.290 INFO:tasks.workunit.client.0.smithi002.stdout:checking whether pthreads work with -pthread... yes 2023-09-13T04:41:45.372 INFO:tasks.workunit.client.0.smithi002.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2023-09-13T04:41:45.372 INFO:tasks.workunit.client.0.smithi002.stdout:checking if more special flags are required for pthreads... no 2023-09-13T04:41:45.372 INFO:tasks.workunit.client.0.smithi002.stdout:checking for cc_r... gcc 2023-09-13T04:41:45.499 INFO:tasks.workunit.client.0.smithi002.stdout:checking for setrlimit... yes 2023-09-13T04:41:45.604 INFO:tasks.workunit.client.0.smithi002.stdout:checking for setproctitle... no 2023-09-13T04:41:45.708 INFO:tasks.workunit.client.0.smithi002.stdout:checking for getopt_long... yes 2023-09-13T04:41:45.820 INFO:tasks.workunit.client.0.smithi002.stdout:checking for strtoull... yes 2023-09-13T04:41:45.930 INFO:tasks.workunit.client.0.smithi002.stdout:checking for strtoq... yes 2023-09-13T04:41:46.042 INFO:tasks.workunit.client.0.smithi002.stdout:checking for snprintf... yes 2023-09-13T04:41:46.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:45 smithi149 ceph-mon[133315]: pgmap v42: 129 pgs: 129 active+clean; 3.8 MiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 245 KiB/s wr, 28 op/s 2023-09-13T04:41:46.150 INFO:tasks.workunit.client.0.smithi002.stdout:checking for vsnprintf... yes 2023-09-13T04:41:46.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:45 smithi156 ceph-mon[133675]: pgmap v42: 129 pgs: 129 active+clean; 3.8 MiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 245 KiB/s wr, 28 op/s 2023-09-13T04:41:46.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:45 smithi002 ceph-mon[97651]: pgmap v42: 129 pgs: 129 active+clean; 3.8 MiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 245 KiB/s wr, 28 op/s 2023-09-13T04:41:46.257 INFO:tasks.workunit.client.0.smithi002.stdout:checking for vfprintf... yes 2023-09-13T04:41:46.364 INFO:tasks.workunit.client.0.smithi002.stdout:checking for gethostname... yes 2023-09-13T04:41:46.472 INFO:tasks.workunit.client.0.smithi002.stdout:checking for setlocale... yes 2023-09-13T04:41:46.579 INFO:tasks.workunit.client.0.smithi002.stdout:checking for sync... yes 2023-09-13T04:41:46.685 INFO:tasks.workunit.client.0.smithi002.stdout:checking whether snprintf is C99 conformant... done 2023-09-13T04:41:46.709 INFO:tasks.workunit.client.0.smithi002.stdout:configure: creating ./config.status 2023-09-13T04:41:47.000 INFO:tasks.workunit.client.0.smithi002.stdout:config.status: creating Makefile 2023-09-13T04:41:47.043 INFO:tasks.workunit.client.0.smithi002.stdout:config.status: creating src/Makefile 2023-09-13T04:41:47.110 INFO:tasks.workunit.client.0.smithi002.stdout:config.status: creating man/Makefile 2023-09-13T04:41:47.130 INFO:tasks.workunit.client.0.smithi002.stdout:config.status: creating config.h 2023-09-13T04:41:47.214 INFO:tasks.workunit.client.0.smithi002.stdout:config.status: executing depfiles commands 2023-09-13T04:41:47.514 INFO:tasks.workunit.client.0.smithi002.stderr:+ make 2023-09-13T04:41:47.527 INFO:tasks.workunit.client.0.smithi002.stdout:make all-recursive 2023-09-13T04:41:47.529 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-09-13T04:41:47.537 INFO:tasks.workunit.client.0.smithi002.stdout:Making all in src 2023-09-13T04:41:47.546 INFO:tasks.workunit.client.0.smithi002.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2023-09-13T04:41:47.546 INFO:tasks.workunit.client.0.smithi002.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2023-09-13T04:41:47.547 INFO:tasks.workunit.client.0.smithi002.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2023-09-13T04:41:47.664 INFO:tasks.workunit.client.0.smithi002.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2023-09-13T04:41:47.664 INFO:tasks.workunit.client.0.smithi002.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2023-09-13T04:41:47.789 INFO:tasks.workunit.client.0.smithi002.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2023-09-13T04:41:47.789 INFO:tasks.workunit.client.0.smithi002.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2023-09-13T04:41:47.870 INFO:tasks.workunit.client.0.smithi002.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2023-09-13T04:41:47.870 INFO:tasks.workunit.client.0.smithi002.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2023-09-13T04:41:47.947 INFO:tasks.workunit.client.0.smithi002.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2023-09-13T04:41:47.948 INFO:tasks.workunit.client.0.smithi002.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2023-09-13T04:41:48.023 INFO:tasks.workunit.client.0.smithi002.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2023-09-13T04:41:48.023 INFO:tasks.workunit.client.0.smithi002.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2023-09-13T04:41:48.095 INFO:tasks.workunit.client.0.smithi002.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2023-09-13T04:41:48.096 INFO:tasks.workunit.client.0.smithi002.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2023-09-13T04:41:48.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:47 smithi149 ceph-mon[133315]: pgmap v43: 129 pgs: 129 active+clean; 3.8 MiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 245 KiB/s wr, 26 op/s 2023-09-13T04:41:48.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:47 smithi156 ceph-mon[133675]: pgmap v43: 129 pgs: 129 active+clean; 3.8 MiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 245 KiB/s wr, 26 op/s 2023-09-13T04:41:48.193 INFO:tasks.workunit.client.0.smithi002.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2023-09-13T04:41:48.194 INFO:tasks.workunit.client.0.smithi002.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2023-09-13T04:41:48.249 INFO:tasks.workunit.client.0.smithi002.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2023-09-13T04:41:48.249 INFO:tasks.workunit.client.0.smithi002.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2023-09-13T04:41:48.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:47 smithi002 ceph-mon[97651]: pgmap v43: 129 pgs: 129 active+clean; 3.8 MiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 245 KiB/s wr, 26 op/s 2023-09-13T04:41:48.304 INFO:tasks.workunit.client.0.smithi002.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2023-09-13T04:41:48.305 INFO:tasks.workunit.client.0.smithi002.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2023-09-13T04:41:48.376 INFO:tasks.workunit.client.0.smithi002.stdout:gcc -g -O2 -pthread -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2023-09-13T04:41:48.424 INFO:tasks.workunit.client.0.smithi002.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2023-09-13T04:41:48.425 INFO:tasks.workunit.client.0.smithi002.stdout:Making all in man 2023-09-13T04:41:48.432 INFO:tasks.workunit.client.0.smithi002.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2023-09-13T04:41:48.432 INFO:tasks.workunit.client.0.smithi002.stdout:make[2]: Nothing to be done for 'all'. 2023-09-13T04:41:48.432 INFO:tasks.workunit.client.0.smithi002.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2023-09-13T04:41:48.435 INFO:tasks.workunit.client.0.smithi002.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-09-13T04:41:48.438 INFO:tasks.workunit.client.0.smithi002.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-09-13T04:41:48.438 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-09-13T04:41:48.439 INFO:tasks.workunit.client.0.smithi002.stderr:+ cd src 2023-09-13T04:41:48.440 INFO:tasks.workunit.client.0.smithi002.stderr:+ mkdir blogtest_in 2023-09-13T04:41:48.444 INFO:tasks.workunit.client.0.smithi002.stdout:running blogbench 2023-09-13T04:41:48.444 INFO:tasks.workunit.client.0.smithi002.stderr:+ echo 'running blogbench' 2023-09-13T04:41:48.444 INFO:tasks.workunit.client.0.smithi002.stderr:+ ./blogbench -d blogtest_in 2023-09-13T04:41:48.445 INFO:tasks.workunit.client.0.smithi002.stdout: 2023-09-13T04:41:48.445 INFO:tasks.workunit.client.0.smithi002.stdout:Frequency = 10 secs 2023-09-13T04:41:48.446 INFO:tasks.workunit.client.0.smithi002.stdout:Scratch dir = [blogtest_in] 2023-09-13T04:41:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.113+0000 7f1cb65ff700 1 -- 172.21.15.2:0/3510413028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cb00680d0 msgr2=0x7f1cb0068550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.113+0000 7f1cb65ff700 1 --2- 172.21.15.2:0/3510413028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cb00680d0 0x7f1cb0068550 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4009990 tx=0x7f1ca4009ca0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.113+0000 7f1cb65ff700 1 -- 172.21.15.2:0/3510413028 shutdown_connections 2023-09-13T04:41:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.113+0000 7f1cb65ff700 1 --2- 172.21.15.2:0/3510413028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cb0068a90 0x7f1cb010db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.113+0000 7f1cb65ff700 1 --2- 172.21.15.2:0/3510413028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cb0106d60 0x7f1cb0107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.113+0000 7f1cb65ff700 1 --2- 172.21.15.2:0/3510413028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cb00680d0 0x7f1cb0068550 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.113+0000 7f1cb65ff700 1 -- 172.21.15.2:0/3510413028 >> 172.21.15.2:0/3510413028 conn(0x7f1cb0076970 msgr2=0x7f1cb0076d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.113+0000 7f1cb65ff700 1 -- 172.21.15.2:0/3510413028 shutdown_connections 2023-09-13T04:41:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.113+0000 7f1cb65ff700 1 -- 172.21.15.2:0/3510413028 wait complete. 2023-09-13T04:41:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.114+0000 7f1cb65ff700 1 Processor -- start 2023-09-13T04:41:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.114+0000 7f1cb65ff700 1 -- start start 2023-09-13T04:41:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.114+0000 7f1cb65ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cb00680d0 0x7f1cb0198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.114+0000 7f1cb65ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cb0068a90 0x7f1cb0199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.115+0000 7f1cb65ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cb0106d60 0x7f1cb019da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.115+0000 7f1cb65ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb019e0d0 con 0x7f1cb0106d60 2023-09-13T04:41:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.115+0000 7f1cb65ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb019e240 con 0x7f1cb00680d0 2023-09-13T04:41:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.115+0000 7f1cb65ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb019e400 con 0x7f1cb0068a90 2023-09-13T04:41:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.115+0000 7f1cb4b9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cb0106d60 0x7f1cb019da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.115+0000 7f1caf7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cb0068a90 0x7f1cb0199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.115+0000 7f1caffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cb00680d0 0x7f1cb0198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.115+0000 7f1caf7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cb0068a90 0x7f1cb0199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:36136/0 (socket says 172.21.15.2:36136) 2023-09-13T04:41:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.115+0000 7f1caffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cb00680d0 0x7f1cb0198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:50818/0 (socket says 172.21.15.2:50818) 2023-09-13T04:41:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.115+0000 7f1cb4b9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cb0106d60 0x7f1cb019da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46822/0 (socket says 172.21.15.2:46822) 2023-09-13T04:41:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.115+0000 7f1caf7fe700 1 -- 172.21.15.2:0/1558004466 learned_addr learned my addr 172.21.15.2:0/1558004466 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.115+0000 7f1caffff700 1 -- 172.21.15.2:0/1558004466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cb0068a90 msgr2=0x7f1cb0199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.115+0000 7f1caffff700 1 --2- 172.21.15.2:0/1558004466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cb0068a90 0x7f1cb0199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.115+0000 7f1caffff700 1 -- 172.21.15.2:0/1558004466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cb0106d60 msgr2=0x7f1cb019da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.115+0000 7f1caffff700 1 --2- 172.21.15.2:0/1558004466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cb0106d60 0x7f1cb019da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.115+0000 7f1caffff700 1 -- 172.21.15.2:0/1558004466 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca40095f0 con 0x7f1cb00680d0 2023-09-13T04:41:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.116+0000 7f1caffff700 1 --2- 172.21.15.2:0/1558004466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cb00680d0 0x7f1cb0198ee0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000e9f0 tx=0x7f1ca000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.116+0000 7f1cad7fa700 1 -- 172.21.15.2:0/1558004466 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca000c9c0 con 0x7f1cb00680d0 2023-09-13T04:41:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.116+0000 7f1cb65ff700 1 -- 172.21.15.2:0/1558004466 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb019e6e0 con 0x7f1cb00680d0 2023-09-13T04:41:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.116+0000 7f1cad7fa700 1 -- 172.21.15.2:0/1558004466 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ca0010430 con 0x7f1cb00680d0 2023-09-13T04:41:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.116+0000 7f1cb65ff700 1 -- 172.21.15.2:0/1558004466 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb01a5f20 con 0x7f1cb00680d0 2023-09-13T04:41:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.116+0000 7f1cad7fa700 1 -- 172.21.15.2:0/1558004466 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0018680 con 0x7f1cb00680d0 2023-09-13T04:41:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.119+0000 7f1cad7fa700 1 -- 172.21.15.2:0/1558004466 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 24) v1 ==== 135491+0+0 (secure 0 0 0) 0x7f1ca0010810 con 0x7f1cb00680d0 2023-09-13T04:41:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.121+0000 7f1cc9f80b80 1 Processor -- start 2023-09-13T04:41:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.122+0000 7f1cc9f80b80 1 -- start start 2023-09-13T04:41:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.122+0000 7f1cad7fa700 1 --2- 172.21.15.2:0/1558004466 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f1c980a08d0 0x7f1c980a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.122+0000 7f1cc9f80b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b6abedbe0 0x556b6abedfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.122+0000 7f1cc9f80b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b6abee500 0x556b6abe9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.122+0000 7f1cc9f80b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abe9840 0x556b6abe9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.122+0000 7f1cc9f80b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b6ab42720 con 0x556b6abe9840 2023-09-13T04:41:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.122+0000 7f1cc9f80b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b6ab40040 con 0x556b6abee500 2023-09-13T04:41:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.122+0000 7f1cc9f80b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b6ab3b930 con 0x556b6abedbe0 2023-09-13T04:41:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.122+0000 7f1cad7fa700 1 -- 172.21.15.2:0/1558004466 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f1ca0014070 con 0x7f1cb00680d0 2023-09-13T04:41:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.122+0000 7f1cb4b9c700 1 --2- 172.21.15.2:0/1558004466 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f1c980a08d0 0x7f1c980a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.123+0000 7f1cb4b9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abe9840 0x556b6abe9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.123+0000 7f1caf7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b6abee500 0x556b6abe9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.123+0000 7f1caffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b6abedbe0 0x556b6abedfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.123+0000 7f1caf7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b6abee500 0x556b6abe9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:50824/0 (socket says 172.21.15.2:50824) 2023-09-13T04:41:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.123+0000 7f1caf7fe700 1 -- 172.21.15.2:0/3844274827 learned_addr learned my addr 172.21.15.2:0/3844274827 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.123+0000 7f1caffff700 1 --2- 172.21.15.2:0/3844274827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b6abedbe0 0x556b6abedfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:36150/0 (socket says 172.21.15.2:36150) 2023-09-13T04:41:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.123+0000 7f1caf7fe700 1 -- 172.21.15.2:0/3844274827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b6abedbe0 msgr2=0x556b6abedfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.123+0000 7f1caf7fe700 1 --2- 172.21.15.2:0/3844274827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b6abedbe0 0x556b6abedfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.123+0000 7f1caf7fe700 1 -- 172.21.15.2:0/3844274827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abe9840 msgr2=0x556b6abe9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.123+0000 7f1caf7fe700 1 --2- 172.21.15.2:0/3844274827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abe9840 0x556b6abe9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.123+0000 7f1caf7fe700 1 -- 172.21.15.2:0/3844274827 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b6a9bb5d0 con 0x556b6abee500 2023-09-13T04:41:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.123+0000 7f1cb4b9c700 1 --2- 172.21.15.2:0/1558004466 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f1c980a08d0 0x7f1c980a2d90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8007740 tx=0x7f1ca8014390 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.123+0000 7f1caf7fe700 1 --2- 172.21.15.2:0/3844274827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b6abee500 0x556b6abe9300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4005bf0 tx=0x7f1ca4005c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8484cd4443ac3bd server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.124+0000 7f1c967fc700 1 -- 172.21.15.2:0/3844274827 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca401d070 con 0x556b6abee500 2023-09-13T04:41:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.124+0000 7f1c967fc700 1 -- 172.21.15.2:0/3844274827 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ca401e040 con 0x556b6abee500 2023-09-13T04:41:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.124+0000 7f1c967fc700 1 -- 172.21.15.2:0/3844274827 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca40049a0 con 0x556b6abee500 2023-09-13T04:41:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.125+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/3844274827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b6abee500 msgr2=0x556b6abe9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.125+0000 7f1cc9f80b80 1 --2- 172.21.15.2:0/3844274827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b6abee500 0x556b6abe9300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4005bf0 tx=0x7f1ca4005c20 comp rx=0 tx=0).stop 2023-09-13T04:41:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.125+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/3844274827 shutdown_connections 2023-09-13T04:41:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.125+0000 7f1cc9f80b80 1 --2- 172.21.15.2:0/3844274827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abe9840 0x556b6abe9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.125+0000 7f1cc9f80b80 1 --2- 172.21.15.2:0/3844274827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b6abedbe0 0x556b6abedfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.125+0000 7f1cc9f80b80 1 --2- 172.21.15.2:0/3844274827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b6abee500 0x556b6abe9300 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.125+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/3844274827 >> 172.21.15.2:0/3844274827 conn(0x556b6ab47c20 msgr2=0x556b6abe3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.125+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/3844274827 shutdown_connections 2023-09-13T04:41:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.125+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/3844274827 wait complete. 2023-09-13T04:41:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.125+0000 7f1cc9f80b80 1 -- start start 2023-09-13T04:41:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.126+0000 7f1cc9f80b80 1 Processor -- start 2023-09-13T04:41:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.127+0000 7f1cc9f80b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b6abe9840 0x556b6abf28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.127+0000 7f1cc9f80b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abedbe0 0x556b6abf2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.127+0000 7f1cc9f80b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b6abee500 0x556b6abf3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.127+0000 7f1cc9f80b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b6aad9b50 con 0x556b6abedbe0 2023-09-13T04:41:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.127+0000 7f1cc9f80b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b6aad9850 con 0x556b6abe9840 2023-09-13T04:41:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.127+0000 7f1cc9f80b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b6a9bd8c0 con 0x556b6abee500 2023-09-13T04:41:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.127+0000 7f1cc9f80b80 10 client.0 _ll_register_callbacks cb 0x556b6aad7620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.127+0000 7f1caf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abedbe0 0x556b6abf2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.127+0000 7f1caffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b6abe9840 0x556b6abf28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.127+0000 7f1cb4b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b6abee500 0x556b6abf3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.127+0000 7f1caf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abedbe0 0x556b6abf2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46838/0 (socket says 172.21.15.2:46838) 2023-09-13T04:41:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.127+0000 7f1caf7fe700 1 -- 172.21.15.2:0/2088112960 learned_addr learned my addr 172.21.15.2:0/2088112960 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.127+0000 7f1caffff700 1 -- 172.21.15.2:0/2088112960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b6abee500 msgr2=0x556b6abf3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.127+0000 7f1caffff700 1 --2- 172.21.15.2:0/2088112960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b6abee500 0x556b6abf3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.127+0000 7f1caffff700 1 -- 172.21.15.2:0/2088112960 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abedbe0 msgr2=0x556b6abf2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.127+0000 7f1caffff700 1 --2- 172.21.15.2:0/2088112960 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abedbe0 0x556b6abf2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.127+0000 7f1caffff700 1 -- 172.21.15.2:0/2088112960 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb010e3c0 con 0x556b6abe9840 2023-09-13T04:41:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.127+0000 7f1caf7fe700 1 --2- 172.21.15.2:0/2088112960 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abedbe0 0x556b6abf2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.128+0000 7f1caffff700 1 --2- 172.21.15.2:0/2088112960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b6abe9840 0x556b6abf28c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00c99c0 tx=0x7f1ca009a4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=401acb323dba9ace server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.128+0000 7f1c8f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:41:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.128+0000 7f1c8f7fe700 1 -- 172.21.15.2:0/2088112960 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca00a1910 con 0x556b6abe9840 2023-09-13T04:41:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.128+0000 7f1c8f7fe700 1 -- 172.21.15.2:0/2088112960 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f1ca0099650 con 0x556b6abe9840 2023-09-13T04:41:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.128+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/2088112960 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556b6abaeef0 con 0x556b6abe9840 2023-09-13T04:41:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.128+0000 7f1c8f7fe700 1 -- 172.21.15.2:0/2088112960 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ca0099fe0 con 0x556b6abe9840 2023-09-13T04:41:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.128+0000 7f1cc9f80b80 10 client.25898 fetch_fsmap learned FSMap version 23 2023-09-13T04:41:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.128+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/2088112960 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556b6ab53740 con 0x556b6abe9840 2023-09-13T04:41:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.128+0000 7f1c8f7fe700 1 -- 172.21.15.2:0/2088112960 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ca00b0030 con 0x556b6abe9840 2023-09-13T04:41:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.129+0000 7f1c8f7fe700 1 -- 172.21.15.2:0/2088112960 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1ca009d0b0 con 0x556b6abe9840 2023-09-13T04:41:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.129+0000 7f1cc9f80b80 10 client.25898 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:41:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.129+0000 7f1cc9f80b80 10 client.25898 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:41:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.129+0000 7f1cc9f80b80 20 client.25898 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:50.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.129+0000 7f1cc9f80b80 1 --2- 172.21.15.2:0/2088112960 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x556b6ac02f40 0x556b6ac05330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:50.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.129+0000 7f1cc9f80b80 4 client.25898 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:41:50.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.129+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/2088112960 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b6ab34910 con 0x556b6ac02f40 2023-09-13T04:41:50.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.129+0000 7f1caf7fe700 1 --2- 172.21.15.2:0/2088112960 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x556b6ac02f40 0x556b6ac05330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:50.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.130+0000 7f1caf7fe700 1 --2- 172.21.15.2:0/2088112960 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x556b6ac02f40 0x556b6ac05330 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a566cadde988b360 server_cookie=96caa621c999d20c in_seq=0 out_seq=0 2023-09-13T04:41:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.130+0000 7f1c8f7fe700 10 client.25898 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.130+0000 7f1c8f7fe700 1 -- 172.21.15.2:0/2088112960 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556b6ab34910 con 0x556b6ac02f40 2023-09-13T04:41:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.130+0000 7f1c8f7fe700 10 client.25898 handle_command_reply: tid=0 2023-09-13T04:41:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.130+0000 7f1cc9f80b80 1 client.25898 shutdown 2023-09-13T04:41:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.131+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/2088112960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b6abe9840 msgr2=0x556b6abf28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.131+0000 7f1cc9f80b80 1 --2- 172.21.15.2:0/2088112960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b6abe9840 0x556b6abf28c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00c99c0 tx=0x7f1ca009a4f0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.131+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/2088112960 shutdown_connections 2023-09-13T04:41:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.131+0000 7f1cc9f80b80 1 --2- 172.21.15.2:0/2088112960 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x556b6ac02f40 0x556b6ac05330 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.131+0000 7f1cc9f80b80 1 --2- 172.21.15.2:0/2088112960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b6abee500 0x556b6abf3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.131+0000 7f1cc9f80b80 1 --2- 172.21.15.2:0/2088112960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b6abe9840 0x556b6abf28c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.131+0000 7f1cc9f80b80 1 --2- 172.21.15.2:0/2088112960 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abedbe0 0x556b6abf2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.131+0000 7f1c8f7fe700 0 client.25898 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.131+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/2088112960 >> 172.21.15.2:0/2088112960 conn(0x556b6ab47c20 msgr2=0x556b6abde630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.131+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/2088112960 shutdown_connections 2023-09-13T04:41:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.131+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/2088112960 wait complete. 2023-09-13T04:41:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.131+0000 7f1cc9f80b80 20 client.25898 trim_cache size 0 max 16384 2023-09-13T04:41:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.146+0000 7f1cc9f80b80 1 Processor -- start 2023-09-13T04:41:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.147+0000 7f1cc9f80b80 1 -- start start 2023-09-13T04:41:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.147+0000 7f1cc9f80b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abe9840 0x556b6abde730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.147+0000 7f1cc9f80b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b6abedbe0 0x556b6abf9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.147+0000 7f1cc9f80b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b6abee500 0x556b6abf3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.147+0000 7f1cc9f80b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b6abf5cd0 con 0x556b6abe9840 2023-09-13T04:41:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.147+0000 7f1cc9f80b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b6abee8e0 con 0x556b6abedbe0 2023-09-13T04:41:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.147+0000 7f1cc9f80b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b6abfa480 con 0x556b6abee500 2023-09-13T04:41:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.147+0000 7f1caffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abe9840 0x556b6abde730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.147+0000 7f1caffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abe9840 0x556b6abde730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46850/0 (socket says 172.21.15.2:46850) 2023-09-13T04:41:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.147+0000 7f1caffff700 1 -- 172.21.15.2:0/3454798661 learned_addr learned my addr 172.21.15.2:0/3454798661 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.147+0000 7f1caf7fe700 1 --2- 172.21.15.2:0/3454798661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b6abedbe0 0x556b6abf9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.147+0000 7f1cb4b9c700 1 --2- 172.21.15.2:0/3454798661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b6abee500 0x556b6abf3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.148+0000 7f1caffff700 1 -- 172.21.15.2:0/3454798661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b6abee500 msgr2=0x556b6abf3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.148+0000 7f1caffff700 1 --2- 172.21.15.2:0/3454798661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b6abee500 0x556b6abf3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.148+0000 7f1caffff700 1 -- 172.21.15.2:0/3454798661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b6abedbe0 msgr2=0x556b6abf9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.148+0000 7f1caffff700 1 --2- 172.21.15.2:0/3454798661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b6abedbe0 0x556b6abf9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.148+0000 7f1caffff700 1 -- 172.21.15.2:0/3454798661 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b6a967d80 con 0x556b6abe9840 2023-09-13T04:41:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.148+0000 7f1caffff700 1 --2- 172.21.15.2:0/3454798661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abe9840 0x556b6abde730 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00a1b20 tx=0x7f1ca0099550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273a53e5afcb5c60 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.148+0000 7f1c96ffd700 1 -- 172.21.15.2:0/3454798661 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca00a1910 con 0x556b6abe9840 2023-09-13T04:41:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.148+0000 7f1c96ffd700 1 -- 172.21.15.2:0/3454798661 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ca00b0070 con 0x556b6abe9840 2023-09-13T04:41:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.148+0000 7f1c96ffd700 1 -- 172.21.15.2:0/3454798661 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca009a660 con 0x556b6abe9840 2023-09-13T04:41:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.149+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/3454798661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abe9840 msgr2=0x556b6abde730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.149+0000 7f1cc9f80b80 1 --2- 172.21.15.2:0/3454798661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abe9840 0x556b6abde730 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00a1b20 tx=0x7f1ca0099550 comp rx=0 tx=0).stop 2023-09-13T04:41:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.149+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/3454798661 shutdown_connections 2023-09-13T04:41:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.149+0000 7f1cc9f80b80 1 --2- 172.21.15.2:0/3454798661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b6abee500 0x556b6abf3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.149+0000 7f1cc9f80b80 1 --2- 172.21.15.2:0/3454798661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abe9840 0x556b6abde730 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.149+0000 7f1cc9f80b80 1 --2- 172.21.15.2:0/3454798661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b6abedbe0 0x556b6abf9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.149+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/3454798661 >> 172.21.15.2:0/3454798661 conn(0x556b6ab47c20 msgr2=0x556b6abe8940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.149+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/3454798661 shutdown_connections 2023-09-13T04:41:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.149+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/3454798661 wait complete. 2023-09-13T04:41:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.149+0000 7f1cc9f80b80 1 -- start start 2023-09-13T04:41:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.149+0000 7f1cc9f80b80 1 Processor -- start 2023-09-13T04:41:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.150+0000 7f1cc9f80b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abe9840 0x556b6abf5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.150+0000 7f1cc9f80b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b6abedbe0 0x556b6abf6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.150+0000 7f1cc9f80b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b6abee500 0x556b6abf69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.150+0000 7f1cc9f80b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b6ac55ea0 con 0x556b6abe9840 2023-09-13T04:41:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.150+0000 7f1cc9f80b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b6ac56010 con 0x556b6abedbe0 2023-09-13T04:41:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.150+0000 7f1cc9f80b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b6ac561d0 con 0x556b6abee500 2023-09-13T04:41:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.150+0000 7f1cc9f80b80 10 client.0 _ll_register_callbacks cb 0x556b6aad7620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.150+0000 7f1caffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abe9840 0x556b6abf5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.150+0000 7f1caffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abe9840 0x556b6abf5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46858/0 (socket says 172.21.15.2:46858) 2023-09-13T04:41:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.150+0000 7f1caffff700 1 -- 172.21.15.2:0/588816878 learned_addr learned my addr 172.21.15.2:0/588816878 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.150+0000 7f1caf7fe700 1 --2- 172.21.15.2:0/588816878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b6abedbe0 0x556b6abf6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.150+0000 7f1cb4b9c700 1 --2- 172.21.15.2:0/588816878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b6abee500 0x556b6abf69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.151+0000 7f1caffff700 1 -- 172.21.15.2:0/588816878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b6abee500 msgr2=0x556b6abf69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.151+0000 7f1caffff700 1 --2- 172.21.15.2:0/588816878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b6abee500 0x556b6abf69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.151+0000 7f1caffff700 1 -- 172.21.15.2:0/588816878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b6abedbe0 msgr2=0x556b6abf6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.151+0000 7f1caffff700 1 --2- 172.21.15.2:0/588816878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b6abedbe0 0x556b6abf6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.151+0000 7f1caffff700 1 -- 172.21.15.2:0/588816878 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b6ab53740 con 0x556b6abe9840 2023-09-13T04:41:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.151+0000 7f1cb4b9c700 1 --2- 172.21.15.2:0/588816878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b6abee500 0x556b6abf69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:41:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.151+0000 7f1caffff700 1 --2- 172.21.15.2:0/588816878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abe9840 0x556b6abf5f20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00a1190 tx=0x7f1ca009d250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa43afb8fa810e5 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.151+0000 7f1c8f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:41:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.151+0000 7f1c8f7fe700 1 -- 172.21.15.2:0/588816878 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca00a1910 con 0x556b6abe9840 2023-09-13T04:41:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.151+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/588816878 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ca401e040 con 0x556b6abe9840 2023-09-13T04:41:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.151+0000 7f1c8f7fe700 1 -- 172.21.15.2:0/588816878 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f1ca00a16c0 con 0x556b6abe9840 2023-09-13T04:41:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.151+0000 7f1c8f7fe700 1 -- 172.21.15.2:0/588816878 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ca0099650 con 0x556b6abe9840 2023-09-13T04:41:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.151+0000 7f1cc9f80b80 10 client.15885 fetch_fsmap learned FSMap version 23 2023-09-13T04:41:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.151+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/588816878 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556b6a950c00 con 0x556b6abe9840 2023-09-13T04:41:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.152+0000 7f1c8f7fe700 1 -- 172.21.15.2:0/588816878 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ca00a6020 con 0x556b6abe9840 2023-09-13T04:41:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.152+0000 7f1c8f7fe700 1 -- 172.21.15.2:0/588816878 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1ca0098b40 con 0x556b6abe9840 2023-09-13T04:41:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.152+0000 7f1cc9f80b80 10 client.15885 fetch_fsmap finished waiting for FSMap version 23 2023-09-13T04:41:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.152+0000 7f1cc9f80b80 10 client.15885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:41:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.152+0000 7f1cc9f80b80 20 client.15885 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.152+0000 7f1cc9f80b80 1 --2- 172.21.15.2:0/588816878 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x556b6ac57410 0x556b6ac59800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.152+0000 7f1cc9f80b80 4 client.15885 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:41:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.152+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/588816878 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556b6aaf4ff0 con 0x556b6ac57410 2023-09-13T04:41:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.152+0000 7f1caf7fe700 1 --2- 172.21.15.2:0/588816878 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x556b6ac57410 0x556b6ac59800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.153+0000 7f1caf7fe700 1 --2- 172.21.15.2:0/588816878 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x556b6ac57410 0x556b6ac59800 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae8e4e06c712e344 server_cookie=c026e5b9f2cfa9b5 in_seq=0 out_seq=0 2023-09-13T04:41:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.153+0000 7f1c8f7fe700 10 client.15885 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.237+0000 7f1c8f7fe700 1 -- 172.21.15.2:0/588816878 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556b6aaf4ff0 con 0x556b6ac57410 2023-09-13T04:41:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.237+0000 7f1c8f7fe700 10 client.15885 handle_command_reply: tid=0 2023-09-13T04:41:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.237+0000 7f1cc9f80b80 1 client.15885 shutdown 2023-09-13T04:41:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.237+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/588816878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abe9840 msgr2=0x556b6abf5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.237+0000 7f1cc9f80b80 1 --2- 172.21.15.2:0/588816878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abe9840 0x556b6abf5f20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00a1190 tx=0x7f1ca009d250 comp rx=0 tx=0).stop 2023-09-13T04:41:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.237+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/588816878 shutdown_connections 2023-09-13T04:41:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.237+0000 7f1cc9f80b80 1 --2- 172.21.15.2:0/588816878 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x556b6ac57410 0x556b6ac59800 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.237+0000 7f1cc9f80b80 1 --2- 172.21.15.2:0/588816878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b6abee500 0x556b6abf69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.237+0000 7f1cc9f80b80 1 --2- 172.21.15.2:0/588816878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b6abe9840 0x556b6abf5f20 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.237+0000 7f1c8f7fe700 0 client.15885 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.237+0000 7f1cc9f80b80 1 --2- 172.21.15.2:0/588816878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b6abedbe0 0x556b6abf6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.237+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/588816878 >> 172.21.15.2:0/588816878 conn(0x556b6ab47c20 msgr2=0x556b6abec740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.237+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/588816878 shutdown_connections 2023-09-13T04:41:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.237+0000 7f1cc9f80b80 1 -- 172.21.15.2:0/588816878 wait complete. 2023-09-13T04:41:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.237+0000 7f1cc9f80b80 20 client.15885 trim_cache size 0 max 16384 2023-09-13T04:41:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:41:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:41:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:41:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:41:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.238+0000 7f1cb65ff700 1 -- 172.21.15.2:0/1558004466 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f1c980a08d0 msgr2=0x7f1c980a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.238+0000 7f1cb65ff700 1 --2- 172.21.15.2:0/1558004466 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f1c980a08d0 0x7f1c980a2d90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8007740 tx=0x7f1ca8014390 comp rx=0 tx=0).stop 2023-09-13T04:41:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.238+0000 7f1cb65ff700 1 -- 172.21.15.2:0/1558004466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cb00680d0 msgr2=0x7f1cb0198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.238+0000 7f1cb65ff700 1 --2- 172.21.15.2:0/1558004466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cb00680d0 0x7f1cb0198ee0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000e9f0 tx=0x7f1ca000ed00 comp rx=0 tx=0).stop 2023-09-13T04:41:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.238+0000 7f1cb65ff700 1 -- 172.21.15.2:0/1558004466 shutdown_connections 2023-09-13T04:41:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.238+0000 7f1cb65ff700 1 --2- 172.21.15.2:0/1558004466 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f1c980a08d0 0x7f1c980a2d90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.238+0000 7f1cb65ff700 1 --2- 172.21.15.2:0/1558004466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cb0106d60 0x7f1cb019da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.238+0000 7f1cb65ff700 1 --2- 172.21.15.2:0/1558004466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cb00680d0 0x7f1cb0198ee0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.238+0000 7f1cb65ff700 1 --2- 172.21.15.2:0/1558004466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cb0068a90 0x7f1cb0199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.238+0000 7f1cb65ff700 1 -- 172.21.15.2:0/1558004466 >> 172.21.15.2:0/1558004466 conn(0x7f1cb0076970 msgr2=0x7f1cb00fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.238+0000 7f1cb65ff700 1 -- 172.21.15.2:0/1558004466 shutdown_connections 2023-09-13T04:41:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:41:50.238+0000 7f1cb65ff700 1 -- 172.21.15.2:0/1558004466 wait complete. 2023-09-13T04:41:50.584 INFO:tasks.workunit.client.0.smithi002.stdout:Spawning 3 writers... 2023-09-13T04:41:50.584 INFO:tasks.workunit.client.0.smithi002.stdout:Spawning 1 rewriters... 2023-09-13T04:41:50.584 INFO:tasks.workunit.client.0.smithi002.stdout:Spawning 5 commenters... 2023-09-13T04:41:50.585 INFO:tasks.workunit.client.0.smithi002.stdout:Spawning 100 readers... 2023-09-13T04:41:50.593 INFO:tasks.workunit.client.0.smithi002.stdout:Benchmarking for 30 iterations. 2023-09-13T04:41:50.593 INFO:tasks.workunit.client.0.smithi002.stdout:The test will run during 5 minutes. 2023-09-13T04:41:50.594 INFO:tasks.workunit.client.0.smithi002.stdout: 2023-09-13T04:41:50.594 INFO:tasks.workunit.client.0.smithi002.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2023-09-13T04:41:50.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:50 smithi149 ceph-mon[133315]: pgmap v44: 129 pgs: 129 active+clean; 6.0 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 467 KiB/s wr, 54 op/s 2023-09-13T04:41:50.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:50 smithi156 ceph-mon[133675]: pgmap v44: 129 pgs: 129 active+clean; 6.0 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 467 KiB/s wr, 54 op/s 2023-09-13T04:41:50.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:50 smithi002 ceph-mon[97651]: pgmap v44: 129 pgs: 129 active+clean; 6.0 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 467 KiB/s wr, 54 op/s 2023-09-13T04:41:52.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:51 smithi149 ceph-mon[133315]: pgmap v45: 129 pgs: 129 active+clean; 6.0 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 449 KiB/s wr, 51 op/s 2023-09-13T04:41:52.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:51 smithi156 ceph-mon[133675]: pgmap v45: 129 pgs: 129 active+clean; 6.0 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 449 KiB/s wr, 51 op/s 2023-09-13T04:41:52.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:51 smithi002 ceph-mon[97651]: pgmap v45: 129 pgs: 129 active+clean; 6.0 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 449 KiB/s wr, 51 op/s 2023-09-13T04:41:54.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:53 smithi149 ceph-mon[133315]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:41:54.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:53 smithi149 ceph-mon[133315]: pgmap v46: 129 pgs: 129 active+clean; 7.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 636 KiB/s wr, 128 op/s 2023-09-13T04:41:54.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:53 smithi156 ceph-mon[133675]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:41:54.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:53 smithi156 ceph-mon[133675]: pgmap v46: 129 pgs: 129 active+clean; 7.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 636 KiB/s wr, 128 op/s 2023-09-13T04:41:54.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:53 smithi002 ceph-mon[97651]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:41:54.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:53 smithi002 ceph-mon[97651]: pgmap v46: 129 pgs: 129 active+clean; 7.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 636 KiB/s wr, 128 op/s 2023-09-13T04:41:55.724 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T04:41:56.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.089+0000 7fc3f699d700 1 -- 172.21.15.2:0/3117522923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3f0108080 msgr2=0x7fc3f010fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:56.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.089+0000 7fc3f699d700 1 --2- 172.21.15.2:0/3117522923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3f0108080 0x7fc3f010fc00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800b0a0 tx=0x7fc3e800b3b0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.090+0000 7fc3f699d700 1 -- 172.21.15.2:0/3117522923 shutdown_connections 2023-09-13T04:41:56.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.090+0000 7fc3f699d700 1 --2- 172.21.15.2:0/3117522923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3f0108080 0x7fc3f010fc00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.090+0000 7fc3f699d700 1 --2- 172.21.15.2:0/3117522923 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3f0102f60 0x7fc3f0103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.090+0000 7fc3f699d700 1 --2- 172.21.15.2:0/3117522923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f0103910 0x7fc3f0107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.090+0000 7fc3f699d700 1 -- 172.21.15.2:0/3117522923 >> 172.21.15.2:0/3117522923 conn(0x7fc3f00fe7c0 msgr2=0x7fc3f0100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:56.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.090+0000 7fc3f699d700 1 -- 172.21.15.2:0/3117522923 shutdown_connections 2023-09-13T04:41:56.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.090+0000 7fc3f699d700 1 -- 172.21.15.2:0/3117522923 wait complete. 2023-09-13T04:41:56.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.091+0000 7fc3f699d700 1 Processor -- start 2023-09-13T04:41:56.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.091+0000 7fc3f699d700 1 -- start start 2023-09-13T04:41:56.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.092+0000 7fc3f699d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3f0102f60 0x7fc3f019e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.092+0000 7fc3f699d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f0108080 0x7fc3f019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.092+0000 7fc3f699d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3f0198a50 0x7fc3f0198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.092+0000 7fc3f699d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f01994d0 con 0x7fc3f0102f60 2023-09-13T04:41:56.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.092+0000 7fc3f699d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f0199610 con 0x7fc3f0198a50 2023-09-13T04:41:56.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.092+0000 7fc3f699d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f01997d0 con 0x7fc3f0108080 2023-09-13T04:41:56.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.092+0000 7fc3f599b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3f0102f60 0x7fc3f019e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.092+0000 7fc3f599b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3f0102f60 0x7fc3f019e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50420/0 (socket says 172.21.15.2:50420) 2023-09-13T04:41:56.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.092+0000 7fc3f599b700 1 -- 172.21.15.2:0/847353040 learned_addr learned my addr 172.21.15.2:0/847353040 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:56.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.093+0000 7fc3f519a700 1 --2- 172.21.15.2:0/847353040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f0108080 0x7fc3f019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.093+0000 7fc3f619c700 1 --2- 172.21.15.2:0/847353040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3f0198a50 0x7fc3f0198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.093+0000 7fc3f599b700 1 -- 172.21.15.2:0/847353040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f0108080 msgr2=0x7fc3f019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:56.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.093+0000 7fc3f599b700 1 --2- 172.21.15.2:0/847353040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f0108080 0x7fc3f019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.093+0000 7fc3f599b700 1 -- 172.21.15.2:0/847353040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3f0198a50 msgr2=0x7fc3f0198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:56.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.093+0000 7fc3f599b700 1 --2- 172.21.15.2:0/847353040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3f0198a50 0x7fc3f0198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.093+0000 7fc3f599b700 1 -- 172.21.15.2:0/847353040 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e8009d00 con 0x7fc3f0102f60 2023-09-13T04:41:56.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.093+0000 7fc3f619c700 1 --2- 172.21.15.2:0/847353040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3f0198a50 0x7fc3f0198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:41:56.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.093+0000 7fc3f599b700 1 --2- 172.21.15.2:0/847353040 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3f0102f60 0x7fc3f019e9e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00e9a0 tx=0x7fc3ec00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:56.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.093+0000 7fc3e6ffd700 1 -- 172.21.15.2:0/847353040 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec009790 con 0x7fc3f0102f60 2023-09-13T04:41:56.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.093+0000 7fc3e6ffd700 1 -- 172.21.15.2:0/847353040 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3ec00cbd0 con 0x7fc3f0102f60 2023-09-13T04:41:56.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.093+0000 7fc3e6ffd700 1 -- 172.21.15.2:0/847353040 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec018680 con 0x7fc3f0102f60 2023-09-13T04:41:56.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.094+0000 7fc3f699d700 1 -- 172.21.15.2:0/847353040 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f01a5850 con 0x7fc3f0102f60 2023-09-13T04:41:56.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.094+0000 7fc3f699d700 1 -- 172.21.15.2:0/847353040 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f01a5d10 con 0x7fc3f0102f60 2023-09-13T04:41:56.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.098+0000 7fc3e6ffd700 1 -- 172.21.15.2:0/847353040 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 24) v1 ==== 135491+0+0 (secure 0 0 0) 0x7fc3ec00cd40 con 0x7fc3f0102f60 2023-09-13T04:41:56.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.100+0000 7fc40b514b80 1 Processor -- start 2023-09-13T04:41:56.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.101+0000 7fc40b514b80 1 -- start start 2023-09-13T04:41:56.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.101+0000 7fc3e6ffd700 1 --2- 172.21.15.2:0/847353040 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc3dc0a09f0 0x7fc3dc0a2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.101+0000 7fc3e6ffd700 1 -- 172.21.15.2:0/847353040 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fc3ec014070 con 0x7fc3f0102f60 2023-09-13T04:41:56.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.101+0000 7fc40b514b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f162c6b0 0x5638f162ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.101+0000 7fc40b514b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638f162cfd0 0x5638f1630dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.101+0000 7fc40b514b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638f1631300 0x5638f1631760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.101+0000 7fc40b514b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f1585780 con 0x5638f1631300 2023-09-13T04:41:56.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.101+0000 7fc40b514b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f15830a0 con 0x5638f162cfd0 2023-09-13T04:41:56.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.101+0000 7fc40b514b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f157e990 con 0x5638f162c6b0 2023-09-13T04:41:56.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.101+0000 7fc3f519a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638f162cfd0 0x5638f1630dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.101+0000 7fc3f519a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638f162cfd0 0x5638f1630dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:46850/0 (socket says 172.21.15.2:46850) 2023-09-13T04:41:56.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.101+0000 7fc3f519a700 1 -- 172.21.15.2:0/1234984486 learned_addr learned my addr 172.21.15.2:0/1234984486 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:56.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.101+0000 7fc3f599b700 1 --2- 172.21.15.2:0/1234984486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f162c6b0 0x5638f162ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.102+0000 7fc3f619c700 1 --2- 172.21.15.2:0/847353040 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc3dc0a09f0 0x7fc3dc0a2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.102+0000 7fc3f519a700 1 -- 172.21.15.2:0/1234984486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f162c6b0 msgr2=0x5638f162ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:56.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.102+0000 7fc3f519a700 1 --2- 172.21.15.2:0/1234984486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f162c6b0 0x5638f162ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.102+0000 7fc3f519a700 1 -- 172.21.15.2:0/1234984486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638f1631300 msgr2=0x5638f1631760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:56.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.102+0000 7fc3f519a700 1 --2- 172.21.15.2:0/1234984486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638f1631300 0x5638f1631760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.102+0000 7fc3f519a700 1 -- 172.21.15.2:0/1234984486 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638f13fe630 con 0x5638f162cfd0 2023-09-13T04:41:56.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.102+0000 7fc3f599b700 1 --2- 172.21.15.2:0/1234984486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f162c6b0 0x5638f162ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:41:56.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.102+0000 7fc3f519a700 1 --2- 172.21.15.2:0/1234984486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638f162cfd0 0x5638f1630dc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009c60 tx=0x7fc3e0009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22624577b9f4e19c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:56.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.102+0000 7fc3dbfff700 1 -- 172.21.15.2:0/1234984486 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0010c90 con 0x5638f162cfd0 2023-09-13T04:41:56.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.102+0000 7fc3dbfff700 1 -- 172.21.15.2:0/1234984486 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3e0015070 con 0x5638f162cfd0 2023-09-13T04:41:56.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.102+0000 7fc3dbfff700 1 -- 172.21.15.2:0/1234984486 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0003e50 con 0x5638f162cfd0 2023-09-13T04:41:56.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.103+0000 7fc40b514b80 1 -- 172.21.15.2:0/1234984486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638f162cfd0 msgr2=0x5638f1630dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:56.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.103+0000 7fc3f619c700 1 --2- 172.21.15.2:0/847353040 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc3dc0a09f0 0x7fc3dc0a2eb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800b780 tx=0x7fc3e8009780 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:56.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.103+0000 7fc40b514b80 1 --2- 172.21.15.2:0/1234984486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638f162cfd0 0x5638f1630dc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009c60 tx=0x7fc3e0009f70 comp rx=0 tx=0).stop 2023-09-13T04:41:56.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.103+0000 7fc40b514b80 1 -- 172.21.15.2:0/1234984486 shutdown_connections 2023-09-13T04:41:56.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.103+0000 7fc40b514b80 1 --2- 172.21.15.2:0/1234984486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638f1631300 0x5638f1631760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.103+0000 7fc40b514b80 1 --2- 172.21.15.2:0/1234984486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f162c6b0 0x5638f162ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.103+0000 7fc40b514b80 1 --2- 172.21.15.2:0/1234984486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638f162cfd0 0x5638f1630dc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.103+0000 7fc40b514b80 1 -- 172.21.15.2:0/1234984486 >> 172.21.15.2:0/1234984486 conn(0x5638f158ac80 msgr2=0x5638f1627300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:56.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.103+0000 7fc40b514b80 1 -- 172.21.15.2:0/1234984486 shutdown_connections 2023-09-13T04:41:56.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.103+0000 7fc40b514b80 1 -- 172.21.15.2:0/1234984486 wait complete. 2023-09-13T04:41:56.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.104+0000 7fc40b514b80 1 -- start start 2023-09-13T04:41:56.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.104+0000 7fc40b514b80 1 Processor -- start 2023-09-13T04:41:56.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.105+0000 7fc40b514b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f162c6b0 0x5638f1635e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.105+0000 7fc40b514b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638f162cfd0 0x5638f1636340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.105+0000 7fc40b514b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638f1631300 0x5638f1636880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.105+0000 7fc40b514b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f151cbb0 con 0x5638f1631300 2023-09-13T04:41:56.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.105+0000 7fc40b514b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f151c8b0 con 0x5638f162cfd0 2023-09-13T04:41:56.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.105+0000 7fc40b514b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f1400920 con 0x5638f162c6b0 2023-09-13T04:41:56.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.105+0000 7fc40b514b80 10 client.0 _ll_register_callbacks cb 0x5638f151a680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:56.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.106+0000 7fc3f599b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f162c6b0 0x5638f1635e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.106+0000 7fc3f599b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f162c6b0 0x5638f1635e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:42990/0 (socket says 172.21.15.2:42990) 2023-09-13T04:41:56.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.106+0000 7fc3f599b700 1 -- 172.21.15.2:0/1815487197 learned_addr learned my addr 172.21.15.2:0/1815487197 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:56.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.106+0000 7fc3f619c700 1 --2- 172.21.15.2:0/1815487197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638f1631300 0x5638f1636880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.106+0000 7fc3f519a700 1 --2- 172.21.15.2:0/1815487197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638f162cfd0 0x5638f1636340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.106+0000 7fc3f599b700 1 -- 172.21.15.2:0/1815487197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638f162cfd0 msgr2=0x5638f1636340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:56.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.106+0000 7fc3f599b700 1 --2- 172.21.15.2:0/1815487197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638f162cfd0 0x5638f1636340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.106+0000 7fc3f599b700 1 -- 172.21.15.2:0/1815487197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638f1631300 msgr2=0x5638f1636880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:56.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.106+0000 7fc3f599b700 1 --2- 172.21.15.2:0/1815487197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638f1631300 0x5638f1636880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.106+0000 7fc3f599b700 1 -- 172.21.15.2:0/1815487197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e0009890 con 0x5638f162c6b0 2023-09-13T04:41:56.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.107+0000 7fc3f599b700 1 --2- 172.21.15.2:0/1815487197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f162c6b0 0x5638f1635e00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec000c00 tx=0x7fc3ec099e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=399437ef7d787817 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:56.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.107+0000 7fc3d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:41:56.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.107+0000 7fc3d97fa700 1 -- 172.21.15.2:0/1815487197 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0a1930 con 0x5638f162c6b0 2023-09-13T04:41:56.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.107+0000 7fc40b514b80 1 -- 172.21.15.2:0/1815487197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638f161f670 con 0x5638f162c6b0 2023-09-13T04:41:56.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.107+0000 7fc3d97fa700 1 -- 172.21.15.2:0/1815487197 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fc3ec0c9f10 con 0x5638f162c6b0 2023-09-13T04:41:56.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.107+0000 7fc3d97fa700 1 -- 172.21.15.2:0/1815487197 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3ec0ca0d0 con 0x5638f162c6b0 2023-09-13T04:41:56.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.107+0000 7fc40b514b80 10 client.25930 fetch_fsmap learned FSMap version 24 2023-09-13T04:41:56.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.107+0000 7fc40b514b80 1 -- 172.21.15.2:0/1815487197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5638f15967a0 con 0x5638f162c6b0 2023-09-13T04:41:56.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.107+0000 7fc3d97fa700 1 -- 172.21.15.2:0/1815487197 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5638f161f670 con 0x5638f162c6b0 2023-09-13T04:41:56.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.108+0000 7fc3d97fa700 1 -- 172.21.15.2:0/1815487197 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 24) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fc3ec0b7b10 con 0x5638f162c6b0 2023-09-13T04:41:56.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.108+0000 7fc40b514b80 10 client.25930 fetch_fsmap finished waiting for FSMap version 24 2023-09-13T04:41:56.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.108+0000 7fc40b514b80 10 client.25930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:41:56.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.108+0000 7fc40b514b80 20 client.25930 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:56.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.108+0000 7fc40b514b80 1 --2- 172.21.15.2:0/1815487197 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5638f1646270 0x5638f1648690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.108+0000 7fc40b514b80 4 client.25930 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:41:56.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.108+0000 7fc40b514b80 1 -- 172.21.15.2:0/1815487197 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638f1577970 con 0x5638f1646270 2023-09-13T04:41:56.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.109+0000 7fc3f519a700 1 --2- 172.21.15.2:0/1815487197 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5638f1646270 0x5638f1648690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.109+0000 7fc3f519a700 1 --2- 172.21.15.2:0/1815487197 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5638f1646270 0x5638f1648690 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f18642bf84fda44 server_cookie=3fb323df152481bf in_seq=0 out_seq=0 2023-09-13T04:41:56.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.109+0000 7fc3d97fa700 10 client.25930 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:56.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.110+0000 7fc3d97fa700 1 -- 172.21.15.2:0/1815487197 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5638f1577970 con 0x5638f1646270 2023-09-13T04:41:56.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.110+0000 7fc3d97fa700 10 client.25930 handle_command_reply: tid=0 2023-09-13T04:41:56.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.110+0000 7fc40b514b80 1 client.25930 shutdown 2023-09-13T04:41:56.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.110+0000 7fc40b514b80 1 -- 172.21.15.2:0/1815487197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f162c6b0 msgr2=0x5638f1635e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:56.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.110+0000 7fc40b514b80 1 --2- 172.21.15.2:0/1815487197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f162c6b0 0x5638f1635e00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec000c00 tx=0x7fc3ec099e00 comp rx=0 tx=0).stop 2023-09-13T04:41:56.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.110+0000 7fc40b514b80 1 -- 172.21.15.2:0/1815487197 shutdown_connections 2023-09-13T04:41:56.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.110+0000 7fc40b514b80 1 --2- 172.21.15.2:0/1815487197 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5638f1646270 0x5638f1648690 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.110+0000 7fc40b514b80 1 --2- 172.21.15.2:0/1815487197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638f1631300 0x5638f1636880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.111+0000 7fc40b514b80 1 --2- 172.21.15.2:0/1815487197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f162c6b0 0x5638f1635e00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.111+0000 7fc40b514b80 1 --2- 172.21.15.2:0/1815487197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638f162cfd0 0x5638f1636340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.111+0000 7fc3d97fa700 0 client.25930 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:56.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.111+0000 7fc40b514b80 1 -- 172.21.15.2:0/1815487197 >> 172.21.15.2:0/1815487197 conn(0x5638f158ac80 msgr2=0x5638f162b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:56.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.111+0000 7fc40b514b80 1 -- 172.21.15.2:0/1815487197 shutdown_connections 2023-09-13T04:41:56.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.111+0000 7fc40b514b80 1 -- 172.21.15.2:0/1815487197 wait complete. 2023-09-13T04:41:56.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.111+0000 7fc40b514b80 20 client.25930 trim_cache size 0 max 16384 2023-09-13T04:41:56.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.146+0000 7fc40b514b80 1 Processor -- start 2023-09-13T04:41:56.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.146+0000 7fc40b514b80 1 -- start start 2023-09-13T04:41:56.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.147+0000 7fc40b514b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638f162c6b0 0x5638f163d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.147+0000 7fc40b514b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638f162cfd0 0x5638f1636700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.147+0000 7fc40b514b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f1631300 0x5638f16336f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.147+0000 7fc40b514b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f1621f20 con 0x5638f162cfd0 2023-09-13T04:41:56.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.147+0000 7fc40b514b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f16238e0 con 0x5638f162c6b0 2023-09-13T04:41:56.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.147+0000 7fc40b514b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f1626e60 con 0x5638f1631300 2023-09-13T04:41:56.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.147+0000 7fc3f519a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638f162cfd0 0x5638f1636700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.147+0000 7fc3f599b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638f162c6b0 0x5638f163d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.147+0000 7fc3f599b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638f162c6b0 0x5638f163d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:46874/0 (socket says 172.21.15.2:46874) 2023-09-13T04:41:56.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.147+0000 7fc3f519a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638f162cfd0 0x5638f1636700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50450/0 (socket says 172.21.15.2:50450) 2023-09-13T04:41:56.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.147+0000 7fc3f599b700 1 -- 172.21.15.2:0/1379571470 learned_addr learned my addr 172.21.15.2:0/1379571470 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:56.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.147+0000 7fc3f619c700 1 --2- 172.21.15.2:0/1379571470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f1631300 0x5638f16336f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.148+0000 7fc3f599b700 1 -- 172.21.15.2:0/1379571470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f1631300 msgr2=0x5638f16336f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:56.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.148+0000 7fc3f599b700 1 --2- 172.21.15.2:0/1379571470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f1631300 0x5638f16336f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.148+0000 7fc3f599b700 1 -- 172.21.15.2:0/1379571470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638f162cfd0 msgr2=0x5638f1636700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:56.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.148+0000 7fc3f599b700 1 --2- 172.21.15.2:0/1379571470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638f162cfd0 0x5638f1636700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.148+0000 7fc3f599b700 1 -- 172.21.15.2:0/1379571470 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638f164d2c0 con 0x5638f162c6b0 2023-09-13T04:41:56.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.148+0000 7fc3f519a700 1 --2- 172.21.15.2:0/1379571470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638f162cfd0 0x5638f1636700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:56.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.148+0000 7fc3f599b700 1 --2- 172.21.15.2:0/1379571470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638f162c6b0 0x5638f163d400 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0995c0 tx=0x7fc3ec099a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddad02769c490f46 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:56.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.148+0000 7fc3e4bff700 1 -- 172.21.15.2:0/1379571470 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0a1930 con 0x5638f162c6b0 2023-09-13T04:41:56.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.148+0000 7fc3e4bff700 1 -- 172.21.15.2:0/1379571470 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3ec0b7b10 con 0x5638f162c6b0 2023-09-13T04:41:56.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.148+0000 7fc3e4bff700 1 -- 172.21.15.2:0/1379571470 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec095910 con 0x5638f162c6b0 2023-09-13T04:41:56.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.149+0000 7fc40b514b80 1 -- 172.21.15.2:0/1379571470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638f162c6b0 msgr2=0x5638f163d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:56.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.149+0000 7fc40b514b80 1 --2- 172.21.15.2:0/1379571470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638f162c6b0 0x5638f163d400 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0995c0 tx=0x7fc3ec099a40 comp rx=0 tx=0).stop 2023-09-13T04:41:56.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.149+0000 7fc40b514b80 1 -- 172.21.15.2:0/1379571470 shutdown_connections 2023-09-13T04:41:56.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.149+0000 7fc40b514b80 1 --2- 172.21.15.2:0/1379571470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f1631300 0x5638f16336f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.149+0000 7fc40b514b80 1 --2- 172.21.15.2:0/1379571470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638f162c6b0 0x5638f163d400 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.149+0000 7fc40b514b80 1 --2- 172.21.15.2:0/1379571470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638f162cfd0 0x5638f1636700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.149+0000 7fc40b514b80 1 -- 172.21.15.2:0/1379571470 >> 172.21.15.2:0/1379571470 conn(0x5638f158ac80 msgr2=0x5638f162b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:56.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.149+0000 7fc40b514b80 1 -- 172.21.15.2:0/1379571470 shutdown_connections 2023-09-13T04:41:56.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.149+0000 7fc40b514b80 1 -- 172.21.15.2:0/1379571470 wait complete. 2023-09-13T04:41:56.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.150+0000 7fc40b514b80 1 -- start start 2023-09-13T04:41:56.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.150+0000 7fc40b514b80 1 Processor -- start 2023-09-13T04:41:56.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.151+0000 7fc40b514b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f162c6b0 0x5638f163aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.151+0000 7fc40b514b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638f162cfd0 0x5638f163af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.151+0000 7fc40b514b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638f1631300 0x5638f163b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.151+0000 7fc40b514b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f1699120 con 0x5638f162cfd0 2023-09-13T04:41:56.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.151+0000 7fc40b514b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f16a1aa0 con 0x5638f1631300 2023-09-13T04:41:56.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.151+0000 7fc40b514b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f16a1c60 con 0x5638f162c6b0 2023-09-13T04:41:56.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.151+0000 7fc40b514b80 10 client.0 _ll_register_callbacks cb 0x5638f151a680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:56.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.151+0000 7fc3f599b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f162c6b0 0x5638f163aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.151+0000 7fc3f599b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f162c6b0 0x5638f163aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:43016/0 (socket says 172.21.15.2:43016) 2023-09-13T04:41:56.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.151+0000 7fc3f599b700 1 -- 172.21.15.2:0/3225415452 learned_addr learned my addr 172.21.15.2:0/3225415452 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:56.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.151+0000 7fc3f619c700 1 --2- 172.21.15.2:0/3225415452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638f1631300 0x5638f163b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.152+0000 7fc3f519a700 1 --2- 172.21.15.2:0/3225415452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638f162cfd0 0x5638f163af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.152+0000 7fc3f599b700 1 -- 172.21.15.2:0/3225415452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638f1631300 msgr2=0x5638f163b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:56.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.152+0000 7fc3f599b700 1 --2- 172.21.15.2:0/3225415452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638f1631300 0x5638f163b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.152+0000 7fc3f599b700 1 -- 172.21.15.2:0/3225415452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638f162cfd0 msgr2=0x5638f163af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:56.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.152+0000 7fc3f599b700 1 --2- 172.21.15.2:0/3225415452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638f162cfd0 0x5638f163af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.152+0000 7fc3f599b700 1 -- 172.21.15.2:0/3225415452 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5638f13fe630 con 0x5638f162c6b0 2023-09-13T04:41:56.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.152+0000 7fc3f519a700 1 --2- 172.21.15.2:0/3225415452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638f162cfd0 0x5638f163af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:41:56.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.152+0000 7fc3f599b700 1 --2- 172.21.15.2:0/3225415452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f162c6b0 0x5638f163aa20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0a1400 tx=0x7fc3ec099e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92b94067b584381c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:56.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.152+0000 7fc3d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:41:56.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.152+0000 7fc3d97fa700 1 -- 172.21.15.2:0/3225415452 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0958b0 con 0x5638f162c6b0 2023-09-13T04:41:56.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.152+0000 7fc3d97fa700 1 -- 172.21.15.2:0/3225415452 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fc3ec095a10 con 0x5638f162c6b0 2023-09-13T04:41:56.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.152+0000 7fc40b514b80 1 -- 172.21.15.2:0/3225415452 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc3e0015070 con 0x5638f162c6b0 2023-09-13T04:41:56.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.152+0000 7fc3d97fa700 1 -- 172.21.15.2:0/3225415452 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3ec0b7c80 con 0x5638f162c6b0 2023-09-13T04:41:56.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.152+0000 7fc40b514b80 10 client.25936 fetch_fsmap learned FSMap version 24 2023-09-13T04:41:56.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.153+0000 7fc40b514b80 1 -- 172.21.15.2:0/3225415452 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5638f15967a0 con 0x5638f162c6b0 2023-09-13T04:41:56.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.153+0000 7fc3d97fa700 1 -- 172.21.15.2:0/3225415452 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3e0015070 con 0x5638f162c6b0 2023-09-13T04:41:56.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.153+0000 7fc3d97fa700 1 -- 172.21.15.2:0/3225415452 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 24) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fc3ec09e4c0 con 0x5638f162c6b0 2023-09-13T04:41:56.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.153+0000 7fc40b514b80 10 client.25936 fetch_fsmap finished waiting for FSMap version 24 2023-09-13T04:41:56.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.153+0000 7fc40b514b80 10 client.25936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:41:56.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.153+0000 7fc40b514b80 20 client.25936 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:56.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.153+0000 7fc40b514b80 1 --2- 172.21.15.2:0/3225415452 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5638f16a2ef0 0x5638f16a5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.153+0000 7fc40b514b80 4 client.25936 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-09-13T04:41:56.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.153+0000 7fc40b514b80 1 -- 172.21.15.2:0/3225415452 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5638f1538050 con 0x5638f16a2ef0 2023-09-13T04:41:56.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.153+0000 7fc3f519a700 1 --2- 172.21.15.2:0/3225415452 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5638f16a2ef0 0x5638f16a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.154+0000 7fc3d97fa700 10 client.25936 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:56.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.154+0000 7fc3f519a700 1 --2- 172.21.15.2:0/3225415452 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5638f16a2ef0 0x5638f16a5310 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3627f7d21bc5eb server_cookie=a09182525fc25398 in_seq=0 out_seq=0 2023-09-13T04:41:56.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.154+0000 7fc3d97fa700 1 -- 172.21.15.2:0/3225415452 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5638f1538050 con 0x5638f16a2ef0 2023-09-13T04:41:56.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.154+0000 7fc3d97fa700 10 client.25936 handle_command_reply: tid=0 2023-09-13T04:41:56.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.154+0000 7fc40b514b80 1 client.25936 shutdown 2023-09-13T04:41:56.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.154+0000 7fc40b514b80 1 -- 172.21.15.2:0/3225415452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f162c6b0 msgr2=0x5638f163aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:56.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.154+0000 7fc40b514b80 1 --2- 172.21.15.2:0/3225415452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f162c6b0 0x5638f163aa20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0a1400 tx=0x7fc3ec099e00 comp rx=0 tx=0).stop 2023-09-13T04:41:56.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.154+0000 7fc40b514b80 1 -- 172.21.15.2:0/3225415452 shutdown_connections 2023-09-13T04:41:56.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.154+0000 7fc40b514b80 1 --2- 172.21.15.2:0/3225415452 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5638f16a2ef0 0x5638f16a5310 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.154+0000 7fc40b514b80 1 --2- 172.21.15.2:0/3225415452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638f1631300 0x5638f163b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.154+0000 7fc3d97fa700 0 client.25936 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:56.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.154+0000 7fc40b514b80 1 --2- 172.21.15.2:0/3225415452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638f162c6b0 0x5638f163aa20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.154+0000 7fc40b514b80 1 --2- 172.21.15.2:0/3225415452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638f162cfd0 0x5638f163af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.154+0000 7fc40b514b80 1 -- 172.21.15.2:0/3225415452 >> 172.21.15.2:0/3225415452 conn(0x5638f158ac80 msgr2=0x5638f16336f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:56.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.154+0000 7fc40b514b80 1 -- 172.21.15.2:0/3225415452 shutdown_connections 2023-09-13T04:41:56.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.154+0000 7fc40b514b80 1 -- 172.21.15.2:0/3225415452 wait complete. 2023-09-13T04:41:56.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.155+0000 7fc40b514b80 20 client.25936 trim_cache size 0 max 16384 2023-09-13T04:41:56.170 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:41:56.170 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2023-09-13T04:41:56.170 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2023-09-13T04:41:56.170 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:41:56.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.155+0000 7fc3f699d700 1 -- 172.21.15.2:0/847353040 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc3dc0a09f0 msgr2=0x7fc3dc0a2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:56.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.155+0000 7fc3f699d700 1 --2- 172.21.15.2:0/847353040 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc3dc0a09f0 0x7fc3dc0a2eb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800b780 tx=0x7fc3e8009780 comp rx=0 tx=0).stop 2023-09-13T04:41:56.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.155+0000 7fc3f699d700 1 -- 172.21.15.2:0/847353040 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3f0102f60 msgr2=0x7fc3f019e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:56.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.155+0000 7fc3f699d700 1 --2- 172.21.15.2:0/847353040 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3f0102f60 0x7fc3f019e9e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00e9a0 tx=0x7fc3ec00ed60 comp rx=0 tx=0).stop 2023-09-13T04:41:56.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.155+0000 7fc3f699d700 1 -- 172.21.15.2:0/847353040 shutdown_connections 2023-09-13T04:41:56.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.155+0000 7fc3f699d700 1 --2- 172.21.15.2:0/847353040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3f0198a50 0x7fc3f0198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.155+0000 7fc3f699d700 1 --2- 172.21.15.2:0/847353040 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc3dc0a09f0 0x7fc3dc0a2eb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.155+0000 7fc3f699d700 1 --2- 172.21.15.2:0/847353040 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3f0102f60 0x7fc3f019e9e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.155+0000 7fc3f699d700 1 --2- 172.21.15.2:0/847353040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f0108080 0x7fc3f019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.155+0000 7fc3f699d700 1 -- 172.21.15.2:0/847353040 >> 172.21.15.2:0/847353040 conn(0x7fc3f00fe7c0 msgr2=0x7fc3f00ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:56.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.156+0000 7fc3f699d700 1 -- 172.21.15.2:0/847353040 shutdown_connections 2023-09-13T04:41:56.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.156+0000 7fc3f699d700 1 -- 172.21.15.2:0/847353040 wait complete. 2023-09-13T04:41:56.174 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:55 smithi002 ceph-mon[97651]: pgmap v47: 129 pgs: 129 active+clean; 12 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 980 KiB/s wr, 129 op/s 2023-09-13T04:41:56.178 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-13T04:41:56.178 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:41:56.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:55 smithi156 ceph-mon[133675]: pgmap v47: 129 pgs: 129 active+clean; 12 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 980 KiB/s wr, 129 op/s 2023-09-13T04:41:56.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:55 smithi149 ceph-mon[133315]: pgmap v47: 129 pgs: 129 active+clean; 12 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 980 KiB/s wr, 129 op/s 2023-09-13T04:41:56.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.402+0000 7f443759e700 1 -- 172.21.15.2:0/2323057626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4438100ab0 msgr2=0x7f4438100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:56.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.402+0000 7f443759e700 1 --2- 172.21.15.2:0/2323057626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4438100ab0 0x7f4438100f30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f442c00faf0 tx=0x7f442c00fe00 comp rx=0 tx=0).stop 2023-09-13T04:41:56.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.402+0000 7f443759e700 1 -- 172.21.15.2:0/2323057626 shutdown_connections 2023-09-13T04:41:56.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.402+0000 7f443759e700 1 --2- 172.21.15.2:0/2323057626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4438101470 0x7f443810d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.402+0000 7f443759e700 1 --2- 172.21.15.2:0/2323057626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4438106ac0 0x7f4438106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.402+0000 7f443759e700 1 --2- 172.21.15.2:0/2323057626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4438100ab0 0x7f4438100f30 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.402+0000 7f443759e700 1 -- 172.21.15.2:0/2323057626 >> 172.21.15.2:0/2323057626 conn(0x7f44380fc5a0 msgr2=0x7f44380fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:56.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.402+0000 7f443759e700 1 -- 172.21.15.2:0/2323057626 shutdown_connections 2023-09-13T04:41:56.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.403+0000 7f443759e700 1 -- 172.21.15.2:0/2323057626 wait complete. 2023-09-13T04:41:56.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.404+0000 7f443759e700 1 Processor -- start 2023-09-13T04:41:56.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.404+0000 7f443759e700 1 -- start start 2023-09-13T04:41:56.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.404+0000 7f443759e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4438100ab0 0x7f443819d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.404+0000 7f443759e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4438101470 0x7f443819d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.404+0000 7f443759e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4438106ac0 0x7f44381a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.404+0000 7f443759e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44381a2060 con 0x7f4438101470 2023-09-13T04:41:56.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.404+0000 7f443759e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44381a21d0 con 0x7f4438100ab0 2023-09-13T04:41:56.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.405+0000 7f443759e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44381a2390 con 0x7f4438106ac0 2023-09-13T04:41:56.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.405+0000 7f443659c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4438100ab0 0x7f443819d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.405+0000 7f443659c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4438100ab0 0x7f443819d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:46910/0 (socket says 172.21.15.2:46910) 2023-09-13T04:41:56.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.405+0000 7f443659c700 1 -- 172.21.15.2:0/2480323220 learned_addr learned my addr 172.21.15.2:0/2480323220 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:56.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.405+0000 7f4436d9d700 1 --2- 172.21.15.2:0/2480323220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4438106ac0 0x7f44381a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.405+0000 7f4435d9b700 1 --2- 172.21.15.2:0/2480323220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4438101470 0x7f443819d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.405+0000 7f443659c700 1 -- 172.21.15.2:0/2480323220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4438106ac0 msgr2=0x7f44381a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:56.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.405+0000 7f443659c700 1 --2- 172.21.15.2:0/2480323220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4438106ac0 0x7f44381a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.405+0000 7f443659c700 1 -- 172.21.15.2:0/2480323220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4438101470 msgr2=0x7f443819d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:56.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.405+0000 7f443659c700 1 --2- 172.21.15.2:0/2480323220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4438101470 0x7f443819d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.405+0000 7f443659c700 1 -- 172.21.15.2:0/2480323220 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4430009c40 con 0x7f4438100ab0 2023-09-13T04:41:56.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.406+0000 7f443659c700 1 --2- 172.21.15.2:0/2480323220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4438100ab0 0x7f443819d080 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f442800eb30 tx=0x7f442800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:56.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.406+0000 7f44277fe700 1 -- 172.21.15.2:0/2480323220 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4428009880 con 0x7f4438100ab0 2023-09-13T04:41:56.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.406+0000 7f44277fe700 1 -- 172.21.15.2:0/2480323220 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44280044e0 con 0x7f4438100ab0 2023-09-13T04:41:56.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.406+0000 7f44277fe700 1 -- 172.21.15.2:0/2480323220 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4428003df0 con 0x7f4438100ab0 2023-09-13T04:41:56.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.406+0000 7f443759e700 1 -- 172.21.15.2:0/2480323220 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4428009520 con 0x7f4438100ab0 2023-09-13T04:41:56.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.406+0000 7f443759e700 1 -- 172.21.15.2:0/2480323220 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f442c00f750 con 0x7f4438100ab0 2023-09-13T04:41:56.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.407+0000 7f443759e700 1 -- 172.21.15.2:0/2480323220 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4438067160 con 0x7f4438100ab0 2023-09-13T04:41:56.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.409+0000 7f44277fe700 1 -- 172.21.15.2:0/2480323220 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 24) v1 ==== 135491+0+0 (secure 0 0 0) 0x7f4428003f90 con 0x7f4438100ab0 2023-09-13T04:41:56.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.411+0000 7f44277fe700 1 --2- 172.21.15.2:0/2480323220 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f44200a0a40 0x7f44200a2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.411+0000 7f44277fe700 1 -- 172.21.15.2:0/2480323220 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f4428014070 con 0x7f4438100ab0 2023-09-13T04:41:56.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.411+0000 7f4435d9b700 1 --2- 172.21.15.2:0/2480323220 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f44200a0a40 0x7f44200a2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.412+0000 7f4435d9b700 1 --2- 172.21.15.2:0/2480323220 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f44200a0a40 0x7f44200a2f00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f442c009870 tx=0x7f442c011040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:56.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.413+0000 7f44277fe700 1 -- 172.21.15.2:0/2480323220 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4428095fc0 con 0x7f4438100ab0 2023-09-13T04:41:56.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.729+0000 7f443759e700 1 -- 172.21.15.2:0/2480323220 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f443819e3d0 con 0x7f4438100ab0 2023-09-13T04:41:56.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.730+0000 7f44277fe700 1 -- 172.21.15.2:0/2480323220 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11304 (secure 0 0 0) 0x7f44280995e0 con 0x7f4438100ab0 2023-09-13T04:41:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:41:56.731 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2469307307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2469307307},{"type":"v1","addr":"172.21.15.2:6839","nonce":2469307307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T04:37:59.110701+0000","modified":"2023-09-13T04:41:52.866081+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24554,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/846579077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":846579077},{"type":"v1","addr":"172.21.15.156:6835","nonce":846579077}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1896833690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1896833690},{"type":"v1","addr":"172.21.15.149:6837","nonce":1896833690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/524563090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":524563090},{"type":"v1","addr":"172.21.15.156:6837","nonce":524563090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6839/1785833227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1785833227},{"type":"v1","addr":"172.21.15.156:6839","nonce":1785833227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.2:6841/482545391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":482545391},{"type":"v1","addr":"172.21.15.2:6841","nonce":482545391}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/389167841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":389167841},{"type":"v1","addr":"172.21.15.149:6839","nonce":389167841}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T04:41:56.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.733+0000 7f443759e700 1 -- 172.21.15.2:0/2480323220 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f44200a0a40 msgr2=0x7f44200a2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:56.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.733+0000 7f443759e700 1 --2- 172.21.15.2:0/2480323220 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f44200a0a40 0x7f44200a2f00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f442c009870 tx=0x7f442c011040 comp rx=0 tx=0).stop 2023-09-13T04:41:56.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.733+0000 7f443759e700 1 -- 172.21.15.2:0/2480323220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4438100ab0 msgr2=0x7f443819d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:56.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.733+0000 7f443759e700 1 --2- 172.21.15.2:0/2480323220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4438100ab0 0x7f443819d080 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f442800eb30 tx=0x7f442800eef0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.733+0000 7f443759e700 1 -- 172.21.15.2:0/2480323220 shutdown_connections 2023-09-13T04:41:56.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.733+0000 7f443759e700 1 --2- 172.21.15.2:0/2480323220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4438106ac0 0x7f44381a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.733+0000 7f443759e700 1 --2- 172.21.15.2:0/2480323220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4438100ab0 0x7f443819d080 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.733+0000 7f443759e700 1 --2- 172.21.15.2:0/2480323220 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f44200a0a40 0x7f44200a2f00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.733+0000 7f443759e700 1 --2- 172.21.15.2:0/2480323220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4438101470 0x7f443819d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.733+0000 7f443759e700 1 -- 172.21.15.2:0/2480323220 >> 172.21.15.2:0/2480323220 conn(0x7f44380fc5a0 msgr2=0x7f44380fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:56.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.733+0000 7f443759e700 1 -- 172.21.15.2:0/2480323220 shutdown_connections 2023-09-13T04:41:56.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.733+0000 7f443759e700 1 -- 172.21.15.2:0/2480323220 wait complete. 2023-09-13T04:41:56.736 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 24 2023-09-13T04:41:56.754 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-13T04:41:56.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.986+0000 7fcb2e19d700 1 -- 172.21.15.2:0/2864861494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb28103910 msgr2=0x7fcb28107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:56.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.986+0000 7fcb2e19d700 1 --2- 172.21.15.2:0/2864861494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb28103910 0x7fcb28107950 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcb18009990 tx=0x7fcb18009ca0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.986+0000 7fcb2e19d700 1 -- 172.21.15.2:0/2864861494 shutdown_connections 2023-09-13T04:41:56.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.986+0000 7fcb2e19d700 1 --2- 172.21.15.2:0/2864861494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb28108080 0x7fcb2810fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.986+0000 7fcb2e19d700 1 --2- 172.21.15.2:0/2864861494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb28102f60 0x7fcb28103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.986+0000 7fcb2e19d700 1 --2- 172.21.15.2:0/2864861494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb28103910 0x7fcb28107950 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.986+0000 7fcb2e19d700 1 -- 172.21.15.2:0/2864861494 >> 172.21.15.2:0/2864861494 conn(0x7fcb280fe7c0 msgr2=0x7fcb28100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:56.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.986+0000 7fcb2e19d700 1 -- 172.21.15.2:0/2864861494 shutdown_connections 2023-09-13T04:41:56.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.986+0000 7fcb2e19d700 1 -- 172.21.15.2:0/2864861494 wait complete. 2023-09-13T04:41:56.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.988+0000 7fcb2e19d700 1 Processor -- start 2023-09-13T04:41:56.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.988+0000 7fcb2e19d700 1 -- start start 2023-09-13T04:41:56.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.989+0000 7fcb2e19d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb28102f60 0x7fcb28198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.989+0000 7fcb2e19d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb28108080 0x7fcb281992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.989+0000 7fcb2e19d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb2819d730 0x7fcb2819dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.989+0000 7fcb2e19d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2819e1b0 con 0x7fcb28102f60 2023-09-13T04:41:56.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.989+0000 7fcb2e19d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2819e320 con 0x7fcb28108080 2023-09-13T04:41:56.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.989+0000 7fcb2e19d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2819e4e0 con 0x7fcb2819d730 2023-09-13T04:41:56.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.989+0000 7fcb2d19b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb28102f60 0x7fcb28198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.989+0000 7fcb2d19b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb28102f60 0x7fcb28198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50496/0 (socket says 172.21.15.2:50496) 2023-09-13T04:41:56.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.989+0000 7fcb2d19b700 1 -- 172.21.15.2:0/1946384064 learned_addr learned my addr 172.21.15.2:0/1946384064 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:56.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.989+0000 7fcb2c99a700 1 --2- 172.21.15.2:0/1946384064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb28108080 0x7fcb281992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.989+0000 7fcb2d99c700 1 --2- 172.21.15.2:0/1946384064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb2819d730 0x7fcb2819dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.990+0000 7fcb2d19b700 1 -- 172.21.15.2:0/1946384064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb2819d730 msgr2=0x7fcb2819dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:56.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.990+0000 7fcb2d19b700 1 --2- 172.21.15.2:0/1946384064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb2819d730 0x7fcb2819dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.990+0000 7fcb2d19b700 1 -- 172.21.15.2:0/1946384064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb28108080 msgr2=0x7fcb281992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:56.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.990+0000 7fcb2d19b700 1 --2- 172.21.15.2:0/1946384064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb28108080 0x7fcb281992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:56.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.990+0000 7fcb2d19b700 1 -- 172.21.15.2:0/1946384064 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb24009520 con 0x7fcb28102f60 2023-09-13T04:41:56.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.990+0000 7fcb2d19b700 1 --2- 172.21.15.2:0/1946384064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb28102f60 0x7fcb28198d90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400eb30 tx=0x7fcb2400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:56.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.990+0000 7fcb1e7fc700 1 -- 172.21.15.2:0/1946384064 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24009880 con 0x7fcb28102f60 2023-09-13T04:41:56.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.990+0000 7fcb1e7fc700 1 -- 172.21.15.2:0/1946384064 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb240044e0 con 0x7fcb28102f60 2023-09-13T04:41:56.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.990+0000 7fcb2e19d700 1 -- 172.21.15.2:0/1946384064 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb180095f0 con 0x7fcb28102f60 2023-09-13T04:41:56.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.990+0000 7fcb1e7fc700 1 -- 172.21.15.2:0/1946384064 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24003d80 con 0x7fcb28102f60 2023-09-13T04:41:56.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.991+0000 7fcb2e19d700 1 -- 172.21.15.2:0/1946384064 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb281a5d90 con 0x7fcb28102f60 2023-09-13T04:41:56.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.995+0000 7fcb1e7fc700 1 -- 172.21.15.2:0/1946384064 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 24) v1 ==== 135491+0+0 (secure 0 0 0) 0x7fcb24003f20 con 0x7fcb28102f60 2023-09-13T04:41:56.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.997+0000 7fcb42d33b80 1 Processor -- start 2023-09-13T04:41:56.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.997+0000 7fcb1e7fc700 1 --2- 172.21.15.2:0/1946384064 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcb140a0970 0x7fcb140a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.997+0000 7fcb42d33b80 1 -- start start 2023-09-13T04:41:56.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.997+0000 7fcb1e7fc700 1 -- 172.21.15.2:0/1946384064 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fcb24014070 con 0x7fcb28102f60 2023-09-13T04:41:56.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.998+0000 7fcb2d99c700 1 --2- 172.21.15.2:0/1946384064 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcb140a0970 0x7fcb140a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:56.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.998+0000 7fcb42d33b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 0x564835a7fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:56.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.998+0000 7fcb42d33b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564835a7ffd0 0x564835a83dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.998+0000 7fcb42d33b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564835a84300 0x564835a84760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.998+0000 7fcb42d33b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648359d8780 con 0x564835a7ffd0 2023-09-13T04:41:57.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.998+0000 7fcb42d33b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648359d60a0 con 0x564835a7f6b0 2023-09-13T04:41:57.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.998+0000 7fcb42d33b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648359d1990 con 0x564835a84300 2023-09-13T04:41:57.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.998+0000 7fcb2d19b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 0x564835a7fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.998+0000 7fcb2d99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564835a84300 0x564835a84760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.998+0000 7fcb2d19b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 0x564835a7fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:46948/0 (socket says 172.21.15.2:46948) 2023-09-13T04:41:57.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.998+0000 7fcb2d19b700 1 -- 172.21.15.2:0/2484351136 learned_addr learned my addr 172.21.15.2:0/2484351136 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:57.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.998+0000 7fcb2d19b700 1 -- 172.21.15.2:0/2484351136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564835a84300 msgr2=0x564835a84760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:57.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.998+0000 7fcb2d19b700 1 --2- 172.21.15.2:0/2484351136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564835a84300 0x564835a84760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.998+0000 7fcb2d19b700 1 -- 172.21.15.2:0/2484351136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564835a7ffd0 msgr2=0x564835a83dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:57.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.998+0000 7fcb2d19b700 1 --2- 172.21.15.2:0/2484351136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564835a7ffd0 0x564835a83dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.998+0000 7fcb2d19b700 1 -- 172.21.15.2:0/2484351136 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564835851630 con 0x564835a7f6b0 2023-09-13T04:41:57.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.999+0000 7fcb2d99c700 1 --2- 172.21.15.2:0/1946384064 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcb140a0970 0x7fcb140a2e30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000f870 tx=0x7fcb2000fc30 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:57.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.999+0000 7fcb2d19b700 1 --2- 172.21.15.2:0/2484351136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 0x564835a7fa90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400eb00 tx=0x7fcb240993e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a3569cb889465eb server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:57.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.999+0000 7fcb137fe700 1 -- 172.21.15.2:0/2484351136 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2409a100 con 0x564835a7f6b0 2023-09-13T04:41:57.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.999+0000 7fcb137fe700 1 -- 172.21.15.2:0/2484351136 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb24099a00 con 0x564835a7f6b0 2023-09-13T04:41:57.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:56.999+0000 7fcb137fe700 1 -- 172.21.15.2:0/2484351136 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24099c60 con 0x564835a7f6b0 2023-09-13T04:41:57.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.000+0000 7fcb42d33b80 1 -- 172.21.15.2:0/2484351136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 msgr2=0x564835a7fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:57.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.000+0000 7fcb42d33b80 1 --2- 172.21.15.2:0/2484351136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 0x564835a7fa90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400eb00 tx=0x7fcb240993e0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.000+0000 7fcb42d33b80 1 -- 172.21.15.2:0/2484351136 shutdown_connections 2023-09-13T04:41:57.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.000+0000 7fcb42d33b80 1 --2- 172.21.15.2:0/2484351136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564835a84300 0x564835a84760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.000+0000 7fcb42d33b80 1 --2- 172.21.15.2:0/2484351136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 0x564835a7fa90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.000+0000 7fcb42d33b80 1 --2- 172.21.15.2:0/2484351136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564835a7ffd0 0x564835a83dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.000+0000 7fcb42d33b80 1 -- 172.21.15.2:0/2484351136 >> 172.21.15.2:0/2484351136 conn(0x5648359ddc80 msgr2=0x564835a7a300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:57.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.000+0000 7fcb42d33b80 1 -- 172.21.15.2:0/2484351136 shutdown_connections 2023-09-13T04:41:57.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.000+0000 7fcb42d33b80 1 -- 172.21.15.2:0/2484351136 wait complete. 2023-09-13T04:41:57.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.001+0000 7fcb42d33b80 1 -- start start 2023-09-13T04:41:57.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.001+0000 7fcb42d33b80 1 Processor -- start 2023-09-13T04:41:57.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.001+0000 7fcb42d33b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 0x564835a88e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.001+0000 7fcb42d33b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564835a7ffd0 0x564835a893c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.001+0000 7fcb42d33b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564835a84300 0x564835a89900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.001+0000 7fcb42d33b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483596fbb0 con 0x564835a7ffd0 2023-09-13T04:41:57.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.001+0000 7fcb42d33b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483596f8b0 con 0x564835a7f6b0 2023-09-13T04:41:57.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.001+0000 7fcb42d33b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564835853920 con 0x564835a84300 2023-09-13T04:41:57.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.001+0000 7fcb42d33b80 10 client.0 _ll_register_callbacks cb 0x56483596d680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:57.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.002+0000 7fcb2d19b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 0x564835a88e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.002+0000 7fcb2d19b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 0x564835a88e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:46958/0 (socket says 172.21.15.2:46958) 2023-09-13T04:41:57.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.002+0000 7fcb2d19b700 1 -- 172.21.15.2:0/2162137196 learned_addr learned my addr 172.21.15.2:0/2162137196 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:57.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.002+0000 7fcb2d99c700 1 --2- 172.21.15.2:0/2162137196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564835a84300 0x564835a89900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.002+0000 7fcb2c99a700 1 --2- 172.21.15.2:0/2162137196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564835a7ffd0 0x564835a893c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.002+0000 7fcb2d19b700 1 -- 172.21.15.2:0/2162137196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564835a84300 msgr2=0x564835a89900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:57.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.002+0000 7fcb2d19b700 1 --2- 172.21.15.2:0/2162137196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564835a84300 0x564835a89900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.002+0000 7fcb2d19b700 1 -- 172.21.15.2:0/2162137196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564835a7ffd0 msgr2=0x564835a893c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:57.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.002+0000 7fcb2d19b700 1 --2- 172.21.15.2:0/2162137196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564835a7ffd0 0x564835a893c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.002+0000 7fcb2d19b700 1 -- 172.21.15.2:0/2162137196 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb281a5d90 con 0x564835a7f6b0 2023-09-13T04:41:57.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.002+0000 7fcb2d19b700 1 --2- 172.21.15.2:0/2162137196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 0x564835a88e80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcb24095f00 tx=0x7fcb2409a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17dbbe8dd808192c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:57.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.002+0000 7fcb10ff9700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:41:57.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.003+0000 7fcb10ff9700 1 -- 172.21.15.2:0/2162137196 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb240a1a70 con 0x564835a7f6b0 2023-09-13T04:41:57.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.003+0000 7fcb10ff9700 1 -- 172.21.15.2:0/2162137196 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fcb2400cdd0 con 0x564835a7f6b0 2023-09-13T04:41:57.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.003+0000 7fcb42d33b80 1 -- 172.21.15.2:0/2162137196 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564835a72670 con 0x564835a7f6b0 2023-09-13T04:41:57.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.003+0000 7fcb10ff9700 1 -- 172.21.15.2:0/2162137196 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb24098ef0 con 0x564835a7f6b0 2023-09-13T04:41:57.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.003+0000 7fcb42d33b80 10 client.25937 fetch_fsmap learned FSMap version 24 2023-09-13T04:41:57.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.003+0000 7fcb42d33b80 1 -- 172.21.15.2:0/2162137196 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5648359e97a0 con 0x564835a7f6b0 2023-09-13T04:41:57.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.003+0000 7fcb10ff9700 1 -- 172.21.15.2:0/2162137196 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564835a72670 con 0x564835a7f6b0 2023-09-13T04:41:57.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.003+0000 7fcb10ff9700 1 -- 172.21.15.2:0/2162137196 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 24) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fcb240a15e0 con 0x564835a7f6b0 2023-09-13T04:41:57.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.003+0000 7fcb42d33b80 10 client.25937 fetch_fsmap finished waiting for FSMap version 24 2023-09-13T04:41:57.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.003+0000 7fcb42d33b80 10 client.25937 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-09-13T04:41:57.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.003+0000 7fcb42d33b80 20 client.25937 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:57.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.003+0000 7fcb42d33b80 1 --2- 172.21.15.2:0/2162137196 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x564835a98fa0 0x564835a9b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.003+0000 7fcb42d33b80 4 client.25937 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:41:57.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.003+0000 7fcb42d33b80 1 -- 172.21.15.2:0/2162137196 --> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648359ca970 con 0x564835a98fa0 2023-09-13T04:41:57.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.004+0000 7fcb2c99a700 1 --2- 172.21.15.2:0/2162137196 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x564835a98fa0 0x564835a9b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.005+0000 7fcb2c99a700 1 --2- 172.21.15.2:0/2162137196 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x564835a98fa0 0x564835a9b390 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8fe97dd914e6cae3 server_cookie=e9756be6922be5bc in_seq=0 out_seq=0 2023-09-13T04:41:57.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.005+0000 7fcb10ff9700 10 client.25937 ms_handle_connect on v2:172.21.15.156:6838/1785833227 2023-09-13T04:41:57.017 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:56 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2480323220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:41:57.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.005+0000 7fcb10ff9700 1 -- 172.21.15.2:0/2162137196 <== mds.2 v2:172.21.15.156:6838/1785833227 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5648359ca970 con 0x564835a98fa0 2023-09-13T04:41:57.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.005+0000 7fcb10ff9700 10 client.25937 handle_command_reply: tid=0 2023-09-13T04:41:57.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.005+0000 7fcb42d33b80 1 client.25937 shutdown 2023-09-13T04:41:57.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.006+0000 7fcb42d33b80 1 -- 172.21.15.2:0/2162137196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 msgr2=0x564835a88e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:57.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.006+0000 7fcb42d33b80 1 --2- 172.21.15.2:0/2162137196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 0x564835a88e80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcb24095f00 tx=0x7fcb2409a180 comp rx=0 tx=0).stop 2023-09-13T04:41:57.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.006+0000 7fcb42d33b80 1 -- 172.21.15.2:0/2162137196 shutdown_connections 2023-09-13T04:41:57.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.006+0000 7fcb42d33b80 1 --2- 172.21.15.2:0/2162137196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564835a84300 0x564835a89900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.006+0000 7fcb42d33b80 1 --2- 172.21.15.2:0/2162137196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 0x564835a88e80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.006+0000 7fcb42d33b80 1 --2- 172.21.15.2:0/2162137196 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x564835a98fa0 0x564835a9b390 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.006+0000 7fcb42d33b80 1 --2- 172.21.15.2:0/2162137196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564835a7ffd0 0x564835a893c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.006+0000 7fcb42d33b80 1 -- 172.21.15.2:0/2162137196 >> 172.21.15.2:0/2162137196 conn(0x5648359ddc80 msgr2=0x564835a7e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:57.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.006+0000 7fcb10ff9700 0 client.25937 ms_handle_reset on v2:172.21.15.156:6838/1785833227 2023-09-13T04:41:57.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.006+0000 7fcb42d33b80 1 -- 172.21.15.2:0/2162137196 shutdown_connections 2023-09-13T04:41:57.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.006+0000 7fcb42d33b80 1 -- 172.21.15.2:0/2162137196 wait complete. 2023-09-13T04:41:57.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.006+0000 7fcb42d33b80 20 client.25937 trim_cache size 0 max 16384 2023-09-13T04:41:57.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.022+0000 7fcb42d33b80 1 Processor -- start 2023-09-13T04:41:57.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.022+0000 7fcb42d33b80 1 -- start start 2023-09-13T04:41:57.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.023+0000 7fcb42d33b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 0x564835a81bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.023+0000 7fcb42d33b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564835a7ffd0 0x564835a90400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.023+0000 7fcb42d33b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564835a84300 0x564835a89730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.023+0000 7fcb42d33b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564835a74f20 con 0x564835a84300 2023-09-13T04:41:57.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.023+0000 7fcb42d33b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564835a803b0 con 0x564835a7f6b0 2023-09-13T04:41:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.023+0000 7fcb42d33b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564835a90940 con 0x564835a7ffd0 2023-09-13T04:41:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.023+0000 7fcb2c99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564835a7ffd0 0x564835a90400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.023+0000 7fcb2c99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564835a7ffd0 0x564835a90400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:43096/0 (socket says 172.21.15.2:43096) 2023-09-13T04:41:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.023+0000 7fcb2c99a700 1 -- 172.21.15.2:0/1618084654 learned_addr learned my addr 172.21.15.2:0/1618084654 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.023+0000 7fcb2d19b700 1 --2- 172.21.15.2:0/1618084654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 0x564835a81bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.023+0000 7fcb2d99c700 1 --2- 172.21.15.2:0/1618084654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564835a84300 0x564835a89730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.024+0000 7fcb2c99a700 1 -- 172.21.15.2:0/1618084654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 msgr2=0x564835a81bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:57.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.024+0000 7fcb2c99a700 1 --2- 172.21.15.2:0/1618084654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 0x564835a81bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.024+0000 7fcb2c99a700 1 -- 172.21.15.2:0/1618084654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564835a84300 msgr2=0x564835a89730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:57.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.024+0000 7fcb2c99a700 1 --2- 172.21.15.2:0/1618084654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564835a84300 0x564835a89730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.024+0000 7fcb2c99a700 1 -- 172.21.15.2:0/1618084654 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648357fdde0 con 0x564835a7ffd0 2023-09-13T04:41:57.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.024+0000 7fcb2d19b700 1 --2- 172.21.15.2:0/1618084654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 0x564835a81bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:57.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.024+0000 7fcb2c99a700 1 --2- 172.21.15.2:0/1618084654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564835a7ffd0 0x564835a90400 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcb28068990 tx=0x7fcb18005960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a0884ccc26c5b26 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:57.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.024+0000 7fcb13fff700 1 -- 172.21.15.2:0/1618084654 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1801d070 con 0x564835a7ffd0 2023-09-13T04:41:57.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.024+0000 7fcb13fff700 1 -- 172.21.15.2:0/1618084654 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb1801e040 con 0x564835a7ffd0 2023-09-13T04:41:57.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.024+0000 7fcb13fff700 1 -- 172.21.15.2:0/1618084654 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1800bae0 con 0x564835a7ffd0 2023-09-13T04:41:57.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.025+0000 7fcb42d33b80 1 -- 172.21.15.2:0/1618084654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564835a7ffd0 msgr2=0x564835a90400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:57.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.025+0000 7fcb42d33b80 1 --2- 172.21.15.2:0/1618084654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564835a7ffd0 0x564835a90400 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcb28068990 tx=0x7fcb18005960 comp rx=0 tx=0).stop 2023-09-13T04:41:57.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.025+0000 7fcb42d33b80 1 -- 172.21.15.2:0/1618084654 shutdown_connections 2023-09-13T04:41:57.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.025+0000 7fcb42d33b80 1 --2- 172.21.15.2:0/1618084654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564835a84300 0x564835a89730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.025+0000 7fcb42d33b80 1 --2- 172.21.15.2:0/1618084654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 0x564835a81bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.025+0000 7fcb42d33b80 1 --2- 172.21.15.2:0/1618084654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564835a7ffd0 0x564835a90400 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.025+0000 7fcb42d33b80 1 -- 172.21.15.2:0/1618084654 >> 172.21.15.2:0/1618084654 conn(0x5648359ddc80 msgr2=0x564835a7eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:57.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.025+0000 7fcb42d33b80 1 -- 172.21.15.2:0/1618084654 shutdown_connections 2023-09-13T04:41:57.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.025+0000 7fcb42d33b80 1 -- 172.21.15.2:0/1618084654 wait complete. 2023-09-13T04:41:57.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.026+0000 7fcb42d33b80 1 -- start start 2023-09-13T04:41:57.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.026+0000 7fcb42d33b80 1 Processor -- start 2023-09-13T04:41:57.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.027+0000 7fcb42d33b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 0x564835a8e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.027+0000 7fcb42d33b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564835a7ffd0 0x564835a8e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.027+0000 7fcb42d33b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564835a84300 0x564835af4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.027+0000 7fcb42d33b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564835af4a50 con 0x564835a84300 2023-09-13T04:41:57.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.027+0000 7fcb42d33b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564835af4bc0 con 0x564835a7f6b0 2023-09-13T04:41:57.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.027+0000 7fcb42d33b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564835af4d80 con 0x564835a7ffd0 2023-09-13T04:41:57.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.027+0000 7fcb42d33b80 10 client.0 _ll_register_callbacks cb 0x56483596d680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:57.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.027+0000 7fcb2d19b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 0x564835a8e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.027+0000 7fcb2d19b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 0x564835a8e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:46980/0 (socket says 172.21.15.2:46980) 2023-09-13T04:41:57.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.027+0000 7fcb2d19b700 1 -- 172.21.15.2:0/75745200 learned_addr learned my addr 172.21.15.2:0/75745200 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:57.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.028+0000 7fcb2d99c700 1 --2- 172.21.15.2:0/75745200 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564835a84300 0x564835af4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.028+0000 7fcb2c99a700 1 --2- 172.21.15.2:0/75745200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564835a7ffd0 0x564835a8e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.028+0000 7fcb2d19b700 1 -- 172.21.15.2:0/75745200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564835a7ffd0 msgr2=0x564835a8e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:57.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.028+0000 7fcb2d19b700 1 --2- 172.21.15.2:0/75745200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564835a7ffd0 0x564835a8e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.028+0000 7fcb2d19b700 1 -- 172.21.15.2:0/75745200 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564835a84300 msgr2=0x564835af4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:57.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.028+0000 7fcb2d19b700 1 --2- 172.21.15.2:0/75745200 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564835a84300 0x564835af4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.028+0000 7fcb2d19b700 1 -- 172.21.15.2:0/75745200 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648359e97a0 con 0x564835a7f6b0 2023-09-13T04:41:57.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.028+0000 7fcb2c99a700 1 --2- 172.21.15.2:0/75745200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564835a7ffd0 0x564835a8e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:41:57.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.028+0000 7fcb2d19b700 1 --2- 172.21.15.2:0/75745200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 0x564835a8e080 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcb240994d0 tx=0x7fcb2409a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=138315c3b807ec53 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:57.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.028+0000 7fcb10ff9700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:41:57.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.028+0000 7fcb10ff9700 1 -- 172.21.15.2:0/75745200 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24099360 con 0x564835a7f6b0 2023-09-13T04:41:57.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.029+0000 7fcb10ff9700 1 -- 172.21.15.2:0/75745200 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fcb2400cdd0 con 0x564835a7f6b0 2023-09-13T04:41:57.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.029+0000 7fcb42d33b80 1 -- 172.21.15.2:0/75745200 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb24099a00 con 0x564835a7f6b0 2023-09-13T04:41:57.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.029+0000 7fcb10ff9700 1 -- 172.21.15.2:0/75745200 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb24099e60 con 0x564835a7f6b0 2023-09-13T04:41:57.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.029+0000 7fcb42d33b80 10 client.25943 fetch_fsmap learned FSMap version 24 2023-09-13T04:41:57.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.029+0000 7fcb42d33b80 1 -- 172.21.15.2:0/75745200 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fcb28110470 con 0x564835a7f6b0 2023-09-13T04:41:57.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.029+0000 7fcb10ff9700 1 -- 172.21.15.2:0/75745200 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb24099a00 con 0x564835a7f6b0 2023-09-13T04:41:57.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.029+0000 7fcb10ff9700 1 -- 172.21.15.2:0/75745200 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 24) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fcb24098bd0 con 0x564835a7f6b0 2023-09-13T04:41:57.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.029+0000 7fcb42d33b80 10 client.25943 fetch_fsmap finished waiting for FSMap version 24 2023-09-13T04:41:57.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.029+0000 7fcb42d33b80 10 client.25943 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-09-13T04:41:57.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.029+0000 7fcb42d33b80 20 client.25943 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:57.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.029+0000 7fcb42d33b80 1 --2- 172.21.15.2:0/75745200 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x564835af6020 0x564835af8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.029+0000 7fcb42d33b80 4 client.25943 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-09-13T04:41:57.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.029+0000 7fcb42d33b80 1 -- 172.21.15.2:0/75745200 --> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56483598b050 con 0x564835af6020 2023-09-13T04:41:57.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.030+0000 7fcb2c99a700 1 --2- 172.21.15.2:0/75745200 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x564835af6020 0x564835af8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.030+0000 7fcb10ff9700 10 client.25943 ms_handle_connect on v2:172.21.15.156:6838/1785833227 2023-09-13T04:41:57.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.030+0000 7fcb2c99a700 1 --2- 172.21.15.2:0/75745200 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x564835af6020 0x564835af8410 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=520170c9f123531b server_cookie=cb879b4739ab98c8 in_seq=0 out_seq=0 2023-09-13T04:41:57.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.031+0000 7fcb10ff9700 1 -- 172.21.15.2:0/75745200 <== mds.2 v2:172.21.15.156:6838/1785833227 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56483598b050 con 0x564835af6020 2023-09-13T04:41:57.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.031+0000 7fcb10ff9700 10 client.25943 handle_command_reply: tid=0 2023-09-13T04:41:57.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.031+0000 7fcb42d33b80 1 client.25943 shutdown 2023-09-13T04:41:57.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.031+0000 7fcb42d33b80 1 -- 172.21.15.2:0/75745200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 msgr2=0x564835a8e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:57.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.031+0000 7fcb42d33b80 1 --2- 172.21.15.2:0/75745200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 0x564835a8e080 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcb240994d0 tx=0x7fcb2409a270 comp rx=0 tx=0).stop 2023-09-13T04:41:57.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.031+0000 7fcb42d33b80 1 -- 172.21.15.2:0/75745200 shutdown_connections 2023-09-13T04:41:57.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.031+0000 7fcb42d33b80 1 --2- 172.21.15.2:0/75745200 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x564835af6020 0x564835af8410 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.031+0000 7fcb42d33b80 1 --2- 172.21.15.2:0/75745200 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564835a84300 0x564835af4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.031+0000 7fcb42d33b80 1 --2- 172.21.15.2:0/75745200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564835a7f6b0 0x564835a8e080 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.031+0000 7fcb42d33b80 1 --2- 172.21.15.2:0/75745200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564835a7ffd0 0x564835a8e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.031+0000 7fcb42d33b80 1 -- 172.21.15.2:0/75745200 >> 172.21.15.2:0/75745200 conn(0x5648359ddc80 msgr2=0x564835a89680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:57.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.031+0000 7fcb10ff9700 0 client.25943 ms_handle_reset on v2:172.21.15.156:6838/1785833227 2023-09-13T04:41:57.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.031+0000 7fcb42d33b80 1 -- 172.21.15.2:0/75745200 shutdown_connections 2023-09-13T04:41:57.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.032+0000 7fcb42d33b80 1 -- 172.21.15.2:0/75745200 wait complete. 2023-09-13T04:41:57.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.032+0000 7fcb42d33b80 20 client.25943 trim_cache size 0 max 16384 2023-09-13T04:41:57.044 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-09-13T04:41:57.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.032+0000 7fcb2e19d700 1 -- 172.21.15.2:0/1946384064 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcb140a0970 msgr2=0x7fcb140a2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:57.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.032+0000 7fcb2e19d700 1 --2- 172.21.15.2:0/1946384064 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcb140a0970 0x7fcb140a2e30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000f870 tx=0x7fcb2000fc30 comp rx=0 tx=0).stop 2023-09-13T04:41:57.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.032+0000 7fcb2e19d700 1 -- 172.21.15.2:0/1946384064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb28102f60 msgr2=0x7fcb28198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:57.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.032+0000 7fcb2e19d700 1 --2- 172.21.15.2:0/1946384064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb28102f60 0x7fcb28198d90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400eb30 tx=0x7fcb2400ee40 comp rx=0 tx=0).stop 2023-09-13T04:41:57.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.032+0000 7fcb2e19d700 1 -- 172.21.15.2:0/1946384064 shutdown_connections 2023-09-13T04:41:57.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.032+0000 7fcb2e19d700 1 --2- 172.21.15.2:0/1946384064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb2819d730 0x7fcb2819dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.032+0000 7fcb2e19d700 1 --2- 172.21.15.2:0/1946384064 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcb140a0970 0x7fcb140a2e30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.032+0000 7fcb2e19d700 1 --2- 172.21.15.2:0/1946384064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb28102f60 0x7fcb28198d90 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.032+0000 7fcb2e19d700 1 --2- 172.21.15.2:0/1946384064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb28108080 0x7fcb281992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.032+0000 7fcb2e19d700 1 -- 172.21.15.2:0/1946384064 >> 172.21.15.2:0/1946384064 conn(0x7fcb280fe7c0 msgr2=0x7fcb280ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:57.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.033+0000 7fcb2e19d700 1 -- 172.21.15.2:0/1946384064 shutdown_connections 2023-09-13T04:41:57.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.033+0000 7fcb2e19d700 1 -- 172.21.15.2:0/1946384064 wait complete. 2023-09-13T04:41:57.053 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-13T04:41:57.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:56 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2480323220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:41:57.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.296+0000 7f7156f9d700 1 -- 172.21.15.2:0/2480542076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71501034f0 msgr2=0x7f7150103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:57.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.296+0000 7f7156f9d700 1 --2- 172.21.15.2:0/2480542076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71501034f0 0x7f7150103970 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f714800b0a0 tx=0x7f714800b3b0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.296+0000 7f7156f9d700 1 -- 172.21.15.2:0/2480542076 shutdown_connections 2023-09-13T04:41:57.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.296+0000 7f7156f9d700 1 --2- 172.21.15.2:0/2480542076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71501034f0 0x7f7150103970 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.296+0000 7f7156f9d700 1 --2- 172.21.15.2:0/2480542076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7150105aa0 0x7f7150067e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.296+0000 7f7156f9d700 1 --2- 172.21.15.2:0/2480542076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7150068380 0x7f71500687e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.296+0000 7f7156f9d700 1 -- 172.21.15.2:0/2480542076 >> 172.21.15.2:0/2480542076 conn(0x7f71500767d0 msgr2=0x7f7150076be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:57.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.296+0000 7f7156f9d700 1 -- 172.21.15.2:0/2480542076 shutdown_connections 2023-09-13T04:41:57.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.296+0000 7f7156f9d700 1 -- 172.21.15.2:0/2480542076 wait complete. 2023-09-13T04:41:57.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.298+0000 7f7156f9d700 1 Processor -- start 2023-09-13T04:41:57.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.298+0000 7f7156f9d700 1 -- start start 2023-09-13T04:41:57.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.299+0000 7f7156f9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7150068380 0x7f715019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.299+0000 7f7156f9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71501034f0 0x7f715019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.299+0000 7f7156f9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7150105aa0 0x7f71501a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.299+0000 7f7156f9d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71501a1f10 con 0x7f7150068380 2023-09-13T04:41:57.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.299+0000 7f7156f9d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71501a2080 con 0x7f71501034f0 2023-09-13T04:41:57.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.299+0000 7f7156f9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71501a2240 con 0x7f7150105aa0 2023-09-13T04:41:57.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.299+0000 7f7155f9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7150068380 0x7f715019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.299+0000 7f715579a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71501034f0 0x7f715019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.299+0000 7f7155f9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7150068380 0x7f715019cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50528/0 (socket says 172.21.15.2:50528) 2023-09-13T04:41:57.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.299+0000 7f7155f9b700 1 -- 172.21.15.2:0/2413038356 learned_addr learned my addr 172.21.15.2:0/2413038356 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:57.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.299+0000 7f715679c700 1 --2- 172.21.15.2:0/2413038356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7150105aa0 0x7f71501a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.300+0000 7f715679c700 1 -- 172.21.15.2:0/2413038356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71501034f0 msgr2=0x7f715019d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:57.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.300+0000 7f715679c700 1 --2- 172.21.15.2:0/2413038356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71501034f0 0x7f715019d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.300+0000 7f715679c700 1 -- 172.21.15.2:0/2413038356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7150068380 msgr2=0x7f715019cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:57.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.300+0000 7f715679c700 1 --2- 172.21.15.2:0/2413038356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7150068380 0x7f715019cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.300+0000 7f715679c700 1 -- 172.21.15.2:0/2413038356 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7148009d00 con 0x7f7150105aa0 2023-09-13T04:41:57.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.300+0000 7f7155f9b700 1 --2- 172.21.15.2:0/2413038356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7150068380 0x7f715019cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:57.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.300+0000 7f715679c700 1 --2- 172.21.15.2:0/2413038356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7150105aa0 0x7f71501a18d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7148009fd0 tx=0x7f71480075c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:57.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.300+0000 7f7146ffd700 1 -- 172.21.15.2:0/2413038356 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714800e040 con 0x7f7150105aa0 2023-09-13T04:41:57.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.300+0000 7f7146ffd700 1 -- 172.21.15.2:0/2413038356 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7148004c50 con 0x7f7150105aa0 2023-09-13T04:41:57.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.300+0000 7f7146ffd700 1 -- 172.21.15.2:0/2413038356 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714801dea0 con 0x7f7150105aa0 2023-09-13T04:41:57.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.300+0000 7f7156f9d700 1 -- 172.21.15.2:0/2413038356 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71501a24c0 con 0x7f7150105aa0 2023-09-13T04:41:57.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.301+0000 7f7156f9d700 1 -- 172.21.15.2:0/2413038356 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71501a9d60 con 0x7f7150105aa0 2023-09-13T04:41:57.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.305+0000 7f716bb0db80 1 Processor -- start 2023-09-13T04:41:57.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.306+0000 7f7146ffd700 1 -- 172.21.15.2:0/2413038356 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 24) v1 ==== 135491+0+0 (secure 0 0 0) 0x7f714801d470 con 0x7f7150105aa0 2023-09-13T04:41:57.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.306+0000 7f716bb0db80 1 -- start start 2023-09-13T04:41:57.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.306+0000 7f716bb0db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53f96b0 0x5568d53f9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.306+0000 7f716bb0db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568d53f9fd0 0x5568d53fddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.306+0000 7f716bb0db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568d53fe300 0x5568d53fe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.306+0000 7f716bb0db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d5352780 con 0x5568d53f9fd0 2023-09-13T04:41:57.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.306+0000 7f716bb0db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d53500a0 con 0x5568d53f96b0 2023-09-13T04:41:57.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.306+0000 7f716bb0db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d534b990 con 0x5568d53fe300 2023-09-13T04:41:57.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.307+0000 7f715579a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568d53fe300 0x5568d53fe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.307+0000 7f715579a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568d53fe300 0x5568d53fe760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:43140/0 (socket says 172.21.15.2:43140) 2023-09-13T04:41:57.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.307+0000 7f715579a700 1 -- 172.21.15.2:0/2907582732 learned_addr learned my addr 172.21.15.2:0/2907582732 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:57.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.307+0000 7f715579a700 1 --2- 172.21.15.2:0/2907582732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53f96b0 0x5568d53f9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.307+0000 7f7155f9b700 1 --2- 172.21.15.2:0/2907582732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568d53f9fd0 0x5568d53fddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.307+0000 7f715579a700 1 -- 172.21.15.2:0/2907582732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53f96b0 msgr2=0x5568d53f9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:57.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.307+0000 7f715579a700 1 --2- 172.21.15.2:0/2907582732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53f96b0 0x5568d53f9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.307+0000 7f715579a700 1 -- 172.21.15.2:0/2907582732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568d53f9fd0 msgr2=0x5568d53fddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:57.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.307+0000 7f715579a700 1 --2- 172.21.15.2:0/2907582732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568d53f9fd0 0x5568d53fddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.307+0000 7f715579a700 1 -- 172.21.15.2:0/2907582732 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568d51cb630 con 0x5568d53fe300 2023-09-13T04:41:57.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.307+0000 7f715579a700 1 --2- 172.21.15.2:0/2907582732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568d53fe300 0x5568d53fe760 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7140011880 tx=0x7f71400058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cbaae3fd5228437 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:57.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.308+0000 7f7146ffd700 1 --2- 172.21.15.2:0/2413038356 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f713c0a09f0 0x7f713c0a2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.308+0000 7f713bfff700 1 -- 172.21.15.2:0/2907582732 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7140011ea0 con 0x5568d53fe300 2023-09-13T04:41:57.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.308+0000 7f713bfff700 1 -- 172.21.15.2:0/2907582732 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71400044f0 con 0x5568d53fe300 2023-09-13T04:41:57.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.308+0000 7f713bfff700 1 -- 172.21.15.2:0/2907582732 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7140005280 con 0x5568d53fe300 2023-09-13T04:41:57.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.308+0000 7f7146ffd700 1 -- 172.21.15.2:0/2413038356 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f71480cdb50 con 0x7f7150105aa0 2023-09-13T04:41:57.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.308+0000 7f7155f9b700 1 --2- 172.21.15.2:0/2413038356 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f713c0a09f0 0x7f713c0a2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.309+0000 7f716bb0db80 1 -- 172.21.15.2:0/2907582732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568d53fe300 msgr2=0x5568d53fe760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:57.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.309+0000 7f716bb0db80 1 --2- 172.21.15.2:0/2907582732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568d53fe300 0x5568d53fe760 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7140011880 tx=0x7f71400058e0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.309+0000 7f716bb0db80 1 -- 172.21.15.2:0/2907582732 shutdown_connections 2023-09-13T04:41:57.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.309+0000 7f716bb0db80 1 --2- 172.21.15.2:0/2907582732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568d53fe300 0x5568d53fe760 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.309+0000 7f716bb0db80 1 --2- 172.21.15.2:0/2907582732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53f96b0 0x5568d53f9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.309+0000 7f716bb0db80 1 --2- 172.21.15.2:0/2907582732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568d53f9fd0 0x5568d53fddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.309+0000 7f716bb0db80 1 -- 172.21.15.2:0/2907582732 >> 172.21.15.2:0/2907582732 conn(0x5568d5357c80 msgr2=0x5568d53f4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:57.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.309+0000 7f7155f9b700 1 --2- 172.21.15.2:0/2413038356 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f713c0a09f0 0x7f713c0a2eb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f714c005950 tx=0x7f714c00a400 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:57.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.309+0000 7f716bb0db80 1 -- 172.21.15.2:0/2907582732 shutdown_connections 2023-09-13T04:41:57.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.309+0000 7f716bb0db80 1 -- 172.21.15.2:0/2907582732 wait complete. 2023-09-13T04:41:57.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.310+0000 7f716bb0db80 1 -- start start 2023-09-13T04:41:57.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.310+0000 7f716bb0db80 1 Processor -- start 2023-09-13T04:41:57.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.311+0000 7f716bb0db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568d53f96b0 0x5568d5402ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.311+0000 7f716bb0db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568d53f9fd0 0x5568d54033e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.311+0000 7f716bb0db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53fe300 0x5568d5403920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.311+0000 7f716bb0db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d52e9bb0 con 0x5568d53f9fd0 2023-09-13T04:41:57.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.311+0000 7f716bb0db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d52e98b0 con 0x5568d53fe300 2023-09-13T04:41:57.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.311+0000 7f716bb0db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d51cd920 con 0x5568d53f96b0 2023-09-13T04:41:57.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.311+0000 7f716bb0db80 10 client.0 _ll_register_callbacks cb 0x5568d52e7680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:57.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.312+0000 7f715679c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53fe300 0x5568d5403920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.312+0000 7f715679c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53fe300 0x5568d5403920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:47022/0 (socket says 172.21.15.2:47022) 2023-09-13T04:41:57.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.312+0000 7f715679c700 1 -- 172.21.15.2:0/1476384531 learned_addr learned my addr 172.21.15.2:0/1476384531 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:57.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.312+0000 7f715579a700 1 --2- 172.21.15.2:0/1476384531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568d53f9fd0 0x5568d54033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.312+0000 7f715679c700 1 -- 172.21.15.2:0/1476384531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568d53f96b0 msgr2=0x5568d5402ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:57.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.312+0000 7f715679c700 1 --2- 172.21.15.2:0/1476384531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568d53f96b0 0x5568d5402ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.312+0000 7f715679c700 1 -- 172.21.15.2:0/1476384531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568d53f9fd0 msgr2=0x5568d54033e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:57.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.312+0000 7f715679c700 1 --2- 172.21.15.2:0/1476384531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568d53f9fd0 0x5568d54033e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.312+0000 7f715679c700 1 -- 172.21.15.2:0/1476384531 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7140019040 con 0x5568d53fe300 2023-09-13T04:41:57.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.312+0000 7f715679c700 1 --2- 172.21.15.2:0/1476384531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53fe300 0x5568d5403920 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f714800b780 tx=0x7f714809b1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4df351160afc038 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:57.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.312+0000 7f71397fa700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:41:57.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.313+0000 7f71397fa700 1 -- 172.21.15.2:0/1476384531 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714809aad0 con 0x5568d53fe300 2023-09-13T04:41:57.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.313+0000 7f71397fa700 1 -- 172.21.15.2:0/1476384531 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f714809e900 con 0x5568d53fe300 2023-09-13T04:41:57.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.313+0000 7f716bb0db80 1 -- 172.21.15.2:0/1476384531 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568d53ec670 con 0x5568d53fe300 2023-09-13T04:41:57.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.313+0000 7f71397fa700 1 -- 172.21.15.2:0/1476384531 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71480a2ab0 con 0x5568d53fe300 2023-09-13T04:41:57.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.313+0000 7f716bb0db80 10 client.25952 fetch_fsmap learned FSMap version 24 2023-09-13T04:41:57.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.313+0000 7f716bb0db80 1 -- 172.21.15.2:0/1476384531 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5568d53637a0 con 0x5568d53fe300 2023-09-13T04:41:57.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.313+0000 7f71397fa700 1 -- 172.21.15.2:0/1476384531 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5568d53ec670 con 0x5568d53fe300 2023-09-13T04:41:57.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.313+0000 7f71397fa700 1 -- 172.21.15.2:0/1476384531 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 24) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f71480cb090 con 0x5568d53fe300 2023-09-13T04:41:57.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.314+0000 7f716bb0db80 10 client.25952 fetch_fsmap finished waiting for FSMap version 24 2023-09-13T04:41:57.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.314+0000 7f716bb0db80 10 client.25952 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-09-13T04:41:57.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.314+0000 7f716bb0db80 20 client.25952 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:57.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.314+0000 7f716bb0db80 1 --2- 172.21.15.2:0/1476384531 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x5568d54131e0 0x5568d5415600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.314+0000 7f716bb0db80 4 client.25952 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:41:57.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.314+0000 7f716bb0db80 1 -- 172.21.15.2:0/1476384531 --> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568d5344970 con 0x5568d54131e0 2023-09-13T04:41:57.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.314+0000 7f7155f9b700 1 --2- 172.21.15.2:0/1476384531 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x5568d54131e0 0x5568d5415600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.315+0000 7f7155f9b700 1 --2- 172.21.15.2:0/1476384531 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x5568d54131e0 0x5568d5415600 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d659d975f0463b0 server_cookie=4538d449171005b4 in_seq=0 out_seq=0 2023-09-13T04:41:57.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.315+0000 7f71397fa700 10 client.25952 ms_handle_connect on v2:172.21.15.2:6840/482545391 2023-09-13T04:41:57.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.318+0000 7f71397fa700 1 -- 172.21.15.2:0/1476384531 <== mds.1 v2:172.21.15.2:6840/482545391 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5568d5344970 con 0x5568d54131e0 2023-09-13T04:41:57.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.318+0000 7f71397fa700 10 client.25952 handle_command_reply: tid=0 2023-09-13T04:41:57.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.318+0000 7f716bb0db80 1 client.25952 shutdown 2023-09-13T04:41:57.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.318+0000 7f716bb0db80 1 -- 172.21.15.2:0/1476384531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53fe300 msgr2=0x5568d5403920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:57.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.318+0000 7f716bb0db80 1 --2- 172.21.15.2:0/1476384531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53fe300 0x5568d5403920 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f714800b780 tx=0x7f714809b1d0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.319+0000 7f716bb0db80 1 -- 172.21.15.2:0/1476384531 shutdown_connections 2023-09-13T04:41:57.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.319+0000 7f716bb0db80 1 --2- 172.21.15.2:0/1476384531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53fe300 0x5568d5403920 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.319+0000 7f716bb0db80 1 --2- 172.21.15.2:0/1476384531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568d53f96b0 0x5568d5402ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.319+0000 7f716bb0db80 1 --2- 172.21.15.2:0/1476384531 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x5568d54131e0 0x5568d5415600 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.319+0000 7f716bb0db80 1 --2- 172.21.15.2:0/1476384531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568d53f9fd0 0x5568d54033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.319+0000 7f716bb0db80 1 -- 172.21.15.2:0/1476384531 >> 172.21.15.2:0/1476384531 conn(0x5568d5357c80 msgr2=0x5568d53f84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:57.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.319+0000 7f71397fa700 0 client.25952 ms_handle_reset on v2:172.21.15.2:6840/482545391 2023-09-13T04:41:57.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.319+0000 7f716bb0db80 1 -- 172.21.15.2:0/1476384531 shutdown_connections 2023-09-13T04:41:57.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.319+0000 7f716bb0db80 1 -- 172.21.15.2:0/1476384531 wait complete. 2023-09-13T04:41:57.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.319+0000 7f716bb0db80 20 client.25952 trim_cache size 0 max 16384 2023-09-13T04:41:57.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.333+0000 7f716bb0db80 1 Processor -- start 2023-09-13T04:41:57.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.333+0000 7f716bb0db80 1 -- start start 2023-09-13T04:41:57.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.334+0000 7f716bb0db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568d53f96b0 0x5568d53fbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.334+0000 7f716bb0db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53f9fd0 0x5568d540a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.334+0000 7f716bb0db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568d53fe300 0x5568d5403750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.334+0000 7f716bb0db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d53f0920 con 0x5568d53f96b0 2023-09-13T04:41:57.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.334+0000 7f716bb0db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d53f5fa0 con 0x5568d53f9fd0 2023-09-13T04:41:57.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.334+0000 7f716bb0db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d53f8b30 con 0x5568d53fe300 2023-09-13T04:41:57.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.334+0000 7f7155f9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568d53f96b0 0x5568d53fbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.335+0000 7f7155f9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568d53f96b0 0x5568d53fbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50558/0 (socket says 172.21.15.2:50558) 2023-09-13T04:41:57.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.335+0000 7f7155f9b700 1 -- 172.21.15.2:0/4071496272 learned_addr learned my addr 172.21.15.2:0/4071496272 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:57.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.335+0000 7f715679c700 1 --2- 172.21.15.2:0/4071496272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568d53fe300 0x5568d5403750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.335+0000 7f715579a700 1 --2- 172.21.15.2:0/4071496272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53f9fd0 0x5568d540a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.335+0000 7f7155f9b700 1 -- 172.21.15.2:0/4071496272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568d53fe300 msgr2=0x5568d5403750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:57.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.335+0000 7f7155f9b700 1 --2- 172.21.15.2:0/4071496272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568d53fe300 0x5568d5403750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.335+0000 7f7155f9b700 1 -- 172.21.15.2:0/4071496272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53f9fd0 msgr2=0x5568d540a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:57.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.335+0000 7f7155f9b700 1 --2- 172.21.15.2:0/4071496272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53f9fd0 0x5568d540a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.335+0000 7f7155f9b700 1 -- 172.21.15.2:0/4071496272 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568d5177de0 con 0x5568d53f96b0 2023-09-13T04:41:57.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.335+0000 7f715679c700 1 --2- 172.21.15.2:0/4071496272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568d53fe300 0x5568d5403750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:57.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.335+0000 7f715579a700 1 --2- 172.21.15.2:0/4071496272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53f9fd0 0x5568d540a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:57.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.335+0000 7f7155f9b700 1 --2- 172.21.15.2:0/4071496272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568d53f96b0 0x5568d53fbbf0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f714c026cb0 tx=0x7f714c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141580e1cf2b3d95 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:57.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.336+0000 7f7144ff9700 1 -- 172.21.15.2:0/4071496272 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c021040 con 0x5568d53f96b0 2023-09-13T04:41:57.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.336+0000 7f7144ff9700 1 -- 172.21.15.2:0/4071496272 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f714c004cf0 con 0x5568d53f96b0 2023-09-13T04:41:57.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.336+0000 7f7144ff9700 1 -- 172.21.15.2:0/4071496272 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c015070 con 0x5568d53f96b0 2023-09-13T04:41:57.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.337+0000 7f716bb0db80 1 -- 172.21.15.2:0/4071496272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568d53f96b0 msgr2=0x5568d53fbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:57.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.337+0000 7f716bb0db80 1 --2- 172.21.15.2:0/4071496272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568d53f96b0 0x5568d53fbbf0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f714c026cb0 tx=0x7f714c004750 comp rx=0 tx=0).stop 2023-09-13T04:41:57.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.337+0000 7f716bb0db80 1 -- 172.21.15.2:0/4071496272 shutdown_connections 2023-09-13T04:41:57.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.337+0000 7f716bb0db80 1 --2- 172.21.15.2:0/4071496272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568d53fe300 0x5568d5403750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.337+0000 7f716bb0db80 1 --2- 172.21.15.2:0/4071496272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568d53f96b0 0x5568d53fbbf0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.337+0000 7f716bb0db80 1 --2- 172.21.15.2:0/4071496272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53f9fd0 0x5568d540a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.337+0000 7f716bb0db80 1 -- 172.21.15.2:0/4071496272 >> 172.21.15.2:0/4071496272 conn(0x5568d5357c80 msgr2=0x5568d54151f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:57.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.337+0000 7f716bb0db80 1 -- 172.21.15.2:0/4071496272 shutdown_connections 2023-09-13T04:41:57.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.337+0000 7f716bb0db80 1 -- 172.21.15.2:0/4071496272 wait complete. 2023-09-13T04:41:57.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.337+0000 7f716bb0db80 1 -- start start 2023-09-13T04:41:57.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.338+0000 7f716bb0db80 1 Processor -- start 2023-09-13T04:41:57.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.338+0000 7f716bb0db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568d53f96b0 0x5568d5407ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.338+0000 7f716bb0db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53f9fd0 0x5568d5408020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.338+0000 7f716bb0db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568d53fe300 0x5568d54085a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.338+0000 7f716bb0db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d5466250 con 0x5568d53f96b0 2023-09-13T04:41:57.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.338+0000 7f716bb0db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d54663c0 con 0x5568d53f9fd0 2023-09-13T04:41:57.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.338+0000 7f716bb0db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d5466580 con 0x5568d53fe300 2023-09-13T04:41:57.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.338+0000 7f716bb0db80 10 client.0 _ll_register_callbacks cb 0x5568d52e7680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:57.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.339+0000 7f7155f9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568d53f96b0 0x5568d5407ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.339+0000 7f715579a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53f9fd0 0x5568d5408020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.339+0000 7f715579a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53f9fd0 0x5568d5408020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:47054/0 (socket says 172.21.15.2:47054) 2023-09-13T04:41:57.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.339+0000 7f715579a700 1 -- 172.21.15.2:0/2857767067 learned_addr learned my addr 172.21.15.2:0/2857767067 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:57.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.339+0000 7f715679c700 1 --2- 172.21.15.2:0/2857767067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568d53fe300 0x5568d54085a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.339+0000 7f7155f9b700 1 -- 172.21.15.2:0/2857767067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568d53fe300 msgr2=0x5568d54085a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:57.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.339+0000 7f7155f9b700 1 --2- 172.21.15.2:0/2857767067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568d53fe300 0x5568d54085a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.339+0000 7f7155f9b700 1 -- 172.21.15.2:0/2857767067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53f9fd0 msgr2=0x5568d5408020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:57.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.339+0000 7f7155f9b700 1 --2- 172.21.15.2:0/2857767067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53f9fd0 0x5568d5408020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.339+0000 7f7155f9b700 1 -- 172.21.15.2:0/2857767067 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7140011e70 con 0x5568d53f96b0 2023-09-13T04:41:57.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.339+0000 7f715679c700 1 --2- 172.21.15.2:0/2857767067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568d53fe300 0x5568d54085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:57.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.339+0000 7f715579a700 1 --2- 172.21.15.2:0/2857767067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53f9fd0 0x5568d5408020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:57.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.340+0000 7f7155f9b700 1 --2- 172.21.15.2:0/2857767067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568d53f96b0 0x5568d5407ae0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f714c017040 tx=0x7f714c01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d042597526f820ee server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:57.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.340+0000 7f71397fa700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:41:57.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.340+0000 7f71397fa700 1 -- 172.21.15.2:0/2857767067 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c021040 con 0x5568d53f96b0 2023-09-13T04:41:57.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.340+0000 7f716bb0db80 1 -- 172.21.15.2:0/2857767067 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71400044f0 con 0x5568d53f96b0 2023-09-13T04:41:57.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.340+0000 7f71397fa700 1 -- 172.21.15.2:0/2857767067 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f714c01cb10 con 0x5568d53f96b0 2023-09-13T04:41:57.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.341+0000 7f71397fa700 1 -- 172.21.15.2:0/2857767067 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f714c01ccd0 con 0x5568d53f96b0 2023-09-13T04:41:57.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.341+0000 7f716bb0db80 10 client.15936 fetch_fsmap learned FSMap version 24 2023-09-13T04:41:57.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.341+0000 7f716bb0db80 1 -- 172.21.15.2:0/2857767067 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5568d53637a0 con 0x5568d53f96b0 2023-09-13T04:41:57.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.341+0000 7f71397fa700 1 -- 172.21.15.2:0/2857767067 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71400044f0 con 0x5568d53f96b0 2023-09-13T04:41:57.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.341+0000 7f71397fa700 1 -- 172.21.15.2:0/2857767067 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 24) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f714c005460 con 0x5568d53f96b0 2023-09-13T04:41:57.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.341+0000 7f716bb0db80 10 client.15936 fetch_fsmap finished waiting for FSMap version 24 2023-09-13T04:41:57.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.341+0000 7f716bb0db80 10 client.15936 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-09-13T04:41:57.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.341+0000 7f716bb0db80 20 client.15936 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:57.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.341+0000 7f716bb0db80 1 --2- 172.21.15.2:0/2857767067 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x5568d5467b70 0x5568d5469f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.341+0000 7f716bb0db80 4 client.15936 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-09-13T04:41:57.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.341+0000 7f716bb0db80 1 -- 172.21.15.2:0/2857767067 --> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5568d5305050 con 0x5568d5467b70 2023-09-13T04:41:57.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.342+0000 7f715579a700 1 --2- 172.21.15.2:0/2857767067 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x5568d5467b70 0x5568d5469f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.342+0000 7f715579a700 1 --2- 172.21.15.2:0/2857767067 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x5568d5467b70 0x5568d5469f90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a6f5964ceee991b5 server_cookie=4bc88e1488aee3c2 in_seq=0 out_seq=0 2023-09-13T04:41:57.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.342+0000 7f71397fa700 10 client.15936 ms_handle_connect on v2:172.21.15.2:6840/482545391 2023-09-13T04:41:57.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.345+0000 7f71397fa700 1 -- 172.21.15.2:0/2857767067 <== mds.1 v2:172.21.15.2:6840/482545391 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5568d5305050 con 0x5568d5467b70 2023-09-13T04:41:57.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.345+0000 7f71397fa700 10 client.15936 handle_command_reply: tid=0 2023-09-13T04:41:57.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.345+0000 7f716bb0db80 1 client.15936 shutdown 2023-09-13T04:41:57.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.345+0000 7f716bb0db80 1 -- 172.21.15.2:0/2857767067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568d53f96b0 msgr2=0x5568d5407ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:57.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.345+0000 7f716bb0db80 1 --2- 172.21.15.2:0/2857767067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568d53f96b0 0x5568d5407ae0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f714c017040 tx=0x7f714c01c8c0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.345+0000 7f716bb0db80 1 -- 172.21.15.2:0/2857767067 shutdown_connections 2023-09-13T04:41:57.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.345+0000 7f716bb0db80 1 --2- 172.21.15.2:0/2857767067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568d53fe300 0x5568d54085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.345+0000 7f716bb0db80 1 --2- 172.21.15.2:0/2857767067 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x5568d5467b70 0x5568d5469f90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.345+0000 7f716bb0db80 1 --2- 172.21.15.2:0/2857767067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568d53f96b0 0x5568d5407ae0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.345+0000 7f716bb0db80 1 --2- 172.21.15.2:0/2857767067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568d53f9fd0 0x5568d5408020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.345+0000 7f71397fa700 0 client.15936 ms_handle_reset on v2:172.21.15.2:6840/482545391 2023-09-13T04:41:57.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.345+0000 7f716bb0db80 1 -- 172.21.15.2:0/2857767067 >> 172.21.15.2:0/2857767067 conn(0x5568d5357c80 msgr2=0x5568d53f3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:57.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.345+0000 7f716bb0db80 1 -- 172.21.15.2:0/2857767067 shutdown_connections 2023-09-13T04:41:57.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.345+0000 7f716bb0db80 1 -- 172.21.15.2:0/2857767067 wait complete. 2023-09-13T04:41:57.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.346+0000 7f716bb0db80 20 client.15936 trim_cache size 0 max 16384 2023-09-13T04:41:57.365 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-09-13T04:41:57.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.346+0000 7f7156f9d700 1 -- 172.21.15.2:0/2413038356 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f713c0a09f0 msgr2=0x7f713c0a2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:57.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.346+0000 7f7156f9d700 1 --2- 172.21.15.2:0/2413038356 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f713c0a09f0 0x7f713c0a2eb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f714c005950 tx=0x7f714c00a400 comp rx=0 tx=0).stop 2023-09-13T04:41:57.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.346+0000 7f7156f9d700 1 -- 172.21.15.2:0/2413038356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7150105aa0 msgr2=0x7f71501a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:57.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.346+0000 7f7156f9d700 1 --2- 172.21.15.2:0/2413038356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7150105aa0 0x7f71501a18d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7148009fd0 tx=0x7f71480075c0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.346+0000 7f7156f9d700 1 -- 172.21.15.2:0/2413038356 shutdown_connections 2023-09-13T04:41:57.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.346+0000 7f7156f9d700 1 --2- 172.21.15.2:0/2413038356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7150105aa0 0x7f71501a18d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.346+0000 7f7156f9d700 1 --2- 172.21.15.2:0/2413038356 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f713c0a09f0 0x7f713c0a2eb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.346+0000 7f7156f9d700 1 --2- 172.21.15.2:0/2413038356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7150068380 0x7f715019cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.346+0000 7f7156f9d700 1 --2- 172.21.15.2:0/2413038356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71501034f0 0x7f715019d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.347+0000 7f7156f9d700 1 -- 172.21.15.2:0/2413038356 >> 172.21.15.2:0/2413038356 conn(0x7f71500767d0 msgr2=0x7f71500fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:57.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.347+0000 7f7156f9d700 1 -- 172.21.15.2:0/2413038356 shutdown_connections 2023-09-13T04:41:57.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.347+0000 7f7156f9d700 1 -- 172.21.15.2:0/2413038356 wait complete. 2023-09-13T04:41:57.369 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-13T04:41:57.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:56 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2480323220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:41:57.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.613+0000 7f86eeb9d700 1 -- 172.21.15.2:0/4245327473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86e8106ac0 msgr2=0x7f86e8106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:57.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.613+0000 7f86eeb9d700 1 --2- 172.21.15.2:0/4245327473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86e8106ac0 0x7f86e8106ea0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009990 tx=0x7f86e4009ca0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.613+0000 7f86eeb9d700 1 -- 172.21.15.2:0/4245327473 shutdown_connections 2023-09-13T04:41:57.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.613+0000 7f86eeb9d700 1 --2- 172.21.15.2:0/4245327473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86e8101470 0x7f86e810d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.613+0000 7f86eeb9d700 1 --2- 172.21.15.2:0/4245327473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86e8106ac0 0x7f86e8106ea0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.613+0000 7f86eeb9d700 1 --2- 172.21.15.2:0/4245327473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86e8100ab0 0x7f86e8100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.613+0000 7f86eeb9d700 1 -- 172.21.15.2:0/4245327473 >> 172.21.15.2:0/4245327473 conn(0x7f86e80fc560 msgr2=0x7f86e80fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:57.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.613+0000 7f86eeb9d700 1 -- 172.21.15.2:0/4245327473 shutdown_connections 2023-09-13T04:41:57.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.613+0000 7f86eeb9d700 1 -- 172.21.15.2:0/4245327473 wait complete. 2023-09-13T04:41:57.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.614+0000 7f86eeb9d700 1 Processor -- start 2023-09-13T04:41:57.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.615+0000 7f86eeb9d700 1 -- start start 2023-09-13T04:41:57.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.615+0000 7f86eeb9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86e8100ab0 0x7f86e8072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.615+0000 7f86eeb9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86e8101470 0x7f86e80734f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.615+0000 7f86eeb9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86e8106ac0 0x7f86e81a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.615+0000 7f86eeb9d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e8073d20 con 0x7f86e8106ac0 2023-09-13T04:41:57.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.615+0000 7f86eeb9d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e81a5190 con 0x7f86e8101470 2023-09-13T04:41:57.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.615+0000 7f86eeb9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e81a52d0 con 0x7f86e8100ab0 2023-09-13T04:41:57.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.615+0000 7f86edb9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86e8100ab0 0x7f86e8072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.615+0000 7f86ed39a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86e8101470 0x7f86e80734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.616+0000 7f86edb9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86e8100ab0 0x7f86e8072fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:43196/0 (socket says 172.21.15.2:43196) 2023-09-13T04:41:57.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.616+0000 7f86edb9b700 1 -- 172.21.15.2:0/1803361874 learned_addr learned my addr 172.21.15.2:0/1803361874 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:57.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.616+0000 7f86ed39a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86e8101470 0x7f86e80734f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:47070/0 (socket says 172.21.15.2:47070) 2023-09-13T04:41:57.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.616+0000 7f86ee39c700 1 --2- 172.21.15.2:0/1803361874 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86e8106ac0 0x7f86e81a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.616+0000 7f86ed39a700 1 -- 172.21.15.2:0/1803361874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86e8100ab0 msgr2=0x7f86e8072fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:57.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.616+0000 7f86ed39a700 1 --2- 172.21.15.2:0/1803361874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86e8100ab0 0x7f86e8072fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.616+0000 7f86ed39a700 1 -- 172.21.15.2:0/1803361874 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86e8106ac0 msgr2=0x7f86e81a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:57.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.616+0000 7f86ed39a700 1 --2- 172.21.15.2:0/1803361874 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86e8106ac0 0x7f86e81a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.616+0000 7f86ed39a700 1 -- 172.21.15.2:0/1803361874 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e40095f0 con 0x7f86e8101470 2023-09-13T04:41:57.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.616+0000 7f86ee39c700 1 --2- 172.21.15.2:0/1803361874 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86e8106ac0 0x7f86e81a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:57.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.616+0000 7f86ed39a700 1 --2- 172.21.15.2:0/1803361874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86e8101470 0x7f86e80734f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f86d800e9f0 tx=0x7f86d800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:57.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.617+0000 7f86deffd700 1 -- 172.21.15.2:0/1803361874 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d800c9c0 con 0x7f86e8101470 2023-09-13T04:41:57.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.617+0000 7f86eeb9d700 1 -- 172.21.15.2:0/1803361874 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e81a5530 con 0x7f86e8101470 2023-09-13T04:41:57.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.617+0000 7f86deffd700 1 -- 172.21.15.2:0/1803361874 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86d8010430 con 0x7f86e8101470 2023-09-13T04:41:57.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.617+0000 7f86deffd700 1 -- 172.21.15.2:0/1803361874 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8018680 con 0x7f86e8101470 2023-09-13T04:41:57.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.617+0000 7f86eeb9d700 1 -- 172.21.15.2:0/1803361874 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e81a5a60 con 0x7f86e8101470 2023-09-13T04:41:57.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.622+0000 7f87037edb80 1 Processor -- start 2023-09-13T04:41:57.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.622+0000 7f87037edb80 1 -- start start 2023-09-13T04:41:57.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.622+0000 7f86deffd700 1 -- 172.21.15.2:0/1803361874 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 24) v1 ==== 135491+0+0 (secure 0 0 0) 0x7f86d8010860 con 0x7f86e8101470 2023-09-13T04:41:57.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.623+0000 7f87037edb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daf7b47900 0x55daf7b47ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.623+0000 7f87037edb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55daf7b48220 0x55daf7b42cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.623+0000 7f87037edb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b43210 0x55daf7b43650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.623+0000 7f87037edb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf7a93780 con 0x55daf7b48220 2023-09-13T04:41:57.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.623+0000 7f87037edb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf7a910a0 con 0x55daf7b47900 2023-09-13T04:41:57.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.623+0000 7f87037edb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf7a8c990 con 0x55daf7b43210 2023-09-13T04:41:57.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.623+0000 7f86ed39a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b43210 0x55daf7b43650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.623+0000 7f86ed39a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b43210 0x55daf7b43650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:43200/0 (socket says 172.21.15.2:43200) 2023-09-13T04:41:57.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.623+0000 7f86edb9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55daf7b48220 0x55daf7b42cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.623+0000 7f86ed39a700 1 -- 172.21.15.2:0/1249919473 learned_addr learned my addr 172.21.15.2:0/1249919473 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:57.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.624+0000 7f86edb9b700 1 -- 172.21.15.2:0/1249919473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b43210 msgr2=0x55daf7b43650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:57.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.624+0000 7f86edb9b700 1 --2- 172.21.15.2:0/1249919473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b43210 0x55daf7b43650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.624+0000 7f86edb9b700 1 -- 172.21.15.2:0/1249919473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daf7b47900 msgr2=0x55daf7b47ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:41:57.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.624+0000 7f86edb9b700 1 --2- 172.21.15.2:0/1249919473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daf7b47900 0x55daf7b47ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.624+0000 7f86edb9b700 1 -- 172.21.15.2:0/1249919473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daf790c630 con 0x55daf7b48220 2023-09-13T04:41:57.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.624+0000 7f86ed39a700 1 --2- 172.21.15.2:0/1249919473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b43210 0x55daf7b43650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:57.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.624+0000 7f86edb9b700 1 --2- 172.21.15.2:0/1249919473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55daf7b48220 0x55daf7b42cd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f86e40059d0 tx=0x7f86e400b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38774698db9b7072 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:57.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.625+0000 7f86deffd700 1 --2- 172.21.15.2:0/1803361874 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f86d40a0970 0x7f86d40a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.625+0000 7f86deffd700 1 -- 172.21.15.2:0/1803361874 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f86d8014070 con 0x7f86e8101470 2023-09-13T04:41:57.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.625+0000 7f86d3fff700 1 -- 172.21.15.2:0/1249919473 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e401d070 con 0x55daf7b48220 2023-09-13T04:41:57.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.625+0000 7f86d3fff700 1 -- 172.21.15.2:0/1249919473 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86e401e040 con 0x55daf7b48220 2023-09-13T04:41:57.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.625+0000 7f86d3fff700 1 -- 172.21.15.2:0/1249919473 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e40049c0 con 0x55daf7b48220 2023-09-13T04:41:57.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.625+0000 7f86ee39c700 1 --2- 172.21.15.2:0/1803361874 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f86d40a0970 0x7f86d40a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.626+0000 7f87037edb80 1 -- 172.21.15.2:0/1249919473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55daf7b48220 msgr2=0x55daf7b42cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:57.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.626+0000 7f87037edb80 1 --2- 172.21.15.2:0/1249919473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55daf7b48220 0x55daf7b42cd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f86e40059d0 tx=0x7f86e400b690 comp rx=0 tx=0).stop 2023-09-13T04:41:57.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.626+0000 7f87037edb80 1 -- 172.21.15.2:0/1249919473 shutdown_connections 2023-09-13T04:41:57.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.626+0000 7f87037edb80 1 --2- 172.21.15.2:0/1249919473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b43210 0x55daf7b43650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.626+0000 7f87037edb80 1 --2- 172.21.15.2:0/1249919473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daf7b47900 0x55daf7b47ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.626+0000 7f87037edb80 1 --2- 172.21.15.2:0/1249919473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55daf7b48220 0x55daf7b42cd0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.626+0000 7f87037edb80 1 -- 172.21.15.2:0/1249919473 >> 172.21.15.2:0/1249919473 conn(0x55daf7a98c80 msgr2=0x55daf7b35300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:57.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.626+0000 7f87037edb80 1 -- 172.21.15.2:0/1249919473 shutdown_connections 2023-09-13T04:41:57.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.626+0000 7f87037edb80 1 -- 172.21.15.2:0/1249919473 wait complete. 2023-09-13T04:41:57.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.626+0000 7f86ee39c700 1 --2- 172.21.15.2:0/1803361874 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f86d40a0970 0x7f86d40a2e30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55daf7b36f70 tx=0x7f86e000b040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:57.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.627+0000 7f87037edb80 1 -- start start 2023-09-13T04:41:57.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.627+0000 7f87037edb80 1 Processor -- start 2023-09-13T04:41:57.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.628+0000 7f87037edb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b43210 0x55daf7b4f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.628+0000 7f87037edb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daf7b47900 0x55daf7b4c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.628+0000 7f87037edb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55daf7b48220 0x55daf7b4ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.628+0000 7f87037edb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf7a2abb0 con 0x55daf7b48220 2023-09-13T04:41:57.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.628+0000 7f87037edb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf7a2a8b0 con 0x55daf7b47900 2023-09-13T04:41:57.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.628+0000 7f87037edb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf790e920 con 0x55daf7b43210 2023-09-13T04:41:57.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.628+0000 7f87037edb80 10 client.0 _ll_register_callbacks cb 0x55daf7a28680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:57.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.628+0000 7f86edb9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b43210 0x55daf7b4f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.628+0000 7f86edb9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b43210 0x55daf7b4f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:43206/0 (socket says 172.21.15.2:43206) 2023-09-13T04:41:57.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.628+0000 7f86edb9b700 1 -- 172.21.15.2:0/2917154275 learned_addr learned my addr 172.21.15.2:0/2917154275 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:57.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.629+0000 7f86ee39c700 1 --2- 172.21.15.2:0/2917154275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55daf7b48220 0x55daf7b4ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.629+0000 7f86ed39a700 1 --2- 172.21.15.2:0/2917154275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daf7b47900 0x55daf7b4c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.629+0000 7f86edb9b700 1 -- 172.21.15.2:0/2917154275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daf7b47900 msgr2=0x55daf7b4c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:57.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.629+0000 7f86edb9b700 1 --2- 172.21.15.2:0/2917154275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daf7b47900 0x55daf7b4c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.629+0000 7f86edb9b700 1 -- 172.21.15.2:0/2917154275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55daf7b48220 msgr2=0x55daf7b4ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:57.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.629+0000 7f86edb9b700 1 --2- 172.21.15.2:0/2917154275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55daf7b48220 0x55daf7b4ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.629+0000 7f86edb9b700 1 -- 172.21.15.2:0/2917154275 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e810e280 con 0x55daf7b43210 2023-09-13T04:41:57.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.629+0000 7f86ed39a700 1 --2- 172.21.15.2:0/2917154275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daf7b47900 0x55daf7b4c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:41:57.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.629+0000 7f86ee39c700 1 --2- 172.21.15.2:0/2917154275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55daf7b48220 0x55daf7b4ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:41:57.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.629+0000 7f86edb9b700 1 --2- 172.21.15.2:0/2917154275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b43210 0x55daf7b4f1f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009ce0 tx=0x7f86e400f720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41eaa4061fc3c5f2 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:57.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.629+0000 7f86d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:41:57.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.629+0000 7f86d17fa700 1 -- 172.21.15.2:0/2917154275 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e401d040 con 0x55daf7b43210 2023-09-13T04:41:57.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.629+0000 7f87037edb80 1 -- 172.21.15.2:0/2917154275 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55daf7b2d670 con 0x55daf7b43210 2023-09-13T04:41:57.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.630+0000 7f86d17fa700 1 -- 172.21.15.2:0/2917154275 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f86e4034460 con 0x55daf7b43210 2023-09-13T04:41:57.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.630+0000 7f86d17fa700 1 -- 172.21.15.2:0/2917154275 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86e4005680 con 0x55daf7b43210 2023-09-13T04:41:57.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.630+0000 7f87037edb80 10 client.25984 fetch_fsmap learned FSMap version 24 2023-09-13T04:41:57.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.630+0000 7f87037edb80 1 -- 172.21.15.2:0/2917154275 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55daf7aa47a0 con 0x55daf7b43210 2023-09-13T04:41:57.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.630+0000 7f86d17fa700 1 -- 172.21.15.2:0/2917154275 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55daf7b2d670 con 0x55daf7b43210 2023-09-13T04:41:57.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.630+0000 7f86d17fa700 1 -- 172.21.15.2:0/2917154275 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 24) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f86e4022490 con 0x55daf7b43210 2023-09-13T04:41:57.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.630+0000 7f87037edb80 10 client.25984 fetch_fsmap finished waiting for FSMap version 24 2023-09-13T04:41:57.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.630+0000 7f87037edb80 10 client.25984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:41:57.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.630+0000 7f87037edb80 20 client.25984 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:57.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.630+0000 7f87037edb80 1 --2- 172.21.15.2:0/2917154275 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55daf7b5c680 0x55daf7b5ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.630+0000 7f87037edb80 4 client.25984 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:41:57.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.630+0000 7f87037edb80 1 -- 172.21.15.2:0/2917154275 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55daf7a85970 con 0x55daf7b5c680 2023-09-13T04:41:57.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.631+0000 7f86ed39a700 1 --2- 172.21.15.2:0/2917154275 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55daf7b5c680 0x55daf7b5ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.632+0000 7f86ed39a700 1 --2- 172.21.15.2:0/2917154275 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55daf7b5c680 0x55daf7b5ea70 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9ca7d4ed2aa719a server_cookie=3b6cd831e01b654 in_seq=0 out_seq=0 2023-09-13T04:41:57.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.632+0000 7f86d17fa700 10 client.25984 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:57.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.632+0000 7f86d17fa700 1 -- 172.21.15.2:0/2917154275 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55daf7a85970 con 0x55daf7b5c680 2023-09-13T04:41:57.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.632+0000 7f86d17fa700 10 client.25984 handle_command_reply: tid=0 2023-09-13T04:41:57.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.632+0000 7f87037edb80 1 client.25984 shutdown 2023-09-13T04:41:57.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.633+0000 7f87037edb80 1 -- 172.21.15.2:0/2917154275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b43210 msgr2=0x55daf7b4f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:57.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.633+0000 7f87037edb80 1 --2- 172.21.15.2:0/2917154275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b43210 0x55daf7b4f1f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009ce0 tx=0x7f86e400f720 comp rx=0 tx=0).stop 2023-09-13T04:41:57.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.633+0000 7f87037edb80 1 -- 172.21.15.2:0/2917154275 shutdown_connections 2023-09-13T04:41:57.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.633+0000 7f87037edb80 1 --2- 172.21.15.2:0/2917154275 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55daf7b5c680 0x55daf7b5ea70 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.633+0000 7f87037edb80 1 --2- 172.21.15.2:0/2917154275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55daf7b48220 0x55daf7b4ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.633+0000 7f87037edb80 1 --2- 172.21.15.2:0/2917154275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b43210 0x55daf7b4f1f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.633+0000 7f87037edb80 1 --2- 172.21.15.2:0/2917154275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daf7b47900 0x55daf7b4c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.633+0000 7f87037edb80 1 -- 172.21.15.2:0/2917154275 >> 172.21.15.2:0/2917154275 conn(0x55daf7a98c80 msgr2=0x55daf7b394b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:57.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.633+0000 7f86d17fa700 0 client.25984 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:57.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.633+0000 7f87037edb80 1 -- 172.21.15.2:0/2917154275 shutdown_connections 2023-09-13T04:41:57.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.633+0000 7f87037edb80 1 -- 172.21.15.2:0/2917154275 wait complete. 2023-09-13T04:41:57.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.633+0000 7f87037edb80 20 client.25984 trim_cache size 0 max 16384 2023-09-13T04:41:57.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.647+0000 7f87037edb80 1 Processor -- start 2023-09-13T04:41:57.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.648+0000 7f87037edb80 1 -- start start 2023-09-13T04:41:57.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.648+0000 7f87037edb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daf7b43210 0x55daf7b4ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.648+0000 7f87037edb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55daf7b47900 0x55daf7b53c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.648+0000 7f87037edb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b48220 0x55daf7b4ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.648+0000 7f87037edb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf7b4f9f0 con 0x55daf7b47900 2023-09-13T04:41:57.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.648+0000 7f87037edb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf7b318e0 con 0x55daf7b43210 2023-09-13T04:41:57.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.648+0000 7f87037edb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf7b36fa0 con 0x55daf7b48220 2023-09-13T04:41:57.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.649+0000 7f86edb9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daf7b43210 0x55daf7b4ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.649+0000 7f86edb9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daf7b43210 0x55daf7b4ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:47084/0 (socket says 172.21.15.2:47084) 2023-09-13T04:41:57.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.649+0000 7f86edb9b700 1 -- 172.21.15.2:0/3360288596 learned_addr learned my addr 172.21.15.2:0/3360288596 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:57.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.649+0000 7f86ed39a700 1 --2- 172.21.15.2:0/3360288596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55daf7b47900 0x55daf7b53c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.649+0000 7f86ee39c700 1 --2- 172.21.15.2:0/3360288596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b48220 0x55daf7b4ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.649+0000 7f86edb9b700 1 -- 172.21.15.2:0/3360288596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b48220 msgr2=0x55daf7b4ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:57.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.649+0000 7f86edb9b700 1 --2- 172.21.15.2:0/3360288596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b48220 0x55daf7b4ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.649+0000 7f86edb9b700 1 -- 172.21.15.2:0/3360288596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55daf7b47900 msgr2=0x55daf7b53c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:57.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.649+0000 7f86edb9b700 1 --2- 172.21.15.2:0/3360288596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55daf7b47900 0x55daf7b53c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.649+0000 7f86edb9b700 1 -- 172.21.15.2:0/3360288596 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daf78b8de0 con 0x55daf7b43210 2023-09-13T04:41:57.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.649+0000 7f86edb9b700 1 --2- 172.21.15.2:0/3360288596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daf7b43210 0x55daf7b4ca30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f86e4000c00 tx=0x7f86e4005910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=959dd1c781fa1f2e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:57.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.650+0000 7f86dcbff700 1 -- 172.21.15.2:0/3360288596 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4005bc0 con 0x55daf7b43210 2023-09-13T04:41:57.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.650+0000 7f86dcbff700 1 -- 172.21.15.2:0/3360288596 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86e40054f0 con 0x55daf7b43210 2023-09-13T04:41:57.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.650+0000 7f86dcbff700 1 -- 172.21.15.2:0/3360288596 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4004a10 con 0x55daf7b43210 2023-09-13T04:41:57.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.650+0000 7f87037edb80 1 -- 172.21.15.2:0/3360288596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daf7b43210 msgr2=0x55daf7b4ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:57.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.650+0000 7f87037edb80 1 --2- 172.21.15.2:0/3360288596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daf7b43210 0x55daf7b4ca30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f86e4000c00 tx=0x7f86e4005910 comp rx=0 tx=0).stop 2023-09-13T04:41:57.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.650+0000 7f87037edb80 1 -- 172.21.15.2:0/3360288596 shutdown_connections 2023-09-13T04:41:57.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.650+0000 7f87037edb80 1 --2- 172.21.15.2:0/3360288596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b48220 0x55daf7b4ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.650+0000 7f87037edb80 1 --2- 172.21.15.2:0/3360288596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daf7b43210 0x55daf7b4ca30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.650+0000 7f87037edb80 1 --2- 172.21.15.2:0/3360288596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55daf7b47900 0x55daf7b53c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.650+0000 7f87037edb80 1 -- 172.21.15.2:0/3360288596 >> 172.21.15.2:0/3360288596 conn(0x55daf7a98c80 msgr2=0x55daf7b583c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:57.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.651+0000 7f87037edb80 1 -- 172.21.15.2:0/3360288596 shutdown_connections 2023-09-13T04:41:57.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.651+0000 7f87037edb80 1 -- 172.21.15.2:0/3360288596 wait complete. 2023-09-13T04:41:57.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.651+0000 7f87037edb80 1 -- start start 2023-09-13T04:41:57.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.651+0000 7f87037edb80 1 Processor -- start 2023-09-13T04:41:57.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.652+0000 7f87037edb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b43210 0x55daf7ba8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.652+0000 7f87037edb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daf7b47900 0x55daf7ba9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.652+0000 7f87037edb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55daf7b48220 0x55daf7b52ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.652+0000 7f87037edb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf7b534a0 con 0x55daf7b48220 2023-09-13T04:41:57.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.652+0000 7f87037edb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf7b53610 con 0x55daf7b47900 2023-09-13T04:41:57.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.652+0000 7f87037edb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf7b537d0 con 0x55daf7b43210 2023-09-13T04:41:57.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.652+0000 7f87037edb80 10 client.0 _ll_register_callbacks cb 0x55daf7a28680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:57.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.652+0000 7f86ed39a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daf7b47900 0x55daf7ba9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.652+0000 7f86ed39a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daf7b47900 0x55daf7ba9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:47094/0 (socket says 172.21.15.2:47094) 2023-09-13T04:41:57.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.652+0000 7f86ed39a700 1 -- 172.21.15.2:0/4111993123 learned_addr learned my addr 172.21.15.2:0/4111993123 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:57.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.652+0000 7f86ee39c700 1 --2- 172.21.15.2:0/4111993123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55daf7b48220 0x55daf7b52ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.652+0000 7f86edb9b700 1 --2- 172.21.15.2:0/4111993123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b43210 0x55daf7ba8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.652+0000 7f86ed39a700 1 -- 172.21.15.2:0/4111993123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b43210 msgr2=0x55daf7ba8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:57.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.652+0000 7f86ed39a700 1 --2- 172.21.15.2:0/4111993123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b43210 0x55daf7ba8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.652+0000 7f86ed39a700 1 -- 172.21.15.2:0/4111993123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55daf7b48220 msgr2=0x55daf7b52ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:57.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.652+0000 7f86ed39a700 1 --2- 172.21.15.2:0/4111993123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55daf7b48220 0x55daf7b52ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.652+0000 7f86ed39a700 1 -- 172.21.15.2:0/4111993123 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55daf7aa47a0 con 0x55daf7b47900 2023-09-13T04:41:57.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.652+0000 7f86edb9b700 1 --2- 172.21.15.2:0/4111993123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b43210 0x55daf7ba8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:41:57.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.653+0000 7f86ed39a700 1 --2- 172.21.15.2:0/4111993123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daf7b47900 0x55daf7ba9300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f86d8010c10 tx=0x7f86d8010c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2ff10ea23665c9e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:57.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.653+0000 7f86d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:41:57.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.653+0000 7f86d17fa700 1 -- 172.21.15.2:0/4111993123 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8098f60 con 0x55daf7b47900 2023-09-13T04:41:57.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.653+0000 7f86d17fa700 1 -- 172.21.15.2:0/4111993123 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f86d8099d00 con 0x55daf7b47900 2023-09-13T04:41:57.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.653+0000 7f87037edb80 1 -- 172.21.15.2:0/4111993123 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86e401e040 con 0x55daf7b47900 2023-09-13T04:41:57.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.653+0000 7f86d17fa700 1 -- 172.21.15.2:0/4111993123 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86d8099600 con 0x55daf7b47900 2023-09-13T04:41:57.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.653+0000 7f87037edb80 10 client.25973 fetch_fsmap learned FSMap version 24 2023-09-13T04:41:57.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.653+0000 7f87037edb80 1 -- 172.21.15.2:0/4111993123 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55daf78a1c60 con 0x55daf7b47900 2023-09-13T04:41:57.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.653+0000 7f86d17fa700 1 -- 172.21.15.2:0/4111993123 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86e401e040 con 0x55daf7b47900 2023-09-13T04:41:57.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.654+0000 7f86d17fa700 1 -- 172.21.15.2:0/4111993123 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 24) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f86d80cc040 con 0x55daf7b47900 2023-09-13T04:41:57.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.654+0000 7f87037edb80 10 client.25973 fetch_fsmap finished waiting for FSMap version 24 2023-09-13T04:41:57.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.654+0000 7f87037edb80 10 client.25973 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:41:57.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.654+0000 7f87037edb80 20 client.25973 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:57.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.654+0000 7f87037edb80 1 --2- 172.21.15.2:0/4111993123 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55daf7bb0c20 0x55daf7bb3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:57.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.654+0000 7f87037edb80 4 client.25973 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-09-13T04:41:57.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.654+0000 7f87037edb80 1 -- 172.21.15.2:0/4111993123 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55daf7a46050 con 0x55daf7bb0c20 2023-09-13T04:41:57.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.654+0000 7f86edb9b700 1 --2- 172.21.15.2:0/4111993123 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55daf7bb0c20 0x55daf7bb3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:57.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.655+0000 7f86edb9b700 1 --2- 172.21.15.2:0/4111993123 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55daf7bb0c20 0x55daf7bb3040 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ead9a72d438316c server_cookie=f3d77cf0413cfb47 in_seq=0 out_seq=0 2023-09-13T04:41:57.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.655+0000 7f86d17fa700 10 client.25973 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:57.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.655+0000 7f86d17fa700 1 -- 172.21.15.2:0/4111993123 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55daf7a46050 con 0x55daf7bb0c20 2023-09-13T04:41:57.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.655+0000 7f86d17fa700 10 client.25973 handle_command_reply: tid=0 2023-09-13T04:41:57.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.655+0000 7f87037edb80 1 client.25973 shutdown 2023-09-13T04:41:57.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.655+0000 7f87037edb80 1 -- 172.21.15.2:0/4111993123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daf7b47900 msgr2=0x55daf7ba9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:57.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.655+0000 7f87037edb80 1 --2- 172.21.15.2:0/4111993123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daf7b47900 0x55daf7ba9300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f86d8010c10 tx=0x7f86d8010c60 comp rx=0 tx=0).stop 2023-09-13T04:41:57.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.655+0000 7f87037edb80 1 -- 172.21.15.2:0/4111993123 shutdown_connections 2023-09-13T04:41:57.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.655+0000 7f87037edb80 1 --2- 172.21.15.2:0/4111993123 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55daf7bb0c20 0x55daf7bb3040 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.655+0000 7f87037edb80 1 --2- 172.21.15.2:0/4111993123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55daf7b48220 0x55daf7b52ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.655+0000 7f87037edb80 1 --2- 172.21.15.2:0/4111993123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf7b43210 0x55daf7ba8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.655+0000 7f87037edb80 1 --2- 172.21.15.2:0/4111993123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daf7b47900 0x55daf7ba9300 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.655+0000 7f87037edb80 1 -- 172.21.15.2:0/4111993123 >> 172.21.15.2:0/4111993123 conn(0x55daf7a98c80 msgr2=0x55daf7b435f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:57.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.655+0000 7f86d17fa700 0 client.25973 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:57.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.655+0000 7f87037edb80 1 -- 172.21.15.2:0/4111993123 shutdown_connections 2023-09-13T04:41:57.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.656+0000 7f87037edb80 1 -- 172.21.15.2:0/4111993123 wait complete. 2023-09-13T04:41:57.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.656+0000 7f87037edb80 20 client.25973 trim_cache size 0 max 16384 2023-09-13T04:41:57.673 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-09-13T04:41:57.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.656+0000 7f86eeb9d700 1 -- 172.21.15.2:0/1803361874 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f86d40a0970 msgr2=0x7f86d40a2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:57.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.656+0000 7f86eeb9d700 1 --2- 172.21.15.2:0/1803361874 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f86d40a0970 0x7f86d40a2e30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55daf7b36f70 tx=0x7f86e000b040 comp rx=0 tx=0).stop 2023-09-13T04:41:57.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.656+0000 7f86eeb9d700 1 -- 172.21.15.2:0/1803361874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86e8101470 msgr2=0x7f86e80734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:57.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.656+0000 7f86eeb9d700 1 --2- 172.21.15.2:0/1803361874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86e8101470 0x7f86e80734f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f86d800e9f0 tx=0x7f86d800ed00 comp rx=0 tx=0).stop 2023-09-13T04:41:57.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.657+0000 7f86eeb9d700 1 -- 172.21.15.2:0/1803361874 shutdown_connections 2023-09-13T04:41:57.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.657+0000 7f86eeb9d700 1 --2- 172.21.15.2:0/1803361874 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f86d40a0970 0x7f86d40a2e30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.657+0000 7f86eeb9d700 1 --2- 172.21.15.2:0/1803361874 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86e8106ac0 0x7f86e81a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.657+0000 7f86eeb9d700 1 --2- 172.21.15.2:0/1803361874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86e8100ab0 0x7f86e8072fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.657+0000 7f86eeb9d700 1 --2- 172.21.15.2:0/1803361874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86e8101470 0x7f86e80734f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:57.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.657+0000 7f86eeb9d700 1 -- 172.21.15.2:0/1803361874 >> 172.21.15.2:0/1803361874 conn(0x7f86e80fc560 msgr2=0x7f86e80fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:57.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.657+0000 7f86eeb9d700 1 -- 172.21.15.2:0/1803361874 shutdown_connections 2023-09-13T04:41:57.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:57.657+0000 7f86eeb9d700 1 -- 172.21.15.2:0/1803361874 wait complete. 2023-09-13T04:41:58.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:57 smithi002 ceph-mon[97651]: pgmap v48: 129 pgs: 129 active+clean; 12 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 811 KiB/s wr, 114 op/s 2023-09-13T04:41:58.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:57 smithi156 ceph-mon[133675]: pgmap v48: 129 pgs: 129 active+clean; 12 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 811 KiB/s wr, 114 op/s 2023-09-13T04:41:58.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:57 smithi149 ceph-mon[133315]: pgmap v48: 129 pgs: 129 active+clean; 12 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 811 KiB/s wr, 114 op/s 2023-09-13T04:41:58.678 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-13T04:41:58.678 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-13T04:41:58.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.915+0000 7ff1da59d700 1 -- 172.21.15.2:0/1056322351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1d40ff510 msgr2=0x7ff1d410d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:58.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.915+0000 7ff1da59d700 1 --2- 172.21.15.2:0/1056322351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1d40ff510 0x7ff1d410d810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00b0a0 tx=0x7ff1cc00b3b0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.915+0000 7ff1da59d700 1 -- 172.21.15.2:0/1056322351 shutdown_connections 2023-09-13T04:41:58.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.915+0000 7ff1da59d700 1 --2- 172.21.15.2:0/1056322351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1d40ff510 0x7ff1d410d810 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.915+0000 7ff1da59d700 1 --2- 172.21.15.2:0/1056322351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1d4105ba0 0x7ff1d4105f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.915+0000 7ff1da59d700 1 --2- 172.21.15.2:0/1056322351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1d40feb50 0x7ff1d40fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.915+0000 7ff1da59d700 1 -- 172.21.15.2:0/1056322351 >> 172.21.15.2:0/1056322351 conn(0x7ff1d4076b10 msgr2=0x7ff1d40fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:58.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.915+0000 7ff1da59d700 1 -- 172.21.15.2:0/1056322351 shutdown_connections 2023-09-13T04:41:58.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.915+0000 7ff1da59d700 1 -- 172.21.15.2:0/1056322351 wait complete. 2023-09-13T04:41:58.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.916+0000 7ff1da59d700 1 Processor -- start 2023-09-13T04:41:58.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.916+0000 7ff1da59d700 1 -- start start 2023-09-13T04:41:58.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.917+0000 7ff1da59d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1d40feb50 0x7ff1d419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:58.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.917+0000 7ff1da59d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1d40ff510 0x7ff1d419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:58.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.917+0000 7ff1da59d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1d4105ba0 0x7ff1d4196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:58.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.917+0000 7ff1da59d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d4196ff0 con 0x7ff1d40ff510 2023-09-13T04:41:58.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.917+0000 7ff1da59d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d4197160 con 0x7ff1d4105ba0 2023-09-13T04:41:58.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.917+0000 7ff1da59d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d4197320 con 0x7ff1d40feb50 2023-09-13T04:41:58.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.917+0000 7ff1d959b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1d40feb50 0x7ff1d419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:58.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.917+0000 7ff1d959b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1d40feb50 0x7ff1d419c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:43232/0 (socket says 172.21.15.2:43232) 2023-09-13T04:41:58.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.917+0000 7ff1d959b700 1 -- 172.21.15.2:0/926264186 learned_addr learned my addr 172.21.15.2:0/926264186 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:58.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.917+0000 7ff1d8d9a700 1 --2- 172.21.15.2:0/926264186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1d40ff510 0x7ff1d419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:58.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.917+0000 7ff1d9d9c700 1 --2- 172.21.15.2:0/926264186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1d4105ba0 0x7ff1d4196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:58.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.918+0000 7ff1d9d9c700 1 -- 172.21.15.2:0/926264186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1d40feb50 msgr2=0x7ff1d419c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:58.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.918+0000 7ff1d9d9c700 1 --2- 172.21.15.2:0/926264186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1d40feb50 0x7ff1d419c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.918+0000 7ff1d9d9c700 1 -- 172.21.15.2:0/926264186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1d40ff510 msgr2=0x7ff1d419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:58.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.918+0000 7ff1d9d9c700 1 --2- 172.21.15.2:0/926264186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1d40ff510 0x7ff1d419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.918+0000 7ff1d9d9c700 1 -- 172.21.15.2:0/926264186 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1cc009d00 con 0x7ff1d4105ba0 2023-09-13T04:41:58.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.918+0000 7ff1d8d9a700 1 --2- 172.21.15.2:0/926264186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1d40ff510 0x7ff1d419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.918+0000 7ff1d9d9c700 1 --2- 172.21.15.2:0/926264186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1d4105ba0 0x7ff1d4196980 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009fd0 tx=0x7ff1cc00bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.918+0000 7ff1ca7fc700 1 -- 172.21.15.2:0/926264186 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc00e040 con 0x7ff1d4105ba0 2023-09-13T04:41:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.918+0000 7ff1ca7fc700 1 -- 172.21.15.2:0/926264186 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1cc0094c0 con 0x7ff1d4105ba0 2023-09-13T04:41:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.918+0000 7ff1da59d700 1 -- 172.21.15.2:0/926264186 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d41975a0 con 0x7ff1d4105ba0 2023-09-13T04:41:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.919+0000 7ff1ca7fc700 1 -- 172.21.15.2:0/926264186 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc005040 con 0x7ff1d4105ba0 2023-09-13T04:41:58.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.919+0000 7ff1da59d700 1 -- 172.21.15.2:0/926264186 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d41a3900 con 0x7ff1d4105ba0 2023-09-13T04:41:58.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.922+0000 7ff1ca7fc700 1 -- 172.21.15.2:0/926264186 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 24) v1 ==== 135491+0+0 (secure 0 0 0) 0x7ff1cc01d470 con 0x7ff1d4105ba0 2023-09-13T04:41:58.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.923+0000 7ff1ca7fc700 1 --2- 172.21.15.2:0/926264186 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7ff1c00a0970 0x7ff1c00a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:58.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.924+0000 7ff1ca7fc700 1 -- 172.21.15.2:0/926264186 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7ff1cc0cdad0 con 0x7ff1d4105ba0 2023-09-13T04:41:58.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.924+0000 7ff1ef0c5b80 1 Processor -- start 2023-09-13T04:41:58.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.924+0000 7ff1d959b700 1 --2- 172.21.15.2:0/926264186 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7ff1c00a0970 0x7ff1c00a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:58.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.924+0000 7ff1ef0c5b80 1 -- start start 2023-09-13T04:41:58.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.925+0000 7ff1ef0c5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1e33730 0x557de1e33b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:58.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.925+0000 7ff1d959b700 1 --2- 172.21.15.2:0/926264186 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7ff1c00a0970 0x7ff1c00a2e30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000bd70 tx=0x7ff1d000b4a0 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:58.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.925+0000 7ff1ef0c5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e34050 0x557de1e37e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:58.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.925+0000 7ff1ef0c5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1e38380 0x557de1e387e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:58.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.925+0000 7ff1ef0c5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de1d8c800 con 0x557de1e33730 2023-09-13T04:41:58.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.925+0000 7ff1ef0c5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de1d8a120 con 0x557de1e38380 2023-09-13T04:41:58.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.925+0000 7ff1ef0c5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de1d85a10 con 0x557de1e34050 2023-09-13T04:41:58.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.925+0000 7ff1d8d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e34050 0x557de1e37e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:58.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.925+0000 7ff1d9d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1e38380 0x557de1e387e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:58.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.925+0000 7ff1d959b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1e33730 0x557de1e33b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:58.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.925+0000 7ff1d8d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e34050 0x557de1e37e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:43236/0 (socket says 172.21.15.2:43236) 2023-09-13T04:41:58.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.925+0000 7ff1d8d9a700 1 -- 172.21.15.2:0/3554273482 learned_addr learned my addr 172.21.15.2:0/3554273482 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:58.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.925+0000 7ff1d9d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1e38380 0x557de1e387e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:47120/0 (socket says 172.21.15.2:47120) 2023-09-13T04:41:58.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.925+0000 7ff1d9d9c700 1 -- 172.21.15.2:0/3554273482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e34050 msgr2=0x557de1e37e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:58.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.925+0000 7ff1d9d9c700 1 --2- 172.21.15.2:0/3554273482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e34050 0x557de1e37e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.925+0000 7ff1d9d9c700 1 -- 172.21.15.2:0/3554273482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1e33730 msgr2=0x557de1e33b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:58.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.925+0000 7ff1d9d9c700 1 --2- 172.21.15.2:0/3554273482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1e33730 0x557de1e33b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.926+0000 7ff1d9d9c700 1 -- 172.21.15.2:0/3554273482 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557de1c056b0 con 0x557de1e38380 2023-09-13T04:41:58.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.926+0000 7ff1d9d9c700 1 --2- 172.21.15.2:0/3554273482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1e38380 0x557de1e387e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0cad30 tx=0x7ff1cc004e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faee55c4fecc6d79 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:58.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.926+0000 7ff1bf7fe700 1 -- 172.21.15.2:0/3554273482 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0ba0b0 con 0x557de1e38380 2023-09-13T04:41:58.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.926+0000 7ff1bf7fe700 1 -- 172.21.15.2:0/3554273482 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1cc09b150 con 0x557de1e38380 2023-09-13T04:41:58.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.926+0000 7ff1bf7fe700 1 -- 172.21.15.2:0/3554273482 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc01d980 con 0x557de1e38380 2023-09-13T04:41:58.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.927+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/3554273482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1e38380 msgr2=0x557de1e387e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:58.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.927+0000 7ff1ef0c5b80 1 --2- 172.21.15.2:0/3554273482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1e38380 0x557de1e387e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0cad30 tx=0x7ff1cc004e60 comp rx=0 tx=0).stop 2023-09-13T04:41:58.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.927+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/3554273482 shutdown_connections 2023-09-13T04:41:58.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.927+0000 7ff1ef0c5b80 1 --2- 172.21.15.2:0/3554273482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1e38380 0x557de1e387e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.927+0000 7ff1ef0c5b80 1 --2- 172.21.15.2:0/3554273482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1e33730 0x557de1e33b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.927+0000 7ff1ef0c5b80 1 --2- 172.21.15.2:0/3554273482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e34050 0x557de1e37e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.927+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/3554273482 >> 172.21.15.2:0/3554273482 conn(0x557de1d91d00 msgr2=0x557de1e2e380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:58.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.927+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/3554273482 shutdown_connections 2023-09-13T04:41:58.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.927+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/3554273482 wait complete. 2023-09-13T04:41:58.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.928+0000 7ff1ef0c5b80 1 -- start start 2023-09-13T04:41:58.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.928+0000 7ff1ef0c5b80 1 Processor -- start 2023-09-13T04:41:58.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.928+0000 7ff1ef0c5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1e33730 0x557de1e3ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:58.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.928+0000 7ff1ef0c5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1e34050 0x557de1e3d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:58.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.928+0000 7ff1ef0c5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e38380 0x557de1e3d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:58.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.929+0000 7ff1ef0c5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de1d23c30 con 0x557de1e33730 2023-09-13T04:41:58.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.929+0000 7ff1ef0c5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de1d23930 con 0x557de1e34050 2023-09-13T04:41:58.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.929+0000 7ff1ef0c5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de1c079a0 con 0x557de1e38380 2023-09-13T04:41:58.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.929+0000 7ff1ef0c5b80 10 client.0 _ll_register_callbacks cb 0x557de1d21700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:58.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.929+0000 7ff1d959b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1e33730 0x557de1e3ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:58.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.929+0000 7ff1d9d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e38380 0x557de1e3d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:58.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.929+0000 7ff1d959b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1e33730 0x557de1e3ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50668/0 (socket says 172.21.15.2:50668) 2023-09-13T04:41:58.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.929+0000 7ff1d959b700 1 -- 172.21.15.2:0/1054479025 learned_addr learned my addr 172.21.15.2:0/1054479025 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:58.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.929+0000 7ff1d8d9a700 1 --2- 172.21.15.2:0/1054479025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1e34050 0x557de1e3d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:58.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.929+0000 7ff1d959b700 1 -- 172.21.15.2:0/1054479025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e38380 msgr2=0x557de1e3d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:58.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.930+0000 7ff1d959b700 1 --2- 172.21.15.2:0/1054479025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e38380 0x557de1e3d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.930+0000 7ff1d959b700 1 -- 172.21.15.2:0/1054479025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1e34050 msgr2=0x557de1e3d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:58.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.930+0000 7ff1d959b700 1 --2- 172.21.15.2:0/1054479025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1e34050 0x557de1e3d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.930+0000 7ff1d959b700 1 -- 172.21.15.2:0/1054479025 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d41a3900 con 0x557de1e33730 2023-09-13T04:41:58.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.930+0000 7ff1d9d9c700 1 --2- 172.21.15.2:0/1054479025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e38380 0x557de1e3d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:58.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.930+0000 7ff1d8d9a700 1 --2- 172.21.15.2:0/1054479025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1e34050 0x557de1e3d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:58.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.930+0000 7ff1d959b700 1 --2- 172.21.15.2:0/1054479025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1e33730 0x557de1e3ce90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00117a0 tx=0x7ff1d00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eeaa699431578b5 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:58.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.930+0000 7ff1bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:41:58.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.930+0000 7ff1bcff9700 1 -- 172.21.15.2:0/1054479025 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0004a30 con 0x557de1e33730 2023-09-13T04:41:58.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.930+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/1054479025 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557de1e266f0 con 0x557de1e33730 2023-09-13T04:41:58.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.930+0000 7ff1bcff9700 1 -- 172.21.15.2:0/1054479025 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7ff1d0017070 con 0x557de1e33730 2023-09-13T04:41:58.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.931+0000 7ff1bcff9700 1 -- 172.21.15.2:0/1054479025 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1d0011e70 con 0x557de1e33730 2023-09-13T04:41:58.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.931+0000 7ff1ef0c5b80 10 client.15966 fetch_fsmap learned FSMap version 25 2023-09-13T04:41:58.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.931+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/1054479025 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557de1d9d820 con 0x557de1e33730 2023-09-13T04:41:58.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.931+0000 7ff1bcff9700 1 -- 172.21.15.2:0/1054479025 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557de1e266f0 con 0x557de1e33730 2023-09-13T04:41:58.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.931+0000 7ff1bcff9700 1 -- 172.21.15.2:0/1054479025 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7ff1d001f630 con 0x557de1e33730 2023-09-13T04:41:58.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.931+0000 7ff1ef0c5b80 10 client.15966 fetch_fsmap finished waiting for FSMap version 25 2023-09-13T04:41:58.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.931+0000 7ff1ef0c5b80 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:41:58.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.931+0000 7ff1ef0c5b80 20 client.15966 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:58.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.931+0000 7ff1ef0c5b80 1 --2- 172.21.15.2:0/1054479025 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x557de1e4d330 0x557de1e4f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:58.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.931+0000 7ff1ef0c5b80 4 client.15966 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:41:58.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.931+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/1054479025 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557de1d7e9f0 con 0x557de1e4d330 2023-09-13T04:41:58.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.932+0000 7ff1d8d9a700 1 --2- 172.21.15.2:0/1054479025 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x557de1e4d330 0x557de1e4f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:58.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.932+0000 7ff1d8d9a700 1 --2- 172.21.15.2:0/1054479025 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x557de1e4d330 0x557de1e4f750 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fca28f86b673674d server_cookie=45b7b9992b163f56 in_seq=0 out_seq=0 2023-09-13T04:41:58.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.932+0000 7ff1bcff9700 10 client.15966 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:58.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.933+0000 7ff1bcff9700 1 -- 172.21.15.2:0/1054479025 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557de1d7e9f0 con 0x557de1e4d330 2023-09-13T04:41:58.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.933+0000 7ff1bcff9700 10 client.15966 handle_command_reply: tid=0 2023-09-13T04:41:58.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.933+0000 7ff1ef0c5b80 1 client.15966 shutdown 2023-09-13T04:41:58.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.934+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/1054479025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1e33730 msgr2=0x557de1e3ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:58.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.934+0000 7ff1ef0c5b80 1 --2- 172.21.15.2:0/1054479025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1e33730 0x557de1e3ce90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00117a0 tx=0x7ff1d00047c0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.934+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/1054479025 shutdown_connections 2023-09-13T04:41:58.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.934+0000 7ff1ef0c5b80 1 --2- 172.21.15.2:0/1054479025 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x557de1e4d330 0x557de1e4f750 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.934+0000 7ff1ef0c5b80 1 --2- 172.21.15.2:0/1054479025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e38380 0x557de1e3d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.934+0000 7ff1ef0c5b80 1 --2- 172.21.15.2:0/1054479025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1e33730 0x557de1e3ce90 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.934+0000 7ff1ef0c5b80 1 --2- 172.21.15.2:0/1054479025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1e34050 0x557de1e3d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.934+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/1054479025 >> 172.21.15.2:0/1054479025 conn(0x557de1d91d00 msgr2=0x557de1e32530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:58.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.934+0000 7ff1bcff9700 0 client.15966 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:58.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.935+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/1054479025 shutdown_connections 2023-09-13T04:41:58.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.935+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/1054479025 wait complete. 2023-09-13T04:41:58.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.935+0000 7ff1ef0c5b80 20 client.15966 trim_cache size 0 max 16384 2023-09-13T04:41:58.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.960+0000 7ff1ef0c5b80 1 Processor -- start 2023-09-13T04:41:58.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.960+0000 7ff1ef0c5b80 1 -- start start 2023-09-13T04:41:58.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.960+0000 7ff1ef0c5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1d91d00 0x557de1e44290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:58.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.960+0000 7ff1ef0c5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e33730 0x557de1e2dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:58.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.960+0000 7ff1ef0c5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1e34050 0x557de1e2e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:58.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.960+0000 7ff1ef0c5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de1e26720 con 0x557de1d91d00 2023-09-13T04:41:58.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.960+0000 7ff1ef0c5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de1e333d0 con 0x557de1e34050 2023-09-13T04:41:58.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.960+0000 7ff1ef0c5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de1e2f070 con 0x557de1e33730 2023-09-13T04:41:58.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.961+0000 7ff1d9d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1e34050 0x557de1e2e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:58.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.961+0000 7ff1d8d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e33730 0x557de1e2dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:58.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.961+0000 7ff1d959b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1d91d00 0x557de1e44290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:58.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.961+0000 7ff1d8d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e33730 0x557de1e2dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:43258/0 (socket says 172.21.15.2:43258) 2023-09-13T04:41:58.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.961+0000 7ff1d9d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1e34050 0x557de1e2e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:47144/0 (socket says 172.21.15.2:47144) 2023-09-13T04:41:58.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.961+0000 7ff1d8d9a700 1 -- 172.21.15.2:0/1835142869 learned_addr learned my addr 172.21.15.2:0/1835142869 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:58.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.961+0000 7ff1d9d9c700 1 -- 172.21.15.2:0/1835142869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e33730 msgr2=0x557de1e2dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:58.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.961+0000 7ff1d9d9c700 1 --2- 172.21.15.2:0/1835142869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e33730 0x557de1e2dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.961+0000 7ff1d9d9c700 1 -- 172.21.15.2:0/1835142869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1d91d00 msgr2=0x557de1e44290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:58.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.961+0000 7ff1d9d9c700 1 --2- 172.21.15.2:0/1835142869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1d91d00 0x557de1e44290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.961+0000 7ff1d9d9c700 1 -- 172.21.15.2:0/1835142869 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557de1e54340 con 0x557de1e34050 2023-09-13T04:41:58.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.961+0000 7ff1d8d9a700 1 --2- 172.21.15.2:0/1835142869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e33730 0x557de1e2dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:58.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.961+0000 7ff1d959b700 1 --2- 172.21.15.2:0/1835142869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1d91d00 0x557de1e44290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:41:58.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.961+0000 7ff1d9d9c700 1 --2- 172.21.15.2:0/1835142869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1e34050 0x557de1e2e420 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00b780 tx=0x7ff1cc01db10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0737c4ad9ffb4fd server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:58.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.962+0000 7ff1bffff700 1 -- 172.21.15.2:0/1835142869 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0ba0b0 con 0x557de1e34050 2023-09-13T04:41:58.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.962+0000 7ff1bffff700 1 -- 172.21.15.2:0/1835142869 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1cc09aa50 con 0x557de1e34050 2023-09-13T04:41:58.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.962+0000 7ff1bffff700 1 -- 172.21.15.2:0/1835142869 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc09acb0 con 0x557de1e34050 2023-09-13T04:41:58.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.963+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/1835142869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1e34050 msgr2=0x557de1e2e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:58.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.963+0000 7ff1ef0c5b80 1 --2- 172.21.15.2:0/1835142869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1e34050 0x557de1e2e420 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00b780 tx=0x7ff1cc01db10 comp rx=0 tx=0).stop 2023-09-13T04:41:58.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.963+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/1835142869 shutdown_connections 2023-09-13T04:41:58.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.963+0000 7ff1ef0c5b80 1 --2- 172.21.15.2:0/1835142869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1e34050 0x557de1e2e420 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.963+0000 7ff1ef0c5b80 1 --2- 172.21.15.2:0/1835142869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1d91d00 0x557de1e44290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.963+0000 7ff1ef0c5b80 1 --2- 172.21.15.2:0/1835142869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e33730 0x557de1e2dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.963+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/1835142869 >> 172.21.15.2:0/1835142869 conn(0x557de1e38ac0 msgr2=0x557de1e36460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:58.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.963+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/1835142869 shutdown_connections 2023-09-13T04:41:58.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.963+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/1835142869 wait complete. 2023-09-13T04:41:58.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.963+0000 7ff1ef0c5b80 1 -- start start 2023-09-13T04:41:58.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.964+0000 7ff1ef0c5b80 1 Processor -- start 2023-09-13T04:41:58.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.964+0000 7ff1ef0c5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1d91d00 0x557de1e43ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:58.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.964+0000 7ff1ef0c5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1e33730 0x557de1e44410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:58.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.964+0000 7ff1ef0c5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e34050 0x557de1e42470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:58.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.964+0000 7ff1ef0c5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de1e42b80 con 0x557de1e33730 2023-09-13T04:41:58.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.964+0000 7ff1ef0c5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de1e42cf0 con 0x557de1d91d00 2023-09-13T04:41:58.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.964+0000 7ff1ef0c5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de1e42eb0 con 0x557de1e34050 2023-09-13T04:41:58.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.964+0000 7ff1ef0c5b80 10 client.0 _ll_register_callbacks cb 0x557de1d21700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:58.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.964+0000 7ff1d8d9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1e33730 0x557de1e44410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:58.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.964+0000 7ff1d8d9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1e33730 0x557de1e44410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50674/0 (socket says 172.21.15.2:50674) 2023-09-13T04:41:58.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.965+0000 7ff1d8d9a700 1 -- 172.21.15.2:0/533884700 learned_addr learned my addr 172.21.15.2:0/533884700 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:58.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.965+0000 7ff1d959b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1d91d00 0x557de1e43ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:58.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.965+0000 7ff1d9d9c700 1 --2- 172.21.15.2:0/533884700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e34050 0x557de1e42470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:58.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.965+0000 7ff1d8d9a700 1 -- 172.21.15.2:0/533884700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e34050 msgr2=0x557de1e42470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:58.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.965+0000 7ff1d8d9a700 1 --2- 172.21.15.2:0/533884700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e34050 0x557de1e42470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.965+0000 7ff1d8d9a700 1 -- 172.21.15.2:0/533884700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1d91d00 msgr2=0x557de1e43ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:58.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.965+0000 7ff1d8d9a700 1 --2- 172.21.15.2:0/533884700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1d91d00 0x557de1e43ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.965+0000 7ff1d8d9a700 1 -- 172.21.15.2:0/533884700 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c4013ae0 con 0x557de1e33730 2023-09-13T04:41:58.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.965+0000 7ff1d9d9c700 1 --2- 172.21.15.2:0/533884700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e34050 0x557de1e42470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:41:58.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.965+0000 7ff1d8d9a700 1 --2- 172.21.15.2:0/533884700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1e33730 0x557de1e44410 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4015010 tx=0x7ff1c40240c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e10f97120c3493 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:58.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.966+0000 7ff1bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:41:58.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.966+0000 7ff1bcff9700 1 -- 172.21.15.2:0/533884700 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4018050 con 0x557de1e33730 2023-09-13T04:41:58.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.966+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/533884700 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1cc09b150 con 0x557de1e33730 2023-09-13T04:41:58.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.966+0000 7ff1bcff9700 1 -- 172.21.15.2:0/533884700 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7ff1c4003680 con 0x557de1e33730 2023-09-13T04:41:58.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.966+0000 7ff1bcff9700 1 -- 172.21.15.2:0/533884700 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1c4003840 con 0x557de1e33730 2023-09-13T04:41:58.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.966+0000 7ff1ef0c5b80 10 client.15972 fetch_fsmap learned FSMap version 25 2023-09-13T04:41:58.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.966+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/533884700 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557de1c056b0 con 0x557de1e33730 2023-09-13T04:41:58.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.966+0000 7ff1bcff9700 1 -- 172.21.15.2:0/533884700 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1cc09b150 con 0x557de1e33730 2023-09-13T04:41:58.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.967+0000 7ff1bcff9700 1 -- 172.21.15.2:0/533884700 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7ff1c402da70 con 0x557de1e33730 2023-09-13T04:41:58.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.967+0000 7ff1ef0c5b80 10 client.15972 fetch_fsmap finished waiting for FSMap version 25 2023-09-13T04:41:58.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.967+0000 7ff1ef0c5b80 10 client.15972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:41:58.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.967+0000 7ff1ef0c5b80 20 client.15972 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:58.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.967+0000 7ff1ef0c5b80 1 --2- 172.21.15.2:0/533884700 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x557de1e38380 0x557de1ea4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:58.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.967+0000 7ff1ef0c5b80 4 client.15972 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-13T04:41:58.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.967+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/533884700 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557de1d3f0d0 con 0x557de1e38380 2023-09-13T04:41:58.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.967+0000 7ff1d959b700 1 --2- 172.21.15.2:0/533884700 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x557de1e38380 0x557de1ea4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:58.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.968+0000 7ff1bcff9700 10 client.15972 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:58.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.968+0000 7ff1d959b700 1 --2- 172.21.15.2:0/533884700 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x557de1e38380 0x557de1ea4620 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa2696d3ee58d101 server_cookie=8beb4e635b23ee54 in_seq=0 out_seq=0 2023-09-13T04:41:58.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.969+0000 7ff1bcff9700 1 -- 172.21.15.2:0/533884700 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557de1d3f0d0 con 0x557de1e38380 2023-09-13T04:41:58.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.969+0000 7ff1bcff9700 10 client.15972 handle_command_reply: tid=0 2023-09-13T04:41:58.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.969+0000 7ff1ef0c5b80 1 client.15972 shutdown 2023-09-13T04:41:58.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.970+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/533884700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1e33730 msgr2=0x557de1e44410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:58.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.970+0000 7ff1ef0c5b80 1 --2- 172.21.15.2:0/533884700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1e33730 0x557de1e44410 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4015010 tx=0x7ff1c40240c0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.970+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/533884700 shutdown_connections 2023-09-13T04:41:58.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.970+0000 7ff1ef0c5b80 1 --2- 172.21.15.2:0/533884700 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x557de1e38380 0x557de1ea4620 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.970+0000 7ff1ef0c5b80 1 --2- 172.21.15.2:0/533884700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557de1e34050 0x557de1e42470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.970+0000 7ff1ef0c5b80 1 --2- 172.21.15.2:0/533884700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557de1d91d00 0x557de1e43ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.970+0000 7ff1ef0c5b80 1 --2- 172.21.15.2:0/533884700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557de1e33730 0x557de1e44410 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.970+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/533884700 >> 172.21.15.2:0/533884700 conn(0x557de1e38ac0 msgr2=0x557de1e36460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:58.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.970+0000 7ff1bcff9700 0 client.15972 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:58.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.970+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/533884700 shutdown_connections 2023-09-13T04:41:58.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.970+0000 7ff1ef0c5b80 1 -- 172.21.15.2:0/533884700 wait complete. 2023-09-13T04:41:58.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.971+0000 7ff1ef0c5b80 20 client.15972 trim_cache size 0 max 16384 2023-09-13T04:41:58.982 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:41:58.982 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2023-09-13T04:41:58.982 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "d95eecf9-65e4-419c-9998-59370b2f6d5a", 2023-09-13T04:41:58.983 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2023-09-13T04:41:58.983 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:41:58.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.971+0000 7ff1da59d700 1 -- 172.21.15.2:0/926264186 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7ff1c00a0970 msgr2=0x7ff1c00a2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:58.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.971+0000 7ff1da59d700 1 --2- 172.21.15.2:0/926264186 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7ff1c00a0970 0x7ff1c00a2e30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000bd70 tx=0x7ff1d000b4a0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.971+0000 7ff1da59d700 1 -- 172.21.15.2:0/926264186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1d4105ba0 msgr2=0x7ff1d4196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:58.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.971+0000 7ff1da59d700 1 --2- 172.21.15.2:0/926264186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1d4105ba0 0x7ff1d4196980 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009fd0 tx=0x7ff1cc00bce0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.972+0000 7ff1da59d700 1 -- 172.21.15.2:0/926264186 shutdown_connections 2023-09-13T04:41:58.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.972+0000 7ff1da59d700 1 --2- 172.21.15.2:0/926264186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1d4105ba0 0x7ff1d4196980 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.972+0000 7ff1da59d700 1 --2- 172.21.15.2:0/926264186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1d40feb50 0x7ff1d419c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.972+0000 7ff1da59d700 1 --2- 172.21.15.2:0/926264186 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7ff1c00a0970 0x7ff1c00a2e30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.972+0000 7ff1da59d700 1 --2- 172.21.15.2:0/926264186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1d40ff510 0x7ff1d419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:58.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.972+0000 7ff1da59d700 1 -- 172.21.15.2:0/926264186 >> 172.21.15.2:0/926264186 conn(0x7ff1d4076b10 msgr2=0x7ff1d40fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:58.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.972+0000 7ff1da59d700 1 -- 172.21.15.2:0/926264186 shutdown_connections 2023-09-13T04:41:58.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:58.972+0000 7ff1da59d700 1 -- 172.21.15.2:0/926264186 wait complete. 2023-09-13T04:41:58.994 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T04:41:59.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:41:58 smithi002 ceph-mon[97651]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:41:59.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:41:58 smithi156 ceph-mon[133675]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:41:59.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.275+0000 7f740dbff700 1 -- 172.21.15.2:0/1005380247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7408073980 msgr2=0x7f740810cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:59.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.275+0000 7f740dbff700 1 --2- 172.21.15.2:0/1005380247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7408073980 0x7f740810cff0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0098f0 tx=0x7f73fc009c00 comp rx=0 tx=0).stop 2023-09-13T04:41:59.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.275+0000 7f740dbff700 1 -- 172.21.15.2:0/1005380247 shutdown_connections 2023-09-13T04:41:59.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.275+0000 7f740dbff700 1 --2- 172.21.15.2:0/1005380247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f740810d530 0x7f740810f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.275+0000 7f740dbff700 1 --2- 172.21.15.2:0/1005380247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7408073060 0x7f7408073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.275+0000 7f740dbff700 1 --2- 172.21.15.2:0/1005380247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7408073980 0x7f740810cff0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.275+0000 7f740dbff700 1 -- 172.21.15.2:0/1005380247 >> 172.21.15.2:0/1005380247 conn(0x7f74080fc760 msgr2=0x7f74080feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:59.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.275+0000 7f740dbff700 1 -- 172.21.15.2:0/1005380247 shutdown_connections 2023-09-13T04:41:59.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.275+0000 7f740dbff700 1 -- 172.21.15.2:0/1005380247 wait complete. 2023-09-13T04:41:59.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.276+0000 7f740dbff700 1 Processor -- start 2023-09-13T04:41:59.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.276+0000 7f740dbff700 1 -- start start 2023-09-13T04:41:59.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.276+0000 7f740dbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7408073060 0x7f7408199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:59.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.276+0000 7f740dbff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7408073980 0x7f7408199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:59.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.276+0000 7f740dbff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f740810d530 0x7f740819d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:59.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.277+0000 7f740dbff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740819e000 con 0x7f7408073980 2023-09-13T04:41:59.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.277+0000 7f740dbff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740819e170 con 0x7f740810d530 2023-09-13T04:41:59.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.277+0000 7f740dbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740819e330 con 0x7f7408073060 2023-09-13T04:41:59.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.277+0000 7f74077fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7408073060 0x7f7408199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:59.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.277+0000 7f74077fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7408073060 0x7f7408199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:43280/0 (socket says 172.21.15.2:43280) 2023-09-13T04:41:59.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.277+0000 7f74077fe700 1 -- 172.21.15.2:0/2043859939 learned_addr learned my addr 172.21.15.2:0/2043859939 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:59.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.277+0000 7f7406ffd700 1 --2- 172.21.15.2:0/2043859939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7408073980 0x7f7408199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:59.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.277+0000 7f7407fff700 1 --2- 172.21.15.2:0/2043859939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f740810d530 0x7f740819d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:59.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.277+0000 7f74077fe700 1 -- 172.21.15.2:0/2043859939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f740810d530 msgr2=0x7f740819d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:59.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.277+0000 7f74077fe700 1 --2- 172.21.15.2:0/2043859939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f740810d530 0x7f740819d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.277+0000 7f74077fe700 1 -- 172.21.15.2:0/2043859939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7408073980 msgr2=0x7f7408199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:59.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.277+0000 7f74077fe700 1 --2- 172.21.15.2:0/2043859939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7408073980 0x7f7408199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.277+0000 7f74077fe700 1 -- 172.21.15.2:0/2043859939 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc009520 con 0x7f7408073060 2023-09-13T04:41:59.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.277+0000 7f7407fff700 1 --2- 172.21.15.2:0/2043859939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f740810d530 0x7f740819d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:41:59.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.278+0000 7f74077fe700 1 --2- 172.21.15.2:0/2043859939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7408073060 0x7f7408199020 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f73f800e9c0 tx=0x7f73f800ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:59.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.278+0000 7f7404ff9700 1 -- 172.21.15.2:0/2043859939 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f800cbd0 con 0x7f7408073060 2023-09-13T04:41:59.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.278+0000 7f7404ff9700 1 -- 172.21.15.2:0/2043859939 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73f800cd30 con 0x7f7408073060 2023-09-13T04:41:59.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.278+0000 7f7404ff9700 1 -- 172.21.15.2:0/2043859939 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f8005580 con 0x7f7408073060 2023-09-13T04:41:59.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.278+0000 7f740dbff700 1 -- 172.21.15.2:0/2043859939 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740819e610 con 0x7f7408073060 2023-09-13T04:41:59.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.278+0000 7f740dbff700 1 -- 172.21.15.2:0/2043859939 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74081a5e50 con 0x7f7408073060 2023-09-13T04:41:59.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.282+0000 7f7404ff9700 1 -- 172.21.15.2:0/2043859939 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 24) v1 ==== 135491+0+0 (secure 0 0 0) 0x7f73f8004750 con 0x7f7408073060 2023-09-13T04:41:59.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.284+0000 7f7421521b80 1 Processor -- start 2023-09-13T04:41:59.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.285+0000 7f7421521b80 1 -- start start 2023-09-13T04:41:59.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.285+0000 7f7421521b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c583cfe0 0x55d1c583d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:59.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.285+0000 7f7421521b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583d900 0x55d1c58374c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:59.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.285+0000 7f7421521b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c5837a00 0x55d1c5837e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:59.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.285+0000 7f7421521b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c5790780 con 0x55d1c5837a00 2023-09-13T04:41:59.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.285+0000 7f7421521b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c578e0a0 con 0x55d1c583cfe0 2023-09-13T04:41:59.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.285+0000 7f7404ff9700 1 --2- 172.21.15.2:0/2043859939 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f73f00a0950 0x7f73f00a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:59.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.285+0000 7f7421521b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c5789990 con 0x55d1c583d900 2023-09-13T04:41:59.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.285+0000 7f74077fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583d900 0x55d1c58374c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:59.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.285+0000 7f7404ff9700 1 -- 172.21.15.2:0/2043859939 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f73f8014070 con 0x7f7408073060 2023-09-13T04:41:59.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.285+0000 7f7407fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c583cfe0 0x55d1c583d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:59.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.285+0000 7f74077fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583d900 0x55d1c58374c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:43286/0 (socket says 172.21.15.2:43286) 2023-09-13T04:41:59.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.285+0000 7f74077fe700 1 -- 172.21.15.2:0/2020321604 learned_addr learned my addr 172.21.15.2:0/2020321604 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:59.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.286+0000 7f7407fff700 1 --2- 172.21.15.2:0/2043859939 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f73f00a0950 0x7f73f00a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:59.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.286+0000 7f74077fe700 1 -- 172.21.15.2:0/2020321604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c583cfe0 msgr2=0x55d1c583d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:59.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.286+0000 7f74077fe700 1 --2- 172.21.15.2:0/2020321604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c583cfe0 0x55d1c583d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.286+0000 7f74077fe700 1 -- 172.21.15.2:0/2020321604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c5837a00 msgr2=0x55d1c5837e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:41:59.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.286+0000 7f74077fe700 1 --2- 172.21.15.2:0/2020321604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c5837a00 0x55d1c5837e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.286+0000 7f74077fe700 1 -- 172.21.15.2:0/2020321604 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1c5609630 con 0x55d1c583d900 2023-09-13T04:41:59.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.286+0000 7f7407fff700 1 --2- 172.21.15.2:0/2020321604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c583cfe0 0x55d1c583d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:41:59.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.286+0000 7f74077fe700 1 --2- 172.21.15.2:0/2020321604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583d900 0x55d1c58374c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f73f800c5b0 tx=0x7f73f80097d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ac9470a7d8aaeba server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:59.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.286+0000 7f73edffb700 1 -- 172.21.15.2:0/2020321604 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f8099720 con 0x55d1c583d900 2023-09-13T04:41:59.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.286+0000 7f73edffb700 1 -- 172.21.15.2:0/2020321604 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73f809d550 con 0x55d1c583d900 2023-09-13T04:41:59.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.286+0000 7f73edffb700 1 -- 172.21.15.2:0/2020321604 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f80c0d90 con 0x55d1c583d900 2023-09-13T04:41:59.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.287+0000 7f7421521b80 1 -- 172.21.15.2:0/2020321604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583d900 msgr2=0x55d1c58374c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:59.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.287+0000 7f7421521b80 1 --2- 172.21.15.2:0/2020321604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583d900 0x55d1c58374c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f73f800c5b0 tx=0x7f73f80097d0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.287+0000 7f7407fff700 1 --2- 172.21.15.2:0/2043859939 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f73f00a0950 0x7f73f00a2e10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f74000060b0 tx=0x7f740000b040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:59.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.288+0000 7f7421521b80 1 -- 172.21.15.2:0/2020321604 shutdown_connections 2023-09-13T04:41:59.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.288+0000 7f7421521b80 1 --2- 172.21.15.2:0/2020321604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c5837a00 0x55d1c5837e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.288+0000 7f7421521b80 1 --2- 172.21.15.2:0/2020321604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c583cfe0 0x55d1c583d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.288+0000 7f7421521b80 1 --2- 172.21.15.2:0/2020321604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583d900 0x55d1c58374c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.288+0000 7f7421521b80 1 -- 172.21.15.2:0/2020321604 >> 172.21.15.2:0/2020321604 conn(0x55d1c5795c80 msgr2=0x55d1c5831f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:59.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.288+0000 7f7421521b80 1 -- 172.21.15.2:0/2020321604 shutdown_connections 2023-09-13T04:41:59.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.288+0000 7f7421521b80 1 -- 172.21.15.2:0/2020321604 wait complete. 2023-09-13T04:41:59.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.288+0000 7f7421521b80 1 -- start start 2023-09-13T04:41:59.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.289+0000 7f7421521b80 1 Processor -- start 2023-09-13T04:41:59.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.289+0000 7f7421521b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c5837a00 0x55d1c58438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:59.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.289+0000 7f7421521b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c583cfe0 0x55d1c5840be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:59.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.290+0000 7f7421521b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583d900 0x55d1c5841320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:59.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.290+0000 7f7421521b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c5727bb0 con 0x55d1c5837a00 2023-09-13T04:41:59.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.290+0000 7f7421521b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c57278b0 con 0x55d1c583cfe0 2023-09-13T04:41:59.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.290+0000 7f7421521b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c560b920 con 0x55d1c583d900 2023-09-13T04:41:59.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.290+0000 7f7421521b80 10 client.0 _ll_register_callbacks cb 0x55d1c5725680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:59.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.290+0000 7f74077fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c5837a00 0x55d1c58438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:59.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.290+0000 7f7406ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c583cfe0 0x55d1c5840be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:59.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.290+0000 7f74077fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c5837a00 0x55d1c58438f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50712/0 (socket says 172.21.15.2:50712) 2023-09-13T04:41:59.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.290+0000 7f74077fe700 1 -- 172.21.15.2:0/2612086991 learned_addr learned my addr 172.21.15.2:0/2612086991 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:59.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.290+0000 7f7407fff700 1 --2- 172.21.15.2:0/2612086991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583d900 0x55d1c5841320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:59.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.291+0000 7f7406ffd700 1 -- 172.21.15.2:0/2612086991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583d900 msgr2=0x55d1c5841320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:59.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.291+0000 7f7406ffd700 1 --2- 172.21.15.2:0/2612086991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583d900 0x55d1c5841320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.291+0000 7f7406ffd700 1 -- 172.21.15.2:0/2612086991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c5837a00 msgr2=0x55d1c58438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:59.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.291+0000 7f7406ffd700 1 --2- 172.21.15.2:0/2612086991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c5837a00 0x55d1c58438f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.291+0000 7f7406ffd700 1 -- 172.21.15.2:0/2612086991 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74081a5e50 con 0x55d1c583cfe0 2023-09-13T04:41:59.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.291+0000 7f7407fff700 1 --2- 172.21.15.2:0/2612086991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583d900 0x55d1c5841320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:41:59.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.291+0000 7f7406ffd700 1 --2- 172.21.15.2:0/2612086991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c583cfe0 0x55d1c5840be0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009810 tx=0x7f73fc009400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cffad1fd7801cb5 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:59.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.291+0000 7f73e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:41:59.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.291+0000 7f73e6ffd700 1 -- 172.21.15.2:0/2612086991 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc01c040 con 0x55d1c583cfe0 2023-09-13T04:41:59.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.291+0000 7f7421521b80 1 -- 172.21.15.2:0/2612086991 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1c582a670 con 0x55d1c583cfe0 2023-09-13T04:41:59.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.291+0000 7f73e6ffd700 1 -- 172.21.15.2:0/2612086991 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f73fc00b970 con 0x55d1c583cfe0 2023-09-13T04:41:59.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.292+0000 7f73e6ffd700 1 -- 172.21.15.2:0/2612086991 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73fc00bb30 con 0x55d1c583cfe0 2023-09-13T04:41:59.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.292+0000 7f7421521b80 10 client.26000 fetch_fsmap learned FSMap version 25 2023-09-13T04:41:59.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.292+0000 7f7421521b80 1 -- 172.21.15.2:0/2612086991 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d1c57a17a0 con 0x55d1c583cfe0 2023-09-13T04:41:59.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.292+0000 7f73e6ffd700 1 -- 172.21.15.2:0/2612086991 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d1c582a670 con 0x55d1c583cfe0 2023-09-13T04:41:59.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.292+0000 7f73e6ffd700 1 -- 172.21.15.2:0/2612086991 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f73fc015020 con 0x55d1c583cfe0 2023-09-13T04:41:59.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.292+0000 7f7421521b80 10 client.26000 fetch_fsmap finished waiting for FSMap version 25 2023-09-13T04:41:59.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.292+0000 7f7421521b80 10 client.26000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:41:59.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.292+0000 7f7421521b80 20 client.26000 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:59.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.292+0000 7f7421521b80 1 --2- 172.21.15.2:0/2612086991 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55d1c58599d0 0x55d1c585bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:59.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.292+0000 7f7421521b80 4 client.26000 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:41:59.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.292+0000 7f7421521b80 1 -- 172.21.15.2:0/2612086991 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1c5782970 con 0x55d1c58599d0 2023-09-13T04:41:59.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.293+0000 7f74077fe700 1 --2- 172.21.15.2:0/2612086991 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55d1c58599d0 0x55d1c585bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:59.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.293+0000 7f74077fe700 1 --2- 172.21.15.2:0/2612086991 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55d1c58599d0 0x55d1c585bdf0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f58e98c9736b14c3 server_cookie=487ceb75e09ac310 in_seq=0 out_seq=0 2023-09-13T04:41:59.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.293+0000 7f73e6ffd700 10 client.26000 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:59.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.294+0000 7f73e6ffd700 1 -- 172.21.15.2:0/2612086991 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d1c5782970 con 0x55d1c58599d0 2023-09-13T04:41:59.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.294+0000 7f73e6ffd700 10 client.26000 handle_command_reply: tid=0 2023-09-13T04:41:59.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.294+0000 7f7421521b80 1 client.26000 shutdown 2023-09-13T04:41:59.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.294+0000 7f7421521b80 1 -- 172.21.15.2:0/2612086991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c583cfe0 msgr2=0x55d1c5840be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:59.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.294+0000 7f7421521b80 1 --2- 172.21.15.2:0/2612086991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c583cfe0 0x55d1c5840be0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009810 tx=0x7f73fc009400 comp rx=0 tx=0).stop 2023-09-13T04:41:59.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.295+0000 7f7421521b80 1 -- 172.21.15.2:0/2612086991 shutdown_connections 2023-09-13T04:41:59.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.295+0000 7f7421521b80 1 --2- 172.21.15.2:0/2612086991 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55d1c58599d0 0x55d1c585bdf0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.295+0000 7f7421521b80 1 --2- 172.21.15.2:0/2612086991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583d900 0x55d1c5841320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.295+0000 7f7421521b80 1 --2- 172.21.15.2:0/2612086991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c5837a00 0x55d1c58438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.295+0000 7f7421521b80 1 --2- 172.21.15.2:0/2612086991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c583cfe0 0x55d1c5840be0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.295+0000 7f7421521b80 1 -- 172.21.15.2:0/2612086991 >> 172.21.15.2:0/2612086991 conn(0x55d1c5795c80 msgr2=0x55d1c582c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:59.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.295+0000 7f73e6ffd700 0 client.26000 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:59.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.295+0000 7f7421521b80 1 -- 172.21.15.2:0/2612086991 shutdown_connections 2023-09-13T04:41:59.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.295+0000 7f7421521b80 1 -- 172.21.15.2:0/2612086991 wait complete. 2023-09-13T04:41:59.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.295+0000 7f7421521b80 20 client.26000 trim_cache size 0 max 16384 2023-09-13T04:41:59.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.328+0000 7f7421521b80 1 Processor -- start 2023-09-13T04:41:59.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.328+0000 7f7421521b80 1 -- start start 2023-09-13T04:41:59.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.328+0000 7f7421521b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c5837a00 0x55d1c5848240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:59.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.328+0000 7f7421521b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c583cfe0 0x55d1c582c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:59.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.328+0000 7f7421521b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583d900 0x55d1c5843490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:59.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.328+0000 7f7421521b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c582a650 con 0x55d1c5837a00 2023-09-13T04:41:59.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.328+0000 7f7421521b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c5857090 con 0x55d1c583cfe0 2023-09-13T04:41:59.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.328+0000 7f7421521b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c5833200 con 0x55d1c583d900 2023-09-13T04:41:59.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.329+0000 7f7406ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c583cfe0 0x55d1c582c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:59.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.329+0000 7f74077fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c5837a00 0x55d1c5848240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:59.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.329+0000 7f7406ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c583cfe0 0x55d1c582c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:47208/0 (socket says 172.21.15.2:47208) 2023-09-13T04:41:59.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.329+0000 7f7406ffd700 1 -- 172.21.15.2:0/267042425 learned_addr learned my addr 172.21.15.2:0/267042425 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:59.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.329+0000 7f7407fff700 1 --2- 172.21.15.2:0/267042425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583d900 0x55d1c5843490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:59.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.329+0000 7f7406ffd700 1 -- 172.21.15.2:0/267042425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583d900 msgr2=0x55d1c5843490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:59.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.329+0000 7f7406ffd700 1 --2- 172.21.15.2:0/267042425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583d900 0x55d1c5843490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.329+0000 7f7406ffd700 1 -- 172.21.15.2:0/267042425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c5837a00 msgr2=0x55d1c5848240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:59.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.329+0000 7f7406ffd700 1 --2- 172.21.15.2:0/267042425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c5837a00 0x55d1c5848240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.329+0000 7f7406ffd700 1 -- 172.21.15.2:0/267042425 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1c5860a20 con 0x55d1c583cfe0 2023-09-13T04:41:59.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.329+0000 7f7406ffd700 1 --2- 172.21.15.2:0/267042425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c583cfe0 0x55d1c582c8f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f73fc005e40 tx=0x7f73fc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95cdb2888fd3da29 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:59.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.330+0000 7f73ee7fc700 1 -- 172.21.15.2:0/267042425 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc01c070 con 0x55d1c583cfe0 2023-09-13T04:41:59.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.330+0000 7f73ee7fc700 1 -- 172.21.15.2:0/267042425 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73fc0048d0 con 0x55d1c583cfe0 2023-09-13T04:41:59.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.330+0000 7f73ee7fc700 1 -- 172.21.15.2:0/267042425 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc005540 con 0x55d1c583cfe0 2023-09-13T04:41:59.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.331+0000 7f7421521b80 1 -- 172.21.15.2:0/267042425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c583cfe0 msgr2=0x55d1c582c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:59.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.331+0000 7f7421521b80 1 --2- 172.21.15.2:0/267042425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c583cfe0 0x55d1c582c8f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f73fc005e40 tx=0x7f73fc009c00 comp rx=0 tx=0).stop 2023-09-13T04:41:59.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.331+0000 7f7421521b80 1 -- 172.21.15.2:0/267042425 shutdown_connections 2023-09-13T04:41:59.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.331+0000 7f7421521b80 1 --2- 172.21.15.2:0/267042425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583d900 0x55d1c5843490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.331+0000 7f7421521b80 1 --2- 172.21.15.2:0/267042425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c5837a00 0x55d1c5848240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.331+0000 7f7421521b80 1 --2- 172.21.15.2:0/267042425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c583cfe0 0x55d1c582c8f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.331+0000 7f7421521b80 1 -- 172.21.15.2:0/267042425 >> 172.21.15.2:0/267042425 conn(0x55d1c5795c80 msgr2=0x55d1c583ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:59.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.331+0000 7f7421521b80 1 -- 172.21.15.2:0/267042425 shutdown_connections 2023-09-13T04:41:59.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.331+0000 7f7421521b80 1 -- 172.21.15.2:0/267042425 wait complete. 2023-09-13T04:41:59.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.331+0000 7f7421521b80 1 -- start start 2023-09-13T04:41:59.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.332+0000 7f7421521b80 1 Processor -- start 2023-09-13T04:41:59.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.332+0000 7f7421521b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c5837a00 0x55d1c5839ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:59.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.332+0000 7f7421521b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583cfe0 0x55d1c583a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:59.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.332+0000 7f7421521b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c583d900 0x55d1c5838a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:59.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.332+0000 7f7421521b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c5838fa0 con 0x55d1c5837a00 2023-09-13T04:41:59.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.332+0000 7f7421521b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c5839110 con 0x55d1c583d900 2023-09-13T04:41:59.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.332+0000 7f7421521b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c58392d0 con 0x55d1c583cfe0 2023-09-13T04:41:59.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.332+0000 7f7421521b80 10 client.0 _ll_register_callbacks cb 0x55d1c584dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:41:59.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.333+0000 7f7406ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583cfe0 0x55d1c583a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:59.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.333+0000 7f7406ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583cfe0 0x55d1c583a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:43318/0 (socket says 172.21.15.2:43318) 2023-09-13T04:41:59.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.333+0000 7f7406ffd700 1 -- 172.21.15.2:0/3777421182 learned_addr learned my addr 172.21.15.2:0/3777421182 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:41:59.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.333+0000 7f74077fe700 1 --2- 172.21.15.2:0/3777421182 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c5837a00 0x55d1c5839ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:59.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.333+0000 7f7406ffd700 1 -- 172.21.15.2:0/3777421182 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c583d900 msgr2=0x55d1c5838a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:59.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.333+0000 7f7406ffd700 1 --2- 172.21.15.2:0/3777421182 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c583d900 0x55d1c5838a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.333+0000 7f7406ffd700 1 -- 172.21.15.2:0/3777421182 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c5837a00 msgr2=0x55d1c5839ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:41:59.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.333+0000 7f7406ffd700 1 --2- 172.21.15.2:0/3777421182 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c5837a00 0x55d1c5839ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.333+0000 7f7406ffd700 1 -- 172.21.15.2:0/3777421182 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1c57a17a0 con 0x55d1c583cfe0 2023-09-13T04:41:59.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.333+0000 7f7406ffd700 1 --2- 172.21.15.2:0/3777421182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583cfe0 0x55d1c583a1e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009440 tx=0x7f73fc00b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24c5491b9572a91 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:41:59.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.333+0000 7f73e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:41:59.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.334+0000 7f73e6ffd700 1 -- 172.21.15.2:0/3777421182 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc01c070 con 0x55d1c583cfe0 2023-09-13T04:41:59.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.334+0000 7f7421521b80 1 -- 172.21.15.2:0/3777421182 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f73f809d550 con 0x55d1c583cfe0 2023-09-13T04:41:59.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.334+0000 7f73e6ffd700 1 -- 172.21.15.2:0/3777421182 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f73fc000e40 con 0x55d1c583cfe0 2023-09-13T04:41:59.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.334+0000 7f73e6ffd700 1 -- 172.21.15.2:0/3777421182 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73fc00bb90 con 0x55d1c583cfe0 2023-09-13T04:41:59.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.334+0000 7f7421521b80 10 client.26026 fetch_fsmap learned FSMap version 25 2023-09-13T04:41:59.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.334+0000 7f7421521b80 1 -- 172.21.15.2:0/3777421182 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d1c55b5de0 con 0x55d1c583cfe0 2023-09-13T04:41:59.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.334+0000 7f73e6ffd700 1 -- 172.21.15.2:0/3777421182 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73f809d550 con 0x55d1c583cfe0 2023-09-13T04:41:59.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.335+0000 7f73e6ffd700 1 -- 172.21.15.2:0/3777421182 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f73fc039030 con 0x55d1c583cfe0 2023-09-13T04:41:59.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.335+0000 7f7421521b80 10 client.26026 fetch_fsmap finished waiting for FSMap version 25 2023-09-13T04:41:59.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.335+0000 7f7421521b80 10 client.26026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:41:59.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.335+0000 7f7421521b80 20 client.26026 populate_metadata read hostname 'smithi002' 2023-09-13T04:41:59.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.335+0000 7f7421521b80 1 --2- 172.21.15.2:0/3777421182 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55d1c58ae2f0 0x55d1c58b0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:41:59.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.335+0000 7f7421521b80 4 client.26026 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-09-13T04:41:59.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.335+0000 7f7421521b80 1 -- 172.21.15.2:0/3777421182 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d1c5743050 con 0x55d1c58ae2f0 2023-09-13T04:41:59.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.335+0000 7f74077fe700 1 --2- 172.21.15.2:0/3777421182 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55d1c58ae2f0 0x55d1c58b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:41:59.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.336+0000 7f74077fe700 1 --2- 172.21.15.2:0/3777421182 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55d1c58ae2f0 0x55d1c58b0710 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fab5538f42381ac server_cookie=e9a5edc672a710ae in_seq=0 out_seq=0 2023-09-13T04:41:59.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.336+0000 7f73e6ffd700 10 client.26026 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:59.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.336+0000 7f73e6ffd700 1 -- 172.21.15.2:0/3777421182 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d1c5743050 con 0x55d1c58ae2f0 2023-09-13T04:41:59.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.336+0000 7f73e6ffd700 10 client.26026 handle_command_reply: tid=0 2023-09-13T04:41:59.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.336+0000 7f7421521b80 1 client.26026 shutdown 2023-09-13T04:41:59.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.336+0000 7f7421521b80 1 -- 172.21.15.2:0/3777421182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583cfe0 msgr2=0x55d1c583a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:59.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.336+0000 7f7421521b80 1 --2- 172.21.15.2:0/3777421182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583cfe0 0x55d1c583a1e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009440 tx=0x7f73fc00b9a0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.336+0000 7f7421521b80 1 -- 172.21.15.2:0/3777421182 shutdown_connections 2023-09-13T04:41:59.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.336+0000 7f7421521b80 1 --2- 172.21.15.2:0/3777421182 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55d1c58ae2f0 0x55d1c58b0710 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.336+0000 7f7421521b80 1 --2- 172.21.15.2:0/3777421182 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c583d900 0x55d1c5838a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.336+0000 7f7421521b80 1 --2- 172.21.15.2:0/3777421182 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c5837a00 0x55d1c5839ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.336+0000 7f73e6ffd700 0 client.26026 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:41:59.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.336+0000 7f7421521b80 1 --2- 172.21.15.2:0/3777421182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1c583cfe0 0x55d1c583a1e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.336+0000 7f7421521b80 1 -- 172.21.15.2:0/3777421182 >> 172.21.15.2:0/3777421182 conn(0x55d1c5795c80 msgr2=0x55d1c583dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:59.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.337+0000 7f7421521b80 1 -- 172.21.15.2:0/3777421182 shutdown_connections 2023-09-13T04:41:59.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.337+0000 7f7421521b80 1 -- 172.21.15.2:0/3777421182 wait complete. 2023-09-13T04:41:59.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.337+0000 7f7421521b80 20 client.26026 trim_cache size 0 max 16384 2023-09-13T04:41:59.349 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:41:59.350 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-13T04:41:59.350 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-09-13T04:41:59.350 INFO:teuthology.orchestra.run.smithi002.stdout: "d95eecf9-65e4-419c-9998-59370b2f6d5a": { 2023-09-13T04:41:59.350 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-09-13T04:41:59.350 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "d95eecf9-65e4-419c-9998-59370b2f6d5a", 2023-09-13T04:41:59.351 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-09-13T04:41:59.351 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:41:59.351 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:41:59.351 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:41:59.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.338+0000 7f740dbff700 1 -- 172.21.15.2:0/2043859939 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f73f00a0950 msgr2=0x7f73f00a2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:59.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.338+0000 7f740dbff700 1 --2- 172.21.15.2:0/2043859939 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f73f00a0950 0x7f73f00a2e10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f74000060b0 tx=0x7f740000b040 comp rx=0 tx=0).stop 2023-09-13T04:41:59.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.338+0000 7f740dbff700 1 -- 172.21.15.2:0/2043859939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7408073060 msgr2=0x7f7408199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:41:59.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.338+0000 7f740dbff700 1 --2- 172.21.15.2:0/2043859939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7408073060 0x7f7408199020 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f73f800e9c0 tx=0x7f73f800ed80 comp rx=0 tx=0).stop 2023-09-13T04:41:59.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.338+0000 7f740dbff700 1 -- 172.21.15.2:0/2043859939 shutdown_connections 2023-09-13T04:41:59.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.338+0000 7f740dbff700 1 --2- 172.21.15.2:0/2043859939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f740810d530 0x7f740819d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.338+0000 7f740dbff700 1 --2- 172.21.15.2:0/2043859939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7408073060 0x7f7408199020 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.338+0000 7f740dbff700 1 --2- 172.21.15.2:0/2043859939 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f73f00a0950 0x7f73f00a2e10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.338+0000 7f740dbff700 1 --2- 172.21.15.2:0/2043859939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7408073980 0x7f7408199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:41:59.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.338+0000 7f740dbff700 1 -- 172.21.15.2:0/2043859939 >> 172.21.15.2:0/2043859939 conn(0x7f74080fc760 msgr2=0x7f740810f670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:41:59.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.338+0000 7f740dbff700 1 -- 172.21.15.2:0/2043859939 shutdown_connections 2023-09-13T04:41:59.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:41:59.338+0000 7f740dbff700 1 -- 172.21.15.2:0/2043859939 wait complete. 2023-09-13T04:41:59.359 INFO:tasks.cephfs.filesystem:scrub status for tag:d95eecf9-65e4-419c-9998-59370b2f6d5a - {'path': '/', 'tag': 'd95eecf9-65e4-419c-9998-59370b2f6d5a', 'options': 'recursive,force'} 2023-09-13T04:41:59.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:41:58 smithi149 ceph-mon[133315]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:42:00.594 INFO:tasks.workunit.client.0.smithi002.stdout: 6 49216 444 27717 427 7712 414 2023-09-13T04:42:00.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:00 smithi149 ceph-mon[133315]: scrub queued for path: / 2023-09-13T04:42:00.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:00 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:00.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:00 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:42:00.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:00 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:42:00.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:00 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:00.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:00 smithi156 ceph-mon[133675]: scrub queued for path: / 2023-09-13T04:42:00.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:00 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:00.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:00 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:42:00.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:00 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:42:00.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:00 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:00.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:00 smithi002 ceph-mon[97651]: scrub queued for path: / 2023-09-13T04:42:00.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:00 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:00.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:00 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:42:00.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:00 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:42:00.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:00 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:01.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:01 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:01.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:01 smithi149 ceph-mon[133315]: pgmap v49: 129 pgs: 129 active+clean; 24 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2023-09-13T04:42:01.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:01 smithi149 ceph-mon[133315]: mgrmap e25: x(active, since 92s), standbys: y, z 2023-09-13T04:42:01.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:01 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:01.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:01 smithi156 ceph-mon[133675]: pgmap v49: 129 pgs: 129 active+clean; 24 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2023-09-13T04:42:01.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:01 smithi156 ceph-mon[133675]: mgrmap e25: x(active, since 92s), standbys: y, z 2023-09-13T04:42:01.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:01 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:01.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:01 smithi002 ceph-mon[97651]: pgmap v49: 129 pgs: 129 active+clean; 24 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2023-09-13T04:42:01.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:01 smithi002 ceph-mon[97651]: mgrmap e25: x(active, since 92s), standbys: y, z 2023-09-13T04:42:02.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:02 smithi149 ceph-mon[133315]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:42:02.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:02 smithi156 ceph-mon[133675]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:42:02.768 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:02 smithi002 ceph-mon[97651]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:42:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.426+0000 7fd89b7ff700 1 -- 172.21.15.2:0/54623470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd894068a90 msgr2=0x7fd89410db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.426+0000 7fd89b7ff700 1 --2- 172.21.15.2:0/54623470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd894068a90 0x7fd89410db00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd88c00b0a0 tx=0x7fd88c00b3b0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.426+0000 7fd89b7ff700 1 -- 172.21.15.2:0/54623470 shutdown_connections 2023-09-13T04:42:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.426+0000 7fd89b7ff700 1 --2- 172.21.15.2:0/54623470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd894068a90 0x7fd89410db00 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.426+0000 7fd89b7ff700 1 --2- 172.21.15.2:0/54623470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd894106d60 0x7fd894107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.426+0000 7fd89b7ff700 1 --2- 172.21.15.2:0/54623470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8940680d0 0x7fd894068550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.426+0000 7fd89b7ff700 1 -- 172.21.15.2:0/54623470 >> 172.21.15.2:0/54623470 conn(0x7fd894076970 msgr2=0x7fd894076d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.427+0000 7fd89b7ff700 1 -- 172.21.15.2:0/54623470 shutdown_connections 2023-09-13T04:42:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.427+0000 7fd89b7ff700 1 -- 172.21.15.2:0/54623470 wait complete. 2023-09-13T04:42:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.428+0000 7fd89b7ff700 1 Processor -- start 2023-09-13T04:42:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.428+0000 7fd89b7ff700 1 -- start start 2023-09-13T04:42:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.429+0000 7fd89b7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8940680d0 0x7fd894100020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.429+0000 7fd89b7ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd894068a90 0x7fd894100560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.429+0000 7fd89b7ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd894106d60 0x7fd8941a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.429+0000 7fd89b7ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8941a5520 con 0x7fd894106d60 2023-09-13T04:42:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.429+0000 7fd89b7ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8941a5660 con 0x7fd894068a90 2023-09-13T04:42:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.429+0000 7fd89b7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8941a57d0 con 0x7fd8940680d0 2023-09-13T04:42:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.429+0000 7fd89959b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8940680d0 0x7fd894100020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.429+0000 7fd898d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd894068a90 0x7fd894100560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.429+0000 7fd898d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd894068a90 0x7fd894100560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:47236/0 (socket says 172.21.15.2:47236) 2023-09-13T04:42:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.429+0000 7fd898d9a700 1 -- 172.21.15.2:0/3038391653 learned_addr learned my addr 172.21.15.2:0/3038391653 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.429+0000 7fd899d9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd894106d60 0x7fd8941a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.430+0000 7fd89959b700 1 -- 172.21.15.2:0/3038391653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd894068a90 msgr2=0x7fd894100560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.430+0000 7fd89959b700 1 --2- 172.21.15.2:0/3038391653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd894068a90 0x7fd894100560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.430+0000 7fd89959b700 1 -- 172.21.15.2:0/3038391653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd894106d60 msgr2=0x7fd8941a4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.430+0000 7fd89959b700 1 --2- 172.21.15.2:0/3038391653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd894106d60 0x7fd8941a4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.430+0000 7fd89959b700 1 -- 172.21.15.2:0/3038391653 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd884009520 con 0x7fd8940680d0 2023-09-13T04:42:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.430+0000 7fd89959b700 1 --2- 172.21.15.2:0/3038391653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8940680d0 0x7fd894100020 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd89000e9f0 tx=0x7fd89000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.430+0000 7fd88a7fc700 1 -- 172.21.15.2:0/3038391653 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89000c9c0 con 0x7fd8940680d0 2023-09-13T04:42:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.430+0000 7fd88a7fc700 1 -- 172.21.15.2:0/3038391653 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd890010430 con 0x7fd8940680d0 2023-09-13T04:42:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.430+0000 7fd89b7ff700 1 -- 172.21.15.2:0/3038391653 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88c009d00 con 0x7fd8940680d0 2023-09-13T04:42:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.430+0000 7fd88a7fc700 1 -- 172.21.15.2:0/3038391653 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd890018680 con 0x7fd8940680d0 2023-09-13T04:42:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.430+0000 7fd89b7ff700 1 -- 172.21.15.2:0/3038391653 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8941a5e30 con 0x7fd8940680d0 2023-09-13T04:42:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.435+0000 7fd88a7fc700 1 -- 172.21.15.2:0/3038391653 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7fd890010860 con 0x7fd8940680d0 2023-09-13T04:42:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.436+0000 7fd8af12fb80 1 Processor -- start 2023-09-13T04:42:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.437+0000 7fd8af12fb80 1 -- start start 2023-09-13T04:42:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.437+0000 7fd88a7fc700 1 --2- 172.21.15.2:0/3038391653 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fd8800a07b0 0x7fd8800a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.438+0000 7fd8af12fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd27ca9be0 0x55dd27ca9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.438+0000 7fd88a7fc700 1 -- 172.21.15.2:0/3038391653 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fd890014070 con 0x7fd8940680d0 2023-09-13T04:42:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.438+0000 7fd8af12fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd27caa500 0x55dd27ca5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.438+0000 7fd8af12fb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd27ca5840 0x55dd27ca5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.438+0000 7fd8af12fb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd27bfe720 con 0x55dd27ca5840 2023-09-13T04:42:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.438+0000 7fd8af12fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd27bfc040 con 0x55dd27caa500 2023-09-13T04:42:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.438+0000 7fd8af12fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd27bf7930 con 0x55dd27ca9be0 2023-09-13T04:42:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.438+0000 7fd89959b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd27ca9be0 0x55dd27ca9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.438+0000 7fd89959b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd27ca9be0 0x55dd27ca9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:43340/0 (socket says 172.21.15.2:43340) 2023-09-13T04:42:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.438+0000 7fd89959b700 1 -- 172.21.15.2:0/790639083 learned_addr learned my addr 172.21.15.2:0/790639083 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.438+0000 7fd898d9a700 1 --2- 172.21.15.2:0/790639083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd27caa500 0x55dd27ca5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:03.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.438+0000 7fd899d9c700 1 --2- 172.21.15.2:0/3038391653 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fd8800a07b0 0x7fd8800a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:03.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.438+0000 7fd899d9c700 1 --2- 172.21.15.2:0/790639083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd27ca5840 0x55dd27ca5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:03.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.438+0000 7fd89959b700 1 -- 172.21.15.2:0/790639083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd27caa500 msgr2=0x55dd27ca5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:03.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.439+0000 7fd89959b700 1 --2- 172.21.15.2:0/790639083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd27caa500 0x55dd27ca5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.439+0000 7fd89959b700 1 -- 172.21.15.2:0/790639083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd27ca5840 msgr2=0x55dd27ca5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.439+0000 7fd89959b700 1 --2- 172.21.15.2:0/790639083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd27ca5840 0x55dd27ca5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.439+0000 7fd89959b700 1 -- 172.21.15.2:0/790639083 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd27a775d0 con 0x55dd27ca9be0 2023-09-13T04:42:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.439+0000 7fd899d9c700 1 --2- 172.21.15.2:0/790639083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd27ca5840 0x55dd27ca5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:42:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.439+0000 7fd898d9a700 1 --2- 172.21.15.2:0/790639083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd27caa500 0x55dd27ca5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:42:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.439+0000 7fd89959b700 1 --2- 172.21.15.2:0/790639083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd27ca9be0 0x55dd27ca9fc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd890095a90 tx=0x7fd8900991f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0dfd452aa5d608f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.439+0000 7fd87f7fe700 1 -- 172.21.15.2:0/790639083 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd890099590 con 0x55dd27ca9be0 2023-09-13T04:42:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.439+0000 7fd87f7fe700 1 -- 172.21.15.2:0/790639083 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd89009d3c0 con 0x55dd27ca9be0 2023-09-13T04:42:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.439+0000 7fd899d9c700 1 --2- 172.21.15.2:0/3038391653 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fd8800a07b0 0x7fd8800a2c70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd88c0060b0 tx=0x7fd88c006000 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.439+0000 7fd87f7fe700 1 -- 172.21.15.2:0/790639083 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8900840b0 con 0x55dd27ca9be0 2023-09-13T04:42:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.440+0000 7fd8af12fb80 1 -- 172.21.15.2:0/790639083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd27ca9be0 msgr2=0x55dd27ca9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.440+0000 7fd8af12fb80 1 --2- 172.21.15.2:0/790639083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd27ca9be0 0x55dd27ca9fc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd890095a90 tx=0x7fd8900991f0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.440+0000 7fd8af12fb80 1 -- 172.21.15.2:0/790639083 shutdown_connections 2023-09-13T04:42:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.440+0000 7fd8af12fb80 1 --2- 172.21.15.2:0/790639083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd27ca5840 0x55dd27ca5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.440+0000 7fd8af12fb80 1 --2- 172.21.15.2:0/790639083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd27ca9be0 0x55dd27ca9fc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.440+0000 7fd8af12fb80 1 --2- 172.21.15.2:0/790639083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd27caa500 0x55dd27ca5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.440+0000 7fd8af12fb80 1 -- 172.21.15.2:0/790639083 >> 172.21.15.2:0/790639083 conn(0x55dd27c03c20 msgr2=0x55dd27c9fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.440+0000 7fd8af12fb80 1 -- 172.21.15.2:0/790639083 shutdown_connections 2023-09-13T04:42:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.440+0000 7fd8af12fb80 1 -- 172.21.15.2:0/790639083 wait complete. 2023-09-13T04:42:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.441+0000 7fd8af12fb80 1 -- start start 2023-09-13T04:42:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.441+0000 7fd8af12fb80 1 Processor -- start 2023-09-13T04:42:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.442+0000 7fd8af12fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd27ca5840 0x55dd27cb1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.442+0000 7fd8af12fb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd27ca9be0 0x55dd27cae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.442+0000 7fd8af12fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd27caa500 0x55dd27caefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.442+0000 7fd8af12fb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd27b95b50 con 0x55dd27ca9be0 2023-09-13T04:42:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.442+0000 7fd8af12fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd27b95850 con 0x55dd27caa500 2023-09-13T04:42:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.442+0000 7fd8af12fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd27a798c0 con 0x55dd27ca5840 2023-09-13T04:42:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.442+0000 7fd8af12fb80 10 client.0 _ll_register_callbacks cb 0x55dd27b93620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:42:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.442+0000 7fd89959b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd27ca5840 0x55dd27cb1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.442+0000 7fd89959b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd27ca5840 0x55dd27cb1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:43342/0 (socket says 172.21.15.2:43342) 2023-09-13T04:42:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.442+0000 7fd89959b700 1 -- 172.21.15.2:0/1682305734 learned_addr learned my addr 172.21.15.2:0/1682305734 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.442+0000 7fd899d9c700 1 --2- 172.21.15.2:0/1682305734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd27caa500 0x55dd27caefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.442+0000 7fd89959b700 1 -- 172.21.15.2:0/1682305734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd27caa500 msgr2=0x55dd27caefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.442+0000 7fd89959b700 1 --2- 172.21.15.2:0/1682305734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd27caa500 0x55dd27caefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.443+0000 7fd89959b700 1 -- 172.21.15.2:0/1682305734 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd27ca9be0 msgr2=0x55dd27cae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:03.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.443+0000 7fd89959b700 1 --2- 172.21.15.2:0/1682305734 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd27ca9be0 0x55dd27cae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.443+0000 7fd89959b700 1 -- 172.21.15.2:0/1682305734 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8941a5e30 con 0x55dd27ca5840 2023-09-13T04:42:03.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.443+0000 7fd89959b700 1 --2- 172.21.15.2:0/1682305734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd27ca5840 0x55dd27cb1600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd89000e9c0 tx=0x7fd8900c97d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=587eb164696715ec server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:03.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.443+0000 7fd87cff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:42:03.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.443+0000 7fd87cff9700 1 -- 172.21.15.2:0/1682305734 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd890099590 con 0x55dd27ca5840 2023-09-13T04:42:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.443+0000 7fd8af12fb80 1 -- 172.21.15.2:0/1682305734 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd27c6aef0 con 0x55dd27ca5840 2023-09-13T04:42:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.443+0000 7fd87cff9700 1 -- 172.21.15.2:0/1682305734 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fd8900997f0 con 0x55dd27ca5840 2023-09-13T04:42:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.443+0000 7fd87cff9700 1 -- 172.21.15.2:0/1682305734 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8900999b0 con 0x55dd27ca5840 2023-09-13T04:42:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.443+0000 7fd8af12fb80 10 client.26038 fetch_fsmap learned FSMap version 26 2023-09-13T04:42:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.443+0000 7fd8af12fb80 1 -- 172.21.15.2:0/1682305734 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55dd27c0f740 con 0x55dd27ca5840 2023-09-13T04:42:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.443+0000 7fd87cff9700 1 -- 172.21.15.2:0/1682305734 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dd27c6aef0 con 0x55dd27ca5840 2023-09-13T04:42:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.444+0000 7fd87cff9700 1 -- 172.21.15.2:0/1682305734 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fd890098bd0 con 0x55dd27ca5840 2023-09-13T04:42:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.444+0000 7fd8af12fb80 10 client.26038 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T04:42:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.444+0000 7fd8af12fb80 10 client.26038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:42:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.444+0000 7fd8af12fb80 20 client.26038 populate_metadata read hostname 'smithi002' 2023-09-13T04:42:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.444+0000 7fd8af12fb80 1 --2- 172.21.15.2:0/1682305734 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55dd27cbeaa0 0x55dd27cc0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.444+0000 7fd8af12fb80 4 client.26038 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:42:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.444+0000 7fd8af12fb80 1 -- 172.21.15.2:0/1682305734 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd27bf0910 con 0x55dd27cbeaa0 2023-09-13T04:42:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.444+0000 7fd898d9a700 1 --2- 172.21.15.2:0/1682305734 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55dd27cbeaa0 0x55dd27cc0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.445+0000 7fd898d9a700 1 --2- 172.21.15.2:0/1682305734 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55dd27cbeaa0 0x55dd27cc0ec0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7955717f4bc8d20f server_cookie=b3c67c65890aab6f in_seq=0 out_seq=0 2023-09-13T04:42:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.445+0000 7fd87cff9700 10 client.26038 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.445+0000 7fd87cff9700 1 -- 172.21.15.2:0/1682305734 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dd27bf0910 con 0x55dd27cbeaa0 2023-09-13T04:42:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.446+0000 7fd87cff9700 10 client.26038 handle_command_reply: tid=0 2023-09-13T04:42:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.446+0000 7fd8af12fb80 1 client.26038 shutdown 2023-09-13T04:42:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.446+0000 7fd8af12fb80 1 -- 172.21.15.2:0/1682305734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd27ca5840 msgr2=0x55dd27cb1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.446+0000 7fd8af12fb80 1 --2- 172.21.15.2:0/1682305734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd27ca5840 0x55dd27cb1600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd89000e9c0 tx=0x7fd8900c97d0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.446+0000 7fd8af12fb80 1 -- 172.21.15.2:0/1682305734 shutdown_connections 2023-09-13T04:42:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.446+0000 7fd8af12fb80 1 --2- 172.21.15.2:0/1682305734 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55dd27cbeaa0 0x55dd27cc0ec0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.446+0000 7fd8af12fb80 1 --2- 172.21.15.2:0/1682305734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd27caa500 0x55dd27caefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.446+0000 7fd8af12fb80 1 --2- 172.21.15.2:0/1682305734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd27ca5840 0x55dd27cb1600 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.446+0000 7fd8af12fb80 1 --2- 172.21.15.2:0/1682305734 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd27ca9be0 0x55dd27cae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.446+0000 7fd8af12fb80 1 -- 172.21.15.2:0/1682305734 >> 172.21.15.2:0/1682305734 conn(0x55dd27c03c20 msgr2=0x55dd27cab330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.446+0000 7fd87cff9700 0 client.26038 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.446+0000 7fd8af12fb80 1 -- 172.21.15.2:0/1682305734 shutdown_connections 2023-09-13T04:42:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.446+0000 7fd8af12fb80 1 -- 172.21.15.2:0/1682305734 wait complete. 2023-09-13T04:42:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.446+0000 7fd8af12fb80 20 client.26038 trim_cache size 0 max 16384 2023-09-13T04:42:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.464+0000 7fd8af12fb80 1 Processor -- start 2023-09-13T04:42:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.464+0000 7fd8af12fb80 1 -- start start 2023-09-13T04:42:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.464+0000 7fd8af12fb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd27ca5840 0x55dd27cb5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.464+0000 7fd8af12fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd27ca9be0 0x55dd27c9a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.464+0000 7fd8af12fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd27caa500 0x55dd27cb11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.464+0000 7fd8af12fb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd27c9c7e0 con 0x55dd27ca5840 2023-09-13T04:42:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.464+0000 7fd8af12fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd27cb6430 con 0x55dd27ca9be0 2023-09-13T04:42:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.464+0000 7fd8af12fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd27ca4440 con 0x55dd27caa500 2023-09-13T04:42:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.465+0000 7fd898d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd27ca9be0 0x55dd27c9a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.465+0000 7fd89959b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd27ca5840 0x55dd27cb5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.465+0000 7fd898d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd27ca9be0 0x55dd27c9a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:47268/0 (socket says 172.21.15.2:47268) 2023-09-13T04:42:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.465+0000 7fd898d9a700 1 -- 172.21.15.2:0/3513261635 learned_addr learned my addr 172.21.15.2:0/3513261635 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.465+0000 7fd899d9c700 1 --2- 172.21.15.2:0/3513261635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd27caa500 0x55dd27cb11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.465+0000 7fd898d9a700 1 -- 172.21.15.2:0/3513261635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd27caa500 msgr2=0x55dd27cb11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.465+0000 7fd898d9a700 1 --2- 172.21.15.2:0/3513261635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd27caa500 0x55dd27cb11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.465+0000 7fd898d9a700 1 -- 172.21.15.2:0/3513261635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd27ca5840 msgr2=0x55dd27cb5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.465+0000 7fd898d9a700 1 --2- 172.21.15.2:0/3513261635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd27ca5840 0x55dd27cb5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.465+0000 7fd898d9a700 1 -- 172.21.15.2:0/3513261635 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd27a23d80 con 0x55dd27ca9be0 2023-09-13T04:42:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.465+0000 7fd898d9a700 1 --2- 172.21.15.2:0/3513261635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd27ca9be0 0x55dd27c9a730 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55dd27cc1790 tx=0x7fd88401ac20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b6cb592ae582a48 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.466+0000 7fd87ffff700 1 -- 172.21.15.2:0/3513261635 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd884016070 con 0x55dd27ca9be0 2023-09-13T04:42:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.466+0000 7fd87ffff700 1 -- 172.21.15.2:0/3513261635 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd884017040 con 0x55dd27ca9be0 2023-09-13T04:42:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.466+0000 7fd87ffff700 1 -- 172.21.15.2:0/3513261635 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88401a730 con 0x55dd27ca9be0 2023-09-13T04:42:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.466+0000 7fd8af12fb80 1 -- 172.21.15.2:0/3513261635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd27ca9be0 msgr2=0x55dd27c9a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.466+0000 7fd8af12fb80 1 --2- 172.21.15.2:0/3513261635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd27ca9be0 0x55dd27c9a730 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55dd27cc1790 tx=0x7fd88401ac20 comp rx=0 tx=0).stop 2023-09-13T04:42:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.467+0000 7fd8af12fb80 1 -- 172.21.15.2:0/3513261635 shutdown_connections 2023-09-13T04:42:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.467+0000 7fd8af12fb80 1 --2- 172.21.15.2:0/3513261635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd27caa500 0x55dd27cb11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.467+0000 7fd8af12fb80 1 --2- 172.21.15.2:0/3513261635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd27ca5840 0x55dd27cb5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.467+0000 7fd8af12fb80 1 --2- 172.21.15.2:0/3513261635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd27ca9be0 0x55dd27c9a730 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.467+0000 7fd8af12fb80 1 -- 172.21.15.2:0/3513261635 >> 172.21.15.2:0/3513261635 conn(0x55dd27c03c20 msgr2=0x55dd27ca3930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.467+0000 7fd8af12fb80 1 -- 172.21.15.2:0/3513261635 shutdown_connections 2023-09-13T04:42:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.467+0000 7fd8af12fb80 1 -- 172.21.15.2:0/3513261635 wait complete. 2023-09-13T04:42:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.467+0000 7fd8af12fb80 1 -- start start 2023-09-13T04:42:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.467+0000 7fd8af12fb80 1 Processor -- start 2023-09-13T04:42:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.468+0000 7fd8af12fb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd27ca5840 0x55dd27d0ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.468+0000 7fd8af12fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd27ca9be0 0x55dd27d0b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.468+0000 7fd8af12fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd27caa500 0x55dd27d06820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.468+0000 7fd8af12fb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd27d06f30 con 0x55dd27ca5840 2023-09-13T04:42:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.468+0000 7fd8af12fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd27d070a0 con 0x55dd27caa500 2023-09-13T04:42:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.468+0000 7fd8af12fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd27d07260 con 0x55dd27ca9be0 2023-09-13T04:42:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.468+0000 7fd8af12fb80 10 client.0 _ll_register_callbacks cb 0x55dd27b93620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:42:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.468+0000 7fd89959b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd27ca5840 0x55dd27d0ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.468+0000 7fd89959b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd27ca5840 0x55dd27d0ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50788/0 (socket says 172.21.15.2:50788) 2023-09-13T04:42:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.468+0000 7fd89959b700 1 -- 172.21.15.2:0/2916209099 learned_addr learned my addr 172.21.15.2:0/2916209099 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.468+0000 7fd899d9c700 1 --2- 172.21.15.2:0/2916209099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd27caa500 0x55dd27d06820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.469+0000 7fd89959b700 1 -- 172.21.15.2:0/2916209099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd27ca9be0 msgr2=0x55dd27d0b150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:42:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.469+0000 7fd89959b700 1 --2- 172.21.15.2:0/2916209099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd27ca9be0 0x55dd27d0b150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.469+0000 7fd89959b700 1 -- 172.21.15.2:0/2916209099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd27caa500 msgr2=0x55dd27d06820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.469+0000 7fd89959b700 1 --2- 172.21.15.2:0/2916209099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd27caa500 0x55dd27d06820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.469+0000 7fd89959b700 1 -- 172.21.15.2:0/2916209099 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd884009c80 con 0x55dd27ca5840 2023-09-13T04:42:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.469+0000 7fd89959b700 1 --2- 172.21.15.2:0/2916209099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd27ca5840 0x55dd27d0ac10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd890099520 tx=0x7fd8900107d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb22ffb100f8494f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.469+0000 7fd87cff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:42:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.469+0000 7fd87cff9700 1 -- 172.21.15.2:0/2916209099 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8900958d0 con 0x55dd27ca5840 2023-09-13T04:42:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.469+0000 7fd87cff9700 1 -- 172.21.15.2:0/2916209099 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fd890099590 con 0x55dd27ca5840 2023-09-13T04:42:03.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.469+0000 7fd8af12fb80 1 -- 172.21.15.2:0/2916209099 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd89009d3c0 con 0x55dd27ca5840 2023-09-13T04:42:03.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.470+0000 7fd87cff9700 1 -- 172.21.15.2:0/2916209099 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8900a1300 con 0x55dd27ca5840 2023-09-13T04:42:03.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.470+0000 7fd8af12fb80 10 client.15999 fetch_fsmap learned FSMap version 26 2023-09-13T04:42:03.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.470+0000 7fd8af12fb80 1 -- 172.21.15.2:0/2916209099 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55dd27a0cc00 con 0x55dd27ca5840 2023-09-13T04:42:03.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.470+0000 7fd87cff9700 1 -- 172.21.15.2:0/2916209099 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd89009d3c0 con 0x55dd27ca5840 2023-09-13T04:42:03.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.470+0000 7fd87cff9700 1 -- 172.21.15.2:0/2916209099 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fd8900b1d20 con 0x55dd27ca5840 2023-09-13T04:42:03.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.470+0000 7fd8af12fb80 10 client.15999 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T04:42:03.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.470+0000 7fd8af12fb80 10 client.15999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:42:03.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.470+0000 7fd8af12fb80 20 client.15999 populate_metadata read hostname 'smithi002' 2023-09-13T04:42:03.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.470+0000 7fd8af12fb80 1 --2- 172.21.15.2:0/2916209099 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55dd27d1be40 0x55dd27d1e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.470+0000 7fd8af12fb80 4 client.15999 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:42:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.470+0000 7fd8af12fb80 1 -- 172.21.15.2:0/2916209099 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dd27bb0ff0 con 0x55dd27d1be40 2023-09-13T04:42:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.470+0000 7fd898d9a700 1 --2- 172.21.15.2:0/2916209099 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55dd27d1be40 0x55dd27d1e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.471+0000 7fd898d9a700 1 --2- 172.21.15.2:0/2916209099 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55dd27d1be40 0x55dd27d1e2e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7d4b5f23c9b1d7a server_cookie=8617ad42e08371cf in_seq=0 out_seq=0 2023-09-13T04:42:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.471+0000 7fd87cff9700 10 client.15999 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.486+0000 7fd87cff9700 1 -- 172.21.15.2:0/2916209099 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dd27bb0ff0 con 0x55dd27d1be40 2023-09-13T04:42:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.486+0000 7fd87cff9700 10 client.15999 handle_command_reply: tid=0 2023-09-13T04:42:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.487+0000 7fd8af12fb80 1 client.15999 shutdown 2023-09-13T04:42:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.487+0000 7fd8af12fb80 1 -- 172.21.15.2:0/2916209099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd27ca5840 msgr2=0x55dd27d0ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.487+0000 7fd8af12fb80 1 --2- 172.21.15.2:0/2916209099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd27ca5840 0x55dd27d0ac10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd890099520 tx=0x7fd8900107d0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.487+0000 7fd8af12fb80 1 -- 172.21.15.2:0/2916209099 shutdown_connections 2023-09-13T04:42:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.487+0000 7fd8af12fb80 1 --2- 172.21.15.2:0/2916209099 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55dd27d1be40 0x55dd27d1e2e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.487+0000 7fd8af12fb80 1 --2- 172.21.15.2:0/2916209099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd27caa500 0x55dd27d06820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.487+0000 7fd8af12fb80 1 --2- 172.21.15.2:0/2916209099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd27ca5840 0x55dd27d0ac10 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.487+0000 7fd87cff9700 0 client.15999 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.487+0000 7fd8af12fb80 1 --2- 172.21.15.2:0/2916209099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd27ca9be0 0x55dd27d0b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.487+0000 7fd8af12fb80 1 -- 172.21.15.2:0/2916209099 >> 172.21.15.2:0/2916209099 conn(0x55dd27c03c20 msgr2=0x55dd27ca4850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.488+0000 7fd8af12fb80 1 -- 172.21.15.2:0/2916209099 shutdown_connections 2023-09-13T04:42:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.488+0000 7fd8af12fb80 1 -- 172.21.15.2:0/2916209099 wait complete. 2023-09-13T04:42:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.489+0000 7fd8af12fb80 20 client.15999 trim_cache size 0 max 16384 2023-09-13T04:42:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:42:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:42:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:42:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:42:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.490+0000 7fd89b7ff700 1 -- 172.21.15.2:0/3038391653 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fd8800a07b0 msgr2=0x7fd8800a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.490+0000 7fd89b7ff700 1 --2- 172.21.15.2:0/3038391653 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fd8800a07b0 0x7fd8800a2c70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd88c0060b0 tx=0x7fd88c006000 comp rx=0 tx=0).stop 2023-09-13T04:42:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.490+0000 7fd89b7ff700 1 -- 172.21.15.2:0/3038391653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8940680d0 msgr2=0x7fd894100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.490+0000 7fd89b7ff700 1 --2- 172.21.15.2:0/3038391653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8940680d0 0x7fd894100020 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd89000e9f0 tx=0x7fd89000ed00 comp rx=0 tx=0).stop 2023-09-13T04:42:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.490+0000 7fd89b7ff700 1 -- 172.21.15.2:0/3038391653 shutdown_connections 2023-09-13T04:42:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.490+0000 7fd89b7ff700 1 --2- 172.21.15.2:0/3038391653 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fd8800a07b0 0x7fd8800a2c70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.490+0000 7fd89b7ff700 1 --2- 172.21.15.2:0/3038391653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd894106d60 0x7fd8941a4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.490+0000 7fd89b7ff700 1 --2- 172.21.15.2:0/3038391653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8940680d0 0x7fd894100020 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.490+0000 7fd89b7ff700 1 --2- 172.21.15.2:0/3038391653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd894068a90 0x7fd894100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.490+0000 7fd89b7ff700 1 -- 172.21.15.2:0/3038391653 >> 172.21.15.2:0/3038391653 conn(0x7fd894076970 msgr2=0x7fd8940fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.493+0000 7fd89b7ff700 1 -- 172.21.15.2:0/3038391653 shutdown_connections 2023-09-13T04:42:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:03.493+0000 7fd89b7ff700 1 -- 172.21.15.2:0/3038391653 wait complete. 2023-09-13T04:42:03.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:03 smithi149 ceph-mon[133315]: pgmap v50: 129 pgs: 129 active+clean; 24 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 153 op/s 2023-09-13T04:42:03.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:03 smithi156 ceph-mon[133675]: pgmap v50: 129 pgs: 129 active+clean; 24 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 153 op/s 2023-09-13T04:42:03.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:03 smithi002 ceph-mon[97651]: pgmap v50: 129 pgs: 129 active+clean; 24 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 153 op/s 2023-09-13T04:42:05.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:05 smithi149 ceph-mon[133315]: pgmap v51: 129 pgs: 129 active+clean; 30 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 244 op/s 2023-09-13T04:42:05.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:05 smithi156 ceph-mon[133675]: pgmap v51: 129 pgs: 129 active+clean; 30 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 244 op/s 2023-09-13T04:42:05.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:05 smithi002 ceph-mon[97651]: pgmap v51: 129 pgs: 129 active+clean; 30 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 244 op/s 2023-09-13T04:42:07.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:07 smithi149 ceph-mon[133315]: pgmap v52: 129 pgs: 129 active+clean; 30 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 166 op/s 2023-09-13T04:42:07.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:07 smithi156 ceph-mon[133675]: pgmap v52: 129 pgs: 129 active+clean; 30 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 166 op/s 2023-09-13T04:42:07.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:07 smithi002 ceph-mon[97651]: pgmap v52: 129 pgs: 129 active+clean; 30 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 166 op/s 2023-09-13T04:42:08.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:08 smithi149 ceph-mon[133315]: scrub summary: idle 2023-09-13T04:42:08.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:08 smithi156 ceph-mon[133675]: scrub summary: idle 2023-09-13T04:42:08.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:08 smithi002 ceph-mon[97651]: scrub summary: idle 2023-09-13T04:42:09.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:09 smithi149 ceph-mon[133315]: pgmap v53: 129 pgs: 129 active+clean; 30 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 157 op/s 2023-09-13T04:42:09.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:09 smithi149 ceph-mon[133315]: scrub summary: idle 2023-09-13T04:42:09.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:09 smithi156 ceph-mon[133675]: pgmap v53: 129 pgs: 129 active+clean; 30 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 157 op/s 2023-09-13T04:42:09.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:09 smithi156 ceph-mon[133675]: scrub summary: idle 2023-09-13T04:42:09.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:09 smithi002 ceph-mon[97651]: pgmap v53: 129 pgs: 129 active+clean; 30 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 157 op/s 2023-09-13T04:42:09.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:09 smithi002 ceph-mon[97651]: scrub summary: idle 2023-09-13T04:42:10.594 INFO:tasks.workunit.client.0.smithi002.stdout: 23 45861 891 31048 807 13663 702 2023-09-13T04:42:11.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:11 smithi156 ceph-mon[133675]: pgmap v54: 129 pgs: 129 active+clean; 41 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 175 op/s 2023-09-13T04:42:11.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:11 smithi002 ceph-mon[97651]: pgmap v54: 129 pgs: 129 active+clean; 41 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 175 op/s 2023-09-13T04:42:11.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:11 smithi149 ceph-mon[133315]: pgmap v54: 129 pgs: 129 active+clean; 41 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 175 op/s 2023-09-13T04:42:13.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:13 smithi156 ceph-mon[133675]: pgmap v55: 129 pgs: 129 active+clean; 41 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 108 op/s 2023-09-13T04:42:13.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:13 smithi002 ceph-mon[97651]: pgmap v55: 129 pgs: 129 active+clean; 41 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 108 op/s 2023-09-13T04:42:13.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:13 smithi149 ceph-mon[133315]: pgmap v55: 129 pgs: 129 active+clean; 41 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 108 op/s 2023-09-13T04:42:15.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:15 smithi156 ceph-mon[133675]: pgmap v56: 129 pgs: 129 active+clean; 72 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 248 op/s 2023-09-13T04:42:15.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:15 smithi002 ceph-mon[97651]: pgmap v56: 129 pgs: 129 active+clean; 72 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 248 op/s 2023-09-13T04:42:15.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:15 smithi149 ceph-mon[133315]: pgmap v56: 129 pgs: 129 active+clean; 72 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 248 op/s 2023-09-13T04:42:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.804+0000 7f23ec5ff700 1 -- 172.21.15.2:0/1496941144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23e410f890 msgr2=0x7f23e41081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.804+0000 7f23ec5ff700 1 --2- 172.21.15.2:0/1496941144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23e410f890 0x7f23e41081f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f23e001c0d0 tx=0x7f23e001c3e0 comp rx=0 tx=0).stop 2023-09-13T04:42:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.820+0000 7f23ec5ff700 1 -- 172.21.15.2:0/1496941144 shutdown_connections 2023-09-13T04:42:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.820+0000 7f23ec5ff700 1 --2- 172.21.15.2:0/1496941144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23e411a490 0x7f23e411c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.820+0000 7f23ec5ff700 1 --2- 172.21.15.2:0/1496941144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e410f420 0x7f23e4107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.820+0000 7f23ec5ff700 1 --2- 172.21.15.2:0/1496941144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23e410f890 0x7f23e41081f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.820+0000 7f23ec5ff700 1 -- 172.21.15.2:0/1496941144 >> 172.21.15.2:0/1496941144 conn(0x7f23e406cda0 msgr2=0x7f23e406d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.827+0000 7f23ec5ff700 1 -- 172.21.15.2:0/1496941144 shutdown_connections 2023-09-13T04:42:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.830+0000 7f23ec5ff700 1 -- 172.21.15.2:0/1496941144 wait complete. 2023-09-13T04:42:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.832+0000 7f23ec5ff700 1 Processor -- start 2023-09-13T04:42:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.833+0000 7f23ec5ff700 1 -- start start 2023-09-13T04:42:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.837+0000 7f23ec5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23e410f420 0x7f23e41157e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.837+0000 7f23ec5ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e410f890 0x7f23e4115d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.837+0000 7f23ec5ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23e411a490 0x7f23e41193f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.837+0000 7f23ec5ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e41199c0 con 0x7f23e410f890 2023-09-13T04:42:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.837+0000 7f23ec5ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e4119b30 con 0x7f23e411a490 2023-09-13T04:42:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.837+0000 7f23ec5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e4119cf0 con 0x7f23e410f420 2023-09-13T04:42:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.838+0000 7f23eab9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23e411a490 0x7f23e41193f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.838+0000 7f23eab9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23e411a490 0x7f23e41193f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:49276/0 (socket says 172.21.15.2:49276) 2023-09-13T04:42:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.838+0000 7f23eab9c700 1 -- 172.21.15.2:0/3476075373 learned_addr learned my addr 172.21.15.2:0/3476075373 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.838+0000 7f23eab9c700 1 -- 172.21.15.2:0/3476075373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23e410f420 msgr2=0x7f23e41157e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-13T04:42:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.838+0000 7f23eab9c700 1 --2- 172.21.15.2:0/3476075373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23e410f420 0x7f23e41157e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.838+0000 7f23eab9c700 1 -- 172.21.15.2:0/3476075373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e410f890 msgr2=0x7f23e4115d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.838+0000 7f23eab9c700 1 --2- 172.21.15.2:0/3476075373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e410f890 0x7f23e4115d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.839+0000 7f23eab9c700 1 -- 172.21.15.2:0/3476075373 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e001ad10 con 0x7f23e411a490 2023-09-13T04:42:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.839+0000 7f23eab9c700 1 --2- 172.21.15.2:0/3476075373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23e411a490 0x7f23e41193f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00d950 tx=0x7f23dc00dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.839+0000 7f23d77fe700 1 -- 172.21.15.2:0/3476075373 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc009940 con 0x7f23e411a490 2023-09-13T04:42:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.839+0000 7f23ec5ff700 1 -- 172.21.15.2:0/3476075373 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e4119fd0 con 0x7f23e411a490 2023-09-13T04:42:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.840+0000 7f23ec5ff700 1 -- 172.21.15.2:0/3476075373 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e41aa4d0 con 0x7f23e411a490 2023-09-13T04:42:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.840+0000 7f23d77fe700 1 -- 172.21.15.2:0/3476075373 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23dc004830 con 0x7f23e411a490 2023-09-13T04:42:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.840+0000 7f23d77fe700 1 -- 172.21.15.2:0/3476075373 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc005440 con 0x7f23e411a490 2023-09-13T04:42:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.852+0000 7f23fff6db80 1 Processor -- start 2023-09-13T04:42:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.854+0000 7f23fff6db80 1 -- start start 2023-09-13T04:42:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.855+0000 7f23fff6db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586712c8be0 0x5586712c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.855+0000 7f23fff6db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586712c9500 0x5586712c4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.855+0000 7f23fff6db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586712c4840 0x5586712c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.855+0000 7f23fff6db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55867121d720 con 0x5586712c8be0 2023-09-13T04:42:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.855+0000 7f23fff6db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55867121b040 con 0x5586712c4840 2023-09-13T04:42:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.855+0000 7f23fff6db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558671216930 con 0x5586712c9500 2023-09-13T04:42:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.855+0000 7f23ea39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586712c9500 0x5586712c4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.855+0000 7f23ea39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586712c9500 0x5586712c4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:53114/0 (socket says 172.21.15.2:53114) 2023-09-13T04:42:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.855+0000 7f23ea39b700 1 -- 172.21.15.2:0/2371789713 learned_addr learned my addr 172.21.15.2:0/2371789713 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.855+0000 7f23d77fe700 1 -- 172.21.15.2:0/3476075373 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f23dc010460 con 0x7f23e411a490 2023-09-13T04:42:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.861+0000 7f23e9b9a700 1 --2- 172.21.15.2:0/2371789713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586712c4840 0x5586712c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.861+0000 7f23e9b9a700 1 --2- 172.21.15.2:0/2371789713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586712c8be0 0x5586712c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.861+0000 7f23ea39b700 1 -- 172.21.15.2:0/2371789713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586712c4840 msgr2=0x5586712c4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.861+0000 7f23ea39b700 1 --2- 172.21.15.2:0/2371789713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586712c4840 0x5586712c4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.861+0000 7f23ea39b700 1 -- 172.21.15.2:0/2371789713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586712c8be0 msgr2=0x5586712c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.861+0000 7f23ea39b700 1 --2- 172.21.15.2:0/2371789713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586712c8be0 0x5586712c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.862+0000 7f23ea39b700 1 -- 172.21.15.2:0/2371789713 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586710965d0 con 0x5586712c9500 2023-09-13T04:42:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.862+0000 7f23ea39b700 1 --2- 172.21.15.2:0/2371789713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586712c9500 0x5586712c4300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009830 tx=0x7f23d8009b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9703531b5a7f7ee5 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.863+0000 7f23d77fe700 1 --2- 172.21.15.2:0/3476075373 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f23d00a08f0 0x7f23d00a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.863+0000 7f23d77fe700 1 -- 172.21.15.2:0/3476075373 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f23dc0cbb40 con 0x7f23e411a490 2023-09-13T04:42:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.868+0000 7f23d4bfe700 1 -- 172.21.15.2:0/2371789713 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8011830 con 0x5586712c9500 2023-09-13T04:42:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.868+0000 7f23d4bfe700 1 -- 172.21.15.2:0/2371789713 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23d8011e50 con 0x5586712c9500 2023-09-13T04:42:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.868+0000 7f23d4bfe700 1 -- 172.21.15.2:0/2371789713 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8010380 con 0x5586712c9500 2023-09-13T04:42:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.870+0000 7f23fff6db80 1 -- 172.21.15.2:0/2371789713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586712c9500 msgr2=0x5586712c4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.870+0000 7f23fff6db80 1 --2- 172.21.15.2:0/2371789713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586712c9500 0x5586712c4300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009830 tx=0x7f23d8009b40 comp rx=0 tx=0).stop 2023-09-13T04:42:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.870+0000 7f23fff6db80 1 -- 172.21.15.2:0/2371789713 shutdown_connections 2023-09-13T04:42:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.870+0000 7f23fff6db80 1 --2- 172.21.15.2:0/2371789713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586712c4840 0x5586712c4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.870+0000 7f23fff6db80 1 --2- 172.21.15.2:0/2371789713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586712c8be0 0x5586712c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.870+0000 7f23fff6db80 1 --2- 172.21.15.2:0/2371789713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586712c9500 0x5586712c4300 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009830 tx=0x7f23d8009b40 comp rx=0 tx=0).stop 2023-09-13T04:42:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.870+0000 7f23fff6db80 1 -- 172.21.15.2:0/2371789713 >> 172.21.15.2:0/2371789713 conn(0x558671222c20 msgr2=0x5586712beb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.876+0000 7f23e9b9a700 1 --2- 172.21.15.2:0/3476075373 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f23d00a08f0 0x7f23d00a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.877+0000 7f23fff6db80 1 -- 172.21.15.2:0/2371789713 shutdown_connections 2023-09-13T04:42:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.880+0000 7f23fff6db80 1 -- 172.21.15.2:0/2371789713 wait complete. 2023-09-13T04:42:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.881+0000 7f23fff6db80 1 -- start start 2023-09-13T04:42:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.881+0000 7f23fff6db80 1 Processor -- start 2023-09-13T04:42:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.882+0000 7f23fff6db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586712cdc40 0x5586712ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.882+0000 7f23fff6db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586712ce560 0x5586712d80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.882+0000 7f23fff6db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586712d85e0 0x5586712da9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.882+0000 7f23fff6db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586711b4b50 con 0x5586712cdc40 2023-09-13T04:42:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.882+0000 7f23fff6db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586711b4850 con 0x5586712d85e0 2023-09-13T04:42:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.882+0000 7f23fff6db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586710988c0 con 0x5586712ce560 2023-09-13T04:42:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.882+0000 7f23fff6db80 10 client.0 _ll_register_callbacks cb 0x5586711b2620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:42:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.886+0000 7f23ea39b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586712cdc40 0x5586712ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.886+0000 7f23ea39b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586712cdc40 0x5586712ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33358/0 (socket says 172.21.15.2:33358) 2023-09-13T04:42:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.886+0000 7f23ea39b700 1 -- 172.21.15.2:0/3681167440 learned_addr learned my addr 172.21.15.2:0/3681167440 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.887+0000 7f23eab9c700 1 --2- 172.21.15.2:0/3681167440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586712d85e0 0x5586712da9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.887+0000 7f23ea39b700 1 -- 172.21.15.2:0/3681167440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586712ce560 msgr2=0x5586712d80a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:42:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.887+0000 7f23ea39b700 1 --2- 172.21.15.2:0/3681167440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586712ce560 0x5586712d80a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.887+0000 7f23ea39b700 1 -- 172.21.15.2:0/3681167440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586712d85e0 msgr2=0x5586712da9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.887+0000 7f23ea39b700 1 --2- 172.21.15.2:0/3681167440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586712d85e0 0x5586712da9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.887+0000 7f23ea39b700 1 -- 172.21.15.2:0/3681167440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d8009460 con 0x5586712cdc40 2023-09-13T04:42:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.888+0000 7f23ea39b700 1 --2- 172.21.15.2:0/3681167440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586712cdc40 0x5586712ce020 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f23d8000c00 tx=0x7f23d800bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c26ac2a8d7aba server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.888+0000 7f23cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:42:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.900+0000 7f23e9b9a700 1 --2- 172.21.15.2:0/3476075373 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f23d00a08f0 0x7f23d00a2db0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x5586712bfb60 tx=0x7f23e0005040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.901+0000 7f23cdffb700 1 -- 172.21.15.2:0/3681167440 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d800b5f0 con 0x5586712cdc40 2023-09-13T04:42:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.901+0000 7f23cdffb700 1 -- 172.21.15.2:0/3681167440 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f23d8021070 con 0x5586712cdc40 2023-09-13T04:42:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.901+0000 7f23fff6db80 1 -- 172.21.15.2:0/3681167440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558671289ef0 con 0x5586712cdc40 2023-09-13T04:42:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.906+0000 7f23cdffb700 1 -- 172.21.15.2:0/3681167440 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23d800b750 con 0x5586712cdc40 2023-09-13T04:42:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.908+0000 7f23fff6db80 10 client.16005 fetch_fsmap learned FSMap version 27 2023-09-13T04:42:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.908+0000 7f23fff6db80 1 -- 172.21.15.2:0/3681167440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55867122e740 con 0x5586712cdc40 2023-09-13T04:42:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.909+0000 7f23cdffb700 1 -- 172.21.15.2:0/3681167440 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558671289ef0 con 0x5586712cdc40 2023-09-13T04:42:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.909+0000 7f23cdffb700 1 -- 172.21.15.2:0/3681167440 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f23d801aa80 con 0x5586712cdc40 2023-09-13T04:42:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.909+0000 7f23fff6db80 10 client.16005 fetch_fsmap finished waiting for FSMap version 27 2023-09-13T04:42:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.909+0000 7f23fff6db80 10 client.16005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:42:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.909+0000 7f23fff6db80 20 client.16005 populate_metadata read hostname 'smithi002' 2023-09-13T04:42:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.909+0000 7f23fff6db80 1 --2- 172.21.15.2:0/3681167440 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5586712ded60 0x5586712e1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.909+0000 7f23fff6db80 4 client.16005 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:42:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.909+0000 7f23fff6db80 1 -- 172.21.15.2:0/3681167440 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55867120f910 con 0x5586712ded60 2023-09-13T04:42:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.910+0000 7f23e9b9a700 1 --2- 172.21.15.2:0/3681167440 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5586712ded60 0x5586712e1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.911+0000 7f23cdffb700 10 client.16005 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.913+0000 7f23e9b9a700 1 --2- 172.21.15.2:0/3681167440 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5586712ded60 0x5586712e1180 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16c0f1ec4f924cdb server_cookie=94d915339c9bdf9c in_seq=0 out_seq=0 2023-09-13T04:42:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.918+0000 7f23cdffb700 1 -- 172.21.15.2:0/3681167440 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55867120f910 con 0x5586712ded60 2023-09-13T04:42:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.918+0000 7f23cdffb700 10 client.16005 handle_command_reply: tid=0 2023-09-13T04:42:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.918+0000 7f23fff6db80 1 client.16005 shutdown 2023-09-13T04:42:16.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.922+0000 7f23fff6db80 1 -- 172.21.15.2:0/3681167440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586712cdc40 msgr2=0x5586712ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:16.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.922+0000 7f23fff6db80 1 --2- 172.21.15.2:0/3681167440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586712cdc40 0x5586712ce020 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f23d8000c00 tx=0x7f23d800bb50 comp rx=0 tx=0).stop 2023-09-13T04:42:16.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.922+0000 7f23fff6db80 1 -- 172.21.15.2:0/3681167440 shutdown_connections 2023-09-13T04:42:16.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.922+0000 7f23fff6db80 1 --2- 172.21.15.2:0/3681167440 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5586712ded60 0x5586712e1180 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:16.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.922+0000 7f23fff6db80 1 --2- 172.21.15.2:0/3681167440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586712d85e0 0x5586712da9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:16.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.922+0000 7f23fff6db80 1 --2- 172.21.15.2:0/3681167440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586712cdc40 0x5586712ce020 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.922+0000 7f23fff6db80 1 --2- 172.21.15.2:0/3681167440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586712ce560 0x5586712d80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.922+0000 7f23fff6db80 1 -- 172.21.15.2:0/3681167440 >> 172.21.15.2:0/3681167440 conn(0x558671222c20 msgr2=0x5586712ca330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.923+0000 7f23cdffb700 0 client.16005 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.928+0000 7f23fff6db80 1 -- 172.21.15.2:0/3681167440 shutdown_connections 2023-09-13T04:42:16.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.937+0000 7f23fff6db80 1 -- 172.21.15.2:0/3681167440 wait complete. 2023-09-13T04:42:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.944+0000 7f23fff6db80 20 client.16005 trim_cache size 0 max 16384 2023-09-13T04:42:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.978+0000 7f23fff6db80 1 Processor -- start 2023-09-13T04:42:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.978+0000 7f23fff6db80 1 -- start start 2023-09-13T04:42:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.979+0000 7f23fff6db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586712cdc40 0x5586712e1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.979+0000 7f23fff6db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586712ce560 0x5586712d5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.979+0000 7f23fff6db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586712d85e0 0x5586712b9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.979+0000 7f23fff6db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586712bff20 con 0x5586712ce560 2023-09-13T04:42:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.979+0000 7f23fff6db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586712d5540 con 0x5586712d85e0 2023-09-13T04:42:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.979+0000 7f23fff6db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586712c3740 con 0x5586712cdc40 2023-09-13T04:42:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.979+0000 7f23ea39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586712cdc40 0x5586712e1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.979+0000 7f23ea39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586712cdc40 0x5586712e1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:53140/0 (socket says 172.21.15.2:53140) 2023-09-13T04:42:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.979+0000 7f23ea39b700 1 -- 172.21.15.2:0/1095695701 learned_addr learned my addr 172.21.15.2:0/1095695701 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.979+0000 7f23eab9c700 1 --2- 172.21.15.2:0/1095695701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586712d85e0 0x5586712b9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.980+0000 7f23e9b9a700 1 --2- 172.21.15.2:0/1095695701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586712ce560 0x5586712d5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.980+0000 7f23eab9c700 1 -- 172.21.15.2:0/1095695701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586712cdc40 msgr2=0x5586712e1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.980+0000 7f23eab9c700 1 --2- 172.21.15.2:0/1095695701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586712cdc40 0x5586712e1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.980+0000 7f23eab9c700 1 -- 172.21.15.2:0/1095695701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586712ce560 msgr2=0x5586712d5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.980+0000 7f23eab9c700 1 --2- 172.21.15.2:0/1095695701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586712ce560 0x5586712d5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.980+0000 7f23eab9c700 1 -- 172.21.15.2:0/1095695701 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558671042d80 con 0x5586712d85e0 2023-09-13T04:42:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.981+0000 7f23eab9c700 1 --2- 172.21.15.2:0/1095695701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586712d85e0 0x5586712b9730 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f23dc0c8f20 tx=0x7f23dc0950f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe4582ab11f95f08 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.981+0000 7f23d53ff700 1 -- 172.21.15.2:0/1095695701 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc014020 con 0x5586712d85e0 2023-09-13T04:42:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.981+0000 7f23d53ff700 1 -- 172.21.15.2:0/1095695701 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23dc09cab0 con 0x5586712d85e0 2023-09-13T04:42:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.981+0000 7f23d53ff700 1 -- 172.21.15.2:0/1095695701 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc0983d0 con 0x5586712d85e0 2023-09-13T04:42:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.982+0000 7f23fff6db80 1 -- 172.21.15.2:0/1095695701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586712d85e0 msgr2=0x5586712b9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.982+0000 7f23fff6db80 1 --2- 172.21.15.2:0/1095695701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586712d85e0 0x5586712b9730 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f23dc0c8f20 tx=0x7f23dc0950f0 comp rx=0 tx=0).stop 2023-09-13T04:42:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.982+0000 7f23fff6db80 1 -- 172.21.15.2:0/1095695701 shutdown_connections 2023-09-13T04:42:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.982+0000 7f23fff6db80 1 --2- 172.21.15.2:0/1095695701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586712d85e0 0x5586712b9730 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.982+0000 7f23fff6db80 1 --2- 172.21.15.2:0/1095695701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586712cdc40 0x5586712e1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.982+0000 7f23fff6db80 1 --2- 172.21.15.2:0/1095695701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586712ce560 0x5586712d5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.982+0000 7f23fff6db80 1 -- 172.21.15.2:0/1095695701 >> 172.21.15.2:0/1095695701 conn(0x558671222c20 msgr2=0x5586712c8250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.982+0000 7f23fff6db80 1 -- 172.21.15.2:0/1095695701 shutdown_connections 2023-09-13T04:42:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.982+0000 7f23fff6db80 1 -- 172.21.15.2:0/1095695701 wait complete. 2023-09-13T04:42:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.983+0000 7f23fff6db80 1 -- start start 2023-09-13T04:42:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.983+0000 7f23fff6db80 1 Processor -- start 2023-09-13T04:42:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.984+0000 7f23fff6db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586712cdc40 0x5586712d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.984+0000 7f23fff6db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586712ce560 0x5586712d3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.984+0000 7f23fff6db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586712d85e0 0x558671331b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.984+0000 7f23fff6db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586712d3e20 con 0x5586712cdc40 2023-09-13T04:42:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.984+0000 7f23fff6db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586713320d0 con 0x5586712d85e0 2023-09-13T04:42:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.984+0000 7f23fff6db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558671332210 con 0x5586712ce560 2023-09-13T04:42:16.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.984+0000 7f23fff6db80 10 client.0 _ll_register_callbacks cb 0x5586711b2620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:42:16.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.985+0000 7f23e9b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586712ce560 0x5586712d3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:16.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.985+0000 7f23e9b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586712ce560 0x5586712d3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:53154/0 (socket says 172.21.15.2:53154) 2023-09-13T04:42:16.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.985+0000 7f23e9b9a700 1 -- 172.21.15.2:0/1391968539 learned_addr learned my addr 172.21.15.2:0/1391968539 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:16.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.985+0000 7f23ea39b700 1 --2- 172.21.15.2:0/1391968539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586712cdc40 0x5586712d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.985+0000 7f23e9b9a700 1 -- 172.21.15.2:0/1391968539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586712d85e0 msgr2=0x558671331b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.985+0000 7f23e9b9a700 1 --2- 172.21.15.2:0/1391968539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586712d85e0 0x558671331b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.985+0000 7f23e9b9a700 1 -- 172.21.15.2:0/1391968539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586712cdc40 msgr2=0x5586712d31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.985+0000 7f23e9b9a700 1 --2- 172.21.15.2:0/1391968539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586712cdc40 0x5586712d31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.985+0000 7f23e9b9a700 1 -- 172.21.15.2:0/1391968539 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e41aa4d0 con 0x5586712ce560 2023-09-13T04:42:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.986+0000 7f23e9b9a700 1 --2- 172.21.15.2:0/1391968539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586712ce560 0x5586712d3710 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f23e001c0d0 tx=0x7f23e001a9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=314a4001be383c70 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.986+0000 7f23cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:42:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.986+0000 7f23cdffb700 1 -- 172.21.15.2:0/1391968539 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e001f040 con 0x5586712ce560 2023-09-13T04:42:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.986+0000 7f23fff6db80 1 -- 172.21.15.2:0/1391968539 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23d8011e50 con 0x5586712ce560 2023-09-13T04:42:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.986+0000 7f23cdffb700 1 -- 172.21.15.2:0/1391968539 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f23e0036040 con 0x5586712ce560 2023-09-13T04:42:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.987+0000 7f23cdffb700 1 -- 172.21.15.2:0/1391968539 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23e00402c0 con 0x5586712ce560 2023-09-13T04:42:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.987+0000 7f23fff6db80 10 client.26053 fetch_fsmap learned FSMap version 27 2023-09-13T04:42:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.987+0000 7f23fff6db80 1 -- 172.21.15.2:0/1391968539 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55867102bc00 con 0x5586712ce560 2023-09-13T04:42:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.988+0000 7f23cdffb700 1 -- 172.21.15.2:0/1391968539 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23e0052020 con 0x5586712ce560 2023-09-13T04:42:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.988+0000 7f23cdffb700 1 -- 172.21.15.2:0/1391968539 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f23e00052c0 con 0x5586712ce560 2023-09-13T04:42:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.988+0000 7f23fff6db80 10 client.26053 fetch_fsmap finished waiting for FSMap version 27 2023-09-13T04:42:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.988+0000 7f23fff6db80 10 client.26053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:42:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.988+0000 7f23fff6db80 20 client.26053 populate_metadata read hostname 'smithi002' 2023-09-13T04:42:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.988+0000 7f23fff6db80 1 --2- 172.21.15.2:0/1391968539 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x558671333ab0 0x558671335ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.988+0000 7f23fff6db80 4 client.26053 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:42:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.988+0000 7f23fff6db80 1 -- 172.21.15.2:0/1391968539 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5586711cfff0 con 0x558671333ab0 2023-09-13T04:42:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.989+0000 7f23ea39b700 1 --2- 172.21.15.2:0/1391968539 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x558671333ab0 0x558671335ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.990+0000 7f23ea39b700 1 --2- 172.21.15.2:0/1391968539 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x558671333ab0 0x558671335ed0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb0a268642a9ca05 server_cookie=410d56e321528ff1 in_seq=0 out_seq=0 2023-09-13T04:42:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:16.990+0000 7f23cdffb700 10 client.26053 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.558+0000 7f23cdffb700 1 -- 172.21.15.2:0/1391968539 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5586711cfff0 con 0x558671333ab0 2023-09-13T04:42:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.558+0000 7f23cdffb700 10 client.26053 handle_command_reply: tid=0 2023-09-13T04:42:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.558+0000 7f23fff6db80 1 client.26053 shutdown 2023-09-13T04:42:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.558+0000 7f23fff6db80 1 -- 172.21.15.2:0/1391968539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586712ce560 msgr2=0x5586712d3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.558+0000 7f23fff6db80 1 --2- 172.21.15.2:0/1391968539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586712ce560 0x5586712d3710 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f23e001c0d0 tx=0x7f23e001a9f0 comp rx=0 tx=0).stop 2023-09-13T04:42:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.558+0000 7f23fff6db80 1 -- 172.21.15.2:0/1391968539 shutdown_connections 2023-09-13T04:42:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.558+0000 7f23fff6db80 1 --2- 172.21.15.2:0/1391968539 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x558671333ab0 0x558671335ed0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.558+0000 7f23fff6db80 1 --2- 172.21.15.2:0/1391968539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586712d85e0 0x558671331b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.558+0000 7f23fff6db80 1 --2- 172.21.15.2:0/1391968539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586712cdc40 0x5586712d31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.558+0000 7f23fff6db80 1 --2- 172.21.15.2:0/1391968539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586712ce560 0x5586712d3710 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.558+0000 7f23cdffb700 0 client.26053 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.558+0000 7f23fff6db80 1 -- 172.21.15.2:0/1391968539 >> 172.21.15.2:0/1391968539 conn(0x558671222c20 msgr2=0x5586712bea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.558+0000 7f23fff6db80 1 -- 172.21.15.2:0/1391968539 shutdown_connections 2023-09-13T04:42:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.559+0000 7f23fff6db80 1 -- 172.21.15.2:0/1391968539 wait complete. 2023-09-13T04:42:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.559+0000 7f23fff6db80 20 client.26053 trim_cache size 0 max 16384 2023-09-13T04:42:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:42:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:42:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:42:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:42:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.559+0000 7f23ec5ff700 1 -- 172.21.15.2:0/3476075373 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f23d00a08f0 msgr2=0x7f23d00a2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.559+0000 7f23ec5ff700 1 --2- 172.21.15.2:0/3476075373 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f23d00a08f0 0x7f23d00a2db0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x5586712bfb60 tx=0x7f23e0005040 comp rx=0 tx=0).stop 2023-09-13T04:42:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.559+0000 7f23ec5ff700 1 -- 172.21.15.2:0/3476075373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23e411a490 msgr2=0x7f23e41193f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.559+0000 7f23ec5ff700 1 --2- 172.21.15.2:0/3476075373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23e411a490 0x7f23e41193f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00d950 tx=0x7f23dc00dd10 comp rx=0 tx=0).stop 2023-09-13T04:42:17.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.560+0000 7f23ec5ff700 1 -- 172.21.15.2:0/3476075373 shutdown_connections 2023-09-13T04:42:17.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.560+0000 7f23ec5ff700 1 --2- 172.21.15.2:0/3476075373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23e411a490 0x7f23e41193f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:17.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.560+0000 7f23ec5ff700 1 --2- 172.21.15.2:0/3476075373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23e410f420 0x7f23e41157e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:17.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.560+0000 7f23ec5ff700 1 --2- 172.21.15.2:0/3476075373 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f23d00a08f0 0x7f23d00a2db0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.560+0000 7f23ec5ff700 1 --2- 172.21.15.2:0/3476075373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e410f890 0x7f23e4115d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.560+0000 7f23ec5ff700 1 -- 172.21.15.2:0/3476075373 >> 172.21.15.2:0/3476075373 conn(0x7f23e406cda0 msgr2=0x7f23e40716b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.560+0000 7f23ec5ff700 1 -- 172.21.15.2:0/3476075373 shutdown_connections 2023-09-13T04:42:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:17.560+0000 7f23ec5ff700 1 -- 172.21.15.2:0/3476075373 wait complete. 2023-09-13T04:42:17.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:17 smithi002 ceph-mon[97651]: pgmap v57: 129 pgs: 129 active+clean; 72 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 157 op/s 2023-09-13T04:42:17.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:17 smithi002 ceph-mon[97651]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:42:17.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:17 smithi149 ceph-mon[133315]: pgmap v57: 129 pgs: 129 active+clean; 72 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 157 op/s 2023-09-13T04:42:17.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:17 smithi149 ceph-mon[133315]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:42:17.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:17 smithi156 ceph-mon[133675]: pgmap v57: 129 pgs: 129 active+clean; 72 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 157 op/s 2023-09-13T04:42:17.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:17 smithi156 ceph-mon[133675]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:42:18.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:18 smithi002 ceph-mon[97651]: pgmap v58: 129 pgs: 129 active+clean; 72 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 157 op/s 2023-09-13T04:42:18.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:18 smithi149 ceph-mon[133315]: pgmap v58: 129 pgs: 129 active+clean; 72 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 157 op/s 2023-09-13T04:42:18.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:18 smithi156 ceph-mon[133675]: pgmap v58: 129 pgs: 129 active+clean; 72 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 157 op/s 2023-09-13T04:42:20.595 INFO:tasks.workunit.client.0.smithi002.stdout: 37 58606 667 37577 711 13315 393 2023-09-13T04:42:20.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:20 smithi149 ceph-mon[133315]: pgmap v59: 129 pgs: 129 active+clean; 116 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.6 MiB/s wr, 478 op/s 2023-09-13T04:42:20.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:20 smithi156 ceph-mon[133675]: pgmap v59: 129 pgs: 129 active+clean; 116 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.6 MiB/s wr, 478 op/s 2023-09-13T04:42:20.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:20 smithi002 ceph-mon[97651]: pgmap v59: 129 pgs: 129 active+clean; 116 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.6 MiB/s wr, 478 op/s 2023-09-13T04:42:22.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:21 smithi156 ceph-mon[133675]: pgmap v60: 129 pgs: 129 active+clean; 116 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.7 MiB/s wr, 460 op/s 2023-09-13T04:42:22.266 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:21 smithi002 ceph-mon[97651]: pgmap v60: 129 pgs: 129 active+clean; 116 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.7 MiB/s wr, 460 op/s 2023-09-13T04:42:22.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:21 smithi149 ceph-mon[133315]: pgmap v60: 129 pgs: 129 active+clean; 116 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.7 MiB/s wr, 460 op/s 2023-09-13T04:42:24.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:23 smithi156 ceph-mon[133675]: pgmap v61: 129 pgs: 129 active+clean; 122 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.2 MiB/s wr, 501 op/s 2023-09-13T04:42:24.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:23 smithi002 ceph-mon[97651]: pgmap v61: 129 pgs: 129 active+clean; 122 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.2 MiB/s wr, 501 op/s 2023-09-13T04:42:24.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:23 smithi149 ceph-mon[133315]: pgmap v61: 129 pgs: 129 active+clean; 122 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.2 MiB/s wr, 501 op/s 2023-09-13T04:42:26.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:25 smithi156 ceph-mon[133675]: pgmap v62: 129 pgs: 129 active+clean; 122 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 361 op/s 2023-09-13T04:42:26.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:25 smithi002 ceph-mon[97651]: pgmap v62: 129 pgs: 129 active+clean; 122 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 361 op/s 2023-09-13T04:42:26.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:25 smithi149 ceph-mon[133315]: pgmap v62: 129 pgs: 129 active+clean; 122 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 361 op/s 2023-09-13T04:42:28.264 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:27 smithi002 ceph-mon[97651]: pgmap v63: 129 pgs: 129 active+clean; 122 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 361 op/s 2023-09-13T04:42:28.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:27 smithi149 ceph-mon[133315]: pgmap v63: 129 pgs: 129 active+clean; 122 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 361 op/s 2023-09-13T04:42:28.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:27 smithi156 ceph-mon[133675]: pgmap v63: 129 pgs: 129 active+clean; 122 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 361 op/s 2023-09-13T04:42:29.360 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T04:42:30.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.464+0000 7f1644dff700 1 -- 172.21.15.2:0/3845523296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f164011a490 msgr2=0x7f164011c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:30.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.464+0000 7f1644dff700 1 --2- 172.21.15.2:0/3845523296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f164011a490 0x7f164011c880 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f163800b0a0 tx=0x7f163800b3b0 comp rx=0 tx=0).stop 2023-09-13T04:42:30.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.469+0000 7f1644dff700 1 -- 172.21.15.2:0/3845523296 shutdown_connections 2023-09-13T04:42:30.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.469+0000 7f1644dff700 1 --2- 172.21.15.2:0/3845523296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f164011a490 0x7f164011c880 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:30.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.469+0000 7f1644dff700 1 --2- 172.21.15.2:0/3845523296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f164010f440 0x7f1640107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:30.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.469+0000 7f1644dff700 1 --2- 172.21.15.2:0/3845523296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16401081f0 0x7f1640108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:30.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.469+0000 7f1644dff700 1 -- 172.21.15.2:0/3845523296 >> 172.21.15.2:0/3845523296 conn(0x7f164006cfa0 msgr2=0x7f164006d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:30.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.473+0000 7f1644dff700 1 -- 172.21.15.2:0/3845523296 shutdown_connections 2023-09-13T04:42:30.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.473+0000 7f1644dff700 1 -- 172.21.15.2:0/3845523296 wait complete. 2023-09-13T04:42:30.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.475+0000 7f1644dff700 1 Processor -- start 2023-09-13T04:42:30.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.475+0000 7f1644dff700 1 -- start start 2023-09-13T04:42:30.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.475+0000 7f1644dff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f164010f440 0x7f16401a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:30.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.475+0000 7f1644dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f164011a490 0x7f16401a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:30.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.476+0000 7f1644dff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16401aa7c0 0x7f16401aac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:30.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.476+0000 7f1644dff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16401ab240 con 0x7f164010f440 2023-09-13T04:42:30.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.476+0000 7f1644dff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16401ab380 con 0x7f16401aa7c0 2023-09-13T04:42:30.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.476+0000 7f1644dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16401ab4f0 con 0x7f164011a490 2023-09-13T04:42:30.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.476+0000 7f163dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f164011a490 0x7f16401a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:30.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.476+0000 7f163dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f164011a490 0x7f16401a6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:50432/0 (socket says 172.21.15.2:50432) 2023-09-13T04:42:30.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.476+0000 7f163dd9b700 1 -- 172.21.15.2:0/2333659648 learned_addr learned my addr 172.21.15.2:0/2333659648 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:30.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.488+0000 7f163ed9d700 1 --2- 172.21.15.2:0/2333659648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16401aa7c0 0x7f16401aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:30.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.488+0000 7f163dd9b700 1 -- 172.21.15.2:0/2333659648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16401aa7c0 msgr2=0x7f16401aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:30.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.488+0000 7f163dd9b700 1 --2- 172.21.15.2:0/2333659648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16401aa7c0 0x7f16401aac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:30.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.488+0000 7f163dd9b700 1 -- 172.21.15.2:0/2333659648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f164010f440 msgr2=0x7f16401a5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:30.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.488+0000 7f163dd9b700 1 --2- 172.21.15.2:0/2333659648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f164010f440 0x7f16401a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:30.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.488+0000 7f163dd9b700 1 -- 172.21.15.2:0/2333659648 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1638009d00 con 0x7f164011a490 2023-09-13T04:42:30.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.507+0000 7f163dd9b700 1 --2- 172.21.15.2:0/2333659648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f164011a490 0x7f16401a6360 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f163000b480 tx=0x7f163000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:30.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.523+0000 7f162f7fe700 1 -- 172.21.15.2:0/2333659648 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1630011830 con 0x7f164011a490 2023-09-13T04:42:30.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.523+0000 7f162f7fe700 1 -- 172.21.15.2:0/2333659648 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1630011e50 con 0x7f164011a490 2023-09-13T04:42:30.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.523+0000 7f162f7fe700 1 -- 172.21.15.2:0/2333659648 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163000f2e0 con 0x7f164011a490 2023-09-13T04:42:30.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.523+0000 7f1644dff700 1 -- 172.21.15.2:0/2333659648 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164010fc70 con 0x7f164011a490 2023-09-13T04:42:30.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.523+0000 7f1644dff700 1 -- 172.21.15.2:0/2333659648 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1640110130 con 0x7f164011a490 2023-09-13T04:42:30.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.533+0000 7f1658768b80 1 Processor -- start 2023-09-13T04:42:30.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.549+0000 7f1658768b80 1 -- start start 2023-09-13T04:42:30.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.550+0000 7f1658768b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561076864fe0 0x5610768653c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:30.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.550+0000 7f1658768b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561076865900 0x56107685f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:30.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.550+0000 7f1658768b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107685fa00 0x56107685fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:30.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.550+0000 7f1658768b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610767b8780 con 0x561076864fe0 2023-09-13T04:42:30.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.550+0000 7f1658768b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610767b60a0 con 0x561076865900 2023-09-13T04:42:30.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.550+0000 7f1658768b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610767b1990 con 0x56107685fa00 2023-09-13T04:42:30.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.563+0000 7f163e59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561076865900 0x56107685f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:30.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.571+0000 7f163e59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561076865900 0x56107685f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:60168/0 (socket says 172.21.15.2:60168) 2023-09-13T04:42:30.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.571+0000 7f163e59c700 1 -- 172.21.15.2:0/3659937162 learned_addr learned my addr 172.21.15.2:0/3659937162 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:30.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.576+0000 7f163ed9d700 1 --2- 172.21.15.2:0/3659937162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561076864fe0 0x5610768653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:30.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.579+0000 7f163e59c700 1 -- 172.21.15.2:0/3659937162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107685fa00 msgr2=0x56107685fe60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:42:30.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.579+0000 7f163e59c700 1 --2- 172.21.15.2:0/3659937162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107685fa00 0x56107685fe60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:30.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.579+0000 7f163e59c700 1 -- 172.21.15.2:0/3659937162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561076864fe0 msgr2=0x5610768653c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:30.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.579+0000 7f163e59c700 1 --2- 172.21.15.2:0/3659937162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561076864fe0 0x5610768653c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:30.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.579+0000 7f163e59c700 1 -- 172.21.15.2:0/3659937162 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561076631630 con 0x561076865900 2023-09-13T04:42:30.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.583+0000 7f163e59c700 1 --2- 172.21.15.2:0/3659937162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561076865900 0x56107685f4c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f163400bb90 tx=0x7f163400bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ce78d3e183797ef server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:30.601 INFO:tasks.workunit.client.0.smithi002.stdout: 44 516424 365 347725 343 126193 848 2023-09-13T04:42:30.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.621+0000 7f162f7fe700 1 -- 172.21.15.2:0/2333659648 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f1630011990 con 0x7f164011a490 2023-09-13T04:42:30.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.624+0000 7f162f7fe700 1 --2- 172.21.15.2:0/2333659648 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f16280a09c0 0x7f16280a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:30.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.624+0000 7f162f7fe700 1 -- 172.21.15.2:0/2333659648 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f16300cbc50 con 0x7f164011a490 2023-09-13T04:42:30.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.627+0000 7f163ed9d700 1 --2- 172.21.15.2:0/2333659648 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f16280a09c0 0x7f16280a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:30.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:30 smithi149 ceph-mon[133315]: pgmap v64: 129 pgs: 129 active+clean; 156 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.5 MiB/s wr, 517 op/s 2023-09-13T04:42:30.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:30.654+0000 7f163ed9d700 1 --2- 172.21.15.2:0/2333659648 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f16280a09c0 0x7f16280a2e80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f16380060b0 tx=0x7f1638006000 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:30.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:30 smithi156 ceph-mon[133675]: pgmap v64: 129 pgs: 129 active+clean; 156 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.5 MiB/s wr, 517 op/s 2023-09-13T04:42:30.786 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:30 smithi002 ceph-mon[97651]: pgmap v64: 129 pgs: 129 active+clean; 156 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.5 MiB/s wr, 517 op/s 2023-09-13T04:42:31.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.021+0000 7f162cff9700 1 -- 172.21.15.2:0/3659937162 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163400d8f0 con 0x561076865900 2023-09-13T04:42:31.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.021+0000 7f162cff9700 1 -- 172.21.15.2:0/3659937162 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1634004830 con 0x561076865900 2023-09-13T04:42:31.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.021+0000 7f162cff9700 1 -- 172.21.15.2:0/3659937162 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16340054e0 con 0x561076865900 2023-09-13T04:42:31.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.022+0000 7f1658768b80 1 -- 172.21.15.2:0/3659937162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561076865900 msgr2=0x56107685f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:31.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.022+0000 7f1658768b80 1 --2- 172.21.15.2:0/3659937162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561076865900 0x56107685f4c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f163400bb90 tx=0x7f163400bea0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.022+0000 7f1658768b80 1 -- 172.21.15.2:0/3659937162 shutdown_connections 2023-09-13T04:42:31.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.022+0000 7f1658768b80 1 --2- 172.21.15.2:0/3659937162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107685fa00 0x56107685fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.022+0000 7f1658768b80 1 --2- 172.21.15.2:0/3659937162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561076864fe0 0x5610768653c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.022+0000 7f1658768b80 1 --2- 172.21.15.2:0/3659937162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561076865900 0x56107685f4c0 secure :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f163400bb90 tx=0x7f163400bea0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.022+0000 7f1658768b80 1 -- 172.21.15.2:0/3659937162 >> 172.21.15.2:0/3659937162 conn(0x5610767bdc80 msgr2=0x561076859f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:31.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.031+0000 7f1658768b80 1 -- 172.21.15.2:0/3659937162 shutdown_connections 2023-09-13T04:42:31.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.031+0000 7f1658768b80 1 -- 172.21.15.2:0/3659937162 wait complete. 2023-09-13T04:42:31.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.032+0000 7f1658768b80 1 -- start start 2023-09-13T04:42:31.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.032+0000 7f1658768b80 1 Processor -- start 2023-09-13T04:42:31.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.050+0000 7f1658768b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107685fa00 0x561076868e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:31.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.050+0000 7f1658768b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561076864fe0 0x561076869380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:31.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.050+0000 7f1658768b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610768698c0 0x561076875360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:31.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.050+0000 7f1658768b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107674fbb0 con 0x561076864fe0 2023-09-13T04:42:31.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.050+0000 7f1658768b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107674f8b0 con 0x5610768698c0 2023-09-13T04:42:31.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.050+0000 7f1658768b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561076633920 con 0x56107685fa00 2023-09-13T04:42:31.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.050+0000 7f1658768b80 10 client.0 _ll_register_callbacks cb 0x56107674d680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:42:31.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.051+0000 7f163e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107685fa00 0x561076868e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:31.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.055+0000 7f163ed9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610768698c0 0x561076875360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:31.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.055+0000 7f163ed9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610768698c0 0x561076875360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:60176/0 (socket says 172.21.15.2:60176) 2023-09-13T04:42:31.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.055+0000 7f163ed9d700 1 -- 172.21.15.2:0/3517438109 learned_addr learned my addr 172.21.15.2:0/3517438109 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:31.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.055+0000 7f163dd9b700 1 --2- 172.21.15.2:0/3517438109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561076864fe0 0x561076869380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:31.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.061+0000 7f163e59c700 1 -- 172.21.15.2:0/3517438109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610768698c0 msgr2=0x561076875360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:31.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.061+0000 7f163e59c700 1 --2- 172.21.15.2:0/3517438109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610768698c0 0x561076875360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.061+0000 7f163e59c700 1 -- 172.21.15.2:0/3517438109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561076864fe0 msgr2=0x561076869380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:31.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.061+0000 7f163e59c700 1 --2- 172.21.15.2:0/3517438109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561076864fe0 0x561076869380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.061+0000 7f163e59c700 1 -- 172.21.15.2:0/3517438109 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163400b7c0 con 0x56107685fa00 2023-09-13T04:42:31.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.083+0000 7f163e59c700 1 --2- 172.21.15.2:0/3517438109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107685fa00 0x561076868e40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x56107685be40 tx=0x7f1634004bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0f0c061b80d7b4 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:31.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.132+0000 7f1625ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:42:31.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.132+0000 7f1625ffb700 1 -- 172.21.15.2:0/3517438109 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163400db00 con 0x56107685fa00 2023-09-13T04:42:31.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.132+0000 7f1625ffb700 1 -- 172.21.15.2:0/3517438109 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f163400dc60 con 0x56107685fa00 2023-09-13T04:42:31.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.132+0000 7f1658768b80 1 -- 172.21.15.2:0/3517438109 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561076852670 con 0x56107685fa00 2023-09-13T04:42:31.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.239+0000 7f1625ffb700 1 -- 172.21.15.2:0/3517438109 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f163400de20 con 0x56107685fa00 2023-09-13T04:42:31.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.239+0000 7f1658768b80 10 client.26062 fetch_fsmap learned FSMap version 27 2023-09-13T04:42:31.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.239+0000 7f1658768b80 1 -- 172.21.15.2:0/3517438109 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5610767c97a0 con 0x56107685fa00 2023-09-13T04:42:31.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.250+0000 7f1625ffb700 1 -- 172.21.15.2:0/3517438109 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561076852670 con 0x56107685fa00 2023-09-13T04:42:31.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.250+0000 7f1625ffb700 1 -- 172.21.15.2:0/3517438109 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f1634025820 con 0x56107685fa00 2023-09-13T04:42:31.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.251+0000 7f1658768b80 10 client.26062 fetch_fsmap finished waiting for FSMap version 27 2023-09-13T04:42:31.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.251+0000 7f1658768b80 10 client.26062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:42:31.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.251+0000 7f1658768b80 20 client.26062 populate_metadata read hostname 'smithi002' 2023-09-13T04:42:31.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.251+0000 7f1658768b80 1 --2- 172.21.15.2:0/3517438109 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5610768796a0 0x56107687bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:31.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.251+0000 7f1658768b80 4 client.26062 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:42:31.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.251+0000 7f1658768b80 1 -- 172.21.15.2:0/3517438109 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5610767aa970 con 0x5610768796a0 2023-09-13T04:42:31.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.251+0000 7f163dd9b700 1 --2- 172.21.15.2:0/3517438109 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5610768796a0 0x56107687bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:31.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.254+0000 7f163dd9b700 1 --2- 172.21.15.2:0/3517438109 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5610768796a0 0x56107687bac0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=148cb1e16fb2266a server_cookie=5cfcca65bca61977 in_seq=0 out_seq=0 2023-09-13T04:42:31.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.254+0000 7f1625ffb700 10 client.26062 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:31.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.256+0000 7f1625ffb700 1 -- 172.21.15.2:0/3517438109 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5610767aa970 con 0x5610768796a0 2023-09-13T04:42:31.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.256+0000 7f1625ffb700 10 client.26062 handle_command_reply: tid=0 2023-09-13T04:42:31.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.256+0000 7f1658768b80 1 client.26062 shutdown 2023-09-13T04:42:31.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.272+0000 7f1658768b80 1 -- 172.21.15.2:0/3517438109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107685fa00 msgr2=0x561076868e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:31.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.272+0000 7f1658768b80 1 --2- 172.21.15.2:0/3517438109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107685fa00 0x561076868e40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x56107685be40 tx=0x7f1634004bd0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.272+0000 7f1658768b80 1 -- 172.21.15.2:0/3517438109 shutdown_connections 2023-09-13T04:42:31.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.272+0000 7f1658768b80 1 --2- 172.21.15.2:0/3517438109 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5610768796a0 0x56107687bac0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.272+0000 7f1658768b80 1 --2- 172.21.15.2:0/3517438109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610768698c0 0x561076875360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.272+0000 7f1658768b80 1 --2- 172.21.15.2:0/3517438109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107685fa00 0x561076868e40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.272+0000 7f1658768b80 1 --2- 172.21.15.2:0/3517438109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561076864fe0 0x561076869380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.272+0000 7f1658768b80 1 -- 172.21.15.2:0/3517438109 >> 172.21.15.2:0/3517438109 conn(0x5610767bdc80 msgr2=0x5610768547f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:31.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.272+0000 7f1625ffb700 0 client.26062 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:31.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.286+0000 7f1658768b80 1 -- 172.21.15.2:0/3517438109 shutdown_connections 2023-09-13T04:42:31.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.286+0000 7f1658768b80 1 -- 172.21.15.2:0/3517438109 wait complete. 2023-09-13T04:42:31.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.287+0000 7f1658768b80 20 client.26062 trim_cache size 0 max 16384 2023-09-13T04:42:31.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.528+0000 7f1658768b80 1 Processor -- start 2023-09-13T04:42:31.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.541+0000 7f1658768b80 1 -- start start 2023-09-13T04:42:31.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.542+0000 7f1658768b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107685fa00 0x5610768702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:31.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.542+0000 7f1658768b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561076864fe0 0x5610768548f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:31.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.542+0000 7f1658768b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610768698c0 0x561076864660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:31.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.542+0000 7f1658768b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610768707f0 con 0x5610768698c0 2023-09-13T04:42:31.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.542+0000 7f1658768b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610768603e0 con 0x56107685fa00 2023-09-13T04:42:31.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.542+0000 7f1658768b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561076867440 con 0x561076864fe0 2023-09-13T04:42:31.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.542+0000 7f163dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561076864fe0 0x5610768548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:31.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.542+0000 7f163e59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107685fa00 0x5610768702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:31.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.543+0000 7f163e59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107685fa00 0x5610768702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:60184/0 (socket says 172.21.15.2:60184) 2023-09-13T04:42:31.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.543+0000 7f163e59c700 1 -- 172.21.15.2:0/2081224354 learned_addr learned my addr 172.21.15.2:0/2081224354 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:31.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.544+0000 7f163ed9d700 1 --2- 172.21.15.2:0/2081224354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610768698c0 0x561076864660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:31.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.549+0000 7f163dd9b700 1 -- 172.21.15.2:0/2081224354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107685fa00 msgr2=0x5610768702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:31.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.549+0000 7f163dd9b700 1 --2- 172.21.15.2:0/2081224354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107685fa00 0x5610768702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.549+0000 7f163dd9b700 1 -- 172.21.15.2:0/2081224354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610768698c0 msgr2=0x561076864660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:31.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.549+0000 7f163dd9b700 1 --2- 172.21.15.2:0/2081224354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610768698c0 0x561076864660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.549+0000 7f163dd9b700 1 -- 172.21.15.2:0/2081224354 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610768806b0 con 0x561076864fe0 2023-09-13T04:42:31.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.558+0000 7f163dd9b700 1 --2- 172.21.15.2:0/2081224354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561076864fe0 0x5610768548f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f16300c9140 tx=0x7f1630095310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85314d2d3ac0aa2c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.615+0000 7f162d7fa700 1 -- 172.21.15.2:0/2081224354 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16300a0f10 con 0x561076864fe0 2023-09-13T04:42:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.615+0000 7f162d7fa700 1 -- 172.21.15.2:0/2081224354 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1630096fe0 con 0x561076864fe0 2023-09-13T04:42:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.615+0000 7f162d7fa700 1 -- 172.21.15.2:0/2081224354 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16300985f0 con 0x561076864fe0 2023-09-13T04:42:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.619+0000 7f1658768b80 1 -- 172.21.15.2:0/2081224354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561076864fe0 msgr2=0x5610768548f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.619+0000 7f1658768b80 1 --2- 172.21.15.2:0/2081224354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561076864fe0 0x5610768548f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f16300c9140 tx=0x7f1630095310 comp rx=0 tx=0).stop 2023-09-13T04:42:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.620+0000 7f1658768b80 1 -- 172.21.15.2:0/2081224354 shutdown_connections 2023-09-13T04:42:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.620+0000 7f1658768b80 1 --2- 172.21.15.2:0/2081224354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610768698c0 0x561076864660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.620+0000 7f1658768b80 1 --2- 172.21.15.2:0/2081224354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107685fa00 0x5610768702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.620+0000 7f1658768b80 1 --2- 172.21.15.2:0/2081224354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561076864fe0 0x5610768548f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.620+0000 7f1658768b80 1 -- 172.21.15.2:0/2081224354 >> 172.21.15.2:0/2081224354 conn(0x5610767bdc80 msgr2=0x561076863f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.620+0000 7f1658768b80 1 -- 172.21.15.2:0/2081224354 shutdown_connections 2023-09-13T04:42:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.621+0000 7f1658768b80 1 -- 172.21.15.2:0/2081224354 wait complete. 2023-09-13T04:42:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.621+0000 7f1658768b80 1 -- start start 2023-09-13T04:42:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.622+0000 7f1658768b80 1 Processor -- start 2023-09-13T04:42:31.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.622+0000 7f1658768b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107685fa00 0x5610768c1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:31.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.622+0000 7f1658768b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561076864fe0 0x5610768c1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:31.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.622+0000 7f1658768b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610768698c0 0x5610768cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:31.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.622+0000 7f1658768b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610768c2270 con 0x561076864fe0 2023-09-13T04:42:31.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.622+0000 7f1658768b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610768cc9e0 con 0x5610768698c0 2023-09-13T04:42:31.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.622+0000 7f1658768b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610768ccb20 con 0x56107685fa00 2023-09-13T04:42:31.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.622+0000 7f1658768b80 10 client.0 _ll_register_callbacks cb 0x56107674d680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:42:31.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.624+0000 7f163dd9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561076864fe0 0x5610768c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:31.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.624+0000 7f163dd9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561076864fe0 0x5610768c1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50078/0 (socket says 172.21.15.2:50078) 2023-09-13T04:42:31.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.624+0000 7f163dd9b700 1 -- 172.21.15.2:0/3554229119 learned_addr learned my addr 172.21.15.2:0/3554229119 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:31.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.626+0000 7f163ed9d700 1 --2- 172.21.15.2:0/3554229119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610768698c0 0x5610768cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:31.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.626+0000 7f163e59c700 1 --2- 172.21.15.2:0/3554229119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107685fa00 0x5610768c1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.638+0000 7f163ed9d700 1 -- 172.21.15.2:0/3554229119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107685fa00 msgr2=0x5610768c1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.638+0000 7f163ed9d700 1 --2- 172.21.15.2:0/3554229119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107685fa00 0x5610768c1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.638+0000 7f163ed9d700 1 -- 172.21.15.2:0/3554229119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561076864fe0 msgr2=0x5610768c1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.638+0000 7f163ed9d700 1 --2- 172.21.15.2:0/3554229119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561076864fe0 0x5610768c1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.638+0000 7f163ed9d700 1 -- 172.21.15.2:0/3554229119 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1640110130 con 0x5610768698c0 2023-09-13T04:42:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.638+0000 7f163ed9d700 1 --2- 172.21.15.2:0/3554229119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610768698c0 0x5610768cc4a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f163800b070 tx=0x7f163800b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a31399d42b8a9bc server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.639+0000 7f1625ffb700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:42:31.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.641+0000 7f1625ffb700 1 -- 172.21.15.2:0/3554229119 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163800e040 con 0x5610768698c0 2023-09-13T04:42:31.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.641+0000 7f1625ffb700 1 -- 172.21.15.2:0/3554229119 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f1638022040 con 0x5610768698c0 2023-09-13T04:42:31.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.642+0000 7f1658768b80 1 -- 172.21.15.2:0/3554229119 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1634004830 con 0x5610768698c0 2023-09-13T04:42:31.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.650+0000 7f1625ffb700 1 -- 172.21.15.2:0/3554229119 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f163801ccb0 con 0x5610768698c0 2023-09-13T04:42:31.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.656+0000 7f1658768b80 10 client.26051 fetch_fsmap learned FSMap version 27 2023-09-13T04:42:31.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.656+0000 7f1658768b80 1 -- 172.21.15.2:0/3554229119 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5610765ddde0 con 0x5610768698c0 2023-09-13T04:42:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.657+0000 7f1625ffb700 1 -- 172.21.15.2:0/3554229119 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1634004830 con 0x5610768698c0 2023-09-13T04:42:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.657+0000 7f1625ffb700 1 -- 172.21.15.2:0/3554229119 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f163802a4a0 con 0x5610768698c0 2023-09-13T04:42:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.658+0000 7f1658768b80 10 client.26051 fetch_fsmap finished waiting for FSMap version 27 2023-09-13T04:42:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.658+0000 7f1658768b80 10 client.26051 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:42:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.658+0000 7f1658768b80 20 client.26051 populate_metadata read hostname 'smithi002' 2023-09-13T04:42:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.658+0000 7f1658768b80 1 --2- 172.21.15.2:0/3554229119 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5610768ce200 0x5610768d0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.658+0000 7f1658768b80 4 client.26051 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-09-13T04:42:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.658+0000 7f1658768b80 1 -- 172.21.15.2:0/3554229119 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56107676b050 con 0x5610768ce200 2023-09-13T04:42:31.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.664+0000 7f163e59c700 1 --2- 172.21.15.2:0/3554229119 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5610768ce200 0x5610768d0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:31.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.674+0000 7f163e59c700 1 --2- 172.21.15.2:0/3554229119 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5610768ce200 0x5610768d0620 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32fd8908fa69ef34 server_cookie=60e078ee5ec66a69 in_seq=0 out_seq=0 2023-09-13T04:42:31.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.675+0000 7f1625ffb700 10 client.26051 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:31.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.687+0000 7f1625ffb700 1 -- 172.21.15.2:0/3554229119 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56107676b050 con 0x5610768ce200 2023-09-13T04:42:31.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.687+0000 7f1625ffb700 10 client.26051 handle_command_reply: tid=0 2023-09-13T04:42:31.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.687+0000 7f1658768b80 1 client.26051 shutdown 2023-09-13T04:42:31.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.696+0000 7f1658768b80 1 -- 172.21.15.2:0/3554229119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610768698c0 msgr2=0x5610768cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:31.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.696+0000 7f1658768b80 1 --2- 172.21.15.2:0/3554229119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610768698c0 0x5610768cc4a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f163800b070 tx=0x7f163800b3b0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.696+0000 7f1658768b80 1 -- 172.21.15.2:0/3554229119 shutdown_connections 2023-09-13T04:42:31.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.696+0000 7f1658768b80 1 --2- 172.21.15.2:0/3554229119 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5610768ce200 0x5610768d0620 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.696+0000 7f1658768b80 1 --2- 172.21.15.2:0/3554229119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610768698c0 0x5610768cc4a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.696+0000 7f1658768b80 1 --2- 172.21.15.2:0/3554229119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107685fa00 0x5610768c1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.696+0000 7f1658768b80 1 --2- 172.21.15.2:0/3554229119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561076864fe0 0x5610768c1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.696+0000 7f1658768b80 1 -- 172.21.15.2:0/3554229119 >> 172.21.15.2:0/3554229119 conn(0x5610767bdc80 msgr2=0x561076874a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:31.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.696+0000 7f1625ffb700 0 client.26051 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:31.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.706+0000 7f1658768b80 1 -- 172.21.15.2:0/3554229119 shutdown_connections 2023-09-13T04:42:31.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.714+0000 7f1658768b80 1 -- 172.21.15.2:0/3554229119 wait complete. 2023-09-13T04:42:31.736 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:42:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2023-09-13T04:42:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2023-09-13T04:42:31.737 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:42:31.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.733+0000 7f1658768b80 20 client.26051 trim_cache size 0 max 16384 2023-09-13T04:42:31.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.753+0000 7f1644dff700 1 -- 172.21.15.2:0/2333659648 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f16280a09c0 msgr2=0x7f16280a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:31.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.753+0000 7f1644dff700 1 --2- 172.21.15.2:0/2333659648 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f16280a09c0 0x7f16280a2e80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f16380060b0 tx=0x7f1638006000 comp rx=0 tx=0).stop 2023-09-13T04:42:31.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.753+0000 7f1644dff700 1 -- 172.21.15.2:0/2333659648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f164011a490 msgr2=0x7f16401a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:31.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.753+0000 7f1644dff700 1 --2- 172.21.15.2:0/2333659648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f164011a490 0x7f16401a6360 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f163000b480 tx=0x7f163000b840 comp rx=0 tx=0).stop 2023-09-13T04:42:31.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.755+0000 7f1644dff700 1 -- 172.21.15.2:0/2333659648 shutdown_connections 2023-09-13T04:42:31.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.755+0000 7f1644dff700 1 --2- 172.21.15.2:0/2333659648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16401aa7c0 0x7f16401aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.755+0000 7f1644dff700 1 --2- 172.21.15.2:0/2333659648 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f16280a09c0 0x7f16280a2e80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.755+0000 7f1644dff700 1 --2- 172.21.15.2:0/2333659648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f164010f440 0x7f16401a5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.755+0000 7f1644dff700 1 --2- 172.21.15.2:0/2333659648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f164011a490 0x7f16401a6360 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.755+0000 7f1644dff700 1 -- 172.21.15.2:0/2333659648 >> 172.21.15.2:0/2333659648 conn(0x7f164006cfa0 msgr2=0x7f1640118480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:31.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.759+0000 7f1644dff700 1 -- 172.21.15.2:0/2333659648 shutdown_connections 2023-09-13T04:42:31.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:31.759+0000 7f1644dff700 1 -- 172.21.15.2:0/2333659648 wait complete. 2023-09-13T04:42:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.785+0000 7f89a57ff700 1 -- 172.21.15.2:0/2786796605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a010f420 msgr2=0x7f89a0107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.785+0000 7f89a57ff700 1 --2- 172.21.15.2:0/2786796605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a010f420 0x7f89a0107cb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8990009f90 tx=0x7f899000b330 comp rx=0 tx=0).stop 2023-09-13T04:42:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.791+0000 7f89a57ff700 1 -- 172.21.15.2:0/2786796605 shutdown_connections 2023-09-13T04:42:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.791+0000 7f89a57ff700 1 --2- 172.21.15.2:0/2786796605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a011a490 0x7f89a011c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.791+0000 7f89a57ff700 1 --2- 172.21.15.2:0/2786796605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a010f420 0x7f89a0107cb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.791+0000 7f89a57ff700 1 --2- 172.21.15.2:0/2786796605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89a010f890 0x7f89a01081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.791+0000 7f89a57ff700 1 -- 172.21.15.2:0/2786796605 >> 172.21.15.2:0/2786796605 conn(0x7f89a006cda0 msgr2=0x7f89a006d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.791+0000 7f89a57ff700 1 -- 172.21.15.2:0/2786796605 shutdown_connections 2023-09-13T04:42:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.791+0000 7f89a57ff700 1 -- 172.21.15.2:0/2786796605 wait complete. 2023-09-13T04:42:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.793+0000 7f89a57ff700 1 Processor -- start 2023-09-13T04:42:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.793+0000 7f89a57ff700 1 -- start start 2023-09-13T04:42:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.814+0000 7f89a57ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89a010f420 0x7f89a0115990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.815+0000 7f89a57ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a010f890 0x7f89a0115ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.815+0000 7f89a57ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a011a490 0x7f89a01a2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.815+0000 7f89a57ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a01a2770 con 0x7f89a011a490 2023-09-13T04:42:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.815+0000 7f89a57ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a01a28b0 con 0x7f89a010f890 2023-09-13T04:42:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.815+0000 7f89a57ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a01a2a70 con 0x7f89a010f420 2023-09-13T04:42:31.819 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-13T04:42:31.819 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:42:31.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.819+0000 7f89967fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a010f890 0x7f89a0115ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:31.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.819+0000 7f899effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89a010f420 0x7f89a0115990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:31.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.819+0000 7f89967fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a010f890 0x7f89a0115ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:60204/0 (socket says 172.21.15.2:60204) 2023-09-13T04:42:31.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.819+0000 7f89967fc700 1 -- 172.21.15.2:0/218697810 learned_addr learned my addr 172.21.15.2:0/218697810 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:31.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.825+0000 7f89967fc700 1 -- 172.21.15.2:0/218697810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89a010f420 msgr2=0x7f89a0115990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:31.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.825+0000 7f89967fc700 1 --2- 172.21.15.2:0/218697810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89a010f420 0x7f89a0115990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.825+0000 7f89967fc700 1 -- 172.21.15.2:0/218697810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a011a490 msgr2=0x7f89a01a2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:31.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.825+0000 7f89967fc700 1 --2- 172.21.15.2:0/218697810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a011a490 0x7f89a01a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.825+0000 7f89967fc700 1 -- 172.21.15.2:0/218697810 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8990009c40 con 0x7f89a010f890 2023-09-13T04:42:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.847+0000 7f89967fc700 1 --2- 172.21.15.2:0/218697810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a010f890 0x7f89a0115ed0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f898800b550 tx=0x7f898800b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.858+0000 7f899cff9700 1 -- 172.21.15.2:0/218697810 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8988011830 con 0x7f89a010f890 2023-09-13T04:42:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.858+0000 7f899cff9700 1 -- 172.21.15.2:0/218697810 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8988011e50 con 0x7f89a010f890 2023-09-13T04:42:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.858+0000 7f899cff9700 1 -- 172.21.15.2:0/218697810 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898800f2e0 con 0x7f89a010f890 2023-09-13T04:42:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.863+0000 7f89a57ff700 1 -- 172.21.15.2:0/218697810 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a01a2d50 con 0x7f89a010f890 2023-09-13T04:42:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.864+0000 7f89a57ff700 1 -- 172.21.15.2:0/218697810 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a01aa4d0 con 0x7f89a010f890 2023-09-13T04:42:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.871+0000 7f89b9067b80 1 Processor -- start 2023-09-13T04:42:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.873+0000 7f89b9067b80 1 -- start start 2023-09-13T04:42:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.873+0000 7f89b9067b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8dc5ccbe0 0x55a8dc5ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.873+0000 7f89b9067b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5cd500 0x55a8dc5c8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.873+0000 7f89b9067b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8dc5c8840 0x55a8dc5c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.873+0000 7f89b9067b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8dc521720 con 0x55a8dc5c8840 2023-09-13T04:42:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.873+0000 7f89b9067b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8dc51f040 con 0x55a8dc5cd500 2023-09-13T04:42:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.873+0000 7f89b9067b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8dc51a930 con 0x55a8dc5ccbe0 2023-09-13T04:42:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.875+0000 7f899f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8dc5ccbe0 0x55a8dc5ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.875+0000 7f899f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8dc5ccbe0 0x55a8dc5ccfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:50480/0 (socket says 172.21.15.2:50480) 2023-09-13T04:42:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.875+0000 7f899f7fe700 1 -- 172.21.15.2:0/3961884388 learned_addr learned my addr 172.21.15.2:0/3961884388 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.875+0000 7f899effd700 1 --2- 172.21.15.2:0/3961884388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5cd500 0x55a8dc5c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.875+0000 7f899f7fe700 1 -- 172.21.15.2:0/3961884388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5cd500 msgr2=0x55a8dc5c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.875+0000 7f899f7fe700 1 --2- 172.21.15.2:0/3961884388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5cd500 0x55a8dc5c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.875+0000 7f899f7fe700 1 -- 172.21.15.2:0/3961884388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8dc5c8840 msgr2=0x55a8dc5c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:31.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.875+0000 7f899f7fe700 1 --2- 172.21.15.2:0/3961884388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8dc5c8840 0x55a8dc5c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.875+0000 7f899f7fe700 1 -- 172.21.15.2:0/3961884388 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8dc39a5d0 con 0x55a8dc5ccbe0 2023-09-13T04:42:31.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.880+0000 7f899cff9700 1 -- 172.21.15.2:0/218697810 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f8988011990 con 0x7f89a010f890 2023-09-13T04:42:31.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.881+0000 7f899f7fe700 1 --2- 172.21.15.2:0/3961884388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8dc5ccbe0 0x55a8dc5ccfc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f899801cad0 tx=0x7f899801cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f19ab00dfedb550 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:31.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.883+0000 7f899cff9700 1 --2- 172.21.15.2:0/218697810 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f898c0a09c0 0x7f898c0a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.884+0000 7f899cff9700 1 -- 172.21.15.2:0/218697810 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f89880cbb90 con 0x7f89a010f890 2023-09-13T04:42:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.888+0000 7f899effd700 1 --2- 172.21.15.2:0/218697810 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f898c0a09c0 0x7f898c0a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.899+0000 7f89957fa700 1 -- 172.21.15.2:0/3961884388 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899801eab0 con 0x55a8dc5ccbe0 2023-09-13T04:42:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.899+0000 7f89957fa700 1 -- 172.21.15.2:0/3961884388 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8998004cf0 con 0x55a8dc5ccbe0 2023-09-13T04:42:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.899+0000 7f89957fa700 1 -- 172.21.15.2:0/3961884388 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89980054a0 con 0x55a8dc5ccbe0 2023-09-13T04:42:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.900+0000 7f89b9067b80 1 -- 172.21.15.2:0/3961884388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8dc5ccbe0 msgr2=0x55a8dc5ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.900+0000 7f89b9067b80 1 --2- 172.21.15.2:0/3961884388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8dc5ccbe0 0x55a8dc5ccfc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f899801cad0 tx=0x7f899801cde0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.904+0000 7f89b9067b80 1 -- 172.21.15.2:0/3961884388 shutdown_connections 2023-09-13T04:42:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.904+0000 7f89b9067b80 1 --2- 172.21.15.2:0/3961884388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8dc5c8840 0x55a8dc5c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.904+0000 7f89b9067b80 1 --2- 172.21.15.2:0/3961884388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8dc5ccbe0 0x55a8dc5ccfc0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.904+0000 7f89b9067b80 1 --2- 172.21.15.2:0/3961884388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5cd500 0x55a8dc5c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.904+0000 7f89b9067b80 1 -- 172.21.15.2:0/3961884388 >> 172.21.15.2:0/3961884388 conn(0x55a8dc526c20 msgr2=0x55a8dc5c2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.907+0000 7f89b9067b80 1 -- 172.21.15.2:0/3961884388 shutdown_connections 2023-09-13T04:42:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.918+0000 7f89b9067b80 1 -- 172.21.15.2:0/3961884388 wait complete. 2023-09-13T04:42:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.918+0000 7f89b9067b80 1 -- start start 2023-09-13T04:42:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.919+0000 7f89b9067b80 1 Processor -- start 2023-09-13T04:42:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.930+0000 7f89b9067b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8dc5c8840 0x55a8dc5d1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.930+0000 7f89b9067b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5ccbe0 0x55a8dc5d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.930+0000 7f89b9067b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8dc5cd500 0x55a8dc5d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.930+0000 7f89b9067b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8dc4b8b50 con 0x55a8dc5cd500 2023-09-13T04:42:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.930+0000 7f89b9067b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8dc4b8850 con 0x55a8dc5ccbe0 2023-09-13T04:42:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.930+0000 7f89b9067b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8dc39c8c0 con 0x55a8dc5c8840 2023-09-13T04:42:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.930+0000 7f89b9067b80 10 client.0 _ll_register_callbacks cb 0x55a8dc4b6620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:42:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.931+0000 7f89967fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5ccbe0 0x55a8dc5d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.931+0000 7f89967fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5ccbe0 0x55a8dc5d1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:60220/0 (socket says 172.21.15.2:60220) 2023-09-13T04:42:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.931+0000 7f89967fc700 1 -- 172.21.15.2:0/3112497214 learned_addr learned my addr 172.21.15.2:0/3112497214 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.939+0000 7f899effd700 1 --2- 172.21.15.2:0/3112497214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8dc5c8840 0x55a8dc5d1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.939+0000 7f89967fc700 1 -- 172.21.15.2:0/3112497214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8dc5c8840 msgr2=0x55a8dc5d1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.939+0000 7f89967fc700 1 --2- 172.21.15.2:0/3112497214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8dc5c8840 0x55a8dc5d1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.939+0000 7f89967fc700 1 -- 172.21.15.2:0/3112497214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8dc5cd500 msgr2=0x55a8dc5d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.939+0000 7f89967fc700 1 --2- 172.21.15.2:0/3112497214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8dc5cd500 0x55a8dc5d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.939+0000 7f89967fc700 1 -- 172.21.15.2:0/3112497214 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899801c730 con 0x55a8dc5ccbe0 2023-09-13T04:42:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.940+0000 7f899effd700 1 --2- 172.21.15.2:0/3112497214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8dc5c8840 0x55a8dc5d1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:42:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.947+0000 7f899effd700 1 --2- 172.21.15.2:0/218697810 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f898c0a09c0 0x7f898c0a2e80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55a8dc5cd900 tx=0x7f8990006000 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.964+0000 7f89967fc700 1 --2- 172.21.15.2:0/3112497214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5ccbe0 0x55a8dc5d1ec0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8988095360 tx=0x7f8988010430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31e7160bad14f84f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.964+0000 7f89867fc700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:42:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.973+0000 7f89867fc700 1 -- 172.21.15.2:0/3112497214 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89880b0070 con 0x55a8dc5ccbe0 2023-09-13T04:42:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.973+0000 7f89867fc700 1 -- 172.21.15.2:0/3112497214 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f898809cc90 con 0x55a8dc5ccbe0 2023-09-13T04:42:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:31.973+0000 7f89b9067b80 1 -- 172.21.15.2:0/3112497214 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8dc58def0 con 0x55a8dc5ccbe0 2023-09-13T04:42:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.009+0000 7f89867fc700 1 -- 172.21.15.2:0/3112497214 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89880b1260 con 0x55a8dc5ccbe0 2023-09-13T04:42:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.015+0000 7f89b9067b80 10 client.26063 fetch_fsmap learned FSMap version 27 2023-09-13T04:42:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.015+0000 7f89b9067b80 1 -- 172.21.15.2:0/3112497214 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a8dc532740 con 0x55a8dc5ccbe0 2023-09-13T04:42:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.028+0000 7f89867fc700 1 -- 172.21.15.2:0/3112497214 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a8dc58def0 con 0x55a8dc5ccbe0 2023-09-13T04:42:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.028+0000 7f89867fc700 1 -- 172.21.15.2:0/3112497214 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f89880990c0 con 0x55a8dc5ccbe0 2023-09-13T04:42:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.033+0000 7f89b9067b80 10 client.26063 fetch_fsmap finished waiting for FSMap version 27 2023-09-13T04:42:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.033+0000 7f89b9067b80 10 client.26063 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:42:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.033+0000 7f89b9067b80 20 client.26063 populate_metadata read hostname 'smithi002' 2023-09-13T04:42:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.033+0000 7f89b9067b80 1 --2- 172.21.15.2:0/3112497214 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a8dc5e2050 0x55a8dc5e4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.033+0000 7f89b9067b80 4 client.26063 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:42:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.033+0000 7f89b9067b80 1 -- 172.21.15.2:0/3112497214 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8dc513910 con 0x55a8dc5e2050 2023-09-13T04:42:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.034+0000 7f899effd700 1 --2- 172.21.15.2:0/3112497214 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a8dc5e2050 0x55a8dc5e4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.034+0000 7f899effd700 1 --2- 172.21.15.2:0/3112497214 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a8dc5e2050 0x55a8dc5e4470 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ed35e1d524310c5 server_cookie=6416b6af8e0e4c24 in_seq=0 out_seq=0 2023-09-13T04:42:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.038+0000 7f89867fc700 10 client.26063 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.038+0000 7f89867fc700 1 -- 172.21.15.2:0/3112497214 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a8dc513910 con 0x55a8dc5e2050 2023-09-13T04:42:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.038+0000 7f89867fc700 10 client.26063 handle_command_reply: tid=0 2023-09-13T04:42:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.045+0000 7f89b9067b80 1 client.26063 shutdown 2023-09-13T04:42:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.046+0000 7f89b9067b80 1 -- 172.21.15.2:0/3112497214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5ccbe0 msgr2=0x55a8dc5d1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.046+0000 7f89b9067b80 1 --2- 172.21.15.2:0/3112497214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5ccbe0 0x55a8dc5d1ec0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8988095360 tx=0x7f8988010430 comp rx=0 tx=0).stop 2023-09-13T04:42:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.051+0000 7f89b9067b80 1 -- 172.21.15.2:0/3112497214 shutdown_connections 2023-09-13T04:42:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.051+0000 7f89b9067b80 1 --2- 172.21.15.2:0/3112497214 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a8dc5e2050 0x55a8dc5e4470 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.051+0000 7f89b9067b80 1 --2- 172.21.15.2:0/3112497214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8dc5cd500 0x55a8dc5d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.051+0000 7f89b9067b80 1 --2- 172.21.15.2:0/3112497214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8dc5c8840 0x55a8dc5d1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.051+0000 7f89b9067b80 1 --2- 172.21.15.2:0/3112497214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5ccbe0 0x55a8dc5d1ec0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.051+0000 7f89b9067b80 1 -- 172.21.15.2:0/3112497214 >> 172.21.15.2:0/3112497214 conn(0x55a8dc526c20 msgr2=0x55a8dc5ce330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.051+0000 7f89867fc700 0 client.26063 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.055+0000 7f89b9067b80 1 -- 172.21.15.2:0/3112497214 shutdown_connections 2023-09-13T04:42:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.055+0000 7f89b9067b80 1 -- 172.21.15.2:0/3112497214 wait complete. 2023-09-13T04:42:32.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.056+0000 7f89b9067b80 20 client.26063 trim_cache size 0 max 16384 2023-09-13T04:42:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.099+0000 7f89b9067b80 1 Processor -- start 2023-09-13T04:42:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.100+0000 7f89b9067b80 1 -- start start 2023-09-13T04:42:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.101+0000 7f89b9067b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8dc5c8840 0x55a8dc5bd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.101+0000 7f89b9067b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5ccbe0 0x55a8dc5cb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.101+0000 7f89b9067b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8dc5cd500 0x55a8dc5cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.101+0000 7f89b9067b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8dc5c3100 con 0x55a8dc5c8840 2023-09-13T04:42:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.101+0000 7f89b9067b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8dc5bb2d0 con 0x55a8dc5ccbe0 2023-09-13T04:42:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.101+0000 7f89b9067b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8dc5df6b0 con 0x55a8dc5cd500 2023-09-13T04:42:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.102+0000 7f89967fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5ccbe0 0x55a8dc5cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.102+0000 7f89967fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5ccbe0 0x55a8dc5cb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:60228/0 (socket says 172.21.15.2:60228) 2023-09-13T04:42:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.102+0000 7f89967fc700 1 -- 172.21.15.2:0/366727306 learned_addr learned my addr 172.21.15.2:0/366727306 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.103+0000 7f89967fc700 1 -- 172.21.15.2:0/366727306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8dc5cd500 msgr2=0x55a8dc5cbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.103+0000 7f89967fc700 1 --2- 172.21.15.2:0/366727306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8dc5cd500 0x55a8dc5cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.103+0000 7f89967fc700 1 -- 172.21.15.2:0/366727306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8dc5c8840 msgr2=0x55a8dc5bd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.103+0000 7f89967fc700 1 --2- 172.21.15.2:0/366727306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8dc5c8840 0x55a8dc5bd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.103+0000 7f89967fc700 1 -- 172.21.15.2:0/366727306 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8dc5d1510 con 0x55a8dc5ccbe0 2023-09-13T04:42:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.105+0000 7f89967fc700 1 --2- 172.21.15.2:0/366727306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5ccbe0 0x55a8dc5cb740 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f89880958d0 tx=0x7f8988098fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f28817afbf79b3ca server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.106+0000 7f8995ffb700 1 -- 172.21.15.2:0/366727306 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89880b0040 con 0x55a8dc5ccbe0 2023-09-13T04:42:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.106+0000 7f8995ffb700 1 -- 172.21.15.2:0/366727306 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f898809cc90 con 0x55a8dc5ccbe0 2023-09-13T04:42:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.106+0000 7f8995ffb700 1 -- 172.21.15.2:0/366727306 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89880990c0 con 0x55a8dc5ccbe0 2023-09-13T04:42:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.107+0000 7f89b9067b80 1 -- 172.21.15.2:0/366727306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5ccbe0 msgr2=0x55a8dc5cb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.107+0000 7f89b9067b80 1 --2- 172.21.15.2:0/366727306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5ccbe0 0x55a8dc5cb740 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f89880958d0 tx=0x7f8988098fe0 comp rx=0 tx=0).stop 2023-09-13T04:42:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.108+0000 7f89b9067b80 1 -- 172.21.15.2:0/366727306 shutdown_connections 2023-09-13T04:42:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.108+0000 7f89b9067b80 1 --2- 172.21.15.2:0/366727306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8dc5cd500 0x55a8dc5cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.108+0000 7f89b9067b80 1 --2- 172.21.15.2:0/366727306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8dc5c8840 0x55a8dc5bd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.108+0000 7f89b9067b80 1 --2- 172.21.15.2:0/366727306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5ccbe0 0x55a8dc5cb740 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.108+0000 7f89b9067b80 1 -- 172.21.15.2:0/366727306 >> 172.21.15.2:0/366727306 conn(0x55a8dc526c20 msgr2=0x55a8dc5dd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.108+0000 7f89b9067b80 1 -- 172.21.15.2:0/366727306 shutdown_connections 2023-09-13T04:42:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.108+0000 7f89b9067b80 1 -- 172.21.15.2:0/366727306 wait complete. 2023-09-13T04:42:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.109+0000 7f89b9067b80 1 -- start start 2023-09-13T04:42:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.109+0000 7f89b9067b80 1 Processor -- start 2023-09-13T04:42:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.110+0000 7f89b9067b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5c8840 0x55a8dc5d6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.110+0000 7f89b9067b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8dc5ccbe0 0x55a8dc5d7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.110+0000 7f89b9067b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8dc5cd500 0x55a8dc5d79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.110+0000 7f89b9067b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8dc634e80 con 0x55a8dc5cd500 2023-09-13T04:42:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.110+0000 7f89b9067b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8dc634fc0 con 0x55a8dc5c8840 2023-09-13T04:42:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.110+0000 7f89b9067b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8dc635100 con 0x55a8dc5ccbe0 2023-09-13T04:42:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.110+0000 7f89b9067b80 10 client.0 _ll_register_callbacks cb 0x55a8dc4b6620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:42:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.111+0000 7f899effd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5c8840 0x55a8dc5d6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.111+0000 7f899effd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5c8840 0x55a8dc5d6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:60232/0 (socket says 172.21.15.2:60232) 2023-09-13T04:42:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.111+0000 7f899effd700 1 -- 172.21.15.2:0/1447864853 learned_addr learned my addr 172.21.15.2:0/1447864853 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.127+0000 7f899f7fe700 1 --2- 172.21.15.2:0/1447864853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8dc5cd500 0x55a8dc5d79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.128+0000 7f89967fc700 1 --2- 172.21.15.2:0/1447864853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8dc5ccbe0 0x55a8dc5d7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.128+0000 7f899effd700 1 -- 172.21.15.2:0/1447864853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8dc5ccbe0 msgr2=0x55a8dc5d7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.128+0000 7f899effd700 1 --2- 172.21.15.2:0/1447864853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8dc5ccbe0 0x55a8dc5d7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.128+0000 7f899effd700 1 -- 172.21.15.2:0/1447864853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8dc5cd500 msgr2=0x55a8dc5d79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.128+0000 7f899effd700 1 --2- 172.21.15.2:0/1447864853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8dc5cd500 0x55a8dc5d79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.128+0000 7f899effd700 1 -- 172.21.15.2:0/1447864853 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8dc532740 con 0x55a8dc5c8840 2023-09-13T04:42:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.142+0000 7f899effd700 1 --2- 172.21.15.2:0/1447864853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5c8840 0x55a8dc5d6f30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f899000bfd0 tx=0x7f899000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ef5630a849eaa2f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.142+0000 7f89867fc700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:42:32.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.142+0000 7f89867fc700 1 -- 172.21.15.2:0/1447864853 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899002f070 con 0x55a8dc5c8840 2023-09-13T04:42:32.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.142+0000 7f89867fc700 1 -- 172.21.15.2:0/1447864853 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f8990007420 con 0x55a8dc5c8840 2023-09-13T04:42:32.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.143+0000 7f89b9067b80 1 -- 172.21.15.2:0/1447864853 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8998004cf0 con 0x55a8dc5c8840 2023-09-13T04:42:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.148+0000 7f89867fc700 1 -- 172.21.15.2:0/1447864853 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f899000e040 con 0x55a8dc5c8840 2023-09-13T04:42:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.151+0000 7f89b9067b80 10 client.26069 fetch_fsmap learned FSMap version 27 2023-09-13T04:42:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.151+0000 7f89b9067b80 1 -- 172.21.15.2:0/1447864853 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a8dc346d80 con 0x55a8dc5c8840 2023-09-13T04:42:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.157+0000 7f89867fc700 1 -- 172.21.15.2:0/1447864853 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8998004cf0 con 0x55a8dc5c8840 2023-09-13T04:42:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.158+0000 7f89867fc700 1 -- 172.21.15.2:0/1447864853 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8990005110 con 0x55a8dc5c8840 2023-09-13T04:42:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.159+0000 7f89b9067b80 10 client.26069 fetch_fsmap finished waiting for FSMap version 27 2023-09-13T04:42:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.159+0000 7f89b9067b80 10 client.26069 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:42:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.159+0000 7f89b9067b80 20 client.26069 populate_metadata read hostname 'smithi002' 2023-09-13T04:42:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.159+0000 7f89b9067b80 1 --2- 172.21.15.2:0/1447864853 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a8dc6367c0 0x55a8dc638be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.159+0000 7f89b9067b80 4 client.26069 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:42:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.159+0000 7f89b9067b80 1 -- 172.21.15.2:0/1447864853 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a8dc4d3ff0 con 0x55a8dc6367c0 2023-09-13T04:42:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.162+0000 7f89967fc700 1 --2- 172.21.15.2:0/1447864853 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a8dc6367c0 0x55a8dc638be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.176+0000 7f89967fc700 1 --2- 172.21.15.2:0/1447864853 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a8dc6367c0 0x55a8dc638be0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbded6e05662d2e3 server_cookie=3edf1d87d89be51a in_seq=0 out_seq=0 2023-09-13T04:42:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:32.176+0000 7f89867fc700 10 client.26069 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:32.189 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:31 smithi002 ceph-mon[97651]: pgmap v65: 129 pgs: 129 active+clean; 156 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 196 op/s 2023-09-13T04:42:32.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:31 smithi149 ceph-mon[133315]: pgmap v65: 129 pgs: 129 active+clean; 156 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 196 op/s 2023-09-13T04:42:32.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:31 smithi156 ceph-mon[133675]: pgmap v65: 129 pgs: 129 active+clean; 156 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 196 op/s 2023-09-13T04:42:32.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.552+0000 7f0f40bff700 1 -- 172.21.15.2:0/201243336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f3c1085d0 msgr2=0x7f0f3c1138c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:32.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.552+0000 7f0f40bff700 1 --2- 172.21.15.2:0/201243336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f3c1085d0 0x7f0f3c1138c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0099c0 tx=0x7f0f2c009cd0 comp rx=0 tx=0).stop 2023-09-13T04:42:32.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.560+0000 7f0f40bff700 1 -- 172.21.15.2:0/201243336 shutdown_connections 2023-09-13T04:42:32.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.560+0000 7f0f40bff700 1 --2- 172.21.15.2:0/201243336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f3c113e00 0x7f0f3c1161f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:32.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.560+0000 7f0f40bff700 1 --2- 172.21.15.2:0/201243336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f3c107cb0 0x7f0f3c108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:32.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.561+0000 7f0f40bff700 1 --2- 172.21.15.2:0/201243336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f3c1085d0 0x7f0f3c1138c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:32.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.561+0000 7f0f40bff700 1 -- 172.21.15.2:0/201243336 >> 172.21.15.2:0/201243336 conn(0x7f0f3c06bd20 msgr2=0x7f0f3c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:32.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.561+0000 7f0f40bff700 1 -- 172.21.15.2:0/201243336 shutdown_connections 2023-09-13T04:42:32.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.561+0000 7f0f40bff700 1 -- 172.21.15.2:0/201243336 wait complete. 2023-09-13T04:42:32.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.562+0000 7f0f40bff700 1 Processor -- start 2023-09-13T04:42:32.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.562+0000 7f0f40bff700 1 -- start start 2023-09-13T04:42:32.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.562+0000 7f0f40bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f3c107cb0 0x7f0f3c19d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:32.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.563+0000 7f0f40bff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f3c1085d0 0x7f0f3c19d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:32.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.563+0000 7f0f40bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f3c113e00 0x7f0f3c1a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:32.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.563+0000 7f0f40bff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c1a2380 con 0x7f0f3c1085d0 2023-09-13T04:42:32.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.563+0000 7f0f40bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c1a24f0 con 0x7f0f3c107cb0 2023-09-13T04:42:32.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.563+0000 7f0f40bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c1a26b0 con 0x7f0f3c113e00 2023-09-13T04:42:32.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.563+0000 7f0f3a59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f3c107cb0 0x7f0f3c19d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:32.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.563+0000 7f0f3a59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f3c107cb0 0x7f0f3c19d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:60242/0 (socket says 172.21.15.2:60242) 2023-09-13T04:42:32.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.563+0000 7f0f3a59c700 1 -- 172.21.15.2:0/801921515 learned_addr learned my addr 172.21.15.2:0/801921515 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:32.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.567+0000 7f0f39d9b700 1 --2- 172.21.15.2:0/801921515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f3c1085d0 0x7f0f3c19d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:32.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.567+0000 7f0f3a59c700 1 -- 172.21.15.2:0/801921515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f3c113e00 msgr2=0x7f0f3c1a1d40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-13T04:42:32.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.567+0000 7f0f3a59c700 1 --2- 172.21.15.2:0/801921515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f3c113e00 0x7f0f3c1a1d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:32.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.567+0000 7f0f3a59c700 1 -- 172.21.15.2:0/801921515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f3c1085d0 msgr2=0x7f0f3c19d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:32.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.567+0000 7f0f3a59c700 1 --2- 172.21.15.2:0/801921515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f3c1085d0 0x7f0f3c19d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:32.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.567+0000 7f0f3a59c700 1 -- 172.21.15.2:0/801921515 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2c0095f0 con 0x7f0f3c107cb0 2023-09-13T04:42:32.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.568+0000 7f0f3a59c700 1 --2- 172.21.15.2:0/801921515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f3c107cb0 0x7f0f3c19d3a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0f340060b0 tx=0x7f0f34007f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:32.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.568+0000 7f0f2b7fe700 1 -- 172.21.15.2:0/801921515 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3400de20 con 0x7f0f3c107cb0 2023-09-13T04:42:32.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.568+0000 7f0f2b7fe700 1 -- 172.21.15.2:0/801921515 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f3400f040 con 0x7f0f3c107cb0 2023-09-13T04:42:32.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.568+0000 7f0f2b7fe700 1 -- 172.21.15.2:0/801921515 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f340146e0 con 0x7f0f3c107cb0 2023-09-13T04:42:32.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.568+0000 7f0f40bff700 1 -- 172.21.15.2:0/801921515 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f3c1a2990 con 0x7f0f3c107cb0 2023-09-13T04:42:32.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.568+0000 7f0f40bff700 1 -- 172.21.15.2:0/801921515 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f3c1aa1d0 con 0x7f0f3c107cb0 2023-09-13T04:42:32.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.569+0000 7f0f40bff700 1 -- 172.21.15.2:0/801921515 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f3c067ad0 con 0x7f0f3c107cb0 2023-09-13T04:42:32.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.575+0000 7f0f2b7fe700 1 -- 172.21.15.2:0/801921515 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f0f34015a30 con 0x7f0f3c107cb0 2023-09-13T04:42:32.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.578+0000 7f0f2b7fe700 1 --2- 172.21.15.2:0/801921515 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f0f240a4d40 0x7f0f240a7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:32.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.578+0000 7f0f2b7fe700 1 -- 172.21.15.2:0/801921515 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f0f340cc1a0 con 0x7f0f3c107cb0 2023-09-13T04:42:32.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.579+0000 7f0f2b7fe700 1 -- 172.21.15.2:0/801921515 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0f34095620 con 0x7f0f3c107cb0 2023-09-13T04:42:32.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.580+0000 7f0f39d9b700 1 --2- 172.21.15.2:0/801921515 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f0f240a4d40 0x7f0f240a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:32.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:32.581+0000 7f0f39d9b700 1 --2- 172.21.15.2:0/801921515 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f0f240a4d40 0x7f0f240a7200 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c009f50 tx=0x7f0f2c005a90 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:34.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:33 smithi002 ceph-mon[97651]: pgmap v66: 129 pgs: 129 active+clean; 175 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.4 MiB/s wr, 459 op/s 2023-09-13T04:42:34.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.340+0000 7f0f40bff700 1 -- 172.21.15.2:0/801921515 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f3c19e6f0 con 0x7f0f3c107cb0 2023-09-13T04:42:34.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.341+0000 7f0f2b7fe700 1 -- 172.21.15.2:0/801921515 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11309 (secure 0 0 0) 0x7f0f3401a090 con 0x7f0f3c107cb0 2023-09-13T04:42:34.342 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:42:34.343 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2469307307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2469307307},{"type":"v1","addr":"172.21.15.2:6839","nonce":2469307307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T04:37:59.110701+0000","modified":"2023-09-13T04:42:15.620529+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24554,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/846579077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":846579077},{"type":"v1","addr":"172.21.15.156:6835","nonce":846579077}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1896833690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1896833690},{"type":"v1","addr":"172.21.15.149:6837","nonce":1896833690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/524563090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":524563090},{"type":"v1","addr":"172.21.15.156:6837","nonce":524563090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6839/1785833227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1785833227},{"type":"v1","addr":"172.21.15.156:6839","nonce":1785833227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.2:6841/482545391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":482545391},{"type":"v1","addr":"172.21.15.2:6841","nonce":482545391}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/389167841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":389167841},{"type":"v1","addr":"172.21.15.149:6839","nonce":389167841}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T04:42:34.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.343+0000 7f0f40bff700 1 -- 172.21.15.2:0/801921515 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f0f240a4d40 msgr2=0x7f0f240a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:34.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.343+0000 7f0f40bff700 1 --2- 172.21.15.2:0/801921515 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f0f240a4d40 0x7f0f240a7200 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c009f50 tx=0x7f0f2c005a90 comp rx=0 tx=0).stop 2023-09-13T04:42:34.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.343+0000 7f0f40bff700 1 -- 172.21.15.2:0/801921515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f3c107cb0 msgr2=0x7f0f3c19d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:34.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.343+0000 7f0f40bff700 1 --2- 172.21.15.2:0/801921515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f3c107cb0 0x7f0f3c19d3a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0f340060b0 tx=0x7f0f34007f90 comp rx=0 tx=0).stop 2023-09-13T04:42:34.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.344+0000 7f0f40bff700 1 -- 172.21.15.2:0/801921515 shutdown_connections 2023-09-13T04:42:34.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.344+0000 7f0f40bff700 1 --2- 172.21.15.2:0/801921515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f3c113e00 0x7f0f3c1a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.344+0000 7f0f40bff700 1 --2- 172.21.15.2:0/801921515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f3c107cb0 0x7f0f3c19d3a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.344+0000 7f0f40bff700 1 --2- 172.21.15.2:0/801921515 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f0f240a4d40 0x7f0f240a7200 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.344+0000 7f0f40bff700 1 --2- 172.21.15.2:0/801921515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f3c1085d0 0x7f0f3c19d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.344+0000 7f0f40bff700 1 -- 172.21.15.2:0/801921515 >> 172.21.15.2:0/801921515 conn(0x7f0f3c06bd20 msgr2=0x7f0f3c06f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:34.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.344+0000 7f0f40bff700 1 -- 172.21.15.2:0/801921515 shutdown_connections 2023-09-13T04:42:34.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.344+0000 7f0f40bff700 1 -- 172.21.15.2:0/801921515 wait complete. 2023-09-13T04:42:34.348 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 27 2023-09-13T04:42:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.357+0000 7f89867fc700 1 -- 172.21.15.2:0/1447864853 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a8dc4d3ff0 con 0x55a8dc6367c0 2023-09-13T04:42:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.357+0000 7f89867fc700 10 client.26069 handle_command_reply: tid=0 2023-09-13T04:42:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.357+0000 7f89b9067b80 1 client.26069 shutdown 2023-09-13T04:42:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.357+0000 7f89b9067b80 1 -- 172.21.15.2:0/1447864853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5c8840 msgr2=0x55a8dc5d6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.357+0000 7f89b9067b80 1 --2- 172.21.15.2:0/1447864853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5c8840 0x55a8dc5d6f30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f899000bfd0 tx=0x7f899000b330 comp rx=0 tx=0).stop 2023-09-13T04:42:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.357+0000 7f89b9067b80 1 -- 172.21.15.2:0/1447864853 shutdown_connections 2023-09-13T04:42:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.357+0000 7f89b9067b80 1 --2- 172.21.15.2:0/1447864853 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a8dc6367c0 0x55a8dc638be0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.357+0000 7f89b9067b80 1 --2- 172.21.15.2:0/1447864853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8dc5cd500 0x55a8dc5d79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.357+0000 7f89b9067b80 1 --2- 172.21.15.2:0/1447864853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8dc5c8840 0x55a8dc5d6f30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.357+0000 7f89867fc700 0 client.26069 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.357+0000 7f89b9067b80 1 --2- 172.21.15.2:0/1447864853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8dc5ccbe0 0x55a8dc5d7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.357+0000 7f89b9067b80 1 -- 172.21.15.2:0/1447864853 >> 172.21.15.2:0/1447864853 conn(0x55a8dc526c20 msgr2=0x55a8dc5cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.358+0000 7f89b9067b80 1 -- 172.21.15.2:0/1447864853 shutdown_connections 2023-09-13T04:42:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.358+0000 7f89b9067b80 1 -- 172.21.15.2:0/1447864853 wait complete. 2023-09-13T04:42:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.358+0000 7f89b9067b80 20 client.26069 trim_cache size 0 max 16384 2023-09-13T04:42:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:42:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:42:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:42:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:42:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.359+0000 7f89a57ff700 1 -- 172.21.15.2:0/218697810 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f898c0a09c0 msgr2=0x7f898c0a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.359+0000 7f89a57ff700 1 --2- 172.21.15.2:0/218697810 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f898c0a09c0 0x7f898c0a2e80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55a8dc5cd900 tx=0x7f8990006000 comp rx=0 tx=0).stop 2023-09-13T04:42:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.359+0000 7f89a57ff700 1 -- 172.21.15.2:0/218697810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a010f890 msgr2=0x7f89a0115ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.359+0000 7f89a57ff700 1 --2- 172.21.15.2:0/218697810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a010f890 0x7f89a0115ed0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f898800b550 tx=0x7f898800b860 comp rx=0 tx=0).stop 2023-09-13T04:42:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.359+0000 7f89a57ff700 1 -- 172.21.15.2:0/218697810 shutdown_connections 2023-09-13T04:42:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.359+0000 7f89a57ff700 1 --2- 172.21.15.2:0/218697810 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f898c0a09c0 0x7f898c0a2e80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.359+0000 7f89a57ff700 1 --2- 172.21.15.2:0/218697810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a011a490 0x7f89a01a2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.359+0000 7f89a57ff700 1 --2- 172.21.15.2:0/218697810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89a010f420 0x7f89a0115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.359+0000 7f89a57ff700 1 --2- 172.21.15.2:0/218697810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a010f890 0x7f89a0115ed0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.359+0000 7f89a57ff700 1 -- 172.21.15.2:0/218697810 >> 172.21.15.2:0/218697810 conn(0x7f89a006cda0 msgr2=0x7f89a011ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.359+0000 7f89a57ff700 1 -- 172.21.15.2:0/218697810 shutdown_connections 2023-09-13T04:42:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:34.359+0000 7f89a57ff700 1 -- 172.21.15.2:0/218697810 wait complete. 2023-09-13T04:42:34.369 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-13T04:42:34.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:33 smithi149 ceph-mon[133315]: pgmap v66: 129 pgs: 129 active+clean; 175 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.4 MiB/s wr, 459 op/s 2023-09-13T04:42:34.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:33 smithi156 ceph-mon[133675]: pgmap v66: 129 pgs: 129 active+clean; 175 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.4 MiB/s wr, 459 op/s 2023-09-13T04:42:34.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.736+0000 7f9cde7ff700 1 -- 172.21.15.2:0/1376575572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cd81073d0 msgr2=0x7f9cd8107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:34.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.736+0000 7f9cde7ff700 1 --2- 172.21.15.2:0/1376575572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cd81073d0 0x7f9cd8107830 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc009990 tx=0x7f9ccc009ca0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.736+0000 7f9cde7ff700 1 -- 172.21.15.2:0/1376575572 shutdown_connections 2023-09-13T04:42:34.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.736+0000 7f9cde7ff700 1 --2- 172.21.15.2:0/1376575572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cd8107d70 0x7f9cd8114160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.736+0000 7f9cde7ff700 1 --2- 172.21.15.2:0/1376575572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cd810d330 0x7f9cd810d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.736+0000 7f9cde7ff700 1 --2- 172.21.15.2:0/1376575572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cd81073d0 0x7f9cd8107830 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.736+0000 7f9cde7ff700 1 -- 172.21.15.2:0/1376575572 >> 172.21.15.2:0/1376575572 conn(0x7f9cd8076b20 msgr2=0x7f9cd8105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:34.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.737+0000 7f9cde7ff700 1 -- 172.21.15.2:0/1376575572 shutdown_connections 2023-09-13T04:42:34.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.737+0000 7f9cde7ff700 1 -- 172.21.15.2:0/1376575572 wait complete. 2023-09-13T04:42:34.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.738+0000 7f9cde7ff700 1 Processor -- start 2023-09-13T04:42:34.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.739+0000 7f9cde7ff700 1 -- start start 2023-09-13T04:42:34.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.739+0000 7f9cde7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cd81073d0 0x7f9cd819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:34.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.739+0000 7f9cde7ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cd8107d70 0x7f9cd819d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:34.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.740+0000 7f9cde7ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cd810d330 0x7f9cd81a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:34.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.740+0000 7f9cde7ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd81a2430 con 0x7f9cd810d330 2023-09-13T04:42:34.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.740+0000 7f9cde7ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd81a25a0 con 0x7f9cd8107d70 2023-09-13T04:42:34.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.740+0000 7f9cde7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd81a2760 con 0x7f9cd81073d0 2023-09-13T04:42:34.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.740+0000 7f9cd77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cd8107d70 0x7f9cd819d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:34.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.740+0000 7f9cd77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cd8107d70 0x7f9cd819d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:46346/0 (socket says 172.21.15.2:46346) 2023-09-13T04:42:34.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.740+0000 7f9cd77fe700 1 -- 172.21.15.2:0/205431551 learned_addr learned my addr 172.21.15.2:0/205431551 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:34.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.740+0000 7f9cd7fff700 1 --2- 172.21.15.2:0/205431551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cd81073d0 0x7f9cd819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:34.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.741+0000 7f9cd77fe700 1 -- 172.21.15.2:0/205431551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cd81073d0 msgr2=0x7f9cd819d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:34.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.741+0000 7f9cd77fe700 1 --2- 172.21.15.2:0/205431551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cd81073d0 0x7f9cd819d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.741+0000 7f9cd77fe700 1 -- 172.21.15.2:0/205431551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cd810d330 msgr2=0x7f9cd81a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:34.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.741+0000 7f9cd77fe700 1 --2- 172.21.15.2:0/205431551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cd810d330 0x7f9cd81a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.741+0000 7f9cd77fe700 1 -- 172.21.15.2:0/205431551 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ccc0095f0 con 0x7f9cd8107d70 2023-09-13T04:42:34.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.742+0000 7f9cd77fe700 1 --2- 172.21.15.2:0/205431551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cd8107d70 0x7f9cd819d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc000c00 tx=0x7f9ccc0057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:34.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.743+0000 7f9cd57fa700 1 -- 172.21.15.2:0/205431551 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc01d070 con 0x7f9cd8107d70 2023-09-13T04:42:34.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.743+0000 7f9cde7ff700 1 -- 172.21.15.2:0/205431551 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd81a29e0 con 0x7f9cd8107d70 2023-09-13T04:42:34.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.743+0000 7f9cd57fa700 1 -- 172.21.15.2:0/205431551 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ccc0173f0 con 0x7f9cd8107d70 2023-09-13T04:42:34.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.743+0000 7f9cde7ff700 1 -- 172.21.15.2:0/205431551 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd81aa280 con 0x7f9cd8107d70 2023-09-13T04:42:34.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.743+0000 7f9cd57fa700 1 -- 172.21.15.2:0/205431551 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc00e640 con 0x7f9cd8107d70 2023-09-13T04:42:34.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.751+0000 7f9cf20f0b80 1 Processor -- start 2023-09-13T04:42:34.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.751+0000 7f9cf20f0b80 1 -- start start 2023-09-13T04:42:34.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.752+0000 7f9cf20f0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f5a2cbfb0 0x564f5a2cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:34.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.752+0000 7f9cf20f0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f5a2cc8d0 0x564f5a2c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:34.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.752+0000 7f9cf20f0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f5a2c6a10 0x564f5a2c6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:34.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.752+0000 7f9cf20f0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5a20e780 con 0x564f5a2cc8d0 2023-09-13T04:42:34.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.752+0000 7f9cf20f0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5a20c0a0 con 0x564f5a2c6a10 2023-09-13T04:42:34.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.752+0000 7f9cf20f0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5a207990 con 0x564f5a2cbfb0 2023-09-13T04:42:34.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.753+0000 7f9cd57fa700 1 -- 172.21.15.2:0/205431551 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f9ccc00e7a0 con 0x7f9cd8107d70 2023-09-13T04:42:34.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.755+0000 7f9cd57fa700 1 --2- 172.21.15.2:0/205431551 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f9cc00a08f0 0x7f9cc00a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:34.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.755+0000 7f9cd57fa700 1 -- 172.21.15.2:0/205431551 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f9ccc0cd780 con 0x7f9cd8107d70 2023-09-13T04:42:34.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.757+0000 7f9cd7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f5a2cc8d0 0x564f5a2c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:34.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.757+0000 7f9cd7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f5a2cc8d0 0x564f5a2c64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37120/0 (socket says 172.21.15.2:37120) 2023-09-13T04:42:34.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.757+0000 7f9cd7fff700 1 -- 172.21.15.2:0/3868293209 learned_addr learned my addr 172.21.15.2:0/3868293209 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:34.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.757+0000 7f9cd7fff700 1 -- 172.21.15.2:0/3868293209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f5a2cbfb0 msgr2=0x564f5a2cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:34.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.757+0000 7f9cd7fff700 1 --2- 172.21.15.2:0/3868293209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f5a2cbfb0 0x564f5a2cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.757+0000 7f9cd7fff700 1 -- 172.21.15.2:0/3868293209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f5a2c6a10 msgr2=0x564f5a2c6e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:42:34.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.757+0000 7f9cd7fff700 1 --2- 172.21.15.2:0/3868293209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f5a2c6a10 0x564f5a2c6e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.757+0000 7f9cd7fff700 1 -- 172.21.15.2:0/3868293209 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f5a087630 con 0x564f5a2cc8d0 2023-09-13T04:42:34.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.758+0000 7f9cd7fff700 1 --2- 172.21.15.2:0/3868293209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f5a2cc8d0 0x564f5a2c64d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009d10 tx=0x7f9cc80116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4662fab6ff1504f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:34.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.758+0000 7f9cbe7fc700 1 -- 172.21.15.2:0/3868293209 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8011ed0 con 0x564f5a2cc8d0 2023-09-13T04:42:34.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.759+0000 7f9cbe7fc700 1 -- 172.21.15.2:0/3868293209 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cc8009e60 con 0x564f5a2cc8d0 2023-09-13T04:42:34.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.759+0000 7f9cbe7fc700 1 -- 172.21.15.2:0/3868293209 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc800f8e0 con 0x564f5a2cc8d0 2023-09-13T04:42:34.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.761+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/3868293209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f5a2cc8d0 msgr2=0x564f5a2c64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:34.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.761+0000 7f9cf20f0b80 1 --2- 172.21.15.2:0/3868293209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f5a2cc8d0 0x564f5a2c64d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009d10 tx=0x7f9cc80116c0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.770+0000 7f9cdcd9c700 1 --2- 172.21.15.2:0/205431551 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f9cc00a08f0 0x7f9cc00a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:34.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.770+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/3868293209 shutdown_connections 2023-09-13T04:42:34.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.770+0000 7f9cf20f0b80 1 --2- 172.21.15.2:0/3868293209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f5a2c6a10 0x564f5a2c6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.770+0000 7f9cf20f0b80 1 --2- 172.21.15.2:0/3868293209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f5a2cbfb0 0x564f5a2cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.770+0000 7f9cf20f0b80 1 --2- 172.21.15.2:0/3868293209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f5a2cc8d0 0x564f5a2c64d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.770+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/3868293209 >> 172.21.15.2:0/3868293209 conn(0x564f5a213c80 msgr2=0x564f5a2aff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:34.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.775+0000 7f9cdcd9c700 1 --2- 172.21.15.2:0/205431551 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f9cc00a08f0 0x7f9cc00a2db0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8067c70 tx=0x7f9cd0001500 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:34.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.776+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/3868293209 shutdown_connections 2023-09-13T04:42:34.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.779+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/3868293209 wait complete. 2023-09-13T04:42:34.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.779+0000 7f9cf20f0b80 1 -- start start 2023-09-13T04:42:34.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.780+0000 7f9cf20f0b80 1 Processor -- start 2023-09-13T04:42:34.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.787+0000 7f9cf20f0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f5a2c6a10 0x564f5a2d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:34.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.787+0000 7f9cf20f0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f5a2cbfb0 0x564f5a2d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:34.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.787+0000 7f9cf20f0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f5a2cc8d0 0x564f5a2be820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:34.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.787+0000 7f9cf20f0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5a1a5bb0 con 0x564f5a2cbfb0 2023-09-13T04:42:34.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.787+0000 7f9cf20f0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5a1a58b0 con 0x564f5a2cc8d0 2023-09-13T04:42:34.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.787+0000 7f9cf20f0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5a089920 con 0x564f5a2c6a10 2023-09-13T04:42:34.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.787+0000 7f9cf20f0b80 10 client.0 _ll_register_callbacks cb 0x564f5a1a3680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:42:34.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.789+0000 7f9cdcd9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f5a2cc8d0 0x564f5a2be820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:34.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.789+0000 7f9cdcd9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f5a2cc8d0 0x564f5a2be820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:46354/0 (socket says 172.21.15.2:46354) 2023-09-13T04:42:34.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.789+0000 7f9cdcd9c700 1 -- 172.21.15.2:0/1567264212 learned_addr learned my addr 172.21.15.2:0/1567264212 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:34.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.789+0000 7f9cd77fe700 1 --2- 172.21.15.2:0/1567264212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f5a2cbfb0 0x564f5a2d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:34.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.790+0000 7f9cdcd9c700 1 -- 172.21.15.2:0/1567264212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f5a2c6a10 msgr2=0x564f5a2d0380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:42:34.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.790+0000 7f9cdcd9c700 1 --2- 172.21.15.2:0/1567264212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f5a2c6a10 0x564f5a2d0380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.790+0000 7f9cdcd9c700 1 -- 172.21.15.2:0/1567264212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f5a2cbfb0 msgr2=0x564f5a2d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:34.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.790+0000 7f9cdcd9c700 1 --2- 172.21.15.2:0/1567264212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f5a2cbfb0 0x564f5a2d08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.790+0000 7f9cdcd9c700 1 -- 172.21.15.2:0/1567264212 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc8009940 con 0x564f5a2cc8d0 2023-09-13T04:42:34.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.791+0000 7f9cdcd9c700 1 --2- 172.21.15.2:0/1567264212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f5a2cc8d0 0x564f5a2be820 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9cd00166d0 tx=0x7f9cd0006d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c665b67b65d1342 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:34.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.792+0000 7f9cb77fe700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:42:34.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.792+0000 7f9cb77fe700 1 -- 172.21.15.2:0/1567264212 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd001b070 con 0x564f5a2cc8d0 2023-09-13T04:42:34.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.792+0000 7f9cb77fe700 1 -- 172.21.15.2:0/1567264212 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f9cd0012440 con 0x564f5a2cc8d0 2023-09-13T04:42:34.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.792+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/1567264212 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f5a2a8670 con 0x564f5a2cc8d0 2023-09-13T04:42:34.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.794+0000 7f9cb77fe700 1 -- 172.21.15.2:0/1567264212 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cd0012600 con 0x564f5a2cc8d0 2023-09-13T04:42:34.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.799+0000 7f9cf20f0b80 10 client.26084 fetch_fsmap learned FSMap version 27 2023-09-13T04:42:34.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.799+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/1567264212 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564f5a21f7a0 con 0x564f5a2cc8d0 2023-09-13T04:42:34.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.804+0000 7f9cb77fe700 1 -- 172.21.15.2:0/1567264212 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564f5a2a8670 con 0x564f5a2cc8d0 2023-09-13T04:42:34.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.804+0000 7f9cb77fe700 1 -- 172.21.15.2:0/1567264212 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9cd0021710 con 0x564f5a2cc8d0 2023-09-13T04:42:34.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.805+0000 7f9cf20f0b80 10 client.26084 fetch_fsmap finished waiting for FSMap version 27 2023-09-13T04:42:34.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.805+0000 7f9cf20f0b80 10 client.26084 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-09-13T04:42:34.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.805+0000 7f9cf20f0b80 20 client.26084 populate_metadata read hostname 'smithi002' 2023-09-13T04:42:34.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.805+0000 7f9cf20f0b80 1 --2- 172.21.15.2:0/1567264212 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x564f5a2c2d50 0x564f5a2c5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:34.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.805+0000 7f9cf20f0b80 4 client.26084 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:42:34.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.805+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/1567264212 --> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f5a200970 con 0x564f5a2c2d50 2023-09-13T04:42:34.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.806+0000 7f9cd7fff700 1 --2- 172.21.15.2:0/1567264212 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x564f5a2c2d50 0x564f5a2c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:34.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.806+0000 7f9cd7fff700 1 --2- 172.21.15.2:0/1567264212 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x564f5a2c2d50 0x564f5a2c5170 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c16c573559f4ab36 server_cookie=795ec9079646b768 in_seq=0 out_seq=0 2023-09-13T04:42:34.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.807+0000 7f9cb77fe700 10 client.26084 ms_handle_connect on v2:172.21.15.156:6838/1785833227 2023-09-13T04:42:34.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.807+0000 7f9cb77fe700 1 -- 172.21.15.2:0/1567264212 <== mds.2 v2:172.21.15.156:6838/1785833227 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564f5a200970 con 0x564f5a2c2d50 2023-09-13T04:42:34.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.807+0000 7f9cb77fe700 10 client.26084 handle_command_reply: tid=0 2023-09-13T04:42:34.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.807+0000 7f9cf20f0b80 1 client.26084 shutdown 2023-09-13T04:42:34.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.822+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/1567264212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f5a2cc8d0 msgr2=0x564f5a2be820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:34.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.822+0000 7f9cf20f0b80 1 --2- 172.21.15.2:0/1567264212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f5a2cc8d0 0x564f5a2be820 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9cd00166d0 tx=0x7f9cd0006d00 comp rx=0 tx=0).stop 2023-09-13T04:42:34.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.824+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/1567264212 shutdown_connections 2023-09-13T04:42:34.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.824+0000 7f9cf20f0b80 1 --2- 172.21.15.2:0/1567264212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f5a2cc8d0 0x564f5a2be820 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.824+0000 7f9cf20f0b80 1 --2- 172.21.15.2:0/1567264212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f5a2c6a10 0x564f5a2d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.824+0000 7f9cf20f0b80 1 --2- 172.21.15.2:0/1567264212 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x564f5a2c2d50 0x564f5a2c5170 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.824+0000 7f9cf20f0b80 1 --2- 172.21.15.2:0/1567264212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f5a2cbfb0 0x564f5a2d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.824+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/1567264212 >> 172.21.15.2:0/1567264212 conn(0x564f5a213c80 msgr2=0x564f5a2aa7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:34.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.824+0000 7f9cb77fe700 0 client.26084 ms_handle_reset on v2:172.21.15.156:6838/1785833227 2023-09-13T04:42:34.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.825+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/1567264212 shutdown_connections 2023-09-13T04:42:34.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.825+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/1567264212 wait complete. 2023-09-13T04:42:34.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.826+0000 7f9cf20f0b80 20 client.26084 trim_cache size 0 max 16384 2023-09-13T04:42:34.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.849+0000 7f9cf20f0b80 1 Processor -- start 2023-09-13T04:42:34.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.850+0000 7f9cf20f0b80 1 -- start start 2023-09-13T04:42:34.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.850+0000 7f9cf20f0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f5a2c6a10 0x564f5a2cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:34.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.850+0000 7f9cf20f0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f5a2cbfb0 0x564f5a2d0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:34.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.850+0000 7f9cf20f0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f5a2cc8d0 0x564f5a2aa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:34.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.850+0000 7f9cf20f0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5a2b04f0 con 0x564f5a2cbfb0 2023-09-13T04:42:34.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.850+0000 7f9cf20f0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5a2b1a50 con 0x564f5a2c6a10 2023-09-13T04:42:34.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.850+0000 7f9cf20f0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5a2b54c0 con 0x564f5a2cc8d0 2023-09-13T04:42:34.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.851+0000 7f9cd7fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f5a2c6a10 0x564f5a2cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:34.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.851+0000 7f9cd7fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f5a2c6a10 0x564f5a2cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:46364/0 (socket says 172.21.15.2:46364) 2023-09-13T04:42:34.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.851+0000 7f9cd7fff700 1 -- 172.21.15.2:0/2876281467 learned_addr learned my addr 172.21.15.2:0/2876281467 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:34.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.851+0000 7f9cdcd9c700 1 --2- 172.21.15.2:0/2876281467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f5a2cc8d0 0x564f5a2aa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:34.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.851+0000 7f9cd7fff700 1 -- 172.21.15.2:0/2876281467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f5a2cc8d0 msgr2=0x564f5a2aa8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:34.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.851+0000 7f9cd7fff700 1 --2- 172.21.15.2:0/2876281467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f5a2cc8d0 0x564f5a2aa8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.851+0000 7f9cd7fff700 1 -- 172.21.15.2:0/2876281467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f5a2cbfb0 msgr2=0x564f5a2d0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:34.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.851+0000 7f9cd7fff700 1 --2- 172.21.15.2:0/2876281467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f5a2cbfb0 0x564f5a2d0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.851+0000 7f9cd7fff700 1 -- 172.21.15.2:0/2876281467 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f5a033de0 con 0x564f5a2c6a10 2023-09-13T04:42:34.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.851+0000 7f9cdcd9c700 1 --2- 172.21.15.2:0/2876281467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f5a2cc8d0 0x564f5a2aa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:42:34.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.851+0000 7f9cd7fff700 1 --2- 172.21.15.2:0/2876281467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f5a2c6a10 0x564f5a2cdd90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9cc801c040 tx=0x7f9cc8005960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eca1980c97881df9 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:34.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.852+0000 7f9cbeffd700 1 -- 172.21.15.2:0/2876281467 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8027ce0 con 0x564f5a2c6a10 2023-09-13T04:42:34.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.852+0000 7f9cbeffd700 1 -- 172.21.15.2:0/2876281467 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cc802b070 con 0x564f5a2c6a10 2023-09-13T04:42:34.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.852+0000 7f9cbeffd700 1 -- 172.21.15.2:0/2876281467 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8003680 con 0x564f5a2c6a10 2023-09-13T04:42:34.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.853+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/2876281467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f5a2c6a10 msgr2=0x564f5a2cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:34.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.853+0000 7f9cf20f0b80 1 --2- 172.21.15.2:0/2876281467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f5a2c6a10 0x564f5a2cdd90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9cc801c040 tx=0x7f9cc8005960 comp rx=0 tx=0).stop 2023-09-13T04:42:34.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.853+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/2876281467 shutdown_connections 2023-09-13T04:42:34.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.853+0000 7f9cf20f0b80 1 --2- 172.21.15.2:0/2876281467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f5a2cc8d0 0x564f5a2aa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.853+0000 7f9cf20f0b80 1 --2- 172.21.15.2:0/2876281467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f5a2c6a10 0x564f5a2cdd90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.853+0000 7f9cf20f0b80 1 --2- 172.21.15.2:0/2876281467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f5a2cbfb0 0x564f5a2d0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.853+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/2876281467 >> 172.21.15.2:0/2876281467 conn(0x564f5a213c80 msgr2=0x564f5a2d0150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:34.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.854+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/2876281467 shutdown_connections 2023-09-13T04:42:34.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.854+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/2876281467 wait complete. 2023-09-13T04:42:34.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.854+0000 7f9cf20f0b80 1 -- start start 2023-09-13T04:42:34.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.854+0000 7f9cf20f0b80 1 Processor -- start 2023-09-13T04:42:34.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.855+0000 7f9cf20f0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f5a2c6a10 0x564f5a2b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:34.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.855+0000 7f9cf20f0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f5a2cbfb0 0x564f5a2b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:34.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.855+0000 7f9cf20f0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f5a2cc8d0 0x564f5a333150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:34.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.855+0000 7f9cf20f0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5a333690 con 0x564f5a2c6a10 2023-09-13T04:42:34.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.855+0000 7f9cf20f0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5a3337d0 con 0x564f5a2cc8d0 2023-09-13T04:42:34.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.855+0000 7f9cf20f0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5a333910 con 0x564f5a2cbfb0 2023-09-13T04:42:34.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.855+0000 7f9cf20f0b80 10 client.0 _ll_register_callbacks cb 0x564f5a2c9f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:42:34.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.856+0000 7f9cd77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f5a2cbfb0 0x564f5a2b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:34.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.856+0000 7f9cd77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f5a2cbfb0 0x564f5a2b69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:51264/0 (socket says 172.21.15.2:51264) 2023-09-13T04:42:34.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.856+0000 7f9cd77fe700 1 -- 172.21.15.2:0/659063977 learned_addr learned my addr 172.21.15.2:0/659063977 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:34.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.856+0000 7f9cd77fe700 1 -- 172.21.15.2:0/659063977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f5a2cc8d0 msgr2=0x564f5a333150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:42:34.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.856+0000 7f9cd77fe700 1 --2- 172.21.15.2:0/659063977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f5a2cc8d0 0x564f5a333150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.856+0000 7f9cd77fe700 1 -- 172.21.15.2:0/659063977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f5a2c6a10 msgr2=0x564f5a2b64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:34.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.856+0000 7f9cd77fe700 1 --2- 172.21.15.2:0/659063977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f5a2c6a10 0x564f5a2b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.856+0000 7f9cd77fe700 1 -- 172.21.15.2:0/659063977 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f5a21f7a0 con 0x564f5a2cbfb0 2023-09-13T04:42:34.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.857+0000 7f9cd77fe700 1 --2- 172.21.15.2:0/659063977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f5a2cbfb0 0x564f5a2b69f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc097330 tx=0x7f9ccc09a840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4e335194ddcd0bb server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:34.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.857+0000 7f9cb77fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:42:34.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.857+0000 7f9cb77fe700 1 -- 172.21.15.2:0/659063977 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc09b290 con 0x564f5a2cbfb0 2023-09-13T04:42:34.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.857+0000 7f9cb77fe700 1 -- 172.21.15.2:0/659063977 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f9ccc09be40 con 0x564f5a2cbfb0 2023-09-13T04:42:34.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.857+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/659063977 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cc8009e60 con 0x564f5a2cbfb0 2023-09-13T04:42:34.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.857+0000 7f9cb77fe700 1 -- 172.21.15.2:0/659063977 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ccc0cd5c0 con 0x564f5a2cbfb0 2023-09-13T04:42:34.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.857+0000 7f9cf20f0b80 10 client.26092 fetch_fsmap learned FSMap version 27 2023-09-13T04:42:34.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.857+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/659063977 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564f5a087630 con 0x564f5a2cbfb0 2023-09-13T04:42:34.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.857+0000 7f9cb77fe700 1 -- 172.21.15.2:0/659063977 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ccc07d020 con 0x564f5a2cbfb0 2023-09-13T04:42:34.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.858+0000 7f9cb77fe700 1 -- 172.21.15.2:0/659063977 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9ccc0a2980 con 0x564f5a2cbfb0 2023-09-13T04:42:34.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.858+0000 7f9cf20f0b80 10 client.26092 fetch_fsmap finished waiting for FSMap version 27 2023-09-13T04:42:34.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.858+0000 7f9cf20f0b80 10 client.26092 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-09-13T04:42:34.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.858+0000 7f9cf20f0b80 20 client.26092 populate_metadata read hostname 'smithi002' 2023-09-13T04:42:34.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.858+0000 7f9cf20f0b80 1 --2- 172.21.15.2:0/659063977 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x564f5a334d80 0x564f5a3371a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:34.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.858+0000 7f9cf20f0b80 4 client.26092 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-09-13T04:42:34.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.858+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/659063977 --> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564f5a1c1050 con 0x564f5a334d80 2023-09-13T04:42:34.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.858+0000 7f9cd7fff700 1 --2- 172.21.15.2:0/659063977 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x564f5a334d80 0x564f5a3371a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:34.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.858+0000 7f9cb77fe700 10 client.26092 ms_handle_connect on v2:172.21.15.156:6838/1785833227 2023-09-13T04:42:34.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.859+0000 7f9cd7fff700 1 --2- 172.21.15.2:0/659063977 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x564f5a334d80 0x564f5a3371a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf47d903aed4ea9e server_cookie=ea64e2fae153d01f in_seq=0 out_seq=0 2023-09-13T04:42:34.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.859+0000 7f9cb77fe700 1 -- 172.21.15.2:0/659063977 <== mds.2 v2:172.21.15.156:6838/1785833227 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564f5a1c1050 con 0x564f5a334d80 2023-09-13T04:42:34.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.859+0000 7f9cb77fe700 10 client.26092 handle_command_reply: tid=0 2023-09-13T04:42:34.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.859+0000 7f9cf20f0b80 1 client.26092 shutdown 2023-09-13T04:42:34.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.859+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/659063977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f5a2cbfb0 msgr2=0x564f5a2b69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:34.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.859+0000 7f9cf20f0b80 1 --2- 172.21.15.2:0/659063977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f5a2cbfb0 0x564f5a2b69f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc097330 tx=0x7f9ccc09a840 comp rx=0 tx=0).stop 2023-09-13T04:42:34.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.859+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/659063977 shutdown_connections 2023-09-13T04:42:34.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.859+0000 7f9cf20f0b80 1 --2- 172.21.15.2:0/659063977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f5a2cc8d0 0x564f5a333150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.859+0000 7f9cf20f0b80 1 --2- 172.21.15.2:0/659063977 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x564f5a334d80 0x564f5a3371a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.860+0000 7f9cb77fe700 0 client.26092 ms_handle_reset on v2:172.21.15.156:6838/1785833227 2023-09-13T04:42:34.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.859+0000 7f9cf20f0b80 1 --2- 172.21.15.2:0/659063977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f5a2c6a10 0x564f5a2b64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.860+0000 7f9cf20f0b80 1 --2- 172.21.15.2:0/659063977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f5a2cbfb0 0x564f5a2b69f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.860+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/659063977 >> 172.21.15.2:0/659063977 conn(0x564f5a213c80 msgr2=0x564f5a2c5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:34.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.860+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/659063977 shutdown_connections 2023-09-13T04:42:34.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.860+0000 7f9cf20f0b80 1 -- 172.21.15.2:0/659063977 wait complete. 2023-09-13T04:42:34.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.860+0000 7f9cf20f0b80 20 client.26092 trim_cache size 0 max 16384 2023-09-13T04:42:34.871 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-09-13T04:42:34.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.861+0000 7f9cde7ff700 1 -- 172.21.15.2:0/205431551 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f9cc00a08f0 msgr2=0x7f9cc00a2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:34.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.861+0000 7f9cde7ff700 1 --2- 172.21.15.2:0/205431551 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f9cc00a08f0 0x7f9cc00a2db0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8067c70 tx=0x7f9cd0001500 comp rx=0 tx=0).stop 2023-09-13T04:42:34.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.861+0000 7f9cde7ff700 1 -- 172.21.15.2:0/205431551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cd8107d70 msgr2=0x7f9cd819d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:34.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.861+0000 7f9cde7ff700 1 --2- 172.21.15.2:0/205431551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cd8107d70 0x7f9cd819d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc000c00 tx=0x7f9ccc0057f0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.861+0000 7f9cde7ff700 1 -- 172.21.15.2:0/205431551 shutdown_connections 2023-09-13T04:42:34.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.861+0000 7f9cde7ff700 1 --2- 172.21.15.2:0/205431551 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f9cc00a08f0 0x7f9cc00a2db0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.861+0000 7f9cde7ff700 1 --2- 172.21.15.2:0/205431551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cd810d330 0x7f9cd81a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.861+0000 7f9cde7ff700 1 --2- 172.21.15.2:0/205431551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cd81073d0 0x7f9cd819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.861+0000 7f9cde7ff700 1 --2- 172.21.15.2:0/205431551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cd8107d70 0x7f9cd819d990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:34.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.861+0000 7f9cde7ff700 1 -- 172.21.15.2:0/205431551 >> 172.21.15.2:0/205431551 conn(0x7f9cd8076b20 msgr2=0x7f9cd8077a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:34.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.861+0000 7f9cde7ff700 1 -- 172.21.15.2:0/205431551 shutdown_connections 2023-09-13T04:42:34.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:34.862+0000 7f9cde7ff700 1 -- 172.21.15.2:0/205431551 wait complete. 2023-09-13T04:42:34.905 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-13T04:42:35.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.262+0000 7f8e1bfff700 1 -- 172.21.15.2:0/2878400134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e14103ab0 msgr2=0x7f8e14107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:35.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.262+0000 7f8e1bfff700 1 --2- 172.21.15.2:0/2878400134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e14103ab0 0x7f8e14107af0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009990 tx=0x7f8e04009ca0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.263+0000 7f8e1bfff700 1 -- 172.21.15.2:0/2878400134 shutdown_connections 2023-09-13T04:42:35.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.263+0000 7f8e1bfff700 1 --2- 172.21.15.2:0/2878400134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e14108220 0x7f8e1410fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.263+0000 7f8e1bfff700 1 --2- 172.21.15.2:0/2878400134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e14103100 0x7f8e141034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.263+0000 7f8e1bfff700 1 --2- 172.21.15.2:0/2878400134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e14103ab0 0x7f8e14107af0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.263+0000 7f8e1bfff700 1 -- 172.21.15.2:0/2878400134 >> 172.21.15.2:0/2878400134 conn(0x7f8e140fe940 msgr2=0x7f8e14100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:35.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.263+0000 7f8e1bfff700 1 -- 172.21.15.2:0/2878400134 shutdown_connections 2023-09-13T04:42:35.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.263+0000 7f8e1bfff700 1 -- 172.21.15.2:0/2878400134 wait complete. 2023-09-13T04:42:35.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.264+0000 7f8e1bfff700 1 Processor -- start 2023-09-13T04:42:35.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.264+0000 7f8e1bfff700 1 -- start start 2023-09-13T04:42:35.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.264+0000 7f8e1bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e14103100 0x7f8e14198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.264+0000 7f8e1bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e14108220 0x7f8e141994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.264+0000 7f8e1bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e1419d8c0 0x7f8e1419dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.264+0000 7f8e1bfff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1419e340 con 0x7f8e14103100 2023-09-13T04:42:35.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.264+0000 7f8e1bfff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1419e4b0 con 0x7f8e1419d8c0 2023-09-13T04:42:35.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.264+0000 7f8e1bfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1419e670 con 0x7f8e14108220 2023-09-13T04:42:35.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.265+0000 7f8e1959a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e14108220 0x7f8e141994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.265+0000 7f8e19d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e14103100 0x7f8e14198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.265+0000 7f8e1959a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e14108220 0x7f8e141994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:51284/0 (socket says 172.21.15.2:51284) 2023-09-13T04:42:35.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.265+0000 7f8e1a59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e1419d8c0 0x7f8e1419dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.265+0000 7f8e19d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e14103100 0x7f8e14198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37162/0 (socket says 172.21.15.2:37162) 2023-09-13T04:42:35.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.265+0000 7f8e19d9b700 1 -- 172.21.15.2:0/3205040380 learned_addr learned my addr 172.21.15.2:0/3205040380 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:35.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.265+0000 7f8e19d9b700 1 -- 172.21.15.2:0/3205040380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e14108220 msgr2=0x7f8e141994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:35.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.265+0000 7f8e19d9b700 1 --2- 172.21.15.2:0/3205040380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e14108220 0x7f8e141994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.265+0000 7f8e19d9b700 1 -- 172.21.15.2:0/3205040380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e1419d8c0 msgr2=0x7f8e1419dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:35.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.265+0000 7f8e19d9b700 1 --2- 172.21.15.2:0/3205040380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e1419d8c0 0x7f8e1419dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.265+0000 7f8e19d9b700 1 -- 172.21.15.2:0/3205040380 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e10009520 con 0x7f8e14103100 2023-09-13T04:42:35.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.265+0000 7f8e1959a700 1 --2- 172.21.15.2:0/3205040380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e14108220 0x7f8e141994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:42:35.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.265+0000 7f8e1a59c700 1 --2- 172.21.15.2:0/3205040380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e1419d8c0 0x7f8e1419dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:42:35.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.266+0000 7f8e19d9b700 1 --2- 172.21.15.2:0/3205040380 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e14103100 0x7f8e14198f70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8e1000eb30 tx=0x7f8e1000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:35.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.266+0000 7f8e0affd700 1 -- 172.21.15.2:0/3205040380 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e10009880 con 0x7f8e14103100 2023-09-13T04:42:35.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.267+0000 7f8e0affd700 1 -- 172.21.15.2:0/3205040380 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e100044e0 con 0x7f8e14103100 2023-09-13T04:42:35.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.267+0000 7f8e0affd700 1 -- 172.21.15.2:0/3205040380 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e10003d80 con 0x7f8e14103100 2023-09-13T04:42:35.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.267+0000 7f8e1bfff700 1 -- 172.21.15.2:0/3205040380 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e040095f0 con 0x7f8e14103100 2023-09-13T04:42:35.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.267+0000 7f8e1bfff700 1 -- 172.21.15.2:0/3205040380 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e141a5ed0 con 0x7f8e14103100 2023-09-13T04:42:35.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.274+0000 7f8e2f9d5b80 1 Processor -- start 2023-09-13T04:42:35.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.279+0000 7f8e2f9d5b80 1 -- start start 2023-09-13T04:42:35.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.280+0000 7f8e2f9d5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645014387f0 0x564501438bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.280+0000 7f8e2f9d5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501439110 0x564501432cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.280+0000 7f8e2f9d5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564501433210 0x564501433670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.280+0000 7f8e2f9d5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564501383780 con 0x5645014387f0 2023-09-13T04:42:35.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.280+0000 7f8e2f9d5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645013810a0 con 0x564501439110 2023-09-13T04:42:35.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.280+0000 7f8e2f9d5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450137c990 con 0x564501433210 2023-09-13T04:42:35.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.280+0000 7f8e1a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645014387f0 0x564501438bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.281+0000 7f8e1a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645014387f0 0x564501438bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37172/0 (socket says 172.21.15.2:37172) 2023-09-13T04:42:35.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.281+0000 7f8e1a59c700 1 -- 172.21.15.2:0/4194051231 learned_addr learned my addr 172.21.15.2:0/4194051231 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:35.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.281+0000 7f8e1a59c700 1 -- 172.21.15.2:0/4194051231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564501433210 msgr2=0x564501433670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:42:35.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.281+0000 7f8e1a59c700 1 --2- 172.21.15.2:0/4194051231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564501433210 0x564501433670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.281+0000 7f8e1a59c700 1 -- 172.21.15.2:0/4194051231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501439110 msgr2=0x564501432cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:42:35.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.281+0000 7f8e1a59c700 1 --2- 172.21.15.2:0/4194051231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501439110 0x564501432cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.281+0000 7f8e1a59c700 1 -- 172.21.15.2:0/4194051231 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645011fc630 con 0x5645014387f0 2023-09-13T04:42:35.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.281+0000 7f8e1a59c700 1 --2- 172.21.15.2:0/4194051231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645014387f0 0x564501438bd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c003eb0 tx=0x7f8e0c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52f7274874b69ce server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:35.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.283+0000 7f8e0affd700 1 -- 172.21.15.2:0/3205040380 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f8e1000cbd0 con 0x7f8e14103100 2023-09-13T04:42:35.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.283+0000 7f8dfffff700 1 -- 172.21.15.2:0/4194051231 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0092d0 con 0x5645014387f0 2023-09-13T04:42:35.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.283+0000 7f8dfffff700 1 -- 172.21.15.2:0/4194051231 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e0c00f040 con 0x5645014387f0 2023-09-13T04:42:35.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.283+0000 7f8dfffff700 1 -- 172.21.15.2:0/4194051231 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c00e8b0 con 0x5645014387f0 2023-09-13T04:42:35.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.284+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/4194051231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645014387f0 msgr2=0x564501438bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:35.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.284+0000 7f8e2f9d5b80 1 --2- 172.21.15.2:0/4194051231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645014387f0 0x564501438bd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c003eb0 tx=0x7f8e0c009920 comp rx=0 tx=0).stop 2023-09-13T04:42:35.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.285+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/4194051231 shutdown_connections 2023-09-13T04:42:35.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.285+0000 7f8e2f9d5b80 1 --2- 172.21.15.2:0/4194051231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564501433210 0x564501433670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.285+0000 7f8e2f9d5b80 1 --2- 172.21.15.2:0/4194051231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645014387f0 0x564501438bd0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.285+0000 7f8e2f9d5b80 1 --2- 172.21.15.2:0/4194051231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501439110 0x564501432cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.285+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/4194051231 >> 172.21.15.2:0/4194051231 conn(0x564501388c80 msgr2=0x564501424f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:35.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.286+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/4194051231 shutdown_connections 2023-09-13T04:42:35.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.286+0000 7f8e0affd700 1 --2- 172.21.15.2:0/3205040380 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f8e000a07c0 0x7f8e000a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.286+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/4194051231 wait complete. 2023-09-13T04:42:35.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.286+0000 7f8e0affd700 1 -- 172.21.15.2:0/3205040380 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f8e10014070 con 0x7f8e14103100 2023-09-13T04:42:35.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.286+0000 7f8e1959a700 1 --2- 172.21.15.2:0/3205040380 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f8e000a07c0 0x7f8e000a2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.289+0000 7f8e2f9d5b80 1 -- start start 2023-09-13T04:42:35.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.289+0000 7f8e2f9d5b80 1 Processor -- start 2023-09-13T04:42:35.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.291+0000 7f8e2f9d5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501433210 0x56450143f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.291+0000 7f8e2f9d5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645014387f0 0x56450143c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.291+0000 7f8e2f9d5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501439110 0x56450143c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.291+0000 7f8e2f9d5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450131abb0 con 0x564501439110 2023-09-13T04:42:35.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.291+0000 7f8e2f9d5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450131a8b0 con 0x564501433210 2023-09-13T04:42:35.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.291+0000 7f8e2f9d5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645011fe920 con 0x5645014387f0 2023-09-13T04:42:35.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.291+0000 7f8e2f9d5b80 10 client.0 _ll_register_callbacks cb 0x564501318680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:42:35.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.292+0000 7f8e19d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501433210 0x56450143f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.292+0000 7f8e1959a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645014387f0 0x56450143c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.292+0000 7f8e1959a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645014387f0 0x56450143c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:51286/0 (socket says 172.21.15.2:51286) 2023-09-13T04:42:35.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.292+0000 7f8e1959a700 1 -- 172.21.15.2:0/790598037 learned_addr learned my addr 172.21.15.2:0/790598037 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:35.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.292+0000 7f8e1959a700 1 --2- 172.21.15.2:0/3205040380 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f8e000a07c0 0x7f8e000a2c80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400b2e0 tx=0x7f8e04005960 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:35.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.292+0000 7f8e1959a700 1 -- 172.21.15.2:0/790598037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501433210 msgr2=0x56450143f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:35.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.292+0000 7f8e1959a700 1 --2- 172.21.15.2:0/790598037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501433210 0x56450143f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.292+0000 7f8e1959a700 1 -- 172.21.15.2:0/790598037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501439110 msgr2=0x56450143c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:35.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.292+0000 7f8e1959a700 1 --2- 172.21.15.2:0/790598037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501439110 0x56450143c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.292+0000 7f8e1959a700 1 -- 172.21.15.2:0/790598037 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e0c003b90 con 0x5645014387f0 2023-09-13T04:42:35.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.293+0000 7f8e1959a700 1 --2- 172.21.15.2:0/790598037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645014387f0 0x56450143c350 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009990 tx=0x7f8e04009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=253e1d6e0d04754c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:35.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.293+0000 7f8dfd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:42:35.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.293+0000 7f8dfd7fa700 1 -- 172.21.15.2:0/790598037 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0400f870 con 0x5645014387f0 2023-09-13T04:42:35.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.293+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/790598037 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56450141d670 con 0x5645014387f0 2023-09-13T04:42:35.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.293+0000 7f8dfd7fa700 1 -- 172.21.15.2:0/790598037 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f8e0400f9d0 con 0x5645014387f0 2023-09-13T04:42:35.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.294+0000 7f8dfd7fa700 1 -- 172.21.15.2:0/790598037 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e04004e60 con 0x5645014387f0 2023-09-13T04:42:35.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.296+0000 7f8e2f9d5b80 10 client.26101 fetch_fsmap learned FSMap version 27 2023-09-13T04:42:35.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.296+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/790598037 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5645013947a0 con 0x5645014387f0 2023-09-13T04:42:35.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.296+0000 7f8dfd7fa700 1 -- 172.21.15.2:0/790598037 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56450141d670 con 0x5645014387f0 2023-09-13T04:42:35.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.296+0000 7f8dfd7fa700 1 -- 172.21.15.2:0/790598037 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8e04024ba0 con 0x5645014387f0 2023-09-13T04:42:35.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.296+0000 7f8e2f9d5b80 10 client.26101 fetch_fsmap finished waiting for FSMap version 27 2023-09-13T04:42:35.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.296+0000 7f8e2f9d5b80 10 client.26101 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-09-13T04:42:35.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.296+0000 7f8e2f9d5b80 20 client.26101 populate_metadata read hostname 'smithi002' 2023-09-13T04:42:35.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.296+0000 7f8e2f9d5b80 1 --2- 172.21.15.2:0/790598037 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x56450145db40 0x56450145ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.296+0000 7f8e2f9d5b80 4 client.26101 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:42:35.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.296+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/790598037 --> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564501375970 con 0x56450145db40 2023-09-13T04:42:35.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.298+0000 7f8e19d9b700 1 --2- 172.21.15.2:0/790598037 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x56450145db40 0x56450145ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.299+0000 7f8e19d9b700 1 --2- 172.21.15.2:0/790598037 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x56450145db40 0x56450145ff60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bd9dc21589ee2c0d server_cookie=37d038b32f34faf0 in_seq=0 out_seq=0 2023-09-13T04:42:35.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.299+0000 7f8dfd7fa700 10 client.26101 ms_handle_connect on v2:172.21.15.2:6840/482545391 2023-09-13T04:42:35.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.300+0000 7f8dfd7fa700 1 -- 172.21.15.2:0/790598037 <== mds.1 v2:172.21.15.2:6840/482545391 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564501375970 con 0x56450145db40 2023-09-13T04:42:35.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.300+0000 7f8dfd7fa700 10 client.26101 handle_command_reply: tid=0 2023-09-13T04:42:35.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.301+0000 7f8e2f9d5b80 1 client.26101 shutdown 2023-09-13T04:42:35.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.301+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/790598037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645014387f0 msgr2=0x56450143c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:35.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.301+0000 7f8e2f9d5b80 1 --2- 172.21.15.2:0/790598037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645014387f0 0x56450143c350 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009990 tx=0x7f8e04009ca0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.301+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/790598037 shutdown_connections 2023-09-13T04:42:35.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.301+0000 7f8e2f9d5b80 1 --2- 172.21.15.2:0/790598037 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x56450145db40 0x56450145ff60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.301+0000 7f8e2f9d5b80 1 --2- 172.21.15.2:0/790598037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501439110 0x56450143c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.301+0000 7f8e2f9d5b80 1 --2- 172.21.15.2:0/790598037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501433210 0x56450143f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.301+0000 7f8e2f9d5b80 1 --2- 172.21.15.2:0/790598037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645014387f0 0x56450143c350 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.301+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/790598037 >> 172.21.15.2:0/790598037 conn(0x564501388c80 msgr2=0x564501424e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:35.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.301+0000 7f8dfd7fa700 0 client.26101 ms_handle_reset on v2:172.21.15.2:6840/482545391 2023-09-13T04:42:35.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.302+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/790598037 shutdown_connections 2023-09-13T04:42:35.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.302+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/790598037 wait complete. 2023-09-13T04:42:35.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.302+0000 7f8e2f9d5b80 20 client.26101 trim_cache size 0 max 16384 2023-09-13T04:42:35.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.321+0000 7f8e2f9d5b80 1 Processor -- start 2023-09-13T04:42:35.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.321+0000 7f8e2f9d5b80 1 -- start start 2023-09-13T04:42:35.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.321+0000 7f8e2f9d5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501433210 0x564501454ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.321+0000 7f8e2f9d5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645014387f0 0x56450141f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.321+0000 7f8e2f9d5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501439110 0x56450143ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.321+0000 7f8e2f9d5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450145a3e0 con 0x564501433210 2023-09-13T04:42:35.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.321+0000 7f8e2f9d5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645014218e0 con 0x564501439110 2023-09-13T04:42:35.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.321+0000 7f8e2f9d5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450141d5e0 con 0x5645014387f0 2023-09-13T04:42:35.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.321+0000 7f8e1959a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645014387f0 0x56450141f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.321+0000 7f8e1959a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645014387f0 0x56450141f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:51292/0 (socket says 172.21.15.2:51292) 2023-09-13T04:42:35.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.322+0000 7f8e1959a700 1 -- 172.21.15.2:0/1446755231 learned_addr learned my addr 172.21.15.2:0/1446755231 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:35.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.322+0000 7f8e1a59c700 1 --2- 172.21.15.2:0/1446755231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501439110 0x56450143ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.322+0000 7f8e1a59c700 1 -- 172.21.15.2:0/1446755231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645014387f0 msgr2=0x56450141f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:35.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.322+0000 7f8e1a59c700 1 --2- 172.21.15.2:0/1446755231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645014387f0 0x56450141f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.322+0000 7f8e1a59c700 1 -- 172.21.15.2:0/1446755231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501433210 msgr2=0x564501454ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:35.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.322+0000 7f8e19d9b700 1 --2- 172.21.15.2:0/1446755231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501433210 0x564501454ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.322+0000 7f8e1a59c700 1 --2- 172.21.15.2:0/1446755231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501433210 0x564501454ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.322+0000 7f8e1a59c700 1 -- 172.21.15.2:0/1446755231 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564501425130 con 0x564501439110 2023-09-13T04:42:35.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.322+0000 7f8e19d9b700 1 --2- 172.21.15.2:0/1446755231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501433210 0x564501454ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:42:35.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.323+0000 7f8e1a59c700 1 --2- 172.21.15.2:0/1446755231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501439110 0x56450143ec70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c003e80 tx=0x7f8e0c0091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=758a44bb3b259183 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:35.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.323+0000 7f8e08ff9700 1 -- 172.21.15.2:0/1446755231 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c004a50 con 0x564501439110 2023-09-13T04:42:35.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.323+0000 7f8e08ff9700 1 -- 172.21.15.2:0/1446755231 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e0c004bb0 con 0x564501439110 2023-09-13T04:42:35.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.323+0000 7f8e08ff9700 1 -- 172.21.15.2:0/1446755231 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c015670 con 0x564501439110 2023-09-13T04:42:35.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.324+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/1446755231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501439110 msgr2=0x56450143ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:35.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.324+0000 7f8e2f9d5b80 1 --2- 172.21.15.2:0/1446755231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501439110 0x56450143ec70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c003e80 tx=0x7f8e0c0091d0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.325+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/1446755231 shutdown_connections 2023-09-13T04:42:35.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.325+0000 7f8e2f9d5b80 1 --2- 172.21.15.2:0/1446755231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501439110 0x56450143ec70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.325+0000 7f8e2f9d5b80 1 --2- 172.21.15.2:0/1446755231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501433210 0x564501454ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.325+0000 7f8e2f9d5b80 1 --2- 172.21.15.2:0/1446755231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645014387f0 0x56450141f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.325+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/1446755231 >> 172.21.15.2:0/1446755231 conn(0x564501388c80 msgr2=0x564501436510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:35.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.325+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/1446755231 shutdown_connections 2023-09-13T04:42:35.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.325+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/1446755231 wait complete. 2023-09-13T04:42:35.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.326+0000 7f8e2f9d5b80 1 -- start start 2023-09-13T04:42:35.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.326+0000 7f8e2f9d5b80 1 Processor -- start 2023-09-13T04:42:35.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.326+0000 7f8e2f9d5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501433210 0x564501435360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.326+0000 7f8e2f9d5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645014387f0 0x5645014358a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.326+0000 7f8e2f9d5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564501439110 0x564501434270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.326+0000 7f8e2f9d5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645014347b0 con 0x5645014387f0 2023-09-13T04:42:35.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.326+0000 7f8e2f9d5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564501434920 con 0x564501433210 2023-09-13T04:42:35.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.326+0000 7f8e2f9d5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564501434ae0 con 0x564501439110 2023-09-13T04:42:35.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.326+0000 7f8e2f9d5b80 10 client.0 _ll_register_callbacks cb 0x564501318680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:42:35.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.327+0000 7f8e1959a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645014387f0 0x5645014358a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.327+0000 7f8e1959a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645014387f0 0x5645014358a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37204/0 (socket says 172.21.15.2:37204) 2023-09-13T04:42:35.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.327+0000 7f8e1959a700 1 -- 172.21.15.2:0/1983142726 learned_addr learned my addr 172.21.15.2:0/1983142726 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:35.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.327+0000 7f8e19d9b700 1 --2- 172.21.15.2:0/1983142726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501433210 0x564501435360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.327+0000 7f8e1a59c700 1 --2- 172.21.15.2:0/1983142726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564501439110 0x564501434270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.328+0000 7f8e1a59c700 1 -- 172.21.15.2:0/1983142726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501433210 msgr2=0x564501435360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:35.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.328+0000 7f8e1a59c700 1 --2- 172.21.15.2:0/1983142726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501433210 0x564501435360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.328+0000 7f8e1a59c700 1 -- 172.21.15.2:0/1983142726 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645014387f0 msgr2=0x5645014358a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:35.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.328+0000 7f8e1a59c700 1 --2- 172.21.15.2:0/1983142726 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645014387f0 0x5645014358a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.328+0000 7f8e1a59c700 1 -- 172.21.15.2:0/1983142726 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645011fc630 con 0x564501439110 2023-09-13T04:42:35.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.328+0000 7f8e1a59c700 1 --2- 172.21.15.2:0/1983142726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564501439110 0x564501434270 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x564501425f50 tx=0x7f8e0c0095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90c00ad79d11c8d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:35.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.328+0000 7f8dfd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:42:35.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.328+0000 7f8dfd7fa700 1 -- 172.21.15.2:0/1983142726 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c00e9b0 con 0x564501439110 2023-09-13T04:42:35.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.328+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/1983142726 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564501433c80 con 0x564501439110 2023-09-13T04:42:35.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.328+0000 7f8dfd7fa700 1 -- 172.21.15.2:0/1983142726 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f8e0c015450 con 0x564501439110 2023-09-13T04:42:35.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.329+0000 7f8dfd7fa700 1 -- 172.21.15.2:0/1983142726 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e0c015610 con 0x564501439110 2023-09-13T04:42:35.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.329+0000 7f8e2f9d5b80 10 client.26107 fetch_fsmap learned FSMap version 27 2023-09-13T04:42:35.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.329+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/1983142726 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5645011a8de0 con 0x564501439110 2023-09-13T04:42:35.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.329+0000 7f8dfd7fa700 1 -- 172.21.15.2:0/1983142726 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564501433c80 con 0x564501439110 2023-09-13T04:42:35.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.329+0000 7f8dfd7fa700 1 -- 172.21.15.2:0/1983142726 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8e0c02d030 con 0x564501439110 2023-09-13T04:42:35.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.329+0000 7f8e2f9d5b80 10 client.26107 fetch_fsmap finished waiting for FSMap version 27 2023-09-13T04:42:35.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.329+0000 7f8e2f9d5b80 10 client.26107 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-09-13T04:42:35.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.329+0000 7f8e2f9d5b80 20 client.26107 populate_metadata read hostname 'smithi002' 2023-09-13T04:42:35.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.329+0000 7f8e2f9d5b80 1 --2- 172.21.15.2:0/1983142726 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x5645014b2350 0x5645014b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.329+0000 7f8e2f9d5b80 4 client.26107 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-09-13T04:42:35.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.329+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/1983142726 --> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f8e0c00f040 con 0x5645014b2350 2023-09-13T04:42:35.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.329+0000 7f8e19d9b700 1 --2- 172.21.15.2:0/1983142726 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x5645014b2350 0x5645014b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.330+0000 7f8e19d9b700 1 --2- 172.21.15.2:0/1983142726 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x5645014b2350 0x5645014b4770 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f5f6cc4663f66ba server_cookie=d8cd18909edebbca in_seq=0 out_seq=0 2023-09-13T04:42:35.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.330+0000 7f8dfd7fa700 10 client.26107 ms_handle_connect on v2:172.21.15.2:6840/482545391 2023-09-13T04:42:35.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.334+0000 7f8dfd7fa700 1 -- 172.21.15.2:0/1983142726 <== mds.1 v2:172.21.15.2:6840/482545391 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f8e0c00f040 con 0x5645014b2350 2023-09-13T04:42:35.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.334+0000 7f8dfd7fa700 10 client.26107 handle_command_reply: tid=0 2023-09-13T04:42:35.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.334+0000 7f8e2f9d5b80 1 client.26107 shutdown 2023-09-13T04:42:35.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.334+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/1983142726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564501439110 msgr2=0x564501434270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:35.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.334+0000 7f8e2f9d5b80 1 --2- 172.21.15.2:0/1983142726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564501439110 0x564501434270 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x564501425f50 tx=0x7f8e0c0095e0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.334+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/1983142726 shutdown_connections 2023-09-13T04:42:35.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.334+0000 7f8e2f9d5b80 1 --2- 172.21.15.2:0/1983142726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564501439110 0x564501434270 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.334+0000 7f8e2f9d5b80 1 --2- 172.21.15.2:0/1983142726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501433210 0x564501435360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.334+0000 7f8e2f9d5b80 1 --2- 172.21.15.2:0/1983142726 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x5645014b2350 0x5645014b4770 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.334+0000 7f8e2f9d5b80 1 --2- 172.21.15.2:0/1983142726 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645014387f0 0x5645014358a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.334+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/1983142726 >> 172.21.15.2:0/1983142726 conn(0x564501388c80 msgr2=0x564501437430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:35.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.335+0000 7f8dfd7fa700 0 client.26107 ms_handle_reset on v2:172.21.15.2:6840/482545391 2023-09-13T04:42:35.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.335+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/1983142726 shutdown_connections 2023-09-13T04:42:35.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.335+0000 7f8e2f9d5b80 1 -- 172.21.15.2:0/1983142726 wait complete. 2023-09-13T04:42:35.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.335+0000 7f8e2f9d5b80 20 client.26107 trim_cache size 0 max 16384 2023-09-13T04:42:35.344 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-09-13T04:42:35.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.336+0000 7f8e1bfff700 1 -- 172.21.15.2:0/3205040380 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f8e000a07c0 msgr2=0x7f8e000a2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:35.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.336+0000 7f8e1bfff700 1 --2- 172.21.15.2:0/3205040380 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f8e000a07c0 0x7f8e000a2c80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400b2e0 tx=0x7f8e04005960 comp rx=0 tx=0).stop 2023-09-13T04:42:35.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.336+0000 7f8e1bfff700 1 -- 172.21.15.2:0/3205040380 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e14103100 msgr2=0x7f8e14198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:35.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.336+0000 7f8e1bfff700 1 --2- 172.21.15.2:0/3205040380 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e14103100 0x7f8e14198f70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8e1000eb30 tx=0x7f8e1000ee40 comp rx=0 tx=0).stop 2023-09-13T04:42:35.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.336+0000 7f8e1bfff700 1 -- 172.21.15.2:0/3205040380 shutdown_connections 2023-09-13T04:42:35.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.336+0000 7f8e1bfff700 1 --2- 172.21.15.2:0/3205040380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e1419d8c0 0x7f8e1419dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.336+0000 7f8e1bfff700 1 --2- 172.21.15.2:0/3205040380 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f8e000a07c0 0x7f8e000a2c80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.336+0000 7f8e1bfff700 1 --2- 172.21.15.2:0/3205040380 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e14103100 0x7f8e14198f70 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.336+0000 7f8e1bfff700 1 --2- 172.21.15.2:0/3205040380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e14108220 0x7f8e141994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.336+0000 7f8e1bfff700 1 -- 172.21.15.2:0/3205040380 >> 172.21.15.2:0/3205040380 conn(0x7f8e140fe940 msgr2=0x7f8e14106ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:35.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.336+0000 7f8e1bfff700 1 -- 172.21.15.2:0/3205040380 shutdown_connections 2023-09-13T04:42:35.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.336+0000 7f8e1bfff700 1 -- 172.21.15.2:0/3205040380 wait complete. 2023-09-13T04:42:35.366 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-13T04:42:35.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:35 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/801921515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:42:35.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.652+0000 7f443edff700 1 -- 172.21.15.2:0/3180789350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4438102810 msgr2=0x7f443810acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:35.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.652+0000 7f443edff700 1 --2- 172.21.15.2:0/3180789350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4438102810 0x7f443810acf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4424009990 tx=0x7f4424009ca0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.652+0000 7f443edff700 1 -- 172.21.15.2:0/3180789350 shutdown_connections 2023-09-13T04:42:35.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.652+0000 7f443edff700 1 --2- 172.21.15.2:0/3180789350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f443810b2c0 0x7f443810d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.652+0000 7f443edff700 1 --2- 172.21.15.2:0/3180789350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4438101ef0 0x7f44381022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.652+0000 7f443edff700 1 --2- 172.21.15.2:0/3180789350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4438102810 0x7f443810acf0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.652+0000 7f443edff700 1 -- 172.21.15.2:0/3180789350 >> 172.21.15.2:0/3180789350 conn(0x7f44380fb690 msgr2=0x7f44380fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:35.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.652+0000 7f443edff700 1 -- 172.21.15.2:0/3180789350 shutdown_connections 2023-09-13T04:42:35.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.652+0000 7f443edff700 1 -- 172.21.15.2:0/3180789350 wait complete. 2023-09-13T04:42:35.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.654+0000 7f443edff700 1 Processor -- start 2023-09-13T04:42:35.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.654+0000 7f443edff700 1 -- start start 2023-09-13T04:42:35.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.655+0000 7f443edff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4438101ef0 0x7f4438198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.655+0000 7f443edff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4438102810 0x7f44381994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.655+0000 7f443edff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f443810b2c0 0x7f443819d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.655+0000 7f443edff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443819df80 con 0x7f4438102810 2023-09-13T04:42:35.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.655+0000 7f443edff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443819e0f0 con 0x7f4438101ef0 2023-09-13T04:42:35.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.655+0000 7f443edff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443819e2b0 con 0x7f443810b2c0 2023-09-13T04:42:35.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.655+0000 7f443d39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f443810b2c0 0x7f443819d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.655+0000 7f443d39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f443810b2c0 0x7f443819d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:51318/0 (socket says 172.21.15.2:51318) 2023-09-13T04:42:35.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.655+0000 7f443d39c700 1 -- 172.21.15.2:0/1328989646 learned_addr learned my addr 172.21.15.2:0/1328989646 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:35.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.655+0000 7f443cb9b700 1 --2- 172.21.15.2:0/1328989646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4438101ef0 0x7f4438198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.655+0000 7f4437fff700 1 --2- 172.21.15.2:0/1328989646 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4438102810 0x7f44381994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.656+0000 7f443cb9b700 1 -- 172.21.15.2:0/1328989646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f443810b2c0 msgr2=0x7f443819d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:35.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.656+0000 7f443cb9b700 1 --2- 172.21.15.2:0/1328989646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f443810b2c0 0x7f443819d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.656+0000 7f443cb9b700 1 -- 172.21.15.2:0/1328989646 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4438102810 msgr2=0x7f44381994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:35.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.656+0000 7f443cb9b700 1 --2- 172.21.15.2:0/1328989646 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4438102810 0x7f44381994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.656+0000 7f443cb9b700 1 -- 172.21.15.2:0/1328989646 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44240095f0 con 0x7f4438101ef0 2023-09-13T04:42:35.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.656+0000 7f443d39c700 1 --2- 172.21.15.2:0/1328989646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f443810b2c0 0x7f443819d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:42:35.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.656+0000 7f443cb9b700 1 --2- 172.21.15.2:0/1328989646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4438101ef0 0x7f4438198fa0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f442c00e9f0 tx=0x7f442c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:35.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.656+0000 7f4435ffb700 1 -- 172.21.15.2:0/1328989646 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c00c9c0 con 0x7f4438101ef0 2023-09-13T04:42:35.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.656+0000 7f4435ffb700 1 -- 172.21.15.2:0/1328989646 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f442c010430 con 0x7f4438101ef0 2023-09-13T04:42:35.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.656+0000 7f4435ffb700 1 -- 172.21.15.2:0/1328989646 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c018680 con 0x7f4438101ef0 2023-09-13T04:42:35.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.656+0000 7f443edff700 1 -- 172.21.15.2:0/1328989646 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443819e590 con 0x7f4438101ef0 2023-09-13T04:42:35.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.657+0000 7f443edff700 1 -- 172.21.15.2:0/1328989646 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44381a5dd0 con 0x7f4438101ef0 2023-09-13T04:42:35.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.661+0000 7f4452647b80 1 Processor -- start 2023-09-13T04:42:35.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.662+0000 7f4452647b80 1 -- start start 2023-09-13T04:42:35.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.662+0000 7f4452647b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ecf47efe0 0x563ecf47f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.662+0000 7f4452647b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ecf47f900 0x563ecf4794c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.662+0000 7f4452647b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ecf479a00 0x563ecf479e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.662+0000 7f4452647b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ecf3d2780 con 0x563ecf47efe0 2023-09-13T04:42:35.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.662+0000 7f4452647b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ecf3d00a0 con 0x563ecf47f900 2023-09-13T04:42:35.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.662+0000 7f4452647b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ecf3cb990 con 0x563ecf479a00 2023-09-13T04:42:35.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.663+0000 7f443d39c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ecf47efe0 0x563ecf47f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.663+0000 7f443d39c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ecf47efe0 0x563ecf47f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37234/0 (socket says 172.21.15.2:37234) 2023-09-13T04:42:35.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.663+0000 7f443d39c700 1 -- 172.21.15.2:0/1983506265 learned_addr learned my addr 172.21.15.2:0/1983506265 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:35.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.663+0000 7f4437fff700 1 --2- 172.21.15.2:0/1983506265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ecf479a00 0x563ecf479e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.663+0000 7f4435ffb700 1 -- 172.21.15.2:0/1328989646 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f442c010860 con 0x7f4438101ef0 2023-09-13T04:42:35.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.663+0000 7f443d39c700 1 -- 172.21.15.2:0/1983506265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ecf479a00 msgr2=0x563ecf479e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:35.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.663+0000 7f443d39c700 1 --2- 172.21.15.2:0/1983506265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ecf479a00 0x563ecf479e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.663+0000 7f443d39c700 1 -- 172.21.15.2:0/1983506265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ecf47f900 msgr2=0x563ecf4794c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:42:35.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.663+0000 7f443d39c700 1 --2- 172.21.15.2:0/1983506265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ecf47f900 0x563ecf4794c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.663+0000 7f443d39c700 1 -- 172.21.15.2:0/1983506265 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ecf24b630 con 0x563ecf47efe0 2023-09-13T04:42:35.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.664+0000 7f4437fff700 1 --2- 172.21.15.2:0/1983506265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ecf479a00 0x563ecf479e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:42:35.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.664+0000 7f443d39c700 1 --2- 172.21.15.2:0/1983506265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ecf47efe0 0x563ecf47f3c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4430009500 tx=0x7f4430009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea46f35c0749ef2 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:35.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.664+0000 7f4422ffd700 1 -- 172.21.15.2:0/1983506265 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443000e900 con 0x563ecf47efe0 2023-09-13T04:42:35.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.664+0000 7f4422ffd700 1 -- 172.21.15.2:0/1983506265 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f443000f040 con 0x563ecf47efe0 2023-09-13T04:42:35.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.664+0000 7f4422ffd700 1 -- 172.21.15.2:0/1983506265 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4430017d50 con 0x563ecf47efe0 2023-09-13T04:42:35.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.665+0000 7f4452647b80 1 -- 172.21.15.2:0/1983506265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ecf47efe0 msgr2=0x563ecf47f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:35.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.665+0000 7f4452647b80 1 --2- 172.21.15.2:0/1983506265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ecf47efe0 0x563ecf47f3c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4430009500 tx=0x7f4430009920 comp rx=0 tx=0).stop 2023-09-13T04:42:35.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.665+0000 7f4452647b80 1 -- 172.21.15.2:0/1983506265 shutdown_connections 2023-09-13T04:42:35.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.665+0000 7f4452647b80 1 --2- 172.21.15.2:0/1983506265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ecf479a00 0x563ecf479e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.665+0000 7f4452647b80 1 --2- 172.21.15.2:0/1983506265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ecf47efe0 0x563ecf47f3c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.665+0000 7f4452647b80 1 --2- 172.21.15.2:0/1983506265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ecf47f900 0x563ecf4794c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.665+0000 7f4452647b80 1 -- 172.21.15.2:0/1983506265 >> 172.21.15.2:0/1983506265 conn(0x563ecf3d7c80 msgr2=0x563ecf473f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:35.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.666+0000 7f4435ffb700 1 --2- 172.21.15.2:0/1328989646 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f44280a0830 0x7f44280a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.666+0000 7f4435ffb700 1 -- 172.21.15.2:0/1328989646 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f442c014070 con 0x7f4438101ef0 2023-09-13T04:42:35.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.666+0000 7f4437fff700 1 --2- 172.21.15.2:0/1328989646 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f44280a0830 0x7f44280a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.667+0000 7f4452647b80 1 -- 172.21.15.2:0/1983506265 shutdown_connections 2023-09-13T04:42:35.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.667+0000 7f4437fff700 1 --2- 172.21.15.2:0/1328989646 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f44280a0830 0x7f44280a2cf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f442400b6f0 tx=0x7f442400b280 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:35.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.667+0000 7f4452647b80 1 -- 172.21.15.2:0/1983506265 wait complete. 2023-09-13T04:42:35.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.668+0000 7f4452647b80 1 -- start start 2023-09-13T04:42:35.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.668+0000 7f4452647b80 1 Processor -- start 2023-09-13T04:42:35.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.669+0000 7f4452647b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ecf479a00 0x563ecf482d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.669+0000 7f4452647b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ecf47efe0 0x563ecf483240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.669+0000 7f4452647b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ecf483780 0x563ecf497ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.669+0000 7f4452647b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ecf369bb0 con 0x563ecf483780 2023-09-13T04:42:35.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.669+0000 7f4452647b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ecf3698b0 con 0x563ecf47efe0 2023-09-13T04:42:35.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.669+0000 7f4452647b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ecf24d920 con 0x563ecf479a00 2023-09-13T04:42:35.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.669+0000 7f4452647b80 10 client.0 _ll_register_callbacks cb 0x563ecf367680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:42:35.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.669+0000 7f443d39c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ecf483780 0x563ecf497ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.669+0000 7f4437fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ecf47efe0 0x563ecf483240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.669+0000 7f443d39c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ecf483780 0x563ecf497ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37246/0 (socket says 172.21.15.2:37246) 2023-09-13T04:42:35.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.669+0000 7f443d39c700 1 -- 172.21.15.2:0/3523471050 learned_addr learned my addr 172.21.15.2:0/3523471050 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:35.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.670+0000 7f4437fff700 1 -- 172.21.15.2:0/3523471050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ecf479a00 msgr2=0x563ecf482d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:35.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.670+0000 7f4437fff700 1 --2- 172.21.15.2:0/3523471050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ecf479a00 0x563ecf482d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.670+0000 7f4437fff700 1 -- 172.21.15.2:0/3523471050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ecf483780 msgr2=0x563ecf497ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:35.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.670+0000 7f4437fff700 1 --2- 172.21.15.2:0/3523471050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ecf483780 0x563ecf497ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.670+0000 7f4437fff700 1 -- 172.21.15.2:0/3523471050 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44300091e0 con 0x563ecf47efe0 2023-09-13T04:42:35.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.670+0000 7f4437fff700 1 --2- 172.21.15.2:0/3523471050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ecf47efe0 0x563ecf483240 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4424006010 tx=0x7f4424009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aef2aa0abf2b472d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:35.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.670+0000 7f4417fff700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:42:35.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.670+0000 7f4417fff700 1 -- 172.21.15.2:0/3523471050 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442401d040 con 0x563ecf47efe0 2023-09-13T04:42:35.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.670+0000 7f4452647b80 1 -- 172.21.15.2:0/3523471050 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ecf46c670 con 0x563ecf47efe0 2023-09-13T04:42:35.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.670+0000 7f4417fff700 1 -- 172.21.15.2:0/3523471050 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f4424004b50 con 0x563ecf47efe0 2023-09-13T04:42:35.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.671+0000 7f4417fff700 1 -- 172.21.15.2:0/3523471050 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4424004d10 con 0x563ecf47efe0 2023-09-13T04:42:35.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.671+0000 7f4452647b80 10 client.26108 fetch_fsmap learned FSMap version 27 2023-09-13T04:42:35.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.671+0000 7f4452647b80 1 -- 172.21.15.2:0/3523471050 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563ecf3e37a0 con 0x563ecf47efe0 2023-09-13T04:42:35.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.671+0000 7f4417fff700 1 -- 172.21.15.2:0/3523471050 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563ecf46c670 con 0x563ecf47efe0 2023-09-13T04:42:35.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.671+0000 7f4417fff700 1 -- 172.21.15.2:0/3523471050 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f442400f460 con 0x563ecf47efe0 2023-09-13T04:42:35.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.671+0000 7f4452647b80 10 client.26108 fetch_fsmap finished waiting for FSMap version 27 2023-09-13T04:42:35.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.671+0000 7f4452647b80 10 client.26108 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:42:35.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.671+0000 7f4452647b80 20 client.26108 populate_metadata read hostname 'smithi002' 2023-09-13T04:42:35.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.671+0000 7f4452647b80 1 --2- 172.21.15.2:0/3523471050 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x563ecf49be90 0x563ecf49e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.671+0000 7f4452647b80 4 client.26108 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:42:35.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.671+0000 7f4452647b80 1 -- 172.21.15.2:0/3523471050 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ecf3c4970 con 0x563ecf49be90 2023-09-13T04:42:35.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.672+0000 7f443cb9b700 1 --2- 172.21.15.2:0/3523471050 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x563ecf49be90 0x563ecf49e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.672+0000 7f4417fff700 10 client.26108 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:35.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.672+0000 7f443cb9b700 1 --2- 172.21.15.2:0/3523471050 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x563ecf49be90 0x563ecf49e2b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfe7568a7a2b4948 server_cookie=1f68a3299fbe0a03 in_seq=0 out_seq=0 2023-09-13T04:42:35.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.676+0000 7f4417fff700 1 -- 172.21.15.2:0/3523471050 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563ecf3c4970 con 0x563ecf49be90 2023-09-13T04:42:35.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.676+0000 7f4417fff700 10 client.26108 handle_command_reply: tid=0 2023-09-13T04:42:35.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.676+0000 7f4452647b80 1 client.26108 shutdown 2023-09-13T04:42:35.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.676+0000 7f4452647b80 1 -- 172.21.15.2:0/3523471050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ecf47efe0 msgr2=0x563ecf483240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:35.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.676+0000 7f4452647b80 1 --2- 172.21.15.2:0/3523471050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ecf47efe0 0x563ecf483240 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4424006010 tx=0x7f4424009ca0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.676+0000 7f4452647b80 1 -- 172.21.15.2:0/3523471050 shutdown_connections 2023-09-13T04:42:35.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.676+0000 7f4452647b80 1 --2- 172.21.15.2:0/3523471050 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x563ecf49be90 0x563ecf49e2b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.676+0000 7f4452647b80 1 --2- 172.21.15.2:0/3523471050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ecf483780 0x563ecf497ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.676+0000 7f4417fff700 0 client.26108 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:35.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.676+0000 7f4452647b80 1 --2- 172.21.15.2:0/3523471050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ecf479a00 0x563ecf482d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.676+0000 7f4452647b80 1 --2- 172.21.15.2:0/3523471050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ecf47efe0 0x563ecf483240 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.676+0000 7f4452647b80 1 -- 172.21.15.2:0/3523471050 >> 172.21.15.2:0/3523471050 conn(0x563ecf3d7c80 msgr2=0x563ecf46e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:35.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.677+0000 7f4452647b80 1 -- 172.21.15.2:0/3523471050 shutdown_connections 2023-09-13T04:42:35.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.677+0000 7f4452647b80 1 -- 172.21.15.2:0/3523471050 wait complete. 2023-09-13T04:42:35.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.677+0000 7f4452647b80 20 client.26108 trim_cache size 0 max 16384 2023-09-13T04:42:35.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:35 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/801921515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:42:35.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.699+0000 7f4452647b80 1 Processor -- start 2023-09-13T04:42:35.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.699+0000 7f4452647b80 1 -- start start 2023-09-13T04:42:35.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.699+0000 7f4452647b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ecf479a00 0x563ecf4794c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.699+0000 7f4452647b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ecf47efe0 0x563ecf497a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.699+0000 7f4452647b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ecf483780 0x563ecf492ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.699+0000 7f4452647b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ecf498920 con 0x563ecf483780 2023-09-13T04:42:35.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.699+0000 7f4452647b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ecf4750a0 con 0x563ecf479a00 2023-09-13T04:42:35.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.699+0000 7f4452647b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ecf474de0 con 0x563ecf47efe0 2023-09-13T04:42:35.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.700+0000 7f443cb9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ecf479a00 0x563ecf4794c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.700+0000 7f4437fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ecf47efe0 0x563ecf497a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.700+0000 7f443cb9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ecf479a00 0x563ecf4794c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:46478/0 (socket says 172.21.15.2:46478) 2023-09-13T04:42:35.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.700+0000 7f443cb9b700 1 -- 172.21.15.2:0/1902626215 learned_addr learned my addr 172.21.15.2:0/1902626215 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:35.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.700+0000 7f443d39c700 1 --2- 172.21.15.2:0/1902626215 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ecf483780 0x563ecf492ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.700+0000 7f4437fff700 1 -- 172.21.15.2:0/1902626215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ecf479a00 msgr2=0x563ecf4794c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:35.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.700+0000 7f4437fff700 1 --2- 172.21.15.2:0/1902626215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ecf479a00 0x563ecf4794c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.700+0000 7f4437fff700 1 -- 172.21.15.2:0/1902626215 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ecf483780 msgr2=0x563ecf492ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:35.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.700+0000 7f4437fff700 1 --2- 172.21.15.2:0/1902626215 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ecf483780 0x563ecf492ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.700+0000 7f4437fff700 1 -- 172.21.15.2:0/1902626215 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ecf4a2ea0 con 0x563ecf47efe0 2023-09-13T04:42:35.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.700+0000 7f443cb9b700 1 --2- 172.21.15.2:0/1902626215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ecf479a00 0x563ecf4794c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:42:35.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.700+0000 7f4437fff700 1 --2- 172.21.15.2:0/1902626215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ecf47efe0 0x563ecf497a90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f442400f840 tx=0x7f44240098c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b0aa7eba5036025 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:35.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.701+0000 7f44237fe700 1 -- 172.21.15.2:0/1902626215 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442401d070 con 0x563ecf47efe0 2023-09-13T04:42:35.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.701+0000 7f44237fe700 1 -- 172.21.15.2:0/1902626215 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f442401e040 con 0x563ecf47efe0 2023-09-13T04:42:35.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.701+0000 7f44237fe700 1 -- 172.21.15.2:0/1902626215 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424024790 con 0x563ecf47efe0 2023-09-13T04:42:35.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.701+0000 7f4452647b80 1 -- 172.21.15.2:0/1902626215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ecf47efe0 msgr2=0x563ecf497a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:35.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.701+0000 7f4452647b80 1 --2- 172.21.15.2:0/1902626215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ecf47efe0 0x563ecf497a90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f442400f840 tx=0x7f44240098c0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.701+0000 7f4452647b80 1 -- 172.21.15.2:0/1902626215 shutdown_connections 2023-09-13T04:42:35.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.701+0000 7f4452647b80 1 --2- 172.21.15.2:0/1902626215 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ecf483780 0x563ecf492ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.701+0000 7f4452647b80 1 --2- 172.21.15.2:0/1902626215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ecf479a00 0x563ecf4794c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.701+0000 7f4452647b80 1 --2- 172.21.15.2:0/1902626215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ecf47efe0 0x563ecf497a90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.701+0000 7f4452647b80 1 -- 172.21.15.2:0/1902626215 >> 172.21.15.2:0/1902626215 conn(0x563ecf3d7c80 msgr2=0x563ecf479de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:35.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.702+0000 7f4452647b80 1 -- 172.21.15.2:0/1902626215 shutdown_connections 2023-09-13T04:42:35.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.702+0000 7f4452647b80 1 -- 172.21.15.2:0/1902626215 wait complete. 2023-09-13T04:42:35.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.702+0000 7f4452647b80 1 -- start start 2023-09-13T04:42:35.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.703+0000 7f4452647b80 1 Processor -- start 2023-09-13T04:42:35.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.703+0000 7f4452647b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ecf479a00 0x563ecf4e74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.703+0000 7f4452647b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ecf47efe0 0x563ecf4e7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.703+0000 7f4452647b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ecf483780 0x563ecf491f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.703+0000 7f4452647b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ecf492640 con 0x563ecf483780 2023-09-13T04:42:35.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.703+0000 7f4452647b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ecf4927b0 con 0x563ecf47efe0 2023-09-13T04:42:35.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.703+0000 7f4452647b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ecf492970 con 0x563ecf479a00 2023-09-13T04:42:35.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.703+0000 7f4452647b80 10 client.0 _ll_register_callbacks cb 0x563ecf367680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:42:35.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.704+0000 7f4437fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ecf47efe0 0x563ecf4e7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.704+0000 7f443cb9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ecf479a00 0x563ecf4e74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.704+0000 7f4437fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ecf47efe0 0x563ecf4e7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:46482/0 (socket says 172.21.15.2:46482) 2023-09-13T04:42:35.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.704+0000 7f4437fff700 1 -- 172.21.15.2:0/2207091068 learned_addr learned my addr 172.21.15.2:0/2207091068 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:35.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.704+0000 7f4437fff700 1 -- 172.21.15.2:0/2207091068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ecf479a00 msgr2=0x563ecf4e74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:35.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.704+0000 7f4437fff700 1 --2- 172.21.15.2:0/2207091068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ecf479a00 0x563ecf4e74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.704+0000 7f443d39c700 1 --2- 172.21.15.2:0/2207091068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ecf483780 0x563ecf491f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.704+0000 7f4437fff700 1 -- 172.21.15.2:0/2207091068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ecf483780 msgr2=0x563ecf491f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:35.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.704+0000 7f4437fff700 1 --2- 172.21.15.2:0/2207091068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ecf483780 0x563ecf491f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.704+0000 7f4437fff700 1 -- 172.21.15.2:0/2207091068 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44381a5dd0 con 0x563ecf47efe0 2023-09-13T04:42:35.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.704+0000 7f443cb9b700 1 --2- 172.21.15.2:0/2207091068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ecf479a00 0x563ecf4e74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:42:35.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.704+0000 7f443d39c700 1 --2- 172.21.15.2:0/2207091068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ecf483780 0x563ecf491f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:42:35.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.704+0000 7f4437fff700 1 --2- 172.21.15.2:0/2207091068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ecf47efe0 0x563ecf4e7a00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f442400f7c0 tx=0x7f442400b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad52ca05f75236b6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:35.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.704+0000 7f4417fff700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:42:35.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.705+0000 7f4417fff700 1 -- 172.21.15.2:0/2207091068 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442401d040 con 0x563ecf47efe0 2023-09-13T04:42:35.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.705+0000 7f4452647b80 1 -- 172.21.15.2:0/2207091068 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f443000f040 con 0x563ecf47efe0 2023-09-13T04:42:35.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.705+0000 7f4417fff700 1 -- 172.21.15.2:0/2207091068 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f44240049d0 con 0x563ecf47efe0 2023-09-13T04:42:35.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.705+0000 7f4417fff700 1 -- 172.21.15.2:0/2207091068 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4424004b90 con 0x563ecf47efe0 2023-09-13T04:42:35.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.705+0000 7f4452647b80 10 client.26114 fetch_fsmap learned FSMap version 27 2023-09-13T04:42:35.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.705+0000 7f4452647b80 1 -- 172.21.15.2:0/2207091068 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563ecf3e37a0 con 0x563ecf47efe0 2023-09-13T04:42:35.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.705+0000 7f4417fff700 1 -- 172.21.15.2:0/2207091068 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f443000f040 con 0x563ecf47efe0 2023-09-13T04:42:35.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.706+0000 7f4417fff700 1 -- 172.21.15.2:0/2207091068 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f442402f430 con 0x563ecf47efe0 2023-09-13T04:42:35.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.706+0000 7f4452647b80 10 client.26114 fetch_fsmap finished waiting for FSMap version 27 2023-09-13T04:42:35.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.706+0000 7f4452647b80 10 client.26114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:42:35.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.706+0000 7f4452647b80 20 client.26114 populate_metadata read hostname 'smithi002' 2023-09-13T04:42:35.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.706+0000 7f4452647b80 1 --2- 172.21.15.2:0/2207091068 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x563ecf4f0030 0x563ecf4f2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:35.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.706+0000 7f4452647b80 4 client.26114 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-09-13T04:42:35.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.706+0000 7f4452647b80 1 -- 172.21.15.2:0/2207091068 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563ecf385050 con 0x563ecf4f0030 2023-09-13T04:42:35.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.706+0000 7f443cb9b700 1 --2- 172.21.15.2:0/2207091068 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x563ecf4f0030 0x563ecf4f2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:35.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.707+0000 7f443cb9b700 1 --2- 172.21.15.2:0/2207091068 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x563ecf4f0030 0x563ecf4f2450 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ee0ec2565ddd171 server_cookie=586fe966d4ca8e16 in_seq=0 out_seq=0 2023-09-13T04:42:35.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.707+0000 7f4417fff700 10 client.26114 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:35.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.708+0000 7f4417fff700 1 -- 172.21.15.2:0/2207091068 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563ecf385050 con 0x563ecf4f0030 2023-09-13T04:42:35.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.708+0000 7f4417fff700 10 client.26114 handle_command_reply: tid=0 2023-09-13T04:42:35.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.708+0000 7f4452647b80 1 client.26114 shutdown 2023-09-13T04:42:35.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.708+0000 7f4452647b80 1 -- 172.21.15.2:0/2207091068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ecf47efe0 msgr2=0x563ecf4e7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:35.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.708+0000 7f4452647b80 1 --2- 172.21.15.2:0/2207091068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ecf47efe0 0x563ecf4e7a00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f442400f7c0 tx=0x7f442400b390 comp rx=0 tx=0).stop 2023-09-13T04:42:35.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.708+0000 7f4452647b80 1 -- 172.21.15.2:0/2207091068 shutdown_connections 2023-09-13T04:42:35.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.708+0000 7f4452647b80 1 --2- 172.21.15.2:0/2207091068 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x563ecf4f0030 0x563ecf4f2450 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.708+0000 7f4452647b80 1 --2- 172.21.15.2:0/2207091068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ecf483780 0x563ecf491f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.708+0000 7f4452647b80 1 --2- 172.21.15.2:0/2207091068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ecf479a00 0x563ecf4e74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.708+0000 7f4417fff700 0 client.26114 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:35.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.708+0000 7f4452647b80 1 --2- 172.21.15.2:0/2207091068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ecf47efe0 0x563ecf4e7a00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.708+0000 7f4452647b80 1 -- 172.21.15.2:0/2207091068 >> 172.21.15.2:0/2207091068 conn(0x563ecf3d7c80 msgr2=0x563ecf47ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:35.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.709+0000 7f4452647b80 1 -- 172.21.15.2:0/2207091068 shutdown_connections 2023-09-13T04:42:35.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.709+0000 7f4452647b80 1 -- 172.21.15.2:0/2207091068 wait complete. 2023-09-13T04:42:35.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.709+0000 7f4452647b80 20 client.26114 trim_cache size 0 max 16384 2023-09-13T04:42:35.720 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-09-13T04:42:35.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.710+0000 7f443edff700 1 -- 172.21.15.2:0/1328989646 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f44280a0830 msgr2=0x7f44280a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:35.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.710+0000 7f443edff700 1 --2- 172.21.15.2:0/1328989646 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f44280a0830 0x7f44280a2cf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f442400b6f0 tx=0x7f442400b280 comp rx=0 tx=0).stop 2023-09-13T04:42:35.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.710+0000 7f443edff700 1 -- 172.21.15.2:0/1328989646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4438101ef0 msgr2=0x7f4438198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:35.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.710+0000 7f443edff700 1 --2- 172.21.15.2:0/1328989646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4438101ef0 0x7f4438198fa0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f442c00e9f0 tx=0x7f442c00ed00 comp rx=0 tx=0).stop 2023-09-13T04:42:35.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.710+0000 7f443edff700 1 -- 172.21.15.2:0/1328989646 shutdown_connections 2023-09-13T04:42:35.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.710+0000 7f443edff700 1 --2- 172.21.15.2:0/1328989646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f443810b2c0 0x7f443819d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.710+0000 7f443edff700 1 --2- 172.21.15.2:0/1328989646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4438101ef0 0x7f4438198fa0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.710+0000 7f443edff700 1 --2- 172.21.15.2:0/1328989646 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f44280a0830 0x7f44280a2cf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.710+0000 7f443edff700 1 --2- 172.21.15.2:0/1328989646 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4438102810 0x7f44381994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:35.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.710+0000 7f443edff700 1 -- 172.21.15.2:0/1328989646 >> 172.21.15.2:0/1328989646 conn(0x7f44380fb690 msgr2=0x7f443810d460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:35.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.710+0000 7f443edff700 1 -- 172.21.15.2:0/1328989646 shutdown_connections 2023-09-13T04:42:35.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:35.711+0000 7f443edff700 1 -- 172.21.15.2:0/1328989646 wait complete. 2023-09-13T04:42:35.729 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:35 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/801921515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:42:36.622 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:36 smithi156 ceph-mon[133675]: pgmap v67: 129 pgs: 129 active+clean; 181 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.4 MiB/s wr, 505 op/s 2023-09-13T04:42:36.623 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:42:36 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[171700]: *** Caught signal (Segmentation fault) ** 2023-09-13T04:42:36.623 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:42:36 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[171700]: in thread 7fe646986700 thread_name:MR_Finisher 2023-09-13T04:42:36.623 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:42:36 smithi156 systemd[1]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.c.service: Main process exited, code=exited, status=139/n/a 2023-09-13T04:42:36.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:36 smithi149 ceph-mon[133315]: pgmap v67: 129 pgs: 129 active+clean; 181 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.4 MiB/s wr, 505 op/s 2023-09-13T04:42:36.732 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-13T04:42:36.733 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-13T04:42:36.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:36 smithi002 ceph-mon[97651]: pgmap v67: 129 pgs: 129 active+clean; 181 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.4 MiB/s wr, 505 op/s 2023-09-13T04:42:36.936 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:42:36 smithi156 systemd[1]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.c.service: Failed with result 'exit-code'. 2023-09-13T04:42:37.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.033+0000 7f56c89ff700 1 -- 172.21.15.2:0/3282799782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56c4100e90 msgr2=0x7f56c4101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:37.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.033+0000 7f56c89ff700 1 --2- 172.21.15.2:0/3282799782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56c4100e90 0x7f56c4101270 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009990 tx=0x7f56bc009ca0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.033+0000 7f56c89ff700 1 -- 172.21.15.2:0/3282799782 shutdown_connections 2023-09-13T04:42:37.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.033+0000 7f56c89ff700 1 --2- 172.21.15.2:0/3282799782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56c4105fb0 0x7f56c410db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.033+0000 7f56c89ff700 1 --2- 172.21.15.2:0/3282799782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56c4100e90 0x7f56c4101270 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.033+0000 7f56c89ff700 1 --2- 172.21.15.2:0/3282799782 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56c4101840 0x7f56c4105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.033+0000 7f56c89ff700 1 -- 172.21.15.2:0/3282799782 >> 172.21.15.2:0/3282799782 conn(0x7f56c40fc730 msgr2=0x7f56c40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:37.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.033+0000 7f56c89ff700 1 -- 172.21.15.2:0/3282799782 shutdown_connections 2023-09-13T04:42:37.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.033+0000 7f56c89ff700 1 -- 172.21.15.2:0/3282799782 wait complete. 2023-09-13T04:42:37.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.034+0000 7f56c89ff700 1 Processor -- start 2023-09-13T04:42:37.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.035+0000 7f56c89ff700 1 -- start start 2023-09-13T04:42:37.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.035+0000 7f56c89ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56c4100e90 0x7f56c419d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.035+0000 7f56c89ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56c4105fb0 0x7f56c419d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.035+0000 7f56c89ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56c41a1d50 0x7f56c41a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.035+0000 7f56c89ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c41a27d0 con 0x7f56c4100e90 2023-09-13T04:42:37.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.035+0000 7f56c89ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c41a2940 con 0x7f56c4105fb0 2023-09-13T04:42:37.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.035+0000 7f56c89ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c41a2b00 con 0x7f56c41a1d50 2023-09-13T04:42:37.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.036+0000 7f56c1d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56c4105fb0 0x7f56c419d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.036+0000 7f56c259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56c4100e90 0x7f56c419d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.036+0000 7f56c1d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56c4105fb0 0x7f56c419d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:46492/0 (socket says 172.21.15.2:46492) 2023-09-13T04:42:37.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.036+0000 7f56c1d9b700 1 -- 172.21.15.2:0/4203642452 learned_addr learned my addr 172.21.15.2:0/4203642452 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:37.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.036+0000 7f56c1d9b700 1 -- 172.21.15.2:0/4203642452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56c41a1d50 msgr2=0x7f56c41a21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:37.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.036+0000 7f56c1d9b700 1 --2- 172.21.15.2:0/4203642452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56c41a1d50 0x7f56c41a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.036+0000 7f56c1d9b700 1 -- 172.21.15.2:0/4203642452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56c4100e90 msgr2=0x7f56c419d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:37.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.036+0000 7f56c1d9b700 1 --2- 172.21.15.2:0/4203642452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56c4100e90 0x7f56c419d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.036+0000 7f56c1d9b700 1 -- 172.21.15.2:0/4203642452 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc0095f0 con 0x7f56c4105fb0 2023-09-13T04:42:37.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.036+0000 7f56c1d9b700 1 --2- 172.21.15.2:0/4203642452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56c4105fb0 0x7f56c419d8f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00bea0 tx=0x7f56ac00bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:37.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.036+0000 7f56bb7fe700 1 -- 172.21.15.2:0/4203642452 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac00dba0 con 0x7f56c4105fb0 2023-09-13T04:42:37.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.037+0000 7f56bb7fe700 1 -- 172.21.15.2:0/4203642452 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56ac00d460 con 0x7f56c4105fb0 2023-09-13T04:42:37.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.037+0000 7f56bb7fe700 1 -- 172.21.15.2:0/4203642452 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac009940 con 0x7f56c4105fb0 2023-09-13T04:42:37.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.037+0000 7f56c89ff700 1 -- 172.21.15.2:0/4203642452 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c41aa060 con 0x7f56c4105fb0 2023-09-13T04:42:37.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.037+0000 7f56c89ff700 1 -- 172.21.15.2:0/4203642452 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c41aa4f0 con 0x7f56c4105fb0 2023-09-13T04:42:37.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.041+0000 7f56bb7fe700 1 -- 172.21.15.2:0/4203642452 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f56ac014070 con 0x7f56c4105fb0 2023-09-13T04:42:37.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.042+0000 7f56dc313b80 1 Processor -- start 2023-09-13T04:42:37.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.042+0000 7f56dc313b80 1 -- start start 2023-09-13T04:42:37.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.043+0000 7f56dc313b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634fe6eb060 0x5634fe6eb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.043+0000 7f56dc313b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634fe6eb980 0x5634fe6e5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.043+0000 7f56dc313b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634fe6e5a80 0x5634fe6e5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.044+0000 7f56dc313b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634fe63e800 con 0x5634fe6eb060 2023-09-13T04:42:37.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.044+0000 7f56dc313b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634fe63c120 con 0x5634fe6eb980 2023-09-13T04:42:37.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.044+0000 7f56dc313b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634fe637a10 con 0x5634fe6e5a80 2023-09-13T04:42:37.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.044+0000 7f56c1d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634fe6e5a80 0x5634fe6e5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.044+0000 7f56c1d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634fe6e5a80 0x5634fe6e5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:51370/0 (socket says 172.21.15.2:51370) 2023-09-13T04:42:37.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.044+0000 7f56c1d9b700 1 -- 172.21.15.2:0/2443156884 learned_addr learned my addr 172.21.15.2:0/2443156884 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:37.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.044+0000 7f56c2d9d700 1 --2- 172.21.15.2:0/2443156884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634fe6eb060 0x5634fe6eb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.044+0000 7f56c259c700 1 --2- 172.21.15.2:0/2443156884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634fe6eb980 0x5634fe6e5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.045+0000 7f56c2d9d700 1 -- 172.21.15.2:0/2443156884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634fe6e5a80 msgr2=0x5634fe6e5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:37.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.045+0000 7f56c2d9d700 1 --2- 172.21.15.2:0/2443156884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634fe6e5a80 0x5634fe6e5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.045+0000 7f56c2d9d700 1 -- 172.21.15.2:0/2443156884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634fe6eb980 msgr2=0x5634fe6e5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:37.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.045+0000 7f56c2d9d700 1 --2- 172.21.15.2:0/2443156884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634fe6eb980 0x5634fe6e5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.045+0000 7f56c2d9d700 1 -- 172.21.15.2:0/2443156884 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634fe4b76b0 con 0x5634fe6eb060 2023-09-13T04:42:37.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.045+0000 7f56bb7fe700 1 --2- 172.21.15.2:0/4203642452 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f56b00a0830 0x7f56b00a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.045+0000 7f56c2d9d700 1 --2- 172.21.15.2:0/2443156884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634fe6eb060 0x5634fe6eb440 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f56b400e410 tx=0x7f56b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e1f4bed38f474 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:37.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.046+0000 7f56bb7fe700 1 -- 172.21.15.2:0/4203642452 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f56ac0cb5b0 con 0x7f56c4105fb0 2023-09-13T04:42:37.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.046+0000 7f56b8ff9700 1 -- 172.21.15.2:0/2443156884 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b4010040 con 0x5634fe6eb060 2023-09-13T04:42:37.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.046+0000 7f56b8ff9700 1 -- 172.21.15.2:0/2443156884 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56b400ee60 con 0x5634fe6eb060 2023-09-13T04:42:37.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.046+0000 7f56b8ff9700 1 -- 172.21.15.2:0/2443156884 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b40043a0 con 0x5634fe6eb060 2023-09-13T04:42:37.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.046+0000 7f56c259c700 1 --2- 172.21.15.2:0/4203642452 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f56b00a0830 0x7f56b00a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.047+0000 7f56dc313b80 1 -- 172.21.15.2:0/2443156884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634fe6eb060 msgr2=0x5634fe6eb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:37.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.047+0000 7f56dc313b80 1 --2- 172.21.15.2:0/2443156884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634fe6eb060 0x5634fe6eb440 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f56b400e410 tx=0x7f56b400b040 comp rx=0 tx=0).stop 2023-09-13T04:42:37.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.047+0000 7f56c259c700 1 --2- 172.21.15.2:0/4203642452 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f56b00a0830 0x7f56b00a2cf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00b690 tx=0x7f56bc005fd0 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:37.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.048+0000 7f56dc313b80 1 -- 172.21.15.2:0/2443156884 shutdown_connections 2023-09-13T04:42:37.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.048+0000 7f56dc313b80 1 --2- 172.21.15.2:0/2443156884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634fe6e5a80 0x5634fe6e5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.048+0000 7f56dc313b80 1 --2- 172.21.15.2:0/2443156884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634fe6eb060 0x5634fe6eb440 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.048+0000 7f56dc313b80 1 --2- 172.21.15.2:0/2443156884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634fe6eb980 0x5634fe6e5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.048+0000 7f56dc313b80 1 -- 172.21.15.2:0/2443156884 >> 172.21.15.2:0/2443156884 conn(0x5634fe643d00 msgr2=0x5634fe6dffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:37.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.048+0000 7f56dc313b80 1 -- 172.21.15.2:0/2443156884 shutdown_connections 2023-09-13T04:42:37.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.048+0000 7f56dc313b80 1 -- 172.21.15.2:0/2443156884 wait complete. 2023-09-13T04:42:37.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.048+0000 7f56dc313b80 1 -- start start 2023-09-13T04:42:37.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.049+0000 7f56dc313b80 1 Processor -- start 2023-09-13T04:42:37.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.049+0000 7f56dc313b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634fe6e5a80 0x5634fe6eec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.049+0000 7f56dc313b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634fe6eb060 0x5634fe6ef1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.049+0000 7f56dc313b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634fe6eb980 0x5634fe6ef700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.049+0000 7f56dc313b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634fe5d5c30 con 0x5634fe6eb060 2023-09-13T04:42:37.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.049+0000 7f56dc313b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634fe5d5930 con 0x5634fe6eb980 2023-09-13T04:42:37.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.049+0000 7f56dc313b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634fe4b99a0 con 0x5634fe6e5a80 2023-09-13T04:42:37.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.049+0000 7f56dc313b80 10 client.0 _ll_register_callbacks cb 0x5634fe5d3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:42:37.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.050+0000 7f56c259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634fe6e5a80 0x5634fe6eec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.050+0000 7f56c259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634fe6e5a80 0x5634fe6eec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:51378/0 (socket says 172.21.15.2:51378) 2023-09-13T04:42:37.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.050+0000 7f56c259c700 1 -- 172.21.15.2:0/2092081767 learned_addr learned my addr 172.21.15.2:0/2092081767 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:37.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.050+0000 7f56c1d9b700 1 --2- 172.21.15.2:0/2092081767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634fe6eb060 0x5634fe6ef1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.050+0000 7f56c2d9d700 1 --2- 172.21.15.2:0/2092081767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634fe6eb980 0x5634fe6ef700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.050+0000 7f56c259c700 1 -- 172.21.15.2:0/2092081767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634fe6eb980 msgr2=0x5634fe6ef700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:37.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.050+0000 7f56c259c700 1 --2- 172.21.15.2:0/2092081767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634fe6eb980 0x5634fe6ef700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.050+0000 7f56c259c700 1 -- 172.21.15.2:0/2092081767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634fe6eb060 msgr2=0x5634fe6ef1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:37.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.050+0000 7f56c259c700 1 --2- 172.21.15.2:0/2092081767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634fe6eb060 0x5634fe6ef1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.050+0000 7f56c259c700 1 -- 172.21.15.2:0/2092081767 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b400e0c0 con 0x5634fe6e5a80 2023-09-13T04:42:37.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.050+0000 7f56c1d9b700 1 --2- 172.21.15.2:0/2092081767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634fe6eb060 0x5634fe6ef1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:42:37.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.051+0000 7f56c259c700 1 --2- 172.21.15.2:0/2092081767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634fe6e5a80 0x5634fe6eec80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f56bc015000 tx=0x7f56bc009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce5444bf438d0057 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:37.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.051+0000 7f56a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:42:37.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.051+0000 7f56a9ffb700 1 -- 172.21.15.2:0/2092081767 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc004dd0 con 0x5634fe6e5a80 2023-09-13T04:42:37.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.051+0000 7f56a9ffb700 1 -- 172.21.15.2:0/2092081767 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f56bc026460 con 0x5634fe6e5a80 2023-09-13T04:42:37.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.051+0000 7f56dc313b80 1 -- 172.21.15.2:0/2092081767 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634fe6d86f0 con 0x5634fe6e5a80 2023-09-13T04:42:37.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.052+0000 7f56a9ffb700 1 -- 172.21.15.2:0/2092081767 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56bc026620 con 0x5634fe6e5a80 2023-09-13T04:42:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.052+0000 7f56dc313b80 10 client.26131 fetch_fsmap learned FSMap version 27 2023-09-13T04:42:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.052+0000 7f56dc313b80 1 -- 172.21.15.2:0/2092081767 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5634fe64f820 con 0x5634fe6e5a80 2023-09-13T04:42:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.052+0000 7f56a9ffb700 1 -- 172.21.15.2:0/2092081767 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5634fe6d86f0 con 0x5634fe6e5a80 2023-09-13T04:42:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.052+0000 7f56a9ffb700 1 -- 172.21.15.2:0/2092081767 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f56bc00f790 con 0x5634fe6e5a80 2023-09-13T04:42:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.052+0000 7f56dc313b80 10 client.26131 fetch_fsmap finished waiting for FSMap version 27 2023-09-13T04:42:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.052+0000 7f56dc313b80 10 client.26131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:42:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.052+0000 7f56dc313b80 20 client.26131 populate_metadata read hostname 'smithi002' 2023-09-13T04:42:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.052+0000 7f56dc313b80 1 --2- 172.21.15.2:0/2092081767 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5634fe710230 0x5634fe712650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.053+0000 7f56dc313b80 4 client.26131 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:42:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.053+0000 7f56dc313b80 1 -- 172.21.15.2:0/2092081767 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634fe6309f0 con 0x5634fe710230 2023-09-13T04:42:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.053+0000 7f56c1d9b700 1 --2- 172.21.15.2:0/2092081767 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5634fe710230 0x5634fe712650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.053+0000 7f56c1d9b700 1 --2- 172.21.15.2:0/2092081767 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5634fe710230 0x5634fe712650 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24d16529cb4c238f server_cookie=64913c7963ef1560 in_seq=0 out_seq=0 2023-09-13T04:42:37.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.053+0000 7f56a9ffb700 10 client.26131 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:37.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.054+0000 7f56a9ffb700 1 -- 172.21.15.2:0/2092081767 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5634fe6309f0 con 0x5634fe710230 2023-09-13T04:42:37.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.054+0000 7f56a9ffb700 10 client.26131 handle_command_reply: tid=0 2023-09-13T04:42:37.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.055+0000 7f56dc313b80 1 client.26131 shutdown 2023-09-13T04:42:37.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.055+0000 7f56dc313b80 1 -- 172.21.15.2:0/2092081767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634fe6e5a80 msgr2=0x5634fe6eec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.055+0000 7f56dc313b80 1 --2- 172.21.15.2:0/2092081767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634fe6e5a80 0x5634fe6eec80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f56bc015000 tx=0x7f56bc009310 comp rx=0 tx=0).stop 2023-09-13T04:42:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.055+0000 7f56dc313b80 1 -- 172.21.15.2:0/2092081767 shutdown_connections 2023-09-13T04:42:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.055+0000 7f56dc313b80 1 --2- 172.21.15.2:0/2092081767 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5634fe710230 0x5634fe712650 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.055+0000 7f56dc313b80 1 --2- 172.21.15.2:0/2092081767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634fe6eb980 0x5634fe6ef700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.055+0000 7f56dc313b80 1 --2- 172.21.15.2:0/2092081767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634fe6e5a80 0x5634fe6eec80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.055+0000 7f56dc313b80 1 --2- 172.21.15.2:0/2092081767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634fe6eb060 0x5634fe6ef1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.055+0000 7f56dc313b80 1 -- 172.21.15.2:0/2092081767 >> 172.21.15.2:0/2092081767 conn(0x5634fe643d00 msgr2=0x5634fe6da870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:37.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.055+0000 7f56a9ffb700 0 client.26131 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:37.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.055+0000 7f56dc313b80 1 -- 172.21.15.2:0/2092081767 shutdown_connections 2023-09-13T04:42:37.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.055+0000 7f56dc313b80 1 -- 172.21.15.2:0/2092081767 wait complete. 2023-09-13T04:42:37.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.056+0000 7f56dc313b80 20 client.26131 trim_cache size 0 max 16384 2023-09-13T04:42:37.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.087+0000 7f56dc313b80 1 Processor -- start 2023-09-13T04:42:37.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.093+0000 7f56dc313b80 1 -- start start 2023-09-13T04:42:37.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.093+0000 7f56dc313b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634fe6e5a80 0x5634fe70c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.093+0000 7f56dc313b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634fe6eb060 0x5634fe6ede10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.093+0000 7f56dc313b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634fe6eb980 0x5634fe6da970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.093+0000 7f56dc313b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634fe6f2080 con 0x5634fe6eb060 2023-09-13T04:42:37.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.093+0000 7f56dc313b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634fe6e0e60 con 0x5634fe6eb980 2023-09-13T04:42:37.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.093+0000 7f56dc313b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634fe6ee350 con 0x5634fe6e5a80 2023-09-13T04:42:37.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.094+0000 7f56c259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634fe6e5a80 0x5634fe70c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.094+0000 7f56c259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634fe6e5a80 0x5634fe70c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:51394/0 (socket says 172.21.15.2:51394) 2023-09-13T04:42:37.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.094+0000 7f56c1d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634fe6eb060 0x5634fe6ede10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.094+0000 7f56c259c700 1 -- 172.21.15.2:0/169130447 learned_addr learned my addr 172.21.15.2:0/169130447 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:37.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.094+0000 7f56c2d9d700 1 --2- 172.21.15.2:0/169130447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634fe6eb980 0x5634fe6da970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.094+0000 7f56c259c700 1 -- 172.21.15.2:0/169130447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634fe6eb980 msgr2=0x5634fe6da970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:37.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.094+0000 7f56c259c700 1 --2- 172.21.15.2:0/169130447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634fe6eb980 0x5634fe6da970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.094+0000 7f56c259c700 1 -- 172.21.15.2:0/169130447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634fe6eb060 msgr2=0x5634fe6ede10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:37.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.094+0000 7f56c259c700 1 --2- 172.21.15.2:0/169130447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634fe6eb060 0x5634fe6ede10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.094+0000 7f56c259c700 1 -- 172.21.15.2:0/169130447 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634fe717240 con 0x5634fe6e5a80 2023-09-13T04:42:37.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.096+0000 7f56c2d9d700 1 --2- 172.21.15.2:0/169130447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634fe6eb980 0x5634fe6da970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:42:37.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.096+0000 7f56c259c700 1 --2- 172.21.15.2:0/169130447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634fe6e5a80 0x5634fe70c830 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f56bc005d70 tx=0x7f56bc0059d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee2a1cc0e4e6d939 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:37.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.097+0000 7f56b97fa700 1 -- 172.21.15.2:0/169130447 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc004dd0 con 0x5634fe6e5a80 2023-09-13T04:42:37.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.097+0000 7f56b97fa700 1 -- 172.21.15.2:0/169130447 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56bc01e040 con 0x5634fe6e5a80 2023-09-13T04:42:37.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.097+0000 7f56b97fa700 1 -- 172.21.15.2:0/169130447 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc024bc0 con 0x5634fe6e5a80 2023-09-13T04:42:37.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.098+0000 7f56dc313b80 1 -- 172.21.15.2:0/169130447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634fe6e5a80 msgr2=0x5634fe70c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:37.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.098+0000 7f56dc313b80 1 --2- 172.21.15.2:0/169130447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634fe6e5a80 0x5634fe70c830 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f56bc005d70 tx=0x7f56bc0059d0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.099+0000 7f56dc313b80 1 -- 172.21.15.2:0/169130447 shutdown_connections 2023-09-13T04:42:37.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.099+0000 7f56dc313b80 1 --2- 172.21.15.2:0/169130447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634fe6eb980 0x5634fe6da970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.099+0000 7f56dc313b80 1 --2- 172.21.15.2:0/169130447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634fe6e5a80 0x5634fe70c830 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.099+0000 7f56dc313b80 1 --2- 172.21.15.2:0/169130447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634fe6eb060 0x5634fe6ede10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.099+0000 7f56dc313b80 1 -- 172.21.15.2:0/169130447 >> 172.21.15.2:0/169130447 conn(0x5634fe643d00 msgr2=0x5634fe6e9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:37.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.102+0000 7f56dc313b80 1 -- 172.21.15.2:0/169130447 shutdown_connections 2023-09-13T04:42:37.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.108+0000 7f56dc313b80 1 -- 172.21.15.2:0/169130447 wait complete. 2023-09-13T04:42:37.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.109+0000 7f56dc313b80 1 -- start start 2023-09-13T04:42:37.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.109+0000 7f56dc313b80 1 Processor -- start 2023-09-13T04:42:37.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.113+0000 7f56dc313b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634fe6e5a80 0x5634fe6f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.114+0000 7f56dc313b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634fe6eb060 0x5634fe6f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.114+0000 7f56dc313b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634fe6eb980 0x5634fe6f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.114+0000 7f56dc313b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634fe6f4d30 con 0x5634fe6e5a80 2023-09-13T04:42:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.114+0000 7f56dc313b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634fe6f4ea0 con 0x5634fe6eb060 2023-09-13T04:42:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.114+0000 7f56dc313b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634fe6f5060 con 0x5634fe6eb980 2023-09-13T04:42:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.114+0000 7f56dc313b80 10 client.0 _ll_register_callbacks cb 0x5634fe5d3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:42:37.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.114+0000 7f56c259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634fe6e5a80 0x5634fe6f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.114+0000 7f56c259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634fe6e5a80 0x5634fe6f5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37314/0 (socket says 172.21.15.2:37314) 2023-09-13T04:42:37.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.114+0000 7f56c259c700 1 -- 172.21.15.2:0/2406037201 learned_addr learned my addr 172.21.15.2:0/2406037201 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:37.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.115+0000 7f56c1d9b700 1 --2- 172.21.15.2:0/2406037201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634fe6eb060 0x5634fe6f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.115+0000 7f56c259c700 1 -- 172.21.15.2:0/2406037201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634fe6eb980 msgr2=0x5634fe6f47f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:42:37.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.115+0000 7f56c259c700 1 --2- 172.21.15.2:0/2406037201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634fe6eb980 0x5634fe6f47f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.115+0000 7f56c259c700 1 -- 172.21.15.2:0/2406037201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634fe6eb060 msgr2=0x5634fe6f42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:37.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.115+0000 7f56c259c700 1 --2- 172.21.15.2:0/2406037201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634fe6eb060 0x5634fe6f42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.115+0000 7f56c259c700 1 -- 172.21.15.2:0/2406037201 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634fe64f820 con 0x5634fe6e5a80 2023-09-13T04:42:37.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.116+0000 7f56c259c700 1 --2- 172.21.15.2:0/2406037201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634fe6e5a80 0x5634fe6f5ff0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f56bc011040 tx=0x7f56bc005600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4347b6c925478d4 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:37.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.116+0000 7f56a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:42:37.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.117+0000 7f56a9ffb700 1 -- 172.21.15.2:0/2406037201 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc00f930 con 0x5634fe6e5a80 2023-09-13T04:42:37.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.117+0000 7f56a9ffb700 1 -- 172.21.15.2:0/2406037201 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f56bc024500 con 0x5634fe6e5a80 2023-09-13T04:42:37.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.117+0000 7f56dc313b80 1 -- 172.21.15.2:0/2406037201 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56b400ee60 con 0x5634fe6e5a80 2023-09-13T04:42:37.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.118+0000 7f56a9ffb700 1 -- 172.21.15.2:0/2406037201 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56bc0246c0 con 0x5634fe6e5a80 2023-09-13T04:42:37.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.118+0000 7f56dc313b80 10 client.16089 fetch_fsmap learned FSMap version 27 2023-09-13T04:42:37.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.118+0000 7f56dc313b80 1 -- 172.21.15.2:0/2406037201 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5634fe463e60 con 0x5634fe6e5a80 2023-09-13T04:42:37.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.118+0000 7f56a9ffb700 1 -- 172.21.15.2:0/2406037201 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56b400ee60 con 0x5634fe6e5a80 2023-09-13T04:42:37.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.119+0000 7f56a9ffb700 1 -- 172.21.15.2:0/2406037201 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f56bc053070 con 0x5634fe6e5a80 2023-09-13T04:42:37.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.119+0000 7f56dc313b80 10 client.16089 fetch_fsmap finished waiting for FSMap version 27 2023-09-13T04:42:37.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.119+0000 7f56dc313b80 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:42:37.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.119+0000 7f56dc313b80 20 client.16089 populate_metadata read hostname 'smithi002' 2023-09-13T04:42:37.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.119+0000 7f56dc313b80 1 --2- 172.21.15.2:0/2406037201 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5634fe75ef00 0x5634fe75f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.119+0000 7f56dc313b80 4 client.16089 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-13T04:42:37.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.119+0000 7f56dc313b80 1 -- 172.21.15.2:0/2406037201 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5634fe5f10d0 con 0x5634fe75ef00 2023-09-13T04:42:37.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.121+0000 7f56c1d9b700 1 --2- 172.21.15.2:0/2406037201 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5634fe75ef00 0x5634fe75f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.122+0000 7f56c1d9b700 1 --2- 172.21.15.2:0/2406037201 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5634fe75ef00 0x5634fe75f310 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42aaaffa9d6558e4 server_cookie=ec04fb77c6229b46 in_seq=0 out_seq=0 2023-09-13T04:42:37.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.122+0000 7f56a9ffb700 10 client.16089 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:37.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.128+0000 7f56a9ffb700 1 -- 172.21.15.2:0/2406037201 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5634fe5f10d0 con 0x5634fe75ef00 2023-09-13T04:42:37.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.128+0000 7f56a9ffb700 10 client.16089 handle_command_reply: tid=0 2023-09-13T04:42:37.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.128+0000 7f56dc313b80 1 client.16089 shutdown 2023-09-13T04:42:37.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.133+0000 7f56dc313b80 1 -- 172.21.15.2:0/2406037201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634fe6e5a80 msgr2=0x5634fe6f5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:37.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.133+0000 7f56dc313b80 1 --2- 172.21.15.2:0/2406037201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634fe6e5a80 0x5634fe6f5ff0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f56bc011040 tx=0x7f56bc005600 comp rx=0 tx=0).stop 2023-09-13T04:42:37.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.135+0000 7f56dc313b80 1 -- 172.21.15.2:0/2406037201 shutdown_connections 2023-09-13T04:42:37.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.135+0000 7f56dc313b80 1 --2- 172.21.15.2:0/2406037201 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5634fe75ef00 0x5634fe75f310 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.135+0000 7f56dc313b80 1 --2- 172.21.15.2:0/2406037201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634fe6eb980 0x5634fe6f47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.135+0000 7f56dc313b80 1 --2- 172.21.15.2:0/2406037201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634fe6e5a80 0x5634fe6f5ff0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.135+0000 7f56dc313b80 1 --2- 172.21.15.2:0/2406037201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634fe6eb060 0x5634fe6f42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.135+0000 7f56a9ffb700 0 client.16089 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:37.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.135+0000 7f56dc313b80 1 -- 172.21.15.2:0/2406037201 >> 172.21.15.2:0/2406037201 conn(0x5634fe643d00 msgr2=0x5634fe6edc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:37.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.135+0000 7f56dc313b80 1 -- 172.21.15.2:0/2406037201 shutdown_connections 2023-09-13T04:42:37.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.135+0000 7f56dc313b80 1 -- 172.21.15.2:0/2406037201 wait complete. 2023-09-13T04:42:37.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.136+0000 7f56dc313b80 20 client.16089 trim_cache size 0 max 16384 2023-09-13T04:42:37.138 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:42:37.138 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2023-09-13T04:42:37.138 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "740235c0-6e1b-4729-b1fe-33345ae948c1", 2023-09-13T04:42:37.139 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2023-09-13T04:42:37.139 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:42:37.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.140+0000 7f56c89ff700 1 -- 172.21.15.2:0/4203642452 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f56b00a0830 msgr2=0x7f56b00a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:37.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.140+0000 7f56c89ff700 1 --2- 172.21.15.2:0/4203642452 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f56b00a0830 0x7f56b00a2cf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00b690 tx=0x7f56bc005fd0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.140+0000 7f56c89ff700 1 -- 172.21.15.2:0/4203642452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56c4105fb0 msgr2=0x7f56c419d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:37.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.140+0000 7f56c89ff700 1 --2- 172.21.15.2:0/4203642452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56c4105fb0 0x7f56c419d8f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00bea0 tx=0x7f56ac00bed0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.142+0000 7f56c89ff700 1 -- 172.21.15.2:0/4203642452 shutdown_connections 2023-09-13T04:42:37.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.142+0000 7f56c89ff700 1 --2- 172.21.15.2:0/4203642452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56c41a1d50 0x7f56c41a21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.142+0000 7f56c89ff700 1 --2- 172.21.15.2:0/4203642452 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f56b00a0830 0x7f56b00a2cf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.142+0000 7f56c89ff700 1 --2- 172.21.15.2:0/4203642452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56c4100e90 0x7f56c419d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.142+0000 7f56c89ff700 1 --2- 172.21.15.2:0/4203642452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56c4105fb0 0x7f56c419d8f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.142+0000 7f56c89ff700 1 -- 172.21.15.2:0/4203642452 >> 172.21.15.2:0/4203642452 conn(0x7f56c40fc730 msgr2=0x7f56c41050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:37.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.148+0000 7f56c89ff700 1 -- 172.21.15.2:0/4203642452 shutdown_connections 2023-09-13T04:42:37.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.148+0000 7f56c89ff700 1 -- 172.21.15.2:0/4203642452 wait complete. 2023-09-13T04:42:37.181 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T04:42:37.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.593+0000 7fc75a5ff700 1 -- 172.21.15.2:0/990939164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc754101860 msgr2=0x7fc7541058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:37.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.593+0000 7fc75a5ff700 1 --2- 172.21.15.2:0/990939164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc754101860 0x7fc7541058a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc748009990 tx=0x7fc748009ca0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.593+0000 7fc75a5ff700 1 -- 172.21.15.2:0/990939164 shutdown_connections 2023-09-13T04:42:37.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.593+0000 7fc75a5ff700 1 --2- 172.21.15.2:0/990939164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc754105fd0 0x7fc75410db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.593+0000 7fc75a5ff700 1 --2- 172.21.15.2:0/990939164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc754100eb0 0x7fc754101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.593+0000 7fc75a5ff700 1 --2- 172.21.15.2:0/990939164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc754101860 0x7fc7541058a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.593+0000 7fc75a5ff700 1 -- 172.21.15.2:0/990939164 >> 172.21.15.2:0/990939164 conn(0x7fc7540fc770 msgr2=0x7fc7540feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:37.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.593+0000 7fc75a5ff700 1 -- 172.21.15.2:0/990939164 shutdown_connections 2023-09-13T04:42:37.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.593+0000 7fc75a5ff700 1 -- 172.21.15.2:0/990939164 wait complete. 2023-09-13T04:42:37.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.594+0000 7fc75a5ff700 1 Processor -- start 2023-09-13T04:42:37.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.594+0000 7fc75a5ff700 1 -- start start 2023-09-13T04:42:37.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.595+0000 7fc75a5ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc754100eb0 0x7fc754196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.595+0000 7fc75a5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc754105fd0 0x7fc7541972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.595+0000 7fc75a5ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc75419b700 0x7fc75419bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.595+0000 7fc75a5ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75419c180 con 0x7fc754100eb0 2023-09-13T04:42:37.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.595+0000 7fc75a5ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75419c2f0 con 0x7fc75419b700 2023-09-13T04:42:37.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.595+0000 7fc75a5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75419c4b0 con 0x7fc754105fd0 2023-09-13T04:42:37.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.595+0000 7fc7537fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc754105fd0 0x7fc7541972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.595+0000 7fc753fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc754100eb0 0x7fc754196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.595+0000 7fc7537fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc754105fd0 0x7fc7541972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:51430/0 (socket says 172.21.15.2:51430) 2023-09-13T04:42:37.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.595+0000 7fc7537fe700 1 -- 172.21.15.2:0/1099233019 learned_addr learned my addr 172.21.15.2:0/1099233019 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:37.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.595+0000 7fc758b9c700 1 --2- 172.21.15.2:0/1099233019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc75419b700 0x7fc75419bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.595+0000 7fc7537fe700 1 -- 172.21.15.2:0/1099233019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc75419b700 msgr2=0x7fc75419bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:37.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.595+0000 7fc7537fe700 1 --2- 172.21.15.2:0/1099233019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc75419b700 0x7fc75419bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.595+0000 7fc7537fe700 1 -- 172.21.15.2:0/1099233019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc754100eb0 msgr2=0x7fc754196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:37.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.595+0000 7fc7537fe700 1 --2- 172.21.15.2:0/1099233019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc754100eb0 0x7fc754196d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.595+0000 7fc7537fe700 1 -- 172.21.15.2:0/1099233019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7480095f0 con 0x7fc754105fd0 2023-09-13T04:42:37.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.596+0000 7fc7537fe700 1 --2- 172.21.15.2:0/1099233019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc754105fd0 0x7fc7541972a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc7480059d0 tx=0x7fc74800b680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:37.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.596+0000 7fc7517fa700 1 -- 172.21.15.2:0/1099233019 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74800fac0 con 0x7fc754105fd0 2023-09-13T04:42:37.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.596+0000 7fc7517fa700 1 -- 172.21.15.2:0/1099233019 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc748023070 con 0x7fc754105fd0 2023-09-13T04:42:37.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.596+0000 7fc75a5ff700 1 -- 172.21.15.2:0/1099233019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc754072ee0 con 0x7fc754105fd0 2023-09-13T04:42:37.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.596+0000 7fc7517fa700 1 -- 172.21.15.2:0/1099233019 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7480173f0 con 0x7fc754105fd0 2023-09-13T04:42:37.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.597+0000 7fc75a5ff700 1 -- 172.21.15.2:0/1099233019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc754073440 con 0x7fc754105fd0 2023-09-13T04:42:37.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.602+0000 7fc7517fa700 1 -- 172.21.15.2:0/1099233019 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7fc748025030 con 0x7fc754105fd0 2023-09-13T04:42:37.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.603+0000 7fc76deefb80 1 Processor -- start 2023-09-13T04:42:37.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.603+0000 7fc76deefb80 1 -- start start 2023-09-13T04:42:37.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.604+0000 7fc76deefb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649300187f0 0x564930018bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.604+0000 7fc76deefb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564930019110 0x564930012cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.604+0000 7fc76deefb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564930013210 0x564930013670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.604+0000 7fc76deefb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56492ff63780 con 0x564930019110 2023-09-13T04:42:37.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.604+0000 7fc76deefb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56492ff610a0 con 0x5649300187f0 2023-09-13T04:42:37.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.604+0000 7fc76deefb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56492ff5c990 con 0x564930013210 2023-09-13T04:42:37.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.604+0000 7fc758b9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649300187f0 0x564930018bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.604+0000 7fc758b9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649300187f0 0x564930018bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:46548/0 (socket says 172.21.15.2:46548) 2023-09-13T04:42:37.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.604+0000 7fc758b9c700 1 -- 172.21.15.2:0/2547131360 learned_addr learned my addr 172.21.15.2:0/2547131360 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:37.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.605+0000 7fc753fff700 1 --2- 172.21.15.2:0/2547131360 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564930019110 0x564930012cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.605+0000 7fc758b9c700 1 -- 172.21.15.2:0/2547131360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564930013210 msgr2=0x564930013670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:42:37.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.605+0000 7fc758b9c700 1 --2- 172.21.15.2:0/2547131360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564930013210 0x564930013670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.605+0000 7fc758b9c700 1 -- 172.21.15.2:0/2547131360 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564930019110 msgr2=0x564930012cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:37.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.605+0000 7fc758b9c700 1 --2- 172.21.15.2:0/2547131360 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564930019110 0x564930012cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.605+0000 7fc758b9c700 1 -- 172.21.15.2:0/2547131360 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56492fddc630 con 0x5649300187f0 2023-09-13T04:42:37.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.606+0000 7fc758b9c700 1 --2- 172.21.15.2:0/2547131360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649300187f0 0x564930018bd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc74c011680 tx=0x7fc74c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=174fb00d842025f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:37.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.606+0000 7fc7517fa700 1 --2- 172.21.15.2:0/1099233019 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc73c0a0830 0x7fc73c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.606+0000 7fc73a7fc700 1 -- 172.21.15.2:0/2547131360 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c00e040 con 0x5649300187f0 2023-09-13T04:42:37.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.606+0000 7fc7517fa700 1 -- 172.21.15.2:0/1099233019 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fc7480d1660 con 0x7fc754105fd0 2023-09-13T04:42:37.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.606+0000 7fc73a7fc700 1 -- 172.21.15.2:0/2547131360 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc74c00f040 con 0x5649300187f0 2023-09-13T04:42:37.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.606+0000 7fc73a7fc700 1 -- 172.21.15.2:0/2547131360 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c0048c0 con 0x5649300187f0 2023-09-13T04:42:37.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.607+0000 7fc753fff700 1 --2- 172.21.15.2:0/1099233019 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc73c0a0830 0x7fc73c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.608+0000 7fc76deefb80 1 -- 172.21.15.2:0/2547131360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649300187f0 msgr2=0x564930018bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:37.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.608+0000 7fc753fff700 1 --2- 172.21.15.2:0/1099233019 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc73c0a0830 0x7fc73c0a2cf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x564930019570 tx=0x7fc74400b410 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:37.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.608+0000 7fc76deefb80 1 --2- 172.21.15.2:0/2547131360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649300187f0 0x564930018bd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc74c011680 tx=0x7fc74c009920 comp rx=0 tx=0).stop 2023-09-13T04:42:37.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.608+0000 7fc76deefb80 1 -- 172.21.15.2:0/2547131360 shutdown_connections 2023-09-13T04:42:37.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.608+0000 7fc76deefb80 1 --2- 172.21.15.2:0/2547131360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564930013210 0x564930013670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.608+0000 7fc76deefb80 1 --2- 172.21.15.2:0/2547131360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649300187f0 0x564930018bd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.608+0000 7fc76deefb80 1 --2- 172.21.15.2:0/2547131360 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564930019110 0x564930012cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.608+0000 7fc76deefb80 1 -- 172.21.15.2:0/2547131360 >> 172.21.15.2:0/2547131360 conn(0x56492ff68c80 msgr2=0x564930004f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:37.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.609+0000 7fc76deefb80 1 -- 172.21.15.2:0/2547131360 shutdown_connections 2023-09-13T04:42:37.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.609+0000 7fc76deefb80 1 -- 172.21.15.2:0/2547131360 wait complete. 2023-09-13T04:42:37.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.610+0000 7fc76deefb80 1 -- start start 2023-09-13T04:42:37.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.611+0000 7fc76deefb80 1 Processor -- start 2023-09-13T04:42:37.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.612+0000 7fc76deefb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564930013210 0x56493001c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.612+0000 7fc76deefb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649300187f0 0x56493001c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.612+0000 7fc76deefb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564930019110 0x56493001ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.612+0000 7fc76deefb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56492fefabb0 con 0x564930013210 2023-09-13T04:42:37.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.612+0000 7fc76deefb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56492fefa8b0 con 0x564930019110 2023-09-13T04:42:37.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.612+0000 7fc76deefb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56492fdde920 con 0x5649300187f0 2023-09-13T04:42:37.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.612+0000 7fc76deefb80 10 client.0 _ll_register_callbacks cb 0x56492fef8680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:42:37.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.613+0000 7fc758b9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564930019110 0x56493001ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.613+0000 7fc758b9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564930019110 0x56493001ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:46556/0 (socket says 172.21.15.2:46556) 2023-09-13T04:42:37.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.613+0000 7fc758b9c700 1 -- 172.21.15.2:0/933430675 learned_addr learned my addr 172.21.15.2:0/933430675 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:37.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.613+0000 7fc753fff700 1 --2- 172.21.15.2:0/933430675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564930013210 0x56493001c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.614+0000 7fc758b9c700 1 -- 172.21.15.2:0/933430675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649300187f0 msgr2=0x56493001c970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:42:37.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.614+0000 7fc758b9c700 1 --2- 172.21.15.2:0/933430675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649300187f0 0x56493001c970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.614+0000 7fc758b9c700 1 -- 172.21.15.2:0/933430675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564930013210 msgr2=0x56493001c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:37.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.614+0000 7fc758b9c700 1 --2- 172.21.15.2:0/933430675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564930013210 0x56493001c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.614+0000 7fc758b9c700 1 -- 172.21.15.2:0/933430675 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc74c011330 con 0x564930019110 2023-09-13T04:42:37.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.615+0000 7fc758b9c700 1 --2- 172.21.15.2:0/933430675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564930019110 0x56493001ceb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc74c000f80 tx=0x7fc74c009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f9c2505ca158a48 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:37.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.615+0000 7fc7337fe700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:42:37.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.615+0000 7fc7337fe700 1 -- 172.21.15.2:0/933430675 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c00e070 con 0x564930019110 2023-09-13T04:42:37.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.615+0000 7fc76deefb80 1 -- 172.21.15.2:0/933430675 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56492fffd670 con 0x564930019110 2023-09-13T04:42:37.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.616+0000 7fc7337fe700 1 -- 172.21.15.2:0/933430675 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fc74c003c10 con 0x564930019110 2023-09-13T04:42:37.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.616+0000 7fc7337fe700 1 -- 172.21.15.2:0/933430675 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc74c003dd0 con 0x564930019110 2023-09-13T04:42:37.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.616+0000 7fc76deefb80 10 client.26144 fetch_fsmap learned FSMap version 27 2023-09-13T04:42:37.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.616+0000 7fc76deefb80 1 -- 172.21.15.2:0/933430675 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56492ff747a0 con 0x564930019110 2023-09-13T04:42:37.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.616+0000 7fc7337fe700 1 -- 172.21.15.2:0/933430675 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56492fffd670 con 0x564930019110 2023-09-13T04:42:37.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.617+0000 7fc7337fe700 1 -- 172.21.15.2:0/933430675 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fc74c027460 con 0x564930019110 2023-09-13T04:42:37.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.617+0000 7fc76deefb80 10 client.26144 fetch_fsmap finished waiting for FSMap version 27 2023-09-13T04:42:37.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.617+0000 7fc76deefb80 10 client.26144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:42:37.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.617+0000 7fc76deefb80 20 client.26144 populate_metadata read hostname 'smithi002' 2023-09-13T04:42:37.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.617+0000 7fc76deefb80 1 --2- 172.21.15.2:0/933430675 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56493003d9f0 0x56493003fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.617+0000 7fc76deefb80 4 client.26144 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:42:37.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.617+0000 7fc76deefb80 1 -- 172.21.15.2:0/933430675 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56492ff55970 con 0x56493003d9f0 2023-09-13T04:42:37.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.617+0000 7fc753fff700 1 --2- 172.21.15.2:0/933430675 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56493003d9f0 0x56493003fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.618+0000 7fc753fff700 1 --2- 172.21.15.2:0/933430675 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56493003d9f0 0x56493003fe10 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1009c35d21cb61fc server_cookie=3ef9625ad16bc333 in_seq=0 out_seq=0 2023-09-13T04:42:37.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.618+0000 7fc7337fe700 10 client.26144 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:37.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.619+0000 7fc7337fe700 1 -- 172.21.15.2:0/933430675 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56492ff55970 con 0x56493003d9f0 2023-09-13T04:42:37.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.619+0000 7fc7337fe700 10 client.26144 handle_command_reply: tid=0 2023-09-13T04:42:37.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.620+0000 7fc76deefb80 1 client.26144 shutdown 2023-09-13T04:42:37.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.620+0000 7fc76deefb80 1 -- 172.21.15.2:0/933430675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564930019110 msgr2=0x56493001ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:37.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.620+0000 7fc76deefb80 1 --2- 172.21.15.2:0/933430675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564930019110 0x56493001ceb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc74c000f80 tx=0x7fc74c009510 comp rx=0 tx=0).stop 2023-09-13T04:42:37.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.620+0000 7fc76deefb80 1 -- 172.21.15.2:0/933430675 shutdown_connections 2023-09-13T04:42:37.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.620+0000 7fc76deefb80 1 --2- 172.21.15.2:0/933430675 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56493003d9f0 0x56493003fe10 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.620+0000 7fc76deefb80 1 --2- 172.21.15.2:0/933430675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564930019110 0x56493001ceb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.620+0000 7fc76deefb80 1 --2- 172.21.15.2:0/933430675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564930013210 0x56493001c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.620+0000 7fc76deefb80 1 --2- 172.21.15.2:0/933430675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649300187f0 0x56493001c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.620+0000 7fc76deefb80 1 -- 172.21.15.2:0/933430675 >> 172.21.15.2:0/933430675 conn(0x56492ff68c80 msgr2=0x56492ffff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:37.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.621+0000 7fc7337fe700 0 client.26144 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:37.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.621+0000 7fc76deefb80 1 -- 172.21.15.2:0/933430675 shutdown_connections 2023-09-13T04:42:37.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.621+0000 7fc76deefb80 1 -- 172.21.15.2:0/933430675 wait complete. 2023-09-13T04:42:37.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.621+0000 7fc76deefb80 20 client.26144 trim_cache size 0 max 16384 2023-09-13T04:42:37.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.650+0000 7fc76deefb80 1 Processor -- start 2023-09-13T04:42:37.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.650+0000 7fc76deefb80 1 -- start start 2023-09-13T04:42:37.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.650+0000 7fc76deefb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564930013210 0x564930023a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.651+0000 7fc76deefb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649300187f0 0x56492ffff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.651+0000 7fc76deefb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564930019110 0x56493001ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.651+0000 7fc76deefb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56492ff55970 con 0x5649300187f0 2023-09-13T04:42:37.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.651+0000 7fc76deefb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564930013c70 con 0x564930019110 2023-09-13T04:42:37.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.651+0000 7fc76deefb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564930005de0 con 0x564930013210 2023-09-13T04:42:37.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.651+0000 7fc758b9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564930019110 0x56493001ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.651+0000 7fc758b9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564930019110 0x56493001ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:46570/0 (socket says 172.21.15.2:46570) 2023-09-13T04:42:37.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.651+0000 7fc758b9c700 1 -- 172.21.15.2:0/1328097967 learned_addr learned my addr 172.21.15.2:0/1328097967 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:37.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.652+0000 7fc758b9c700 1 -- 172.21.15.2:0/1328097967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564930013210 msgr2=0x564930023a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:42:37.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.652+0000 7fc758b9c700 1 --2- 172.21.15.2:0/1328097967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564930013210 0x564930023a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.652+0000 7fc758b9c700 1 -- 172.21.15.2:0/1328097967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649300187f0 msgr2=0x56492ffff8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:37.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.652+0000 7fc758b9c700 1 --2- 172.21.15.2:0/1328097967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649300187f0 0x56492ffff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.652+0000 7fc758b9c700 1 -- 172.21.15.2:0/1328097967 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564930005130 con 0x564930019110 2023-09-13T04:42:37.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.652+0000 7fc758b9c700 1 --2- 172.21.15.2:0/1328097967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564930019110 0x56493001ad10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc74c004490 tx=0x7fc74c006160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feabb71cfc5a9cbf server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:37.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.652+0000 7fc73affd700 1 -- 172.21.15.2:0/1328097967 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c00e070 con 0x564930019110 2023-09-13T04:42:37.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.652+0000 7fc73affd700 1 -- 172.21.15.2:0/1328097967 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc74c027bb0 con 0x564930019110 2023-09-13T04:42:37.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.652+0000 7fc73affd700 1 -- 172.21.15.2:0/1328097967 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c0041e0 con 0x564930019110 2023-09-13T04:42:37.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.654+0000 7fc76deefb80 1 -- 172.21.15.2:0/1328097967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564930019110 msgr2=0x56493001ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:37.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.654+0000 7fc76deefb80 1 --2- 172.21.15.2:0/1328097967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564930019110 0x56493001ad10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc74c004490 tx=0x7fc74c006160 comp rx=0 tx=0).stop 2023-09-13T04:42:37.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.654+0000 7fc76deefb80 1 -- 172.21.15.2:0/1328097967 shutdown_connections 2023-09-13T04:42:37.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.654+0000 7fc76deefb80 1 --2- 172.21.15.2:0/1328097967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564930019110 0x56493001ad10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.654+0000 7fc76deefb80 1 --2- 172.21.15.2:0/1328097967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564930013210 0x564930023a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.654+0000 7fc76deefb80 1 --2- 172.21.15.2:0/1328097967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649300187f0 0x56492ffff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.654+0000 7fc76deefb80 1 -- 172.21.15.2:0/1328097967 >> 172.21.15.2:0/1328097967 conn(0x56492ff68c80 msgr2=0x564930016590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.658+0000 7fc76deefb80 1 -- 172.21.15.2:0/1328097967 shutdown_connections 2023-09-13T04:42:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.667+0000 7fc76deefb80 1 -- 172.21.15.2:0/1328097967 wait complete. 2023-09-13T04:42:37.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.668+0000 7fc76deefb80 1 -- start start 2023-09-13T04:42:37.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.668+0000 7fc76deefb80 1 Processor -- start 2023-09-13T04:42:37.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.669+0000 7fc76deefb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649300187f0 0x5649300236e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.669+0000 7fc76deefb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564930019110 0x564930023c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.669+0000 7fc76deefb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564930021b70 0x564930021ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.669+0000 7fc76deefb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564930022730 con 0x564930019110 2023-09-13T04:42:37.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.669+0000 7fc76deefb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649300228a0 con 0x5649300187f0 2023-09-13T04:42:37.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.669+0000 7fc76deefb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649300074a0 con 0x564930021b70 2023-09-13T04:42:37.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.669+0000 7fc76deefb80 10 client.0 _ll_register_callbacks cb 0x56492fef8680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:42:37.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.670+0000 7fc758b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564930021b70 0x564930021ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.670+0000 7fc758b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564930021b70 0x564930021ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:51466/0 (socket says 172.21.15.2:51466) 2023-09-13T04:42:37.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.670+0000 7fc758b9c700 1 -- 172.21.15.2:0/3614492220 learned_addr learned my addr 172.21.15.2:0/3614492220 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:37.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.671+0000 7fc758b9c700 1 -- 172.21.15.2:0/3614492220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649300187f0 msgr2=0x5649300236e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:42:37.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.671+0000 7fc758b9c700 1 --2- 172.21.15.2:0/3614492220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649300187f0 0x5649300236e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.671+0000 7fc758b9c700 1 -- 172.21.15.2:0/3614492220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564930019110 msgr2=0x564930023c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:42:37.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.672+0000 7fc7537fe700 1 --2- 172.21.15.2:0/3614492220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564930019110 0x564930023c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.672+0000 7fc758b9c700 1 --2- 172.21.15.2:0/3614492220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564930019110 0x564930023c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.672+0000 7fc758b9c700 1 -- 172.21.15.2:0/3614492220 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56492ff747a0 con 0x564930021b70 2023-09-13T04:42:37.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.673+0000 7fc758b9c700 1 --2- 172.21.15.2:0/3614492220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564930021b70 0x564930021ff0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc74c027710 tx=0x7fc74c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78aca7573b8be9d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:37.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.684+0000 7fc7337fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:42:37.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.684+0000 7fc7337fe700 1 -- 172.21.15.2:0/3614492220 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c00e070 con 0x564930021b70 2023-09-13T04:42:37.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.684+0000 7fc7337fe700 1 -- 172.21.15.2:0/3614492220 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fc74c004e40 con 0x564930021b70 2023-09-13T04:42:37.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.685+0000 7fc76deefb80 1 -- 172.21.15.2:0/3614492220 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc74c00f040 con 0x564930021b70 2023-09-13T04:42:37.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.687+0000 7fc7337fe700 1 -- 172.21.15.2:0/3614492220 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc74c003e20 con 0x564930021b70 2023-09-13T04:42:37.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.687+0000 7fc76deefb80 10 client.26143 fetch_fsmap learned FSMap version 27 2023-09-13T04:42:37.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.687+0000 7fc76deefb80 1 -- 172.21.15.2:0/3614492220 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56493001bfa0 con 0x564930021b70 2023-09-13T04:42:37.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.693+0000 7fc7337fe700 1 -- 172.21.15.2:0/3614492220 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc74c00f040 con 0x564930021b70 2023-09-13T04:42:37.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.693+0000 7fc7337fe700 1 -- 172.21.15.2:0/3614492220 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fc74c007500 con 0x564930021b70 2023-09-13T04:42:37.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.697+0000 7fc76deefb80 10 client.26143 fetch_fsmap finished waiting for FSMap version 27 2023-09-13T04:42:37.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.697+0000 7fc76deefb80 10 client.26143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:42:37.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.697+0000 7fc76deefb80 20 client.26143 populate_metadata read hostname 'smithi002' 2023-09-13T04:42:37.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.697+0000 7fc76deefb80 1 --2- 172.21.15.2:0/3614492220 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5649300086c0 0x56493000aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:37.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.697+0000 7fc76deefb80 4 client.26143 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-09-13T04:42:37.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.697+0000 7fc76deefb80 1 -- 172.21.15.2:0/3614492220 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56492ff16050 con 0x5649300086c0 2023-09-13T04:42:37.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.699+0000 7fc753fff700 1 --2- 172.21.15.2:0/3614492220 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5649300086c0 0x56493000aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:37.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.699+0000 7fc753fff700 1 --2- 172.21.15.2:0/3614492220 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5649300086c0 0x56493000aae0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e30df678c72ca35e server_cookie=2ed397544f9aaf42 in_seq=0 out_seq=0 2023-09-13T04:42:37.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.699+0000 7fc7337fe700 10 client.26143 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:37.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.700+0000 7fc7337fe700 1 -- 172.21.15.2:0/3614492220 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56492ff16050 con 0x5649300086c0 2023-09-13T04:42:37.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.700+0000 7fc7337fe700 10 client.26143 handle_command_reply: tid=0 2023-09-13T04:42:37.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.700+0000 7fc76deefb80 1 client.26143 shutdown 2023-09-13T04:42:37.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.701+0000 7fc76deefb80 1 -- 172.21.15.2:0/3614492220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564930021b70 msgr2=0x564930021ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:37.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.701+0000 7fc76deefb80 1 --2- 172.21.15.2:0/3614492220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564930021b70 0x564930021ff0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc74c027710 tx=0x7fc74c009920 comp rx=0 tx=0).stop 2023-09-13T04:42:37.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.701+0000 7fc76deefb80 1 -- 172.21.15.2:0/3614492220 shutdown_connections 2023-09-13T04:42:37.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.701+0000 7fc76deefb80 1 --2- 172.21.15.2:0/3614492220 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5649300086c0 0x56493000aae0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.701+0000 7fc76deefb80 1 --2- 172.21.15.2:0/3614492220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564930021b70 0x564930021ff0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.701+0000 7fc76deefb80 1 --2- 172.21.15.2:0/3614492220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649300187f0 0x5649300236e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.701+0000 7fc76deefb80 1 --2- 172.21.15.2:0/3614492220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564930019110 0x564930023c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.701+0000 7fc76deefb80 1 -- 172.21.15.2:0/3614492220 >> 172.21.15.2:0/3614492220 conn(0x56492ff68c80 msgr2=0x56493001ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:37.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.701+0000 7fc7337fe700 0 client.26143 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:37.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.703+0000 7fc76deefb80 1 -- 172.21.15.2:0/3614492220 shutdown_connections 2023-09-13T04:42:37.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.703+0000 7fc76deefb80 1 -- 172.21.15.2:0/3614492220 wait complete. 2023-09-13T04:42:37.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.711+0000 7fc76deefb80 20 client.26143 trim_cache size 0 max 16384 2023-09-13T04:42:37.712 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:42:37.712 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-13T04:42:37.712 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-09-13T04:42:37.712 INFO:teuthology.orchestra.run.smithi002.stdout: "740235c0-6e1b-4729-b1fe-33345ae948c1": { 2023-09-13T04:42:37.712 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-09-13T04:42:37.713 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "740235c0-6e1b-4729-b1fe-33345ae948c1", 2023-09-13T04:42:37.713 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-09-13T04:42:37.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:42:37.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:42:37.713 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:42:37.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.723+0000 7fc75a5ff700 1 -- 172.21.15.2:0/1099233019 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc73c0a0830 msgr2=0x7fc73c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:37.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.723+0000 7fc75a5ff700 1 --2- 172.21.15.2:0/1099233019 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc73c0a0830 0x7fc73c0a2cf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x564930019570 tx=0x7fc74400b410 comp rx=0 tx=0).stop 2023-09-13T04:42:37.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.723+0000 7fc75a5ff700 1 -- 172.21.15.2:0/1099233019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc754105fd0 msgr2=0x7fc7541972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:37.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.723+0000 7fc75a5ff700 1 --2- 172.21.15.2:0/1099233019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc754105fd0 0x7fc7541972a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc7480059d0 tx=0x7fc74800b680 comp rx=0 tx=0).stop 2023-09-13T04:42:37.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.723+0000 7fc75a5ff700 1 -- 172.21.15.2:0/1099233019 shutdown_connections 2023-09-13T04:42:37.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.723+0000 7fc75a5ff700 1 --2- 172.21.15.2:0/1099233019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc75419b700 0x7fc75419bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.723+0000 7fc75a5ff700 1 --2- 172.21.15.2:0/1099233019 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc73c0a0830 0x7fc73c0a2cf0 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x564930019570 tx=0x7fc74400b410 comp rx=0 tx=0).stop 2023-09-13T04:42:37.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.723+0000 7fc75a5ff700 1 --2- 172.21.15.2:0/1099233019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc754100eb0 0x7fc754196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.723+0000 7fc75a5ff700 1 --2- 172.21.15.2:0/1099233019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc754105fd0 0x7fc7541972a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:37.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.723+0000 7fc75a5ff700 1 -- 172.21.15.2:0/1099233019 >> 172.21.15.2:0/1099233019 conn(0x7fc7540fc770 msgr2=0x7fc7540fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:37.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.729+0000 7fc75a5ff700 1 -- 172.21.15.2:0/1099233019 shutdown_connections 2023-09-13T04:42:37.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:42:37.736+0000 7fc75a5ff700 1 -- 172.21.15.2:0/1099233019 wait complete. 2023-09-13T04:42:37.789 INFO:tasks.cephfs.filesystem:scrub status for tag:740235c0-6e1b-4729-b1fe-33345ae948c1 - {'path': '/', 'tag': '740235c0-6e1b-4729-b1fe-33345ae948c1', 'options': 'recursive,force'} 2023-09-13T04:42:38.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:37 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:42:38.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:37 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:42:38.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:37 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:42:39.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:38 smithi149 ceph-mon[133315]: scrub queued for path: / 2023-09-13T04:42:39.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:38 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:39.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:38 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:42:39.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:38 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:42:39.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:38 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:39.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:38 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:42:39.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:38 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:42:39.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:38 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:39.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:38 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:39.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:38 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:42:39.139 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:38 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:39.139 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:38 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:42:39.139 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:38 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:39.139 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:38 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:42:39.139 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:38 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:39.140 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:38 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:42:39.140 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:38 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:39.140 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:38 smithi149 ceph-mon[133315]: pgmap v68: 129 pgs: 129 active+clean; 181 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.4 MiB/s wr, 505 op/s 2023-09-13T04:42:39.140 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:38 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:42:39.140 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:38 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:42:39.141 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:38 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:42:39.141 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:38 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:42:39.141 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:38 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:42:39.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:38 smithi156 ceph-mon[133675]: scrub queued for path: / 2023-09-13T04:42:39.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:38 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:39.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:38 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:42:39.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:38 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:42:39.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:38 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:39.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:38 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:42:39.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:38 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:42:39.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:38 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:39.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:38 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:39.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:38 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:42:39.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:38 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:39.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:38 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:42:39.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:38 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:39.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:38 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:42:39.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:38 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:39.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:38 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:42:39.190 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:38 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:39.190 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:38 smithi156 ceph-mon[133675]: pgmap v68: 129 pgs: 129 active+clean; 181 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.4 MiB/s wr, 505 op/s 2023-09-13T04:42:39.190 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:38 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:42:39.190 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:38 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:42:39.190 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:38 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:42:39.191 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:38 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:42:39.191 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:38 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:42:39.262 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:38 smithi002 ceph-mon[97651]: scrub queued for path: / 2023-09-13T04:42:39.262 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:38 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:39.263 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:38 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:42:39.263 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:38 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:42:39.263 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:38 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:39.264 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:38 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:42:39.264 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:38 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:42:39.264 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:38 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:39.265 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:38 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:39.265 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:38 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:42:39.265 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:38 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:39.266 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:38 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:42:39.266 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:38 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:39.266 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:38 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:42:39.267 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:38 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:39.267 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:38 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:42:39.267 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:38 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:39.268 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:38 smithi002 ceph-mon[97651]: pgmap v68: 129 pgs: 129 active+clean; 181 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.4 MiB/s wr, 505 op/s 2023-09-13T04:42:39.268 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:38 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:42:39.268 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:38 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:42:39.268 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:38 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:42:39.268 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:38 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:42:39.269 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:38 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:42:40.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:39 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:40.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:39 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:40.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:39 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:42:40.597 INFO:tasks.workunit.client.0.smithi002.stdout: 50 424164 352 294106 370 134300 486 2023-09-13T04:42:41.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:40 smithi149 ceph-mon[133315]: pgmap v69: 129 pgs: 129 active+clean; 187 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.1 MiB/s wr, 621 op/s 2023-09-13T04:42:41.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:40 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:42:41.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:40 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:42:41.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:40 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:42:41.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:40 smithi156 ceph-mon[133675]: pgmap v69: 129 pgs: 129 active+clean; 187 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.1 MiB/s wr, 621 op/s 2023-09-13T04:42:41.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:40 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:42:41.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:40 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:42:41.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:40 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:42:41.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:40 smithi002 ceph-mon[97651]: pgmap v69: 129 pgs: 129 active+clean; 187 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.1 MiB/s wr, 621 op/s 2023-09-13T04:42:41.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:40 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:42:41.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:40 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:42:41.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:40 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:42:43.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:42 smithi156 ceph-mon[133675]: pgmap v70: 129 pgs: 129 active+clean; 187 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 465 op/s 2023-09-13T04:42:43.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:42 smithi002 ceph-mon[97651]: pgmap v70: 129 pgs: 129 active+clean; 187 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 465 op/s 2023-09-13T04:42:43.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:42 smithi149 ceph-mon[133315]: pgmap v70: 129 pgs: 129 active+clean; 187 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 465 op/s 2023-09-13T04:42:44.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:43 smithi002 ceph-mon[97651]: pgmap v71: 129 pgs: 129 active+clean; 190 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 516 op/s 2023-09-13T04:42:44.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:43 smithi149 ceph-mon[133315]: pgmap v71: 129 pgs: 129 active+clean; 190 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 516 op/s 2023-09-13T04:42:44.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:43 smithi156 ceph-mon[133675]: pgmap v71: 129 pgs: 129 active+clean; 190 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 516 op/s 2023-09-13T04:42:46.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:46 smithi002 ceph-mon[97651]: pgmap v72: 129 pgs: 129 active+clean; 190 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 261 op/s 2023-09-13T04:42:46.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:46 smithi149 ceph-mon[133315]: pgmap v72: 129 pgs: 129 active+clean; 190 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 261 op/s 2023-09-13T04:42:46.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:46 smithi156 ceph-mon[133675]: pgmap v72: 129 pgs: 129 active+clean; 190 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 261 op/s 2023-09-13T04:42:47.225 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:42:46 smithi156 systemd[1]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.c.service: Service RestartSec=10s expired, scheduling restart. 2023-09-13T04:42:47.225 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:42:46 smithi156 systemd[1]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.c.service: Scheduled restart job, restart counter is at 1. 2023-09-13T04:42:47.226 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:42:46 smithi156 systemd[1]: Stopped Ceph mds.c for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:42:47.226 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:42:46 smithi156 systemd[1]: Starting Ceph mds.c for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:42:47.504 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:42:47 smithi156 podman[175887]: 2023-09-13T04:42:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.811+0000 7fcb497ff700 1 -- 172.21.15.2:0/590826364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb4410f420 msgr2=0x7fcb44107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.811+0000 7fcb497ff700 1 --2- 172.21.15.2:0/590826364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb4410f420 0x7fcb44107cb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcb3801a9a0 tx=0x7fcb3801acb0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.815+0000 7fcb497ff700 1 -- 172.21.15.2:0/590826364 shutdown_connections 2023-09-13T04:42:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.815+0000 7fcb497ff700 1 --2- 172.21.15.2:0/590826364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb4411a490 0x7fcb4411c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.815+0000 7fcb497ff700 1 --2- 172.21.15.2:0/590826364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb4410f420 0x7fcb44107cb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.815+0000 7fcb497ff700 1 --2- 172.21.15.2:0/590826364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb4410f890 0x7fcb441081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.815+0000 7fcb497ff700 1 -- 172.21.15.2:0/590826364 >> 172.21.15.2:0/590826364 conn(0x7fcb4406cda0 msgr2=0x7fcb4406d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.815+0000 7fcb497ff700 1 -- 172.21.15.2:0/590826364 shutdown_connections 2023-09-13T04:42:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.815+0000 7fcb497ff700 1 -- 172.21.15.2:0/590826364 wait complete. 2023-09-13T04:42:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.817+0000 7fcb497ff700 1 Processor -- start 2023-09-13T04:42:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.818+0000 7fcb497ff700 1 -- start start 2023-09-13T04:42:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.818+0000 7fcb497ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb4410f420 0x7fcb44115970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.818+0000 7fcb497ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb4410f890 0x7fcb44115ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.818+0000 7fcb497ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb4411a490 0x7fcb441a2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.818+0000 7fcb497ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb441a2820 con 0x7fcb4410f890 2023-09-13T04:42:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.818+0000 7fcb497ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb441a2990 con 0x7fcb4410f420 2023-09-13T04:42:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.818+0000 7fcb497ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb441a2b50 con 0x7fcb4411a490 2023-09-13T04:42:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.819+0000 7fcb437fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb4411a490 0x7fcb441a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.819+0000 7fcb437fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb4411a490 0x7fcb441a2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:41784/0 (socket says 172.21.15.2:41784) 2023-09-13T04:42:47.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.819+0000 7fcb437fe700 1 -- 172.21.15.2:0/1332147106 learned_addr learned my addr 172.21.15.2:0/1332147106 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:47.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.819+0000 7fcb42ffd700 1 --2- 172.21.15.2:0/1332147106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb4410f420 0x7fcb44115970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:47.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.819+0000 7fcb427fc700 1 --2- 172.21.15.2:0/1332147106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb4410f890 0x7fcb44115ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:47.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.819+0000 7fcb437fe700 1 -- 172.21.15.2:0/1332147106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb4410f420 msgr2=0x7fcb44115970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:47.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.819+0000 7fcb437fe700 1 --2- 172.21.15.2:0/1332147106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb4410f420 0x7fcb44115970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.819+0000 7fcb437fe700 1 -- 172.21.15.2:0/1332147106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb4410f890 msgr2=0x7fcb44115ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:47.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.819+0000 7fcb437fe700 1 --2- 172.21.15.2:0/1332147106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb4410f890 0x7fcb44115ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.819+0000 7fcb437fe700 1 -- 172.21.15.2:0/1332147106 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3801a600 con 0x7fcb4411a490 2023-09-13T04:42:47.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.819+0000 7fcb427fc700 1 --2- 172.21.15.2:0/1332147106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb4410f890 0x7fcb44115ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:42:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.820+0000 7fcb437fe700 1 --2- 172.21.15.2:0/1332147106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb4411a490 0x7fcb441a2110 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c00b7b0 tx=0x7fcb3c00bac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.820+0000 7fcb2bfff700 1 -- 172.21.15.2:0/1332147106 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c010800 con 0x7fcb4411a490 2023-09-13T04:42:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.820+0000 7fcb2bfff700 1 -- 172.21.15.2:0/1332147106 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb3c010e20 con 0x7fcb4411a490 2023-09-13T04:42:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.820+0000 7fcb2bfff700 1 -- 172.21.15.2:0/1332147106 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c017e70 con 0x7fcb4411a490 2023-09-13T04:42:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.821+0000 7fcb497ff700 1 -- 172.21.15.2:0/1332147106 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb441a2e30 con 0x7fcb4411a490 2023-09-13T04:42:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.821+0000 7fcb497ff700 1 -- 172.21.15.2:0/1332147106 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb441aa510 con 0x7fcb4411a490 2023-09-13T04:42:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.826+0000 7fcb2bfff700 1 -- 172.21.15.2:0/1332147106 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7fcb3c0174d0 con 0x7fcb4411a490 2023-09-13T04:42:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.827+0000 7fcb5d122b80 1 Processor -- start 2023-09-13T04:42:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.828+0000 7fcb5d122b80 1 -- start start 2023-09-13T04:42:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.828+0000 7fcb5d122b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8e0be0 0x5615ab8e0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.828+0000 7fcb5d122b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615ab8e1500 0x5615ab8dc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.828+0000 7fcb5d122b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5615ab8dc840 0x5615ab8dcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.828+0000 7fcb5d122b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ab835720 con 0x5615ab8dc840 2023-09-13T04:42:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.828+0000 7fcb5d122b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ab833040 con 0x5615ab8e0be0 2023-09-13T04:42:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.828+0000 7fcb5d122b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ab82e930 con 0x5615ab8e1500 2023-09-13T04:42:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.829+0000 7fcb2bfff700 1 --2- 172.21.15.2:0/1332147106 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcb2c0a07b0 0x7fcb2c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.829+0000 7fcb437fe700 1 --2- 172.21.15.2:0/1332147106 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcb2c0a07b0 0x7fcb2c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.829+0000 7fcb427fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8e0be0 0x5615ab8e0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.829+0000 7fcb2bfff700 1 -- 172.21.15.2:0/1332147106 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fcb3c0cbb20 con 0x7fcb4411a490 2023-09-13T04:42:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.829+0000 7fcb427fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8e0be0 0x5615ab8e0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:41190/0 (socket says 172.21.15.2:41190) 2023-09-13T04:42:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.829+0000 7fcb427fc700 1 -- 172.21.15.2:0/2110618721 learned_addr learned my addr 172.21.15.2:0/2110618721 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.830+0000 7fcb437fe700 1 --2- 172.21.15.2:0/1332147106 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcb2c0a07b0 0x7fcb2c0a2c70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0c9010 tx=0x7fcb3c0949f0 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.830+0000 7fcb42ffd700 1 --2- 172.21.15.2:0/2110618721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615ab8e1500 0x5615ab8dc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.830+0000 7fcb427fc700 1 -- 172.21.15.2:0/2110618721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615ab8e1500 msgr2=0x5615ab8dc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.830+0000 7fcb427fc700 1 --2- 172.21.15.2:0/2110618721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615ab8e1500 0x5615ab8dc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.830+0000 7fcb427fc700 1 -- 172.21.15.2:0/2110618721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5615ab8dc840 msgr2=0x5615ab8dcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.830+0000 7fcb427fc700 1 --2- 172.21.15.2:0/2110618721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5615ab8dc840 0x5615ab8dcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.830+0000 7fcb427fc700 1 -- 172.21.15.2:0/2110618721 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615ab6ae5d0 con 0x5615ab8e0be0 2023-09-13T04:42:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.831+0000 7fcb427fc700 1 --2- 172.21.15.2:0/2110618721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8e0be0 0x5615ab8e0fc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400bce0 tx=0x7fcb34010370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88d7c725f94db06 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.831+0000 7fcb297fa700 1 -- 172.21.15.2:0/2110618721 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34010c30 con 0x5615ab8e0be0 2023-09-13T04:42:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.831+0000 7fcb297fa700 1 -- 172.21.15.2:0/2110618721 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb340044e0 con 0x5615ab8e0be0 2023-09-13T04:42:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.831+0000 7fcb297fa700 1 -- 172.21.15.2:0/2110618721 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34012450 con 0x5615ab8e0be0 2023-09-13T04:42:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.832+0000 7fcb5d122b80 1 -- 172.21.15.2:0/2110618721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8e0be0 msgr2=0x5615ab8e0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.832+0000 7fcb5d122b80 1 --2- 172.21.15.2:0/2110618721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8e0be0 0x5615ab8e0fc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400bce0 tx=0x7fcb34010370 comp rx=0 tx=0).stop 2023-09-13T04:42:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.832+0000 7fcb5d122b80 1 -- 172.21.15.2:0/2110618721 shutdown_connections 2023-09-13T04:42:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.832+0000 7fcb5d122b80 1 --2- 172.21.15.2:0/2110618721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5615ab8dc840 0x5615ab8dcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.832+0000 7fcb5d122b80 1 --2- 172.21.15.2:0/2110618721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8e0be0 0x5615ab8e0fc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.832+0000 7fcb5d122b80 1 --2- 172.21.15.2:0/2110618721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615ab8e1500 0x5615ab8dc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.832+0000 7fcb5d122b80 1 -- 172.21.15.2:0/2110618721 >> 172.21.15.2:0/2110618721 conn(0x5615ab83ac20 msgr2=0x5615ab8d6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.832+0000 7fcb5d122b80 1 -- 172.21.15.2:0/2110618721 shutdown_connections 2023-09-13T04:42:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.833+0000 7fcb5d122b80 1 -- 172.21.15.2:0/2110618721 wait complete. 2023-09-13T04:42:47.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.833+0000 7fcb5d122b80 1 -- start start 2023-09-13T04:42:47.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.834+0000 7fcb5d122b80 1 Processor -- start 2023-09-13T04:42:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.836+0000 7fcb5d122b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8dc840 0x5615ab8e5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.836+0000 7fcb5d122b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615ab8e0be0 0x5615ab8e5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.836+0000 7fcb5d122b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5615ab8e1500 0x5615ab8e63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.836+0000 7fcb5d122b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ab7ccb50 con 0x5615ab8e1500 2023-09-13T04:42:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.836+0000 7fcb5d122b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ab7cc850 con 0x5615ab8dc840 2023-09-13T04:42:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.836+0000 7fcb5d122b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ab6b08c0 con 0x5615ab8e0be0 2023-09-13T04:42:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.836+0000 7fcb5d122b80 10 client.0 _ll_register_callbacks cb 0x5615ab7ca620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:42:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.836+0000 7fcb427fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8dc840 0x5615ab8e5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.836+0000 7fcb427fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8dc840 0x5615ab8e5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:41206/0 (socket says 172.21.15.2:41206) 2023-09-13T04:42:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.836+0000 7fcb427fc700 1 -- 172.21.15.2:0/2482535608 learned_addr learned my addr 172.21.15.2:0/2482535608 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.836+0000 7fcb427fc700 1 --2- 172.21.15.2:0/2482535608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5615ab8e1500 0x5615ab8e63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.836+0000 7fcb42ffd700 1 --2- 172.21.15.2:0/2482535608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615ab8e0be0 0x5615ab8e5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.837+0000 7fcb427fc700 1 -- 172.21.15.2:0/2482535608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615ab8e0be0 msgr2=0x5615ab8e5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.837+0000 7fcb427fc700 1 --2- 172.21.15.2:0/2482535608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615ab8e0be0 0x5615ab8e5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.837+0000 7fcb427fc700 1 -- 172.21.15.2:0/2482535608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5615ab8e1500 msgr2=0x5615ab8e63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.837+0000 7fcb427fc700 1 --2- 172.21.15.2:0/2482535608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5615ab8e1500 0x5615ab8e63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.837+0000 7fcb427fc700 1 -- 172.21.15.2:0/2482535608 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3400b9c0 con 0x5615ab8dc840 2023-09-13T04:42:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.837+0000 7fcb427fc700 1 --2- 172.21.15.2:0/2482535608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8dc840 0x5615ab8e5960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400b460 tx=0x7fcb34005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffd8fe717ef179e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.837+0000 7fcb26ff5700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:42:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.837+0000 7fcb26ff5700 1 -- 172.21.15.2:0/2482535608 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34005840 con 0x5615ab8dc840 2023-09-13T04:42:47.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.838+0000 7fcb26ff5700 1 -- 172.21.15.2:0/2482535608 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fcb34010a30 con 0x5615ab8dc840 2023-09-13T04:42:47.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.838+0000 7fcb5d122b80 1 -- 172.21.15.2:0/2482535608 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615ab8a1ef0 con 0x5615ab8dc840 2023-09-13T04:42:47.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.838+0000 7fcb26ff5700 1 -- 172.21.15.2:0/2482535608 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb34010bf0 con 0x5615ab8dc840 2023-09-13T04:42:47.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.838+0000 7fcb5d122b80 10 client.26159 fetch_fsmap learned FSMap version 29 2023-09-13T04:42:47.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.838+0000 7fcb5d122b80 1 -- 172.21.15.2:0/2482535608 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5615ab846740 con 0x5615ab8dc840 2023-09-13T04:42:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.838+0000 7fcb26ff5700 1 -- 172.21.15.2:0/2482535608 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5615ab8a1ef0 con 0x5615ab8dc840 2023-09-13T04:42:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.838+0000 7fcb26ff5700 1 -- 172.21.15.2:0/2482535608 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fcb3401a400 con 0x5615ab8dc840 2023-09-13T04:42:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.839+0000 7fcb5d122b80 10 client.26159 fetch_fsmap finished waiting for FSMap version 29 2023-09-13T04:42:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.839+0000 7fcb5d122b80 10 client.26159 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:42:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.839+0000 7fcb5d122b80 20 client.26159 populate_metadata read hostname 'smithi002' 2023-09-13T04:42:47.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.839+0000 7fcb5d122b80 1 --2- 172.21.15.2:0/2482535608 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5615ab8f5f50 0x5615ab8f8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:47.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.839+0000 7fcb5d122b80 4 client.26159 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:42:47.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.839+0000 7fcb5d122b80 1 -- 172.21.15.2:0/2482535608 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615ab827910 con 0x5615ab8f5f50 2023-09-13T04:42:47.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.839+0000 7fcb42ffd700 1 --2- 172.21.15.2:0/2482535608 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5615ab8f5f50 0x5615ab8f8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:47.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.840+0000 7fcb26ff5700 10 client.26159 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:47.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.840+0000 7fcb42ffd700 1 --2- 172.21.15.2:0/2482535608 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5615ab8f5f50 0x5615ab8f8370 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8bd4335801d3cc6 server_cookie=bc891cb2295471ae in_seq=0 out_seq=0 2023-09-13T04:42:47.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.840+0000 7fcb26ff5700 1 -- 172.21.15.2:0/2482535608 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5615ab827910 con 0x5615ab8f5f50 2023-09-13T04:42:47.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.840+0000 7fcb26ff5700 10 client.26159 handle_command_reply: tid=0 2023-09-13T04:42:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.841+0000 7fcb5d122b80 1 client.26159 shutdown 2023-09-13T04:42:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.841+0000 7fcb5d122b80 1 -- 172.21.15.2:0/2482535608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8dc840 msgr2=0x5615ab8e5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.841+0000 7fcb5d122b80 1 --2- 172.21.15.2:0/2482535608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8dc840 0x5615ab8e5960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400b460 tx=0x7fcb34005ce0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.841+0000 7fcb5d122b80 1 -- 172.21.15.2:0/2482535608 shutdown_connections 2023-09-13T04:42:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.841+0000 7fcb5d122b80 1 --2- 172.21.15.2:0/2482535608 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5615ab8f5f50 0x5615ab8f8370 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.841+0000 7fcb5d122b80 1 --2- 172.21.15.2:0/2482535608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5615ab8e1500 0x5615ab8e63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.841+0000 7fcb5d122b80 1 --2- 172.21.15.2:0/2482535608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8dc840 0x5615ab8e5960 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.841+0000 7fcb26ff5700 0 client.26159 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.841+0000 7fcb5d122b80 1 --2- 172.21.15.2:0/2482535608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615ab8e0be0 0x5615ab8e5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.841+0000 7fcb5d122b80 1 -- 172.21.15.2:0/2482535608 >> 172.21.15.2:0/2482535608 conn(0x5615ab83ac20 msgr2=0x5615ab8e2330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:47.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.841+0000 7fcb5d122b80 1 -- 172.21.15.2:0/2482535608 shutdown_connections 2023-09-13T04:42:47.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.841+0000 7fcb5d122b80 1 -- 172.21.15.2:0/2482535608 wait complete. 2023-09-13T04:42:47.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.842+0000 7fcb5d122b80 20 client.26159 trim_cache size 0 max 16384 2023-09-13T04:42:47.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.860+0000 7fcb5d122b80 1 Processor -- start 2023-09-13T04:42:47.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.860+0000 7fcb5d122b80 1 -- start start 2023-09-13T04:42:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.860+0000 7fcb5d122b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5615ab8dc840 0x5615ab8d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.860+0000 7fcb5d122b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8e0be0 0x5615ab8e6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.860+0000 7fcb5d122b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615ab8e1500 0x5615ab8da970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:47.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.860+0000 7fcb5d122b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ab8d87a0 con 0x5615ab8dc840 2023-09-13T04:42:47.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.860+0000 7fcb5d122b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ab8d1c70 con 0x5615ab8e0be0 2023-09-13T04:42:47.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.860+0000 7fcb5d122b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ab8d89a0 con 0x5615ab8e1500 2023-09-13T04:42:47.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.861+0000 7fcb42ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8e0be0 0x5615ab8e6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:47.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.861+0000 7fcb42ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8e0be0 0x5615ab8e6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:41212/0 (socket says 172.21.15.2:41212) 2023-09-13T04:42:47.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.861+0000 7fcb42ffd700 1 -- 172.21.15.2:0/1308682713 learned_addr learned my addr 172.21.15.2:0/1308682713 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.861+0000 7fcb427fc700 1 --2- 172.21.15.2:0/1308682713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615ab8e1500 0x5615ab8da970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.861+0000 7fcb427fc700 1 --2- 172.21.15.2:0/1308682713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5615ab8dc840 0x5615ab8d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.861+0000 7fcb42ffd700 1 -- 172.21.15.2:0/1308682713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615ab8e1500 msgr2=0x5615ab8da970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.861+0000 7fcb42ffd700 1 --2- 172.21.15.2:0/1308682713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615ab8e1500 0x5615ab8da970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.861+0000 7fcb42ffd700 1 -- 172.21.15.2:0/1308682713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5615ab8dc840 msgr2=0x5615ab8d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.861+0000 7fcb42ffd700 1 --2- 172.21.15.2:0/1308682713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5615ab8dc840 0x5615ab8d1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.861+0000 7fcb42ffd700 1 -- 172.21.15.2:0/1308682713 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615ab8d7110 con 0x5615ab8e0be0 2023-09-13T04:42:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.861+0000 7fcb427fc700 1 --2- 172.21.15.2:0/1308682713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615ab8e1500 0x5615ab8da970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:42:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.862+0000 7fcb42ffd700 1 --2- 172.21.15.2:0/1308682713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8e0be0 0x5615ab8e6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcb38000c00 tx=0x7fcb3801c550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=509d3f3a44d08fb3 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:47.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.862+0000 7fcb29ffb700 1 -- 172.21.15.2:0/1308682713 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3802d040 con 0x5615ab8e0be0 2023-09-13T04:42:47.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.862+0000 7fcb29ffb700 1 -- 172.21.15.2:0/1308682713 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb3802e040 con 0x5615ab8e0be0 2023-09-13T04:42:47.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.862+0000 7fcb29ffb700 1 -- 172.21.15.2:0/1308682713 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb380052a0 con 0x5615ab8e0be0 2023-09-13T04:42:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.863+0000 7fcb5d122b80 1 -- 172.21.15.2:0/1308682713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8e0be0 msgr2=0x5615ab8e6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.863+0000 7fcb5d122b80 1 --2- 172.21.15.2:0/1308682713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8e0be0 0x5615ab8e6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcb38000c00 tx=0x7fcb3801c550 comp rx=0 tx=0).stop 2023-09-13T04:42:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.863+0000 7fcb5d122b80 1 -- 172.21.15.2:0/1308682713 shutdown_connections 2023-09-13T04:42:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.863+0000 7fcb5d122b80 1 --2- 172.21.15.2:0/1308682713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615ab8e1500 0x5615ab8da970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.863+0000 7fcb5d122b80 1 --2- 172.21.15.2:0/1308682713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5615ab8dc840 0x5615ab8d1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.863+0000 7fcb5d122b80 1 --2- 172.21.15.2:0/1308682713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8e0be0 0x5615ab8e6210 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.863+0000 7fcb5d122b80 1 -- 172.21.15.2:0/1308682713 >> 172.21.15.2:0/1308682713 conn(0x5615ab83ac20 msgr2=0x5615ab8df740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.863+0000 7fcb5d122b80 1 -- 172.21.15.2:0/1308682713 shutdown_connections 2023-09-13T04:42:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.863+0000 7fcb5d122b80 1 -- 172.21.15.2:0/1308682713 wait complete. 2023-09-13T04:42:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.863+0000 7fcb5d122b80 1 -- start start 2023-09-13T04:42:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.864+0000 7fcb5d122b80 1 Processor -- start 2023-09-13T04:42:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.864+0000 7fcb5d122b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615ab8dc840 0x5615ab8ecd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.864+0000 7fcb5d122b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5615ab8e0be0 0x5615ab8eb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.864+0000 7fcb5d122b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8e1500 0x5615ab8eb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.864+0000 7fcb5d122b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ab8ebb30 con 0x5615ab8e0be0 2023-09-13T04:42:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.864+0000 7fcb5d122b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ab8ebca0 con 0x5615ab8e1500 2023-09-13T04:42:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.864+0000 7fcb5d122b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ab8ebe60 con 0x5615ab8dc840 2023-09-13T04:42:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.864+0000 7fcb5d122b80 10 client.0 _ll_register_callbacks cb 0x5615ab7ca620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:42:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.864+0000 7fcb427fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615ab8dc840 0x5615ab8ecd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.865+0000 7fcb427fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615ab8dc840 0x5615ab8ecd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:41818/0 (socket says 172.21.15.2:41818) 2023-09-13T04:42:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.865+0000 7fcb427fc700 1 -- 172.21.15.2:0/1017484775 learned_addr learned my addr 172.21.15.2:0/1017484775 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:42:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.865+0000 7fcb427fc700 1 --2- 172.21.15.2:0/1017484775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8e1500 0x5615ab8eb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.865+0000 7fcb42ffd700 1 --2- 172.21.15.2:0/1017484775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5615ab8e0be0 0x5615ab8eb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.865+0000 7fcb427fc700 1 -- 172.21.15.2:0/1017484775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8e1500 msgr2=0x5615ab8eb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.865+0000 7fcb427fc700 1 --2- 172.21.15.2:0/1017484775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8e1500 0x5615ab8eb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.865+0000 7fcb427fc700 1 -- 172.21.15.2:0/1017484775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5615ab8e0be0 msgr2=0x5615ab8eb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:42:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.865+0000 7fcb427fc700 1 --2- 172.21.15.2:0/1017484775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5615ab8e0be0 0x5615ab8eb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.865+0000 7fcb427fc700 1 -- 172.21.15.2:0/1017484775 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5615ab846740 con 0x5615ab8dc840 2023-09-13T04:42:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.865+0000 7fcb427fc700 1 --2- 172.21.15.2:0/1017484775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615ab8dc840 0x5615ab8ecd80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcb340044b0 tx=0x7fcb3400b2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=338978763e626fa server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:42:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.865+0000 7fcb26ff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:42:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.865+0000 7fcb26ff5700 1 -- 172.21.15.2:0/1017484775 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34010c30 con 0x5615ab8dc840 2023-09-13T04:42:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.865+0000 7fcb5d122b80 1 -- 172.21.15.2:0/1017484775 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb340044e0 con 0x5615ab8dc840 2023-09-13T04:42:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.865+0000 7fcb26ff5700 1 -- 172.21.15.2:0/1017484775 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fcb34000e40 con 0x5615ab8dc840 2023-09-13T04:42:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.866+0000 7fcb26ff5700 1 -- 172.21.15.2:0/1017484775 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb340106c0 con 0x5615ab8dc840 2023-09-13T04:42:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.866+0000 7fcb5d122b80 10 client.26164 fetch_fsmap learned FSMap version 29 2023-09-13T04:42:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.866+0000 7fcb5d122b80 1 -- 172.21.15.2:0/1017484775 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fcb4411d1b0 con 0x5615ab8dc840 2023-09-13T04:42:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.866+0000 7fcb26ff5700 1 -- 172.21.15.2:0/1017484775 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb340044e0 con 0x5615ab8dc840 2023-09-13T04:42:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.866+0000 7fcb26ff5700 1 -- 172.21.15.2:0/1017484775 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fcb34012a90 con 0x5615ab8dc840 2023-09-13T04:42:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.866+0000 7fcb5d122b80 10 client.26164 fetch_fsmap finished waiting for FSMap version 29 2023-09-13T04:42:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.866+0000 7fcb5d122b80 10 client.26164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:42:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.866+0000 7fcb5d122b80 20 client.26164 populate_metadata read hostname 'smithi002' 2023-09-13T04:42:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.866+0000 7fcb5d122b80 1 --2- 172.21.15.2:0/1017484775 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5615ab94ad70 0x5615ab94d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:42:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.866+0000 7fcb5d122b80 4 client.26164 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:42:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.866+0000 7fcb5d122b80 1 -- 172.21.15.2:0/1017484775 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5615ab7e7ff0 con 0x5615ab94ad70 2023-09-13T04:42:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.867+0000 7fcb42ffd700 1 --2- 172.21.15.2:0/1017484775 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5615ab94ad70 0x5615ab94d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:42:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.867+0000 7fcb42ffd700 1 --2- 172.21.15.2:0/1017484775 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5615ab94ad70 0x5615ab94d190 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef95b58edd32703a server_cookie=746ed4d7def5ce1d in_seq=0 out_seq=0 2023-09-13T04:42:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:47.867+0000 7fcb26ff5700 10 client.26164 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:47.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:47 smithi156 ceph-mon[133675]: daemon mds.c restarted 2023-09-13T04:42:47.936 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:42:47 smithi156 bash[175887]: f90ca0e1c78bd249df35601d7674b376cdbb6a54d0cbf200a8f0806f3923506a 2023-09-13T04:42:47.937 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:42:47 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: starting mds.c at 2023-09-13T04:42:47.937 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:42:47 smithi156 systemd[1]: Started Ceph mds.c for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:42:48.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:47 smithi002 ceph-mon[97651]: daemon mds.c restarted 2023-09-13T04:42:48.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:47 smithi149 ceph-mon[133315]: daemon mds.c restarted 2023-09-13T04:42:49.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:48 smithi002 ceph-mon[97651]: osdmap e94: 12 total, 12 up, 12 in 2023-09-13T04:42:49.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:48 smithi002 ceph-mon[97651]: mds.? [v2:172.21.15.156:6836/2098926245,v1:172.21.15.156:6837/2098926245] up:boot 2023-09-13T04:42:49.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:48 smithi002 ceph-mon[97651]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2023-09-13T04:42:49.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:48 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-13T04:42:49.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:48 smithi002 ceph-mon[97651]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:42:49.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:48 smithi002 ceph-mon[97651]: pgmap v74: 129 pgs: 129 active+clean; 190 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 210 op/s 2023-09-13T04:42:49.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:48 smithi149 ceph-mon[133315]: osdmap e94: 12 total, 12 up, 12 in 2023-09-13T04:42:49.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:48 smithi149 ceph-mon[133315]: mds.? [v2:172.21.15.156:6836/2098926245,v1:172.21.15.156:6837/2098926245] up:boot 2023-09-13T04:42:49.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:48 smithi149 ceph-mon[133315]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2023-09-13T04:42:49.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:48 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-13T04:42:49.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:48 smithi149 ceph-mon[133315]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:42:49.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:48 smithi149 ceph-mon[133315]: pgmap v74: 129 pgs: 129 active+clean; 190 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 210 op/s 2023-09-13T04:42:49.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:48 smithi156 ceph-mon[133675]: osdmap e94: 12 total, 12 up, 12 in 2023-09-13T04:42:49.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:48 smithi156 ceph-mon[133675]: mds.? [v2:172.21.15.156:6836/2098926245,v1:172.21.15.156:6837/2098926245] up:boot 2023-09-13T04:42:49.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:48 smithi156 ceph-mon[133675]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2023-09-13T04:42:49.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:48 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-13T04:42:49.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:48 smithi156 ceph-mon[133675]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:42:49.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:48 smithi156 ceph-mon[133675]: pgmap v74: 129 pgs: 129 active+clean; 190 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 210 op/s 2023-09-13T04:42:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.708+0000 7fcb26ff5700 1 -- 172.21.15.2:0/1017484775 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5615ab7e7ff0 con 0x5615ab94ad70 2023-09-13T04:42:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.708+0000 7fcb26ff5700 10 client.26164 handle_command_reply: tid=0 2023-09-13T04:42:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.708+0000 7fcb5d122b80 1 client.26164 shutdown 2023-09-13T04:42:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.709+0000 7fcb5d122b80 1 -- 172.21.15.2:0/1017484775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615ab8dc840 msgr2=0x5615ab8ecd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.709+0000 7fcb5d122b80 1 --2- 172.21.15.2:0/1017484775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615ab8dc840 0x5615ab8ecd80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcb340044b0 tx=0x7fcb3400b2e0 comp rx=0 tx=0).stop 2023-09-13T04:42:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.709+0000 7fcb5d122b80 1 -- 172.21.15.2:0/1017484775 shutdown_connections 2023-09-13T04:42:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.709+0000 7fcb5d122b80 1 --2- 172.21.15.2:0/1017484775 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5615ab94ad70 0x5615ab94d190 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.709+0000 7fcb5d122b80 1 --2- 172.21.15.2:0/1017484775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615ab8e1500 0x5615ab8eb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.709+0000 7fcb26ff5700 0 client.26164 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:42:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.709+0000 7fcb5d122b80 1 --2- 172.21.15.2:0/1017484775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615ab8dc840 0x5615ab8ecd80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.709+0000 7fcb5d122b80 1 --2- 172.21.15.2:0/1017484775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5615ab8e0be0 0x5615ab8eb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.709+0000 7fcb5d122b80 1 -- 172.21.15.2:0/1017484775 >> 172.21.15.2:0/1017484775 conn(0x5615ab83ac20 msgr2=0x5615ab8da970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.709+0000 7fcb5d122b80 1 -- 172.21.15.2:0/1017484775 shutdown_connections 2023-09-13T04:42:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.709+0000 7fcb5d122b80 1 -- 172.21.15.2:0/1017484775 wait complete. 2023-09-13T04:42:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.710+0000 7fcb5d122b80 20 client.26164 trim_cache size 0 max 16384 2023-09-13T04:42:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:42:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:42:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:42:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:42:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.710+0000 7fcb497ff700 1 -- 172.21.15.2:0/1332147106 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcb2c0a07b0 msgr2=0x7fcb2c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.710+0000 7fcb497ff700 1 --2- 172.21.15.2:0/1332147106 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcb2c0a07b0 0x7fcb2c0a2c70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0c9010 tx=0x7fcb3c0949f0 comp rx=0 tx=0).stop 2023-09-13T04:42:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.710+0000 7fcb497ff700 1 -- 172.21.15.2:0/1332147106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb4411a490 msgr2=0x7fcb441a2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:42:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.710+0000 7fcb497ff700 1 --2- 172.21.15.2:0/1332147106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb4411a490 0x7fcb441a2110 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c00b7b0 tx=0x7fcb3c00bac0 comp rx=0 tx=0).stop 2023-09-13T04:42:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.710+0000 7fcb497ff700 1 -- 172.21.15.2:0/1332147106 shutdown_connections 2023-09-13T04:42:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.710+0000 7fcb497ff700 1 --2- 172.21.15.2:0/1332147106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb4411a490 0x7fcb441a2110 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.710+0000 7fcb497ff700 1 --2- 172.21.15.2:0/1332147106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb4410f420 0x7fcb44115970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.710+0000 7fcb497ff700 1 --2- 172.21.15.2:0/1332147106 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcb2c0a07b0 0x7fcb2c0a2c70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.710+0000 7fcb497ff700 1 --2- 172.21.15.2:0/1332147106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb4410f890 0x7fcb44115ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:42:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.710+0000 7fcb497ff700 1 -- 172.21.15.2:0/1332147106 >> 172.21.15.2:0/1332147106 conn(0x7fcb4406cda0 msgr2=0x7fcb440716b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:42:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.711+0000 7fcb497ff700 1 -- 172.21.15.2:0/1332147106 shutdown_connections 2023-09-13T04:42:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:42:49.711+0000 7fcb497ff700 1 -- 172.21.15.2:0/1332147106 wait complete. 2023-09-13T04:42:50.604 INFO:tasks.workunit.client.0.smithi002.stdout: 53 625944 222 445447 321 204547 507 2023-09-13T04:42:50.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:50 smithi156 ceph-mon[133675]: pgmap v75: 129 pgs: 129 active+clean; 207 MiB data, 978 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 211 op/s 2023-09-13T04:42:50.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:50 smithi002 ceph-mon[97651]: pgmap v75: 129 pgs: 129 active+clean; 207 MiB data, 978 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 211 op/s 2023-09-13T04:42:50.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:50 smithi149 ceph-mon[133315]: pgmap v75: 129 pgs: 129 active+clean; 207 MiB data, 978 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 211 op/s 2023-09-13T04:42:52.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:52 smithi149 ceph-mon[133315]: pgmap v76: 129 pgs: 129 active+clean; 207 MiB data, 978 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 211 op/s 2023-09-13T04:42:52.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:52 smithi156 ceph-mon[133675]: pgmap v76: 129 pgs: 129 active+clean; 207 MiB data, 978 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 211 op/s 2023-09-13T04:42:52.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:52 smithi002 ceph-mon[97651]: pgmap v76: 129 pgs: 129 active+clean; 207 MiB data, 978 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 211 op/s 2023-09-13T04:42:55.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:54 smithi002 ceph-mon[97651]: pgmap v77: 129 pgs: 129 active+clean; 220 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 3.7 MiB/s wr, 400 op/s 2023-09-13T04:42:55.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:54 smithi149 ceph-mon[133315]: pgmap v77: 129 pgs: 129 active+clean; 220 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 3.7 MiB/s wr, 400 op/s 2023-09-13T04:42:55.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:54 smithi156 ceph-mon[133675]: pgmap v77: 129 pgs: 129 active+clean; 220 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 3.7 MiB/s wr, 400 op/s 2023-09-13T04:42:56.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:56 smithi149 ceph-mon[133315]: pgmap v78: 129 pgs: 129 active+clean; 230 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 4.7 MiB/s wr, 499 op/s 2023-09-13T04:42:56.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:56 smithi156 ceph-mon[133675]: pgmap v78: 129 pgs: 129 active+clean; 230 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 4.7 MiB/s wr, 499 op/s 2023-09-13T04:42:56.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:56 smithi002 ceph-mon[97651]: pgmap v78: 129 pgs: 129 active+clean; 230 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 4.7 MiB/s wr, 499 op/s 2023-09-13T04:42:59.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:42:58 smithi002 ceph-mon[97651]: pgmap v79: 129 pgs: 129 active+clean; 230 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 4.6 MiB/s wr, 489 op/s 2023-09-13T04:42:59.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:42:58 smithi149 ceph-mon[133315]: pgmap v79: 129 pgs: 129 active+clean; 230 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 4.6 MiB/s wr, 489 op/s 2023-09-13T04:42:59.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:42:58 smithi156 ceph-mon[133675]: pgmap v79: 129 pgs: 129 active+clean; 230 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 4.6 MiB/s wr, 489 op/s 2023-09-13T04:43:00.598 INFO:tasks.workunit.client.0.smithi002.stdout: 60 625400 313 460779 420 226275 299 2023-09-13T04:43:00.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:00 smithi149 ceph-mon[133315]: pgmap v80: 129 pgs: 129 active+clean; 236 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 475 op/s 2023-09-13T04:43:00.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:00 smithi156 ceph-mon[133675]: pgmap v80: 129 pgs: 129 active+clean; 236 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 475 op/s 2023-09-13T04:43:00.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:00 smithi002 ceph-mon[97651]: pgmap v80: 129 pgs: 129 active+clean; 236 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 475 op/s 2023-09-13T04:43:03.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:02 smithi002 ceph-mon[97651]: pgmap v81: 129 pgs: 129 active+clean; 236 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 3.1 MiB/s wr, 358 op/s 2023-09-13T04:43:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.272+0000 7fe21abff700 1 -- 172.21.15.2:0/4142949020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2140739a0 msgr2=0x7fe21410cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.272+0000 7fe21abff700 1 --2- 172.21.15.2:0/4142949020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2140739a0 0x7fe21410cfc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe200009990 tx=0x7fe200009ca0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.272+0000 7fe21abff700 1 -- 172.21.15.2:0/4142949020 shutdown_connections 2023-09-13T04:43:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.272+0000 7fe21abff700 1 --2- 172.21.15.2:0/4142949020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe21410d500 0x7fe21410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.272+0000 7fe21abff700 1 --2- 172.21.15.2:0/4142949020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe214073080 0x7fe214073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.272+0000 7fe21abff700 1 --2- 172.21.15.2:0/4142949020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2140739a0 0x7fe21410cfc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.272+0000 7fe21abff700 1 -- 172.21.15.2:0/4142949020 >> 172.21.15.2:0/4142949020 conn(0x7fe2140fc730 msgr2=0x7fe2140feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.272+0000 7fe21abff700 1 -- 172.21.15.2:0/4142949020 shutdown_connections 2023-09-13T04:43:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.272+0000 7fe21abff700 1 -- 172.21.15.2:0/4142949020 wait complete. 2023-09-13T04:43:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.274+0000 7fe21abff700 1 Processor -- start 2023-09-13T04:43:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.274+0000 7fe21abff700 1 -- start start 2023-09-13T04:43:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.274+0000 7fe21abff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe214073080 0x7fe214199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.274+0000 7fe21abff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2140739a0 0x7fe214199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.274+0000 7fe21abff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe21410d500 0x7fe21419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.274+0000 7fe21abff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21419e200 con 0x7fe21410d500 2023-09-13T04:43:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.274+0000 7fe21abff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21419e370 con 0x7fe2140739a0 2023-09-13T04:43:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.274+0000 7fe21abff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21419e530 con 0x7fe214073080 2023-09-13T04:43:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.274+0000 7fe21899b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe214073080 0x7fe214199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.274+0000 7fe213fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2140739a0 0x7fe214199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.274+0000 7fe21919c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe21410d500 0x7fe21419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.275+0000 7fe21899b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe214073080 0x7fe214199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:58338/0 (socket says 172.21.15.2:58338) 2023-09-13T04:43:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.275+0000 7fe21899b700 1 -- 172.21.15.2:0/107061909 learned_addr learned my addr 172.21.15.2:0/107061909 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.275+0000 7fe21899b700 1 -- 172.21.15.2:0/107061909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2140739a0 msgr2=0x7fe214199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.275+0000 7fe21899b700 1 --2- 172.21.15.2:0/107061909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2140739a0 0x7fe214199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.275+0000 7fe21899b700 1 -- 172.21.15.2:0/107061909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe21410d500 msgr2=0x7fe21419dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.275+0000 7fe21899b700 1 --2- 172.21.15.2:0/107061909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe21410d500 0x7fe21419dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.275+0000 7fe21899b700 1 -- 172.21.15.2:0/107061909 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2000095f0 con 0x7fe214073080 2023-09-13T04:43:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.275+0000 7fe213fff700 1 --2- 172.21.15.2:0/107061909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2140739a0 0x7fe214199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:43:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.275+0000 7fe21919c700 1 --2- 172.21.15.2:0/107061909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe21410d500 0x7fe21419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:43:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.275+0000 7fe21899b700 1 --2- 172.21.15.2:0/107061909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe214073080 0x7fe214199010 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe20800e9a0 tx=0x7fe20800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.276+0000 7fe211ffb700 1 -- 172.21.15.2:0/107061909 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe208009790 con 0x7fe214073080 2023-09-13T04:43:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.276+0000 7fe211ffb700 1 -- 172.21.15.2:0/107061909 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe208010430 con 0x7fe214073080 2023-09-13T04:43:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.276+0000 7fe21abff700 1 -- 172.21.15.2:0/107061909 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21419e810 con 0x7fe214073080 2023-09-13T04:43:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.276+0000 7fe211ffb700 1 -- 172.21.15.2:0/107061909 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe208018680 con 0x7fe214073080 2023-09-13T04:43:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.276+0000 7fe21abff700 1 -- 172.21.15.2:0/107061909 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2141a6050 con 0x7fe214073080 2023-09-13T04:43:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.279+0000 7fe211ffb700 1 -- 172.21.15.2:0/107061909 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7fe2080105a0 con 0x7fe214073080 2023-09-13T04:43:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.280+0000 7fe22e5f3b80 1 Processor -- start 2023-09-13T04:43:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.280+0000 7fe22e5f3b80 1 -- start start 2023-09-13T04:43:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.280+0000 7fe211ffb700 1 --2- 172.21.15.2:0/107061909 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe2040a07b0 0x7fe2040a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.281+0000 7fe211ffb700 1 -- 172.21.15.2:0/107061909 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fe208014070 con 0x7fe214073080 2023-09-13T04:43:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.281+0000 7fe22e5f3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f6be0 0x555e952f6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.281+0000 7fe22e5f3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f7500 0x555e952f2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.281+0000 7fe22e5f3b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e952f2840 0x555e952f2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.281+0000 7fe22e5f3b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9524b720 con 0x555e952f2840 2023-09-13T04:43:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.281+0000 7fe22e5f3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e95249040 con 0x555e952f6be0 2023-09-13T04:43:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.281+0000 7fe22e5f3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e95244930 con 0x555e952f7500 2023-09-13T04:43:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.281+0000 7fe21919c700 1 --2- 172.21.15.2:0/107061909 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe2040a07b0 0x7fe2040a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.281+0000 7fe21899b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f6be0 0x555e952f6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.281+0000 7fe21899b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f6be0 0x555e952f6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:35490/0 (socket says 172.21.15.2:35490) 2023-09-13T04:43:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.281+0000 7fe21899b700 1 -- 172.21.15.2:0/1264785045 learned_addr learned my addr 172.21.15.2:0/1264785045 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.281+0000 7fe213fff700 1 --2- 172.21.15.2:0/1264785045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f7500 0x555e952f2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.281+0000 7fe21899b700 1 -- 172.21.15.2:0/1264785045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f7500 msgr2=0x555e952f2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.281+0000 7fe21899b700 1 --2- 172.21.15.2:0/1264785045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f7500 0x555e952f2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.281+0000 7fe21899b700 1 -- 172.21.15.2:0/1264785045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e952f2840 msgr2=0x555e952f2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.281+0000 7fe21899b700 1 --2- 172.21.15.2:0/1264785045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e952f2840 0x555e952f2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.281+0000 7fe21899b700 1 -- 172.21.15.2:0/1264785045 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e950c45d0 con 0x555e952f6be0 2023-09-13T04:43:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.282+0000 7fe213fff700 1 --2- 172.21.15.2:0/1264785045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f7500 0x555e952f2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:43:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.282+0000 7fe21899b700 1 --2- 172.21.15.2:0/1264785045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f6be0 0x555e952f6fc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe208096050 tx=0x7fe208099010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f863021cf6b7444d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.282+0000 7fe1feffd700 1 -- 172.21.15.2:0/1264785045 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20809d410 con 0x555e952f6be0 2023-09-13T04:43:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.282+0000 7fe21919c700 1 --2- 172.21.15.2:0/107061909 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe2040a07b0 0x7fe2040a2c70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x555e952eeae0 tx=0x7fe20c009920 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.282+0000 7fe1feffd700 1 -- 172.21.15.2:0/1264785045 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2080a1650 con 0x555e952f6be0 2023-09-13T04:43:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.282+0000 7fe1feffd700 1 -- 172.21.15.2:0/1264785045 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe208099840 con 0x555e952f6be0 2023-09-13T04:43:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.283+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/1264785045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f6be0 msgr2=0x555e952f6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.283+0000 7fe22e5f3b80 1 --2- 172.21.15.2:0/1264785045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f6be0 0x555e952f6fc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe208096050 tx=0x7fe208099010 comp rx=0 tx=0).stop 2023-09-13T04:43:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.283+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/1264785045 shutdown_connections 2023-09-13T04:43:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.283+0000 7fe22e5f3b80 1 --2- 172.21.15.2:0/1264785045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e952f2840 0x555e952f2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.283+0000 7fe22e5f3b80 1 --2- 172.21.15.2:0/1264785045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f6be0 0x555e952f6fc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.283+0000 7fe22e5f3b80 1 --2- 172.21.15.2:0/1264785045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f7500 0x555e952f2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.283+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/1264785045 >> 172.21.15.2:0/1264785045 conn(0x555e95250c20 msgr2=0x555e952ecb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.283+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/1264785045 shutdown_connections 2023-09-13T04:43:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.283+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/1264785045 wait complete. 2023-09-13T04:43:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.283+0000 7fe22e5f3b80 1 -- start start 2023-09-13T04:43:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.284+0000 7fe22e5f3b80 1 Processor -- start 2023-09-13T04:43:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.285+0000 7fe22e5f3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f2840 0x555e952fb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.285+0000 7fe22e5f3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f6be0 0x555e952fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.285+0000 7fe22e5f3b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e952f7500 0x555e952fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.285+0000 7fe22e5f3b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e951e2b50 con 0x555e952f7500 2023-09-13T04:43:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.285+0000 7fe22e5f3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e951e2850 con 0x555e952f2840 2023-09-13T04:43:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.285+0000 7fe22e5f3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e950c68c0 con 0x555e952f6be0 2023-09-13T04:43:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.285+0000 7fe22e5f3b80 10 client.0 _ll_register_callbacks cb 0x555e951e0620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:43:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.285+0000 7fe21899b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f2840 0x555e952fb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.285+0000 7fe21899b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f2840 0x555e952fb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:35494/0 (socket says 172.21.15.2:35494) 2023-09-13T04:43:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.285+0000 7fe21899b700 1 -- 172.21.15.2:0/3986180817 learned_addr learned my addr 172.21.15.2:0/3986180817 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.285+0000 7fe213fff700 1 --2- 172.21.15.2:0/3986180817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f6be0 0x555e952fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.285+0000 7fe21919c700 1 --2- 172.21.15.2:0/3986180817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e952f7500 0x555e952fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.285+0000 7fe21899b700 1 -- 172.21.15.2:0/3986180817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f6be0 msgr2=0x555e952fbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.285+0000 7fe21899b700 1 --2- 172.21.15.2:0/3986180817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f6be0 0x555e952fbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.285+0000 7fe21899b700 1 -- 172.21.15.2:0/3986180817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e952f7500 msgr2=0x555e952fc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.285+0000 7fe21899b700 1 --2- 172.21.15.2:0/3986180817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e952f7500 0x555e952fc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.285+0000 7fe21899b700 1 -- 172.21.15.2:0/3986180817 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2141a6050 con 0x555e952f2840 2023-09-13T04:43:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.285+0000 7fe21899b700 1 --2- 172.21.15.2:0/3986180817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f2840 0x555e952fb960 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe2080a1120 tx=0x7fe20809d4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95caceac9464688e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.285+0000 7fe1f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:43:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.286+0000 7fe1f3fff700 1 -- 172.21.15.2:0/3986180817 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2080b1be0 con 0x555e952f2840 2023-09-13T04:43:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.286+0000 7fe1f3fff700 1 -- 172.21.15.2:0/3986180817 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fe208098ad0 con 0x555e952f2840 2023-09-13T04:43:03.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.286+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/3986180817 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e952b7ef0 con 0x555e952f2840 2023-09-13T04:43:03.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.286+0000 7fe1f3fff700 1 -- 172.21.15.2:0/3986180817 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe208098c90 con 0x555e952f2840 2023-09-13T04:43:03.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.286+0000 7fe22e5f3b80 10 client.26177 fetch_fsmap learned FSMap version 29 2023-09-13T04:43:03.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.286+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/3986180817 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555e9525c740 con 0x555e952f2840 2023-09-13T04:43:03.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.286+0000 7fe1f3fff700 1 -- 172.21.15.2:0/3986180817 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555e952b7ef0 con 0x555e952f2840 2023-09-13T04:43:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.286+0000 7fe1f3fff700 1 -- 172.21.15.2:0/3986180817 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe208098540 con 0x555e952f2840 2023-09-13T04:43:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.286+0000 7fe22e5f3b80 10 client.26177 fetch_fsmap finished waiting for FSMap version 29 2023-09-13T04:43:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.286+0000 7fe22e5f3b80 10 client.26177 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:43:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.286+0000 7fe22e5f3b80 20 client.26177 populate_metadata read hostname 'smithi002' 2023-09-13T04:43:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.286+0000 7fe22e5f3b80 1 --2- 172.21.15.2:0/3986180817 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x555e9530bba0 0x555e9530dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.286+0000 7fe22e5f3b80 4 client.26177 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:43:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.286+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/3986180817 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e9523d910 con 0x555e9530bba0 2023-09-13T04:43:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.287+0000 7fe213fff700 1 --2- 172.21.15.2:0/3986180817 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x555e9530bba0 0x555e9530dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.287+0000 7fe213fff700 1 --2- 172.21.15.2:0/3986180817 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x555e9530bba0 0x555e9530dfc0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6d563428b7c8174 server_cookie=9920ee849d956a10 in_seq=0 out_seq=0 2023-09-13T04:43:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.287+0000 7fe1f3fff700 10 client.26177 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:43:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.288+0000 7fe1f3fff700 1 -- 172.21.15.2:0/3986180817 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555e9523d910 con 0x555e9530bba0 2023-09-13T04:43:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.288+0000 7fe1f3fff700 10 client.26177 handle_command_reply: tid=0 2023-09-13T04:43:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.288+0000 7fe22e5f3b80 1 client.26177 shutdown 2023-09-13T04:43:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.288+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/3986180817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f2840 msgr2=0x555e952fb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.288+0000 7fe22e5f3b80 1 --2- 172.21.15.2:0/3986180817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f2840 0x555e952fb960 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe2080a1120 tx=0x7fe20809d4c0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.288+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/3986180817 shutdown_connections 2023-09-13T04:43:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.288+0000 7fe22e5f3b80 1 --2- 172.21.15.2:0/3986180817 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x555e9530bba0 0x555e9530dfc0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.288+0000 7fe22e5f3b80 1 --2- 172.21.15.2:0/3986180817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e952f7500 0x555e952fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.288+0000 7fe22e5f3b80 1 --2- 172.21.15.2:0/3986180817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f2840 0x555e952fb960 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.288+0000 7fe22e5f3b80 1 --2- 172.21.15.2:0/3986180817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f6be0 0x555e952fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.288+0000 7fe1f3fff700 0 client.26177 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:43:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.288+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/3986180817 >> 172.21.15.2:0/3986180817 conn(0x555e95250c20 msgr2=0x555e952f8330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.288+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/3986180817 shutdown_connections 2023-09-13T04:43:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.288+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/3986180817 wait complete. 2023-09-13T04:43:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.288+0000 7fe22e5f3b80 20 client.26177 trim_cache size 0 max 16384 2023-09-13T04:43:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.304+0000 7fe22e5f3b80 1 Processor -- start 2023-09-13T04:43:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.304+0000 7fe22e5f3b80 1 -- start start 2023-09-13T04:43:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.304+0000 7fe22e5f3b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e952f2840 0x555e952e7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.304+0000 7fe22e5f3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f6be0 0x555e95302ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.304+0000 7fe22e5f3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f7500 0x555e952fc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.305+0000 7fe22e5f3b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e952ede20 con 0x555e952f2840 2023-09-13T04:43:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.305+0000 7fe22e5f3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e952e7c70 con 0x555e952f7500 2023-09-13T04:43:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.305+0000 7fe22e5f3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e95303430 con 0x555e952f6be0 2023-09-13T04:43:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.305+0000 7fe213fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f6be0 0x555e95302ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.305+0000 7fe21899b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e952f2840 0x555e952e7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.305+0000 7fe21919c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f7500 0x555e952fc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.305+0000 7fe213fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f6be0 0x555e95302ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:58370/0 (socket says 172.21.15.2:58370) 2023-09-13T04:43:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.305+0000 7fe213fff700 1 -- 172.21.15.2:0/2722383346 learned_addr learned my addr 172.21.15.2:0/2722383346 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.305+0000 7fe213fff700 1 -- 172.21.15.2:0/2722383346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f7500 msgr2=0x555e952fc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.305+0000 7fe213fff700 1 --2- 172.21.15.2:0/2722383346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f7500 0x555e952fc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.305+0000 7fe213fff700 1 -- 172.21.15.2:0/2722383346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e952f2840 msgr2=0x555e952e7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.305+0000 7fe213fff700 1 --2- 172.21.15.2:0/2722383346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e952f2840 0x555e952e7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.305+0000 7fe213fff700 1 -- 172.21.15.2:0/2722383346 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e95070d80 con 0x555e952f6be0 2023-09-13T04:43:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.305+0000 7fe21919c700 1 --2- 172.21.15.2:0/2722383346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f7500 0x555e952fc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:43:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.305+0000 7fe213fff700 1 --2- 172.21.15.2:0/2722383346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f6be0 0x555e95302ef0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe2000098e0 tx=0x7fe200009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a3bf87c457eeca5 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.306+0000 7fe1ff7fe700 1 -- 172.21.15.2:0/2722383346 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20001d070 con 0x555e952f6be0 2023-09-13T04:43:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.306+0000 7fe1ff7fe700 1 -- 172.21.15.2:0/2722383346 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe20001e040 con 0x555e952f6be0 2023-09-13T04:43:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.306+0000 7fe1ff7fe700 1 -- 172.21.15.2:0/2722383346 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2000273e0 con 0x555e952f6be0 2023-09-13T04:43:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.306+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/2722383346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f6be0 msgr2=0x555e95302ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.306+0000 7fe22e5f3b80 1 --2- 172.21.15.2:0/2722383346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f6be0 0x555e95302ef0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe2000098e0 tx=0x7fe200009310 comp rx=0 tx=0).stop 2023-09-13T04:43:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.306+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/2722383346 shutdown_connections 2023-09-13T04:43:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.306+0000 7fe22e5f3b80 1 --2- 172.21.15.2:0/2722383346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f7500 0x555e952fc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.306+0000 7fe22e5f3b80 1 --2- 172.21.15.2:0/2722383346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e952f2840 0x555e952e7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.306+0000 7fe22e5f3b80 1 --2- 172.21.15.2:0/2722383346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f6be0 0x555e95302ef0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.306+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/2722383346 >> 172.21.15.2:0/2722383346 conn(0x555e95250c20 msgr2=0x555e952eca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.306+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/2722383346 shutdown_connections 2023-09-13T04:43:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.306+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/2722383346 wait complete. 2023-09-13T04:43:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.307+0000 7fe22e5f3b80 1 -- start start 2023-09-13T04:43:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.307+0000 7fe22e5f3b80 1 Processor -- start 2023-09-13T04:43:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.307+0000 7fe22e5f3b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e952f2840 0x555e95300ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.308+0000 7fe22e5f3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f6be0 0x555e953010e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.308+0000 7fe22e5f3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f7500 0x555e95358960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.308+0000 7fe22e5f3b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e95358ea0 con 0x555e952f2840 2023-09-13T04:43:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.308+0000 7fe22e5f3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e95358fe0 con 0x555e952f7500 2023-09-13T04:43:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.308+0000 7fe22e5f3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e953591a0 con 0x555e952f6be0 2023-09-13T04:43:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.308+0000 7fe22e5f3b80 10 client.0 _ll_register_callbacks cb 0x555e951e0620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:43:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.308+0000 7fe213fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f6be0 0x555e953010e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.308+0000 7fe213fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f6be0 0x555e953010e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:58378/0 (socket says 172.21.15.2:58378) 2023-09-13T04:43:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.308+0000 7fe213fff700 1 -- 172.21.15.2:0/3681319800 learned_addr learned my addr 172.21.15.2:0/3681319800 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.308+0000 7fe21899b700 1 --2- 172.21.15.2:0/3681319800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e952f2840 0x555e95300ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.308+0000 7fe21919c700 1 --2- 172.21.15.2:0/3681319800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f7500 0x555e95358960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.308+0000 7fe213fff700 1 -- 172.21.15.2:0/3681319800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f7500 msgr2=0x555e95358960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.308+0000 7fe213fff700 1 --2- 172.21.15.2:0/3681319800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f7500 0x555e95358960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.308+0000 7fe213fff700 1 -- 172.21.15.2:0/3681319800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e952f2840 msgr2=0x555e95300ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.308+0000 7fe213fff700 1 --2- 172.21.15.2:0/3681319800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e952f2840 0x555e95300ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.308+0000 7fe213fff700 1 -- 172.21.15.2:0/3681319800 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe20c0047b0 con 0x555e952f6be0 2023-09-13T04:43:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.308+0000 7fe21919c700 1 --2- 172.21.15.2:0/3681319800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f7500 0x555e95358960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:43:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.308+0000 7fe213fff700 1 --2- 172.21.15.2:0/3681319800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f6be0 0x555e953010e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe200005f20 tx=0x7fe2000058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad4ba8bd4f3a7e3a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.308+0000 7fe1f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:43:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.309+0000 7fe1f3fff700 1 -- 172.21.15.2:0/3681319800 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20001d070 con 0x555e952f6be0 2023-09-13T04:43:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.309+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/3681319800 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2080a1650 con 0x555e952f6be0 2023-09-13T04:43:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.309+0000 7fe1f3fff700 1 -- 172.21.15.2:0/3681319800 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fe200027a70 con 0x555e952f6be0 2023-09-13T04:43:03.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.309+0000 7fe1f3fff700 1 -- 172.21.15.2:0/3681319800 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe200027ea0 con 0x555e952f6be0 2023-09-13T04:43:03.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.309+0000 7fe22e5f3b80 10 client.26182 fetch_fsmap learned FSMap version 29 2023-09-13T04:43:03.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.309+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/3681319800 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fe214110640 con 0x555e952f6be0 2023-09-13T04:43:03.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.309+0000 7fe1f3fff700 1 -- 172.21.15.2:0/3681319800 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2080a1650 con 0x555e952f6be0 2023-09-13T04:43:03.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.310+0000 7fe1f3fff700 1 -- 172.21.15.2:0/3681319800 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe2000046e0 con 0x555e952f6be0 2023-09-13T04:43:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.310+0000 7fe22e5f3b80 10 client.26182 fetch_fsmap finished waiting for FSMap version 29 2023-09-13T04:43:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.310+0000 7fe22e5f3b80 10 client.26182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:43:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.310+0000 7fe22e5f3b80 20 client.26182 populate_metadata read hostname 'smithi002' 2023-09-13T04:43:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.310+0000 7fe22e5f3b80 1 --2- 172.21.15.2:0/3681319800 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x555e95360730 0x555e95362b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.310+0000 7fe22e5f3b80 4 client.26182 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:43:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.310+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/3681319800 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555e951fdff0 con 0x555e95360730 2023-09-13T04:43:03.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.310+0000 7fe21899b700 1 --2- 172.21.15.2:0/3681319800 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x555e95360730 0x555e95362b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:03.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.311+0000 7fe21899b700 1 --2- 172.21.15.2:0/3681319800 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x555e95360730 0x555e95362b50 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6d7413b609dcaf9 server_cookie=9475d673b6cfbd5c in_seq=0 out_seq=0 2023-09-13T04:43:03.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:03.311+0000 7fe1f3fff700 10 client.26182 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:43:03.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:02 smithi149 ceph-mon[133315]: pgmap v81: 129 pgs: 129 active+clean; 236 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 3.1 MiB/s wr, 358 op/s 2023-09-13T04:43:03.439 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:02 smithi156 ceph-mon[133675]: pgmap v81: 129 pgs: 129 active+clean; 236 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 3.1 MiB/s wr, 358 op/s 2023-09-13T04:43:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.069+0000 7fe1f3fff700 1 -- 172.21.15.2:0/3681319800 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555e951fdff0 con 0x555e95360730 2023-09-13T04:43:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.069+0000 7fe1f3fff700 10 client.26182 handle_command_reply: tid=0 2023-09-13T04:43:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.069+0000 7fe22e5f3b80 1 client.26182 shutdown 2023-09-13T04:43:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.069+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/3681319800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f6be0 msgr2=0x555e953010e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.069+0000 7fe22e5f3b80 1 --2- 172.21.15.2:0/3681319800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f6be0 0x555e953010e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe200005f20 tx=0x7fe2000058e0 comp rx=0 tx=0).stop 2023-09-13T04:43:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.069+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/3681319800 shutdown_connections 2023-09-13T04:43:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.069+0000 7fe22e5f3b80 1 --2- 172.21.15.2:0/3681319800 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x555e95360730 0x555e95362b50 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.069+0000 7fe22e5f3b80 1 --2- 172.21.15.2:0/3681319800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e952f7500 0x555e95358960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.069+0000 7fe22e5f3b80 1 --2- 172.21.15.2:0/3681319800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e952f2840 0x555e95300ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.069+0000 7fe1f3fff700 0 client.26182 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:43:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.069+0000 7fe22e5f3b80 1 --2- 172.21.15.2:0/3681319800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e952f6be0 0x555e953010e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.069+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/3681319800 >> 172.21.15.2:0/3681319800 conn(0x555e95250c20 msgr2=0x555e952fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.070+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/3681319800 shutdown_connections 2023-09-13T04:43:04.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.071+0000 7fe22e5f3b80 1 -- 172.21.15.2:0/3681319800 wait complete. 2023-09-13T04:43:04.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.071+0000 7fe22e5f3b80 20 client.26182 trim_cache size 0 max 16384 2023-09-13T04:43:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:43:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:43:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:43:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:43:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.072+0000 7fe21abff700 1 -- 172.21.15.2:0/107061909 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe2040a07b0 msgr2=0x7fe2040a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.072+0000 7fe21abff700 1 --2- 172.21.15.2:0/107061909 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe2040a07b0 0x7fe2040a2c70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x555e952eeae0 tx=0x7fe20c009920 comp rx=0 tx=0).stop 2023-09-13T04:43:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.072+0000 7fe21abff700 1 -- 172.21.15.2:0/107061909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe214073080 msgr2=0x7fe214199010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.072+0000 7fe21abff700 1 --2- 172.21.15.2:0/107061909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe214073080 0x7fe214199010 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe20800e9a0 tx=0x7fe20800ecb0 comp rx=0 tx=0).stop 2023-09-13T04:43:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.072+0000 7fe21abff700 1 -- 172.21.15.2:0/107061909 shutdown_connections 2023-09-13T04:43:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.072+0000 7fe21abff700 1 --2- 172.21.15.2:0/107061909 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe2040a07b0 0x7fe2040a2c70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.072+0000 7fe21abff700 1 --2- 172.21.15.2:0/107061909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe21410d500 0x7fe21419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.072+0000 7fe21abff700 1 --2- 172.21.15.2:0/107061909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe214073080 0x7fe214199010 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.072+0000 7fe21abff700 1 --2- 172.21.15.2:0/107061909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2140739a0 0x7fe214199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.072+0000 7fe21abff700 1 -- 172.21.15.2:0/107061909 >> 172.21.15.2:0/107061909 conn(0x7fe2140fc730 msgr2=0x7fe21410f620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.072+0000 7fe21abff700 1 -- 172.21.15.2:0/107061909 shutdown_connections 2023-09-13T04:43:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:04.072+0000 7fe21abff700 1 -- 172.21.15.2:0/107061909 wait complete. 2023-09-13T04:43:05.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:05 smithi149 ceph-mon[133315]: pgmap v82: 129 pgs: 129 active+clean; 257 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 572 op/s 2023-09-13T04:43:05.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:05 smithi156 ceph-mon[133675]: pgmap v82: 129 pgs: 129 active+clean; 257 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 572 op/s 2023-09-13T04:43:05.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:05 smithi002 ceph-mon[97651]: pgmap v82: 129 pgs: 129 active+clean; 257 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 572 op/s 2023-09-13T04:43:06.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:06 smithi149 ceph-mon[133315]: pgmap v83: 129 pgs: 129 active+clean; 263 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 448 op/s 2023-09-13T04:43:06.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:06 smithi156 ceph-mon[133675]: pgmap v83: 129 pgs: 129 active+clean; 263 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 448 op/s 2023-09-13T04:43:06.513 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:06 smithi002 ceph-mon[97651]: pgmap v83: 129 pgs: 129 active+clean; 263 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 448 op/s 2023-09-13T04:43:07.790 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T04:43:09.263 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:08 smithi002 ceph-mon[97651]: pgmap v84: 129 pgs: 129 active+clean; 263 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 357 op/s 2023-09-13T04:43:09.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:08 smithi149 ceph-mon[133315]: pgmap v84: 129 pgs: 129 active+clean; 263 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 357 op/s 2023-09-13T04:43:09.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:08 smithi156 ceph-mon[133675]: pgmap v84: 129 pgs: 129 active+clean; 263 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 357 op/s 2023-09-13T04:43:10.598 INFO:tasks.workunit.client.0.smithi002.stdout: 61 507184 203 381559 142 175241 140 2023-09-13T04:43:10.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:10 smithi156 ceph-mon[133675]: pgmap v85: 129 pgs: 129 active+clean; 268 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 422 op/s 2023-09-13T04:43:10.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:10 smithi002 ceph-mon[97651]: pgmap v85: 129 pgs: 129 active+clean; 268 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 422 op/s 2023-09-13T04:43:10.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:10 smithi149 ceph-mon[133315]: pgmap v85: 129 pgs: 129 active+clean; 268 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 422 op/s 2023-09-13T04:43:11.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.107+0000 7f5d6dd9d700 1 -- 172.21.15.2:0/1625016162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d68073050 msgr2=0x7f5d68073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:11.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.107+0000 7f5d6dd9d700 1 --2- 172.21.15.2:0/1625016162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d68073050 0x7f5d68073430 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009990 tx=0x7f5d5c009ca0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.108+0000 7f5d6dd9d700 1 -- 172.21.15.2:0/1625016162 shutdown_connections 2023-09-13T04:43:11.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.108+0000 7f5d6dd9d700 1 --2- 172.21.15.2:0/1625016162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d68111ab0 0x7f5d68113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.108+0000 7f5d6dd9d700 1 --2- 172.21.15.2:0/1625016162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d68073050 0x7f5d68073430 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.108+0000 7f5d6dd9d700 1 --2- 172.21.15.2:0/1625016162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d68073970 0x7f5d68111570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.108+0000 7f5d6dd9d700 1 -- 172.21.15.2:0/1625016162 >> 172.21.15.2:0/1625016162 conn(0x7f5d680fc550 msgr2=0x7f5d680fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:11.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.108+0000 7f5d6dd9d700 1 -- 172.21.15.2:0/1625016162 shutdown_connections 2023-09-13T04:43:11.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.108+0000 7f5d6dd9d700 1 -- 172.21.15.2:0/1625016162 wait complete. 2023-09-13T04:43:11.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.109+0000 7f5d6dd9d700 1 Processor -- start 2023-09-13T04:43:11.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.109+0000 7f5d6dd9d700 1 -- start start 2023-09-13T04:43:11.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.110+0000 7f5d6dd9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d68073050 0x7f5d6819d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:11.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.110+0000 7f5d6dd9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d68073970 0x7f5d6819d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:11.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.110+0000 7f5d6dd9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d68111ab0 0x7f5d681a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:11.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.110+0000 7f5d6dd9d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d681a21e0 con 0x7f5d68073050 2023-09-13T04:43:11.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.110+0000 7f5d6dd9d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d681a2350 con 0x7f5d68073970 2023-09-13T04:43:11.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.110+0000 7f5d6dd9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d681a2510 con 0x7f5d68111ab0 2023-09-13T04:43:11.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.110+0000 7f5d67fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d68073970 0x7f5d6819d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:11.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.110+0000 7f5d6cd9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d68073050 0x7f5d6819d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:11.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.110+0000 7f5d67fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d68073970 0x7f5d6819d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:45584/0 (socket says 172.21.15.2:45584) 2023-09-13T04:43:11.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.110+0000 7f5d6cd9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d68073050 0x7f5d6819d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46362/0 (socket says 172.21.15.2:46362) 2023-09-13T04:43:11.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.110+0000 7f5d67fff700 1 -- 172.21.15.2:0/1798038941 learned_addr learned my addr 172.21.15.2:0/1798038941 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:11.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.110+0000 7f5d6d59c700 1 --2- 172.21.15.2:0/1798038941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d68111ab0 0x7f5d681a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:11.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.111+0000 7f5d67fff700 1 -- 172.21.15.2:0/1798038941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d68111ab0 msgr2=0x7f5d681a1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:11.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.111+0000 7f5d67fff700 1 --2- 172.21.15.2:0/1798038941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d68111ab0 0x7f5d681a1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.111+0000 7f5d67fff700 1 -- 172.21.15.2:0/1798038941 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d68073050 msgr2=0x7f5d6819d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:11.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.111+0000 7f5d67fff700 1 --2- 172.21.15.2:0/1798038941 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d68073050 0x7f5d6819d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.111+0000 7f5d67fff700 1 -- 172.21.15.2:0/1798038941 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5c0095f0 con 0x7f5d68073970 2023-09-13T04:43:11.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.111+0000 7f5d67fff700 1 --2- 172.21.15.2:0/1798038941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d68073970 0x7f5d6819d740 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400d950 tx=0x7f5d5400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:11.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.111+0000 7f5d65ffb700 1 -- 172.21.15.2:0/1798038941 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5400f890 con 0x7f5d68073970 2023-09-13T04:43:11.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.112+0000 7f5d65ffb700 1 -- 172.21.15.2:0/1798038941 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d54010460 con 0x7f5d68073970 2023-09-13T04:43:11.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.112+0000 7f5d65ffb700 1 -- 172.21.15.2:0/1798038941 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5400e610 con 0x7f5d68073970 2023-09-13T04:43:11.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.112+0000 7f5d6dd9d700 1 -- 172.21.15.2:0/1798038941 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d681a27f0 con 0x7f5d68073970 2023-09-13T04:43:11.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.112+0000 7f5d6dd9d700 1 -- 172.21.15.2:0/1798038941 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d68104f20 con 0x7f5d68073970 2023-09-13T04:43:11.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.117+0000 7f5d828bbb80 1 Processor -- start 2023-09-13T04:43:11.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.119+0000 7f5d828bbb80 1 -- start start 2023-09-13T04:43:11.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.119+0000 7f5d828bbb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56172202d6b0 0x56172202da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:11.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.119+0000 7f5d828bbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56172202dfd0 0x561722031dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:11.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.119+0000 7f5d828bbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561722032300 0x561722032760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:11.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.120+0000 7f5d828bbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561721f86780 con 0x56172202dfd0 2023-09-13T04:43:11.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.120+0000 7f5d828bbb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561721f840a0 con 0x56172202d6b0 2023-09-13T04:43:11.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.120+0000 7f5d828bbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561721f7f990 con 0x561722032300 2023-09-13T04:43:11.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.120+0000 7f5d6cd9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56172202dfd0 0x561722031dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:11.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.120+0000 7f5d6cd9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56172202dfd0 0x561722031dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46364/0 (socket says 172.21.15.2:46364) 2023-09-13T04:43:11.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.120+0000 7f5d6cd9b700 1 -- 172.21.15.2:0/596267320 learned_addr learned my addr 172.21.15.2:0/596267320 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:11.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.121+0000 7f5d65ffb700 1 -- 172.21.15.2:0/1798038941 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f5d5400f9f0 con 0x7f5d68073970 2023-09-13T04:43:11.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.121+0000 7f5d6cd9b700 1 -- 172.21.15.2:0/596267320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561722032300 msgr2=0x561722032760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:43:11.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.121+0000 7f5d6cd9b700 1 --2- 172.21.15.2:0/596267320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561722032300 0x561722032760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.121+0000 7f5d6cd9b700 1 -- 172.21.15.2:0/596267320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56172202d6b0 msgr2=0x56172202da90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:43:11.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.121+0000 7f5d6cd9b700 1 --2- 172.21.15.2:0/596267320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56172202d6b0 0x56172202da90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.121+0000 7f5d6cd9b700 1 -- 172.21.15.2:0/596267320 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561721dff630 con 0x56172202dfd0 2023-09-13T04:43:11.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.122+0000 7f5d6cd9b700 1 --2- 172.21.15.2:0/596267320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56172202dfd0 0x561722031dc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009fb0 tx=0x7f5d5c005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2febb621f856c048 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:11.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.123+0000 7f5d52ffd700 1 -- 172.21.15.2:0/596267320 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c01d030 con 0x56172202dfd0 2023-09-13T04:43:11.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.123+0000 7f5d65ffb700 1 --2- 172.21.15.2:0/1798038941 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f5d580a08d0 0x7f5d580a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:11.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.123+0000 7f5d52ffd700 1 -- 172.21.15.2:0/596267320 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d5c01e040 con 0x56172202dfd0 2023-09-13T04:43:11.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.123+0000 7f5d52ffd700 1 -- 172.21.15.2:0/596267320 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c005600 con 0x56172202dfd0 2023-09-13T04:43:11.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.124+0000 7f5d65ffb700 1 -- 172.21.15.2:0/1798038941 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f5d540cbf30 con 0x7f5d68073970 2023-09-13T04:43:11.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.124+0000 7f5d828bbb80 1 -- 172.21.15.2:0/596267320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56172202dfd0 msgr2=0x561722031dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:11.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.124+0000 7f5d828bbb80 1 --2- 172.21.15.2:0/596267320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56172202dfd0 0x561722031dc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009fb0 tx=0x7f5d5c005bc0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.125+0000 7f5d6d59c700 1 --2- 172.21.15.2:0/1798038941 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f5d580a08d0 0x7f5d580a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:11.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.125+0000 7f5d828bbb80 1 -- 172.21.15.2:0/596267320 shutdown_connections 2023-09-13T04:43:11.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.125+0000 7f5d828bbb80 1 --2- 172.21.15.2:0/596267320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561722032300 0x561722032760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.125+0000 7f5d828bbb80 1 --2- 172.21.15.2:0/596267320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56172202d6b0 0x56172202da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.125+0000 7f5d828bbb80 1 --2- 172.21.15.2:0/596267320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56172202dfd0 0x561722031dc0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.125+0000 7f5d828bbb80 1 -- 172.21.15.2:0/596267320 >> 172.21.15.2:0/596267320 conn(0x561721f8bc80 msgr2=0x561722028300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:11.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.128+0000 7f5d828bbb80 1 -- 172.21.15.2:0/596267320 shutdown_connections 2023-09-13T04:43:11.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.128+0000 7f5d6d59c700 1 --2- 172.21.15.2:0/1798038941 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f5d580a08d0 0x7f5d580a2d90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5d6806a340 tx=0x7f5d6000d040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:11.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.128+0000 7f5d828bbb80 1 -- 172.21.15.2:0/596267320 wait complete. 2023-09-13T04:43:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.129+0000 7f5d828bbb80 1 -- start start 2023-09-13T04:43:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.129+0000 7f5d828bbb80 1 Processor -- start 2023-09-13T04:43:11.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.130+0000 7f5d828bbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56172202d6b0 0x561722039b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:11.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.130+0000 7f5d828bbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56172202dfd0 0x561722036de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:11.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.130+0000 7f5d828bbb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561722032300 0x561722037520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:11.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.130+0000 7f5d828bbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561721f1dbb0 con 0x56172202dfd0 2023-09-13T04:43:11.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.130+0000 7f5d828bbb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561721f1d8b0 con 0x561722032300 2023-09-13T04:43:11.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.130+0000 7f5d828bbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561721e01920 con 0x56172202d6b0 2023-09-13T04:43:11.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.130+0000 7f5d828bbb80 10 client.0 _ll_register_callbacks cb 0x561721f1b680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:43:11.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.132+0000 7f5d6d59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561722032300 0x561722037520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:11.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.132+0000 7f5d6d59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561722032300 0x561722037520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:45604/0 (socket says 172.21.15.2:45604) 2023-09-13T04:43:11.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.132+0000 7f5d6d59c700 1 -- 172.21.15.2:0/1952786447 learned_addr learned my addr 172.21.15.2:0/1952786447 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:11.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.132+0000 7f5d6d59c700 1 -- 172.21.15.2:0/1952786447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56172202d6b0 msgr2=0x561722039b60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:43:11.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.132+0000 7f5d6d59c700 1 --2- 172.21.15.2:0/1952786447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56172202d6b0 0x561722039b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.132+0000 7f5d6d59c700 1 -- 172.21.15.2:0/1952786447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56172202dfd0 msgr2=0x561722036de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:11.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.132+0000 7f5d6d59c700 1 --2- 172.21.15.2:0/1952786447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56172202dfd0 0x561722036de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.132+0000 7f5d6d59c700 1 -- 172.21.15.2:0/1952786447 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d68114850 con 0x561722032300 2023-09-13T04:43:11.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.133+0000 7f5d6d59c700 1 --2- 172.21.15.2:0/1952786447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561722032300 0x561722037520 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009880 tx=0x7f5d60004be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4891edaba707737 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:11.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.136+0000 7f5d47fff700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:43:11.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.136+0000 7f5d47fff700 1 -- 172.21.15.2:0/1952786447 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60010040 con 0x561722032300 2023-09-13T04:43:11.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.136+0000 7f5d47fff700 1 -- 172.21.15.2:0/1952786447 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f5d60006e40 con 0x561722032300 2023-09-13T04:43:11.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.136+0000 7f5d828bbb80 1 -- 172.21.15.2:0/1952786447 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561722020670 con 0x561722032300 2023-09-13T04:43:11.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.136+0000 7f5d47fff700 1 -- 172.21.15.2:0/1952786447 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d60016e90 con 0x561722032300 2023-09-13T04:43:11.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.136+0000 7f5d828bbb80 10 client.26192 fetch_fsmap learned FSMap version 29 2023-09-13T04:43:11.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.136+0000 7f5d828bbb80 1 -- 172.21.15.2:0/1952786447 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561721f977a0 con 0x561722032300 2023-09-13T04:43:11.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.137+0000 7f5d47fff700 1 -- 172.21.15.2:0/1952786447 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561722020670 con 0x561722032300 2023-09-13T04:43:11.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.137+0000 7f5d47fff700 1 -- 172.21.15.2:0/1952786447 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f5d6001f070 con 0x561722032300 2023-09-13T04:43:11.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.137+0000 7f5d828bbb80 10 client.26192 fetch_fsmap finished waiting for FSMap version 29 2023-09-13T04:43:11.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.137+0000 7f5d828bbb80 10 client.26192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:43:11.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.137+0000 7f5d828bbb80 20 client.26192 populate_metadata read hostname 'smithi002' 2023-09-13T04:43:11.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.137+0000 7f5d828bbb80 1 --2- 172.21.15.2:0/1952786447 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x561722058570 0x56172205a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:11.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.137+0000 7f5d828bbb80 4 client.26192 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:43:11.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.137+0000 7f5d828bbb80 1 -- 172.21.15.2:0/1952786447 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561721f78970 con 0x561722058570 2023-09-13T04:43:11.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.137+0000 7f5d6cd9b700 1 --2- 172.21.15.2:0/1952786447 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x561722058570 0x56172205a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:11.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.138+0000 7f5d6cd9b700 1 --2- 172.21.15.2:0/1952786447 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x561722058570 0x56172205a9b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10d938c086d53c14 server_cookie=1394ff17e79b5aad in_seq=0 out_seq=0 2023-09-13T04:43:11.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.138+0000 7f5d47fff700 10 client.26192 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:43:11.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.139+0000 7f5d47fff700 1 -- 172.21.15.2:0/1952786447 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561721f78970 con 0x561722058570 2023-09-13T04:43:11.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.139+0000 7f5d47fff700 10 client.26192 handle_command_reply: tid=0 2023-09-13T04:43:11.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.139+0000 7f5d828bbb80 1 client.26192 shutdown 2023-09-13T04:43:11.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.139+0000 7f5d828bbb80 1 -- 172.21.15.2:0/1952786447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561722032300 msgr2=0x561722037520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:11.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.139+0000 7f5d828bbb80 1 --2- 172.21.15.2:0/1952786447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561722032300 0x561722037520 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009880 tx=0x7f5d60004be0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.139+0000 7f5d828bbb80 1 -- 172.21.15.2:0/1952786447 shutdown_connections 2023-09-13T04:43:11.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.139+0000 7f5d828bbb80 1 --2- 172.21.15.2:0/1952786447 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x561722058570 0x56172205a9b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.139+0000 7f5d828bbb80 1 --2- 172.21.15.2:0/1952786447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561722032300 0x561722037520 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.139+0000 7f5d828bbb80 1 --2- 172.21.15.2:0/1952786447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56172202d6b0 0x561722039b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.139+0000 7f5d828bbb80 1 --2- 172.21.15.2:0/1952786447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56172202dfd0 0x561722036de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.139+0000 7f5d828bbb80 1 -- 172.21.15.2:0/1952786447 >> 172.21.15.2:0/1952786447 conn(0x561721f8bc80 msgr2=0x56172202c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:11.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.140+0000 7f5d47fff700 0 client.26192 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:43:11.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.140+0000 7f5d828bbb80 1 -- 172.21.15.2:0/1952786447 shutdown_connections 2023-09-13T04:43:11.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.140+0000 7f5d828bbb80 1 -- 172.21.15.2:0/1952786447 wait complete. 2023-09-13T04:43:11.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.140+0000 7f5d828bbb80 20 client.26192 trim_cache size 0 max 16384 2023-09-13T04:43:11.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.168+0000 7f5d828bbb80 1 Processor -- start 2023-09-13T04:43:11.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.173+0000 7f5d828bbb80 1 -- start start 2023-09-13T04:43:11.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.173+0000 7f5d828bbb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56172202d6b0 0x56172202c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:11.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.173+0000 7f5d828bbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56172202dfd0 0x561722039700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:11.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.173+0000 7f5d828bbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561722032300 0x561722027e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:11.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.173+0000 7f5d828bbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561722029fa0 con 0x56172202dfd0 2023-09-13T04:43:11.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.173+0000 7f5d828bbb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172203a180 con 0x56172202d6b0 2023-09-13T04:43:11.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.173+0000 7f5d828bbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617220293d0 con 0x561722032300 2023-09-13T04:43:11.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.174+0000 7f5d6d59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561722032300 0x561722027e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:11.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.174+0000 7f5d6d59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561722032300 0x561722027e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:48462/0 (socket says 172.21.15.2:48462) 2023-09-13T04:43:11.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.174+0000 7f5d6d59c700 1 -- 172.21.15.2:0/1948789776 learned_addr learned my addr 172.21.15.2:0/1948789776 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:11.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.174+0000 7f5d6cd9b700 1 --2- 172.21.15.2:0/1948789776 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56172202d6b0 0x56172202c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:11.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.174+0000 7f5d6d59c700 1 -- 172.21.15.2:0/1948789776 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56172202d6b0 msgr2=0x56172202c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:11.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.174+0000 7f5d6d59c700 1 --2- 172.21.15.2:0/1948789776 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56172202d6b0 0x56172202c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.174+0000 7f5d6d59c700 1 -- 172.21.15.2:0/1948789776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56172202dfd0 msgr2=0x561722039700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:11.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.174+0000 7f5d6d59c700 1 --2- 172.21.15.2:0/1948789776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56172202dfd0 0x561722039700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.174+0000 7f5d6d59c700 1 -- 172.21.15.2:0/1948789776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561721dabde0 con 0x561722032300 2023-09-13T04:43:11.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.174+0000 7f5d6cd9b700 1 --2- 172.21.15.2:0/1948789776 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56172202d6b0 0x56172202c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:43:11.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.175+0000 7f5d6d59c700 1 --2- 172.21.15.2:0/1948789776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561722032300 0x561722027e60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009850 tx=0x7f5d6000abd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba792279e3ac386 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:11.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.175+0000 7f5d537fe700 1 -- 172.21.15.2:0/1948789776 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60010040 con 0x561722032300 2023-09-13T04:43:11.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.175+0000 7f5d537fe700 1 -- 172.21.15.2:0/1948789776 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d60016e90 con 0x561722032300 2023-09-13T04:43:11.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.175+0000 7f5d537fe700 1 -- 172.21.15.2:0/1948789776 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60020650 con 0x561722032300 2023-09-13T04:43:11.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.177+0000 7f5d828bbb80 1 -- 172.21.15.2:0/1948789776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561722032300 msgr2=0x561722027e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:11.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.177+0000 7f5d828bbb80 1 --2- 172.21.15.2:0/1948789776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561722032300 0x561722027e60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009850 tx=0x7f5d6000abd0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.177+0000 7f5d828bbb80 1 -- 172.21.15.2:0/1948789776 shutdown_connections 2023-09-13T04:43:11.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.177+0000 7f5d828bbb80 1 --2- 172.21.15.2:0/1948789776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561722032300 0x561722027e60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.177+0000 7f5d828bbb80 1 --2- 172.21.15.2:0/1948789776 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56172202d6b0 0x56172202c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.177+0000 7f5d828bbb80 1 --2- 172.21.15.2:0/1948789776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56172202dfd0 0x561722039700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.177+0000 7f5d828bbb80 1 -- 172.21.15.2:0/1948789776 >> 172.21.15.2:0/1948789776 conn(0x561721f8bc80 msgr2=0x561722054770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:11.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.177+0000 7f5d828bbb80 1 -- 172.21.15.2:0/1948789776 shutdown_connections 2023-09-13T04:43:11.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.177+0000 7f5d828bbb80 1 -- 172.21.15.2:0/1948789776 wait complete. 2023-09-13T04:43:11.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.178+0000 7f5d828bbb80 1 -- start start 2023-09-13T04:43:11.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.178+0000 7f5d828bbb80 1 Processor -- start 2023-09-13T04:43:11.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.179+0000 7f5d828bbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56172202d6b0 0x56172203e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:11.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.179+0000 7f5d828bbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56172202dfd0 0x5617220356f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:11.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.179+0000 7f5d828bbb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561722032300 0x561722035c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:11.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.179+0000 7f5d828bbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561722036380 con 0x56172202d6b0 2023-09-13T04:43:11.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.179+0000 7f5d828bbb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617220364f0 con 0x561722032300 2023-09-13T04:43:11.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.179+0000 7f5d828bbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617220ac400 con 0x56172202dfd0 2023-09-13T04:43:11.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.180+0000 7f5d828bbb80 10 client.0 _ll_register_callbacks cb 0x561721f1b680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:43:11.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.180+0000 7f5d6d59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561722032300 0x561722035c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:11.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.180+0000 7f5d6d59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561722032300 0x561722035c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:45630/0 (socket says 172.21.15.2:45630) 2023-09-13T04:43:11.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.180+0000 7f5d6d59c700 1 -- 172.21.15.2:0/2830492254 learned_addr learned my addr 172.21.15.2:0/2830492254 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:11.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.180+0000 7f5d67fff700 1 --2- 172.21.15.2:0/2830492254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56172202dfd0 0x5617220356f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:11.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.181+0000 7f5d67fff700 1 -- 172.21.15.2:0/2830492254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561722032300 msgr2=0x561722035c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:11.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.181+0000 7f5d67fff700 1 --2- 172.21.15.2:0/2830492254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561722032300 0x561722035c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.181+0000 7f5d67fff700 1 -- 172.21.15.2:0/2830492254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56172202d6b0 msgr2=0x56172203e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:11.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.181+0000 7f5d67fff700 1 --2- 172.21.15.2:0/2830492254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56172202d6b0 0x56172203e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.181+0000 7f5d67fff700 1 -- 172.21.15.2:0/2830492254 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561721f977a0 con 0x56172202dfd0 2023-09-13T04:43:11.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.181+0000 7f5d67fff700 1 --2- 172.21.15.2:0/2830492254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56172202dfd0 0x5617220356f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5d540c9310 tx=0x7f5d54099770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fe7d26ab9b1b170 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:11.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.181+0000 7f5d47fff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:43:11.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.181+0000 7f5d47fff700 1 -- 172.21.15.2:0/2830492254 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d540994d0 con 0x56172202dfd0 2023-09-13T04:43:11.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.182+0000 7f5d47fff700 1 -- 172.21.15.2:0/2830492254 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f5d5400feb0 con 0x56172202dfd0 2023-09-13T04:43:11.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.182+0000 7f5d828bbb80 1 -- 172.21.15.2:0/2830492254 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d5c01e040 con 0x56172202dfd0 2023-09-13T04:43:11.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.182+0000 7f5d47fff700 1 -- 172.21.15.2:0/2830492254 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d5409cea0 con 0x56172202dfd0 2023-09-13T04:43:11.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.182+0000 7f5d828bbb80 10 client.26194 fetch_fsmap learned FSMap version 29 2023-09-13T04:43:11.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.182+0000 7f5d828bbb80 1 -- 172.21.15.2:0/2830492254 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561721d94c60 con 0x56172202dfd0 2023-09-13T04:43:11.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.183+0000 7f5d47fff700 1 -- 172.21.15.2:0/2830492254 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d5c01e040 con 0x56172202dfd0 2023-09-13T04:43:11.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.183+0000 7f5d47fff700 1 -- 172.21.15.2:0/2830492254 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f5d540b10c0 con 0x56172202dfd0 2023-09-13T04:43:11.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.183+0000 7f5d828bbb80 10 client.26194 fetch_fsmap finished waiting for FSMap version 29 2023-09-13T04:43:11.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.183+0000 7f5d828bbb80 10 client.26194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:43:11.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.183+0000 7f5d828bbb80 20 client.26194 populate_metadata read hostname 'smithi002' 2023-09-13T04:43:11.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.183+0000 7f5d828bbb80 1 --2- 172.21.15.2:0/2830492254 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5617220adab0 0x5617220afed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:11.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.183+0000 7f5d828bbb80 4 client.26194 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-09-13T04:43:11.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.183+0000 7f5d828bbb80 1 -- 172.21.15.2:0/2830492254 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561721f39050 con 0x5617220adab0 2023-09-13T04:43:11.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.184+0000 7f5d6cd9b700 1 --2- 172.21.15.2:0/2830492254 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5617220adab0 0x5617220afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:11.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.185+0000 7f5d6cd9b700 1 --2- 172.21.15.2:0/2830492254 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5617220adab0 0x5617220afed0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78f4df75c523be6d server_cookie=9423508de470568b in_seq=0 out_seq=0 2023-09-13T04:43:11.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.185+0000 7f5d47fff700 10 client.26194 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:43:11.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.185+0000 7f5d47fff700 1 -- 172.21.15.2:0/2830492254 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561721f39050 con 0x5617220adab0 2023-09-13T04:43:11.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.185+0000 7f5d47fff700 10 client.26194 handle_command_reply: tid=0 2023-09-13T04:43:11.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.185+0000 7f5d828bbb80 1 client.26194 shutdown 2023-09-13T04:43:11.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.185+0000 7f5d828bbb80 1 -- 172.21.15.2:0/2830492254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56172202dfd0 msgr2=0x5617220356f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:11.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.185+0000 7f5d828bbb80 1 --2- 172.21.15.2:0/2830492254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56172202dfd0 0x5617220356f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5d540c9310 tx=0x7f5d54099770 comp rx=0 tx=0).stop 2023-09-13T04:43:11.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.186+0000 7f5d828bbb80 1 -- 172.21.15.2:0/2830492254 shutdown_connections 2023-09-13T04:43:11.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.186+0000 7f5d828bbb80 1 --2- 172.21.15.2:0/2830492254 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5617220adab0 0x5617220afed0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.186+0000 7f5d828bbb80 1 --2- 172.21.15.2:0/2830492254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561722032300 0x561722035c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.186+0000 7f5d828bbb80 1 --2- 172.21.15.2:0/2830492254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56172202d6b0 0x56172203e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.186+0000 7f5d828bbb80 1 --2- 172.21.15.2:0/2830492254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56172202dfd0 0x5617220356f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.186+0000 7f5d828bbb80 1 -- 172.21.15.2:0/2830492254 >> 172.21.15.2:0/2830492254 conn(0x561721f8bc80 msgr2=0x56172202c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:11.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.186+0000 7f5d47fff700 0 client.26194 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:43:11.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.187+0000 7f5d828bbb80 1 -- 172.21.15.2:0/2830492254 shutdown_connections 2023-09-13T04:43:11.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.187+0000 7f5d828bbb80 1 -- 172.21.15.2:0/2830492254 wait complete. 2023-09-13T04:43:11.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.187+0000 7f5d828bbb80 20 client.26194 trim_cache size 0 max 16384 2023-09-13T04:43:11.194 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:43:11.195 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-13T04:43:11.195 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-09-13T04:43:11.195 INFO:teuthology.orchestra.run.smithi002.stdout: "740235c0-6e1b-4729-b1fe-33345ae948c1": { 2023-09-13T04:43:11.195 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-09-13T04:43:11.195 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "740235c0-6e1b-4729-b1fe-33345ae948c1", 2023-09-13T04:43:11.196 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-09-13T04:43:11.196 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:43:11.196 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:43:11.196 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:43:11.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.188+0000 7f5d6dd9d700 1 -- 172.21.15.2:0/1798038941 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f5d580a08d0 msgr2=0x7f5d580a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:11.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.188+0000 7f5d6dd9d700 1 --2- 172.21.15.2:0/1798038941 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f5d580a08d0 0x7f5d580a2d90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5d6806a340 tx=0x7f5d6000d040 comp rx=0 tx=0).stop 2023-09-13T04:43:11.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.188+0000 7f5d6dd9d700 1 -- 172.21.15.2:0/1798038941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d68073970 msgr2=0x7f5d6819d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:11.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.188+0000 7f5d6dd9d700 1 --2- 172.21.15.2:0/1798038941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d68073970 0x7f5d6819d740 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400d950 tx=0x7f5d5400dc60 comp rx=0 tx=0).stop 2023-09-13T04:43:11.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.189+0000 7f5d6dd9d700 1 -- 172.21.15.2:0/1798038941 shutdown_connections 2023-09-13T04:43:11.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.189+0000 7f5d6dd9d700 1 --2- 172.21.15.2:0/1798038941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d68111ab0 0x7f5d681a1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.189+0000 7f5d6dd9d700 1 --2- 172.21.15.2:0/1798038941 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f5d580a08d0 0x7f5d580a2d90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.189+0000 7f5d6dd9d700 1 --2- 172.21.15.2:0/1798038941 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d68073050 0x7f5d6819d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.189+0000 7f5d6dd9d700 1 --2- 172.21.15.2:0/1798038941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d68073970 0x7f5d6819d740 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:11.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.189+0000 7f5d6dd9d700 1 -- 172.21.15.2:0/1798038941 >> 172.21.15.2:0/1798038941 conn(0x7f5d680fc550 msgr2=0x7f5d680fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:11.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.189+0000 7f5d6dd9d700 1 -- 172.21.15.2:0/1798038941 shutdown_connections 2023-09-13T04:43:11.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:11.189+0000 7f5d6dd9d700 1 -- 172.21.15.2:0/1798038941 wait complete. 2023-09-13T04:43:11.213 INFO:tasks.cephfs.filesystem:scrub status for tag:740235c0-6e1b-4729-b1fe-33345ae948c1 - {'path': '/', 'tag': '740235c0-6e1b-4729-b1fe-33345ae948c1', 'options': 'recursive,force'} 2023-09-13T04:43:13.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:12 smithi002 ceph-mon[97651]: pgmap v86: 129 pgs: 129 active+clean; 268 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 364 op/s 2023-09-13T04:43:13.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:12 smithi149 ceph-mon[133315]: pgmap v86: 129 pgs: 129 active+clean; 268 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 364 op/s 2023-09-13T04:43:13.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:12 smithi156 ceph-mon[133675]: pgmap v86: 129 pgs: 129 active+clean; 268 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 364 op/s 2023-09-13T04:43:15.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:14 smithi002 ceph-mon[97651]: pgmap v87: 129 pgs: 129 active+clean; 272 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 388 op/s 2023-09-13T04:43:15.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:14 smithi149 ceph-mon[133315]: pgmap v87: 129 pgs: 129 active+clean; 272 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 388 op/s 2023-09-13T04:43:15.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:14 smithi156 ceph-mon[133675]: pgmap v87: 129 pgs: 129 active+clean; 272 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 388 op/s 2023-09-13T04:43:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.312+0000 7f77403ff700 1 -- 172.21.15.2:0/905537883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7738073080 msgr2=0x7f7738073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.312+0000 7f77403ff700 1 --2- 172.21.15.2:0/905537883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7738073080 0x7f7738073460 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f773400b0a0 tx=0x7f773400b3b0 comp rx=0 tx=0).stop 2023-09-13T04:43:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.312+0000 7f77403ff700 1 -- 172.21.15.2:0/905537883 shutdown_connections 2023-09-13T04:43:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.312+0000 7f77403ff700 1 --2- 172.21.15.2:0/905537883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f773810d4f0 0x7f773810f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.312+0000 7f77403ff700 1 --2- 172.21.15.2:0/905537883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7738073080 0x7f7738073460 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.312+0000 7f77403ff700 1 --2- 172.21.15.2:0/905537883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77380739a0 0x7f773810cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.312+0000 7f77403ff700 1 -- 172.21.15.2:0/905537883 >> 172.21.15.2:0/905537883 conn(0x7f77380fc740 msgr2=0x7f77380feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.313+0000 7f77403ff700 1 -- 172.21.15.2:0/905537883 shutdown_connections 2023-09-13T04:43:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.313+0000 7f77403ff700 1 -- 172.21.15.2:0/905537883 wait complete. 2023-09-13T04:43:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.314+0000 7f77403ff700 1 Processor -- start 2023-09-13T04:43:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.315+0000 7f77403ff700 1 -- start start 2023-09-13T04:43:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.315+0000 7f77403ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7738073080 0x7f7738106530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.315+0000 7f77403ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77380739a0 0x7f7738106a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.315+0000 7f77403ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f773810d4f0 0x7f77381013e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.315+0000 7f77403ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7738101bb0 con 0x7f7738073080 2023-09-13T04:43:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.315+0000 7f77403ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7738101d20 con 0x7f773810d4f0 2023-09-13T04:43:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.315+0000 7f77403ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7738101ee0 con 0x7f77380739a0 2023-09-13T04:43:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.316+0000 7f773d99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77380739a0 0x7f7738106a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.316+0000 7f773d99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77380739a0 0x7f7738106a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:34430/0 (socket says 172.21.15.2:34430) 2023-09-13T04:43:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.316+0000 7f773d99a700 1 -- 172.21.15.2:0/2765894686 learned_addr learned my addr 172.21.15.2:0/2765894686 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.316+0000 7f773e99c700 1 --2- 172.21.15.2:0/2765894686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f773810d4f0 0x7f77381013e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.316+0000 7f773e19b700 1 --2- 172.21.15.2:0/2765894686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7738073080 0x7f7738106530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.316+0000 7f773d99a700 1 -- 172.21.15.2:0/2765894686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f773810d4f0 msgr2=0x7f77381013e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.316+0000 7f773d99a700 1 --2- 172.21.15.2:0/2765894686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f773810d4f0 0x7f77381013e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.316+0000 7f773d99a700 1 -- 172.21.15.2:0/2765894686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7738073080 msgr2=0x7f7738106530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.317+0000 7f773d99a700 1 --2- 172.21.15.2:0/2765894686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7738073080 0x7f7738106530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.317+0000 7f773d99a700 1 -- 172.21.15.2:0/2765894686 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7734009d00 con 0x7f77380739a0 2023-09-13T04:43:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.317+0000 7f773d99a700 1 --2- 172.21.15.2:0/2765894686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77380739a0 0x7f7738106a90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f772800d950 tx=0x7f772800dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.317+0000 7f772f7fe700 1 -- 172.21.15.2:0/2765894686 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7728009b40 con 0x7f77380739a0 2023-09-13T04:43:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.317+0000 7f772f7fe700 1 -- 172.21.15.2:0/2765894686 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7728009ca0 con 0x7f77380739a0 2023-09-13T04:43:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.317+0000 7f772f7fe700 1 -- 172.21.15.2:0/2765894686 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772800f620 con 0x7f77380739a0 2023-09-13T04:43:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.318+0000 7f77403ff700 1 -- 172.21.15.2:0/2765894686 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77381021c0 con 0x7f77380739a0 2023-09-13T04:43:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.318+0000 7f77403ff700 1 -- 172.21.15.2:0/2765894686 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f773810c580 con 0x7f77380739a0 2023-09-13T04:43:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.324+0000 7f7753de5b80 1 Processor -- start 2023-09-13T04:43:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.325+0000 7f7753de5b80 1 -- start start 2023-09-13T04:43:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.325+0000 7f7753de5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0fba65be0 0x55b0fba65fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.325+0000 7f7753de5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba66500 0x55b0fba61300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.325+0000 7f7753de5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0fba61840 0x55b0fba61c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.325+0000 7f7753de5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0fb9ba720 con 0x55b0fba61840 2023-09-13T04:43:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.325+0000 7f7753de5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0fb9b8040 con 0x55b0fba66500 2023-09-13T04:43:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.325+0000 7f7753de5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0fb9b3930 con 0x55b0fba65be0 2023-09-13T04:43:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.326+0000 7f773e99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0fba65be0 0x55b0fba65fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.327+0000 7f773e99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0fba65be0 0x55b0fba65fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:34438/0 (socket says 172.21.15.2:34438) 2023-09-13T04:43:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.327+0000 7f773e99c700 1 -- 172.21.15.2:0/2488999260 learned_addr learned my addr 172.21.15.2:0/2488999260 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.327+0000 7f773e19b700 1 --2- 172.21.15.2:0/2488999260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba66500 0x55b0fba61300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.327+0000 7f773e19b700 1 -- 172.21.15.2:0/2488999260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0fba65be0 msgr2=0x55b0fba65fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.327+0000 7f773e19b700 1 --2- 172.21.15.2:0/2488999260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0fba65be0 0x55b0fba65fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.327+0000 7f773e19b700 1 -- 172.21.15.2:0/2488999260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0fba61840 msgr2=0x55b0fba61c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.327+0000 7f773e19b700 1 --2- 172.21.15.2:0/2488999260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0fba61840 0x55b0fba61c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.327+0000 7f773e19b700 1 -- 172.21.15.2:0/2488999260 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0fb8335d0 con 0x55b0fba66500 2023-09-13T04:43:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.328+0000 7f773e19b700 1 --2- 172.21.15.2:0/2488999260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba66500 0x55b0fba61300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7734009fd0 tx=0x7f7734009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a15d540b6725163 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.328+0000 7f772cbfe700 1 -- 172.21.15.2:0/2488999260 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773400e050 con 0x55b0fba66500 2023-09-13T04:43:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.328+0000 7f772cbfe700 1 -- 172.21.15.2:0/2488999260 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77340074e0 con 0x55b0fba66500 2023-09-13T04:43:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.329+0000 7f772cbfe700 1 -- 172.21.15.2:0/2488999260 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7734005630 con 0x55b0fba66500 2023-09-13T04:43:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.330+0000 7f7753de5b80 1 -- 172.21.15.2:0/2488999260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba66500 msgr2=0x55b0fba61300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.330+0000 7f7753de5b80 1 --2- 172.21.15.2:0/2488999260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba66500 0x55b0fba61300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7734009fd0 tx=0x7f7734009250 comp rx=0 tx=0).stop 2023-09-13T04:43:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.330+0000 7f7753de5b80 1 -- 172.21.15.2:0/2488999260 shutdown_connections 2023-09-13T04:43:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.330+0000 7f7753de5b80 1 --2- 172.21.15.2:0/2488999260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0fba61840 0x55b0fba61c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.330+0000 7f7753de5b80 1 --2- 172.21.15.2:0/2488999260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0fba65be0 0x55b0fba65fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.330+0000 7f7753de5b80 1 --2- 172.21.15.2:0/2488999260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba66500 0x55b0fba61300 secure :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7734009fd0 tx=0x7f7734009250 comp rx=0 tx=0).stop 2023-09-13T04:43:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.330+0000 7f7753de5b80 1 -- 172.21.15.2:0/2488999260 >> 172.21.15.2:0/2488999260 conn(0x55b0fb9bfc20 msgr2=0x55b0fba5bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.330+0000 7f772f7fe700 1 -- 172.21.15.2:0/2765894686 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f7728010460 con 0x7f77380739a0 2023-09-13T04:43:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.331+0000 7f7753de5b80 1 -- 172.21.15.2:0/2488999260 shutdown_connections 2023-09-13T04:43:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.331+0000 7f7753de5b80 1 -- 172.21.15.2:0/2488999260 wait complete. 2023-09-13T04:43:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.332+0000 7f7753de5b80 1 -- start start 2023-09-13T04:43:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.332+0000 7f7753de5b80 1 Processor -- start 2023-09-13T04:43:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.333+0000 7f7753de5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba65be0 0x55b0fba6abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.333+0000 7f7753de5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0fba6b0f0 0x55b0fba6b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.333+0000 7f7753de5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0fba86020 0x55b0fba88410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.333+0000 7f7753de5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0fb951b50 con 0x55b0fba86020 2023-09-13T04:43:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.333+0000 7f7753de5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0fb951850 con 0x55b0fba65be0 2023-09-13T04:43:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.333+0000 7f7753de5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0fb8358c0 con 0x55b0fba6b0f0 2023-09-13T04:43:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.333+0000 7f7753de5b80 10 client.0 _ll_register_callbacks cb 0x55b0fb94f620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:43:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.334+0000 7f773e19b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0fba6b0f0 0x55b0fba6b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.334+0000 7f773e19b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0fba6b0f0 0x55b0fba6b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:34446/0 (socket says 172.21.15.2:34446) 2023-09-13T04:43:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.334+0000 7f773e19b700 1 -- 172.21.15.2:0/990890715 learned_addr learned my addr 172.21.15.2:0/990890715 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.334+0000 7f773e99c700 1 --2- 172.21.15.2:0/990890715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba65be0 0x55b0fba6abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.334+0000 7f772f7fe700 1 --2- 172.21.15.2:0/2765894686 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f77240a0780 0x7f77240a2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.334+0000 7f773e19b700 1 -- 172.21.15.2:0/990890715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba65be0 msgr2=0x55b0fba6abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.334+0000 7f773e19b700 1 --2- 172.21.15.2:0/990890715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba65be0 0x55b0fba6abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.334+0000 7f773e19b700 1 -- 172.21.15.2:0/990890715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0fba86020 msgr2=0x55b0fba88410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.334+0000 7f773e19b700 1 --2- 172.21.15.2:0/990890715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0fba86020 0x55b0fba88410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.334+0000 7f773e19b700 1 -- 172.21.15.2:0/990890715 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7738110630 con 0x55b0fba6b0f0 2023-09-13T04:43:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.334+0000 7f772f7fe700 1 -- 172.21.15.2:0/2765894686 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f77280ce280 con 0x7f77380739a0 2023-09-13T04:43:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.335+0000 7f773e99c700 1 --2- 172.21.15.2:0/990890715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba65be0 0x55b0fba6abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:43:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.335+0000 7f773e19b700 1 --2- 172.21.15.2:0/990890715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0fba6b0f0 0x55b0fba6b570 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7734000f80 tx=0x7f77340060f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d802f98876906905 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.335+0000 7f7721ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:43:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.335+0000 7f773e99c700 1 --2- 172.21.15.2:0/2765894686 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f77240a0780 0x7f77240a2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.335+0000 7f7721ffb700 1 -- 172.21.15.2:0/990890715 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773400e040 con 0x55b0fba6b0f0 2023-09-13T04:43:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.335+0000 7f7721ffb700 1 -- 172.21.15.2:0/990890715 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f7734009440 con 0x55b0fba6b0f0 2023-09-13T04:43:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.335+0000 7f7753de5b80 1 -- 172.21.15.2:0/990890715 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0fba26ef0 con 0x55b0fba6b0f0 2023-09-13T04:43:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.336+0000 7f7721ffb700 1 -- 172.21.15.2:0/990890715 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77340090d0 con 0x55b0fba6b0f0 2023-09-13T04:43:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.336+0000 7f7753de5b80 10 client.26203 fetch_fsmap learned FSMap version 29 2023-09-13T04:43:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.336+0000 7f7753de5b80 1 -- 172.21.15.2:0/990890715 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b0fb9cb740 con 0x55b0fba6b0f0 2023-09-13T04:43:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.336+0000 7f773e99c700 1 --2- 172.21.15.2:0/2765894686 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f77240a0780 0x7f77240a2c40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55b0fba5da70 tx=0x7f7730014040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.337+0000 7f7721ffb700 1 -- 172.21.15.2:0/990890715 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7734034030 con 0x55b0fba6b0f0 2023-09-13T04:43:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.337+0000 7f7721ffb700 1 -- 172.21.15.2:0/990890715 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f773400b0d0 con 0x55b0fba6b0f0 2023-09-13T04:43:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.337+0000 7f7753de5b80 10 client.26203 fetch_fsmap finished waiting for FSMap version 29 2023-09-13T04:43:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.337+0000 7f7753de5b80 10 client.26203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:43:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.337+0000 7f7753de5b80 20 client.26203 populate_metadata read hostname 'smithi002' 2023-09-13T04:43:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.337+0000 7f7753de5b80 1 --2- 172.21.15.2:0/990890715 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55b0fba8c640 0x55b0fba8ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.337+0000 7f7753de5b80 4 client.26203 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:43:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.337+0000 7f7753de5b80 1 -- 172.21.15.2:0/990890715 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b0fb9ac910 con 0x55b0fba8c640 2023-09-13T04:43:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.338+0000 7f773d99a700 1 --2- 172.21.15.2:0/990890715 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55b0fba8c640 0x55b0fba8ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.338+0000 7f773d99a700 1 --2- 172.21.15.2:0/990890715 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55b0fba8c640 0x55b0fba8ea60 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d394b35540c400bc server_cookie=d94cb91012e41e02 in_seq=0 out_seq=0 2023-09-13T04:43:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.338+0000 7f7721ffb700 10 client.26203 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:43:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.339+0000 7f7721ffb700 1 -- 172.21.15.2:0/990890715 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b0fb9ac910 con 0x55b0fba8c640 2023-09-13T04:43:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.339+0000 7f7721ffb700 10 client.26203 handle_command_reply: tid=0 2023-09-13T04:43:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.340+0000 7f7753de5b80 1 client.26203 shutdown 2023-09-13T04:43:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.340+0000 7f7753de5b80 1 -- 172.21.15.2:0/990890715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0fba6b0f0 msgr2=0x55b0fba6b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.340+0000 7f7753de5b80 1 --2- 172.21.15.2:0/990890715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0fba6b0f0 0x55b0fba6b570 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7734000f80 tx=0x7f77340060f0 comp rx=0 tx=0).stop 2023-09-13T04:43:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.340+0000 7f7753de5b80 1 -- 172.21.15.2:0/990890715 shutdown_connections 2023-09-13T04:43:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.340+0000 7f7753de5b80 1 --2- 172.21.15.2:0/990890715 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55b0fba8c640 0x55b0fba8ea60 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.340+0000 7f7753de5b80 1 --2- 172.21.15.2:0/990890715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0fba86020 0x55b0fba88410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.340+0000 7f7753de5b80 1 --2- 172.21.15.2:0/990890715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba65be0 0x55b0fba6abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.340+0000 7f7721ffb700 0 client.26203 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:43:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.340+0000 7f7753de5b80 1 --2- 172.21.15.2:0/990890715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0fba6b0f0 0x55b0fba6b570 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.340+0000 7f7753de5b80 1 -- 172.21.15.2:0/990890715 >> 172.21.15.2:0/990890715 conn(0x55b0fb9bfc20 msgr2=0x55b0fba67330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.341+0000 7f7753de5b80 1 -- 172.21.15.2:0/990890715 shutdown_connections 2023-09-13T04:43:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.341+0000 7f7753de5b80 1 -- 172.21.15.2:0/990890715 wait complete. 2023-09-13T04:43:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.341+0000 7f7753de5b80 20 client.26203 trim_cache size 0 max 16384 2023-09-13T04:43:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.359+0000 7f7753de5b80 1 Processor -- start 2023-09-13T04:43:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.359+0000 7f7753de5b80 1 -- start start 2023-09-13T04:43:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.360+0000 7f7753de5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba65be0 0x55b0fba6b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.360+0000 7f7753de5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0fba6b0f0 0x55b0fba5ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.360+0000 7f7753de5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0fba86020 0x55b0fba5fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.360+0000 7f7753de5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0fba6c1d0 con 0x55b0fba6b0f0 2023-09-13T04:43:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.360+0000 7f7753de5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0fba7ece0 con 0x55b0fba65be0 2023-09-13T04:43:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.360+0000 7f7753de5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0fba68b40 con 0x55b0fba86020 2023-09-13T04:43:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.360+0000 7f773e19b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba65be0 0x55b0fba6b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.360+0000 7f773e19b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba65be0 0x55b0fba6b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:48942/0 (socket says 172.21.15.2:48942) 2023-09-13T04:43:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.360+0000 7f773e19b700 1 -- 172.21.15.2:0/1594258876 learned_addr learned my addr 172.21.15.2:0/1594258876 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.360+0000 7f773d99a700 1 --2- 172.21.15.2:0/1594258876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0fba6b0f0 0x55b0fba5ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.360+0000 7f773e99c700 1 --2- 172.21.15.2:0/1594258876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0fba86020 0x55b0fba5fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.360+0000 7f773e19b700 1 -- 172.21.15.2:0/1594258876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0fba86020 msgr2=0x55b0fba5fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.360+0000 7f773e19b700 1 --2- 172.21.15.2:0/1594258876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0fba86020 0x55b0fba5fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.360+0000 7f773e19b700 1 -- 172.21.15.2:0/1594258876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0fba6b0f0 msgr2=0x55b0fba5ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.360+0000 7f773e19b700 1 --2- 172.21.15.2:0/1594258876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0fba6b0f0 0x55b0fba5ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.360+0000 7f773e19b700 1 -- 172.21.15.2:0/1594258876 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0fb7dfd80 con 0x55b0fba65be0 2023-09-13T04:43:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.361+0000 7f773d99a700 1 --2- 172.21.15.2:0/1594258876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0fba6b0f0 0x55b0fba5ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:43:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.361+0000 7f773e19b700 1 --2- 172.21.15.2:0/1594258876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba65be0 0x55b0fba6b4d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7734009cd0 tx=0x7f7734009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d72ad7911ed6c9c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.361+0000 7f772d3ff700 1 -- 172.21.15.2:0/1594258876 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773401bc10 con 0x55b0fba65be0 2023-09-13T04:43:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.361+0000 7f772d3ff700 1 -- 172.21.15.2:0/1594258876 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7734034070 con 0x55b0fba65be0 2023-09-13T04:43:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.361+0000 7f772d3ff700 1 -- 172.21.15.2:0/1594258876 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773401b640 con 0x55b0fba65be0 2023-09-13T04:43:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.362+0000 7f7753de5b80 1 -- 172.21.15.2:0/1594258876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba65be0 msgr2=0x55b0fba6b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.362+0000 7f7753de5b80 1 --2- 172.21.15.2:0/1594258876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba65be0 0x55b0fba6b4d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7734009cd0 tx=0x7f7734009540 comp rx=0 tx=0).stop 2023-09-13T04:43:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.362+0000 7f7753de5b80 1 -- 172.21.15.2:0/1594258876 shutdown_connections 2023-09-13T04:43:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.362+0000 7f7753de5b80 1 --2- 172.21.15.2:0/1594258876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0fba86020 0x55b0fba5fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.362+0000 7f7753de5b80 1 --2- 172.21.15.2:0/1594258876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba65be0 0x55b0fba6b4d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.362+0000 7f7753de5b80 1 --2- 172.21.15.2:0/1594258876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0fba6b0f0 0x55b0fba5ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.362+0000 7f7753de5b80 1 -- 172.21.15.2:0/1594258876 >> 172.21.15.2:0/1594258876 conn(0x55b0fb9bfc20 msgr2=0x55b0fba93380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.362+0000 7f7753de5b80 1 -- 172.21.15.2:0/1594258876 shutdown_connections 2023-09-13T04:43:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.362+0000 7f7753de5b80 1 -- 172.21.15.2:0/1594258876 wait complete. 2023-09-13T04:43:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.363+0000 7f7753de5b80 1 -- start start 2023-09-13T04:43:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.363+0000 7f7753de5b80 1 Processor -- start 2023-09-13T04:43:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.363+0000 7f7753de5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba65be0 0x55b0fba83170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.363+0000 7f7753de5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0fba6b0f0 0x55b0fba80f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.363+0000 7f7753de5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0fba86020 0x55b0fba814a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.363+0000 7f7753de5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0fba81bb0 con 0x55b0fba6b0f0 2023-09-13T04:43:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.363+0000 7f7753de5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0fba81d20 con 0x55b0fba65be0 2023-09-13T04:43:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.363+0000 7f7753de5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0fbadf430 con 0x55b0fba86020 2023-09-13T04:43:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.364+0000 7f7753de5b80 10 client.0 _ll_register_callbacks cb 0x55b0fba62140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:43:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.364+0000 7f773e19b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba65be0 0x55b0fba83170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.364+0000 7f773e19b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba65be0 0x55b0fba83170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:48958/0 (socket says 172.21.15.2:48958) 2023-09-13T04:43:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.364+0000 7f773e19b700 1 -- 172.21.15.2:0/203670561 learned_addr learned my addr 172.21.15.2:0/203670561 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.364+0000 7f773e99c700 1 --2- 172.21.15.2:0/203670561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0fba86020 0x55b0fba814a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.364+0000 7f773e19b700 1 -- 172.21.15.2:0/203670561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0fba86020 msgr2=0x55b0fba814a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.364+0000 7f773e19b700 1 --2- 172.21.15.2:0/203670561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0fba86020 0x55b0fba814a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.364+0000 7f773e19b700 1 -- 172.21.15.2:0/203670561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0fba6b0f0 msgr2=0x55b0fba80f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.364+0000 7f773e19b700 1 --2- 172.21.15.2:0/203670561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0fba6b0f0 0x55b0fba80f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.364+0000 7f773e19b700 1 -- 172.21.15.2:0/203670561 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b0fb9cb740 con 0x55b0fba65be0 2023-09-13T04:43:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.365+0000 7f773e19b700 1 --2- 172.21.15.2:0/203670561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba65be0 0x55b0fba83170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f773400d010 tx=0x7f773401b4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7373756374a0ba1 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.365+0000 7f7721ffb700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:43:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.365+0000 7f7721ffb700 1 -- 172.21.15.2:0/203670561 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773400e030 con 0x55b0fba65be0 2023-09-13T04:43:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.365+0000 7f7753de5b80 1 -- 172.21.15.2:0/203670561 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77340074e0 con 0x55b0fba65be0 2023-09-13T04:43:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.365+0000 7f7721ffb700 1 -- 172.21.15.2:0/203670561 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f7734004dc0 con 0x55b0fba65be0 2023-09-13T04:43:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.365+0000 7f7721ffb700 1 -- 172.21.15.2:0/203670561 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7734004f80 con 0x55b0fba65be0 2023-09-13T04:43:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.365+0000 7f7753de5b80 10 client.26213 fetch_fsmap learned FSMap version 29 2023-09-13T04:43:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.365+0000 7f7753de5b80 1 -- 172.21.15.2:0/203670561 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b0fb7c8c00 con 0x55b0fba65be0 2023-09-13T04:43:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.365+0000 7f7721ffb700 1 -- 172.21.15.2:0/203670561 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77340074e0 con 0x55b0fba65be0 2023-09-13T04:43:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.366+0000 7f7721ffb700 1 -- 172.21.15.2:0/203670561 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f7734030800 con 0x55b0fba65be0 2023-09-13T04:43:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.366+0000 7f7753de5b80 10 client.26213 fetch_fsmap finished waiting for FSMap version 29 2023-09-13T04:43:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.366+0000 7f7753de5b80 10 client.26213 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:43:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.366+0000 7f7753de5b80 20 client.26213 populate_metadata read hostname 'smithi002' 2023-09-13T04:43:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.366+0000 7f7753de5b80 1 --2- 172.21.15.2:0/203670561 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55b0fbae0720 0x55b0fbae2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.366+0000 7f7753de5b80 4 client.26213 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:43:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.366+0000 7f7753de5b80 1 -- 172.21.15.2:0/203670561 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b0fb96cff0 con 0x55b0fbae0720 2023-09-13T04:43:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.366+0000 7f773d99a700 1 --2- 172.21.15.2:0/203670561 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55b0fbae0720 0x55b0fbae2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.367+0000 7f7721ffb700 10 client.26213 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:43:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:17.367+0000 7f773d99a700 1 --2- 172.21.15.2:0/203670561 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55b0fbae0720 0x55b0fbae2b40 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=471d78882333bedb server_cookie=50975abb3f2f85f7 in_seq=0 out_seq=0 2023-09-13T04:43:17.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:17 smithi149 ceph-mon[133315]: pgmap v88: 129 pgs: 129 active+clean; 273 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 189 op/s 2023-09-13T04:43:17.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:17 smithi156 ceph-mon[133675]: pgmap v88: 129 pgs: 129 active+clean; 273 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 189 op/s 2023-09-13T04:43:17.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:17 smithi002 ceph-mon[97651]: pgmap v88: 129 pgs: 129 active+clean; 273 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 189 op/s 2023-09-13T04:43:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.034+0000 7f7721ffb700 1 -- 172.21.15.2:0/203670561 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b0fb96cff0 con 0x55b0fbae0720 2023-09-13T04:43:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.034+0000 7f7721ffb700 10 client.26213 handle_command_reply: tid=0 2023-09-13T04:43:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.034+0000 7f7753de5b80 1 client.26213 shutdown 2023-09-13T04:43:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.034+0000 7f7753de5b80 1 -- 172.21.15.2:0/203670561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba65be0 msgr2=0x55b0fba83170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.034+0000 7f7753de5b80 1 --2- 172.21.15.2:0/203670561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba65be0 0x55b0fba83170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f773400d010 tx=0x7f773401b4c0 comp rx=0 tx=0).stop 2023-09-13T04:43:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.034+0000 7f7753de5b80 1 -- 172.21.15.2:0/203670561 shutdown_connections 2023-09-13T04:43:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.034+0000 7f7753de5b80 1 --2- 172.21.15.2:0/203670561 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55b0fbae0720 0x55b0fbae2b40 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.034+0000 7f7753de5b80 1 --2- 172.21.15.2:0/203670561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0fba86020 0x55b0fba814a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.034+0000 7f7721ffb700 0 client.26213 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:43:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.034+0000 7f7753de5b80 1 --2- 172.21.15.2:0/203670561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0fba65be0 0x55b0fba83170 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.034+0000 7f7753de5b80 1 --2- 172.21.15.2:0/203670561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0fba6b0f0 0x55b0fba80f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.034+0000 7f7753de5b80 1 -- 172.21.15.2:0/203670561 >> 172.21.15.2:0/203670561 conn(0x55b0fb9bfc20 msgr2=0x55b0fba5ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.035+0000 7f7753de5b80 1 -- 172.21.15.2:0/203670561 shutdown_connections 2023-09-13T04:43:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.035+0000 7f7753de5b80 1 -- 172.21.15.2:0/203670561 wait complete. 2023-09-13T04:43:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.035+0000 7f7753de5b80 20 client.26213 trim_cache size 0 max 16384 2023-09-13T04:43:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:43:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:43:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:43:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:43:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.036+0000 7f77403ff700 1 -- 172.21.15.2:0/2765894686 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f77240a0780 msgr2=0x7f77240a2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.036+0000 7f77403ff700 1 --2- 172.21.15.2:0/2765894686 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f77240a0780 0x7f77240a2c40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55b0fba5da70 tx=0x7f7730014040 comp rx=0 tx=0).stop 2023-09-13T04:43:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.036+0000 7f77403ff700 1 -- 172.21.15.2:0/2765894686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77380739a0 msgr2=0x7f7738106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.036+0000 7f77403ff700 1 --2- 172.21.15.2:0/2765894686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77380739a0 0x7f7738106a90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f772800d950 tx=0x7f772800dd10 comp rx=0 tx=0).stop 2023-09-13T04:43:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.036+0000 7f77403ff700 1 -- 172.21.15.2:0/2765894686 shutdown_connections 2023-09-13T04:43:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.036+0000 7f77403ff700 1 --2- 172.21.15.2:0/2765894686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f773810d4f0 0x7f77381013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.036+0000 7f77403ff700 1 --2- 172.21.15.2:0/2765894686 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f77240a0780 0x7f77240a2c40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.036+0000 7f77403ff700 1 --2- 172.21.15.2:0/2765894686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7738073080 0x7f7738106530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.036+0000 7f77403ff700 1 --2- 172.21.15.2:0/2765894686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77380739a0 0x7f7738106a90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.036+0000 7f77403ff700 1 -- 172.21.15.2:0/2765894686 >> 172.21.15.2:0/2765894686 conn(0x7f77380fc740 msgr2=0x7f773810f600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.038+0000 7f77403ff700 1 -- 172.21.15.2:0/2765894686 shutdown_connections 2023-09-13T04:43:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:18.038+0000 7f77403ff700 1 -- 172.21.15.2:0/2765894686 wait complete. 2023-09-13T04:43:19.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:19 smithi149 ceph-mon[133315]: pgmap v89: 129 pgs: 129 active+clean; 273 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 943 KiB/s wr, 105 op/s 2023-09-13T04:43:19.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:19 smithi156 ceph-mon[133675]: pgmap v89: 129 pgs: 129 active+clean; 273 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 943 KiB/s wr, 105 op/s 2023-09-13T04:43:19.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:19 smithi002 ceph-mon[97651]: pgmap v89: 129 pgs: 129 active+clean; 273 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 943 KiB/s wr, 105 op/s 2023-09-13T04:43:20.598 INFO:tasks.workunit.client.0.smithi002.stdout: 71 381954 657 282240 635 133590 693 2023-09-13T04:43:20.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:20 smithi149 ceph-mon[133315]: pgmap v90: 129 pgs: 129 active+clean; 276 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 346 op/s 2023-09-13T04:43:20.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:20 smithi156 ceph-mon[133675]: pgmap v90: 129 pgs: 129 active+clean; 276 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 346 op/s 2023-09-13T04:43:20.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:20 smithi002 ceph-mon[97651]: pgmap v90: 129 pgs: 129 active+clean; 276 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 346 op/s 2023-09-13T04:43:23.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:22 smithi002 ceph-mon[97651]: pgmap v91: 129 pgs: 129 active+clean; 276 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 281 op/s 2023-09-13T04:43:23.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:22 smithi149 ceph-mon[133315]: pgmap v91: 129 pgs: 129 active+clean; 276 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 281 op/s 2023-09-13T04:43:23.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:22 smithi156 ceph-mon[133675]: pgmap v91: 129 pgs: 129 active+clean; 276 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 281 op/s 2023-09-13T04:43:25.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:25 smithi149 ceph-mon[133315]: pgmap v92: 129 pgs: 129 active+clean; 282 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 320 op/s 2023-09-13T04:43:25.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:25 smithi156 ceph-mon[133675]: pgmap v92: 129 pgs: 129 active+clean; 282 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 320 op/s 2023-09-13T04:43:25.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:25 smithi002 ceph-mon[97651]: pgmap v92: 129 pgs: 129 active+clean; 282 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 320 op/s 2023-09-13T04:43:26.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:26 smithi149 ceph-mon[133315]: pgmap v93: 129 pgs: 129 active+clean; 282 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 314 op/s 2023-09-13T04:43:26.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:26 smithi156 ceph-mon[133675]: pgmap v93: 129 pgs: 129 active+clean; 282 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 314 op/s 2023-09-13T04:43:26.511 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:26 smithi002 ceph-mon[97651]: pgmap v93: 129 pgs: 129 active+clean; 282 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 314 op/s 2023-09-13T04:43:29.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:29 smithi149 ceph-mon[133315]: pgmap v94: 129 pgs: 129 active+clean; 282 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 299 op/s 2023-09-13T04:43:29.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:29 smithi156 ceph-mon[133675]: pgmap v94: 129 pgs: 129 active+clean; 282 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 299 op/s 2023-09-13T04:43:29.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:29 smithi002 ceph-mon[97651]: pgmap v94: 129 pgs: 129 active+clean; 282 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 299 op/s 2023-09-13T04:43:30.600 INFO:tasks.workunit.client.0.smithi002.stdout: 77 540221 358 393009 398 176357 136 2023-09-13T04:43:30.784 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:30 smithi002 ceph-mon[97651]: pgmap v95: 129 pgs: 129 active+clean; 327 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 531 op/s 2023-09-13T04:43:30.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:30 smithi149 ceph-mon[133315]: pgmap v95: 129 pgs: 129 active+clean; 327 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 531 op/s 2023-09-13T04:43:30.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:30 smithi156 ceph-mon[133675]: pgmap v95: 129 pgs: 129 active+clean; 327 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 531 op/s 2023-09-13T04:43:33.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:33 smithi149 ceph-mon[133315]: pgmap v96: 129 pgs: 129 active+clean; 327 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 291 op/s 2023-09-13T04:43:33.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:33 smithi156 ceph-mon[133675]: pgmap v96: 129 pgs: 129 active+clean; 327 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 291 op/s 2023-09-13T04:43:33.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:33 smithi002 ceph-mon[97651]: pgmap v96: 129 pgs: 129 active+clean; 327 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 291 op/s 2023-09-13T04:43:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.018+0000 7f3abddff700 1 -- 172.21.15.2:0/3853438021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ab810f890 msgr2=0x7f3ab81081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.018+0000 7f3abddff700 1 --2- 172.21.15.2:0/3853438021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ab810f890 0x7f3ab81081f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009990 tx=0x7f3aac009ca0 comp rx=0 tx=0).stop 2023-09-13T04:43:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.019+0000 7f3abddff700 1 -- 172.21.15.2:0/3853438021 shutdown_connections 2023-09-13T04:43:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.019+0000 7f3abddff700 1 --2- 172.21.15.2:0/3853438021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ab811a490 0x7f3ab811c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.019+0000 7f3abddff700 1 --2- 172.21.15.2:0/3853438021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab810f420 0x7f3ab8107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.019+0000 7f3abddff700 1 --2- 172.21.15.2:0/3853438021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ab810f890 0x7f3ab81081f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.019+0000 7f3abddff700 1 -- 172.21.15.2:0/3853438021 >> 172.21.15.2:0/3853438021 conn(0x7f3ab806cda0 msgr2=0x7f3ab806d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.020+0000 7f3abddff700 1 -- 172.21.15.2:0/3853438021 shutdown_connections 2023-09-13T04:43:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.020+0000 7f3abddff700 1 -- 172.21.15.2:0/3853438021 wait complete. 2023-09-13T04:43:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.022+0000 7f3abddff700 1 Processor -- start 2023-09-13T04:43:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.022+0000 7f3abddff700 1 -- start start 2023-09-13T04:43:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.022+0000 7f3abddff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab810f420 0x7f3ab8118800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.022+0000 7f3abddff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ab810f890 0x7f3ab8118d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.023+0000 7f3abddff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ab811a490 0x7f3ab8111700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.023+0000 7f3abddff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab8111ed0 con 0x7f3ab811a490 2023-09-13T04:43:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.023+0000 7f3abddff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab8112040 con 0x7f3ab810f890 2023-09-13T04:43:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.023+0000 7f3abddff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab8112200 con 0x7f3ab810f420 2023-09-13T04:43:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.024+0000 7f3ab6ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ab810f890 0x7f3ab8118d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.024+0000 7f3ab77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab810f420 0x7f3ab8118800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.024+0000 7f3ab6ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ab810f890 0x7f3ab8118d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:42262/0 (socket says 172.21.15.2:42262) 2023-09-13T04:43:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.024+0000 7f3ab6ffd700 1 -- 172.21.15.2:0/224707689 learned_addr learned my addr 172.21.15.2:0/224707689 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.024+0000 7f3ab7fff700 1 --2- 172.21.15.2:0/224707689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ab811a490 0x7f3ab8111700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:34.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.028+0000 7f3ab77fe700 1 -- 172.21.15.2:0/224707689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ab810f890 msgr2=0x7f3ab8118d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:34.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.028+0000 7f3ab77fe700 1 --2- 172.21.15.2:0/224707689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ab810f890 0x7f3ab8118d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.028+0000 7f3ab77fe700 1 -- 172.21.15.2:0/224707689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ab811a490 msgr2=0x7f3ab8111700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.028+0000 7f3ab77fe700 1 --2- 172.21.15.2:0/224707689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ab811a490 0x7f3ab8111700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.028+0000 7f3ab77fe700 1 -- 172.21.15.2:0/224707689 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac0095f0 con 0x7f3ab810f420 2023-09-13T04:43:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.040+0000 7f3ab77fe700 1 --2- 172.21.15.2:0/224707689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab810f420 0x7f3ab8118800 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800e9f0 tx=0x7f3aa800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.178+0000 7f3ab4ff9700 1 -- 172.21.15.2:0/224707689 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa800c9c0 con 0x7f3ab810f420 2023-09-13T04:43:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.178+0000 7f3ab4ff9700 1 -- 172.21.15.2:0/224707689 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3aa8010430 con 0x7f3ab810f420 2023-09-13T04:43:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.178+0000 7f3ab4ff9700 1 -- 172.21.15.2:0/224707689 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8018680 con 0x7f3ab810f420 2023-09-13T04:43:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.178+0000 7f3abddff700 1 -- 172.21.15.2:0/224707689 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab81124e0 con 0x7f3ab810f420 2023-09-13T04:43:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.178+0000 7f3abddff700 1 -- 172.21.15.2:0/224707689 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab81166c0 con 0x7f3ab810f420 2023-09-13T04:43:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.210+0000 7f3ad1778b80 1 Processor -- start 2023-09-13T04:43:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.210+0000 7f3ad1778b80 1 -- start start 2023-09-13T04:43:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.211+0000 7f3ad1778b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc1a953be0 0x55bc1a953fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.211+0000 7f3ad1778b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc1a954500 0x55bc1a94f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.211+0000 7f3ad1778b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc1a94f840 0x55bc1a94fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.211+0000 7f3ad1778b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc1a8a8720 con 0x55bc1a953be0 2023-09-13T04:43:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.211+0000 7f3ad1778b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc1a8a6040 con 0x55bc1a94f840 2023-09-13T04:43:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.211+0000 7f3ad1778b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc1a8a1930 con 0x55bc1a954500 2023-09-13T04:43:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.212+0000 7f3ab6ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc1a94f840 0x55bc1a94fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.212+0000 7f3ab6ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc1a94f840 0x55bc1a94fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:42264/0 (socket says 172.21.15.2:42264) 2023-09-13T04:43:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.212+0000 7f3ab6ffd700 1 -- 172.21.15.2:0/1425405358 learned_addr learned my addr 172.21.15.2:0/1425405358 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.212+0000 7f3ab4ff9700 1 -- 172.21.15.2:0/224707689 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f3aa800cb20 con 0x7f3ab810f420 2023-09-13T04:43:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.212+0000 7f3ab6ffd700 1 -- 172.21.15.2:0/1425405358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc1a954500 msgr2=0x55bc1a94f300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:43:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.212+0000 7f3ab6ffd700 1 --2- 172.21.15.2:0/1425405358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc1a954500 0x55bc1a94f300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.212+0000 7f3ab6ffd700 1 -- 172.21.15.2:0/1425405358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc1a953be0 msgr2=0x55bc1a953fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.212+0000 7f3ab6ffd700 1 --2- 172.21.15.2:0/1425405358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc1a953be0 0x55bc1a953fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.212+0000 7f3ab6ffd700 1 -- 172.21.15.2:0/1425405358 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc1a7215d0 con 0x55bc1a94f840 2023-09-13T04:43:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.213+0000 7f3ab6ffd700 1 --2- 172.21.15.2:0/1425405358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc1a94f840 0x55bc1a94fc80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3aac005c70 tx=0x7f3aac00b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faa04dca7a28ef40 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.244+0000 7f3ab4ff9700 1 --2- 172.21.15.2:0/224707689 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f3aa00a09c0 0x7f3aa00a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.244+0000 7f3ab4ff9700 1 -- 172.21.15.2:0/224707689 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f3aa8014070 con 0x7f3ab810f420 2023-09-13T04:43:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.249+0000 7f3ab7fff700 1 --2- 172.21.15.2:0/224707689 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f3aa00a09c0 0x7f3aa00a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.294+0000 7f3ab7fff700 1 --2- 172.21.15.2:0/224707689 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f3aa00a09c0 0x7f3aa00a2e80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55bc1a94ba70 tx=0x7f3ab000d040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.332+0000 7f3a9dffb700 1 -- 172.21.15.2:0/1425405358 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac01d070 con 0x55bc1a94f840 2023-09-13T04:43:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.332+0000 7f3a9dffb700 1 -- 172.21.15.2:0/1425405358 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3aac01e040 con 0x55bc1a94f840 2023-09-13T04:43:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.332+0000 7f3a9dffb700 1 -- 172.21.15.2:0/1425405358 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac004be0 con 0x55bc1a94f840 2023-09-13T04:43:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.333+0000 7f3ad1778b80 1 -- 172.21.15.2:0/1425405358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc1a94f840 msgr2=0x55bc1a94fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.333+0000 7f3ad1778b80 1 --2- 172.21.15.2:0/1425405358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc1a94f840 0x55bc1a94fc80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3aac005c70 tx=0x7f3aac00b850 comp rx=0 tx=0).stop 2023-09-13T04:43:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.333+0000 7f3ad1778b80 1 -- 172.21.15.2:0/1425405358 shutdown_connections 2023-09-13T04:43:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.333+0000 7f3ad1778b80 1 --2- 172.21.15.2:0/1425405358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc1a94f840 0x55bc1a94fc80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.333+0000 7f3ad1778b80 1 --2- 172.21.15.2:0/1425405358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc1a953be0 0x55bc1a953fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.333+0000 7f3ad1778b80 1 --2- 172.21.15.2:0/1425405358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc1a954500 0x55bc1a94f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.333+0000 7f3ad1778b80 1 -- 172.21.15.2:0/1425405358 >> 172.21.15.2:0/1425405358 conn(0x55bc1a8adc20 msgr2=0x55bc1a949b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.335+0000 7f3ad1778b80 1 -- 172.21.15.2:0/1425405358 shutdown_connections 2023-09-13T04:43:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.335+0000 7f3ad1778b80 1 -- 172.21.15.2:0/1425405358 wait complete. 2023-09-13T04:43:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.336+0000 7f3ad1778b80 1 -- start start 2023-09-13T04:43:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.336+0000 7f3ad1778b80 1 Processor -- start 2023-09-13T04:43:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.337+0000 7f3ad1778b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc1a94f840 0x55bc1a958970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.337+0000 7f3ad1778b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc1a953be0 0x55bc1a958eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.337+0000 7f3ad1778b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc1a954500 0x55bc1a9593f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.337+0000 7f3ad1778b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc1a83fb50 con 0x55bc1a954500 2023-09-13T04:43:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.337+0000 7f3ad1778b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc1a83f850 con 0x55bc1a953be0 2023-09-13T04:43:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.337+0000 7f3ad1778b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc1a7238c0 con 0x55bc1a94f840 2023-09-13T04:43:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.337+0000 7f3ad1778b80 10 client.0 _ll_register_callbacks cb 0x55bc1a83d620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:43:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.338+0000 7f3ab77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc1a94f840 0x55bc1a958970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.338+0000 7f3ab77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc1a94f840 0x55bc1a958970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:50472/0 (socket says 172.21.15.2:50472) 2023-09-13T04:43:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.338+0000 7f3ab77fe700 1 -- 172.21.15.2:0/598946170 learned_addr learned my addr 172.21.15.2:0/598946170 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:34.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.339+0000 7f3ab6ffd700 1 --2- 172.21.15.2:0/598946170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc1a953be0 0x55bc1a958eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.350+0000 7f3ab6ffd700 1 -- 172.21.15.2:0/598946170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc1a94f840 msgr2=0x55bc1a958970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.350+0000 7f3ab6ffd700 1 --2- 172.21.15.2:0/598946170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc1a94f840 0x55bc1a958970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.350+0000 7f3ab6ffd700 1 -- 172.21.15.2:0/598946170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc1a954500 msgr2=0x55bc1a9593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.350+0000 7f3ab6ffd700 1 --2- 172.21.15.2:0/598946170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc1a954500 0x55bc1a9593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.350+0000 7f3ab6ffd700 1 -- 172.21.15.2:0/598946170 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab811d1b0 con 0x55bc1a953be0 2023-09-13T04:43:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.359+0000 7f3ab6ffd700 1 --2- 172.21.15.2:0/598946170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc1a953be0 0x55bc1a958eb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3aac005b00 tx=0x7f3aac023ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1940676941893c8d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:34.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:34 smithi156 ceph-mon[133675]: pgmap v97: 129 pgs: 129 active+clean; 340 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.7 MiB/s wr, 361 op/s 2023-09-13T04:43:34.514 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:34 smithi002 ceph-mon[97651]: pgmap v97: 129 pgs: 129 active+clean; 340 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.7 MiB/s wr, 361 op/s 2023-09-13T04:43:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.527+0000 7f3a96ffd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:43:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.527+0000 7f3a96ffd700 1 -- 172.21.15.2:0/598946170 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac01d040 con 0x55bc1a953be0 2023-09-13T04:43:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.527+0000 7f3a96ffd700 1 -- 172.21.15.2:0/598946170 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f3aac00f460 con 0x55bc1a953be0 2023-09-13T04:43:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.532+0000 7f3ad1778b80 1 -- 172.21.15.2:0/598946170 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc1a914ef0 con 0x55bc1a953be0 2023-09-13T04:43:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.550+0000 7f3a96ffd700 1 -- 172.21.15.2:0/598946170 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3aac00f640 con 0x55bc1a953be0 2023-09-13T04:43:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.551+0000 7f3ad1778b80 10 client.26225 fetch_fsmap learned FSMap version 29 2023-09-13T04:43:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.551+0000 7f3ad1778b80 1 -- 172.21.15.2:0/598946170 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bc1a8b9740 con 0x55bc1a953be0 2023-09-13T04:43:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.558+0000 7f3a96ffd700 1 -- 172.21.15.2:0/598946170 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bc1a914ef0 con 0x55bc1a953be0 2023-09-13T04:43:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.558+0000 7f3a96ffd700 1 -- 172.21.15.2:0/598946170 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f3aac036560 con 0x55bc1a953be0 2023-09-13T04:43:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.559+0000 7f3ad1778b80 10 client.26225 fetch_fsmap finished waiting for FSMap version 29 2023-09-13T04:43:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.559+0000 7f3ad1778b80 10 client.26225 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:43:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.559+0000 7f3ad1778b80 20 client.26225 populate_metadata read hostname 'smithi002' 2023-09-13T04:43:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.559+0000 7f3ad1778b80 1 --2- 172.21.15.2:0/598946170 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55bc1a968b40 0x55bc1a96af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.559+0000 7f3ad1778b80 4 client.26225 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:43:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.559+0000 7f3ad1778b80 1 -- 172.21.15.2:0/598946170 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc1a89a910 con 0x55bc1a968b40 2023-09-13T04:43:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.567+0000 7f3ab77fe700 1 --2- 172.21.15.2:0/598946170 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55bc1a968b40 0x55bc1a96af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.587+0000 7f3ab77fe700 1 --2- 172.21.15.2:0/598946170 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55bc1a968b40 0x55bc1a96af30 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15f285337244338d server_cookie=ab9ee5e26b941834 in_seq=0 out_seq=0 2023-09-13T04:43:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.587+0000 7f3a96ffd700 10 client.26225 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:43:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.596+0000 7f3a96ffd700 1 -- 172.21.15.2:0/598946170 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bc1a89a910 con 0x55bc1a968b40 2023-09-13T04:43:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.596+0000 7f3a96ffd700 10 client.26225 handle_command_reply: tid=0 2023-09-13T04:43:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.597+0000 7f3ad1778b80 1 client.26225 shutdown 2023-09-13T04:43:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.597+0000 7f3ad1778b80 1 -- 172.21.15.2:0/598946170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc1a953be0 msgr2=0x55bc1a958eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.597+0000 7f3ad1778b80 1 --2- 172.21.15.2:0/598946170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc1a953be0 0x55bc1a958eb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3aac005b00 tx=0x7f3aac023ab0 comp rx=0 tx=0).stop 2023-09-13T04:43:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.602+0000 7f3ad1778b80 1 -- 172.21.15.2:0/598946170 shutdown_connections 2023-09-13T04:43:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.602+0000 7f3ad1778b80 1 --2- 172.21.15.2:0/598946170 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55bc1a968b40 0x55bc1a96af30 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.602+0000 7f3ad1778b80 1 --2- 172.21.15.2:0/598946170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc1a954500 0x55bc1a9593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.602+0000 7f3ad1778b80 1 --2- 172.21.15.2:0/598946170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc1a94f840 0x55bc1a958970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.602+0000 7f3ad1778b80 1 --2- 172.21.15.2:0/598946170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc1a953be0 0x55bc1a958eb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.602+0000 7f3ad1778b80 1 -- 172.21.15.2:0/598946170 >> 172.21.15.2:0/598946170 conn(0x55bc1a8adc20 msgr2=0x55bc1a955330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:34.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.613+0000 7f3a96ffd700 0 client.26225 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:43:34.633 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.632+0000 7f3ad1778b80 1 -- 172.21.15.2:0/598946170 shutdown_connections 2023-09-13T04:43:34.633 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.632+0000 7f3ad1778b80 1 -- 172.21.15.2:0/598946170 wait complete. 2023-09-13T04:43:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.633+0000 7f3ad1778b80 20 client.26225 trim_cache size 0 max 16384 2023-09-13T04:43:34.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:34 smithi149 ceph-mon[133315]: pgmap v97: 129 pgs: 129 active+clean; 340 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.7 MiB/s wr, 361 op/s 2023-09-13T04:43:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.907+0000 7f3ad1778b80 1 Processor -- start 2023-09-13T04:43:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.922+0000 7f3ad1778b80 1 -- start start 2023-09-13T04:43:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.922+0000 7f3ad1778b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc1a94f840 0x55bc1a944730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.922+0000 7f3ad1778b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc1a953be0 0x55bc1a959270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.922+0000 7f3ad1778b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc1a954500 0x55bc1a952740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.922+0000 7f3ad1778b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc1a959a80 con 0x55bc1a953be0 2023-09-13T04:43:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.922+0000 7f3ad1778b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc1a94e560 con 0x55bc1a94f840 2023-09-13T04:43:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.922+0000 7f3ad1778b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc1a944c70 con 0x55bc1a954500 2023-09-13T04:43:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.923+0000 7f3ab77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc1a94f840 0x55bc1a944730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.960+0000 7f3ab77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc1a94f840 0x55bc1a944730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:42288/0 (socket says 172.21.15.2:42288) 2023-09-13T04:43:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.960+0000 7f3ab77fe700 1 -- 172.21.15.2:0/2013184226 learned_addr learned my addr 172.21.15.2:0/2013184226 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.934+0000 7f3ab7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc1a954500 0x55bc1a952740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.934+0000 7f3ab6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc1a953be0 0x55bc1a959270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.961+0000 7f3ab7fff700 1 -- 172.21.15.2:0/2013184226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc1a94f840 msgr2=0x55bc1a944730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.961+0000 7f3ab7fff700 1 --2- 172.21.15.2:0/2013184226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc1a94f840 0x55bc1a944730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.961+0000 7f3ab7fff700 1 -- 172.21.15.2:0/2013184226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc1a953be0 msgr2=0x55bc1a959270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.972+0000 7f3ab7fff700 1 --2- 172.21.15.2:0/2013184226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc1a953be0 0x55bc1a959270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.972+0000 7f3ab7fff700 1 -- 172.21.15.2:0/2013184226 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc1a94fef0 con 0x55bc1a954500 2023-09-13T04:43:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.972+0000 7f3ab6ffd700 1 --2- 172.21.15.2:0/2013184226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc1a953be0 0x55bc1a959270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:43:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:34.976+0000 7f3ab7fff700 1 --2- 172.21.15.2:0/2013184226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc1a954500 0x55bc1a952740 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3ab806a050 tx=0x7f3ab0009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2abdc276f55dca8a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.036+0000 7f3a9e7fc700 1 -- 172.21.15.2:0/2013184226 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0010040 con 0x55bc1a954500 2023-09-13T04:43:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.036+0000 7f3a9e7fc700 1 -- 172.21.15.2:0/2013184226 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ab001c040 con 0x55bc1a954500 2023-09-13T04:43:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.036+0000 7f3a9e7fc700 1 -- 172.21.15.2:0/2013184226 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab00156f0 con 0x55bc1a954500 2023-09-13T04:43:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.044+0000 7f3ad1778b80 1 -- 172.21.15.2:0/2013184226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc1a954500 msgr2=0x55bc1a952740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.044+0000 7f3ad1778b80 1 --2- 172.21.15.2:0/2013184226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc1a954500 0x55bc1a952740 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3ab806a050 tx=0x7f3ab0009650 comp rx=0 tx=0).stop 2023-09-13T04:43:35.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.049+0000 7f3ad1778b80 1 -- 172.21.15.2:0/2013184226 shutdown_connections 2023-09-13T04:43:35.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.049+0000 7f3ad1778b80 1 --2- 172.21.15.2:0/2013184226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc1a954500 0x55bc1a952740 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:35.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.049+0000 7f3ad1778b80 1 --2- 172.21.15.2:0/2013184226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc1a94f840 0x55bc1a944730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:35.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.049+0000 7f3ad1778b80 1 --2- 172.21.15.2:0/2013184226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc1a953be0 0x55bc1a959270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.049+0000 7f3ad1778b80 1 -- 172.21.15.2:0/2013184226 >> 172.21.15.2:0/2013184226 conn(0x55bc1a8adc20 msgr2=0x55bc1a9646d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.050+0000 7f3ad1778b80 1 -- 172.21.15.2:0/2013184226 shutdown_connections 2023-09-13T04:43:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.050+0000 7f3ad1778b80 1 -- 172.21.15.2:0/2013184226 wait complete. 2023-09-13T04:43:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.051+0000 7f3ad1778b80 1 -- start start 2023-09-13T04:43:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.051+0000 7f3ad1778b80 1 Processor -- start 2023-09-13T04:43:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.052+0000 7f3ad1778b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc1a94f840 0x55bc1a95df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.052+0000 7f3ad1778b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc1a953be0 0x55bc1a95e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.052+0000 7f3ad1778b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc1a954500 0x55bc1a95e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.052+0000 7f3ad1778b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc1a960510 con 0x55bc1a954500 2023-09-13T04:43:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.052+0000 7f3ad1778b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc1a9bb910 con 0x55bc1a953be0 2023-09-13T04:43:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.052+0000 7f3ad1778b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc1a9bba50 con 0x55bc1a94f840 2023-09-13T04:43:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.052+0000 7f3ad1778b80 10 client.0 _ll_register_callbacks cb 0x55bc1a83d620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:43:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.053+0000 7f3ab7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc1a954500 0x55bc1a95e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.053+0000 7f3ab7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc1a954500 0x55bc1a95e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51306/0 (socket says 172.21.15.2:51306) 2023-09-13T04:43:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.053+0000 7f3ab7fff700 1 -- 172.21.15.2:0/2557774696 learned_addr learned my addr 172.21.15.2:0/2557774696 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.058+0000 7f3ab7fff700 1 -- 172.21.15.2:0/2557774696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc1a94f840 msgr2=0x55bc1a95df60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:43:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.058+0000 7f3ab7fff700 1 --2- 172.21.15.2:0/2557774696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc1a94f840 0x55bc1a95df60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.058+0000 7f3ab7fff700 1 -- 172.21.15.2:0/2557774696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc1a953be0 msgr2=0x55bc1a95e4a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:43:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.058+0000 7f3ab7fff700 1 --2- 172.21.15.2:0/2557774696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc1a953be0 0x55bc1a95e4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.058+0000 7f3ab7fff700 1 -- 172.21.15.2:0/2557774696 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab000ad60 con 0x55bc1a954500 2023-09-13T04:43:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.067+0000 7f3ab7fff700 1 --2- 172.21.15.2:0/2557774696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc1a954500 0x55bc1a95e9e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000a010 tx=0x7f3ab0020bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3405ea3c10932201 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.183+0000 7f3a96ffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:43:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.183+0000 7f3a96ffd700 1 -- 172.21.15.2:0/2557774696 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0032600 con 0x55bc1a954500 2023-09-13T04:43:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.183+0000 7f3a96ffd700 1 -- 172.21.15.2:0/2557774696 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f3ab0006e40 con 0x55bc1a954500 2023-09-13T04:43:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.183+0000 7f3ad1778b80 1 -- 172.21.15.2:0/2557774696 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3aac01e040 con 0x55bc1a954500 2023-09-13T04:43:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.198+0000 7f3a96ffd700 1 -- 172.21.15.2:0/2557774696 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ab0032760 con 0x55bc1a954500 2023-09-13T04:43:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.207+0000 7f3ad1778b80 10 client.16149 fetch_fsmap learned FSMap version 29 2023-09-13T04:43:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.207+0000 7f3ad1778b80 1 -- 172.21.15.2:0/2557774696 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bc1a6cdd80 con 0x55bc1a954500 2023-09-13T04:43:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.212+0000 7f3a96ffd700 1 -- 172.21.15.2:0/2557774696 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3aac01e040 con 0x55bc1a954500 2023-09-13T04:43:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.212+0000 7f3a96ffd700 1 -- 172.21.15.2:0/2557774696 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f3ab0030030 con 0x55bc1a954500 2023-09-13T04:43:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.212+0000 7f3ad1778b80 10 client.16149 fetch_fsmap finished waiting for FSMap version 29 2023-09-13T04:43:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.213+0000 7f3ad1778b80 10 client.16149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:43:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.213+0000 7f3ad1778b80 20 client.16149 populate_metadata read hostname 'smithi002' 2023-09-13T04:43:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.213+0000 7f3ad1778b80 1 --2- 172.21.15.2:0/2557774696 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55bc1a9bd100 0x55bc1a9bf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.213+0000 7f3ad1778b80 4 client.16149 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:43:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.213+0000 7f3ad1778b80 1 -- 172.21.15.2:0/2557774696 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bc1a85aff0 con 0x55bc1a9bd100 2023-09-13T04:43:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.221+0000 7f3ab77fe700 1 --2- 172.21.15.2:0/2557774696 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55bc1a9bd100 0x55bc1a9bf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.248+0000 7f3ab77fe700 1 --2- 172.21.15.2:0/2557774696 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55bc1a9bd100 0x55bc1a9bf520 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3253f087805d6d6 server_cookie=d3961788f7715220 in_seq=0 out_seq=0 2023-09-13T04:43:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.269+0000 7f3a96ffd700 10 client.16149 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:43:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.865+0000 7f3a96ffd700 1 -- 172.21.15.2:0/2557774696 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bc1a85aff0 con 0x55bc1a9bd100 2023-09-13T04:43:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.865+0000 7f3a96ffd700 10 client.16149 handle_command_reply: tid=0 2023-09-13T04:43:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.865+0000 7f3ad1778b80 1 client.16149 shutdown 2023-09-13T04:43:35.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.866+0000 7f3ad1778b80 1 -- 172.21.15.2:0/2557774696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc1a954500 msgr2=0x55bc1a95e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:35.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.866+0000 7f3ad1778b80 1 --2- 172.21.15.2:0/2557774696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc1a954500 0x55bc1a95e9e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000a010 tx=0x7f3ab0020bb0 comp rx=0 tx=0).stop 2023-09-13T04:43:35.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.867+0000 7f3ad1778b80 1 -- 172.21.15.2:0/2557774696 shutdown_connections 2023-09-13T04:43:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.867+0000 7f3ad1778b80 1 --2- 172.21.15.2:0/2557774696 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55bc1a9bd100 0x55bc1a9bf520 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.867+0000 7f3ad1778b80 1 --2- 172.21.15.2:0/2557774696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc1a954500 0x55bc1a95e9e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.867+0000 7f3ad1778b80 1 --2- 172.21.15.2:0/2557774696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc1a94f840 0x55bc1a95df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.867+0000 7f3ad1778b80 1 --2- 172.21.15.2:0/2557774696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc1a953be0 0x55bc1a95e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.867+0000 7f3ad1778b80 1 -- 172.21.15.2:0/2557774696 >> 172.21.15.2:0/2557774696 conn(0x55bc1a8adc20 msgr2=0x55bc1a952740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.867+0000 7f3a96ffd700 0 client.16149 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:43:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.878+0000 7f3ad1778b80 1 -- 172.21.15.2:0/2557774696 shutdown_connections 2023-09-13T04:43:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.879+0000 7f3ad1778b80 1 -- 172.21.15.2:0/2557774696 wait complete. 2023-09-13T04:43:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.879+0000 7f3ad1778b80 20 client.16149 trim_cache size 0 max 16384 2023-09-13T04:43:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:43:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:43:35.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:43:35.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:43:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.916+0000 7f3abddff700 1 -- 172.21.15.2:0/224707689 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f3aa00a09c0 msgr2=0x7f3aa00a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.916+0000 7f3abddff700 1 --2- 172.21.15.2:0/224707689 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f3aa00a09c0 0x7f3aa00a2e80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55bc1a94ba70 tx=0x7f3ab000d040 comp rx=0 tx=0).stop 2023-09-13T04:43:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.916+0000 7f3abddff700 1 -- 172.21.15.2:0/224707689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab810f420 msgr2=0x7f3ab8118800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.916+0000 7f3abddff700 1 --2- 172.21.15.2:0/224707689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab810f420 0x7f3ab8118800 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800e9f0 tx=0x7f3aa800ed00 comp rx=0 tx=0).stop 2023-09-13T04:43:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.916+0000 7f3abddff700 1 -- 172.21.15.2:0/224707689 shutdown_connections 2023-09-13T04:43:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.916+0000 7f3abddff700 1 --2- 172.21.15.2:0/224707689 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f3aa00a09c0 0x7f3aa00a2e80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.916+0000 7f3abddff700 1 --2- 172.21.15.2:0/224707689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ab811a490 0x7f3ab8111700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.916+0000 7f3abddff700 1 --2- 172.21.15.2:0/224707689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab810f420 0x7f3ab8118800 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.916+0000 7f3abddff700 1 --2- 172.21.15.2:0/224707689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ab810f890 0x7f3ab8118d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.916+0000 7f3abddff700 1 -- 172.21.15.2:0/224707689 >> 172.21.15.2:0/224707689 conn(0x7f3ab806cda0 msgr2=0x7f3ab806f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.917+0000 7f3abddff700 1 -- 172.21.15.2:0/224707689 shutdown_connections 2023-09-13T04:43:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:35.917+0000 7f3abddff700 1 -- 172.21.15.2:0/224707689 wait complete. 2023-09-13T04:43:37.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:37 smithi149 ceph-mon[133315]: pgmap v98: 129 pgs: 129 active+clean; 346 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.6 MiB/s wr, 346 op/s 2023-09-13T04:43:37.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:37 smithi156 ceph-mon[133675]: pgmap v98: 129 pgs: 129 active+clean; 346 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.6 MiB/s wr, 346 op/s 2023-09-13T04:43:37.513 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:37 smithi002 ceph-mon[97651]: pgmap v98: 129 pgs: 129 active+clean; 346 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.6 MiB/s wr, 346 op/s 2023-09-13T04:43:38.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:38 smithi149 ceph-mon[133315]: pgmap v99: 129 pgs: 129 active+clean; 346 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 5.6 MiB/s wr, 328 op/s 2023-09-13T04:43:38.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:38 smithi156 ceph-mon[133675]: pgmap v99: 129 pgs: 129 active+clean; 346 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 5.6 MiB/s wr, 328 op/s 2023-09-13T04:43:38.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:38 smithi002 ceph-mon[97651]: pgmap v99: 129 pgs: 129 active+clean; 346 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 5.6 MiB/s wr, 328 op/s 2023-09-13T04:43:40.633 INFO:tasks.workunit.client.0.smithi002.stdout: 79 719296 237 532229 235 228405 243 2023-09-13T04:43:40.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:40 smithi002 ceph-mon[97651]: pgmap v100: 129 pgs: 129 active+clean; 349 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.9 MiB/s wr, 462 op/s 2023-09-13T04:43:40.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:40 smithi149 ceph-mon[133315]: pgmap v100: 129 pgs: 129 active+clean; 349 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.9 MiB/s wr, 462 op/s 2023-09-13T04:43:40.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:40 smithi156 ceph-mon[133675]: pgmap v100: 129 pgs: 129 active+clean; 349 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.9 MiB/s wr, 462 op/s 2023-09-13T04:43:41.214 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T04:43:41.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:41 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:43:41.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:41 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:43:42.016 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:41 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:43:42.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:42 smithi149 ceph-mon[133315]: pgmap v101: 129 pgs: 129 active+clean; 349 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 2.0 MiB/s wr, 229 op/s 2023-09-13T04:43:42.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:42 smithi156 ceph-mon[133675]: pgmap v101: 129 pgs: 129 active+clean; 349 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 2.0 MiB/s wr, 229 op/s 2023-09-13T04:43:43.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:42 smithi002 ceph-mon[97651]: pgmap v101: 129 pgs: 129 active+clean; 349 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 2.0 MiB/s wr, 229 op/s 2023-09-13T04:43:45.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:45 smithi149 ceph-mon[133315]: pgmap v102: 129 pgs: 129 active+clean; 352 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 280 op/s 2023-09-13T04:43:45.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:45 smithi156 ceph-mon[133675]: pgmap v102: 129 pgs: 129 active+clean; 352 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 280 op/s 2023-09-13T04:43:45.516 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:45 smithi002 ceph-mon[97651]: pgmap v102: 129 pgs: 129 active+clean; 352 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 280 op/s 2023-09-13T04:43:46.573 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:46 smithi002 ceph-mon[97651]: pgmap v103: 129 pgs: 129 active+clean; 352 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 215 op/s 2023-09-13T04:43:46.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:46 smithi149 ceph-mon[133315]: pgmap v103: 129 pgs: 129 active+clean; 352 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 215 op/s 2023-09-13T04:43:46.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:46 smithi156 ceph-mon[133675]: pgmap v103: 129 pgs: 129 active+clean; 352 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 215 op/s 2023-09-13T04:43:48.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.008+0000 7f92213ff700 1 -- 172.21.15.2:0/3991059369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f921c10f440 msgr2=0x7f921c107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:48.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.008+0000 7f92213ff700 1 --2- 172.21.15.2:0/3991059369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f921c10f440 0x7f921c107cb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f920c009990 tx=0x7f920c009ca0 comp rx=0 tx=0).stop 2023-09-13T04:43:48.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.089+0000 7f92213ff700 1 -- 172.21.15.2:0/3991059369 shutdown_connections 2023-09-13T04:43:48.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.089+0000 7f92213ff700 1 --2- 172.21.15.2:0/3991059369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f921c11a490 0x7f921c11c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:48.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.089+0000 7f92213ff700 1 --2- 172.21.15.2:0/3991059369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f921c10f440 0x7f921c107cb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:48.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.089+0000 7f92213ff700 1 --2- 172.21.15.2:0/3991059369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f921c1081f0 0x7f921c108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:48.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.089+0000 7f92213ff700 1 -- 172.21.15.2:0/3991059369 >> 172.21.15.2:0/3991059369 conn(0x7f921c06cfa0 msgr2=0x7f921c06d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:48.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.090+0000 7f92213ff700 1 -- 172.21.15.2:0/3991059369 shutdown_connections 2023-09-13T04:43:48.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.090+0000 7f92213ff700 1 -- 172.21.15.2:0/3991059369 wait complete. 2023-09-13T04:43:48.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.092+0000 7f92213ff700 1 Processor -- start 2023-09-13T04:43:48.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.093+0000 7f92213ff700 1 -- start start 2023-09-13T04:43:48.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.093+0000 7f92213ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f921c1081f0 0x7f921c116530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:48.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.093+0000 7f92213ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f921c10f440 0x7f921c116a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:48.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.093+0000 7f92213ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f921c11a490 0x7f921c1116b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:48.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.093+0000 7f92213ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c111d60 con 0x7f921c11a490 2023-09-13T04:43:48.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.093+0000 7f92213ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c111ed0 con 0x7f921c10f440 2023-09-13T04:43:48.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.093+0000 7f92213ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c112090 con 0x7f921c1081f0 2023-09-13T04:43:48.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.094+0000 7f921affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f921c1081f0 0x7f921c116530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:48.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.094+0000 7f921affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f921c1081f0 0x7f921c116530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:33828/0 (socket says 172.21.15.2:33828) 2023-09-13T04:43:48.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.094+0000 7f921affd700 1 -- 172.21.15.2:0/1657392610 learned_addr learned my addr 172.21.15.2:0/1657392610 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:48.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.094+0000 7f921a7fc700 1 --2- 172.21.15.2:0/1657392610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f921c10f440 0x7f921c116a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:48.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.094+0000 7f921affd700 1 -- 172.21.15.2:0/1657392610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f921c10f440 msgr2=0x7f921c116a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:48.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.094+0000 7f921affd700 1 --2- 172.21.15.2:0/1657392610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f921c10f440 0x7f921c116a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:48.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.094+0000 7f921affd700 1 -- 172.21.15.2:0/1657392610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f921c11a490 msgr2=0x7f921c1116b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-13T04:43:48.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.094+0000 7f921affd700 1 --2- 172.21.15.2:0/1657392610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f921c11a490 0x7f921c1116b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:48.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.094+0000 7f921affd700 1 -- 172.21.15.2:0/1657392610 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920c0095f0 con 0x7f921c1081f0 2023-09-13T04:43:48.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.100+0000 7f921affd700 1 --2- 172.21.15.2:0/1657392610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f921c1081f0 0x7f921c116530 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f920c003c60 tx=0x7f920c003c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:48.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.274+0000 7f920bfff700 1 -- 172.21.15.2:0/1657392610 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c01d070 con 0x7f921c1081f0 2023-09-13T04:43:48.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.274+0000 7f920bfff700 1 -- 172.21.15.2:0/1657392610 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f920c003e60 con 0x7f921c1081f0 2023-09-13T04:43:48.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.274+0000 7f920bfff700 1 -- 172.21.15.2:0/1657392610 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c0173f0 con 0x7f921c1081f0 2023-09-13T04:43:48.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.281+0000 7f92213ff700 1 -- 172.21.15.2:0/1657392610 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f921c112310 con 0x7f921c1081f0 2023-09-13T04:43:48.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.282+0000 7f92213ff700 1 -- 172.21.15.2:0/1657392610 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921c115470 con 0x7f921c1081f0 2023-09-13T04:43:48.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.288+0000 7f9234ca0b80 1 Processor -- start 2023-09-13T04:43:48.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.289+0000 7f9234ca0b80 1 -- start start 2023-09-13T04:43:48.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.289+0000 7f9234ca0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a739fe0 0x56107a73a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:48.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.290+0000 7f9234ca0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107a73a900 0x56107a7344c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:48.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.290+0000 7f9234ca0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56107a734a00 0x56107a734e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:48.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.290+0000 7f9234ca0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107a68d780 con 0x56107a734a00 2023-09-13T04:43:48.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.290+0000 7f9234ca0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107a68b0a0 con 0x56107a739fe0 2023-09-13T04:43:48.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.290+0000 7f9234ca0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107a686990 con 0x56107a73a900 2023-09-13T04:43:48.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.294+0000 7f921b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a739fe0 0x56107a73a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:48.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.294+0000 7f921b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a739fe0 0x56107a73a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:57422/0 (socket says 172.21.15.2:57422) 2023-09-13T04:43:48.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.294+0000 7f921b7fe700 1 -- 172.21.15.2:0/1263941436 learned_addr learned my addr 172.21.15.2:0/1263941436 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:48.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.295+0000 7f921a7fc700 1 --2- 172.21.15.2:0/1263941436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56107a734a00 0x56107a734e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:48.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.296+0000 7f921affd700 1 --2- 172.21.15.2:0/1263941436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107a73a900 0x56107a7344c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:48.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.305+0000 7f921b7fe700 1 -- 172.21.15.2:0/1263941436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107a73a900 msgr2=0x56107a7344c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:48.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.305+0000 7f921b7fe700 1 --2- 172.21.15.2:0/1263941436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107a73a900 0x56107a7344c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:48.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.305+0000 7f921b7fe700 1 -- 172.21.15.2:0/1263941436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56107a734a00 msgr2=0x56107a734e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:48.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.305+0000 7f921b7fe700 1 --2- 172.21.15.2:0/1263941436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56107a734a00 0x56107a734e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:48.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.305+0000 7f921b7fe700 1 -- 172.21.15.2:0/1263941436 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56107a506630 con 0x56107a739fe0 2023-09-13T04:43:48.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.306+0000 7f920bfff700 1 -- 172.21.15.2:0/1657392610 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f920c017550 con 0x7f921c1081f0 2023-09-13T04:43:48.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.308+0000 7f920bfff700 1 --2- 172.21.15.2:0/1657392610 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f92040a09c0 0x7f92040a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:48.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.309+0000 7f920bfff700 1 -- 172.21.15.2:0/1657392610 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f920c0d0510 con 0x7f921c1081f0 2023-09-13T04:43:48.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.318+0000 7f921b7fe700 1 --2- 172.21.15.2:0/1263941436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a739fe0 0x56107a73a3c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9214009c20 tx=0x7f9214009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2725d0b82abd28a4 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:48.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.323+0000 7f921affd700 1 --2- 172.21.15.2:0/1657392610 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f92040a09c0 0x7f92040a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:48.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.370+0000 7f921affd700 1 --2- 172.21.15.2:0/1657392610 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f92040a09c0 0x7f92040a2e80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x56107a73ad60 tx=0x7f920c009ca0 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:48.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.372+0000 7f92097fa700 1 -- 172.21.15.2:0/1263941436 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921400e070 con 0x56107a739fe0 2023-09-13T04:43:48.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.372+0000 7f92097fa700 1 -- 172.21.15.2:0/1263941436 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9214009750 con 0x56107a739fe0 2023-09-13T04:43:48.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.372+0000 7f92097fa700 1 -- 172.21.15.2:0/1263941436 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9214012ea0 con 0x56107a739fe0 2023-09-13T04:43:48.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.373+0000 7f9234ca0b80 1 -- 172.21.15.2:0/1263941436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a739fe0 msgr2=0x56107a73a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:48.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.373+0000 7f9234ca0b80 1 --2- 172.21.15.2:0/1263941436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a739fe0 0x56107a73a3c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9214009c20 tx=0x7f9214009f30 comp rx=0 tx=0).stop 2023-09-13T04:43:48.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.373+0000 7f9234ca0b80 1 -- 172.21.15.2:0/1263941436 shutdown_connections 2023-09-13T04:43:48.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.374+0000 7f9234ca0b80 1 --2- 172.21.15.2:0/1263941436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56107a734a00 0x56107a734e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:48.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.374+0000 7f9234ca0b80 1 --2- 172.21.15.2:0/1263941436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a739fe0 0x56107a73a3c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:48.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.374+0000 7f9234ca0b80 1 --2- 172.21.15.2:0/1263941436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107a73a900 0x56107a7344c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:48.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.374+0000 7f9234ca0b80 1 -- 172.21.15.2:0/1263941436 >> 172.21.15.2:0/1263941436 conn(0x56107a692c80 msgr2=0x56107a72ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:48.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.374+0000 7f9234ca0b80 1 -- 172.21.15.2:0/1263941436 shutdown_connections 2023-09-13T04:43:48.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.374+0000 7f9234ca0b80 1 -- 172.21.15.2:0/1263941436 wait complete. 2023-09-13T04:43:48.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.375+0000 7f9234ca0b80 1 -- start start 2023-09-13T04:43:48.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.376+0000 7f9234ca0b80 1 Processor -- start 2023-09-13T04:43:48.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.376+0000 7f9234ca0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a734a00 0x56107a7408c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:48.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.376+0000 7f9234ca0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56107a739fe0 0x56107a73db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:48.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.377+0000 7f9234ca0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107a73a900 0x56107a73e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:48.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.377+0000 7f9234ca0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107a624bb0 con 0x56107a739fe0 2023-09-13T04:43:48.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.377+0000 7f9234ca0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107a6248b0 con 0x56107a734a00 2023-09-13T04:43:48.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.377+0000 7f9234ca0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107a508920 con 0x56107a73a900 2023-09-13T04:43:48.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.377+0000 7f9234ca0b80 10 client.0 _ll_register_callbacks cb 0x56107a622680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:43:48.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.378+0000 7f921a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56107a739fe0 0x56107a73db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:48.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.378+0000 7f921a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56107a739fe0 0x56107a73db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60732/0 (socket says 172.21.15.2:60732) 2023-09-13T04:43:48.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.378+0000 7f921a7fc700 1 -- 172.21.15.2:0/3019272812 learned_addr learned my addr 172.21.15.2:0/3019272812 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:48.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.377+0000 7f921b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a734a00 0x56107a7408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:48.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.378+0000 7f921b7fe700 1 --2- 172.21.15.2:0/3019272812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107a73a900 0x56107a73e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:48.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.393+0000 7f921b7fe700 1 -- 172.21.15.2:0/3019272812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107a73a900 msgr2=0x56107a73e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:48.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.393+0000 7f921b7fe700 1 --2- 172.21.15.2:0/3019272812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107a73a900 0x56107a73e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:48.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.393+0000 7f921b7fe700 1 -- 172.21.15.2:0/3019272812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56107a739fe0 msgr2=0x56107a73db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:48.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.393+0000 7f921b7fe700 1 --2- 172.21.15.2:0/3019272812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56107a739fe0 0x56107a73db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:48.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.394+0000 7f921b7fe700 1 -- 172.21.15.2:0/3019272812 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92140098d0 con 0x56107a734a00 2023-09-13T04:43:48.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.394+0000 7f921b7fe700 1 --2- 172.21.15.2:0/3019272812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a734a00 0x56107a7408c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f92140098a0 tx=0x7f9214012720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93c0edb0ef68ab44 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:48.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.481+0000 7f92027fc700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:43:48.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.481+0000 7f92027fc700 1 -- 172.21.15.2:0/3019272812 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921400e040 con 0x56107a734a00 2023-09-13T04:43:48.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.481+0000 7f92027fc700 1 -- 172.21.15.2:0/3019272812 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f92140124b0 con 0x56107a734a00 2023-09-13T04:43:48.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.484+0000 7f9234ca0b80 1 -- 172.21.15.2:0/3019272812 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56107a67f970 con 0x56107a734a00 2023-09-13T04:43:48.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.495+0000 7f92027fc700 1 -- 172.21.15.2:0/3019272812 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f921401fa70 con 0x56107a734a00 2023-09-13T04:43:48.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.495+0000 7f9234ca0b80 10 client.26237 fetch_fsmap learned FSMap version 29 2023-09-13T04:43:48.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.495+0000 7f9234ca0b80 1 -- 172.21.15.2:0/3019272812 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56107a69e7a0 con 0x56107a734a00 2023-09-13T04:43:48.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.506+0000 7f92027fc700 1 -- 172.21.15.2:0/3019272812 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56107a67f970 con 0x56107a734a00 2023-09-13T04:43:48.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.506+0000 7f92027fc700 1 -- 172.21.15.2:0/3019272812 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f921401b390 con 0x56107a734a00 2023-09-13T04:43:48.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.506+0000 7f9234ca0b80 10 client.26237 fetch_fsmap finished waiting for FSMap version 29 2023-09-13T04:43:48.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.506+0000 7f9234ca0b80 10 client.26237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:43:48.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.506+0000 7f9234ca0b80 20 client.26237 populate_metadata read hostname 'smithi002' 2023-09-13T04:43:48.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.506+0000 7f9234ca0b80 1 --2- 172.21.15.2:0/3019272812 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56107a74e0b0 0x56107a7504d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:48.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.506+0000 7f9234ca0b80 4 client.26237 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:43:48.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.506+0000 7f9234ca0b80 1 -- 172.21.15.2:0/3019272812 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56107a640050 con 0x56107a74e0b0 2023-09-13T04:43:48.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.508+0000 7f921a7fc700 1 --2- 172.21.15.2:0/3019272812 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56107a74e0b0 0x56107a7504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:48.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.513+0000 7f92027fc700 10 client.26237 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:43:48.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.513+0000 7f921a7fc700 1 --2- 172.21.15.2:0/3019272812 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56107a74e0b0 0x56107a7504d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=886bfd4db1d83861 server_cookie=a86f8f52c6995f4f in_seq=0 out_seq=0 2023-09-13T04:43:48.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.514+0000 7f92027fc700 1 -- 172.21.15.2:0/3019272812 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56107a640050 con 0x56107a74e0b0 2023-09-13T04:43:48.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.514+0000 7f92027fc700 10 client.26237 handle_command_reply: tid=0 2023-09-13T04:43:48.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.514+0000 7f9234ca0b80 1 client.26237 shutdown 2023-09-13T04:43:48.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.515+0000 7f9234ca0b80 1 -- 172.21.15.2:0/3019272812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a734a00 msgr2=0x56107a7408c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:48.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.515+0000 7f9234ca0b80 1 --2- 172.21.15.2:0/3019272812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a734a00 0x56107a7408c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f92140098a0 tx=0x7f9214012720 comp rx=0 tx=0).stop 2023-09-13T04:43:48.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.516+0000 7f9234ca0b80 1 -- 172.21.15.2:0/3019272812 shutdown_connections 2023-09-13T04:43:48.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.516+0000 7f9234ca0b80 1 --2- 172.21.15.2:0/3019272812 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56107a74e0b0 0x56107a7504d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:48.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.516+0000 7f9234ca0b80 1 --2- 172.21.15.2:0/3019272812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107a73a900 0x56107a73e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:48.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.516+0000 7f9234ca0b80 1 --2- 172.21.15.2:0/3019272812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a734a00 0x56107a7408c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:48.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.516+0000 7f9234ca0b80 1 --2- 172.21.15.2:0/3019272812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56107a739fe0 0x56107a73db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:48.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.516+0000 7f9234ca0b80 1 -- 172.21.15.2:0/3019272812 >> 172.21.15.2:0/3019272812 conn(0x56107a692c80 msgr2=0x56107a72ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:48.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.516+0000 7f92027fc700 0 client.26237 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:43:48.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.516+0000 7f9234ca0b80 1 -- 172.21.15.2:0/3019272812 shutdown_connections 2023-09-13T04:43:48.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.516+0000 7f9234ca0b80 1 -- 172.21.15.2:0/3019272812 wait complete. 2023-09-13T04:43:48.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:48.528+0000 7f9234ca0b80 20 client.26237 trim_cache size 0 max 16384 2023-09-13T04:43:49.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.001+0000 7f9234ca0b80 1 Processor -- start 2023-09-13T04:43:49.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.002+0000 7f9234ca0b80 1 -- start start 2023-09-13T04:43:49.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.002+0000 7f9234ca0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a734a00 0x56107a7452a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:49.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.003+0000 7f9234ca0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107a739fe0 0x56107a7298f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:49.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.003+0000 7f921b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a734a00 0x56107a7452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:49.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.003+0000 7f9234ca0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56107a73a900 0x56107a740460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:49.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.003+0000 7f9234ca0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107a73f120 con 0x56107a73a900 2023-09-13T04:43:49.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.003+0000 7f9234ca0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107a73e7e0 con 0x56107a734a00 2023-09-13T04:43:49.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.003+0000 7f9234ca0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107a72b8e0 con 0x56107a739fe0 2023-09-13T04:43:49.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.003+0000 7f921b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a734a00 0x56107a7452a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:57432/0 (socket says 172.21.15.2:57432) 2023-09-13T04:43:49.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.003+0000 7f921b7fe700 1 -- 172.21.15.2:0/2209218856 learned_addr learned my addr 172.21.15.2:0/2209218856 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:49.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.004+0000 7f921a7fc700 1 --2- 172.21.15.2:0/2209218856 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107a739fe0 0x56107a7298f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:49.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.004+0000 7f921b7fe700 1 -- 172.21.15.2:0/2209218856 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107a739fe0 msgr2=0x56107a7298f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:49.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.004+0000 7f921b7fe700 1 --2- 172.21.15.2:0/2209218856 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107a739fe0 0x56107a7298f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:49.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.004+0000 7f921b7fe700 1 -- 172.21.15.2:0/2209218856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56107a73a900 msgr2=0x56107a740460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:49.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.004+0000 7f921b7fe700 1 --2- 172.21.15.2:0/2209218856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56107a73a900 0x56107a740460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:49.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.004+0000 7f921b7fe700 1 -- 172.21.15.2:0/2209218856 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56107a755100 con 0x56107a734a00 2023-09-13T04:43:49.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.005+0000 7f921b7fe700 1 --2- 172.21.15.2:0/2209218856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a734a00 0x56107a7452a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f921400d010 tx=0x7f9214009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9ba8793034f2b60 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:49.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.063+0000 7f9209ffb700 1 -- 172.21.15.2:0/2209218856 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921400e040 con 0x56107a734a00 2023-09-13T04:43:49.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.063+0000 7f9209ffb700 1 -- 172.21.15.2:0/2209218856 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92140124b0 con 0x56107a734a00 2023-09-13T04:43:49.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.063+0000 7f9209ffb700 1 -- 172.21.15.2:0/2209218856 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921401c430 con 0x56107a734a00 2023-09-13T04:43:49.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.104+0000 7f9234ca0b80 1 -- 172.21.15.2:0/2209218856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a734a00 msgr2=0x56107a7452a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:49.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.104+0000 7f9234ca0b80 1 --2- 172.21.15.2:0/2209218856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a734a00 0x56107a7452a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f921400d010 tx=0x7f9214009ba0 comp rx=0 tx=0).stop 2023-09-13T04:43:49.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.118+0000 7f9234ca0b80 1 -- 172.21.15.2:0/2209218856 shutdown_connections 2023-09-13T04:43:49.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.118+0000 7f9234ca0b80 1 --2- 172.21.15.2:0/2209218856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56107a73a900 0x56107a740460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:49.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.118+0000 7f9234ca0b80 1 --2- 172.21.15.2:0/2209218856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a734a00 0x56107a7452a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:49.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.118+0000 7f9234ca0b80 1 --2- 172.21.15.2:0/2209218856 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107a739fe0 0x56107a7298f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:49.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.118+0000 7f9234ca0b80 1 -- 172.21.15.2:0/2209218856 >> 172.21.15.2:0/2209218856 conn(0x56107a692c80 msgr2=0x56107a74a540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:49.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.119+0000 7f9234ca0b80 1 -- 172.21.15.2:0/2209218856 shutdown_connections 2023-09-13T04:43:49.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.119+0000 7f9234ca0b80 1 -- 172.21.15.2:0/2209218856 wait complete. 2023-09-13T04:43:49.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.119+0000 7f9234ca0b80 1 -- start start 2023-09-13T04:43:49.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.120+0000 7f9234ca0b80 1 Processor -- start 2023-09-13T04:43:49.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.121+0000 7f9234ca0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56107a734a00 0x56107a736ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:49.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.121+0000 7f9234ca0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107a739fe0 0x56107a737020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:49.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.121+0000 7f9234ca0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a73a900 0x56107a735a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:49.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.121+0000 7f9234ca0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107a737560 con 0x56107a734a00 2023-09-13T04:43:49.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.121+0000 7f9234ca0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107a7376d0 con 0x56107a73a900 2023-09-13T04:43:49.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.121+0000 7f9234ca0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107a735fa0 con 0x56107a739fe0 2023-09-13T04:43:49.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.121+0000 7f9234ca0b80 10 client.0 _ll_register_callbacks cb 0x56107a622680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:43:49.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.121+0000 7f921a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107a739fe0 0x56107a737020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:49.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.122+0000 7f921a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107a739fe0 0x56107a737020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:33874/0 (socket says 172.21.15.2:33874) 2023-09-13T04:43:49.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.122+0000 7f921a7fc700 1 -- 172.21.15.2:0/425170543 learned_addr learned my addr 172.21.15.2:0/425170543 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:49.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.122+0000 7f921b7fe700 1 --2- 172.21.15.2:0/425170543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a73a900 0x56107a735a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:49.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.122+0000 7f921a7fc700 1 -- 172.21.15.2:0/425170543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a73a900 msgr2=0x56107a735a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:49.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.122+0000 7f921a7fc700 1 --2- 172.21.15.2:0/425170543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a73a900 0x56107a735a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:49.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.122+0000 7f921a7fc700 1 -- 172.21.15.2:0/425170543 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56107a734a00 msgr2=0x56107a736ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:49.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.122+0000 7f921a7fc700 1 --2- 172.21.15.2:0/425170543 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56107a734a00 0x56107a736ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:49.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.122+0000 7f921a7fc700 1 -- 172.21.15.2:0/425170543 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56107a69e7a0 con 0x56107a739fe0 2023-09-13T04:43:49.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.133+0000 7f921a7fc700 1 --2- 172.21.15.2:0/425170543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107a739fe0 0x56107a737020 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f92100093b0 tx=0x7f9210009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e53e8c5945fb088 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:49.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.197+0000 7f92027fc700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:43:49.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.197+0000 7f92027fc700 1 -- 172.21.15.2:0/425170543 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921001b040 con 0x56107a739fe0 2023-09-13T04:43:49.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.197+0000 7f92027fc700 1 -- 172.21.15.2:0/425170543 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f921001ee10 con 0x56107a739fe0 2023-09-13T04:43:49.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.197+0000 7f9234ca0b80 1 -- 172.21.15.2:0/425170543 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56107a727670 con 0x56107a739fe0 2023-09-13T04:43:49.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.209+0000 7f92027fc700 1 -- 172.21.15.2:0/425170543 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f921001e4a0 con 0x56107a739fe0 2023-09-13T04:43:49.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.209+0000 7f9234ca0b80 10 client.26239 fetch_fsmap learned FSMap version 29 2023-09-13T04:43:49.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.209+0000 7f9234ca0b80 1 -- 172.21.15.2:0/425170543 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56107a4b2de0 con 0x56107a739fe0 2023-09-13T04:43:49.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.210+0000 7f92027fc700 1 -- 172.21.15.2:0/425170543 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56107a727670 con 0x56107a739fe0 2023-09-13T04:43:49.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.210+0000 7f92027fc700 1 -- 172.21.15.2:0/425170543 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9210006800 con 0x56107a739fe0 2023-09-13T04:43:49.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.211+0000 7f9234ca0b80 10 client.26239 fetch_fsmap finished waiting for FSMap version 29 2023-09-13T04:43:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.212+0000 7f9234ca0b80 10 client.26239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:43:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.212+0000 7f9234ca0b80 20 client.26239 populate_metadata read hostname 'smithi002' 2023-09-13T04:43:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.212+0000 7f9234ca0b80 1 --2- 172.21.15.2:0/425170543 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56107a7a2980 0x56107a7a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.212+0000 7f9234ca0b80 4 client.26239 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-09-13T04:43:49.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.212+0000 7f9234ca0b80 1 -- 172.21.15.2:0/425170543 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56107a4d3f70 con 0x56107a7a2980 2023-09-13T04:43:49.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.212+0000 7f921b7fe700 1 --2- 172.21.15.2:0/425170543 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56107a7a2980 0x56107a7a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:49.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.230+0000 7f921b7fe700 1 --2- 172.21.15.2:0/425170543 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56107a7a2980 0x56107a7a4da0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d63044d89d43cf server_cookie=e0f97cbfb2f7c6c2 in_seq=0 out_seq=0 2023-09-13T04:43:49.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.230+0000 7f92027fc700 10 client.26239 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:43:49.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.230+0000 7f92027fc700 1 -- 172.21.15.2:0/425170543 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56107a4d3f70 con 0x56107a7a2980 2023-09-13T04:43:49.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.230+0000 7f92027fc700 10 client.26239 handle_command_reply: tid=0 2023-09-13T04:43:49.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.230+0000 7f9234ca0b80 1 client.26239 shutdown 2023-09-13T04:43:49.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.231+0000 7f9234ca0b80 1 -- 172.21.15.2:0/425170543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107a739fe0 msgr2=0x56107a737020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:49.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.231+0000 7f9234ca0b80 1 --2- 172.21.15.2:0/425170543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107a739fe0 0x56107a737020 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f92100093b0 tx=0x7f9210009fd0 comp rx=0 tx=0).stop 2023-09-13T04:43:49.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.231+0000 7f9234ca0b80 1 -- 172.21.15.2:0/425170543 shutdown_connections 2023-09-13T04:43:49.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.231+0000 7f9234ca0b80 1 --2- 172.21.15.2:0/425170543 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56107a7a2980 0x56107a7a4da0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:49.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.231+0000 7f9234ca0b80 1 --2- 172.21.15.2:0/425170543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56107a73a900 0x56107a735a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:49.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.231+0000 7f9234ca0b80 1 --2- 172.21.15.2:0/425170543 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56107a734a00 0x56107a736ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:49.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.231+0000 7f9234ca0b80 1 --2- 172.21.15.2:0/425170543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56107a739fe0 0x56107a737020 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:49.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.231+0000 7f9234ca0b80 1 -- 172.21.15.2:0/425170543 >> 172.21.15.2:0/425170543 conn(0x56107a692c80 msgr2=0x56107a738bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:49.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.231+0000 7f92027fc700 0 client.26239 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:43:49.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.232+0000 7f9234ca0b80 1 -- 172.21.15.2:0/425170543 shutdown_connections 2023-09-13T04:43:49.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.232+0000 7f9234ca0b80 1 -- 172.21.15.2:0/425170543 wait complete. 2023-09-13T04:43:49.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.232+0000 7f9234ca0b80 20 client.26239 trim_cache size 0 max 16384 2023-09-13T04:43:49.237 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:43:49.237 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-13T04:43:49.238 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-09-13T04:43:49.238 INFO:teuthology.orchestra.run.smithi002.stdout: "740235c0-6e1b-4729-b1fe-33345ae948c1": { 2023-09-13T04:43:49.238 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-09-13T04:43:49.239 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "740235c0-6e1b-4729-b1fe-33345ae948c1", 2023-09-13T04:43:49.239 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-09-13T04:43:49.240 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:43:49.240 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:43:49.240 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:43:49.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.276+0000 7f92213ff700 1 -- 172.21.15.2:0/1657392610 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f92040a09c0 msgr2=0x7f92040a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:49.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.276+0000 7f92213ff700 1 --2- 172.21.15.2:0/1657392610 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f92040a09c0 0x7f92040a2e80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x56107a73ad60 tx=0x7f920c009ca0 comp rx=0 tx=0).stop 2023-09-13T04:43:49.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.276+0000 7f92213ff700 1 -- 172.21.15.2:0/1657392610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f921c1081f0 msgr2=0x7f921c116530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:49.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.276+0000 7f92213ff700 1 --2- 172.21.15.2:0/1657392610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f921c1081f0 0x7f921c116530 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f920c003c60 tx=0x7f920c003c90 comp rx=0 tx=0).stop 2023-09-13T04:43:49.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.277+0000 7f92213ff700 1 -- 172.21.15.2:0/1657392610 shutdown_connections 2023-09-13T04:43:49.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.277+0000 7f92213ff700 1 --2- 172.21.15.2:0/1657392610 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f92040a09c0 0x7f92040a2e80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:49.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.277+0000 7f92213ff700 1 --2- 172.21.15.2:0/1657392610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f921c11a490 0x7f921c1116b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:49.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.277+0000 7f92213ff700 1 --2- 172.21.15.2:0/1657392610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f921c1081f0 0x7f921c116530 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:49.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.277+0000 7f92213ff700 1 --2- 172.21.15.2:0/1657392610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f921c10f440 0x7f921c116a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:49.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.277+0000 7f92213ff700 1 -- 172.21.15.2:0/1657392610 >> 172.21.15.2:0/1657392610 conn(0x7f921c06cfa0 msgr2=0x7f921c06f670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:49.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.277+0000 7f92213ff700 1 -- 172.21.15.2:0/1657392610 shutdown_connections 2023-09-13T04:43:49.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:43:49.278+0000 7f92213ff700 1 -- 172.21.15.2:0/1657392610 wait complete. 2023-09-13T04:43:49.342 INFO:tasks.cephfs.filesystem:scrub status for tag:740235c0-6e1b-4729-b1fe-33345ae948c1 - {'path': '/', 'tag': '740235c0-6e1b-4729-b1fe-33345ae948c1', 'options': 'recursive,force'} 2023-09-13T04:43:49.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:49 smithi149 ceph-mon[133315]: pgmap v104: 129 pgs: 129 active+clean; 352 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 683 KiB/s wr, 190 op/s 2023-09-13T04:43:49.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:49 smithi156 ceph-mon[133675]: pgmap v104: 129 pgs: 129 active+clean; 352 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 683 KiB/s wr, 190 op/s 2023-09-13T04:43:49.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:49 smithi002 ceph-mon[97651]: pgmap v104: 129 pgs: 129 active+clean; 352 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 683 KiB/s wr, 190 op/s 2023-09-13T04:43:50.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.510+0000 7f419b7ff700 1 -- 172.21.15.2:0/1407845087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4194106d80 msgr2=0x7f4194107160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:50.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.510+0000 7f419b7ff700 1 --2- 172.21.15.2:0/1407845087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4194106d80 0x7f4194107160 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4190009990 tx=0x7f4190009ca0 comp rx=0 tx=0).stop 2023-09-13T04:43:50.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.511+0000 7f419b7ff700 1 -- 172.21.15.2:0/1407845087 shutdown_connections 2023-09-13T04:43:50.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.511+0000 7f419b7ff700 1 --2- 172.21.15.2:0/1407845087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4194068a90 0x7f419410db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:50.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.511+0000 7f419b7ff700 1 --2- 172.21.15.2:0/1407845087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4194106d80 0x7f4194107160 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:50.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.511+0000 7f419b7ff700 1 --2- 172.21.15.2:0/1407845087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41940680d0 0x7f4194068550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:50.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.511+0000 7f419b7ff700 1 -- 172.21.15.2:0/1407845087 >> 172.21.15.2:0/1407845087 conn(0x7f4194076970 msgr2=0x7f4194076d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:50.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.511+0000 7f419b7ff700 1 -- 172.21.15.2:0/1407845087 shutdown_connections 2023-09-13T04:43:50.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.511+0000 7f419b7ff700 1 -- 172.21.15.2:0/1407845087 wait complete. 2023-09-13T04:43:50.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.513+0000 7f419b7ff700 1 Processor -- start 2023-09-13T04:43:50.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.513+0000 7f419b7ff700 1 -- start start 2023-09-13T04:43:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.513+0000 7f419b7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41940680d0 0x7f419419f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.513+0000 7f419b7ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4194068a90 0x7f419419fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.514+0000 7f419b7ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4194106d80 0x7f419410c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.514+0000 7f419b7ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419410c6b0 con 0x7f4194068a90 2023-09-13T04:43:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.514+0000 7f419b7ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419410c820 con 0x7f4194106d80 2023-09-13T04:43:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.514+0000 7f419b7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419410c9e0 con 0x7f41940680d0 2023-09-13T04:43:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.514+0000 7f419959b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41940680d0 0x7f419419f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.514+0000 7f419959b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41940680d0 0x7f419419f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:33898/0 (socket says 172.21.15.2:33898) 2023-09-13T04:43:50.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.514+0000 7f419959b700 1 -- 172.21.15.2:0/2736092457 learned_addr learned my addr 172.21.15.2:0/2736092457 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:50.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.514+0000 7f419959b700 1 -- 172.21.15.2:0/2736092457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4194106d80 msgr2=0x7f419410c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:50.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.514+0000 7f4199d9c700 1 --2- 172.21.15.2:0/2736092457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4194106d80 0x7f419410c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:50.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.514+0000 7f419959b700 1 --2- 172.21.15.2:0/2736092457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4194106d80 0x7f419410c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:50.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.514+0000 7f419959b700 1 -- 172.21.15.2:0/2736092457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4194068a90 msgr2=0x7f419419fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:50.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.514+0000 7f419959b700 1 --2- 172.21.15.2:0/2736092457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4194068a90 0x7f419419fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:50.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.514+0000 7f419959b700 1 -- 172.21.15.2:0/2736092457 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41900095f0 con 0x7f41940680d0 2023-09-13T04:43:50.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.515+0000 7f419959b700 1 --2- 172.21.15.2:0/2736092457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41940680d0 0x7f419419f570 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4190000c00 tx=0x7f41900057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:50.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.515+0000 7f41867fc700 1 -- 172.21.15.2:0/2736092457 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419001d070 con 0x7f41940680d0 2023-09-13T04:43:50.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.515+0000 7f41867fc700 1 -- 172.21.15.2:0/2736092457 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4190021410 con 0x7f41940680d0 2023-09-13T04:43:50.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.515+0000 7f41867fc700 1 -- 172.21.15.2:0/2736092457 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4190017650 con 0x7f41940680d0 2023-09-13T04:43:50.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.516+0000 7f419b7ff700 1 -- 172.21.15.2:0/2736092457 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f419410cc60 con 0x7f41940680d0 2023-09-13T04:43:50.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.517+0000 7f419b7ff700 1 -- 172.21.15.2:0/2736092457 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4194111180 con 0x7f41940680d0 2023-09-13T04:43:50.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.524+0000 7f41af029b80 1 Processor -- start 2023-09-13T04:43:50.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.525+0000 7f41af029b80 1 -- start start 2023-09-13T04:43:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.525+0000 7f41af029b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fff071abe0 0x55fff071afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.525+0000 7f41af029b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fff071b500 0x55fff0716300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.525+0000 7f41af029b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fff0716840 0x55fff0716c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.525+0000 7f41af029b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fff066f720 con 0x55fff0716840 2023-09-13T04:43:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.525+0000 7f41af029b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fff066d040 con 0x55fff071abe0 2023-09-13T04:43:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.525+0000 7f41af029b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fff0668930 con 0x55fff071b500 2023-09-13T04:43:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.526+0000 7f4198d9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fff0716840 0x55fff0716c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.526+0000 7f4198d9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fff0716840 0x55fff0716c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60784/0 (socket says 172.21.15.2:60784) 2023-09-13T04:43:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.526+0000 7f4198d9a700 1 -- 172.21.15.2:0/2470669762 learned_addr learned my addr 172.21.15.2:0/2470669762 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.527+0000 7f4198d9a700 1 -- 172.21.15.2:0/2470669762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fff071b500 msgr2=0x55fff0716300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:43:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.527+0000 7f4198d9a700 1 --2- 172.21.15.2:0/2470669762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fff071b500 0x55fff0716300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.527+0000 7f4198d9a700 1 -- 172.21.15.2:0/2470669762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fff071abe0 msgr2=0x55fff071afc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:43:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.527+0000 7f4198d9a700 1 --2- 172.21.15.2:0/2470669762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fff071abe0 0x55fff071afc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.527+0000 7f4198d9a700 1 -- 172.21.15.2:0/2470669762 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fff04e85d0 con 0x55fff0716840 2023-09-13T04:43:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.527+0000 7f4198d9a700 1 --2- 172.21.15.2:0/2470669762 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fff0716840 0x55fff0716c80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4188009c70 tx=0x7f4188009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb3682038278cb9 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.528+0000 7f41867fc700 1 -- 172.21.15.2:0/2736092457 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f4190021580 con 0x7f41940680d0 2023-09-13T04:43:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.528+0000 7f417f7fe700 1 -- 172.21.15.2:0/2470669762 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418800ec10 con 0x55fff0716840 2023-09-13T04:43:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.528+0000 7f417f7fe700 1 -- 172.21.15.2:0/2470669762 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f418800ed70 con 0x55fff0716840 2023-09-13T04:43:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.528+0000 7f417f7fe700 1 -- 172.21.15.2:0/2470669762 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4188010430 con 0x55fff0716840 2023-09-13T04:43:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.529+0000 7f41af029b80 1 -- 172.21.15.2:0/2470669762 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fff0716840 msgr2=0x55fff0716c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.529+0000 7f41af029b80 1 --2- 172.21.15.2:0/2470669762 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fff0716840 0x55fff0716c80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4188009c70 tx=0x7f4188009f80 comp rx=0 tx=0).stop 2023-09-13T04:43:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.530+0000 7f41867fc700 1 --2- 172.21.15.2:0/2736092457 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f41800a09c0 0x7f41800a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.531+0000 7f41867fc700 1 -- 172.21.15.2:0/2736092457 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f41900cd270 con 0x7f41940680d0 2023-09-13T04:43:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.531+0000 7f41af029b80 1 -- 172.21.15.2:0/2470669762 shutdown_connections 2023-09-13T04:43:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.531+0000 7f41af029b80 1 --2- 172.21.15.2:0/2470669762 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fff0716840 0x55fff0716c80 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.531+0000 7f41af029b80 1 --2- 172.21.15.2:0/2470669762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fff071abe0 0x55fff071afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.531+0000 7f41af029b80 1 --2- 172.21.15.2:0/2470669762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fff071b500 0x55fff0716300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.531+0000 7f41af029b80 1 -- 172.21.15.2:0/2470669762 >> 172.21.15.2:0/2470669762 conn(0x55fff0674c20 msgr2=0x55fff0710b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.532+0000 7f41af029b80 1 -- 172.21.15.2:0/2470669762 shutdown_connections 2023-09-13T04:43:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.532+0000 7f41af029b80 1 -- 172.21.15.2:0/2470669762 wait complete. 2023-09-13T04:43:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.532+0000 7f41af029b80 1 -- start start 2023-09-13T04:43:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.533+0000 7f41af029b80 1 Processor -- start 2023-09-13T04:43:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.534+0000 7f41af029b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fff0716840 0x55fff071f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:50.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.534+0000 7f41af029b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fff071abe0 0x55fff071ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:50.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.534+0000 7f41af029b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fff071b500 0x55fff0720460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:50.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.534+0000 7f41af029b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fff0606b50 con 0x55fff071b500 2023-09-13T04:43:50.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.534+0000 7f41af029b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fff0606850 con 0x55fff071abe0 2023-09-13T04:43:50.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.534+0000 7f41af029b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fff04ea8c0 con 0x55fff0716840 2023-09-13T04:43:50.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.534+0000 7f41af029b80 10 client.0 _ll_register_callbacks cb 0x55fff0604620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:43:50.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.534+0000 7f4199d9c700 1 --2- 172.21.15.2:0/2736092457 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f41800a09c0 0x7f41800a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:50.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.536+0000 7f4199d9c700 1 --2- 172.21.15.2:0/2736092457 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f41800a09c0 0x7f41800a2e80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55fff0712a70 tx=0x7f418c009360 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:50.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.537+0000 7f4198d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fff071abe0 0x55fff071ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:50.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.537+0000 7f4198d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fff071abe0 0x55fff071ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:57458/0 (socket says 172.21.15.2:57458) 2023-09-13T04:43:50.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.537+0000 7f4198d9a700 1 -- 172.21.15.2:0/3139445443 learned_addr learned my addr 172.21.15.2:0/3139445443 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:50.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.537+0000 7f419959b700 1 --2- 172.21.15.2:0/3139445443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fff0716840 0x55fff071f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:50.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.537+0000 7f4199d9c700 1 --2- 172.21.15.2:0/3139445443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fff071b500 0x55fff0720460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:50.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.538+0000 7f4198d9a700 1 -- 172.21.15.2:0/3139445443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fff0716840 msgr2=0x55fff071f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:50.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.538+0000 7f4198d9a700 1 --2- 172.21.15.2:0/3139445443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fff0716840 0x55fff071f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.538+0000 7f4198d9a700 1 -- 172.21.15.2:0/3139445443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fff071b500 msgr2=0x55fff0720460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.538+0000 7f4198d9a700 1 --2- 172.21.15.2:0/3139445443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fff071b500 0x55fff0720460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.538+0000 7f4198d9a700 1 -- 172.21.15.2:0/3139445443 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4188009920 con 0x55fff071abe0 2023-09-13T04:43:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.538+0000 7f4198d9a700 1 --2- 172.21.15.2:0/3139445443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fff071abe0 0x55fff071ff20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f418800ed40 tx=0x7f4188005650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3acf459510ae1ce server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.538+0000 7f417cff9700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:43:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.539+0000 7f417cff9700 1 -- 172.21.15.2:0/3139445443 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4188004a30 con 0x55fff071abe0 2023-09-13T04:43:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.539+0000 7f41af029b80 1 -- 172.21.15.2:0/3139445443 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fff06dbef0 con 0x55fff071abe0 2023-09-13T04:43:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.540+0000 7f417cff9700 1 -- 172.21.15.2:0/3139445443 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f4188014070 con 0x55fff071abe0 2023-09-13T04:43:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.540+0000 7f417cff9700 1 -- 172.21.15.2:0/3139445443 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4188005130 con 0x55fff071abe0 2023-09-13T04:43:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.540+0000 7f41af029b80 10 client.26246 fetch_fsmap learned FSMap version 29 2023-09-13T04:43:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.540+0000 7f41af029b80 1 -- 172.21.15.2:0/3139445443 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fff0680740 con 0x55fff071abe0 2023-09-13T04:43:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.540+0000 7f417cff9700 1 -- 172.21.15.2:0/3139445443 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fff06dbef0 con 0x55fff071abe0 2023-09-13T04:43:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.541+0000 7f417cff9700 1 -- 172.21.15.2:0/3139445443 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f4188018ac0 con 0x55fff071abe0 2023-09-13T04:43:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.541+0000 7f41af029b80 10 client.26246 fetch_fsmap finished waiting for FSMap version 29 2023-09-13T04:43:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.541+0000 7f41af029b80 10 client.26246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:43:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.541+0000 7f41af029b80 20 client.26246 populate_metadata read hostname 'smithi002' 2023-09-13T04:43:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.541+0000 7f41af029b80 1 --2- 172.21.15.2:0/3139445443 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55fff0741090 0x55fff07434b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.541+0000 7f41af029b80 4 client.26246 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:43:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.541+0000 7f41af029b80 1 -- 172.21.15.2:0/3139445443 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fff0661910 con 0x55fff0741090 2023-09-13T04:43:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.542+0000 7f419959b700 1 --2- 172.21.15.2:0/3139445443 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55fff0741090 0x55fff07434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.542+0000 7f419959b700 1 --2- 172.21.15.2:0/3139445443 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55fff0741090 0x55fff07434b0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ed5b81c66ff87ac server_cookie=e20110e7dffa9d5 in_seq=0 out_seq=0 2023-09-13T04:43:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.542+0000 7f417cff9700 10 client.26246 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:43:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.543+0000 7f417cff9700 1 -- 172.21.15.2:0/3139445443 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fff0661910 con 0x55fff0741090 2023-09-13T04:43:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.543+0000 7f417cff9700 10 client.26246 handle_command_reply: tid=0 2023-09-13T04:43:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.543+0000 7f41af029b80 1 client.26246 shutdown 2023-09-13T04:43:50.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.544+0000 7f41af029b80 1 -- 172.21.15.2:0/3139445443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fff071abe0 msgr2=0x55fff071ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:50.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.544+0000 7f41af029b80 1 --2- 172.21.15.2:0/3139445443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fff071abe0 0x55fff071ff20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f418800ed40 tx=0x7f4188005650 comp rx=0 tx=0).stop 2023-09-13T04:43:50.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.545+0000 7f41af029b80 1 -- 172.21.15.2:0/3139445443 shutdown_connections 2023-09-13T04:43:50.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.545+0000 7f41af029b80 1 --2- 172.21.15.2:0/3139445443 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55fff0741090 0x55fff07434b0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:50.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.545+0000 7f41af029b80 1 --2- 172.21.15.2:0/3139445443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fff071b500 0x55fff0720460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:50.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.545+0000 7f41af029b80 1 --2- 172.21.15.2:0/3139445443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fff0716840 0x55fff071f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.545+0000 7f41af029b80 1 --2- 172.21.15.2:0/3139445443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fff071abe0 0x55fff071ff20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.545+0000 7f41af029b80 1 -- 172.21.15.2:0/3139445443 >> 172.21.15.2:0/3139445443 conn(0x55fff0674c20 msgr2=0x55fff071c330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.545+0000 7f417cff9700 0 client.26246 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:43:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.545+0000 7f41af029b80 1 -- 172.21.15.2:0/3139445443 shutdown_connections 2023-09-13T04:43:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.545+0000 7f41af029b80 1 -- 172.21.15.2:0/3139445443 wait complete. 2023-09-13T04:43:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.547+0000 7f41af029b80 20 client.26246 trim_cache size 0 max 16384 2023-09-13T04:43:50.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.580+0000 7f41af029b80 1 Processor -- start 2023-09-13T04:43:50.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.581+0000 7f41af029b80 1 -- start start 2023-09-13T04:43:50.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.581+0000 7f41af029b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fff0716840 0x55fff0714a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:50.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.581+0000 7f41af029b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fff071abe0 0x55fff0714f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:50.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.581+0000 7f41af029b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fff071b500 0x55fff0715990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:50.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.581+0000 7f41af029b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fff07124e0 con 0x55fff071b500 2023-09-13T04:43:50.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.581+0000 7f41af029b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fff070d7e0 con 0x55fff071abe0 2023-09-13T04:43:50.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.581+0000 7f41af029b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fff07092d0 con 0x55fff0716840 2023-09-13T04:43:50.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.582+0000 7f4198d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fff071abe0 0x55fff0714f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:50.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.582+0000 7f4198d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fff071abe0 0x55fff0714f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:57466/0 (socket says 172.21.15.2:57466) 2023-09-13T04:43:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.582+0000 7f4198d9a700 1 -- 172.21.15.2:0/2967948398 learned_addr learned my addr 172.21.15.2:0/2967948398 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.582+0000 7f4199d9c700 1 --2- 172.21.15.2:0/2967948398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fff071b500 0x55fff0715990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.583+0000 7f419959b700 1 --2- 172.21.15.2:0/2967948398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fff0716840 0x55fff0714a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.583+0000 7f4199d9c700 1 -- 172.21.15.2:0/2967948398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fff0716840 msgr2=0x55fff0714a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.583+0000 7f4199d9c700 1 --2- 172.21.15.2:0/2967948398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fff0716840 0x55fff0714a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:50.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.583+0000 7f4199d9c700 1 -- 172.21.15.2:0/2967948398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fff071abe0 msgr2=0x55fff0714f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:50.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.583+0000 7f4199d9c700 1 --2- 172.21.15.2:0/2967948398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fff071abe0 0x55fff0714f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:50.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.583+0000 7f4199d9c700 1 -- 172.21.15.2:0/2967948398 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fff0494d80 con 0x55fff071b500 2023-09-13T04:43:50.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.583+0000 7f4199d9c700 1 --2- 172.21.15.2:0/2967948398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fff071b500 0x55fff0715990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f418c007f00 tx=0x7f418c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1797a84d6d2f137 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.595+0000 7f417ffff700 1 -- 172.21.15.2:0/2967948398 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c01d040 con 0x55fff071b500 2023-09-13T04:43:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.595+0000 7f417ffff700 1 -- 172.21.15.2:0/2967948398 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f418c018e70 con 0x55fff071b500 2023-09-13T04:43:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.595+0000 7f417ffff700 1 -- 172.21.15.2:0/2967948398 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c0177b0 con 0x55fff071b500 2023-09-13T04:43:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.596+0000 7f41af029b80 1 -- 172.21.15.2:0/2967948398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fff071b500 msgr2=0x55fff0715990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.596+0000 7f41af029b80 1 --2- 172.21.15.2:0/2967948398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fff071b500 0x55fff0715990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f418c007f00 tx=0x7f418c015000 comp rx=0 tx=0).stop 2023-09-13T04:43:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.596+0000 7f41af029b80 1 -- 172.21.15.2:0/2967948398 shutdown_connections 2023-09-13T04:43:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.596+0000 7f41af029b80 1 --2- 172.21.15.2:0/2967948398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fff071b500 0x55fff0715990 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.596+0000 7f41af029b80 1 --2- 172.21.15.2:0/2967948398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fff0716840 0x55fff0714a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.596+0000 7f41af029b80 1 --2- 172.21.15.2:0/2967948398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fff071abe0 0x55fff0714f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:50.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.596+0000 7f41af029b80 1 -- 172.21.15.2:0/2967948398 >> 172.21.15.2:0/2967948398 conn(0x55fff0674c20 msgr2=0x55fff073c740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:50.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.598+0000 7f41af029b80 1 -- 172.21.15.2:0/2967948398 shutdown_connections 2023-09-13T04:43:50.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.598+0000 7f41af029b80 1 -- 172.21.15.2:0/2967948398 wait complete. 2023-09-13T04:43:50.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.598+0000 7f41af029b80 1 -- start start 2023-09-13T04:43:50.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.599+0000 7f41af029b80 1 Processor -- start 2023-09-13T04:43:50.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.600+0000 7f41af029b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fff0716840 0x55fff0735fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:50.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.600+0000 7f41af029b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fff071abe0 0x55fff0736510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:50.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.600+0000 7f41af029b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fff071b500 0x55fff0736a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:50.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.600+0000 7f41af029b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fff0793f40 con 0x55fff071b500 2023-09-13T04:43:50.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.600+0000 7f41af029b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fff07940b0 con 0x55fff0716840 2023-09-13T04:43:50.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.600+0000 7f41af029b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fff0794270 con 0x55fff071abe0 2023-09-13T04:43:50.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.600+0000 7f41af029b80 10 client.0 _ll_register_callbacks cb 0x55fff0604620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:43:50.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.601+0000 7f4198d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fff071abe0 0x55fff0736510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:50.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.601+0000 7f4198d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fff071abe0 0x55fff0736510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:33928/0 (socket says 172.21.15.2:33928) 2023-09-13T04:43:50.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.601+0000 7f4198d9a700 1 -- 172.21.15.2:0/466786006 learned_addr learned my addr 172.21.15.2:0/466786006 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:43:50.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.601+0000 7f4198d9a700 1 -- 172.21.15.2:0/466786006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fff0716840 msgr2=0x55fff0735fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:43:50.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.601+0000 7f4198d9a700 1 --2- 172.21.15.2:0/466786006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fff0716840 0x55fff0735fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:50.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.601+0000 7f4198d9a700 1 -- 172.21.15.2:0/466786006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fff071b500 msgr2=0x55fff0736a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:43:50.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.601+0000 7f4198d9a700 1 --2- 172.21.15.2:0/466786006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fff071b500 0x55fff0736a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:50.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.601+0000 7f4198d9a700 1 -- 172.21.15.2:0/466786006 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f418c007be0 con 0x55fff071abe0 2023-09-13T04:43:50.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.602+0000 7f4198d9a700 1 --2- 172.21.15.2:0/466786006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fff071abe0 0x55fff0736510 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f418800e760 tx=0x7f4188010480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b8b699a6d584cb8 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:43:50.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.602+0000 7f417cff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:43:50.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.603+0000 7f417cff9700 1 -- 172.21.15.2:0/466786006 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4188004240 con 0x55fff071abe0 2023-09-13T04:43:50.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.603+0000 7f417cff9700 1 -- 172.21.15.2:0/466786006 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f4188014070 con 0x55fff071abe0 2023-09-13T04:43:50.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.604+0000 7f41af029b80 1 -- 172.21.15.2:0/466786006 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f418800ed70 con 0x55fff071abe0 2023-09-13T04:43:50.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.605+0000 7f417cff9700 1 -- 172.21.15.2:0/466786006 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41880043a0 con 0x55fff071abe0 2023-09-13T04:43:50.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.605+0000 7f41af029b80 10 client.26254 fetch_fsmap learned FSMap version 29 2023-09-13T04:43:50.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.605+0000 7f41af029b80 1 -- 172.21.15.2:0/466786006 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fff047dc00 con 0x55fff071abe0 2023-09-13T04:43:50.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.606+0000 7f417cff9700 1 -- 172.21.15.2:0/466786006 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f418800ed70 con 0x55fff071abe0 2023-09-13T04:43:50.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.606+0000 7f417cff9700 1 -- 172.21.15.2:0/466786006 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f4188003990 con 0x55fff071abe0 2023-09-13T04:43:50.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.606+0000 7f41af029b80 10 client.26254 fetch_fsmap finished waiting for FSMap version 29 2023-09-13T04:43:50.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.606+0000 7f41af029b80 10 client.26254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:43:50.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.606+0000 7f41af029b80 20 client.26254 populate_metadata read hostname 'smithi002' 2023-09-13T04:43:50.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.606+0000 7f41af029b80 1 --2- 172.21.15.2:0/466786006 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55fff07956b0 0x55fff0797af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:43:50.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.606+0000 7f41af029b80 4 client.26254 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:43:50.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.606+0000 7f41af029b80 1 -- 172.21.15.2:0/466786006 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fff0621ff0 con 0x55fff07956b0 2023-09-13T04:43:50.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.607+0000 7f419959b700 1 --2- 172.21.15.2:0/466786006 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55fff07956b0 0x55fff0797af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:43:50.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.608+0000 7f417cff9700 10 client.26254 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:43:50.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:50.608+0000 7f419959b700 1 --2- 172.21.15.2:0/466786006 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55fff07956b0 0x55fff0797af0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcc63b9041cb9c38 server_cookie=2109db6c9822980d in_seq=0 out_seq=0 2023-09-13T04:43:50.633 INFO:tasks.workunit.client.0.smithi002.stdout: 83 693350 325 519615 186 236167 426 2023-09-13T04:43:50.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:50 smithi002 ceph-mon[97651]: pgmap v105: 129 pgs: 129 active+clean; 381 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 334 op/s 2023-09-13T04:43:50.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:50 smithi149 ceph-mon[133315]: pgmap v105: 129 pgs: 129 active+clean; 381 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 334 op/s 2023-09-13T04:43:50.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:50 smithi156 ceph-mon[133675]: pgmap v105: 129 pgs: 129 active+clean; 381 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 334 op/s 2023-09-13T04:43:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.569+0000 7f417cff9700 1 -- 172.21.15.2:0/466786006 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fff0621ff0 con 0x55fff07956b0 2023-09-13T04:43:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.569+0000 7f417cff9700 10 client.26254 handle_command_reply: tid=0 2023-09-13T04:43:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.579+0000 7f41af029b80 1 client.26254 shutdown 2023-09-13T04:43:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:43:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:43:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:43:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:43:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.583+0000 7f41af029b80 1 -- 172.21.15.2:0/466786006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fff071abe0 msgr2=0x55fff0736510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.583+0000 7f41af029b80 1 --2- 172.21.15.2:0/466786006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fff071abe0 0x55fff0736510 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f418800e760 tx=0x7f4188010480 comp rx=0 tx=0).stop 2023-09-13T04:43:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.583+0000 7f41af029b80 1 -- 172.21.15.2:0/466786006 shutdown_connections 2023-09-13T04:43:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.583+0000 7f41af029b80 1 --2- 172.21.15.2:0/466786006 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55fff07956b0 0x55fff0797af0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.583+0000 7f41af029b80 1 --2- 172.21.15.2:0/466786006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fff071b500 0x55fff0736a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.583+0000 7f41af029b80 1 --2- 172.21.15.2:0/466786006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fff0716840 0x55fff0735fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.583+0000 7f41af029b80 1 --2- 172.21.15.2:0/466786006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fff071abe0 0x55fff0736510 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.583+0000 7f41af029b80 1 -- 172.21.15.2:0/466786006 >> 172.21.15.2:0/466786006 conn(0x55fff0674c20 msgr2=0x55fff0714a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.583+0000 7f417cff9700 0 client.26254 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:43:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.587+0000 7f41af029b80 1 -- 172.21.15.2:0/466786006 shutdown_connections 2023-09-13T04:43:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.587+0000 7f41af029b80 1 -- 172.21.15.2:0/466786006 wait complete. 2023-09-13T04:43:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.588+0000 7f41af029b80 20 client.26254 trim_cache size 0 max 16384 2023-09-13T04:43:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.589+0000 7f419b7ff700 1 -- 172.21.15.2:0/2736092457 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f41800a09c0 msgr2=0x7f41800a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.589+0000 7f419b7ff700 1 --2- 172.21.15.2:0/2736092457 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f41800a09c0 0x7f41800a2e80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55fff0712a70 tx=0x7f418c009360 comp rx=0 tx=0).stop 2023-09-13T04:43:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.589+0000 7f419b7ff700 1 -- 172.21.15.2:0/2736092457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41940680d0 msgr2=0x7f419419f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:43:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.589+0000 7f419b7ff700 1 --2- 172.21.15.2:0/2736092457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41940680d0 0x7f419419f570 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4190000c00 tx=0x7f41900057f0 comp rx=0 tx=0).stop 2023-09-13T04:43:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.591+0000 7f419b7ff700 1 -- 172.21.15.2:0/2736092457 shutdown_connections 2023-09-13T04:43:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.591+0000 7f419b7ff700 1 --2- 172.21.15.2:0/2736092457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4194106d80 0x7f419410c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.591+0000 7f419b7ff700 1 --2- 172.21.15.2:0/2736092457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41940680d0 0x7f419419f570 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.591+0000 7f419b7ff700 1 --2- 172.21.15.2:0/2736092457 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f41800a09c0 0x7f41800a2e80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.591+0000 7f419b7ff700 1 --2- 172.21.15.2:0/2736092457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4194068a90 0x7f419419fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:43:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.591+0000 7f419b7ff700 1 -- 172.21.15.2:0/2736092457 >> 172.21.15.2:0/2736092457 conn(0x7f4194076970 msgr2=0x7f41940fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:43:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.594+0000 7f419b7ff700 1 -- 172.21.15.2:0/2736092457 shutdown_connections 2023-09-13T04:43:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:43:51.594+0000 7f419b7ff700 1 -- 172.21.15.2:0/2736092457 wait complete. 2023-09-13T04:43:51.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:51 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:43:51.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:51 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:43:51.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:51 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:43:51.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:51 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:43:51.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:51 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:43:51.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:51 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:43:52.014 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:51 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:43:52.015 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:51 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:43:52.015 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:51 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:43:52.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:52 smithi149 ceph-mon[133315]: pgmap v106: 129 pgs: 129 active+clean; 381 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 199 op/s 2023-09-13T04:43:52.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:52 smithi156 ceph-mon[133675]: pgmap v106: 129 pgs: 129 active+clean; 381 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 199 op/s 2023-09-13T04:43:53.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:52 smithi002 ceph-mon[97651]: pgmap v106: 129 pgs: 129 active+clean; 381 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 199 op/s 2023-09-13T04:43:55.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:55 smithi149 ceph-mon[133315]: pgmap v107: 129 pgs: 129 active+clean; 382 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 338 op/s 2023-09-13T04:43:55.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:55 smithi156 ceph-mon[133675]: pgmap v107: 129 pgs: 129 active+clean; 382 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 338 op/s 2023-09-13T04:43:55.511 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:55 smithi002 ceph-mon[97651]: pgmap v107: 129 pgs: 129 active+clean; 382 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 338 op/s 2023-09-13T04:43:57.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:57 smithi156 ceph-mon[133675]: pgmap v108: 129 pgs: 129 active+clean; 386 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 3.4 MiB/s wr, 333 op/s 2023-09-13T04:43:57.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:57 smithi002 ceph-mon[97651]: pgmap v108: 129 pgs: 129 active+clean; 386 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 3.4 MiB/s wr, 333 op/s 2023-09-13T04:43:57.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:57 smithi149 ceph-mon[133315]: pgmap v108: 129 pgs: 129 active+clean; 386 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 3.4 MiB/s wr, 333 op/s 2023-09-13T04:43:58.510 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:43:58 smithi002 ceph-mon[97651]: pgmap v109: 129 pgs: 129 active+clean; 386 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 3.4 MiB/s wr, 328 op/s 2023-09-13T04:43:58.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:43:58 smithi149 ceph-mon[133315]: pgmap v109: 129 pgs: 129 active+clean; 386 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 3.4 MiB/s wr, 328 op/s 2023-09-13T04:43:58.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:43:58 smithi156 ceph-mon[133675]: pgmap v109: 129 pgs: 129 active+clean; 386 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 3.4 MiB/s wr, 328 op/s 2023-09-13T04:44:00.648 INFO:tasks.workunit.client.0.smithi002.stdout: 86 671766 252 492364 217 239164 504 2023-09-13T04:44:00.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:00 smithi149 ceph-mon[133315]: pgmap v110: 129 pgs: 129 active+clean; 389 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.7 MiB/s wr, 359 op/s 2023-09-13T04:44:00.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:00 smithi156 ceph-mon[133675]: pgmap v110: 129 pgs: 129 active+clean; 389 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.7 MiB/s wr, 359 op/s 2023-09-13T04:44:01.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:00 smithi002 ceph-mon[97651]: pgmap v110: 129 pgs: 129 active+clean; 389 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.7 MiB/s wr, 359 op/s 2023-09-13T04:44:03.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:03 smithi156 ceph-mon[133675]: pgmap v111: 129 pgs: 129 active+clean; 389 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.1 MiB/s wr, 216 op/s 2023-09-13T04:44:03.512 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:03 smithi002 ceph-mon[97651]: pgmap v111: 129 pgs: 129 active+clean; 389 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.1 MiB/s wr, 216 op/s 2023-09-13T04:44:03.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:03 smithi149 ceph-mon[133315]: pgmap v111: 129 pgs: 129 active+clean; 389 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.1 MiB/s wr, 216 op/s 2023-09-13T04:44:05.509 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:05 smithi002 ceph-mon[97651]: pgmap v112: 129 pgs: 129 active+clean; 393 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 248 op/s 2023-09-13T04:44:05.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:05 smithi149 ceph-mon[133315]: pgmap v112: 129 pgs: 129 active+clean; 393 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 248 op/s 2023-09-13T04:44:05.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:05 smithi156 ceph-mon[133675]: pgmap v112: 129 pgs: 129 active+clean; 393 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 248 op/s 2023-09-13T04:44:06.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:06 smithi149 ceph-mon[133315]: pgmap v113: 129 pgs: 129 active+clean; 393 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2023-09-13T04:44:06.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:06 smithi156 ceph-mon[133675]: pgmap v113: 129 pgs: 129 active+clean; 393 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2023-09-13T04:44:06.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:06 smithi002 ceph-mon[97651]: pgmap v113: 129 pgs: 129 active+clean; 393 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2023-09-13T04:44:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.917+0000 7fa966bff700 1 -- 172.21.15.2:0/1543878178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa96010c5a0 msgr2=0x7fa960078fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.917+0000 7fa966bff700 1 --2- 172.21.15.2:0/1543878178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa96010c5a0 0x7fa960078fc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa95800b0a0 tx=0x7fa95800b3b0 comp rx=0 tx=0).stop 2023-09-13T04:44:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.918+0000 7fa966bff700 1 -- 172.21.15.2:0/1543878178 shutdown_connections 2023-09-13T04:44:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.918+0000 7fa966bff700 1 --2- 172.21.15.2:0/1543878178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa96010c5a0 0x7fa960078fc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.918+0000 7fa966bff700 1 --2- 172.21.15.2:0/1543878178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa960107f20 0x7fa960108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.918+0000 7fa966bff700 1 --2- 172.21.15.2:0/1543878178 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa960108840 0x7fa96010bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.918+0000 7fa966bff700 1 -- 172.21.15.2:0/1543878178 >> 172.21.15.2:0/1543878178 conn(0x7fa96006cda0 msgr2=0x7fa96006d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.927+0000 7fa966bff700 1 -- 172.21.15.2:0/1543878178 shutdown_connections 2023-09-13T04:44:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.927+0000 7fa966bff700 1 -- 172.21.15.2:0/1543878178 wait complete. 2023-09-13T04:44:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.928+0000 7fa966bff700 1 Processor -- start 2023-09-13T04:44:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.929+0000 7fa966bff700 1 -- start start 2023-09-13T04:44:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.929+0000 7fa966bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa960107f20 0x7fa960083600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.929+0000 7fa966bff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa960108840 0x7fa96007d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.929+0000 7fa966bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa96010c5a0 0x7fa96007d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.929+0000 7fa966bff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96007df60 con 0x7fa960108840 2023-09-13T04:44:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.929+0000 7fa966bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96007e0d0 con 0x7fa96010c5a0 2023-09-13T04:44:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.929+0000 7fa966bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96007e290 con 0x7fa960107f20 2023-09-13T04:44:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.935+0000 7fa96519c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa96010c5a0 0x7fa96007d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.935+0000 7fa96519c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa96010c5a0 0x7fa96007d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:37422/0 (socket says 172.21.15.2:37422) 2023-09-13T04:44:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.935+0000 7fa96519c700 1 -- 172.21.15.2:0/4011540624 learned_addr learned my addr 172.21.15.2:0/4011540624 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.936+0000 7fa96499b700 1 --2- 172.21.15.2:0/4011540624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa960107f20 0x7fa960083600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.945+0000 7fa95ffff700 1 --2- 172.21.15.2:0/4011540624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa960108840 0x7fa96007d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.947+0000 7fa96519c700 1 -- 172.21.15.2:0/4011540624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa960107f20 msgr2=0x7fa960083600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.947+0000 7fa96519c700 1 --2- 172.21.15.2:0/4011540624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa960107f20 0x7fa960083600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.947+0000 7fa96519c700 1 -- 172.21.15.2:0/4011540624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa960108840 msgr2=0x7fa96007d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.947+0000 7fa96519c700 1 --2- 172.21.15.2:0/4011540624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa960108840 0x7fa96007d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.947+0000 7fa96519c700 1 -- 172.21.15.2:0/4011540624 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa958009d00 con 0x7fa96010c5a0 2023-09-13T04:44:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:06.947+0000 7fa96519c700 1 --2- 172.21.15.2:0/4011540624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa96010c5a0 0x7fa96007d910 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa95800b070 tx=0x7fa958007b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.104+0000 7fa95dffb700 1 -- 172.21.15.2:0/4011540624 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95800e040 con 0x7fa96010c5a0 2023-09-13T04:44:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.104+0000 7fa95dffb700 1 -- 172.21.15.2:0/4011540624 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa958004e60 con 0x7fa96010c5a0 2023-09-13T04:44:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.104+0000 7fa95dffb700 1 -- 172.21.15.2:0/4011540624 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9580056d0 con 0x7fa96010c5a0 2023-09-13T04:44:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.104+0000 7fa966bff700 1 -- 172.21.15.2:0/4011540624 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9601bafa0 con 0x7fa96010c5a0 2023-09-13T04:44:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.104+0000 7fa966bff700 1 -- 172.21.15.2:0/4011540624 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9601bb400 con 0x7fa96010c5a0 2023-09-13T04:44:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.124+0000 7fa97a4b7b80 1 Processor -- start 2023-09-13T04:44:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.129+0000 7fa97a4b7b80 1 -- start start 2023-09-13T04:44:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.129+0000 7fa97a4b7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e34a08dbe0 0x55e34a08dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.129+0000 7fa97a4b7b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e34a08e500 0x55e34a089300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.129+0000 7fa97a4b7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a089840 0x55e34a089c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.129+0000 7fa97a4b7b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e349fe2720 con 0x55e34a08e500 2023-09-13T04:44:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.129+0000 7fa97a4b7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e349fe0040 con 0x55e34a089840 2023-09-13T04:44:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.129+0000 7fa97a4b7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e349fdb930 con 0x55e34a08dbe0 2023-09-13T04:44:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.130+0000 7fa95ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e34a08dbe0 0x55e34a08dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.130+0000 7fa95ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e34a08dbe0 0x55e34a08dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:54910/0 (socket says 172.21.15.2:54910) 2023-09-13T04:44:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.130+0000 7fa95ffff700 1 -- 172.21.15.2:0/307727499 learned_addr learned my addr 172.21.15.2:0/307727499 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.130+0000 7fa95ffff700 1 --2- 172.21.15.2:0/307727499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a089840 0x55e34a089c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.130+0000 7fa95ffff700 1 -- 172.21.15.2:0/307727499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a089840 msgr2=0x55e34a089c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.130+0000 7fa95ffff700 1 --2- 172.21.15.2:0/307727499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a089840 0x55e34a089c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.130+0000 7fa95ffff700 1 -- 172.21.15.2:0/307727499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e34a08e500 msgr2=0x55e34a089300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:44:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.130+0000 7fa95ffff700 1 --2- 172.21.15.2:0/307727499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e34a08e500 0x55e34a089300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.130+0000 7fa95ffff700 1 -- 172.21.15.2:0/307727499 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e349e5b5d0 con 0x55e34a08dbe0 2023-09-13T04:44:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.131+0000 7fa95ffff700 1 --2- 172.21.15.2:0/307727499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e34a08dbe0 0x55e34a08dfc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa950007e30 tx=0x7fa9500071a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cb67f090fc4af7b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:07.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.137+0000 7fa95dffb700 1 -- 172.21.15.2:0/4011540624 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7fa958005830 con 0x7fa96010c5a0 2023-09-13T04:44:07.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.139+0000 7fa95dffb700 1 --2- 172.21.15.2:0/4011540624 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fa9480a09b0 0x7fa9480a2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.139+0000 7fa95dffb700 1 -- 172.21.15.2:0/4011540624 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fa9580ce380 con 0x7fa96010c5a0 2023-09-13T04:44:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.144+0000 7fa946ffd700 1 -- 172.21.15.2:0/307727499 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa950011e30 con 0x55e34a08dbe0 2023-09-13T04:44:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.145+0000 7fa946ffd700 1 -- 172.21.15.2:0/307727499 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9500178c0 con 0x55e34a08dbe0 2023-09-13T04:44:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.145+0000 7fa946ffd700 1 -- 172.21.15.2:0/307727499 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa950014700 con 0x55e34a08dbe0 2023-09-13T04:44:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.146+0000 7fa96499b700 1 --2- 172.21.15.2:0/4011540624 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fa9480a09b0 0x7fa9480a2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.151+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/307727499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e34a08dbe0 msgr2=0x55e34a08dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.151+0000 7fa97a4b7b80 1 --2- 172.21.15.2:0/307727499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e34a08dbe0 0x55e34a08dfc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa950007e30 tx=0x7fa9500071a0 comp rx=0 tx=0).stop 2023-09-13T04:44:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.161+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/307727499 shutdown_connections 2023-09-13T04:44:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.161+0000 7fa97a4b7b80 1 --2- 172.21.15.2:0/307727499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a089840 0x55e34a089c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.161+0000 7fa97a4b7b80 1 --2- 172.21.15.2:0/307727499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e34a08dbe0 0x55e34a08dfc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.161+0000 7fa97a4b7b80 1 --2- 172.21.15.2:0/307727499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e34a08e500 0x55e34a089300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.161+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/307727499 >> 172.21.15.2:0/307727499 conn(0x55e349fe7c20 msgr2=0x55e34a083b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.162+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/307727499 shutdown_connections 2023-09-13T04:44:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.162+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/307727499 wait complete. 2023-09-13T04:44:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.163+0000 7fa97a4b7b80 1 -- start start 2023-09-13T04:44:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.163+0000 7fa97a4b7b80 1 Processor -- start 2023-09-13T04:44:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.164+0000 7fa97a4b7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a089840 0x55e34a0929b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.164+0000 7fa97a4b7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e34a08dbe0 0x55e34a092ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.164+0000 7fa97a4b7b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e34a08e500 0x55e34a093430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.164+0000 7fa97a4b7b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e349f79b50 con 0x55e34a08e500 2023-09-13T04:44:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.164+0000 7fa97a4b7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e349f79850 con 0x55e34a089840 2023-09-13T04:44:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.164+0000 7fa97a4b7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e349e5d8c0 con 0x55e34a08dbe0 2023-09-13T04:44:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.164+0000 7fa97a4b7b80 10 client.0 _ll_register_callbacks cb 0x55e349f77620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:44:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.166+0000 7fa96519c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e34a08e500 0x55e34a093430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:07.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.172+0000 7fa96499b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a089840 0x55e34a0929b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:07.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.172+0000 7fa96499b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a089840 0x55e34a0929b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:37436/0 (socket says 172.21.15.2:37436) 2023-09-13T04:44:07.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.172+0000 7fa96499b700 1 -- 172.21.15.2:0/2640801157 learned_addr learned my addr 172.21.15.2:0/2640801157 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:07.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.172+0000 7fa95ffff700 1 --2- 172.21.15.2:0/2640801157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e34a08dbe0 0x55e34a092ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.173+0000 7fa96499b700 1 -- 172.21.15.2:0/2640801157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e34a08dbe0 msgr2=0x55e34a092ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.173+0000 7fa96499b700 1 --2- 172.21.15.2:0/2640801157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e34a08dbe0 0x55e34a092ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.173+0000 7fa96499b700 1 -- 172.21.15.2:0/2640801157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e34a08e500 msgr2=0x55e34a093430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.173+0000 7fa96499b700 1 --2- 172.21.15.2:0/2640801157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e34a08e500 0x55e34a093430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.173+0000 7fa96499b700 1 -- 172.21.15.2:0/2640801157 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa950007ad0 con 0x55e34a089840 2023-09-13T04:44:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.174+0000 7fa96499b700 1 --2- 172.21.15.2:0/2640801157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a089840 0x55e34a0929b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa954017940 tx=0x7fa954004800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff5f589f1bed14b1 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.193+0000 7fa96499b700 1 --2- 172.21.15.2:0/4011540624 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fa9480a09b0 0x7fa9480a2e70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa954009dd0 tx=0x7fa954014570 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.284+0000 7fa9447f8700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:44:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.284+0000 7fa9447f8700 1 -- 172.21.15.2:0/2640801157 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa954009480 con 0x55e34a089840 2023-09-13T04:44:07.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.284+0000 7fa9447f8700 1 -- 172.21.15.2:0/2640801157 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fa954016890 con 0x55e34a089840 2023-09-13T04:44:07.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.284+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/2640801157 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e34a04eef0 con 0x55e34a089840 2023-09-13T04:44:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.305+0000 7fa9447f8700 1 -- 172.21.15.2:0/2640801157 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa954016a50 con 0x55e34a089840 2023-09-13T04:44:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.305+0000 7fa97a4b7b80 10 client.26258 fetch_fsmap learned FSMap version 29 2023-09-13T04:44:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.305+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/2640801157 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e349ff3740 con 0x55e34a089840 2023-09-13T04:44:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.306+0000 7fa9447f8700 1 -- 172.21.15.2:0/2640801157 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e34a04eef0 con 0x55e34a089840 2023-09-13T04:44:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.306+0000 7fa9447f8700 1 -- 172.21.15.2:0/2640801157 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa954022430 con 0x55e34a089840 2023-09-13T04:44:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.307+0000 7fa97a4b7b80 10 client.26258 fetch_fsmap finished waiting for FSMap version 29 2023-09-13T04:44:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.307+0000 7fa97a4b7b80 10 client.26258 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:44:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.307+0000 7fa97a4b7b80 20 client.26258 populate_metadata read hostname 'smithi002' 2023-09-13T04:44:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.307+0000 7fa97a4b7b80 1 --2- 172.21.15.2:0/2640801157 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55e34a0a3190 0x55e34a0a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.307+0000 7fa97a4b7b80 4 client.26258 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:44:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.307+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/2640801157 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e349fd4910 con 0x55e34a0a3190 2023-09-13T04:44:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.311+0000 7fa95ffff700 1 --2- 172.21.15.2:0/2640801157 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55e34a0a3190 0x55e34a0a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.341+0000 7fa95ffff700 1 --2- 172.21.15.2:0/2640801157 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55e34a0a3190 0x55e34a0a55b0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=905061cb5949c6e6 server_cookie=a505df13b4dc2c99 in_seq=0 out_seq=0 2023-09-13T04:44:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.341+0000 7fa9447f8700 10 client.26258 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:44:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.341+0000 7fa9447f8700 1 -- 172.21.15.2:0/2640801157 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e349fd4910 con 0x55e34a0a3190 2023-09-13T04:44:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.341+0000 7fa9447f8700 10 client.26258 handle_command_reply: tid=0 2023-09-13T04:44:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.342+0000 7fa97a4b7b80 1 client.26258 shutdown 2023-09-13T04:44:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.345+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/2640801157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a089840 msgr2=0x55e34a0929b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.345+0000 7fa97a4b7b80 1 --2- 172.21.15.2:0/2640801157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a089840 0x55e34a0929b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa954017940 tx=0x7fa954004800 comp rx=0 tx=0).stop 2023-09-13T04:44:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.345+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/2640801157 shutdown_connections 2023-09-13T04:44:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.345+0000 7fa97a4b7b80 1 --2- 172.21.15.2:0/2640801157 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55e34a0a3190 0x55e34a0a55b0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.345+0000 7fa97a4b7b80 1 --2- 172.21.15.2:0/2640801157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e34a08e500 0x55e34a093430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.345+0000 7fa97a4b7b80 1 --2- 172.21.15.2:0/2640801157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a089840 0x55e34a0929b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.345+0000 7fa97a4b7b80 1 --2- 172.21.15.2:0/2640801157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e34a08dbe0 0x55e34a092ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.345+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/2640801157 >> 172.21.15.2:0/2640801157 conn(0x55e349fe7c20 msgr2=0x55e34a07e630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.349+0000 7fa9447f8700 0 client.26258 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:44:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.350+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/2640801157 shutdown_connections 2023-09-13T04:44:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.350+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/2640801157 wait complete. 2023-09-13T04:44:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.351+0000 7fa97a4b7b80 20 client.26258 trim_cache size 0 max 16384 2023-09-13T04:44:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.629+0000 7fa97a4b7b80 1 Processor -- start 2023-09-13T04:44:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.629+0000 7fa97a4b7b80 1 -- start start 2023-09-13T04:44:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.661+0000 7fa97a4b7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e34a089840 0x55e34a099fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.661+0000 7fa97a4b7b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e34a08dbe0 0x55e34a07e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.661+0000 7fa97a4b7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a08e500 0x55e34a093260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.661+0000 7fa97a4b7b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e34a084ac0 con 0x55e34a08dbe0 2023-09-13T04:44:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.661+0000 7fa97a4b7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e34a08e8e0 con 0x55e34a08e500 2023-09-13T04:44:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.661+0000 7fa97a4b7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e34a088550 con 0x55e34a089840 2023-09-13T04:44:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.663+0000 7fa96519c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a08e500 0x55e34a093260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.664+0000 7fa96519c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a08e500 0x55e34a093260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:37444/0 (socket says 172.21.15.2:37444) 2023-09-13T04:44:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.664+0000 7fa96519c700 1 -- 172.21.15.2:0/2945163124 learned_addr learned my addr 172.21.15.2:0/2945163124 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.672+0000 7fa96519c700 1 -- 172.21.15.2:0/2945163124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e34a089840 msgr2=0x55e34a099fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:44:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.672+0000 7fa96519c700 1 --2- 172.21.15.2:0/2945163124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e34a089840 0x55e34a099fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.672+0000 7fa96519c700 1 -- 172.21.15.2:0/2945163124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e34a08dbe0 msgr2=0x55e34a07e730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:44:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.672+0000 7fa96519c700 1 --2- 172.21.15.2:0/2945163124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e34a08dbe0 0x55e34a07e730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.672+0000 7fa96519c700 1 -- 172.21.15.2:0/2945163124 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e34a084110 con 0x55e34a08e500 2023-09-13T04:44:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.681+0000 7fa96519c700 1 --2- 172.21.15.2:0/2945163124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a08e500 0x55e34a093260 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa958097a50 tx=0x7fa9580979a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f91c6fc20f6c65bd server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.888+0000 7fa9477fe700 1 -- 172.21.15.2:0/2945163124 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95809f160 con 0x55e34a08e500 2023-09-13T04:44:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.888+0000 7fa9477fe700 1 -- 172.21.15.2:0/2945163124 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa95809bc50 con 0x55e34a08e500 2023-09-13T04:44:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.888+0000 7fa9477fe700 1 -- 172.21.15.2:0/2945163124 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9580cbc20 con 0x55e34a08e500 2023-09-13T04:44:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.891+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/2945163124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a08e500 msgr2=0x55e34a093260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.891+0000 7fa97a4b7b80 1 --2- 172.21.15.2:0/2945163124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a08e500 0x55e34a093260 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa958097a50 tx=0x7fa9580979a0 comp rx=0 tx=0).stop 2023-09-13T04:44:07.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.897+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/2945163124 shutdown_connections 2023-09-13T04:44:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.897+0000 7fa97a4b7b80 1 --2- 172.21.15.2:0/2945163124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a08e500 0x55e34a093260 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.897+0000 7fa97a4b7b80 1 --2- 172.21.15.2:0/2945163124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e34a089840 0x55e34a099fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.897+0000 7fa97a4b7b80 1 --2- 172.21.15.2:0/2945163124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e34a08dbe0 0x55e34a07e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.897+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/2945163124 >> 172.21.15.2:0/2945163124 conn(0x55e349fe7c20 msgr2=0x55e34a09e730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.898+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/2945163124 shutdown_connections 2023-09-13T04:44:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.898+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/2945163124 wait complete. 2023-09-13T04:44:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.898+0000 7fa97a4b7b80 1 -- start start 2023-09-13T04:44:07.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.899+0000 7fa97a4b7b80 1 Processor -- start 2023-09-13T04:44:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.900+0000 7fa97a4b7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e34a089840 0x55e34a08ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.900+0000 7fa97a4b7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a08dbe0 0x55e34a08f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.900+0000 7fa97a4b7b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e34a08e500 0x55e34a08f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.900+0000 7fa97a4b7b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e34a08fc50 con 0x55e34a08e500 2023-09-13T04:44:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.900+0000 7fa97a4b7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e34a08fdc0 con 0x55e34a08dbe0 2023-09-13T04:44:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.900+0000 7fa97a4b7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e34a08ff80 con 0x55e34a089840 2023-09-13T04:44:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.900+0000 7fa97a4b7b80 10 client.0 _ll_register_callbacks cb 0x55e349f77620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:44:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.900+0000 7fa95ffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a08dbe0 0x55e34a08f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.900+0000 7fa95ffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a08dbe0 0x55e34a08f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:37448/0 (socket says 172.21.15.2:37448) 2023-09-13T04:44:07.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.900+0000 7fa95ffff700 1 -- 172.21.15.2:0/507105722 learned_addr learned my addr 172.21.15.2:0/507105722 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:07.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.901+0000 7fa96499b700 1 --2- 172.21.15.2:0/507105722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e34a089840 0x55e34a08ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.906+0000 7fa95ffff700 1 -- 172.21.15.2:0/507105722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e34a089840 msgr2=0x55e34a08ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.906+0000 7fa95ffff700 1 --2- 172.21.15.2:0/507105722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e34a089840 0x55e34a08ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.906+0000 7fa95ffff700 1 -- 172.21.15.2:0/507105722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e34a08e500 msgr2=0x55e34a08f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.906+0000 7fa95ffff700 1 --2- 172.21.15.2:0/507105722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e34a08e500 0x55e34a08f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.906+0000 7fa95ffff700 1 -- 172.21.15.2:0/507105722 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa958097040 con 0x55e34a08dbe0 2023-09-13T04:44:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:07.909+0000 7fa95ffff700 1 --2- 172.21.15.2:0/507105722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a08dbe0 0x55e34a08f1d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa950006b30 tx=0x7fa950004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89ff8c45a8c4ac server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:08.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:08.034+0000 7fa9447f8700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:44:08.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:08.034+0000 7fa9447f8700 1 -- 172.21.15.2:0/507105722 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa950029570 con 0x55e34a08dbe0 2023-09-13T04:44:08.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:08.034+0000 7fa9447f8700 1 -- 172.21.15.2:0/507105722 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fa950027070 con 0x55e34a08dbe0 2023-09-13T04:44:08.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:08.034+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/507105722 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9500178c0 con 0x55e34a08dbe0 2023-09-13T04:44:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:08.035+0000 7fa9447f8700 1 -- 172.21.15.2:0/507105722 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa950011c30 con 0x55e34a08dbe0 2023-09-13T04:44:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:08.035+0000 7fa97a4b7b80 10 client.26264 fetch_fsmap learned FSMap version 30 2023-09-13T04:44:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:08.035+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/507105722 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e349e07d80 con 0x55e34a08dbe0 2023-09-13T04:44:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:08.041+0000 7fa9447f8700 1 -- 172.21.15.2:0/507105722 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9500178c0 con 0x55e34a08dbe0 2023-09-13T04:44:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:08.041+0000 7fa9447f8700 1 -- 172.21.15.2:0/507105722 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fa950029b50 con 0x55e34a08dbe0 2023-09-13T04:44:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:08.042+0000 7fa97a4b7b80 10 client.26264 fetch_fsmap finished waiting for FSMap version 30 2023-09-13T04:44:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:08.042+0000 7fa97a4b7b80 10 client.26264 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:44:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:08.042+0000 7fa97a4b7b80 20 client.26264 populate_metadata read hostname 'smithi002' 2023-09-13T04:44:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:08.042+0000 7fa97a4b7b80 1 --2- 172.21.15.2:0/507105722 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55e34a0f78a0 0x55e34a0f9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:08.042+0000 7fa97a4b7b80 4 client.26264 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:44:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:08.042+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/507105722 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e349f94ff0 con 0x55e34a0f78a0 2023-09-13T04:44:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:08.047+0000 7fa96499b700 1 --2- 172.21.15.2:0/507105722 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55e34a0f78a0 0x55e34a0f9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:08.061+0000 7fa96499b700 1 --2- 172.21.15.2:0/507105722 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55e34a0f78a0 0x55e34a0f9cc0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbc24b3b52ac9332 server_cookie=aeec35a76fbef989 in_seq=0 out_seq=0 2023-09-13T04:44:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:08.061+0000 7fa9447f8700 10 client.26264 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:44:09.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.032+0000 7fa9447f8700 1 -- 172.21.15.2:0/507105722 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e349f94ff0 con 0x55e34a0f78a0 2023-09-13T04:44:09.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.032+0000 7fa9447f8700 10 client.26264 handle_command_reply: tid=0 2023-09-13T04:44:09.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.032+0000 7fa97a4b7b80 1 client.26264 shutdown 2023-09-13T04:44:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.033+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/507105722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a08dbe0 msgr2=0x55e34a08f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.033+0000 7fa97a4b7b80 1 --2- 172.21.15.2:0/507105722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a08dbe0 0x55e34a08f1d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa950006b30 tx=0x7fa950004860 comp rx=0 tx=0).stop 2023-09-13T04:44:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.039+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/507105722 shutdown_connections 2023-09-13T04:44:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.039+0000 7fa97a4b7b80 1 --2- 172.21.15.2:0/507105722 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55e34a0f78a0 0x55e34a0f9cc0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.039+0000 7fa97a4b7b80 1 --2- 172.21.15.2:0/507105722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e34a08e500 0x55e34a08f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.039+0000 7fa97a4b7b80 1 --2- 172.21.15.2:0/507105722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e34a089840 0x55e34a08ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.039+0000 7fa97a4b7b80 1 --2- 172.21.15.2:0/507105722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34a08dbe0 0x55e34a08f1d0 secure :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa950006b30 tx=0x7fa950004860 comp rx=0 tx=0).stop 2023-09-13T04:44:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.039+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/507105722 >> 172.21.15.2:0/507105722 conn(0x55e349fe7c20 msgr2=0x55e34a088a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.039+0000 7fa9447f8700 0 client.26264 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:44:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.042+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/507105722 shutdown_connections 2023-09-13T04:44:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.048+0000 7fa97a4b7b80 1 -- 172.21.15.2:0/507105722 wait complete. 2023-09-13T04:44:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.049+0000 7fa97a4b7b80 20 client.26264 trim_cache size 0 max 16384 2023-09-13T04:44:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:44:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:44:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:44:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:44:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.134+0000 7fa966bff700 1 -- 172.21.15.2:0/4011540624 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fa9480a09b0 msgr2=0x7fa9480a2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.134+0000 7fa966bff700 1 --2- 172.21.15.2:0/4011540624 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fa9480a09b0 0x7fa9480a2e70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa954009dd0 tx=0x7fa954014570 comp rx=0 tx=0).stop 2023-09-13T04:44:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.134+0000 7fa966bff700 1 -- 172.21.15.2:0/4011540624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa96010c5a0 msgr2=0x7fa96007d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.134+0000 7fa966bff700 1 --2- 172.21.15.2:0/4011540624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa96010c5a0 0x7fa96007d910 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa95800b070 tx=0x7fa958007b60 comp rx=0 tx=0).stop 2023-09-13T04:44:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.145+0000 7fa966bff700 1 -- 172.21.15.2:0/4011540624 shutdown_connections 2023-09-13T04:44:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.145+0000 7fa966bff700 1 --2- 172.21.15.2:0/4011540624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa96010c5a0 0x7fa96007d910 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.145+0000 7fa966bff700 1 --2- 172.21.15.2:0/4011540624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa960107f20 0x7fa960083600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.145+0000 7fa966bff700 1 --2- 172.21.15.2:0/4011540624 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fa9480a09b0 0x7fa9480a2e70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.145+0000 7fa966bff700 1 --2- 172.21.15.2:0/4011540624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa960108840 0x7fa96007d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.145+0000 7fa966bff700 1 -- 172.21.15.2:0/4011540624 >> 172.21.15.2:0/4011540624 conn(0x7fa96006cda0 msgr2=0x7fa960070780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.149+0000 7fa966bff700 1 -- 172.21.15.2:0/4011540624 shutdown_connections 2023-09-13T04:44:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:09.166+0000 7fa966bff700 1 -- 172.21.15.2:0/4011540624 wait complete. 2023-09-13T04:44:09.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:08 smithi002 ceph-mon[97651]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:44:09.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:08 smithi002 ceph-mon[97651]: pgmap v114: 129 pgs: 129 active+clean; 393 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 754 KiB/s wr, 81 op/s 2023-09-13T04:44:09.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:08 smithi149 ceph-mon[133315]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:44:09.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:08 smithi149 ceph-mon[133315]: pgmap v114: 129 pgs: 129 active+clean; 393 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 754 KiB/s wr, 81 op/s 2023-09-13T04:44:09.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:08 smithi156 ceph-mon[133675]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:44:09.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:08 smithi156 ceph-mon[133675]: pgmap v114: 129 pgs: 129 active+clean; 393 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 754 KiB/s wr, 81 op/s 2023-09-13T04:44:10.643 INFO:tasks.workunit.client.0.smithi002.stdout: 89 696442 138 519609 262 256848 409 2023-09-13T04:44:10.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:10 smithi002 ceph-mon[97651]: pgmap v115: 129 pgs: 129 active+clean; 397 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 1.1 MiB/s wr, 250 op/s 2023-09-13T04:44:10.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:10 smithi149 ceph-mon[133315]: pgmap v115: 129 pgs: 129 active+clean; 397 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 1.1 MiB/s wr, 250 op/s 2023-09-13T04:44:10.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:10 smithi156 ceph-mon[133675]: pgmap v115: 129 pgs: 129 active+clean; 397 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 1.1 MiB/s wr, 250 op/s 2023-09-13T04:44:13.525 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:13 smithi002 ceph-mon[97651]: pgmap v116: 129 pgs: 129 active+clean; 397 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 836 KiB/s wr, 219 op/s 2023-09-13T04:44:13.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:13 smithi149 ceph-mon[133315]: pgmap v116: 129 pgs: 129 active+clean; 397 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 836 KiB/s wr, 219 op/s 2023-09-13T04:44:13.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:13 smithi156 ceph-mon[133675]: pgmap v116: 129 pgs: 129 active+clean; 397 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 836 KiB/s wr, 219 op/s 2023-09-13T04:44:14.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:14 smithi149 ceph-mon[133315]: pgmap v117: 129 pgs: 129 active+clean; 417 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 375 op/s 2023-09-13T04:44:14.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:14 smithi156 ceph-mon[133675]: pgmap v117: 129 pgs: 129 active+clean; 417 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 375 op/s 2023-09-13T04:44:14.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:14 smithi002 ceph-mon[97651]: pgmap v117: 129 pgs: 129 active+clean; 417 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 375 op/s 2023-09-13T04:44:17.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:17 smithi156 ceph-mon[133675]: pgmap v118: 129 pgs: 129 active+clean; 419 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.3 MiB/s wr, 358 op/s 2023-09-13T04:44:17.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:17 smithi002 ceph-mon[97651]: pgmap v118: 129 pgs: 129 active+clean; 419 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.3 MiB/s wr, 358 op/s 2023-09-13T04:44:17.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:17 smithi149 ceph-mon[133315]: pgmap v118: 129 pgs: 129 active+clean; 419 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.3 MiB/s wr, 358 op/s 2023-09-13T04:44:19.343 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T04:44:19.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:19 smithi156 ceph-mon[133675]: pgmap v119: 129 pgs: 129 active+clean; 419 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 2.3 MiB/s wr, 341 op/s 2023-09-13T04:44:19.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:19 smithi149 ceph-mon[133315]: pgmap v119: 129 pgs: 129 active+clean; 419 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 2.3 MiB/s wr, 341 op/s 2023-09-13T04:44:19.709 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:19 smithi002 ceph-mon[97651]: pgmap v119: 129 pgs: 129 active+clean; 419 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 2.3 MiB/s wr, 341 op/s 2023-09-13T04:44:20.652 INFO:tasks.workunit.client.0.smithi002.stdout: 94 462558 235 343575 141 180622 434 2023-09-13T04:44:20.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:20 smithi149 ceph-mon[133315]: pgmap v120: 129 pgs: 129 active+clean; 442 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 475 op/s 2023-09-13T04:44:20.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:20 smithi156 ceph-mon[133675]: pgmap v120: 129 pgs: 129 active+clean; 442 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 475 op/s 2023-09-13T04:44:21.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:20 smithi002 ceph-mon[97651]: pgmap v120: 129 pgs: 129 active+clean; 442 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 475 op/s 2023-09-13T04:44:23.185 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:23 smithi002 ceph-mon[97651]: pgmap v121: 129 pgs: 129 active+clean; 442 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 305 op/s 2023-09-13T04:44:23.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:23 smithi156 ceph-mon[133675]: pgmap v121: 129 pgs: 129 active+clean; 442 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 305 op/s 2023-09-13T04:44:23.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:23 smithi149 ceph-mon[133315]: pgmap v121: 129 pgs: 129 active+clean; 442 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 305 op/s 2023-09-13T04:44:25.524 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:25 smithi002 ceph-mon[97651]: pgmap v122: 129 pgs: 129 active+clean; 444 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 338 op/s 2023-09-13T04:44:25.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:25 smithi149 ceph-mon[133315]: pgmap v122: 129 pgs: 129 active+clean; 444 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 338 op/s 2023-09-13T04:44:25.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:25 smithi156 ceph-mon[133675]: pgmap v122: 129 pgs: 129 active+clean; 444 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 338 op/s 2023-09-13T04:44:26.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.639+0000 7f94ad5ff700 1 -- 172.21.15.2:0/866127065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94a81081f0 msgr2=0x7f94a8108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:26.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.639+0000 7f94ad5ff700 1 --2- 172.21.15.2:0/866127065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94a81081f0 0x7f94a8108670 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009f90 tx=0x7f94a000b330 comp rx=0 tx=0).stop 2023-09-13T04:44:26.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.639+0000 7f94ad5ff700 1 -- 172.21.15.2:0/866127065 shutdown_connections 2023-09-13T04:44:26.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.639+0000 7f94ad5ff700 1 --2- 172.21.15.2:0/866127065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94a811a490 0x7f94a811c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:26.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.639+0000 7f94ad5ff700 1 --2- 172.21.15.2:0/866127065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94a810f440 0x7f94a8107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:26.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.639+0000 7f94ad5ff700 1 --2- 172.21.15.2:0/866127065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94a81081f0 0x7f94a8108670 secure :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009f90 tx=0x7f94a000b330 comp rx=0 tx=0).stop 2023-09-13T04:44:26.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.639+0000 7f94ad5ff700 1 -- 172.21.15.2:0/866127065 >> 172.21.15.2:0/866127065 conn(0x7f94a806cfa0 msgr2=0x7f94a806d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.657+0000 7f72a57ff700 1 -- 172.21.15.2:0/1960064660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72a010f420 msgr2=0x7f72a0107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.657+0000 7f72a57ff700 1 --2- 172.21.15.2:0/1960064660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72a010f420 0x7f72a0107cb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7290009990 tx=0x7f7290009ca0 comp rx=0 tx=0).stop 2023-09-13T04:44:26.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.653+0000 7f94ad5ff700 1 -- 172.21.15.2:0/866127065 shutdown_connections 2023-09-13T04:44:26.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.663+0000 7f94ad5ff700 1 -- 172.21.15.2:0/866127065 wait complete. 2023-09-13T04:44:26.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.664+0000 7f94ad5ff700 1 Processor -- start 2023-09-13T04:44:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.680+0000 7f72a57ff700 1 -- 172.21.15.2:0/1960064660 shutdown_connections 2023-09-13T04:44:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.680+0000 7f72a57ff700 1 --2- 172.21.15.2:0/1960064660 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72a011a490 0x7f72a011c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.680+0000 7f72a57ff700 1 --2- 172.21.15.2:0/1960064660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72a010f420 0x7f72a0107cb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.680+0000 7f72a57ff700 1 --2- 172.21.15.2:0/1960064660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72a010f890 0x7f72a01081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.680+0000 7f72a57ff700 1 -- 172.21.15.2:0/1960064660 >> 172.21.15.2:0/1960064660 conn(0x7f72a006cda0 msgr2=0x7f72a006d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:26.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.665+0000 7f94ad5ff700 1 -- start start 2023-09-13T04:44:26.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.665+0000 7f94ad5ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94a810f440 0x7f94a81a5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:26.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.665+0000 7f94ad5ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94a811a490 0x7f94a81a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:26.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.665+0000 7f94ad5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94a81aa8f0 0x7f94a81aad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:26.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.665+0000 7f94ad5ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a81ab370 con 0x7f94a811a490 2023-09-13T04:44:26.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.665+0000 7f94ad5ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a81ab4b0 con 0x7f94a810f440 2023-09-13T04:44:26.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.665+0000 7f94ad5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a81ab620 con 0x7f94a81aa8f0 2023-09-13T04:44:26.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.667+0000 7f94a67fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94a811a490 0x7f94a81a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:26.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.667+0000 7f94a67fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94a811a490 0x7f94a81a6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54866/0 (socket says 172.21.15.2:54866) 2023-09-13T04:44:26.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.667+0000 7f94a67fc700 1 -- 172.21.15.2:0/500657641 learned_addr learned my addr 172.21.15.2:0/500657641 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:26.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.676+0000 7f94a77fe700 1 --2- 172.21.15.2:0/500657641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94a81aa8f0 0x7f94a81aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:26.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.676+0000 7f94a6ffd700 1 --2- 172.21.15.2:0/500657641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94a810f440 0x7f94a81a5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:26.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.676+0000 7f94a6ffd700 1 -- 172.21.15.2:0/500657641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94a81aa8f0 msgr2=0x7f94a81aad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:26.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.676+0000 7f94a6ffd700 1 --2- 172.21.15.2:0/500657641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94a81aa8f0 0x7f94a81aad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:26.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.676+0000 7f94a6ffd700 1 -- 172.21.15.2:0/500657641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94a811a490 msgr2=0x7f94a81a6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:26.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.676+0000 7f94a6ffd700 1 --2- 172.21.15.2:0/500657641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94a811a490 0x7f94a81a6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:26.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.676+0000 7f94a6ffd700 1 -- 172.21.15.2:0/500657641 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a0009c40 con 0x7f94a810f440 2023-09-13T04:44:26.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.677+0000 7f94a6ffd700 1 --2- 172.21.15.2:0/500657641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94a810f440 0x7f94a81a5f50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f949c006fd0 tx=0x7f949c00b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.680+0000 7f72a57ff700 1 -- 172.21.15.2:0/1960064660 shutdown_connections 2023-09-13T04:44:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.680+0000 7f72a57ff700 1 -- 172.21.15.2:0/1960064660 wait complete. 2023-09-13T04:44:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.682+0000 7f72a57ff700 1 Processor -- start 2023-09-13T04:44:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.683+0000 7f72a57ff700 1 -- start start 2023-09-13T04:44:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.683+0000 7f72a57ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72a010f420 0x7f72a01187b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.683+0000 7f72a57ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72a010f890 0x7f72a0118d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.683+0000 7f72a57ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72a011a490 0x7f72a0111670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.683+0000 7f72a57ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a0111e40 con 0x7f72a010f890 2023-09-13T04:44:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.683+0000 7f72a57ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a0111fb0 con 0x7f72a010f420 2023-09-13T04:44:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.683+0000 7f72a57ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a0112170 con 0x7f72a011a490 2023-09-13T04:44:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.684+0000 7f729f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72a011a490 0x7f72a0111670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.684+0000 7f729f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72a011a490 0x7f72a0111670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:47716/0 (socket says 172.21.15.2:47716) 2023-09-13T04:44:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.684+0000 7f729f7fe700 1 -- 172.21.15.2:0/1450801934 learned_addr learned my addr 172.21.15.2:0/1450801934 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.685+0000 7f729f7fe700 1 -- 172.21.15.2:0/1450801934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72a010f420 msgr2=0x7f72a01187b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-13T04:44:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.685+0000 7f729f7fe700 1 --2- 172.21.15.2:0/1450801934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72a010f420 0x7f72a01187b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.685+0000 7f729f7fe700 1 -- 172.21.15.2:0/1450801934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72a010f890 msgr2=0x7f72a0118d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.685+0000 7f729f7fe700 1 --2- 172.21.15.2:0/1450801934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72a010f890 0x7f72a0118d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.685+0000 7f729f7fe700 1 -- 172.21.15.2:0/1450801934 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72900095f0 con 0x7f72a011a490 2023-09-13T04:44:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.692+0000 7f729f7fe700 1 --2- 172.21.15.2:0/1450801934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72a011a490 0x7f72a0111670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f729800b9e0 tx=0x7f729800bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.750+0000 7f728ffff700 1 -- 172.21.15.2:0/1450801934 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729800c9b0 con 0x7f72a011a490 2023-09-13T04:44:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.751+0000 7f728ffff700 1 -- 172.21.15.2:0/1450801934 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7298014440 con 0x7f72a011a490 2023-09-13T04:44:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.751+0000 7f728ffff700 1 -- 172.21.15.2:0/1450801934 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72980125a0 con 0x7f72a011a490 2023-09-13T04:44:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.751+0000 7f72a57ff700 1 -- 172.21.15.2:0/1450801934 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a0112450 con 0x7f72a011a490 2023-09-13T04:44:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.751+0000 7f72a57ff700 1 -- 172.21.15.2:0/1450801934 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a0116610 con 0x7f72a011a490 2023-09-13T04:44:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.763+0000 7f728ffff700 1 -- 172.21.15.2:0/1450801934 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f7298012700 con 0x7f72a011a490 2023-09-13T04:44:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.763+0000 7f72b90b4b80 1 Processor -- start 2023-09-13T04:44:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.764+0000 7f72b90b4b80 1 -- start start 2023-09-13T04:44:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.765+0000 7f72b90b4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c5be0 0x55a3b79c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.765+0000 7f72b90b4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3b79c6500 0x55a3b79c1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.765+0000 7f72b90b4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3b79c1840 0x55a3b79c1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.765+0000 7f72b90b4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3b791a720 con 0x55a3b79c1840 2023-09-13T04:44:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.765+0000 7f72b90b4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3b7918040 con 0x55a3b79c6500 2023-09-13T04:44:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.765+0000 7f72b90b4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3b7913930 con 0x55a3b79c5be0 2023-09-13T04:44:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.766+0000 7f729e7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c5be0 0x55a3b79c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.766+0000 7f729e7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c5be0 0x55a3b79c5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:47722/0 (socket says 172.21.15.2:47722) 2023-09-13T04:44:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.766+0000 7f729e7fc700 1 -- 172.21.15.2:0/1573657393 learned_addr learned my addr 172.21.15.2:0/1573657393 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.766+0000 7f729effd700 1 --2- 172.21.15.2:0/1573657393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3b79c6500 0x55a3b79c1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.769+0000 7f729effd700 1 -- 172.21.15.2:0/1573657393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c5be0 msgr2=0x55a3b79c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.769+0000 7f729effd700 1 --2- 172.21.15.2:0/1573657393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c5be0 0x55a3b79c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.769+0000 7f729effd700 1 -- 172.21.15.2:0/1573657393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3b79c1840 msgr2=0x55a3b79c1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.769+0000 7f729effd700 1 --2- 172.21.15.2:0/1573657393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3b79c1840 0x55a3b79c1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.769+0000 7f729effd700 1 -- 172.21.15.2:0/1573657393 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3b77935d0 con 0x55a3b79c6500 2023-09-13T04:44:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.769+0000 7f729effd700 1 --2- 172.21.15.2:0/1573657393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3b79c6500 0x55a3b79c1300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f72900098e0 tx=0x7f7290005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bda76c887ef208af server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.771+0000 7f728ffff700 1 --2- 172.21.15.2:0/1450801934 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f72880a09c0 0x7f72880a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.771+0000 7f728ffff700 1 -- 172.21.15.2:0/1450801934 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f7298006e40 con 0x7f72a011a490 2023-09-13T04:44:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.773+0000 7f729e7fc700 1 --2- 172.21.15.2:0/1450801934 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f72880a09c0 0x7f72880a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.798+0000 7f729e7fc700 1 --2- 172.21.15.2:0/1450801934 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f72880a09c0 0x7f72880a2e80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55a3b79bdae0 tx=0x7f7294009790 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:26.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.829+0000 7f948ffff700 1 -- 172.21.15.2:0/500657641 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c011830 con 0x7f94a810f440 2023-09-13T04:44:26.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.829+0000 7f948ffff700 1 -- 172.21.15.2:0/500657641 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f949c011e50 con 0x7f94a810f440 2023-09-13T04:44:26.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.829+0000 7f948ffff700 1 -- 172.21.15.2:0/500657641 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c010d20 con 0x7f94a810f440 2023-09-13T04:44:26.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.829+0000 7f94ad5ff700 1 -- 172.21.15.2:0/500657641 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a810fc70 con 0x7f94a810f440 2023-09-13T04:44:26.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.830+0000 7f94ad5ff700 1 -- 172.21.15.2:0/500657641 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a8110130 con 0x7f94a810f440 2023-09-13T04:44:26.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.848+0000 7f94c0f5eb80 1 Processor -- start 2023-09-13T04:44:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.848+0000 7f948ffff700 1 -- 172.21.15.2:0/500657641 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f949c01a400 con 0x7f94a810f440 2023-09-13T04:44:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.851+0000 7f948ffff700 1 --2- 172.21.15.2:0/500657641 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f94900a09c0 0x7f94900a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.851+0000 7f948ffff700 1 -- 172.21.15.2:0/500657641 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f949c0cc9e0 con 0x7f94a810f440 2023-09-13T04:44:26.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.859+0000 7f94c0f5eb80 1 -- start start 2023-09-13T04:44:26.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.859+0000 7f94c0f5eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b55e4c8fe0 0x55b55e4c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:26.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.859+0000 7f94c0f5eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b55e4c9900 0x55b55e4c34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:26.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.859+0000 7f94c0f5eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b55e4c3a00 0x55b55e4c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:26.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.859+0000 7f94c0f5eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55e41c780 con 0x55b55e4c9900 2023-09-13T04:44:26.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.859+0000 7f94c0f5eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55e41a0a0 con 0x55b55e4c3a00 2023-09-13T04:44:26.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.859+0000 7f94c0f5eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55e415990 con 0x55b55e4c8fe0 2023-09-13T04:44:26.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.861+0000 7f94a67fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b55e4c9900 0x55b55e4c34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:26.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.861+0000 7f94a67fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b55e4c9900 0x55b55e4c34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54886/0 (socket says 172.21.15.2:54886) 2023-09-13T04:44:26.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.861+0000 7f94a67fc700 1 -- 172.21.15.2:0/594437342 learned_addr learned my addr 172.21.15.2:0/594437342 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:26.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.861+0000 7f94a77fe700 1 --2- 172.21.15.2:0/594437342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b55e4c3a00 0x55b55e4c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:26.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.861+0000 7f94a77fe700 1 --2- 172.21.15.2:0/500657641 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f94900a09c0 0x7f94900a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:26.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.862+0000 7f94a6ffd700 1 --2- 172.21.15.2:0/594437342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b55e4c8fe0 0x55b55e4c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:26.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.862+0000 7f94a77fe700 1 -- 172.21.15.2:0/594437342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b55e4c8fe0 msgr2=0x55b55e4c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:26.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.862+0000 7f94a77fe700 1 --2- 172.21.15.2:0/594437342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b55e4c8fe0 0x55b55e4c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:26.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.862+0000 7f94a77fe700 1 -- 172.21.15.2:0/594437342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b55e4c9900 msgr2=0x55b55e4c34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:26.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.862+0000 7f94a77fe700 1 --2- 172.21.15.2:0/594437342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b55e4c9900 0x55b55e4c34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:26.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.862+0000 7f94a77fe700 1 -- 172.21.15.2:0/594437342 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b55e295630 con 0x55b55e4c3a00 2023-09-13T04:44:26.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.864+0000 7f94a77fe700 1 --2- 172.21.15.2:0/594437342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b55e4c3a00 0x55b55e4c3e60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9498005fd0 tx=0x7f9498010ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f45dabec921184a8 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:26.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.881+0000 7f94a77fe700 1 --2- 172.21.15.2:0/500657641 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f94900a09c0 0x7f94900a2e80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9498022000 tx=0x7f949800e6a0 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.906+0000 7f728d7fa700 1 -- 172.21.15.2:0/1573657393 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729001d070 con 0x55a3b79c6500 2023-09-13T04:44:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.906+0000 7f728d7fa700 1 -- 172.21.15.2:0/1573657393 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f729000bd10 con 0x55a3b79c6500 2023-09-13T04:44:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.906+0000 7f728d7fa700 1 -- 172.21.15.2:0/1573657393 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729000fcf0 con 0x55a3b79c6500 2023-09-13T04:44:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.907+0000 7f72b90b4b80 1 -- 172.21.15.2:0/1573657393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3b79c6500 msgr2=0x55a3b79c1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.907+0000 7f72b90b4b80 1 --2- 172.21.15.2:0/1573657393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3b79c6500 0x55a3b79c1300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f72900098e0 tx=0x7f7290005c00 comp rx=0 tx=0).stop 2023-09-13T04:44:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.907+0000 7f72b90b4b80 1 -- 172.21.15.2:0/1573657393 shutdown_connections 2023-09-13T04:44:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.907+0000 7f72b90b4b80 1 --2- 172.21.15.2:0/1573657393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3b79c1840 0x55a3b79c1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.907+0000 7f72b90b4b80 1 --2- 172.21.15.2:0/1573657393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c5be0 0x55a3b79c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.907+0000 7f72b90b4b80 1 --2- 172.21.15.2:0/1573657393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3b79c6500 0x55a3b79c1300 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.907+0000 7f72b90b4b80 1 -- 172.21.15.2:0/1573657393 >> 172.21.15.2:0/1573657393 conn(0x55a3b791fc20 msgr2=0x55a3b79bbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.908+0000 7f72b90b4b80 1 -- 172.21.15.2:0/1573657393 shutdown_connections 2023-09-13T04:44:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.908+0000 7f72b90b4b80 1 -- 172.21.15.2:0/1573657393 wait complete. 2023-09-13T04:44:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.909+0000 7f72b90b4b80 1 -- start start 2023-09-13T04:44:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.910+0000 7f72b90b4b80 1 Processor -- start 2023-09-13T04:44:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.911+0000 7f72b90b4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c1840 0x55a3b79ca860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.911+0000 7f72b90b4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3b79c5be0 0x55a3b79cada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.911+0000 7f72b90b4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3b79c6500 0x55a3b79cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.911+0000 7f72b90b4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3b78b1b50 con 0x55a3b79c6500 2023-09-13T04:44:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.911+0000 7f72b90b4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3b78b1850 con 0x55a3b79c5be0 2023-09-13T04:44:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.911+0000 7f72b90b4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3b77958c0 con 0x55a3b79c1840 2023-09-13T04:44:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.911+0000 7f72b90b4b80 10 client.0 _ll_register_callbacks cb 0x55a3b78af620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:44:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.912+0000 7f729effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c1840 0x55a3b79ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.912+0000 7f729e7fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3b79c5be0 0x55a3b79cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.912+0000 7f729effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c1840 0x55a3b79ca860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:47740/0 (socket says 172.21.15.2:47740) 2023-09-13T04:44:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.912+0000 7f729effd700 1 -- 172.21.15.2:0/1994430553 learned_addr learned my addr 172.21.15.2:0/1994430553 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.921+0000 7f729f7fe700 1 --2- 172.21.15.2:0/1994430553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3b79c6500 0x55a3b79cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.921+0000 7f729effd700 1 -- 172.21.15.2:0/1994430553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3b79c5be0 msgr2=0x55a3b79cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.921+0000 7f729effd700 1 --2- 172.21.15.2:0/1994430553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3b79c5be0 0x55a3b79cada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.921+0000 7f729effd700 1 -- 172.21.15.2:0/1994430553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3b79c6500 msgr2=0x55a3b79cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.921+0000 7f729effd700 1 --2- 172.21.15.2:0/1994430553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3b79c6500 0x55a3b79cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.921+0000 7f729effd700 1 -- 172.21.15.2:0/1994430553 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a011d1b0 con 0x55a3b79c1840 2023-09-13T04:44:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.933+0000 7f729effd700 1 --2- 172.21.15.2:0/1994430553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c1840 0x55a3b79ca860 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7290000c00 tx=0x7f729000bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c5cfd29c86f117b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.937+0000 7f72867fc700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:44:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.937+0000 7f72867fc700 1 -- 172.21.15.2:0/1994430553 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729001d070 con 0x55a3b79c1840 2023-09-13T04:44:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.937+0000 7f72867fc700 1 -- 172.21.15.2:0/1994430553 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f729000fa60 con 0x55a3b79c1840 2023-09-13T04:44:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.938+0000 7f72b90b4b80 1 -- 172.21.15.2:0/1994430553 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3b7986ef0 con 0x55a3b79c1840 2023-09-13T04:44:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.961+0000 7f72867fc700 1 -- 172.21.15.2:0/1994430553 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f729000fc40 con 0x55a3b79c1840 2023-09-13T04:44:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.961+0000 7f72b90b4b80 10 client.26287 fetch_fsmap learned FSMap version 30 2023-09-13T04:44:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.961+0000 7f72b90b4b80 1 -- 172.21.15.2:0/1994430553 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a3b792b740 con 0x55a3b79c1840 2023-09-13T04:44:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.972+0000 7f72867fc700 1 -- 172.21.15.2:0/1994430553 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a3b7986ef0 con 0x55a3b79c1840 2023-09-13T04:44:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.972+0000 7f72867fc700 1 -- 172.21.15.2:0/1994430553 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f729002a3f0 con 0x55a3b79c1840 2023-09-13T04:44:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.973+0000 7f72b90b4b80 10 client.26287 fetch_fsmap finished waiting for FSMap version 30 2023-09-13T04:44:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.973+0000 7f72b90b4b80 10 client.26287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:44:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.973+0000 7f72b90b4b80 20 client.26287 populate_metadata read hostname 'smithi002' 2023-09-13T04:44:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.973+0000 7f72b90b4b80 1 --2- 172.21.15.2:0/1994430553 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a3b79daac0 0x55a3b79dcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.973+0000 7f72b90b4b80 4 client.26287 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:44:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.973+0000 7f72b90b4b80 1 -- 172.21.15.2:0/1994430553 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3b790c910 con 0x55a3b79daac0 2023-09-13T04:44:26.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.979+0000 7f729e7fc700 1 --2- 172.21.15.2:0/1994430553 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a3b79daac0 0x55a3b79dcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:26.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.990+0000 7f948d7fa700 1 -- 172.21.15.2:0/594437342 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498022030 con 0x55b55e4c3a00 2023-09-13T04:44:26.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.990+0000 7f948d7fa700 1 -- 172.21.15.2:0/594437342 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f949800fba0 con 0x55b55e4c3a00 2023-09-13T04:44:26.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.990+0000 7f948d7fa700 1 -- 172.21.15.2:0/594437342 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498016070 con 0x55b55e4c3a00 2023-09-13T04:44:26.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.991+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/594437342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b55e4c3a00 msgr2=0x55b55e4c3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:26.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.991+0000 7f94c0f5eb80 1 --2- 172.21.15.2:0/594437342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b55e4c3a00 0x55b55e4c3e60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9498005fd0 tx=0x7f9498010ab0 comp rx=0 tx=0).stop 2023-09-13T04:44:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.992+0000 7f729e7fc700 1 --2- 172.21.15.2:0/1994430553 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a3b79daac0 0x55a3b79dcee0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7559f917767ac7e9 server_cookie=ac0d6286954c0c0d in_seq=0 out_seq=0 2023-09-13T04:44:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.992+0000 7f72867fc700 10 client.26287 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:44:26.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.992+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/594437342 shutdown_connections 2023-09-13T04:44:26.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.992+0000 7f94c0f5eb80 1 --2- 172.21.15.2:0/594437342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b55e4c3a00 0x55b55e4c3e60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:26.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.992+0000 7f94c0f5eb80 1 --2- 172.21.15.2:0/594437342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b55e4c8fe0 0x55b55e4c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:26.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.992+0000 7f94c0f5eb80 1 --2- 172.21.15.2:0/594437342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b55e4c9900 0x55b55e4c34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:26.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:26.992+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/594437342 >> 172.21.15.2:0/594437342 conn(0x55b55e421c80 msgr2=0x55b55e4bdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.994+0000 7f72867fc700 1 -- 172.21.15.2:0/1994430553 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a3b790c910 con 0x55a3b79daac0 2023-09-13T04:44:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.994+0000 7f72867fc700 10 client.26287 handle_command_reply: tid=0 2023-09-13T04:44:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:26.994+0000 7f72b90b4b80 1 client.26287 shutdown 2023-09-13T04:44:27.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.003+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/594437342 shutdown_connections 2023-09-13T04:44:27.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.004+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/594437342 wait complete. 2023-09-13T04:44:27.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.004+0000 7f94c0f5eb80 1 -- start start 2023-09-13T04:44:27.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.005+0000 7f94c0f5eb80 1 Processor -- start 2023-09-13T04:44:27.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.006+0000 7f94c0f5eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b55e4c3a00 0x55b55e4ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:27.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.006+0000 7f94c0f5eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b55e4c8fe0 0x55b55e4cd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:27.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.006+0000 7f94c0f5eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b55e4c9900 0x55b55e4cd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:27.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.006+0000 7f94c0f5eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55e3b3bb0 con 0x55b55e4c8fe0 2023-09-13T04:44:27.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.006+0000 7f94c0f5eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55e3b38b0 con 0x55b55e4c9900 2023-09-13T04:44:27.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.006+0000 7f94c0f5eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55e297920 con 0x55b55e4c3a00 2023-09-13T04:44:27.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.006+0000 7f94c0f5eb80 10 client.0 _ll_register_callbacks cb 0x55b55e3b1680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:44:27.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.007+0000 7f94a77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b55e4c9900 0x55b55e4cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:27.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.007+0000 7f94a77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b55e4c9900 0x55b55e4cd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:37998/0 (socket says 172.21.15.2:37998) 2023-09-13T04:44:27.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.007+0000 7f94a77fe700 1 -- 172.21.15.2:0/1117688521 learned_addr learned my addr 172.21.15.2:0/1117688521 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:27.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.007+0000 7f94a6ffd700 1 --2- 172.21.15.2:0/1117688521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b55e4c3a00 0x55b55e4ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:27.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.008+0000 7f94a67fc700 1 --2- 172.21.15.2:0/1117688521 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b55e4c8fe0 0x55b55e4cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:27.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.009+0000 7f94a6ffd700 1 -- 172.21.15.2:0/1117688521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b55e4c9900 msgr2=0x55b55e4cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:27.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.009+0000 7f94a6ffd700 1 --2- 172.21.15.2:0/1117688521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b55e4c9900 0x55b55e4cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:27.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.009+0000 7f94a6ffd700 1 -- 172.21.15.2:0/1117688521 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b55e4c8fe0 msgr2=0x55b55e4cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:27.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.009+0000 7f94a6ffd700 1 --2- 172.21.15.2:0/1117688521 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b55e4c8fe0 0x55b55e4cd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:27.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.009+0000 7f94a6ffd700 1 -- 172.21.15.2:0/1117688521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9498010430 con 0x55b55e4c3a00 2023-09-13T04:44:27.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.018+0000 7f94a6ffd700 1 --2- 172.21.15.2:0/1117688521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b55e4c3a00 0x55b55e4ccbd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f949c0c9ed0 tx=0x7f949c099610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e19f438bb79da264 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.044+0000 7f72b90b4b80 1 -- 172.21.15.2:0/1994430553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c1840 msgr2=0x55a3b79ca860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.044+0000 7f72b90b4b80 1 --2- 172.21.15.2:0/1994430553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c1840 0x55a3b79ca860 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7290000c00 tx=0x7f729000bc80 comp rx=0 tx=0).stop 2023-09-13T04:44:27.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.044+0000 7f72b90b4b80 1 -- 172.21.15.2:0/1994430553 shutdown_connections 2023-09-13T04:44:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.044+0000 7f72b90b4b80 1 --2- 172.21.15.2:0/1994430553 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a3b79daac0 0x55a3b79dcee0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.044+0000 7f72b90b4b80 1 --2- 172.21.15.2:0/1994430553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3b79c6500 0x55a3b79cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.044+0000 7f72b90b4b80 1 --2- 172.21.15.2:0/1994430553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c1840 0x55a3b79ca860 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.044+0000 7f72b90b4b80 1 --2- 172.21.15.2:0/1994430553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3b79c5be0 0x55a3b79cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.044+0000 7f72b90b4b80 1 -- 172.21.15.2:0/1994430553 >> 172.21.15.2:0/1994430553 conn(0x55a3b791fc20 msgr2=0x55a3b79b6630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.044+0000 7f72867fc700 0 client.26287 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:44:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.045+0000 7f72b90b4b80 1 -- 172.21.15.2:0/1994430553 shutdown_connections 2023-09-13T04:44:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.045+0000 7f72b90b4b80 1 -- 172.21.15.2:0/1994430553 wait complete. 2023-09-13T04:44:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.130+0000 7f72b90b4b80 20 client.26287 trim_cache size 0 max 16384 2023-09-13T04:44:27.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.136+0000 7f948aff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:44:27.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.136+0000 7f948aff5700 1 -- 172.21.15.2:0/1117688521 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c09da60 con 0x55b55e4c3a00 2023-09-13T04:44:27.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.136+0000 7f948aff5700 1 -- 172.21.15.2:0/1117688521 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f949c0a1ca0 con 0x55b55e4c3a00 2023-09-13T04:44:27.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.136+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/1117688521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b55e4b6670 con 0x55b55e4c3a00 2023-09-13T04:44:27.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.140+0000 7f948aff5700 1 -- 172.21.15.2:0/1117688521 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f949c010e80 con 0x55b55e4c3a00 2023-09-13T04:44:27.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.155+0000 7f94c0f5eb80 10 client.26290 fetch_fsmap learned FSMap version 30 2023-09-13T04:44:27.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.155+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/1117688521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b55e42d7a0 con 0x55b55e4c3a00 2023-09-13T04:44:27.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.172+0000 7f948aff5700 1 -- 172.21.15.2:0/1117688521 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b55e4b6670 con 0x55b55e4c3a00 2023-09-13T04:44:27.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.172+0000 7f948aff5700 1 -- 172.21.15.2:0/1117688521 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f949c0a1810 con 0x55b55e4c3a00 2023-09-13T04:44:27.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.172+0000 7f94c0f5eb80 10 client.26290 fetch_fsmap finished waiting for FSMap version 30 2023-09-13T04:44:27.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.172+0000 7f94c0f5eb80 10 client.26290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:44:27.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.172+0000 7f94c0f5eb80 20 client.26290 populate_metadata read hostname 'smithi002' 2023-09-13T04:44:27.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.172+0000 7f94c0f5eb80 1 --2- 172.21.15.2:0/1117688521 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55b55e4dd2a0 0x55b55e4df6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:27.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.172+0000 7f94c0f5eb80 4 client.26290 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:44:27.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.172+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/1117688521 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b55e40e970 con 0x55b55e4dd2a0 2023-09-13T04:44:27.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.182+0000 7f94a67fc700 1 --2- 172.21.15.2:0/1117688521 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55b55e4dd2a0 0x55b55e4df6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:27.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.193+0000 7f94a67fc700 1 --2- 172.21.15.2:0/1117688521 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55b55e4dd2a0 0x55b55e4df6c0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b326d03ba5c968a7 server_cookie=b89002c28b0d1e3a in_seq=0 out_seq=0 2023-09-13T04:44:27.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.194+0000 7f948aff5700 10 client.26290 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:44:27.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.198+0000 7f948aff5700 1 -- 172.21.15.2:0/1117688521 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b55e40e970 con 0x55b55e4dd2a0 2023-09-13T04:44:27.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.198+0000 7f948aff5700 10 client.26290 handle_command_reply: tid=0 2023-09-13T04:44:27.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.207+0000 7f94c0f5eb80 1 client.26290 shutdown 2023-09-13T04:44:27.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.207+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/1117688521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b55e4c3a00 msgr2=0x55b55e4ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:27.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.207+0000 7f94c0f5eb80 1 --2- 172.21.15.2:0/1117688521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b55e4c3a00 0x55b55e4ccbd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f949c0c9ed0 tx=0x7f949c099610 comp rx=0 tx=0).stop 2023-09-13T04:44:27.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.208+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/1117688521 shutdown_connections 2023-09-13T04:44:27.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.208+0000 7f94c0f5eb80 1 --2- 172.21.15.2:0/1117688521 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55b55e4dd2a0 0x55b55e4df6c0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:27.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.208+0000 7f94c0f5eb80 1 --2- 172.21.15.2:0/1117688521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b55e4c9900 0x55b55e4cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:27.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.208+0000 7f94c0f5eb80 1 --2- 172.21.15.2:0/1117688521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b55e4c3a00 0x55b55e4ccbd0 secure :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f949c0c9ed0 tx=0x7f949c099610 comp rx=0 tx=0).stop 2023-09-13T04:44:27.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.208+0000 7f94c0f5eb80 1 --2- 172.21.15.2:0/1117688521 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b55e4c8fe0 0x55b55e4cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:27.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.208+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/1117688521 >> 172.21.15.2:0/1117688521 conn(0x55b55e421c80 msgr2=0x55b55e4b87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:27.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.208+0000 7f948aff5700 0 client.26290 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:44:27.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.227+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/1117688521 shutdown_connections 2023-09-13T04:44:27.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.227+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/1117688521 wait complete. 2023-09-13T04:44:27.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.229+0000 7f94c0f5eb80 20 client.26290 trim_cache size 0 max 16384 2023-09-13T04:44:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.293+0000 7f72b90b4b80 1 Processor -- start 2023-09-13T04:44:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.325+0000 7f72b90b4b80 1 -- start start 2023-09-13T04:44:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.326+0000 7f72b90b4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3b79c1840 0x55a3b79b6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.326+0000 7f72b90b4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c5be0 0x55a3b79d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.326+0000 7f72b90b4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3b79c6500 0x55a3b79cb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.326+0000 7f72b90b4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3b79c6920 con 0x55a3b79c6500 2023-09-13T04:44:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.326+0000 7f72b90b4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3b79bcd90 con 0x55a3b79c1840 2023-09-13T04:44:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.326+0000 7f72b90b4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3b79cbd20 con 0x55a3b79c5be0 2023-09-13T04:44:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.333+0000 7f729e7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c5be0 0x55a3b79d1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.333+0000 7f729e7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c5be0 0x55a3b79d1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:47754/0 (socket says 172.21.15.2:47754) 2023-09-13T04:44:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.333+0000 7f729e7fc700 1 -- 172.21.15.2:0/35881839 learned_addr learned my addr 172.21.15.2:0/35881839 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.333+0000 7f729effd700 1 --2- 172.21.15.2:0/35881839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3b79c1840 0x55a3b79b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.338+0000 7f729effd700 1 -- 172.21.15.2:0/35881839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c5be0 msgr2=0x55a3b79d1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.338+0000 7f729effd700 1 --2- 172.21.15.2:0/35881839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c5be0 0x55a3b79d1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.338+0000 7f729effd700 1 -- 172.21.15.2:0/35881839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3b79c6500 msgr2=0x55a3b79cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.338+0000 7f729effd700 1 --2- 172.21.15.2:0/35881839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3b79c6500 0x55a3b79cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.338+0000 7f729effd700 1 -- 172.21.15.2:0/35881839 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3b773fd80 con 0x55a3b79c1840 2023-09-13T04:44:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.343+0000 7f729effd700 1 --2- 172.21.15.2:0/35881839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3b79c1840 0x55a3b79b6730 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f729000f820 tx=0x7f72900058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28397d538bc0a132 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.432+0000 7f728dffb700 1 -- 172.21.15.2:0/35881839 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729001d020 con 0x55a3b79c1840 2023-09-13T04:44:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.432+0000 7f728dffb700 1 -- 172.21.15.2:0/35881839 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f729002a5f0 con 0x55a3b79c1840 2023-09-13T04:44:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.432+0000 7f728dffb700 1 -- 172.21.15.2:0/35881839 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7290021870 con 0x55a3b79c1840 2023-09-13T04:44:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.433+0000 7f72b90b4b80 1 -- 172.21.15.2:0/35881839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3b79c1840 msgr2=0x55a3b79b6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.433+0000 7f72b90b4b80 1 --2- 172.21.15.2:0/35881839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3b79c1840 0x55a3b79b6730 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f729000f820 tx=0x7f72900058e0 comp rx=0 tx=0).stop 2023-09-13T04:44:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.445+0000 7f72b90b4b80 1 -- 172.21.15.2:0/35881839 shutdown_connections 2023-09-13T04:44:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.445+0000 7f72b90b4b80 1 --2- 172.21.15.2:0/35881839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3b79c6500 0x55a3b79cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.445+0000 7f72b90b4b80 1 --2- 172.21.15.2:0/35881839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3b79c1840 0x55a3b79b6730 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.445+0000 7f72b90b4b80 1 --2- 172.21.15.2:0/35881839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c5be0 0x55a3b79d1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.445+0000 7f72b90b4b80 1 -- 172.21.15.2:0/35881839 >> 172.21.15.2:0/35881839 conn(0x55a3b791fc20 msgr2=0x55a3b79d7250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:27.510 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:27 smithi002 ceph-mon[97651]: pgmap v123: 129 pgs: 129 active+clean; 445 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 2.5 MiB/s wr, 192 op/s 2023-09-13T04:44:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.574+0000 7f72b90b4b80 1 -- 172.21.15.2:0/35881839 shutdown_connections 2023-09-13T04:44:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.575+0000 7f72b90b4b80 1 -- 172.21.15.2:0/35881839 wait complete. 2023-09-13T04:44:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.575+0000 7f72b90b4b80 1 -- start start 2023-09-13T04:44:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.576+0000 7f72b90b4b80 1 Processor -- start 2023-09-13T04:44:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.576+0000 7f72b90b4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3b79c1840 0x55a3b79cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.577+0000 7f72b90b4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c5be0 0x55a3b79ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.577+0000 7f72b90b4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3b79c6500 0x55a3b7a2d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.577+0000 7f72b90b4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3b79ced70 con 0x55a3b79c1840 2023-09-13T04:44:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.577+0000 7f72b90b4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3b7a2dde0 con 0x55a3b79c6500 2023-09-13T04:44:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.577+0000 7f72b90b4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3b7a2df20 con 0x55a3b79c5be0 2023-09-13T04:44:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.577+0000 7f72b90b4b80 10 client.0 _ll_register_callbacks cb 0x55a3b78af620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:44:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.578+0000 7f729e7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c5be0 0x55a3b79ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.578+0000 7f729e7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c5be0 0x55a3b79ce490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:47766/0 (socket says 172.21.15.2:47766) 2023-09-13T04:44:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.578+0000 7f729e7fc700 1 -- 172.21.15.2:0/368859875 learned_addr learned my addr 172.21.15.2:0/368859875 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.578+0000 7f729f7fe700 1 --2- 172.21.15.2:0/368859875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3b79c6500 0x55a3b7a2d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.579+0000 7f729effd700 1 --2- 172.21.15.2:0/368859875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3b79c1840 0x55a3b79cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:27.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.583+0000 7f729e7fc700 1 -- 172.21.15.2:0/368859875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3b79c6500 msgr2=0x55a3b7a2d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:27.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.583+0000 7f729e7fc700 1 --2- 172.21.15.2:0/368859875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3b79c6500 0x55a3b7a2d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.583+0000 7f729e7fc700 1 -- 172.21.15.2:0/368859875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3b79c1840 msgr2=0x55a3b79cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.583+0000 7f729e7fc700 1 --2- 172.21.15.2:0/368859875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3b79c1840 0x55a3b79cdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.584+0000 7f729e7fc700 1 -- 172.21.15.2:0/368859875 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3b792b740 con 0x55a3b79c5be0 2023-09-13T04:44:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.586+0000 7f729e7fc700 1 --2- 172.21.15.2:0/368859875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c5be0 0x55a3b79ce490 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55a3b79bda70 tx=0x7f7294009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=289080282b5a3078 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:27.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:27 smithi149 ceph-mon[133315]: pgmap v123: 129 pgs: 129 active+clean; 445 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 2.5 MiB/s wr, 192 op/s 2023-09-13T04:44:27.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:27 smithi156 ceph-mon[133675]: pgmap v123: 129 pgs: 129 active+clean; 445 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 2.5 MiB/s wr, 192 op/s 2023-09-13T04:44:27.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.775+0000 7f94c0f5eb80 1 Processor -- start 2023-09-13T04:44:27.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.784+0000 7f94c0f5eb80 1 -- start start 2023-09-13T04:44:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.785+0000 7f94c0f5eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b55e4c8fe0 0x55b55e4c34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.785+0000 7f94c0f5eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b55e4c9900 0x55b55e4d42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.785+0000 7f94c0f5eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b55e4c73b0 0x55b55e4b88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.785+0000 7f94c0f5eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55e4cb180 con 0x55b55e4c73b0 2023-09-13T04:44:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.785+0000 7f94c0f5eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55e4d9c60 con 0x55b55e4c8fe0 2023-09-13T04:44:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.785+0000 7f94c0f5eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55e4c7830 con 0x55b55e4c9900 2023-09-13T04:44:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.785+0000 7f94a67fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b55e4c9900 0x55b55e4d42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.785+0000 7f94a6ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b55e4c8fe0 0x55b55e4c34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.786+0000 7f94a67fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b55e4c9900 0x55b55e4d42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:47778/0 (socket says 172.21.15.2:47778) 2023-09-13T04:44:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.786+0000 7f94a67fc700 1 -- 172.21.15.2:0/3628850796 learned_addr learned my addr 172.21.15.2:0/3628850796 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:27.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.792+0000 7f94a77fe700 1 --2- 172.21.15.2:0/3628850796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b55e4c73b0 0x55b55e4b88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.804+0000 7f72867fc700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:44:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.804+0000 7f72867fc700 1 -- 172.21.15.2:0/368859875 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294022040 con 0x55a3b79c5be0 2023-09-13T04:44:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.804+0000 7f72867fc700 1 -- 172.21.15.2:0/368859875 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f7294027510 con 0x55a3b79c5be0 2023-09-13T04:44:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.804+0000 7f72b90b4b80 1 -- 172.21.15.2:0/368859875 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f729000bd10 con 0x55a3b79c5be0 2023-09-13T04:44:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.803+0000 7f94a77fe700 1 -- 172.21.15.2:0/3628850796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b55e4c9900 msgr2=0x55b55e4d42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:27.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.803+0000 7f94a77fe700 1 --2- 172.21.15.2:0/3628850796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b55e4c9900 0x55b55e4d42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:27.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.803+0000 7f94a77fe700 1 -- 172.21.15.2:0/3628850796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b55e4c8fe0 msgr2=0x55b55e4c34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:27.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.803+0000 7f94a77fe700 1 --2- 172.21.15.2:0/3628850796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b55e4c8fe0 0x55b55e4c34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:27.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.803+0000 7f94a77fe700 1 -- 172.21.15.2:0/3628850796 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b55e4be130 con 0x55b55e4c73b0 2023-09-13T04:44:27.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.818+0000 7f94a77fe700 1 --2- 172.21.15.2:0/3628850796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b55e4c73b0 0x55b55e4b88f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f949801d7f0 tx=0x7f9498028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcaa1ebc16ee412 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:27.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.821+0000 7f72867fc700 1 -- 172.21.15.2:0/368859875 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7294027900 con 0x55a3b79c5be0 2023-09-13T04:44:27.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.821+0000 7f72b90b4b80 10 client.26296 fetch_fsmap learned FSMap version 30 2023-09-13T04:44:27.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.821+0000 7f72b90b4b80 1 -- 172.21.15.2:0/368859875 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a3b7728c00 con 0x55a3b79c5be0 2023-09-13T04:44:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.832+0000 7f72867fc700 1 -- 172.21.15.2:0/368859875 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f729000bd10 con 0x55a3b79c5be0 2023-09-13T04:44:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.832+0000 7f72867fc700 1 -- 172.21.15.2:0/368859875 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f729402d020 con 0x55a3b79c5be0 2023-09-13T04:44:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.833+0000 7f72b90b4b80 10 client.26296 fetch_fsmap finished waiting for FSMap version 30 2023-09-13T04:44:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.833+0000 7f72b90b4b80 10 client.26296 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:44:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.833+0000 7f72b90b4b80 20 client.26296 populate_metadata read hostname 'smithi002' 2023-09-13T04:44:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.833+0000 7f72b90b4b80 1 --2- 172.21.15.2:0/368859875 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a3b7a2f5c0 0x55a3b7a319e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.833+0000 7f72b90b4b80 4 client.26296 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:44:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.833+0000 7f72b90b4b80 1 -- 172.21.15.2:0/368859875 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a3b78ccff0 con 0x55a3b7a2f5c0 2023-09-13T04:44:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.833+0000 7f729effd700 1 --2- 172.21.15.2:0/368859875 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a3b7a2f5c0 0x55a3b7a319e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.864+0000 7f729effd700 1 --2- 172.21.15.2:0/368859875 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a3b7a2f5c0 0x55a3b7a319e0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a63b6a6874c5920 server_cookie=eb4a116f6a58dc37 in_seq=0 out_seq=0 2023-09-13T04:44:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:27.864+0000 7f72867fc700 10 client.26296 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:44:27.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.875+0000 7f948dffb700 1 -- 172.21.15.2:0/3628850796 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498016040 con 0x55b55e4c73b0 2023-09-13T04:44:27.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.875+0000 7f948dffb700 1 -- 172.21.15.2:0/3628850796 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9498005bc0 con 0x55b55e4c73b0 2023-09-13T04:44:27.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.875+0000 7f948dffb700 1 -- 172.21.15.2:0/3628850796 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949800edf0 con 0x55b55e4c73b0 2023-09-13T04:44:27.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.878+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/3628850796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b55e4c73b0 msgr2=0x55b55e4b88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:27.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.878+0000 7f94c0f5eb80 1 --2- 172.21.15.2:0/3628850796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b55e4c73b0 0x55b55e4b88f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f949801d7f0 tx=0x7f9498028000 comp rx=0 tx=0).stop 2023-09-13T04:44:27.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.963+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/3628850796 shutdown_connections 2023-09-13T04:44:27.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.963+0000 7f94c0f5eb80 1 --2- 172.21.15.2:0/3628850796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b55e4c73b0 0x55b55e4b88f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:27.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.963+0000 7f94c0f5eb80 1 --2- 172.21.15.2:0/3628850796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b55e4c8fe0 0x55b55e4c34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:27.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.963+0000 7f94c0f5eb80 1 --2- 172.21.15.2:0/3628850796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b55e4c9900 0x55b55e4d42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:27.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.963+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/3628850796 >> 172.21.15.2:0/3628850796 conn(0x55b55e421c80 msgr2=0x55b55e4c6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:27.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.964+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/3628850796 shutdown_connections 2023-09-13T04:44:27.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.966+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/3628850796 wait complete. 2023-09-13T04:44:27.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.966+0000 7f94c0f5eb80 1 -- start start 2023-09-13T04:44:27.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.967+0000 7f94c0f5eb80 1 Processor -- start 2023-09-13T04:44:27.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.967+0000 7f94c0f5eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b55e4c73b0 0x55b55e4c6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:27.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.967+0000 7f94c0f5eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b55e4c8fe0 0x55b55e4c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:27.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.968+0000 7f94c0f5eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b55e4c9900 0x55b55e4c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:27.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.968+0000 7f94c0f5eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55e4c5550 con 0x55b55e4c73b0 2023-09-13T04:44:27.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.968+0000 7f94c0f5eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55e4c56c0 con 0x55b55e4c9900 2023-09-13T04:44:27.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.968+0000 7f94c0f5eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55e4c5880 con 0x55b55e4c8fe0 2023-09-13T04:44:27.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.968+0000 7f94c0f5eb80 10 client.0 _ll_register_callbacks cb 0x55b55e3b1680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:44:27.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.968+0000 7f94a67fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b55e4c8fe0 0x55b55e4c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:27.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.968+0000 7f94a67fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b55e4c8fe0 0x55b55e4c4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:47784/0 (socket says 172.21.15.2:47784) 2023-09-13T04:44:27.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.968+0000 7f94a67fc700 1 -- 172.21.15.2:0/3955705540 learned_addr learned my addr 172.21.15.2:0/3955705540 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:27.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.969+0000 7f94a67fc700 1 -- 172.21.15.2:0/3955705540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b55e4c9900 msgr2=0x55b55e4c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:27.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.969+0000 7f94a67fc700 1 --2- 172.21.15.2:0/3955705540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b55e4c9900 0x55b55e4c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:27.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.969+0000 7f94a67fc700 1 -- 172.21.15.2:0/3955705540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b55e4c73b0 msgr2=0x55b55e4c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:27.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.969+0000 7f94a67fc700 1 --2- 172.21.15.2:0/3955705540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b55e4c73b0 0x55b55e4c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:27.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.969+0000 7f94a67fc700 1 -- 172.21.15.2:0/3955705540 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b55e295630 con 0x55b55e4c8fe0 2023-09-13T04:44:27.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:27.978+0000 7f94a67fc700 1 --2- 172.21.15.2:0/3955705540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b55e4c8fe0 0x55b55e4c4a60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009040 tx=0x7f94a0000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c5ad5a7ed98825d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.086+0000 7f948aff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:44:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.087+0000 7f948aff5700 1 -- 172.21.15.2:0/3955705540 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a0022040 con 0x55b55e4c8fe0 2023-09-13T04:44:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.087+0000 7f948aff5700 1 -- 172.21.15.2:0/3955705540 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f94a000b9f0 con 0x55b55e4c8fe0 2023-09-13T04:44:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.087+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/3955705540 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f949800fba0 con 0x55b55e4c8fe0 2023-09-13T04:44:28.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.098+0000 7f948aff5700 1 -- 172.21.15.2:0/3955705540 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94a000e040 con 0x55b55e4c8fe0 2023-09-13T04:44:28.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.098+0000 7f94c0f5eb80 10 client.26302 fetch_fsmap learned FSMap version 30 2023-09-13T04:44:28.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.098+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/3955705540 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b55e4e42b0 con 0x55b55e4c8fe0 2023-09-13T04:44:28.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.105+0000 7f948aff5700 1 -- 172.21.15.2:0/3955705540 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f949800fba0 con 0x55b55e4c8fe0 2023-09-13T04:44:28.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.105+0000 7f948aff5700 1 -- 172.21.15.2:0/3955705540 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f94a000b050 con 0x55b55e4c8fe0 2023-09-13T04:44:28.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.106+0000 7f94c0f5eb80 10 client.26302 fetch_fsmap finished waiting for FSMap version 30 2023-09-13T04:44:28.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.106+0000 7f94c0f5eb80 10 client.26302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:44:28.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.106+0000 7f94c0f5eb80 20 client.26302 populate_metadata read hostname 'smithi002' 2023-09-13T04:44:28.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.106+0000 7f94c0f5eb80 1 --2- 172.21.15.2:0/3955705540 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55b55e532660 0x55b55e534aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:28.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.106+0000 7f94c0f5eb80 4 client.26302 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-09-13T04:44:28.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.106+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/3955705540 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b55e3cf050 con 0x55b55e532660 2023-09-13T04:44:28.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.107+0000 7f94a6ffd700 1 --2- 172.21.15.2:0/3955705540 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55b55e532660 0x55b55e534aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:28.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.153+0000 7f94a6ffd700 1 --2- 172.21.15.2:0/3955705540 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55b55e532660 0x55b55e534aa0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46c32f28d58eaa63 server_cookie=d0903cc418cd4314 in_seq=0 out_seq=0 2023-09-13T04:44:28.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.159+0000 7f948aff5700 10 client.26302 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:44:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:28.915+0000 7f72867fc700 1 -- 172.21.15.2:0/368859875 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a3b78ccff0 con 0x55a3b7a2f5c0 2023-09-13T04:44:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:28.915+0000 7f72867fc700 10 client.26296 handle_command_reply: tid=0 2023-09-13T04:44:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:28.915+0000 7f72b90b4b80 1 client.26296 shutdown 2023-09-13T04:44:28.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.923+0000 7f948aff5700 1 -- 172.21.15.2:0/3955705540 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b55e3cf050 con 0x55b55e532660 2023-09-13T04:44:28.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.923+0000 7f948aff5700 10 client.26302 handle_command_reply: tid=0 2023-09-13T04:44:28.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.923+0000 7f94c0f5eb80 1 client.26302 shutdown 2023-09-13T04:44:28.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.924+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/3955705540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b55e4c8fe0 msgr2=0x55b55e4c4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:28.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.924+0000 7f94c0f5eb80 1 --2- 172.21.15.2:0/3955705540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b55e4c8fe0 0x55b55e4c4a60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009040 tx=0x7f94a0000f40 comp rx=0 tx=0).stop 2023-09-13T04:44:28.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.924+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/3955705540 shutdown_connections 2023-09-13T04:44:28.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.924+0000 7f94c0f5eb80 1 --2- 172.21.15.2:0/3955705540 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55b55e532660 0x55b55e534aa0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:28.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.924+0000 7f94c0f5eb80 1 --2- 172.21.15.2:0/3955705540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b55e4c9900 0x55b55e4c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:28.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.924+0000 7f94c0f5eb80 1 --2- 172.21.15.2:0/3955705540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b55e4c73b0 0x55b55e4c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:28.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.924+0000 7f94c0f5eb80 1 --2- 172.21.15.2:0/3955705540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b55e4c8fe0 0x55b55e4c4a60 secure :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009040 tx=0x7f94a0000f40 comp rx=0 tx=0).stop 2023-09-13T04:44:28.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.924+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/3955705540 >> 172.21.15.2:0/3955705540 conn(0x55b55e421c80 msgr2=0x55b55e4c7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:28.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.924+0000 7f948aff5700 0 client.26302 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:44:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:28.927+0000 7f72b90b4b80 1 -- 172.21.15.2:0/368859875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c5be0 msgr2=0x55a3b79ce490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:28.927+0000 7f72b90b4b80 1 --2- 172.21.15.2:0/368859875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c5be0 0x55a3b79ce490 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55a3b79bda70 tx=0x7f7294009260 comp rx=0 tx=0).stop 2023-09-13T04:44:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:28.927+0000 7f72b90b4b80 1 -- 172.21.15.2:0/368859875 shutdown_connections 2023-09-13T04:44:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:28.927+0000 7f72b90b4b80 1 --2- 172.21.15.2:0/368859875 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a3b7a2f5c0 0x55a3b7a319e0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:28.927+0000 7f72b90b4b80 1 --2- 172.21.15.2:0/368859875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3b79c6500 0x55a3b7a2d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:28.927+0000 7f72b90b4b80 1 --2- 172.21.15.2:0/368859875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3b79c1840 0x55a3b79cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:28.927+0000 7f72b90b4b80 1 --2- 172.21.15.2:0/368859875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3b79c5be0 0x55a3b79ce490 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:28.927+0000 7f72b90b4b80 1 -- 172.21.15.2:0/368859875 >> 172.21.15.2:0/368859875 conn(0x55a3b791fc20 msgr2=0x55a3b79c4740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:28.928+0000 7f72867fc700 0 client.26296 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:44:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:28.928+0000 7f72b90b4b80 1 -- 172.21.15.2:0/368859875 shutdown_connections 2023-09-13T04:44:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:28.928+0000 7f72b90b4b80 1 -- 172.21.15.2:0/368859875 wait complete. 2023-09-13T04:44:28.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.930+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/3955705540 shutdown_connections 2023-09-13T04:44:28.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.930+0000 7f94c0f5eb80 1 -- 172.21.15.2:0/3955705540 wait complete. 2023-09-13T04:44:28.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:28.930+0000 7f94c0f5eb80 20 client.26302 trim_cache size 0 max 16384 2023-09-13T04:44:28.934 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:44:28.935 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-13T04:44:28.935 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-09-13T04:44:28.935 INFO:teuthology.orchestra.run.smithi002.stdout: "740235c0-6e1b-4729-b1fe-33345ae948c1": { 2023-09-13T04:44:28.935 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-09-13T04:44:28.936 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "740235c0-6e1b-4729-b1fe-33345ae948c1", 2023-09-13T04:44:28.936 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-09-13T04:44:28.936 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:44:28.936 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:44:28.936 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:44:28.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:44:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:44:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:44:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:44:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:28.937+0000 7f72b90b4b80 20 client.26296 trim_cache size 0 max 16384 2023-09-13T04:44:29.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:29.122+0000 7f94ad5ff700 1 -- 172.21.15.2:0/500657641 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f94900a09c0 msgr2=0x7f94900a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:29.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:29.122+0000 7f94ad5ff700 1 --2- 172.21.15.2:0/500657641 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f94900a09c0 0x7f94900a2e80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9498022000 tx=0x7f949800e6a0 comp rx=0 tx=0).stop 2023-09-13T04:44:29.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:29.122+0000 7f94ad5ff700 1 -- 172.21.15.2:0/500657641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94a810f440 msgr2=0x7f94a81a5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:29.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:29.122+0000 7f94ad5ff700 1 --2- 172.21.15.2:0/500657641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94a810f440 0x7f94a81a5f50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f949c006fd0 tx=0x7f949c00b730 comp rx=0 tx=0).stop 2023-09-13T04:44:29.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:29.130+0000 7f94ad5ff700 1 -- 172.21.15.2:0/500657641 shutdown_connections 2023-09-13T04:44:29.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:29.130+0000 7f94ad5ff700 1 --2- 172.21.15.2:0/500657641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94a81aa8f0 0x7f94a81aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:29.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:29.130+0000 7f94ad5ff700 1 --2- 172.21.15.2:0/500657641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94a810f440 0x7f94a81a5f50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:29.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:29.130+0000 7f94ad5ff700 1 --2- 172.21.15.2:0/500657641 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f94900a09c0 0x7f94900a2e80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:29.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:29.130+0000 7f94ad5ff700 1 --2- 172.21.15.2:0/500657641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94a811a490 0x7f94a81a6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:29.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:29.130+0000 7f94ad5ff700 1 -- 172.21.15.2:0/500657641 >> 172.21.15.2:0/500657641 conn(0x7f94a806cfa0 msgr2=0x7f94a8118480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:29.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:29.131+0000 7f94ad5ff700 1 -- 172.21.15.2:0/500657641 shutdown_connections 2023-09-13T04:44:29.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:44:29.131+0000 7f94ad5ff700 1 -- 172.21.15.2:0/500657641 wait complete. 2023-09-13T04:44:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:29.179+0000 7f72a57ff700 1 -- 172.21.15.2:0/1450801934 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f72880a09c0 msgr2=0x7f72880a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:29.179+0000 7f72a57ff700 1 --2- 172.21.15.2:0/1450801934 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f72880a09c0 0x7f72880a2e80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55a3b79bdae0 tx=0x7f7294009790 comp rx=0 tx=0).stop 2023-09-13T04:44:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:29.179+0000 7f72a57ff700 1 -- 172.21.15.2:0/1450801934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72a011a490 msgr2=0x7f72a0111670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:29.179+0000 7f72a57ff700 1 --2- 172.21.15.2:0/1450801934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72a011a490 0x7f72a0111670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f729800b9e0 tx=0x7f729800bda0 comp rx=0 tx=0).stop 2023-09-13T04:44:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:29.228+0000 7f72a57ff700 1 -- 172.21.15.2:0/1450801934 shutdown_connections 2023-09-13T04:44:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:29.228+0000 7f72a57ff700 1 --2- 172.21.15.2:0/1450801934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72a011a490 0x7f72a0111670 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:29.228+0000 7f72a57ff700 1 --2- 172.21.15.2:0/1450801934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72a010f420 0x7f72a01187b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:29.228+0000 7f72a57ff700 1 --2- 172.21.15.2:0/1450801934 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f72880a09c0 0x7f72880a2e80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:29.228+0000 7f72a57ff700 1 --2- 172.21.15.2:0/1450801934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72a010f890 0x7f72a0118d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:29.228+0000 7f72a57ff700 1 -- 172.21.15.2:0/1450801934 >> 172.21.15.2:0/1450801934 conn(0x7f72a006cda0 msgr2=0x7f72a006f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:29.232+0000 7f72a57ff700 1 -- 172.21.15.2:0/1450801934 shutdown_connections 2023-09-13T04:44:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:29.232+0000 7f72a57ff700 1 -- 172.21.15.2:0/1450801934 wait complete. 2023-09-13T04:44:29.430 INFO:tasks.cephfs.filesystem:scrub status for tag:740235c0-6e1b-4729-b1fe-33345ae948c1 - {'path': '/', 'tag': '740235c0-6e1b-4729-b1fe-33345ae948c1', 'options': 'recursive,force'} 2023-09-13T04:44:29.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:29 smithi149 ceph-mon[133315]: pgmap v124: 129 pgs: 129 active+clean; 445 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 2.4 MiB/s wr, 177 op/s 2023-09-13T04:44:29.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:29 smithi156 ceph-mon[133675]: pgmap v124: 129 pgs: 129 active+clean; 445 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 2.4 MiB/s wr, 177 op/s 2023-09-13T04:44:29.760 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:29 smithi002 ceph-mon[97651]: pgmap v124: 129 pgs: 129 active+clean; 445 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 2.4 MiB/s wr, 177 op/s 2023-09-13T04:44:30.652 INFO:tasks.workunit.client.0.smithi002.stdout: 97 717791 288 534886 210 271258 357 2023-09-13T04:44:30.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:30 smithi149 ceph-mon[133315]: pgmap v125: 129 pgs: 129 active+clean; 445 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 404 op/s 2023-09-13T04:44:30.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:30 smithi156 ceph-mon[133675]: pgmap v125: 129 pgs: 129 active+clean; 445 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 404 op/s 2023-09-13T04:44:31.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:30 smithi002 ceph-mon[97651]: pgmap v125: 129 pgs: 129 active+clean; 445 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 404 op/s 2023-09-13T04:44:33.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:33 smithi002 ceph-mon[97651]: pgmap v126: 129 pgs: 129 active+clean; 445 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 754 KiB/s wr, 270 op/s 2023-09-13T04:44:33.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:33 smithi149 ceph-mon[133315]: pgmap v126: 129 pgs: 129 active+clean; 445 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 754 KiB/s wr, 270 op/s 2023-09-13T04:44:33.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:33 smithi156 ceph-mon[133675]: pgmap v126: 129 pgs: 129 active+clean; 445 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 754 KiB/s wr, 270 op/s 2023-09-13T04:44:35.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:35 smithi149 ceph-mon[133315]: pgmap v127: 129 pgs: 129 active+clean; 463 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 374 op/s 2023-09-13T04:44:35.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:35 smithi156 ceph-mon[133675]: pgmap v127: 129 pgs: 129 active+clean; 463 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 374 op/s 2023-09-13T04:44:35.761 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:35 smithi002 ceph-mon[97651]: pgmap v127: 129 pgs: 129 active+clean; 463 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 374 op/s 2023-09-13T04:44:37.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:37 smithi149 ceph-mon[133315]: pgmap v128: 129 pgs: 129 active+clean; 465 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 2.3 MiB/s wr, 351 op/s 2023-09-13T04:44:37.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:37 smithi156 ceph-mon[133675]: pgmap v128: 129 pgs: 129 active+clean; 465 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 2.3 MiB/s wr, 351 op/s 2023-09-13T04:44:37.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:37 smithi002 ceph-mon[97651]: pgmap v128: 129 pgs: 129 active+clean; 465 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 2.3 MiB/s wr, 351 op/s 2023-09-13T04:44:39.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:39 smithi149 ceph-mon[133315]: pgmap v129: 129 pgs: 129 active+clean; 465 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.1 MiB/s wr, 341 op/s 2023-09-13T04:44:39.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:39 smithi156 ceph-mon[133675]: pgmap v129: 129 pgs: 129 active+clean; 465 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.1 MiB/s wr, 341 op/s 2023-09-13T04:44:39.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:39 smithi002 ceph-mon[97651]: pgmap v129: 129 pgs: 129 active+clean; 465 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.1 MiB/s wr, 341 op/s 2023-09-13T04:44:40.652 INFO:tasks.workunit.client.0.smithi002.stdout: 101 683450 223 505993 221 267498 415 2023-09-13T04:44:40.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:40 smithi149 ceph-mon[133315]: pgmap v130: 129 pgs: 129 active+clean; 479 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 464 op/s 2023-09-13T04:44:40.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:40 smithi156 ceph-mon[133675]: pgmap v130: 129 pgs: 129 active+clean; 479 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 464 op/s 2023-09-13T04:44:41.030 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:40 smithi002 ceph-mon[97651]: pgmap v130: 129 pgs: 129 active+clean; 479 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 464 op/s 2023-09-13T04:44:43.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:43 smithi149 ceph-mon[133315]: pgmap v131: 129 pgs: 129 active+clean; 479 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 3.0 MiB/s wr, 237 op/s 2023-09-13T04:44:43.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:43 smithi156 ceph-mon[133675]: pgmap v131: 129 pgs: 129 active+clean; 479 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 3.0 MiB/s wr, 237 op/s 2023-09-13T04:44:43.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:43 smithi002 ceph-mon[97651]: pgmap v131: 129 pgs: 129 active+clean; 479 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 3.0 MiB/s wr, 237 op/s 2023-09-13T04:44:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.237+0000 7f77e97ff700 1 -- 172.21.15.2:0/104836464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77e411a490 msgr2=0x7f77e411c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.237+0000 7f77e97ff700 1 --2- 172.21.15.2:0/104836464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77e411a490 0x7f77e411c880 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f77d80098f0 tx=0x7f77d8009c00 comp rx=0 tx=0).stop 2023-09-13T04:44:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.238+0000 7f77e97ff700 1 -- 172.21.15.2:0/104836464 shutdown_connections 2023-09-13T04:44:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.238+0000 7f77e97ff700 1 --2- 172.21.15.2:0/104836464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77e411a490 0x7f77e411c880 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.238+0000 7f77e97ff700 1 --2- 172.21.15.2:0/104836464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77e410f420 0x7f77e4107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.238+0000 7f77e97ff700 1 --2- 172.21.15.2:0/104836464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77e410f890 0x7f77e41081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.238+0000 7f77e97ff700 1 -- 172.21.15.2:0/104836464 >> 172.21.15.2:0/104836464 conn(0x7f77e406cda0 msgr2=0x7f77e406d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.274+0000 7f77e97ff700 1 -- 172.21.15.2:0/104836464 shutdown_connections 2023-09-13T04:44:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.287+0000 7f77e97ff700 1 -- 172.21.15.2:0/104836464 wait complete. 2023-09-13T04:44:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.288+0000 7f77e97ff700 1 Processor -- start 2023-09-13T04:44:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.289+0000 7f77e97ff700 1 -- start start 2023-09-13T04:44:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.289+0000 7f77e97ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77e410f890 0x7f77e41a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.289+0000 7f77e97ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77e411a490 0x7f77e41a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.289+0000 7f77e97ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77e41aa990 0x7f77e41aae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.289+0000 7f77e97ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e41ab410 con 0x7f77e41aa990 2023-09-13T04:44:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.289+0000 7f77e97ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e41ab550 con 0x7f77e411a490 2023-09-13T04:44:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.289+0000 7f77e97ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e41ab6c0 con 0x7f77e410f890 2023-09-13T04:44:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.298+0000 7f77e37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77e41aa990 0x7f77e41aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.298+0000 7f77e37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77e41aa990 0x7f77e41aae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37768/0 (socket says 172.21.15.2:37768) 2023-09-13T04:44:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.298+0000 7f77e37fe700 1 -- 172.21.15.2:0/1341911145 learned_addr learned my addr 172.21.15.2:0/1341911145 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.298+0000 7f77e2ffd700 1 --2- 172.21.15.2:0/1341911145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77e410f890 0x7f77e41a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.306+0000 7f77e27fc700 1 --2- 172.21.15.2:0/1341911145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77e411a490 0x7f77e41a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.319+0000 7f77e2ffd700 1 -- 172.21.15.2:0/1341911145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77e411a490 msgr2=0x7f77e41a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.319+0000 7f77e2ffd700 1 --2- 172.21.15.2:0/1341911145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77e411a490 0x7f77e41a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.319+0000 7f77e2ffd700 1 -- 172.21.15.2:0/1341911145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77e41aa990 msgr2=0x7f77e41aae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.319+0000 7f77e2ffd700 1 --2- 172.21.15.2:0/1341911145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77e41aa990 0x7f77e41aae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.319+0000 7f77e2ffd700 1 -- 172.21.15.2:0/1341911145 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d8009520 con 0x7f77e410f890 2023-09-13T04:44:44.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.321+0000 7f77e2ffd700 1 --2- 172.21.15.2:0/1341911145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77e410f890 0x7f77e41a5de0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f77e406d9a0 tx=0x7f77dc009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:44.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.321+0000 7f77cbfff700 1 -- 172.21.15.2:0/1341911145 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc00cc90 con 0x7f77e410f890 2023-09-13T04:44:44.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.321+0000 7f77cbfff700 1 -- 172.21.15.2:0/1341911145 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77dc00cdf0 con 0x7f77e410f890 2023-09-13T04:44:44.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.321+0000 7f77cbfff700 1 -- 172.21.15.2:0/1341911145 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc0127a0 con 0x7f77e410f890 2023-09-13T04:44:44.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.321+0000 7f77e97ff700 1 -- 172.21.15.2:0/1341911145 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e410fc70 con 0x7f77e410f890 2023-09-13T04:44:44.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.321+0000 7f77e97ff700 1 -- 172.21.15.2:0/1341911145 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e4110130 con 0x7f77e410f890 2023-09-13T04:44:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.329+0000 7f77fd130b80 1 Processor -- start 2023-09-13T04:44:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.348+0000 7f77cbfff700 1 -- 172.21.15.2:0/1341911145 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f77dc002020 con 0x7f77e410f890 2023-09-13T04:44:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.349+0000 7f77fd130b80 1 -- start start 2023-09-13T04:44:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.349+0000 7f77fd130b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd86be0 0x55c6cbd86fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.350+0000 7f77fd130b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6cbd87500 0x55c6cbd82300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.350+0000 7f77fd130b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6cbd82840 0x55c6cbd82c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.350+0000 7f77fd130b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6cbcdb720 con 0x55c6cbd82840 2023-09-13T04:44:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.350+0000 7f77fd130b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6cbcd9040 con 0x55c6cbd87500 2023-09-13T04:44:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.350+0000 7f77fd130b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6cbcd4930 con 0x55c6cbd86be0 2023-09-13T04:44:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.350+0000 7f77e37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd86be0 0x55c6cbd86fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.350+0000 7f77e37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd86be0 0x55c6cbd86fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:51248/0 (socket says 172.21.15.2:51248) 2023-09-13T04:44:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.350+0000 7f77e37fe700 1 -- 172.21.15.2:0/2470052012 learned_addr learned my addr 172.21.15.2:0/2470052012 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.351+0000 7f77e2ffd700 1 --2- 172.21.15.2:0/2470052012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6cbd87500 0x55c6cbd82300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.351+0000 7f77e37fe700 1 -- 172.21.15.2:0/2470052012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6cbd87500 msgr2=0x55c6cbd82300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.351+0000 7f77e37fe700 1 --2- 172.21.15.2:0/2470052012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6cbd87500 0x55c6cbd82300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.351+0000 7f77e37fe700 1 -- 172.21.15.2:0/2470052012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6cbd82840 msgr2=0x55c6cbd82c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:44:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.351+0000 7f77e37fe700 1 --2- 172.21.15.2:0/2470052012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6cbd82840 0x55c6cbd82c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.351+0000 7f77e37fe700 1 -- 172.21.15.2:0/2470052012 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6cbb545d0 con 0x55c6cbd86be0 2023-09-13T04:44:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.354+0000 7f77cbfff700 1 --2- 172.21.15.2:0/1341911145 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f77cc0a09c0 0x7f77cc0a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.354+0000 7f77cbfff700 1 -- 172.21.15.2:0/1341911145 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f77dc0cb8a0 con 0x7f77e410f890 2023-09-13T04:44:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.361+0000 7f77e2ffd700 1 --2- 172.21.15.2:0/1341911145 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f77cc0a09c0 0x7f77cc0a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.377+0000 7f77e2ffd700 1 --2- 172.21.15.2:0/1341911145 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f77cc0a09c0 0x7f77cc0a2e80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55c6cbd87900 tx=0x7f77dc00e040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.396+0000 7f77e37fe700 1 --2- 172.21.15.2:0/2470052012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd86be0 0x55c6cbd86fc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f77d800f510 tx=0x7f77d800f540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40e12397f64c1970 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.494+0000 7f77c97fa700 1 -- 172.21.15.2:0/2470052012 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d801d070 con 0x55c6cbd86be0 2023-09-13T04:44:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.494+0000 7f77c97fa700 1 -- 172.21.15.2:0/2470052012 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77d800f7d0 con 0x55c6cbd86be0 2023-09-13T04:44:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.494+0000 7f77c97fa700 1 -- 172.21.15.2:0/2470052012 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d80055e0 con 0x55c6cbd86be0 2023-09-13T04:44:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.495+0000 7f77fd130b80 1 -- 172.21.15.2:0/2470052012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd86be0 msgr2=0x55c6cbd86fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.495+0000 7f77fd130b80 1 --2- 172.21.15.2:0/2470052012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd86be0 0x55c6cbd86fc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f77d800f510 tx=0x7f77d800f540 comp rx=0 tx=0).stop 2023-09-13T04:44:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.496+0000 7f77fd130b80 1 -- 172.21.15.2:0/2470052012 shutdown_connections 2023-09-13T04:44:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.496+0000 7f77fd130b80 1 --2- 172.21.15.2:0/2470052012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6cbd82840 0x55c6cbd82c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.496+0000 7f77fd130b80 1 --2- 172.21.15.2:0/2470052012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd86be0 0x55c6cbd86fc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.496+0000 7f77fd130b80 1 --2- 172.21.15.2:0/2470052012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6cbd87500 0x55c6cbd82300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.496+0000 7f77fd130b80 1 -- 172.21.15.2:0/2470052012 >> 172.21.15.2:0/2470052012 conn(0x55c6cbce0c20 msgr2=0x55c6cbd7cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.497+0000 7f77fd130b80 1 -- 172.21.15.2:0/2470052012 shutdown_connections 2023-09-13T04:44:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.497+0000 7f77fd130b80 1 -- 172.21.15.2:0/2470052012 wait complete. 2023-09-13T04:44:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.497+0000 7f77fd130b80 1 -- start start 2023-09-13T04:44:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.498+0000 7f77fd130b80 1 Processor -- start 2023-09-13T04:44:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.499+0000 7f77fd130b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6cbd82840 0x55c6cbd8b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.499+0000 7f77fd130b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6cbd86be0 0x55c6cbd8bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.499+0000 7f77fd130b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd87500 0x55c6cbd8c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.499+0000 7f77fd130b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6cbc72b50 con 0x55c6cbd82840 2023-09-13T04:44:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.499+0000 7f77fd130b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6cbc72850 con 0x55c6cbd86be0 2023-09-13T04:44:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.499+0000 7f77fd130b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6cbb568c0 con 0x55c6cbd87500 2023-09-13T04:44:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.500+0000 7f77fd130b80 10 client.0 _ll_register_callbacks cb 0x55c6cbc70620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:44:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.504+0000 7f77e27fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6cbd86be0 0x55c6cbd8bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.504+0000 7f77e37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd87500 0x55c6cbd8c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.504+0000 7f77e37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd87500 0x55c6cbd8c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:51258/0 (socket says 172.21.15.2:51258) 2023-09-13T04:44:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.504+0000 7f77e37fe700 1 -- 172.21.15.2:0/3838145060 learned_addr learned my addr 172.21.15.2:0/3838145060 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.515+0000 7f77e37fe700 1 -- 172.21.15.2:0/3838145060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6cbd86be0 msgr2=0x55c6cbd8bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.515+0000 7f77e37fe700 1 --2- 172.21.15.2:0/3838145060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6cbd86be0 0x55c6cbd8bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.515+0000 7f77e37fe700 1 -- 172.21.15.2:0/3838145060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6cbd82840 msgr2=0x55c6cbd8b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.515+0000 7f77e37fe700 1 --2- 172.21.15.2:0/3838145060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6cbd82840 0x55c6cbd8b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.515+0000 7f77e37fe700 1 -- 172.21.15.2:0/3838145060 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e411d1b0 con 0x55c6cbd87500 2023-09-13T04:44:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.515+0000 7f77e37fe700 1 --2- 172.21.15.2:0/3838145060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd87500 0x55c6cbd8c320 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f77d800f660 tx=0x7f77d8005bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=defc938bb486bfb0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.516+0000 7f77c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:44:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.516+0000 7f77c6ff5700 1 -- 172.21.15.2:0/3838145060 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d801d040 con 0x55c6cbd87500 2023-09-13T04:44:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.516+0000 7f77c6ff5700 1 -- 172.21.15.2:0/3838145060 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f77d8005610 con 0x55c6cbd87500 2023-09-13T04:44:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.516+0000 7f77fd130b80 1 -- 172.21.15.2:0/3838145060 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6cbd47ef0 con 0x55c6cbd87500 2023-09-13T04:44:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.527+0000 7f77c6ff5700 1 -- 172.21.15.2:0/3838145060 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77d8003680 con 0x55c6cbd87500 2023-09-13T04:44:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.527+0000 7f77fd130b80 10 client.26314 fetch_fsmap learned FSMap version 30 2023-09-13T04:44:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.527+0000 7f77fd130b80 1 -- 172.21.15.2:0/3838145060 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c6cbcec740 con 0x55c6cbd87500 2023-09-13T04:44:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.535+0000 7f77c6ff5700 1 -- 172.21.15.2:0/3838145060 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c6cbd47ef0 con 0x55c6cbd87500 2023-09-13T04:44:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.535+0000 7f77c6ff5700 1 -- 172.21.15.2:0/3838145060 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f77d8015020 con 0x55c6cbd87500 2023-09-13T04:44:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.536+0000 7f77fd130b80 10 client.26314 fetch_fsmap finished waiting for FSMap version 30 2023-09-13T04:44:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.536+0000 7f77fd130b80 10 client.26314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:44:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.536+0000 7f77fd130b80 20 client.26314 populate_metadata read hostname 'smithi002' 2023-09-13T04:44:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.537+0000 7f77fd130b80 1 --2- 172.21.15.2:0/3838145060 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55c6cbd9bb00 0x55c6cbd9df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.537+0000 7f77fd130b80 4 client.26314 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:44:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.537+0000 7f77fd130b80 1 -- 172.21.15.2:0/3838145060 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6cbccd910 con 0x55c6cbd9bb00 2023-09-13T04:44:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.540+0000 7f77e27fc700 1 --2- 172.21.15.2:0/3838145060 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55c6cbd9bb00 0x55c6cbd9df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.564+0000 7f77e27fc700 1 --2- 172.21.15.2:0/3838145060 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55c6cbd9bb00 0x55c6cbd9df20 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76a7653a7531f45d server_cookie=41ef96e3297910d9 in_seq=0 out_seq=0 2023-09-13T04:44:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.564+0000 7f77c6ff5700 10 client.26314 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:44:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.580+0000 7f77c6ff5700 1 -- 172.21.15.2:0/3838145060 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c6cbccd910 con 0x55c6cbd9bb00 2023-09-13T04:44:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.580+0000 7f77c6ff5700 10 client.26314 handle_command_reply: tid=0 2023-09-13T04:44:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.580+0000 7f77fd130b80 1 client.26314 shutdown 2023-09-13T04:44:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.589+0000 7f77fd130b80 1 -- 172.21.15.2:0/3838145060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd87500 msgr2=0x55c6cbd8c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.589+0000 7f77fd130b80 1 --2- 172.21.15.2:0/3838145060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd87500 0x55c6cbd8c320 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f77d800f660 tx=0x7f77d8005bb0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.590+0000 7f77fd130b80 1 -- 172.21.15.2:0/3838145060 shutdown_connections 2023-09-13T04:44:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.590+0000 7f77fd130b80 1 --2- 172.21.15.2:0/3838145060 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55c6cbd9bb00 0x55c6cbd9df20 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.590+0000 7f77fd130b80 1 --2- 172.21.15.2:0/3838145060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd87500 0x55c6cbd8c320 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.590+0000 7f77fd130b80 1 --2- 172.21.15.2:0/3838145060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6cbd82840 0x55c6cbd8b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.590+0000 7f77fd130b80 1 --2- 172.21.15.2:0/3838145060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6cbd86be0 0x55c6cbd8bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.590+0000 7f77fd130b80 1 -- 172.21.15.2:0/3838145060 >> 172.21.15.2:0/3838145060 conn(0x55c6cbce0c20 msgr2=0x55c6cbd77630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.590+0000 7f77c6ff5700 0 client.26314 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:44:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.590+0000 7f77fd130b80 1 -- 172.21.15.2:0/3838145060 shutdown_connections 2023-09-13T04:44:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.590+0000 7f77fd130b80 1 -- 172.21.15.2:0/3838145060 wait complete. 2023-09-13T04:44:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.599+0000 7f77fd130b80 20 client.26314 trim_cache size 0 max 16384 2023-09-13T04:44:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.671+0000 7f77fd130b80 1 Processor -- start 2023-09-13T04:44:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.672+0000 7f77fd130b80 1 -- start start 2023-09-13T04:44:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.673+0000 7f77fd130b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd82840 0x55c6cbd92f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.673+0000 7f77fd130b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6cbd86be0 0x55c6cbd77730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:44.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.673+0000 7f77fd130b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6cbd87500 0x55c6cbd8b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:44.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.673+0000 7f77fd130b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6cbd8e950 con 0x55c6cbd87500 2023-09-13T04:44:44.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.673+0000 7f77fd130b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6cbd934b0 con 0x55c6cbd86be0 2023-09-13T04:44:44.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.673+0000 7f77fd130b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6cbd89e20 con 0x55c6cbd82840 2023-09-13T04:44:44.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.674+0000 7f77e37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6cbd87500 0x55c6cbd8b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:44.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.674+0000 7f77e37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd82840 0x55c6cbd92f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.679+0000 7f77e37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd82840 0x55c6cbd92f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:51266/0 (socket says 172.21.15.2:51266) 2023-09-13T04:44:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.679+0000 7f77e37fe700 1 -- 172.21.15.2:0/584750203 learned_addr learned my addr 172.21.15.2:0/584750203 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.680+0000 7f77e27fc700 1 --2- 172.21.15.2:0/584750203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6cbd86be0 0x55c6cbd77730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.680+0000 7f77e37fe700 1 -- 172.21.15.2:0/584750203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6cbd86be0 msgr2=0x55c6cbd77730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.680+0000 7f77e37fe700 1 --2- 172.21.15.2:0/584750203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6cbd86be0 0x55c6cbd77730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.680+0000 7f77e37fe700 1 -- 172.21.15.2:0/584750203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6cbd87500 msgr2=0x55c6cbd8b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.680+0000 7f77e37fe700 1 --2- 172.21.15.2:0/584750203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6cbd87500 0x55c6cbd8b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.680+0000 7f77e37fe700 1 -- 172.21.15.2:0/584750203 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6cbd82ef0 con 0x55c6cbd82840 2023-09-13T04:44:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.681+0000 7f77e37fe700 1 --2- 172.21.15.2:0/584750203 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd82840 0x55c6cbd92f70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f77d802cc80 tx=0x7f77d80050e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d35f488edbb19ce server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.746+0000 7f77c9ffb700 1 -- 172.21.15.2:0/584750203 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d8005610 con 0x55c6cbd82840 2023-09-13T04:44:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.747+0000 7f77c9ffb700 1 -- 172.21.15.2:0/584750203 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77d8005770 con 0x55c6cbd82840 2023-09-13T04:44:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.747+0000 7f77c9ffb700 1 -- 172.21.15.2:0/584750203 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d8022b20 con 0x55c6cbd82840 2023-09-13T04:44:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.748+0000 7f77fd130b80 1 -- 172.21.15.2:0/584750203 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd82840 msgr2=0x55c6cbd92f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.748+0000 7f77fd130b80 1 --2- 172.21.15.2:0/584750203 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd82840 0x55c6cbd92f70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f77d802cc80 tx=0x7f77d80050e0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.748+0000 7f77fd130b80 1 -- 172.21.15.2:0/584750203 shutdown_connections 2023-09-13T04:44:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.748+0000 7f77fd130b80 1 --2- 172.21.15.2:0/584750203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6cbd87500 0x55c6cbd8b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.748+0000 7f77fd130b80 1 --2- 172.21.15.2:0/584750203 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd82840 0x55c6cbd92f70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.748+0000 7f77fd130b80 1 --2- 172.21.15.2:0/584750203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6cbd86be0 0x55c6cbd77730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.748+0000 7f77fd130b80 1 -- 172.21.15.2:0/584750203 >> 172.21.15.2:0/584750203 conn(0x55c6cbce0c20 msgr2=0x55c6cbd976d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.748+0000 7f77fd130b80 1 -- 172.21.15.2:0/584750203 shutdown_connections 2023-09-13T04:44:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.749+0000 7f77fd130b80 1 -- 172.21.15.2:0/584750203 wait complete. 2023-09-13T04:44:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.749+0000 7f77fd130b80 1 -- start start 2023-09-13T04:44:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.749+0000 7f77fd130b80 1 Processor -- start 2023-09-13T04:44:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.750+0000 7f77fd130b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd82840 0x55c6cbd8ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.750+0000 7f77fd130b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6cbd86be0 0x55c6cbd8f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.750+0000 7f77fd130b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6cbd87500 0x55c6cbd8f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.750+0000 7f77fd130b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6cbd93340 con 0x55c6cbd87500 2023-09-13T04:44:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.750+0000 7f77fd130b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6cbdee920 con 0x55c6cbd86be0 2023-09-13T04:44:44.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.750+0000 7f77fd130b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6cbdeeae0 con 0x55c6cbd82840 2023-09-13T04:44:44.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.750+0000 7f77fd130b80 10 client.0 _ll_register_callbacks cb 0x55c6cbc70620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:44:44.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.751+0000 7f77e27fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6cbd86be0 0x55c6cbd8f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:44.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.751+0000 7f77e27fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6cbd86be0 0x55c6cbd8f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:51990/0 (socket says 172.21.15.2:51990) 2023-09-13T04:44:44.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.751+0000 7f77e27fc700 1 -- 172.21.15.2:0/1591238546 learned_addr learned my addr 172.21.15.2:0/1591238546 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.752+0000 7f77e37fe700 1 --2- 172.21.15.2:0/1591238546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd82840 0x55c6cbd8ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.752+0000 7f77e27fc700 1 -- 172.21.15.2:0/1591238546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd82840 msgr2=0x55c6cbd8ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.752+0000 7f77e27fc700 1 --2- 172.21.15.2:0/1591238546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd82840 0x55c6cbd8ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.752+0000 7f77e27fc700 1 -- 172.21.15.2:0/1591238546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6cbd87500 msgr2=0x55c6cbd8f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.752+0000 7f77e27fc700 1 --2- 172.21.15.2:0/1591238546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6cbd87500 0x55c6cbd8f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.752+0000 7f77e27fc700 1 -- 172.21.15.2:0/1591238546 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6cbcec740 con 0x55c6cbd86be0 2023-09-13T04:44:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.757+0000 7f77e27fc700 1 --2- 172.21.15.2:0/1591238546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6cbd86be0 0x55c6cbd8f490 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f77d4004790 tx=0x7f77d401aaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bac3604476bfae10 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.771+0000 7f77c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:44:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.771+0000 7f77c6ff5700 1 -- 172.21.15.2:0/1591238546 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d4018070 con 0x55c6cbd86be0 2023-09-13T04:44:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.771+0000 7f77c6ff5700 1 -- 172.21.15.2:0/1591238546 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f77d4004050 con 0x55c6cbd86be0 2023-09-13T04:44:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.772+0000 7f77fd130b80 1 -- 172.21.15.2:0/1591238546 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77d800f7d0 con 0x55c6cbd86be0 2023-09-13T04:44:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.777+0000 7f77c6ff5700 1 -- 172.21.15.2:0/1591238546 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77d4004210 con 0x55c6cbd86be0 2023-09-13T04:44:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.777+0000 7f77fd130b80 10 client.26303 fetch_fsmap learned FSMap version 30 2023-09-13T04:44:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.777+0000 7f77fd130b80 1 -- 172.21.15.2:0/1591238546 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c6cbd7d110 con 0x55c6cbd86be0 2023-09-13T04:44:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.781+0000 7f77c6ff5700 1 -- 172.21.15.2:0/1591238546 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77d800f7d0 con 0x55c6cbd86be0 2023-09-13T04:44:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.781+0000 7f77c6ff5700 1 -- 172.21.15.2:0/1591238546 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f77d4012450 con 0x55c6cbd86be0 2023-09-13T04:44:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.781+0000 7f77fd130b80 10 client.26303 fetch_fsmap finished waiting for FSMap version 30 2023-09-13T04:44:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.781+0000 7f77fd130b80 10 client.26303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:44:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.781+0000 7f77fd130b80 20 client.26303 populate_metadata read hostname 'smithi002' 2023-09-13T04:44:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.781+0000 7f77fd130b80 1 --2- 172.21.15.2:0/1591238546 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55c6cbdf0380 0x55c6cbdf27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.781+0000 7f77fd130b80 4 client.26303 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:44:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.781+0000 7f77fd130b80 1 -- 172.21.15.2:0/1591238546 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c6cbc8dff0 con 0x55c6cbdf0380 2023-09-13T04:44:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.782+0000 7f77e37fe700 1 --2- 172.21.15.2:0/1591238546 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55c6cbdf0380 0x55c6cbdf27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:44.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.785+0000 7f77e37fe700 1 --2- 172.21.15.2:0/1591238546 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55c6cbdf0380 0x55c6cbdf27a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db4e96e81bf40a27 server_cookie=4cfe12da45185696 in_seq=0 out_seq=0 2023-09-13T04:44:44.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:44.785+0000 7f77c6ff5700 10 client.26303 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:44:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.563+0000 7f77c6ff5700 1 -- 172.21.15.2:0/1591238546 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c6cbc8dff0 con 0x55c6cbdf0380 2023-09-13T04:44:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.563+0000 7f77c6ff5700 10 client.26303 handle_command_reply: tid=0 2023-09-13T04:44:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.563+0000 7f77fd130b80 1 client.26303 shutdown 2023-09-13T04:44:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.570+0000 7f77fd130b80 1 -- 172.21.15.2:0/1591238546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6cbd86be0 msgr2=0x55c6cbd8f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.570+0000 7f77fd130b80 1 --2- 172.21.15.2:0/1591238546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6cbd86be0 0x55c6cbd8f490 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f77d4004790 tx=0x7f77d401aaf0 comp rx=0 tx=0).stop 2023-09-13T04:44:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.571+0000 7f77fd130b80 1 -- 172.21.15.2:0/1591238546 shutdown_connections 2023-09-13T04:44:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.571+0000 7f77fd130b80 1 --2- 172.21.15.2:0/1591238546 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55c6cbdf0380 0x55c6cbdf27a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.571+0000 7f77fd130b80 1 --2- 172.21.15.2:0/1591238546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6cbd87500 0x55c6cbd8f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.571+0000 7f77fd130b80 1 --2- 172.21.15.2:0/1591238546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6cbd82840 0x55c6cbd8ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:45.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.571+0000 7f77fd130b80 1 --2- 172.21.15.2:0/1591238546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6cbd86be0 0x55c6cbd8f490 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:45.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.571+0000 7f77fd130b80 1 -- 172.21.15.2:0/1591238546 >> 172.21.15.2:0/1591238546 conn(0x55c6cbce0c20 msgr2=0x55c6cbd8c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:45.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.571+0000 7f77c6ff5700 0 client.26303 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:44:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.572+0000 7f77fd130b80 1 -- 172.21.15.2:0/1591238546 shutdown_connections 2023-09-13T04:44:45.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.572+0000 7f77fd130b80 1 -- 172.21.15.2:0/1591238546 wait complete. 2023-09-13T04:44:45.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.572+0000 7f77fd130b80 20 client.26303 trim_cache size 0 max 16384 2023-09-13T04:44:45.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:44:45.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:44:45.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:44:45.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:44:45.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.574+0000 7f77e97ff700 1 -- 172.21.15.2:0/1341911145 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f77cc0a09c0 msgr2=0x7f77cc0a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:45.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.574+0000 7f77e97ff700 1 --2- 172.21.15.2:0/1341911145 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f77cc0a09c0 0x7f77cc0a2e80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55c6cbd87900 tx=0x7f77dc00e040 comp rx=0 tx=0).stop 2023-09-13T04:44:45.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.574+0000 7f77e97ff700 1 -- 172.21.15.2:0/1341911145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77e410f890 msgr2=0x7f77e41a5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:45.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.574+0000 7f77e97ff700 1 --2- 172.21.15.2:0/1341911145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77e410f890 0x7f77e41a5de0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f77e406d9a0 tx=0x7f77dc009f70 comp rx=0 tx=0).stop 2023-09-13T04:44:45.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.574+0000 7f77e97ff700 1 -- 172.21.15.2:0/1341911145 shutdown_connections 2023-09-13T04:44:45.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.574+0000 7f77e97ff700 1 --2- 172.21.15.2:0/1341911145 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f77cc0a09c0 0x7f77cc0a2e80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:45.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.574+0000 7f77e97ff700 1 --2- 172.21.15.2:0/1341911145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77e41aa990 0x7f77e41aae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:45.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.574+0000 7f77e97ff700 1 --2- 172.21.15.2:0/1341911145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77e410f890 0x7f77e41a5de0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.574+0000 7f77e97ff700 1 --2- 172.21.15.2:0/1341911145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77e411a490 0x7f77e41a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.574+0000 7f77e97ff700 1 -- 172.21.15.2:0/1341911145 >> 172.21.15.2:0/1341911145 conn(0x7f77e406cda0 msgr2=0x7f77e4118480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.574+0000 7f77e97ff700 1 -- 172.21.15.2:0/1341911145 shutdown_connections 2023-09-13T04:44:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:45.575+0000 7f77e97ff700 1 -- 172.21.15.2:0/1341911145 wait complete. 2023-09-13T04:44:45.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:45 smithi149 ceph-mon[133315]: pgmap v132: 129 pgs: 129 active+clean; 482 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 284 op/s 2023-09-13T04:44:45.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:45 smithi156 ceph-mon[133675]: pgmap v132: 129 pgs: 129 active+clean; 482 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 284 op/s 2023-09-13T04:44:45.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:45 smithi002 ceph-mon[97651]: pgmap v132: 129 pgs: 129 active+clean; 482 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 284 op/s 2023-09-13T04:44:47.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:47 smithi149 ceph-mon[133315]: pgmap v133: 129 pgs: 129 active+clean; 482 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 1.8 MiB/s wr, 182 op/s 2023-09-13T04:44:47.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:47 smithi156 ceph-mon[133675]: pgmap v133: 129 pgs: 129 active+clean; 482 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 1.8 MiB/s wr, 182 op/s 2023-09-13T04:44:47.760 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:47 smithi002 ceph-mon[97651]: pgmap v133: 129 pgs: 129 active+clean; 482 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 1.8 MiB/s wr, 182 op/s 2023-09-13T04:44:49.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:49 smithi156 ceph-mon[133675]: pgmap v134: 129 pgs: 129 active+clean; 482 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 1.6 MiB/s wr, 171 op/s 2023-09-13T04:44:49.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:49 smithi156 ceph-mon[133675]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:44:49.765 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:49 smithi002 ceph-mon[97651]: pgmap v134: 129 pgs: 129 active+clean; 482 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 1.6 MiB/s wr, 171 op/s 2023-09-13T04:44:49.765 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:49 smithi002 ceph-mon[97651]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:44:49.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:49 smithi149 ceph-mon[133315]: pgmap v134: 129 pgs: 129 active+clean; 482 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 1.6 MiB/s wr, 171 op/s 2023-09-13T04:44:49.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:49 smithi149 ceph-mon[133315]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:44:50.660 INFO:tasks.workunit.client.0.smithi002.stdout: 103 721872 236 530683 206 293361 385 2023-09-13T04:44:50.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:50 smithi149 ceph-mon[133315]: pgmap v135: 129 pgs: 129 active+clean; 484 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 324 op/s 2023-09-13T04:44:50.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:50 smithi156 ceph-mon[133675]: pgmap v135: 129 pgs: 129 active+clean; 484 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 324 op/s 2023-09-13T04:44:51.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:50 smithi002 ceph-mon[97651]: pgmap v135: 129 pgs: 129 active+clean; 484 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 324 op/s 2023-09-13T04:44:52.444 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:52 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:44:52.518 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:52 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:44:52.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:52 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:44:53.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:53 smithi156 ceph-mon[133675]: pgmap v136: 129 pgs: 129 active+clean; 484 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 596 KiB/s wr, 202 op/s 2023-09-13T04:44:53.510 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:53 smithi002 ceph-mon[97651]: pgmap v136: 129 pgs: 129 active+clean; 484 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 596 KiB/s wr, 202 op/s 2023-09-13T04:44:53.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:53 smithi149 ceph-mon[133315]: pgmap v136: 129 pgs: 129 active+clean; 484 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 596 KiB/s wr, 202 op/s 2023-09-13T04:44:54.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:54 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:44:54.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:54 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:44:54.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:54 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:44:54.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:54 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:44:54.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:54 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:44:54.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:54 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:44:54.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:54 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:44:54.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:54 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:44:54.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:54 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:44:55.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:55 smithi156 ceph-mon[133675]: pgmap v137: 129 pgs: 129 active+clean; 504 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 317 op/s 2023-09-13T04:44:55.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:55 smithi002 ceph-mon[97651]: pgmap v137: 129 pgs: 129 active+clean; 504 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 317 op/s 2023-09-13T04:44:55.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:55 smithi149 ceph-mon[133315]: pgmap v137: 129 pgs: 129 active+clean; 504 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 317 op/s 2023-09-13T04:44:57.511 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:57 smithi002 ceph-mon[97651]: pgmap v138: 129 pgs: 129 active+clean; 506 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 2.2 MiB/s wr, 284 op/s 2023-09-13T04:44:57.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:57 smithi149 ceph-mon[133315]: pgmap v138: 129 pgs: 129 active+clean; 506 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 2.2 MiB/s wr, 284 op/s 2023-09-13T04:44:57.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:57 smithi156 ceph-mon[133675]: pgmap v138: 129 pgs: 129 active+clean; 506 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 2.2 MiB/s wr, 284 op/s 2023-09-13T04:44:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.779+0000 7f49937ff700 1 -- 172.21.15.2:0/3616951500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f498c106d60 msgr2=0x7f498c107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.779+0000 7f49937ff700 1 --2- 172.21.15.2:0/3616951500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f498c106d60 0x7f498c107140 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4988009990 tx=0x7f4988009ca0 comp rx=0 tx=0).stop 2023-09-13T04:44:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.779+0000 7f49937ff700 1 -- 172.21.15.2:0/3616951500 shutdown_connections 2023-09-13T04:44:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.779+0000 7f49937ff700 1 --2- 172.21.15.2:0/3616951500 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f498c068a90 0x7f498c10db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.779+0000 7f49937ff700 1 --2- 172.21.15.2:0/3616951500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f498c106d60 0x7f498c107140 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.779+0000 7f49937ff700 1 --2- 172.21.15.2:0/3616951500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f498c0680d0 0x7f498c068550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.779+0000 7f49937ff700 1 -- 172.21.15.2:0/3616951500 >> 172.21.15.2:0/3616951500 conn(0x7f498c076970 msgr2=0x7f498c076d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.779+0000 7f49937ff700 1 -- 172.21.15.2:0/3616951500 shutdown_connections 2023-09-13T04:44:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.779+0000 7f49937ff700 1 -- 172.21.15.2:0/3616951500 wait complete. 2023-09-13T04:44:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.780+0000 7f49937ff700 1 Processor -- start 2023-09-13T04:44:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.780+0000 7f49937ff700 1 -- start start 2023-09-13T04:44:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.781+0000 7f49937ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f498c0680d0 0x7f498c198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.781+0000 7f49937ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f498c068a90 0x7f498c199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.781+0000 7f49937ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f498c106d60 0x7f498c19da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.781+0000 7f49937ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c19e0d0 con 0x7f498c106d60 2023-09-13T04:44:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.781+0000 7f49937ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c19e240 con 0x7f498c0680d0 2023-09-13T04:44:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.781+0000 7f49937ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c19e400 con 0x7f498c068a90 2023-09-13T04:44:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.781+0000 7f499159b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f498c0680d0 0x7f498c198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.781+0000 7f499159b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f498c0680d0 0x7f498c198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:40744/0 (socket says 172.21.15.2:40744) 2023-09-13T04:44:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.781+0000 7f499159b700 1 -- 172.21.15.2:0/546519036 learned_addr learned my addr 172.21.15.2:0/546519036 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.781+0000 7f4990d9a700 1 --2- 172.21.15.2:0/546519036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f498c068a90 0x7f498c199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.782+0000 7f499159b700 1 -- 172.21.15.2:0/546519036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f498c068a90 msgr2=0x7f498c199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.782+0000 7f499159b700 1 --2- 172.21.15.2:0/546519036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f498c068a90 0x7f498c199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.782+0000 7f499159b700 1 -- 172.21.15.2:0/546519036 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f498c106d60 msgr2=0x7f498c19da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.782+0000 7f499159b700 1 --2- 172.21.15.2:0/546519036 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f498c106d60 0x7f498c19da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.782+0000 7f499159b700 1 -- 172.21.15.2:0/546519036 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49880095f0 con 0x7f498c0680d0 2023-09-13T04:44:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.782+0000 7f499159b700 1 --2- 172.21.15.2:0/546519036 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f498c0680d0 0x7f498c198ee0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4988005910 tx=0x7f4988004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.782+0000 7f49827fc700 1 -- 172.21.15.2:0/546519036 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498801d070 con 0x7f498c0680d0 2023-09-13T04:44:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.782+0000 7f49827fc700 1 -- 172.21.15.2:0/546519036 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4988022470 con 0x7f498c0680d0 2023-09-13T04:44:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.782+0000 7f49937ff700 1 -- 172.21.15.2:0/546519036 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f498c19e680 con 0x7f498c0680d0 2023-09-13T04:44:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.782+0000 7f49827fc700 1 -- 172.21.15.2:0/546519036 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4988017650 con 0x7f498c0680d0 2023-09-13T04:44:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.783+0000 7f49937ff700 1 -- 172.21.15.2:0/546519036 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498c1a5f20 con 0x7f498c0680d0 2023-09-13T04:44:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.787+0000 7f49827fc700 1 -- 172.21.15.2:0/546519036 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f498800fad0 con 0x7f498c0680d0 2023-09-13T04:44:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.789+0000 7f49a706fb80 1 Processor -- start 2023-09-13T04:44:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.789+0000 7f49a706fb80 1 -- start start 2023-09-13T04:44:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.789+0000 7f49a706fb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a551be0 0x561f1a551fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.789+0000 7f49a706fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a552500 0x561f1a54d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.789+0000 7f49a706fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f1a54d840 0x561f1a54dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.789+0000 7f49a706fb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a4a6720 con 0x561f1a551be0 2023-09-13T04:44:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.789+0000 7f49a706fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a4a4040 con 0x561f1a552500 2023-09-13T04:44:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.789+0000 7f49a706fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a49f930 con 0x561f1a54d840 2023-09-13T04:44:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.790+0000 7f4991d9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a551be0 0x561f1a551fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.790+0000 7f4991d9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a551be0 0x561f1a551fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60626/0 (socket says 172.21.15.2:60626) 2023-09-13T04:44:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.790+0000 7f4991d9c700 1 -- 172.21.15.2:0/1552528832 learned_addr learned my addr 172.21.15.2:0/1552528832 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.790+0000 7f4990d9a700 1 --2- 172.21.15.2:0/1552528832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f1a54d840 0x561f1a54dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.791+0000 7f4991d9c700 1 -- 172.21.15.2:0/1552528832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f1a54d840 msgr2=0x561f1a54dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.791+0000 7f4991d9c700 1 --2- 172.21.15.2:0/1552528832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f1a54d840 0x561f1a54dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.791+0000 7f4991d9c700 1 -- 172.21.15.2:0/1552528832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a552500 msgr2=0x561f1a54d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.791+0000 7f499159b700 1 --2- 172.21.15.2:0/1552528832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a552500 0x561f1a54d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.791+0000 7f49827fc700 1 --2- 172.21.15.2:0/546519036 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f49780a0900 0x7f49780a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.791+0000 7f4991d9c700 1 --2- 172.21.15.2:0/1552528832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a552500 0x561f1a54d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.791+0000 7f4991d9c700 1 -- 172.21.15.2:0/1552528832 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f1a31f5d0 con 0x561f1a551be0 2023-09-13T04:44:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.791+0000 7f499159b700 1 --2- 172.21.15.2:0/1552528832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a552500 0x561f1a54d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:44:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.791+0000 7f49827fc700 1 -- 172.21.15.2:0/546519036 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f49880cd720 con 0x7f498c0680d0 2023-09-13T04:44:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.791+0000 7f4990d9a700 1 --2- 172.21.15.2:0/546519036 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f49780a0900 0x7f49780a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.791+0000 7f4991d9c700 1 --2- 172.21.15.2:0/1552528832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a551be0 0x561f1a551fc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4984008fb0 tx=0x7f498400d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560015aae71fad3 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.792+0000 7f49777fe700 1 -- 172.21.15.2:0/1552528832 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984013420 con 0x561f1a551be0 2023-09-13T04:44:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.792+0000 7f49777fe700 1 -- 172.21.15.2:0/1552528832 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4984008a40 con 0x561f1a551be0 2023-09-13T04:44:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.792+0000 7f4990d9a700 1 --2- 172.21.15.2:0/546519036 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f49780a0900 0x7f49780a2dc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x561f1a549ae0 tx=0x7f497c009260 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.792+0000 7f49777fe700 1 -- 172.21.15.2:0/1552528832 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984012a20 con 0x561f1a551be0 2023-09-13T04:44:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.793+0000 7f49a706fb80 1 -- 172.21.15.2:0/1552528832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a551be0 msgr2=0x561f1a551fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.793+0000 7f49a706fb80 1 --2- 172.21.15.2:0/1552528832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a551be0 0x561f1a551fc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4984008fb0 tx=0x7f498400d3c0 comp rx=0 tx=0).stop 2023-09-13T04:44:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.793+0000 7f49a706fb80 1 -- 172.21.15.2:0/1552528832 shutdown_connections 2023-09-13T04:44:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.793+0000 7f49a706fb80 1 --2- 172.21.15.2:0/1552528832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f1a54d840 0x561f1a54dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.793+0000 7f49a706fb80 1 --2- 172.21.15.2:0/1552528832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a551be0 0x561f1a551fc0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.793+0000 7f49a706fb80 1 --2- 172.21.15.2:0/1552528832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a552500 0x561f1a54d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.793+0000 7f49a706fb80 1 -- 172.21.15.2:0/1552528832 >> 172.21.15.2:0/1552528832 conn(0x561f1a4abc20 msgr2=0x561f1a547b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.793+0000 7f49a706fb80 1 -- 172.21.15.2:0/1552528832 shutdown_connections 2023-09-13T04:44:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.794+0000 7f49a706fb80 1 -- 172.21.15.2:0/1552528832 wait complete. 2023-09-13T04:44:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.794+0000 7f49a706fb80 1 -- start start 2023-09-13T04:44:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.794+0000 7f49a706fb80 1 Processor -- start 2023-09-13T04:44:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.795+0000 7f49a706fb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a54d840 0x561f1a559630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.795+0000 7f49a706fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f1a551be0 0x561f1a556840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.795+0000 7f49a706fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a552500 0x561f1a556f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.795+0000 7f49a706fb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a43db50 con 0x561f1a54d840 2023-09-13T04:44:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.795+0000 7f49a706fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a43d850 con 0x561f1a552500 2023-09-13T04:44:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.795+0000 7f49a706fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a3218c0 con 0x561f1a551be0 2023-09-13T04:44:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.795+0000 7f49a706fb80 10 client.0 _ll_register_callbacks cb 0x561f1a43b620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:44:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.795+0000 7f4991d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a552500 0x561f1a556f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.795+0000 7f4991d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a552500 0x561f1a556f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:40760/0 (socket says 172.21.15.2:40760) 2023-09-13T04:44:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.795+0000 7f4991d9c700 1 -- 172.21.15.2:0/3788988776 learned_addr learned my addr 172.21.15.2:0/3788988776 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:58.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.796+0000 7f4990d9a700 1 --2- 172.21.15.2:0/3788988776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f1a551be0 0x561f1a556840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:58.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.796+0000 7f4990d9a700 1 -- 172.21.15.2:0/3788988776 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a552500 msgr2=0x561f1a556f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:58.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.796+0000 7f4990d9a700 1 --2- 172.21.15.2:0/3788988776 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a552500 0x561f1a556f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:58.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.796+0000 7f4990d9a700 1 -- 172.21.15.2:0/3788988776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a54d840 msgr2=0x561f1a559630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.796+0000 7f4990d9a700 1 --2- 172.21.15.2:0/3788988776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a54d840 0x561f1a559630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.796+0000 7f4990d9a700 1 -- 172.21.15.2:0/3788988776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4984008be0 con 0x561f1a551be0 2023-09-13T04:44:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.796+0000 7f4990d9a700 1 --2- 172.21.15.2:0/3788988776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f1a551be0 0x561f1a556840 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f497c003900 tx=0x7f497c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8234f226bb3d179 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.797+0000 7f4974ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:44:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.798+0000 7f4974ff9700 1 -- 172.21.15.2:0/3788988776 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c011880 con 0x561f1a551be0 2023-09-13T04:44:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.798+0000 7f4974ff9700 1 -- 172.21.15.2:0/3788988776 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f497c0119e0 con 0x561f1a551be0 2023-09-13T04:44:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.802+0000 7f49a706fb80 1 -- 172.21.15.2:0/3788988776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f1a512ef0 con 0x561f1a551be0 2023-09-13T04:44:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.802+0000 7f4974ff9700 1 -- 172.21.15.2:0/3788988776 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f497c011ba0 con 0x561f1a551be0 2023-09-13T04:44:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.802+0000 7f49a706fb80 10 client.26329 fetch_fsmap learned FSMap version 31 2023-09-13T04:44:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.802+0000 7f49a706fb80 1 -- 172.21.15.2:0/3788988776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x561f1a4b7740 con 0x561f1a551be0 2023-09-13T04:44:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.802+0000 7f4974ff9700 1 -- 172.21.15.2:0/3788988776 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561f1a512ef0 con 0x561f1a551be0 2023-09-13T04:44:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.802+0000 7f4974ff9700 1 -- 172.21.15.2:0/3788988776 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f497c031420 con 0x561f1a551be0 2023-09-13T04:44:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.803+0000 7f49a706fb80 10 client.26329 fetch_fsmap finished waiting for FSMap version 31 2023-09-13T04:44:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.803+0000 7f49a706fb80 10 client.26329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:44:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.803+0000 7f49a706fb80 20 client.26329 populate_metadata read hostname 'smithi002' 2023-09-13T04:44:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.803+0000 7f49a706fb80 1 --2- 172.21.15.2:0/3788988776 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x561f1a566f90 0x561f1a5693b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.803+0000 7f49a706fb80 4 client.26329 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:44:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.803+0000 7f49a706fb80 1 -- 172.21.15.2:0/3788988776 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f1a498910 con 0x561f1a566f90 2023-09-13T04:44:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.803+0000 7f499159b700 1 --2- 172.21.15.2:0/3788988776 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x561f1a566f90 0x561f1a5693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.804+0000 7f499159b700 1 --2- 172.21.15.2:0/3788988776 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x561f1a566f90 0x561f1a5693b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d22f34bce9ca4c server_cookie=6d7ad5d500ef871d in_seq=0 out_seq=0 2023-09-13T04:44:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.804+0000 7f4974ff9700 10 client.26329 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:44:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.805+0000 7f4974ff9700 1 -- 172.21.15.2:0/3788988776 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561f1a498910 con 0x561f1a566f90 2023-09-13T04:44:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.805+0000 7f4974ff9700 10 client.26329 handle_command_reply: tid=0 2023-09-13T04:44:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.805+0000 7f49a706fb80 1 client.26329 shutdown 2023-09-13T04:44:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.805+0000 7f49a706fb80 1 -- 172.21.15.2:0/3788988776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f1a551be0 msgr2=0x561f1a556840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.805+0000 7f49a706fb80 1 --2- 172.21.15.2:0/3788988776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f1a551be0 0x561f1a556840 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f497c003900 tx=0x7f497c004750 comp rx=0 tx=0).stop 2023-09-13T04:44:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.805+0000 7f49a706fb80 1 -- 172.21.15.2:0/3788988776 shutdown_connections 2023-09-13T04:44:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.805+0000 7f49a706fb80 1 --2- 172.21.15.2:0/3788988776 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x561f1a566f90 0x561f1a5693b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.805+0000 7f49a706fb80 1 --2- 172.21.15.2:0/3788988776 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a552500 0x561f1a556f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.805+0000 7f4974ff9700 0 client.26329 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:44:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.805+0000 7f49a706fb80 1 --2- 172.21.15.2:0/3788988776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a54d840 0x561f1a559630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.805+0000 7f49a706fb80 1 --2- 172.21.15.2:0/3788988776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f1a551be0 0x561f1a556840 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.805+0000 7f49a706fb80 1 -- 172.21.15.2:0/3788988776 >> 172.21.15.2:0/3788988776 conn(0x561f1a4abc20 msgr2=0x561f1a553330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.805+0000 7f49a706fb80 1 -- 172.21.15.2:0/3788988776 shutdown_connections 2023-09-13T04:44:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.806+0000 7f49a706fb80 1 -- 172.21.15.2:0/3788988776 wait complete. 2023-09-13T04:44:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.807+0000 7f49a706fb80 20 client.26329 trim_cache size 0 max 16384 2023-09-13T04:44:58.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.839+0000 7f49a706fb80 1 Processor -- start 2023-09-13T04:44:58.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.842+0000 7f49a706fb80 1 -- start start 2023-09-13T04:44:58.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.842+0000 7f49a706fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a54d840 0x561f1a542730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.842+0000 7f49a706fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f1a551be0 0x561f1a55dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.842+0000 7f49a706fb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a552500 0x561f1a547a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.843+0000 7f49a706fb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a557340 con 0x561f1a552500 2023-09-13T04:44:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.843+0000 7f49a706fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a55e4f0 con 0x561f1a54d840 2023-09-13T04:44:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.843+0000 7f49a706fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a554b40 con 0x561f1a551be0 2023-09-13T04:44:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.843+0000 7f4990d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f1a551be0 0x561f1a55dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.843+0000 7f499159b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a54d840 0x561f1a542730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.844+0000 7f499159b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a54d840 0x561f1a542730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:40774/0 (socket says 172.21.15.2:40774) 2023-09-13T04:44:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.844+0000 7f499159b700 1 -- 172.21.15.2:0/3470395038 learned_addr learned my addr 172.21.15.2:0/3470395038 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.844+0000 7f4991d9c700 1 --2- 172.21.15.2:0/3470395038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a552500 0x561f1a547a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.845+0000 7f499159b700 1 -- 172.21.15.2:0/3470395038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f1a551be0 msgr2=0x561f1a55dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.845+0000 7f499159b700 1 --2- 172.21.15.2:0/3470395038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f1a551be0 0x561f1a55dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.845+0000 7f499159b700 1 -- 172.21.15.2:0/3470395038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a552500 msgr2=0x561f1a547a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.845+0000 7f499159b700 1 --2- 172.21.15.2:0/3470395038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a552500 0x561f1a547a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.845+0000 7f499159b700 1 -- 172.21.15.2:0/3470395038 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f1a2cbd80 con 0x561f1a54d840 2023-09-13T04:44:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.845+0000 7f499159b700 1 --2- 172.21.15.2:0/3470395038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a54d840 0x561f1a542730 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4988005f50 tx=0x7f49880cfb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70ec4d858ffb752a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:58.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.856+0000 7f4977fff700 1 -- 172.21.15.2:0/3470395038 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498809a8f0 con 0x561f1a54d840 2023-09-13T04:44:58.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.856+0000 7f4977fff700 1 -- 172.21.15.2:0/3470395038 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f498809e720 con 0x561f1a54d840 2023-09-13T04:44:58.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.856+0000 7f4977fff700 1 -- 172.21.15.2:0/3470395038 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498800fe90 con 0x561f1a54d840 2023-09-13T04:44:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.857+0000 7f49a706fb80 1 -- 172.21.15.2:0/3470395038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a54d840 msgr2=0x561f1a542730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:44:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.857+0000 7f49a706fb80 1 --2- 172.21.15.2:0/3470395038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a54d840 0x561f1a542730 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4988005f50 tx=0x7f49880cfb30 comp rx=0 tx=0).stop 2023-09-13T04:44:58.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.859+0000 7f49a706fb80 1 -- 172.21.15.2:0/3470395038 shutdown_connections 2023-09-13T04:44:58.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.859+0000 7f49a706fb80 1 --2- 172.21.15.2:0/3470395038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a552500 0x561f1a547a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:58.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.859+0000 7f49a706fb80 1 --2- 172.21.15.2:0/3470395038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a54d840 0x561f1a542730 secure :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4988005f50 tx=0x7f49880cfb30 comp rx=0 tx=0).stop 2023-09-13T04:44:58.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.859+0000 7f49a706fb80 1 --2- 172.21.15.2:0/3470395038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f1a551be0 0x561f1a55dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:58.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.859+0000 7f49a706fb80 1 -- 172.21.15.2:0/3470395038 >> 172.21.15.2:0/3470395038 conn(0x561f1a4abc20 msgr2=0x561f1a56dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:44:58.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.861+0000 7f49a706fb80 1 -- 172.21.15.2:0/3470395038 shutdown_connections 2023-09-13T04:44:58.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.861+0000 7f49a706fb80 1 -- 172.21.15.2:0/3470395038 wait complete. 2023-09-13T04:44:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.862+0000 7f49a706fb80 1 -- start start 2023-09-13T04:44:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.862+0000 7f49a706fb80 1 Processor -- start 2023-09-13T04:44:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.863+0000 7f49a706fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a551be0 0x561f1a55c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.863+0000 7f49a706fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f1a552500 0x561f1a55c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.863+0000 7f49a706fb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a5be5b0 0x561f1a5bea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.863+0000 7f49a706fb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a5bef70 con 0x561f1a5be5b0 2023-09-13T04:44:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.863+0000 7f49a706fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a5bf0e0 con 0x561f1a551be0 2023-09-13T04:44:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.863+0000 7f49a706fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a5bf2a0 con 0x561f1a552500 2023-09-13T04:44:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.863+0000 7f49a706fb80 10 client.0 _ll_register_callbacks cb 0x561f1a43b620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:44:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.864+0000 7f499159b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a551be0 0x561f1a55c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.864+0000 7f499159b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a551be0 0x561f1a55c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:40788/0 (socket says 172.21.15.2:40788) 2023-09-13T04:44:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.864+0000 7f499159b700 1 -- 172.21.15.2:0/3741666913 learned_addr learned my addr 172.21.15.2:0/3741666913 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:44:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.864+0000 7f4990d9a700 1 --2- 172.21.15.2:0/3741666913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f1a552500 0x561f1a55c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.868+0000 7f499159b700 1 -- 172.21.15.2:0/3741666913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f1a552500 msgr2=0x561f1a55c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.868+0000 7f499159b700 1 --2- 172.21.15.2:0/3741666913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f1a552500 0x561f1a55c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.868+0000 7f499159b700 1 -- 172.21.15.2:0/3741666913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a5be5b0 msgr2=0x561f1a5bea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:44:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.868+0000 7f499159b700 1 --2- 172.21.15.2:0/3741666913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a5be5b0 0x561f1a5bea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:44:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.868+0000 7f499159b700 1 -- 172.21.15.2:0/3741666913 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498c1a5f20 con 0x561f1a551be0 2023-09-13T04:44:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.878+0000 7f499159b700 1 --2- 172.21.15.2:0/3741666913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a551be0 0x561f1a55c180 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f49880a23a0 tx=0x7f4988022c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=707f8897c7c4ff94 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:44:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.882+0000 7f4974ff9700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:44:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.887+0000 7f4974ff9700 1 -- 172.21.15.2:0/3741666913 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4988098a30 con 0x561f1a551be0 2023-09-13T04:44:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.887+0000 7f4974ff9700 1 -- 172.21.15.2:0/3741666913 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f498809a8f0 con 0x561f1a551be0 2023-09-13T04:44:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.887+0000 7f49a706fb80 1 -- 172.21.15.2:0/3741666913 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4984008a40 con 0x561f1a551be0 2023-09-13T04:44:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.894+0000 7f4974ff9700 1 -- 172.21.15.2:0/3741666913 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49880225e0 con 0x561f1a551be0 2023-09-13T04:44:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.901+0000 7f49a706fb80 10 client.26318 fetch_fsmap learned FSMap version 31 2023-09-13T04:44:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.901+0000 7f49a706fb80 1 -- 172.21.15.2:0/3741666913 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x561f1a2b4c00 con 0x561f1a551be0 2023-09-13T04:44:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.901+0000 7f4974ff9700 1 -- 172.21.15.2:0/3741666913 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4984008a40 con 0x561f1a551be0 2023-09-13T04:44:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.903+0000 7f4974ff9700 1 -- 172.21.15.2:0/3741666913 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f49880cd070 con 0x561f1a551be0 2023-09-13T04:44:58.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.903+0000 7f49a706fb80 10 client.26318 fetch_fsmap finished waiting for FSMap version 31 2023-09-13T04:44:58.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.903+0000 7f49a706fb80 10 client.26318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:44:58.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.903+0000 7f49a706fb80 20 client.26318 populate_metadata read hostname 'smithi002' 2023-09-13T04:44:58.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.903+0000 7f49a706fb80 1 --2- 172.21.15.2:0/3741666913 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x561f1a5d53a0 0x561f1a5d77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:44:58.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.903+0000 7f49a706fb80 4 client.26318 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:44:58.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.904+0000 7f49a706fb80 1 -- 172.21.15.2:0/3741666913 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561f1a458ff0 con 0x561f1a5d53a0 2023-09-13T04:44:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.908+0000 7f4990d9a700 1 --2- 172.21.15.2:0/3741666913 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x561f1a5d53a0 0x561f1a5d77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:44:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.934+0000 7f4990d9a700 1 --2- 172.21.15.2:0/3741666913 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x561f1a5d53a0 0x561f1a5d77c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1744c17ce8e33ea7 server_cookie=d491c77b8b3b46bb in_seq=0 out_seq=0 2023-09-13T04:44:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:44:58.935+0000 7f4974ff9700 10 client.26318 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:44:59.431 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T04:44:59.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:44:59 smithi149 ceph-mon[133315]: pgmap v139: 129 pgs: 129 active+clean; 506 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 2.2 MiB/s wr, 282 op/s 2023-09-13T04:44:59.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:44:59 smithi156 ceph-mon[133675]: pgmap v139: 129 pgs: 129 active+clean; 506 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 2.2 MiB/s wr, 282 op/s 2023-09-13T04:44:59.736 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:44:59 smithi002 ceph-mon[97651]: pgmap v139: 129 pgs: 129 active+clean; 506 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 2.2 MiB/s wr, 282 op/s 2023-09-13T04:45:00.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.076+0000 7f57331ff700 1 -- 172.21.15.2:0/602458716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f572c11a490 msgr2=0x7f572c11c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:00.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.076+0000 7f57331ff700 1 --2- 172.21.15.2:0/602458716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f572c11a490 0x7f572c11c880 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f572001a9a0 tx=0x7f572001acb0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.095+0000 7f57331ff700 1 -- 172.21.15.2:0/602458716 shutdown_connections 2023-09-13T04:45:00.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.095+0000 7f57331ff700 1 --2- 172.21.15.2:0/602458716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f572c11a490 0x7f572c11c880 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.095+0000 7f57331ff700 1 --2- 172.21.15.2:0/602458716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f572c10f440 0x7f572c107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.095+0000 7f57331ff700 1 --2- 172.21.15.2:0/602458716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f572c1081f0 0x7f572c108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.095+0000 7f57331ff700 1 -- 172.21.15.2:0/602458716 >> 172.21.15.2:0/602458716 conn(0x7f572c06cfa0 msgr2=0x7f572c06d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:00.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.108+0000 7f57331ff700 1 -- 172.21.15.2:0/602458716 shutdown_connections 2023-09-13T04:45:00.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.108+0000 7f57331ff700 1 -- 172.21.15.2:0/602458716 wait complete. 2023-09-13T04:45:00.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.109+0000 7f57331ff700 1 Processor -- start 2023-09-13T04:45:00.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.110+0000 7f57331ff700 1 -- start start 2023-09-13T04:45:00.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.110+0000 7f57331ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f572c1081f0 0x7f572c115860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:00.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.110+0000 7f57331ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f572c10f440 0x7f572c115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:00.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.110+0000 7f57331ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f572c11a490 0x7f572c1a91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:00.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.110+0000 7f57331ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572c1a97a0 con 0x7f572c11a490 2023-09-13T04:45:00.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.110+0000 7f57331ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572c1a98e0 con 0x7f572c10f440 2023-09-13T04:45:00.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.110+0000 7f57331ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572c1a9aa0 con 0x7f572c1081f0 2023-09-13T04:45:00.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.112+0000 7f5730f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f572c1081f0 0x7f572c115860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:00.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.112+0000 7f5730f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f572c1081f0 0x7f572c115860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:54122/0 (socket says 172.21.15.2:54122) 2023-09-13T04:45:00.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.112+0000 7f5730f9b700 1 -- 172.21.15.2:0/967506374 learned_addr learned my addr 172.21.15.2:0/967506374 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:00.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.113+0000 7f573179c700 1 --2- 172.21.15.2:0/967506374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f572c11a490 0x7f572c1a91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:00.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.113+0000 7f5730f9b700 1 -- 172.21.15.2:0/967506374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f572c10f440 msgr2=0x7f572c115da0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-13T04:45:00.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.113+0000 7f5730f9b700 1 --2- 172.21.15.2:0/967506374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f572c10f440 0x7f572c115da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.113+0000 7f5730f9b700 1 -- 172.21.15.2:0/967506374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f572c11a490 msgr2=0x7f572c1a91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:00.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.113+0000 7f5730f9b700 1 --2- 172.21.15.2:0/967506374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f572c11a490 0x7f572c1a91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.113+0000 7f5730f9b700 1 -- 172.21.15.2:0/967506374 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f572001a600 con 0x7f572c1081f0 2023-09-13T04:45:00.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.114+0000 7f5730f9b700 1 --2- 172.21.15.2:0/967506374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f572c1081f0 0x7f572c115860 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f572400b870 tx=0x7f572400bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:00.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.118+0000 7f5729ffb700 1 -- 172.21.15.2:0/967506374 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572400d8e0 con 0x7f572c1081f0 2023-09-13T04:45:00.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.118+0000 7f5729ffb700 1 -- 172.21.15.2:0/967506374 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5724009950 con 0x7f572c1081f0 2023-09-13T04:45:00.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.118+0000 7f5729ffb700 1 -- 172.21.15.2:0/967506374 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57240143d0 con 0x7f572c1081f0 2023-09-13T04:45:00.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.122+0000 7f57331ff700 1 -- 172.21.15.2:0/967506374 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f572c1a9d80 con 0x7f572c1081f0 2023-09-13T04:45:00.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.122+0000 7f57331ff700 1 -- 172.21.15.2:0/967506374 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f572c1aa1b0 con 0x7f572c1081f0 2023-09-13T04:45:00.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.129+0000 7f5746aadb80 1 Processor -- start 2023-09-13T04:45:00.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.132+0000 7f5729ffb700 1 -- 172.21.15.2:0/967506374 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f5724015620 con 0x7f572c1081f0 2023-09-13T04:45:00.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.132+0000 7f5746aadb80 1 -- start start 2023-09-13T04:45:00.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.133+0000 7f5746aadb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564855825fe0 0x5648558263c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:00.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.133+0000 7f5746aadb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564855826900 0x5648558204c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:00.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.133+0000 7f5746aadb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855820a00 0x564855820e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:00.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.133+0000 7f5746aadb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564855779780 con 0x564855826900 2023-09-13T04:45:00.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.133+0000 7f5746aadb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648557770a0 con 0x564855825fe0 2023-09-13T04:45:00.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.133+0000 7f5746aadb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564855772990 con 0x564855820a00 2023-09-13T04:45:00.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.134+0000 7f572bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855820a00 0x564855820e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:00.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.134+0000 7f572bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855820a00 0x564855820e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:54130/0 (socket says 172.21.15.2:54130) 2023-09-13T04:45:00.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.134+0000 7f572bfff700 1 -- 172.21.15.2:0/2459239468 learned_addr learned my addr 172.21.15.2:0/2459239468 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:00.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.135+0000 7f5730f9b700 1 --2- 172.21.15.2:0/2459239468 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564855826900 0x5648558204c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:00.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.135+0000 7f573179c700 1 --2- 172.21.15.2:0/2459239468 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564855825fe0 0x5648558263c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:00.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.135+0000 7f5729ffb700 1 --2- 172.21.15.2:0/967506374 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f57140a09c0 0x7f57140a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:00.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.135+0000 7f5729ffb700 1 -- 172.21.15.2:0/967506374 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f5724011070 con 0x7f572c1081f0 2023-09-13T04:45:00.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.136+0000 7f572bfff700 1 -- 172.21.15.2:0/2459239468 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564855825fe0 msgr2=0x5648558263c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:00.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.136+0000 7f572bfff700 1 --2- 172.21.15.2:0/2459239468 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564855825fe0 0x5648558263c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.136+0000 7f572bfff700 1 -- 172.21.15.2:0/2459239468 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564855826900 msgr2=0x5648558204c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:00.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.136+0000 7f572bfff700 1 --2- 172.21.15.2:0/2459239468 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564855826900 0x5648558204c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.136+0000 7f572bfff700 1 -- 172.21.15.2:0/2459239468 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648555f2630 con 0x564855820a00 2023-09-13T04:45:00.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.137+0000 7f572bfff700 1 --2- 172.21.15.2:0/2459239468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855820a00 0x564855820e60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f571c009800 tx=0x7f571c009b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a43310f198be61 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:00.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.139+0000 7f573179c700 1 --2- 172.21.15.2:0/967506374 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f57140a09c0 0x7f57140a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:00.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.139+0000 7f5712ffd700 1 -- 172.21.15.2:0/2459239468 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571c011830 con 0x564855820a00 2023-09-13T04:45:00.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.139+0000 7f5712ffd700 1 -- 172.21.15.2:0/2459239468 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f571c011e50 con 0x564855820a00 2023-09-13T04:45:00.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.139+0000 7f5712ffd700 1 -- 172.21.15.2:0/2459239468 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571c00b300 con 0x564855820a00 2023-09-13T04:45:00.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.141+0000 7f5746aadb80 1 -- 172.21.15.2:0/2459239468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855820a00 msgr2=0x564855820e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:00.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.141+0000 7f5746aadb80 1 --2- 172.21.15.2:0/2459239468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855820a00 0x564855820e60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f571c009800 tx=0x7f571c009b10 comp rx=0 tx=0).stop 2023-09-13T04:45:00.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.142+0000 7f5746aadb80 1 -- 172.21.15.2:0/2459239468 shutdown_connections 2023-09-13T04:45:00.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.142+0000 7f5746aadb80 1 --2- 172.21.15.2:0/2459239468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855820a00 0x564855820e60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.142+0000 7f5746aadb80 1 --2- 172.21.15.2:0/2459239468 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564855825fe0 0x5648558263c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.142+0000 7f5746aadb80 1 --2- 172.21.15.2:0/2459239468 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564855826900 0x5648558204c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.142+0000 7f5746aadb80 1 -- 172.21.15.2:0/2459239468 >> 172.21.15.2:0/2459239468 conn(0x56485577ec80 msgr2=0x56485581af20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:00.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.143+0000 7f573179c700 1 --2- 172.21.15.2:0/967506374 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f57140a09c0 0x7f57140a2e80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f572001af50 tx=0x7f572001a320 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:00.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.148+0000 7f5746aadb80 1 -- 172.21.15.2:0/2459239468 shutdown_connections 2023-09-13T04:45:00.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.148+0000 7f5746aadb80 1 -- 172.21.15.2:0/2459239468 wait complete. 2023-09-13T04:45:00.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.149+0000 7f5746aadb80 1 -- start start 2023-09-13T04:45:00.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.149+0000 7f5746aadb80 1 Processor -- start 2023-09-13T04:45:00.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.150+0000 7f5746aadb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855820a00 0x56485582c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:00.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.150+0000 7f5746aadb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564855825fe0 0x564855829c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:00.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.150+0000 7f5746aadb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564855826900 0x56485582a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:00.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.150+0000 7f5746aadb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564855710bb0 con 0x564855825fe0 2023-09-13T04:45:00.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.150+0000 7f5746aadb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648557108b0 con 0x564855826900 2023-09-13T04:45:00.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.150+0000 7f5746aadb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648555f4920 con 0x564855820a00 2023-09-13T04:45:00.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.150+0000 7f5746aadb80 10 client.0 _ll_register_callbacks cb 0x56485570e680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:45:00.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.151+0000 7f5730f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855820a00 0x56485582c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:00.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.151+0000 7f5730f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855820a00 0x56485582c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:54140/0 (socket says 172.21.15.2:54140) 2023-09-13T04:45:00.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.151+0000 7f5730f9b700 1 -- 172.21.15.2:0/2660307633 learned_addr learned my addr 172.21.15.2:0/2660307633 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:00.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.151+0000 7f5730f9b700 1 -- 172.21.15.2:0/2660307633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564855826900 msgr2=0x56485582a360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:45:00.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.151+0000 7f5730f9b700 1 --2- 172.21.15.2:0/2660307633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564855826900 0x56485582a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.151+0000 7f5730f9b700 1 -- 172.21.15.2:0/2660307633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564855825fe0 msgr2=0x564855829c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:00.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.151+0000 7f5730f9b700 1 --2- 172.21.15.2:0/2660307633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564855825fe0 0x564855829c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.151+0000 7f5730f9b700 1 -- 172.21.15.2:0/2660307633 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571c009430 con 0x564855820a00 2023-09-13T04:45:00.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.153+0000 7f5730f9b700 1 --2- 172.21.15.2:0/2660307633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855820a00 0x56485582c930 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5724098ce0 tx=0x7f572401b2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70c655ca173ce650 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:00.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.153+0000 7f570bfff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:45:00.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.153+0000 7f570bfff700 1 -- 172.21.15.2:0/2660307633 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5724099450 con 0x564855820a00 2023-09-13T04:45:00.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.153+0000 7f570bfff700 1 -- 172.21.15.2:0/2660307633 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f5724098d50 con 0x564855820a00 2023-09-13T04:45:00.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.153+0000 7f5746aadb80 1 -- 172.21.15.2:0/2660307633 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564855813670 con 0x564855820a00 2023-09-13T04:45:00.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.154+0000 7f570bfff700 1 -- 172.21.15.2:0/2660307633 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f572409cb80 con 0x564855820a00 2023-09-13T04:45:00.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.157+0000 7f5746aadb80 10 client.26341 fetch_fsmap learned FSMap version 31 2023-09-13T04:45:00.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.157+0000 7f5746aadb80 1 -- 172.21.15.2:0/2660307633 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56485578a7a0 con 0x564855820a00 2023-09-13T04:45:00.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.165+0000 7f570bfff700 1 -- 172.21.15.2:0/2660307633 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564855813670 con 0x564855820a00 2023-09-13T04:45:00.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.165+0000 7f570bfff700 1 -- 172.21.15.2:0/2660307633 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f57240b80b0 con 0x564855820a00 2023-09-13T04:45:00.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.170+0000 7f5746aadb80 10 client.26341 fetch_fsmap finished waiting for FSMap version 31 2023-09-13T04:45:00.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.170+0000 7f5746aadb80 10 client.26341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:45:00.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.170+0000 7f5746aadb80 20 client.26341 populate_metadata read hostname 'smithi002' 2023-09-13T04:45:00.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.170+0000 7f5746aadb80 1 --2- 172.21.15.2:0/2660307633 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56485583a1c0 0x56485583c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:00.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.170+0000 7f5746aadb80 4 client.26341 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:45:00.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.170+0000 7f5746aadb80 1 -- 172.21.15.2:0/2660307633 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56485576b970 con 0x56485583a1c0 2023-09-13T04:45:00.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.170+0000 7f572bfff700 1 --2- 172.21.15.2:0/2660307633 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56485583a1c0 0x56485583c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:00.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.177+0000 7f572bfff700 1 --2- 172.21.15.2:0/2660307633 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56485583a1c0 0x56485583c5e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afc6c0e00429223e server_cookie=246107d22b0d6151 in_seq=0 out_seq=0 2023-09-13T04:45:00.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.177+0000 7f570bfff700 10 client.26341 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:45:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.560+0000 7f4974ff9700 1 -- 172.21.15.2:0/3741666913 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561f1a458ff0 con 0x561f1a5d53a0 2023-09-13T04:45:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.560+0000 7f4974ff9700 10 client.26318 handle_command_reply: tid=0 2023-09-13T04:45:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.561+0000 7f49a706fb80 1 client.26318 shutdown 2023-09-13T04:45:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.562+0000 7f49a706fb80 1 -- 172.21.15.2:0/3741666913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a551be0 msgr2=0x561f1a55c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.562+0000 7f49a706fb80 1 --2- 172.21.15.2:0/3741666913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a551be0 0x561f1a55c180 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f49880a23a0 tx=0x7f4988022c60 comp rx=0 tx=0).stop 2023-09-13T04:45:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.563+0000 7f49a706fb80 1 -- 172.21.15.2:0/3741666913 shutdown_connections 2023-09-13T04:45:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.563+0000 7f49a706fb80 1 --2- 172.21.15.2:0/3741666913 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x561f1a5d53a0 0x561f1a5d77c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.563+0000 7f49a706fb80 1 --2- 172.21.15.2:0/3741666913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a5be5b0 0x561f1a5bea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.563+0000 7f49a706fb80 1 --2- 172.21.15.2:0/3741666913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f1a551be0 0x561f1a55c180 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.563+0000 7f49a706fb80 1 --2- 172.21.15.2:0/3741666913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f1a552500 0x561f1a55c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.563+0000 7f49a706fb80 1 -- 172.21.15.2:0/3741666913 >> 172.21.15.2:0/3741666913 conn(0x561f1a4abc20 msgr2=0x561f1a547a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.563+0000 7f4974ff9700 0 client.26318 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:45:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.563+0000 7f49a706fb80 1 -- 172.21.15.2:0/3741666913 shutdown_connections 2023-09-13T04:45:00.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.563+0000 7f570bfff700 1 -- 172.21.15.2:0/2660307633 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56485576b970 con 0x56485583a1c0 2023-09-13T04:45:00.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.563+0000 7f570bfff700 10 client.26341 handle_command_reply: tid=0 2023-09-13T04:45:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.565+0000 7f49a706fb80 1 -- 172.21.15.2:0/3741666913 wait complete. 2023-09-13T04:45:00.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.564+0000 7f5746aadb80 1 client.26341 shutdown 2023-09-13T04:45:00.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.564+0000 7f5746aadb80 1 -- 172.21.15.2:0/2660307633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855820a00 msgr2=0x56485582c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:00.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.564+0000 7f5746aadb80 1 --2- 172.21.15.2:0/2660307633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855820a00 0x56485582c930 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5724098ce0 tx=0x7f572401b2c0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.564+0000 7f5746aadb80 1 -- 172.21.15.2:0/2660307633 shutdown_connections 2023-09-13T04:45:00.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.564+0000 7f5746aadb80 1 --2- 172.21.15.2:0/2660307633 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56485583a1c0 0x56485583c5e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.565+0000 7f5746aadb80 1 --2- 172.21.15.2:0/2660307633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564855826900 0x56485582a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.565+0000 7f5746aadb80 1 --2- 172.21.15.2:0/2660307633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855820a00 0x56485582c930 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.565+0000 7f5746aadb80 1 --2- 172.21.15.2:0/2660307633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564855825fe0 0x564855829c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.565+0000 7f5746aadb80 1 -- 172.21.15.2:0/2660307633 >> 172.21.15.2:0/2660307633 conn(0x56485577ec80 msgr2=0x5648558157f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:00.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.565+0000 7f570bfff700 0 client.26341 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:45:00.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.565+0000 7f5746aadb80 1 -- 172.21.15.2:0/2660307633 shutdown_connections 2023-09-13T04:45:00.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.569+0000 7f5746aadb80 1 -- 172.21.15.2:0/2660307633 wait complete. 2023-09-13T04:45:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.570+0000 7f49a706fb80 20 client.26318 trim_cache size 0 max 16384 2023-09-13T04:45:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:45:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:45:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:45:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:45:00.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.575+0000 7f5746aadb80 20 client.26341 trim_cache size 0 max 16384 2023-09-13T04:45:00.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.594+0000 7f49937ff700 1 -- 172.21.15.2:0/546519036 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f49780a0900 msgr2=0x7f49780a2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:00.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.594+0000 7f49937ff700 1 --2- 172.21.15.2:0/546519036 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f49780a0900 0x7f49780a2dc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x561f1a549ae0 tx=0x7f497c009260 comp rx=0 tx=0).stop 2023-09-13T04:45:00.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.594+0000 7f49937ff700 1 -- 172.21.15.2:0/546519036 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f498c0680d0 msgr2=0x7f498c198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:00.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.594+0000 7f49937ff700 1 --2- 172.21.15.2:0/546519036 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f498c0680d0 0x7f498c198ee0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4988005910 tx=0x7f4988004730 comp rx=0 tx=0).stop 2023-09-13T04:45:00.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.594+0000 7f49937ff700 1 -- 172.21.15.2:0/546519036 shutdown_connections 2023-09-13T04:45:00.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.594+0000 7f49937ff700 1 --2- 172.21.15.2:0/546519036 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f49780a0900 0x7f49780a2dc0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.594+0000 7f49937ff700 1 --2- 172.21.15.2:0/546519036 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f498c106d60 0x7f498c19da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.594+0000 7f49937ff700 1 --2- 172.21.15.2:0/546519036 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f498c0680d0 0x7f498c198ee0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.594+0000 7f49937ff700 1 --2- 172.21.15.2:0/546519036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f498c068a90 0x7f498c199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.594+0000 7f49937ff700 1 -- 172.21.15.2:0/546519036 >> 172.21.15.2:0/546519036 conn(0x7f498c076970 msgr2=0x7f498c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.605+0000 7f49937ff700 1 -- 172.21.15.2:0/546519036 shutdown_connections 2023-09-13T04:45:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:00.605+0000 7f49937ff700 1 -- 172.21.15.2:0/546519036 wait complete. 2023-09-13T04:45:00.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.613+0000 7f5746aadb80 1 Processor -- start 2023-09-13T04:45:00.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.613+0000 7f5746aadb80 1 -- start start 2023-09-13T04:45:00.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.614+0000 7f5746aadb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564855820a00 0x564855831240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:00.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.614+0000 7f5746aadb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855825fe0 0x5648558158f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:00.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.614+0000 7f5746aadb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564855826900 0x56485582c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:00.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.614+0000 7f5746aadb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485581b4f0 con 0x564855826900 2023-09-13T04:45:00.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.614+0000 7f5746aadb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485582b1f0 con 0x564855820a00 2023-09-13T04:45:00.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.614+0000 7f5746aadb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485583a020 con 0x564855825fe0 2023-09-13T04:45:00.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.614+0000 7f572bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855825fe0 0x5648558158f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:00.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.614+0000 7f572bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855825fe0 0x5648558158f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:54148/0 (socket says 172.21.15.2:54148) 2023-09-13T04:45:00.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.614+0000 7f572bfff700 1 -- 172.21.15.2:0/3467795223 learned_addr learned my addr 172.21.15.2:0/3467795223 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:00.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.615+0000 7f573179c700 1 --2- 172.21.15.2:0/3467795223 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564855826900 0x56485582c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:00.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.618+0000 7f573179c700 1 -- 172.21.15.2:0/3467795223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855825fe0 msgr2=0x5648558158f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:00.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.618+0000 7f573179c700 1 --2- 172.21.15.2:0/3467795223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855825fe0 0x5648558158f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.618+0000 7f573179c700 1 -- 172.21.15.2:0/3467795223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564855820a00 msgr2=0x564855831240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:45:00.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.618+0000 7f573179c700 1 --2- 172.21.15.2:0/3467795223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564855820a00 0x564855831240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.618+0000 7f573179c700 1 -- 172.21.15.2:0/3467795223 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56485559ede0 con 0x564855826900 2023-09-13T04:45:00.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.619+0000 7f573179c700 1 --2- 172.21.15.2:0/3467795223 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564855826900 0x56485582c4d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f572001a8f0 tx=0x7f5720031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68612596e252d5f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:00.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.626+0000 7f57137fe700 1 -- 172.21.15.2:0/3467795223 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5720004860 con 0x564855826900 2023-09-13T04:45:00.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.626+0000 7f57137fe700 1 -- 172.21.15.2:0/3467795223 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5720004e80 con 0x564855826900 2023-09-13T04:45:00.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.626+0000 7f57137fe700 1 -- 172.21.15.2:0/3467795223 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5720004260 con 0x564855826900 2023-09-13T04:45:00.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.627+0000 7f5746aadb80 1 -- 172.21.15.2:0/3467795223 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564855826900 msgr2=0x56485582c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:00.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.627+0000 7f5746aadb80 1 --2- 172.21.15.2:0/3467795223 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564855826900 0x56485582c4d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f572001a8f0 tx=0x7f5720031000 comp rx=0 tx=0).stop 2023-09-13T04:45:00.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.627+0000 7f5746aadb80 1 -- 172.21.15.2:0/3467795223 shutdown_connections 2023-09-13T04:45:00.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.627+0000 7f5746aadb80 1 --2- 172.21.15.2:0/3467795223 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564855826900 0x56485582c4d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.627+0000 7f5746aadb80 1 --2- 172.21.15.2:0/3467795223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564855820a00 0x564855831240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.627+0000 7f5746aadb80 1 --2- 172.21.15.2:0/3467795223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855825fe0 0x5648558158f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.627+0000 7f5746aadb80 1 -- 172.21.15.2:0/3467795223 >> 172.21.15.2:0/3467795223 conn(0x56485577ec80 msgr2=0x564855823d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:00.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.641+0000 7f5746aadb80 1 -- 172.21.15.2:0/3467795223 shutdown_connections 2023-09-13T04:45:00.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.650+0000 7f5746aadb80 1 -- 172.21.15.2:0/3467795223 wait complete. 2023-09-13T04:45:00.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.650+0000 7f5746aadb80 1 -- start start 2023-09-13T04:45:00.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.651+0000 7f5746aadb80 1 Processor -- start 2023-09-13T04:45:00.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.653+0000 7f5746aadb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855820a00 0x56485582d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:00.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.653+0000 7f5746aadb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564855825fe0 0x56485582d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:00.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.653+0000 7f5746aadb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564855826900 0x56485588d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:00.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.653+0000 7f5746aadb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485582e010 con 0x564855826900 2023-09-13T04:45:00.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.654+0000 7f5746aadb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485588d560 con 0x564855825fe0 2023-09-13T04:45:00.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.654+0000 7f5746aadb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485588d6a0 con 0x564855820a00 2023-09-13T04:45:00.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.654+0000 7f5746aadb80 10 client.0 _ll_register_callbacks cb 0x56485570e680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:45:00.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.654+0000 7f5730f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855820a00 0x56485582d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:00.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.654+0000 7f5730f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855820a00 0x56485582d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:54160/0 (socket says 172.21.15.2:54160) 2023-09-13T04:45:00.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.654+0000 7f5730f9b700 1 -- 172.21.15.2:0/3252355682 learned_addr learned my addr 172.21.15.2:0/3252355682 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:00.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.654+0000 7f572bfff700 1 --2- 172.21.15.2:0/3252355682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564855825fe0 0x56485582d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:00.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.655+0000 7f5730f9b700 1 -- 172.21.15.2:0/3252355682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564855825fe0 msgr2=0x56485582d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:00.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.655+0000 7f5730f9b700 1 --2- 172.21.15.2:0/3252355682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564855825fe0 0x56485582d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.655+0000 7f5730f9b700 1 -- 172.21.15.2:0/3252355682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564855826900 msgr2=0x56485588d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:00.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.655+0000 7f5730f9b700 1 --2- 172.21.15.2:0/3252355682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564855826900 0x56485588d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.655+0000 7f5730f9b700 1 -- 172.21.15.2:0/3252355682 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f572c11d230 con 0x564855820a00 2023-09-13T04:45:00.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.655+0000 7f5730f9b700 1 --2- 172.21.15.2:0/3252355682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855820a00 0x56485582d410 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f57240143a0 tx=0x7f57240c90d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=397f99956a7aa3fe server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:00.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.655+0000 7f570bfff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:45:00.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.655+0000 7f570bfff700 1 -- 172.21.15.2:0/3252355682 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57240b82b0 con 0x564855820a00 2023-09-13T04:45:00.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.655+0000 7f570bfff700 1 -- 172.21.15.2:0/3252355682 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f5724098d50 con 0x564855820a00 2023-09-13T04:45:00.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.655+0000 7f5746aadb80 1 -- 172.21.15.2:0/3252355682 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f571c011e50 con 0x564855820a00 2023-09-13T04:45:00.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.656+0000 7f570bfff700 1 -- 172.21.15.2:0/3252355682 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5724097d00 con 0x564855820a00 2023-09-13T04:45:00.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.656+0000 7f5746aadb80 10 client.26347 fetch_fsmap learned FSMap version 31 2023-09-13T04:45:00.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.656+0000 7f5746aadb80 1 -- 172.21.15.2:0/3252355682 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x564855587c60 con 0x564855820a00 2023-09-13T04:45:00.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.656+0000 7f570bfff700 1 -- 172.21.15.2:0/3252355682 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f571c011e50 con 0x564855820a00 2023-09-13T04:45:00.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.656+0000 7f570bfff700 1 -- 172.21.15.2:0/3252355682 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f572409d9e0 con 0x564855820a00 2023-09-13T04:45:00.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.656+0000 7f5746aadb80 10 client.26347 fetch_fsmap finished waiting for FSMap version 31 2023-09-13T04:45:00.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.656+0000 7f5746aadb80 10 client.26347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:45:00.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.656+0000 7f5746aadb80 20 client.26347 populate_metadata read hostname 'smithi002' 2023-09-13T04:45:00.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.656+0000 7f5746aadb80 1 --2- 172.21.15.2:0/3252355682 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56485588e950 0x564855890d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:00.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.656+0000 7f5746aadb80 4 client.26347 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-09-13T04:45:00.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.656+0000 7f5746aadb80 1 -- 172.21.15.2:0/3252355682 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56485572c050 con 0x56485588e950 2023-09-13T04:45:00.662 INFO:tasks.workunit.client.0.smithi002.stdout: 106 374553 123 276531 196 151781 301 2023-09-13T04:45:00.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.659+0000 7f572bfff700 1 --2- 172.21.15.2:0/3252355682 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56485588e950 0x564855890d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:00.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.666+0000 7f572bfff700 1 --2- 172.21.15.2:0/3252355682 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56485588e950 0x564855890d70 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70dffd713f721356 server_cookie=4f164f02c6e2b797 in_seq=0 out_seq=0 2023-09-13T04:45:00.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.666+0000 7f570bfff700 10 client.26347 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:45:00.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.666+0000 7f570bfff700 1 -- 172.21.15.2:0/3252355682 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56485572c050 con 0x56485588e950 2023-09-13T04:45:00.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.666+0000 7f570bfff700 10 client.26347 handle_command_reply: tid=0 2023-09-13T04:45:00.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.666+0000 7f5746aadb80 1 client.26347 shutdown 2023-09-13T04:45:00.679 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:45:00.679 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-13T04:45:00.679 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-09-13T04:45:00.679 INFO:teuthology.orchestra.run.smithi002.stdout: "740235c0-6e1b-4729-b1fe-33345ae948c1": { 2023-09-13T04:45:00.680 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-09-13T04:45:00.680 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "740235c0-6e1b-4729-b1fe-33345ae948c1", 2023-09-13T04:45:00.680 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-09-13T04:45:00.680 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:45:00.680 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:45:00.681 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:45:00.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.670+0000 7f5746aadb80 1 -- 172.21.15.2:0/3252355682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855820a00 msgr2=0x56485582d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:00.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.670+0000 7f5746aadb80 1 --2- 172.21.15.2:0/3252355682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855820a00 0x56485582d410 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f57240143a0 tx=0x7f57240c90d0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.673+0000 7f5746aadb80 1 -- 172.21.15.2:0/3252355682 shutdown_connections 2023-09-13T04:45:00.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.673+0000 7f5746aadb80 1 --2- 172.21.15.2:0/3252355682 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56485588e950 0x564855890d70 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.673+0000 7f5746aadb80 1 --2- 172.21.15.2:0/3252355682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564855826900 0x56485588d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.673+0000 7f5746aadb80 1 --2- 172.21.15.2:0/3252355682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564855820a00 0x56485582d410 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.673+0000 7f5746aadb80 1 --2- 172.21.15.2:0/3252355682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564855825fe0 0x56485582d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.673+0000 7f5746aadb80 1 -- 172.21.15.2:0/3252355682 >> 172.21.15.2:0/3252355682 conn(0x56485577ec80 msgr2=0x564855824ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:00.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.673+0000 7f570bfff700 0 client.26347 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:45:00.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.676+0000 7f5746aadb80 1 -- 172.21.15.2:0/3252355682 shutdown_connections 2023-09-13T04:45:00.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.676+0000 7f5746aadb80 1 -- 172.21.15.2:0/3252355682 wait complete. 2023-09-13T04:45:00.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.677+0000 7f5746aadb80 20 client.26347 trim_cache size 0 max 16384 2023-09-13T04:45:00.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.679+0000 7f57331ff700 1 -- 172.21.15.2:0/967506374 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f57140a09c0 msgr2=0x7f57140a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:00.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.679+0000 7f57331ff700 1 --2- 172.21.15.2:0/967506374 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f57140a09c0 0x7f57140a2e80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f572001af50 tx=0x7f572001a320 comp rx=0 tx=0).stop 2023-09-13T04:45:00.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.679+0000 7f57331ff700 1 -- 172.21.15.2:0/967506374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f572c1081f0 msgr2=0x7f572c115860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:00.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.679+0000 7f57331ff700 1 --2- 172.21.15.2:0/967506374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f572c1081f0 0x7f572c115860 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f572400b870 tx=0x7f572400bc30 comp rx=0 tx=0).stop 2023-09-13T04:45:00.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.680+0000 7f57331ff700 1 -- 172.21.15.2:0/967506374 shutdown_connections 2023-09-13T04:45:00.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.680+0000 7f57331ff700 1 --2- 172.21.15.2:0/967506374 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f57140a09c0 0x7f57140a2e80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.680+0000 7f57331ff700 1 --2- 172.21.15.2:0/967506374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f572c11a490 0x7f572c1a91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.680+0000 7f57331ff700 1 --2- 172.21.15.2:0/967506374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f572c1081f0 0x7f572c115860 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.680+0000 7f57331ff700 1 --2- 172.21.15.2:0/967506374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f572c10f440 0x7f572c115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:00.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.680+0000 7f57331ff700 1 -- 172.21.15.2:0/967506374 >> 172.21.15.2:0/967506374 conn(0x7f572c06cfa0 msgr2=0x7f572c11ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:00.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.681+0000 7f57331ff700 1 -- 172.21.15.2:0/967506374 shutdown_connections 2023-09-13T04:45:00.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:00.681+0000 7f57331ff700 1 -- 172.21.15.2:0/967506374 wait complete. 2023-09-13T04:45:00.726 INFO:tasks.cephfs.filesystem:scrub status for tag:740235c0-6e1b-4729-b1fe-33345ae948c1 - {'path': '/', 'tag': '740235c0-6e1b-4729-b1fe-33345ae948c1', 'options': 'recursive,force'} 2023-09-13T04:45:00.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:00 smithi149 ceph-mon[133315]: pgmap v140: 129 pgs: 129 active+clean; 521 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 423 op/s 2023-09-13T04:45:00.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:00 smithi156 ceph-mon[133675]: pgmap v140: 129 pgs: 129 active+clean; 521 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 423 op/s 2023-09-13T04:45:01.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:00 smithi002 ceph-mon[97651]: pgmap v140: 129 pgs: 129 active+clean; 521 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 423 op/s 2023-09-13T04:45:03.509 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:03 smithi002 ceph-mon[97651]: pgmap v141: 129 pgs: 129 active+clean; 521 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 3.3 MiB/s wr, 270 op/s 2023-09-13T04:45:03.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:03 smithi149 ceph-mon[133315]: pgmap v141: 129 pgs: 129 active+clean; 521 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 3.3 MiB/s wr, 270 op/s 2023-09-13T04:45:03.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:03 smithi156 ceph-mon[133675]: pgmap v141: 129 pgs: 129 active+clean; 521 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 3.3 MiB/s wr, 270 op/s 2023-09-13T04:45:05.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:05 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:45:05.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:05 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:45:05.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:05 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:45:05.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:05 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:45:05.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:05 smithi149 ceph-mon[133315]: pgmap v142: 129 pgs: 129 active+clean; 520 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 780 op/s 2023-09-13T04:45:05.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:05 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:45:05.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:05 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:45:05.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:05 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:45:05.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:05 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:45:05.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:05 smithi156 ceph-mon[133675]: pgmap v142: 129 pgs: 129 active+clean; 520 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 780 op/s 2023-09-13T04:45:05.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:05 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:45:05.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:05 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:45:05.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:05 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:45:05.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:05 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:45:05.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:05 smithi002 ceph-mon[97651]: pgmap v142: 129 pgs: 129 active+clean; 520 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 780 op/s 2023-09-13T04:45:07.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:07 smithi149 ceph-mon[133315]: pgmap v143: 129 pgs: 129 active+clean; 523 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 2.2 MiB/s wr, 768 op/s 2023-09-13T04:45:07.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:07 smithi156 ceph-mon[133675]: pgmap v143: 129 pgs: 129 active+clean; 523 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 2.2 MiB/s wr, 768 op/s 2023-09-13T04:45:07.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:07 smithi002 ceph-mon[97651]: pgmap v143: 129 pgs: 129 active+clean; 523 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 2.2 MiB/s wr, 768 op/s 2023-09-13T04:45:09.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:09 smithi149 ceph-mon[133315]: pgmap v144: 129 pgs: 129 active+clean; 523 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.0 MiB/s wr, 754 op/s 2023-09-13T04:45:09.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:09 smithi156 ceph-mon[133675]: pgmap v144: 129 pgs: 129 active+clean; 523 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.0 MiB/s wr, 754 op/s 2023-09-13T04:45:09.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:09 smithi002 ceph-mon[97651]: pgmap v144: 129 pgs: 129 active+clean; 523 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.0 MiB/s wr, 754 op/s 2023-09-13T04:45:10.659 INFO:tasks.workunit.client.0.smithi002.stdout: 112 323213 263 240048 320 127682 61 2023-09-13T04:45:10.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:10 smithi149 ceph-mon[133315]: pgmap v145: 129 pgs: 129 active+clean; 526 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 841 op/s 2023-09-13T04:45:10.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:10 smithi156 ceph-mon[133675]: pgmap v145: 129 pgs: 129 active+clean; 526 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 841 op/s 2023-09-13T04:45:11.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:10 smithi002 ceph-mon[97651]: pgmap v145: 129 pgs: 129 active+clean; 526 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 841 op/s 2023-09-13T04:45:13.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:13 smithi002 ceph-mon[97651]: pgmap v146: 129 pgs: 129 active+clean; 526 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 932 KiB/s wr, 700 op/s 2023-09-13T04:45:13.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:13 smithi149 ceph-mon[133315]: pgmap v146: 129 pgs: 129 active+clean; 526 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 932 KiB/s wr, 700 op/s 2023-09-13T04:45:13.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:13 smithi156 ceph-mon[133675]: pgmap v146: 129 pgs: 129 active+clean; 526 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 932 KiB/s wr, 700 op/s 2023-09-13T04:45:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.812+0000 7f85075ff700 1 -- 172.21.15.2:0/2122479175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85001014b0 msgr2=0x7f850010d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.812+0000 7f85075ff700 1 --2- 172.21.15.2:0/2122479175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85001014b0 0x7f850010d9c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f84f800b0a0 tx=0x7f84f800b3b0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.812+0000 7f85075ff700 1 -- 172.21.15.2:0/2122479175 shutdown_connections 2023-09-13T04:45:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.812+0000 7f85075ff700 1 --2- 172.21.15.2:0/2122479175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85001014b0 0x7f850010d9c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.812+0000 7f85075ff700 1 --2- 172.21.15.2:0/2122479175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8500106b90 0x7f8500106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.812+0000 7f85075ff700 1 --2- 172.21.15.2:0/2122479175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8500100af0 0x7f8500100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.812+0000 7f85075ff700 1 -- 172.21.15.2:0/2122479175 >> 172.21.15.2:0/2122479175 conn(0x7f85000fc5c0 msgr2=0x7f85000fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.812+0000 7f85075ff700 1 -- 172.21.15.2:0/2122479175 shutdown_connections 2023-09-13T04:45:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.812+0000 7f85075ff700 1 -- 172.21.15.2:0/2122479175 wait complete. 2023-09-13T04:45:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.813+0000 7f85075ff700 1 Processor -- start 2023-09-13T04:45:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.814+0000 7f85075ff700 1 -- start start 2023-09-13T04:45:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.814+0000 7f85075ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8500100af0 0x7f8500198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.814+0000 7f85075ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85001014b0 0x7f85001992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.814+0000 7f85075ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8500106b90 0x7f850019d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.814+0000 7f85075ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850019dd60 con 0x7f8500100af0 2023-09-13T04:45:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.814+0000 7f85075ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850019ded0 con 0x7f85001014b0 2023-09-13T04:45:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.814+0000 7f85075ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850019e090 con 0x7f8500106b90 2023-09-13T04:45:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.814+0000 7f8504b9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85001014b0 0x7f85001992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.814+0000 7f8504b9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85001014b0 0x7f85001992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:50840/0 (socket says 172.21.15.2:50840) 2023-09-13T04:45:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.814+0000 7f8504b9a700 1 -- 172.21.15.2:0/2935422610 learned_addr learned my addr 172.21.15.2:0/2935422610 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.814+0000 7f8505b9c700 1 --2- 172.21.15.2:0/2935422610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8500106b90 0x7f850019d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.814+0000 7f850539b700 1 --2- 172.21.15.2:0/2935422610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8500100af0 0x7f8500198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.815+0000 7f8505b9c700 1 -- 172.21.15.2:0/2935422610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85001014b0 msgr2=0x7f85001992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.815+0000 7f8505b9c700 1 --2- 172.21.15.2:0/2935422610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85001014b0 0x7f85001992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.815+0000 7f8505b9c700 1 -- 172.21.15.2:0/2935422610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8500100af0 msgr2=0x7f8500198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.815+0000 7f8505b9c700 1 --2- 172.21.15.2:0/2935422610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8500100af0 0x7f8500198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.815+0000 7f8505b9c700 1 -- 172.21.15.2:0/2935422610 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f8009d00 con 0x7f8500106b90 2023-09-13T04:45:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.815+0000 7f8505b9c700 1 --2- 172.21.15.2:0/2935422610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8500106b90 0x7f850019d720 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f84f80062a0 tx=0x7f84f80077f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.816+0000 7f84f67fc700 1 -- 172.21.15.2:0/2935422610 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f800e040 con 0x7f8500106b90 2023-09-13T04:45:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.816+0000 7f84f67fc700 1 -- 172.21.15.2:0/2935422610 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84f80094c0 con 0x7f8500106b90 2023-09-13T04:45:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.816+0000 7f85075ff700 1 -- 172.21.15.2:0/2935422610 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850019e310 con 0x7f8500106b90 2023-09-13T04:45:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.816+0000 7f84f67fc700 1 -- 172.21.15.2:0/2935422610 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8005040 con 0x7f8500106b90 2023-09-13T04:45:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.816+0000 7f85075ff700 1 -- 172.21.15.2:0/2935422610 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85001a5bb0 con 0x7f8500106b90 2023-09-13T04:45:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.821+0000 7f84f67fc700 1 -- 172.21.15.2:0/2935422610 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f84f801d470 con 0x7f8500106b90 2023-09-13T04:45:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.822+0000 7f851ae42b80 1 Processor -- start 2023-09-13T04:45:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.823+0000 7f851ae42b80 1 -- start start 2023-09-13T04:45:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.823+0000 7f84f67fc700 1 --2- 172.21.15.2:0/2935422610 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f84ec0a0830 0x7f84ec0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.823+0000 7f851ae42b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e69ce0 0x55f930e6a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.823+0000 7f851ae42b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f930e6a600 0x55f930e640b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.823+0000 7f851ae42b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f930e64640 0x55f930e64ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.823+0000 7f851ae42b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f930dbd720 con 0x55f930e64640 2023-09-13T04:45:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.823+0000 7f851ae42b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f930dbb040 con 0x55f930e6a600 2023-09-13T04:45:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.823+0000 7f851ae42b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f930db6930 con 0x55f930e69ce0 2023-09-13T04:45:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.824+0000 7f84f67fc700 1 -- 172.21.15.2:0/2935422610 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f84f80cd7f0 con 0x7f8500106b90 2023-09-13T04:45:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.824+0000 7f850539b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e69ce0 0x55f930e6a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.824+0000 7f850539b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e69ce0 0x55f930e6a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:47730/0 (socket says 172.21.15.2:47730) 2023-09-13T04:45:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.824+0000 7f850539b700 1 -- 172.21.15.2:0/2496070334 learned_addr learned my addr 172.21.15.2:0/2496070334 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.824+0000 7f8504b9a700 1 --2- 172.21.15.2:0/2935422610 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f84ec0a0830 0x7f84ec0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.824+0000 7f8505b9c700 1 --2- 172.21.15.2:0/2496070334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f930e64640 0x55f930e64ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.824+0000 7f8504b9a700 1 --2- 172.21.15.2:0/2496070334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f930e6a600 0x55f930e640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.824+0000 7f850539b700 1 -- 172.21.15.2:0/2496070334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f930e6a600 msgr2=0x55f930e640b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.824+0000 7f850539b700 1 --2- 172.21.15.2:0/2496070334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f930e6a600 0x55f930e640b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.824+0000 7f850539b700 1 -- 172.21.15.2:0/2496070334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f930e64640 msgr2=0x55f930e64ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.824+0000 7f850539b700 1 --2- 172.21.15.2:0/2496070334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f930e64640 0x55f930e64ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.824+0000 7f850539b700 1 -- 172.21.15.2:0/2496070334 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f930c365d0 con 0x55f930e69ce0 2023-09-13T04:45:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.825+0000 7f850539b700 1 --2- 172.21.15.2:0/2496070334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e69ce0 0x55f930e6a0c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f84fc009d00 tx=0x7f84fc011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf3364ab3161ff8 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.825+0000 7f8504b9a700 1 --2- 172.21.15.2:0/2935422610 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f84ec0a0830 0x7f84ec0a2cf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55f930e5fb40 tx=0x7f84f0009970 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.825+0000 7f84eb7fe700 1 -- 172.21.15.2:0/2496070334 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc004230 con 0x55f930e69ce0 2023-09-13T04:45:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.825+0000 7f84eb7fe700 1 -- 172.21.15.2:0/2496070334 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84fc004390 con 0x55f930e69ce0 2023-09-13T04:45:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.825+0000 7f84eb7fe700 1 -- 172.21.15.2:0/2496070334 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc019880 con 0x55f930e69ce0 2023-09-13T04:45:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.826+0000 7f851ae42b80 1 -- 172.21.15.2:0/2496070334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e69ce0 msgr2=0x55f930e6a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.826+0000 7f851ae42b80 1 --2- 172.21.15.2:0/2496070334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e69ce0 0x55f930e6a0c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f84fc009d00 tx=0x7f84fc011720 comp rx=0 tx=0).stop 2023-09-13T04:45:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.826+0000 7f851ae42b80 1 -- 172.21.15.2:0/2496070334 shutdown_connections 2023-09-13T04:45:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.826+0000 7f851ae42b80 1 --2- 172.21.15.2:0/2496070334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f930e64640 0x55f930e64ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.826+0000 7f851ae42b80 1 --2- 172.21.15.2:0/2496070334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e69ce0 0x55f930e6a0c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.826+0000 7f851ae42b80 1 --2- 172.21.15.2:0/2496070334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f930e6a600 0x55f930e640b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.826+0000 7f851ae42b80 1 -- 172.21.15.2:0/2496070334 >> 172.21.15.2:0/2496070334 conn(0x55f930dc2c20 msgr2=0x55f930e5ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.826+0000 7f851ae42b80 1 -- 172.21.15.2:0/2496070334 shutdown_connections 2023-09-13T04:45:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.826+0000 7f851ae42b80 1 -- 172.21.15.2:0/2496070334 wait complete. 2023-09-13T04:45:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.827+0000 7f851ae42b80 1 -- start start 2023-09-13T04:45:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.827+0000 7f851ae42b80 1 Processor -- start 2023-09-13T04:45:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.828+0000 7f851ae42b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e64640 0x55f930e6d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.828+0000 7f851ae42b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f930e69ce0 0x55f930e6ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.828+0000 7f851ae42b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f930e6a600 0x55f930e6e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.828+0000 7f851ae42b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f930d54b50 con 0x55f930e6a600 2023-09-13T04:45:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.828+0000 7f851ae42b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f930d54850 con 0x55f930e69ce0 2023-09-13T04:45:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.828+0000 7f851ae42b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f930c388c0 con 0x55f930e64640 2023-09-13T04:45:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.828+0000 7f851ae42b80 10 client.0 _ll_register_callbacks cb 0x55f930d52620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:45:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.828+0000 7f850539b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e64640 0x55f930e6d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.828+0000 7f8504b9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f930e69ce0 0x55f930e6ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.828+0000 7f850539b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e64640 0x55f930e6d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:47734/0 (socket says 172.21.15.2:47734) 2023-09-13T04:45:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.828+0000 7f850539b700 1 -- 172.21.15.2:0/2931443371 learned_addr learned my addr 172.21.15.2:0/2931443371 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.828+0000 7f8505b9c700 1 --2- 172.21.15.2:0/2931443371 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f930e6a600 0x55f930e6e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.828+0000 7f850539b700 1 -- 172.21.15.2:0/2931443371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f930e69ce0 msgr2=0x55f930e6ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.828+0000 7f850539b700 1 --2- 172.21.15.2:0/2931443371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f930e69ce0 0x55f930e6ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.828+0000 7f850539b700 1 -- 172.21.15.2:0/2931443371 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f930e6a600 msgr2=0x55f930e6e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.828+0000 7f850539b700 1 --2- 172.21.15.2:0/2931443371 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f930e6a600 0x55f930e6e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.828+0000 7f850539b700 1 -- 172.21.15.2:0/2931443371 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84fc009960 con 0x55f930e64640 2023-09-13T04:45:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.828+0000 7f8505b9c700 1 --2- 172.21.15.2:0/2931443371 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f930e6a600 0x55f930e6e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:45:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.829+0000 7f850539b700 1 --2- 172.21.15.2:0/2931443371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e64640 0x55f930e6d890 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f84fc000c00 tx=0x7f84fc0038b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36d62dfff374078c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.829+0000 7f84e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:45:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.829+0000 7f84e8ff9700 1 -- 172.21.15.2:0/2931443371 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc005ce0 con 0x55f930e64640 2023-09-13T04:45:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.829+0000 7f851ae42b80 1 -- 172.21.15.2:0/2931443371 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f930e57260 con 0x55f930e64640 2023-09-13T04:45:13.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.829+0000 7f84e8ff9700 1 -- 172.21.15.2:0/2931443371 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f84fc005250 con 0x55f930e64640 2023-09-13T04:45:13.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.830+0000 7f84e8ff9700 1 -- 172.21.15.2:0/2931443371 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84fc005410 con 0x55f930e64640 2023-09-13T04:45:13.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.830+0000 7f851ae42b80 10 client.26359 fetch_fsmap learned FSMap version 31 2023-09-13T04:45:13.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.830+0000 7f851ae42b80 1 -- 172.21.15.2:0/2931443371 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f930dce740 con 0x55f930e64640 2023-09-13T04:45:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.830+0000 7f84e8ff9700 1 -- 172.21.15.2:0/2931443371 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f930e57260 con 0x55f930e64640 2023-09-13T04:45:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.830+0000 7f84e8ff9700 1 -- 172.21.15.2:0/2931443371 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f84fc028020 con 0x55f930e64640 2023-09-13T04:45:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.830+0000 7f851ae42b80 10 client.26359 fetch_fsmap finished waiting for FSMap version 31 2023-09-13T04:45:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.830+0000 7f851ae42b80 10 client.26359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:45:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.830+0000 7f851ae42b80 20 client.26359 populate_metadata read hostname 'smithi002' 2023-09-13T04:45:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.830+0000 7f851ae42b80 1 --2- 172.21.15.2:0/2931443371 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f930e7de70 0x55f930e80290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.830+0000 7f851ae42b80 4 client.26359 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:45:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.830+0000 7f851ae42b80 1 -- 172.21.15.2:0/2931443371 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f930daf910 con 0x55f930e7de70 2023-09-13T04:45:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.831+0000 7f8504b9a700 1 --2- 172.21.15.2:0/2931443371 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f930e7de70 0x55f930e80290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.831+0000 7f8504b9a700 1 --2- 172.21.15.2:0/2931443371 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f930e7de70 0x55f930e80290 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efb4c8434cc03607 server_cookie=df558f2b57021aa6 in_seq=0 out_seq=0 2023-09-13T04:45:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.831+0000 7f84e8ff9700 10 client.26359 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:45:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.832+0000 7f84e8ff9700 1 -- 172.21.15.2:0/2931443371 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f930daf910 con 0x55f930e7de70 2023-09-13T04:45:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.832+0000 7f84e8ff9700 10 client.26359 handle_command_reply: tid=0 2023-09-13T04:45:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.832+0000 7f851ae42b80 1 client.26359 shutdown 2023-09-13T04:45:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.833+0000 7f851ae42b80 1 -- 172.21.15.2:0/2931443371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e64640 msgr2=0x55f930e6d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.833+0000 7f851ae42b80 1 --2- 172.21.15.2:0/2931443371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e64640 0x55f930e6d890 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f84fc000c00 tx=0x7f84fc0038b0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.833+0000 7f851ae42b80 1 -- 172.21.15.2:0/2931443371 shutdown_connections 2023-09-13T04:45:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.833+0000 7f851ae42b80 1 --2- 172.21.15.2:0/2931443371 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f930e7de70 0x55f930e80290 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.833+0000 7f851ae42b80 1 --2- 172.21.15.2:0/2931443371 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f930e6a600 0x55f930e6e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.833+0000 7f851ae42b80 1 --2- 172.21.15.2:0/2931443371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e64640 0x55f930e6d890 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.833+0000 7f84e8ff9700 0 client.26359 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:45:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.833+0000 7f851ae42b80 1 --2- 172.21.15.2:0/2931443371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f930e69ce0 0x55f930e6ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.833+0000 7f851ae42b80 1 -- 172.21.15.2:0/2931443371 >> 172.21.15.2:0/2931443371 conn(0x55f930dc2c20 msgr2=0x55f930e630a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.833+0000 7f851ae42b80 1 -- 172.21.15.2:0/2931443371 shutdown_connections 2023-09-13T04:45:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.833+0000 7f851ae42b80 1 -- 172.21.15.2:0/2931443371 wait complete. 2023-09-13T04:45:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.833+0000 7f851ae42b80 20 client.26359 trim_cache size 0 max 16384 2023-09-13T04:45:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.855+0000 7f851ae42b80 1 Processor -- start 2023-09-13T04:45:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.855+0000 7f851ae42b80 1 -- start start 2023-09-13T04:45:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.855+0000 7f851ae42b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f930e64640 0x55f930e67df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.855+0000 7f851ae42b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f930e69ce0 0x55f930e6e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.855+0000 7f851ae42b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e6a600 0x55f930e68950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.855+0000 7f851ae42b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f930e68330 con 0x55f930e69ce0 2023-09-13T04:45:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.855+0000 7f851ae42b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f930e6bdf0 con 0x55f930e64640 2023-09-13T04:45:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.855+0000 7f851ae42b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f930e68540 con 0x55f930e6a600 2023-09-13T04:45:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.855+0000 7f850539b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f930e64640 0x55f930e67df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.855+0000 7f8505b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e6a600 0x55f930e68950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.856+0000 7f850539b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f930e64640 0x55f930e67df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:50864/0 (socket says 172.21.15.2:50864) 2023-09-13T04:45:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.856+0000 7f850539b700 1 -- 172.21.15.2:0/611809729 learned_addr learned my addr 172.21.15.2:0/611809729 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.856+0000 7f8504b9a700 1 --2- 172.21.15.2:0/611809729 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f930e69ce0 0x55f930e6e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.856+0000 7f850539b700 1 -- 172.21.15.2:0/611809729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e6a600 msgr2=0x55f930e68950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.856+0000 7f850539b700 1 --2- 172.21.15.2:0/611809729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e6a600 0x55f930e68950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.856+0000 7f850539b700 1 -- 172.21.15.2:0/611809729 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f930e69ce0 msgr2=0x55f930e6e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.856+0000 7f850539b700 1 --2- 172.21.15.2:0/611809729 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f930e69ce0 0x55f930e6e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.856+0000 7f850539b700 1 -- 172.21.15.2:0/611809729 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f930be2d80 con 0x55f930e64640 2023-09-13T04:45:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.856+0000 7f8505b9c700 1 --2- 172.21.15.2:0/611809729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e6a600 0x55f930e68950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:45:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.856+0000 7f850539b700 1 --2- 172.21.15.2:0/611809729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f930e64640 0x55f930e67df0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f84fc005a80 tx=0x7f84fc003ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6889497278742b8 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.856+0000 7f84ebfff700 1 -- 172.21.15.2:0/611809729 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc011b20 con 0x55f930e64640 2023-09-13T04:45:13.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.857+0000 7f84ebfff700 1 -- 172.21.15.2:0/611809729 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84fc005c20 con 0x55f930e64640 2023-09-13T04:45:13.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.857+0000 7f84ebfff700 1 -- 172.21.15.2:0/611809729 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc004760 con 0x55f930e64640 2023-09-13T04:45:13.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.857+0000 7f851ae42b80 1 -- 172.21.15.2:0/611809729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f930e64640 msgr2=0x55f930e67df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:13.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.857+0000 7f851ae42b80 1 --2- 172.21.15.2:0/611809729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f930e64640 0x55f930e67df0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f84fc005a80 tx=0x7f84fc003ea0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.858+0000 7f851ae42b80 1 -- 172.21.15.2:0/611809729 shutdown_connections 2023-09-13T04:45:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.858+0000 7f851ae42b80 1 --2- 172.21.15.2:0/611809729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e6a600 0x55f930e68950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.858+0000 7f851ae42b80 1 --2- 172.21.15.2:0/611809729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f930e64640 0x55f930e67df0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.858+0000 7f851ae42b80 1 --2- 172.21.15.2:0/611809729 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f930e69ce0 0x55f930e6e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.858+0000 7f851ae42b80 1 -- 172.21.15.2:0/611809729 >> 172.21.15.2:0/611809729 conn(0x55f930dc2c20 msgr2=0x55f930e676e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.858+0000 7f851ae42b80 1 -- 172.21.15.2:0/611809729 shutdown_connections 2023-09-13T04:45:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.858+0000 7f851ae42b80 1 -- 172.21.15.2:0/611809729 wait complete. 2023-09-13T04:45:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.858+0000 7f851ae42b80 1 -- start start 2023-09-13T04:45:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.858+0000 7f851ae42b80 1 Processor -- start 2023-09-13T04:45:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.859+0000 7f851ae42b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f930e64640 0x55f930e72b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.859+0000 7f851ae42b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f930e69ce0 0x55f930e73060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.859+0000 7f851ae42b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e6a600 0x55f930ed9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.859+0000 7f851ae42b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f930e73770 con 0x55f930e64640 2023-09-13T04:45:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.859+0000 7f851ae42b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f930ed99b0 con 0x55f930e69ce0 2023-09-13T04:45:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.859+0000 7f851ae42b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f930ed9b20 con 0x55f930e6a600 2023-09-13T04:45:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.859+0000 7f851ae42b80 10 client.0 _ll_register_callbacks cb 0x55f930d52620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:45:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.859+0000 7f8504b9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f930e69ce0 0x55f930e73060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.859+0000 7f8504b9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f930e69ce0 0x55f930e73060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:50870/0 (socket says 172.21.15.2:50870) 2023-09-13T04:45:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.859+0000 7f8504b9a700 1 -- 172.21.15.2:0/2343854758 learned_addr learned my addr 172.21.15.2:0/2343854758 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.859+0000 7f8505b9c700 1 --2- 172.21.15.2:0/2343854758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e6a600 0x55f930ed9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.860+0000 7f850539b700 1 --2- 172.21.15.2:0/2343854758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f930e64640 0x55f930e72b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.860+0000 7f8505b9c700 1 -- 172.21.15.2:0/2343854758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f930e69ce0 msgr2=0x55f930e73060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.860+0000 7f8505b9c700 1 --2- 172.21.15.2:0/2343854758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f930e69ce0 0x55f930e73060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.860+0000 7f8505b9c700 1 -- 172.21.15.2:0/2343854758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f930e64640 msgr2=0x55f930e72b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.860+0000 7f8505b9c700 1 --2- 172.21.15.2:0/2343854758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f930e64640 0x55f930e72b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.860+0000 7f8505b9c700 1 -- 172.21.15.2:0/2343854758 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85001a5bb0 con 0x55f930e6a600 2023-09-13T04:45:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.860+0000 7f8505b9c700 1 --2- 172.21.15.2:0/2343854758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e6a600 0x55f930ed9470 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f84f80cabd0 tx=0x7f84f809a320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecfadb50dec262a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.860+0000 7f84e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:45:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.860+0000 7f84e8ff9700 1 -- 172.21.15.2:0/2343854758 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f809b0c0 con 0x55f930e6a600 2023-09-13T04:45:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.860+0000 7f84e8ff9700 1 -- 172.21.15.2:0/2343854758 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f84f809a9c0 con 0x55f930e6a600 2023-09-13T04:45:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.861+0000 7f851ae42b80 1 -- 172.21.15.2:0/2343854758 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84fc004390 con 0x55f930e6a600 2023-09-13T04:45:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.861+0000 7f84e8ff9700 1 -- 172.21.15.2:0/2343854758 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84f809e7f0 con 0x55f930e6a600 2023-09-13T04:45:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.861+0000 7f851ae42b80 10 client.26365 fetch_fsmap learned FSMap version 31 2023-09-13T04:45:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.861+0000 7f851ae42b80 1 -- 172.21.15.2:0/2343854758 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f930dce740 con 0x55f930e6a600 2023-09-13T04:45:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.861+0000 7f84e8ff9700 1 -- 172.21.15.2:0/2343854758 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84fc004390 con 0x55f930e6a600 2023-09-13T04:45:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.862+0000 7f84e8ff9700 1 -- 172.21.15.2:0/2343854758 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f84f809ac20 con 0x55f930e6a600 2023-09-13T04:45:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.862+0000 7f851ae42b80 10 client.26365 fetch_fsmap finished waiting for FSMap version 31 2023-09-13T04:45:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.862+0000 7f851ae42b80 10 client.26365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:45:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.862+0000 7f851ae42b80 20 client.26365 populate_metadata read hostname 'smithi002' 2023-09-13T04:45:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.862+0000 7f851ae42b80 1 --2- 172.21.15.2:0/2343854758 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f930edb0b0 0x55f930edd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.862+0000 7f851ae42b80 4 client.26365 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:45:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.862+0000 7f851ae42b80 1 -- 172.21.15.2:0/2343854758 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f930d6fff0 con 0x55f930edb0b0 2023-09-13T04:45:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.862+0000 7f850539b700 1 --2- 172.21.15.2:0/2343854758 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f930edb0b0 0x55f930edd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.863+0000 7f850539b700 1 --2- 172.21.15.2:0/2343854758 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f930edb0b0 0x55f930edd4a0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90c0cc37803dce91 server_cookie=1413152311555b5c in_seq=0 out_seq=0 2023-09-13T04:45:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:13.863+0000 7f84e8ff9700 10 client.26365 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:45:15.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:15 smithi149 ceph-mon[133315]: pgmap v147: 129 pgs: 129 active+clean; 526 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 1.0 MiB/s wr, 709 op/s 2023-09-13T04:45:15.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:15 smithi156 ceph-mon[133675]: pgmap v147: 129 pgs: 129 active+clean; 526 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 1.0 MiB/s wr, 709 op/s 2023-09-13T04:45:15.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:15 smithi002 ceph-mon[97651]: pgmap v147: 129 pgs: 129 active+clean; 526 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 1.0 MiB/s wr, 709 op/s 2023-09-13T04:45:17.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:17 smithi149 ceph-mon[133315]: pgmap v148: 129 pgs: 129 active+clean; 526 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 684 KiB/s wr, 209 op/s 2023-09-13T04:45:17.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:17 smithi156 ceph-mon[133675]: pgmap v148: 129 pgs: 129 active+clean; 526 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 684 KiB/s wr, 209 op/s 2023-09-13T04:45:17.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:17 smithi002 ceph-mon[97651]: pgmap v148: 129 pgs: 129 active+clean; 526 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 684 KiB/s wr, 209 op/s 2023-09-13T04:45:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.875+0000 7f84e8ff9700 1 -- 172.21.15.2:0/2343854758 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f930d6fff0 con 0x55f930edb0b0 2023-09-13T04:45:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.875+0000 7f84e8ff9700 10 client.26365 handle_command_reply: tid=0 2023-09-13T04:45:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.875+0000 7f851ae42b80 1 client.26365 shutdown 2023-09-13T04:45:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.875+0000 7f851ae42b80 1 -- 172.21.15.2:0/2343854758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e6a600 msgr2=0x55f930ed9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.875+0000 7f851ae42b80 1 --2- 172.21.15.2:0/2343854758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e6a600 0x55f930ed9470 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f84f80cabd0 tx=0x7f84f809a320 comp rx=0 tx=0).stop 2023-09-13T04:45:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.875+0000 7f851ae42b80 1 -- 172.21.15.2:0/2343854758 shutdown_connections 2023-09-13T04:45:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.875+0000 7f851ae42b80 1 --2- 172.21.15.2:0/2343854758 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f930edb0b0 0x55f930edd4a0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.875+0000 7f851ae42b80 1 --2- 172.21.15.2:0/2343854758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f930e6a600 0x55f930ed9470 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.875+0000 7f851ae42b80 1 --2- 172.21.15.2:0/2343854758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f930e64640 0x55f930e72b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.875+0000 7f84e8ff9700 0 client.26365 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:45:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.875+0000 7f851ae42b80 1 --2- 172.21.15.2:0/2343854758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f930e69ce0 0x55f930e73060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.875+0000 7f851ae42b80 1 -- 172.21.15.2:0/2343854758 >> 172.21.15.2:0/2343854758 conn(0x55f930dc2c20 msgr2=0x55f930e6e050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.875+0000 7f851ae42b80 1 -- 172.21.15.2:0/2343854758 shutdown_connections 2023-09-13T04:45:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.875+0000 7f851ae42b80 1 -- 172.21.15.2:0/2343854758 wait complete. 2023-09-13T04:45:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.876+0000 7f851ae42b80 20 client.26365 trim_cache size 0 max 16384 2023-09-13T04:45:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:45:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:45:17.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:45:17.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:45:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.876+0000 7f85075ff700 1 -- 172.21.15.2:0/2935422610 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f84ec0a0830 msgr2=0x7f84ec0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.876+0000 7f85075ff700 1 --2- 172.21.15.2:0/2935422610 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f84ec0a0830 0x7f84ec0a2cf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55f930e5fb40 tx=0x7f84f0009970 comp rx=0 tx=0).stop 2023-09-13T04:45:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.876+0000 7f85075ff700 1 -- 172.21.15.2:0/2935422610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8500106b90 msgr2=0x7f850019d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.876+0000 7f85075ff700 1 --2- 172.21.15.2:0/2935422610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8500106b90 0x7f850019d720 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f84f80062a0 tx=0x7f84f80077f0 comp rx=0 tx=0).stop 2023-09-13T04:45:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.876+0000 7f85075ff700 1 -- 172.21.15.2:0/2935422610 shutdown_connections 2023-09-13T04:45:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.876+0000 7f85075ff700 1 --2- 172.21.15.2:0/2935422610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8500106b90 0x7f850019d720 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.876+0000 7f85075ff700 1 --2- 172.21.15.2:0/2935422610 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f84ec0a0830 0x7f84ec0a2cf0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.876+0000 7f85075ff700 1 --2- 172.21.15.2:0/2935422610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8500100af0 0x7f8500198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.876+0000 7f85075ff700 1 --2- 172.21.15.2:0/2935422610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85001014b0 0x7f85001992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:17.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.876+0000 7f85075ff700 1 -- 172.21.15.2:0/2935422610 >> 172.21.15.2:0/2935422610 conn(0x7f85000fc5c0 msgr2=0x7f85000fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:17.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.877+0000 7f85075ff700 1 -- 172.21.15.2:0/2935422610 shutdown_connections 2023-09-13T04:45:17.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:17.877+0000 7f85075ff700 1 -- 172.21.15.2:0/2935422610 wait complete. 2023-09-13T04:45:19.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:19 smithi149 ceph-mon[133315]: pgmap v149: 129 pgs: 129 active+clean; 526 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 373 KiB/s wr, 106 op/s 2023-09-13T04:45:19.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:19 smithi156 ceph-mon[133675]: pgmap v149: 129 pgs: 129 active+clean; 526 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 373 KiB/s wr, 106 op/s 2023-09-13T04:45:19.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:19 smithi002 ceph-mon[97651]: pgmap v149: 129 pgs: 129 active+clean; 526 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 373 KiB/s wr, 106 op/s 2023-09-13T04:45:20.659 INFO:tasks.workunit.client.0.smithi002.stdout: 113 288096 107 215505 27 121318 110 2023-09-13T04:45:20.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:20 smithi149 ceph-mon[133315]: pgmap v150: 129 pgs: 129 active+clean; 552 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 2.9 MiB/s wr, 263 op/s 2023-09-13T04:45:20.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:20 smithi156 ceph-mon[133675]: pgmap v150: 129 pgs: 129 active+clean; 552 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 2.9 MiB/s wr, 263 op/s 2023-09-13T04:45:21.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:20 smithi002 ceph-mon[97651]: pgmap v150: 129 pgs: 129 active+clean; 552 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 2.9 MiB/s wr, 263 op/s 2023-09-13T04:45:23.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:23 smithi149 ceph-mon[133315]: pgmap v151: 129 pgs: 129 active+clean; 552 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 2.6 MiB/s wr, 176 op/s 2023-09-13T04:45:23.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:23 smithi156 ceph-mon[133675]: pgmap v151: 129 pgs: 129 active+clean; 552 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 2.6 MiB/s wr, 176 op/s 2023-09-13T04:45:23.773 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:23 smithi002 ceph-mon[97651]: pgmap v151: 129 pgs: 129 active+clean; 552 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 2.6 MiB/s wr, 176 op/s 2023-09-13T04:45:25.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:25 smithi149 ceph-mon[133315]: pgmap v152: 129 pgs: 129 active+clean; 558 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 232 op/s 2023-09-13T04:45:25.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:25 smithi156 ceph-mon[133675]: pgmap v152: 129 pgs: 129 active+clean; 558 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 232 op/s 2023-09-13T04:45:25.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:25 smithi002 ceph-mon[97651]: pgmap v152: 129 pgs: 129 active+clean; 558 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 232 op/s 2023-09-13T04:45:27.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:27 smithi149 ceph-mon[133315]: pgmap v153: 129 pgs: 129 active+clean; 561 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 240 op/s 2023-09-13T04:45:27.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:27 smithi156 ceph-mon[133675]: pgmap v153: 129 pgs: 129 active+clean; 561 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 240 op/s 2023-09-13T04:45:27.812 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:27 smithi002 ceph-mon[97651]: pgmap v153: 129 pgs: 129 active+clean; 561 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 240 op/s 2023-09-13T04:45:29.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:29 smithi156 ceph-mon[133675]: pgmap v154: 129 pgs: 129 active+clean; 561 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 230 op/s 2023-09-13T04:45:29.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:29 smithi002 ceph-mon[97651]: pgmap v154: 129 pgs: 129 active+clean; 561 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 230 op/s 2023-09-13T04:45:29.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:29 smithi149 ceph-mon[133315]: pgmap v154: 129 pgs: 129 active+clean; 561 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 230 op/s 2023-09-13T04:45:30.663 INFO:tasks.workunit.client.0.smithi002.stdout: 122 727273 586 534445 450 276243 789 2023-09-13T04:45:30.727 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T04:45:30.861 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:30 smithi002 ceph-mon[97651]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:45:30.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:30 smithi149 ceph-mon[133315]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:45:30.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:30 smithi156 ceph-mon[133675]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:45:31.761 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:31 smithi002 ceph-mon[97651]: pgmap v155: 129 pgs: 129 active+clean; 570 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 284 op/s 2023-09-13T04:45:31.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:31 smithi149 ceph-mon[133315]: pgmap v155: 129 pgs: 129 active+clean; 570 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 284 op/s 2023-09-13T04:45:31.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:31 smithi156 ceph-mon[133675]: pgmap v155: 129 pgs: 129 active+clean; 570 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 284 op/s 2023-09-13T04:45:32.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:32 smithi149 ceph-mon[133315]: pgmap v156: 129 pgs: 129 active+clean; 570 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 127 op/s 2023-09-13T04:45:32.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:32 smithi156 ceph-mon[133675]: pgmap v156: 129 pgs: 129 active+clean; 570 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 127 op/s 2023-09-13T04:45:33.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:32 smithi002 ceph-mon[97651]: pgmap v156: 129 pgs: 129 active+clean; 570 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 127 op/s 2023-09-13T04:45:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.820+0000 7f607ebff700 1 -- 172.21.15.2:0/44503951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f607810f890 msgr2=0x7f60781081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.821+0000 7f607ebff700 1 --2- 172.21.15.2:0/44503951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f607810f890 0x7f60781081f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f607000b0a0 tx=0x7f607000b3b0 comp rx=0 tx=0).stop 2023-09-13T04:45:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.825+0000 7f607ebff700 1 -- 172.21.15.2:0/44503951 shutdown_connections 2023-09-13T04:45:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.825+0000 7f607ebff700 1 --2- 172.21.15.2:0/44503951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f607811a490 0x7f607811c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.825+0000 7f607ebff700 1 --2- 172.21.15.2:0/44503951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f607810f420 0x7f6078107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.825+0000 7f607ebff700 1 --2- 172.21.15.2:0/44503951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f607810f890 0x7f60781081f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.825+0000 7f607ebff700 1 -- 172.21.15.2:0/44503951 >> 172.21.15.2:0/44503951 conn(0x7f607806cda0 msgr2=0x7f607806d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.826+0000 7f607ebff700 1 -- 172.21.15.2:0/44503951 shutdown_connections 2023-09-13T04:45:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.826+0000 7f607ebff700 1 -- 172.21.15.2:0/44503951 wait complete. 2023-09-13T04:45:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.827+0000 7f607ebff700 1 Processor -- start 2023-09-13T04:45:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.828+0000 7f607ebff700 1 -- start start 2023-09-13T04:45:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.828+0000 7f607ebff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f607810f420 0x7f6078119960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.828+0000 7f607ebff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f607810f890 0x7f6078119ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.828+0000 7f607ebff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f607811a490 0x7f60781138c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.828+0000 7f607ebff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6078114090 con 0x7f607810f420 2023-09-13T04:45:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.828+0000 7f607ebff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6078114200 con 0x7f607811a490 2023-09-13T04:45:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.828+0000 7f607ebff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60781143c0 con 0x7f607810f890 2023-09-13T04:45:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.829+0000 7f6077fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f607810f890 0x7f6078119ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.829+0000 7f6077fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f607810f890 0x7f6078119ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:51788/0 (socket says 172.21.15.2:51788) 2023-09-13T04:45:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.829+0000 7f6077fff700 1 -- 172.21.15.2:0/883555920 learned_addr learned my addr 172.21.15.2:0/883555920 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.829+0000 7f6077fff700 1 -- 172.21.15.2:0/883555920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f607811a490 msgr2=0x7f60781138c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-13T04:45:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.829+0000 7f6077fff700 1 --2- 172.21.15.2:0/883555920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f607811a490 0x7f60781138c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.829+0000 7f6077fff700 1 -- 172.21.15.2:0/883555920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f607810f420 msgr2=0x7f6078119960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.829+0000 7f6077fff700 1 --2- 172.21.15.2:0/883555920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f607810f420 0x7f6078119960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.829+0000 7f6077fff700 1 -- 172.21.15.2:0/883555920 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6070009d00 con 0x7f607810f890 2023-09-13T04:45:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.830+0000 7f6077fff700 1 --2- 172.21.15.2:0/883555920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f607810f890 0x7f6078119ea0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f607000b070 tx=0x7f6070005b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.875+0000 7f6075ffb700 1 -- 172.21.15.2:0/883555920 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607000e040 con 0x7f607810f890 2023-09-13T04:45:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.875+0000 7f6075ffb700 1 -- 172.21.15.2:0/883555920 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6070012ae0 con 0x7f607810f890 2023-09-13T04:45:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.875+0000 7f6075ffb700 1 -- 172.21.15.2:0/883555920 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6070005710 con 0x7f607810f890 2023-09-13T04:45:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.889+0000 7f607ebff700 1 -- 172.21.15.2:0/883555920 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60781bb160 con 0x7f607810f890 2023-09-13T04:45:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.889+0000 7f607ebff700 1 -- 172.21.15.2:0/883555920 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60781bb4d0 con 0x7f607810f890 2023-09-13T04:45:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.917+0000 7f6075ffb700 1 -- 172.21.15.2:0/883555920 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f607001d480 con 0x7f607810f890 2023-09-13T04:45:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.922+0000 7f6075ffb700 1 --2- 172.21.15.2:0/883555920 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f60600a09c0 0x7f60600a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.922+0000 7f6075ffb700 1 -- 172.21.15.2:0/883555920 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f60700ce6b0 con 0x7f607810f890 2023-09-13T04:45:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.924+0000 7f6092506b80 1 Processor -- start 2023-09-13T04:45:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.926+0000 7f607c99b700 1 --2- 172.21.15.2:0/883555920 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f60600a09c0 0x7f60600a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.926+0000 7f6092506b80 1 -- start start 2023-09-13T04:45:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.928+0000 7f6092506b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a250e06be0 0x55a250e06fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.928+0000 7f6092506b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a250e07500 0x55a250e02300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.928+0000 7f6092506b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e02840 0x55a250e02c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.928+0000 7f6092506b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a250d5b720 con 0x55a250e06be0 2023-09-13T04:45:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.928+0000 7f6092506b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a250d59040 con 0x55a250e07500 2023-09-13T04:45:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.928+0000 7f6092506b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a250d54930 con 0x55a250e02840 2023-09-13T04:45:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.930+0000 7f6077fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a250e07500 0x55a250e02300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.930+0000 7f6077fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a250e07500 0x55a250e02300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:41268/0 (socket says 172.21.15.2:41268) 2023-09-13T04:45:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.930+0000 7f6077fff700 1 -- 172.21.15.2:0/2899377642 learned_addr learned my addr 172.21.15.2:0/2899377642 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.939+0000 7f607c99b700 1 --2- 172.21.15.2:0/2899377642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a250e06be0 0x55a250e06fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:33.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.940+0000 7f607d19c700 1 --2- 172.21.15.2:0/2899377642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e02840 0x55a250e02c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:33.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.947+0000 7f6077fff700 1 -- 172.21.15.2:0/2899377642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e02840 msgr2=0x55a250e02c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.947+0000 7f6077fff700 1 --2- 172.21.15.2:0/2899377642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e02840 0x55a250e02c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.947+0000 7f6077fff700 1 -- 172.21.15.2:0/2899377642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a250e06be0 msgr2=0x55a250e06fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.947+0000 7f6077fff700 1 --2- 172.21.15.2:0/2899377642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a250e06be0 0x55a250e06fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.947+0000 7f6077fff700 1 -- 172.21.15.2:0/2899377642 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a250bd45d0 con 0x55a250e07500 2023-09-13T04:45:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.948+0000 7f607c99b700 1 --2- 172.21.15.2:0/883555920 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f60600a09c0 0x7f60600a2e80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6068009770 tx=0x7f6068006cd0 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:34.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:33.975+0000 7f6077fff700 1 --2- 172.21.15.2:0/2899377642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a250e07500 0x55a250e02300 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f60700a33c0 tx=0x7f607009b370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd6467a8a0144150 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.199+0000 7f6066ffd700 1 -- 172.21.15.2:0/2899377642 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60700a38f0 con 0x55a250e07500 2023-09-13T04:45:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.199+0000 7f6066ffd700 1 -- 172.21.15.2:0/2899377642 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6070005870 con 0x55a250e07500 2023-09-13T04:45:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.199+0000 7f6066ffd700 1 -- 172.21.15.2:0/2899377642 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60700cbd10 con 0x55a250e07500 2023-09-13T04:45:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.200+0000 7f6092506b80 1 -- 172.21.15.2:0/2899377642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a250e07500 msgr2=0x55a250e02300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.200+0000 7f6092506b80 1 --2- 172.21.15.2:0/2899377642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a250e07500 0x55a250e02300 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f60700a33c0 tx=0x7f607009b370 comp rx=0 tx=0).stop 2023-09-13T04:45:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.202+0000 7f6092506b80 1 -- 172.21.15.2:0/2899377642 shutdown_connections 2023-09-13T04:45:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.202+0000 7f6092506b80 1 --2- 172.21.15.2:0/2899377642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e02840 0x55a250e02c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.202+0000 7f6092506b80 1 --2- 172.21.15.2:0/2899377642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a250e06be0 0x55a250e06fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.202+0000 7f6092506b80 1 --2- 172.21.15.2:0/2899377642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a250e07500 0x55a250e02300 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.202+0000 7f6092506b80 1 -- 172.21.15.2:0/2899377642 >> 172.21.15.2:0/2899377642 conn(0x55a250d60c20 msgr2=0x55a250dfcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.208+0000 7f6092506b80 1 -- 172.21.15.2:0/2899377642 shutdown_connections 2023-09-13T04:45:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.208+0000 7f6092506b80 1 -- 172.21.15.2:0/2899377642 wait complete. 2023-09-13T04:45:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.209+0000 7f6092506b80 1 -- start start 2023-09-13T04:45:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.209+0000 7f6092506b80 1 Processor -- start 2023-09-13T04:45:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.210+0000 7f6092506b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a250e02840 0x55a250e0e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.210+0000 7f6092506b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a250e06be0 0x55a250e0b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.210+0000 7f6092506b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e07500 0x55a250e0bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.210+0000 7f6092506b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a250cf2b50 con 0x55a250e02840 2023-09-13T04:45:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.210+0000 7f6092506b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a250cf2850 con 0x55a250e06be0 2023-09-13T04:45:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.210+0000 7f6092506b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a250bd68c0 con 0x55a250e07500 2023-09-13T04:45:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.210+0000 7f6092506b80 10 client.0 _ll_register_callbacks cb 0x55a250cf0620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:45:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.212+0000 7f607d19c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e07500 0x55a250e0bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.212+0000 7f607d19c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e07500 0x55a250e0bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:48244/0 (socket says 172.21.15.2:48244) 2023-09-13T04:45:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.212+0000 7f607d19c700 1 -- 172.21.15.2:0/3801070097 learned_addr learned my addr 172.21.15.2:0/3801070097 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.226+0000 7f607c99b700 1 --2- 172.21.15.2:0/3801070097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a250e02840 0x55a250e0e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.227+0000 7f6077fff700 1 --2- 172.21.15.2:0/3801070097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a250e06be0 0x55a250e0b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.227+0000 7f607d19c700 1 -- 172.21.15.2:0/3801070097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a250e06be0 msgr2=0x55a250e0b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.227+0000 7f607d19c700 1 --2- 172.21.15.2:0/3801070097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a250e06be0 0x55a250e0b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.227+0000 7f607d19c700 1 -- 172.21.15.2:0/3801070097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a250e02840 msgr2=0x55a250e0e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.227+0000 7f607d19c700 1 --2- 172.21.15.2:0/3801070097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a250e02840 0x55a250e0e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.227+0000 7f607d19c700 1 -- 172.21.15.2:0/3801070097 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60781bb4d0 con 0x55a250e07500 2023-09-13T04:45:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.231+0000 7f607d19c700 1 --2- 172.21.15.2:0/3801070097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e07500 0x55a250e0bff0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f606c00aee0 tx=0x7f606c00c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1907417d729c821a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.603+0000 7f605bfff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:45:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.603+0000 7f605bfff700 1 -- 172.21.15.2:0/3801070097 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c0113f0 con 0x55a250e07500 2023-09-13T04:45:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.603+0000 7f605bfff700 1 -- 172.21.15.2:0/3801070097 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f606c011550 con 0x55a250e07500 2023-09-13T04:45:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.603+0000 7f6092506b80 1 -- 172.21.15.2:0/3801070097 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a250dc7ef0 con 0x55a250e07500 2023-09-13T04:45:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.702+0000 7f605bfff700 1 -- 172.21.15.2:0/3801070097 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f606c011710 con 0x55a250e07500 2023-09-13T04:45:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.702+0000 7f6092506b80 10 client.26371 fetch_fsmap learned FSMap version 33 2023-09-13T04:45:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.702+0000 7f6092506b80 1 -- 172.21.15.2:0/3801070097 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55a250d6c740 con 0x55a250e07500 2023-09-13T04:45:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.709+0000 7f605bfff700 1 -- 172.21.15.2:0/3801070097 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a250dc7ef0 con 0x55a250e07500 2023-09-13T04:45:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.709+0000 7f605bfff700 1 -- 172.21.15.2:0/3801070097 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f606c01aae0 con 0x55a250e07500 2023-09-13T04:45:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.709+0000 7f6092506b80 10 client.26371 fetch_fsmap finished waiting for FSMap version 33 2023-09-13T04:45:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.709+0000 7f6092506b80 10 client.26371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:45:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.709+0000 7f6092506b80 20 client.26371 populate_metadata read hostname 'smithi002' 2023-09-13T04:45:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.709+0000 7f6092506b80 1 --2- 172.21.15.2:0/3801070097 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a250e1bd50 0x55a250e1e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.709+0000 7f6092506b80 4 client.26371 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:45:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.709+0000 7f6092506b80 1 -- 172.21.15.2:0/3801070097 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a250d4d910 con 0x55a250e1bd50 2023-09-13T04:45:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.710+0000 7f607c99b700 1 --2- 172.21.15.2:0/3801070097 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a250e1bd50 0x55a250e1e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:34.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.724+0000 7f607c99b700 1 --2- 172.21.15.2:0/3801070097 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a250e1bd50 0x55a250e1e170 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67304b170a642bbd server_cookie=91ec6351a529cb48 in_seq=0 out_seq=0 2023-09-13T04:45:34.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.725+0000 7f605bfff700 10 client.26371 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:45:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.736+0000 7f605bfff700 1 -- 172.21.15.2:0/3801070097 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a250d4d910 con 0x55a250e1bd50 2023-09-13T04:45:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.736+0000 7f605bfff700 10 client.26371 handle_command_reply: tid=0 2023-09-13T04:45:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.736+0000 7f6092506b80 1 client.26371 shutdown 2023-09-13T04:45:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.739+0000 7f6092506b80 1 -- 172.21.15.2:0/3801070097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e07500 msgr2=0x55a250e0bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.739+0000 7f6092506b80 1 --2- 172.21.15.2:0/3801070097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e07500 0x55a250e0bff0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f606c00aee0 tx=0x7f606c00c430 comp rx=0 tx=0).stop 2023-09-13T04:45:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.740+0000 7f6092506b80 1 -- 172.21.15.2:0/3801070097 shutdown_connections 2023-09-13T04:45:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.740+0000 7f6092506b80 1 --2- 172.21.15.2:0/3801070097 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a250e1bd50 0x55a250e1e170 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.740+0000 7f6092506b80 1 --2- 172.21.15.2:0/3801070097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e07500 0x55a250e0bff0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.740+0000 7f6092506b80 1 --2- 172.21.15.2:0/3801070097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a250e02840 0x55a250e0e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.740+0000 7f6092506b80 1 --2- 172.21.15.2:0/3801070097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a250e06be0 0x55a250e0b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:34.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.740+0000 7f6092506b80 1 -- 172.21.15.2:0/3801070097 >> 172.21.15.2:0/3801070097 conn(0x55a250d60c20 msgr2=0x55a250e08330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:34.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.740+0000 7f605bfff700 0 client.26371 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:45:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.741+0000 7f6092506b80 1 -- 172.21.15.2:0/3801070097 shutdown_connections 2023-09-13T04:45:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.741+0000 7f6092506b80 1 -- 172.21.15.2:0/3801070097 wait complete. 2023-09-13T04:45:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.741+0000 7f6092506b80 20 client.26371 trim_cache size 0 max 16384 2023-09-13T04:45:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:34.989+0000 7f6092506b80 1 Processor -- start 2023-09-13T04:45:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.165+0000 7f6092506b80 1 -- start start 2023-09-13T04:45:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.165+0000 7f6092506b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a250e02840 0x55a250e12d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.165+0000 7f6092506b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e06be0 0x55a250e0e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.165+0000 7f6092506b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a250e07500 0x55a250e0e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.165+0000 7f6092506b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a250dfd9b0 con 0x55a250e07500 2023-09-13T04:45:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.165+0000 7f6092506b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a250e132a0 con 0x55a250e02840 2023-09-13T04:45:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.165+0000 7f6092506b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a250df52a0 con 0x55a250e06be0 2023-09-13T04:45:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.176+0000 7f607c99b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a250e02840 0x55a250e12d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.176+0000 7f607c99b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a250e02840 0x55a250e12d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:55816/0 (socket says 172.21.15.2:55816) 2023-09-13T04:45:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.176+0000 7f607c99b700 1 -- 172.21.15.2:0/611493028 learned_addr learned my addr 172.21.15.2:0/611493028 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.177+0000 7f6077fff700 1 --2- 172.21.15.2:0/611493028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e06be0 0x55a250e0e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.178+0000 7f6077fff700 1 -- 172.21.15.2:0/611493028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a250e02840 msgr2=0x55a250e12d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.178+0000 7f6077fff700 1 --2- 172.21.15.2:0/611493028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a250e02840 0x55a250e12d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.178+0000 7f6077fff700 1 -- 172.21.15.2:0/611493028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a250e07500 msgr2=0x55a250e0e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.178+0000 7f6077fff700 1 --2- 172.21.15.2:0/611493028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a250e07500 0x55a250e0e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.178+0000 7f6077fff700 1 -- 172.21.15.2:0/611493028 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a250e02ef0 con 0x55a250e06be0 2023-09-13T04:45:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.178+0000 7f6077fff700 1 --2- 172.21.15.2:0/611493028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e06be0 0x55a250e0e0a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55a250e0ebc0 tx=0x7f6070005ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba406c3c4e050b3a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.233+0000 7f60677fe700 1 -- 172.21.15.2:0/611493028 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60700a38f0 con 0x55a250e06be0 2023-09-13T04:45:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.233+0000 7f60677fe700 1 -- 172.21.15.2:0/611493028 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6070099a50 con 0x55a250e06be0 2023-09-13T04:45:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.233+0000 7f60677fe700 1 -- 172.21.15.2:0/611493028 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6070099dd0 con 0x55a250e06be0 2023-09-13T04:45:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.262+0000 7f6092506b80 1 -- 172.21.15.2:0/611493028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e06be0 msgr2=0x55a250e0e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.262+0000 7f6092506b80 1 --2- 172.21.15.2:0/611493028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e06be0 0x55a250e0e0a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55a250e0ebc0 tx=0x7f6070005ba0 comp rx=0 tx=0).stop 2023-09-13T04:45:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.271+0000 7f6092506b80 1 -- 172.21.15.2:0/611493028 shutdown_connections 2023-09-13T04:45:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.271+0000 7f6092506b80 1 --2- 172.21.15.2:0/611493028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a250e07500 0x55a250e0e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.271+0000 7f6092506b80 1 --2- 172.21.15.2:0/611493028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a250e02840 0x55a250e12d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.271+0000 7f6092506b80 1 --2- 172.21.15.2:0/611493028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e06be0 0x55a250e0e0a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.271+0000 7f6092506b80 1 -- 172.21.15.2:0/611493028 >> 172.21.15.2:0/611493028 conn(0x55a250d60c20 msgr2=0x55a250e17eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.271+0000 7f6092506b80 1 -- 172.21.15.2:0/611493028 shutdown_connections 2023-09-13T04:45:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.271+0000 7f6092506b80 1 -- 172.21.15.2:0/611493028 wait complete. 2023-09-13T04:45:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.274+0000 7f6092506b80 1 -- start start 2023-09-13T04:45:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.274+0000 7f6092506b80 1 Processor -- start 2023-09-13T04:45:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.275+0000 7f6092506b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a250e02840 0x55a250e0b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.275+0000 7f6092506b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e06be0 0x55a250e10d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.275+0000 7f6092506b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a250e07500 0x55a250e11280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.275+0000 7f6092506b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a250e11970 con 0x55a250e02840 2023-09-13T04:45:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.275+0000 7f6092506b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a250e11ae0 con 0x55a250e07500 2023-09-13T04:45:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.275+0000 7f6092506b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a250e6eb40 con 0x55a250e06be0 2023-09-13T04:45:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.275+0000 7f6092506b80 10 client.0 _ll_register_callbacks cb 0x55a250cf0620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:45:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.277+0000 7f6077fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e06be0 0x55a250e10d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.277+0000 7f6077fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e06be0 0x55a250e10d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:48252/0 (socket says 172.21.15.2:48252) 2023-09-13T04:45:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.277+0000 7f6077fff700 1 -- 172.21.15.2:0/1644158405 learned_addr learned my addr 172.21.15.2:0/1644158405 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.277+0000 7f607d19c700 1 --2- 172.21.15.2:0/1644158405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a250e07500 0x55a250e11280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.277+0000 7f6077fff700 1 -- 172.21.15.2:0/1644158405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a250e07500 msgr2=0x55a250e11280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.277+0000 7f6077fff700 1 --2- 172.21.15.2:0/1644158405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a250e07500 0x55a250e11280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.277+0000 7f6077fff700 1 -- 172.21.15.2:0/1644158405 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a250e02840 msgr2=0x55a250e0b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.277+0000 7f6077fff700 1 --2- 172.21.15.2:0/1644158405 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a250e02840 0x55a250e0b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.277+0000 7f6077fff700 1 -- 172.21.15.2:0/1644158405 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a250bd45d0 con 0x55a250e06be0 2023-09-13T04:45:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.292+0000 7f6077fff700 1 --2- 172.21.15.2:0/1644158405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e06be0 0x55a250e10d40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f60700a33c0 tx=0x7f6070099ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ece906b1b9f3004 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.368+0000 7f605bfff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:45:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.368+0000 7f605bfff700 1 -- 172.21.15.2:0/1644158405 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607009b270 con 0x55a250e06be0 2023-09-13T04:45:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.368+0000 7f605bfff700 1 -- 172.21.15.2:0/1644158405 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f60700a38f0 con 0x55a250e06be0 2023-09-13T04:45:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.368+0000 7f6092506b80 1 -- 172.21.15.2:0/1644158405 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6070005870 con 0x55a250e06be0 2023-09-13T04:45:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.401+0000 7f605bfff700 1 -- 172.21.15.2:0/1644158405 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6070012e50 con 0x55a250e06be0 2023-09-13T04:45:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.401+0000 7f6092506b80 10 client.26377 fetch_fsmap learned FSMap version 33 2023-09-13T04:45:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.401+0000 7f6092506b80 1 -- 172.21.15.2:0/1644158405 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55a250b80d80 con 0x55a250e06be0 2023-09-13T04:45:35.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.410+0000 7f605bfff700 1 -- 172.21.15.2:0/1644158405 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6070005870 con 0x55a250e06be0 2023-09-13T04:45:35.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.410+0000 7f605bfff700 1 -- 172.21.15.2:0/1644158405 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f607009bb70 con 0x55a250e06be0 2023-09-13T04:45:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.411+0000 7f6092506b80 10 client.26377 fetch_fsmap finished waiting for FSMap version 33 2023-09-13T04:45:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.411+0000 7f6092506b80 10 client.26377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:45:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.411+0000 7f6092506b80 20 client.26377 populate_metadata read hostname 'smithi002' 2023-09-13T04:45:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.411+0000 7f6092506b80 1 --2- 172.21.15.2:0/1644158405 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a250e6fcd0 0x55a250e720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.411+0000 7f6092506b80 4 client.26377 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:45:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.411+0000 7f6092506b80 1 -- 172.21.15.2:0/1644158405 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a250d0dff0 con 0x55a250e6fcd0 2023-09-13T04:45:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.412+0000 7f607c99b700 1 --2- 172.21.15.2:0/1644158405 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a250e6fcd0 0x55a250e720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.422+0000 7f607c99b700 1 --2- 172.21.15.2:0/1644158405 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a250e6fcd0 0x55a250e720c0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37979552a33b5f51 server_cookie=9422fe8bba619bf7 in_seq=0 out_seq=0 2023-09-13T04:45:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:35.422+0000 7f605bfff700 10 client.26377 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:45:35.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:35 smithi149 ceph-mon[133315]: pgmap v157: 129 pgs: 129 active+clean; 577 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 176 op/s 2023-09-13T04:45:35.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:35 smithi149 ceph-mon[133315]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:45:35.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:35 smithi156 ceph-mon[133675]: pgmap v157: 129 pgs: 129 active+clean; 577 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 176 op/s 2023-09-13T04:45:35.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:35 smithi156 ceph-mon[133675]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:45:36.011 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:35 smithi002 ceph-mon[97651]: pgmap v157: 129 pgs: 129 active+clean; 577 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 176 op/s 2023-09-13T04:45:36.011 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:35 smithi002 ceph-mon[97651]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:45:37.063 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:36 smithi002 ceph-mon[97651]: pgmap v158: 129 pgs: 129 active+clean; 582 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 155 op/s 2023-09-13T04:45:37.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:36 smithi149 ceph-mon[133315]: pgmap v158: 129 pgs: 129 active+clean; 582 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 155 op/s 2023-09-13T04:45:37.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:36 smithi156 ceph-mon[133675]: pgmap v158: 129 pgs: 129 active+clean; 582 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 155 op/s 2023-09-13T04:45:38.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.411+0000 7fe223fff700 1 -- 172.21.15.2:0/3671067230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe21c1081f0 msgr2=0x7fe21c108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:38.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.411+0000 7fe223fff700 1 --2- 172.21.15.2:0/3671067230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe21c1081f0 0x7fe21c108670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe218009990 tx=0x7fe218009ca0 comp rx=0 tx=0).stop 2023-09-13T04:45:38.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.411+0000 7fe223fff700 1 -- 172.21.15.2:0/3671067230 shutdown_connections 2023-09-13T04:45:38.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.411+0000 7fe223fff700 1 --2- 172.21.15.2:0/3671067230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe21c11a490 0x7fe21c11c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:38.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.411+0000 7fe223fff700 1 --2- 172.21.15.2:0/3671067230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe21c10f440 0x7fe21c107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:38.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.411+0000 7fe223fff700 1 --2- 172.21.15.2:0/3671067230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe21c1081f0 0x7fe21c108670 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:38.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.411+0000 7fe223fff700 1 -- 172.21.15.2:0/3671067230 >> 172.21.15.2:0/3671067230 conn(0x7fe21c06cfa0 msgr2=0x7fe21c06d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:38.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.412+0000 7fe223fff700 1 -- 172.21.15.2:0/3671067230 shutdown_connections 2023-09-13T04:45:38.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.412+0000 7fe223fff700 1 -- 172.21.15.2:0/3671067230 wait complete. 2023-09-13T04:45:38.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.413+0000 7fe223fff700 1 Processor -- start 2023-09-13T04:45:38.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.414+0000 7fe223fff700 1 -- start start 2023-09-13T04:45:38.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.414+0000 7fe223fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe21c1081f0 0x7fe21c1aba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:38.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.414+0000 7fe223fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe21c10f440 0x7fe21c1abfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:38.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.414+0000 7fe223fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe21c11a490 0x7fe21c1a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.414+0000 7fe223fff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c1a6240 con 0x7fe21c1081f0 2023-09-13T04:45:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.415+0000 7fe223fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c1a6380 con 0x7fe21c10f440 2023-09-13T04:45:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.415+0000 7fe223fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c1a6540 con 0x7fe21c11a490 2023-09-13T04:45:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.415+0000 7fe22259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe21c11a490 0x7fe21c1a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.415+0000 7fe22259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe21c11a490 0x7fe21c1a5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:48258/0 (socket says 172.21.15.2:48258) 2023-09-13T04:45:38.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.415+0000 7fe22259c700 1 -- 172.21.15.2:0/3762549771 learned_addr learned my addr 172.21.15.2:0/3762549771 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:38.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.415+0000 7fe22159a700 1 --2- 172.21.15.2:0/3762549771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe21c10f440 0x7fe21c1abfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:38.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.416+0000 7fe22259c700 1 -- 172.21.15.2:0/3762549771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe21c10f440 msgr2=0x7fe21c1abfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:38.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.416+0000 7fe22259c700 1 --2- 172.21.15.2:0/3762549771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe21c10f440 0x7fe21c1abfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:38.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.416+0000 7fe22259c700 1 -- 172.21.15.2:0/3762549771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe21c1081f0 msgr2=0x7fe21c1aba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:38.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.416+0000 7fe22259c700 1 --2- 172.21.15.2:0/3762549771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe21c1081f0 0x7fe21c1aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:38.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.416+0000 7fe22259c700 1 -- 172.21.15.2:0/3762549771 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2180095f0 con 0x7fe21c11a490 2023-09-13T04:45:38.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.432+0000 7fe22259c700 1 --2- 172.21.15.2:0/3762549771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe21c11a490 0x7fe21c1a5a70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00d6a0 tx=0x7fe20c00d9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:38.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.457+0000 7fe212ffd700 1 -- 172.21.15.2:0/3762549771 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c004a30 con 0x7fe21c11a490 2023-09-13T04:45:38.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.457+0000 7fe212ffd700 1 -- 172.21.15.2:0/3762549771 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe20c0054e0 con 0x7fe21c11a490 2023-09-13T04:45:38.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.457+0000 7fe212ffd700 1 -- 172.21.15.2:0/3762549771 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c00fbf0 con 0x7fe21c11a490 2023-09-13T04:45:38.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.457+0000 7fe223fff700 1 -- 172.21.15.2:0/3762549771 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21c1a6820 con 0x7fe21c11a490 2023-09-13T04:45:38.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.458+0000 7fe223fff700 1 -- 172.21.15.2:0/3762549771 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21c10fe50 con 0x7fe21c11a490 2023-09-13T04:45:38.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.469+0000 7fe237855b80 1 Processor -- start 2023-09-13T04:45:38.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.469+0000 7fe237855b80 1 -- start start 2023-09-13T04:45:38.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.471+0000 7fe237855b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f19702dfe0 0x55f19702e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:38.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.471+0000 7fe237855b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f19702e900 0x55f1970284c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:38.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.471+0000 7fe237855b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f197028a00 0x55f197028e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:38.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.471+0000 7fe237855b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f196f81780 con 0x55f197028a00 2023-09-13T04:45:38.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.471+0000 7fe237855b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f196f7f0a0 con 0x55f19702e900 2023-09-13T04:45:38.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.471+0000 7fe237855b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f196f7a990 con 0x55f19702dfe0 2023-09-13T04:45:38.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.471+0000 7fe22159a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f19702dfe0 0x55f19702e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:38.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.471+0000 7fe22159a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f19702dfe0 0x55f19702e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:48268/0 (socket says 172.21.15.2:48268) 2023-09-13T04:45:38.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.471+0000 7fe22159a700 1 -- 172.21.15.2:0/4216955207 learned_addr learned my addr 172.21.15.2:0/4216955207 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:38.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.472+0000 7fe221d9b700 1 --2- 172.21.15.2:0/4216955207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f19702e900 0x55f1970284c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:38.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.472+0000 7fe22159a700 1 -- 172.21.15.2:0/4216955207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f19702e900 msgr2=0x55f1970284c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:38.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.472+0000 7fe22159a700 1 --2- 172.21.15.2:0/4216955207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f19702e900 0x55f1970284c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:38.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.472+0000 7fe22159a700 1 -- 172.21.15.2:0/4216955207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f197028a00 msgr2=0x55f197028e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:38.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.472+0000 7fe22159a700 1 --2- 172.21.15.2:0/4216955207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f197028a00 0x55f197028e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:38.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.472+0000 7fe22159a700 1 -- 172.21.15.2:0/4216955207 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f196dfa630 con 0x55f19702dfe0 2023-09-13T04:45:38.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.490+0000 7fe22159a700 1 --2- 172.21.15.2:0/4216955207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f19702dfe0 0x55f19702e3c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe218009990 tx=0x7fe218005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=580c95afe440c9a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:38.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.491+0000 7fe212ffd700 1 -- 172.21.15.2:0/3762549771 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7fe20c00fd50 con 0x7fe21c11a490 2023-09-13T04:45:38.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.494+0000 7fe212ffd700 1 --2- 172.21.15.2:0/3762549771 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe2080a09c0 0x7fe2080a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:38.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.494+0000 7fe212ffd700 1 -- 172.21.15.2:0/3762549771 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fe20c0cba40 con 0x7fe21c11a490 2023-09-13T04:45:38.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.550+0000 7fe221d9b700 1 --2- 172.21.15.2:0/3762549771 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe2080a09c0 0x7fe2080a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:38.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.567+0000 7fe207fff700 1 -- 172.21.15.2:0/4216955207 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21800b850 con 0x55f19702dfe0 2023-09-13T04:45:38.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.567+0000 7fe207fff700 1 -- 172.21.15.2:0/4216955207 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe21800be70 con 0x55f19702dfe0 2023-09-13T04:45:38.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.567+0000 7fe207fff700 1 -- 172.21.15.2:0/4216955207 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21800fe60 con 0x55f19702dfe0 2023-09-13T04:45:38.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.575+0000 7fe237855b80 1 -- 172.21.15.2:0/4216955207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f19702dfe0 msgr2=0x55f19702e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:38.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.575+0000 7fe237855b80 1 --2- 172.21.15.2:0/4216955207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f19702dfe0 0x55f19702e3c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe218009990 tx=0x7fe218005bc0 comp rx=0 tx=0).stop 2023-09-13T04:45:38.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.705+0000 7fe237855b80 1 -- 172.21.15.2:0/4216955207 shutdown_connections 2023-09-13T04:45:38.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.705+0000 7fe237855b80 1 --2- 172.21.15.2:0/4216955207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f197028a00 0x55f197028e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:38.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.705+0000 7fe237855b80 1 --2- 172.21.15.2:0/4216955207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f19702dfe0 0x55f19702e3c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:38.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.705+0000 7fe237855b80 1 --2- 172.21.15.2:0/4216955207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f19702e900 0x55f1970284c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:38.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.705+0000 7fe237855b80 1 -- 172.21.15.2:0/4216955207 >> 172.21.15.2:0/4216955207 conn(0x55f196f86c80 msgr2=0x55f197022f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:38.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.725+0000 7fe237855b80 1 -- 172.21.15.2:0/4216955207 shutdown_connections 2023-09-13T04:45:38.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.725+0000 7fe237855b80 1 -- 172.21.15.2:0/4216955207 wait complete. 2023-09-13T04:45:38.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.726+0000 7fe237855b80 1 -- start start 2023-09-13T04:45:38.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.726+0000 7fe237855b80 1 Processor -- start 2023-09-13T04:45:38.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.727+0000 7fe237855b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f197028a00 0x55f197034880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:38.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.727+0000 7fe237855b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f19702dfe0 0x55f197031b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:38.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.727+0000 7fe237855b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f19702e900 0x55f197032160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:38.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.727+0000 7fe237855b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f196f18bb0 con 0x55f19702e900 2023-09-13T04:45:38.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.727+0000 7fe237855b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f196f188b0 con 0x55f19702dfe0 2023-09-13T04:45:38.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.727+0000 7fe237855b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f196dfc920 con 0x55f197028a00 2023-09-13T04:45:38.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.727+0000 7fe237855b80 10 client.0 _ll_register_callbacks cb 0x55f196f16680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:45:38.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.737+0000 7fe22159a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f19702dfe0 0x55f197031b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:38.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.737+0000 7fe22159a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f19702dfe0 0x55f197031b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:55866/0 (socket says 172.21.15.2:55866) 2023-09-13T04:45:38.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.737+0000 7fe22159a700 1 -- 172.21.15.2:0/1144991489 learned_addr learned my addr 172.21.15.2:0/1144991489 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:38.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.737+0000 7fe221d9b700 1 --2- 172.21.15.2:0/1144991489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f197028a00 0x55f197034880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:38.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.746+0000 7fe22159a700 1 -- 172.21.15.2:0/1144991489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f197028a00 msgr2=0x55f197034880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:38.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.746+0000 7fe22159a700 1 --2- 172.21.15.2:0/1144991489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f197028a00 0x55f197034880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:38.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.746+0000 7fe22159a700 1 -- 172.21.15.2:0/1144991489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f19702e900 msgr2=0x55f197032160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:38.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.746+0000 7fe22159a700 1 --2- 172.21.15.2:0/1144991489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f19702e900 0x55f197032160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:38.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.746+0000 7fe22159a700 1 -- 172.21.15.2:0/1144991489 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe218020040 con 0x55f19702dfe0 2023-09-13T04:45:38.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.762+0000 7fe22159a700 1 --2- 172.21.15.2:0/1144991489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f19702dfe0 0x55f197031b00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe218009350 tx=0x7fe21800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc71e061fec074f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:38.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.763+0000 7fe2057fa700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:45:38.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.763+0000 7fe2057fa700 1 -- 172.21.15.2:0/1144991489 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21800fc80 con 0x55f19702dfe0 2023-09-13T04:45:38.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.763+0000 7fe2057fa700 1 -- 172.21.15.2:0/1144991489 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fe21800fde0 con 0x55f19702dfe0 2023-09-13T04:45:38.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.763+0000 7fe237855b80 1 -- 172.21.15.2:0/1144991489 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f19701b670 con 0x55f19702dfe0 2023-09-13T04:45:38.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.773+0000 7fe2057fa700 1 -- 172.21.15.2:0/1144991489 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe21800b850 con 0x55f19702dfe0 2023-09-13T04:45:38.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.819+0000 7fe237855b80 10 client.26366 fetch_fsmap learned FSMap version 33 2023-09-13T04:45:38.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.819+0000 7fe237855b80 1 -- 172.21.15.2:0/1144991489 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f196f927a0 con 0x55f19702dfe0 2023-09-13T04:45:38.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.830+0000 7fe2057fa700 1 -- 172.21.15.2:0/1144991489 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f19701b670 con 0x55f19702dfe0 2023-09-13T04:45:38.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.830+0000 7fe2057fa700 1 -- 172.21.15.2:0/1144991489 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 33) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe21802e3f0 con 0x55f19702dfe0 2023-09-13T04:45:38.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.831+0000 7fe237855b80 10 client.26366 fetch_fsmap finished waiting for FSMap version 33 2023-09-13T04:45:38.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.831+0000 7fe237855b80 10 client.26366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:45:38.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.831+0000 7fe237855b80 20 client.26366 populate_metadata read hostname 'smithi002' 2023-09-13T04:45:38.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.831+0000 7fe237855b80 1 --2- 172.21.15.2:0/1144991489 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f197041cc0 0x55f1970440e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:38.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.831+0000 7fe237855b80 4 client.26366 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:45:38.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.831+0000 7fe237855b80 1 -- 172.21.15.2:0/1144991489 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f196f73970 con 0x55f197041cc0 2023-09-13T04:45:38.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.832+0000 7fe221d9b700 1 --2- 172.21.15.2:0/1144991489 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f197041cc0 0x55f1970440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:38.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.855+0000 7fe221d9b700 1 --2- 172.21.15.2:0/1144991489 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f197041cc0 0x55f1970440e0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8385596ec151b7e4 server_cookie=f4a27b935c380035 in_seq=0 out_seq=0 2023-09-13T04:45:38.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.855+0000 7fe2057fa700 10 client.26366 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:45:38.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:38.865+0000 7fe221d9b700 1 --2- 172.21.15.2:0/3762549771 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe2080a09c0 0x7fe2080a2e80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55f19702ed60 tx=0x7fe214009340 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:39.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:39 smithi149 ceph-mon[133315]: pgmap v159: 129 pgs: 129 active+clean; 582 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 138 op/s 2023-09-13T04:45:39.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:39 smithi156 ceph-mon[133675]: pgmap v159: 129 pgs: 129 active+clean; 582 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 138 op/s 2023-09-13T04:45:39.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:39 smithi002 ceph-mon[97651]: pgmap v159: 129 pgs: 129 active+clean; 582 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 138 op/s 2023-09-13T04:45:40.663 INFO:tasks.workunit.client.0.smithi002.stdout: 126 618967 121 453049 247 242891 488 2023-09-13T04:45:42.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:41 smithi002 ceph-mon[97651]: pgmap v160: 129 pgs: 129 active+clean; 613 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 644 op/s 2023-09-13T04:45:42.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:41 smithi149 ceph-mon[133315]: pgmap v160: 129 pgs: 129 active+clean; 613 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 644 op/s 2023-09-13T04:45:42.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:41 smithi156 ceph-mon[133675]: pgmap v160: 129 pgs: 129 active+clean; 613 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 644 op/s 2023-09-13T04:45:42.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:42.926+0000 7fe2057fa700 1 -- 172.21.15.2:0/1144991489 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f196f73970 con 0x55f197041cc0 2023-09-13T04:45:42.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:42.926+0000 7fe2057fa700 10 client.26366 handle_command_reply: tid=0 2023-09-13T04:45:42.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:42.927+0000 7fe237855b80 1 client.26366 shutdown 2023-09-13T04:45:42.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:42.927+0000 7fe237855b80 1 -- 172.21.15.2:0/1144991489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f19702dfe0 msgr2=0x55f197031b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:42.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:42.927+0000 7fe237855b80 1 --2- 172.21.15.2:0/1144991489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f19702dfe0 0x55f197031b00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe218009350 tx=0x7fe21800f690 comp rx=0 tx=0).stop 2023-09-13T04:45:42.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:42.929+0000 7fe237855b80 1 -- 172.21.15.2:0/1144991489 shutdown_connections 2023-09-13T04:45:42.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:42.929+0000 7fe237855b80 1 --2- 172.21.15.2:0/1144991489 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f197041cc0 0x55f1970440e0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:42.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:42.929+0000 7fe237855b80 1 --2- 172.21.15.2:0/1144991489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f19702e900 0x55f197032160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:42.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:42.929+0000 7fe237855b80 1 --2- 172.21.15.2:0/1144991489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f197028a00 0x55f197034880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:42.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:42.929+0000 7fe237855b80 1 --2- 172.21.15.2:0/1144991489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f19702dfe0 0x55f197031b00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:42.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:42.929+0000 7fe237855b80 1 -- 172.21.15.2:0/1144991489 >> 172.21.15.2:0/1144991489 conn(0x55f196f86c80 msgr2=0x55f19701d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:42.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:42.929+0000 7fe2057fa700 0 client.26366 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:45:42.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:42.930+0000 7fe237855b80 1 -- 172.21.15.2:0/1144991489 shutdown_connections 2023-09-13T04:45:42.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:42.930+0000 7fe237855b80 1 -- 172.21.15.2:0/1144991489 wait complete. 2023-09-13T04:45:42.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:42.930+0000 7fe237855b80 20 client.26366 trim_cache size 0 max 16384 2023-09-13T04:45:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:45:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:45:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:45:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:45:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:42.983+0000 7f605bfff700 1 -- 172.21.15.2:0/1644158405 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a250d0dff0 con 0x55a250e6fcd0 2023-09-13T04:45:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:42.983+0000 7f605bfff700 10 client.26377 handle_command_reply: tid=0 2023-09-13T04:45:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:42.983+0000 7f6092506b80 1 client.26377 shutdown 2023-09-13T04:45:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:42.993+0000 7f6092506b80 1 -- 172.21.15.2:0/1644158405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e06be0 msgr2=0x55a250e10d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:42.993+0000 7f6092506b80 1 --2- 172.21.15.2:0/1644158405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e06be0 0x55a250e10d40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f60700a33c0 tx=0x7f6070099ec0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:42.993+0000 7f6092506b80 1 -- 172.21.15.2:0/1644158405 shutdown_connections 2023-09-13T04:45:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:42.993+0000 7f6092506b80 1 --2- 172.21.15.2:0/1644158405 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a250e6fcd0 0x55a250e720c0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:42.993+0000 7f6092506b80 1 --2- 172.21.15.2:0/1644158405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a250e07500 0x55a250e11280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:42.993+0000 7f6092506b80 1 --2- 172.21.15.2:0/1644158405 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a250e02840 0x55a250e0b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:42.993+0000 7f6092506b80 1 --2- 172.21.15.2:0/1644158405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a250e06be0 0x55a250e10d40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:42.993+0000 7f6092506b80 1 -- 172.21.15.2:0/1644158405 >> 172.21.15.2:0/1644158405 conn(0x55a250d60c20 msgr2=0x55a250e009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:42.993+0000 7f605bfff700 0 client.26377 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:45:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:42.994+0000 7f6092506b80 1 -- 172.21.15.2:0/1644158405 shutdown_connections 2023-09-13T04:45:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:42.994+0000 7f6092506b80 1 -- 172.21.15.2:0/1644158405 wait complete. 2023-09-13T04:45:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:42.994+0000 7f6092506b80 20 client.26377 trim_cache size 0 max 16384 2023-09-13T04:45:43.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:42 smithi149 ceph-mon[133315]: pgmap v161: 129 pgs: 129 active+clean; 617 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 4.3 MiB/s wr, 742 op/s 2023-09-13T04:45:43.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:43.142+0000 7f607ebff700 1 -- 172.21.15.2:0/883555920 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f60600a09c0 msgr2=0x7f60600a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:43.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:43.142+0000 7f607ebff700 1 --2- 172.21.15.2:0/883555920 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f60600a09c0 0x7f60600a2e80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6068009770 tx=0x7f6068006cd0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:43.142+0000 7f607ebff700 1 -- 172.21.15.2:0/883555920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f607810f890 msgr2=0x7f6078119ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:43.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:43.142+0000 7f607ebff700 1 --2- 172.21.15.2:0/883555920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f607810f890 0x7f6078119ea0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f607000b070 tx=0x7f6070005b30 comp rx=0 tx=0).stop 2023-09-13T04:45:43.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:43.143+0000 7f607ebff700 1 -- 172.21.15.2:0/883555920 shutdown_connections 2023-09-13T04:45:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:43.143+0000 7f607ebff700 1 --2- 172.21.15.2:0/883555920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f607811a490 0x7f60781138c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:43.143+0000 7f607ebff700 1 --2- 172.21.15.2:0/883555920 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f60600a09c0 0x7f60600a2e80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:43.143+0000 7f607ebff700 1 --2- 172.21.15.2:0/883555920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f607810f420 0x7f6078119960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:43.143+0000 7f607ebff700 1 --2- 172.21.15.2:0/883555920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f607810f890 0x7f6078119ea0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:43.143+0000 7f607ebff700 1 -- 172.21.15.2:0/883555920 >> 172.21.15.2:0/883555920 conn(0x7f607806cda0 msgr2=0x7f607811bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:43.143+0000 7f607ebff700 1 -- 172.21.15.2:0/883555920 shutdown_connections 2023-09-13T04:45:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:43.144+0000 7f607ebff700 1 -- 172.21.15.2:0/883555920 wait complete. 2023-09-13T04:45:43.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.177+0000 7fe237855b80 1 Processor -- start 2023-09-13T04:45:43.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:42 smithi156 ceph-mon[133675]: pgmap v161: 129 pgs: 129 active+clean; 617 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 4.3 MiB/s wr, 742 op/s 2023-09-13T04:45:43.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.187+0000 7fe237855b80 1 -- start start 2023-09-13T04:45:43.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.190+0000 7fe237855b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f197028a00 0x55f1970284c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:43.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.190+0000 7fe237855b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f19702dfe0 0x55f197039240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:43.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.190+0000 7fe237855b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f19702e900 0x55f19701d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:43.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.190+0000 7fe237855b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1970234f0 con 0x55f19702e900 2023-09-13T04:45:43.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.190+0000 7fe237855b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f197039780 con 0x55f19702dfe0 2023-09-13T04:45:43.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.190+0000 7fe237855b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f197030080 con 0x55f197028a00 2023-09-13T04:45:43.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.195+0000 7fe22259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f19702e900 0x55f19701d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:43.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.195+0000 7fe22259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f19702e900 0x55f19701d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57886/0 (socket says 172.21.15.2:57886) 2023-09-13T04:45:43.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.195+0000 7fe22259c700 1 -- 172.21.15.2:0/1336456418 learned_addr learned my addr 172.21.15.2:0/1336456418 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:43.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.200+0000 7fe22159a700 1 --2- 172.21.15.2:0/1336456418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f19702dfe0 0x55f197039240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:43.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.200+0000 7fe221d9b700 1 --2- 172.21.15.2:0/1336456418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f197028a00 0x55f1970284c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:43.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.201+0000 7fe22159a700 1 -- 172.21.15.2:0/1336456418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f197028a00 msgr2=0x55f1970284c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:43.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.201+0000 7fe22159a700 1 --2- 172.21.15.2:0/1336456418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f197028a00 0x55f1970284c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.201+0000 7fe22159a700 1 -- 172.21.15.2:0/1336456418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f19702e900 msgr2=0x55f19701d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:43.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.201+0000 7fe22159a700 1 --2- 172.21.15.2:0/1336456418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f19702e900 0x55f19701d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.201+0000 7fe22159a700 1 -- 172.21.15.2:0/1336456418 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f196da6de0 con 0x55f19702dfe0 2023-09-13T04:45:43.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.202+0000 7fe22159a700 1 --2- 172.21.15.2:0/1336456418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f19702dfe0 0x55f197039240 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe218005950 tx=0x7fe21802eaa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25aa3ce80060fcff server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:43.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.203+0000 7fe210ff9700 1 -- 172.21.15.2:0/1336456418 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21802eb90 con 0x55f19702dfe0 2023-09-13T04:45:43.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.203+0000 7fe210ff9700 1 -- 172.21.15.2:0/1336456418 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe21802e3f0 con 0x55f19702dfe0 2023-09-13T04:45:43.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.203+0000 7fe210ff9700 1 -- 172.21.15.2:0/1336456418 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2180259a0 con 0x55f19702dfe0 2023-09-13T04:45:43.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.214+0000 7fe237855b80 1 -- 172.21.15.2:0/1336456418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f19702dfe0 msgr2=0x55f197039240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:43.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.214+0000 7fe237855b80 1 --2- 172.21.15.2:0/1336456418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f19702dfe0 0x55f197039240 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe218005950 tx=0x7fe21802eaa0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.230+0000 7fe237855b80 1 -- 172.21.15.2:0/1336456418 shutdown_connections 2023-09-13T04:45:43.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.230+0000 7fe237855b80 1 --2- 172.21.15.2:0/1336456418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f19702e900 0x55f19701d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.230+0000 7fe237855b80 1 --2- 172.21.15.2:0/1336456418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f197028a00 0x55f1970284c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.230+0000 7fe237855b80 1 --2- 172.21.15.2:0/1336456418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f19702dfe0 0x55f197039240 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.230+0000 7fe237855b80 1 -- 172.21.15.2:0/1336456418 >> 172.21.15.2:0/1336456418 conn(0x55f196f86c80 msgr2=0x55f19703e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:43.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.232+0000 7fe237855b80 1 -- 172.21.15.2:0/1336456418 shutdown_connections 2023-09-13T04:45:43.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.232+0000 7fe237855b80 1 -- 172.21.15.2:0/1336456418 wait complete. 2023-09-13T04:45:43.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.233+0000 7fe237855b80 1 -- start start 2023-09-13T04:45:43.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.233+0000 7fe237855b80 1 Processor -- start 2023-09-13T04:45:43.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.235+0000 7fe237855b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f197028a00 0x55f19702b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:43.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.235+0000 7fe237855b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f19702dfe0 0x55f197029a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:43.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.235+0000 7fe237855b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f19702e900 0x55f197029fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:43.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.235+0000 7fe237855b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f19702b7c0 con 0x55f197028a00 2023-09-13T04:45:43.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.235+0000 7fe237855b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f19702a4e0 con 0x55f19702e900 2023-09-13T04:45:43.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.235+0000 7fe237855b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f19702a6a0 con 0x55f19702dfe0 2023-09-13T04:45:43.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.235+0000 7fe237855b80 10 client.0 _ll_register_callbacks cb 0x55f196f16680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:45:43.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.235+0000 7fe22159a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f19702dfe0 0x55f197029a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.235+0000 7fe22159a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f19702dfe0 0x55f197029a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:48296/0 (socket says 172.21.15.2:48296) 2023-09-13T04:45:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.235+0000 7fe22159a700 1 -- 172.21.15.2:0/3220454139 learned_addr learned my addr 172.21.15.2:0/3220454139 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:43.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.236+0000 7fe22259c700 1 --2- 172.21.15.2:0/3220454139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f19702e900 0x55f197029fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:43.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.236+0000 7fe22159a700 1 -- 172.21.15.2:0/3220454139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f19702e900 msgr2=0x55f197029fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:43.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.236+0000 7fe22159a700 1 --2- 172.21.15.2:0/3220454139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f19702e900 0x55f197029fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.236+0000 7fe22159a700 1 -- 172.21.15.2:0/3220454139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f197028a00 msgr2=0x55f19702b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:43.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.236+0000 7fe22159a700 1 --2- 172.21.15.2:0/3220454139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f197028a00 0x55f19702b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.236+0000 7fe22159a700 1 -- 172.21.15.2:0/3220454139 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f196f927a0 con 0x55f19702dfe0 2023-09-13T04:45:43.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.244+0000 7fe22159a700 1 --2- 172.21.15.2:0/3220454139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f19702dfe0 0x55f197029a60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe21800f930 tx=0x7fe218005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d69341afe1317977 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:43.264 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:42 smithi002 ceph-mon[97651]: pgmap v161: 129 pgs: 129 active+clean; 617 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 4.3 MiB/s wr, 742 op/s 2023-09-13T04:45:43.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.264+0000 7fe2057fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:45:43.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.264+0000 7fe2057fa700 1 -- 172.21.15.2:0/3220454139 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe218025440 con 0x55f19702dfe0 2023-09-13T04:45:43.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.264+0000 7fe2057fa700 1 -- 172.21.15.2:0/3220454139 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fe218000e40 con 0x55f19702dfe0 2023-09-13T04:45:43.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.273+0000 7fe237855b80 1 -- 172.21.15.2:0/3220454139 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe21800be70 con 0x55f19702dfe0 2023-09-13T04:45:43.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.287+0000 7fe2057fa700 1 -- 172.21.15.2:0/3220454139 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2180255a0 con 0x55f19702dfe0 2023-09-13T04:45:43.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.287+0000 7fe237855b80 10 client.26395 fetch_fsmap learned FSMap version 33 2023-09-13T04:45:43.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.288+0000 7fe237855b80 1 -- 172.21.15.2:0/3220454139 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f196d8fc60 con 0x55f19702dfe0 2023-09-13T04:45:43.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.295+0000 7fe2057fa700 1 -- 172.21.15.2:0/3220454139 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe21800be70 con 0x55f19702dfe0 2023-09-13T04:45:43.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.295+0000 7fe2057fa700 1 -- 172.21.15.2:0/3220454139 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe21804f050 con 0x55f19702dfe0 2023-09-13T04:45:43.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.295+0000 7fe237855b80 10 client.26395 fetch_fsmap finished waiting for FSMap version 33 2023-09-13T04:45:43.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.295+0000 7fe237855b80 10 client.26395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:45:43.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.295+0000 7fe237855b80 20 client.26395 populate_metadata read hostname 'smithi002' 2023-09-13T04:45:43.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.295+0000 7fe237855b80 1 --2- 172.21.15.2:0/3220454139 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f1970968c0 0x55f197098cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:43.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.295+0000 7fe237855b80 4 client.26395 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-09-13T04:45:43.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.295+0000 7fe237855b80 1 -- 172.21.15.2:0/3220454139 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f196f34050 con 0x55f1970968c0 2023-09-13T04:45:43.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.297+0000 7fe221d9b700 1 --2- 172.21.15.2:0/3220454139 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f1970968c0 0x55f197098cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:43.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.302+0000 7fe221d9b700 1 --2- 172.21.15.2:0/3220454139 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f1970968c0 0x55f197098cb0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96b6b0070ce4dd29 server_cookie=437ff4aefd33936c in_seq=0 out_seq=0 2023-09-13T04:45:43.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.302+0000 7fe2057fa700 10 client.26395 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:45:43.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.303+0000 7fe2057fa700 1 -- 172.21.15.2:0/3220454139 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55f196f34050 con 0x55f1970968c0 2023-09-13T04:45:43.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.303+0000 7fe2057fa700 10 client.26395 handle_command_reply: tid=0 2023-09-13T04:45:43.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.303+0000 7fe237855b80 1 client.26395 shutdown 2023-09-13T04:45:43.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.303+0000 7fe237855b80 1 -- 172.21.15.2:0/3220454139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f19702dfe0 msgr2=0x55f197029a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:43.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.303+0000 7fe237855b80 1 --2- 172.21.15.2:0/3220454139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f19702dfe0 0x55f197029a60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe21800f930 tx=0x7fe218005fb0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.303+0000 7fe237855b80 1 -- 172.21.15.2:0/3220454139 shutdown_connections 2023-09-13T04:45:43.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.303+0000 7fe237855b80 1 --2- 172.21.15.2:0/3220454139 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f1970968c0 0x55f197098cb0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.303+0000 7fe237855b80 1 --2- 172.21.15.2:0/3220454139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f19702e900 0x55f197029fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.303+0000 7fe237855b80 1 --2- 172.21.15.2:0/3220454139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f197028a00 0x55f19702b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.304+0000 7fe237855b80 1 --2- 172.21.15.2:0/3220454139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f19702dfe0 0x55f197029a60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.304+0000 7fe237855b80 1 -- 172.21.15.2:0/3220454139 >> 172.21.15.2:0/3220454139 conn(0x55f196f86c80 msgr2=0x55f19702bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:43.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.304+0000 7fe2057fa700 0 client.26395 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:45:43.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.304+0000 7fe237855b80 1 -- 172.21.15.2:0/3220454139 shutdown_connections 2023-09-13T04:45:43.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.304+0000 7fe237855b80 1 -- 172.21.15.2:0/3220454139 wait complete. 2023-09-13T04:45:43.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.305+0000 7fe237855b80 20 client.26395 trim_cache size 0 max 16384 2023-09-13T04:45:43.308 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:45:43.308 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "scrub active (3 inodes in the stack)", 2023-09-13T04:45:43.308 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-09-13T04:45:43.308 INFO:teuthology.orchestra.run.smithi002.stdout: "740235c0-6e1b-4729-b1fe-33345ae948c1": { 2023-09-13T04:45:43.309 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-09-13T04:45:43.309 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "740235c0-6e1b-4729-b1fe-33345ae948c1", 2023-09-13T04:45:43.309 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-09-13T04:45:43.309 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:45:43.309 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:45:43.310 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:45:43.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.319+0000 7fe223fff700 1 -- 172.21.15.2:0/3762549771 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe2080a09c0 msgr2=0x7fe2080a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:43.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.319+0000 7fe223fff700 1 --2- 172.21.15.2:0/3762549771 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe2080a09c0 0x7fe2080a2e80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55f19702ed60 tx=0x7fe214009340 comp rx=0 tx=0).stop 2023-09-13T04:45:43.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.319+0000 7fe223fff700 1 -- 172.21.15.2:0/3762549771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe21c11a490 msgr2=0x7fe21c1a5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:43.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.319+0000 7fe223fff700 1 --2- 172.21.15.2:0/3762549771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe21c11a490 0x7fe21c1a5a70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00d6a0 tx=0x7fe20c00d9b0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.320+0000 7fe223fff700 1 -- 172.21.15.2:0/3762549771 shutdown_connections 2023-09-13T04:45:43.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.320+0000 7fe223fff700 1 --2- 172.21.15.2:0/3762549771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe21c11a490 0x7fe21c1a5a70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.320+0000 7fe223fff700 1 --2- 172.21.15.2:0/3762549771 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe2080a09c0 0x7fe2080a2e80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.320+0000 7fe223fff700 1 --2- 172.21.15.2:0/3762549771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe21c1081f0 0x7fe21c1aba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.320+0000 7fe223fff700 1 --2- 172.21.15.2:0/3762549771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe21c10f440 0x7fe21c1abfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:43.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.320+0000 7fe223fff700 1 -- 172.21.15.2:0/3762549771 >> 172.21.15.2:0/3762549771 conn(0x7fe21c06cfa0 msgr2=0x7fe21c118480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:43.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.320+0000 7fe223fff700 1 -- 172.21.15.2:0/3762549771 shutdown_connections 2023-09-13T04:45:43.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:45:43.320+0000 7fe223fff700 1 -- 172.21.15.2:0/3762549771 wait complete. 2023-09-13T04:45:43.431 INFO:tasks.cephfs.filesystem:scrub status for tag:740235c0-6e1b-4729-b1fe-33345ae948c1 - {'path': '/', 'tag': '740235c0-6e1b-4729-b1fe-33345ae948c1', 'options': 'recursive,force'} 2023-09-13T04:45:45.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:45 smithi149 ceph-mon[133315]: pgmap v162: 129 pgs: 129 active+clean; 609 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.7 MiB/s wr, 867 op/s 2023-09-13T04:45:45.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:45 smithi156 ceph-mon[133675]: pgmap v162: 129 pgs: 129 active+clean; 609 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.7 MiB/s wr, 867 op/s 2023-09-13T04:45:46.025 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:45 smithi002 ceph-mon[97651]: pgmap v162: 129 pgs: 129 active+clean; 609 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.7 MiB/s wr, 867 op/s 2023-09-13T04:45:47.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:47 smithi149 ceph-mon[133315]: pgmap v163: 129 pgs: 129 active+clean; 615 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 935 op/s 2023-09-13T04:45:47.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:47 smithi156 ceph-mon[133675]: pgmap v163: 129 pgs: 129 active+clean; 615 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 935 op/s 2023-09-13T04:45:48.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:47 smithi002 ceph-mon[97651]: pgmap v163: 129 pgs: 129 active+clean; 615 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 935 op/s 2023-09-13T04:45:49.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:49 smithi156 ceph-mon[133675]: pgmap v164: 129 pgs: 129 active+clean; 615 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 900 op/s 2023-09-13T04:45:50.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:49 smithi002 ceph-mon[97651]: pgmap v164: 129 pgs: 129 active+clean; 615 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 900 op/s 2023-09-13T04:45:50.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:49 smithi149 ceph-mon[133315]: pgmap v164: 129 pgs: 129 active+clean; 615 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 900 op/s 2023-09-13T04:45:50.663 INFO:tasks.workunit.client.0.smithi002.stdout: 127 615346 234 451370 45 246700 290 2023-09-13T04:45:51.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:51 smithi156 ceph-mon[133675]: pgmap v165: 129 pgs: 129 active+clean; 617 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 917 op/s 2023-09-13T04:45:52.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:51 smithi002 ceph-mon[97651]: pgmap v165: 129 pgs: 129 active+clean; 617 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 917 op/s 2023-09-13T04:45:52.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:51 smithi149 ceph-mon[133315]: pgmap v165: 129 pgs: 129 active+clean; 617 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 917 op/s 2023-09-13T04:45:53.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:53 smithi156 ceph-mon[133675]: pgmap v166: 129 pgs: 129 active+clean; 617 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.5 MiB/s wr, 416 op/s 2023-09-13T04:45:54.024 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:53 smithi002 ceph-mon[97651]: pgmap v166: 129 pgs: 129 active+clean; 617 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.5 MiB/s wr, 416 op/s 2023-09-13T04:45:54.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:53 smithi149 ceph-mon[133315]: pgmap v166: 129 pgs: 129 active+clean; 617 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.5 MiB/s wr, 416 op/s 2023-09-13T04:45:55.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:54 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:45:55.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:54 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:45:55.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:54 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:45:56.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:55 smithi149 ceph-mon[133315]: pgmap v167: 129 pgs: 129 active+clean; 621 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 282 op/s 2023-09-13T04:45:56.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:55 smithi156 ceph-mon[133675]: pgmap v167: 129 pgs: 129 active+clean; 621 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 282 op/s 2023-09-13T04:45:56.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:55 smithi002 ceph-mon[97651]: pgmap v167: 129 pgs: 129 active+clean; 621 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 282 op/s 2023-09-13T04:45:57.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:56 smithi149 ceph-mon[133315]: pgmap v168: 129 pgs: 129 active+clean; 623 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 170 op/s 2023-09-13T04:45:57.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:56 smithi156 ceph-mon[133675]: pgmap v168: 129 pgs: 129 active+clean; 623 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 170 op/s 2023-09-13T04:45:57.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:56 smithi002 ceph-mon[97651]: pgmap v168: 129 pgs: 129 active+clean; 623 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 170 op/s 2023-09-13T04:45:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.528+0000 7fcd247ff700 1 -- 172.21.15.2:0/2860101398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd1c10c590 msgr2=0x7fcd1c078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.528+0000 7fcd247ff700 1 --2- 172.21.15.2:0/2860101398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd1c10c590 0x7fcd1c078da0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009d40 tx=0x7fcd1400b540 comp rx=0 tx=0).stop 2023-09-13T04:45:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.542+0000 7fcd247ff700 1 -- 172.21.15.2:0/2860101398 shutdown_connections 2023-09-13T04:45:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.542+0000 7fcd247ff700 1 --2- 172.21.15.2:0/2860101398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd1c10c590 0x7fcd1c078da0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.542+0000 7fcd247ff700 1 --2- 172.21.15.2:0/2860101398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd1c107d00 0x7fcd1c1080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.542+0000 7fcd247ff700 1 --2- 172.21.15.2:0/2860101398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd1c108620 0x7fcd1c10bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.542+0000 7fcd247ff700 1 -- 172.21.15.2:0/2860101398 >> 172.21.15.2:0/2860101398 conn(0x7fcd1c06cd90 msgr2=0x7fcd1c06d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.543+0000 7fcd247ff700 1 -- 172.21.15.2:0/2860101398 shutdown_connections 2023-09-13T04:45:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.543+0000 7fcd247ff700 1 -- 172.21.15.2:0/2860101398 wait complete. 2023-09-13T04:45:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.544+0000 7fcd247ff700 1 Processor -- start 2023-09-13T04:45:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.545+0000 7fcd247ff700 1 -- start start 2023-09-13T04:45:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.545+0000 7fcd247ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd1c107d00 0x7fcd1c083080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.545+0000 7fcd247ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd1c108620 0x7fcd1c0835c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.545+0000 7fcd247ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd1c10c590 0x7fcd1c07d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.545+0000 7fcd247ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c07d890 con 0x7fcd1c108620 2023-09-13T04:45:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.546+0000 7fcd247ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c07da00 con 0x7fcd1c10c590 2023-09-13T04:45:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.546+0000 7fcd247ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c07dbc0 con 0x7fcd1c107d00 2023-09-13T04:45:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.546+0000 7fcd21d9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd1c108620 0x7fcd1c0835c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.546+0000 7fcd21d9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd1c108620 0x7fcd1c0835c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48576/0 (socket says 172.21.15.2:48576) 2023-09-13T04:45:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.546+0000 7fcd21d9a700 1 -- 172.21.15.2:0/695427531 learned_addr learned my addr 172.21.15.2:0/695427531 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.553+0000 7fcd22d9c700 1 --2- 172.21.15.2:0/695427531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd1c10c590 0x7fcd1c07d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.555+0000 7fcd21d9a700 1 -- 172.21.15.2:0/695427531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd1c107d00 msgr2=0x7fcd1c083080 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-13T04:45:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.555+0000 7fcd21d9a700 1 --2- 172.21.15.2:0/695427531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd1c107d00 0x7fcd1c083080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.555+0000 7fcd21d9a700 1 -- 172.21.15.2:0/695427531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd1c10c590 msgr2=0x7fcd1c07d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.555+0000 7fcd21d9a700 1 --2- 172.21.15.2:0/695427531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd1c10c590 0x7fcd1c07d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.555+0000 7fcd21d9a700 1 -- 172.21.15.2:0/695427531 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd14009520 con 0x7fcd1c108620 2023-09-13T04:45:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.564+0000 7fcd21d9a700 1 --2- 172.21.15.2:0/695427531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd1c108620 0x7fcd1c0835c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800c0c0 tx=0x7fcd1800c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:59.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:45:59 smithi149 ceph-mon[133315]: pgmap v169: 129 pgs: 129 active+clean; 623 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 723 KiB/s wr, 53 op/s 2023-09-13T04:45:59.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:45:59 smithi156 ceph-mon[133675]: pgmap v169: 129 pgs: 129 active+clean; 623 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 723 KiB/s wr, 53 op/s 2023-09-13T04:45:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.710+0000 7fcd137fe700 1 -- 172.21.15.2:0/695427531 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd18018070 con 0x7fcd1c108620 2023-09-13T04:45:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.710+0000 7fcd137fe700 1 -- 172.21.15.2:0/695427531 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd1800f040 con 0x7fcd1c108620 2023-09-13T04:45:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.710+0000 7fcd137fe700 1 -- 172.21.15.2:0/695427531 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1800e020 con 0x7fcd1c108620 2023-09-13T04:45:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.713+0000 7fcd247ff700 1 -- 172.21.15.2:0/695427531 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1c07de40 con 0x7fcd1c108620 2023-09-13T04:45:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.714+0000 7fcd247ff700 1 -- 172.21.15.2:0/695427531 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1c1303b0 con 0x7fcd1c108620 2023-09-13T04:45:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.721+0000 7fcd38021b80 1 Processor -- start 2023-09-13T04:45:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.725+0000 7fcd38021b80 1 -- start start 2023-09-13T04:45:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.726+0000 7fcd38021b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605ee893be0 0x5605ee893fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.726+0000 7fcd38021b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5605ee894500 0x5605ee88f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.726+0000 7fcd38021b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ee88f840 0x5605ee88fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.726+0000 7fcd38021b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ee7e8720 con 0x5605ee894500 2023-09-13T04:45:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.726+0000 7fcd38021b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ee7e6040 con 0x5605ee893be0 2023-09-13T04:45:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.726+0000 7fcd38021b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ee7e1930 con 0x5605ee88f840 2023-09-13T04:45:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.729+0000 7fcd137fe700 1 -- 172.21.15.2:0/695427531 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7fcd180075c0 con 0x7fcd1c108620 2023-09-13T04:45:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.730+0000 7fcd21d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ee88f840 0x5605ee88fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.730+0000 7fcd21d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ee88f840 0x5605ee88fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:36988/0 (socket says 172.21.15.2:36988) 2023-09-13T04:45:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.730+0000 7fcd21d9a700 1 -- 172.21.15.2:0/2717092425 learned_addr learned my addr 172.21.15.2:0/2717092425 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.731+0000 7fcd137fe700 1 --2- 172.21.15.2:0/695427531 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcd040a09c0 0x7fcd040a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.732+0000 7fcd137fe700 1 -- 172.21.15.2:0/695427531 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fcd180cbac0 con 0x7fcd1c108620 2023-09-13T04:45:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.732+0000 7fcd22d9c700 1 --2- 172.21.15.2:0/2717092425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605ee893be0 0x5605ee893fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.739+0000 7fcd2259b700 1 --2- 172.21.15.2:0/2717092425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5605ee894500 0x5605ee88f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.740+0000 7fcd22d9c700 1 -- 172.21.15.2:0/2717092425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ee88f840 msgr2=0x5605ee88fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.740+0000 7fcd22d9c700 1 --2- 172.21.15.2:0/2717092425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ee88f840 0x5605ee88fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.740+0000 7fcd22d9c700 1 -- 172.21.15.2:0/2717092425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5605ee894500 msgr2=0x5605ee88f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.740+0000 7fcd22d9c700 1 --2- 172.21.15.2:0/2717092425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5605ee894500 0x5605ee88f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.740+0000 7fcd22d9c700 1 -- 172.21.15.2:0/2717092425 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605ee6615d0 con 0x5605ee893be0 2023-09-13T04:45:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.740+0000 7fcd22d9c700 1 --2- 172.21.15.2:0/695427531 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcd040a09c0 0x7fcd040a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.752+0000 7fcd22d9c700 1 --2- 172.21.15.2:0/2717092425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605ee893be0 0x5605ee893fc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009230 tx=0x7fcd14009940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f6c42c25b773262 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.753+0000 7fcd10ff9700 1 -- 172.21.15.2:0/2717092425 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd14029030 con 0x5605ee893be0 2023-09-13T04:45:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.753+0000 7fcd10ff9700 1 -- 172.21.15.2:0/2717092425 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd14005500 con 0x5605ee893be0 2023-09-13T04:45:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.753+0000 7fcd10ff9700 1 -- 172.21.15.2:0/2717092425 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd14017500 con 0x5605ee893be0 2023-09-13T04:45:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.754+0000 7fcd38021b80 1 -- 172.21.15.2:0/2717092425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605ee893be0 msgr2=0x5605ee893fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.754+0000 7fcd38021b80 1 --2- 172.21.15.2:0/2717092425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605ee893be0 0x5605ee893fc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009230 tx=0x7fcd14009940 comp rx=0 tx=0).stop 2023-09-13T04:45:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.754+0000 7fcd38021b80 1 -- 172.21.15.2:0/2717092425 shutdown_connections 2023-09-13T04:45:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.754+0000 7fcd38021b80 1 --2- 172.21.15.2:0/2717092425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ee88f840 0x5605ee88fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.754+0000 7fcd38021b80 1 --2- 172.21.15.2:0/2717092425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605ee893be0 0x5605ee893fc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.754+0000 7fcd38021b80 1 --2- 172.21.15.2:0/2717092425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5605ee894500 0x5605ee88f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.754+0000 7fcd38021b80 1 -- 172.21.15.2:0/2717092425 >> 172.21.15.2:0/2717092425 conn(0x5605ee7edc20 msgr2=0x5605ee889b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.755+0000 7fcd38021b80 1 -- 172.21.15.2:0/2717092425 shutdown_connections 2023-09-13T04:45:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.755+0000 7fcd38021b80 1 -- 172.21.15.2:0/2717092425 wait complete. 2023-09-13T04:45:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.755+0000 7fcd38021b80 1 -- start start 2023-09-13T04:45:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.756+0000 7fcd38021b80 1 Processor -- start 2023-09-13T04:45:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.757+0000 7fcd38021b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5605ee88f840 0x5605ee898990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.757+0000 7fcd38021b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ee893be0 0x5605ee898ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:59.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.757+0000 7fcd38021b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605ee894500 0x5605ee899410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:59.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.757+0000 7fcd38021b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ee77fb50 con 0x5605ee88f840 2023-09-13T04:45:59.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.757+0000 7fcd38021b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ee77f850 con 0x5605ee894500 2023-09-13T04:45:59.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.757+0000 7fcd38021b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ee6638c0 con 0x5605ee893be0 2023-09-13T04:45:59.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.757+0000 7fcd38021b80 10 client.0 _ll_register_callbacks cb 0x5605ee77d620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:45:59.766 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:45:59 smithi002 ceph-mon[97651]: pgmap v169: 129 pgs: 129 active+clean; 623 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 723 KiB/s wr, 53 op/s 2023-09-13T04:45:59.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.759+0000 7fcd22d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605ee894500 0x5605ee899410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:59.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.759+0000 7fcd22d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605ee894500 0x5605ee899410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:53324/0 (socket says 172.21.15.2:53324) 2023-09-13T04:45:59.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.759+0000 7fcd22d9c700 1 -- 172.21.15.2:0/4110607141 learned_addr learned my addr 172.21.15.2:0/4110607141 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:59.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.759+0000 7fcd2259b700 1 --2- 172.21.15.2:0/4110607141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5605ee88f840 0x5605ee898990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:59.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.759+0000 7fcd22d9c700 1 -- 172.21.15.2:0/4110607141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ee893be0 msgr2=0x5605ee898ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:45:59.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.759+0000 7fcd22d9c700 1 --2- 172.21.15.2:0/4110607141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ee893be0 0x5605ee898ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.759+0000 7fcd22d9c700 1 -- 172.21.15.2:0/4110607141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5605ee88f840 msgr2=0x5605ee898990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.759+0000 7fcd22d9c700 1 --2- 172.21.15.2:0/4110607141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5605ee88f840 0x5605ee898990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.759+0000 7fcd22d9c700 1 -- 172.21.15.2:0/4110607141 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1c079660 con 0x5605ee894500 2023-09-13T04:45:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.760+0000 7fcd22d9c700 1 --2- 172.21.15.2:0/4110607141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605ee894500 0x5605ee899410 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcd1402d000 tx=0x7fcd14005b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=957c237e3c0e62f2 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.770+0000 7fcd22d9c700 1 --2- 172.21.15.2:0/695427531 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcd040a09c0 0x7fcd040a2e80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009230 tx=0x7fcd14009940 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.782+0000 7fcd01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:45:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.782+0000 7fcd01ffb700 1 -- 172.21.15.2:0/4110607141 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd14029070 con 0x5605ee894500 2023-09-13T04:45:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.782+0000 7fcd01ffb700 1 -- 172.21.15.2:0/4110607141 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fcd1402a070 con 0x5605ee894500 2023-09-13T04:45:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.783+0000 7fcd38021b80 1 -- 172.21.15.2:0/4110607141 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605ee854ef0 con 0x5605ee894500 2023-09-13T04:45:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.787+0000 7fcd01ffb700 1 -- 172.21.15.2:0/4110607141 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd14005320 con 0x5605ee894500 2023-09-13T04:45:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.787+0000 7fcd38021b80 10 client.26381 fetch_fsmap learned FSMap version 33 2023-09-13T04:45:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.787+0000 7fcd38021b80 1 -- 172.21.15.2:0/4110607141 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5605ee7f9740 con 0x5605ee894500 2023-09-13T04:45:59.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.791+0000 7fcd01ffb700 1 -- 172.21.15.2:0/4110607141 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5605ee854ef0 con 0x5605ee894500 2023-09-13T04:45:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.791+0000 7fcd01ffb700 1 -- 172.21.15.2:0/4110607141 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 33) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fcd1400f610 con 0x5605ee894500 2023-09-13T04:45:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.792+0000 7fcd38021b80 10 client.26381 fetch_fsmap finished waiting for FSMap version 33 2023-09-13T04:45:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.792+0000 7fcd38021b80 10 client.26381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:45:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.792+0000 7fcd38021b80 20 client.26381 populate_metadata read hostname 'smithi002' 2023-09-13T04:45:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.792+0000 7fcd38021b80 1 --2- 172.21.15.2:0/4110607141 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5605ee8a8d00 0x5605ee8ab120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.792+0000 7fcd38021b80 4 client.26381 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:45:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.792+0000 7fcd38021b80 1 -- 172.21.15.2:0/4110607141 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605ee7da910 con 0x5605ee8a8d00 2023-09-13T04:45:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.793+0000 7fcd2259b700 1 --2- 172.21.15.2:0/4110607141 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5605ee8a8d00 0x5605ee8ab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.817+0000 7fcd2259b700 1 --2- 172.21.15.2:0/4110607141 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5605ee8a8d00 0x5605ee8ab120 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=332c57567b837c0 server_cookie=ed65299ce38d2407 in_seq=0 out_seq=0 2023-09-13T04:45:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.817+0000 7fcd01ffb700 10 client.26381 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:45:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.824+0000 7fcd01ffb700 1 -- 172.21.15.2:0/4110607141 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5605ee7da910 con 0x5605ee8a8d00 2023-09-13T04:45:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.824+0000 7fcd01ffb700 10 client.26381 handle_command_reply: tid=0 2023-09-13T04:45:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.824+0000 7fcd38021b80 1 client.26381 shutdown 2023-09-13T04:45:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.825+0000 7fcd38021b80 1 -- 172.21.15.2:0/4110607141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605ee894500 msgr2=0x5605ee899410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.825+0000 7fcd38021b80 1 --2- 172.21.15.2:0/4110607141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605ee894500 0x5605ee899410 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcd1402d000 tx=0x7fcd14005b40 comp rx=0 tx=0).stop 2023-09-13T04:45:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.825+0000 7fcd38021b80 1 -- 172.21.15.2:0/4110607141 shutdown_connections 2023-09-13T04:45:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.825+0000 7fcd38021b80 1 --2- 172.21.15.2:0/4110607141 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5605ee8a8d00 0x5605ee8ab120 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.825+0000 7fcd38021b80 1 --2- 172.21.15.2:0/4110607141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605ee894500 0x5605ee899410 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.825+0000 7fcd38021b80 1 --2- 172.21.15.2:0/4110607141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5605ee88f840 0x5605ee898990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.825+0000 7fcd38021b80 1 --2- 172.21.15.2:0/4110607141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ee893be0 0x5605ee898ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.825+0000 7fcd38021b80 1 -- 172.21.15.2:0/4110607141 >> 172.21.15.2:0/4110607141 conn(0x5605ee7edc20 msgr2=0x5605ee884630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.825+0000 7fcd01ffb700 0 client.26381 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:45:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.826+0000 7fcd38021b80 1 -- 172.21.15.2:0/4110607141 shutdown_connections 2023-09-13T04:45:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.826+0000 7fcd38021b80 1 -- 172.21.15.2:0/4110607141 wait complete. 2023-09-13T04:45:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.827+0000 7fcd38021b80 20 client.26381 trim_cache size 0 max 16384 2023-09-13T04:45:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.882+0000 7fcd38021b80 1 Processor -- start 2023-09-13T04:45:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.882+0000 7fcd38021b80 1 -- start start 2023-09-13T04:45:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.883+0000 7fcd38021b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605ee88f840 0x5605ee884730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.883+0000 7fcd38021b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ee893be0 0x5605ee8a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.883+0000 7fcd38021b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5605ee894500 0x5605ee88d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.883+0000 7fcd38021b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ee89bd30 con 0x5605ee894500 2023-09-13T04:45:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.883+0000 7fcd38021b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ee88f400 con 0x5605ee88f840 2023-09-13T04:45:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.883+0000 7fcd38021b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ee886440 con 0x5605ee893be0 2023-09-13T04:45:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.883+0000 7fcd22d9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5605ee894500 0x5605ee88d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.883+0000 7fcd22d9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5605ee894500 0x5605ee88d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48612/0 (socket says 172.21.15.2:48612) 2023-09-13T04:45:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.883+0000 7fcd22d9c700 1 -- 172.21.15.2:0/2464595697 learned_addr learned my addr 172.21.15.2:0/2464595697 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.884+0000 7fcd2259b700 1 --2- 172.21.15.2:0/2464595697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605ee88f840 0x5605ee884730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.884+0000 7fcd21d9a700 1 --2- 172.21.15.2:0/2464595697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ee893be0 0x5605ee8a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.884+0000 7fcd22d9c700 1 -- 172.21.15.2:0/2464595697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ee893be0 msgr2=0x5605ee8a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.884+0000 7fcd22d9c700 1 --2- 172.21.15.2:0/2464595697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ee893be0 0x5605ee8a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.884+0000 7fcd22d9c700 1 -- 172.21.15.2:0/2464595697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605ee88f840 msgr2=0x5605ee884730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.884+0000 7fcd22d9c700 1 --2- 172.21.15.2:0/2464595697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605ee88f840 0x5605ee884730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.884+0000 7fcd22d9c700 1 -- 172.21.15.2:0/2464595697 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605ee88fef0 con 0x5605ee894500 2023-09-13T04:45:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.885+0000 7fcd22d9c700 1 --2- 172.21.15.2:0/2464595697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5605ee894500 0x5605ee88d2f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fcd140092a0 tx=0x7fcd1400beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fb09ad3286114 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.886+0000 7fcd117fa700 1 -- 172.21.15.2:0/2464595697 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd14029070 con 0x5605ee894500 2023-09-13T04:45:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.886+0000 7fcd117fa700 1 -- 172.21.15.2:0/2464595697 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd1400fb90 con 0x5605ee894500 2023-09-13T04:45:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.886+0000 7fcd117fa700 1 -- 172.21.15.2:0/2464595697 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd14017820 con 0x5605ee894500 2023-09-13T04:45:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.887+0000 7fcd38021b80 1 -- 172.21.15.2:0/2464595697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5605ee894500 msgr2=0x5605ee88d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:45:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.887+0000 7fcd38021b80 1 --2- 172.21.15.2:0/2464595697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5605ee894500 0x5605ee88d2f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fcd140092a0 tx=0x7fcd1400beb0 comp rx=0 tx=0).stop 2023-09-13T04:45:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.887+0000 7fcd38021b80 1 -- 172.21.15.2:0/2464595697 shutdown_connections 2023-09-13T04:45:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.887+0000 7fcd38021b80 1 --2- 172.21.15.2:0/2464595697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5605ee894500 0x5605ee88d2f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.887+0000 7fcd38021b80 1 --2- 172.21.15.2:0/2464595697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605ee88f840 0x5605ee884730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:59.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.887+0000 7fcd38021b80 1 --2- 172.21.15.2:0/2464595697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ee893be0 0x5605ee8a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:59.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.887+0000 7fcd38021b80 1 -- 172.21.15.2:0/2464595697 >> 172.21.15.2:0/2464595697 conn(0x5605ee7edc20 msgr2=0x5605ee88ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:45:59.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.888+0000 7fcd38021b80 1 -- 172.21.15.2:0/2464595697 shutdown_connections 2023-09-13T04:45:59.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.888+0000 7fcd38021b80 1 -- 172.21.15.2:0/2464595697 wait complete. 2023-09-13T04:45:59.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.889+0000 7fcd38021b80 1 -- start start 2023-09-13T04:45:59.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.890+0000 7fcd38021b80 1 Processor -- start 2023-09-13T04:45:59.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.891+0000 7fcd38021b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5605ee88f840 0x5605ee89fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:59.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.891+0000 7fcd38021b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ee893be0 0x5605ee8a0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:59.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.891+0000 7fcd38021b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605ee894500 0x5605ee89e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:59.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.891+0000 7fcd38021b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ee89e810 con 0x5605ee88f840 2023-09-13T04:45:59.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.891+0000 7fcd38021b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ee89e980 con 0x5605ee894500 2023-09-13T04:45:59.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.891+0000 7fcd38021b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ee89eb40 con 0x5605ee893be0 2023-09-13T04:45:59.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.891+0000 7fcd38021b80 10 client.0 _ll_register_callbacks cb 0x5605ee77d620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:45:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.892+0000 7fcd21d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ee893be0 0x5605ee8a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.892+0000 7fcd22d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605ee894500 0x5605ee89e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.892+0000 7fcd22d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605ee894500 0x5605ee89e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:53354/0 (socket says 172.21.15.2:53354) 2023-09-13T04:45:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.892+0000 7fcd22d9c700 1 -- 172.21.15.2:0/335274311 learned_addr learned my addr 172.21.15.2:0/335274311 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:45:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.892+0000 7fcd2259b700 1 --2- 172.21.15.2:0/335274311 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5605ee88f840 0x5605ee89fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.892+0000 7fcd21d9a700 1 -- 172.21.15.2:0/335274311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605ee894500 msgr2=0x5605ee89e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.892+0000 7fcd21d9a700 1 --2- 172.21.15.2:0/335274311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605ee894500 0x5605ee89e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.892+0000 7fcd21d9a700 1 -- 172.21.15.2:0/335274311 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5605ee88f840 msgr2=0x5605ee89fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:45:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.892+0000 7fcd21d9a700 1 --2- 172.21.15.2:0/335274311 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5605ee88f840 0x5605ee89fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:45:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.892+0000 7fcd21d9a700 1 -- 172.21.15.2:0/335274311 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605ee7f9740 con 0x5605ee893be0 2023-09-13T04:45:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.895+0000 7fcd21d9a700 1 --2- 172.21.15.2:0/335274311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ee893be0 0x5605ee8a0110 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x5605ee88afa0 tx=0x7fcd18007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=279a106748ee1ec server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:45:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.896+0000 7fcd01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:45:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.896+0000 7fcd01ffb700 1 -- 172.21.15.2:0/335274311 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd18094f40 con 0x5605ee893be0 2023-09-13T04:45:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.896+0000 7fcd01ffb700 1 -- 172.21.15.2:0/335274311 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fcd18099300 con 0x5605ee893be0 2023-09-13T04:45:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.896+0000 7fcd38021b80 1 -- 172.21.15.2:0/335274311 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605ee79aff0 con 0x5605ee893be0 2023-09-13T04:45:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.897+0000 7fcd01ffb700 1 -- 172.21.15.2:0/335274311 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd180989d0 con 0x5605ee893be0 2023-09-13T04:45:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.897+0000 7fcd38021b80 10 client.26407 fetch_fsmap learned FSMap version 33 2023-09-13T04:45:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.897+0000 7fcd38021b80 1 -- 172.21.15.2:0/335274311 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5605ee88a110 con 0x5605ee893be0 2023-09-13T04:45:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.912+0000 7fcd01ffb700 1 -- 172.21.15.2:0/335274311 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5605ee79aff0 con 0x5605ee893be0 2023-09-13T04:45:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.912+0000 7fcd01ffb700 1 -- 172.21.15.2:0/335274311 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fcd180a07e0 con 0x5605ee893be0 2023-09-13T04:45:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.913+0000 7fcd38021b80 10 client.26407 fetch_fsmap finished waiting for FSMap version 33 2023-09-13T04:45:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.913+0000 7fcd38021b80 10 client.26407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:45:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.913+0000 7fcd38021b80 20 client.26407 populate_metadata read hostname 'smithi002' 2023-09-13T04:45:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.913+0000 7fcd38021b80 1 --2- 172.21.15.2:0/335274311 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5605ee8fdee0 0x5605ee900300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:45:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.913+0000 7fcd38021b80 4 client.26407 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:45:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.913+0000 7fcd38021b80 1 -- 172.21.15.2:0/335274311 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5605ee62ef10 con 0x5605ee8fdee0 2023-09-13T04:45:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.913+0000 7fcd2259b700 1 --2- 172.21.15.2:0/335274311 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5605ee8fdee0 0x5605ee900300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:45:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.921+0000 7fcd2259b700 1 --2- 172.21.15.2:0/335274311 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5605ee8fdee0 0x5605ee900300 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f74499cf148d329b server_cookie=beb6c13025b27a56 in_seq=0 out_seq=0 2023-09-13T04:45:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:45:59.921+0000 7fcd01ffb700 10 client.26407 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:46:00.708 INFO:tasks.workunit.client.0.smithi002.stdout: 130 600666 323 440368 192 251654 488 2023-09-13T04:46:02.012 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:01 smithi002 ceph-mon[97651]: pgmap v170: 129 pgs: 129 active+clean; 626 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 992 KiB/s wr, 74 op/s 2023-09-13T04:46:02.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:01 smithi149 ceph-mon[133315]: pgmap v170: 129 pgs: 129 active+clean; 626 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 992 KiB/s wr, 74 op/s 2023-09-13T04:46:02.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:01 smithi156 ceph-mon[133675]: pgmap v170: 129 pgs: 129 active+clean; 626 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 992 KiB/s wr, 74 op/s 2023-09-13T04:46:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.011+0000 7fcd01ffb700 1 -- 172.21.15.2:0/335274311 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5605ee62ef10 con 0x5605ee8fdee0 2023-09-13T04:46:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.011+0000 7fcd01ffb700 10 client.26407 handle_command_reply: tid=0 2023-09-13T04:46:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.018+0000 7fcd38021b80 1 client.26407 shutdown 2023-09-13T04:46:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.019+0000 7fcd38021b80 1 -- 172.21.15.2:0/335274311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ee893be0 msgr2=0x5605ee8a0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.019+0000 7fcd38021b80 1 --2- 172.21.15.2:0/335274311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ee893be0 0x5605ee8a0110 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x5605ee88afa0 tx=0x7fcd18007f90 comp rx=0 tx=0).stop 2023-09-13T04:46:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.019+0000 7fcd38021b80 1 -- 172.21.15.2:0/335274311 shutdown_connections 2023-09-13T04:46:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.019+0000 7fcd38021b80 1 --2- 172.21.15.2:0/335274311 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5605ee8fdee0 0x5605ee900300 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.019+0000 7fcd38021b80 1 --2- 172.21.15.2:0/335274311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605ee894500 0x5605ee89e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.019+0000 7fcd38021b80 1 --2- 172.21.15.2:0/335274311 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5605ee88f840 0x5605ee89fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.019+0000 7fcd38021b80 1 --2- 172.21.15.2:0/335274311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ee893be0 0x5605ee8a0110 secure :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0x5605ee88afa0 tx=0x7fcd18007f90 comp rx=0 tx=0).stop 2023-09-13T04:46:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.019+0000 7fcd38021b80 1 -- 172.21.15.2:0/335274311 >> 172.21.15.2:0/335274311 conn(0x5605ee7edc20 msgr2=0x5605ee89a230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.029+0000 7fcd01ffb700 0 client.26407 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:46:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.030+0000 7fcd38021b80 1 -- 172.21.15.2:0/335274311 shutdown_connections 2023-09-13T04:46:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.030+0000 7fcd38021b80 1 -- 172.21.15.2:0/335274311 wait complete. 2023-09-13T04:46:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:46:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:46:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:46:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:46:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.033+0000 7fcd38021b80 20 client.26407 trim_cache size 0 max 16384 2023-09-13T04:46:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.120+0000 7fcd247ff700 1 -- 172.21.15.2:0/695427531 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcd040a09c0 msgr2=0x7fcd040a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.120+0000 7fcd247ff700 1 --2- 172.21.15.2:0/695427531 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcd040a09c0 0x7fcd040a2e80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009230 tx=0x7fcd14009940 comp rx=0 tx=0).stop 2023-09-13T04:46:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.120+0000 7fcd247ff700 1 -- 172.21.15.2:0/695427531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd1c108620 msgr2=0x7fcd1c0835c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.120+0000 7fcd247ff700 1 --2- 172.21.15.2:0/695427531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd1c108620 0x7fcd1c0835c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800c0c0 tx=0x7fcd1800c480 comp rx=0 tx=0).stop 2023-09-13T04:46:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.125+0000 7fcd247ff700 1 -- 172.21.15.2:0/695427531 shutdown_connections 2023-09-13T04:46:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.125+0000 7fcd247ff700 1 --2- 172.21.15.2:0/695427531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd1c10c590 0x7fcd1c07d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.125+0000 7fcd247ff700 1 --2- 172.21.15.2:0/695427531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd1c107d00 0x7fcd1c083080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.125+0000 7fcd247ff700 1 --2- 172.21.15.2:0/695427531 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcd040a09c0 0x7fcd040a2e80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.125+0000 7fcd247ff700 1 --2- 172.21.15.2:0/695427531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd1c108620 0x7fcd1c0835c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.125+0000 7fcd247ff700 1 -- 172.21.15.2:0/695427531 >> 172.21.15.2:0/695427531 conn(0x7fcd1c06cd90 msgr2=0x7fcd1c0707c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:04.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:03 smithi149 ceph-mon[133315]: pgmap v171: 129 pgs: 129 active+clean; 626 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 912 KiB/s wr, 70 op/s 2023-09-13T04:46:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.133+0000 7fcd247ff700 1 -- 172.21.15.2:0/695427531 shutdown_connections 2023-09-13T04:46:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:04.137+0000 7fcd247ff700 1 -- 172.21.15.2:0/695427531 wait complete. 2023-09-13T04:46:04.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:03 smithi156 ceph-mon[133675]: pgmap v171: 129 pgs: 129 active+clean; 626 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 912 KiB/s wr, 70 op/s 2023-09-13T04:46:04.285 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:03 smithi002 ceph-mon[97651]: pgmap v171: 129 pgs: 129 active+clean; 626 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 912 KiB/s wr, 70 op/s 2023-09-13T04:46:05.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:05 smithi156 ceph-mon[133675]: pgmap v172: 129 pgs: 129 active+clean; 653 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 3.2 MiB/s wr, 319 op/s 2023-09-13T04:46:06.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:05 smithi002 ceph-mon[97651]: pgmap v172: 129 pgs: 129 active+clean; 653 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 3.2 MiB/s wr, 319 op/s 2023-09-13T04:46:06.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:05 smithi149 ceph-mon[133315]: pgmap v172: 129 pgs: 129 active+clean; 653 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 3.2 MiB/s wr, 319 op/s 2023-09-13T04:46:07.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:06 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:46:07.010 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:06 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:46:07.010 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:06 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:46:07.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:06 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:46:07.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:06 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:46:07.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:06 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:46:07.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:06 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:46:07.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:06 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:46:07.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:06 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:46:08.017 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:07 smithi002 ceph-mon[97651]: pgmap v173: 129 pgs: 129 active+clean; 657 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 3.2 MiB/s wr, 343 op/s 2023-09-13T04:46:08.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:07 smithi149 ceph-mon[133315]: pgmap v173: 129 pgs: 129 active+clean; 657 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 3.2 MiB/s wr, 343 op/s 2023-09-13T04:46:08.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:07 smithi156 ceph-mon[133675]: pgmap v173: 129 pgs: 129 active+clean; 657 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 3.2 MiB/s wr, 343 op/s 2023-09-13T04:46:10.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:09 smithi149 ceph-mon[133315]: pgmap v174: 129 pgs: 129 active+clean; 657 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 3.1 MiB/s wr, 344 op/s 2023-09-13T04:46:10.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:09 smithi156 ceph-mon[133675]: pgmap v174: 129 pgs: 129 active+clean; 657 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 3.1 MiB/s wr, 344 op/s 2023-09-13T04:46:10.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:09 smithi002 ceph-mon[97651]: pgmap v174: 129 pgs: 129 active+clean; 657 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 3.1 MiB/s wr, 344 op/s 2023-09-13T04:46:10.664 INFO:tasks.workunit.client.0.smithi002.stdout: 135 658790 241 473099 370 271916 449 2023-09-13T04:46:12.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:11 smithi149 ceph-mon[133315]: pgmap v175: 129 pgs: 129 active+clean; 666 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 415 op/s 2023-09-13T04:46:12.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:11 smithi156 ceph-mon[133675]: pgmap v175: 129 pgs: 129 active+clean; 666 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 415 op/s 2023-09-13T04:46:12.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:11 smithi002 ceph-mon[97651]: pgmap v175: 129 pgs: 129 active+clean; 666 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 415 op/s 2023-09-13T04:46:13.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:12 smithi149 ceph-mon[133315]: pgmap v176: 129 pgs: 129 active+clean; 670 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 403 op/s 2023-09-13T04:46:13.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:12 smithi156 ceph-mon[133675]: pgmap v176: 129 pgs: 129 active+clean; 670 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 403 op/s 2023-09-13T04:46:13.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:12 smithi002 ceph-mon[97651]: pgmap v176: 129 pgs: 129 active+clean; 670 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 403 op/s 2023-09-13T04:46:13.432 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T04:46:15.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:15 smithi156 ceph-mon[133675]: pgmap v177: 129 pgs: 129 active+clean; 672 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 424 op/s 2023-09-13T04:46:16.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:15 smithi002 ceph-mon[97651]: pgmap v177: 129 pgs: 129 active+clean; 672 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 424 op/s 2023-09-13T04:46:16.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:15 smithi149 ceph-mon[133315]: pgmap v177: 129 pgs: 129 active+clean; 672 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 424 op/s 2023-09-13T04:46:18.108 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:17 smithi002 ceph-mon[97651]: pgmap v178: 129 pgs: 129 active+clean; 672 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 174 op/s 2023-09-13T04:46:18.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:17 smithi149 ceph-mon[133315]: pgmap v178: 129 pgs: 129 active+clean; 672 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 174 op/s 2023-09-13T04:46:18.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:17 smithi156 ceph-mon[133675]: pgmap v178: 129 pgs: 129 active+clean; 672 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 174 op/s 2023-09-13T04:46:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.942+0000 7f2cbcbff700 1 -- 172.21.15.2:0/932979484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cb811a490 msgr2=0x7f2cb811c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.942+0000 7f2cbcbff700 1 --2- 172.21.15.2:0/932979484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cb811a490 0x7f2cb811c880 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000b0a0 tx=0x7f2cb000b3b0 comp rx=0 tx=0).stop 2023-09-13T04:46:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.943+0000 7f2cbcbff700 1 -- 172.21.15.2:0/932979484 shutdown_connections 2023-09-13T04:46:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.943+0000 7f2cbcbff700 1 --2- 172.21.15.2:0/932979484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cb811a490 0x7f2cb811c880 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.943+0000 7f2cbcbff700 1 --2- 172.21.15.2:0/932979484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2cb810f420 0x7f2cb8107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.943+0000 7f2cbcbff700 1 --2- 172.21.15.2:0/932979484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cb810f890 0x7f2cb81081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.943+0000 7f2cbcbff700 1 -- 172.21.15.2:0/932979484 >> 172.21.15.2:0/932979484 conn(0x7f2cb806cda0 msgr2=0x7f2cb806d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.943+0000 7f2cbcbff700 1 -- 172.21.15.2:0/932979484 shutdown_connections 2023-09-13T04:46:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.943+0000 7f2cbcbff700 1 -- 172.21.15.2:0/932979484 wait complete. 2023-09-13T04:46:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.945+0000 7f2cbcbff700 1 Processor -- start 2023-09-13T04:46:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.946+0000 7f2cbcbff700 1 -- start start 2023-09-13T04:46:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.946+0000 7f2cbcbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cb810f420 0x7f2cb8118840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.946+0000 7f2cbcbff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2cb810f890 0x7f2cb8118dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.946+0000 7f2cbcbff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cb811a490 0x7f2cb81116e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.946+0000 7f2cbcbff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb8111eb0 con 0x7f2cb810f890 2023-09-13T04:46:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.946+0000 7f2cbcbff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb8112020 con 0x7f2cb811a490 2023-09-13T04:46:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.946+0000 7f2cbcbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb81121e0 con 0x7f2cb810f420 2023-09-13T04:46:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.948+0000 7f2cb6d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cb811a490 0x7f2cb81116e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.948+0000 7f2cb6d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cb811a490 0x7f2cb81116e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:53050/0 (socket says 172.21.15.2:53050) 2023-09-13T04:46:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.948+0000 7f2cb6d9d700 1 -- 172.21.15.2:0/1986935115 learned_addr learned my addr 172.21.15.2:0/1986935115 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.948+0000 7f2cb659c700 1 --2- 172.21.15.2:0/1986935115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cb810f420 0x7f2cb8118840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.949+0000 7f2cb6d9d700 1 -- 172.21.15.2:0/1986935115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cb810f420 msgr2=0x7f2cb8118840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.949+0000 7f2cb6d9d700 1 --2- 172.21.15.2:0/1986935115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cb810f420 0x7f2cb8118840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.950+0000 7f2cb6d9d700 1 -- 172.21.15.2:0/1986935115 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2cb810f890 msgr2=0x7f2cb8118dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.950+0000 7f2cb6d9d700 1 --2- 172.21.15.2:0/1986935115 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2cb810f890 0x7f2cb8118dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.950+0000 7f2cb6d9d700 1 -- 172.21.15.2:0/1986935115 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb0009d00 con 0x7f2cb811a490 2023-09-13T04:46:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.971+0000 7f2cb6d9d700 1 --2- 172.21.15.2:0/1986935115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cb811a490 0x7f2cb81116e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0009040 tx=0x7f2cb00078a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.971+0000 7f2ca77fe700 1 -- 172.21.15.2:0/1986935115 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb000e040 con 0x7f2cb811a490 2023-09-13T04:46:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.971+0000 7f2ca77fe700 1 -- 172.21.15.2:0/1986935115 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cb0004e60 con 0x7f2cb811a490 2023-09-13T04:46:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.971+0000 7f2ca77fe700 1 -- 172.21.15.2:0/1986935115 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb001b6c0 con 0x7f2cb811a490 2023-09-13T04:46:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.972+0000 7f2cbcbff700 1 -- 172.21.15.2:0/1986935115 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb8112460 con 0x7f2cb811a490 2023-09-13T04:46:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.972+0000 7f2cbcbff700 1 -- 172.21.15.2:0/1986935115 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb8116610 con 0x7f2cb811a490 2023-09-13T04:46:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.979+0000 7f2cd04b7b80 1 Processor -- start 2023-09-13T04:46:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.980+0000 7f2cd04b7b80 1 -- start start 2023-09-13T04:46:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.980+0000 7f2cd04b7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5349cdbe0 0x55f5349cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.980+0000 7f2cd04b7b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5349ce500 0x55f5349c9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.980+0000 7f2cd04b7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349c9840 0x55f5349c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.980+0000 7f2cd04b7b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f534922720 con 0x55f5349ce500 2023-09-13T04:46:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.981+0000 7f2cd04b7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f534920040 con 0x55f5349cdbe0 2023-09-13T04:46:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.981+0000 7f2cd04b7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f53491b930 con 0x55f5349c9840 2023-09-13T04:46:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.981+0000 7f2cb5d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5349cdbe0 0x55f5349cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.981+0000 7f2cb5d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5349cdbe0 0x55f5349cdfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:53054/0 (socket says 172.21.15.2:53054) 2023-09-13T04:46:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.981+0000 7f2cb5d9b700 1 -- 172.21.15.2:0/1772491293 learned_addr learned my addr 172.21.15.2:0/1772491293 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.981+0000 7f2cb5d9b700 1 --2- 172.21.15.2:0/1772491293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349c9840 0x55f5349c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.982+0000 7f2cb5d9b700 1 -- 172.21.15.2:0/1772491293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349c9840 msgr2=0x55f5349c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.982+0000 7f2cb5d9b700 1 --2- 172.21.15.2:0/1772491293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349c9840 0x55f5349c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.982+0000 7f2cb5d9b700 1 -- 172.21.15.2:0/1772491293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5349ce500 msgr2=0x55f5349c9300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:46:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.982+0000 7f2cb5d9b700 1 --2- 172.21.15.2:0/1772491293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5349ce500 0x55f5349c9300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.982+0000 7f2cb5d9b700 1 -- 172.21.15.2:0/1772491293 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f53479b5d0 con 0x55f5349cdbe0 2023-09-13T04:46:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:18.982+0000 7f2cb5d9b700 1 --2- 172.21.15.2:0/1772491293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5349cdbe0 0x55f5349cdfc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2cac009fd0 tx=0x7f2cac009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42a3a54a00e3f058 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:19.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.000+0000 7f2ca77fe700 1 -- 172.21.15.2:0/1986935115 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f2cb001d470 con 0x7f2cb811a490 2023-09-13T04:46:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.003+0000 7f2ca77fe700 1 --2- 172.21.15.2:0/1986935115 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f2ca00a09c0 0x7f2ca00a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.003+0000 7f2ca77fe700 1 -- 172.21.15.2:0/1986935115 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f2cb00cd380 con 0x7f2cb811a490 2023-09-13T04:46:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.007+0000 7f2cb659c700 1 --2- 172.21.15.2:0/1986935115 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f2ca00a09c0 0x7f2ca00a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.024+0000 7f2cb659c700 1 --2- 172.21.15.2:0/1986935115 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f2ca00a09c0 0x7f2ca00a2e80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8006fd0 tx=0x7f2ca8008040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.024+0000 7f2ca4ff9700 1 -- 172.21.15.2:0/1772491293 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac004a30 con 0x55f5349cdbe0 2023-09-13T04:46:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.024+0000 7f2ca4ff9700 1 -- 172.21.15.2:0/1772491293 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cac019070 con 0x55f5349cdbe0 2023-09-13T04:46:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.024+0000 7f2ca4ff9700 1 -- 172.21.15.2:0/1772491293 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac00f6e0 con 0x55f5349cdbe0 2023-09-13T04:46:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.025+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/1772491293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5349cdbe0 msgr2=0x55f5349cdfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.025+0000 7f2cd04b7b80 1 --2- 172.21.15.2:0/1772491293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5349cdbe0 0x55f5349cdfc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2cac009fd0 tx=0x7f2cac009770 comp rx=0 tx=0).stop 2023-09-13T04:46:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.026+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/1772491293 shutdown_connections 2023-09-13T04:46:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.026+0000 7f2cd04b7b80 1 --2- 172.21.15.2:0/1772491293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349c9840 0x55f5349c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.026+0000 7f2cd04b7b80 1 --2- 172.21.15.2:0/1772491293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5349cdbe0 0x55f5349cdfc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.026+0000 7f2cd04b7b80 1 --2- 172.21.15.2:0/1772491293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5349ce500 0x55f5349c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.026+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/1772491293 >> 172.21.15.2:0/1772491293 conn(0x55f534927c20 msgr2=0x55f5349c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.026+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/1772491293 shutdown_connections 2023-09-13T04:46:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.026+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/1772491293 wait complete. 2023-09-13T04:46:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.027+0000 7f2cd04b7b80 1 -- start start 2023-09-13T04:46:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.028+0000 7f2cd04b7b80 1 Processor -- start 2023-09-13T04:46:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.028+0000 7f2cd04b7b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5349c9840 0x55f5349d2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.028+0000 7f2cd04b7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349cdbe0 0x55f5349d2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.028+0000 7f2cd04b7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5349ce500 0x55f5349d3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.030+0000 7f2cb5d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349cdbe0 0x55f5349d2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.031+0000 7f2cb5d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349cdbe0 0x55f5349d2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:33836/0 (socket says 172.21.15.2:33836) 2023-09-13T04:46:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.031+0000 7f2cd04b7b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5348b9b50 con 0x55f5349c9840 2023-09-13T04:46:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.031+0000 7f2cd04b7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5348b9850 con 0x55f5349ce500 2023-09-13T04:46:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.031+0000 7f2cd04b7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f53479d8c0 con 0x55f5349cdbe0 2023-09-13T04:46:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.031+0000 7f2cd04b7b80 10 client.0 _ll_register_callbacks cb 0x55f5348b7620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:46:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.031+0000 7f2cb5d9b700 1 -- 172.21.15.2:0/2095872235 learned_addr learned my addr 172.21.15.2:0/2095872235 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.039+0000 7f2cb5d9b700 1 -- 172.21.15.2:0/2095872235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5349ce500 msgr2=0x55f5349d3400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:46:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.039+0000 7f2cb5d9b700 1 --2- 172.21.15.2:0/2095872235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5349ce500 0x55f5349d3400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.039+0000 7f2cb5d9b700 1 -- 172.21.15.2:0/2095872235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5349c9840 msgr2=0x55f5349d2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.039+0000 7f2cb5d9b700 1 --2- 172.21.15.2:0/2095872235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5349c9840 0x55f5349d2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.039+0000 7f2cb5d9b700 1 -- 172.21.15.2:0/2095872235 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cac013520 con 0x55f5349cdbe0 2023-09-13T04:46:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.055+0000 7f2cb5d9b700 1 --2- 172.21.15.2:0/2095872235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349cdbe0 0x55f5349d2ec0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2cac005cb0 tx=0x7f2cac009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28fb3919e6d77cda server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.125+0000 7f2c9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:46:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.125+0000 7f2c9dffb700 1 -- 172.21.15.2:0/2095872235 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac004230 con 0x55f5349cdbe0 2023-09-13T04:46:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.125+0000 7f2c9dffb700 1 -- 172.21.15.2:0/2095872235 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f2cac004a40 con 0x55f5349cdbe0 2023-09-13T04:46:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.125+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/2095872235 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f53498eef0 con 0x55f5349cdbe0 2023-09-13T04:46:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.147+0000 7f2c9dffb700 1 -- 172.21.15.2:0/2095872235 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cac004390 con 0x55f5349cdbe0 2023-09-13T04:46:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.148+0000 7f2cd04b7b80 10 client.26419 fetch_fsmap learned FSMap version 33 2023-09-13T04:46:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.148+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/2095872235 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f534933740 con 0x55f5349cdbe0 2023-09-13T04:46:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.156+0000 7f2c9dffb700 1 -- 172.21.15.2:0/2095872235 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cac02e030 con 0x55f5349cdbe0 2023-09-13T04:46:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.157+0000 7f2c9dffb700 1 -- 172.21.15.2:0/2095872235 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f2cac004de0 con 0x55f5349cdbe0 2023-09-13T04:46:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.157+0000 7f2cd04b7b80 10 client.26419 fetch_fsmap finished waiting for FSMap version 33 2023-09-13T04:46:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.157+0000 7f2cd04b7b80 10 client.26419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:46:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.157+0000 7f2cd04b7b80 20 client.26419 populate_metadata read hostname 'smithi002' 2023-09-13T04:46:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.157+0000 7f2cd04b7b80 1 --2- 172.21.15.2:0/2095872235 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f5349e2ef0 0x55f5349e5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.157+0000 7f2cd04b7b80 4 client.26419 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:46:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.157+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/2095872235 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f534914910 con 0x55f5349e2ef0 2023-09-13T04:46:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.157+0000 7f2cb659c700 1 --2- 172.21.15.2:0/2095872235 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f5349e2ef0 0x55f5349e5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.168+0000 7f2cb659c700 1 --2- 172.21.15.2:0/2095872235 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f5349e2ef0 0x55f5349e5310 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7baaeed837cd264 server_cookie=4f5b4737a5d5548a in_seq=0 out_seq=0 2023-09-13T04:46:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.168+0000 7f2c9dffb700 10 client.26419 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:46:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.180+0000 7f2c9dffb700 1 -- 172.21.15.2:0/2095872235 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f534914910 con 0x55f5349e2ef0 2023-09-13T04:46:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.180+0000 7f2c9dffb700 10 client.26419 handle_command_reply: tid=0 2023-09-13T04:46:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.181+0000 7f2cd04b7b80 1 client.26419 shutdown 2023-09-13T04:46:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.181+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/2095872235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349cdbe0 msgr2=0x55f5349d2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.181+0000 7f2cd04b7b80 1 --2- 172.21.15.2:0/2095872235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349cdbe0 0x55f5349d2ec0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2cac005cb0 tx=0x7f2cac009da0 comp rx=0 tx=0).stop 2023-09-13T04:46:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.182+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/2095872235 shutdown_connections 2023-09-13T04:46:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.182+0000 7f2cd04b7b80 1 --2- 172.21.15.2:0/2095872235 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f5349e2ef0 0x55f5349e5310 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.182+0000 7f2cd04b7b80 1 --2- 172.21.15.2:0/2095872235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5349ce500 0x55f5349d3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.182+0000 7f2cd04b7b80 1 --2- 172.21.15.2:0/2095872235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5349c9840 0x55f5349d2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.182+0000 7f2cd04b7b80 1 --2- 172.21.15.2:0/2095872235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349cdbe0 0x55f5349d2ec0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.182+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/2095872235 >> 172.21.15.2:0/2095872235 conn(0x55f534927c20 msgr2=0x55f5349be630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.182+0000 7f2c9dffb700 0 client.26419 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:46:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.182+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/2095872235 shutdown_connections 2023-09-13T04:46:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.182+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/2095872235 wait complete. 2023-09-13T04:46:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.286+0000 7f2cd04b7b80 20 client.26419 trim_cache size 0 max 16384 2023-09-13T04:46:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.604+0000 7f2cd04b7b80 1 Processor -- start 2023-09-13T04:46:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.632+0000 7f2cd04b7b80 1 -- start start 2023-09-13T04:46:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.633+0000 7f2cd04b7b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5349c9840 0x55f5349be730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.633+0000 7f2cd04b7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5349cdbe0 0x55f5349d9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.633+0000 7f2cd04b7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349ce500 0x55f5349d2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.633+0000 7f2cd04b7b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5349d59c0 con 0x55f5349c9840 2023-09-13T04:46:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.633+0000 7f2cd04b7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5349d3950 con 0x55f5349cdbe0 2023-09-13T04:46:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.633+0000 7f2cd04b7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5349bec70 con 0x55f5349ce500 2023-09-13T04:46:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.636+0000 7f2cb5d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5349cdbe0 0x55f5349d9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.636+0000 7f2cb5d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5349cdbe0 0x55f5349d9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:53080/0 (socket says 172.21.15.2:53080) 2023-09-13T04:46:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.636+0000 7f2cb5d9b700 1 -- 172.21.15.2:0/204313425 learned_addr learned my addr 172.21.15.2:0/204313425 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.636+0000 7f2cb6d9d700 1 --2- 172.21.15.2:0/204313425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349ce500 0x55f5349d2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.647+0000 7f2cb6d9d700 1 -- 172.21.15.2:0/204313425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5349cdbe0 msgr2=0x55f5349d9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.647+0000 7f2cb6d9d700 1 --2- 172.21.15.2:0/204313425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5349cdbe0 0x55f5349d9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.647+0000 7f2cb6d9d700 1 -- 172.21.15.2:0/204313425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5349c9840 msgr2=0x55f5349be730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.647+0000 7f2cb6d9d700 1 --2- 172.21.15.2:0/204313425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5349c9840 0x55f5349be730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.647+0000 7f2cb6d9d700 1 -- 172.21.15.2:0/204313425 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f534747d80 con 0x55f5349ce500 2023-09-13T04:46:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.648+0000 7f2cb6d9d700 1 --2- 172.21.15.2:0/204313425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349ce500 0x55f5349d2140 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2cb00ca760 tx=0x7f2cb009a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2df5a1cdfacde1fd server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.995+0000 7f2ca57fa700 1 -- 172.21.15.2:0/204313425 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb009e170 con 0x55f5349ce500 2023-09-13T04:46:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.995+0000 7f2ca57fa700 1 -- 172.21.15.2:0/204313425 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cb009ac50 con 0x55f5349ce500 2023-09-13T04:46:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.995+0000 7f2ca57fa700 1 -- 172.21.15.2:0/204313425 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb00ca9e0 con 0x55f5349ce500 2023-09-13T04:46:19.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.998+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/204313425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349ce500 msgr2=0x55f5349d2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:20.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:19.998+0000 7f2cd04b7b80 1 --2- 172.21.15.2:0/204313425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349ce500 0x55f5349d2140 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2cb00ca760 tx=0x7f2cb009a3f0 comp rx=0 tx=0).stop 2023-09-13T04:46:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.001+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/204313425 shutdown_connections 2023-09-13T04:46:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.001+0000 7f2cd04b7b80 1 --2- 172.21.15.2:0/204313425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349ce500 0x55f5349d2140 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.001+0000 7f2cd04b7b80 1 --2- 172.21.15.2:0/204313425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5349c9840 0x55f5349be730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.001+0000 7f2cd04b7b80 1 --2- 172.21.15.2:0/204313425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5349cdbe0 0x55f5349d9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.001+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/204313425 >> 172.21.15.2:0/204313425 conn(0x55f534927c20 msgr2=0x55f5349de6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.012+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/204313425 shutdown_connections 2023-09-13T04:46:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.012+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/204313425 wait complete. 2023-09-13T04:46:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.013+0000 7f2cd04b7b80 1 -- start start 2023-09-13T04:46:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.013+0000 7f2cd04b7b80 1 Processor -- start 2023-09-13T04:46:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.018+0000 7f2cd04b7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349c9840 0x55f5349d5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.018+0000 7f2cd04b7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5349cdbe0 0x55f5349d6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.018+0000 7f2cd04b7b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5349ce500 0x55f5349d69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.018+0000 7f2cd04b7b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5349da360 con 0x55f5349ce500 2023-09-13T04:46:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.018+0000 7f2cd04b7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f534a35d30 con 0x55f5349cdbe0 2023-09-13T04:46:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.018+0000 7f2cd04b7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f534a35ef0 con 0x55f5349c9840 2023-09-13T04:46:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.018+0000 7f2cd04b7b80 10 client.0 _ll_register_callbacks cb 0x55f5348b7620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:46:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.022+0000 7f2cb5d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5349cdbe0 0x55f5349d6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.022+0000 7f2cb659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349c9840 0x55f5349d5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.022+0000 7f2cb659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349c9840 0x55f5349d5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:33854/0 (socket says 172.21.15.2:33854) 2023-09-13T04:46:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.022+0000 7f2cb659c700 1 -- 172.21.15.2:0/3897006610 learned_addr learned my addr 172.21.15.2:0/3897006610 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:20.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.024+0000 7f2cb659c700 1 -- 172.21.15.2:0/3897006610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5349cdbe0 msgr2=0x55f5349d6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:20.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.024+0000 7f2cb659c700 1 --2- 172.21.15.2:0/3897006610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5349cdbe0 0x55f5349d6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:20.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.025+0000 7f2cb659c700 1 -- 172.21.15.2:0/3897006610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5349ce500 msgr2=0x55f5349d69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.025+0000 7f2cb659c700 1 --2- 172.21.15.2:0/3897006610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5349ce500 0x55f5349d69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.025+0000 7f2cb659c700 1 -- 172.21.15.2:0/3897006610 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb0096040 con 0x55f5349c9840 2023-09-13T04:46:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.035+0000 7f2cb659c700 1 --2- 172.21.15.2:0/3897006610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349c9840 0x55f5349d5f50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8022cb0 tx=0x7f2ca8022ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab35255c2f7d9d66 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.105+0000 7f2c9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:46:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.105+0000 7f2c9dffb700 1 -- 172.21.15.2:0/3897006610 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8022e00 con 0x55f5349c9840 2023-09-13T04:46:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.105+0000 7f2c9dffb700 1 -- 172.21.15.2:0/3897006610 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f2ca8027030 con 0x55f5349c9840 2023-09-13T04:46:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.106+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/3897006610 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2cac019070 con 0x55f5349c9840 2023-09-13T04:46:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.127+0000 7f2c9dffb700 1 -- 172.21.15.2:0/3897006610 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ca8005a50 con 0x55f5349c9840 2023-09-13T04:46:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.127+0000 7f2cd04b7b80 10 client.26425 fetch_fsmap learned FSMap version 33 2023-09-13T04:46:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.127+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/3897006610 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f534730c00 con 0x55f5349c9840 2023-09-13T04:46:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.128+0000 7f2c9dffb700 1 -- 172.21.15.2:0/3897006610 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ca8030030 con 0x55f5349c9840 2023-09-13T04:46:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.129+0000 7f2c9dffb700 1 -- 172.21.15.2:0/3897006610 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f2ca8014b00 con 0x55f5349c9840 2023-09-13T04:46:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.130+0000 7f2cd04b7b80 10 client.26425 fetch_fsmap finished waiting for FSMap version 33 2023-09-13T04:46:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.130+0000 7f2cd04b7b80 10 client.26425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:46:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.130+0000 7f2cd04b7b80 20 client.26425 populate_metadata read hostname 'smithi002' 2023-09-13T04:46:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.130+0000 7f2cd04b7b80 1 --2- 172.21.15.2:0/3897006610 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f534a376b0 0x55f534a39ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.130+0000 7f2cd04b7b80 4 client.26425 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:46:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.130+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/3897006610 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f5348d4ff0 con 0x55f534a376b0 2023-09-13T04:46:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.131+0000 7f2cb5d9b700 1 --2- 172.21.15.2:0/3897006610 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f534a376b0 0x55f534a39ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:20.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:19 smithi149 ceph-mon[133315]: pgmap v179: 129 pgs: 129 active+clean; 675 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 146 op/s 2023-09-13T04:46:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.158+0000 7f2cb5d9b700 1 --2- 172.21.15.2:0/3897006610 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f534a376b0 0x55f534a39ad0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bb38d7da34f20f6 server_cookie=4e4ad6cf6d8f9d63 in_seq=0 out_seq=0 2023-09-13T04:46:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:20.161+0000 7f2c9dffb700 10 client.26425 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:46:20.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:19 smithi156 ceph-mon[133675]: pgmap v179: 129 pgs: 129 active+clean; 675 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 146 op/s 2023-09-13T04:46:20.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:19 smithi002 ceph-mon[97651]: pgmap v179: 129 pgs: 129 active+clean; 675 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 146 op/s 2023-09-13T04:46:20.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.418+0000 7f47d259d700 1 -- 172.21.15.2:0/711525240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47cc10f230 msgr2=0x7f47cc10f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:20.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.418+0000 7f47d259d700 1 --2- 172.21.15.2:0/711525240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47cc10f230 0x7f47cc10f610 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009990 tx=0x7f47c8009ca0 comp rx=0 tx=0).stop 2023-09-13T04:46:20.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.665+0000 7f47d259d700 1 -- 172.21.15.2:0/711525240 shutdown_connections 2023-09-13T04:46:20.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.665+0000 7f47d259d700 1 --2- 172.21.15.2:0/711525240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47cc10d8b0 0x7f47cc11c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:20.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.665+0000 7f47d259d700 1 --2- 172.21.15.2:0/711525240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47cc10f230 0x7f47cc10f610 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:20.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.665+0000 7f47d259d700 1 --2- 172.21.15.2:0/711525240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47cc10cd90 0x7f47cc10d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:20.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.665+0000 7f47d259d700 1 -- 172.21.15.2:0/711525240 >> 172.21.15.2:0/711525240 conn(0x7f47cc06cec0 msgr2=0x7f47cc06d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:20.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.665+0000 7f47d259d700 1 -- 172.21.15.2:0/711525240 shutdown_connections 2023-09-13T04:46:20.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.665+0000 7f47d259d700 1 -- 172.21.15.2:0/711525240 wait complete. 2023-09-13T04:46:20.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.667+0000 7f47d259d700 1 Processor -- start 2023-09-13T04:46:20.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.668+0000 7f47d259d700 1 -- start start 2023-09-13T04:46:20.670 INFO:tasks.workunit.client.0.smithi002.stdout: 142 696157 301 502628 446 286579 293 2023-09-13T04:46:20.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.722+0000 7f47d259d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47cc10cd90 0x7f47cc1146c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:20.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.723+0000 7f47d259d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47cc10d8b0 0x7f47cc114c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:20.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.723+0000 7f47d259d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47cc10f230 0x7f47cc1b9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:20.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.723+0000 7f47d259d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc1ba450 con 0x7f47cc10d8b0 2023-09-13T04:46:20.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.723+0000 7f47d259d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc1ba590 con 0x7f47cc10f230 2023-09-13T04:46:20.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.723+0000 7f47d259d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc1ba700 con 0x7f47cc10cd90 2023-09-13T04:46:20.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.725+0000 7f47d0d9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47cc10d8b0 0x7f47cc114c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:20.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.725+0000 7f47d0d9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47cc10d8b0 0x7f47cc114c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44342/0 (socket says 172.21.15.2:44342) 2023-09-13T04:46:20.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.725+0000 7f47d0d9a700 1 -- 172.21.15.2:0/949341832 learned_addr learned my addr 172.21.15.2:0/949341832 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:20.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.737+0000 7f47d0d9a700 1 -- 172.21.15.2:0/949341832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47cc10cd90 msgr2=0x7f47cc1146c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-13T04:46:20.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.737+0000 7f47d0d9a700 1 --2- 172.21.15.2:0/949341832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47cc10cd90 0x7f47cc1146c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:20.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.737+0000 7f47d0d9a700 1 -- 172.21.15.2:0/949341832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47cc10f230 msgr2=0x7f47cc1b9f10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-13T04:46:20.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.737+0000 7f47d0d9a700 1 --2- 172.21.15.2:0/949341832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47cc10f230 0x7f47cc1b9f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:20.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.738+0000 7f47d0d9a700 1 -- 172.21.15.2:0/949341832 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c80095f0 con 0x7f47cc10d8b0 2023-09-13T04:46:20.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.751+0000 7f47d0d9a700 1 --2- 172.21.15.2:0/949341832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47cc10d8b0 0x7f47cc114c00 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00e9a0 tx=0x7f47bc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:20.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.826+0000 7f47c27fc700 1 -- 172.21.15.2:0/949341832 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc00cbd0 con 0x7f47cc10d8b0 2023-09-13T04:46:20.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.826+0000 7f47c27fc700 1 -- 172.21.15.2:0/949341832 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47bc00cd30 con 0x7f47cc10d8b0 2023-09-13T04:46:20.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.826+0000 7f47c27fc700 1 -- 172.21.15.2:0/949341832 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc018710 con 0x7f47cc10d8b0 2023-09-13T04:46:20.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.827+0000 7f47d259d700 1 -- 172.21.15.2:0/949341832 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47cc1ba9e0 con 0x7f47cc10d8b0 2023-09-13T04:46:20.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.829+0000 7f47d259d700 1 -- 172.21.15.2:0/949341832 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47cc1baea0 con 0x7f47cc10d8b0 2023-09-13T04:46:20.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.851+0000 7f47e7162b80 1 Processor -- start 2023-09-13T04:46:20.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.851+0000 7f47e7162b80 1 -- start start 2023-09-13T04:46:20.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.852+0000 7f47e7162b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56416356c6b0 0x56416356ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:20.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.852+0000 7f47e7162b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416356cfd0 0x564163570e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:20.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.852+0000 7f47e7162b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564163571350 0x564163571790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:20.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.852+0000 7f47e7162b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641634c5780 con 0x56416356c6b0 2023-09-13T04:46:20.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.852+0000 7f47e7162b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641634c30a0 con 0x56416356cfd0 2023-09-13T04:46:20.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.852+0000 7f47e7162b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641634be990 con 0x564163571350 2023-09-13T04:46:20.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.866+0000 7f47d0d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564163571350 0x564163571790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:20.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.871+0000 7f47d0d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564163571350 0x564163571790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:33880/0 (socket says 172.21.15.2:33880) 2023-09-13T04:46:20.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.871+0000 7f47d0d9a700 1 -- 172.21.15.2:0/1261798079 learned_addr learned my addr 172.21.15.2:0/1261798079 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:20.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.876+0000 7f47d159b700 1 --2- 172.21.15.2:0/1261798079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416356cfd0 0x564163570e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:20.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.876+0000 7f47d1d9c700 1 --2- 172.21.15.2:0/1261798079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56416356c6b0 0x56416356ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:20.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.877+0000 7f47c27fc700 1 -- 172.21.15.2:0/949341832 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f47bc010430 con 0x7f47cc10d8b0 2023-09-13T04:46:20.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.879+0000 7f47d159b700 1 -- 172.21.15.2:0/1261798079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564163571350 msgr2=0x564163571790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:20.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.879+0000 7f47d159b700 1 --2- 172.21.15.2:0/1261798079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564163571350 0x564163571790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:20.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.879+0000 7f47d159b700 1 -- 172.21.15.2:0/1261798079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56416356c6b0 msgr2=0x56416356ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:20.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.879+0000 7f47d159b700 1 --2- 172.21.15.2:0/1261798079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56416356c6b0 0x56416356ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:20.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.879+0000 7f47d159b700 1 -- 172.21.15.2:0/1261798079 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56416333e630 con 0x56416356cfd0 2023-09-13T04:46:20.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.880+0000 7f47c27fc700 1 --2- 172.21.15.2:0/949341832 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f47b80a0a80 0x7f47b80a2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:20.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.880+0000 7f47c27fc700 1 -- 172.21.15.2:0/949341832 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f47bc014070 con 0x7f47cc10d8b0 2023-09-13T04:46:20.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.885+0000 7f47d159b700 1 --2- 172.21.15.2:0/1261798079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416356cfd0 0x564163570e10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f47c80098e0 tx=0x7f47c8009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c529a7594ec7dd2 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:20.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:20.923+0000 7f47d1d9c700 1 --2- 172.21.15.2:0/949341832 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f47b80a0a80 0x7f47b80a2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:21.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.017+0000 7f47d1d9c700 1 --2- 172.21.15.2:0/949341832 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f47b80a0a80 0x7f47b80a2f40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009b90 tx=0x7f47c4009040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:21.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.114+0000 7f47b77fe700 1 -- 172.21.15.2:0/1261798079 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c801d070 con 0x56416356cfd0 2023-09-13T04:46:21.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.115+0000 7f47b77fe700 1 -- 172.21.15.2:0/1261798079 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47c800bd10 con 0x56416356cfd0 2023-09-13T04:46:21.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.115+0000 7f47b77fe700 1 -- 172.21.15.2:0/1261798079 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c800fb80 con 0x56416356cfd0 2023-09-13T04:46:21.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.116+0000 7f47e7162b80 1 -- 172.21.15.2:0/1261798079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416356cfd0 msgr2=0x564163570e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:21.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.116+0000 7f47e7162b80 1 --2- 172.21.15.2:0/1261798079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416356cfd0 0x564163570e10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f47c80098e0 tx=0x7f47c8009530 comp rx=0 tx=0).stop 2023-09-13T04:46:21.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.117+0000 7f47e7162b80 1 -- 172.21.15.2:0/1261798079 shutdown_connections 2023-09-13T04:46:21.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.117+0000 7f47e7162b80 1 --2- 172.21.15.2:0/1261798079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564163571350 0x564163571790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:21.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.117+0000 7f47e7162b80 1 --2- 172.21.15.2:0/1261798079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56416356c6b0 0x56416356ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:21.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.117+0000 7f47e7162b80 1 --2- 172.21.15.2:0/1261798079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416356cfd0 0x564163570e10 secure :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f47c80098e0 tx=0x7f47c8009530 comp rx=0 tx=0).stop 2023-09-13T04:46:21.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.117+0000 7f47e7162b80 1 -- 172.21.15.2:0/1261798079 >> 172.21.15.2:0/1261798079 conn(0x5641634cac80 msgr2=0x564163567330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:21.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.128+0000 7f47e7162b80 1 -- 172.21.15.2:0/1261798079 shutdown_connections 2023-09-13T04:46:21.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.128+0000 7f47e7162b80 1 -- 172.21.15.2:0/1261798079 wait complete. 2023-09-13T04:46:21.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.129+0000 7f47e7162b80 1 -- start start 2023-09-13T04:46:21.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.129+0000 7f47e7162b80 1 Processor -- start 2023-09-13T04:46:21.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.131+0000 7f47e7162b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56416356c6b0 0x564163576080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:21.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.131+0000 7f47e7162b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564163571350 0x5641635765c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:21.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.131+0000 7f47e7162b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564163576b00 0x5641635824f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:21.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.131+0000 7f47e7162b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56416345cbb0 con 0x564163576b00 2023-09-13T04:46:21.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.131+0000 7f47e7162b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56416345c8b0 con 0x564163571350 2023-09-13T04:46:21.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.131+0000 7f47e7162b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564163340920 con 0x56416356c6b0 2023-09-13T04:46:21.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.131+0000 7f47e7162b80 10 client.0 _ll_register_callbacks cb 0x56416345a680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:46:21.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.132+0000 7f47d0d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564163571350 0x5641635765c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:21.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.132+0000 7f47d0d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564163571350 0x5641635765c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:53140/0 (socket says 172.21.15.2:53140) 2023-09-13T04:46:21.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.132+0000 7f47d0d9a700 1 -- 172.21.15.2:0/965067304 learned_addr learned my addr 172.21.15.2:0/965067304 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:21.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.132+0000 7f47d159b700 1 --2- 172.21.15.2:0/965067304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56416356c6b0 0x564163576080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:21.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.141+0000 7f47d0d9a700 1 -- 172.21.15.2:0/965067304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56416356c6b0 msgr2=0x564163576080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:21.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.141+0000 7f47d0d9a700 1 --2- 172.21.15.2:0/965067304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56416356c6b0 0x564163576080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:21.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.141+0000 7f47d0d9a700 1 -- 172.21.15.2:0/965067304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564163576b00 msgr2=0x5641635824f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:21.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.141+0000 7f47d0d9a700 1 --2- 172.21.15.2:0/965067304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564163576b00 0x5641635824f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:21.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.141+0000 7f47d0d9a700 1 -- 172.21.15.2:0/965067304 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47cc11d010 con 0x564163571350 2023-09-13T04:46:21.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.152+0000 7f47d0d9a700 1 --2- 172.21.15.2:0/965067304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564163571350 0x5641635765c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0a4770 tx=0x7f47bc099080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b78d29804b0d8a1d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:21.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.263+0000 7f47b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:46:21.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.263+0000 7f47b4ff9700 1 -- 172.21.15.2:0/965067304 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc09d330 con 0x564163571350 2023-09-13T04:46:21.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.263+0000 7f47b4ff9700 1 -- 172.21.15.2:0/965067304 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f47bc09cc30 con 0x564163571350 2023-09-13T04:46:21.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.263+0000 7f47e7162b80 1 -- 172.21.15.2:0/965067304 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56416355f670 con 0x564163571350 2023-09-13T04:46:21.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.266+0000 7f47b4ff9700 1 -- 172.21.15.2:0/965067304 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47bc0a0a60 con 0x564163571350 2023-09-13T04:46:21.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.269+0000 7f47e7162b80 10 client.26411 fetch_fsmap learned FSMap version 33 2023-09-13T04:46:21.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.269+0000 7f47e7162b80 1 -- 172.21.15.2:0/965067304 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5641634d67a0 con 0x564163571350 2023-09-13T04:46:21.292 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:20 smithi002 ceph-mon[97651]: pgmap v180: 129 pgs: 129 active+clean; 703 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 257 op/s 2023-09-13T04:46:21.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.291+0000 7f47b4ff9700 1 -- 172.21.15.2:0/965067304 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56416355f670 con 0x564163571350 2023-09-13T04:46:21.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.291+0000 7f47b4ff9700 1 -- 172.21.15.2:0/965067304 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 33) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f47bc09c120 con 0x564163571350 2023-09-13T04:46:21.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.292+0000 7f47e7162b80 10 client.26411 fetch_fsmap finished waiting for FSMap version 33 2023-09-13T04:46:21.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.292+0000 7f47e7162b80 10 client.26411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:46:21.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.292+0000 7f47e7162b80 20 client.26411 populate_metadata read hostname 'smithi002' 2023-09-13T04:46:21.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.292+0000 7f47e7162b80 1 --2- 172.21.15.2:0/965067304 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5641635869e0 0x564163588e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:21.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.292+0000 7f47e7162b80 4 client.26411 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:46:21.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.292+0000 7f47e7162b80 1 -- 172.21.15.2:0/965067304 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641634b7970 con 0x5641635869e0 2023-09-13T04:46:21.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.293+0000 7f47d159b700 1 --2- 172.21.15.2:0/965067304 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5641635869e0 0x564163588e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:21.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.309+0000 7f47d159b700 1 --2- 172.21.15.2:0/965067304 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5641635869e0 0x564163588e00 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6844cf1359dc47b0 server_cookie=bbfd78058bb2501c in_seq=0 out_seq=0 2023-09-13T04:46:21.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:21.310+0000 7f47b4ff9700 10 client.26411 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:46:21.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:20 smithi149 ceph-mon[133315]: pgmap v180: 129 pgs: 129 active+clean; 703 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 257 op/s 2023-09-13T04:46:21.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:20 smithi156 ceph-mon[133675]: pgmap v180: 129 pgs: 129 active+clean; 703 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 257 op/s 2023-09-13T04:46:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.194+0000 7f2c9dffb700 1 -- 172.21.15.2:0/3897006610 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f5348d4ff0 con 0x55f534a376b0 2023-09-13T04:46:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.194+0000 7f2c9dffb700 10 client.26425 handle_command_reply: tid=0 2023-09-13T04:46:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.194+0000 7f2cd04b7b80 1 client.26425 shutdown 2023-09-13T04:46:23.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.198+0000 7f47b4ff9700 1 -- 172.21.15.2:0/965067304 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5641634b7970 con 0x5641635869e0 2023-09-13T04:46:23.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.198+0000 7f47b4ff9700 10 client.26411 handle_command_reply: tid=0 2023-09-13T04:46:23.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.198+0000 7f47e7162b80 1 client.26411 shutdown 2023-09-13T04:46:23.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.199+0000 7f47e7162b80 1 -- 172.21.15.2:0/965067304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564163571350 msgr2=0x5641635765c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:23.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.199+0000 7f47e7162b80 1 --2- 172.21.15.2:0/965067304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564163571350 0x5641635765c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0a4770 tx=0x7f47bc099080 comp rx=0 tx=0).stop 2023-09-13T04:46:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.201+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/3897006610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349c9840 msgr2=0x55f5349d5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.201+0000 7f2cd04b7b80 1 --2- 172.21.15.2:0/3897006610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349c9840 0x55f5349d5f50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8022cb0 tx=0x7f2ca8022ce0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.201+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/3897006610 shutdown_connections 2023-09-13T04:46:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.201+0000 7f2cd04b7b80 1 --2- 172.21.15.2:0/3897006610 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f534a376b0 0x55f534a39ad0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.201+0000 7f2cd04b7b80 1 --2- 172.21.15.2:0/3897006610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5349ce500 0x55f5349d69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.201+0000 7f2cd04b7b80 1 --2- 172.21.15.2:0/3897006610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5349c9840 0x55f5349d5f50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.201+0000 7f2cd04b7b80 1 --2- 172.21.15.2:0/3897006610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5349cdbe0 0x55f5349d6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.201+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/3897006610 >> 172.21.15.2:0/3897006610 conn(0x55f534927c20 msgr2=0x55f5349c8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.201+0000 7f2c9dffb700 0 client.26425 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:46:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.201+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/3897006610 shutdown_connections 2023-09-13T04:46:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.202+0000 7f2cd04b7b80 1 -- 172.21.15.2:0/3897006610 wait complete. 2023-09-13T04:46:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.202+0000 7f2cd04b7b80 20 client.26425 trim_cache size 0 max 16384 2023-09-13T04:46:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:46:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:46:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:46:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:46:23.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.207+0000 7f47e7162b80 1 -- 172.21.15.2:0/965067304 shutdown_connections 2023-09-13T04:46:23.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.207+0000 7f47e7162b80 1 --2- 172.21.15.2:0/965067304 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5641635869e0 0x564163588e00 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.207+0000 7f47e7162b80 1 --2- 172.21.15.2:0/965067304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564163576b00 0x5641635824f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.207+0000 7f47e7162b80 1 --2- 172.21.15.2:0/965067304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56416356c6b0 0x564163576080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.207+0000 7f47e7162b80 1 --2- 172.21.15.2:0/965067304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564163571350 0x5641635765c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.208+0000 7f47e7162b80 1 -- 172.21.15.2:0/965067304 >> 172.21.15.2:0/965067304 conn(0x5641634cac80 msgr2=0x564163567080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:23.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.208+0000 7f47b4ff9700 0 client.26411 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:46:23.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.208+0000 7f47e7162b80 1 -- 172.21.15.2:0/965067304 shutdown_connections 2023-09-13T04:46:23.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.208+0000 7f47e7162b80 1 -- 172.21.15.2:0/965067304 wait complete. 2023-09-13T04:46:23.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.209+0000 7f47e7162b80 20 client.26411 trim_cache size 0 max 16384 2023-09-13T04:46:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.282+0000 7f2cbcbff700 1 -- 172.21.15.2:0/1986935115 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f2ca00a09c0 msgr2=0x7f2ca00a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.282+0000 7f2cbcbff700 1 --2- 172.21.15.2:0/1986935115 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f2ca00a09c0 0x7f2ca00a2e80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8006fd0 tx=0x7f2ca8008040 comp rx=0 tx=0).stop 2023-09-13T04:46:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.282+0000 7f2cbcbff700 1 -- 172.21.15.2:0/1986935115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cb811a490 msgr2=0x7f2cb81116e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.282+0000 7f2cbcbff700 1 --2- 172.21.15.2:0/1986935115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cb811a490 0x7f2cb81116e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0009040 tx=0x7f2cb00078a0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.289+0000 7f2cbcbff700 1 -- 172.21.15.2:0/1986935115 shutdown_connections 2023-09-13T04:46:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.289+0000 7f2cbcbff700 1 --2- 172.21.15.2:0/1986935115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cb811a490 0x7f2cb81116e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.289+0000 7f2cbcbff700 1 --2- 172.21.15.2:0/1986935115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cb810f420 0x7f2cb8118840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.289+0000 7f2cbcbff700 1 --2- 172.21.15.2:0/1986935115 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f2ca00a09c0 0x7f2ca00a2e80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.289+0000 7f2cbcbff700 1 --2- 172.21.15.2:0/1986935115 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2cb810f890 0x7f2cb8118dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.289+0000 7f2cbcbff700 1 -- 172.21.15.2:0/1986935115 >> 172.21.15.2:0/1986935115 conn(0x7f2cb806cda0 msgr2=0x7f2cb806f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.296+0000 7f2cbcbff700 1 -- 172.21.15.2:0/1986935115 shutdown_connections 2023-09-13T04:46:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:23.304+0000 7f2cbcbff700 1 -- 172.21.15.2:0/1986935115 wait complete. 2023-09-13T04:46:23.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.348+0000 7f47e7162b80 1 Processor -- start 2023-09-13T04:46:23.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.367+0000 7f47e7162b80 1 -- start start 2023-09-13T04:46:23.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.368+0000 7f47e7162b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56416356c6b0 0x5641635706f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:23.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.368+0000 7f47e7162b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564163571350 0x564163570c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:23.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.368+0000 7f47e7162b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564163576b00 0x564163566ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:23.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.368+0000 7f47e7162b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564163584160 con 0x56416356c6b0 2023-09-13T04:46:23.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.368+0000 7f47e7162b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564163579220 con 0x564163576b00 2023-09-13T04:46:23.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.368+0000 7f47e7162b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564163568a10 con 0x564163571350 2023-09-13T04:46:23.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.378+0000 7f47d0d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564163571350 0x564163570c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:23.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.378+0000 7f47d1d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564163576b00 0x564163566ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:23.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.378+0000 7f47d0d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564163571350 0x564163570c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:33890/0 (socket says 172.21.15.2:33890) 2023-09-13T04:46:23.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.378+0000 7f47d0d9a700 1 -- 172.21.15.2:0/1337364468 learned_addr learned my addr 172.21.15.2:0/1337364468 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:23.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.379+0000 7f47d0d9a700 1 -- 172.21.15.2:0/1337364468 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564163576b00 msgr2=0x564163566ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:23.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.379+0000 7f47d0d9a700 1 --2- 172.21.15.2:0/1337364468 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564163576b00 0x564163566ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.379+0000 7f47d0d9a700 1 -- 172.21.15.2:0/1337364468 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56416356c6b0 msgr2=0x5641635706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:23.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.379+0000 7f47d0d9a700 1 --2- 172.21.15.2:0/1337364468 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56416356c6b0 0x5641635706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.379+0000 7f47d0d9a700 1 -- 172.21.15.2:0/1337364468 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641632eade0 con 0x564163571350 2023-09-13T04:46:23.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.405+0000 7f47d0d9a700 1 --2- 172.21.15.2:0/1337364468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564163571350 0x564163570c30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f47bc09c480 tx=0x7f47bc0108f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=baeef9ecd2fb5272 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:23.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.416+0000 7f47b7fff700 1 -- 172.21.15.2:0/1337364468 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0bae80 con 0x564163571350 2023-09-13T04:46:23.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.416+0000 7f47b7fff700 1 -- 172.21.15.2:0/1337364468 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47bc09d330 con 0x564163571350 2023-09-13T04:46:23.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.416+0000 7f47b7fff700 1 -- 172.21.15.2:0/1337364468 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0a0af0 con 0x564163571350 2023-09-13T04:46:23.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.423+0000 7f47e7162b80 1 -- 172.21.15.2:0/1337364468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564163571350 msgr2=0x564163570c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:23.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.423+0000 7f47e7162b80 1 --2- 172.21.15.2:0/1337364468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564163571350 0x564163570c30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f47bc09c480 tx=0x7f47bc0108f0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.427+0000 7f47e7162b80 1 -- 172.21.15.2:0/1337364468 shutdown_connections 2023-09-13T04:46:23.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.427+0000 7f47e7162b80 1 --2- 172.21.15.2:0/1337364468 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564163576b00 0x564163566ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.427+0000 7f47e7162b80 1 --2- 172.21.15.2:0/1337364468 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56416356c6b0 0x5641635706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.427+0000 7f47e7162b80 1 --2- 172.21.15.2:0/1337364468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564163571350 0x564163570c30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.427+0000 7f47e7162b80 1 -- 172.21.15.2:0/1337364468 >> 172.21.15.2:0/1337364468 conn(0x5641634cac80 msgr2=0x564163576040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:23.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.427+0000 7f47e7162b80 1 -- 172.21.15.2:0/1337364468 shutdown_connections 2023-09-13T04:46:23.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.427+0000 7f47e7162b80 1 -- 172.21.15.2:0/1337364468 wait complete. 2023-09-13T04:46:23.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.428+0000 7f47e7162b80 1 -- start start 2023-09-13T04:46:23.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.428+0000 7f47e7162b80 1 Processor -- start 2023-09-13T04:46:23.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.429+0000 7f47e7162b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416356c6b0 0x56416356ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:23.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.429+0000 7f47e7162b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564163571350 0x56416356f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:23.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.429+0000 7f47e7162b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564163576b00 0x56416356dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:23.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.429+0000 7f47e7162b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56416356f9e0 con 0x564163571350 2023-09-13T04:46:23.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.429+0000 7f47e7162b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56416356e170 con 0x56416356c6b0 2023-09-13T04:46:23.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.429+0000 7f47e7162b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56416356e330 con 0x564163576b00 2023-09-13T04:46:23.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.429+0000 7f47e7162b80 10 client.0 _ll_register_callbacks cb 0x56416345a680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:46:23.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.430+0000 7f47d159b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416356c6b0 0x56416356ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:23.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.430+0000 7f47d159b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416356c6b0 0x56416356ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:53164/0 (socket says 172.21.15.2:53164) 2023-09-13T04:46:23.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.430+0000 7f47d159b700 1 -- 172.21.15.2:0/3105458823 learned_addr learned my addr 172.21.15.2:0/3105458823 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:23.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.430+0000 7f47d1d9c700 1 --2- 172.21.15.2:0/3105458823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564163576b00 0x56416356dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:23.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.434+0000 7f47d0d9a700 1 --2- 172.21.15.2:0/3105458823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564163571350 0x56416356f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:23.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.439+0000 7f47d1d9c700 1 -- 172.21.15.2:0/3105458823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416356c6b0 msgr2=0x56416356ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:23.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.439+0000 7f47d1d9c700 1 --2- 172.21.15.2:0/3105458823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416356c6b0 0x56416356ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.439+0000 7f47d1d9c700 1 -- 172.21.15.2:0/3105458823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564163571350 msgr2=0x56416356f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:23.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.439+0000 7f47d1d9c700 1 --2- 172.21.15.2:0/3105458823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564163571350 0x56416356f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.439+0000 7f47d1d9c700 1 -- 172.21.15.2:0/3105458823 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641634d67a0 con 0x564163576b00 2023-09-13T04:46:23.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.441+0000 7f47d1d9c700 1 --2- 172.21.15.2:0/3105458823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564163576b00 0x56416356dc30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009340 tx=0x7f47c4016a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=692cbbc14b03bcea server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:23.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.441+0000 7f47b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:46:23.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.443+0000 7f47b4ff9700 1 -- 172.21.15.2:0/3105458823 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c4003c10 con 0x564163576b00 2023-09-13T04:46:23.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.443+0000 7f47b4ff9700 1 -- 172.21.15.2:0/3105458823 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f47c4003d70 con 0x564163576b00 2023-09-13T04:46:23.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.444+0000 7f47e7162b80 1 -- 172.21.15.2:0/3105458823 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47c800bd10 con 0x564163576b00 2023-09-13T04:46:23.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.444+0000 7f47b4ff9700 1 -- 172.21.15.2:0/3105458823 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47c4004050 con 0x564163576b00 2023-09-13T04:46:23.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.444+0000 7f47e7162b80 10 client.26440 fetch_fsmap learned FSMap version 33 2023-09-13T04:46:23.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.444+0000 7f47e7162b80 1 -- 172.21.15.2:0/3105458823 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5641632d3c60 con 0x564163576b00 2023-09-13T04:46:23.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.453+0000 7f47b4ff9700 1 -- 172.21.15.2:0/3105458823 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47c800bd10 con 0x564163576b00 2023-09-13T04:46:23.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.453+0000 7f47b4ff9700 1 -- 172.21.15.2:0/3105458823 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f47c4021070 con 0x564163576b00 2023-09-13T04:46:23.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.455+0000 7f47e7162b80 10 client.26440 fetch_fsmap finished waiting for FSMap version 33 2023-09-13T04:46:23.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.455+0000 7f47e7162b80 10 client.26440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:46:23.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.455+0000 7f47e7162b80 20 client.26440 populate_metadata read hostname 'smithi002' 2023-09-13T04:46:23.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.455+0000 7f47e7162b80 1 --2- 172.21.15.2:0/3105458823 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5641635db4c0 0x5641635dd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:23.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.455+0000 7f47e7162b80 4 client.26440 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-09-13T04:46:23.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.455+0000 7f47e7162b80 1 -- 172.21.15.2:0/3105458823 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564163478050 con 0x5641635db4c0 2023-09-13T04:46:23.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.455+0000 7f47d159b700 1 --2- 172.21.15.2:0/3105458823 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5641635db4c0 0x5641635dd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:23.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.473+0000 7f47d159b700 1 --2- 172.21.15.2:0/3105458823 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5641635db4c0 0x5641635dd8e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ef47ab66d7833d7 server_cookie=80afd3672c032051 in_seq=0 out_seq=0 2023-09-13T04:46:23.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.483+0000 7f47b4ff9700 10 client.26440 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:46:23.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.512+0000 7f47b4ff9700 1 -- 172.21.15.2:0/3105458823 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x564163478050 con 0x5641635db4c0 2023-09-13T04:46:23.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.512+0000 7f47b4ff9700 10 client.26440 handle_command_reply: tid=0 2023-09-13T04:46:23.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.512+0000 7f47e7162b80 1 client.26440 shutdown 2023-09-13T04:46:23.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.521+0000 7f47e7162b80 1 -- 172.21.15.2:0/3105458823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564163576b00 msgr2=0x56416356dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:23.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.521+0000 7f47e7162b80 1 --2- 172.21.15.2:0/3105458823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564163576b00 0x56416356dc30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009340 tx=0x7f47c4016a70 comp rx=0 tx=0).stop 2023-09-13T04:46:23.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.526+0000 7f47e7162b80 1 -- 172.21.15.2:0/3105458823 shutdown_connections 2023-09-13T04:46:23.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.526+0000 7f47e7162b80 1 --2- 172.21.15.2:0/3105458823 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5641635db4c0 0x5641635dd8e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.526+0000 7f47e7162b80 1 --2- 172.21.15.2:0/3105458823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564163576b00 0x56416356dc30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.526+0000 7f47e7162b80 1 --2- 172.21.15.2:0/3105458823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416356c6b0 0x56416356ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.526+0000 7f47e7162b80 1 --2- 172.21.15.2:0/3105458823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564163571350 0x56416356f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.526+0000 7f47e7162b80 1 -- 172.21.15.2:0/3105458823 >> 172.21.15.2:0/3105458823 conn(0x5641634cac80 msgr2=0x564163576020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:23.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.526+0000 7f47b4ff9700 0 client.26440 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:46:23.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.563+0000 7f47e7162b80 1 -- 172.21.15.2:0/3105458823 shutdown_connections 2023-09-13T04:46:23.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.570+0000 7f47e7162b80 1 -- 172.21.15.2:0/3105458823 wait complete. 2023-09-13T04:46:23.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.571+0000 7f47e7162b80 20 client.26440 trim_cache size 0 max 16384 2023-09-13T04:46:23.573 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:46:23.574 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "scrub active (3 inodes in the stack)", 2023-09-13T04:46:23.574 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-09-13T04:46:23.574 INFO:teuthology.orchestra.run.smithi002.stdout: "740235c0-6e1b-4729-b1fe-33345ae948c1": { 2023-09-13T04:46:23.574 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-09-13T04:46:23.574 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "740235c0-6e1b-4729-b1fe-33345ae948c1", 2023-09-13T04:46:23.575 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-09-13T04:46:23.575 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:46:23.575 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:46:23.575 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:46:23.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.683+0000 7f47d259d700 1 -- 172.21.15.2:0/949341832 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f47b80a0a80 msgr2=0x7f47b80a2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:23.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.683+0000 7f47d259d700 1 --2- 172.21.15.2:0/949341832 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f47b80a0a80 0x7f47b80a2f40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009b90 tx=0x7f47c4009040 comp rx=0 tx=0).stop 2023-09-13T04:46:23.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.683+0000 7f47d259d700 1 -- 172.21.15.2:0/949341832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47cc10d8b0 msgr2=0x7f47cc114c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:23.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.683+0000 7f47d259d700 1 --2- 172.21.15.2:0/949341832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47cc10d8b0 0x7f47cc114c00 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00e9a0 tx=0x7f47bc00ed60 comp rx=0 tx=0).stop 2023-09-13T04:46:23.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.683+0000 7f47d259d700 1 -- 172.21.15.2:0/949341832 shutdown_connections 2023-09-13T04:46:23.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.683+0000 7f47d259d700 1 --2- 172.21.15.2:0/949341832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47cc10f230 0x7f47cc1b9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.683+0000 7f47d259d700 1 --2- 172.21.15.2:0/949341832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47cc10cd90 0x7f47cc1146c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.683+0000 7f47d259d700 1 --2- 172.21.15.2:0/949341832 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f47b80a0a80 0x7f47b80a2f40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.683+0000 7f47d259d700 1 --2- 172.21.15.2:0/949341832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47cc10d8b0 0x7f47cc114c00 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:23.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.683+0000 7f47d259d700 1 -- 172.21.15.2:0/949341832 >> 172.21.15.2:0/949341832 conn(0x7f47cc06cec0 msgr2=0x7f47cc0703b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:23.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.684+0000 7f47d259d700 1 -- 172.21.15.2:0/949341832 shutdown_connections 2023-09-13T04:46:23.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:23.684+0000 7f47d259d700 1 -- 172.21.15.2:0/949341832 wait complete. 2023-09-13T04:46:23.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:23 smithi156 ceph-mon[133675]: pgmap v181: 129 pgs: 129 active+clean; 703 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 185 op/s 2023-09-13T04:46:23.761 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:23 smithi002 ceph-mon[97651]: pgmap v181: 129 pgs: 129 active+clean; 703 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 185 op/s 2023-09-13T04:46:23.879 INFO:tasks.cephfs.filesystem:scrub status for tag:740235c0-6e1b-4729-b1fe-33345ae948c1 - {'path': '/', 'tag': '740235c0-6e1b-4729-b1fe-33345ae948c1', 'options': 'recursive,force'} 2023-09-13T04:46:23.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:23 smithi149 ceph-mon[133315]: pgmap v181: 129 pgs: 129 active+clean; 703 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 185 op/s 2023-09-13T04:46:26.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:25 smithi149 ceph-mon[133315]: pgmap v182: 129 pgs: 129 active+clean; 710 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 523 op/s 2023-09-13T04:46:26.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:25 smithi156 ceph-mon[133675]: pgmap v182: 129 pgs: 129 active+clean; 710 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 523 op/s 2023-09-13T04:46:26.268 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:25 smithi002 ceph-mon[97651]: pgmap v182: 129 pgs: 129 active+clean; 710 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 523 op/s 2023-09-13T04:46:28.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:27 smithi149 ceph-mon[133315]: pgmap v183: 129 pgs: 129 active+clean; 711 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 529 op/s 2023-09-13T04:46:28.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:27 smithi156 ceph-mon[133675]: pgmap v183: 129 pgs: 129 active+clean; 711 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 529 op/s 2023-09-13T04:46:28.263 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:27 smithi002 ceph-mon[97651]: pgmap v183: 129 pgs: 129 active+clean; 711 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 529 op/s 2023-09-13T04:46:30.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:29 smithi149 ceph-mon[133315]: pgmap v184: 129 pgs: 129 active+clean; 711 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 533 op/s 2023-09-13T04:46:30.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:29 smithi156 ceph-mon[133675]: pgmap v184: 129 pgs: 129 active+clean; 711 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 533 op/s 2023-09-13T04:46:30.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:29 smithi002 ceph-mon[97651]: pgmap v184: 129 pgs: 129 active+clean; 711 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 533 op/s 2023-09-13T04:46:30.670 INFO:tasks.workunit.client.0.smithi002.stdout: 150 691933 479 506593 398 279860 760 2023-09-13T04:46:32.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:31 smithi149 ceph-mon[133315]: pgmap v185: 129 pgs: 129 active+clean; 722 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 574 op/s 2023-09-13T04:46:32.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:31 smithi156 ceph-mon[133675]: pgmap v185: 129 pgs: 129 active+clean; 722 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 574 op/s 2023-09-13T04:46:32.264 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:31 smithi002 ceph-mon[97651]: pgmap v185: 129 pgs: 129 active+clean; 722 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 574 op/s 2023-09-13T04:46:34.276 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:33 smithi002 ceph-mon[97651]: pgmap v186: 129 pgs: 129 active+clean; 722 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 450 op/s 2023-09-13T04:46:34.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:33 smithi149 ceph-mon[133315]: pgmap v186: 129 pgs: 129 active+clean; 722 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 450 op/s 2023-09-13T04:46:34.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:33 smithi156 ceph-mon[133675]: pgmap v186: 129 pgs: 129 active+clean; 722 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 450 op/s 2023-09-13T04:46:36.016 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:35 smithi002 ceph-mon[97651]: pgmap v187: 129 pgs: 129 active+clean; 729 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 513 op/s 2023-09-13T04:46:36.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:35 smithi149 ceph-mon[133315]: pgmap v187: 129 pgs: 129 active+clean; 729 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 513 op/s 2023-09-13T04:46:36.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:35 smithi156 ceph-mon[133675]: pgmap v187: 129 pgs: 129 active+clean; 729 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 513 op/s 2023-09-13T04:46:38.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:37 smithi156 ceph-mon[133675]: pgmap v188: 129 pgs: 129 active+clean; 729 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 170 op/s 2023-09-13T04:46:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.248+0000 7f69fd1ff700 1 -- 172.21.15.2:0/2266236846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69f810f420 msgr2=0x7f69f8107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.248+0000 7f69fd1ff700 1 --2- 172.21.15.2:0/2266236846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69f810f420 0x7f69f8107cb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f69f000b0a0 tx=0x7f69f000b3b0 comp rx=0 tx=0).stop 2023-09-13T04:46:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.252+0000 7f69fd1ff700 1 -- 172.21.15.2:0/2266236846 shutdown_connections 2023-09-13T04:46:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.252+0000 7f69fd1ff700 1 --2- 172.21.15.2:0/2266236846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69f811a490 0x7f69f811c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.252+0000 7f69fd1ff700 1 --2- 172.21.15.2:0/2266236846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69f810f420 0x7f69f8107cb0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:38.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.252+0000 7f69fd1ff700 1 --2- 172.21.15.2:0/2266236846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69f810f890 0x7f69f81081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:38.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.252+0000 7f69fd1ff700 1 -- 172.21.15.2:0/2266236846 >> 172.21.15.2:0/2266236846 conn(0x7f69f806cda0 msgr2=0x7f69f806d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:38.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.252+0000 7f69fd1ff700 1 -- 172.21.15.2:0/2266236846 shutdown_connections 2023-09-13T04:46:38.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.252+0000 7f69fd1ff700 1 -- 172.21.15.2:0/2266236846 wait complete. 2023-09-13T04:46:38.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.255+0000 7f69fd1ff700 1 Processor -- start 2023-09-13T04:46:38.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:37 smithi002 ceph-mon[97651]: pgmap v188: 129 pgs: 129 active+clean; 729 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 170 op/s 2023-09-13T04:46:38.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.255+0000 7f69fd1ff700 1 -- start start 2023-09-13T04:46:38.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.256+0000 7f69fd1ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69f810f420 0x7f69f81188d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:38.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.256+0000 7f69fd1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69f810f890 0x7f69f8118e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:38.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.256+0000 7f69fd1ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69f811a490 0x7f69f8111840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:38.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.256+0000 7f69fd1ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f8111fd0 con 0x7f69f811a490 2023-09-13T04:46:38.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.256+0000 7f69fd1ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f8112140 con 0x7f69f810f420 2023-09-13T04:46:38.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.256+0000 7f69fd1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f8112300 con 0x7f69f810f890 2023-09-13T04:46:38.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.259+0000 7f69f759e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69f811a490 0x7f69f8111840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:38.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.259+0000 7f69f759e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69f811a490 0x7f69f8111840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47516/0 (socket says 172.21.15.2:47516) 2023-09-13T04:46:38.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.259+0000 7f69f759e700 1 -- 172.21.15.2:0/3609652131 learned_addr learned my addr 172.21.15.2:0/3609652131 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.266+0000 7f69f6d9d700 1 --2- 172.21.15.2:0/3609652131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69f810f420 0x7f69f81188d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.266+0000 7f69f659c700 1 --2- 172.21.15.2:0/3609652131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69f810f890 0x7f69f8118e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.272+0000 7f69f6d9d700 1 -- 172.21.15.2:0/3609652131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69f810f890 msgr2=0x7f69f8118e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.272+0000 7f69f6d9d700 1 --2- 172.21.15.2:0/3609652131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69f810f890 0x7f69f8118e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.272+0000 7f69f6d9d700 1 -- 172.21.15.2:0/3609652131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69f811a490 msgr2=0x7f69f8111840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.272+0000 7f69f6d9d700 1 --2- 172.21.15.2:0/3609652131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69f811a490 0x7f69f8111840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.272+0000 7f69f6d9d700 1 -- 172.21.15.2:0/3609652131 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f0009d00 con 0x7f69f810f420 2023-09-13T04:46:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.277+0000 7f69f6d9d700 1 --2- 172.21.15.2:0/3609652131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69f810f420 0x7f69f81188d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f69f0006070 tx=0x7f69f0007b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.278+0000 7f69dffff700 1 -- 172.21.15.2:0/3609652131 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f000e040 con 0x7f69f810f420 2023-09-13T04:46:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.278+0000 7f69dffff700 1 -- 172.21.15.2:0/3609652131 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69f001dd50 con 0x7f69f810f420 2023-09-13T04:46:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.278+0000 7f69dffff700 1 -- 172.21.15.2:0/3609652131 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f001b450 con 0x7f69f810f420 2023-09-13T04:46:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.278+0000 7f69fd1ff700 1 -- 172.21.15.2:0/3609652131 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f8116460 con 0x7f69f810f420 2023-09-13T04:46:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.278+0000 7f69fd1ff700 1 -- 172.21.15.2:0/3609652131 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f8116920 con 0x7f69f810f420 2023-09-13T04:46:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.305+0000 7f69dffff700 1 -- 172.21.15.2:0/3609652131 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f69f001d890 con 0x7f69f810f420 2023-09-13T04:46:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.308+0000 7f69dffff700 1 --2- 172.21.15.2:0/3609652131 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f69e00a09c0 0x7f69e00a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.308+0000 7f69dffff700 1 -- 172.21.15.2:0/3609652131 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f69f00cdd60 con 0x7f69f810f420 2023-09-13T04:46:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.310+0000 7f6a10a4cb80 1 Processor -- start 2023-09-13T04:46:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.313+0000 7f69f659c700 1 --2- 172.21.15.2:0/3609652131 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f69e00a09c0 0x7f69e00a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.314+0000 7f6a10a4cb80 1 -- start start 2023-09-13T04:46:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.314+0000 7f6a10a4cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3e23aabe0 0x55c3e23aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.314+0000 7f6a10a4cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3e23ab500 0x55c3e23a6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.314+0000 7f6a10a4cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23a6840 0x55c3e23a6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.314+0000 7f6a10a4cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3e22ff720 con 0x55c3e23aabe0 2023-09-13T04:46:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.314+0000 7f6a10a4cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3e22fd040 con 0x55c3e23a6840 2023-09-13T04:46:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.314+0000 7f6a10a4cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3e22f8930 con 0x55c3e23ab500 2023-09-13T04:46:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.315+0000 7f69f759e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23a6840 0x55c3e23a6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.315+0000 7f69f759e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23a6840 0x55c3e23a6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:50002/0 (socket says 172.21.15.2:50002) 2023-09-13T04:46:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.315+0000 7f69f759e700 1 -- 172.21.15.2:0/2268560454 learned_addr learned my addr 172.21.15.2:0/2268560454 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.315+0000 7f69f759e700 1 -- 172.21.15.2:0/2268560454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3e23ab500 msgr2=0x55c3e23a6300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:46:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.315+0000 7f69f759e700 1 --2- 172.21.15.2:0/2268560454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3e23ab500 0x55c3e23a6300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.315+0000 7f69f759e700 1 -- 172.21.15.2:0/2268560454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3e23aabe0 msgr2=0x55c3e23aafc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:46:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.315+0000 7f69f759e700 1 --2- 172.21.15.2:0/2268560454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3e23aabe0 0x55c3e23aafc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.315+0000 7f69f759e700 1 -- 172.21.15.2:0/2268560454 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3e21785d0 con 0x55c3e23a6840 2023-09-13T04:46:38.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:37 smithi149 ceph-mon[133315]: pgmap v188: 129 pgs: 129 active+clean; 729 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 170 op/s 2023-09-13T04:46:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.327+0000 7f69f759e700 1 --2- 172.21.15.2:0/2268560454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23a6840 0x55c3e23a6c80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f69e800ace0 tx=0x7f69e800f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5cc5133daa080d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.367+0000 7f69dd7fa700 1 -- 172.21.15.2:0/2268560454 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e800a7e0 con 0x55c3e23a6840 2023-09-13T04:46:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.367+0000 7f69dd7fa700 1 -- 172.21.15.2:0/2268560454 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69e80044e0 con 0x55c3e23a6840 2023-09-13T04:46:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.367+0000 7f69dd7fa700 1 -- 172.21.15.2:0/2268560454 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e80186a0 con 0x55c3e23a6840 2023-09-13T04:46:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.368+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/2268560454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23a6840 msgr2=0x55c3e23a6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.368+0000 7f6a10a4cb80 1 --2- 172.21.15.2:0/2268560454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23a6840 0x55c3e23a6c80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f69e800ace0 tx=0x7f69e800f3c0 comp rx=0 tx=0).stop 2023-09-13T04:46:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.604+0000 7f69f659c700 1 --2- 172.21.15.2:0/3609652131 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f69e00a09c0 0x7f69e00a2e80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55c3e23ab900 tx=0x7f69ec009190 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.624+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/2268560454 shutdown_connections 2023-09-13T04:46:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.624+0000 7f6a10a4cb80 1 --2- 172.21.15.2:0/2268560454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23a6840 0x55c3e23a6c80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.624+0000 7f6a10a4cb80 1 --2- 172.21.15.2:0/2268560454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3e23aabe0 0x55c3e23aafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.624+0000 7f6a10a4cb80 1 --2- 172.21.15.2:0/2268560454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3e23ab500 0x55c3e23a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.624+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/2268560454 >> 172.21.15.2:0/2268560454 conn(0x55c3e2304c20 msgr2=0x55c3e23a0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.630+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/2268560454 shutdown_connections 2023-09-13T04:46:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.631+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/2268560454 wait complete. 2023-09-13T04:46:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.632+0000 7f6a10a4cb80 1 -- start start 2023-09-13T04:46:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.632+0000 7f6a10a4cb80 1 Processor -- start 2023-09-13T04:46:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.635+0000 7f6a10a4cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3e23a6840 0x55c3e23af920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.635+0000 7f6a10a4cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3e23aabe0 0x55c3e23afe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.635+0000 7f6a10a4cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23ab500 0x55c3e23b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.635+0000 7f6a10a4cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3e2296b50 con 0x55c3e23aabe0 2023-09-13T04:46:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.635+0000 7f6a10a4cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3e2296850 con 0x55c3e23ab500 2023-09-13T04:46:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.635+0000 7f6a10a4cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3e217a8c0 con 0x55c3e23a6840 2023-09-13T04:46:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.636+0000 7f6a10a4cb80 10 client.0 _ll_register_callbacks cb 0x55c3e2294620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:46:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.637+0000 7f69f759e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23ab500 0x55c3e23b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.637+0000 7f69f759e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23ab500 0x55c3e23b03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:50012/0 (socket says 172.21.15.2:50012) 2023-09-13T04:46:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.637+0000 7f69f759e700 1 -- 172.21.15.2:0/493811558 learned_addr learned my addr 172.21.15.2:0/493811558 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.637+0000 7f69f6d9d700 1 --2- 172.21.15.2:0/493811558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3e23a6840 0x55c3e23af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.644+0000 7f69f659c700 1 --2- 172.21.15.2:0/493811558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3e23aabe0 0x55c3e23afe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:38.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.648+0000 7f69f6d9d700 1 -- 172.21.15.2:0/493811558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23ab500 msgr2=0x55c3e23b03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.648+0000 7f69f6d9d700 1 --2- 172.21.15.2:0/493811558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23ab500 0x55c3e23b03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.649+0000 7f69f6d9d700 1 -- 172.21.15.2:0/493811558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3e23aabe0 msgr2=0x55c3e23afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.649+0000 7f69f6d9d700 1 --2- 172.21.15.2:0/493811558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3e23aabe0 0x55c3e23afe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.649+0000 7f69f6d9d700 1 -- 172.21.15.2:0/493811558 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e800a940 con 0x55c3e23a6840 2023-09-13T04:46:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.649+0000 7f69f6d9d700 1 --2- 172.21.15.2:0/493811558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3e23a6840 0x55c3e23af920 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f69f00cb140 tx=0x7f69f009ad80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=915472b2b91cdfb6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.649+0000 7f69daff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:46:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.649+0000 7f69daff5700 1 -- 172.21.15.2:0/493811558 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f009b390 con 0x55c3e23a6840 2023-09-13T04:46:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.650+0000 7f69daff5700 1 -- 172.21.15.2:0/493811558 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f69f009b630 con 0x55c3e23a6840 2023-09-13T04:46:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.651+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/493811558 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3e236bef0 con 0x55c3e23a6840 2023-09-13T04:46:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.651+0000 7f69daff5700 1 -- 172.21.15.2:0/493811558 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69f009af30 con 0x55c3e23a6840 2023-09-13T04:46:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.651+0000 7f6a10a4cb80 10 client.26446 fetch_fsmap learned FSMap version 33 2023-09-13T04:46:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.651+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/493811558 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55c3e2310740 con 0x55c3e23a6840 2023-09-13T04:46:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.652+0000 7f69daff5700 1 -- 172.21.15.2:0/493811558 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69f00b7070 con 0x55c3e23a6840 2023-09-13T04:46:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.652+0000 7f69daff5700 1 -- 172.21.15.2:0/493811558 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f69f009a420 con 0x55c3e23a6840 2023-09-13T04:46:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.653+0000 7f6a10a4cb80 10 client.26446 fetch_fsmap finished waiting for FSMap version 33 2023-09-13T04:46:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.654+0000 7f6a10a4cb80 10 client.26446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:46:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.654+0000 7f6a10a4cb80 20 client.26446 populate_metadata read hostname 'smithi002' 2023-09-13T04:46:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.654+0000 7f6a10a4cb80 1 --2- 172.21.15.2:0/493811558 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55c3e23bfdf0 0x55c3e23c2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.654+0000 7f6a10a4cb80 4 client.26446 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:46:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.654+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/493811558 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3e22f1910 con 0x55c3e23bfdf0 2023-09-13T04:46:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.654+0000 7f69f659c700 1 --2- 172.21.15.2:0/493811558 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55c3e23bfdf0 0x55c3e23c2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.677+0000 7f69f659c700 1 --2- 172.21.15.2:0/493811558 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55c3e23bfdf0 0x55c3e23c2210 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5df96f7fce5536f server_cookie=e973caf6bc58905a in_seq=0 out_seq=0 2023-09-13T04:46:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.677+0000 7f69daff5700 10 client.26446 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:46:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.682+0000 7f69daff5700 1 -- 172.21.15.2:0/493811558 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c3e22f1910 con 0x55c3e23bfdf0 2023-09-13T04:46:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.682+0000 7f69daff5700 10 client.26446 handle_command_reply: tid=0 2023-09-13T04:46:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.682+0000 7f6a10a4cb80 1 client.26446 shutdown 2023-09-13T04:46:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.803+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/493811558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3e23a6840 msgr2=0x55c3e23af920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.803+0000 7f6a10a4cb80 1 --2- 172.21.15.2:0/493811558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3e23a6840 0x55c3e23af920 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f69f00cb140 tx=0x7f69f009ad80 comp rx=0 tx=0).stop 2023-09-13T04:46:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.804+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/493811558 shutdown_connections 2023-09-13T04:46:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.804+0000 7f6a10a4cb80 1 --2- 172.21.15.2:0/493811558 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55c3e23bfdf0 0x55c3e23c2210 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.804+0000 7f6a10a4cb80 1 --2- 172.21.15.2:0/493811558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23ab500 0x55c3e23b03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.804+0000 7f6a10a4cb80 1 --2- 172.21.15.2:0/493811558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3e23a6840 0x55c3e23af920 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.804+0000 7f6a10a4cb80 1 --2- 172.21.15.2:0/493811558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3e23aabe0 0x55c3e23afe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.804+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/493811558 >> 172.21.15.2:0/493811558 conn(0x55c3e2304c20 msgr2=0x55c3e23ac330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.804+0000 7f69daff5700 0 client.26446 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:46:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.805+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/493811558 shutdown_connections 2023-09-13T04:46:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.805+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/493811558 wait complete. 2023-09-13T04:46:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:38.806+0000 7f6a10a4cb80 20 client.26446 trim_cache size 0 max 16384 2023-09-13T04:46:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.190+0000 7f6a10a4cb80 1 Processor -- start 2023-09-13T04:46:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.196+0000 7f6a10a4cb80 1 -- start start 2023-09-13T04:46:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.197+0000 7f6a10a4cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23a6840 0x55c3e23b6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.197+0000 7f6a10a4cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3e23aabe0 0x55c3e239b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.197+0000 7f6a10a4cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3e23ab500 0x55c3e23b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.197+0000 7f6a10a4cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3e23a1eb0 con 0x55c3e23aabe0 2023-09-13T04:46:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.197+0000 7f6a10a4cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3e23a28f0 con 0x55c3e23a6840 2023-09-13T04:46:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.197+0000 7f6a10a4cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3e23b7550 con 0x55c3e23ab500 2023-09-13T04:46:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.198+0000 7f69f6d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23a6840 0x55c3e23b6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.198+0000 7f69f759e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3e23ab500 0x55c3e23b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.198+0000 7f69f759e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3e23ab500 0x55c3e23b00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:57836/0 (socket says 172.21.15.2:57836) 2023-09-13T04:46:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.198+0000 7f69f759e700 1 -- 172.21.15.2:0/1782772467 learned_addr learned my addr 172.21.15.2:0/1782772467 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.200+0000 7f69f659c700 1 --2- 172.21.15.2:0/1782772467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3e23aabe0 0x55c3e239b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.207+0000 7f69f6d9d700 1 -- 172.21.15.2:0/1782772467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3e23ab500 msgr2=0x55c3e23b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.207+0000 7f69f6d9d700 1 --2- 172.21.15.2:0/1782772467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3e23ab500 0x55c3e23b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.207+0000 7f69f6d9d700 1 -- 172.21.15.2:0/1782772467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3e23aabe0 msgr2=0x55c3e239b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.207+0000 7f69f6d9d700 1 --2- 172.21.15.2:0/1782772467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3e23aabe0 0x55c3e239b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.207+0000 7f69f6d9d700 1 -- 172.21.15.2:0/1782772467 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3e23aa840 con 0x55c3e23a6840 2023-09-13T04:46:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.208+0000 7f69f6d9d700 1 --2- 172.21.15.2:0/1782772467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23a6840 0x55c3e23b6fe0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f69f009a7a0 tx=0x7f69f00994e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bbedba34f909649 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:39.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.278+0000 7f69ddffb700 1 -- 172.21.15.2:0/1782772467 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f009b630 con 0x55c3e23a6840 2023-09-13T04:46:39.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.278+0000 7f69ddffb700 1 -- 172.21.15.2:0/1782772467 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69f00b7050 con 0x55c3e23a6840 2023-09-13T04:46:39.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.278+0000 7f69ddffb700 1 -- 172.21.15.2:0/1782772467 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f009edf0 con 0x55c3e23a6840 2023-09-13T04:46:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.289+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/1782772467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23a6840 msgr2=0x55c3e23b6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.289+0000 7f6a10a4cb80 1 --2- 172.21.15.2:0/1782772467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23a6840 0x55c3e23b6fe0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f69f009a7a0 tx=0x7f69f00994e0 comp rx=0 tx=0).stop 2023-09-13T04:46:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.299+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/1782772467 shutdown_connections 2023-09-13T04:46:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.299+0000 7f6a10a4cb80 1 --2- 172.21.15.2:0/1782772467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3e23ab500 0x55c3e23b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.299+0000 7f6a10a4cb80 1 --2- 172.21.15.2:0/1782772467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23a6840 0x55c3e23b6fe0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.299+0000 7f6a10a4cb80 1 --2- 172.21.15.2:0/1782772467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3e23aabe0 0x55c3e239b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.299+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/1782772467 >> 172.21.15.2:0/1782772467 conn(0x55c3e2304c20 msgr2=0x55c3e23bb730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.299+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/1782772467 shutdown_connections 2023-09-13T04:46:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.299+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/1782772467 wait complete. 2023-09-13T04:46:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.300+0000 7f6a10a4cb80 1 -- start start 2023-09-13T04:46:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.300+0000 7f6a10a4cb80 1 Processor -- start 2023-09-13T04:46:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.301+0000 7f6a10a4cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3e23a6840 0x55c3e2407b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.301+0000 7f6a10a4cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3e23aabe0 0x55c3e24080b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.301+0000 7f6a10a4cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23ab500 0x55c3e24085f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.301+0000 7f6a10a4cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3e2412bf0 con 0x55c3e23a6840 2023-09-13T04:46:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.301+0000 7f6a10a4cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3e2412d30 con 0x55c3e23ab500 2023-09-13T04:46:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.301+0000 7f6a10a4cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3e2412ef0 con 0x55c3e23aabe0 2023-09-13T04:46:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.301+0000 7f6a10a4cb80 10 client.0 _ll_register_callbacks cb 0x55c3e2294620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:46:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.303+0000 7f69f6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3e23a6840 0x55c3e2407b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:39.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.303+0000 7f69f759e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23ab500 0x55c3e24085f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:39.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.303+0000 7f69f759e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23ab500 0x55c3e24085f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:50030/0 (socket says 172.21.15.2:50030) 2023-09-13T04:46:39.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.303+0000 7f69f759e700 1 -- 172.21.15.2:0/1609527680 learned_addr learned my addr 172.21.15.2:0/1609527680 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.303+0000 7f69f659c700 1 --2- 172.21.15.2:0/1609527680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3e23aabe0 0x55c3e24080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.303+0000 7f69f659c700 1 -- 172.21.15.2:0/1609527680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23ab500 msgr2=0x55c3e24085f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.303+0000 7f69f659c700 1 --2- 172.21.15.2:0/1609527680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23ab500 0x55c3e24085f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.303+0000 7f69f659c700 1 -- 172.21.15.2:0/1609527680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3e23a6840 msgr2=0x55c3e2407b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.303+0000 7f69f659c700 1 --2- 172.21.15.2:0/1609527680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3e23a6840 0x55c3e2407b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.303+0000 7f69f659c700 1 -- 172.21.15.2:0/1609527680 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3e2310740 con 0x55c3e23aabe0 2023-09-13T04:46:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.304+0000 7f69f659c700 1 --2- 172.21.15.2:0/1609527680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3e23aabe0 0x55c3e24080b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55c3e23b0fd0 tx=0x7f69ec004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1afc8b0588245d6c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.434+0000 7f69daff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:46:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.434+0000 7f69daff5700 1 -- 172.21.15.2:0/1609527680 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec02a070 con 0x55c3e23aabe0 2023-09-13T04:46:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.434+0000 7f69daff5700 1 -- 172.21.15.2:0/1609527680 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f69ec0253a0 con 0x55c3e23aabe0 2023-09-13T04:46:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.434+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/1609527680 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f69e80044e0 con 0x55c3e23aabe0 2023-09-13T04:46:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.441+0000 7f69daff5700 1 -- 172.21.15.2:0/1609527680 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69ec025560 con 0x55c3e23aabe0 2023-09-13T04:46:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.441+0000 7f6a10a4cb80 10 client.26452 fetch_fsmap learned FSMap version 34 2023-09-13T04:46:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.441+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/1609527680 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c3e2124d80 con 0x55c3e23aabe0 2023-09-13T04:46:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.446+0000 7f69daff5700 1 -- 172.21.15.2:0/1609527680 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69e80044e0 con 0x55c3e23aabe0 2023-09-13T04:46:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.446+0000 7f69daff5700 1 -- 172.21.15.2:0/1609527680 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 34) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f69ec037070 con 0x55c3e23aabe0 2023-09-13T04:46:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.446+0000 7f6a10a4cb80 10 client.26452 fetch_fsmap finished waiting for FSMap version 34 2023-09-13T04:46:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.446+0000 7f6a10a4cb80 10 client.26452 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:46:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.446+0000 7f6a10a4cb80 20 client.26452 populate_metadata read hostname 'smithi002' 2023-09-13T04:46:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.446+0000 7f6a10a4cb80 1 --2- 172.21.15.2:0/1609527680 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55c3e24146c0 0x55c3e2416ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.446+0000 7f6a10a4cb80 4 client.26452 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:46:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.446+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/1609527680 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c3e22b1ff0 con 0x55c3e24146c0 2023-09-13T04:46:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.448+0000 7f69f6d9d700 1 --2- 172.21.15.2:0/1609527680 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55c3e24146c0 0x55c3e2416ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.532+0000 7f69f6d9d700 1 --2- 172.21.15.2:0/1609527680 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55c3e24146c0 0x55c3e2416ae0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73a448ac7eaf063a server_cookie=7a1236a21b3ab7ef in_seq=0 out_seq=0 2023-09-13T04:46:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:39.536+0000 7f69daff5700 10 client.26452 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:46:40.263 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:39 smithi002 ceph-mon[97651]: pgmap v189: 129 pgs: 129 active+clean; 730 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 138 op/s 2023-09-13T04:46:40.264 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:39 smithi002 ceph-mon[97651]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:46:40.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:39 smithi149 ceph-mon[133315]: pgmap v189: 129 pgs: 129 active+clean; 730 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 138 op/s 2023-09-13T04:46:40.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:39 smithi149 ceph-mon[133315]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:46:40.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:39 smithi156 ceph-mon[133675]: pgmap v189: 129 pgs: 129 active+clean; 730 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 138 op/s 2023-09-13T04:46:40.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:39 smithi156 ceph-mon[133675]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:46:40.703 INFO:tasks.workunit.client.0.smithi002.stdout: 154 674846 320 485846 380 269647 679 2023-09-13T04:46:41.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:41 smithi156 ceph-mon[133675]: pgmap v190: 129 pgs: 129 active+clean; 761 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 275 op/s 2023-09-13T04:46:41.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:41 smithi002 ceph-mon[97651]: pgmap v190: 129 pgs: 129 active+clean; 761 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 275 op/s 2023-09-13T04:46:41.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:41 smithi149 ceph-mon[133315]: pgmap v190: 129 pgs: 129 active+clean; 761 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 275 op/s 2023-09-13T04:46:42.050 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:41 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:41.744+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cc2 [6,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-22.xml auth{0=1} v1280 ap=1 s=4932 n(v0 rc2023-09-13T04:43:53.782740+0000 b4932 1=1+0) (ifile excl) (iversion lock) cr={25348=0-4194304@4} caps={25348=pAsLsXsFsxcrwb/pFscr@1},l=25348 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648bf1b3080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000cc2:[<0x20000000748/comment-22.xml v1219>,<0x1000000033e/blog-13 v118734>,<0x100000001fc/blogtest_in v1985>,<0x100000001f7/src v1902>,<0x10000000001/blogbench-1.0 v1628>,<0x10000000000/tmp v77>,<0x1/client.0 v77>]//[3]","memoryvalue":"(4)0x10000000cc2:[<0x20000000748/comment-22.xml v1280>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:42.050 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:41 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:41.746+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000ce7 [6,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-2.xml auth{0=1} v1278 ap=1 s=2738 n(v0 rc2023-09-13T04:43:54.835717+0000 b2738 1=1+0) (ifile excl) (iversion lock) cr={25348=0-4194304@4} caps={25348=pAsLsXsFsxcrwb/pFscr@1},l=25348 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648bf1b4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000ce7:[<0x20000000748/comment-2.xml v1253>,<0x1000000033e/blog-13 v118734>,<0x100000001fc/blogtest_in v1985>,<0x100000001f7/src v1902>,<0x10000000001/blogbench-1.0 v1628>,<0x10000000000/tmp v77>,<0x1/client.0 v77>]//[3]","memoryvalue":"(4)0x10000000ce7:[<0x20000000748/comment-2.xml v1278>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:42.051 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:41 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:41.747+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000008b4 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-44.xml auth{0=1} v1161 ap=1 s=9533 n(v0 rc2023-09-13T04:42:52.095080+0000 b9533 1=1+0) (ifile excl) (iversion lock) cr={25348=0-4194304@3} caps={25348=pAsLsXsFsxcrwb/pFscr@1},l=25348 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648bf1b4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000008b4:[<0x20000000748/comment-44.xml v1043>,<0x1000000033e/blog-13 v77268>,<0x100000001fc/blogtest_in v1012>,<0x100000001f7/src v1888>,<0x10000000001/blogbench-1.0 v1614>,<0x10000000000/tmp v63>,<0x1/client.0 v63>]//[3]","memoryvalue":"(4)0x100000008b4:[<0x20000000748/comment-44.xml v1161>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:42.051 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:41 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:41.779+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000eba [6,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-43.xml auth{0=1} v1292 ap=1 s=8004 n(v0 rc2023-09-13T04:44:31.834916+0000 b8004 1=1+0) (ifile excl) (iversion lock) cr={25348=0-4194304@5} caps={25348=pAsLsXsFsxcrwb/pFscr@1},l=25348 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648bf1b5180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000eba:[<0x20000000748/article-43.xml v1292>,<0x1000000033e/blog-13 v135499>,<0x100000001fc/blogtest_in v2494>,<0x100000001f7/src v1908>,<0x10000000001/blogbench-1.0 v1634>,<0x10000000000/tmp v83>,<0x1/client.0 v83>]//[3]","memoryvalue":"(4)0x10000000eba:[<0x20000000748/article-43.xml v1292>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:42.051 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:41 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:41.779+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000b3f [5,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-33.xml auth{0=1} v1188 ap=1 s=28416 n(v0 rc2023-09-13T04:43:18.802500+0000 b28416 1=1+0) (ifile excl) (iversion lock) cr={25348=0-4194304@4} caps={25348=pAsLsXsFsxcrwb/pFscr@1},l=25348 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c08bc000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000b3f:[<0x20000000748/article-33.xml v1188>,<0x1000000033e/blog-13 v99726>,<0x100000001fc/blogtest_in v1381>,<0x100000001f7/src v1894>,<0x10000000001/blogbench-1.0 v1620>,<0x10000000000/tmp v69>,<0x1/client.0 v69>]//[3]","memoryvalue":"(4)0x10000000b3f:[<0x20000000748/article-33.xml v1188>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:42.051 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:41 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:41.788+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000005c2 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-57.xml auth{0=1} v838 ap=1 s=9387 n(v0 rc2023-09-13T04:42:05.997091+0000 b9387 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c08bdb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x200000005c2:[<0x20000000748/article-57.xml v838>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x200000005c2:[<0x20000000748/article-57.xml v838>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:42.052 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:41 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:41.798+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cbb [6,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-27.xml auth{0=1} v1272 ap=1 s=10188 n(v0 rc2023-09-13T04:43:53.556745+0000 b10188 1=1+0) (ifile excl) (iversion lock) cr={25348=0-4194304@5} caps={25348=pAsLsXsFsxcrwb/pFscr@1},l=25348 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c08bd600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000cbb:[<0x20000000748/comment-27.xml v1203>,<0x1000000033e/blog-13 v118734>,<0x100000001fc/blogtest_in v1985>,<0x100000001f7/src v1902>,<0x10000000001/blogbench-1.0 v1628>,<0x10000000000/tmp v77>,<0x1/client.0 v77>]//[3]","memoryvalue":"(4)0x10000000cbb:[<0x20000000748/comment-27.xml v1272>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:42.052 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:41 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:41.799+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000661 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-5.jpg auth{0=1} v956 ap=1 s=17825 n(v0 rc2023-09-13T04:42:06.529080+0000 b17825 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c08bc580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000661:[<0x20000000748/picture-5.jpg v956>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x20000000661:[<0x20000000748/picture-5.jpg v956>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:42.052 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:41 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:41.799+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000008a3 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-52.xml auth{0=1} v1147 ap=1 s=6788 n(v0 rc2023-09-13T04:42:51.814086+0000 b6788 1=1+0) (ifile excl) (iversion lock) cr={25348=0-4194304@3} caps={25348=pAsLsXsFsxcrwb/pFscr@1},l=25348 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c08be100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000008a3:[<0x20000000748/comment-52.xml v1039>,<0x1000000033e/blog-13 v77268>,<0x100000001fc/blogtest_in v1012>,<0x100000001f7/src v1888>,<0x10000000001/blogbench-1.0 v1614>,<0x10000000000/tmp v63>,<0x1/client.0 v63>]//[3]","memoryvalue":"(4)0x100000008a3:[<0x20000000748/comment-52.xml v1147>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:42.052 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:41 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:41.819+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000ce0 [6,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-7.xml auth{0=1} v1264 ap=1 s=5577 n(v0 rc2023-09-13T04:43:54.715719+0000 b5577 1=1+0) (ifile excl) (iversion lock) cr={25348=0-4194304@5} caps={25348=pAsLsXsFsxcrwb/pFscr@1},l=25348 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c08bf180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000ce0:[<0x20000000748/comment-7.xml v1245>,<0x1000000033e/blog-13 v118734>,<0x100000001fc/blogtest_in v1985>,<0x100000001f7/src v1902>,<0x10000000001/blogbench-1.0 v1628>,<0x10000000000/tmp v77>,<0x1/client.0 v77>]//[3]","memoryvalue":"(4)0x10000000ce0:[<0x20000000748/comment-7.xml v1264>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:42.052 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:41 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:41.836+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000ccb [6,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-15.xml auth{0=1} v1260 ap=1 s=2130 n(v0 rc2023-09-13T04:43:54.079733+0000 b2130 1=1+0) (ifile excl) (iversion lock) cr={25348=0-4194304@5} caps={25348=pAsLsXsFsxcrwb/pFscr@1},l=25348 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c061c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000ccb:[<0x20000000748/comment-15.xml v1229>,<0x1000000033e/blog-13 v118734>,<0x100000001fc/blogtest_in v1985>,<0x100000001f7/src v1902>,<0x10000000001/blogbench-1.0 v1628>,<0x10000000000/tmp v77>,<0x1/client.0 v77>]//[3]","memoryvalue":"(4)0x10000000ccb:[<0x20000000748/comment-15.xml v1260>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:42.053 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:41 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:41.836+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000005bf [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-58.xml auth{0=1} v812 ap=1 s=17501 n(v0 rc2023-09-13T04:42:05.991091+0000 b17501 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c061cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x200000005bf:[<0x20000000748/article-58.xml v812>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x200000005bf:[<0x20000000748/article-58.xml v812>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:42.053 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:41 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:41.847+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cc7 [6,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-17.xml auth{0=1} v1268 ap=1 s=7522 n(v0 rc2023-09-13T04:43:53.944736+0000 b7522 1=1+0) (ifile excl) (iversion lock) cr={25348=0-4194304@5} caps={25348=pAsLsXsFsxcrwb/pFscr@1},l=25348 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c061e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000cc7:[<0x20000000748/comment-17.xml v1226>,<0x1000000033e/blog-13 v118734>,<0x100000001fc/blogtest_in v1985>,<0x100000001f7/src v1902>,<0x10000000001/blogbench-1.0 v1628>,<0x10000000000/tmp v77>,<0x1/client.0 v77>]//[3]","memoryvalue":"(4)0x10000000cc7:[<0x20000000748/comment-17.xml v1268>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:42.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:42 smithi002 ceph-mon[97651]: Scrub error on inode 0x10000000cc2 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-22.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:42.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:42 smithi002 ceph-mon[97651]: Scrub error on inode 0x10000000ce7 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-2.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:42.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:42 smithi002 ceph-mon[97651]: Scrub error on inode 0x100000008b4 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-44.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:42.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:42 smithi002 ceph-mon[97651]: Scrub error on inode 0x10000000eba (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-43.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:42.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:42 smithi002 ceph-mon[97651]: Scrub error on inode 0x10000000b3f (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-33.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:42.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:42 smithi002 ceph-mon[97651]: Scrub error on inode 0x200000005c2 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-57.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:42.508 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:42 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:42.044+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000603 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-28.xml auth{0=1} v854 ap=1 s=24557 n(v0 rc2023-09-13T04:42:06.200087+0000 b24557 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c061ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000603:[<0x20000000748/article-28.xml v854>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x20000000603:[<0x20000000748/article-28.xml v854>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:42.508 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:42 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:42.046+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000005ce [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-53.xml auth{0=1} v920 ap=1 s=27779 n(v0 rc2023-09-13T04:42:06.021091+0000 b27779 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c061d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x200000005ce:[<0x20000000748/article-53.xml v920>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x200000005ce:[<0x20000000748/article-53.xml v920>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:42.508 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:42 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:42.048+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000008b5 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-43.xml auth{0=1} v1163 ap=1 s=6583 n(v0 rc2023-09-13T04:42:52.109080+0000 b6583 1=1+0) (ifile excl) (iversion lock) cr={25348=0-4194304@3} caps={25348=pAsLsXsFsxcrwb/pFscr@1},l=25348 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c061d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000008b5:[<0x20000000748/comment-43.xml v1053>,<0x1000000033e/blog-13 v77268>,<0x100000001fc/blogtest_in v1012>,<0x100000001f7/src v1888>,<0x10000000001/blogbench-1.0 v1614>,<0x10000000000/tmp v63>,<0x1/client.0 v63>]//[3]","memoryvalue":"(4)0x100000008b5:[<0x20000000748/comment-43.xml v1163>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:42.509 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:42 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:42.165+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000008b2 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-45.xml auth{0=1} v1153 ap=1 s=8334 n(v0 rc2023-09-13T04:42:52.084081+0000 b8334 1=1+0) (ifile excl) (iversion lock) cr={25348=0-4194304@3} caps={25348=pAsLsXsFsxcrwb/pFscr@1},l=25348 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c0faeb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000008b2:[<0x20000000748/comment-45.xml v1041>,<0x1000000033e/blog-13 v77268>,<0x100000001fc/blogtest_in v1012>,<0x100000001f7/src v1888>,<0x10000000001/blogbench-1.0 v1614>,<0x10000000000/tmp v63>,<0x1/client.0 v63>]//[3]","memoryvalue":"(4)0x100000008b2:[<0x20000000748/comment-45.xml v1153>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:42.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:42 smithi149 ceph-mon[133315]: Scrub error on inode 0x10000000cc2 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-22.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:42.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:42 smithi149 ceph-mon[133315]: Scrub error on inode 0x10000000ce7 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-2.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:42.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:42 smithi149 ceph-mon[133315]: Scrub error on inode 0x100000008b4 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-44.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:42.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:42 smithi149 ceph-mon[133315]: Scrub error on inode 0x10000000eba (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-43.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:42.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:42 smithi149 ceph-mon[133315]: Scrub error on inode 0x10000000b3f (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-33.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:42.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:42 smithi149 ceph-mon[133315]: Scrub error on inode 0x200000005c2 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-57.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:42.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:42 smithi156 ceph-mon[133675]: Scrub error on inode 0x10000000cc2 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-22.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:42.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:42 smithi156 ceph-mon[133675]: Scrub error on inode 0x10000000ce7 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-2.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:42.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:42 smithi156 ceph-mon[133675]: Scrub error on inode 0x100000008b4 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-44.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:42.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:42 smithi156 ceph-mon[133675]: Scrub error on inode 0x10000000eba (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-43.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:42.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:42 smithi156 ceph-mon[133675]: Scrub error on inode 0x10000000b3f (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-33.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:42.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:42 smithi156 ceph-mon[133675]: Scrub error on inode 0x200000005c2 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-57.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:43 smithi149 ceph-mon[133315]: Scrub error on inode 0x10000000cbb (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-27.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:43 smithi149 ceph-mon[133315]: Scrub error on inode 0x20000000661 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-5.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:43 smithi149 ceph-mon[133315]: Scrub error on inode 0x100000008a3 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-52.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:43 smithi149 ceph-mon[133315]: Scrub error on inode 0x10000000ce0 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-7.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:43 smithi149 ceph-mon[133315]: Scrub error on inode 0x10000000ccb (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-15.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:43 smithi149 ceph-mon[133315]: Scrub error on inode 0x200000005bf (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-58.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:43 smithi149 ceph-mon[133315]: Scrub error on inode 0x10000000cc7 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-17.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:43 smithi149 ceph-mon[133315]: Scrub error on inode 0x20000000603 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-28.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:43 smithi149 ceph-mon[133315]: Scrub error on inode 0x200000005ce (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-53.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:43 smithi149 ceph-mon[133315]: Scrub error on inode 0x100000008b5 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-43.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:43 smithi149 ceph-mon[133315]: Scrub error on inode 0x100000008b2 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-45.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:43 smithi149 ceph-mon[133315]: pgmap v191: 129 pgs: 129 active+clean; 771 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 272 op/s 2023-09-13T04:46:43.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:43 smithi156 ceph-mon[133675]: Scrub error on inode 0x10000000cbb (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-27.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:43 smithi156 ceph-mon[133675]: Scrub error on inode 0x20000000661 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-5.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:43 smithi156 ceph-mon[133675]: Scrub error on inode 0x100000008a3 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-52.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:43 smithi156 ceph-mon[133675]: Scrub error on inode 0x10000000ce0 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-7.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:43 smithi156 ceph-mon[133675]: Scrub error on inode 0x10000000ccb (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-15.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:43 smithi156 ceph-mon[133675]: Scrub error on inode 0x200000005bf (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-58.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:43 smithi156 ceph-mon[133675]: Scrub error on inode 0x10000000cc7 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-17.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:43 smithi156 ceph-mon[133675]: Scrub error on inode 0x20000000603 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-28.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:43 smithi156 ceph-mon[133675]: Scrub error on inode 0x200000005ce (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-53.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:43 smithi156 ceph-mon[133675]: Scrub error on inode 0x100000008b5 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-43.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:43 smithi156 ceph-mon[133675]: Scrub error on inode 0x100000008b2 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-45.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:43 smithi156 ceph-mon[133675]: pgmap v191: 129 pgs: 129 active+clean; 771 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 272 op/s 2023-09-13T04:46:43.741 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-mon[97651]: Scrub error on inode 0x10000000cbb (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-27.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.741 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-mon[97651]: Scrub error on inode 0x20000000661 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-5.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.742 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-mon[97651]: Scrub error on inode 0x100000008a3 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-52.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.742 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-mon[97651]: Scrub error on inode 0x10000000ce0 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-7.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.742 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-mon[97651]: Scrub error on inode 0x10000000ccb (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-15.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.742 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-mon[97651]: Scrub error on inode 0x200000005bf (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-58.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.742 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-mon[97651]: Scrub error on inode 0x10000000cc7 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-17.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-mon[97651]: Scrub error on inode 0x20000000603 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-28.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-mon[97651]: Scrub error on inode 0x200000005ce (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-53.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-mon[97651]: Scrub error on inode 0x100000008b5 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-43.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-mon[97651]: Scrub error on inode 0x100000008b2 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-45.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:43.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-mon[97651]: pgmap v191: 129 pgs: 129 active+clean; 771 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 272 op/s 2023-09-13T04:46:43.744 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:43.354+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cdc [6,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-9.xml auth{0=1} v1276 ap=1 s=7453 n(v0 rc2023-09-13T04:43:54.659721+0000 b7453 1=1+0) (ifile excl) (iversion lock) cr={25348=0-4194304@5} caps={25348=pAsLsXsFsxcrwb/pFscr@1},l=25348 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c0fae000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000cdc:[<0x20000000748/comment-9.xml v1236>,<0x1000000033e/blog-13 v118734>,<0x100000001fc/blogtest_in v1985>,<0x100000001f7/src v1902>,<0x10000000001/blogbench-1.0 v1628>,<0x10000000000/tmp v77>,<0x1/client.0 v77>]//[3]","memoryvalue":"(4)0x10000000cdc:[<0x20000000748/comment-9.xml v1276>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:43.744 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:43.354+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000005d1 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-52.xml auth{0=1} v938 ap=1 s=1450 n(v0 rc2023-09-13T04:42:06.027091+0000 b1450 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c0fae580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x200000005d1:[<0x20000000748/article-52.xml v938>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x200000005d1:[<0x20000000748/article-52.xml v938>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:43.744 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:43.727+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000008a2 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-53.xml auth{0=1} v1134 ap=1 s=3819 n(v0 rc2023-09-13T04:42:51.752088+0000 b3819 1=1+0) (ifile excl) (iversion lock) cr={25348=0-4194304@3} caps={25348=pAsLsXsFsxcrwb/pFscr@1},l=25348 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648bc321b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000008a2:[<0x20000000748/comment-53.xml v1036>,<0x1000000033e/blog-13 v77268>,<0x100000001fc/blogtest_in v1012>,<0x100000001f7/src v1888>,<0x10000000001/blogbench-1.0 v1614>,<0x10000000000/tmp v63>,<0x1/client.0 v63>]//[3]","memoryvalue":"(4)0x100000008a2:[<0x20000000748/comment-53.xml v1134>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:43.745 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:43.728+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cd4 [6,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-10.xml auth{0=1} v1251 ap=1 s=3101 n(v0 rc2023-09-13T04:43:54.429726+0000 b3101 1=1+0) (ifile excl) (iversion lock) cr={25348=0-4194304@5} caps={25348=pAsLsXsFsxcrwb/pFscr@1},l=25348 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648bc321080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000cd4:[<0x20000000748/comment-10.xml v1226>,<0x1000000033e/blog-13 v118734>,<0x100000001fc/blogtest_in v1985>,<0x100000001f7/src v1902>,<0x10000000001/blogbench-1.0 v1628>,<0x10000000000/tmp v77>,<0x1/client.0 v77>]//[3]","memoryvalue":"(4)0x10000000cd4:[<0x20000000748/comment-10.xml v1251>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:44.020 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:43.748+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000665 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-3.jpg auth{0=1} v928 ap=1 s=44505 n(v0 rc2023-09-13T04:42:06.544079+0000 b44505 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648bc321600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000665:[<0x20000000748/picture-3.jpg v928>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x20000000665:[<0x20000000748/picture-3.jpg v928>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:44.020 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:43.749+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000ce2 [6,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-6.xml auth{0=1} v1247 ap=1 s=2470 n(v0 rc2023-09-13T04:43:54.737719+0000 b2470 1=1+0) (ifile excl) (iversion lock) cr={25348=0-4194304@5} caps={25348=pAsLsXsFsxcrwb/pFscr@1},l=25348 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648bc322c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000ce2:[<0x20000000748/comment-6.xml v1240>,<0x1000000033e/blog-13 v118734>,<0x100000001fc/blogtest_in v1985>,<0x100000001f7/src v1902>,<0x10000000001/blogbench-1.0 v1628>,<0x10000000000/tmp v77>,<0x1/client.0 v77>]//[3]","memoryvalue":"(4)0x10000000ce2:[<0x20000000748/comment-6.xml v1247>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:44.020 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:43.769+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000005f7 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-34.xml auth{0=1} v870 ap=1 s=25783 n(v0 rc2023-09-13T04:42:06.152088+0000 b25783 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c0d26000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x200000005f7:[<0x20000000748/article-34.xml v870>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x200000005f7:[<0x20000000748/article-34.xml v870>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:44.020 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:43.769+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000062f [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-5.xml auth{0=1} v830 ap=1 s=9854 n(v0 rc2023-09-13T04:42:06.347084+0000 b9854 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648bc323700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x2000000062f:[<0x20000000748/article-5.xml v830>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x2000000062f:[<0x20000000748/article-5.xml v830>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:44.021 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:43.782+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000008c6 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-35.xml auth{0=1} v1135 ap=1 s=2969 n(v0 rc2023-09-13T04:42:52.519071+0000 b2969 1=1+0) (ifile excl) (iversion lock) cr={25348=0-4194304@3} caps={25348=pAsLsXsFsxcrwb/pFscr@1},l=25348 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c0d26580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000008c6:[<0x20000000748/comment-35.xml v1064>,<0x1000000033e/blog-13 v77268>,<0x100000001fc/blogtest_in v1012>,<0x100000001f7/src v1888>,<0x10000000001/blogbench-1.0 v1614>,<0x10000000000/tmp v63>,<0x1/client.0 v63>]//[3]","memoryvalue":"(4)0x100000008c6:[<0x20000000748/comment-35.xml v1135>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:44.021 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:43.832+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cba [6,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-28.xml auth{0=1} v1255 ap=1 s=4115 n(v0 rc2023-09-13T04:43:53.528745+0000 b4115 1=1+0) (ifile excl) (iversion lock) cr={25348=0-4194304@5} caps={25348=pAsLsXsFsxcrwb/pFscr@1},l=25348 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c0d27600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000cba:[<0x20000000748/comment-28.xml v1199>,<0x1000000033e/blog-13 v118734>,<0x100000001fc/blogtest_in v1985>,<0x100000001f7/src v1902>,<0x10000000001/blogbench-1.0 v1628>,<0x10000000000/tmp v77>,<0x1/client.0 v77>]//[3]","memoryvalue":"(4)0x10000000cba:[<0x20000000748/comment-28.xml v1255>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:44.021 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:43.850+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000601 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-29.xml auth{0=1} v890 ap=1 s=29862 n(v0 rc2023-09-13T04:42:06.191087+0000 b29862 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c0d28100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000601:[<0x20000000748/article-29.xml v890>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x20000000601:[<0x20000000748/article-29.xml v890>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:44.021 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:43.855+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000005b9 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-60.xml auth{0=1} v844 ap=1 s=5502 n(v0 rc2023-09-13T04:42:05.979092+0000 b5502 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c0d28680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x200000005b9:[<0x20000000748/article-60.xml v844>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x200000005b9:[<0x20000000748/article-60.xml v844>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:44.022 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:43.860+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000060f [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-22.xml auth{0=1} v972 ap=1 s=32356 n(v0 rc2023-09-13T04:42:06.245086+0000 b32356 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c0d28c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x2000000060f:[<0x20000000748/article-22.xml v972>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x2000000060f:[<0x20000000748/article-22.xml v972>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:44.022 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:43.980+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000008bf [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-38.xml auth{0=1} v1143 ap=1 s=4597 n(v0 rc2023-09-13T04:42:52.359075+0000 b4597 1=1+0) (ifile excl) (iversion lock) cr={25348=0-4194304@3} caps={25348=pAsLsXsFsxcrwb/pFscr@1},l=25348 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c0d29180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000008bf:[<0x20000000748/comment-38.xml v1050>,<0x1000000033e/blog-13 v77268>,<0x100000001fc/blogtest_in v1012>,<0x100000001f7/src v1888>,<0x10000000001/blogbench-1.0 v1614>,<0x10000000000/tmp v63>,<0x1/client.0 v63>]//[3]","memoryvalue":"(4)0x100000008bf:[<0x20000000748/comment-38.xml v1143>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:44.022 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:43 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:43.991+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000005d9 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-49.xml auth{0=1} v864 ap=1 s=30280 n(v0 rc2023-09-13T04:42:06.046090+0000 b30280 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c0d29700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x200000005d9:[<0x20000000748/article-49.xml v864>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x200000005d9:[<0x20000000748/article-49.xml v864>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3068>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:44.364 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:44 smithi002 ceph-mon[97651]: Scrub error on inode 0x10000000cdc (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-9.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.365 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:44 smithi002 ceph-mon[97651]: Scrub error on inode 0x200000005d1 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-52.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.365 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:44 smithi002 ceph-mon[97651]: Scrub error on inode 0x100000008a2 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-53.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.365 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:44 smithi002 ceph-mon[97651]: Scrub error on inode 0x10000000cd4 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-10.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.365 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:44 smithi002 ceph-mon[97651]: Scrub error on inode 0x20000000665 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-3.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.366 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:44 smithi002 ceph-mon[97651]: Scrub error on inode 0x10000000ce2 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-6.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.366 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:44 smithi002 ceph-mon[97651]: Scrub error on inode 0x200000005f7 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-34.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.366 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:44 smithi002 ceph-mon[97651]: Scrub error on inode 0x2000000062f (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-5.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.366 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:44 smithi002 ceph-mon[97651]: Scrub error on inode 0x100000008c6 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-35.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:44 smithi149 ceph-mon[133315]: Scrub error on inode 0x10000000cdc (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-9.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:44 smithi149 ceph-mon[133315]: Scrub error on inode 0x200000005d1 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-52.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:44 smithi149 ceph-mon[133315]: Scrub error on inode 0x100000008a2 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-53.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:44 smithi149 ceph-mon[133315]: Scrub error on inode 0x10000000cd4 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-10.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:44 smithi149 ceph-mon[133315]: Scrub error on inode 0x20000000665 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-3.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:44 smithi149 ceph-mon[133315]: Scrub error on inode 0x10000000ce2 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-6.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:44 smithi149 ceph-mon[133315]: Scrub error on inode 0x200000005f7 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-34.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:44 smithi149 ceph-mon[133315]: Scrub error on inode 0x2000000062f (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-5.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:44 smithi149 ceph-mon[133315]: Scrub error on inode 0x100000008c6 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-35.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:44 smithi156 ceph-mon[133675]: Scrub error on inode 0x10000000cdc (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-9.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:44 smithi156 ceph-mon[133675]: Scrub error on inode 0x200000005d1 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-52.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:44 smithi156 ceph-mon[133675]: Scrub error on inode 0x100000008a2 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-53.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:44 smithi156 ceph-mon[133675]: Scrub error on inode 0x10000000cd4 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-10.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:44 smithi156 ceph-mon[133675]: Scrub error on inode 0x20000000665 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-3.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:44 smithi156 ceph-mon[133675]: Scrub error on inode 0x10000000ce2 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-6.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:44 smithi156 ceph-mon[133675]: Scrub error on inode 0x200000005f7 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-34.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:44 smithi156 ceph-mon[133675]: Scrub error on inode 0x2000000062f (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-5.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:44.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:44 smithi156 ceph-mon[133675]: Scrub error on inode 0x100000008c6 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-35.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:45.198 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:44 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:44.940+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000655 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-11.jpg auth{0=1} v976 ap=1 s=48251 n(v0 rc2023-09-13T04:42:06.485081+0000 b48251 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648bc323180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000655:[<0x20000000748/picture-11.jpg v976>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x20000000655:[<0x20000000748/picture-11.jpg v976>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3070>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:45.198 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:44 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:44.940+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000b3e [5,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-99.jpg auth v1187 ap=1 s=27041 n(v0 rc2023-09-13T04:43:18.800500+0000 b27041 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x5648c0d27b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000b3e:[<0x20000000748/picture-99.jpg v1187>,<0x1000000033e/blog-13 v99726>,<0x100000001fc/blogtest_in v1381>,<0x100000001f7/src v1894>,<0x10000000001/blogbench-1.0 v1620>,<0x10000000000/tmp v69>,<0x1/client.0 v69>]//[3]","memoryvalue":"(4)0x10000000b3e:[<0x20000000748/picture-99.jpg v1187>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3070>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:45.198 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:44 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:44.940+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cdf [6,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-8.xml auth{0=1} v1257 ap=1 s=8955 n(v0 rc2023-09-13T04:43:54.687720+0000 b8955 1=1+0) (ifile excl) (iversion lock) cr={25348=0-4194304@4} caps={25348=pAsLsXsFsxcrwb/pFscr@1},l=25348 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c0c34b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000cdf:[<0x20000000748/comment-8.xml v1235>,<0x1000000033e/blog-13 v118734>,<0x100000001fc/blogtest_in v1985>,<0x100000001f7/src v1902>,<0x10000000001/blogbench-1.0 v1628>,<0x10000000000/tmp v77>,<0x1/client.0 v77>]//[3]","memoryvalue":"(4)0x10000000cdf:[<0x20000000748/comment-8.xml v1257>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3070>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:45.198 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:44 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:44.979+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000061d [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-14.xml auth{0=1} v878 ap=1 s=32387 n(v0 rc2023-09-13T04:42:06.290085+0000 b32387 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c07e0b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x2000000061d:[<0x20000000748/article-14.xml v878>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x2000000061d:[<0x20000000748/article-14.xml v878>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3070>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:45.199 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:44.997+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000639 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-23.jpg auth{0=1} v804 ap=1 s=56974 n(v0 rc2023-09-13T04:42:06.386083+0000 b56974 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c0db7700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000639:[<0x20000000748/picture-23.jpg v804>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x20000000639:[<0x20000000748/picture-23.jpg v804>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3070>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:45.199 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:45.056+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000092f [5,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-79.jpg auth v1153 ap=1 s=32335 n(v0 rc2023-09-13T04:43:00.414899+0000 b32335 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x5648c07e0580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000092f:[<0x20000000748/picture-79.jpg v1137>,<0x1000000033e/blog-13 v77268>,<0x100000001fc/blogtest_in v1012>,<0x100000001f7/src v1888>,<0x10000000001/blogbench-1.0 v1614>,<0x10000000000/tmp v63>,<0x1/client.0 v63>]//[3]","memoryvalue":"(4)0x1000000092f:[<0x20000000748/picture-79.jpg v1153>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3070>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:45.199 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:45.063+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000005df [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-46.xml auth{0=1} v912 ap=1 s=4945 n(v0 rc2023-09-13T04:42:06.064090+0000 b4945 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c07e1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x200000005df:[<0x20000000748/article-46.xml v912>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x200000005df:[<0x20000000748/article-46.xml v912>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3070>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:45.199 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:45.063+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000005dd [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-47.xml auth{0=1} v908 ap=1 s=13509 n(v0 rc2023-09-13T04:42:06.058090+0000 b13509 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c07e1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x200000005dd:[<0x20000000748/article-47.xml v908>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x200000005dd:[<0x20000000748/article-47.xml v908>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3070>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:45.200 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:45.103+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000598 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-71.xml auth{0=1} v810 ap=1 s=18583 n(v0 rc2023-09-13T04:42:05.912093+0000 b18583 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c07e1b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000598:[<0x20000000748/article-71.xml v810>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x20000000598:[<0x20000000748/article-71.xml v810>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3070>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:45.200 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:45.105+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000063b [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-22.jpg auth{0=1} v966 ap=1 s=25761 n(v0 rc2023-09-13T04:42:06.396083+0000 b25761 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c07e2100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x2000000063b:[<0x20000000748/picture-22.jpg v966>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x2000000063b:[<0x20000000748/picture-22.jpg v966>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3070>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:45.200 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:45.120+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e93 [6,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-16.xml auth{0=1} v1268 ap=1 s=29143 n(v0 rc2023-09-13T04:44:22.379120+0000 b29143 1=1+0) (ifile excl) (iversion lock) cr={25348=0-4194304@5} caps={25348=pAsLsXsFsxcrwb/pFscr@1},l=25348 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c07e3180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000e93:[<0x20000000748/article-16.xml v1263>,<0x1000000033e/blog-13 v126212>,<0x100000001fc/blogtest_in v2266>,<0x100000001f7/src v1906>,<0x10000000001/blogbench-1.0 v1632>,<0x10000000000/tmp v81>,<0x1/client.0 v81>]//[3]","memoryvalue":"(4)0x10000000e93:[<0x20000000748/article-16.xml v1268>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3070>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:45.200 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:45.120+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000005cb [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-54.xml auth{0=1} v898 ap=1 s=6811 n(v0 rc2023-09-13T04:42:06.015091+0000 b6811 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c07e3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x200000005cb:[<0x20000000748/article-54.xml v898>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x200000005cb:[<0x20000000748/article-54.xml v898>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3070>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:45.200 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:45.128+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000607 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-26.xml auth{0=1} v874 ap=1 s=29145 n(v0 rc2023-09-13T04:42:06.215086+0000 b29145 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c0eec000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000607:[<0x20000000748/article-26.xml v874>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x20000000607:[<0x20000000748/article-26.xml v874>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3070>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:45.201 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:45.129+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000064d [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-15.jpg auth{0=1} v886 ap=1 s=44002 n(v0 rc2023-09-13T04:42:06.452081+0000 b44002 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c0eec580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x2000000064d:[<0x20000000748/picture-15.jpg v886>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x2000000064d:[<0x20000000748/picture-15.jpg v886>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3070>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:45.201 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:45.140+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000653 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-12.jpg auth{0=1} v882 ap=1 s=36427 n(v0 rc2023-09-13T04:42:06.477081+0000 b36427 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c0eed080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000653:[<0x20000000748/picture-12.jpg v882>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x20000000653:[<0x20000000748/picture-12.jpg v882>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3070>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:45.201 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:45.144+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000625 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-10.xml auth{0=1} v824 ap=1 s=3146 n(v0 rc2023-09-13T04:42:06.315084+0000 b3146 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c0eecb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000625:[<0x20000000748/article-10.xml v824>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x20000000625:[<0x20000000748/article-10.xml v824>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3070>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:45.201 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:45.175+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000008a8 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-50.xml auth{0=1} v1147 ap=1 s=2602 n(v0 rc2023-09-13T04:42:51.925084+0000 b2602 1=1+0) (ifile excl) (iversion lock) cr={25348=0-4194304@3} caps={25348=pAsLsXsFsxcrwb/pFscr@1},l=25348 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c0eed600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000008a8:[<0x20000000748/comment-50.xml v1044>,<0x1000000033e/blog-13 v77268>,<0x100000001fc/blogtest_in v1012>,<0x100000001f7/src v1888>,<0x10000000001/blogbench-1.0 v1614>,<0x10000000000/tmp v63>,<0x1/client.0 v63>]//[3]","memoryvalue":"(4)0x100000008a8:[<0x20000000748/comment-50.xml v1147>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3070>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:45.201 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:45.175+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000005db [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-48.xml auth{0=1} v796 ap=1 s=4574 n(v0 rc2023-09-13T04:42:06.052090+0000 b4574 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c0eedb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x200000005db:[<0x20000000748/article-48.xml v796>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x200000005db:[<0x20000000748/article-48.xml v796>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3070>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.409+0000 7f69daff5700 1 -- 172.21.15.2:0/1609527680 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c3e22b1ff0 con 0x55c3e24146c0 2023-09-13T04:46:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.409+0000 7f69daff5700 10 client.26452 handle_command_reply: tid=0 2023-09-13T04:46:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.409+0000 7f6a10a4cb80 1 client.26452 shutdown 2023-09-13T04:46:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.413+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/1609527680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3e23aabe0 msgr2=0x55c3e24080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.413+0000 7f6a10a4cb80 1 --2- 172.21.15.2:0/1609527680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3e23aabe0 0x55c3e24080b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55c3e23b0fd0 tx=0x7f69ec004770 comp rx=0 tx=0).stop 2023-09-13T04:46:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.414+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/1609527680 shutdown_connections 2023-09-13T04:46:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.414+0000 7f6a10a4cb80 1 --2- 172.21.15.2:0/1609527680 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55c3e24146c0 0x55c3e2416ae0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.414+0000 7f6a10a4cb80 1 --2- 172.21.15.2:0/1609527680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3e23ab500 0x55c3e24085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.414+0000 7f6a10a4cb80 1 --2- 172.21.15.2:0/1609527680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3e23a6840 0x55c3e2407b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.414+0000 7f6a10a4cb80 1 --2- 172.21.15.2:0/1609527680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3e23aabe0 0x55c3e24080b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.414+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/1609527680 >> 172.21.15.2:0/1609527680 conn(0x55c3e2304c20 msgr2=0x55c3e23b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.414+0000 7f69daff5700 0 client.26452 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:46:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.414+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/1609527680 shutdown_connections 2023-09-13T04:46:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.423+0000 7f6a10a4cb80 1 -- 172.21.15.2:0/1609527680 wait complete. 2023-09-13T04:46:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.423+0000 7f6a10a4cb80 20 client.26452 trim_cache size 0 max 16384 2023-09-13T04:46:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:46:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:46:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:46:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:46:45.584 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:45.198+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000631 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-4.xml auth{0=1} v914 ap=1 s=27955 n(v0 rc2023-09-13T04:42:06.354083+0000 b27955 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c0eee680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000631:[<0x20000000748/article-4.xml v914>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x20000000631:[<0x20000000748/article-4.xml v914>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3070>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:45.584 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:45.198+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000008ac [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-48.xml auth{0=1} v1145 ap=1 s=8937 n(v0 rc2023-09-13T04:42:52.005082+0000 b8937 1=1+0) (ifile excl) (iversion lock) cr={25348=0-4194304@3} caps={25348=pAsLsXsFsxcrwb/pFscr@1},l=25348 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c0eee100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000008ac:[<0x20000000748/comment-48.xml v1046>,<0x1000000033e/blog-13 v77268>,<0x100000001fc/blogtest_in v1012>,<0x100000001f7/src v1888>,<0x10000000001/blogbench-1.0 v1614>,<0x10000000000/tmp v63>,<0x1/client.0 v63>]//[3]","memoryvalue":"(4)0x100000008ac:[<0x20000000748/comment-48.xml v1145>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3070>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:45.585 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:45.211+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000005ef [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-38.xml auth{0=1} v970 ap=1 s=21041 n(v0 rc2023-09-13T04:42:06.122089+0000 b21041 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c0eef180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x200000005ef:[<0x20000000748/article-38.xml v970>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x200000005ef:[<0x20000000748/article-38.xml v970>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3070>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:45.585 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:45.212+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000609 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-25.xml auth{0=1} v982 ap=1 s=32387 n(v0 rc2023-09-13T04:42:06.223086+0000 b32387 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c0eeec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000609:[<0x20000000748/article-25.xml v982>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x20000000609:[<0x20000000748/article-25.xml v982>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3070>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:45.585 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:45.300+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000008ce [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-31.xml auth{0=1} v1149 ap=1 s=2723 n(v0 rc2023-09-13T04:42:52.976061+0000 b2723 1=1+0) (ifile excl) (iversion lock) cr={25348=0-4194304@3} caps={25348=pAsLsXsFsxcrwb/pFscr@1},l=25348 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c0eef700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000008ce:[<0x20000000748/comment-31.xml v1075>,<0x1000000033e/blog-13 v77268>,<0x100000001fc/blogtest_in v1012>,<0x100000001f7/src v1888>,<0x10000000001/blogbench-1.0 v1614>,<0x10000000000/tmp v63>,<0x1/client.0 v63>]//[3]","memoryvalue":"(4)0x100000008ce:[<0x20000000748/comment-31.xml v1149>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3072>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:45.585 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:46:45.335+0000 7f8e28b96700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000623 [4,head] /client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-11.xml auth{0=1} v978 ap=1 s=11922 n(v0 rc2023-09-13T04:42:06.309084+0000 b11922 1=1+0) (iversion lock) caps={25348=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5648c07e2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000623:[<0x20000000748/article-11.xml v978>,<0x1000000033e/blog-13 v44500>,<0x100000001fc/blogtest_in v569>,<0x100000001f7/src v1878>,<0x10000000001/blogbench-1.0 v1604>,<0x10000000000/tmp v53>,<0x1/client.0 v53>]//[3]","memoryvalue":"(4)0x20000000623:[<0x20000000748/article-11.xml v978>,<0x1000000033e/blog-13 v144866>,<0x100000001fc/blogtest_in v3072>,<0x100000001f7/src v1932>,<0x10000000001/blogbench-1.0 v1595>,<0x10000000000/tmp v51>,<0x1/client.0 v38>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T04:46:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.655+0000 7f69fd1ff700 1 -- 172.21.15.2:0/3609652131 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f69e00a09c0 msgr2=0x7f69e00a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.655+0000 7f69fd1ff700 1 --2- 172.21.15.2:0/3609652131 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f69e00a09c0 0x7f69e00a2e80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55c3e23ab900 tx=0x7f69ec009190 comp rx=0 tx=0).stop 2023-09-13T04:46:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.655+0000 7f69fd1ff700 1 -- 172.21.15.2:0/3609652131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69f810f420 msgr2=0x7f69f81188d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.655+0000 7f69fd1ff700 1 --2- 172.21.15.2:0/3609652131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69f810f420 0x7f69f81188d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f69f0006070 tx=0x7f69f0007b80 comp rx=0 tx=0).stop 2023-09-13T04:46:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.655+0000 7f69fd1ff700 1 -- 172.21.15.2:0/3609652131 shutdown_connections 2023-09-13T04:46:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.655+0000 7f69fd1ff700 1 --2- 172.21.15.2:0/3609652131 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f69e00a09c0 0x7f69e00a2e80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.655+0000 7f69fd1ff700 1 --2- 172.21.15.2:0/3609652131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69f811a490 0x7f69f8111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.655+0000 7f69fd1ff700 1 --2- 172.21.15.2:0/3609652131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69f810f420 0x7f69f81188d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.655+0000 7f69fd1ff700 1 --2- 172.21.15.2:0/3609652131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69f810f890 0x7f69f8118e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.655+0000 7f69fd1ff700 1 -- 172.21.15.2:0/3609652131 >> 172.21.15.2:0/3609652131 conn(0x7f69f806cda0 msgr2=0x7f69f806f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.658+0000 7f69fd1ff700 1 -- 172.21.15.2:0/3609652131 shutdown_connections 2023-09-13T04:46:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:45.658+0000 7f69fd1ff700 1 -- 172.21.15.2:0/3609652131 wait complete. 2023-09-13T04:46:46.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:45 smithi149 ceph-mon[133315]: Scrub error on inode 0x10000000cba (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-28.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:46.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:45 smithi149 ceph-mon[133315]: Scrub error on inode 0x20000000601 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-29.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:46.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:45 smithi149 ceph-mon[133315]: Scrub error on inode 0x200000005b9 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-60.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:46.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:45 smithi149 ceph-mon[133315]: Scrub error on inode 0x2000000060f (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-22.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:46.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:45 smithi149 ceph-mon[133315]: Scrub error on inode 0x100000008bf (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-38.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:46.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:45 smithi149 ceph-mon[133315]: Scrub error on inode 0x200000005d9 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-49.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:46.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:45 smithi149 ceph-mon[133315]: pgmap v192: 129 pgs: 129 active+clean; 786 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.6 MiB/s wr, 553 op/s 2023-09-13T04:46:46.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:45 smithi156 ceph-mon[133675]: Scrub error on inode 0x10000000cba (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-28.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:46.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:45 smithi156 ceph-mon[133675]: Scrub error on inode 0x20000000601 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-29.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:46.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:45 smithi156 ceph-mon[133675]: Scrub error on inode 0x200000005b9 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-60.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:46.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:45 smithi156 ceph-mon[133675]: Scrub error on inode 0x2000000060f (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-22.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:46.190 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:45 smithi156 ceph-mon[133675]: Scrub error on inode 0x100000008bf (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-38.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:46.190 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:45 smithi156 ceph-mon[133675]: Scrub error on inode 0x200000005d9 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-49.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:46.190 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:45 smithi156 ceph-mon[133675]: pgmap v192: 129 pgs: 129 active+clean; 786 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.6 MiB/s wr, 553 op/s 2023-09-13T04:46:46.279 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-mon[97651]: Scrub error on inode 0x10000000cba (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-28.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:46.279 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-mon[97651]: Scrub error on inode 0x20000000601 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-29.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:46.279 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-mon[97651]: Scrub error on inode 0x200000005b9 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-60.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:46.279 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-mon[97651]: Scrub error on inode 0x2000000060f (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-22.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:46.280 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-mon[97651]: Scrub error on inode 0x100000008bf (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-38.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:46.280 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-mon[97651]: Scrub error on inode 0x200000005d9 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-49.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:46.280 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:45 smithi002 ceph-mon[97651]: pgmap v192: 129 pgs: 129 active+clean; 786 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.6 MiB/s wr, 553 op/s 2023-09-13T04:46:47.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:46 smithi156 ceph-mon[133675]: Scrub error on inode 0x20000000655 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-11.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:46 smithi156 ceph-mon[133675]: Scrub error on inode 0x10000000b3e (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-99.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:46 smithi156 ceph-mon[133675]: Scrub error on inode 0x10000000cdf (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-8.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:46 smithi156 ceph-mon[133675]: Scrub error on inode 0x2000000061d (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-14.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:46 smithi156 ceph-mon[133675]: Scrub error on inode 0x20000000639 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-23.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:46 smithi156 ceph-mon[133675]: Scrub error on inode 0x1000000092f (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-79.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:46 smithi156 ceph-mon[133675]: Scrub error on inode 0x200000005df (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-46.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:46 smithi156 ceph-mon[133675]: Scrub error on inode 0x200000005dd (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-47.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:46 smithi156 ceph-mon[133675]: Scrub error on inode 0x20000000598 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-71.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:46 smithi156 ceph-mon[133675]: Scrub error on inode 0x2000000063b (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-22.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:46 smithi156 ceph-mon[133675]: Scrub error on inode 0x10000000e93 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-16.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:46 smithi156 ceph-mon[133675]: Scrub error on inode 0x200000005cb (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-54.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:46 smithi156 ceph-mon[133675]: Scrub error on inode 0x20000000607 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-26.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:46 smithi156 ceph-mon[133675]: Scrub error on inode 0x2000000064d (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-15.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:46 smithi156 ceph-mon[133675]: Scrub error on inode 0x20000000653 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-12.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.190 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:46 smithi156 ceph-mon[133675]: Scrub error on inode 0x20000000625 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-10.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.190 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:46 smithi156 ceph-mon[133675]: Scrub error on inode 0x100000008a8 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-50.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.190 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:46 smithi156 ceph-mon[133675]: Scrub error on inode 0x200000005db (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-48.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.190 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:46 smithi156 ceph-mon[133675]: Scrub error on inode 0x20000000631 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-4.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.190 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:46 smithi156 ceph-mon[133675]: Scrub error on inode 0x100000008ac (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-48.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.191 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:46 smithi156 ceph-mon[133675]: Scrub error on inode 0x200000005ef (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-38.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.191 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:46 smithi156 ceph-mon[133675]: Scrub error on inode 0x20000000609 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-25.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.191 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:46 smithi156 ceph-mon[133675]: Scrub error on inode 0x100000008ce (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-31.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.191 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:46 smithi156 ceph-mon[133675]: Scrub error on inode 0x20000000623 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-11.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.318 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:46 smithi002 ceph-mon[97651]: Scrub error on inode 0x20000000655 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-11.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.318 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:46 smithi002 ceph-mon[97651]: Scrub error on inode 0x10000000b3e (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-99.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.319 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:46 smithi002 ceph-mon[97651]: Scrub error on inode 0x10000000cdf (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-8.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.319 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:46 smithi002 ceph-mon[97651]: Scrub error on inode 0x2000000061d (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-14.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.319 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:46 smithi002 ceph-mon[97651]: Scrub error on inode 0x20000000639 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-23.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.319 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:46 smithi002 ceph-mon[97651]: Scrub error on inode 0x1000000092f (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-79.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.319 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:46 smithi002 ceph-mon[97651]: Scrub error on inode 0x200000005df (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-46.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.320 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:46 smithi002 ceph-mon[97651]: Scrub error on inode 0x200000005dd (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-47.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.320 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:46 smithi002 ceph-mon[97651]: Scrub error on inode 0x20000000598 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-71.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.320 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:46 smithi002 ceph-mon[97651]: Scrub error on inode 0x2000000063b (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-22.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.320 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:46 smithi002 ceph-mon[97651]: Scrub error on inode 0x10000000e93 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-16.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.320 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:46 smithi002 ceph-mon[97651]: Scrub error on inode 0x200000005cb (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-54.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.321 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:46 smithi002 ceph-mon[97651]: Scrub error on inode 0x20000000607 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-26.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.321 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:46 smithi002 ceph-mon[97651]: Scrub error on inode 0x2000000064d (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-15.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.321 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:46 smithi002 ceph-mon[97651]: Scrub error on inode 0x20000000653 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-12.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.321 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:46 smithi002 ceph-mon[97651]: Scrub error on inode 0x20000000625 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-10.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.321 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:46 smithi002 ceph-mon[97651]: Scrub error on inode 0x100000008a8 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-50.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.322 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:46 smithi002 ceph-mon[97651]: Scrub error on inode 0x200000005db (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-48.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.322 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:46 smithi002 ceph-mon[97651]: Scrub error on inode 0x20000000631 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-4.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.322 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:46 smithi002 ceph-mon[97651]: Scrub error on inode 0x100000008ac (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-48.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.322 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:46 smithi002 ceph-mon[97651]: Scrub error on inode 0x200000005ef (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-38.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.322 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:46 smithi002 ceph-mon[97651]: Scrub error on inode 0x20000000609 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-25.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:46 smithi002 ceph-mon[97651]: Scrub error on inode 0x100000008ce (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-31.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:46 smithi002 ceph-mon[97651]: Scrub error on inode 0x20000000623 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-11.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:46 smithi149 ceph-mon[133315]: Scrub error on inode 0x20000000655 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-11.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:46 smithi149 ceph-mon[133315]: Scrub error on inode 0x10000000b3e (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-99.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:46 smithi149 ceph-mon[133315]: Scrub error on inode 0x10000000cdf (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-8.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:46 smithi149 ceph-mon[133315]: Scrub error on inode 0x2000000061d (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-14.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:46 smithi149 ceph-mon[133315]: Scrub error on inode 0x20000000639 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-23.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:46 smithi149 ceph-mon[133315]: Scrub error on inode 0x1000000092f (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-79.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:46 smithi149 ceph-mon[133315]: Scrub error on inode 0x200000005df (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-46.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:46 smithi149 ceph-mon[133315]: Scrub error on inode 0x200000005dd (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-47.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:46 smithi149 ceph-mon[133315]: Scrub error on inode 0x20000000598 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-71.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:46 smithi149 ceph-mon[133315]: Scrub error on inode 0x2000000063b (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-22.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:46 smithi149 ceph-mon[133315]: Scrub error on inode 0x10000000e93 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-16.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:46 smithi149 ceph-mon[133315]: Scrub error on inode 0x200000005cb (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-54.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:46 smithi149 ceph-mon[133315]: Scrub error on inode 0x20000000607 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-26.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:46 smithi149 ceph-mon[133315]: Scrub error on inode 0x2000000064d (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-15.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:46 smithi149 ceph-mon[133315]: Scrub error on inode 0x20000000653 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-12.jpg) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.390 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:46 smithi149 ceph-mon[133315]: Scrub error on inode 0x20000000625 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-10.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.390 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:46 smithi149 ceph-mon[133315]: Scrub error on inode 0x100000008a8 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-50.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.390 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:46 smithi149 ceph-mon[133315]: Scrub error on inode 0x200000005db (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-48.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.390 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:46 smithi149 ceph-mon[133315]: Scrub error on inode 0x20000000631 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-4.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.390 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:46 smithi149 ceph-mon[133315]: Scrub error on inode 0x100000008ac (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-48.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.391 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:46 smithi149 ceph-mon[133315]: Scrub error on inode 0x200000005ef (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-38.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.391 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:46 smithi149 ceph-mon[133315]: Scrub error on inode 0x20000000609 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-25.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.391 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:46 smithi149 ceph-mon[133315]: Scrub error on inode 0x100000008ce (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-31.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:47.391 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:46 smithi149 ceph-mon[133315]: Scrub error on inode 0x20000000623 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-11.xml) see mds.a log and `damage ls` output for details 2023-09-13T04:46:48.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:47 smithi156 ceph-mon[133675]: pgmap v193: 129 pgs: 129 active+clean; 791 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.5 MiB/s wr, 535 op/s 2023-09-13T04:46:48.266 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:47 smithi002 ceph-mon[97651]: pgmap v193: 129 pgs: 129 active+clean; 791 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.5 MiB/s wr, 535 op/s 2023-09-13T04:46:48.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:47 smithi149 ceph-mon[133315]: pgmap v193: 129 pgs: 129 active+clean; 791 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.5 MiB/s wr, 535 op/s 2023-09-13T04:46:49.265 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:48 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mon-a[97627]: 2023-09-13T04:46:48.927+0000 7fbd28d3d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-09-13T04:46:49.266 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:48 smithi002 ceph-mon[97651]: pgmap v194: 129 pgs: 129 active+clean; 794 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.8 MiB/s wr, 579 op/s 2023-09-13T04:46:49.266 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:48 smithi002 ceph-mon[97651]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-09-13T04:46:49.266 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:48 smithi002 ceph-mon[97651]: mds.? [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] up:active 2023-09-13T04:46:49.266 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:48 smithi002 ceph-mon[97651]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:46:49.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:48 smithi149 ceph-mon[133315]: pgmap v194: 129 pgs: 129 active+clean; 794 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.8 MiB/s wr, 579 op/s 2023-09-13T04:46:49.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:48 smithi149 ceph-mon[133315]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-09-13T04:46:49.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:48 smithi149 ceph-mon[133315]: mds.? [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] up:active 2023-09-13T04:46:49.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:48 smithi149 ceph-mon[133315]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:46:49.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:48 smithi156 ceph-mon[133675]: pgmap v194: 129 pgs: 129 active+clean; 794 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.8 MiB/s wr, 579 op/s 2023-09-13T04:46:49.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:48 smithi156 ceph-mon[133675]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-09-13T04:46:49.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:48 smithi156 ceph-mon[133675]: mds.? [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] up:active 2023-09-13T04:46:49.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:48 smithi156 ceph-mon[133675]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:46:50.713 INFO:tasks.workunit.client.0.smithi002.stdout: 160 676444 407 495824 237 279359 695 2023-09-13T04:46:52.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:51 smithi149 ceph-mon[133315]: pgmap v195: 129 pgs: 129 active+clean; 788 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.3 MiB/s wr, 670 op/s 2023-09-13T04:46:52.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:51 smithi156 ceph-mon[133675]: pgmap v195: 129 pgs: 129 active+clean; 788 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.3 MiB/s wr, 670 op/s 2023-09-13T04:46:52.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:51 smithi002 ceph-mon[97651]: pgmap v195: 129 pgs: 129 active+clean; 788 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.3 MiB/s wr, 670 op/s 2023-09-13T04:46:52.902 INFO:tasks.workunit.client.0.smithi002.stdout: 2023-09-13T04:46:52.902 INFO:tasks.workunit.client.0.smithi002.stdout:Final score for writes: 160 2023-09-13T04:46:52.902 INFO:tasks.workunit.client.0.smithi002.stdout:Final score for reads : 315874 2023-09-13T04:46:52.903 INFO:tasks.workunit.client.0.smithi002.stdout: 2023-09-13T04:46:52.904 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-09-13T04:46:52.904 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-09-13T04:46:53.880 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T04:46:54.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:53 smithi149 ceph-mon[133315]: pgmap v196: 129 pgs: 129 active+clean; 790 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 532 op/s 2023-09-13T04:46:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.136+0000 7fb3a17ff700 1 -- 172.21.15.2:0/226626582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb39c067160 msgr2=0x7fb39c103ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.136+0000 7fb3a17ff700 1 --2- 172.21.15.2:0/226626582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb39c067160 0x7fb39c103ac0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb39400b0a0 tx=0x7fb39400b3b0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.136+0000 7fb3a17ff700 1 -- 172.21.15.2:0/226626582 shutdown_connections 2023-09-13T04:46:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.136+0000 7fb3a17ff700 1 --2- 172.21.15.2:0/226626582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb39c067160 0x7fb39c103ac0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.136+0000 7fb3a17ff700 1 --2- 172.21.15.2:0/226626582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb39c0ff120 0x7fb39c0ff500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.137+0000 7fb3a17ff700 1 --2- 172.21.15.2:0/226626582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb39c0ffad0 0x7fb39c104350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.137+0000 7fb3a17ff700 1 -- 172.21.15.2:0/226626582 >> 172.21.15.2:0/226626582 conn(0x7fb39c075c30 msgr2=0x7fb39c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:54.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.138+0000 7fb3a17ff700 1 -- 172.21.15.2:0/226626582 shutdown_connections 2023-09-13T04:46:54.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.139+0000 7fb3a17ff700 1 -- 172.21.15.2:0/226626582 wait complete. 2023-09-13T04:46:54.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.141+0000 7fb3a17ff700 1 Processor -- start 2023-09-13T04:46:54.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.141+0000 7fb3a17ff700 1 -- start start 2023-09-13T04:46:54.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.142+0000 7fb3a17ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb39c067160 0x7fb39c109360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:54.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.142+0000 7fb3a17ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb39c0ff120 0x7fb39c106370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:54.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.142+0000 7fb3a17ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb39c1068b0 0x7fb39c106d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:54.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.142+0000 7fb3a17ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c109b60 con 0x7fb39c067160 2023-09-13T04:46:54.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.142+0000 7fb3a17ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c071720 con 0x7fb39c1068b0 2023-09-13T04:46:54.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.142+0000 7fb3a17ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c0718e0 con 0x7fb39c0ff120 2023-09-13T04:46:54.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.142+0000 7fb39b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb39c1068b0 0x7fb39c106d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:54.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.142+0000 7fb39b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb39c1068b0 0x7fb39c106d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:53044/0 (socket says 172.21.15.2:53044) 2023-09-13T04:46:54.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.142+0000 7fb39b7fe700 1 -- 172.21.15.2:0/3045133376 learned_addr learned my addr 172.21.15.2:0/3045133376 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:54.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.142+0000 7fb39a7fc700 1 --2- 172.21.15.2:0/3045133376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb39c0ff120 0x7fb39c106370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:54.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.143+0000 7fb39b7fe700 1 -- 172.21.15.2:0/3045133376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb39c0ff120 msgr2=0x7fb39c106370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:54.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.143+0000 7fb39b7fe700 1 --2- 172.21.15.2:0/3045133376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb39c0ff120 0x7fb39c106370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.143+0000 7fb39b7fe700 1 -- 172.21.15.2:0/3045133376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb39c067160 msgr2=0x7fb39c109360 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-13T04:46:54.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.143+0000 7fb39b7fe700 1 --2- 172.21.15.2:0/3045133376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb39c067160 0x7fb39c109360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.143+0000 7fb39b7fe700 1 -- 172.21.15.2:0/3045133376 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb394009d00 con 0x7fb39c1068b0 2023-09-13T04:46:54.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.147+0000 7fb39b7fe700 1 --2- 172.21.15.2:0/3045133376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb39c1068b0 0x7fb39c106d60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb39400b830 tx=0x7fb394007b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:54.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.148+0000 7fb383fff700 1 -- 172.21.15.2:0/3045133376 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39400e040 con 0x7fb39c1068b0 2023-09-13T04:46:54.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.148+0000 7fb383fff700 1 -- 172.21.15.2:0/3045133376 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb394012dc0 con 0x7fb39c1068b0 2023-09-13T04:46:54.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.148+0000 7fb383fff700 1 -- 172.21.15.2:0/3045133376 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39401b450 con 0x7fb39c1068b0 2023-09-13T04:46:54.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.148+0000 7fb3a17ff700 1 -- 172.21.15.2:0/3045133376 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb39c071b60 con 0x7fb39c1068b0 2023-09-13T04:46:54.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.149+0000 7fb3a17ff700 1 -- 172.21.15.2:0/3045133376 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39c072020 con 0x7fb39c1068b0 2023-09-13T04:46:54.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.156+0000 7fb3b50cab80 1 Processor -- start 2023-09-13T04:46:54.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.160+0000 7fb383fff700 1 -- 172.21.15.2:0/3045133376 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7fb394012900 con 0x7fb39c1068b0 2023-09-13T04:46:54.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.163+0000 7fb383fff700 1 --2- 172.21.15.2:0/3045133376 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fb3840a0980 0x7fb3840a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:54.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.163+0000 7fb383fff700 1 -- 172.21.15.2:0/3045133376 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fb3940cdf60 con 0x7fb39c1068b0 2023-09-13T04:46:54.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.163+0000 7fb3b50cab80 1 -- start start 2023-09-13T04:46:54.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.164+0000 7fb39a7fc700 1 --2- 172.21.15.2:0/3045133376 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fb3840a0980 0x7fb3840a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:54.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.164+0000 7fb3b50cab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556070763fe0 0x5560707643c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:54.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.164+0000 7fb3b50cab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556070764900 0x55607075e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:54.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.164+0000 7fb3b50cab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55607075ea00 0x55607075ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:54.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.164+0000 7fb3b50cab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560706b7780 con 0x556070764900 2023-09-13T04:46:54.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.164+0000 7fb3b50cab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560706b50a0 con 0x55607075ea00 2023-09-13T04:46:54.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.164+0000 7fb3b50cab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560706b0990 con 0x556070763fe0 2023-09-13T04:46:54.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.164+0000 7fb39b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55607075ea00 0x55607075ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:54.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.164+0000 7fb39affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556070763fe0 0x5560707643c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:54.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.164+0000 7fb39affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556070763fe0 0x5560707643c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:60354/0 (socket says 172.21.15.2:60354) 2023-09-13T04:46:54.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.165+0000 7fb39affd700 1 -- 172.21.15.2:0/77587339 learned_addr learned my addr 172.21.15.2:0/77587339 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:54.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.165+0000 7fb39a7fc700 1 --2- 172.21.15.2:0/77587339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556070764900 0x55607075e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:54.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.166+0000 7fb39a7fc700 1 --2- 172.21.15.2:0/3045133376 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fb3840a0980 0x7fb3840a2e40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb39c107b60 tx=0x7fb39000a770 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:54.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.166+0000 7fb39a7fc700 1 -- 172.21.15.2:0/77587339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556070763fe0 msgr2=0x5560707643c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:54.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.166+0000 7fb39a7fc700 1 --2- 172.21.15.2:0/77587339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556070763fe0 0x5560707643c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.166+0000 7fb39a7fc700 1 -- 172.21.15.2:0/77587339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55607075ea00 msgr2=0x55607075ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:54.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.166+0000 7fb39a7fc700 1 --2- 172.21.15.2:0/77587339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55607075ea00 0x55607075ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.166+0000 7fb39a7fc700 1 -- 172.21.15.2:0/77587339 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556070530630 con 0x556070764900 2023-09-13T04:46:54.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.166+0000 7fb39a7fc700 1 --2- 172.21.15.2:0/77587339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556070764900 0x55607075e4c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb39c10cd80 tx=0x7fb39001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6433a439653170 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:54.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.167+0000 7fb3817fa700 1 -- 172.21.15.2:0/77587339 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb390012e40 con 0x556070764900 2023-09-13T04:46:54.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.167+0000 7fb3817fa700 1 -- 172.21.15.2:0/77587339 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb390024900 con 0x556070764900 2023-09-13T04:46:54.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.167+0000 7fb3817fa700 1 -- 172.21.15.2:0/77587339 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39001ca10 con 0x556070764900 2023-09-13T04:46:54.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.168+0000 7fb3b50cab80 1 -- 172.21.15.2:0/77587339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556070764900 msgr2=0x55607075e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.168+0000 7fb3b50cab80 1 --2- 172.21.15.2:0/77587339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556070764900 0x55607075e4c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb39c10cd80 tx=0x7fb39001a000 comp rx=0 tx=0).stop 2023-09-13T04:46:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.170+0000 7fb3b50cab80 1 -- 172.21.15.2:0/77587339 shutdown_connections 2023-09-13T04:46:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.170+0000 7fb3b50cab80 1 --2- 172.21.15.2:0/77587339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55607075ea00 0x55607075ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.170+0000 7fb3b50cab80 1 --2- 172.21.15.2:0/77587339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556070763fe0 0x5560707643c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.170+0000 7fb3b50cab80 1 --2- 172.21.15.2:0/77587339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556070764900 0x55607075e4c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.170+0000 7fb3b50cab80 1 -- 172.21.15.2:0/77587339 >> 172.21.15.2:0/77587339 conn(0x5560706bcc80 msgr2=0x556070758f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.171+0000 7fb3b50cab80 1 -- 172.21.15.2:0/77587339 shutdown_connections 2023-09-13T04:46:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.171+0000 7fb3b50cab80 1 -- 172.21.15.2:0/77587339 wait complete. 2023-09-13T04:46:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.172+0000 7fb3b50cab80 1 -- start start 2023-09-13T04:46:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.173+0000 7fb3b50cab80 1 Processor -- start 2023-09-13T04:46:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.173+0000 7fb3b50cab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55607075ea00 0x55607076a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.173+0000 7fb3b50cab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556070764900 0x556070767be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:54.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.174+0000 7fb3b50cab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556070768280 0x5560707743a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:54.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.174+0000 7fb3b50cab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607064ebb0 con 0x55607075ea00 2023-09-13T04:46:54.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.174+0000 7fb3b50cab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607064e8b0 con 0x556070768280 2023-09-13T04:46:54.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.174+0000 7fb3b50cab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556070532920 con 0x556070764900 2023-09-13T04:46:54.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.174+0000 7fb3b50cab80 10 client.0 _ll_register_callbacks cb 0x55607064c680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:46:54.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.174+0000 7fb39affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55607075ea00 0x55607076a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:54.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.174+0000 7fb39affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55607075ea00 0x55607076a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54244/0 (socket says 172.21.15.2:54244) 2023-09-13T04:46:54.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.174+0000 7fb39affd700 1 -- 172.21.15.2:0/1555689174 learned_addr learned my addr 172.21.15.2:0/1555689174 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:54.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.175+0000 7fb39a7fc700 1 --2- 172.21.15.2:0/1555689174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556070764900 0x556070767be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:54.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.175+0000 7fb39affd700 1 -- 172.21.15.2:0/1555689174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556070764900 msgr2=0x556070767be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:54.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.175+0000 7fb39affd700 1 --2- 172.21.15.2:0/1555689174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556070764900 0x556070767be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.175+0000 7fb39affd700 1 -- 172.21.15.2:0/1555689174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556070768280 msgr2=0x5560707743a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:46:54.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.175+0000 7fb39affd700 1 --2- 172.21.15.2:0/1555689174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556070768280 0x5560707743a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.175+0000 7fb39affd700 1 -- 172.21.15.2:0/1555689174 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39000a970 con 0x55607075ea00 2023-09-13T04:46:54.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.175+0000 7fb39a7fc700 1 --2- 172.21.15.2:0/1555689174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556070764900 0x556070767be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:46:54.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.176+0000 7fb39affd700 1 --2- 172.21.15.2:0/1555689174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55607075ea00 0x55607076a960 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00d160 tx=0x7fb38c00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ccffefa239bfe3f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:54.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.176+0000 7fb37eff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:46:54.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.176+0000 7fb37eff5700 1 -- 172.21.15.2:0/1555689174 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c00de60 con 0x55607075ea00 2023-09-13T04:46:54.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.177+0000 7fb37eff5700 1 -- 172.21.15.2:0/1555689174 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fb38c014430 con 0x55607075ea00 2023-09-13T04:46:54.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.179+0000 7fb3b50cab80 1 -- 172.21.15.2:0/1555689174 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556070751670 con 0x55607075ea00 2023-09-13T04:46:54.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.180+0000 7fb37eff5700 1 -- 172.21.15.2:0/1555689174 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb38c00b380 con 0x55607075ea00 2023-09-13T04:46:54.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.181+0000 7fb3b50cab80 10 client.16293 fetch_fsmap learned FSMap version 35 2023-09-13T04:46:54.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.181+0000 7fb3b50cab80 1 -- 172.21.15.2:0/1555689174 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5560706c87a0 con 0x55607075ea00 2023-09-13T04:46:54.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:53 smithi156 ceph-mon[133675]: pgmap v196: 129 pgs: 129 active+clean; 790 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 532 op/s 2023-09-13T04:46:54.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.191+0000 7fb37eff5700 1 -- 172.21.15.2:0/1555689174 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556070751670 con 0x55607075ea00 2023-09-13T04:46:54.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.191+0000 7fb37eff5700 1 -- 172.21.15.2:0/1555689174 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 35) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fb38c0128b0 con 0x55607075ea00 2023-09-13T04:46:54.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.192+0000 7fb3b50cab80 10 client.16293 fetch_fsmap finished waiting for FSMap version 35 2023-09-13T04:46:54.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.192+0000 7fb3b50cab80 10 client.16293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:46:54.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.192+0000 7fb3b50cab80 20 client.16293 populate_metadata read hostname 'smithi002' 2023-09-13T04:46:54.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.192+0000 7fb3b50cab80 1 --2- 172.21.15.2:0/1555689174 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5560707897a0 0x55607078bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:54.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.192+0000 7fb3b50cab80 4 client.16293 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:46:54.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.192+0000 7fb3b50cab80 1 -- 172.21.15.2:0/1555689174 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5560706a9970 con 0x5560707897a0 2023-09-13T04:46:54.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.192+0000 7fb39a7fc700 1 --2- 172.21.15.2:0/1555689174 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5560707897a0 0x55607078bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:54.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.197+0000 7fb39a7fc700 1 --2- 172.21.15.2:0/1555689174 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5560707897a0 0x55607078bb90 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64a62e9e21e5c19 server_cookie=425701b8b309337a in_seq=0 out_seq=0 2023-09-13T04:46:54.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.199+0000 7fb37eff5700 10 client.16293 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:46:54.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.205+0000 7fb37eff5700 1 -- 172.21.15.2:0/1555689174 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5560706a9970 con 0x5560707897a0 2023-09-13T04:46:54.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.205+0000 7fb37eff5700 10 client.16293 handle_command_reply: tid=0 2023-09-13T04:46:54.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.206+0000 7fb3b50cab80 1 client.16293 shutdown 2023-09-13T04:46:54.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.207+0000 7fb3b50cab80 1 -- 172.21.15.2:0/1555689174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55607075ea00 msgr2=0x55607076a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:54.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.207+0000 7fb3b50cab80 1 --2- 172.21.15.2:0/1555689174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55607075ea00 0x55607076a960 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00d160 tx=0x7fb38c00d520 comp rx=0 tx=0).stop 2023-09-13T04:46:54.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.208+0000 7fb3b50cab80 1 -- 172.21.15.2:0/1555689174 shutdown_connections 2023-09-13T04:46:54.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.208+0000 7fb3b50cab80 1 --2- 172.21.15.2:0/1555689174 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5560707897a0 0x55607078bb90 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.208+0000 7fb3b50cab80 1 --2- 172.21.15.2:0/1555689174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556070768280 0x5560707743a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.208+0000 7fb3b50cab80 1 --2- 172.21.15.2:0/1555689174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55607075ea00 0x55607076a960 secure :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00d160 tx=0x7fb38c00d520 comp rx=0 tx=0).stop 2023-09-13T04:46:54.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.208+0000 7fb3b50cab80 1 --2- 172.21.15.2:0/1555689174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556070764900 0x556070767be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.208+0000 7fb3b50cab80 1 -- 172.21.15.2:0/1555689174 >> 172.21.15.2:0/1555689174 conn(0x5560706bcc80 msgr2=0x5560707537f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:54.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.208+0000 7fb37eff5700 0 client.16293 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:46:54.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.211+0000 7fb3b50cab80 1 -- 172.21.15.2:0/1555689174 shutdown_connections 2023-09-13T04:46:54.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.216+0000 7fb3b50cab80 1 -- 172.21.15.2:0/1555689174 wait complete. 2023-09-13T04:46:54.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.221+0000 7fb3b50cab80 20 client.16293 trim_cache size 0 max 16384 2023-09-13T04:46:54.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:53 smithi002 ceph-mon[97651]: pgmap v196: 129 pgs: 129 active+clean; 790 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 532 op/s 2023-09-13T04:46:54.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.261+0000 7fb3b50cab80 1 Processor -- start 2023-09-13T04:46:54.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.261+0000 7fb3b50cab80 1 -- start start 2023-09-13T04:46:54.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.262+0000 7fb3b50cab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556070764900 0x55607075e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:54.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.262+0000 7fb3b50cab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556070768280 0x55607076f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:54.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.262+0000 7fb3b50cab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556070767d40 0x5560707538f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:54.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.262+0000 7fb3b50cab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560707594f0 con 0x556070768280 2023-09-13T04:46:54.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.262+0000 7fb3b50cab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556070759cf0 con 0x556070767d40 2023-09-13T04:46:54.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.262+0000 7fb3b50cab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556070786d30 con 0x556070764900 2023-09-13T04:46:54.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.262+0000 7fb39affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556070764900 0x55607075e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:54.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.262+0000 7fb39affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556070764900 0x55607075e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:60374/0 (socket says 172.21.15.2:60374) 2023-09-13T04:46:54.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.262+0000 7fb39affd700 1 -- 172.21.15.2:0/1382034197 learned_addr learned my addr 172.21.15.2:0/1382034197 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:54.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.263+0000 7fb39b7fe700 1 --2- 172.21.15.2:0/1382034197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556070767d40 0x5560707538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:54.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.263+0000 7fb39a7fc700 1 --2- 172.21.15.2:0/1382034197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556070768280 0x55607076f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:54.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.264+0000 7fb39b7fe700 1 -- 172.21.15.2:0/1382034197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556070764900 msgr2=0x55607075e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:54.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.264+0000 7fb39b7fe700 1 --2- 172.21.15.2:0/1382034197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556070764900 0x55607075e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.264+0000 7fb39b7fe700 1 -- 172.21.15.2:0/1382034197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556070768280 msgr2=0x55607076f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:54.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.264+0000 7fb39b7fe700 1 --2- 172.21.15.2:0/1382034197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556070768280 0x55607076f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.264+0000 7fb39b7fe700 1 -- 172.21.15.2:0/1382034197 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560704dcde0 con 0x556070767d40 2023-09-13T04:46:54.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.264+0000 7fb39b7fe700 1 --2- 172.21.15.2:0/1382034197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556070767d40 0x5560707538f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb3940cb340 tx=0x7fb394009040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdb7d769ea635907 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:54.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.266+0000 7fb381ffb700 1 -- 172.21.15.2:0/1382034197 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39409b830 con 0x556070767d40 2023-09-13T04:46:54.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.266+0000 7fb381ffb700 1 -- 172.21.15.2:0/1382034197 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb39409d070 con 0x556070767d40 2023-09-13T04:46:54.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.266+0000 7fb381ffb700 1 -- 172.21.15.2:0/1382034197 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3940cb5c0 con 0x556070767d40 2023-09-13T04:46:54.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.267+0000 7fb3b50cab80 1 -- 172.21.15.2:0/1382034197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556070767d40 msgr2=0x5560707538f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:54.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.267+0000 7fb3b50cab80 1 --2- 172.21.15.2:0/1382034197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556070767d40 0x5560707538f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb3940cb340 tx=0x7fb394009040 comp rx=0 tx=0).stop 2023-09-13T04:46:54.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.268+0000 7fb3b50cab80 1 -- 172.21.15.2:0/1382034197 shutdown_connections 2023-09-13T04:46:54.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.268+0000 7fb3b50cab80 1 --2- 172.21.15.2:0/1382034197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556070767d40 0x5560707538f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.268+0000 7fb3b50cab80 1 --2- 172.21.15.2:0/1382034197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556070764900 0x55607075e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.268+0000 7fb3b50cab80 1 --2- 172.21.15.2:0/1382034197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556070768280 0x55607076f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.268+0000 7fb3b50cab80 1 -- 172.21.15.2:0/1382034197 >> 172.21.15.2:0/1382034197 conn(0x5560706bcc80 msgr2=0x556070773a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:54.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.269+0000 7fb3b50cab80 1 -- 172.21.15.2:0/1382034197 shutdown_connections 2023-09-13T04:46:54.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.269+0000 7fb3b50cab80 1 -- 172.21.15.2:0/1382034197 wait complete. 2023-09-13T04:46:54.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.269+0000 7fb3b50cab80 1 -- start start 2023-09-13T04:46:54.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.270+0000 7fb3b50cab80 1 Processor -- start 2023-09-13T04:46:54.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.270+0000 7fb3b50cab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556070764900 0x55607076f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:54.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.270+0000 7fb3b50cab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556070767d40 0x55607075fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:54.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.270+0000 7fb3b50cab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556070768280 0x556070760350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:54.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.270+0000 7fb3b50cab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556070760a20 con 0x556070767d40 2023-09-13T04:46:54.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.270+0000 7fb3b50cab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556070760b90 con 0x556070764900 2023-09-13T04:46:54.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.270+0000 7fb3b50cab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560707dd570 con 0x556070768280 2023-09-13T04:46:54.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.271+0000 7fb3b50cab80 10 client.0 _ll_register_callbacks cb 0x55607064c680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:46:54.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.272+0000 7fb39a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556070767d40 0x55607075fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:54.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.273+0000 7fb39a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556070767d40 0x55607075fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54260/0 (socket says 172.21.15.2:54260) 2023-09-13T04:46:54.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.273+0000 7fb39a7fc700 1 -- 172.21.15.2:0/376039626 learned_addr learned my addr 172.21.15.2:0/376039626 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:54.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.273+0000 7fb39affd700 1 --2- 172.21.15.2:0/376039626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556070764900 0x55607076f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:54.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.273+0000 7fb39affd700 1 -- 172.21.15.2:0/376039626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556070768280 msgr2=0x556070760350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T04:46:54.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.273+0000 7fb39affd700 1 --2- 172.21.15.2:0/376039626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556070768280 0x556070760350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.273+0000 7fb39affd700 1 -- 172.21.15.2:0/376039626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556070767d40 msgr2=0x55607075fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:54.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.273+0000 7fb39affd700 1 --2- 172.21.15.2:0/376039626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556070767d40 0x55607075fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.273+0000 7fb39affd700 1 -- 172.21.15.2:0/376039626 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39c072020 con 0x556070764900 2023-09-13T04:46:54.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.275+0000 7fb39affd700 1 --2- 172.21.15.2:0/376039626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556070764900 0x55607076f4e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00d920 tx=0x7fb38c0134b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd880d00a7feb904 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:54.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.277+0000 7fb37eff5700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:46:54.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.281+0000 7fb37eff5700 1 -- 172.21.15.2:0/376039626 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c00a040 con 0x556070764900 2023-09-13T04:46:54.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.282+0000 7fb37eff5700 1 -- 172.21.15.2:0/376039626 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fb38c00b320 con 0x556070764900 2023-09-13T04:46:54.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.282+0000 7fb3b50cab80 1 -- 172.21.15.2:0/376039626 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb390024900 con 0x556070764900 2023-09-13T04:46:54.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.285+0000 7fb37eff5700 1 -- 172.21.15.2:0/376039626 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb38c0125f0 con 0x556070764900 2023-09-13T04:46:54.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.285+0000 7fb3b50cab80 10 client.26444 fetch_fsmap learned FSMap version 35 2023-09-13T04:46:54.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.285+0000 7fb3b50cab80 1 -- 172.21.15.2:0/376039626 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5560704c5c60 con 0x556070764900 2023-09-13T04:46:54.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.288+0000 7fb37eff5700 1 -- 172.21.15.2:0/376039626 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb390024900 con 0x556070764900 2023-09-13T04:46:54.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.288+0000 7fb37eff5700 1 -- 172.21.15.2:0/376039626 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fb38c01e450 con 0x556070764900 2023-09-13T04:46:54.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.289+0000 7fb3b50cab80 10 client.26444 fetch_fsmap finished waiting for FSMap version 35 2023-09-13T04:46:54.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.289+0000 7fb3b50cab80 10 client.26444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:46:54.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.289+0000 7fb3b50cab80 20 client.26444 populate_metadata read hostname 'smithi002' 2023-09-13T04:46:54.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.289+0000 7fb3b50cab80 1 --2- 172.21.15.2:0/376039626 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5560707dec00 0x5560707e1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:54.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.289+0000 7fb3b50cab80 4 client.26444 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-09-13T04:46:54.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.289+0000 7fb3b50cab80 1 -- 172.21.15.2:0/376039626 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55607066a050 con 0x5560707dec00 2023-09-13T04:46:54.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.290+0000 7fb39a7fc700 1 --2- 172.21.15.2:0/376039626 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5560707dec00 0x5560707e1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:54.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.291+0000 7fb39a7fc700 1 --2- 172.21.15.2:0/376039626 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5560707dec00 0x5560707e1020 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77b49fd9080402c2 server_cookie=4f5bb859dfdab004 in_seq=0 out_seq=0 2023-09-13T04:46:54.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.291+0000 7fb37eff5700 10 client.26444 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:46:54.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.291+0000 7fb37eff5700 1 -- 172.21.15.2:0/376039626 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55607066a050 con 0x5560707dec00 2023-09-13T04:46:54.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.291+0000 7fb37eff5700 10 client.26444 handle_command_reply: tid=0 2023-09-13T04:46:54.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.292+0000 7fb3b50cab80 1 client.26444 shutdown 2023-09-13T04:46:54.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.292+0000 7fb3b50cab80 1 -- 172.21.15.2:0/376039626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556070764900 msgr2=0x55607076f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:54.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.292+0000 7fb3b50cab80 1 --2- 172.21.15.2:0/376039626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556070764900 0x55607076f4e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00d920 tx=0x7fb38c0134b0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.296+0000 7fb3b50cab80 1 -- 172.21.15.2:0/376039626 shutdown_connections 2023-09-13T04:46:54.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.296+0000 7fb3b50cab80 1 --2- 172.21.15.2:0/376039626 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5560707dec00 0x5560707e1020 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.296+0000 7fb3b50cab80 1 --2- 172.21.15.2:0/376039626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556070768280 0x556070760350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.296+0000 7fb3b50cab80 1 --2- 172.21.15.2:0/376039626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556070764900 0x55607076f4e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.296+0000 7fb3b50cab80 1 --2- 172.21.15.2:0/376039626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556070767d40 0x55607075fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.296+0000 7fb3b50cab80 1 -- 172.21.15.2:0/376039626 >> 172.21.15.2:0/376039626 conn(0x5560706bcc80 msgr2=0x556070764d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:54.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.296+0000 7fb37eff5700 0 client.26444 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:46:54.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.297+0000 7fb3b50cab80 1 -- 172.21.15.2:0/376039626 shutdown_connections 2023-09-13T04:46:54.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.298+0000 7fb3b50cab80 1 -- 172.21.15.2:0/376039626 wait complete. 2023-09-13T04:46:54.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.301+0000 7fb3b50cab80 20 client.26444 trim_cache size 0 max 16384 2023-09-13T04:46:54.303 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:46:54.303 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "scrub active (3 inodes in the stack)", 2023-09-13T04:46:54.303 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-09-13T04:46:54.303 INFO:teuthology.orchestra.run.smithi002.stdout: "740235c0-6e1b-4729-b1fe-33345ae948c1": { 2023-09-13T04:46:54.304 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-09-13T04:46:54.304 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "740235c0-6e1b-4729-b1fe-33345ae948c1", 2023-09-13T04:46:54.304 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-09-13T04:46:54.304 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:46:54.304 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:46:54.305 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:46:54.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.303+0000 7fb3a17ff700 1 -- 172.21.15.2:0/3045133376 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fb3840a0980 msgr2=0x7fb3840a2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:54.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.303+0000 7fb3a17ff700 1 --2- 172.21.15.2:0/3045133376 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fb3840a0980 0x7fb3840a2e40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb39c107b60 tx=0x7fb39000a770 comp rx=0 tx=0).stop 2023-09-13T04:46:54.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.303+0000 7fb3a17ff700 1 -- 172.21.15.2:0/3045133376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb39c1068b0 msgr2=0x7fb39c106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:54.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.303+0000 7fb3a17ff700 1 --2- 172.21.15.2:0/3045133376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb39c1068b0 0x7fb39c106d60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb39400b830 tx=0x7fb394007b60 comp rx=0 tx=0).stop 2023-09-13T04:46:54.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.303+0000 7fb3a17ff700 1 -- 172.21.15.2:0/3045133376 shutdown_connections 2023-09-13T04:46:54.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.303+0000 7fb3a17ff700 1 --2- 172.21.15.2:0/3045133376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb39c1068b0 0x7fb39c106d60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.303+0000 7fb3a17ff700 1 --2- 172.21.15.2:0/3045133376 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fb3840a0980 0x7fb3840a2e40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.303+0000 7fb3a17ff700 1 --2- 172.21.15.2:0/3045133376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb39c067160 0x7fb39c109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.303+0000 7fb3a17ff700 1 --2- 172.21.15.2:0/3045133376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb39c0ff120 0x7fb39c106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:54.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.303+0000 7fb3a17ff700 1 -- 172.21.15.2:0/3045133376 >> 172.21.15.2:0/3045133376 conn(0x7fb39c075c30 msgr2=0x7fb39c101570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:54.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.306+0000 7fb3a17ff700 1 -- 172.21.15.2:0/3045133376 shutdown_connections 2023-09-13T04:46:54.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:46:54.307+0000 7fb3a17ff700 1 -- 172.21.15.2:0/3045133376 wait complete. 2023-09-13T04:46:54.328 INFO:tasks.cephfs.filesystem:scrub status for tag:740235c0-6e1b-4729-b1fe-33345ae948c1 - {'path': '/', 'tag': '740235c0-6e1b-4729-b1fe-33345ae948c1', 'options': 'recursive,force'} 2023-09-13T04:46:56.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:55 smithi149 ceph-mon[133315]: pgmap v197: 129 pgs: 129 active+clean; 796 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 534 op/s 2023-09-13T04:46:56.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:55 smithi156 ceph-mon[133675]: pgmap v197: 129 pgs: 129 active+clean; 796 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 534 op/s 2023-09-13T04:46:56.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:55 smithi002 ceph-mon[97651]: pgmap v197: 129 pgs: 129 active+clean; 796 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 534 op/s 2023-09-13T04:46:58.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:57 smithi149 ceph-mon[133315]: pgmap v198: 129 pgs: 129 active+clean; 797 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 264 op/s 2023-09-13T04:46:58.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:57 smithi156 ceph-mon[133675]: pgmap v198: 129 pgs: 129 active+clean; 797 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 264 op/s 2023-09-13T04:46:58.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:57 smithi002 ceph-mon[97651]: pgmap v198: 129 pgs: 129 active+clean; 797 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 264 op/s 2023-09-13T04:46:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.189+0000 7f0feb7ff700 1 -- 172.21.15.2:0/4294881573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fe4101880 msgr2=0x7f0fe41058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.189+0000 7f0feb7ff700 1 --2- 172.21.15.2:0/4294881573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fe4101880 0x7f0fe41058c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4009990 tx=0x7f0fd4009ca0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.189+0000 7f0feb7ff700 1 -- 172.21.15.2:0/4294881573 shutdown_connections 2023-09-13T04:46:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.189+0000 7f0feb7ff700 1 --2- 172.21.15.2:0/4294881573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fe4105ff0 0x7f0fe410db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.189+0000 7f0feb7ff700 1 --2- 172.21.15.2:0/4294881573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fe4100ed0 0x7f0fe41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.189+0000 7f0feb7ff700 1 --2- 172.21.15.2:0/4294881573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fe4101880 0x7f0fe41058c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.189+0000 7f0feb7ff700 1 -- 172.21.15.2:0/4294881573 >> 172.21.15.2:0/4294881573 conn(0x7f0fe40fc750 msgr2=0x7f0fe40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.190+0000 7f0feb7ff700 1 -- 172.21.15.2:0/4294881573 shutdown_connections 2023-09-13T04:46:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.190+0000 7f0feb7ff700 1 -- 172.21.15.2:0/4294881573 wait complete. 2023-09-13T04:46:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.191+0000 7f0feb7ff700 1 Processor -- start 2023-09-13T04:46:59.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.191+0000 7f0feb7ff700 1 -- start start 2023-09-13T04:46:59.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.192+0000 7f0feb7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fe4100ed0 0x7f0fe4196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:59.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.192+0000 7f0feb7ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fe4105ff0 0x7f0fe4197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:59.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.192+0000 7f0feb7ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fe419b990 0x7f0fe419be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.192+0000 7f0feb7ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe419c410 con 0x7f0fe4105ff0 2023-09-13T04:46:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.192+0000 7f0feb7ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe419c580 con 0x7f0fe419b990 2023-09-13T04:46:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.192+0000 7f0feb7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe419c740 con 0x7f0fe4100ed0 2023-09-13T04:46:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.192+0000 7f0fe959b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fe4100ed0 0x7f0fe4196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.192+0000 7f0fe959b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fe4100ed0 0x7f0fe4196de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:60402/0 (socket says 172.21.15.2:60402) 2023-09-13T04:46:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.192+0000 7f0fe959b700 1 -- 172.21.15.2:0/954594225 learned_addr learned my addr 172.21.15.2:0/954594225 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.192+0000 7f0fe8d9a700 1 --2- 172.21.15.2:0/954594225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fe4105ff0 0x7f0fe4197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.192+0000 7f0fe9d9c700 1 --2- 172.21.15.2:0/954594225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fe419b990 0x7f0fe419be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.193+0000 7f0fe959b700 1 -- 172.21.15.2:0/954594225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fe419b990 msgr2=0x7f0fe419be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.193+0000 7f0fe959b700 1 --2- 172.21.15.2:0/954594225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fe419b990 0x7f0fe419be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.193+0000 7f0fe959b700 1 -- 172.21.15.2:0/954594225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fe4105ff0 msgr2=0x7f0fe4197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.193+0000 7f0fe959b700 1 --2- 172.21.15.2:0/954594225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fe4105ff0 0x7f0fe4197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.193+0000 7f0fe959b700 1 -- 172.21.15.2:0/954594225 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd40095f0 con 0x7f0fe4100ed0 2023-09-13T04:46:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.193+0000 7f0fe9d9c700 1 --2- 172.21.15.2:0/954594225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fe419b990 0x7f0fe419be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:46:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.193+0000 7f0fe959b700 1 --2- 172.21.15.2:0/954594225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fe4100ed0 0x7f0fe4196de0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000e9a0 tx=0x7f0fe000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.193+0000 7f0fda7fc700 1 -- 172.21.15.2:0/954594225 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0009790 con 0x7f0fe4100ed0 2023-09-13T04:46:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.193+0000 7f0feb7ff700 1 -- 172.21.15.2:0/954594225 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe41a3ca0 con 0x7f0fe4100ed0 2023-09-13T04:46:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.193+0000 7f0fda7fc700 1 -- 172.21.15.2:0/954594225 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fe000cbd0 con 0x7f0fe4100ed0 2023-09-13T04:46:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.193+0000 7f0fda7fc700 1 -- 172.21.15.2:0/954594225 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0018680 con 0x7f0fe4100ed0 2023-09-13T04:46:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.193+0000 7f0feb7ff700 1 -- 172.21.15.2:0/954594225 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe41a41d0 con 0x7f0fe4100ed0 2023-09-13T04:46:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.198+0000 7f0fff0a1b80 1 Processor -- start 2023-09-13T04:46:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.198+0000 7f0fff0a1b80 1 -- start start 2023-09-13T04:46:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.199+0000 7f0fda7fc700 1 -- 172.21.15.2:0/954594225 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f0fe000cd40 con 0x7f0fe4100ed0 2023-09-13T04:46:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.199+0000 7f0fff0a1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b702c0f160 0x55b702c0f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.199+0000 7f0fff0a1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b702c0fa80 0x55b702c09710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.199+0000 7f0fff0a1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b702c09ca0 0x55b702c0a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.199+0000 7f0fff0a1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b702b5e720 con 0x55b702c0fa80 2023-09-13T04:46:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.199+0000 7f0fff0a1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b702b5c040 con 0x55b702c09ca0 2023-09-13T04:46:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.199+0000 7f0fff0a1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b702b57930 con 0x55b702c0f160 2023-09-13T04:46:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.199+0000 7f0fe9d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b702c0f160 0x55b702c0f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.199+0000 7f0fe9d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b702c0f160 0x55b702c0f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:60412/0 (socket says 172.21.15.2:60412) 2023-09-13T04:46:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.199+0000 7f0fe9d9c700 1 -- 172.21.15.2:0/3323080628 learned_addr learned my addr 172.21.15.2:0/3323080628 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.199+0000 7f0fe8d9a700 1 --2- 172.21.15.2:0/3323080628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b702c09ca0 0x55b702c0a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.199+0000 7f0fe959b700 1 --2- 172.21.15.2:0/3323080628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b702c0fa80 0x55b702c09710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.200+0000 7f0fe9d9c700 1 -- 172.21.15.2:0/3323080628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b702c09ca0 msgr2=0x55b702c0a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.200+0000 7f0fe9d9c700 1 --2- 172.21.15.2:0/3323080628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b702c09ca0 0x55b702c0a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.200+0000 7f0fe9d9c700 1 -- 172.21.15.2:0/3323080628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b702c0fa80 msgr2=0x55b702c09710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.200+0000 7f0fe9d9c700 1 --2- 172.21.15.2:0/3323080628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b702c0fa80 0x55b702c09710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.200+0000 7f0fe9d9c700 1 -- 172.21.15.2:0/3323080628 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7029d75d0 con 0x55b702c0f160 2023-09-13T04:46:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.200+0000 7f0fe9d9c700 1 --2- 172.21.15.2:0/3323080628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b702c0f160 0x55b702c0f540 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009440 tx=0x7f0fdc009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47ce38cd48aea282 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.200+0000 7f0fcf7fe700 1 -- 172.21.15.2:0/3323080628 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc00e070 con 0x55b702c0f160 2023-09-13T04:46:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.200+0000 7f0fcf7fe700 1 -- 172.21.15.2:0/3323080628 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fdc00f040 con 0x55b702c0f160 2023-09-13T04:46:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.200+0000 7f0fcf7fe700 1 -- 172.21.15.2:0/3323080628 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc015c90 con 0x55b702c0f160 2023-09-13T04:46:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.201+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/3323080628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b702c0f160 msgr2=0x55b702c0f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.201+0000 7f0fff0a1b80 1 --2- 172.21.15.2:0/3323080628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b702c0f160 0x55b702c0f540 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009440 tx=0x7f0fdc009920 comp rx=0 tx=0).stop 2023-09-13T04:46:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.201+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/3323080628 shutdown_connections 2023-09-13T04:46:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.201+0000 7f0fff0a1b80 1 --2- 172.21.15.2:0/3323080628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b702c09ca0 0x55b702c0a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.201+0000 7f0fff0a1b80 1 --2- 172.21.15.2:0/3323080628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b702c0f160 0x55b702c0f540 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.201+0000 7f0fff0a1b80 1 --2- 172.21.15.2:0/3323080628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b702c0fa80 0x55b702c09710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.201+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/3323080628 >> 172.21.15.2:0/3323080628 conn(0x55b702b63c20 msgr2=0x55b702bffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.201+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/3323080628 shutdown_connections 2023-09-13T04:46:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.201+0000 7f0fda7fc700 1 --2- 172.21.15.2:0/954594225 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f0fd00a07b0 0x7f0fd00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.202+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/3323080628 wait complete. 2023-09-13T04:46:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.202+0000 7f0fda7fc700 1 -- 172.21.15.2:0/954594225 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f0fe0014070 con 0x7f0fe4100ed0 2023-09-13T04:46:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.202+0000 7f0fe8d9a700 1 --2- 172.21.15.2:0/954594225 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f0fd00a07b0 0x7f0fd00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.202+0000 7f0fff0a1b80 1 -- start start 2023-09-13T04:46:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.203+0000 7f0fe8d9a700 1 --2- 172.21.15.2:0/954594225 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f0fd00a07b0 0x7f0fd00a2c70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400b2e0 tx=0x7f0fd4005960 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.203+0000 7f0fff0a1b80 1 Processor -- start 2023-09-13T04:46:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.203+0000 7f0fff0a1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b702c09ca0 0x55b702c093a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.203+0000 7f0fff0a1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b702c0f160 0x55b702c02730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.203+0000 7f0fff0a1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b702c0fa80 0x55b702c02c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.204+0000 7f0fff0a1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b702af5b50 con 0x55b702c0f160 2023-09-13T04:46:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.204+0000 7f0fff0a1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b702af5850 con 0x55b702c0fa80 2023-09-13T04:46:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.204+0000 7f0fff0a1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7029d98c0 con 0x55b702c09ca0 2023-09-13T04:46:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.204+0000 7f0fff0a1b80 10 client.0 _ll_register_callbacks cb 0x55b702af3620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:46:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.204+0000 7f0fe8d9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b702c0f160 0x55b702c02730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.204+0000 7f0fe8d9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b702c0f160 0x55b702c02730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54298/0 (socket says 172.21.15.2:54298) 2023-09-13T04:46:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.204+0000 7f0fe8d9a700 1 -- 172.21.15.2:0/1192220181 learned_addr learned my addr 172.21.15.2:0/1192220181 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.204+0000 7f0fe959b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b702c09ca0 0x55b702c093a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.204+0000 7f0fe9d9c700 1 --2- 172.21.15.2:0/1192220181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b702c0fa80 0x55b702c02c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.204+0000 7f0fe959b700 1 -- 172.21.15.2:0/1192220181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b702c0fa80 msgr2=0x55b702c02c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.204+0000 7f0fe959b700 1 --2- 172.21.15.2:0/1192220181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b702c0fa80 0x55b702c02c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.204+0000 7f0fe959b700 1 -- 172.21.15.2:0/1192220181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b702c0f160 msgr2=0x55b702c02730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.204+0000 7f0fe959b700 1 --2- 172.21.15.2:0/1192220181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b702c0f160 0x55b702c02730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.204+0000 7f0fe959b700 1 -- 172.21.15.2:0/1192220181 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fdc0090f0 con 0x55b702c09ca0 2023-09-13T04:46:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.204+0000 7f0fe8d9a700 1 --2- 172.21.15.2:0/1192220181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b702c0f160 0x55b702c02730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:46:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.205+0000 7f0fe959b700 1 --2- 172.21.15.2:0/1192220181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b702c09ca0 0x55b702c093a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0fe00c9950 tx=0x7f0fe0099500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb2dc168253580f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.205+0000 7f0fccff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:46:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.205+0000 7f0fccff9700 1 -- 172.21.15.2:0/1192220181 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe00842f0 con 0x55b702c09ca0 2023-09-13T04:46:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.205+0000 7f0fccff9700 1 -- 172.21.15.2:0/1192220181 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f0fe0099db0 con 0x55b702c09ca0 2023-09-13T04:46:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.205+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/1192220181 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b702bcaef0 con 0x55b702c09ca0 2023-09-13T04:46:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.205+0000 7f0fccff9700 1 -- 172.21.15.2:0/1192220181 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fe00cc3b0 con 0x55b702c09ca0 2023-09-13T04:46:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.205+0000 7f0fff0a1b80 10 client.26476 fetch_fsmap learned FSMap version 35 2023-09-13T04:46:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.205+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/1192220181 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b702b6f740 con 0x55b702c09ca0 2023-09-13T04:46:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.205+0000 7f0fccff9700 1 -- 172.21.15.2:0/1192220181 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b702bcaef0 con 0x55b702c09ca0 2023-09-13T04:46:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.206+0000 7f0fccff9700 1 -- 172.21.15.2:0/1192220181 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 35) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f0fe009d1a0 con 0x55b702c09ca0 2023-09-13T04:46:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.206+0000 7f0fff0a1b80 10 client.26476 fetch_fsmap finished waiting for FSMap version 35 2023-09-13T04:46:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.206+0000 7f0fff0a1b80 10 client.26476 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:46:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.206+0000 7f0fff0a1b80 20 client.26476 populate_metadata read hostname 'smithi002' 2023-09-13T04:46:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.206+0000 7f0fff0a1b80 1 --2- 172.21.15.2:0/1192220181 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55b702c27800 0x55b702c29c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.206+0000 7f0fff0a1b80 4 client.26476 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:46:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.206+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/1192220181 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b702b50910 con 0x55b702c27800 2023-09-13T04:46:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.206+0000 7f0fe8d9a700 1 --2- 172.21.15.2:0/1192220181 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55b702c27800 0x55b702c29c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.207+0000 7f0fe8d9a700 1 --2- 172.21.15.2:0/1192220181 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55b702c27800 0x55b702c29c20 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=290d483c29ce42c0 server_cookie=aa57c087a4e5b996 in_seq=0 out_seq=0 2023-09-13T04:46:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.207+0000 7f0fccff9700 10 client.26476 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:46:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.208+0000 7f0fccff9700 1 -- 172.21.15.2:0/1192220181 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b702b50910 con 0x55b702c27800 2023-09-13T04:46:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.208+0000 7f0fccff9700 10 client.26476 handle_command_reply: tid=0 2023-09-13T04:46:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.208+0000 7f0fff0a1b80 1 client.26476 shutdown 2023-09-13T04:46:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.208+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/1192220181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b702c09ca0 msgr2=0x55b702c093a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.208+0000 7f0fff0a1b80 1 --2- 172.21.15.2:0/1192220181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b702c09ca0 0x55b702c093a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0fe00c9950 tx=0x7f0fe0099500 comp rx=0 tx=0).stop 2023-09-13T04:46:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.208+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/1192220181 shutdown_connections 2023-09-13T04:46:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.208+0000 7f0fff0a1b80 1 --2- 172.21.15.2:0/1192220181 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55b702c27800 0x55b702c29c20 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.208+0000 7f0fff0a1b80 1 --2- 172.21.15.2:0/1192220181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b702c0fa80 0x55b702c02c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.208+0000 7f0fff0a1b80 1 --2- 172.21.15.2:0/1192220181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b702c09ca0 0x55b702c093a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.208+0000 7f0fccff9700 0 client.26476 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:46:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.208+0000 7f0fff0a1b80 1 --2- 172.21.15.2:0/1192220181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b702c0f160 0x55b702c02730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.208+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/1192220181 >> 172.21.15.2:0/1192220181 conn(0x55b702b63c20 msgr2=0x55b702bfa630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.208+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/1192220181 shutdown_connections 2023-09-13T04:46:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.208+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/1192220181 wait complete. 2023-09-13T04:46:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.209+0000 7f0fff0a1b80 20 client.26476 trim_cache size 0 max 16384 2023-09-13T04:46:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.224+0000 7f0fff0a1b80 1 Processor -- start 2023-09-13T04:46:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.224+0000 7f0fff0a1b80 1 -- start start 2023-09-13T04:46:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.225+0000 7f0fff0a1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b702c09ca0 0x55b702c02b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.225+0000 7f0fff0a1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b702c0f160 0x55b702bfa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.225+0000 7f0fff0a1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b702c0fa80 0x55b702c0db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.225+0000 7f0fff0a1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b702c0eda0 con 0x55b702c09ca0 2023-09-13T04:46:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.225+0000 7f0fff0a1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b702c036d0 con 0x55b702c0f160 2023-09-13T04:46:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.225+0000 7f0fff0a1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b702c147c0 con 0x55b702c0fa80 2023-09-13T04:46:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.225+0000 7f0fe959b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b702c09ca0 0x55b702c02b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.225+0000 7f0fe8d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b702c0f160 0x55b702bfa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.225+0000 7f0fe8d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b702c0f160 0x55b702bfa730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:53128/0 (socket says 172.21.15.2:53128) 2023-09-13T04:46:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.225+0000 7f0fe8d9a700 1 -- 172.21.15.2:0/2475277581 learned_addr learned my addr 172.21.15.2:0/2475277581 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.225+0000 7f0fe9d9c700 1 --2- 172.21.15.2:0/2475277581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b702c0fa80 0x55b702c0db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.225+0000 7f0fe8d9a700 1 -- 172.21.15.2:0/2475277581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b702c0fa80 msgr2=0x55b702c0db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.225+0000 7f0fe8d9a700 1 --2- 172.21.15.2:0/2475277581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b702c0fa80 0x55b702c0db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.225+0000 7f0fe8d9a700 1 -- 172.21.15.2:0/2475277581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b702c09ca0 msgr2=0x55b702c02b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.225+0000 7f0fe8d9a700 1 --2- 172.21.15.2:0/2475277581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b702c09ca0 0x55b702c02b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.225+0000 7f0fe8d9a700 1 -- 172.21.15.2:0/2475277581 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b702c00110 con 0x55b702c0f160 2023-09-13T04:46:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.226+0000 7f0fe8d9a700 1 --2- 172.21.15.2:0/2475277581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b702c0f160 0x55b702bfa730 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4006010 tx=0x7f0fd401a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab27330188713faf server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.226+0000 7f0fcffff700 1 -- 172.21.15.2:0/2475277581 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4004bc0 con 0x55b702c0f160 2023-09-13T04:46:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.226+0000 7f0fcffff700 1 -- 172.21.15.2:0/2475277581 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fd401e040 con 0x55b702c0f160 2023-09-13T04:46:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.226+0000 7f0fcffff700 1 -- 172.21.15.2:0/2475277581 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4026460 con 0x55b702c0f160 2023-09-13T04:46:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.227+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/2475277581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b702c0f160 msgr2=0x55b702bfa730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.227+0000 7f0fff0a1b80 1 --2- 172.21.15.2:0/2475277581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b702c0f160 0x55b702bfa730 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4006010 tx=0x7f0fd401a050 comp rx=0 tx=0).stop 2023-09-13T04:46:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.227+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/2475277581 shutdown_connections 2023-09-13T04:46:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.227+0000 7f0fff0a1b80 1 --2- 172.21.15.2:0/2475277581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b702c0fa80 0x55b702c0db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.227+0000 7f0fff0a1b80 1 --2- 172.21.15.2:0/2475277581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b702c09ca0 0x55b702c02b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.227+0000 7f0fff0a1b80 1 --2- 172.21.15.2:0/2475277581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b702c0f160 0x55b702bfa730 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.227+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/2475277581 >> 172.21.15.2:0/2475277581 conn(0x55b702b63c20 msgr2=0x55b702c0ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.227+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/2475277581 shutdown_connections 2023-09-13T04:46:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.227+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/2475277581 wait complete. 2023-09-13T04:46:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.227+0000 7f0fff0a1b80 1 -- start start 2023-09-13T04:46:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.228+0000 7f0fff0a1b80 1 Processor -- start 2023-09-13T04:46:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.228+0000 7f0fff0a1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b702c09ca0 0x55b702c23e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.229+0000 7f0fff0a1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b702c0f160 0x55b702c243c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.229+0000 7f0fff0a1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b702c0fa80 0x55b702c124e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.229+0000 7f0fff0a1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b702c12a70 con 0x55b702c09ca0 2023-09-13T04:46:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.229+0000 7f0fff0a1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b702c12be0 con 0x55b702c0fa80 2023-09-13T04:46:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.229+0000 7f0fff0a1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b702c12da0 con 0x55b702c0f160 2023-09-13T04:46:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.229+0000 7f0fff0a1b80 10 client.0 _ll_register_callbacks cb 0x55b702af3620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:46:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.229+0000 7f0fe959b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b702c09ca0 0x55b702c23e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.229+0000 7f0fe959b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b702c09ca0 0x55b702c23e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54316/0 (socket says 172.21.15.2:54316) 2023-09-13T04:46:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.229+0000 7f0fe959b700 1 -- 172.21.15.2:0/1197424507 learned_addr learned my addr 172.21.15.2:0/1197424507 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:46:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.229+0000 7f0fe9d9c700 1 --2- 172.21.15.2:0/1197424507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b702c0fa80 0x55b702c124e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.229+0000 7f0fe8d9a700 1 --2- 172.21.15.2:0/1197424507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b702c0f160 0x55b702c243c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.229+0000 7f0fe959b700 1 -- 172.21.15.2:0/1197424507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b702c0f160 msgr2=0x55b702c243c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.229+0000 7f0fe959b700 1 --2- 172.21.15.2:0/1197424507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b702c0f160 0x55b702c243c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.229+0000 7f0fe959b700 1 -- 172.21.15.2:0/1197424507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b702c0fa80 msgr2=0x55b702c124e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:46:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.229+0000 7f0fe959b700 1 --2- 172.21.15.2:0/1197424507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b702c0fa80 0x55b702c124e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.229+0000 7f0fe959b700 1 -- 172.21.15.2:0/1197424507 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe410e3e0 con 0x55b702c09ca0 2023-09-13T04:46:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.229+0000 7f0fe9d9c700 1 --2- 172.21.15.2:0/1197424507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b702c0fa80 0x55b702c124e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:46:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.230+0000 7f0fe959b700 1 --2- 172.21.15.2:0/1197424507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b702c09ca0 0x55b702c23e80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0096120 tx=0x7f0fe0084320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84cf2d2668356b5 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:46:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.230+0000 7f0fccff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:46:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.230+0000 7f0fccff9700 1 -- 172.21.15.2:0/1197424507 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe00cfb50 con 0x55b702c09ca0 2023-09-13T04:46:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.230+0000 7f0fccff9700 1 -- 172.21.15.2:0/1197424507 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f0fe0099db0 con 0x55b702c09ca0 2023-09-13T04:46:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.230+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/1197424507 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0fdc00f040 con 0x55b702c09ca0 2023-09-13T04:46:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.230+0000 7f0fccff9700 1 -- 172.21.15.2:0/1197424507 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fe00991f0 con 0x55b702c09ca0 2023-09-13T04:46:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.230+0000 7f0fff0a1b80 10 client.16317 fetch_fsmap learned FSMap version 35 2023-09-13T04:46:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.230+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/1197424507 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b702983d80 con 0x55b702c09ca0 2023-09-13T04:46:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.231+0000 7f0fccff9700 1 -- 172.21.15.2:0/1197424507 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fdc00f040 con 0x55b702c09ca0 2023-09-13T04:46:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.231+0000 7f0fccff9700 1 -- 172.21.15.2:0/1197424507 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 35) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f0fe0098ba0 con 0x55b702c09ca0 2023-09-13T04:46:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.231+0000 7f0fff0a1b80 10 client.16317 fetch_fsmap finished waiting for FSMap version 35 2023-09-13T04:46:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.231+0000 7f0fff0a1b80 10 client.16317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:46:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.231+0000 7f0fff0a1b80 20 client.16317 populate_metadata read hostname 'smithi002' 2023-09-13T04:46:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.231+0000 7f0fff0a1b80 1 --2- 172.21.15.2:0/1197424507 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55b702c7c610 0x55b702c7ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:46:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.231+0000 7f0fff0a1b80 4 client.16317 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:46:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.231+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/1197424507 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b702b10ff0 con 0x55b702c7c610 2023-09-13T04:46:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.231+0000 7f0fe8d9a700 1 --2- 172.21.15.2:0/1197424507 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55b702c7c610 0x55b702c7ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:46:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.232+0000 7f0fccff9700 10 client.16317 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:46:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.232+0000 7f0fe8d9a700 1 --2- 172.21.15.2:0/1197424507 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55b702c7c610 0x55b702c7ea00 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ad48d5785c99cb server_cookie=e2eb0247da1e840 in_seq=0 out_seq=0 2023-09-13T04:46:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.647+0000 7f0fccff9700 1 -- 172.21.15.2:0/1197424507 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b702b10ff0 con 0x55b702c7c610 2023-09-13T04:46:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.647+0000 7f0fccff9700 10 client.16317 handle_command_reply: tid=0 2023-09-13T04:46:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.647+0000 7f0fff0a1b80 1 client.16317 shutdown 2023-09-13T04:46:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.648+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/1197424507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b702c09ca0 msgr2=0x55b702c23e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.648+0000 7f0fff0a1b80 1 --2- 172.21.15.2:0/1197424507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b702c09ca0 0x55b702c23e80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0096120 tx=0x7f0fe0084320 comp rx=0 tx=0).stop 2023-09-13T04:46:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.648+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/1197424507 shutdown_connections 2023-09-13T04:46:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.648+0000 7f0fff0a1b80 1 --2- 172.21.15.2:0/1197424507 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55b702c7c610 0x55b702c7ea00 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.648+0000 7f0fff0a1b80 1 --2- 172.21.15.2:0/1197424507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b702c0fa80 0x55b702c124e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.648+0000 7f0fff0a1b80 1 --2- 172.21.15.2:0/1197424507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b702c09ca0 0x55b702c23e80 secure :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0096120 tx=0x7f0fe0084320 comp rx=0 tx=0).stop 2023-09-13T04:46:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.648+0000 7f0fff0a1b80 1 --2- 172.21.15.2:0/1197424507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b702c0f160 0x55b702c243c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.648+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/1197424507 >> 172.21.15.2:0/1197424507 conn(0x55b702b63c20 msgr2=0x55b702c029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.648+0000 7f0fccff9700 0 client.16317 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:46:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.651+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/1197424507 shutdown_connections 2023-09-13T04:46:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.651+0000 7f0fff0a1b80 1 -- 172.21.15.2:0/1197424507 wait complete. 2023-09-13T04:46:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.652+0000 7f0fff0a1b80 20 client.16317 trim_cache size 0 max 16384 2023-09-13T04:46:59.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:46:59.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:46:59.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:46:59.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:46:59.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.656+0000 7f0feb7ff700 1 -- 172.21.15.2:0/954594225 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f0fd00a07b0 msgr2=0x7f0fd00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:59.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.656+0000 7f0feb7ff700 1 --2- 172.21.15.2:0/954594225 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f0fd00a07b0 0x7f0fd00a2c70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400b2e0 tx=0x7f0fd4005960 comp rx=0 tx=0).stop 2023-09-13T04:46:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.656+0000 7f0feb7ff700 1 -- 172.21.15.2:0/954594225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fe4100ed0 msgr2=0x7f0fe4196de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:46:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.656+0000 7f0feb7ff700 1 --2- 172.21.15.2:0/954594225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fe4100ed0 0x7f0fe4196de0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000e9a0 tx=0x7f0fe000ecb0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.656+0000 7f0feb7ff700 1 -- 172.21.15.2:0/954594225 shutdown_connections 2023-09-13T04:46:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.656+0000 7f0feb7ff700 1 --2- 172.21.15.2:0/954594225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fe419b990 0x7f0fe419be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.656+0000 7f0feb7ff700 1 --2- 172.21.15.2:0/954594225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fe4100ed0 0x7f0fe4196de0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.656+0000 7f0feb7ff700 1 --2- 172.21.15.2:0/954594225 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f0fd00a07b0 0x7f0fd00a2c70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.656+0000 7f0feb7ff700 1 --2- 172.21.15.2:0/954594225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fe4105ff0 0x7f0fe4197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:46:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.656+0000 7f0feb7ff700 1 -- 172.21.15.2:0/954594225 >> 172.21.15.2:0/954594225 conn(0x7f0fe40fc750 msgr2=0x7f0fe40fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:46:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.660+0000 7f0feb7ff700 1 -- 172.21.15.2:0/954594225 shutdown_connections 2023-09-13T04:46:59.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:46:59.664+0000 7f0feb7ff700 1 -- 172.21.15.2:0/954594225 wait complete. 2023-09-13T04:47:00.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:46:59 smithi149 ceph-mon[133315]: pgmap v199: 129 pgs: 129 active+clean; 797 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 241 op/s 2023-09-13T04:47:00.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:46:59 smithi156 ceph-mon[133675]: pgmap v199: 129 pgs: 129 active+clean; 797 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 241 op/s 2023-09-13T04:47:00.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:46:59 smithi002 ceph-mon[97651]: pgmap v199: 129 pgs: 129 active+clean; 797 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 241 op/s 2023-09-13T04:47:02.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:01 smithi149 ceph-mon[133315]: pgmap v200: 129 pgs: 129 active+clean; 803 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 340 op/s 2023-09-13T04:47:02.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:01 smithi156 ceph-mon[133675]: pgmap v200: 129 pgs: 129 active+clean; 803 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 340 op/s 2023-09-13T04:47:02.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:01 smithi002 ceph-mon[97651]: pgmap v200: 129 pgs: 129 active+clean; 803 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 340 op/s 2023-09-13T04:47:04.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:03 smithi149 ceph-mon[133315]: pgmap v201: 129 pgs: 129 active+clean; 804 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 290 op/s 2023-09-13T04:47:04.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:03 smithi156 ceph-mon[133675]: pgmap v201: 129 pgs: 129 active+clean; 804 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 290 op/s 2023-09-13T04:47:04.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:03 smithi002 ceph-mon[97651]: pgmap v201: 129 pgs: 129 active+clean; 804 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 290 op/s 2023-09-13T04:47:05.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:04 smithi149 ceph-mon[133315]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:47:05.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:04 smithi156 ceph-mon[133675]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:47:05.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:04 smithi002 ceph-mon[97651]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:47:06.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:05 smithi149 ceph-mon[133315]: pgmap v202: 129 pgs: 129 active+clean; 819 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 597 op/s 2023-09-13T04:47:06.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:05 smithi156 ceph-mon[133675]: pgmap v202: 129 pgs: 129 active+clean; 819 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 597 op/s 2023-09-13T04:47:06.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:05 smithi002 ceph-mon[97651]: pgmap v202: 129 pgs: 129 active+clean; 819 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 597 op/s 2023-09-13T04:47:08.202 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:07 smithi156 ceph-mon[133675]: pgmap v203: 129 pgs: 129 active+clean; 823 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 566 op/s 2023-09-13T04:47:08.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:07 smithi002 ceph-mon[97651]: pgmap v203: 129 pgs: 129 active+clean; 823 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 566 op/s 2023-09-13T04:47:08.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:07 smithi149 ceph-mon[133315]: pgmap v203: 129 pgs: 129 active+clean; 823 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 566 op/s 2023-09-13T04:47:10.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:09 smithi156 ceph-mon[133675]: pgmap v204: 129 pgs: 129 active+clean; 837 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 599 op/s 2023-09-13T04:47:10.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:09 smithi156 ceph-mon[133675]: osdmap e95: 12 total, 12 up, 12 in 2023-09-13T04:47:10.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:09 smithi002 ceph-mon[97651]: pgmap v204: 129 pgs: 129 active+clean; 837 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 599 op/s 2023-09-13T04:47:10.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:09 smithi002 ceph-mon[97651]: osdmap e95: 12 total, 12 up, 12 in 2023-09-13T04:47:10.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:09 smithi149 ceph-mon[133315]: pgmap v204: 129 pgs: 129 active+clean; 837 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 599 op/s 2023-09-13T04:47:10.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:09 smithi149 ceph-mon[133315]: osdmap e95: 12 total, 12 up, 12 in 2023-09-13T04:47:11.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:10 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:47:11.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:10 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:47:11.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:10 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:47:12.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:11 smithi156 ceph-mon[133675]: pgmap v206: 129 pgs: 129 active+clean; 852 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.0 MiB/s wr, 693 op/s 2023-09-13T04:47:12.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:11 smithi002 ceph-mon[97651]: pgmap v206: 129 pgs: 129 active+clean; 852 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.0 MiB/s wr, 693 op/s 2023-09-13T04:47:12.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:11 smithi149 ceph-mon[133315]: pgmap v206: 129 pgs: 129 active+clean; 852 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.0 MiB/s wr, 693 op/s 2023-09-13T04:47:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.832+0000 7fd5c51ff700 1 -- 172.21.15.2:0/857271920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5c0106d60 msgr2=0x7fd5c0107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.832+0000 7fd5c51ff700 1 --2- 172.21.15.2:0/857271920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5c0106d60 0x7fd5c0107140 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009990 tx=0x7fd5b0009ca0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.833+0000 7fd5c51ff700 1 -- 172.21.15.2:0/857271920 shutdown_connections 2023-09-13T04:47:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.833+0000 7fd5c51ff700 1 --2- 172.21.15.2:0/857271920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5c0068a90 0x7fd5c010db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.833+0000 7fd5c51ff700 1 --2- 172.21.15.2:0/857271920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5c0106d60 0x7fd5c0107140 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.833+0000 7fd5c51ff700 1 --2- 172.21.15.2:0/857271920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5c00680d0 0x7fd5c0068550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.833+0000 7fd5c51ff700 1 -- 172.21.15.2:0/857271920 >> 172.21.15.2:0/857271920 conn(0x7fd5c0076970 msgr2=0x7fd5c0076d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.833+0000 7fd5c51ff700 1 -- 172.21.15.2:0/857271920 shutdown_connections 2023-09-13T04:47:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.833+0000 7fd5c51ff700 1 -- 172.21.15.2:0/857271920 wait complete. 2023-09-13T04:47:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.834+0000 7fd5c51ff700 1 Processor -- start 2023-09-13T04:47:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.834+0000 7fd5c51ff700 1 -- start start 2023-09-13T04:47:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.835+0000 7fd5c51ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5c00680d0 0x7fd5c0100020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.835+0000 7fd5c51ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5c0068a90 0x7fd5c0100560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.835+0000 7fd5c51ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5c0106d60 0x7fd5c01a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.835+0000 7fd5c51ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c01a5450 con 0x7fd5c0068a90 2023-09-13T04:47:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.835+0000 7fd5c51ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c01a5590 con 0x7fd5c0106d60 2023-09-13T04:47:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.835+0000 7fd5c51ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c01a5700 con 0x7fd5c00680d0 2023-09-13T04:47:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.835+0000 7fd5bed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5c00680d0 0x7fd5c0100020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.835+0000 7fd5be59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5c0068a90 0x7fd5c0100560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.835+0000 7fd5bed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5c00680d0 0x7fd5c0100020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:39402/0 (socket says 172.21.15.2:39402) 2023-09-13T04:47:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.835+0000 7fd5bed9d700 1 -- 172.21.15.2:0/4265492917 learned_addr learned my addr 172.21.15.2:0/4265492917 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.835+0000 7fd5bf59e700 1 --2- 172.21.15.2:0/4265492917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5c0106d60 0x7fd5c01a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.836+0000 7fd5bed9d700 1 -- 172.21.15.2:0/4265492917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5c0106d60 msgr2=0x7fd5c01a4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.836+0000 7fd5bed9d700 1 --2- 172.21.15.2:0/4265492917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5c0106d60 0x7fd5c01a4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.836+0000 7fd5bed9d700 1 -- 172.21.15.2:0/4265492917 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5c0068a90 msgr2=0x7fd5c0100560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.836+0000 7fd5bed9d700 1 --2- 172.21.15.2:0/4265492917 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5c0068a90 0x7fd5c0100560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.836+0000 7fd5bed9d700 1 -- 172.21.15.2:0/4265492917 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b00095f0 con 0x7fd5c00680d0 2023-09-13T04:47:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.836+0000 7fd5bf59e700 1 --2- 172.21.15.2:0/4265492917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5c0106d60 0x7fd5c01a4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:47:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.836+0000 7fd5be59c700 1 --2- 172.21.15.2:0/4265492917 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5c0068a90 0x7fd5c0100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:47:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.836+0000 7fd5bed9d700 1 --2- 172.21.15.2:0/4265492917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5c00680d0 0x7fd5c0100020 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0005910 tx=0x7fd5b0005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.837+0000 7fd5a7fff700 1 -- 172.21.15.2:0/4265492917 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b001d070 con 0x7fd5c00680d0 2023-09-13T04:47:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.837+0000 7fd5a7fff700 1 -- 172.21.15.2:0/4265492917 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5b0022470 con 0x7fd5c00680d0 2023-09-13T04:47:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.837+0000 7fd5c51ff700 1 -- 172.21.15.2:0/4265492917 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c01a5980 con 0x7fd5c00680d0 2023-09-13T04:47:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.837+0000 7fd5a7fff700 1 -- 172.21.15.2:0/4265492917 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0017650 con 0x7fd5c00680d0 2023-09-13T04:47:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.837+0000 7fd5c51ff700 1 -- 172.21.15.2:0/4265492917 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c01a5da0 con 0x7fd5c00680d0 2023-09-13T04:47:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.840+0000 7fd5a7fff700 1 -- 172.21.15.2:0/4265492917 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7fd5b00225e0 con 0x7fd5c00680d0 2023-09-13T04:47:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.842+0000 7fd5a7fff700 1 --2- 172.21.15.2:0/4265492917 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fd5a80a07b0 0x7fd5a80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.842+0000 7fd5a7fff700 1 -- 172.21.15.2:0/4265492917 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9483+0+0 (secure 0 0 0) 0x7fd5b00cd7f0 con 0x7fd5c00680d0 2023-09-13T04:47:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.842+0000 7fd5be59c700 1 --2- 172.21.15.2:0/4265492917 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fd5a80a07b0 0x7fd5a80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.843+0000 7fd5be59c700 1 --2- 172.21.15.2:0/4265492917 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fd5a80a07b0 0x7fd5a80a2c70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0101840 tx=0x7fd5b4009260 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.843+0000 7fd5d8adab80 1 Processor -- start 2023-09-13T04:47:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.843+0000 7fd5d8adab80 1 -- start start 2023-09-13T04:47:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.844+0000 7fd5d8adab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd75be0 0x55bddbd75fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.844+0000 7fd5d8adab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bddbd76500 0x55bddbd71300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.844+0000 7fd5d8adab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bddbd71840 0x55bddbd71c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.844+0000 7fd5d8adab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bddbcca720 con 0x55bddbd71840 2023-09-13T04:47:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.844+0000 7fd5d8adab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bddbcc8040 con 0x55bddbd76500 2023-09-13T04:47:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.844+0000 7fd5d8adab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bddbcc3930 con 0x55bddbd75be0 2023-09-13T04:47:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.844+0000 7fd5bed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd75be0 0x55bddbd75fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.844+0000 7fd5be59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bddbd76500 0x55bddbd71300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.844+0000 7fd5bed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd75be0 0x55bddbd75fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:39414/0 (socket says 172.21.15.2:39414) 2023-09-13T04:47:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.844+0000 7fd5bed9d700 1 -- 172.21.15.2:0/2070621359 learned_addr learned my addr 172.21.15.2:0/2070621359 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.844+0000 7fd5bf59e700 1 --2- 172.21.15.2:0/2070621359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bddbd71840 0x55bddbd71c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.844+0000 7fd5bed9d700 1 -- 172.21.15.2:0/2070621359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bddbd76500 msgr2=0x55bddbd71300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.844+0000 7fd5bed9d700 1 --2- 172.21.15.2:0/2070621359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bddbd76500 0x55bddbd71300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.845+0000 7fd5bed9d700 1 -- 172.21.15.2:0/2070621359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bddbd71840 msgr2=0x55bddbd71c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.845+0000 7fd5bed9d700 1 --2- 172.21.15.2:0/2070621359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bddbd71840 0x55bddbd71c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.845+0000 7fd5bed9d700 1 -- 172.21.15.2:0/2070621359 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bddbb435d0 con 0x55bddbd75be0 2023-09-13T04:47:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.845+0000 7fd5bf59e700 1 --2- 172.21.15.2:0/2070621359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bddbd71840 0x55bddbd71c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:47:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.845+0000 7fd5be59c700 1 --2- 172.21.15.2:0/2070621359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bddbd76500 0x55bddbd71300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:47:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.845+0000 7fd5bed9d700 1 --2- 172.21.15.2:0/2070621359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd75be0 0x55bddbd75fc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd5b00a24e0 tx=0x7fd5b0009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd304d14076ae6be server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.845+0000 7fd5a57fa700 1 -- 172.21.15.2:0/2070621359 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b009a390 con 0x55bddbd75be0 2023-09-13T04:47:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.845+0000 7fd5a57fa700 1 -- 172.21.15.2:0/2070621359 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5b009b130 con 0x55bddbd75be0 2023-09-13T04:47:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.845+0000 7fd5a57fa700 1 -- 172.21.15.2:0/2070621359 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b00cd5e0 con 0x55bddbd75be0 2023-09-13T04:47:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.846+0000 7fd5d8adab80 1 -- 172.21.15.2:0/2070621359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd75be0 msgr2=0x55bddbd75fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.846+0000 7fd5d8adab80 1 --2- 172.21.15.2:0/2070621359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd75be0 0x55bddbd75fc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd5b00a24e0 tx=0x7fd5b0009ca0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.846+0000 7fd5d8adab80 1 -- 172.21.15.2:0/2070621359 shutdown_connections 2023-09-13T04:47:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.846+0000 7fd5d8adab80 1 --2- 172.21.15.2:0/2070621359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bddbd71840 0x55bddbd71c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.846+0000 7fd5d8adab80 1 --2- 172.21.15.2:0/2070621359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd75be0 0x55bddbd75fc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.846+0000 7fd5d8adab80 1 --2- 172.21.15.2:0/2070621359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bddbd76500 0x55bddbd71300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.846+0000 7fd5d8adab80 1 -- 172.21.15.2:0/2070621359 >> 172.21.15.2:0/2070621359 conn(0x55bddbccfc20 msgr2=0x55bddbd6bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.846+0000 7fd5d8adab80 1 -- 172.21.15.2:0/2070621359 shutdown_connections 2023-09-13T04:47:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.846+0000 7fd5d8adab80 1 -- 172.21.15.2:0/2070621359 wait complete. 2023-09-13T04:47:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.847+0000 7fd5d8adab80 1 -- start start 2023-09-13T04:47:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.847+0000 7fd5d8adab80 1 Processor -- start 2023-09-13T04:47:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.848+0000 7fd5d8adab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd71840 0x55bddbd7d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.848+0000 7fd5d8adab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bddbd75be0 0x55bddbd7a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.848+0000 7fd5d8adab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bddbd76500 0x55bddbd7af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.848+0000 7fd5d8adab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bddbc61b50 con 0x55bddbd76500 2023-09-13T04:47:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.848+0000 7fd5d8adab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bddbc61850 con 0x55bddbd75be0 2023-09-13T04:47:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.848+0000 7fd5d8adab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bddbb458c0 con 0x55bddbd71840 2023-09-13T04:47:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.848+0000 7fd5d8adab80 10 client.0 _ll_register_callbacks cb 0x55bddbc5f620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:47:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.848+0000 7fd5bed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd71840 0x55bddbd7d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.848+0000 7fd5be59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bddbd75be0 0x55bddbd7a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.848+0000 7fd5bed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd71840 0x55bddbd7d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:39424/0 (socket says 172.21.15.2:39424) 2023-09-13T04:47:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.848+0000 7fd5bed9d700 1 -- 172.21.15.2:0/1288279564 learned_addr learned my addr 172.21.15.2:0/1288279564 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.849+0000 7fd5bf59e700 1 --2- 172.21.15.2:0/1288279564 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bddbd76500 0x55bddbd7af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.849+0000 7fd5bed9d700 1 -- 172.21.15.2:0/1288279564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bddbd75be0 msgr2=0x55bddbd7a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.849+0000 7fd5bed9d700 1 --2- 172.21.15.2:0/1288279564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bddbd75be0 0x55bddbd7a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.849+0000 7fd5bed9d700 1 -- 172.21.15.2:0/1288279564 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bddbd76500 msgr2=0x55bddbd7af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.849+0000 7fd5bed9d700 1 --2- 172.21.15.2:0/1288279564 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bddbd76500 0x55bddbd7af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.849+0000 7fd5bed9d700 1 -- 172.21.15.2:0/1288279564 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c01a5da0 con 0x55bddbd71840 2023-09-13T04:47:12.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.849+0000 7fd5bf59e700 1 --2- 172.21.15.2:0/1288279564 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bddbd76500 0x55bddbd7af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:47:12.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.849+0000 7fd5bed9d700 1 --2- 172.21.15.2:0/1288279564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd71840 0x55bddbd7d5e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0098d80 tx=0x7fd5b0009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb8968f4067106ae server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:12.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.849+0000 7fd5a2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:47:12.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.849+0000 7fd5a2ff5700 1 -- 172.21.15.2:0/1288279564 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b009e860 con 0x55bddbd71840 2023-09-13T04:47:12.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.849+0000 7fd5a2ff5700 1 -- 172.21.15.2:0/1288279564 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9483+0+0 (secure 0 0 0) 0x7fd5b00a2a10 con 0x55bddbd71840 2023-09-13T04:47:12.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.849+0000 7fd5d8adab80 1 -- 172.21.15.2:0/1288279564 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bddbd36ef0 con 0x55bddbd71840 2023-09-13T04:47:12.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.850+0000 7fd5a2ff5700 1 -- 172.21.15.2:0/1288279564 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5b009ac90 con 0x55bddbd71840 2023-09-13T04:47:12.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.850+0000 7fd5d8adab80 10 client.26494 fetch_fsmap learned FSMap version 36 2023-09-13T04:47:12.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.850+0000 7fd5d8adab80 1 -- 172.21.15.2:0/1288279564 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55bddbcdb740 con 0x55bddbd71840 2023-09-13T04:47:12.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.850+0000 7fd5a2ff5700 1 -- 172.21.15.2:0/1288279564 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bddbd36ef0 con 0x55bddbd71840 2023-09-13T04:47:12.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.850+0000 7fd5a2ff5700 1 -- 172.21.15.2:0/1288279564 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fd5b00a2580 con 0x55bddbd71840 2023-09-13T04:47:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.850+0000 7fd5d8adab80 10 client.26494 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:47:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.850+0000 7fd5d8adab80 10 client.26494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:47:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.850+0000 7fd5d8adab80 20 client.26494 populate_metadata read hostname 'smithi002' 2023-09-13T04:47:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.850+0000 7fd5d8adab80 1 --2- 172.21.15.2:0/1288279564 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55bddbd8a7c0 0x55bddbd8cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.850+0000 7fd5d8adab80 4 client.26494 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:47:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.850+0000 7fd5d8adab80 1 -- 172.21.15.2:0/1288279564 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bddbcbc910 con 0x55bddbd8a7c0 2023-09-13T04:47:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.851+0000 7fd5be59c700 1 --2- 172.21.15.2:0/1288279564 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55bddbd8a7c0 0x55bddbd8cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.851+0000 7fd5be59c700 1 --2- 172.21.15.2:0/1288279564 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55bddbd8a7c0 0x55bddbd8cbe0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77af8b08ece58e9d server_cookie=2c7cf31f4c87d4e4 in_seq=0 out_seq=0 2023-09-13T04:47:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.851+0000 7fd5a2ff5700 10 client.26494 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:47:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.852+0000 7fd5a2ff5700 1 -- 172.21.15.2:0/1288279564 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bddbcbc910 con 0x55bddbd8a7c0 2023-09-13T04:47:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.852+0000 7fd5a2ff5700 10 client.26494 handle_command_reply: tid=0 2023-09-13T04:47:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.852+0000 7fd5d8adab80 1 client.26494 shutdown 2023-09-13T04:47:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.853+0000 7fd5d8adab80 1 -- 172.21.15.2:0/1288279564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd71840 msgr2=0x55bddbd7d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.853+0000 7fd5d8adab80 1 --2- 172.21.15.2:0/1288279564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd71840 0x55bddbd7d5e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0098d80 tx=0x7fd5b0009310 comp rx=0 tx=0).stop 2023-09-13T04:47:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.853+0000 7fd5d8adab80 1 -- 172.21.15.2:0/1288279564 shutdown_connections 2023-09-13T04:47:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.853+0000 7fd5d8adab80 1 --2- 172.21.15.2:0/1288279564 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55bddbd8a7c0 0x55bddbd8cbe0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.853+0000 7fd5d8adab80 1 --2- 172.21.15.2:0/1288279564 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bddbd76500 0x55bddbd7af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.853+0000 7fd5d8adab80 1 --2- 172.21.15.2:0/1288279564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd71840 0x55bddbd7d5e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.853+0000 7fd5d8adab80 1 --2- 172.21.15.2:0/1288279564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bddbd75be0 0x55bddbd7a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.853+0000 7fd5d8adab80 1 -- 172.21.15.2:0/1288279564 >> 172.21.15.2:0/1288279564 conn(0x55bddbccfc20 msgr2=0x55bddbd77330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.853+0000 7fd5a2ff5700 0 client.26494 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:47:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.853+0000 7fd5d8adab80 1 -- 172.21.15.2:0/1288279564 shutdown_connections 2023-09-13T04:47:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.853+0000 7fd5d8adab80 1 -- 172.21.15.2:0/1288279564 wait complete. 2023-09-13T04:47:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.853+0000 7fd5d8adab80 20 client.26494 trim_cache size 0 max 16384 2023-09-13T04:47:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.873+0000 7fd5d8adab80 1 Processor -- start 2023-09-13T04:47:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.873+0000 7fd5d8adab80 1 -- start start 2023-09-13T04:47:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.873+0000 7fd5d8adab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd71840 0x55bddbd66730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.873+0000 7fd5d8adab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bddbd75be0 0x55bddbd7d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.873+0000 7fd5d8adab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bddbd76500 0x55bddbd7a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.873+0000 7fd5d8adab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bddbd6c100 con 0x55bddbd76500 2023-09-13T04:47:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.873+0000 7fd5d8adab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bddbd6d4e0 con 0x55bddbd75be0 2023-09-13T04:47:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.873+0000 7fd5d8adab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bddbd6ca50 con 0x55bddbd71840 2023-09-13T04:47:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.874+0000 7fd5bed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd71840 0x55bddbd66730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.874+0000 7fd5be59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bddbd75be0 0x55bddbd7d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.874+0000 7fd5bed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd71840 0x55bddbd66730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:39436/0 (socket says 172.21.15.2:39436) 2023-09-13T04:47:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.874+0000 7fd5be59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bddbd75be0 0x55bddbd7d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:34834/0 (socket says 172.21.15.2:34834) 2023-09-13T04:47:12.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.874+0000 7fd5bed9d700 1 -- 172.21.15.2:0/1121601264 learned_addr learned my addr 172.21.15.2:0/1121601264 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:12.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.874+0000 7fd5bf59e700 1 --2- 172.21.15.2:0/1121601264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bddbd76500 0x55bddbd7a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:12.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.874+0000 7fd5bed9d700 1 -- 172.21.15.2:0/1121601264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bddbd75be0 msgr2=0x55bddbd7d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:12.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.874+0000 7fd5bed9d700 1 --2- 172.21.15.2:0/1121601264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bddbd75be0 0x55bddbd7d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.874+0000 7fd5bed9d700 1 -- 172.21.15.2:0/1121601264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bddbd76500 msgr2=0x55bddbd7a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.874+0000 7fd5bed9d700 1 --2- 172.21.15.2:0/1121601264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bddbd76500 0x55bddbd7a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.874+0000 7fd5bed9d700 1 -- 172.21.15.2:0/1121601264 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bddbaefd80 con 0x55bddbd71840 2023-09-13T04:47:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.874+0000 7fd5bf59e700 1 --2- 172.21.15.2:0/1121601264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bddbd76500 0x55bddbd7a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:47:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.874+0000 7fd5be59c700 1 --2- 172.21.15.2:0/1121601264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bddbd75be0 0x55bddbd7d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:47:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.874+0000 7fd5bed9d700 1 --2- 172.21.15.2:0/1121601264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd71840 0x55bddbd66730 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd5b009a730 tx=0x7fd5b00cb080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dab013f5513a401 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:12.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.875+0000 7fd5a5ffb700 1 -- 172.21.15.2:0/1121601264 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b009aa30 con 0x55bddbd71840 2023-09-13T04:47:12.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.875+0000 7fd5a5ffb700 1 -- 172.21.15.2:0/1121601264 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5b00a2a10 con 0x55bddbd71840 2023-09-13T04:47:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.875+0000 7fd5a5ffb700 1 -- 172.21.15.2:0/1121601264 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b009ac90 con 0x55bddbd71840 2023-09-13T04:47:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.876+0000 7fd5d8adab80 1 -- 172.21.15.2:0/1121601264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd71840 msgr2=0x55bddbd66730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.876+0000 7fd5d8adab80 1 --2- 172.21.15.2:0/1121601264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd71840 0x55bddbd66730 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd5b009a730 tx=0x7fd5b00cb080 comp rx=0 tx=0).stop 2023-09-13T04:47:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.876+0000 7fd5d8adab80 1 -- 172.21.15.2:0/1121601264 shutdown_connections 2023-09-13T04:47:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.876+0000 7fd5d8adab80 1 --2- 172.21.15.2:0/1121601264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bddbd76500 0x55bddbd7a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.876+0000 7fd5d8adab80 1 --2- 172.21.15.2:0/1121601264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd71840 0x55bddbd66730 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.876+0000 7fd5d8adab80 1 --2- 172.21.15.2:0/1121601264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bddbd75be0 0x55bddbd7d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.876+0000 7fd5d8adab80 1 -- 172.21.15.2:0/1121601264 >> 172.21.15.2:0/1121601264 conn(0x55bddbccfc20 msgr2=0x55bddbd6f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.877+0000 7fd5d8adab80 1 -- 172.21.15.2:0/1121601264 shutdown_connections 2023-09-13T04:47:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.877+0000 7fd5d8adab80 1 -- 172.21.15.2:0/1121601264 wait complete. 2023-09-13T04:47:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.877+0000 7fd5d8adab80 1 -- start start 2023-09-13T04:47:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.878+0000 7fd5d8adab80 1 Processor -- start 2023-09-13T04:47:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.878+0000 7fd5d8adab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd71840 0x55bddbd7fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.878+0000 7fd5d8adab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bddbd75be0 0x55bddbd80220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.878+0000 7fd5d8adab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bddbd76500 0x55bddbd80760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.878+0000 7fd5d8adab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bddbddd5f0 con 0x55bddbd75be0 2023-09-13T04:47:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.878+0000 7fd5d8adab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bddbddd730 con 0x55bddbd76500 2023-09-13T04:47:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.878+0000 7fd5d8adab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bddbddd8c0 con 0x55bddbd71840 2023-09-13T04:47:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.878+0000 7fd5d8adab80 10 client.0 _ll_register_callbacks cb 0x55bddbc5f620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:47:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.879+0000 7fd5bed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd71840 0x55bddbd7fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.879+0000 7fd5bed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd71840 0x55bddbd7fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:39450/0 (socket says 172.21.15.2:39450) 2023-09-13T04:47:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.879+0000 7fd5bed9d700 1 -- 172.21.15.2:0/3561877853 learned_addr learned my addr 172.21.15.2:0/3561877853 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.879+0000 7fd5be59c700 1 --2- 172.21.15.2:0/3561877853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bddbd75be0 0x55bddbd80220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.879+0000 7fd5bf59e700 1 --2- 172.21.15.2:0/3561877853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bddbd76500 0x55bddbd80760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.879+0000 7fd5bed9d700 1 -- 172.21.15.2:0/3561877853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bddbd76500 msgr2=0x55bddbd80760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.879+0000 7fd5bed9d700 1 --2- 172.21.15.2:0/3561877853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bddbd76500 0x55bddbd80760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.879+0000 7fd5bed9d700 1 -- 172.21.15.2:0/3561877853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bddbd75be0 msgr2=0x55bddbd80220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.879+0000 7fd5bed9d700 1 --2- 172.21.15.2:0/3561877853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bddbd75be0 0x55bddbd80220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.879+0000 7fd5bed9d700 1 -- 172.21.15.2:0/3561877853 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b40041f0 con 0x55bddbd71840 2023-09-13T04:47:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.879+0000 7fd5be59c700 1 --2- 172.21.15.2:0/3561877853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bddbd75be0 0x55bddbd80220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:47:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.880+0000 7fd5bed9d700 1 --2- 172.21.15.2:0/3561877853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd71840 0x55bddbd7fce0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd5b009a9c0 tx=0x7fd5b0009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67bb52a35770fec3 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.880+0000 7fd5a2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:47:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.880+0000 7fd5a2ff5700 1 -- 172.21.15.2:0/3561877853 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b009aa30 con 0x55bddbd71840 2023-09-13T04:47:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.880+0000 7fd5a2ff5700 1 -- 172.21.15.2:0/3561877853 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9483+0+0 (secure 0 0 0) 0x7fd5b0098e70 con 0x55bddbd71840 2023-09-13T04:47:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.880+0000 7fd5d8adab80 1 -- 172.21.15.2:0/3561877853 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5b009b130 con 0x55bddbd71840 2023-09-13T04:47:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.880+0000 7fd5a2ff5700 1 -- 172.21.15.2:0/3561877853 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5b009e860 con 0x55bddbd71840 2023-09-13T04:47:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.880+0000 7fd5d8adab80 10 client.26500 fetch_fsmap learned FSMap version 36 2023-09-13T04:47:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.880+0000 7fd5d8adab80 1 -- 172.21.15.2:0/3561877853 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55bddbcdb740 con 0x55bddbd71840 2023-09-13T04:47:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.880+0000 7fd5a2ff5700 1 -- 172.21.15.2:0/3561877853 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5b009b130 con 0x55bddbd71840 2023-09-13T04:47:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.881+0000 7fd5a2ff5700 1 -- 172.21.15.2:0/3561877853 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fd5b0099ab0 con 0x55bddbd71840 2023-09-13T04:47:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.881+0000 7fd5d8adab80 10 client.26500 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:47:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.881+0000 7fd5d8adab80 10 client.26500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:47:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.881+0000 7fd5d8adab80 20 client.26500 populate_metadata read hostname 'smithi002' 2023-09-13T04:47:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.881+0000 7fd5d8adab80 1 --2- 172.21.15.2:0/3561877853 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55bddbddea90 0x55bddbde0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.881+0000 7fd5d8adab80 4 client.26500 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:47:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.881+0000 7fd5d8adab80 1 -- 172.21.15.2:0/3561877853 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bddbc7cff0 con 0x55bddbddea90 2023-09-13T04:47:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.881+0000 7fd5be59c700 1 --2- 172.21.15.2:0/3561877853 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55bddbddea90 0x55bddbde0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.882+0000 7fd5be59c700 1 --2- 172.21.15.2:0/3561877853 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55bddbddea90 0x55bddbde0e80 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0f24945818b7777 server_cookie=8a12ff01a1fdd41b in_seq=0 out_seq=0 2023-09-13T04:47:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:12.882+0000 7fd5a2ff5700 10 client.26500 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:47:13.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:12 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:47:13.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:12 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:47:13.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:12 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:47:13.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:12 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:47:13.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:12 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:47:13.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:12 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:47:13.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:12 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:47:13.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:12 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:47:13.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:12 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:47:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.527+0000 7fd5a2ff5700 1 -- 172.21.15.2:0/3561877853 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bddbc7cff0 con 0x55bddbddea90 2023-09-13T04:47:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.527+0000 7fd5a2ff5700 10 client.26500 handle_command_reply: tid=0 2023-09-13T04:47:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.527+0000 7fd5d8adab80 1 client.26500 shutdown 2023-09-13T04:47:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.528+0000 7fd5d8adab80 1 -- 172.21.15.2:0/3561877853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd71840 msgr2=0x55bddbd7fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.528+0000 7fd5d8adab80 1 --2- 172.21.15.2:0/3561877853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd71840 0x55bddbd7fce0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd5b009a9c0 tx=0x7fd5b0009310 comp rx=0 tx=0).stop 2023-09-13T04:47:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.529+0000 7fd5d8adab80 1 -- 172.21.15.2:0/3561877853 shutdown_connections 2023-09-13T04:47:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.529+0000 7fd5d8adab80 1 --2- 172.21.15.2:0/3561877853 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55bddbddea90 0x55bddbde0e80 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.529+0000 7fd5d8adab80 1 --2- 172.21.15.2:0/3561877853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bddbd76500 0x55bddbd80760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.529+0000 7fd5d8adab80 1 --2- 172.21.15.2:0/3561877853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bddbd71840 0x55bddbd7fce0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.529+0000 7fd5d8adab80 1 --2- 172.21.15.2:0/3561877853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bddbd75be0 0x55bddbd80220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.529+0000 7fd5d8adab80 1 -- 172.21.15.2:0/3561877853 >> 172.21.15.2:0/3561877853 conn(0x55bddbccfc20 msgr2=0x55bddbd70800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.529+0000 7fd5a2ff5700 0 client.26500 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:47:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.529+0000 7fd5d8adab80 1 -- 172.21.15.2:0/3561877853 shutdown_connections 2023-09-13T04:47:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.529+0000 7fd5d8adab80 1 -- 172.21.15.2:0/3561877853 wait complete. 2023-09-13T04:47:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.530+0000 7fd5d8adab80 20 client.26500 trim_cache size 0 max 16384 2023-09-13T04:47:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:47:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:47:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:47:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:47:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.531+0000 7fd5c51ff700 1 -- 172.21.15.2:0/4265492917 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fd5a80a07b0 msgr2=0x7fd5a80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.531+0000 7fd5c51ff700 1 --2- 172.21.15.2:0/4265492917 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fd5a80a07b0 0x7fd5a80a2c70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0101840 tx=0x7fd5b4009260 comp rx=0 tx=0).stop 2023-09-13T04:47:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.531+0000 7fd5c51ff700 1 -- 172.21.15.2:0/4265492917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5c00680d0 msgr2=0x7fd5c0100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.531+0000 7fd5c51ff700 1 --2- 172.21.15.2:0/4265492917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5c00680d0 0x7fd5c0100020 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0005910 tx=0x7fd5b0005720 comp rx=0 tx=0).stop 2023-09-13T04:47:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.531+0000 7fd5c51ff700 1 -- 172.21.15.2:0/4265492917 shutdown_connections 2023-09-13T04:47:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.531+0000 7fd5c51ff700 1 --2- 172.21.15.2:0/4265492917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5c0106d60 0x7fd5c01a4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.531+0000 7fd5c51ff700 1 --2- 172.21.15.2:0/4265492917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5c00680d0 0x7fd5c0100020 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.531+0000 7fd5c51ff700 1 --2- 172.21.15.2:0/4265492917 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fd5a80a07b0 0x7fd5a80a2c70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.531+0000 7fd5c51ff700 1 --2- 172.21.15.2:0/4265492917 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5c0068a90 0x7fd5c0100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.531+0000 7fd5c51ff700 1 -- 172.21.15.2:0/4265492917 >> 172.21.15.2:0/4265492917 conn(0x7fd5c0076970 msgr2=0x7fd5c00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.532+0000 7fd5c51ff700 1 -- 172.21.15.2:0/4265492917 shutdown_connections 2023-09-13T04:47:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:13.532+0000 7fd5c51ff700 1 -- 172.21.15.2:0/4265492917 wait complete. 2023-09-13T04:47:14.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:13 smithi002 ceph-mon[97651]: pgmap v207: 129 pgs: 2 active+clean+snaptrim, 120 active+clean, 7 active+clean+snaptrim_wait; 856 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.3 MiB/s wr, 682 op/s 2023-09-13T04:47:14.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:13 smithi149 ceph-mon[133315]: pgmap v207: 129 pgs: 2 active+clean+snaptrim, 120 active+clean, 7 active+clean+snaptrim_wait; 856 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.3 MiB/s wr, 682 op/s 2023-09-13T04:47:14.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:13 smithi156 ceph-mon[133675]: pgmap v207: 129 pgs: 2 active+clean+snaptrim, 120 active+clean, 7 active+clean+snaptrim_wait; 856 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.3 MiB/s wr, 682 op/s 2023-09-13T04:47:16.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:15 smithi002 ceph-mon[97651]: pgmap v208: 129 pgs: 2 active+clean+snaptrim, 120 active+clean, 7 active+clean+snaptrim_wait; 859 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 435 op/s 2023-09-13T04:47:16.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:15 smithi149 ceph-mon[133315]: pgmap v208: 129 pgs: 2 active+clean+snaptrim, 120 active+clean, 7 active+clean+snaptrim_wait; 859 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 435 op/s 2023-09-13T04:47:16.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:15 smithi156 ceph-mon[133675]: pgmap v208: 129 pgs: 2 active+clean+snaptrim, 120 active+clean, 7 active+clean+snaptrim_wait; 859 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 435 op/s 2023-09-13T04:47:18.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:17 smithi002 ceph-mon[97651]: pgmap v209: 129 pgs: 129 active+clean; 855 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 426 op/s 2023-09-13T04:47:18.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:17 smithi149 ceph-mon[133315]: pgmap v209: 129 pgs: 129 active+clean; 855 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 426 op/s 2023-09-13T04:47:18.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:17 smithi156 ceph-mon[133675]: pgmap v209: 129 pgs: 129 active+clean; 855 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 426 op/s 2023-09-13T04:47:19.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:18 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:47:19.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:18 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:47:19.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:18 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:47:20.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:19 smithi002 ceph-mon[97651]: pgmap v210: 129 pgs: 129 active+clean; 857 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 413 op/s 2023-09-13T04:47:20.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:19 smithi002 ceph-mon[97651]: osdmap e96: 12 total, 12 up, 12 in 2023-09-13T04:47:20.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:19 smithi149 ceph-mon[133315]: pgmap v210: 129 pgs: 129 active+clean; 857 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 413 op/s 2023-09-13T04:47:20.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:19 smithi149 ceph-mon[133315]: osdmap e96: 12 total, 12 up, 12 in 2023-09-13T04:47:20.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:19 smithi156 ceph-mon[133675]: pgmap v210: 129 pgs: 129 active+clean; 857 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 413 op/s 2023-09-13T04:47:20.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:19 smithi156 ceph-mon[133675]: osdmap e96: 12 total, 12 up, 12 in 2023-09-13T04:47:22.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:21 smithi002 ceph-mon[97651]: pgmap v212: 129 pgs: 129 active+clean; 863 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 333 op/s 2023-09-13T04:47:22.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:21 smithi149 ceph-mon[133315]: pgmap v212: 129 pgs: 129 active+clean; 863 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 333 op/s 2023-09-13T04:47:22.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:21 smithi156 ceph-mon[133675]: pgmap v212: 129 pgs: 129 active+clean; 863 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 333 op/s 2023-09-13T04:47:24.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:23 smithi002 ceph-mon[97651]: pgmap v213: 129 pgs: 129 active+clean; 869 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 316 op/s 2023-09-13T04:47:24.329 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T04:47:24.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:23 smithi149 ceph-mon[133315]: pgmap v213: 129 pgs: 129 active+clean; 869 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 316 op/s 2023-09-13T04:47:24.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:23 smithi156 ceph-mon[133675]: pgmap v213: 129 pgs: 129 active+clean; 869 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 316 op/s 2023-09-13T04:47:24.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.542+0000 7f197fdff700 1 -- 172.21.15.2:0/62399224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1978100eb0 msgr2=0x7f1978101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:24.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.542+0000 7f197fdff700 1 --2- 172.21.15.2:0/62399224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1978100eb0 0x7f1978101290 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1974009990 tx=0x7f1974009ca0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.542+0000 7f197fdff700 1 -- 172.21.15.2:0/62399224 shutdown_connections 2023-09-13T04:47:24.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.542+0000 7f197fdff700 1 --2- 172.21.15.2:0/62399224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1978105fd0 0x7f197810db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.542+0000 7f197fdff700 1 --2- 172.21.15.2:0/62399224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1978100eb0 0x7f1978101290 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.542+0000 7f197fdff700 1 --2- 172.21.15.2:0/62399224 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1978101860 0x7f19781058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.542+0000 7f197fdff700 1 -- 172.21.15.2:0/62399224 >> 172.21.15.2:0/62399224 conn(0x7f19780fc750 msgr2=0x7f19780feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:24.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.542+0000 7f197fdff700 1 -- 172.21.15.2:0/62399224 shutdown_connections 2023-09-13T04:47:24.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.542+0000 7f197fdff700 1 -- 172.21.15.2:0/62399224 wait complete. 2023-09-13T04:47:24.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.543+0000 7f197fdff700 1 Processor -- start 2023-09-13T04:47:24.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.543+0000 7f197fdff700 1 -- start start 2023-09-13T04:47:24.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.544+0000 7f197fdff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1978100eb0 0x7f1978072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:24.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.544+0000 7f197fdff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1978105fd0 0x7f19780734f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:24.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.544+0000 7f197fdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1978073a30 0x7f19781a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:24.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.544+0000 7f197fdff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19781a5760 con 0x7f1978100eb0 2023-09-13T04:47:24.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.544+0000 7f197fdff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19781a58d0 con 0x7f1978105fd0 2023-09-13T04:47:24.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.544+0000 7f197fdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19781a5a90 con 0x7f1978073a30 2023-09-13T04:47:24.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.544+0000 7f197db9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1978100eb0 0x7f1978072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:24.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.544+0000 7f197d39a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1978105fd0 0x7f19780734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:24.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.544+0000 7f197e39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1978073a30 0x7f19781a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:24.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.544+0000 7f197d39a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1978105fd0 0x7f19780734f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:48240/0 (socket says 172.21.15.2:48240) 2023-09-13T04:47:24.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.544+0000 7f197d39a700 1 -- 172.21.15.2:0/2098549882 learned_addr learned my addr 172.21.15.2:0/2098549882 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:24.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.544+0000 7f197db9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1978100eb0 0x7f1978072fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34514/0 (socket says 172.21.15.2:34514) 2023-09-13T04:47:24.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.544+0000 7f197db9b700 1 -- 172.21.15.2:0/2098549882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1978073a30 msgr2=0x7f19781a50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:24.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.544+0000 7f197db9b700 1 --2- 172.21.15.2:0/2098549882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1978073a30 0x7f19781a50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.544+0000 7f197db9b700 1 -- 172.21.15.2:0/2098549882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1978105fd0 msgr2=0x7f19780734f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:24.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.544+0000 7f197db9b700 1 --2- 172.21.15.2:0/2098549882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1978105fd0 0x7f19780734f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.544+0000 7f197db9b700 1 -- 172.21.15.2:0/2098549882 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1970009c40 con 0x7f1978100eb0 2023-09-13T04:47:24.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.544+0000 7f197db9b700 1 --2- 172.21.15.2:0/2098549882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1978100eb0 0x7f1978072fb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1974000c00 tx=0x7f1974005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:24.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.545+0000 7f196effd700 1 -- 172.21.15.2:0/2098549882 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197401d070 con 0x7f1978100eb0 2023-09-13T04:47:24.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.545+0000 7f196effd700 1 -- 172.21.15.2:0/2098549882 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1974021410 con 0x7f1978100eb0 2023-09-13T04:47:24.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.545+0000 7f197fdff700 1 -- 172.21.15.2:0/2098549882 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19740095f0 con 0x7f1978100eb0 2023-09-13T04:47:24.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.545+0000 7f196effd700 1 -- 172.21.15.2:0/2098549882 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1974017650 con 0x7f1978100eb0 2023-09-13T04:47:24.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.545+0000 7f197fdff700 1 -- 172.21.15.2:0/2098549882 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19781a6000 con 0x7f1978100eb0 2023-09-13T04:47:24.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.548+0000 7f196effd700 1 -- 172.21.15.2:0/2098549882 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f1974021580 con 0x7f1978100eb0 2023-09-13T04:47:24.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.549+0000 7f19937b9b80 1 Processor -- start 2023-09-13T04:47:24.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.549+0000 7f19937b9b80 1 -- start start 2023-09-13T04:47:24.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.549+0000 7f196effd700 1 --2- 172.21.15.2:0/2098549882 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f19640a0830 0x7f19640a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:24.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.550+0000 7f196effd700 1 -- 172.21.15.2:0/2098549882 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f19740cd400 con 0x7f1978100eb0 2023-09-13T04:47:24.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.550+0000 7f19937b9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfdf8a7fe0 0x55dfdf8a83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:24.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.550+0000 7f19937b9b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a8900 0x55dfdf8a24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:24.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.550+0000 7f19937b9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfdf8a2a00 0x55dfdf8a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:24.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.550+0000 7f19937b9b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdf7fb780 con 0x55dfdf8a8900 2023-09-13T04:47:24.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.550+0000 7f19937b9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdf7f90a0 con 0x55dfdf8a7fe0 2023-09-13T04:47:24.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.550+0000 7f19937b9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdf7f4990 con 0x55dfdf8a2a00 2023-09-13T04:47:24.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.550+0000 7f197e39c700 1 --2- 172.21.15.2:0/2098549882 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f19640a0830 0x7f19640a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:24.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.550+0000 7f197db9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfdf8a7fe0 0x55dfdf8a83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:24.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.550+0000 7f197db9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfdf8a7fe0 0x55dfdf8a83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:48252/0 (socket says 172.21.15.2:48252) 2023-09-13T04:47:24.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.550+0000 7f197db9b700 1 -- 172.21.15.2:0/1625715767 learned_addr learned my addr 172.21.15.2:0/1625715767 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:24.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.550+0000 7f197d39a700 1 --2- 172.21.15.2:0/1625715767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a8900 0x55dfdf8a24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:24.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.550+0000 7f197e39c700 1 --2- 172.21.15.2:0/1625715767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfdf8a2a00 0x55dfdf8a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:24.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.550+0000 7f197db9b700 1 -- 172.21.15.2:0/1625715767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfdf8a2a00 msgr2=0x55dfdf8a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:24.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.550+0000 7f197db9b700 1 --2- 172.21.15.2:0/1625715767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfdf8a2a00 0x55dfdf8a2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.550+0000 7f197db9b700 1 -- 172.21.15.2:0/1625715767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a8900 msgr2=0x55dfdf8a24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:24.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.550+0000 7f197db9b700 1 --2- 172.21.15.2:0/1625715767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a8900 0x55dfdf8a24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.551+0000 7f197db9b700 1 -- 172.21.15.2:0/1625715767 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfdf674630 con 0x55dfdf8a7fe0 2023-09-13T04:47:24.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.551+0000 7f197e39c700 1 --2- 172.21.15.2:0/2098549882 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f19640a0830 0x7f19640a2cf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1970009fd0 tx=0x7f197000fdd0 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:24.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.551+0000 7f197db9b700 1 --2- 172.21.15.2:0/1625715767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfdf8a7fe0 0x55dfdf8a83c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1974096b80 tx=0x7f197409a570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aafe1b70f5817e40 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:24.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.551+0000 7f1963fff700 1 -- 172.21.15.2:0/1625715767 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197409a680 con 0x55dfdf8a7fe0 2023-09-13T04:47:24.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.551+0000 7f1963fff700 1 -- 172.21.15.2:0/1625715767 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f197409e4b0 con 0x55dfdf8a7fe0 2023-09-13T04:47:24.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.551+0000 7f1963fff700 1 -- 172.21.15.2:0/1625715767 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1974021880 con 0x55dfdf8a7fe0 2023-09-13T04:47:24.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.552+0000 7f19937b9b80 1 -- 172.21.15.2:0/1625715767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfdf8a7fe0 msgr2=0x55dfdf8a83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:24.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.552+0000 7f19937b9b80 1 --2- 172.21.15.2:0/1625715767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfdf8a7fe0 0x55dfdf8a83c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1974096b80 tx=0x7f197409a570 comp rx=0 tx=0).stop 2023-09-13T04:47:24.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.552+0000 7f19937b9b80 1 -- 172.21.15.2:0/1625715767 shutdown_connections 2023-09-13T04:47:24.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.552+0000 7f19937b9b80 1 --2- 172.21.15.2:0/1625715767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfdf8a2a00 0x55dfdf8a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.552+0000 7f19937b9b80 1 --2- 172.21.15.2:0/1625715767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfdf8a7fe0 0x55dfdf8a83c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.552+0000 7f19937b9b80 1 --2- 172.21.15.2:0/1625715767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a8900 0x55dfdf8a24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.552+0000 7f19937b9b80 1 -- 172.21.15.2:0/1625715767 >> 172.21.15.2:0/1625715767 conn(0x55dfdf800c80 msgr2=0x55dfdf89cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:24.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.552+0000 7f19937b9b80 1 -- 172.21.15.2:0/1625715767 shutdown_connections 2023-09-13T04:47:24.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.552+0000 7f19937b9b80 1 -- 172.21.15.2:0/1625715767 wait complete. 2023-09-13T04:47:24.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.552+0000 7f19937b9b80 1 -- start start 2023-09-13T04:47:24.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.553+0000 7f19937b9b80 1 Processor -- start 2023-09-13T04:47:24.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.553+0000 7f19937b9b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a2a00 0x55dfdf8ae800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:24.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.553+0000 7f19937b9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfdf8a7fe0 0x55dfdf8abb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:24.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.553+0000 7f19937b9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfdf8a8900 0x55dfdf8ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:24.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.553+0000 7f19937b9b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdf792bb0 con 0x55dfdf8a2a00 2023-09-13T04:47:24.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.553+0000 7f19937b9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdf7928b0 con 0x55dfdf8a8900 2023-09-13T04:47:24.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.553+0000 7f19937b9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdf676920 con 0x55dfdf8a7fe0 2023-09-13T04:47:24.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.553+0000 7f19937b9b80 10 client.0 _ll_register_callbacks cb 0x55dfdf790680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:47:24.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.553+0000 7f197db9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a2a00 0x55dfdf8ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:24.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.553+0000 7f197e39c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfdf8a8900 0x55dfdf8ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:24.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.553+0000 7f197d39a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfdf8a7fe0 0x55dfdf8abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:24.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.553+0000 7f197db9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a2a00 0x55dfdf8ae800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34522/0 (socket says 172.21.15.2:34522) 2023-09-13T04:47:24.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.553+0000 7f197db9b700 1 -- 172.21.15.2:0/4158518997 learned_addr learned my addr 172.21.15.2:0/4158518997 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:24.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.554+0000 7f197db9b700 1 -- 172.21.15.2:0/4158518997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfdf8a7fe0 msgr2=0x55dfdf8abb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:24.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.554+0000 7f197db9b700 1 --2- 172.21.15.2:0/4158518997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfdf8a7fe0 0x55dfdf8abb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.554+0000 7f197db9b700 1 -- 172.21.15.2:0/4158518997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfdf8a8900 msgr2=0x55dfdf8ac0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:24.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.554+0000 7f197db9b700 1 --2- 172.21.15.2:0/4158518997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfdf8a8900 0x55dfdf8ac0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.554+0000 7f197db9b700 1 -- 172.21.15.2:0/4158518997 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19781a6000 con 0x55dfdf8a2a00 2023-09-13T04:47:24.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.554+0000 7f197e39c700 1 --2- 172.21.15.2:0/4158518997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfdf8a8900 0x55dfdf8ac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:47:24.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.554+0000 7f197db9b700 1 --2- 172.21.15.2:0/4158518997 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a2a00 0x55dfdf8ae800 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1974005cd0 tx=0x7f19740cd2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d015d39051957f34 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:24.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.554+0000 7f19617fa700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:47:24.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.554+0000 7f19617fa700 1 -- 172.21.15.2:0/4158518997 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1974021a90 con 0x55dfdf8a2a00 2023-09-13T04:47:24.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.554+0000 7f19617fa700 1 -- 172.21.15.2:0/4158518997 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f19740a2660 con 0x55dfdf8a2a00 2023-09-13T04:47:24.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.554+0000 7f19937b9b80 1 -- 172.21.15.2:0/4158518997 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dfdf895670 con 0x55dfdf8a2a00 2023-09-13T04:47:24.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.554+0000 7f19617fa700 1 -- 172.21.15.2:0/4158518997 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f197409a680 con 0x55dfdf8a2a00 2023-09-13T04:47:24.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.555+0000 7f19937b9b80 10 client.16341 fetch_fsmap learned FSMap version 36 2023-09-13T04:47:24.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.555+0000 7f19937b9b80 1 -- 172.21.15.2:0/4158518997 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55dfdf80c7a0 con 0x55dfdf8a2a00 2023-09-13T04:47:24.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.555+0000 7f19617fa700 1 -- 172.21.15.2:0/4158518997 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dfdf895670 con 0x55dfdf8a2a00 2023-09-13T04:47:24.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.555+0000 7f19617fa700 1 -- 172.21.15.2:0/4158518997 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f19740a21d0 con 0x55dfdf8a2a00 2023-09-13T04:47:24.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.555+0000 7f19937b9b80 10 client.16341 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:47:24.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.555+0000 7f19937b9b80 10 client.16341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:47:24.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.555+0000 7f19937b9b80 20 client.16341 populate_metadata read hostname 'smithi002' 2023-09-13T04:47:24.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.555+0000 7f19937b9b80 1 --2- 172.21.15.2:0/4158518997 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55dfdf8bba30 0x55dfdf8bde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:24.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.555+0000 7f19937b9b80 4 client.16341 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:47:24.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.555+0000 7f19937b9b80 1 -- 172.21.15.2:0/4158518997 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dfdf7ed970 con 0x55dfdf8bba30 2023-09-13T04:47:24.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.555+0000 7f197d39a700 1 --2- 172.21.15.2:0/4158518997 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55dfdf8bba30 0x55dfdf8bde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:24.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.556+0000 7f197d39a700 1 --2- 172.21.15.2:0/4158518997 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55dfdf8bba30 0x55dfdf8bde50 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60d3004e7627ac1b server_cookie=a258e818ec7a544c in_seq=0 out_seq=0 2023-09-13T04:47:24.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.556+0000 7f19617fa700 10 client.16341 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:47:24.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.557+0000 7f19617fa700 1 -- 172.21.15.2:0/4158518997 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dfdf7ed970 con 0x55dfdf8bba30 2023-09-13T04:47:24.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.557+0000 7f19617fa700 10 client.16341 handle_command_reply: tid=0 2023-09-13T04:47:24.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.557+0000 7f19937b9b80 1 client.16341 shutdown 2023-09-13T04:47:24.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.557+0000 7f19937b9b80 1 -- 172.21.15.2:0/4158518997 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a2a00 msgr2=0x55dfdf8ae800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:24.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.557+0000 7f19937b9b80 1 --2- 172.21.15.2:0/4158518997 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a2a00 0x55dfdf8ae800 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1974005cd0 tx=0x7f19740cd2f0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.557+0000 7f19937b9b80 1 -- 172.21.15.2:0/4158518997 shutdown_connections 2023-09-13T04:47:24.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.557+0000 7f19937b9b80 1 --2- 172.21.15.2:0/4158518997 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55dfdf8bba30 0x55dfdf8bde50 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.557+0000 7f19937b9b80 1 --2- 172.21.15.2:0/4158518997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfdf8a8900 0x55dfdf8ac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.557+0000 7f19937b9b80 1 --2- 172.21.15.2:0/4158518997 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a2a00 0x55dfdf8ae800 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.557+0000 7f19937b9b80 1 --2- 172.21.15.2:0/4158518997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfdf8a7fe0 0x55dfdf8abb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.557+0000 7f19617fa700 0 client.16341 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:47:24.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.557+0000 7f19937b9b80 1 -- 172.21.15.2:0/4158518997 >> 172.21.15.2:0/4158518997 conn(0x55dfdf800c80 msgr2=0x55dfdf8977f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:24.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.557+0000 7f19937b9b80 1 -- 172.21.15.2:0/4158518997 shutdown_connections 2023-09-13T04:47:24.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.557+0000 7f19937b9b80 1 -- 172.21.15.2:0/4158518997 wait complete. 2023-09-13T04:47:24.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.558+0000 7f19937b9b80 20 client.16341 trim_cache size 0 max 16384 2023-09-13T04:47:24.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.575+0000 7f19937b9b80 1 Processor -- start 2023-09-13T04:47:24.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.575+0000 7f19937b9b80 1 -- start start 2023-09-13T04:47:24.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.575+0000 7f19937b9b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a2a00 0x55dfdf8a24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:24.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.575+0000 7f19937b9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfdf8a7fe0 0x55dfdf8a7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:24.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.575+0000 7f19937b9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfdf8a8900 0x55dfdf8ae330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:24.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.575+0000 7f19937b9b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdf8b88b0 con 0x55dfdf8a2a00 2023-09-13T04:47:24.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.575+0000 7f19937b9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdf7ed970 con 0x55dfdf8a8900 2023-09-13T04:47:24.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.575+0000 7f19937b9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdf89ea50 con 0x55dfdf8a7fe0 2023-09-13T04:47:24.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.576+0000 7f197d39a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfdf8a7fe0 0x55dfdf8a7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:24.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.576+0000 7f197e39c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfdf8a8900 0x55dfdf8ae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:24.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.576+0000 7f197db9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a2a00 0x55dfdf8a24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:24.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.576+0000 7f197d39a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfdf8a7fe0 0x55dfdf8a7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:47594/0 (socket says 172.21.15.2:47594) 2023-09-13T04:47:24.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.576+0000 7f197e39c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfdf8a8900 0x55dfdf8ae330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:48268/0 (socket says 172.21.15.2:48268) 2023-09-13T04:47:24.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.576+0000 7f197d39a700 1 -- 172.21.15.2:0/1106142101 learned_addr learned my addr 172.21.15.2:0/1106142101 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:24.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.576+0000 7f197db9b700 1 -- 172.21.15.2:0/1106142101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfdf8a7fe0 msgr2=0x55dfdf8a7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:24.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.576+0000 7f197db9b700 1 --2- 172.21.15.2:0/1106142101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfdf8a7fe0 0x55dfdf8a7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.576+0000 7f197db9b700 1 -- 172.21.15.2:0/1106142101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfdf8a8900 msgr2=0x55dfdf8ae330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:24.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.576+0000 7f197db9b700 1 --2- 172.21.15.2:0/1106142101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfdf8a8900 0x55dfdf8ae330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.576+0000 7f197db9b700 1 -- 172.21.15.2:0/1106142101 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfdf620de0 con 0x55dfdf8a2a00 2023-09-13T04:47:24.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.576+0000 7f197db9b700 1 --2- 172.21.15.2:0/1106142101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a2a00 0x55dfdf8a24c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f197409a380 tx=0x7f1974021c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44b86be23cbe93b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:24.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.576+0000 7f196cbff700 1 -- 172.21.15.2:0/1106142101 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1974021a90 con 0x55dfdf8a2a00 2023-09-13T04:47:24.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.576+0000 7f196cbff700 1 -- 172.21.15.2:0/1106142101 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1974099e30 con 0x55dfdf8a2a00 2023-09-13T04:47:24.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.576+0000 7f196cbff700 1 -- 172.21.15.2:0/1106142101 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19740177b0 con 0x55dfdf8a2a00 2023-09-13T04:47:24.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.577+0000 7f19937b9b80 1 -- 172.21.15.2:0/1106142101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a2a00 msgr2=0x55dfdf8a24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:24.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.577+0000 7f19937b9b80 1 --2- 172.21.15.2:0/1106142101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a2a00 0x55dfdf8a24c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f197409a380 tx=0x7f1974021c20 comp rx=0 tx=0).stop 2023-09-13T04:47:24.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.577+0000 7f19937b9b80 1 -- 172.21.15.2:0/1106142101 shutdown_connections 2023-09-13T04:47:24.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.577+0000 7f19937b9b80 1 --2- 172.21.15.2:0/1106142101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfdf8a8900 0x55dfdf8ae330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.577+0000 7f19937b9b80 1 --2- 172.21.15.2:0/1106142101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a2a00 0x55dfdf8a24c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.577+0000 7f19937b9b80 1 --2- 172.21.15.2:0/1106142101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfdf8a7fe0 0x55dfdf8a7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.577+0000 7f19937b9b80 1 -- 172.21.15.2:0/1106142101 >> 172.21.15.2:0/1106142101 conn(0x55dfdf800c80 msgr2=0x55dfdf8a8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:24.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.577+0000 7f19937b9b80 1 -- 172.21.15.2:0/1106142101 shutdown_connections 2023-09-13T04:47:24.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.577+0000 7f19937b9b80 1 -- 172.21.15.2:0/1106142101 wait complete. 2023-09-13T04:47:24.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.577+0000 7f19937b9b80 1 -- start start 2023-09-13T04:47:24.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.578+0000 7f19937b9b80 1 Processor -- start 2023-09-13T04:47:24.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.578+0000 7f19937b9b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a2a00 0x55dfdf8a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:24.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.578+0000 7f19937b9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfdf8a7fe0 0x55dfdf8a5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:24.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.578+0000 7f19937b9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfdf8a8900 0x55dfdf8a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:24.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.578+0000 7f19937b9b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdf8b33f0 con 0x55dfdf8a2a00 2023-09-13T04:47:24.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.578+0000 7f19937b9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdf8b3560 con 0x55dfdf8a8900 2023-09-13T04:47:24.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.578+0000 7f19937b9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdf8a3f90 con 0x55dfdf8a7fe0 2023-09-13T04:47:24.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.578+0000 7f19937b9b80 10 client.0 _ll_register_callbacks cb 0x55dfdf790680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:47:24.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.578+0000 7f197db9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a2a00 0x55dfdf8a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:24.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.578+0000 7f197db9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a2a00 0x55dfdf8a4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34546/0 (socket says 172.21.15.2:34546) 2023-09-13T04:47:24.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.578+0000 7f197db9b700 1 -- 172.21.15.2:0/2760624922 learned_addr learned my addr 172.21.15.2:0/2760624922 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:24.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.578+0000 7f197d39a700 1 --2- 172.21.15.2:0/2760624922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfdf8a7fe0 0x55dfdf8a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:24.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.578+0000 7f197e39c700 1 --2- 172.21.15.2:0/2760624922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfdf8a8900 0x55dfdf8a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:24.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.579+0000 7f197db9b700 1 -- 172.21.15.2:0/2760624922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfdf8a7fe0 msgr2=0x55dfdf8a5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:24.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.579+0000 7f197db9b700 1 --2- 172.21.15.2:0/2760624922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfdf8a7fe0 0x55dfdf8a5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.579+0000 7f197db9b700 1 -- 172.21.15.2:0/2760624922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfdf8a8900 msgr2=0x55dfdf8a3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:24.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.579+0000 7f197db9b700 1 --2- 172.21.15.2:0/2760624922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfdf8a8900 0x55dfdf8a3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.579+0000 7f197db9b700 1 -- 172.21.15.2:0/2760624922 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dfdf80c7a0 con 0x55dfdf8a2a00 2023-09-13T04:47:24.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.579+0000 7f197db9b700 1 --2- 172.21.15.2:0/2760624922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a2a00 0x55dfdf8a4bd0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f19740cad80 tx=0x7f1974098c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85582f16abc282b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:24.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.579+0000 7f19617fa700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:47:24.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.579+0000 7f19617fa700 1 -- 172.21.15.2:0/2760624922 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19740852d0 con 0x55dfdf8a2a00 2023-09-13T04:47:24.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.579+0000 7f19937b9b80 1 -- 172.21.15.2:0/2760624922 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f197409e4b0 con 0x55dfdf8a2a00 2023-09-13T04:47:24.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.579+0000 7f19617fa700 1 -- 172.21.15.2:0/2760624922 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f1974099b70 con 0x55dfdf8a2a00 2023-09-13T04:47:24.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.579+0000 7f19617fa700 1 -- 172.21.15.2:0/2760624922 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1974099fe0 con 0x55dfdf8a2a00 2023-09-13T04:47:24.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.579+0000 7f19937b9b80 10 client.16347 fetch_fsmap learned FSMap version 36 2023-09-13T04:47:24.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.579+0000 7f19937b9b80 1 -- 172.21.15.2:0/2760624922 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55dfdf609c60 con 0x55dfdf8a2a00 2023-09-13T04:47:24.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.580+0000 7f19617fa700 1 -- 172.21.15.2:0/2760624922 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f197409e4b0 con 0x55dfdf8a2a00 2023-09-13T04:47:24.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.580+0000 7f19617fa700 1 -- 172.21.15.2:0/2760624922 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f19740a0270 con 0x55dfdf8a2a00 2023-09-13T04:47:24.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.580+0000 7f19937b9b80 10 client.16347 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:47:24.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.580+0000 7f19937b9b80 10 client.16347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:47:24.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.580+0000 7f19937b9b80 20 client.16347 populate_metadata read hostname 'smithi002' 2023-09-13T04:47:24.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.580+0000 7f19937b9b80 1 --2- 172.21.15.2:0/2760624922 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55dfdf90ffe0 0x55dfdf9123d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:24.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.580+0000 7f19937b9b80 4 client.16347 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-09-13T04:47:24.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.580+0000 7f19937b9b80 1 -- 172.21.15.2:0/2760624922 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dfdf7ae050 con 0x55dfdf90ffe0 2023-09-13T04:47:24.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.580+0000 7f197d39a700 1 --2- 172.21.15.2:0/2760624922 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55dfdf90ffe0 0x55dfdf9123d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:24.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.581+0000 7f19617fa700 10 client.16347 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:47:24.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.581+0000 7f197d39a700 1 --2- 172.21.15.2:0/2760624922 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55dfdf90ffe0 0x55dfdf9123d0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d7715179438d7d2 server_cookie=b05b0a9010bb7fb6 in_seq=0 out_seq=0 2023-09-13T04:47:24.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.582+0000 7f19617fa700 1 -- 172.21.15.2:0/2760624922 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55dfdf7ae050 con 0x55dfdf90ffe0 2023-09-13T04:47:24.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.582+0000 7f19617fa700 10 client.16347 handle_command_reply: tid=0 2023-09-13T04:47:24.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.582+0000 7f19937b9b80 1 client.16347 shutdown 2023-09-13T04:47:24.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.582+0000 7f19937b9b80 1 -- 172.21.15.2:0/2760624922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a2a00 msgr2=0x55dfdf8a4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:24.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.582+0000 7f19937b9b80 1 --2- 172.21.15.2:0/2760624922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a2a00 0x55dfdf8a4bd0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f19740cad80 tx=0x7f1974098c30 comp rx=0 tx=0).stop 2023-09-13T04:47:24.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.582+0000 7f19937b9b80 1 -- 172.21.15.2:0/2760624922 shutdown_connections 2023-09-13T04:47:24.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.582+0000 7f19937b9b80 1 --2- 172.21.15.2:0/2760624922 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55dfdf90ffe0 0x55dfdf9123d0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.582+0000 7f19937b9b80 1 --2- 172.21.15.2:0/2760624922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfdf8a8900 0x55dfdf8a3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.582+0000 7f19617fa700 0 client.16347 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:47:24.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.582+0000 7f19937b9b80 1 --2- 172.21.15.2:0/2760624922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfdf8a2a00 0x55dfdf8a4bd0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.582+0000 7f19937b9b80 1 --2- 172.21.15.2:0/2760624922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfdf8a7fe0 0x55dfdf8a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.582+0000 7f19937b9b80 1 -- 172.21.15.2:0/2760624922 >> 172.21.15.2:0/2760624922 conn(0x55dfdf800c80 msgr2=0x55dfdf8a72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:24.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.582+0000 7f19937b9b80 1 -- 172.21.15.2:0/2760624922 shutdown_connections 2023-09-13T04:47:24.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.582+0000 7f19937b9b80 1 -- 172.21.15.2:0/2760624922 wait complete. 2023-09-13T04:47:24.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.582+0000 7f19937b9b80 20 client.16347 trim_cache size 0 max 16384 2023-09-13T04:47:24.600 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:47:24.600 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "scrub active (3 inodes in the stack)", 2023-09-13T04:47:24.600 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-09-13T04:47:24.600 INFO:teuthology.orchestra.run.smithi002.stdout: "740235c0-6e1b-4729-b1fe-33345ae948c1": { 2023-09-13T04:47:24.600 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-09-13T04:47:24.601 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "740235c0-6e1b-4729-b1fe-33345ae948c1", 2023-09-13T04:47:24.601 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-09-13T04:47:24.601 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:47:24.601 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:47:24.601 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:47:24.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.583+0000 7f197fdff700 1 -- 172.21.15.2:0/2098549882 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f19640a0830 msgr2=0x7f19640a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:24.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.583+0000 7f197fdff700 1 --2- 172.21.15.2:0/2098549882 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f19640a0830 0x7f19640a2cf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1970009fd0 tx=0x7f197000fdd0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.583+0000 7f197fdff700 1 -- 172.21.15.2:0/2098549882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1978100eb0 msgr2=0x7f1978072fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:24.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.583+0000 7f197fdff700 1 --2- 172.21.15.2:0/2098549882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1978100eb0 0x7f1978072fb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1974000c00 tx=0x7f1974005740 comp rx=0 tx=0).stop 2023-09-13T04:47:24.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.583+0000 7f197fdff700 1 -- 172.21.15.2:0/2098549882 shutdown_connections 2023-09-13T04:47:24.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.583+0000 7f197fdff700 1 --2- 172.21.15.2:0/2098549882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1978073a30 0x7f19781a50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.583+0000 7f197fdff700 1 --2- 172.21.15.2:0/2098549882 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f19640a0830 0x7f19640a2cf0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.583+0000 7f197fdff700 1 --2- 172.21.15.2:0/2098549882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1978100eb0 0x7f1978072fb0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.583+0000 7f197fdff700 1 --2- 172.21.15.2:0/2098549882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1978105fd0 0x7f19780734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:24.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.583+0000 7f197fdff700 1 -- 172.21.15.2:0/2098549882 >> 172.21.15.2:0/2098549882 conn(0x7f19780fc750 msgr2=0x7f19780fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:24.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.583+0000 7f197fdff700 1 -- 172.21.15.2:0/2098549882 shutdown_connections 2023-09-13T04:47:24.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:24.583+0000 7f197fdff700 1 -- 172.21.15.2:0/2098549882 wait complete. 2023-09-13T04:47:24.605 INFO:tasks.cephfs.filesystem:scrub status for tag:740235c0-6e1b-4729-b1fe-33345ae948c1 - {'path': '/', 'tag': '740235c0-6e1b-4729-b1fe-33345ae948c1', 'options': 'recursive,force'} 2023-09-13T04:47:26.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:25 smithi002 ceph-mon[97651]: pgmap v214: 129 pgs: 129 active+clean; 881 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 264 op/s 2023-09-13T04:47:26.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:25 smithi149 ceph-mon[133315]: pgmap v214: 129 pgs: 129 active+clean; 881 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 264 op/s 2023-09-13T04:47:26.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:25 smithi156 ceph-mon[133675]: pgmap v214: 129 pgs: 129 active+clean; 881 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 264 op/s 2023-09-13T04:47:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.726+0000 7f30b31ff700 1 -- 172.21.15.2:0/491978830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30ac111c60 msgr2=0x7f30ac114050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:26.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.726+0000 7f30b31ff700 1 --2- 172.21.15.2:0/491978830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30ac111c60 0x7f30ac114050 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f30a40098f0 tx=0x7f30a4009c00 comp rx=0 tx=0).stop 2023-09-13T04:47:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.731+0000 7f30b31ff700 1 -- 172.21.15.2:0/491978830 shutdown_connections 2023-09-13T04:47:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.731+0000 7f30b31ff700 1 --2- 172.21.15.2:0/491978830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30ac111c60 0x7f30ac114050 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.731+0000 7f30b31ff700 1 --2- 172.21.15.2:0/491978830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30ac073050 0x7f30ac073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.731+0000 7f30b31ff700 1 --2- 172.21.15.2:0/491978830 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30ac073970 0x7f30ac111720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.731+0000 7f30b31ff700 1 -- 172.21.15.2:0/491978830 >> 172.21.15.2:0/491978830 conn(0x7f30ac0fc740 msgr2=0x7f30ac0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:26.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.732+0000 7f30b31ff700 1 -- 172.21.15.2:0/491978830 shutdown_connections 2023-09-13T04:47:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.736+0000 7f30b31ff700 1 -- 172.21.15.2:0/491978830 wait complete. 2023-09-13T04:47:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.737+0000 7f30b31ff700 1 Processor -- start 2023-09-13T04:47:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.739+0000 7f30b31ff700 1 -- start start 2023-09-13T04:47:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.739+0000 7f30b31ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30ac073050 0x7f30ac19d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.739+0000 7f30b31ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30ac073970 0x7f30ac19d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.739+0000 7f30b31ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30ac111c60 0x7f30ac1a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:26.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.739+0000 7f30b31ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac1a2520 con 0x7f30ac073970 2023-09-13T04:47:26.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.740+0000 7f30b31ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac1a2690 con 0x7f30ac111c60 2023-09-13T04:47:26.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.740+0000 7f30b31ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac1a2850 con 0x7f30ac073050 2023-09-13T04:47:26.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.740+0000 7f30b179c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30ac111c60 0x7f30ac1a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:26.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.740+0000 7f30b179c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30ac111c60 0x7f30ac1a1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:48296/0 (socket says 172.21.15.2:48296) 2023-09-13T04:47:26.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.740+0000 7f30b179c700 1 -- 172.21.15.2:0/3960120565 learned_addr learned my addr 172.21.15.2:0/3960120565 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:26.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.741+0000 7f30b0f9b700 1 --2- 172.21.15.2:0/3960120565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30ac073050 0x7f30ac19d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:26.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.741+0000 7f30b179c700 1 -- 172.21.15.2:0/3960120565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30ac073050 msgr2=0x7f30ac19d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:26.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.741+0000 7f30b179c700 1 --2- 172.21.15.2:0/3960120565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30ac073050 0x7f30ac19d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:26.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.741+0000 7f30b179c700 1 -- 172.21.15.2:0/3960120565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30ac073970 msgr2=0x7f30ac19d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:26.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.741+0000 7f30b179c700 1 --2- 172.21.15.2:0/3960120565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30ac073970 0x7f30ac19d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.741+0000 7f30b179c700 1 -- 172.21.15.2:0/3960120565 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a4009520 con 0x7f30ac111c60 2023-09-13T04:47:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.742+0000 7f30b179c700 1 --2- 172.21.15.2:0/3960120565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30ac111c60 0x7f30ac1a1ee0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f30a4009230 tx=0x7f30a400b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.749+0000 7f30a1ffb700 1 -- 172.21.15.2:0/3960120565 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a401c070 con 0x7f30ac111c60 2023-09-13T04:47:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.749+0000 7f30a1ffb700 1 -- 172.21.15.2:0/3960120565 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30a4005040 con 0x7f30ac111c60 2023-09-13T04:47:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.749+0000 7f30a1ffb700 1 -- 172.21.15.2:0/3960120565 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a4004a70 con 0x7f30ac111c60 2023-09-13T04:47:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.750+0000 7f30b31ff700 1 -- 172.21.15.2:0/3960120565 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30ac1a2ad0 con 0x7f30ac111c60 2023-09-13T04:47:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.750+0000 7f30b31ff700 1 -- 172.21.15.2:0/3960120565 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30ac1050d0 con 0x7f30ac111c60 2023-09-13T04:47:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.759+0000 7f30c6ae1b80 1 Processor -- start 2023-09-13T04:47:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.760+0000 7f30a1ffb700 1 -- 172.21.15.2:0/3960120565 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f30a4020070 con 0x7f30ac111c60 2023-09-13T04:47:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.762+0000 7f30c6ae1b80 1 -- start start 2023-09-13T04:47:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.762+0000 7f30c6ae1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d1612be0 0x55a7d1612fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.762+0000 7f30c6ae1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7d1613500 0x55a7d160e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.762+0000 7f30c6ae1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a7d160e840 0x55a7d160ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.762+0000 7f30c6ae1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d1567720 con 0x55a7d1613500 2023-09-13T04:47:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.762+0000 7f30c6ae1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d1565040 con 0x55a7d1612be0 2023-09-13T04:47:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.762+0000 7f30c6ae1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d1560930 con 0x55a7d160e840 2023-09-13T04:47:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.763+0000 7f30a3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d1612be0 0x55a7d1612fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.763+0000 7f30a3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d1612be0 0x55a7d1612fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:48302/0 (socket says 172.21.15.2:48302) 2023-09-13T04:47:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.763+0000 7f30a3fff700 1 -- 172.21.15.2:0/841809352 learned_addr learned my addr 172.21.15.2:0/841809352 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.763+0000 7f30a3fff700 1 -- 172.21.15.2:0/841809352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a7d160e840 msgr2=0x55a7d160ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.763+0000 7f30a3fff700 1 --2- 172.21.15.2:0/841809352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a7d160e840 0x55a7d160ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.763+0000 7f30a3fff700 1 -- 172.21.15.2:0/841809352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7d1613500 msgr2=0x55a7d160e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.763+0000 7f30a3fff700 1 --2- 172.21.15.2:0/841809352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7d1613500 0x55a7d160e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.763+0000 7f30a3fff700 1 -- 172.21.15.2:0/841809352 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7d13e05d0 con 0x55a7d1612be0 2023-09-13T04:47:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.763+0000 7f30a1ffb700 1 --2- 172.21.15.2:0/3960120565 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f30940a0980 0x7f30940a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.764+0000 7f30a3fff700 1 --2- 172.21.15.2:0/841809352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d1612be0 0x55a7d1612fc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f309c005b40 tx=0x7f309c009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e7fd97d9d416de5 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.764+0000 7f30a1ffb700 1 -- 172.21.15.2:0/3960120565 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f30a40ccfc0 con 0x7f30ac111c60 2023-09-13T04:47:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.765+0000 7f30b0f9b700 1 --2- 172.21.15.2:0/3960120565 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f30940a0980 0x7f30940a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.765+0000 7f3092ffd700 1 -- 172.21.15.2:0/841809352 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c010b80 con 0x55a7d1612be0 2023-09-13T04:47:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.766+0000 7f3092ffd700 1 -- 172.21.15.2:0/841809352 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f309c004cf0 con 0x55a7d1612be0 2023-09-13T04:47:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.766+0000 7f3092ffd700 1 -- 172.21.15.2:0/841809352 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c012480 con 0x55a7d1612be0 2023-09-13T04:47:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.767+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/841809352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d1612be0 msgr2=0x55a7d1612fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.767+0000 7f30c6ae1b80 1 --2- 172.21.15.2:0/841809352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d1612be0 0x55a7d1612fc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f309c005b40 tx=0x7f309c009f50 comp rx=0 tx=0).stop 2023-09-13T04:47:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.767+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/841809352 shutdown_connections 2023-09-13T04:47:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.767+0000 7f30c6ae1b80 1 --2- 172.21.15.2:0/841809352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a7d160e840 0x55a7d160ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.767+0000 7f30c6ae1b80 1 --2- 172.21.15.2:0/841809352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d1612be0 0x55a7d1612fc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.767+0000 7f30c6ae1b80 1 --2- 172.21.15.2:0/841809352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7d1613500 0x55a7d160e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.767+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/841809352 >> 172.21.15.2:0/841809352 conn(0x55a7d156cc20 msgr2=0x55a7d1608b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.768+0000 7f30b0f9b700 1 --2- 172.21.15.2:0/3960120565 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f30940a0980 0x7f30940a2e40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55a7d1613900 tx=0x7f30a8009040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.769+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/841809352 shutdown_connections 2023-09-13T04:47:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.776+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/841809352 wait complete. 2023-09-13T04:47:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.776+0000 7f30c6ae1b80 1 -- start start 2023-09-13T04:47:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.777+0000 7f30c6ae1b80 1 Processor -- start 2023-09-13T04:47:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.778+0000 7f30c6ae1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d160e840 0x55a7d16178a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.778+0000 7f30c6ae1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a7d1612be0 0x55a7d1617de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.778+0000 7f30c6ae1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7d1613500 0x55a7d1618320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.778+0000 7f30c6ae1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d14feb50 con 0x55a7d1613500 2023-09-13T04:47:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.778+0000 7f30c6ae1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d14fe850 con 0x55a7d160e840 2023-09-13T04:47:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.778+0000 7f30c6ae1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d13e28c0 con 0x55a7d1612be0 2023-09-13T04:47:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.778+0000 7f30c6ae1b80 10 client.0 _ll_register_callbacks cb 0x55a7d14fc620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:47:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.778+0000 7f30a3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a7d1612be0 0x55a7d1617de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.778+0000 7f30a3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a7d1612be0 0x55a7d1617de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:47626/0 (socket says 172.21.15.2:47626) 2023-09-13T04:47:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.778+0000 7f30a3fff700 1 -- 172.21.15.2:0/3912463848 learned_addr learned my addr 172.21.15.2:0/3912463848 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.779+0000 7f30b0f9b700 1 --2- 172.21.15.2:0/3912463848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d160e840 0x55a7d16178a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.779+0000 7f30b179c700 1 --2- 172.21.15.2:0/3912463848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7d1613500 0x55a7d1618320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.779+0000 7f30a3fff700 1 -- 172.21.15.2:0/3912463848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d160e840 msgr2=0x55a7d16178a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.779+0000 7f30a3fff700 1 --2- 172.21.15.2:0/3912463848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d160e840 0x55a7d16178a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.779+0000 7f30a3fff700 1 -- 172.21.15.2:0/3912463848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7d1613500 msgr2=0x55a7d1618320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.779+0000 7f30a3fff700 1 --2- 172.21.15.2:0/3912463848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7d1613500 0x55a7d1618320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.779+0000 7f30a3fff700 1 -- 172.21.15.2:0/3912463848 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309c009950 con 0x55a7d1612be0 2023-09-13T04:47:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.781+0000 7f30a3fff700 1 --2- 172.21.15.2:0/3912463848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a7d1612be0 0x55a7d1617de0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f309c009820 tx=0x7f309c0051e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf45aa54104f9d61 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.781+0000 7f308bfff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:47:26.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.784+0000 7f308bfff700 1 -- 172.21.15.2:0/3912463848 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c010980 con 0x55a7d1612be0 2023-09-13T04:47:26.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.784+0000 7f308bfff700 1 -- 172.21.15.2:0/3912463848 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f309c010ae0 con 0x55a7d1612be0 2023-09-13T04:47:26.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.786+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/3912463848 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7d15d3ef0 con 0x55a7d1612be0 2023-09-13T04:47:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.789+0000 7f308bfff700 1 -- 172.21.15.2:0/3912463848 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f309c010cc0 con 0x55a7d1612be0 2023-09-13T04:47:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.789+0000 7f30c6ae1b80 10 client.26524 fetch_fsmap learned FSMap version 36 2023-09-13T04:47:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.789+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/3912463848 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55a7d1578740 con 0x55a7d1612be0 2023-09-13T04:47:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.789+0000 7f308bfff700 1 -- 172.21.15.2:0/3912463848 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a7d15d3ef0 con 0x55a7d1612be0 2023-09-13T04:47:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.790+0000 7f308bfff700 1 -- 172.21.15.2:0/3912463848 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f309c018030 con 0x55a7d1612be0 2023-09-13T04:47:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.790+0000 7f30c6ae1b80 10 client.26524 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:47:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.790+0000 7f30c6ae1b80 10 client.26524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:47:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.790+0000 7f30c6ae1b80 20 client.26524 populate_metadata read hostname 'smithi002' 2023-09-13T04:47:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.790+0000 7f30c6ae1b80 1 --2- 172.21.15.2:0/3912463848 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a7d1638fa0 0x55a7d163b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.790+0000 7f30c6ae1b80 4 client.26524 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:47:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.791+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/3912463848 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7d1559910 con 0x55a7d1638fa0 2023-09-13T04:47:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.791+0000 7f30b0f9b700 1 --2- 172.21.15.2:0/3912463848 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a7d1638fa0 0x55a7d163b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.794+0000 7f30b0f9b700 1 --2- 172.21.15.2:0/3912463848 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a7d1638fa0 0x55a7d163b3c0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e09acba2a2f6bcff server_cookie=7d54bfae9897be64 in_seq=0 out_seq=0 2023-09-13T04:47:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.794+0000 7f308bfff700 10 client.26524 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:47:26.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.801+0000 7f308bfff700 1 -- 172.21.15.2:0/3912463848 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a7d1559910 con 0x55a7d1638fa0 2023-09-13T04:47:26.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.801+0000 7f308bfff700 10 client.26524 handle_command_reply: tid=0 2023-09-13T04:47:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.802+0000 7f30c6ae1b80 1 client.26524 shutdown 2023-09-13T04:47:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.802+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/3912463848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a7d1612be0 msgr2=0x55a7d1617de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.802+0000 7f30c6ae1b80 1 --2- 172.21.15.2:0/3912463848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a7d1612be0 0x55a7d1617de0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f309c009820 tx=0x7f309c0051e0 comp rx=0 tx=0).stop 2023-09-13T04:47:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.802+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/3912463848 shutdown_connections 2023-09-13T04:47:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.802+0000 7f30c6ae1b80 1 --2- 172.21.15.2:0/3912463848 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a7d1638fa0 0x55a7d163b3c0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.802+0000 7f30c6ae1b80 1 --2- 172.21.15.2:0/3912463848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7d1613500 0x55a7d1618320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.802+0000 7f30c6ae1b80 1 --2- 172.21.15.2:0/3912463848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d160e840 0x55a7d16178a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.802+0000 7f30c6ae1b80 1 --2- 172.21.15.2:0/3912463848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a7d1612be0 0x55a7d1617de0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.802+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/3912463848 >> 172.21.15.2:0/3912463848 conn(0x55a7d156cc20 msgr2=0x55a7d1614330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.803+0000 7f308bfff700 0 client.26524 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:47:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.803+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/3912463848 shutdown_connections 2023-09-13T04:47:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.803+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/3912463848 wait complete. 2023-09-13T04:47:26.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.803+0000 7f30c6ae1b80 20 client.26524 trim_cache size 0 max 16384 2023-09-13T04:47:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.834+0000 7f30c6ae1b80 1 Processor -- start 2023-09-13T04:47:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.835+0000 7f30c6ae1b80 1 -- start start 2023-09-13T04:47:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.835+0000 7f30c6ae1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d160e840 0x55a7d162ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.835+0000 7f30c6ae1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7d1612be0 0x55a7d1603730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.835+0000 7f30c6ae1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a7d1613500 0x55a7d1618150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.835+0000 7f30c6ae1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d1615e20 con 0x55a7d1612be0 2023-09-13T04:47:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.835+0000 7f30c6ae1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d160d460 con 0x55a7d160e840 2023-09-13T04:47:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.835+0000 7f30c6ae1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d1609dc0 con 0x55a7d1613500 2023-09-13T04:47:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.835+0000 7f30b179c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a7d1613500 0x55a7d1618150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.836+0000 7f30b179c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a7d1613500 0x55a7d1618150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:47636/0 (socket says 172.21.15.2:47636) 2023-09-13T04:47:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.836+0000 7f30b179c700 1 -- 172.21.15.2:0/1955987321 learned_addr learned my addr 172.21.15.2:0/1955987321 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.836+0000 7f30b0f9b700 1 --2- 172.21.15.2:0/1955987321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d160e840 0x55a7d162ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.836+0000 7f30b0f9b700 1 -- 172.21.15.2:0/1955987321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a7d1613500 msgr2=0x55a7d1618150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.836+0000 7f30b0f9b700 1 --2- 172.21.15.2:0/1955987321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a7d1613500 0x55a7d1618150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.836+0000 7f30b0f9b700 1 -- 172.21.15.2:0/1955987321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7d1612be0 msgr2=0x55a7d1603730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.836+0000 7f30a3fff700 1 --2- 172.21.15.2:0/1955987321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7d1612be0 0x55a7d1603730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.836+0000 7f30b0f9b700 1 --2- 172.21.15.2:0/1955987321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7d1612be0 0x55a7d1603730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.836+0000 7f30b0f9b700 1 -- 172.21.15.2:0/1955987321 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7d163fff0 con 0x55a7d160e840 2023-09-13T04:47:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.837+0000 7f30a3fff700 1 --2- 172.21.15.2:0/1955987321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7d1612be0 0x55a7d1603730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:47:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.837+0000 7f30b0f9b700 1 --2- 172.21.15.2:0/1955987321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d160e840 0x55a7d162ffe0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55a7d163bca0 tx=0x7f30a8010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=217a0ef422efd42b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.838+0000 7f30937fe700 1 -- 172.21.15.2:0/1955987321 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a8025150 con 0x55a7d160e840 2023-09-13T04:47:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.838+0000 7f30937fe700 1 -- 172.21.15.2:0/1955987321 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30a80252b0 con 0x55a7d160e840 2023-09-13T04:47:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.838+0000 7f30937fe700 1 -- 172.21.15.2:0/1955987321 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a8021660 con 0x55a7d160e840 2023-09-13T04:47:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.839+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/1955987321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d160e840 msgr2=0x55a7d162ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.839+0000 7f30c6ae1b80 1 --2- 172.21.15.2:0/1955987321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d160e840 0x55a7d162ffe0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55a7d163bca0 tx=0x7f30a8010000 comp rx=0 tx=0).stop 2023-09-13T04:47:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.841+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/1955987321 shutdown_connections 2023-09-13T04:47:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.841+0000 7f30c6ae1b80 1 --2- 172.21.15.2:0/1955987321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a7d1613500 0x55a7d1618150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.841+0000 7f30c6ae1b80 1 --2- 172.21.15.2:0/1955987321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d160e840 0x55a7d162ffe0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.841+0000 7f30c6ae1b80 1 --2- 172.21.15.2:0/1955987321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7d1612be0 0x55a7d1603730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.841+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/1955987321 >> 172.21.15.2:0/1955987321 conn(0x55a7d156cc20 msgr2=0x55a7d163afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.846+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/1955987321 shutdown_connections 2023-09-13T04:47:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.846+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/1955987321 wait complete. 2023-09-13T04:47:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.847+0000 7f30c6ae1b80 1 -- start start 2023-09-13T04:47:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.847+0000 7f30c6ae1b80 1 Processor -- start 2023-09-13T04:47:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.848+0000 7f30c6ae1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a7d160e840 0x55a7d162dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.848+0000 7f30c6ae1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d1612be0 0x55a7d162e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.848+0000 7f30c6ae1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7d1613500 0x55a7d162ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.848+0000 7f30c6ae1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d16303c0 con 0x55a7d1613500 2023-09-13T04:47:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.848+0000 7f30c6ae1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d168be00 con 0x55a7d1612be0 2023-09-13T04:47:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.848+0000 7f30c6ae1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d168bfc0 con 0x55a7d160e840 2023-09-13T04:47:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.848+0000 7f30c6ae1b80 10 client.0 _ll_register_callbacks cb 0x55a7d14fc620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:47:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.849+0000 7f30a3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d1612be0 0x55a7d162e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.849+0000 7f30a3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d1612be0 0x55a7d162e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:48328/0 (socket says 172.21.15.2:48328) 2023-09-13T04:47:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.849+0000 7f30a3fff700 1 -- 172.21.15.2:0/2968669029 learned_addr learned my addr 172.21.15.2:0/2968669029 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.849+0000 7f30b0f9b700 1 --2- 172.21.15.2:0/2968669029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a7d160e840 0x55a7d162dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.850+0000 7f30b179c700 1 --2- 172.21.15.2:0/2968669029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7d1613500 0x55a7d162ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.850+0000 7f30a3fff700 1 -- 172.21.15.2:0/2968669029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a7d160e840 msgr2=0x55a7d162dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.850+0000 7f30a3fff700 1 --2- 172.21.15.2:0/2968669029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a7d160e840 0x55a7d162dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.851+0000 7f30a3fff700 1 -- 172.21.15.2:0/2968669029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7d1613500 msgr2=0x55a7d162ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.851+0000 7f30a3fff700 1 --2- 172.21.15.2:0/2968669029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7d1613500 0x55a7d162ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.851+0000 7f30a3fff700 1 -- 172.21.15.2:0/2968669029 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a800ddc0 con 0x55a7d1612be0 2023-09-13T04:47:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.851+0000 7f30a3fff700 1 --2- 172.21.15.2:0/2968669029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d1612be0 0x55a7d162e510 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f309c004790 tx=0x7f309c009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f05dfd5713d288d7 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.851+0000 7f308bfff700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:47:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.852+0000 7f308bfff700 1 -- 172.21.15.2:0/2968669029 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c010b80 con 0x55a7d1612be0 2023-09-13T04:47:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.852+0000 7f308bfff700 1 -- 172.21.15.2:0/2968669029 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f309c010ce0 con 0x55a7d1612be0 2023-09-13T04:47:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.852+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/2968669029 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f309c004cf0 con 0x55a7d1612be0 2023-09-13T04:47:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.852+0000 7f308bfff700 1 -- 172.21.15.2:0/2968669029 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f309c030730 con 0x55a7d1612be0 2023-09-13T04:47:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.853+0000 7f30c6ae1b80 10 client.26516 fetch_fsmap learned FSMap version 36 2023-09-13T04:47:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.853+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/2968669029 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55a7d138cd80 con 0x55a7d1612be0 2023-09-13T04:47:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.853+0000 7f308bfff700 1 -- 172.21.15.2:0/2968669029 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f309c004cf0 con 0x55a7d1612be0 2023-09-13T04:47:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.854+0000 7f308bfff700 1 -- 172.21.15.2:0/2968669029 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f309c037020 con 0x55a7d1612be0 2023-09-13T04:47:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.854+0000 7f30c6ae1b80 10 client.26516 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:47:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.854+0000 7f30c6ae1b80 10 client.26516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:47:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.854+0000 7f30c6ae1b80 20 client.26516 populate_metadata read hostname 'smithi002' 2023-09-13T04:47:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.854+0000 7f30c6ae1b80 1 --2- 172.21.15.2:0/2968669029 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a7d161af40 0x55a7d161d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.854+0000 7f30c6ae1b80 4 client.26516 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:47:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.854+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/2968669029 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a7d1519ff0 con 0x55a7d161af40 2023-09-13T04:47:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.855+0000 7f30b0f9b700 1 --2- 172.21.15.2:0/2968669029 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a7d161af40 0x55a7d161d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.860+0000 7f308bfff700 10 client.26516 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:47:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:26.860+0000 7f30b0f9b700 1 --2- 172.21.15.2:0/2968669029 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a7d161af40 0x55a7d161d360 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52acf9e669e249f server_cookie=6437a7a072df44f0 in_seq=0 out_seq=0 2023-09-13T04:47:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.159+0000 7f308bfff700 1 -- 172.21.15.2:0/2968669029 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a7d1519ff0 con 0x55a7d161af40 2023-09-13T04:47:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.159+0000 7f308bfff700 10 client.26516 handle_command_reply: tid=0 2023-09-13T04:47:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.159+0000 7f30c6ae1b80 1 client.26516 shutdown 2023-09-13T04:47:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.160+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/2968669029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d1612be0 msgr2=0x55a7d162e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.160+0000 7f30c6ae1b80 1 --2- 172.21.15.2:0/2968669029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d1612be0 0x55a7d162e510 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f309c004790 tx=0x7f309c009fd0 comp rx=0 tx=0).stop 2023-09-13T04:47:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.160+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/2968669029 shutdown_connections 2023-09-13T04:47:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.160+0000 7f30c6ae1b80 1 --2- 172.21.15.2:0/2968669029 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55a7d161af40 0x55a7d161d360 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.160+0000 7f30c6ae1b80 1 --2- 172.21.15.2:0/2968669029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7d1613500 0x55a7d162ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.160+0000 7f30c6ae1b80 1 --2- 172.21.15.2:0/2968669029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a7d160e840 0x55a7d162dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.160+0000 7f308bfff700 0 client.26516 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:47:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.160+0000 7f30c6ae1b80 1 --2- 172.21.15.2:0/2968669029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d1612be0 0x55a7d162e510 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.160+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/2968669029 >> 172.21.15.2:0/2968669029 conn(0x55a7d156cc20 msgr2=0x55a7d160c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.160+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/2968669029 shutdown_connections 2023-09-13T04:47:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.160+0000 7f30c6ae1b80 1 -- 172.21.15.2:0/2968669029 wait complete. 2023-09-13T04:47:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.160+0000 7f30c6ae1b80 20 client.26516 trim_cache size 0 max 16384 2023-09-13T04:47:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:47:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:47:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:47:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:47:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.161+0000 7f30b31ff700 1 -- 172.21.15.2:0/3960120565 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f30940a0980 msgr2=0x7f30940a2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.161+0000 7f30b31ff700 1 --2- 172.21.15.2:0/3960120565 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f30940a0980 0x7f30940a2e40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55a7d1613900 tx=0x7f30a8009040 comp rx=0 tx=0).stop 2023-09-13T04:47:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.161+0000 7f30b31ff700 1 -- 172.21.15.2:0/3960120565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30ac111c60 msgr2=0x7f30ac1a1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.161+0000 7f30b31ff700 1 --2- 172.21.15.2:0/3960120565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30ac111c60 0x7f30ac1a1ee0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f30a4009230 tx=0x7f30a400b560 comp rx=0 tx=0).stop 2023-09-13T04:47:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.161+0000 7f30b31ff700 1 -- 172.21.15.2:0/3960120565 shutdown_connections 2023-09-13T04:47:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.161+0000 7f30b31ff700 1 --2- 172.21.15.2:0/3960120565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30ac111c60 0x7f30ac1a1ee0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.161+0000 7f30b31ff700 1 --2- 172.21.15.2:0/3960120565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30ac073050 0x7f30ac19d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.161+0000 7f30b31ff700 1 --2- 172.21.15.2:0/3960120565 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f30940a0980 0x7f30940a2e40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.161+0000 7f30b31ff700 1 --2- 172.21.15.2:0/3960120565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30ac073970 0x7f30ac19d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.161+0000 7f30b31ff700 1 -- 172.21.15.2:0/3960120565 >> 172.21.15.2:0/3960120565 conn(0x7f30ac0fc740 msgr2=0x7f30ac0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.161+0000 7f30b31ff700 1 -- 172.21.15.2:0/3960120565 shutdown_connections 2023-09-13T04:47:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:27.161+0000 7f30b31ff700 1 -- 172.21.15.2:0/3960120565 wait complete. 2023-09-13T04:47:27.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:27 smithi149 ceph-mon[133315]: pgmap v215: 129 pgs: 129 active+clean; 882 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 259 op/s 2023-09-13T04:47:27.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:27 smithi156 ceph-mon[133675]: pgmap v215: 129 pgs: 129 active+clean; 882 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 259 op/s 2023-09-13T04:47:27.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:27 smithi002 ceph-mon[97651]: pgmap v215: 129 pgs: 129 active+clean; 882 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 259 op/s 2023-09-13T04:47:29.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:29 smithi002 ceph-mon[97651]: pgmap v216: 129 pgs: 129 active+clean; 885 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 263 op/s 2023-09-13T04:47:29.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:29 smithi149 ceph-mon[133315]: pgmap v216: 129 pgs: 129 active+clean; 885 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 263 op/s 2023-09-13T04:47:29.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:29 smithi156 ceph-mon[133675]: pgmap v216: 129 pgs: 129 active+clean; 885 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 263 op/s 2023-09-13T04:47:32.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:31 smithi149 ceph-mon[133315]: pgmap v217: 129 pgs: 129 active+clean; 893 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.3 MiB/s wr, 339 op/s 2023-09-13T04:47:32.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:31 smithi156 ceph-mon[133675]: pgmap v217: 129 pgs: 129 active+clean; 893 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.3 MiB/s wr, 339 op/s 2023-09-13T04:47:32.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:31 smithi002 ceph-mon[97651]: pgmap v217: 129 pgs: 129 active+clean; 893 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.3 MiB/s wr, 339 op/s 2023-09-13T04:47:34.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:33 smithi149 ceph-mon[133315]: pgmap v218: 129 pgs: 129 active+clean; 893 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 340 op/s 2023-09-13T04:47:34.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:33 smithi156 ceph-mon[133675]: pgmap v218: 129 pgs: 129 active+clean; 893 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 340 op/s 2023-09-13T04:47:34.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:33 smithi002 ceph-mon[97651]: pgmap v218: 129 pgs: 129 active+clean; 893 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 340 op/s 2023-09-13T04:47:36.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:35 smithi149 ceph-mon[133315]: pgmap v219: 129 pgs: 129 active+clean; 900 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 455 op/s 2023-09-13T04:47:36.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:35 smithi156 ceph-mon[133675]: pgmap v219: 129 pgs: 129 active+clean; 900 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 455 op/s 2023-09-13T04:47:36.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:35 smithi002 ceph-mon[97651]: pgmap v219: 129 pgs: 129 active+clean; 900 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 455 op/s 2023-09-13T04:47:38.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:37 smithi149 ceph-mon[133315]: pgmap v220: 129 pgs: 129 active+clean; 900 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 407 op/s 2023-09-13T04:47:38.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:37 smithi156 ceph-mon[133675]: pgmap v220: 129 pgs: 129 active+clean; 900 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 407 op/s 2023-09-13T04:47:38.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:37 smithi002 ceph-mon[97651]: pgmap v220: 129 pgs: 129 active+clean; 900 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 407 op/s 2023-09-13T04:47:40.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:39 smithi149 ceph-mon[133315]: pgmap v221: 129 pgs: 129 active+clean; 900 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 431 op/s 2023-09-13T04:47:40.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:39 smithi156 ceph-mon[133675]: pgmap v221: 129 pgs: 129 active+clean; 900 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 431 op/s 2023-09-13T04:47:40.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:39 smithi002 ceph-mon[97651]: pgmap v221: 129 pgs: 129 active+clean; 900 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 431 op/s 2023-09-13T04:47:40.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.318+0000 7f4c977ff700 1 -- 172.21.15.2:0/2114345991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c90100ed0 msgr2=0x7f4c901012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:40.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.318+0000 7f4c977ff700 1 --2- 172.21.15.2:0/2114345991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c90100ed0 0x7f4c901012b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009990 tx=0x7f4c8c009ca0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.318+0000 7f4c977ff700 1 -- 172.21.15.2:0/2114345991 shutdown_connections 2023-09-13T04:47:40.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.318+0000 7f4c977ff700 1 --2- 172.21.15.2:0/2114345991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c90105ff0 0x7f4c9010db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.318+0000 7f4c977ff700 1 --2- 172.21.15.2:0/2114345991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c90100ed0 0x7f4c901012b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.318+0000 7f4c977ff700 1 --2- 172.21.15.2:0/2114345991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c90101880 0x7f4c901058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.318+0000 7f4c977ff700 1 -- 172.21.15.2:0/2114345991 >> 172.21.15.2:0/2114345991 conn(0x7f4c900fc750 msgr2=0x7f4c900feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.318+0000 7f4c977ff700 1 -- 172.21.15.2:0/2114345991 shutdown_connections 2023-09-13T04:47:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.318+0000 7f4c977ff700 1 -- 172.21.15.2:0/2114345991 wait complete. 2023-09-13T04:47:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.319+0000 7f4c977ff700 1 Processor -- start 2023-09-13T04:47:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.320+0000 7f4c977ff700 1 -- start start 2023-09-13T04:47:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.320+0000 7f4c977ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c90100ed0 0x7f4c90196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.320+0000 7f4c977ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c90105ff0 0x7f4c90197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.320+0000 7f4c977ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c9019b970 0x7f4c9019bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.320+0000 7f4c977ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9019c3f0 con 0x7f4c9019b970 2023-09-13T04:47:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.320+0000 7f4c977ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9019c560 con 0x7f4c90100ed0 2023-09-13T04:47:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.320+0000 7f4c977ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9019c720 con 0x7f4c90105ff0 2023-09-13T04:47:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.321+0000 7f4c94d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c90105ff0 0x7f4c90197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.321+0000 7f4c9559b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c90100ed0 0x7f4c90196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.321+0000 7f4c94d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c90105ff0 0x7f4c90197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:44176/0 (socket says 172.21.15.2:44176) 2023-09-13T04:47:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.321+0000 7f4c95d9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c9019b970 0x7f4c9019bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.321+0000 7f4c94d9a700 1 -- 172.21.15.2:0/392683092 learned_addr learned my addr 172.21.15.2:0/392683092 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:40.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.321+0000 7f4c9559b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c90100ed0 0x7f4c90196dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:59528/0 (socket says 172.21.15.2:59528) 2023-09-13T04:47:40.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.321+0000 7f4c94d9a700 1 -- 172.21.15.2:0/392683092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c90100ed0 msgr2=0x7f4c90196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:40.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.321+0000 7f4c94d9a700 1 --2- 172.21.15.2:0/392683092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c90100ed0 0x7f4c90196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.321+0000 7f4c94d9a700 1 -- 172.21.15.2:0/392683092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c9019b970 msgr2=0x7f4c9019bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.321+0000 7f4c94d9a700 1 --2- 172.21.15.2:0/392683092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c9019b970 0x7f4c9019bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.321+0000 7f4c94d9a700 1 -- 172.21.15.2:0/392683092 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c0095f0 con 0x7f4c90105ff0 2023-09-13T04:47:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.321+0000 7f4c94d9a700 1 --2- 172.21.15.2:0/392683092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c90105ff0 0x7f4c90197300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400e970 tx=0x7f4c8400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.321+0000 7f4c827fc700 1 -- 172.21.15.2:0/392683092 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c84009790 con 0x7f4c90105ff0 2023-09-13T04:47:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.321+0000 7f4c977ff700 1 -- 172.21.15.2:0/392683092 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c90072f40 con 0x7f4c90105ff0 2023-09-13T04:47:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.321+0000 7f4c827fc700 1 -- 172.21.15.2:0/392683092 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c84005020 con 0x7f4c90105ff0 2023-09-13T04:47:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.321+0000 7f4c827fc700 1 -- 172.21.15.2:0/392683092 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8400cb80 con 0x7f4c90105ff0 2023-09-13T04:47:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.322+0000 7f4c977ff700 1 -- 172.21.15.2:0/392683092 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c90073450 con 0x7f4c90105ff0 2023-09-13T04:47:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.325+0000 7f4c827fc700 1 -- 172.21.15.2:0/392683092 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f4c84005490 con 0x7f4c90105ff0 2023-09-13T04:47:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.326+0000 7f4cab08cb80 1 Processor -- start 2023-09-13T04:47:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.326+0000 7f4cab08cb80 1 -- start start 2023-09-13T04:47:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.326+0000 7f4c827fc700 1 --2- 172.21.15.2:0/392683092 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f4c7c0a07b0 0x7f4c7c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.326+0000 7f4c827fc700 1 -- 172.21.15.2:0/392683092 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f4c84014070 con 0x7f4c90105ff0 2023-09-13T04:47:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.326+0000 7f4c95d9c700 1 --2- 172.21.15.2:0/392683092 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f4c7c0a07b0 0x7f4c7c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.326+0000 7f4cab08cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f119533be0 0x55f119533fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.327+0000 7f4cab08cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119534500 0x55f11952f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.327+0000 7f4cab08cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11952f840 0x55f11952fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.327+0000 7f4cab08cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f119488720 con 0x55f119533be0 2023-09-13T04:47:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.327+0000 7f4cab08cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f119486040 con 0x55f119534500 2023-09-13T04:47:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.327+0000 7f4cab08cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f119481930 con 0x55f11952f840 2023-09-13T04:47:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.327+0000 7f4c9559b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f119533be0 0x55f119533fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.327+0000 7f4c9559b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f119533be0 0x55f119533fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39526/0 (socket says 172.21.15.2:39526) 2023-09-13T04:47:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.327+0000 7f4c94d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119534500 0x55f11952f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.327+0000 7f4c9559b700 1 -- 172.21.15.2:0/1670750747 learned_addr learned my addr 172.21.15.2:0/1670750747 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.327+0000 7f4c95d9c700 1 --2- 172.21.15.2:0/1670750747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11952f840 0x55f11952fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.327+0000 7f4c94d9a700 1 -- 172.21.15.2:0/1670750747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11952f840 msgr2=0x55f11952fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.327+0000 7f4c94d9a700 1 --2- 172.21.15.2:0/1670750747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11952f840 0x55f11952fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.327+0000 7f4c94d9a700 1 -- 172.21.15.2:0/1670750747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f119533be0 msgr2=0x55f119533fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.327+0000 7f4c94d9a700 1 --2- 172.21.15.2:0/1670750747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f119533be0 0x55f119533fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.327+0000 7f4c94d9a700 1 -- 172.21.15.2:0/1670750747 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1193015d0 con 0x55f119534500 2023-09-13T04:47:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.328+0000 7f4c95d9c700 1 --2- 172.21.15.2:0/392683092 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f4c7c0a07b0 0x7f4c7c0a2c70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4c88010b10 tx=0x7f4c88009920 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.328+0000 7f4c94d9a700 1 --2- 172.21.15.2:0/1670750747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119534500 0x55f11952f300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4c840c9880 tx=0x7f4c84099ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33e21d39345197a4 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.328+0000 7f4c7b7fe700 1 -- 172.21.15.2:0/1670750747 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c840a1650 con 0x55f119534500 2023-09-13T04:47:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.328+0000 7f4c7b7fe700 1 -- 172.21.15.2:0/1670750747 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c84097980 con 0x55f119534500 2023-09-13T04:47:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.328+0000 7f4c7b7fe700 1 -- 172.21.15.2:0/1670750747 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c840c9b00 con 0x55f119534500 2023-09-13T04:47:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.328+0000 7f4cab08cb80 1 -- 172.21.15.2:0/1670750747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119534500 msgr2=0x55f11952f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.328+0000 7f4cab08cb80 1 --2- 172.21.15.2:0/1670750747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119534500 0x55f11952f300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4c840c9880 tx=0x7f4c84099ce0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.329+0000 7f4cab08cb80 1 -- 172.21.15.2:0/1670750747 shutdown_connections 2023-09-13T04:47:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.329+0000 7f4cab08cb80 1 --2- 172.21.15.2:0/1670750747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11952f840 0x55f11952fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.329+0000 7f4cab08cb80 1 --2- 172.21.15.2:0/1670750747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f119533be0 0x55f119533fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.329+0000 7f4cab08cb80 1 --2- 172.21.15.2:0/1670750747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119534500 0x55f11952f300 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.329+0000 7f4cab08cb80 1 -- 172.21.15.2:0/1670750747 >> 172.21.15.2:0/1670750747 conn(0x55f11948dc20 msgr2=0x55f119529b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.329+0000 7f4cab08cb80 1 -- 172.21.15.2:0/1670750747 shutdown_connections 2023-09-13T04:47:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.329+0000 7f4cab08cb80 1 -- 172.21.15.2:0/1670750747 wait complete. 2023-09-13T04:47:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.329+0000 7f4cab08cb80 1 -- start start 2023-09-13T04:47:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.330+0000 7f4cab08cb80 1 Processor -- start 2023-09-13T04:47:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.330+0000 7f4cab08cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f11952f840 0x55f11953b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.330+0000 7f4cab08cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119533be0 0x55f1195388c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.330+0000 7f4cab08cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f119534500 0x55f119539000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.330+0000 7f4cab08cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11941fb50 con 0x55f11952f840 2023-09-13T04:47:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.330+0000 7f4cab08cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11941f850 con 0x55f119533be0 2023-09-13T04:47:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.330+0000 7f4cab08cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1193038c0 con 0x55f119534500 2023-09-13T04:47:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.330+0000 7f4cab08cb80 10 client.0 _ll_register_callbacks cb 0x55f11941d620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:47:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.330+0000 7f4c9559b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f11952f840 0x55f11953b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.330+0000 7f4c94d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119533be0 0x55f1195388c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.330+0000 7f4c94d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119533be0 0x55f1195388c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:59550/0 (socket says 172.21.15.2:59550) 2023-09-13T04:47:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.330+0000 7f4c94d9a700 1 -- 172.21.15.2:0/4114692348 learned_addr learned my addr 172.21.15.2:0/4114692348 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.330+0000 7f4c95d9c700 1 --2- 172.21.15.2:0/4114692348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f119534500 0x55f119539000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.331+0000 7f4c94d9a700 1 -- 172.21.15.2:0/4114692348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f119534500 msgr2=0x55f119539000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.331+0000 7f4c94d9a700 1 --2- 172.21.15.2:0/4114692348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f119534500 0x55f119539000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.331+0000 7f4c94d9a700 1 -- 172.21.15.2:0/4114692348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f11952f840 msgr2=0x55f11953b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.331+0000 7f4c94d9a700 1 --2- 172.21.15.2:0/4114692348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f11952f840 0x55f11953b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.331+0000 7f4c94d9a700 1 -- 172.21.15.2:0/4114692348 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c90073450 con 0x55f119533be0 2023-09-13T04:47:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.331+0000 7f4c94d9a700 1 --2- 172.21.15.2:0/4114692348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119533be0 0x55f1195388c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4c840a1120 tx=0x7f4c8409d540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dced50bccced80bd server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.331+0000 7f4c78ff9700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:47:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.331+0000 7f4c78ff9700 1 -- 172.21.15.2:0/4114692348 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c840a16d0 con 0x55f119533be0 2023-09-13T04:47:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.331+0000 7f4cab08cb80 1 -- 172.21.15.2:0/4114692348 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1194f4ef0 con 0x55f119533be0 2023-09-13T04:47:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.331+0000 7f4c78ff9700 1 -- 172.21.15.2:0/4114692348 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f4c840c9b00 con 0x55f119533be0 2023-09-13T04:47:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.331+0000 7f4c78ff9700 1 -- 172.21.15.2:0/4114692348 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c840c9cc0 con 0x55f119533be0 2023-09-13T04:47:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.331+0000 7f4cab08cb80 10 client.26528 fetch_fsmap learned FSMap version 36 2023-09-13T04:47:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.331+0000 7f4cab08cb80 1 -- 172.21.15.2:0/4114692348 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55f119499740 con 0x55f119533be0 2023-09-13T04:47:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.332+0000 7f4c78ff9700 1 -- 172.21.15.2:0/4114692348 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c8407e080 con 0x55f119533be0 2023-09-13T04:47:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.332+0000 7f4c78ff9700 1 -- 172.21.15.2:0/4114692348 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f4c84005190 con 0x55f119533be0 2023-09-13T04:47:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.332+0000 7f4cab08cb80 10 client.26528 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:47:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.332+0000 7f4cab08cb80 10 client.26528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:47:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.332+0000 7f4cab08cb80 20 client.26528 populate_metadata read hostname 'smithi002' 2023-09-13T04:47:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.332+0000 7f4cab08cb80 1 --2- 172.21.15.2:0/4114692348 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f119548b60 0x55f11954af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.332+0000 7f4cab08cb80 4 client.26528 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:47:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.332+0000 7f4cab08cb80 1 -- 172.21.15.2:0/4114692348 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f11947a910 con 0x55f119548b60 2023-09-13T04:47:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.332+0000 7f4c9559b700 1 --2- 172.21.15.2:0/4114692348 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f119548b60 0x55f11954af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.333+0000 7f4c9559b700 1 --2- 172.21.15.2:0/4114692348 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f119548b60 0x55f11954af80 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ddce22c099841c7 server_cookie=590d29c97a3192b0 in_seq=0 out_seq=0 2023-09-13T04:47:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.333+0000 7f4c78ff9700 10 client.26528 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:47:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.336+0000 7f4c78ff9700 1 -- 172.21.15.2:0/4114692348 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f11947a910 con 0x55f119548b60 2023-09-13T04:47:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.336+0000 7f4c78ff9700 10 client.26528 handle_command_reply: tid=0 2023-09-13T04:47:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.336+0000 7f4cab08cb80 1 client.26528 shutdown 2023-09-13T04:47:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.336+0000 7f4cab08cb80 1 -- 172.21.15.2:0/4114692348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119533be0 msgr2=0x55f1195388c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.336+0000 7f4cab08cb80 1 --2- 172.21.15.2:0/4114692348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119533be0 0x55f1195388c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4c840a1120 tx=0x7f4c8409d540 comp rx=0 tx=0).stop 2023-09-13T04:47:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.336+0000 7f4cab08cb80 1 -- 172.21.15.2:0/4114692348 shutdown_connections 2023-09-13T04:47:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.336+0000 7f4cab08cb80 1 --2- 172.21.15.2:0/4114692348 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f119548b60 0x55f11954af80 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.336+0000 7f4cab08cb80 1 --2- 172.21.15.2:0/4114692348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f119534500 0x55f119539000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.336+0000 7f4cab08cb80 1 --2- 172.21.15.2:0/4114692348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f11952f840 0x55f11953b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.336+0000 7f4cab08cb80 1 --2- 172.21.15.2:0/4114692348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119533be0 0x55f1195388c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.336+0000 7f4c78ff9700 0 client.26528 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:47:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.336+0000 7f4cab08cb80 1 -- 172.21.15.2:0/4114692348 >> 172.21.15.2:0/4114692348 conn(0x55f11948dc20 msgr2=0x55f119535330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.336+0000 7f4cab08cb80 1 -- 172.21.15.2:0/4114692348 shutdown_connections 2023-09-13T04:47:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.336+0000 7f4cab08cb80 1 -- 172.21.15.2:0/4114692348 wait complete. 2023-09-13T04:47:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.337+0000 7f4cab08cb80 20 client.26528 trim_cache size 0 max 16384 2023-09-13T04:47:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.353+0000 7f4cab08cb80 1 Processor -- start 2023-09-13T04:47:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.354+0000 7f4cab08cb80 1 -- start start 2023-09-13T04:47:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.354+0000 7f4cab08cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f11952f840 0x55f11953ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.354+0000 7f4cab08cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119533be0 0x55f119524730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.354+0000 7f4cab08cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f119534500 0x55f119529a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.354+0000 7f4cab08cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f119540560 con 0x55f11952f840 2023-09-13T04:47:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.354+0000 7f4cab08cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f119524ca0 con 0x55f119533be0 2023-09-13T04:47:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.354+0000 7f4cab08cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11954af50 con 0x55f119534500 2023-09-13T04:47:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.354+0000 7f4c94d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119533be0 0x55f119524730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.354+0000 7f4c9559b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f11952f840 0x55f11953ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.354+0000 7f4c95d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f119534500 0x55f119529a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.354+0000 7f4c94d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119533be0 0x55f119524730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:59564/0 (socket says 172.21.15.2:59564) 2023-09-13T04:47:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.354+0000 7f4c9559b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f11952f840 0x55f11953ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39550/0 (socket says 172.21.15.2:39550) 2023-09-13T04:47:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.354+0000 7f4c94d9a700 1 -- 172.21.15.2:0/2825189447 learned_addr learned my addr 172.21.15.2:0/2825189447 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.354+0000 7f4c94d9a700 1 -- 172.21.15.2:0/2825189447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f119534500 msgr2=0x55f119529a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.354+0000 7f4c94d9a700 1 --2- 172.21.15.2:0/2825189447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f119534500 0x55f119529a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.354+0000 7f4c94d9a700 1 -- 172.21.15.2:0/2825189447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f11952f840 msgr2=0x55f11953ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.354+0000 7f4c94d9a700 1 --2- 172.21.15.2:0/2825189447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f11952f840 0x55f11953ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.355+0000 7f4c94d9a700 1 -- 172.21.15.2:0/2825189447 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1192add80 con 0x55f119533be0 2023-09-13T04:47:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.355+0000 7f4c95d9c700 1 --2- 172.21.15.2:0/2825189447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f119534500 0x55f119529a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:47:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.355+0000 7f4c9559b700 1 --2- 172.21.15.2:0/2825189447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f11952f840 0x55f11953ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:47:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.355+0000 7f4c94d9a700 1 --2- 172.21.15.2:0/2825189447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119533be0 0x55f119524730 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4c84099570 tx=0x7f4c840959d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=616c0ae8f2d1cce1 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.355+0000 7f4c7bfff700 1 -- 172.21.15.2:0/2825189447 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c840b7630 con 0x55f119533be0 2023-09-13T04:47:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.355+0000 7f4c7bfff700 1 -- 172.21.15.2:0/2825189447 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f1194f4ef0 con 0x55f119533be0 2023-09-13T04:47:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.355+0000 7f4c7bfff700 1 -- 172.21.15.2:0/2825189447 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c84005190 con 0x55f119533be0 2023-09-13T04:47:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.356+0000 7f4cab08cb80 1 -- 172.21.15.2:0/2825189447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119533be0 msgr2=0x55f119524730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.356+0000 7f4cab08cb80 1 --2- 172.21.15.2:0/2825189447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119533be0 0x55f119524730 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4c84099570 tx=0x7f4c840959d0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.356+0000 7f4cab08cb80 1 -- 172.21.15.2:0/2825189447 shutdown_connections 2023-09-13T04:47:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.356+0000 7f4cab08cb80 1 --2- 172.21.15.2:0/2825189447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f119534500 0x55f119529a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.356+0000 7f4cab08cb80 1 --2- 172.21.15.2:0/2825189447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f11952f840 0x55f11953ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.356+0000 7f4cab08cb80 1 --2- 172.21.15.2:0/2825189447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119533be0 0x55f119524730 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.356+0000 7f4cab08cb80 1 -- 172.21.15.2:0/2825189447 >> 172.21.15.2:0/2825189447 conn(0x55f11948dc20 msgr2=0x55f11954f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.356+0000 7f4cab08cb80 1 -- 172.21.15.2:0/2825189447 shutdown_connections 2023-09-13T04:47:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.356+0000 7f4cab08cb80 1 -- 172.21.15.2:0/2825189447 wait complete. 2023-09-13T04:47:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.356+0000 7f4cab08cb80 1 -- start start 2023-09-13T04:47:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.356+0000 7f4cab08cb80 1 Processor -- start 2023-09-13T04:47:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.357+0000 7f4cab08cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f11952f840 0x55f11953e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.357+0000 7f4cab08cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f119533be0 0x55f11953e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.357+0000 7f4cab08cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119534500 0x55f11959b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.357+0000 7f4cab08cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11959be90 con 0x55f11952f840 2023-09-13T04:47:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.357+0000 7f4cab08cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11959bfd0 con 0x55f119534500 2023-09-13T04:47:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.357+0000 7f4cab08cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11959c140 con 0x55f119533be0 2023-09-13T04:47:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.357+0000 7f4cab08cb80 10 client.0 _ll_register_callbacks cb 0x55f11941d620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:47:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.357+0000 7f4c9559b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f11952f840 0x55f11953e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.357+0000 7f4c94d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f119533be0 0x55f11953e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.357+0000 7f4c95d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119534500 0x55f11959b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.357+0000 7f4c94d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f119533be0 0x55f11953e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:44208/0 (socket says 172.21.15.2:44208) 2023-09-13T04:47:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.357+0000 7f4c94d9a700 1 -- 172.21.15.2:0/157207607 learned_addr learned my addr 172.21.15.2:0/157207607 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.357+0000 7f4c94d9a700 1 -- 172.21.15.2:0/157207607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119534500 msgr2=0x55f11959b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.357+0000 7f4c94d9a700 1 --2- 172.21.15.2:0/157207607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119534500 0x55f11959b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.357+0000 7f4c94d9a700 1 -- 172.21.15.2:0/157207607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f11952f840 msgr2=0x55f11953e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.357+0000 7f4c94d9a700 1 --2- 172.21.15.2:0/157207607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f11952f840 0x55f11953e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.357+0000 7f4c94d9a700 1 -- 172.21.15.2:0/157207607 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9010e3e0 con 0x55f119533be0 2023-09-13T04:47:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.358+0000 7f4c94d9a700 1 --2- 172.21.15.2:0/157207607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f119533be0 0x55f11953e690 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4c84099d20 tx=0x7f4c840b1ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e88b7f891a99b945 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.358+0000 7f4c78ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:47:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.358+0000 7f4c78ff9700 1 -- 172.21.15.2:0/157207607 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c840995e0 con 0x55f119533be0 2023-09-13T04:47:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.358+0000 7f4c78ff9700 1 -- 172.21.15.2:0/157207607 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f4c840a1650 con 0x55f119533be0 2023-09-13T04:47:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.358+0000 7f4cab08cb80 1 -- 172.21.15.2:0/157207607 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c84097980 con 0x55f119533be0 2023-09-13T04:47:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.358+0000 7f4c78ff9700 1 -- 172.21.15.2:0/157207607 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c84095cf0 con 0x55f119533be0 2023-09-13T04:47:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.358+0000 7f4cab08cb80 10 client.26548 fetch_fsmap learned FSMap version 36 2023-09-13T04:47:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.358+0000 7f4cab08cb80 1 -- 172.21.15.2:0/157207607 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f4c88009030 con 0x55f119533be0 2023-09-13T04:47:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.358+0000 7f4c78ff9700 1 -- 172.21.15.2:0/157207607 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c84097980 con 0x55f119533be0 2023-09-13T04:47:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.358+0000 7f4c78ff9700 1 -- 172.21.15.2:0/157207607 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f4c840a1250 con 0x55f119533be0 2023-09-13T04:47:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.359+0000 7f4cab08cb80 10 client.26548 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:47:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.359+0000 7f4cab08cb80 10 client.26548 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:47:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.359+0000 7f4cab08cb80 20 client.26548 populate_metadata read hostname 'smithi002' 2023-09-13T04:47:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.359+0000 7f4cab08cb80 1 --2- 172.21.15.2:0/157207607 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f11959d4d0 0x55f11959f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.359+0000 7f4cab08cb80 4 client.26548 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:47:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.359+0000 7f4cab08cb80 1 -- 172.21.15.2:0/157207607 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f11943aff0 con 0x55f11959d4d0 2023-09-13T04:47:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.359+0000 7f4c9559b700 1 --2- 172.21.15.2:0/157207607 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f11959d4d0 0x55f11959f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.359+0000 7f4c9559b700 1 --2- 172.21.15.2:0/157207607 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f11959d4d0 0x55f11959f8f0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f6f19db705b3b08 server_cookie=7ff4fea75b26324 in_seq=0 out_seq=0 2023-09-13T04:47:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:40.359+0000 7f4c78ff9700 10 client.26548 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:47:41.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.014+0000 7f4c78ff9700 1 -- 172.21.15.2:0/157207607 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f11943aff0 con 0x55f11959d4d0 2023-09-13T04:47:41.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.014+0000 7f4c78ff9700 10 client.26548 handle_command_reply: tid=0 2023-09-13T04:47:41.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.014+0000 7f4cab08cb80 1 client.26548 shutdown 2023-09-13T04:47:41.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.014+0000 7f4cab08cb80 1 -- 172.21.15.2:0/157207607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f119533be0 msgr2=0x55f11953e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:41.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.014+0000 7f4cab08cb80 1 --2- 172.21.15.2:0/157207607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f119533be0 0x55f11953e690 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4c84099d20 tx=0x7f4c840b1ea0 comp rx=0 tx=0).stop 2023-09-13T04:47:41.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.014+0000 7f4cab08cb80 1 -- 172.21.15.2:0/157207607 shutdown_connections 2023-09-13T04:47:41.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.014+0000 7f4cab08cb80 1 --2- 172.21.15.2:0/157207607 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f11959d4d0 0x55f11959f8f0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:41.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.014+0000 7f4cab08cb80 1 --2- 172.21.15.2:0/157207607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f119534500 0x55f11959b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:41.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.014+0000 7f4cab08cb80 1 --2- 172.21.15.2:0/157207607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f11952f840 0x55f11953e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:41.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.014+0000 7f4cab08cb80 1 --2- 172.21.15.2:0/157207607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f119533be0 0x55f11953e690 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:41.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.014+0000 7f4c78ff9700 0 client.26548 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:47:41.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.014+0000 7f4cab08cb80 1 -- 172.21.15.2:0/157207607 >> 172.21.15.2:0/157207607 conn(0x55f11948dc20 msgr2=0x55f119529a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:41.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.014+0000 7f4cab08cb80 1 -- 172.21.15.2:0/157207607 shutdown_connections 2023-09-13T04:47:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.014+0000 7f4cab08cb80 1 -- 172.21.15.2:0/157207607 wait complete. 2023-09-13T04:47:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.015+0000 7f4cab08cb80 20 client.26548 trim_cache size 0 max 16384 2023-09-13T04:47:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:47:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:47:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:47:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:47:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.015+0000 7f4c977ff700 1 -- 172.21.15.2:0/392683092 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f4c7c0a07b0 msgr2=0x7f4c7c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.015+0000 7f4c977ff700 1 --2- 172.21.15.2:0/392683092 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f4c7c0a07b0 0x7f4c7c0a2c70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4c88010b10 tx=0x7f4c88009920 comp rx=0 tx=0).stop 2023-09-13T04:47:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.015+0000 7f4c977ff700 1 -- 172.21.15.2:0/392683092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c90105ff0 msgr2=0x7f4c90197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.015+0000 7f4c977ff700 1 --2- 172.21.15.2:0/392683092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c90105ff0 0x7f4c90197300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400e970 tx=0x7f4c8400ec80 comp rx=0 tx=0).stop 2023-09-13T04:47:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.016+0000 7f4c977ff700 1 -- 172.21.15.2:0/392683092 shutdown_connections 2023-09-13T04:47:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.016+0000 7f4c977ff700 1 --2- 172.21.15.2:0/392683092 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f4c7c0a07b0 0x7f4c7c0a2c70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.016+0000 7f4c977ff700 1 --2- 172.21.15.2:0/392683092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c9019b970 0x7f4c9019bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.016+0000 7f4c977ff700 1 --2- 172.21.15.2:0/392683092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c90100ed0 0x7f4c90196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.016+0000 7f4c977ff700 1 --2- 172.21.15.2:0/392683092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c90105ff0 0x7f4c90197300 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.016+0000 7f4c977ff700 1 -- 172.21.15.2:0/392683092 >> 172.21.15.2:0/392683092 conn(0x7f4c900fc750 msgr2=0x7f4c90104900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.016+0000 7f4c977ff700 1 -- 172.21.15.2:0/392683092 shutdown_connections 2023-09-13T04:47:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:41.016+0000 7f4c977ff700 1 -- 172.21.15.2:0/392683092 wait complete. 2023-09-13T04:47:42.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:41 smithi149 ceph-mon[133315]: pgmap v222: 129 pgs: 129 active+clean; 904 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 548 op/s 2023-09-13T04:47:42.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:41 smithi156 ceph-mon[133675]: pgmap v222: 129 pgs: 129 active+clean; 904 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 548 op/s 2023-09-13T04:47:42.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:41 smithi002 ceph-mon[97651]: pgmap v222: 129 pgs: 129 active+clean; 904 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 548 op/s 2023-09-13T04:47:44.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:43 smithi149 ceph-mon[133315]: pgmap v223: 129 pgs: 129 active+clean; 908 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 449 op/s 2023-09-13T04:47:44.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:43 smithi156 ceph-mon[133675]: pgmap v223: 129 pgs: 129 active+clean; 908 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 449 op/s 2023-09-13T04:47:44.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:43 smithi002 ceph-mon[97651]: pgmap v223: 129 pgs: 129 active+clean; 908 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 449 op/s 2023-09-13T04:47:46.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:45 smithi149 ceph-mon[133315]: pgmap v224: 129 pgs: 129 active+clean; 904 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 654 op/s 2023-09-13T04:47:46.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:45 smithi156 ceph-mon[133675]: pgmap v224: 129 pgs: 129 active+clean; 904 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 654 op/s 2023-09-13T04:47:46.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:45 smithi002 ceph-mon[97651]: pgmap v224: 129 pgs: 129 active+clean; 904 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 654 op/s 2023-09-13T04:47:48.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:47 smithi149 ceph-mon[133315]: pgmap v225: 129 pgs: 129 active+clean; 900 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 532 op/s 2023-09-13T04:47:48.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:47 smithi156 ceph-mon[133675]: pgmap v225: 129 pgs: 129 active+clean; 900 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 532 op/s 2023-09-13T04:47:48.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:47 smithi002 ceph-mon[97651]: pgmap v225: 129 pgs: 129 active+clean; 900 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 532 op/s 2023-09-13T04:47:50.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:49 smithi149 ceph-mon[133315]: pgmap v226: 129 pgs: 129 active+clean; 906 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 535 op/s 2023-09-13T04:47:50.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:49 smithi156 ceph-mon[133675]: pgmap v226: 129 pgs: 129 active+clean; 906 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 535 op/s 2023-09-13T04:47:50.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:49 smithi002 ceph-mon[97651]: pgmap v226: 129 pgs: 129 active+clean; 906 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 535 op/s 2023-09-13T04:47:52.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:51 smithi149 ceph-mon[133315]: pgmap v227: 129 pgs: 129 active+clean; 908 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 571 op/s 2023-09-13T04:47:52.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:51 smithi156 ceph-mon[133675]: pgmap v227: 129 pgs: 129 active+clean; 908 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 571 op/s 2023-09-13T04:47:52.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:51 smithi002 ceph-mon[97651]: pgmap v227: 129 pgs: 129 active+clean; 908 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 571 op/s 2023-09-13T04:47:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.167+0000 7f1f221ff700 1 -- 172.21.15.2:0/3981470647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f1c101880 msgr2=0x7f1f1c1058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.167+0000 7f1f221ff700 1 --2- 172.21.15.2:0/3981470647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f1c101880 0x7f1f1c1058c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009990 tx=0x7f1f10009ca0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.167+0000 7f1f221ff700 1 -- 172.21.15.2:0/3981470647 shutdown_connections 2023-09-13T04:47:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.167+0000 7f1f221ff700 1 --2- 172.21.15.2:0/3981470647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f1c105ff0 0x7f1f1c10db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.167+0000 7f1f221ff700 1 --2- 172.21.15.2:0/3981470647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f1c100ed0 0x7f1f1c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.167+0000 7f1f221ff700 1 --2- 172.21.15.2:0/3981470647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f1c101880 0x7f1f1c1058c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.167+0000 7f1f221ff700 1 -- 172.21.15.2:0/3981470647 >> 172.21.15.2:0/3981470647 conn(0x7f1f1c0fc750 msgr2=0x7f1f1c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.167+0000 7f1f221ff700 1 -- 172.21.15.2:0/3981470647 shutdown_connections 2023-09-13T04:47:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.167+0000 7f1f221ff700 1 -- 172.21.15.2:0/3981470647 wait complete. 2023-09-13T04:47:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.169+0000 7f1f221ff700 1 Processor -- start 2023-09-13T04:47:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.169+0000 7f1f221ff700 1 -- start start 2023-09-13T04:47:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.169+0000 7f1f221ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f1c100ed0 0x7f1f1c199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.170+0000 7f1f221ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f1c105ff0 0x7f1f1c199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.170+0000 7f1f221ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f1c19dbf0 0x7f1f1c19e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.170+0000 7f1f221ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c19e670 con 0x7f1f1c19dbf0 2023-09-13T04:47:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.170+0000 7f1f221ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c19e7e0 con 0x7f1f1c100ed0 2023-09-13T04:47:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.170+0000 7f1f221ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c19e9a0 con 0x7f1f1c105ff0 2023-09-13T04:47:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.170+0000 7f1f1b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f1c100ed0 0x7f1f1c199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.170+0000 7f1f1affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f1c105ff0 0x7f1f1c199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.170+0000 7f1f1b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f1c100ed0 0x7f1f1c199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:37338/0 (socket says 172.21.15.2:37338) 2023-09-13T04:47:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.170+0000 7f1f1b7fe700 1 -- 172.21.15.2:0/4277056913 learned_addr learned my addr 172.21.15.2:0/4277056913 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.170+0000 7f1f1bfff700 1 --2- 172.21.15.2:0/4277056913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f1c19dbf0 0x7f1f1c19e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.170+0000 7f1f1b7fe700 1 -- 172.21.15.2:0/4277056913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f1c105ff0 msgr2=0x7f1f1c199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.170+0000 7f1f1b7fe700 1 --2- 172.21.15.2:0/4277056913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f1c105ff0 0x7f1f1c199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.170+0000 7f1f1b7fe700 1 -- 172.21.15.2:0/4277056913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f1c19dbf0 msgr2=0x7f1f1c19e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.170+0000 7f1f1b7fe700 1 --2- 172.21.15.2:0/4277056913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f1c19dbf0 0x7f1f1c19e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.170+0000 7f1f1b7fe700 1 -- 172.21.15.2:0/4277056913 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f100095f0 con 0x7f1f1c100ed0 2023-09-13T04:47:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.170+0000 7f1f1bfff700 1 --2- 172.21.15.2:0/4277056913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f1c19dbf0 0x7f1f1c19e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:47:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.171+0000 7f1f1b7fe700 1 --2- 172.21.15.2:0/4277056913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f1c100ed0 0x7f1f1c199040 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c00e9a0 tx=0x7f1f0c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.171+0000 7f1f18ff9700 1 -- 172.21.15.2:0/4277056913 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c009790 con 0x7f1f1c100ed0 2023-09-13T04:47:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.171+0000 7f1f18ff9700 1 -- 172.21.15.2:0/4277056913 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f0c010430 con 0x7f1f1c100ed0 2023-09-13T04:47:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.171+0000 7f1f221ff700 1 -- 172.21.15.2:0/4277056913 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f1c1a5f00 con 0x7f1f1c100ed0 2023-09-13T04:47:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.171+0000 7f1f18ff9700 1 -- 172.21.15.2:0/4277056913 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c018680 con 0x7f1f1c100ed0 2023-09-13T04:47:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.171+0000 7f1f221ff700 1 -- 172.21.15.2:0/4277056913 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1c1a6380 con 0x7f1f1c100ed0 2023-09-13T04:47:54.190 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:53 smithi156 ceph-mon[133675]: pgmap v228: 129 pgs: 129 active+clean; 910 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 438 op/s 2023-09-13T04:47:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.174+0000 7f1f18ff9700 1 -- 172.21.15.2:0/4277056913 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f1f0c0105a0 con 0x7f1f1c100ed0 2023-09-13T04:47:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.176+0000 7f1f35af6b80 1 Processor -- start 2023-09-13T04:47:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.176+0000 7f1f35af6b80 1 -- start start 2023-09-13T04:47:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.177+0000 7f1f35af6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616916b3be0 0x5616916b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.177+0000 7f1f35af6b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616916b4500 0x5616916af300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.177+0000 7f1f35af6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916af840 0x5616916afc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.177+0000 7f1f35af6b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561691608720 con 0x5616916b4500 2023-09-13T04:47:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.177+0000 7f1f35af6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561691606040 con 0x5616916af840 2023-09-13T04:47:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.177+0000 7f1f35af6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561691601930 con 0x5616916b3be0 2023-09-13T04:47:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.177+0000 7f1f18ff9700 1 --2- 172.21.15.2:0/4277056913 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f1f040a07b0 0x7f1f040a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.177+0000 7f1f18ff9700 1 -- 172.21.15.2:0/4277056913 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f1f0c014070 con 0x7f1f1c100ed0 2023-09-13T04:47:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.177+0000 7f1f1affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916af840 0x5616916afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.177+0000 7f1f1affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916af840 0x5616916afc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:37350/0 (socket says 172.21.15.2:37350) 2023-09-13T04:47:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.177+0000 7f1f1affd700 1 -- 172.21.15.2:0/3392324882 learned_addr learned my addr 172.21.15.2:0/3392324882 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.177+0000 7f1f1bfff700 1 --2- 172.21.15.2:0/3392324882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616916b3be0 0x5616916b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.177+0000 7f1f1b7fe700 1 --2- 172.21.15.2:0/3392324882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616916b4500 0x5616916af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.177+0000 7f1f1bfff700 1 --2- 172.21.15.2:0/4277056913 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f1f040a07b0 0x7f1f040a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.178+0000 7f1f1affd700 1 -- 172.21.15.2:0/3392324882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616916b3be0 msgr2=0x5616916b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.178+0000 7f1f1affd700 1 --2- 172.21.15.2:0/3392324882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616916b3be0 0x5616916b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.178+0000 7f1f1affd700 1 -- 172.21.15.2:0/3392324882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616916b4500 msgr2=0x5616916af300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.178+0000 7f1f1affd700 1 --2- 172.21.15.2:0/3392324882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616916b4500 0x5616916af300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.178+0000 7f1f1affd700 1 -- 172.21.15.2:0/3392324882 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616914815d0 con 0x5616916af840 2023-09-13T04:47:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.178+0000 7f1f1b7fe700 1 --2- 172.21.15.2:0/3392324882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616916b4500 0x5616916af300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:47:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.178+0000 7f1f1affd700 1 --2- 172.21.15.2:0/3392324882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916af840 0x5616916afc80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009990 tx=0x7f1f10005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7671b4966c48f8be server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.178+0000 7f1f1bfff700 1 --2- 172.21.15.2:0/4277056913 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f1f040a07b0 0x7f1f040a2c70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x5616916aba70 tx=0x7f1f1400b040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.178+0000 7f1f01ffb700 1 -- 172.21.15.2:0/3392324882 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1001d070 con 0x5616916af840 2023-09-13T04:47:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.178+0000 7f1f01ffb700 1 -- 172.21.15.2:0/3392324882 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f1001e040 con 0x5616916af840 2023-09-13T04:47:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.178+0000 7f1f01ffb700 1 -- 172.21.15.2:0/3392324882 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f100047f0 con 0x5616916af840 2023-09-13T04:47:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.179+0000 7f1f35af6b80 1 -- 172.21.15.2:0/3392324882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916af840 msgr2=0x5616916afc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.179+0000 7f1f35af6b80 1 --2- 172.21.15.2:0/3392324882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916af840 0x5616916afc80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009990 tx=0x7f1f10005ea0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.179+0000 7f1f35af6b80 1 -- 172.21.15.2:0/3392324882 shutdown_connections 2023-09-13T04:47:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.179+0000 7f1f35af6b80 1 --2- 172.21.15.2:0/3392324882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916af840 0x5616916afc80 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.179+0000 7f1f35af6b80 1 --2- 172.21.15.2:0/3392324882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616916b3be0 0x5616916b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.179+0000 7f1f35af6b80 1 --2- 172.21.15.2:0/3392324882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616916b4500 0x5616916af300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.179+0000 7f1f35af6b80 1 -- 172.21.15.2:0/3392324882 >> 172.21.15.2:0/3392324882 conn(0x56169160dc20 msgr2=0x5616916a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.179+0000 7f1f35af6b80 1 -- 172.21.15.2:0/3392324882 shutdown_connections 2023-09-13T04:47:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.179+0000 7f1f35af6b80 1 -- 172.21.15.2:0/3392324882 wait complete. 2023-09-13T04:47:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.180+0000 7f1f35af6b80 1 -- start start 2023-09-13T04:47:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.180+0000 7f1f35af6b80 1 Processor -- start 2023-09-13T04:47:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.181+0000 7f1f35af6b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616916af840 0x5616916bb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.181+0000 7f1f35af6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616916b3be0 0x5616916b8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.181+0000 7f1f35af6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916b4500 0x5616916b8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.181+0000 7f1f35af6b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56169159fb50 con 0x5616916af840 2023-09-13T04:47:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.181+0000 7f1f35af6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56169159f850 con 0x5616916b4500 2023-09-13T04:47:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.181+0000 7f1f35af6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616914838c0 con 0x5616916b3be0 2023-09-13T04:47:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.181+0000 7f1f35af6b80 10 client.0 _ll_register_callbacks cb 0x56169159d620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:47:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.181+0000 7f1f1bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916b4500 0x5616916b8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.181+0000 7f1f1affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616916b3be0 0x5616916b8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.181+0000 7f1f1bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916b4500 0x5616916b8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:37366/0 (socket says 172.21.15.2:37366) 2023-09-13T04:47:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.181+0000 7f1f1bfff700 1 -- 172.21.15.2:0/401626922 learned_addr learned my addr 172.21.15.2:0/401626922 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.181+0000 7f1f1b7fe700 1 --2- 172.21.15.2:0/401626922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616916af840 0x5616916bb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.182+0000 7f1f1bfff700 1 -- 172.21.15.2:0/401626922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616916b3be0 msgr2=0x5616916b8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.182+0000 7f1f1bfff700 1 --2- 172.21.15.2:0/401626922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616916b3be0 0x5616916b8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.182+0000 7f1f1bfff700 1 -- 172.21.15.2:0/401626922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616916af840 msgr2=0x5616916bb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.182+0000 7f1f1bfff700 1 --2- 172.21.15.2:0/401626922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616916af840 0x5616916bb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.182+0000 7f1f1bfff700 1 -- 172.21.15.2:0/401626922 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1c10e3e0 con 0x5616916b4500 2023-09-13T04:47:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.182+0000 7f1f1bfff700 1 --2- 172.21.15.2:0/401626922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916b4500 0x5616916b8f50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1f1400ee60 tx=0x7f1f14004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=791e4fef05ac2c91 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.182+0000 7f1efaffd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:47:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.182+0000 7f1efaffd700 1 -- 172.21.15.2:0/401626922 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f14004bb0 con 0x5616916b4500 2023-09-13T04:47:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.182+0000 7f1f35af6b80 1 -- 172.21.15.2:0/401626922 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561691674ef0 con 0x5616916b4500 2023-09-13T04:47:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.183+0000 7f1efaffd700 1 -- 172.21.15.2:0/401626922 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f1f14020040 con 0x5616916b4500 2023-09-13T04:47:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.183+0000 7f1efaffd700 1 -- 172.21.15.2:0/401626922 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f14004d10 con 0x5616916b4500 2023-09-13T04:47:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.183+0000 7f1f35af6b80 10 client.26546 fetch_fsmap learned FSMap version 36 2023-09-13T04:47:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.183+0000 7f1f35af6b80 1 -- 172.21.15.2:0/401626922 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x561691619740 con 0x5616916b4500 2023-09-13T04:47:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.183+0000 7f1efaffd700 1 -- 172.21.15.2:0/401626922 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f14031030 con 0x5616916b4500 2023-09-13T04:47:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.183+0000 7f1efaffd700 1 -- 172.21.15.2:0/401626922 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f1f1401b940 con 0x5616916b4500 2023-09-13T04:47:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.183+0000 7f1f35af6b80 10 client.26546 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:47:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.183+0000 7f1f35af6b80 10 client.26546 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:47:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.183+0000 7f1f35af6b80 20 client.26546 populate_metadata read hostname 'smithi002' 2023-09-13T04:47:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.183+0000 7f1f35af6b80 1 --2- 172.21.15.2:0/401626922 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5616916c9020 0x5616916cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.183+0000 7f1f35af6b80 4 client.26546 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:47:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.183+0000 7f1f35af6b80 1 -- 172.21.15.2:0/401626922 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616915fa910 con 0x5616916c9020 2023-09-13T04:47:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.184+0000 7f1f1b7fe700 1 --2- 172.21.15.2:0/401626922 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5616916c9020 0x5616916cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.184+0000 7f1f1b7fe700 1 --2- 172.21.15.2:0/401626922 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5616916c9020 0x5616916cb440 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e413771b6396fae3 server_cookie=982497ef3455e156 in_seq=0 out_seq=0 2023-09-13T04:47:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.184+0000 7f1efaffd700 10 client.26546 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:47:54.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.185+0000 7f1efaffd700 1 -- 172.21.15.2:0/401626922 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5616915fa910 con 0x5616916c9020 2023-09-13T04:47:54.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.185+0000 7f1efaffd700 10 client.26546 handle_command_reply: tid=0 2023-09-13T04:47:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.185+0000 7f1f35af6b80 1 client.26546 shutdown 2023-09-13T04:47:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.185+0000 7f1f35af6b80 1 -- 172.21.15.2:0/401626922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916b4500 msgr2=0x5616916b8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.185+0000 7f1f35af6b80 1 --2- 172.21.15.2:0/401626922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916b4500 0x5616916b8f50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1f1400ee60 tx=0x7f1f14004ad0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.185+0000 7f1f35af6b80 1 -- 172.21.15.2:0/401626922 shutdown_connections 2023-09-13T04:47:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.185+0000 7f1f35af6b80 1 --2- 172.21.15.2:0/401626922 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5616916c9020 0x5616916cb440 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.185+0000 7f1f35af6b80 1 --2- 172.21.15.2:0/401626922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916b4500 0x5616916b8f50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.185+0000 7f1f35af6b80 1 --2- 172.21.15.2:0/401626922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616916af840 0x5616916bb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.185+0000 7f1f35af6b80 1 --2- 172.21.15.2:0/401626922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616916b3be0 0x5616916b8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.185+0000 7f1efaffd700 0 client.26546 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:47:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.185+0000 7f1f35af6b80 1 -- 172.21.15.2:0/401626922 >> 172.21.15.2:0/401626922 conn(0x56169160dc20 msgr2=0x5616916b5330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.185+0000 7f1f35af6b80 1 -- 172.21.15.2:0/401626922 shutdown_connections 2023-09-13T04:47:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.185+0000 7f1f35af6b80 1 -- 172.21.15.2:0/401626922 wait complete. 2023-09-13T04:47:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.186+0000 7f1f35af6b80 20 client.26546 trim_cache size 0 max 16384 2023-09-13T04:47:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.201+0000 7f1f35af6b80 1 Processor -- start 2023-09-13T04:47:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.201+0000 7f1f35af6b80 1 -- start start 2023-09-13T04:47:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.202+0000 7f1f35af6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916af840 0x5616916bff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.202+0000 7f1f35af6b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616916b3be0 0x5616916c4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.202+0000 7f1f35af6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616916b4500 0x5616916bb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.202+0000 7f1f35af6b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616916abe70 con 0x5616916b3be0 2023-09-13T04:47:54.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.202+0000 7f1f35af6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616916c0460 con 0x5616916af840 2023-09-13T04:47:54.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.202+0000 7f1f35af6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616916b6dc0 con 0x5616916b4500 2023-09-13T04:47:54.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.202+0000 7f1f1bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616916b4500 0x5616916bb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.202+0000 7f1f1affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616916b3be0 0x5616916c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.202+0000 7f1f1bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616916b4500 0x5616916bb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:34784/0 (socket says 172.21.15.2:34784) 2023-09-13T04:47:54.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.202+0000 7f1f1bfff700 1 -- 172.21.15.2:0/1090254575 learned_addr learned my addr 172.21.15.2:0/1090254575 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:54.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.202+0000 7f1f1b7fe700 1 --2- 172.21.15.2:0/1090254575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916af840 0x5616916bff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.202+0000 7f1f1bfff700 1 -- 172.21.15.2:0/1090254575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916af840 msgr2=0x5616916bff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:54.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.202+0000 7f1f1bfff700 1 --2- 172.21.15.2:0/1090254575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916af840 0x5616916bff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.202+0000 7f1f1bfff700 1 -- 172.21.15.2:0/1090254575 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616916b3be0 msgr2=0x5616916c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:54.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.202+0000 7f1f1bfff700 1 --2- 172.21.15.2:0/1090254575 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616916b3be0 0x5616916c4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.202+0000 7f1f1bfff700 1 -- 172.21.15.2:0/1090254575 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56169142dd80 con 0x5616916b4500 2023-09-13T04:47:54.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.202+0000 7f1f1bfff700 1 --2- 172.21.15.2:0/1090254575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616916b4500 0x5616916bb180 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009ec0 tx=0x7f1f14006000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93f03994b0ea475d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:54.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.203+0000 7f1f027fc700 1 -- 172.21.15.2:0/1090254575 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1400ee90 con 0x5616916b4500 2023-09-13T04:47:54.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.203+0000 7f1f027fc700 1 -- 172.21.15.2:0/1090254575 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f14031070 con 0x5616916b4500 2023-09-13T04:47:54.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.203+0000 7f1f027fc700 1 -- 172.21.15.2:0/1090254575 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1401b920 con 0x5616916b4500 2023-09-13T04:47:54.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.203+0000 7f1f35af6b80 1 -- 172.21.15.2:0/1090254575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616916b4500 msgr2=0x5616916bb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:54.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.203+0000 7f1f35af6b80 1 --2- 172.21.15.2:0/1090254575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616916b4500 0x5616916bb180 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009ec0 tx=0x7f1f14006000 comp rx=0 tx=0).stop 2023-09-13T04:47:54.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.203+0000 7f1f35af6b80 1 -- 172.21.15.2:0/1090254575 shutdown_connections 2023-09-13T04:47:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.203+0000 7f1f35af6b80 1 --2- 172.21.15.2:0/1090254575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616916b4500 0x5616916bb180 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.203+0000 7f1f35af6b80 1 --2- 172.21.15.2:0/1090254575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916af840 0x5616916bff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.203+0000 7f1f35af6b80 1 --2- 172.21.15.2:0/1090254575 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616916b3be0 0x5616916c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.203+0000 7f1f35af6b80 1 -- 172.21.15.2:0/1090254575 >> 172.21.15.2:0/1090254575 conn(0x56169160dc20 msgr2=0x5616916c5170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.203+0000 7f1f35af6b80 1 -- 172.21.15.2:0/1090254575 shutdown_connections 2023-09-13T04:47:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.203+0000 7f1f35af6b80 1 -- 172.21.15.2:0/1090254575 wait complete. 2023-09-13T04:47:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.204+0000 7f1f35af6b80 1 -- start start 2023-09-13T04:47:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.204+0000 7f1f35af6b80 1 Processor -- start 2023-09-13T04:47:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.204+0000 7f1f35af6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916af840 0x5616916bf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.204+0000 7f1f35af6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616916b3be0 0x5616916bf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.204+0000 7f1f35af6b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616916b4500 0x56169171be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.204+0000 7f1f35af6b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56169171c370 con 0x5616916b4500 2023-09-13T04:47:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.205+0000 7f1f35af6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56169171c4b0 con 0x5616916af840 2023-09-13T04:47:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.205+0000 7f1f35af6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56169171c5f0 con 0x5616916b3be0 2023-09-13T04:47:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.205+0000 7f1f35af6b80 10 client.0 _ll_register_callbacks cb 0x56169159d620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:47:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.205+0000 7f1f1affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616916b3be0 0x5616916bf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.205+0000 7f1f1b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916af840 0x5616916bf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.205+0000 7f1f1affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616916b3be0 0x5616916bf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:34800/0 (socket says 172.21.15.2:34800) 2023-09-13T04:47:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.205+0000 7f1f1affd700 1 -- 172.21.15.2:0/3472558623 learned_addr learned my addr 172.21.15.2:0/3472558623 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.205+0000 7f1f1bfff700 1 --2- 172.21.15.2:0/3472558623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616916b4500 0x56169171be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.205+0000 7f1f1b7fe700 1 -- 172.21.15.2:0/3472558623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616916b3be0 msgr2=0x5616916bf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.205+0000 7f1f1b7fe700 1 --2- 172.21.15.2:0/3472558623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616916b3be0 0x5616916bf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.205+0000 7f1f1b7fe700 1 -- 172.21.15.2:0/3472558623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616916b4500 msgr2=0x56169171be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.205+0000 7f1f1b7fe700 1 --2- 172.21.15.2:0/3472558623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616916b4500 0x56169171be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.205+0000 7f1f1b7fe700 1 -- 172.21.15.2:0/3472558623 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561691619740 con 0x5616916af840 2023-09-13T04:47:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.205+0000 7f1f1b7fe700 1 --2- 172.21.15.2:0/3472558623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916af840 0x5616916bf2c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0c9870 tx=0x7f1f0c084040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dbf0a345b73376f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.205+0000 7f1efaffd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:47:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.206+0000 7f1efaffd700 1 -- 172.21.15.2:0/3472558623 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c0995d0 con 0x5616916af840 2023-09-13T04:47:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.206+0000 7f1f35af6b80 1 -- 172.21.15.2:0/3472558623 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f1001e040 con 0x5616916af840 2023-09-13T04:47:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.206+0000 7f1efaffd700 1 -- 172.21.15.2:0/3472558623 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f1f0c09d400 con 0x5616916af840 2023-09-13T04:47:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.206+0000 7f1efaffd700 1 -- 172.21.15.2:0/3472558623 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f0c0a1640 con 0x5616916af840 2023-09-13T04:47:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.206+0000 7f1f35af6b80 10 client.26552 fetch_fsmap learned FSMap version 36 2023-09-13T04:47:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.206+0000 7f1f35af6b80 1 -- 172.21.15.2:0/3472558623 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x561691416c00 con 0x5616916af840 2023-09-13T04:47:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.206+0000 7f1efaffd700 1 -- 172.21.15.2:0/3472558623 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f1001e040 con 0x5616916af840 2023-09-13T04:47:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.206+0000 7f1efaffd700 1 -- 172.21.15.2:0/3472558623 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f1f0c0840b0 con 0x5616916af840 2023-09-13T04:47:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.206+0000 7f1f35af6b80 10 client.26552 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:47:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.206+0000 7f1f35af6b80 10 client.26552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:47:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.206+0000 7f1f35af6b80 20 client.26552 populate_metadata read hostname 'smithi002' 2023-09-13T04:47:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.206+0000 7f1f35af6b80 1 --2- 172.21.15.2:0/3472558623 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56169171dcd0 0x5616917200f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.206+0000 7f1f35af6b80 4 client.26552 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:47:54.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.206+0000 7f1f35af6b80 1 -- 172.21.15.2:0/3472558623 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5616915baff0 con 0x56169171dcd0 2023-09-13T04:47:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.236+0000 7f1f1affd700 1 --2- 172.21.15.2:0/3472558623 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56169171dcd0 0x5616917200f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.237+0000 7f1f1affd700 1 --2- 172.21.15.2:0/3472558623 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56169171dcd0 0x5616917200f0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63b669e42aebf66e server_cookie=7ec6d4b2e452df07 in_seq=0 out_seq=0 2023-09-13T04:47:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.237+0000 7f1efaffd700 10 client.26552 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:47:54.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:53 smithi002 ceph-mon[97651]: pgmap v228: 129 pgs: 129 active+clean; 910 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 438 op/s 2023-09-13T04:47:54.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:53 smithi149 ceph-mon[133315]: pgmap v228: 129 pgs: 129 active+clean; 910 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 438 op/s 2023-09-13T04:47:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.511+0000 7f1efaffd700 1 -- 172.21.15.2:0/3472558623 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5616915baff0 con 0x56169171dcd0 2023-09-13T04:47:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.511+0000 7f1efaffd700 10 client.26552 handle_command_reply: tid=0 2023-09-13T04:47:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.511+0000 7f1f35af6b80 1 client.26552 shutdown 2023-09-13T04:47:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.512+0000 7f1f35af6b80 1 -- 172.21.15.2:0/3472558623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916af840 msgr2=0x5616916bf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.512+0000 7f1f35af6b80 1 --2- 172.21.15.2:0/3472558623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916af840 0x5616916bf2c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0c9870 tx=0x7f1f0c084040 comp rx=0 tx=0).stop 2023-09-13T04:47:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.512+0000 7f1f35af6b80 1 -- 172.21.15.2:0/3472558623 shutdown_connections 2023-09-13T04:47:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.512+0000 7f1f35af6b80 1 --2- 172.21.15.2:0/3472558623 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x56169171dcd0 0x5616917200f0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.512+0000 7f1f35af6b80 1 --2- 172.21.15.2:0/3472558623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616916b4500 0x56169171be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.512+0000 7f1f35af6b80 1 --2- 172.21.15.2:0/3472558623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5616916af840 0x5616916bf2c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.512+0000 7f1f35af6b80 1 --2- 172.21.15.2:0/3472558623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616916b3be0 0x5616916bf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.512+0000 7f1efaffd700 0 client.26552 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:47:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.512+0000 7f1f35af6b80 1 -- 172.21.15.2:0/3472558623 >> 172.21.15.2:0/3472558623 conn(0x56169160dc20 msgr2=0x5616916c4670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.512+0000 7f1f35af6b80 1 -- 172.21.15.2:0/3472558623 shutdown_connections 2023-09-13T04:47:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.512+0000 7f1f35af6b80 1 -- 172.21.15.2:0/3472558623 wait complete. 2023-09-13T04:47:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.512+0000 7f1f35af6b80 20 client.26552 trim_cache size 0 max 16384 2023-09-13T04:47:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:47:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:47:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:47:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:47:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.513+0000 7f1f221ff700 1 -- 172.21.15.2:0/4277056913 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f1f040a07b0 msgr2=0x7f1f040a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.513+0000 7f1f221ff700 1 --2- 172.21.15.2:0/4277056913 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f1f040a07b0 0x7f1f040a2c70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x5616916aba70 tx=0x7f1f1400b040 comp rx=0 tx=0).stop 2023-09-13T04:47:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.513+0000 7f1f221ff700 1 -- 172.21.15.2:0/4277056913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f1c100ed0 msgr2=0x7f1f1c199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.513+0000 7f1f221ff700 1 --2- 172.21.15.2:0/4277056913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f1c100ed0 0x7f1f1c199040 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c00e9a0 tx=0x7f1f0c00ecb0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.513+0000 7f1f221ff700 1 -- 172.21.15.2:0/4277056913 shutdown_connections 2023-09-13T04:47:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.513+0000 7f1f221ff700 1 --2- 172.21.15.2:0/4277056913 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f1f040a07b0 0x7f1f040a2c70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.513+0000 7f1f221ff700 1 --2- 172.21.15.2:0/4277056913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f1c19dbf0 0x7f1f1c19e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.513+0000 7f1f221ff700 1 --2- 172.21.15.2:0/4277056913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f1c100ed0 0x7f1f1c199040 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.513+0000 7f1f221ff700 1 --2- 172.21.15.2:0/4277056913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f1c105ff0 0x7f1f1c199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.513+0000 7f1f221ff700 1 -- 172.21.15.2:0/4277056913 >> 172.21.15.2:0/4277056913 conn(0x7f1f1c0fc750 msgr2=0x7f1f1c0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.513+0000 7f1f221ff700 1 -- 172.21.15.2:0/4277056913 shutdown_connections 2023-09-13T04:47:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:47:54.513+0000 7f1f221ff700 1 -- 172.21.15.2:0/4277056913 wait complete. 2023-09-13T04:47:54.607 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T04:47:54.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.818+0000 7ff71d3ff700 1 -- 172.21.15.2:0/2593028905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff718100eb0 msgr2=0x7ff718101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:54.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.818+0000 7ff71d3ff700 1 --2- 172.21.15.2:0/2593028905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff718100eb0 0x7ff718101290 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff708009990 tx=0x7ff708009ca0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.818+0000 7ff71d3ff700 1 -- 172.21.15.2:0/2593028905 shutdown_connections 2023-09-13T04:47:54.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.818+0000 7ff71d3ff700 1 --2- 172.21.15.2:0/2593028905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff718105fd0 0x7ff71810db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.818+0000 7ff71d3ff700 1 --2- 172.21.15.2:0/2593028905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff718100eb0 0x7ff718101290 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.818+0000 7ff71d3ff700 1 --2- 172.21.15.2:0/2593028905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff718101860 0x7ff7181058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.818+0000 7ff71d3ff700 1 -- 172.21.15.2:0/2593028905 >> 172.21.15.2:0/2593028905 conn(0x7ff7180fc770 msgr2=0x7ff7180feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:54.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.818+0000 7ff71d3ff700 1 -- 172.21.15.2:0/2593028905 shutdown_connections 2023-09-13T04:47:54.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.818+0000 7ff71d3ff700 1 -- 172.21.15.2:0/2593028905 wait complete. 2023-09-13T04:47:54.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.819+0000 7ff71d3ff700 1 Processor -- start 2023-09-13T04:47:54.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.820+0000 7ff71d3ff700 1 -- start start 2023-09-13T04:47:54.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.820+0000 7ff71d3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff718100eb0 0x7ff718198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.820+0000 7ff71d3ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff718105fd0 0x7ff718199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.820+0000 7ff71d3ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff71819d980 0x7ff71819de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.820+0000 7ff71d3ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71819e400 con 0x7ff718105fd0 2023-09-13T04:47:54.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.820+0000 7ff71d3ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71819e570 con 0x7ff71819d980 2023-09-13T04:47:54.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.820+0000 7ff71d3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71819e730 con 0x7ff718100eb0 2023-09-13T04:47:54.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.821+0000 7ff716ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff718100eb0 0x7ff718198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.821+0000 7ff7167fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff718105fd0 0x7ff718199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.821+0000 7ff716ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff718100eb0 0x7ff718198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:34820/0 (socket says 172.21.15.2:34820) 2023-09-13T04:47:54.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.821+0000 7ff716ffd700 1 -- 172.21.15.2:0/3629091189 learned_addr learned my addr 172.21.15.2:0/3629091189 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:54.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.821+0000 7ff7177fe700 1 --2- 172.21.15.2:0/3629091189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff71819d980 0x7ff71819de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.821+0000 7ff716ffd700 1 -- 172.21.15.2:0/3629091189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff71819d980 msgr2=0x7ff71819de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:54.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.821+0000 7ff716ffd700 1 --2- 172.21.15.2:0/3629091189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff71819d980 0x7ff71819de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.821+0000 7ff716ffd700 1 -- 172.21.15.2:0/3629091189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff718105fd0 msgr2=0x7ff718199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:54.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.821+0000 7ff716ffd700 1 --2- 172.21.15.2:0/3629091189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff718105fd0 0x7ff718199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.821+0000 7ff716ffd700 1 -- 172.21.15.2:0/3629091189 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7080095f0 con 0x7ff718100eb0 2023-09-13T04:47:54.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.821+0000 7ff7167fc700 1 --2- 172.21.15.2:0/3629091189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff718105fd0 0x7ff718199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:47:54.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.821+0000 7ff716ffd700 1 --2- 172.21.15.2:0/3629091189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff718100eb0 0x7ff718198fe0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff708005910 tx=0x7ff708005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:54.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.822+0000 7ff6fffff700 1 -- 172.21.15.2:0/3629091189 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70801d070 con 0x7ff718100eb0 2023-09-13T04:47:54.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.822+0000 7ff6fffff700 1 -- 172.21.15.2:0/3629091189 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff708022470 con 0x7ff718100eb0 2023-09-13T04:47:54.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.822+0000 7ff71d3ff700 1 -- 172.21.15.2:0/3629091189 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7181a5c30 con 0x7ff718100eb0 2023-09-13T04:47:54.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.822+0000 7ff6fffff700 1 -- 172.21.15.2:0/3629091189 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff708017650 con 0x7ff718100eb0 2023-09-13T04:47:54.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.822+0000 7ff71d3ff700 1 -- 172.21.15.2:0/3629091189 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7181a6050 con 0x7ff718100eb0 2023-09-13T04:47:54.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.825+0000 7ff6fffff700 1 -- 172.21.15.2:0/3629091189 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7ff7080225e0 con 0x7ff718100eb0 2023-09-13T04:47:54.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.826+0000 7ff6fffff700 1 --2- 172.21.15.2:0/3629091189 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7ff7000a07b0 0x7ff7000a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.826+0000 7ff6fffff700 1 -- 172.21.15.2:0/3629091189 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9395+0+0 (secure 0 0 0) 0x7ff7080cd660 con 0x7ff718100eb0 2023-09-13T04:47:54.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.827+0000 7ff7167fc700 1 --2- 172.21.15.2:0/3629091189 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7ff7000a07b0 0x7ff7000a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.827+0000 7ff730d3eb80 1 Processor -- start 2023-09-13T04:47:54.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.827+0000 7ff730d3eb80 1 -- start start 2023-09-13T04:47:54.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.827+0000 7ff7167fc700 1 --2- 172.21.15.2:0/3629091189 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7ff7000a07b0 0x7ff7000a2c70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff71819a5f0 tx=0x7ff70c009260 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:54.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.827+0000 7ff730d3eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbad9fe0 0x556ddbada3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.827+0000 7ff730d3eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556ddbada900 0x556ddbad44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.827+0000 7ff730d3eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 0x556ddbad4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.827+0000 7ff730d3eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ddba2d780 con 0x556ddbad4a00 2023-09-13T04:47:54.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.827+0000 7ff730d3eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ddba2b0a0 con 0x556ddbada900 2023-09-13T04:47:54.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.827+0000 7ff730d3eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ddba26990 con 0x556ddbad9fe0 2023-09-13T04:47:54.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.828+0000 7ff716ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbad9fe0 0x556ddbada3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.828+0000 7ff7167fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556ddbada900 0x556ddbad44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.828+0000 7ff7167fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556ddbada900 0x556ddbad44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:37422/0 (socket says 172.21.15.2:37422) 2023-09-13T04:47:54.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.828+0000 7ff716ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbad9fe0 0x556ddbada3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:34828/0 (socket says 172.21.15.2:34828) 2023-09-13T04:47:54.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.828+0000 7ff7167fc700 1 -- 172.21.15.2:0/3878576835 learned_addr learned my addr 172.21.15.2:0/3878576835 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:54.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.828+0000 7ff7177fe700 1 --2- 172.21.15.2:0/3878576835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 0x556ddbad4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.828+0000 7ff7167fc700 1 -- 172.21.15.2:0/3878576835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbad9fe0 msgr2=0x556ddbada3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:54.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.828+0000 7ff7167fc700 1 --2- 172.21.15.2:0/3878576835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbad9fe0 0x556ddbada3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.828+0000 7ff7167fc700 1 -- 172.21.15.2:0/3878576835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 msgr2=0x556ddbad4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:54.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.828+0000 7ff7167fc700 1 --2- 172.21.15.2:0/3878576835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 0x556ddbad4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.828+0000 7ff7167fc700 1 -- 172.21.15.2:0/3878576835 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ddb8a6630 con 0x556ddbada900 2023-09-13T04:47:54.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.828+0000 7ff716ffd700 1 --2- 172.21.15.2:0/3878576835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbad9fe0 0x556ddbada3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:47:54.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.828+0000 7ff7177fe700 1 --2- 172.21.15.2:0/3878576835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 0x556ddbad4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:47:54.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.828+0000 7ff7167fc700 1 --2- 172.21.15.2:0/3878576835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556ddbada900 0x556ddbad44c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff70c0043c0 tx=0x7ff70c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e252a911386c4c33 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:54.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.829+0000 7ff6fd7fa700 1 -- 172.21.15.2:0/3878576835 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c003d10 con 0x556ddbada900 2023-09-13T04:47:54.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.829+0000 7ff6fd7fa700 1 -- 172.21.15.2:0/3878576835 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff70c003e70 con 0x556ddbada900 2023-09-13T04:47:54.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.829+0000 7ff6fd7fa700 1 -- 172.21.15.2:0/3878576835 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c01e780 con 0x556ddbada900 2023-09-13T04:47:54.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.829+0000 7ff730d3eb80 1 -- 172.21.15.2:0/3878576835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556ddbada900 msgr2=0x556ddbad44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:54.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.829+0000 7ff730d3eb80 1 --2- 172.21.15.2:0/3878576835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556ddbada900 0x556ddbad44c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff70c0043c0 tx=0x7ff70c014000 comp rx=0 tx=0).stop 2023-09-13T04:47:54.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.829+0000 7ff730d3eb80 1 -- 172.21.15.2:0/3878576835 shutdown_connections 2023-09-13T04:47:54.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.829+0000 7ff730d3eb80 1 --2- 172.21.15.2:0/3878576835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 0x556ddbad4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.829+0000 7ff730d3eb80 1 --2- 172.21.15.2:0/3878576835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbad9fe0 0x556ddbada3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.829+0000 7ff730d3eb80 1 --2- 172.21.15.2:0/3878576835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556ddbada900 0x556ddbad44c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.829+0000 7ff730d3eb80 1 -- 172.21.15.2:0/3878576835 >> 172.21.15.2:0/3878576835 conn(0x556ddba32c80 msgr2=0x556ddbacef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:54.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.830+0000 7ff730d3eb80 1 -- 172.21.15.2:0/3878576835 shutdown_connections 2023-09-13T04:47:54.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.830+0000 7ff730d3eb80 1 -- 172.21.15.2:0/3878576835 wait complete. 2023-09-13T04:47:54.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.830+0000 7ff730d3eb80 1 -- start start 2023-09-13T04:47:54.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.830+0000 7ff730d3eb80 1 Processor -- start 2023-09-13T04:47:54.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.831+0000 7ff730d3eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 0x556ddbaddcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.831+0000 7ff730d3eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbad9fe0 0x556ddbade1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.831+0000 7ff730d3eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556ddbada900 0x556ddbade730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.831+0000 7ff730d3eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ddb9c4bb0 con 0x556ddbad4a00 2023-09-13T04:47:54.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.831+0000 7ff730d3eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ddb9c48b0 con 0x556ddbada900 2023-09-13T04:47:54.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.831+0000 7ff730d3eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ddb8a8920 con 0x556ddbad9fe0 2023-09-13T04:47:54.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.831+0000 7ff730d3eb80 10 client.0 _ll_register_callbacks cb 0x556ddb9c2680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:47:54.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.831+0000 7ff7167fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbad9fe0 0x556ddbade1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.831+0000 7ff716ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 0x556ddbaddcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.831+0000 7ff7167fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbad9fe0 0x556ddbade1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:34844/0 (socket says 172.21.15.2:34844) 2023-09-13T04:47:54.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.831+0000 7ff7167fc700 1 -- 172.21.15.2:0/1721318230 learned_addr learned my addr 172.21.15.2:0/1721318230 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:54.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.831+0000 7ff7177fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556ddbada900 0x556ddbade730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.831+0000 7ff7167fc700 1 -- 172.21.15.2:0/1721318230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556ddbada900 msgr2=0x556ddbade730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:54.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.831+0000 7ff7167fc700 1 --2- 172.21.15.2:0/1721318230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556ddbada900 0x556ddbade730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.831+0000 7ff7167fc700 1 -- 172.21.15.2:0/1721318230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 msgr2=0x556ddbaddcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:54.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.831+0000 7ff7167fc700 1 --2- 172.21.15.2:0/1721318230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 0x556ddbaddcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.831+0000 7ff7167fc700 1 -- 172.21.15.2:0/1721318230 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff70c017070 con 0x556ddbad9fe0 2023-09-13T04:47:54.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.831+0000 7ff716ffd700 1 --2- 172.21.15.2:0/1721318230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 0x556ddbaddcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:47:54.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.832+0000 7ff7167fc700 1 --2- 172.21.15.2:0/1721318230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbad9fe0 0x556ddbade1f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff70c0097c0 tx=0x7ff70c026400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52a6764512a4da52 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:54.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.832+0000 7ff6faff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:47:54.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.832+0000 7ff6faff5700 1 -- 172.21.15.2:0/1721318230 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c02b070 con 0x556ddbad9fe0 2023-09-13T04:47:54.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.832+0000 7ff730d3eb80 1 -- 172.21.15.2:0/1721318230 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ddbac7670 con 0x556ddbad9fe0 2023-09-13T04:47:54.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.832+0000 7ff6faff5700 1 -- 172.21.15.2:0/1721318230 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9395+0+0 (secure 0 0 0) 0x7ff70c0267b0 con 0x556ddbad9fe0 2023-09-13T04:47:54.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.832+0000 7ff6faff5700 1 -- 172.21.15.2:0/1721318230 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff70c026970 con 0x556ddbad9fe0 2023-09-13T04:47:54.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.832+0000 7ff730d3eb80 10 client.26578 fetch_fsmap learned FSMap version 36 2023-09-13T04:47:54.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.832+0000 7ff730d3eb80 1 -- 172.21.15.2:0/1721318230 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x556ddba3e7a0 con 0x556ddbad9fe0 2023-09-13T04:47:54.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.832+0000 7ff6faff5700 1 -- 172.21.15.2:0/1721318230 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556ddbac7670 con 0x556ddbad9fe0 2023-09-13T04:47:54.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.833+0000 7ff6faff5700 1 -- 172.21.15.2:0/1721318230 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7ff70c024020 con 0x556ddbad9fe0 2023-09-13T04:47:54.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.833+0000 7ff730d3eb80 10 client.26578 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:47:54.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.833+0000 7ff730d3eb80 10 client.26578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:47:54.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.833+0000 7ff730d3eb80 20 client.26578 populate_metadata read hostname 'smithi002' 2023-09-13T04:47:54.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.833+0000 7ff730d3eb80 1 --2- 172.21.15.2:0/1721318230 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x556ddbaee1c0 0x556ddbaf05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.833+0000 7ff730d3eb80 4 client.26578 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:47:54.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.833+0000 7ff730d3eb80 1 -- 172.21.15.2:0/1721318230 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ddba1f970 con 0x556ddbaee1c0 2023-09-13T04:47:54.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.833+0000 7ff716ffd700 1 --2- 172.21.15.2:0/1721318230 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x556ddbaee1c0 0x556ddbaf05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.833+0000 7ff716ffd700 1 --2- 172.21.15.2:0/1721318230 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x556ddbaee1c0 0x556ddbaf05e0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d9a58ee6a221d3c server_cookie=8afdcad40c297765 in_seq=0 out_seq=0 2023-09-13T04:47:54.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.833+0000 7ff6faff5700 10 client.26578 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:47:54.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.835+0000 7ff6faff5700 1 -- 172.21.15.2:0/1721318230 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556ddba1f970 con 0x556ddbaee1c0 2023-09-13T04:47:54.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.835+0000 7ff6faff5700 10 client.26578 handle_command_reply: tid=0 2023-09-13T04:47:54.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.835+0000 7ff730d3eb80 1 client.26578 shutdown 2023-09-13T04:47:54.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.835+0000 7ff730d3eb80 1 -- 172.21.15.2:0/1721318230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbad9fe0 msgr2=0x556ddbade1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:54.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.835+0000 7ff730d3eb80 1 --2- 172.21.15.2:0/1721318230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbad9fe0 0x556ddbade1f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff70c0097c0 tx=0x7ff70c026400 comp rx=0 tx=0).stop 2023-09-13T04:47:54.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.835+0000 7ff730d3eb80 1 -- 172.21.15.2:0/1721318230 shutdown_connections 2023-09-13T04:47:54.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.835+0000 7ff730d3eb80 1 --2- 172.21.15.2:0/1721318230 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x556ddbaee1c0 0x556ddbaf05e0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.835+0000 7ff730d3eb80 1 --2- 172.21.15.2:0/1721318230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556ddbada900 0x556ddbade730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.835+0000 7ff730d3eb80 1 --2- 172.21.15.2:0/1721318230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 0x556ddbaddcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.835+0000 7ff6faff5700 0 client.26578 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:47:54.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.835+0000 7ff730d3eb80 1 --2- 172.21.15.2:0/1721318230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbad9fe0 0x556ddbade1f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.835+0000 7ff730d3eb80 1 -- 172.21.15.2:0/1721318230 >> 172.21.15.2:0/1721318230 conn(0x556ddba32c80 msgr2=0x556ddbac97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:54.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.835+0000 7ff730d3eb80 1 -- 172.21.15.2:0/1721318230 shutdown_connections 2023-09-13T04:47:54.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.835+0000 7ff730d3eb80 1 -- 172.21.15.2:0/1721318230 wait complete. 2023-09-13T04:47:54.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.835+0000 7ff730d3eb80 20 client.26578 trim_cache size 0 max 16384 2023-09-13T04:47:54.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.852+0000 7ff730d3eb80 1 Processor -- start 2023-09-13T04:47:54.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.852+0000 7ff730d3eb80 1 -- start start 2023-09-13T04:47:54.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.853+0000 7ff730d3eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 0x556ddbad44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.853+0000 7ff730d3eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556ddbad9fe0 0x556ddbae5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.853+0000 7ff730d3eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbada900 0x556ddbac98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.853+0000 7ff730d3eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ddbacfde0 con 0x556ddbad4a00 2023-09-13T04:47:54.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.853+0000 7ff730d3eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ddbad0200 con 0x556ddbad9fe0 2023-09-13T04:47:54.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.853+0000 7ff730d3eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ddbae57b0 con 0x556ddbada900 2023-09-13T04:47:54.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.853+0000 7ff716ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 0x556ddbad44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.853+0000 7ff7167fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556ddbad9fe0 0x556ddbae5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.853+0000 7ff7177fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbada900 0x556ddbac98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.853+0000 7ff716ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 0x556ddbad44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55080/0 (socket says 172.21.15.2:55080) 2023-09-13T04:47:54.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.853+0000 7ff716ffd700 1 -- 172.21.15.2:0/1949274097 learned_addr learned my addr 172.21.15.2:0/1949274097 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:54.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.853+0000 7ff7167fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556ddbad9fe0 0x556ddbae5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:37438/0 (socket says 172.21.15.2:37438) 2023-09-13T04:47:54.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.853+0000 7ff7167fc700 1 -- 172.21.15.2:0/1949274097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbada900 msgr2=0x556ddbac98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:54.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.853+0000 7ff7167fc700 1 --2- 172.21.15.2:0/1949274097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbada900 0x556ddbac98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.853+0000 7ff7167fc700 1 -- 172.21.15.2:0/1949274097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 msgr2=0x556ddbad44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:54.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.853+0000 7ff7167fc700 1 --2- 172.21.15.2:0/1949274097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 0x556ddbad44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.853+0000 7ff7167fc700 1 -- 172.21.15.2:0/1949274097 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ddb852de0 con 0x556ddbad9fe0 2023-09-13T04:47:54.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.853+0000 7ff7177fe700 1 --2- 172.21.15.2:0/1949274097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbada900 0x556ddbac98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:47:54.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.853+0000 7ff716ffd700 1 --2- 172.21.15.2:0/1949274097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 0x556ddbad44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:47:54.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.854+0000 7ff7167fc700 1 --2- 172.21.15.2:0/1949274097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556ddbad9fe0 0x556ddbae5270 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff70c01e470 tx=0x7ff70c009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae87b464648c52c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:54.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.854+0000 7ff6fdffb700 1 -- 172.21.15.2:0/1949274097 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c02b030 con 0x556ddbad9fe0 2023-09-13T04:47:54.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.854+0000 7ff6fdffb700 1 -- 172.21.15.2:0/1949274097 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff70c01e860 con 0x556ddbad9fe0 2023-09-13T04:47:54.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.854+0000 7ff6fdffb700 1 -- 172.21.15.2:0/1949274097 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c0306d0 con 0x556ddbad9fe0 2023-09-13T04:47:54.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.854+0000 7ff730d3eb80 1 -- 172.21.15.2:0/1949274097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556ddbad9fe0 msgr2=0x556ddbae5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:54.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.854+0000 7ff730d3eb80 1 --2- 172.21.15.2:0/1949274097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556ddbad9fe0 0x556ddbae5270 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff70c01e470 tx=0x7ff70c009750 comp rx=0 tx=0).stop 2023-09-13T04:47:54.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.855+0000 7ff730d3eb80 1 -- 172.21.15.2:0/1949274097 shutdown_connections 2023-09-13T04:47:54.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.855+0000 7ff730d3eb80 1 --2- 172.21.15.2:0/1949274097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbada900 0x556ddbac98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.855+0000 7ff730d3eb80 1 --2- 172.21.15.2:0/1949274097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 0x556ddbad44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.855+0000 7ff730d3eb80 1 --2- 172.21.15.2:0/1949274097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556ddbad9fe0 0x556ddbae5270 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.855+0000 7ff730d3eb80 1 -- 172.21.15.2:0/1949274097 >> 172.21.15.2:0/1949274097 conn(0x556ddba32c80 msgr2=0x556ddbad7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:54.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.855+0000 7ff730d3eb80 1 -- 172.21.15.2:0/1949274097 shutdown_connections 2023-09-13T04:47:54.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.855+0000 7ff730d3eb80 1 -- 172.21.15.2:0/1949274097 wait complete. 2023-09-13T04:47:54.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.855+0000 7ff730d3eb80 1 -- start start 2023-09-13T04:47:54.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.855+0000 7ff730d3eb80 1 Processor -- start 2023-09-13T04:47:54.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.856+0000 7ff730d3eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 0x556ddbae3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.856+0000 7ff730d3eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbad9fe0 0x556ddbae37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.856+0000 7ff730d3eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556ddbada900 0x556ddbae3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.856+0000 7ff730d3eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ddbb41100 con 0x556ddbad4a00 2023-09-13T04:47:54.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.856+0000 7ff730d3eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ddbb41270 con 0x556ddbada900 2023-09-13T04:47:54.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.856+0000 7ff730d3eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ddbb41430 con 0x556ddbad9fe0 2023-09-13T04:47:54.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.856+0000 7ff730d3eb80 10 client.0 _ll_register_callbacks cb 0x556ddb9c2680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:47:54.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.856+0000 7ff716ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 0x556ddbae3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.856+0000 7ff7167fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbad9fe0 0x556ddbae37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.856+0000 7ff716ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 0x556ddbae3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55090/0 (socket says 172.21.15.2:55090) 2023-09-13T04:47:54.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.856+0000 7ff716ffd700 1 -- 172.21.15.2:0/1344930822 learned_addr learned my addr 172.21.15.2:0/1344930822 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:47:54.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.856+0000 7ff7177fe700 1 --2- 172.21.15.2:0/1344930822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556ddbada900 0x556ddbae3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.856+0000 7ff716ffd700 1 -- 172.21.15.2:0/1344930822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbad9fe0 msgr2=0x556ddbae37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:54.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.856+0000 7ff716ffd700 1 --2- 172.21.15.2:0/1344930822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbad9fe0 0x556ddbae37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.856+0000 7ff716ffd700 1 -- 172.21.15.2:0/1344930822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556ddbada900 msgr2=0x556ddbae3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:47:54.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.856+0000 7ff716ffd700 1 --2- 172.21.15.2:0/1344930822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556ddbada900 0x556ddbae3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.856+0000 7ff716ffd700 1 -- 172.21.15.2:0/1344930822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7181a5c30 con 0x556ddbad4a00 2023-09-13T04:47:54.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.857+0000 7ff716ffd700 1 --2- 172.21.15.2:0/1344930822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 0x556ddbae3260 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff708096df0 tx=0x7ff708009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa0705f7de53273 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:47:54.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.857+0000 7ff6faff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:47:54.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.857+0000 7ff6faff5700 1 -- 172.21.15.2:0/1344930822 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7080cc040 con 0x556ddbad4a00 2023-09-13T04:47:54.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.857+0000 7ff730d3eb80 1 -- 172.21.15.2:0/1344930822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff70c003e70 con 0x556ddbad4a00 2023-09-13T04:47:54.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.857+0000 7ff6faff5700 1 -- 172.21.15.2:0/1344930822 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9395+0+0 (secure 0 0 0) 0x7ff7080b0040 con 0x556ddbad4a00 2023-09-13T04:47:54.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.857+0000 7ff6faff5700 1 -- 172.21.15.2:0/1344930822 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff70809a8f0 con 0x556ddbad4a00 2023-09-13T04:47:54.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.857+0000 7ff730d3eb80 10 client.16407 fetch_fsmap learned FSMap version 36 2023-09-13T04:47:54.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.857+0000 7ff730d3eb80 1 -- 172.21.15.2:0/1344930822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7ff710009160 con 0x556ddbad4a00 2023-09-13T04:47:54.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.857+0000 7ff6faff5700 1 -- 172.21.15.2:0/1344930822 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff70c003e70 con 0x556ddbad4a00 2023-09-13T04:47:54.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.858+0000 7ff6faff5700 1 -- 172.21.15.2:0/1344930822 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7ff70809ac70 con 0x556ddbad4a00 2023-09-13T04:47:54.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.858+0000 7ff730d3eb80 10 client.16407 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:47:54.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.858+0000 7ff730d3eb80 10 client.16407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:47:54.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.858+0000 7ff730d3eb80 20 client.16407 populate_metadata read hostname 'smithi002' 2023-09-13T04:47:54.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.858+0000 7ff730d3eb80 1 --2- 172.21.15.2:0/1344930822 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x556ddbb42880 0x556ddbb44ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:47:54.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.858+0000 7ff730d3eb80 4 client.16407 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-09-13T04:47:54.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.858+0000 7ff730d3eb80 1 -- 172.21.15.2:0/1344930822 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556ddb9e0050 con 0x556ddbb42880 2023-09-13T04:47:54.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.858+0000 7ff7167fc700 1 --2- 172.21.15.2:0/1344930822 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x556ddbb42880 0x556ddbb44ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:47:54.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.858+0000 7ff7167fc700 1 --2- 172.21.15.2:0/1344930822 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x556ddbb42880 0x556ddbb44ca0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22f2a5cdc4fcbd0c server_cookie=dc2d5e16dc4669fc in_seq=0 out_seq=0 2023-09-13T04:47:54.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.858+0000 7ff6faff5700 10 client.16407 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:47:54.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.859+0000 7ff6faff5700 1 -- 172.21.15.2:0/1344930822 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x556ddb9e0050 con 0x556ddbb42880 2023-09-13T04:47:54.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.859+0000 7ff6faff5700 10 client.16407 handle_command_reply: tid=0 2023-09-13T04:47:54.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.859+0000 7ff730d3eb80 1 client.16407 shutdown 2023-09-13T04:47:54.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.859+0000 7ff730d3eb80 1 -- 172.21.15.2:0/1344930822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 msgr2=0x556ddbae3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:54.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.859+0000 7ff730d3eb80 1 --2- 172.21.15.2:0/1344930822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 0x556ddbae3260 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff708096df0 tx=0x7ff708009ca0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.859+0000 7ff730d3eb80 1 -- 172.21.15.2:0/1344930822 shutdown_connections 2023-09-13T04:47:54.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.859+0000 7ff730d3eb80 1 --2- 172.21.15.2:0/1344930822 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x556ddbb42880 0x556ddbb44ca0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.859+0000 7ff730d3eb80 1 --2- 172.21.15.2:0/1344930822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556ddbada900 0x556ddbae3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.859+0000 7ff730d3eb80 1 --2- 172.21.15.2:0/1344930822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ddbad4a00 0x556ddbae3260 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.859+0000 7ff730d3eb80 1 --2- 172.21.15.2:0/1344930822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ddbad9fe0 0x556ddbae37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.859+0000 7ff6faff5700 0 client.16407 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:47:54.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.859+0000 7ff730d3eb80 1 -- 172.21.15.2:0/1344930822 >> 172.21.15.2:0/1344930822 conn(0x556ddba32c80 msgr2=0x556ddbad8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:54.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.859+0000 7ff730d3eb80 1 -- 172.21.15.2:0/1344930822 shutdown_connections 2023-09-13T04:47:54.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.859+0000 7ff730d3eb80 1 -- 172.21.15.2:0/1344930822 wait complete. 2023-09-13T04:47:54.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.860+0000 7ff730d3eb80 20 client.16407 trim_cache size 0 max 16384 2023-09-13T04:47:54.881 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:47:54.881 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "scrub active (3 inodes in the stack)", 2023-09-13T04:47:54.881 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-09-13T04:47:54.881 INFO:teuthology.orchestra.run.smithi002.stdout: "740235c0-6e1b-4729-b1fe-33345ae948c1": { 2023-09-13T04:47:54.882 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-09-13T04:47:54.882 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "740235c0-6e1b-4729-b1fe-33345ae948c1", 2023-09-13T04:47:54.882 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-09-13T04:47:54.882 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:47:54.883 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:47:54.883 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:47:54.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.860+0000 7ff71d3ff700 1 -- 172.21.15.2:0/3629091189 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7ff7000a07b0 msgr2=0x7ff7000a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:54.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.860+0000 7ff71d3ff700 1 --2- 172.21.15.2:0/3629091189 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7ff7000a07b0 0x7ff7000a2c70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff71819a5f0 tx=0x7ff70c009260 comp rx=0 tx=0).stop 2023-09-13T04:47:54.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.860+0000 7ff71d3ff700 1 -- 172.21.15.2:0/3629091189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff718100eb0 msgr2=0x7ff718198fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:47:54.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.860+0000 7ff71d3ff700 1 --2- 172.21.15.2:0/3629091189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff718100eb0 0x7ff718198fe0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff708005910 tx=0x7ff708005740 comp rx=0 tx=0).stop 2023-09-13T04:47:54.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.860+0000 7ff71d3ff700 1 -- 172.21.15.2:0/3629091189 shutdown_connections 2023-09-13T04:47:54.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.860+0000 7ff71d3ff700 1 --2- 172.21.15.2:0/3629091189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff71819d980 0x7ff71819de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.860+0000 7ff71d3ff700 1 --2- 172.21.15.2:0/3629091189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff718100eb0 0x7ff718198fe0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.860+0000 7ff71d3ff700 1 --2- 172.21.15.2:0/3629091189 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7ff7000a07b0 0x7ff7000a2c70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.860+0000 7ff71d3ff700 1 --2- 172.21.15.2:0/3629091189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff718105fd0 0x7ff718199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:47:54.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.860+0000 7ff71d3ff700 1 -- 172.21.15.2:0/3629091189 >> 172.21.15.2:0/3629091189 conn(0x7ff7180fc770 msgr2=0x7ff71810ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:47:54.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.860+0000 7ff71d3ff700 1 -- 172.21.15.2:0/3629091189 shutdown_connections 2023-09-13T04:47:54.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:47:54.860+0000 7ff71d3ff700 1 -- 172.21.15.2:0/3629091189 wait complete. 2023-09-13T04:47:54.887 INFO:tasks.cephfs.filesystem:scrub status for tag:740235c0-6e1b-4729-b1fe-33345ae948c1 - {'path': '/', 'tag': '740235c0-6e1b-4729-b1fe-33345ae948c1', 'options': 'recursive,force'} 2023-09-13T04:47:56.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:55 smithi156 ceph-mon[133675]: pgmap v229: 129 pgs: 129 active+clean; 919 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 463 op/s 2023-09-13T04:47:56.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:55 smithi002 ceph-mon[97651]: pgmap v229: 129 pgs: 129 active+clean; 919 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 463 op/s 2023-09-13T04:47:56.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:55 smithi149 ceph-mon[133315]: pgmap v229: 129 pgs: 129 active+clean; 919 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 463 op/s 2023-09-13T04:47:58.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:57 smithi156 ceph-mon[133675]: pgmap v230: 129 pgs: 129 active+clean; 919 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 233 op/s 2023-09-13T04:47:58.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:57 smithi002 ceph-mon[97651]: pgmap v230: 129 pgs: 129 active+clean; 919 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 233 op/s 2023-09-13T04:47:58.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:57 smithi149 ceph-mon[133315]: pgmap v230: 129 pgs: 129 active+clean; 919 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 233 op/s 2023-09-13T04:48:00.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:47:59 smithi156 ceph-mon[133675]: pgmap v231: 129 pgs: 129 active+clean; 922 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 244 op/s 2023-09-13T04:48:00.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:47:59 smithi002 ceph-mon[97651]: pgmap v231: 129 pgs: 129 active+clean; 922 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 244 op/s 2023-09-13T04:48:00.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:47:59 smithi149 ceph-mon[133315]: pgmap v231: 129 pgs: 129 active+clean; 922 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 244 op/s 2023-09-13T04:48:01.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:00 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:48:01.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:00 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:48:01.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:00 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:48:02.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:01 smithi156 ceph-mon[133675]: pgmap v232: 129 pgs: 129 active+clean; 940 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 304 op/s 2023-09-13T04:48:02.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:01 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:02.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:01 smithi002 ceph-mon[97651]: pgmap v232: 129 pgs: 129 active+clean; 940 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 304 op/s 2023-09-13T04:48:02.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:01 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:02.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:01 smithi149 ceph-mon[133315]: pgmap v232: 129 pgs: 129 active+clean; 940 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 304 op/s 2023-09-13T04:48:02.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:01 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:03.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:02 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:48:03.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:02 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:03.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:02 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:48:03.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:02 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:03.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:02 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:48:03.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:02 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:03.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:02 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:48:03.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:02 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:03.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:02 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:48:03.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:02 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:03.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:02 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:48:03.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:02 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:03.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:02 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:48:03.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:02 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:03.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:02 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:48:03.390 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:02 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:03.390 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:02 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:48:03.390 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:02 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:03.390 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:02 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:48:03.391 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:02 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:03.391 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:02 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:48:03.391 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:02 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:03.391 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:02 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:48:03.391 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:02 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:03.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:02 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:48:03.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:02 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:03.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:02 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:48:03.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:02 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:03.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:02 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:48:03.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:02 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:03.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:02 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:48:03.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:02 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:03.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:02 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:48:03.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:02 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:03.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:02 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:48:03.439 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:02 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:04.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:03 smithi002 ceph-mon[97651]: pgmap v233: 129 pgs: 129 active+clean; 940 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 252 op/s 2023-09-13T04:48:04.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:03 smithi002 ceph-mon[97651]: osdmap e97: 12 total, 12 up, 12 in 2023-09-13T04:48:04.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:03 smithi149 ceph-mon[133315]: pgmap v233: 129 pgs: 129 active+clean; 940 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 252 op/s 2023-09-13T04:48:04.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:03 smithi149 ceph-mon[133315]: osdmap e97: 12 total, 12 up, 12 in 2023-09-13T04:48:04.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:03 smithi156 ceph-mon[133675]: pgmap v233: 129 pgs: 129 active+clean; 940 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 252 op/s 2023-09-13T04:48:04.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:03 smithi156 ceph-mon[133675]: osdmap e97: 12 total, 12 up, 12 in 2023-09-13T04:48:06.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:05 smithi002 ceph-mon[97651]: pgmap v235: 129 pgs: 15 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 72 active+clean; 954 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 272 op/s 2023-09-13T04:48:06.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:05 smithi149 ceph-mon[133315]: pgmap v235: 129 pgs: 15 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 72 active+clean; 954 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 272 op/s 2023-09-13T04:48:06.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:05 smithi156 ceph-mon[133675]: pgmap v235: 129 pgs: 15 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 72 active+clean; 954 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 272 op/s 2023-09-13T04:48:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.720+0000 7fc928dff700 1 -- 172.21.15.2:0/1435896250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc924105ff0 msgr2=0x7fc92410db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.720+0000 7fc928dff700 1 --2- 172.21.15.2:0/1435896250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc924105ff0 0x7fc92410db70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc91c009f90 tx=0x7fc91c00b330 comp rx=0 tx=0).stop 2023-09-13T04:48:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.720+0000 7fc928dff700 1 -- 172.21.15.2:0/1435896250 shutdown_connections 2023-09-13T04:48:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.720+0000 7fc928dff700 1 --2- 172.21.15.2:0/1435896250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc924105ff0 0x7fc92410db70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.720+0000 7fc928dff700 1 --2- 172.21.15.2:0/1435896250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc924100ed0 0x7fc9241012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.720+0000 7fc928dff700 1 --2- 172.21.15.2:0/1435896250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc924101880 0x7fc9241058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.720+0000 7fc928dff700 1 -- 172.21.15.2:0/1435896250 >> 172.21.15.2:0/1435896250 conn(0x7fc9240fc750 msgr2=0x7fc9240feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.720+0000 7fc928dff700 1 -- 172.21.15.2:0/1435896250 shutdown_connections 2023-09-13T04:48:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.720+0000 7fc928dff700 1 -- 172.21.15.2:0/1435896250 wait complete. 2023-09-13T04:48:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.721+0000 7fc928dff700 1 Processor -- start 2023-09-13T04:48:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.721+0000 7fc928dff700 1 -- start start 2023-09-13T04:48:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.722+0000 7fc928dff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc924100ed0 0x7fc924196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.722+0000 7fc928dff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc924105ff0 0x7fc924197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.722+0000 7fc928dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92419baa0 0x7fc92419bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.722+0000 7fc928dff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92419c520 con 0x7fc924100ed0 2023-09-13T04:48:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.722+0000 7fc928dff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92419c690 con 0x7fc924105ff0 2023-09-13T04:48:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.722+0000 7fc928dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92419c850 con 0x7fc92419baa0 2023-09-13T04:48:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.722+0000 7fc921d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc924105ff0 0x7fc924197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.722+0000 7fc921d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc924105ff0 0x7fc924197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:50674/0 (socket says 172.21.15.2:50674) 2023-09-13T04:48:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.722+0000 7fc921d9b700 1 -- 172.21.15.2:0/4003451045 learned_addr learned my addr 172.21.15.2:0/4003451045 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.722+0000 7fc922d9d700 1 --2- 172.21.15.2:0/4003451045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92419baa0 0x7fc92419bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.722+0000 7fc92259c700 1 --2- 172.21.15.2:0/4003451045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc924100ed0 0x7fc924196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.723+0000 7fc922d9d700 1 -- 172.21.15.2:0/4003451045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc924105ff0 msgr2=0x7fc924197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.723+0000 7fc922d9d700 1 --2- 172.21.15.2:0/4003451045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc924105ff0 0x7fc924197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.723+0000 7fc922d9d700 1 -- 172.21.15.2:0/4003451045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc924100ed0 msgr2=0x7fc924196ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.723+0000 7fc922d9d700 1 --2- 172.21.15.2:0/4003451045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc924100ed0 0x7fc924196ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.723+0000 7fc922d9d700 1 -- 172.21.15.2:0/4003451045 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91c009c40 con 0x7fc92419baa0 2023-09-13T04:48:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.723+0000 7fc92259c700 1 --2- 172.21.15.2:0/4003451045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc924100ed0 0x7fc924196ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:48:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.723+0000 7fc922d9d700 1 --2- 172.21.15.2:0/4003451045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92419baa0 0x7fc92419bf20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc91c00b6c0 tx=0x7fc91c004d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.723+0000 7fc9137fe700 1 -- 172.21.15.2:0/4003451045 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c00e040 con 0x7fc92419baa0 2023-09-13T04:48:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.723+0000 7fc9137fe700 1 -- 172.21.15.2:0/4003451045 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc91c008e80 con 0x7fc92419baa0 2023-09-13T04:48:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.723+0000 7fc928dff700 1 -- 172.21.15.2:0/4003451045 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9241a3d50 con 0x7fc92419baa0 2023-09-13T04:48:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.723+0000 7fc9137fe700 1 -- 172.21.15.2:0/4003451045 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c0052a0 con 0x7fc92419baa0 2023-09-13T04:48:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.724+0000 7fc928dff700 1 -- 172.21.15.2:0/4003451045 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9241a4220 con 0x7fc92419baa0 2023-09-13T04:48:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.728+0000 7fc9137fe700 1 -- 172.21.15.2:0/4003451045 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7fc91c01d470 con 0x7fc92419baa0 2023-09-13T04:48:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.730+0000 7fc93c7deb80 1 Processor -- start 2023-09-13T04:48:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.731+0000 7fc93c7deb80 1 -- start start 2023-09-13T04:48:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.731+0000 7fc9137fe700 1 --2- 172.21.15.2:0/4003451045 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc90c0a0830 0x7fc90c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.731+0000 7fc9137fe700 1 -- 172.21.15.2:0/4003451045 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9483+0+0 (secure 0 0 0) 0x7fc91c0cd670 con 0x7fc92419baa0 2023-09-13T04:48:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.731+0000 7fc93c7deb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136736be0 0x561136736fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.731+0000 7fc921d9b700 1 --2- 172.21.15.2:0/4003451045 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc90c0a0830 0x7fc90c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.731+0000 7fc93c7deb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561136737500 0x561136732300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.731+0000 7fc93c7deb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561136732840 0x561136732c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.731+0000 7fc93c7deb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56113668b720 con 0x561136732840 2023-09-13T04:48:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.731+0000 7fc93c7deb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561136689040 con 0x561136737500 2023-09-13T04:48:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.731+0000 7fc93c7deb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561136684930 con 0x561136736be0 2023-09-13T04:48:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.732+0000 7fc921d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561136737500 0x561136732300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.732+0000 7fc92259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136736be0 0x561136736fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.732+0000 7fc92259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136736be0 0x561136736fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:50356/0 (socket says 172.21.15.2:50356) 2023-09-13T04:48:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.732+0000 7fc92259c700 1 -- 172.21.15.2:0/286904704 learned_addr learned my addr 172.21.15.2:0/286904704 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.732+0000 7fc921d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561136737500 0x561136732300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:50684/0 (socket says 172.21.15.2:50684) 2023-09-13T04:48:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.732+0000 7fc922d9d700 1 --2- 172.21.15.2:0/286904704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561136732840 0x561136732c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.732+0000 7fc92259c700 1 -- 172.21.15.2:0/286904704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561136737500 msgr2=0x561136732300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.732+0000 7fc92259c700 1 --2- 172.21.15.2:0/286904704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561136737500 0x561136732300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.732+0000 7fc92259c700 1 -- 172.21.15.2:0/286904704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561136732840 msgr2=0x561136732c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.732+0000 7fc92259c700 1 --2- 172.21.15.2:0/286904704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561136732840 0x561136732c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.732+0000 7fc92259c700 1 -- 172.21.15.2:0/286904704 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611365045d0 con 0x561136736be0 2023-09-13T04:48:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.732+0000 7fc921d9b700 1 --2- 172.21.15.2:0/4003451045 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc90c0a0830 0x7fc90c0a2cf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc924198710 tx=0x7fc918011900 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.732+0000 7fc92259c700 1 --2- 172.21.15.2:0/286904704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136736be0 0x561136736fc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc914009de0 tx=0x7fc9140116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d7558028ff66055 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.733+0000 7fc910ff9700 1 -- 172.21.15.2:0/286904704 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914011da0 con 0x561136736be0 2023-09-13T04:48:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.733+0000 7fc910ff9700 1 -- 172.21.15.2:0/286904704 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc914012910 con 0x561136736be0 2023-09-13T04:48:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.733+0000 7fc910ff9700 1 -- 172.21.15.2:0/286904704 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91400f840 con 0x561136736be0 2023-09-13T04:48:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.733+0000 7fc93c7deb80 1 -- 172.21.15.2:0/286904704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136736be0 msgr2=0x561136736fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.733+0000 7fc93c7deb80 1 --2- 172.21.15.2:0/286904704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136736be0 0x561136736fc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc914009de0 tx=0x7fc9140116c0 comp rx=0 tx=0).stop 2023-09-13T04:48:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.733+0000 7fc93c7deb80 1 -- 172.21.15.2:0/286904704 shutdown_connections 2023-09-13T04:48:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.733+0000 7fc93c7deb80 1 --2- 172.21.15.2:0/286904704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561136732840 0x561136732c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.733+0000 7fc93c7deb80 1 --2- 172.21.15.2:0/286904704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136736be0 0x561136736fc0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.733+0000 7fc93c7deb80 1 --2- 172.21.15.2:0/286904704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561136737500 0x561136732300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.733+0000 7fc93c7deb80 1 -- 172.21.15.2:0/286904704 >> 172.21.15.2:0/286904704 conn(0x561136690c20 msgr2=0x56113672cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.734+0000 7fc93c7deb80 1 -- 172.21.15.2:0/286904704 shutdown_connections 2023-09-13T04:48:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.734+0000 7fc93c7deb80 1 -- 172.21.15.2:0/286904704 wait complete. 2023-09-13T04:48:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.734+0000 7fc93c7deb80 1 -- start start 2023-09-13T04:48:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.735+0000 7fc93c7deb80 1 Processor -- start 2023-09-13T04:48:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.735+0000 7fc93c7deb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561136732840 0x561136742650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.735+0000 7fc93c7deb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136736be0 0x56113673b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.735+0000 7fc93c7deb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561136737500 0x56113673bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.735+0000 7fc93c7deb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561136622b50 con 0x561136732840 2023-09-13T04:48:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.735+0000 7fc93c7deb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561136622850 con 0x561136737500 2023-09-13T04:48:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.735+0000 7fc93c7deb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611365068c0 con 0x561136736be0 2023-09-13T04:48:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.735+0000 7fc93c7deb80 10 client.0 _ll_register_callbacks cb 0x561136620620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:48:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.735+0000 7fc922d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561136737500 0x56113673bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.735+0000 7fc92259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561136732840 0x561136742650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.735+0000 7fc922d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561136737500 0x56113673bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:50690/0 (socket says 172.21.15.2:50690) 2023-09-13T04:48:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.735+0000 7fc922d9d700 1 -- 172.21.15.2:0/3279559790 learned_addr learned my addr 172.21.15.2:0/3279559790 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.735+0000 7fc921d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136736be0 0x56113673b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.736+0000 7fc922d9d700 1 -- 172.21.15.2:0/3279559790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136736be0 msgr2=0x56113673b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.736+0000 7fc922d9d700 1 --2- 172.21.15.2:0/3279559790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136736be0 0x56113673b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.736+0000 7fc922d9d700 1 -- 172.21.15.2:0/3279559790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561136732840 msgr2=0x561136742650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.736+0000 7fc922d9d700 1 --2- 172.21.15.2:0/3279559790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561136732840 0x561136742650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.736+0000 7fc922d9d700 1 -- 172.21.15.2:0/3279559790 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc914009a10 con 0x561136737500 2023-09-13T04:48:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.736+0000 7fc921d9b700 1 --2- 172.21.15.2:0/3279559790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136736be0 0x56113673b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:48:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.736+0000 7fc922d9d700 1 --2- 172.21.15.2:0/3279559790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561136737500 0x56113673bd60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0060b0 tx=0x7fc91c09a350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=654473c44d0872c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.736+0000 7fc909ffb700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:48:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.736+0000 7fc909ffb700 1 -- 172.21.15.2:0/3279559790 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c09a790 con 0x561136737500 2023-09-13T04:48:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.736+0000 7fc93c7deb80 1 -- 172.21.15.2:0/3279559790 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611366f7ef0 con 0x561136737500 2023-09-13T04:48:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.736+0000 7fc909ffb700 1 -- 172.21.15.2:0/3279559790 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9483+0+0 (secure 0 0 0) 0x7fc91c09e5c0 con 0x561136737500 2023-09-13T04:48:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.736+0000 7fc909ffb700 1 -- 172.21.15.2:0/3279559790 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc91c0a2770 con 0x561136737500 2023-09-13T04:48:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.737+0000 7fc93c7deb80 10 client.26582 fetch_fsmap learned FSMap version 36 2023-09-13T04:48:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.737+0000 7fc93c7deb80 1 -- 172.21.15.2:0/3279559790 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x56113669c740 con 0x561136737500 2023-09-13T04:48:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.737+0000 7fc909ffb700 1 -- 172.21.15.2:0/3279559790 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5611366f7ef0 con 0x561136737500 2023-09-13T04:48:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.737+0000 7fc909ffb700 1 -- 172.21.15.2:0/3279559790 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fc91c0cd1f0 con 0x561136737500 2023-09-13T04:48:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.737+0000 7fc93c7deb80 10 client.26582 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:48:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.737+0000 7fc93c7deb80 10 client.26582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:48:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.737+0000 7fc93c7deb80 20 client.26582 populate_metadata read hostname 'smithi002' 2023-09-13T04:48:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.737+0000 7fc93c7deb80 1 --2- 172.21.15.2:0/3279559790 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5611367545b0 0x5611367569d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.737+0000 7fc93c7deb80 4 client.26582 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:48:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.737+0000 7fc93c7deb80 1 -- 172.21.15.2:0/3279559790 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56113667d910 con 0x5611367545b0 2023-09-13T04:48:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.737+0000 7fc92259c700 1 --2- 172.21.15.2:0/3279559790 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5611367545b0 0x5611367569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.738+0000 7fc92259c700 1 --2- 172.21.15.2:0/3279559790 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5611367545b0 0x5611367569d0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4709454e4726421 server_cookie=4a118e7e64ff3ea4 in_seq=0 out_seq=0 2023-09-13T04:48:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.738+0000 7fc909ffb700 10 client.26582 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.738+0000 7fc909ffb700 1 -- 172.21.15.2:0/3279559790 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56113667d910 con 0x5611367545b0 2023-09-13T04:48:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.738+0000 7fc909ffb700 10 client.26582 handle_command_reply: tid=0 2023-09-13T04:48:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.739+0000 7fc93c7deb80 1 client.26582 shutdown 2023-09-13T04:48:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.739+0000 7fc93c7deb80 1 -- 172.21.15.2:0/3279559790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561136737500 msgr2=0x56113673bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.739+0000 7fc93c7deb80 1 --2- 172.21.15.2:0/3279559790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561136737500 0x56113673bd60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0060b0 tx=0x7fc91c09a350 comp rx=0 tx=0).stop 2023-09-13T04:48:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.739+0000 7fc93c7deb80 1 -- 172.21.15.2:0/3279559790 shutdown_connections 2023-09-13T04:48:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.739+0000 7fc93c7deb80 1 --2- 172.21.15.2:0/3279559790 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5611367545b0 0x5611367569d0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.739+0000 7fc93c7deb80 1 --2- 172.21.15.2:0/3279559790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561136737500 0x56113673bd60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.739+0000 7fc93c7deb80 1 --2- 172.21.15.2:0/3279559790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561136732840 0x561136742650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.739+0000 7fc93c7deb80 1 --2- 172.21.15.2:0/3279559790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136736be0 0x56113673b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.739+0000 7fc909ffb700 0 client.26582 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.739+0000 7fc93c7deb80 1 -- 172.21.15.2:0/3279559790 >> 172.21.15.2:0/3279559790 conn(0x561136690c20 msgr2=0x561136727630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.739+0000 7fc93c7deb80 1 -- 172.21.15.2:0/3279559790 shutdown_connections 2023-09-13T04:48:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.739+0000 7fc93c7deb80 1 -- 172.21.15.2:0/3279559790 wait complete. 2023-09-13T04:48:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.739+0000 7fc93c7deb80 20 client.26582 trim_cache size 0 max 16384 2023-09-13T04:48:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.766+0000 7fc93c7deb80 1 Processor -- start 2023-09-13T04:48:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.766+0000 7fc93c7deb80 1 -- start start 2023-09-13T04:48:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.767+0000 7fc93c7deb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136732840 0x561136740150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.767+0000 7fc93c7deb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561136736be0 0x561136727730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.767+0000 7fc93c7deb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561136737500 0x561136742130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.767+0000 7fc93c7deb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561136732410 con 0x561136737500 2023-09-13T04:48:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.767+0000 7fc93c7deb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561136727c70 con 0x561136736be0 2023-09-13T04:48:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.767+0000 7fc93c7deb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56113672d8d0 con 0x561136732840 2023-09-13T04:48:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.767+0000 7fc92259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136732840 0x561136740150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.767+0000 7fc921d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561136736be0 0x561136727730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.767+0000 7fc92259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136732840 0x561136740150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:50374/0 (socket says 172.21.15.2:50374) 2023-09-13T04:48:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.767+0000 7fc92259c700 1 -- 172.21.15.2:0/2694716753 learned_addr learned my addr 172.21.15.2:0/2694716753 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.767+0000 7fc922d9d700 1 --2- 172.21.15.2:0/2694716753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561136737500 0x561136742130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.768+0000 7fc92259c700 1 -- 172.21.15.2:0/2694716753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561136736be0 msgr2=0x561136727730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.768+0000 7fc92259c700 1 --2- 172.21.15.2:0/2694716753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561136736be0 0x561136727730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.768+0000 7fc92259c700 1 -- 172.21.15.2:0/2694716753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561136737500 msgr2=0x561136742130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.768+0000 7fc92259c700 1 --2- 172.21.15.2:0/2694716753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561136737500 0x561136742130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.768+0000 7fc92259c700 1 -- 172.21.15.2:0/2694716753 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611364b0d80 con 0x561136732840 2023-09-13T04:48:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.768+0000 7fc921d9b700 1 --2- 172.21.15.2:0/2694716753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561136736be0 0x561136727730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:48:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.768+0000 7fc92259c700 1 --2- 172.21.15.2:0/2694716753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136732840 0x561136740150 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc914009de0 tx=0x7fc914005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5df6bebe6cf0058c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.768+0000 7fc9117fa700 1 -- 172.21.15.2:0/2694716753 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91402a070 con 0x561136732840 2023-09-13T04:48:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.768+0000 7fc9117fa700 1 -- 172.21.15.2:0/2694716753 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc91402b250 con 0x561136732840 2023-09-13T04:48:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.768+0000 7fc9117fa700 1 -- 172.21.15.2:0/2694716753 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914012a80 con 0x561136732840 2023-09-13T04:48:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.769+0000 7fc93c7deb80 1 -- 172.21.15.2:0/2694716753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136732840 msgr2=0x561136740150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.769+0000 7fc93c7deb80 1 --2- 172.21.15.2:0/2694716753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136732840 0x561136740150 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc914009de0 tx=0x7fc914005040 comp rx=0 tx=0).stop 2023-09-13T04:48:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.769+0000 7fc93c7deb80 1 -- 172.21.15.2:0/2694716753 shutdown_connections 2023-09-13T04:48:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.769+0000 7fc93c7deb80 1 --2- 172.21.15.2:0/2694716753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561136737500 0x561136742130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.769+0000 7fc93c7deb80 1 --2- 172.21.15.2:0/2694716753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136732840 0x561136740150 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.769+0000 7fc93c7deb80 1 --2- 172.21.15.2:0/2694716753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561136736be0 0x561136727730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.769+0000 7fc93c7deb80 1 -- 172.21.15.2:0/2694716753 >> 172.21.15.2:0/2694716753 conn(0x561136690c20 msgr2=0x56113672ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.769+0000 7fc93c7deb80 1 -- 172.21.15.2:0/2694716753 shutdown_connections 2023-09-13T04:48:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.769+0000 7fc93c7deb80 1 -- 172.21.15.2:0/2694716753 wait complete. 2023-09-13T04:48:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.770+0000 7fc93c7deb80 1 -- start start 2023-09-13T04:48:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.770+0000 7fc93c7deb80 1 Processor -- start 2023-09-13T04:48:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.771+0000 7fc93c7deb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136732840 0x56113674d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.771+0000 7fc93c7deb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561136736be0 0x56113674d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.771+0000 7fc93c7deb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561136737500 0x5611367a73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.771+0000 7fc93c7deb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611367a7910 con 0x561136737500 2023-09-13T04:48:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.771+0000 7fc93c7deb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611367a7a50 con 0x561136736be0 2023-09-13T04:48:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.771+0000 7fc93c7deb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611367a7bc0 con 0x561136732840 2023-09-13T04:48:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.771+0000 7fc93c7deb80 10 client.0 _ll_register_callbacks cb 0x561136620620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:48:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.771+0000 7fc92259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136732840 0x56113674d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.771+0000 7fc92259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136732840 0x56113674d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:50382/0 (socket says 172.21.15.2:50382) 2023-09-13T04:48:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.771+0000 7fc92259c700 1 -- 172.21.15.2:0/2379632658 learned_addr learned my addr 172.21.15.2:0/2379632658 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.771+0000 7fc921d9b700 1 --2- 172.21.15.2:0/2379632658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561136736be0 0x56113674d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.771+0000 7fc922d9d700 1 --2- 172.21.15.2:0/2379632658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561136737500 0x5611367a73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.771+0000 7fc92259c700 1 -- 172.21.15.2:0/2379632658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561136736be0 msgr2=0x56113674d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.771+0000 7fc92259c700 1 --2- 172.21.15.2:0/2379632658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561136736be0 0x56113674d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.771+0000 7fc92259c700 1 -- 172.21.15.2:0/2379632658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561136737500 msgr2=0x5611367a73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.771+0000 7fc92259c700 1 --2- 172.21.15.2:0/2379632658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561136737500 0x5611367a73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.771+0000 7fc92259c700 1 -- 172.21.15.2:0/2379632658 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc918005da0 con 0x561136732840 2023-09-13T04:48:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.772+0000 7fc92259c700 1 --2- 172.21.15.2:0/2379632658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136732840 0x56113674d290 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc914011a10 tx=0x7fc914009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e24af543452fe89f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.772+0000 7fc909ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:48:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.772+0000 7fc909ffb700 1 -- 172.21.15.2:0/2379632658 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91402a070 con 0x561136732840 2023-09-13T04:48:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.772+0000 7fc909ffb700 1 -- 172.21.15.2:0/2379632658 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9483+0+0 (secure 0 0 0) 0x7fc914012660 con 0x561136732840 2023-09-13T04:48:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.772+0000 7fc93c7deb80 1 -- 172.21.15.2:0/2379632658 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc914012910 con 0x561136732840 2023-09-13T04:48:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.772+0000 7fc909ffb700 1 -- 172.21.15.2:0/2379632658 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc914012a80 con 0x561136732840 2023-09-13T04:48:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.772+0000 7fc93c7deb80 10 client.26602 fetch_fsmap learned FSMap version 36 2023-09-13T04:48:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.772+0000 7fc93c7deb80 1 -- 172.21.15.2:0/2379632658 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5611365045d0 con 0x561136732840 2023-09-13T04:48:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.773+0000 7fc909ffb700 1 -- 172.21.15.2:0/2379632658 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc914012910 con 0x561136732840 2023-09-13T04:48:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.773+0000 7fc909ffb700 1 -- 172.21.15.2:0/2379632658 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fc914022070 con 0x561136732840 2023-09-13T04:48:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.773+0000 7fc93c7deb80 10 client.26602 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:48:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.773+0000 7fc93c7deb80 10 client.26602 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:48:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.773+0000 7fc93c7deb80 20 client.26602 populate_metadata read hostname 'smithi002' 2023-09-13T04:48:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.773+0000 7fc93c7deb80 1 --2- 172.21.15.2:0/2379632658 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5611367a9040 0x5611367ab460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.773+0000 7fc93c7deb80 4 client.26602 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:48:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.773+0000 7fc93c7deb80 1 -- 172.21.15.2:0/2379632658 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56113663dff0 con 0x5611367a9040 2023-09-13T04:48:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.773+0000 7fc921d9b700 1 --2- 172.21.15.2:0/2379632658 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5611367a9040 0x5611367ab460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.774+0000 7fc921d9b700 1 --2- 172.21.15.2:0/2379632658 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5611367a9040 0x5611367ab460 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f04e96e1ae07deb server_cookie=9a7314e788f8eb63 in_seq=0 out_seq=0 2023-09-13T04:48:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:07.774+0000 7fc909ffb700 10 client.26602 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:08.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:07 smithi002 ceph-mon[97651]: pgmap v236: 129 pgs: 17 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 63 active+clean; 954 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 261 op/s 2023-09-13T04:48:08.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:07 smithi149 ceph-mon[133315]: pgmap v236: 129 pgs: 17 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 63 active+clean; 954 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 261 op/s 2023-09-13T04:48:08.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:07 smithi156 ceph-mon[133675]: pgmap v236: 129 pgs: 17 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 63 active+clean; 954 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 261 op/s 2023-09-13T04:48:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.722+0000 7fc909ffb700 1 -- 172.21.15.2:0/2379632658 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56113663dff0 con 0x5611367a9040 2023-09-13T04:48:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.722+0000 7fc909ffb700 10 client.26602 handle_command_reply: tid=0 2023-09-13T04:48:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.723+0000 7fc93c7deb80 1 client.26602 shutdown 2023-09-13T04:48:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.723+0000 7fc93c7deb80 1 -- 172.21.15.2:0/2379632658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136732840 msgr2=0x56113674d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.723+0000 7fc93c7deb80 1 --2- 172.21.15.2:0/2379632658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136732840 0x56113674d290 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc914011a10 tx=0x7fc914009260 comp rx=0 tx=0).stop 2023-09-13T04:48:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.723+0000 7fc93c7deb80 1 -- 172.21.15.2:0/2379632658 shutdown_connections 2023-09-13T04:48:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.723+0000 7fc93c7deb80 1 --2- 172.21.15.2:0/2379632658 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5611367a9040 0x5611367ab460 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.723+0000 7fc93c7deb80 1 --2- 172.21.15.2:0/2379632658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561136737500 0x5611367a73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.723+0000 7fc93c7deb80 1 --2- 172.21.15.2:0/2379632658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561136732840 0x56113674d290 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.723+0000 7fc93c7deb80 1 --2- 172.21.15.2:0/2379632658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561136736be0 0x56113674d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.723+0000 7fc93c7deb80 1 -- 172.21.15.2:0/2379632658 >> 172.21.15.2:0/2379632658 conn(0x561136690c20 msgr2=0x561136739fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.723+0000 7fc909ffb700 0 client.26602 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.724+0000 7fc93c7deb80 1 -- 172.21.15.2:0/2379632658 shutdown_connections 2023-09-13T04:48:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.724+0000 7fc93c7deb80 1 -- 172.21.15.2:0/2379632658 wait complete. 2023-09-13T04:48:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.727+0000 7fc93c7deb80 20 client.26602 trim_cache size 0 max 16384 2023-09-13T04:48:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:48:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:48:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:48:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:48:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.732+0000 7fc928dff700 1 -- 172.21.15.2:0/4003451045 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc90c0a0830 msgr2=0x7fc90c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.733+0000 7fc928dff700 1 --2- 172.21.15.2:0/4003451045 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc90c0a0830 0x7fc90c0a2cf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc924198710 tx=0x7fc918011900 comp rx=0 tx=0).stop 2023-09-13T04:48:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.733+0000 7fc928dff700 1 -- 172.21.15.2:0/4003451045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92419baa0 msgr2=0x7fc92419bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.733+0000 7fc928dff700 1 --2- 172.21.15.2:0/4003451045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92419baa0 0x7fc92419bf20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc91c00b6c0 tx=0x7fc91c004d30 comp rx=0 tx=0).stop 2023-09-13T04:48:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.733+0000 7fc928dff700 1 -- 172.21.15.2:0/4003451045 shutdown_connections 2023-09-13T04:48:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.733+0000 7fc928dff700 1 --2- 172.21.15.2:0/4003451045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92419baa0 0x7fc92419bf20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.733+0000 7fc928dff700 1 --2- 172.21.15.2:0/4003451045 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc90c0a0830 0x7fc90c0a2cf0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.733+0000 7fc928dff700 1 --2- 172.21.15.2:0/4003451045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc924100ed0 0x7fc924196ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.733+0000 7fc928dff700 1 --2- 172.21.15.2:0/4003451045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc924105ff0 0x7fc924197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.733+0000 7fc928dff700 1 -- 172.21.15.2:0/4003451045 >> 172.21.15.2:0/4003451045 conn(0x7fc9240fc750 msgr2=0x7fc9240fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.737+0000 7fc928dff700 1 -- 172.21.15.2:0/4003451045 shutdown_connections 2023-09-13T04:48:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:08.737+0000 7fc928dff700 1 -- 172.21.15.2:0/4003451045 wait complete. 2023-09-13T04:48:10.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:09 smithi002 ceph-mon[97651]: pgmap v237: 129 pgs: 15 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 72 active+clean; 955 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 261 op/s 2023-09-13T04:48:10.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:09 smithi149 ceph-mon[133315]: pgmap v237: 129 pgs: 15 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 72 active+clean; 955 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 261 op/s 2023-09-13T04:48:10.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:09 smithi156 ceph-mon[133675]: pgmap v237: 129 pgs: 15 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 72 active+clean; 955 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 261 op/s 2023-09-13T04:48:11.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:11 smithi149 ceph-mon[133315]: pgmap v238: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 942 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 501 op/s 2023-09-13T04:48:11.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:11 smithi156 ceph-mon[133675]: pgmap v238: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 942 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 501 op/s 2023-09-13T04:48:11.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:11 smithi002 ceph-mon[97651]: pgmap v238: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 942 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 501 op/s 2023-09-13T04:48:12.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:12 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:48:12.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:12 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:48:13.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:12 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:48:13.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:13 smithi149 ceph-mon[133315]: pgmap v239: 129 pgs: 129 active+clean; 947 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 569 op/s 2023-09-13T04:48:13.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:13 smithi156 ceph-mon[133675]: pgmap v239: 129 pgs: 129 active+clean; 947 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 569 op/s 2023-09-13T04:48:14.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:13 smithi002 ceph-mon[97651]: pgmap v239: 129 pgs: 129 active+clean; 947 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 569 op/s 2023-09-13T04:48:14.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:14 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:14.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:14 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:48:14.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:14 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:48:14.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:14 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:48:14.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:14 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:14.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:14 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:48:14.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:14 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:48:14.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:14 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:48:15.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:14 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:15.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:14 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:48:15.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:14 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:48:15.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:14 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:48:16.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:15 smithi149 ceph-mon[133315]: pgmap v240: 129 pgs: 129 active+clean; 954 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 579 op/s 2023-09-13T04:48:16.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:15 smithi149 ceph-mon[133315]: osdmap e98: 12 total, 12 up, 12 in 2023-09-13T04:48:16.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:15 smithi156 ceph-mon[133675]: pgmap v240: 129 pgs: 129 active+clean; 954 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 579 op/s 2023-09-13T04:48:16.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:15 smithi156 ceph-mon[133675]: osdmap e98: 12 total, 12 up, 12 in 2023-09-13T04:48:16.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:15 smithi002 ceph-mon[97651]: pgmap v240: 129 pgs: 129 active+clean; 954 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 579 op/s 2023-09-13T04:48:16.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:15 smithi002 ceph-mon[97651]: osdmap e98: 12 total, 12 up, 12 in 2023-09-13T04:48:18.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:17 smithi149 ceph-mon[133315]: pgmap v242: 129 pgs: 129 active+clean; 956 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 571 op/s 2023-09-13T04:48:18.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:17 smithi156 ceph-mon[133675]: pgmap v242: 129 pgs: 129 active+clean; 956 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 571 op/s 2023-09-13T04:48:18.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:17 smithi002 ceph-mon[97651]: pgmap v242: 129 pgs: 129 active+clean; 956 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 571 op/s 2023-09-13T04:48:20.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:19 smithi149 ceph-mon[133315]: pgmap v243: 129 pgs: 129 active+clean; 956 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 569 op/s 2023-09-13T04:48:20.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:19 smithi156 ceph-mon[133675]: pgmap v243: 129 pgs: 129 active+clean; 956 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 569 op/s 2023-09-13T04:48:20.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:19 smithi002 ceph-mon[97651]: pgmap v243: 129 pgs: 129 active+clean; 956 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 569 op/s 2023-09-13T04:48:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.008+0000 7f14e33ff700 1 -- 172.21.15.2:0/3959165527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14dc0ff570 msgr2=0x7f14dc0ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.008+0000 7f14e33ff700 1 --2- 172.21.15.2:0/3959165527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14dc0ff570 0x7f14dc0ff950 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009990 tx=0x7f14d8009ca0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.008+0000 7f14e33ff700 1 -- 172.21.15.2:0/3959165527 shutdown_connections 2023-09-13T04:48:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.008+0000 7f14e33ff700 1 --2- 172.21.15.2:0/3959165527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14dc111970 0x7f14dc113d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.008+0000 7f14e33ff700 1 --2- 172.21.15.2:0/3959165527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14dc0ff570 0x7f14dc0ff950 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.008+0000 7f14e33ff700 1 --2- 172.21.15.2:0/3959165527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14dc0fff20 0x7f14dc111430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.008+0000 7f14e33ff700 1 -- 172.21.15.2:0/3959165527 >> 172.21.15.2:0/3959165527 conn(0x7f14dc074e60 msgr2=0x7f14dc075270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.009+0000 7f14e33ff700 1 -- 172.21.15.2:0/3959165527 shutdown_connections 2023-09-13T04:48:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.009+0000 7f14e33ff700 1 -- 172.21.15.2:0/3959165527 wait complete. 2023-09-13T04:48:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.010+0000 7f14e33ff700 1 Processor -- start 2023-09-13T04:48:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.010+0000 7f14e33ff700 1 -- start start 2023-09-13T04:48:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.011+0000 7f14e33ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14dc0ff570 0x7f14dc072320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.011+0000 7f14e33ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14dc0fff20 0x7f14dc072860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.011+0000 7f14e33ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14dc111970 0x7f14dc06c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.011+0000 7f14e33ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc06c9b0 con 0x7f14dc0fff20 2023-09-13T04:48:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.011+0000 7f14e33ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc06cb20 con 0x7f14dc0ff570 2023-09-13T04:48:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.011+0000 7f14e33ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc06cce0 con 0x7f14dc111970 2023-09-13T04:48:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.012+0000 7f14e199c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14dc111970 0x7f14dc06c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.012+0000 7f14e199c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14dc111970 0x7f14dc06c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:34004/0 (socket says 172.21.15.2:34004) 2023-09-13T04:48:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.012+0000 7f14e199c700 1 -- 172.21.15.2:0/2761184031 learned_addr learned my addr 172.21.15.2:0/2761184031 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.012+0000 7f14e099a700 1 --2- 172.21.15.2:0/2761184031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14dc0fff20 0x7f14dc072860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.012+0000 7f14e119b700 1 --2- 172.21.15.2:0/2761184031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14dc0ff570 0x7f14dc072320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.012+0000 7f14e199c700 1 -- 172.21.15.2:0/2761184031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14dc0ff570 msgr2=0x7f14dc072320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.012+0000 7f14e199c700 1 --2- 172.21.15.2:0/2761184031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14dc0ff570 0x7f14dc072320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.012+0000 7f14e199c700 1 -- 172.21.15.2:0/2761184031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14dc0fff20 msgr2=0x7f14dc072860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.012+0000 7f14e199c700 1 --2- 172.21.15.2:0/2761184031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14dc0fff20 0x7f14dc072860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.012+0000 7f14e199c700 1 -- 172.21.15.2:0/2761184031 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d80095f0 con 0x7f14dc111970 2023-09-13T04:48:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.013+0000 7f14e199c700 1 --2- 172.21.15.2:0/2761184031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14dc111970 0x7f14dc06c1e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f14d400ef90 tx=0x7f14d400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.013+0000 7f14d27fc700 1 -- 172.21.15.2:0/2761184031 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d4018070 con 0x7f14dc111970 2023-09-13T04:48:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.014+0000 7f14d27fc700 1 -- 172.21.15.2:0/2761184031 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14d400f040 con 0x7f14dc111970 2023-09-13T04:48:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.014+0000 7f14d27fc700 1 -- 172.21.15.2:0/2761184031 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d4003e00 con 0x7f14dc111970 2023-09-13T04:48:22.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.014+0000 7f14e33ff700 1 -- 172.21.15.2:0/2761184031 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14dc06cf60 con 0x7f14dc111970 2023-09-13T04:48:22.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.014+0000 7f14e33ff700 1 -- 172.21.15.2:0/2761184031 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14dc106f40 con 0x7f14dc111970 2023-09-13T04:48:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.027+0000 7f14f6ca4b80 1 Processor -- start 2023-09-13T04:48:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.030+0000 7f14f6ca4b80 1 -- start start 2023-09-13T04:48:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.031+0000 7f14f6ca4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd4c1e0be0 0x55cd4c1e0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.031+0000 7f14f6ca4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd4c1e1500 0x55cd4c1dc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.031+0000 7f14f6ca4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd4c1dc840 0x55cd4c1dcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.031+0000 7f14f6ca4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd4c135720 con 0x55cd4c1e1500 2023-09-13T04:48:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.031+0000 7f14f6ca4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd4c133040 con 0x55cd4c1dc840 2023-09-13T04:48:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.031+0000 7f14f6ca4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd4c12e930 con 0x55cd4c1e0be0 2023-09-13T04:48:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.032+0000 7f14e119b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd4c1e1500 0x55cd4c1dc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.032+0000 7f14e119b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd4c1e1500 0x55cd4c1dc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56296/0 (socket says 172.21.15.2:56296) 2023-09-13T04:48:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.032+0000 7f14e119b700 1 -- 172.21.15.2:0/940387888 learned_addr learned my addr 172.21.15.2:0/940387888 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.032+0000 7f14e099a700 1 --2- 172.21.15.2:0/940387888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd4c1dc840 0x55cd4c1dcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.032+0000 7f14e099a700 1 --2- 172.21.15.2:0/940387888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd4c1e0be0 0x55cd4c1e0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.033+0000 7f14e099a700 1 -- 172.21.15.2:0/940387888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd4c1e0be0 msgr2=0x55cd4c1e0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.033+0000 7f14e099a700 1 --2- 172.21.15.2:0/940387888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd4c1e0be0 0x55cd4c1e0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.033+0000 7f14e099a700 1 -- 172.21.15.2:0/940387888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd4c1e1500 msgr2=0x55cd4c1dc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.033+0000 7f14e099a700 1 --2- 172.21.15.2:0/940387888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd4c1e1500 0x55cd4c1dc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.033+0000 7f14e099a700 1 -- 172.21.15.2:0/940387888 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd4bfae5d0 con 0x55cd4c1dc840 2023-09-13T04:48:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.034+0000 7f14e099a700 1 --2- 172.21.15.2:0/940387888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd4c1dc840 0x55cd4c1dcc80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f14cc006fd0 tx=0x7f14cc00a2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ad42aecf34c5ebd server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.034+0000 7f14c77fe700 1 -- 172.21.15.2:0/940387888 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc016ac0 con 0x55cd4c1dc840 2023-09-13T04:48:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.034+0000 7f14c77fe700 1 -- 172.21.15.2:0/940387888 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14cc00c030 con 0x55cd4c1dc840 2023-09-13T04:48:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.034+0000 7f14c77fe700 1 -- 172.21.15.2:0/940387888 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc018dd0 con 0x55cd4c1dc840 2023-09-13T04:48:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.035+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/940387888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd4c1dc840 msgr2=0x55cd4c1dcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.035+0000 7f14f6ca4b80 1 --2- 172.21.15.2:0/940387888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd4c1dc840 0x55cd4c1dcc80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f14cc006fd0 tx=0x7f14cc00a2b0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.035+0000 7f14d27fc700 1 -- 172.21.15.2:0/2761184031 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f14d400ca50 con 0x7f14dc111970 2023-09-13T04:48:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.036+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/940387888 shutdown_connections 2023-09-13T04:48:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.036+0000 7f14f6ca4b80 1 --2- 172.21.15.2:0/940387888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd4c1dc840 0x55cd4c1dcc80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.036+0000 7f14f6ca4b80 1 --2- 172.21.15.2:0/940387888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd4c1e0be0 0x55cd4c1e0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.036+0000 7f14f6ca4b80 1 --2- 172.21.15.2:0/940387888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd4c1e1500 0x55cd4c1dc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.036+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/940387888 >> 172.21.15.2:0/940387888 conn(0x55cd4c13ac20 msgr2=0x55cd4c1d6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.036+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/940387888 shutdown_connections 2023-09-13T04:48:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.038+0000 7f14d27fc700 1 --2- 172.21.15.2:0/2761184031 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f14c80a0800 0x7f14c80a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.038+0000 7f14d27fc700 1 -- 172.21.15.2:0/2761184031 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f14d40cbe20 con 0x7f14dc111970 2023-09-13T04:48:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.038+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/940387888 wait complete. 2023-09-13T04:48:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.039+0000 7f14f6ca4b80 1 -- start start 2023-09-13T04:48:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.039+0000 7f14e119b700 1 --2- 172.21.15.2:0/2761184031 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f14c80a0800 0x7f14c80a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.040+0000 7f14f6ca4b80 1 Processor -- start 2023-09-13T04:48:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.041+0000 7f14f6ca4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd4c1dc840 0x55cd4c1e8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.041+0000 7f14f6ca4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd4c1e0be0 0x55cd4c1e5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.041+0000 7f14f6ca4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd4c1e1500 0x55cd4c1e5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.041+0000 7f14f6ca4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd4c0ccb50 con 0x55cd4c1e0be0 2023-09-13T04:48:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.041+0000 7f14f6ca4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd4c0cc850 con 0x55cd4c1e1500 2023-09-13T04:48:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.041+0000 7f14f6ca4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd4bfb08c0 con 0x55cd4c1dc840 2023-09-13T04:48:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.041+0000 7f14f6ca4b80 10 client.0 _ll_register_callbacks cb 0x55cd4c0ca620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:48:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.042+0000 7f14e119b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd4c1dc840 0x55cd4c1e8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.042+0000 7f14e119b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd4c1dc840 0x55cd4c1e8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:34026/0 (socket says 172.21.15.2:34026) 2023-09-13T04:48:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.042+0000 7f14e119b700 1 -- 172.21.15.2:0/1169291109 learned_addr learned my addr 172.21.15.2:0/1169291109 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.043+0000 7f14e119b700 1 --2- 172.21.15.2:0/2761184031 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f14c80a0800 0x7f14c80a2cc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f14d8000c00 tx=0x7f14d8005fd0 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.043+0000 7f14e119b700 1 -- 172.21.15.2:0/1169291109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd4c1e1500 msgr2=0x55cd4c1e5ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:48:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.043+0000 7f14e119b700 1 --2- 172.21.15.2:0/1169291109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd4c1e1500 0x55cd4c1e5ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.043+0000 7f14e119b700 1 -- 172.21.15.2:0/1169291109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd4c1e0be0 msgr2=0x55cd4c1e5800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:48:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.043+0000 7f14e119b700 1 --2- 172.21.15.2:0/1169291109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd4c1e0be0 0x55cd4c1e5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.043+0000 7f14e119b700 1 -- 172.21.15.2:0/1169291109 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14cc00e040 con 0x55cd4c1dc840 2023-09-13T04:48:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.044+0000 7f14e119b700 1 --2- 172.21.15.2:0/1169291109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd4c1dc840 0x55cd4c1e8580 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f14d800bd00 tx=0x7f14d8009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e789e76aff4c6591 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.044+0000 7f14c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:48:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.044+0000 7f14c4ff9700 1 -- 172.21.15.2:0/1169291109 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d801d050 con 0x55cd4c1dc840 2023-09-13T04:48:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.044+0000 7f14c4ff9700 1 -- 172.21.15.2:0/1169291109 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f14d800f460 con 0x55cd4c1dc840 2023-09-13T04:48:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.044+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/1169291109 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd4c1a1ef0 con 0x55cd4c1dc840 2023-09-13T04:48:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.048+0000 7f14c4ff9700 1 -- 172.21.15.2:0/1169291109 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14d800f620 con 0x55cd4c1dc840 2023-09-13T04:48:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.049+0000 7f14f6ca4b80 10 client.26614 fetch_fsmap learned FSMap version 36 2023-09-13T04:48:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.049+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/1169291109 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55cd4c146740 con 0x55cd4c1dc840 2023-09-13T04:48:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.052+0000 7f14c4ff9700 1 -- 172.21.15.2:0/1169291109 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cd4c1a1ef0 con 0x55cd4c1dc840 2023-09-13T04:48:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.053+0000 7f14c4ff9700 1 -- 172.21.15.2:0/1169291109 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f14d80249a0 con 0x55cd4c1dc840 2023-09-13T04:48:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.053+0000 7f14f6ca4b80 10 client.26614 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:48:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.053+0000 7f14f6ca4b80 10 client.26614 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:48:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.053+0000 7f14f6ca4b80 20 client.26614 populate_metadata read hostname 'smithi002' 2023-09-13T04:48:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.053+0000 7f14f6ca4b80 1 --2- 172.21.15.2:0/1169291109 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55cd4c1f6070 0x55cd4c1f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.053+0000 7f14f6ca4b80 4 client.26614 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:48:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.053+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/1169291109 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd4c127910 con 0x55cd4c1f6070 2023-09-13T04:48:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.055+0000 7f14e099a700 1 --2- 172.21.15.2:0/1169291109 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55cd4c1f6070 0x55cd4c1f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.064+0000 7f14e099a700 1 --2- 172.21.15.2:0/1169291109 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55cd4c1f6070 0x55cd4c1f84a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb2ee11478be50cd server_cookie=31a8bcedb374db63 in_seq=0 out_seq=0 2023-09-13T04:48:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.064+0000 7f14c4ff9700 10 client.26614 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.067+0000 7f14c4ff9700 1 -- 172.21.15.2:0/1169291109 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cd4c127910 con 0x55cd4c1f6070 2023-09-13T04:48:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.067+0000 7f14c4ff9700 10 client.26614 handle_command_reply: tid=0 2023-09-13T04:48:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.068+0000 7f14f6ca4b80 1 client.26614 shutdown 2023-09-13T04:48:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.070+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/1169291109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd4c1dc840 msgr2=0x55cd4c1e8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.070+0000 7f14f6ca4b80 1 --2- 172.21.15.2:0/1169291109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd4c1dc840 0x55cd4c1e8580 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f14d800bd00 tx=0x7f14d8009ca0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.073+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/1169291109 shutdown_connections 2023-09-13T04:48:22.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.073+0000 7f14f6ca4b80 1 --2- 172.21.15.2:0/1169291109 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55cd4c1f6070 0x55cd4c1f84a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.073+0000 7f14f6ca4b80 1 --2- 172.21.15.2:0/1169291109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd4c1e1500 0x55cd4c1e5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.073+0000 7f14f6ca4b80 1 --2- 172.21.15.2:0/1169291109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd4c1dc840 0x55cd4c1e8580 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.073+0000 7f14f6ca4b80 1 --2- 172.21.15.2:0/1169291109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd4c1e0be0 0x55cd4c1e5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.073+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/1169291109 >> 172.21.15.2:0/1169291109 conn(0x55cd4c13ac20 msgr2=0x55cd4c1e2330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.074+0000 7f14c4ff9700 0 client.26614 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.075+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/1169291109 shutdown_connections 2023-09-13T04:48:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.075+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/1169291109 wait complete. 2023-09-13T04:48:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.075+0000 7f14f6ca4b80 20 client.26614 trim_cache size 0 max 16384 2023-09-13T04:48:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.096+0000 7f14f6ca4b80 1 Processor -- start 2023-09-13T04:48:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.096+0000 7f14f6ca4b80 1 -- start start 2023-09-13T04:48:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.097+0000 7f14f6ca4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd4c1dc840 0x55cd4c1ecfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.097+0000 7f14f6ca4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd4c1e0be0 0x55cd4c1e8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.097+0000 7f14f6ca4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd4c1e1500 0x55cd4c1df740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.097+0000 7f14f6ca4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd4c1e0800 con 0x55cd4c1e1500 2023-09-13T04:48:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.097+0000 7f14f6ca4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd4c1ed520 con 0x55cd4c1e0be0 2023-09-13T04:48:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.097+0000 7f14f6ca4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd4c1e67b0 con 0x55cd4c1dc840 2023-09-13T04:48:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.097+0000 7f14e099a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd4c1e0be0 0x55cd4c1e8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.097+0000 7f14e099a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd4c1e0be0 0x55cd4c1e8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:58852/0 (socket says 172.21.15.2:58852) 2023-09-13T04:48:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.097+0000 7f14e099a700 1 -- 172.21.15.2:0/1287801911 learned_addr learned my addr 172.21.15.2:0/1287801911 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.097+0000 7f14e119b700 1 --2- 172.21.15.2:0/1287801911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd4c1dc840 0x55cd4c1ecfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.097+0000 7f14e199c700 1 --2- 172.21.15.2:0/1287801911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd4c1e1500 0x55cd4c1df740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.097+0000 7f14e099a700 1 -- 172.21.15.2:0/1287801911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd4c1dc840 msgr2=0x55cd4c1ecfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.097+0000 7f14e099a700 1 --2- 172.21.15.2:0/1287801911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd4c1dc840 0x55cd4c1ecfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.098+0000 7f14e099a700 1 -- 172.21.15.2:0/1287801911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd4c1e1500 msgr2=0x55cd4c1df740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.098+0000 7f14e099a700 1 --2- 172.21.15.2:0/1287801911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd4c1e1500 0x55cd4c1df740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.098+0000 7f14e099a700 1 -- 172.21.15.2:0/1287801911 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd4bf5ad80 con 0x55cd4c1e0be0 2023-09-13T04:48:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.098+0000 7f14e119b700 1 --2- 172.21.15.2:0/1287801911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd4c1dc840 0x55cd4c1ecfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:48:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.098+0000 7f14e199c700 1 --2- 172.21.15.2:0/1287801911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd4c1e1500 0x55cd4c1df740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:48:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.098+0000 7f14e099a700 1 --2- 172.21.15.2:0/1287801911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd4c1e0be0 0x55cd4c1e8120 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f14cc006fd0 tx=0x7f14cc000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa6ad9b4904e1ead server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.098+0000 7f14c7fff700 1 -- 172.21.15.2:0/1287801911 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc00a0d0 con 0x55cd4c1e0be0 2023-09-13T04:48:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.098+0000 7f14c7fff700 1 -- 172.21.15.2:0/1287801911 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14cc016b00 con 0x55cd4c1e0be0 2023-09-13T04:48:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.098+0000 7f14c7fff700 1 -- 172.21.15.2:0/1287801911 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc018480 con 0x55cd4c1e0be0 2023-09-13T04:48:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.099+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/1287801911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd4c1e0be0 msgr2=0x55cd4c1e8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.099+0000 7f14f6ca4b80 1 --2- 172.21.15.2:0/1287801911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd4c1e0be0 0x55cd4c1e8120 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f14cc006fd0 tx=0x7f14cc000bc0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.099+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/1287801911 shutdown_connections 2023-09-13T04:48:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.099+0000 7f14f6ca4b80 1 --2- 172.21.15.2:0/1287801911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd4c1e1500 0x55cd4c1df740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.099+0000 7f14f6ca4b80 1 --2- 172.21.15.2:0/1287801911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd4c1dc840 0x55cd4c1ecfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.099+0000 7f14f6ca4b80 1 --2- 172.21.15.2:0/1287801911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd4c1e0be0 0x55cd4c1e8120 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.099+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/1287801911 >> 172.21.15.2:0/1287801911 conn(0x55cd4c13ac20 msgr2=0x55cd4c1f8080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.099+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/1287801911 shutdown_connections 2023-09-13T04:48:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.099+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/1287801911 wait complete. 2023-09-13T04:48:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.100+0000 7f14f6ca4b80 1 -- start start 2023-09-13T04:48:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.100+0000 7f14f6ca4b80 1 Processor -- start 2023-09-13T04:48:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.101+0000 7f14f6ca4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd4c1dc840 0x55cd4c1eafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.101+0000 7f14f6ca4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd4c1e0be0 0x55cd4c1eb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.101+0000 7f14f6ca4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd4c1e1500 0x55cd4c1eba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.101+0000 7f14f6ca4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd4c248f00 con 0x55cd4c1e1500 2023-09-13T04:48:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.101+0000 7f14f6ca4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd4c249070 con 0x55cd4c1e0be0 2023-09-13T04:48:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.101+0000 7f14f6ca4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd4c249230 con 0x55cd4c1dc840 2023-09-13T04:48:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.101+0000 7f14f6ca4b80 10 client.0 _ll_register_callbacks cb 0x55cd4c0ca620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:48:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.101+0000 7f14e099a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd4c1e0be0 0x55cd4c1eb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.101+0000 7f14e099a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd4c1e0be0 0x55cd4c1eb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:58856/0 (socket says 172.21.15.2:58856) 2023-09-13T04:48:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.101+0000 7f14e099a700 1 -- 172.21.15.2:0/1845290842 learned_addr learned my addr 172.21.15.2:0/1845290842 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.101+0000 7f14e119b700 1 --2- 172.21.15.2:0/1845290842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd4c1dc840 0x55cd4c1eafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.101+0000 7f14e199c700 1 --2- 172.21.15.2:0/1845290842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd4c1e1500 0x55cd4c1eba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.101+0000 7f14e099a700 1 -- 172.21.15.2:0/1845290842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd4c1dc840 msgr2=0x55cd4c1eafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.101+0000 7f14e099a700 1 --2- 172.21.15.2:0/1845290842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd4c1dc840 0x55cd4c1eafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.101+0000 7f14e099a700 1 -- 172.21.15.2:0/1845290842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd4c1e1500 msgr2=0x55cd4c1eba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.101+0000 7f14e099a700 1 --2- 172.21.15.2:0/1845290842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd4c1e1500 0x55cd4c1eba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.101+0000 7f14e099a700 1 -- 172.21.15.2:0/1845290842 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd4bfae5d0 con 0x55cd4c1e0be0 2023-09-13T04:48:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.102+0000 7f14e099a700 1 --2- 172.21.15.2:0/1845290842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd4c1e0be0 0x55cd4c1eb500 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0164d0 tx=0x7f14cc027210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cac088f521d4da3 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.102+0000 7f14c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:48:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.102+0000 7f14c4ff9700 1 -- 172.21.15.2:0/1845290842 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc027300 con 0x55cd4c1e0be0 2023-09-13T04:48:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.102+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/1845290842 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14cc00c030 con 0x55cd4c1e0be0 2023-09-13T04:48:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.102+0000 7f14c4ff9700 1 -- 172.21.15.2:0/1845290842 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f14cc027460 con 0x55cd4c1e0be0 2023-09-13T04:48:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.102+0000 7f14c4ff9700 1 -- 172.21.15.2:0/1845290842 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14cc027ad0 con 0x55cd4c1e0be0 2023-09-13T04:48:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.102+0000 7f14f6ca4b80 10 client.26603 fetch_fsmap learned FSMap version 36 2023-09-13T04:48:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.102+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/1845290842 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55cd4bf43c00 con 0x55cd4c1e0be0 2023-09-13T04:48:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.103+0000 7f14c4ff9700 1 -- 172.21.15.2:0/1845290842 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14cc00c030 con 0x55cd4c1e0be0 2023-09-13T04:48:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.103+0000 7f14c4ff9700 1 -- 172.21.15.2:0/1845290842 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f14cc018cd0 con 0x55cd4c1e0be0 2023-09-13T04:48:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.103+0000 7f14f6ca4b80 10 client.26603 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:48:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.103+0000 7f14f6ca4b80 10 client.26603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:48:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.103+0000 7f14f6ca4b80 20 client.26603 populate_metadata read hostname 'smithi002' 2023-09-13T04:48:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.103+0000 7f14f6ca4b80 1 --2- 172.21.15.2:0/1845290842 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55cd4c24a570 0x55cd4c24c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.103+0000 7f14f6ca4b80 4 client.26603 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:48:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.103+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/1845290842 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cd4c0e7ff0 con 0x55cd4c24a570 2023-09-13T04:48:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.103+0000 7f14e119b700 1 --2- 172.21.15.2:0/1845290842 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55cd4c24a570 0x55cd4c24c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.104+0000 7f14e119b700 1 --2- 172.21.15.2:0/1845290842 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55cd4c24a570 0x55cd4c24c990 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bc3f1c18b5bada1 server_cookie=5afbb42799f3ef7d in_seq=0 out_seq=0 2023-09-13T04:48:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.104+0000 7f14c4ff9700 10 client.26603 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:22.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:21 smithi149 ceph-mon[133315]: pgmap v244: 129 pgs: 129 active+clean; 966 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 365 op/s 2023-09-13T04:48:22.190 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:21 smithi156 ceph-mon[133675]: pgmap v244: 129 pgs: 129 active+clean; 966 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 365 op/s 2023-09-13T04:48:22.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:21 smithi002 ceph-mon[97651]: pgmap v244: 129 pgs: 129 active+clean; 966 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 365 op/s 2023-09-13T04:48:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.360+0000 7f14c4ff9700 1 -- 172.21.15.2:0/1845290842 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cd4c0e7ff0 con 0x55cd4c24a570 2023-09-13T04:48:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.360+0000 7f14c4ff9700 10 client.26603 handle_command_reply: tid=0 2023-09-13T04:48:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.360+0000 7f14f6ca4b80 1 client.26603 shutdown 2023-09-13T04:48:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.361+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/1845290842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd4c1e0be0 msgr2=0x55cd4c1eb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.361+0000 7f14f6ca4b80 1 --2- 172.21.15.2:0/1845290842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd4c1e0be0 0x55cd4c1eb500 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0164d0 tx=0x7f14cc027210 comp rx=0 tx=0).stop 2023-09-13T04:48:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.361+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/1845290842 shutdown_connections 2023-09-13T04:48:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.361+0000 7f14f6ca4b80 1 --2- 172.21.15.2:0/1845290842 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55cd4c24a570 0x55cd4c24c990 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.361+0000 7f14f6ca4b80 1 --2- 172.21.15.2:0/1845290842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd4c1e1500 0x55cd4c1eba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.361+0000 7f14f6ca4b80 1 --2- 172.21.15.2:0/1845290842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd4c1dc840 0x55cd4c1eafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.361+0000 7f14f6ca4b80 1 --2- 172.21.15.2:0/1845290842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd4c1e0be0 0x55cd4c1eb500 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.361+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/1845290842 >> 172.21.15.2:0/1845290842 conn(0x55cd4c13ac20 msgr2=0x55cd4c1df740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.362+0000 7f14c4ff9700 0 client.26603 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.362+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/1845290842 shutdown_connections 2023-09-13T04:48:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.362+0000 7f14f6ca4b80 1 -- 172.21.15.2:0/1845290842 wait complete. 2023-09-13T04:48:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.362+0000 7f14f6ca4b80 20 client.26603 trim_cache size 0 max 16384 2023-09-13T04:48:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:48:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:48:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:48:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:48:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.363+0000 7f14e33ff700 1 -- 172.21.15.2:0/2761184031 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f14c80a0800 msgr2=0x7f14c80a2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.363+0000 7f14e33ff700 1 --2- 172.21.15.2:0/2761184031 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f14c80a0800 0x7f14c80a2cc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f14d8000c00 tx=0x7f14d8005fd0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.363+0000 7f14e33ff700 1 -- 172.21.15.2:0/2761184031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14dc111970 msgr2=0x7f14dc06c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.363+0000 7f14e33ff700 1 --2- 172.21.15.2:0/2761184031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14dc111970 0x7f14dc06c1e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f14d400ef90 tx=0x7f14d400c3a0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.363+0000 7f14e33ff700 1 -- 172.21.15.2:0/2761184031 shutdown_connections 2023-09-13T04:48:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.363+0000 7f14e33ff700 1 --2- 172.21.15.2:0/2761184031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14dc111970 0x7f14dc06c1e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.363+0000 7f14e33ff700 1 --2- 172.21.15.2:0/2761184031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14dc0ff570 0x7f14dc072320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.363+0000 7f14e33ff700 1 --2- 172.21.15.2:0/2761184031 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f14c80a0800 0x7f14c80a2cc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.363+0000 7f14e33ff700 1 --2- 172.21.15.2:0/2761184031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14dc0fff20 0x7f14dc072860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.363+0000 7f14e33ff700 1 -- 172.21.15.2:0/2761184031 >> 172.21.15.2:0/2761184031 conn(0x7f14dc074e60 msgr2=0x7f14dc10f420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.363+0000 7f14e33ff700 1 -- 172.21.15.2:0/2761184031 shutdown_connections 2023-09-13T04:48:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:22.363+0000 7f14e33ff700 1 -- 172.21.15.2:0/2761184031 wait complete. 2023-09-13T04:48:24.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:23 smithi156 ceph-mon[133675]: pgmap v245: 129 pgs: 129 active+clean; 966 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 331 op/s 2023-09-13T04:48:24.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:23 smithi002 ceph-mon[97651]: pgmap v245: 129 pgs: 129 active+clean; 966 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 331 op/s 2023-09-13T04:48:24.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:23 smithi149 ceph-mon[133315]: pgmap v245: 129 pgs: 129 active+clean; 966 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 331 op/s 2023-09-13T04:48:24.888 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T04:48:25.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.100+0000 7f48ed5ff700 1 -- 172.21.15.2:0/1259144102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48e8073970 msgr2=0x7f48e8111710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:25.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.100+0000 7f48ed5ff700 1 --2- 172.21.15.2:0/1259144102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48e8073970 0x7f48e8111710 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f48dc009990 tx=0x7f48dc009ca0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.100+0000 7f48ed5ff700 1 -- 172.21.15.2:0/1259144102 shutdown_connections 2023-09-13T04:48:25.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.100+0000 7f48ed5ff700 1 --2- 172.21.15.2:0/1259144102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48e8111c50 0x7f48e8114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.100+0000 7f48ed5ff700 1 --2- 172.21.15.2:0/1259144102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e8073050 0x7f48e8073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.100+0000 7f48ed5ff700 1 --2- 172.21.15.2:0/1259144102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48e8073970 0x7f48e8111710 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.100+0000 7f48ed5ff700 1 -- 172.21.15.2:0/1259144102 >> 172.21.15.2:0/1259144102 conn(0x7f48e80fc730 msgr2=0x7f48e80feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:25.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.100+0000 7f48ed5ff700 1 -- 172.21.15.2:0/1259144102 shutdown_connections 2023-09-13T04:48:25.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.100+0000 7f48ed5ff700 1 -- 172.21.15.2:0/1259144102 wait complete. 2023-09-13T04:48:25.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.101+0000 7f48ed5ff700 1 Processor -- start 2023-09-13T04:48:25.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.101+0000 7f48ed5ff700 1 -- start start 2023-09-13T04:48:25.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.102+0000 7f48ed5ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48e8073050 0x7f48e819d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:25.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.102+0000 7f48ed5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e8073970 0x7f48e819d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:25.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.102+0000 7f48ed5ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48e8111c50 0x7f48e81a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:25.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.102+0000 7f48ed5ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e81a23a0 con 0x7f48e8073050 2023-09-13T04:48:25.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.102+0000 7f48ed5ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e81a2510 con 0x7f48e8111c50 2023-09-13T04:48:25.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.102+0000 7f48ed5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e81a26d0 con 0x7f48e8073970 2023-09-13T04:48:25.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.102+0000 7f48e6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48e8073050 0x7f48e819d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:25.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.102+0000 7f48e67fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e8073970 0x7f48e819d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:25.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.102+0000 7f48e67fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e8073970 0x7f48e819d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:41302/0 (socket says 172.21.15.2:41302) 2023-09-13T04:48:25.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.102+0000 7f48e6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48e8073050 0x7f48e819d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34260/0 (socket says 172.21.15.2:34260) 2023-09-13T04:48:25.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.102+0000 7f48e67fc700 1 -- 172.21.15.2:0/651141255 learned_addr learned my addr 172.21.15.2:0/651141255 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:25.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.102+0000 7f48e77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48e8111c50 0x7f48e81a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:25.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.102+0000 7f48e67fc700 1 -- 172.21.15.2:0/651141255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48e8111c50 msgr2=0x7f48e81a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:25.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.102+0000 7f48e67fc700 1 --2- 172.21.15.2:0/651141255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48e8111c50 0x7f48e81a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.102+0000 7f48e67fc700 1 -- 172.21.15.2:0/651141255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48e8073050 msgr2=0x7f48e819d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:25.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.102+0000 7f48e67fc700 1 --2- 172.21.15.2:0/651141255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48e8073050 0x7f48e819d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.102+0000 7f48e67fc700 1 -- 172.21.15.2:0/651141255 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d8009520 con 0x7f48e8073970 2023-09-13T04:48:25.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.102+0000 7f48e6ffd700 1 --2- 172.21.15.2:0/651141255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48e8073050 0x7f48e819d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:48:25.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.103+0000 7f48e67fc700 1 --2- 172.21.15.2:0/651141255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e8073970 0x7f48e819d900 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f48dc000c00 tx=0x7f48dc005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:25.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.103+0000 7f48cffff700 1 -- 172.21.15.2:0/651141255 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc01d070 con 0x7f48e8073970 2023-09-13T04:48:25.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.103+0000 7f48cffff700 1 -- 172.21.15.2:0/651141255 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48dc022470 con 0x7f48e8073970 2023-09-13T04:48:25.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.103+0000 7f48ed5ff700 1 -- 172.21.15.2:0/651141255 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e0009c40 con 0x7f48e8073970 2023-09-13T04:48:25.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.103+0000 7f48cffff700 1 -- 172.21.15.2:0/651141255 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc017650 con 0x7f48e8073970 2023-09-13T04:48:25.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.103+0000 7f48ed5ff700 1 -- 172.21.15.2:0/651141255 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48dc0095f0 con 0x7f48e8073970 2023-09-13T04:48:25.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.106+0000 7f48cffff700 1 -- 172.21.15.2:0/651141255 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f48dc0225e0 con 0x7f48e8073970 2023-09-13T04:48:25.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.108+0000 7f4900fefb80 1 Processor -- start 2023-09-13T04:48:25.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.108+0000 7f4900fefb80 1 -- start start 2023-09-13T04:48:25.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.108+0000 7f48cffff700 1 --2- 172.21.15.2:0/651141255 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f48d00a07b0 0x7f48d00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:25.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.108+0000 7f48cffff700 1 -- 172.21.15.2:0/651141255 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f48dc0cd820 con 0x7f48e8073970 2023-09-13T04:48:25.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.108+0000 7f48e77fe700 1 --2- 172.21.15.2:0/651141255 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f48d00a07b0 0x7f48d00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:25.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.108+0000 7f4900fefb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7a6735fe0 0x55f7a67363c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:25.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.108+0000 7f4900fefb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6736900 0x55f7a67304c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:25.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.108+0000 7f4900fefb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7a6730a00 0x55f7a6730e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:25.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.109+0000 7f4900fefb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a6689780 con 0x55f7a6735fe0 2023-09-13T04:48:25.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.109+0000 7f4900fefb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a66870a0 con 0x55f7a6736900 2023-09-13T04:48:25.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.109+0000 7f4900fefb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a6682990 con 0x55f7a6730a00 2023-09-13T04:48:25.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.109+0000 7f48e67fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6736900 0x55f7a67304c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:25.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.109+0000 7f48e6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7a6735fe0 0x55f7a67363c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:25.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.109+0000 7f48e67fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6736900 0x55f7a67304c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:35818/0 (socket says 172.21.15.2:35818) 2023-09-13T04:48:25.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.109+0000 7f48e67fc700 1 -- 172.21.15.2:0/4030125580 learned_addr learned my addr 172.21.15.2:0/4030125580 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:25.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.109+0000 7f48e77fe700 1 --2- 172.21.15.2:0/4030125580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7a6730a00 0x55f7a6730e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:25.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.109+0000 7f48e67fc700 1 -- 172.21.15.2:0/4030125580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7a6730a00 msgr2=0x55f7a6730e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:25.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.109+0000 7f48e67fc700 1 --2- 172.21.15.2:0/4030125580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7a6730a00 0x55f7a6730e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.109+0000 7f48e67fc700 1 -- 172.21.15.2:0/4030125580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7a6735fe0 msgr2=0x55f7a67363c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:25.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.109+0000 7f48e67fc700 1 --2- 172.21.15.2:0/4030125580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7a6735fe0 0x55f7a67363c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.109+0000 7f48e77fe700 1 --2- 172.21.15.2:0/651141255 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f48d00a07b0 0x7f48d00a2c70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f48e0009fd0 tx=0x7f48e0009920 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:25.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.109+0000 7f48e67fc700 1 -- 172.21.15.2:0/4030125580 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7a6502630 con 0x55f7a6736900 2023-09-13T04:48:25.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.109+0000 7f48e6ffd700 1 --2- 172.21.15.2:0/4030125580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7a6735fe0 0x55f7a67363c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:48:25.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.109+0000 7f48e67fc700 1 --2- 172.21.15.2:0/4030125580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6736900 0x55f7a67304c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f48dc0a2560 tx=0x7f48dc09a670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=830a20d30fbbc7f3 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:25.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.110+0000 7f48cd7fa700 1 -- 172.21.15.2:0/4030125580 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc09aab0 con 0x55f7a6736900 2023-09-13T04:48:25.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.110+0000 7f48cd7fa700 1 -- 172.21.15.2:0/4030125580 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48dc09e8e0 con 0x55f7a6736900 2023-09-13T04:48:25.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.110+0000 7f48cd7fa700 1 -- 172.21.15.2:0/4030125580 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc09ad10 con 0x55f7a6736900 2023-09-13T04:48:25.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.110+0000 7f4900fefb80 1 -- 172.21.15.2:0/4030125580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6736900 msgr2=0x55f7a67304c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:25.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.110+0000 7f4900fefb80 1 --2- 172.21.15.2:0/4030125580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6736900 0x55f7a67304c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f48dc0a2560 tx=0x7f48dc09a670 comp rx=0 tx=0).stop 2023-09-13T04:48:25.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.110+0000 7f4900fefb80 1 -- 172.21.15.2:0/4030125580 shutdown_connections 2023-09-13T04:48:25.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.110+0000 7f4900fefb80 1 --2- 172.21.15.2:0/4030125580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7a6730a00 0x55f7a6730e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.110+0000 7f4900fefb80 1 --2- 172.21.15.2:0/4030125580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7a6735fe0 0x55f7a67363c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.110+0000 7f4900fefb80 1 --2- 172.21.15.2:0/4030125580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6736900 0x55f7a67304c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.110+0000 7f4900fefb80 1 -- 172.21.15.2:0/4030125580 >> 172.21.15.2:0/4030125580 conn(0x55f7a668ec80 msgr2=0x55f7a672af20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:25.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.111+0000 7f4900fefb80 1 -- 172.21.15.2:0/4030125580 shutdown_connections 2023-09-13T04:48:25.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.111+0000 7f4900fefb80 1 -- 172.21.15.2:0/4030125580 wait complete. 2023-09-13T04:48:25.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.111+0000 7f4900fefb80 1 -- start start 2023-09-13T04:48:25.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.111+0000 7f4900fefb80 1 Processor -- start 2023-09-13T04:48:25.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.112+0000 7f4900fefb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7a6730a00 0x55f7a673c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:25.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.112+0000 7f4900fefb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6735fe0 0x55f7a6739bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:25.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.112+0000 7f4900fefb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7a6736900 0x55f7a673a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:25.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.112+0000 7f4900fefb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a6620bb0 con 0x55f7a6736900 2023-09-13T04:48:25.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.112+0000 7f4900fefb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a66208b0 con 0x55f7a6735fe0 2023-09-13T04:48:25.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.112+0000 7f4900fefb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a6504920 con 0x55f7a6730a00 2023-09-13T04:48:25.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.112+0000 7f4900fefb80 10 client.0 _ll_register_callbacks cb 0x55f7a661e680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:48:25.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.112+0000 7f48e67fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6735fe0 0x55f7a6739bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:25.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.112+0000 7f48e67fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6735fe0 0x55f7a6739bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:35826/0 (socket says 172.21.15.2:35826) 2023-09-13T04:48:25.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.112+0000 7f48e67fc700 1 -- 172.21.15.2:0/2829315252 learned_addr learned my addr 172.21.15.2:0/2829315252 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:25.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.112+0000 7f48e6ffd700 1 --2- 172.21.15.2:0/2829315252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7a6730a00 0x55f7a673c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:25.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.112+0000 7f48e77fe700 1 --2- 172.21.15.2:0/2829315252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7a6736900 0x55f7a673a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:25.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.112+0000 7f48e67fc700 1 -- 172.21.15.2:0/2829315252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7a6730a00 msgr2=0x55f7a673c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:25.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.112+0000 7f48e67fc700 1 --2- 172.21.15.2:0/2829315252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7a6730a00 0x55f7a673c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.112+0000 7f48e67fc700 1 -- 172.21.15.2:0/2829315252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7a6736900 msgr2=0x55f7a673a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:25.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.112+0000 7f48e67fc700 1 --2- 172.21.15.2:0/2829315252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7a6736900 0x55f7a673a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.113+0000 7f48e67fc700 1 -- 172.21.15.2:0/2829315252 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48dc0095f0 con 0x55f7a6735fe0 2023-09-13T04:48:25.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.113+0000 7f48e67fc700 1 --2- 172.21.15.2:0/2829315252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6735fe0 0x55f7a6739bc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f48dc009ce0 tx=0x7f48dc0cd630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57988c33e1d8837b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:25.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.113+0000 7f48caff5700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:48:25.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.113+0000 7f48caff5700 1 -- 172.21.15.2:0/2829315252 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc0a2a90 con 0x55f7a6735fe0 2023-09-13T04:48:25.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.113+0000 7f4900fefb80 1 -- 172.21.15.2:0/2829315252 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7a6723670 con 0x55f7a6735fe0 2023-09-13T04:48:25.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.113+0000 7f48caff5700 1 -- 172.21.15.2:0/2829315252 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f48dc09aab0 con 0x55f7a6735fe0 2023-09-13T04:48:25.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.113+0000 7f48caff5700 1 -- 172.21.15.2:0/2829315252 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48dc09a1b0 con 0x55f7a6735fe0 2023-09-13T04:48:25.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.113+0000 7f4900fefb80 10 client.26615 fetch_fsmap learned FSMap version 36 2023-09-13T04:48:25.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.113+0000 7f4900fefb80 1 -- 172.21.15.2:0/2829315252 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55f7a669a7a0 con 0x55f7a6735fe0 2023-09-13T04:48:25.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.113+0000 7f48caff5700 1 -- 172.21.15.2:0/2829315252 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f7a6723670 con 0x55f7a6735fe0 2023-09-13T04:48:25.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.114+0000 7f48caff5700 1 -- 172.21.15.2:0/2829315252 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f48dc09bac0 con 0x55f7a6735fe0 2023-09-13T04:48:25.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.114+0000 7f4900fefb80 10 client.26615 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:48:25.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.114+0000 7f4900fefb80 10 client.26615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:48:25.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.114+0000 7f4900fefb80 20 client.26615 populate_metadata read hostname 'smithi002' 2023-09-13T04:48:25.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.114+0000 7f4900fefb80 1 --2- 172.21.15.2:0/2829315252 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f7a6749ae0 0x55f7a674bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:25.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.114+0000 7f4900fefb80 4 client.26615 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:48:25.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.114+0000 7f4900fefb80 1 -- 172.21.15.2:0/2829315252 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7a667b970 con 0x55f7a6749ae0 2023-09-13T04:48:25.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.114+0000 7f48e6ffd700 1 --2- 172.21.15.2:0/2829315252 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f7a6749ae0 0x55f7a674bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:25.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.115+0000 7f48e6ffd700 1 --2- 172.21.15.2:0/2829315252 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f7a6749ae0 0x55f7a674bf00 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c94df9f7179aec3 server_cookie=361e91d449769fbd in_seq=0 out_seq=0 2023-09-13T04:48:25.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.115+0000 7f48caff5700 10 client.26615 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:25.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.115+0000 7f48caff5700 1 -- 172.21.15.2:0/2829315252 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f7a667b970 con 0x55f7a6749ae0 2023-09-13T04:48:25.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.115+0000 7f48caff5700 10 client.26615 handle_command_reply: tid=0 2023-09-13T04:48:25.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.115+0000 7f4900fefb80 1 client.26615 shutdown 2023-09-13T04:48:25.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.116+0000 7f4900fefb80 1 -- 172.21.15.2:0/2829315252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6735fe0 msgr2=0x55f7a6739bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:25.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.116+0000 7f4900fefb80 1 --2- 172.21.15.2:0/2829315252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6735fe0 0x55f7a6739bc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f48dc009ce0 tx=0x7f48dc0cd630 comp rx=0 tx=0).stop 2023-09-13T04:48:25.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.116+0000 7f4900fefb80 1 -- 172.21.15.2:0/2829315252 shutdown_connections 2023-09-13T04:48:25.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.116+0000 7f4900fefb80 1 --2- 172.21.15.2:0/2829315252 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f7a6749ae0 0x55f7a674bf00 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.116+0000 7f4900fefb80 1 --2- 172.21.15.2:0/2829315252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7a6736900 0x55f7a673a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.116+0000 7f4900fefb80 1 --2- 172.21.15.2:0/2829315252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7a6730a00 0x55f7a673c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.116+0000 7f4900fefb80 1 --2- 172.21.15.2:0/2829315252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6735fe0 0x55f7a6739bc0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.116+0000 7f48caff5700 0 client.26615 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:25.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.116+0000 7f4900fefb80 1 -- 172.21.15.2:0/2829315252 >> 172.21.15.2:0/2829315252 conn(0x55f7a668ec80 msgr2=0x55f7a67257f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:25.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.116+0000 7f4900fefb80 1 -- 172.21.15.2:0/2829315252 shutdown_connections 2023-09-13T04:48:25.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.116+0000 7f4900fefb80 1 -- 172.21.15.2:0/2829315252 wait complete. 2023-09-13T04:48:25.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.116+0000 7f4900fefb80 20 client.26615 trim_cache size 0 max 16384 2023-09-13T04:48:25.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.133+0000 7f4900fefb80 1 Processor -- start 2023-09-13T04:48:25.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.133+0000 7f4900fefb80 1 -- start start 2023-09-13T04:48:25.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.134+0000 7f4900fefb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6730a00 0x55f7a67363c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:25.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.134+0000 7f4900fefb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7a6735fe0 0x55f7a6741020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:25.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.134+0000 7f4900fefb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7a6736900 0x55f7a67258f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:25.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.134+0000 7f4900fefb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a672c800 con 0x55f7a6735fe0 2023-09-13T04:48:25.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.134+0000 7f4900fefb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a667b970 con 0x55f7a6730a00 2023-09-13T04:48:25.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.134+0000 7f4900fefb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a6723650 con 0x55f7a6736900 2023-09-13T04:48:25.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.134+0000 7f48e67fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7a6735fe0 0x55f7a6741020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:25.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.134+0000 7f48e67fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7a6735fe0 0x55f7a6741020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34294/0 (socket says 172.21.15.2:34294) 2023-09-13T04:48:25.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.134+0000 7f48e67fc700 1 -- 172.21.15.2:0/2244441882 learned_addr learned my addr 172.21.15.2:0/2244441882 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:25.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.134+0000 7f48e6ffd700 1 --2- 172.21.15.2:0/2244441882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6730a00 0x55f7a67363c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:25.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.134+0000 7f48e77fe700 1 --2- 172.21.15.2:0/2244441882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7a6736900 0x55f7a67258f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:25.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.135+0000 7f48e67fc700 1 -- 172.21.15.2:0/2244441882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7a6736900 msgr2=0x55f7a67258f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:25.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.135+0000 7f48e67fc700 1 --2- 172.21.15.2:0/2244441882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7a6736900 0x55f7a67258f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.135+0000 7f48e67fc700 1 -- 172.21.15.2:0/2244441882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6730a00 msgr2=0x55f7a67363c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:25.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.135+0000 7f48e67fc700 1 --2- 172.21.15.2:0/2244441882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6730a00 0x55f7a67363c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.135+0000 7f48e67fc700 1 -- 172.21.15.2:0/2244441882 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7a64aede0 con 0x55f7a6735fe0 2023-09-13T04:48:25.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.135+0000 7f48e6ffd700 1 --2- 172.21.15.2:0/2244441882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6730a00 0x55f7a67363c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:48:25.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.135+0000 7f48e77fe700 1 --2- 172.21.15.2:0/2244441882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7a6736900 0x55f7a67258f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:48:25.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.135+0000 7f48e67fc700 1 --2- 172.21.15.2:0/2244441882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7a6735fe0 0x55f7a6741020 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f48dc09a3a0 tx=0x7f48dc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0595b62b3c14d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:25.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.135+0000 7f48cdffb700 1 -- 172.21.15.2:0/2244441882 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc099a60 con 0x55f7a6735fe0 2023-09-13T04:48:25.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.135+0000 7f48cdffb700 1 -- 172.21.15.2:0/2244441882 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48dc09aab0 con 0x55f7a6735fe0 2023-09-13T04:48:25.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.135+0000 7f48cdffb700 1 -- 172.21.15.2:0/2244441882 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc0a2600 con 0x55f7a6735fe0 2023-09-13T04:48:25.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.136+0000 7f4900fefb80 1 -- 172.21.15.2:0/2244441882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7a6735fe0 msgr2=0x55f7a6741020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:25.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.136+0000 7f4900fefb80 1 --2- 172.21.15.2:0/2244441882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7a6735fe0 0x55f7a6741020 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f48dc09a3a0 tx=0x7f48dc009310 comp rx=0 tx=0).stop 2023-09-13T04:48:25.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.136+0000 7f4900fefb80 1 -- 172.21.15.2:0/2244441882 shutdown_connections 2023-09-13T04:48:25.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.136+0000 7f4900fefb80 1 --2- 172.21.15.2:0/2244441882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7a6736900 0x55f7a67258f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.136+0000 7f4900fefb80 1 --2- 172.21.15.2:0/2244441882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6730a00 0x55f7a67363c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.136+0000 7f4900fefb80 1 --2- 172.21.15.2:0/2244441882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7a6735fe0 0x55f7a6741020 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.136+0000 7f4900fefb80 1 -- 172.21.15.2:0/2244441882 >> 172.21.15.2:0/2244441882 conn(0x55f7a668ec80 msgr2=0x55f7a6733d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:25.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.136+0000 7f4900fefb80 1 -- 172.21.15.2:0/2244441882 shutdown_connections 2023-09-13T04:48:25.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.136+0000 7f4900fefb80 1 -- 172.21.15.2:0/2244441882 wait complete. 2023-09-13T04:48:25.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.137+0000 7f4900fefb80 1 -- start start 2023-09-13T04:48:25.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.137+0000 7f4900fefb80 1 Processor -- start 2023-09-13T04:48:25.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.138+0000 7f4900fefb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7a6730a00 0x55f7a6731a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:25.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.138+0000 7f4900fefb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6735fe0 0x55f7a6731f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:25.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.138+0000 7f4900fefb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7a6736900 0x55f7a6732480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:25.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.138+0000 7f4900fefb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a6741600 con 0x55f7a6736900 2023-09-13T04:48:25.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.138+0000 7f4900fefb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a67329f0 con 0x55f7a6735fe0 2023-09-13T04:48:25.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.138+0000 7f4900fefb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a6732bb0 con 0x55f7a6730a00 2023-09-13T04:48:25.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.138+0000 7f4900fefb80 10 client.0 _ll_register_callbacks cb 0x55f7a661e680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:48:25.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.138+0000 7f48e6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7a6730a00 0x55f7a6731a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:25.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.138+0000 7f48e67fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6735fe0 0x55f7a6731f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:25.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.138+0000 7f48e6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7a6730a00 0x55f7a6731a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:41334/0 (socket says 172.21.15.2:41334) 2023-09-13T04:48:25.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.138+0000 7f48e6ffd700 1 -- 172.21.15.2:0/2426271654 learned_addr learned my addr 172.21.15.2:0/2426271654 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:25.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.138+0000 7f48e67fc700 1 --2- 172.21.15.2:0/2426271654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6735fe0 0x55f7a6731f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:35842/0 (socket says 172.21.15.2:35842) 2023-09-13T04:48:25.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.138+0000 7f48e77fe700 1 --2- 172.21.15.2:0/2426271654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7a6736900 0x55f7a6732480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:25.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.139+0000 7f48e6ffd700 1 -- 172.21.15.2:0/2426271654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6735fe0 msgr2=0x55f7a6731f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:25.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.139+0000 7f48e6ffd700 1 --2- 172.21.15.2:0/2426271654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6735fe0 0x55f7a6731f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.139+0000 7f48e6ffd700 1 -- 172.21.15.2:0/2426271654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7a6736900 msgr2=0x55f7a6732480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:25.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.139+0000 7f48e6ffd700 1 --2- 172.21.15.2:0/2426271654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7a6736900 0x55f7a6732480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.139+0000 7f48e6ffd700 1 -- 172.21.15.2:0/2426271654 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d8021a00 con 0x55f7a6730a00 2023-09-13T04:48:25.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.139+0000 7f48e67fc700 1 --2- 172.21.15.2:0/2426271654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6735fe0 0x55f7a6731f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:48:25.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.139+0000 7f48e6ffd700 1 --2- 172.21.15.2:0/2426271654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7a6730a00 0x55f7a6731a00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f48d8004790 tx=0x7f48d8012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=441bf759ffa9da7a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:25.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.139+0000 7f48caff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:48:25.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.139+0000 7f48caff5700 1 -- 172.21.15.2:0/2426271654 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d8017040 con 0x55f7a6730a00 2023-09-13T04:48:25.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.140+0000 7f48caff5700 1 -- 172.21.15.2:0/2426271654 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f48d80212b0 con 0x55f7a6730a00 2023-09-13T04:48:25.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.140+0000 7f4900fefb80 1 -- 172.21.15.2:0/2426271654 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48dc09e8e0 con 0x55f7a6730a00 2023-09-13T04:48:25.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.140+0000 7f48caff5700 1 -- 172.21.15.2:0/2426271654 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48d8021470 con 0x55f7a6730a00 2023-09-13T04:48:25.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.140+0000 7f4900fefb80 10 client.26635 fetch_fsmap learned FSMap version 36 2023-09-13T04:48:25.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.140+0000 7f4900fefb80 1 -- 172.21.15.2:0/2426271654 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55f7a669a7a0 con 0x55f7a6730a00 2023-09-13T04:48:25.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.140+0000 7f48caff5700 1 -- 172.21.15.2:0/2426271654 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48dc09e8e0 con 0x55f7a6730a00 2023-09-13T04:48:25.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.141+0000 7f48caff5700 1 -- 172.21.15.2:0/2426271654 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f48d8005380 con 0x55f7a6730a00 2023-09-13T04:48:25.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.141+0000 7f4900fefb80 10 client.26635 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:48:25.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.141+0000 7f4900fefb80 10 client.26635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:48:25.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.141+0000 7f4900fefb80 20 client.26635 populate_metadata read hostname 'smithi002' 2023-09-13T04:48:25.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.141+0000 7f4900fefb80 1 --2- 172.21.15.2:0/2426271654 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f7a679e2b0 0x55f7a67a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:25.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.141+0000 7f4900fefb80 4 client.26635 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-09-13T04:48:25.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.141+0000 7f4900fefb80 1 -- 172.21.15.2:0/2426271654 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f7a663c050 con 0x55f7a679e2b0 2023-09-13T04:48:25.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.141+0000 7f48e67fc700 1 --2- 172.21.15.2:0/2426271654 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f7a679e2b0 0x55f7a67a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:25.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.142+0000 7f48e67fc700 1 --2- 172.21.15.2:0/2426271654 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f7a679e2b0 0x55f7a67a06d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a2ae34702b57134 server_cookie=e268261c3d9f1277 in_seq=0 out_seq=0 2023-09-13T04:48:25.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.142+0000 7f48caff5700 10 client.26635 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:25.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.142+0000 7f48caff5700 1 -- 172.21.15.2:0/2426271654 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f7a663c050 con 0x55f7a679e2b0 2023-09-13T04:48:25.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.142+0000 7f48caff5700 10 client.26635 handle_command_reply: tid=0 2023-09-13T04:48:25.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.142+0000 7f4900fefb80 1 client.26635 shutdown 2023-09-13T04:48:25.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.142+0000 7f4900fefb80 1 -- 172.21.15.2:0/2426271654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7a6730a00 msgr2=0x55f7a6731a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:25.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.142+0000 7f4900fefb80 1 --2- 172.21.15.2:0/2426271654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7a6730a00 0x55f7a6731a00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f48d8004790 tx=0x7f48d8012b60 comp rx=0 tx=0).stop 2023-09-13T04:48:25.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.142+0000 7f4900fefb80 1 -- 172.21.15.2:0/2426271654 shutdown_connections 2023-09-13T04:48:25.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.142+0000 7f4900fefb80 1 --2- 172.21.15.2:0/2426271654 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f7a679e2b0 0x55f7a67a06d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.142+0000 7f4900fefb80 1 --2- 172.21.15.2:0/2426271654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7a6736900 0x55f7a6732480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.142+0000 7f4900fefb80 1 --2- 172.21.15.2:0/2426271654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7a6730a00 0x55f7a6731a00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.142+0000 7f48caff5700 0 client.26635 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:25.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.142+0000 7f4900fefb80 1 --2- 172.21.15.2:0/2426271654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7a6735fe0 0x55f7a6731f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.142+0000 7f4900fefb80 1 -- 172.21.15.2:0/2426271654 >> 172.21.15.2:0/2426271654 conn(0x55f7a668ec80 msgr2=0x55f7a6734c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:25.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.143+0000 7f4900fefb80 1 -- 172.21.15.2:0/2426271654 shutdown_connections 2023-09-13T04:48:25.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.143+0000 7f4900fefb80 1 -- 172.21.15.2:0/2426271654 wait complete. 2023-09-13T04:48:25.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.143+0000 7f4900fefb80 20 client.26635 trim_cache size 0 max 16384 2023-09-13T04:48:25.164 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:48:25.165 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-13T04:48:25.165 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-09-13T04:48:25.165 INFO:teuthology.orchestra.run.smithi002.stdout: "740235c0-6e1b-4729-b1fe-33345ae948c1": { 2023-09-13T04:48:25.165 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-09-13T04:48:25.166 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "740235c0-6e1b-4729-b1fe-33345ae948c1", 2023-09-13T04:48:25.166 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-09-13T04:48:25.166 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:48:25.166 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:48:25.167 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:48:25.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.143+0000 7f48ed5ff700 1 -- 172.21.15.2:0/651141255 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f48d00a07b0 msgr2=0x7f48d00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:25.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.143+0000 7f48ed5ff700 1 --2- 172.21.15.2:0/651141255 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f48d00a07b0 0x7f48d00a2c70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f48e0009fd0 tx=0x7f48e0009920 comp rx=0 tx=0).stop 2023-09-13T04:48:25.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.144+0000 7f48ed5ff700 1 -- 172.21.15.2:0/651141255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e8073970 msgr2=0x7f48e819d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:25.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.144+0000 7f48ed5ff700 1 --2- 172.21.15.2:0/651141255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e8073970 0x7f48e819d900 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f48dc000c00 tx=0x7f48dc005740 comp rx=0 tx=0).stop 2023-09-13T04:48:25.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.144+0000 7f48ed5ff700 1 -- 172.21.15.2:0/651141255 shutdown_connections 2023-09-13T04:48:25.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.144+0000 7f48ed5ff700 1 --2- 172.21.15.2:0/651141255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48e8111c50 0x7f48e81a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.144+0000 7f48ed5ff700 1 --2- 172.21.15.2:0/651141255 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f48d00a07b0 0x7f48d00a2c70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.144+0000 7f48ed5ff700 1 --2- 172.21.15.2:0/651141255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48e8073050 0x7f48e819d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.144+0000 7f48ed5ff700 1 --2- 172.21.15.2:0/651141255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e8073970 0x7f48e819d900 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:25.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.144+0000 7f48ed5ff700 1 -- 172.21.15.2:0/651141255 >> 172.21.15.2:0/651141255 conn(0x7f48e80fc730 msgr2=0x7f48e80fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:25.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.144+0000 7f48ed5ff700 1 -- 172.21.15.2:0/651141255 shutdown_connections 2023-09-13T04:48:25.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:25.144+0000 7f48ed5ff700 1 -- 172.21.15.2:0/651141255 wait complete. 2023-09-13T04:48:25.170 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:24 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:48:25.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:24 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:25.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:24 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:48:25.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:24 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:25.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:24 smithi002 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-13T04:48:25.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:24 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:25.172 INFO:tasks.cephfs.filesystem:scrub status for tag:740235c0-6e1b-4729-b1fe-33345ae948c1 - {'path': '/', 'tag': '740235c0-6e1b-4729-b1fe-33345ae948c1', 'options': 'recursive,force'} 2023-09-13T04:48:25.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:24 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:48:25.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:24 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:25.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:24 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:48:25.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:24 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:25.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:24 smithi156 ceph-mon[133675]: scrub summary: active paths [/] 2023-09-13T04:48:25.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:24 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:25.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:24 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:48:25.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:24 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:25.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:24 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:48:25.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:24 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:25.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:24 smithi149 ceph-mon[133315]: scrub summary: active paths [/] 2023-09-13T04:48:25.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:24 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:26.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:25 smithi156 ceph-mon[133675]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:26.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:25 smithi156 ceph-mon[133675]: pgmap v246: 129 pgs: 129 active+clean; 962 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 556 op/s 2023-09-13T04:48:26.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:25 smithi002 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:26.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:25 smithi002 ceph-mon[97651]: pgmap v246: 129 pgs: 129 active+clean; 962 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 556 op/s 2023-09-13T04:48:26.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:25 smithi149 ceph-mon[133315]: scrub summary: idle+waiting paths [/] 2023-09-13T04:48:26.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:25 smithi149 ceph-mon[133315]: pgmap v246: 129 pgs: 129 active+clean; 962 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 556 op/s 2023-09-13T04:48:28.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:27 smithi156 ceph-mon[133675]: pgmap v247: 129 pgs: 129 active+clean; 962 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 517 op/s 2023-09-13T04:48:28.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:27 smithi002 ceph-mon[97651]: pgmap v247: 129 pgs: 129 active+clean; 962 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 517 op/s 2023-09-13T04:48:28.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:27 smithi149 ceph-mon[133315]: pgmap v247: 129 pgs: 129 active+clean; 962 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 517 op/s 2023-09-13T04:48:30.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:29 smithi156 ceph-mon[133675]: pgmap v248: 129 pgs: 129 active+clean; 964 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 541 op/s 2023-09-13T04:48:30.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:29 smithi002 ceph-mon[97651]: pgmap v248: 129 pgs: 129 active+clean; 964 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 541 op/s 2023-09-13T04:48:30.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:29 smithi149 ceph-mon[133315]: pgmap v248: 129 pgs: 129 active+clean; 964 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 541 op/s 2023-09-13T04:48:32.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:31 smithi156 ceph-mon[133675]: pgmap v249: 129 pgs: 129 active+clean; 969 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 708 op/s 2023-09-13T04:48:32.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:31 smithi002 ceph-mon[97651]: pgmap v249: 129 pgs: 129 active+clean; 969 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 708 op/s 2023-09-13T04:48:32.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:31 smithi149 ceph-mon[133315]: pgmap v249: 129 pgs: 129 active+clean; 969 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 708 op/s 2023-09-13T04:48:34.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:33 smithi156 ceph-mon[133675]: pgmap v250: 129 pgs: 129 active+clean; 973 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 636 op/s 2023-09-13T04:48:34.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:33 smithi002 ceph-mon[97651]: pgmap v250: 129 pgs: 129 active+clean; 973 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 636 op/s 2023-09-13T04:48:34.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:33 smithi149 ceph-mon[133315]: pgmap v250: 129 pgs: 129 active+clean; 973 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 636 op/s 2023-09-13T04:48:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.560+0000 7fe2103ff700 1 -- 172.21.15.2:0/971611492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe208103100 msgr2=0x7fe2081034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.560+0000 7fe2103ff700 1 --2- 172.21.15.2:0/971611492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe208103100 0x7fe2081034e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe204009990 tx=0x7fe204009ca0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.560+0000 7fe2103ff700 1 -- 172.21.15.2:0/971611492 shutdown_connections 2023-09-13T04:48:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.560+0000 7fe2103ff700 1 --2- 172.21.15.2:0/971611492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe208108220 0x7fe20810fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.560+0000 7fe2103ff700 1 --2- 172.21.15.2:0/971611492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe208103100 0x7fe2081034e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.560+0000 7fe2103ff700 1 --2- 172.21.15.2:0/971611492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe208103ab0 0x7fe208107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.560+0000 7fe2103ff700 1 -- 172.21.15.2:0/971611492 >> 172.21.15.2:0/971611492 conn(0x7fe2080fe960 msgr2=0x7fe208100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.561+0000 7fe2103ff700 1 -- 172.21.15.2:0/971611492 shutdown_connections 2023-09-13T04:48:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.561+0000 7fe2103ff700 1 -- 172.21.15.2:0/971611492 wait complete. 2023-09-13T04:48:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.561+0000 7fe2103ff700 1 Processor -- start 2023-09-13T04:48:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.562+0000 7fe2103ff700 1 -- start start 2023-09-13T04:48:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.562+0000 7fe2103ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe208103100 0x7fe208198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.562+0000 7fe2103ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe208108220 0x7fe2081994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.562+0000 7fe2103ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe20819db20 0x7fe20819dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.562+0000 7fe2103ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20819e5a0 con 0x7fe20819db20 2023-09-13T04:48:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.562+0000 7fe2103ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20819e710 con 0x7fe208108220 2023-09-13T04:48:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.562+0000 7fe2103ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20819e8d0 con 0x7fe208103100 2023-09-13T04:48:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.562+0000 7fe20d99a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe208108220 0x7fe2081994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.562+0000 7fe20d99a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe208108220 0x7fe2081994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:51506/0 (socket says 172.21.15.2:51506) 2023-09-13T04:48:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.562+0000 7fe20d99a700 1 -- 172.21.15.2:0/3614280135 learned_addr learned my addr 172.21.15.2:0/3614280135 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.562+0000 7fe20e99c700 1 --2- 172.21.15.2:0/3614280135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe20819db20 0x7fe20819dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.562+0000 7fe20e19b700 1 --2- 172.21.15.2:0/3614280135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe208103100 0x7fe208198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.563+0000 7fe20d99a700 1 -- 172.21.15.2:0/3614280135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe208103100 msgr2=0x7fe208198f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.563+0000 7fe20d99a700 1 --2- 172.21.15.2:0/3614280135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe208103100 0x7fe208198f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.563+0000 7fe20d99a700 1 -- 172.21.15.2:0/3614280135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe20819db20 msgr2=0x7fe20819dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.563+0000 7fe20d99a700 1 --2- 172.21.15.2:0/3614280135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe20819db20 0x7fe20819dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.563+0000 7fe20d99a700 1 -- 172.21.15.2:0/3614280135 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe200009c40 con 0x7fe208108220 2023-09-13T04:48:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.563+0000 7fe20e19b700 1 --2- 172.21.15.2:0/3614280135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe208103100 0x7fe208198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:48:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.563+0000 7fe20d99a700 1 --2- 172.21.15.2:0/3614280135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe208108220 0x7fe2081994b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800e970 tx=0x7fe1f800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.563+0000 7fe1ff7fe700 1 -- 172.21.15.2:0/3614280135 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8009790 con 0x7fe208108220 2023-09-13T04:48:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.563+0000 7fe1ff7fe700 1 -- 172.21.15.2:0/3614280135 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1f800eeb0 con 0x7fe208108220 2023-09-13T04:48:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.563+0000 7fe2103ff700 1 -- 172.21.15.2:0/3614280135 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2040095f0 con 0x7fe208108220 2023-09-13T04:48:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.563+0000 7fe1ff7fe700 1 -- 172.21.15.2:0/3614280135 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f80055c0 con 0x7fe208108220 2023-09-13T04:48:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.563+0000 7fe2103ff700 1 -- 172.21.15.2:0/3614280135 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2081a6180 con 0x7fe208108220 2023-09-13T04:48:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.567+0000 7fe1ff7fe700 1 -- 172.21.15.2:0/3614280135 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7fe1f8004750 con 0x7fe208108220 2023-09-13T04:48:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.568+0000 7fe223c62b80 1 Processor -- start 2023-09-13T04:48:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.568+0000 7fe223c62b80 1 -- start start 2023-09-13T04:48:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.568+0000 7fe223c62b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d9f48ebe0 0x557d9f48efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.568+0000 7fe223c62b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d9f48f500 0x557d9f48a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.568+0000 7fe223c62b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d9f48a840 0x557d9f48ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.568+0000 7fe223c62b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9f3e3720 con 0x557d9f48a840 2023-09-13T04:48:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.568+0000 7fe223c62b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9f3e1040 con 0x557d9f48ebe0 2023-09-13T04:48:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.568+0000 7fe223c62b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9f3dc930 con 0x557d9f48f500 2023-09-13T04:48:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.568+0000 7fe1ff7fe700 1 --2- 172.21.15.2:0/3614280135 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe1f40a07b0 0x7fe1f40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.568+0000 7fe20e99c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d9f48ebe0 0x557d9f48efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.568+0000 7fe20e99c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d9f48ebe0 0x557d9f48efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:51520/0 (socket says 172.21.15.2:51520) 2023-09-13T04:48:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.568+0000 7fe20e99c700 1 -- 172.21.15.2:0/461988147 learned_addr learned my addr 172.21.15.2:0/461988147 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.569+0000 7fe20e19b700 1 --2- 172.21.15.2:0/461988147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d9f48f500 0x557d9f48a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.569+0000 7fe1ff7fe700 1 -- 172.21.15.2:0/3614280135 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7fe1f8014070 con 0x7fe208108220 2023-09-13T04:48:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.569+0000 7fe20d99a700 1 --2- 172.21.15.2:0/461988147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d9f48a840 0x557d9f48ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.569+0000 7fe20e99c700 1 -- 172.21.15.2:0/461988147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d9f48f500 msgr2=0x557d9f48a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.569+0000 7fe20e99c700 1 --2- 172.21.15.2:0/461988147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d9f48f500 0x557d9f48a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.569+0000 7fe20e99c700 1 -- 172.21.15.2:0/461988147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d9f48a840 msgr2=0x557d9f48ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.569+0000 7fe20e99c700 1 --2- 172.21.15.2:0/461988147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d9f48a840 0x557d9f48ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.569+0000 7fe20e99c700 1 -- 172.21.15.2:0/461988147 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d9f25c5d0 con 0x557d9f48ebe0 2023-09-13T04:48:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.569+0000 7fe20e99c700 1 --2- 172.21.15.2:0/3614280135 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe1f40a07b0 0x7fe1f40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.569+0000 7fe20e99c700 1 --2- 172.21.15.2:0/461988147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d9f48ebe0 0x557d9f48efc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe200009b50 tx=0x7fe20000ead0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23ba1f8191ed07e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.570+0000 7fe1fcbfe700 1 -- 172.21.15.2:0/461988147 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20000e040 con 0x557d9f48ebe0 2023-09-13T04:48:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.570+0000 7fe1fcbfe700 1 -- 172.21.15.2:0/461988147 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe20000f040 con 0x557d9f48ebe0 2023-09-13T04:48:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.570+0000 7fe1fcbfe700 1 -- 172.21.15.2:0/461988147 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe200007b60 con 0x557d9f48ebe0 2023-09-13T04:48:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.570+0000 7fe223c62b80 1 -- 172.21.15.2:0/461988147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d9f48ebe0 msgr2=0x557d9f48efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.570+0000 7fe20e99c700 1 --2- 172.21.15.2:0/3614280135 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe1f40a07b0 0x7fe1f40a2c70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe200009550 tx=0x7fe200017750 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.570+0000 7fe223c62b80 1 --2- 172.21.15.2:0/461988147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d9f48ebe0 0x557d9f48efc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe200009b50 tx=0x7fe20000ead0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.570+0000 7fe223c62b80 1 -- 172.21.15.2:0/461988147 shutdown_connections 2023-09-13T04:48:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.570+0000 7fe223c62b80 1 --2- 172.21.15.2:0/461988147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d9f48a840 0x557d9f48ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.570+0000 7fe223c62b80 1 --2- 172.21.15.2:0/461988147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d9f48ebe0 0x557d9f48efc0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.570+0000 7fe223c62b80 1 --2- 172.21.15.2:0/461988147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d9f48f500 0x557d9f48a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.570+0000 7fe223c62b80 1 -- 172.21.15.2:0/461988147 >> 172.21.15.2:0/461988147 conn(0x557d9f3e8c20 msgr2=0x557d9f484b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.570+0000 7fe223c62b80 1 -- 172.21.15.2:0/461988147 shutdown_connections 2023-09-13T04:48:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.571+0000 7fe223c62b80 1 -- 172.21.15.2:0/461988147 wait complete. 2023-09-13T04:48:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.571+0000 7fe223c62b80 1 -- start start 2023-09-13T04:48:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.571+0000 7fe223c62b80 1 Processor -- start 2023-09-13T04:48:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.572+0000 7fe223c62b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d9f48a840 0x557d9f4965c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.572+0000 7fe223c62b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d9f48ebe0 0x557d9f4937d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.572+0000 7fe223c62b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d9f48f500 0x557d9f493f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.572+0000 7fe223c62b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9f37ab50 con 0x557d9f48a840 2023-09-13T04:48:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.572+0000 7fe223c62b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9f37a850 con 0x557d9f48f500 2023-09-13T04:48:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.572+0000 7fe223c62b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9f25e8c0 con 0x557d9f48ebe0 2023-09-13T04:48:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.572+0000 7fe223c62b80 10 client.0 _ll_register_callbacks cb 0x557d9f378620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:48:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.572+0000 7fe20e19b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d9f48a840 0x557d9f4965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.572+0000 7fe20d99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d9f48ebe0 0x557d9f4937d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.572+0000 7fe20e99c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d9f48f500 0x557d9f493f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.572+0000 7fe20e19b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d9f48a840 0x557d9f4965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53556/0 (socket says 172.21.15.2:53556) 2023-09-13T04:48:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.572+0000 7fe20e19b700 1 -- 172.21.15.2:0/3871587842 learned_addr learned my addr 172.21.15.2:0/3871587842 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.572+0000 7fe20d99a700 1 --2- 172.21.15.2:0/3871587842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d9f48ebe0 0x557d9f4937d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:56146/0 (socket says 172.21.15.2:56146) 2023-09-13T04:48:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.572+0000 7fe20d99a700 1 -- 172.21.15.2:0/3871587842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d9f48f500 msgr2=0x557d9f493f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.572+0000 7fe20d99a700 1 --2- 172.21.15.2:0/3871587842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d9f48f500 0x557d9f493f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.572+0000 7fe20d99a700 1 -- 172.21.15.2:0/3871587842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d9f48a840 msgr2=0x557d9f4965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.572+0000 7fe20d99a700 1 --2- 172.21.15.2:0/3871587842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d9f48a840 0x557d9f4965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.573+0000 7fe20d99a700 1 -- 172.21.15.2:0/3871587842 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe20000e4d0 con 0x557d9f48ebe0 2023-09-13T04:48:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.573+0000 7fe20e19b700 1 --2- 172.21.15.2:0/3871587842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d9f48a840 0x557d9f4965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:48:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.573+0000 7fe20d99a700 1 --2- 172.21.15.2:0/3871587842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d9f48ebe0 0x557d9f4937d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe1f80c9950 tx=0x7fe1f8005020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6420a627a6b36b5 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.573+0000 7fe1f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:48:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.573+0000 7fe1f1ffb700 1 -- 172.21.15.2:0/3871587842 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f80cc3c0 con 0x557d9f48ebe0 2023-09-13T04:48:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.573+0000 7fe223c62b80 1 -- 172.21.15.2:0/3871587842 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d9f44fef0 con 0x557d9f48ebe0 2023-09-13T04:48:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.573+0000 7fe1f1ffb700 1 -- 172.21.15.2:0/3871587842 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7fe1f8099db0 con 0x557d9f48ebe0 2023-09-13T04:48:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.573+0000 7fe1f1ffb700 1 -- 172.21.15.2:0/3871587842 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1f80996b0 con 0x557d9f48ebe0 2023-09-13T04:48:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.573+0000 7fe223c62b80 10 client.26647 fetch_fsmap learned FSMap version 36 2023-09-13T04:48:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.573+0000 7fe223c62b80 1 -- 172.21.15.2:0/3871587842 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x557d9f3f4740 con 0x557d9f48ebe0 2023-09-13T04:48:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.574+0000 7fe1f1ffb700 1 -- 172.21.15.2:0/3871587842 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557d9f44fef0 con 0x557d9f48ebe0 2023-09-13T04:48:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.574+0000 7fe1f1ffb700 1 -- 172.21.15.2:0/3871587842 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fe1f8099910 con 0x557d9f48ebe0 2023-09-13T04:48:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.574+0000 7fe223c62b80 10 client.26647 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:48:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.574+0000 7fe223c62b80 10 client.26647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:48:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.574+0000 7fe223c62b80 20 client.26647 populate_metadata read hostname 'smithi002' 2023-09-13T04:48:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.574+0000 7fe223c62b80 1 --2- 172.21.15.2:0/3871587842 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x557d9f4a3eb0 0x557d9f4a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.574+0000 7fe223c62b80 4 client.26647 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:48:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.574+0000 7fe223c62b80 1 -- 172.21.15.2:0/3871587842 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d9f3d5910 con 0x557d9f4a3eb0 2023-09-13T04:48:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.574+0000 7fe20e19b700 1 --2- 172.21.15.2:0/3871587842 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x557d9f4a3eb0 0x557d9f4a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.575+0000 7fe1f1ffb700 10 client.26647 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.575+0000 7fe20e19b700 1 --2- 172.21.15.2:0/3871587842 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x557d9f4a3eb0 0x557d9f4a62d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4121ecd630014144 server_cookie=60ec539f731c349f in_seq=0 out_seq=0 2023-09-13T04:48:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.576+0000 7fe1f1ffb700 1 -- 172.21.15.2:0/3871587842 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557d9f3d5910 con 0x557d9f4a3eb0 2023-09-13T04:48:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.576+0000 7fe1f1ffb700 10 client.26647 handle_command_reply: tid=0 2023-09-13T04:48:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.576+0000 7fe223c62b80 1 client.26647 shutdown 2023-09-13T04:48:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.576+0000 7fe223c62b80 1 -- 172.21.15.2:0/3871587842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d9f48ebe0 msgr2=0x557d9f4937d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.576+0000 7fe223c62b80 1 --2- 172.21.15.2:0/3871587842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d9f48ebe0 0x557d9f4937d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe1f80c9950 tx=0x7fe1f8005020 comp rx=0 tx=0).stop 2023-09-13T04:48:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.576+0000 7fe223c62b80 1 -- 172.21.15.2:0/3871587842 shutdown_connections 2023-09-13T04:48:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.576+0000 7fe223c62b80 1 --2- 172.21.15.2:0/3871587842 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x557d9f4a3eb0 0x557d9f4a62d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.576+0000 7fe223c62b80 1 --2- 172.21.15.2:0/3871587842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d9f48f500 0x557d9f493f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.576+0000 7fe223c62b80 1 --2- 172.21.15.2:0/3871587842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d9f48a840 0x557d9f4965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.576+0000 7fe223c62b80 1 --2- 172.21.15.2:0/3871587842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d9f48ebe0 0x557d9f4937d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.576+0000 7fe1f1ffb700 0 client.26647 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.576+0000 7fe223c62b80 1 -- 172.21.15.2:0/3871587842 >> 172.21.15.2:0/3871587842 conn(0x557d9f3e8c20 msgr2=0x557d9f47f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.576+0000 7fe223c62b80 1 -- 172.21.15.2:0/3871587842 shutdown_connections 2023-09-13T04:48:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.577+0000 7fe223c62b80 1 -- 172.21.15.2:0/3871587842 wait complete. 2023-09-13T04:48:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.577+0000 7fe223c62b80 20 client.26647 trim_cache size 0 max 16384 2023-09-13T04:48:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.593+0000 7fe223c62b80 1 Processor -- start 2023-09-13T04:48:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.593+0000 7fe223c62b80 1 -- start start 2023-09-13T04:48:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.593+0000 7fe223c62b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d9f48a840 0x557d9f49af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.593+0000 7fe223c62b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d9f48ebe0 0x557d9f47f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.593+0000 7fe223c62b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d9f48f500 0x557d9f4960f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.593+0000 7fe223c62b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9f4a1540 con 0x557d9f48f500 2023-09-13T04:48:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.593+0000 7fe223c62b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9f486790 con 0x557d9f48ebe0 2023-09-13T04:48:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.593+0000 7fe223c62b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9f485100 con 0x557d9f48a840 2023-09-13T04:48:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.593+0000 7fe20e19b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d9f48a840 0x557d9f49af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.593+0000 7fe20e19b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d9f48a840 0x557d9f49af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:56152/0 (socket says 172.21.15.2:56152) 2023-09-13T04:48:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.593+0000 7fe20d99a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d9f48ebe0 0x557d9f47f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.593+0000 7fe20e19b700 1 -- 172.21.15.2:0/916338937 learned_addr learned my addr 172.21.15.2:0/916338937 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.593+0000 7fe20e99c700 1 --2- 172.21.15.2:0/916338937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d9f48f500 0x557d9f4960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.594+0000 7fe20e19b700 1 -- 172.21.15.2:0/916338937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d9f48ebe0 msgr2=0x557d9f47f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.594+0000 7fe20e19b700 1 --2- 172.21.15.2:0/916338937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d9f48ebe0 0x557d9f47f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.594+0000 7fe20e19b700 1 -- 172.21.15.2:0/916338937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d9f48f500 msgr2=0x557d9f4960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.594+0000 7fe20e19b700 1 --2- 172.21.15.2:0/916338937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d9f48f500 0x557d9f4960f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.594+0000 7fe20e19b700 1 -- 172.21.15.2:0/916338937 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d9f208d80 con 0x557d9f48a840 2023-09-13T04:48:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.594+0000 7fe20d99a700 1 --2- 172.21.15.2:0/916338937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d9f48ebe0 0x557d9f47f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:48:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.594+0000 7fe20e19b700 1 --2- 172.21.15.2:0/916338937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d9f48a840 0x557d9f49af80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe20400b6f0 tx=0x7fe20400bb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c183e173ec809cdf server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.594+0000 7fe1fd3ff700 1 -- 172.21.15.2:0/916338937 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204005240 con 0x557d9f48a840 2023-09-13T04:48:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.594+0000 7fe1fd3ff700 1 -- 172.21.15.2:0/916338937 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe20401e040 con 0x557d9f48a840 2023-09-13T04:48:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.594+0000 7fe1fd3ff700 1 -- 172.21.15.2:0/916338937 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20400f460 con 0x557d9f48a840 2023-09-13T04:48:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.595+0000 7fe223c62b80 1 -- 172.21.15.2:0/916338937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d9f48a840 msgr2=0x557d9f49af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.595+0000 7fe223c62b80 1 --2- 172.21.15.2:0/916338937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d9f48a840 0x557d9f49af80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe20400b6f0 tx=0x7fe20400bb60 comp rx=0 tx=0).stop 2023-09-13T04:48:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.595+0000 7fe223c62b80 1 -- 172.21.15.2:0/916338937 shutdown_connections 2023-09-13T04:48:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.595+0000 7fe223c62b80 1 --2- 172.21.15.2:0/916338937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d9f48f500 0x557d9f4960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.595+0000 7fe223c62b80 1 --2- 172.21.15.2:0/916338937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d9f48a840 0x557d9f49af80 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.595+0000 7fe223c62b80 1 --2- 172.21.15.2:0/916338937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d9f48ebe0 0x557d9f47f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.595+0000 7fe223c62b80 1 -- 172.21.15.2:0/916338937 >> 172.21.15.2:0/916338937 conn(0x557d9f3e8c20 msgr2=0x557d9f4898d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.595+0000 7fe223c62b80 1 -- 172.21.15.2:0/916338937 shutdown_connections 2023-09-13T04:48:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.595+0000 7fe223c62b80 1 -- 172.21.15.2:0/916338937 wait complete. 2023-09-13T04:48:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.595+0000 7fe223c62b80 1 -- start start 2023-09-13T04:48:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.596+0000 7fe223c62b80 1 Processor -- start 2023-09-13T04:48:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.596+0000 7fe223c62b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d9f48a840 0x557d9f498ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.596+0000 7fe223c62b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d9f48ebe0 0x557d9f499010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.596+0000 7fe223c62b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d9f48f500 0x557d9f4f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.596+0000 7fe223c62b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9f4f7260 con 0x557d9f48ebe0 2023-09-13T04:48:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.596+0000 7fe223c62b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9f4f73a0 con 0x557d9f48a840 2023-09-13T04:48:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.596+0000 7fe223c62b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9f4f74e0 con 0x557d9f48f500 2023-09-13T04:48:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.596+0000 7fe223c62b80 10 client.0 _ll_register_callbacks cb 0x557d9f378620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:48:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.596+0000 7fe20d99a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d9f48ebe0 0x557d9f499010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.597+0000 7fe20d99a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d9f48ebe0 0x557d9f499010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53566/0 (socket says 172.21.15.2:53566) 2023-09-13T04:48:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.597+0000 7fe20d99a700 1 -- 172.21.15.2:0/3001450758 learned_addr learned my addr 172.21.15.2:0/3001450758 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.597+0000 7fe20e99c700 1 --2- 172.21.15.2:0/3001450758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d9f48f500 0x557d9f4f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.597+0000 7fe20e19b700 1 --2- 172.21.15.2:0/3001450758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d9f48a840 0x557d9f498ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.597+0000 7fe20d99a700 1 -- 172.21.15.2:0/3001450758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d9f48f500 msgr2=0x557d9f4f6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.597+0000 7fe20d99a700 1 --2- 172.21.15.2:0/3001450758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d9f48f500 0x557d9f4f6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.597+0000 7fe20d99a700 1 -- 172.21.15.2:0/3001450758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d9f48a840 msgr2=0x557d9f498ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.597+0000 7fe20d99a700 1 --2- 172.21.15.2:0/3001450758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d9f48a840 0x557d9f498ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.597+0000 7fe20d99a700 1 -- 172.21.15.2:0/3001450758 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d9f3f4740 con 0x557d9f48ebe0 2023-09-13T04:48:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.597+0000 7fe20d99a700 1 --2- 172.21.15.2:0/3001450758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d9f48ebe0 0x557d9f499010 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800c5b0 tx=0x7fe1f80165c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b320c8158df35aa server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.597+0000 7fe1f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:48:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.597+0000 7fe1f1ffb700 1 -- 172.21.15.2:0/3001450758 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8099db0 con 0x557d9f48ebe0 2023-09-13T04:48:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.597+0000 7fe1f1ffb700 1 -- 172.21.15.2:0/3001450758 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7fe1f80996b0 con 0x557d9f48ebe0 2023-09-13T04:48:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.597+0000 7fe223c62b80 1 -- 172.21.15.2:0/3001450758 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe20000f040 con 0x557d9f48ebe0 2023-09-13T04:48:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.598+0000 7fe1f1ffb700 1 -- 172.21.15.2:0/3001450758 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1f809d4e0 con 0x557d9f48ebe0 2023-09-13T04:48:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.598+0000 7fe223c62b80 10 client.16473 fetch_fsmap learned FSMap version 36 2023-09-13T04:48:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.598+0000 7fe223c62b80 1 -- 172.21.15.2:0/3001450758 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7fe208110610 con 0x557d9f48ebe0 2023-09-13T04:48:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.598+0000 7fe1f1ffb700 1 -- 172.21.15.2:0/3001450758 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe20000f040 con 0x557d9f48ebe0 2023-09-13T04:48:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.598+0000 7fe1f1ffb700 1 -- 172.21.15.2:0/3001450758 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fe1f8065080 con 0x557d9f48ebe0 2023-09-13T04:48:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.598+0000 7fe223c62b80 10 client.16473 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:48:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.598+0000 7fe223c62b80 10 client.16473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:48:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.598+0000 7fe223c62b80 20 client.16473 populate_metadata read hostname 'smithi002' 2023-09-13T04:48:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.598+0000 7fe223c62b80 1 --2- 172.21.15.2:0/3001450758 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x557d9f4f8910 0x557d9f4fad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.598+0000 7fe223c62b80 4 client.16473 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:48:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.598+0000 7fe223c62b80 1 -- 172.21.15.2:0/3001450758 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557d9f395ff0 con 0x557d9f4f8910 2023-09-13T04:48:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.598+0000 7fe20e19b700 1 --2- 172.21.15.2:0/3001450758 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x557d9f4f8910 0x557d9f4fad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.599+0000 7fe20e19b700 1 --2- 172.21.15.2:0/3001450758 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x557d9f4f8910 0x557d9f4fad30 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e3af9636aac6f1c server_cookie=5232395df5e10a15 in_seq=0 out_seq=0 2023-09-13T04:48:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.599+0000 7fe1f1ffb700 10 client.16473 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.992+0000 7fe1f1ffb700 1 -- 172.21.15.2:0/3001450758 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557d9f395ff0 con 0x557d9f4f8910 2023-09-13T04:48:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.992+0000 7fe1f1ffb700 10 client.16473 handle_command_reply: tid=0 2023-09-13T04:48:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.992+0000 7fe223c62b80 1 client.16473 shutdown 2023-09-13T04:48:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.992+0000 7fe223c62b80 1 -- 172.21.15.2:0/3001450758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d9f48ebe0 msgr2=0x557d9f499010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.992+0000 7fe223c62b80 1 --2- 172.21.15.2:0/3001450758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d9f48ebe0 0x557d9f499010 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800c5b0 tx=0x7fe1f80165c0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.993+0000 7fe223c62b80 1 -- 172.21.15.2:0/3001450758 shutdown_connections 2023-09-13T04:48:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.993+0000 7fe223c62b80 1 --2- 172.21.15.2:0/3001450758 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x557d9f4f8910 0x557d9f4fad30 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.993+0000 7fe223c62b80 1 --2- 172.21.15.2:0/3001450758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d9f48f500 0x557d9f4f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.993+0000 7fe223c62b80 1 --2- 172.21.15.2:0/3001450758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d9f48a840 0x557d9f498ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.993+0000 7fe223c62b80 1 --2- 172.21.15.2:0/3001450758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d9f48ebe0 0x557d9f499010 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.993+0000 7fe223c62b80 1 -- 172.21.15.2:0/3001450758 >> 172.21.15.2:0/3001450758 conn(0x557d9f3e8c20 msgr2=0x557d9f48d740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.993+0000 7fe1f1ffb700 0 client.16473 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.993+0000 7fe223c62b80 1 -- 172.21.15.2:0/3001450758 shutdown_connections 2023-09-13T04:48:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.993+0000 7fe223c62b80 1 -- 172.21.15.2:0/3001450758 wait complete. 2023-09-13T04:48:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.993+0000 7fe223c62b80 20 client.16473 trim_cache size 0 max 16384 2023-09-13T04:48:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:48:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:48:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:48:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:48:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.994+0000 7fe2103ff700 1 -- 172.21.15.2:0/3614280135 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe1f40a07b0 msgr2=0x7fe1f40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.994+0000 7fe2103ff700 1 --2- 172.21.15.2:0/3614280135 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe1f40a07b0 0x7fe1f40a2c70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe200009550 tx=0x7fe200017750 comp rx=0 tx=0).stop 2023-09-13T04:48:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.994+0000 7fe2103ff700 1 -- 172.21.15.2:0/3614280135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe208108220 msgr2=0x7fe2081994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.994+0000 7fe2103ff700 1 --2- 172.21.15.2:0/3614280135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe208108220 0x7fe2081994b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800e970 tx=0x7fe1f800ed30 comp rx=0 tx=0).stop 2023-09-13T04:48:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.994+0000 7fe2103ff700 1 -- 172.21.15.2:0/3614280135 shutdown_connections 2023-09-13T04:48:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.994+0000 7fe2103ff700 1 --2- 172.21.15.2:0/3614280135 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe1f40a07b0 0x7fe1f40a2c70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.994+0000 7fe2103ff700 1 --2- 172.21.15.2:0/3614280135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe20819db20 0x7fe20819dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.994+0000 7fe2103ff700 1 --2- 172.21.15.2:0/3614280135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe208103100 0x7fe208198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.994+0000 7fe2103ff700 1 --2- 172.21.15.2:0/3614280135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe208108220 0x7fe2081994b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.994+0000 7fe2103ff700 1 -- 172.21.15.2:0/3614280135 >> 172.21.15.2:0/3614280135 conn(0x7fe2080fe960 msgr2=0x7fe2080ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.995+0000 7fe2103ff700 1 -- 172.21.15.2:0/3614280135 shutdown_connections 2023-09-13T04:48:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:35.995+0000 7fe2103ff700 1 -- 172.21.15.2:0/3614280135 wait complete. 2023-09-13T04:48:36.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:35 smithi002 ceph-mon[97651]: pgmap v251: 129 pgs: 129 active+clean; 977 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 707 op/s 2023-09-13T04:48:36.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:35 smithi149 ceph-mon[133315]: pgmap v251: 129 pgs: 129 active+clean; 977 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 707 op/s 2023-09-13T04:48:36.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:35 smithi156 ceph-mon[133675]: pgmap v251: 129 pgs: 129 active+clean; 977 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 707 op/s 2023-09-13T04:48:38.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:37 smithi002 ceph-mon[97651]: pgmap v252: 129 pgs: 129 active+clean; 977 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 460 op/s 2023-09-13T04:48:38.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:37 smithi149 ceph-mon[133315]: pgmap v252: 129 pgs: 129 active+clean; 977 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 460 op/s 2023-09-13T04:48:38.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:37 smithi156 ceph-mon[133675]: pgmap v252: 129 pgs: 129 active+clean; 977 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 460 op/s 2023-09-13T04:48:39.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:38 smithi002 ceph-mon[97651]: scrub summary: idle 2023-09-13T04:48:39.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:38 smithi002 ceph-mon[97651]: scrub summary: idle 2023-09-13T04:48:39.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:38 smithi002 ceph-mon[97651]: scrub summary: idle 2023-09-13T04:48:39.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:38 smithi149 ceph-mon[133315]: scrub summary: idle 2023-09-13T04:48:39.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:38 smithi149 ceph-mon[133315]: scrub summary: idle 2023-09-13T04:48:39.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:38 smithi149 ceph-mon[133315]: scrub summary: idle 2023-09-13T04:48:39.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:38 smithi156 ceph-mon[133675]: scrub summary: idle 2023-09-13T04:48:39.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:38 smithi156 ceph-mon[133675]: scrub summary: idle 2023-09-13T04:48:39.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:38 smithi156 ceph-mon[133675]: scrub summary: idle 2023-09-13T04:48:40.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:39 smithi002 ceph-mon[97651]: pgmap v253: 129 pgs: 129 active+clean; 979 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 475 op/s 2023-09-13T04:48:40.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:39 smithi149 ceph-mon[133315]: pgmap v253: 129 pgs: 129 active+clean; 979 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 475 op/s 2023-09-13T04:48:40.439 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:39 smithi156 ceph-mon[133675]: pgmap v253: 129 pgs: 129 active+clean; 979 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 475 op/s 2023-09-13T04:48:42.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:41 smithi002 ceph-mon[97651]: pgmap v254: 129 pgs: 129 active+clean; 968 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 619 op/s 2023-09-13T04:48:42.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:41 smithi149 ceph-mon[133315]: pgmap v254: 129 pgs: 129 active+clean; 968 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 619 op/s 2023-09-13T04:48:42.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:41 smithi156 ceph-mon[133675]: pgmap v254: 129 pgs: 129 active+clean; 968 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 619 op/s 2023-09-13T04:48:44.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:43 smithi002 ceph-mon[97651]: pgmap v255: 129 pgs: 129 active+clean; 976 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 468 op/s 2023-09-13T04:48:44.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:43 smithi149 ceph-mon[133315]: pgmap v255: 129 pgs: 129 active+clean; 976 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 468 op/s 2023-09-13T04:48:44.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:43 smithi156 ceph-mon[133675]: pgmap v255: 129 pgs: 129 active+clean; 976 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 468 op/s 2023-09-13T04:48:46.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:45 smithi002 ceph-mon[97651]: pgmap v256: 129 pgs: 129 active+clean; 986 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 510 op/s 2023-09-13T04:48:46.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:45 smithi149 ceph-mon[133315]: pgmap v256: 129 pgs: 129 active+clean; 986 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 510 op/s 2023-09-13T04:48:46.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:45 smithi156 ceph-mon[133675]: pgmap v256: 129 pgs: 129 active+clean; 986 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 510 op/s 2023-09-13T04:48:48.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:47 smithi002 ceph-mon[97651]: pgmap v257: 129 pgs: 129 active+clean; 987 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 404 op/s 2023-09-13T04:48:48.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:47 smithi149 ceph-mon[133315]: pgmap v257: 129 pgs: 129 active+clean; 987 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 404 op/s 2023-09-13T04:48:48.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:47 smithi156 ceph-mon[133675]: pgmap v257: 129 pgs: 129 active+clean; 987 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 404 op/s 2023-09-13T04:48:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.176+0000 7fef56bff700 1 -- 172.21.15.2:0/390233588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef5010d920 msgr2=0x7fef5010fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.176+0000 7fef56bff700 1 --2- 172.21.15.2:0/390233588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef5010d920 0x7fef5010fd10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fef480099c0 tx=0x7fef48009cd0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.176+0000 7fef56bff700 1 -- 172.21.15.2:0/390233588 shutdown_connections 2023-09-13T04:48:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.176+0000 7fef56bff700 1 --2- 172.21.15.2:0/390233588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef5010d920 0x7fef5010fd10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.176+0000 7fef56bff700 1 --2- 172.21.15.2:0/390233588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef500680d0 0x7fef500684b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.176+0000 7fef56bff700 1 --2- 172.21.15.2:0/390233588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef500689f0 0x7fef5010d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.176+0000 7fef56bff700 1 -- 172.21.15.2:0/390233588 >> 172.21.15.2:0/390233588 conn(0x7fef50076970 msgr2=0x7fef50076d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.176+0000 7fef56bff700 1 -- 172.21.15.2:0/390233588 shutdown_connections 2023-09-13T04:48:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.176+0000 7fef56bff700 1 -- 172.21.15.2:0/390233588 wait complete. 2023-09-13T04:48:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.178+0000 7fef56bff700 1 Processor -- start 2023-09-13T04:48:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.178+0000 7fef56bff700 1 -- start start 2023-09-13T04:48:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.179+0000 7fef56bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef500680d0 0x7fef50198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.179+0000 7fef56bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef500689f0 0x7fef50199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.179+0000 7fef56bff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef5010d920 0x7fef5019da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.179+0000 7fef56bff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5019e0d0 con 0x7fef5010d920 2023-09-13T04:48:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.179+0000 7fef56bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5019e240 con 0x7fef500680d0 2023-09-13T04:48:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.179+0000 7fef56bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5019e400 con 0x7fef500689f0 2023-09-13T04:48:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.179+0000 7fef4ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef500689f0 0x7fef50199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.179+0000 7fef5499b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef500680d0 0x7fef50198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.179+0000 7fef4ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef500689f0 0x7fef50199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:48756/0 (socket says 172.21.15.2:48756) 2023-09-13T04:48:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.179+0000 7fef4ffff700 1 -- 172.21.15.2:0/213017938 learned_addr learned my addr 172.21.15.2:0/213017938 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.179+0000 7fef5519c700 1 --2- 172.21.15.2:0/213017938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef5010d920 0x7fef5019da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.179+0000 7fef4ffff700 1 -- 172.21.15.2:0/213017938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef500680d0 msgr2=0x7fef50198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.179+0000 7fef4ffff700 1 --2- 172.21.15.2:0/213017938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef500680d0 0x7fef50198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.179+0000 7fef4ffff700 1 -- 172.21.15.2:0/213017938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef5010d920 msgr2=0x7fef5019da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.179+0000 7fef4ffff700 1 --2- 172.21.15.2:0/213017938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef5010d920 0x7fef5019da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.179+0000 7fef4ffff700 1 -- 172.21.15.2:0/213017938 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef480095f0 con 0x7fef500689f0 2023-09-13T04:48:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.180+0000 7fef4ffff700 1 --2- 172.21.15.2:0/213017938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef500689f0 0x7fef50199420 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fef3c00e9a0 tx=0x7fef3c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.180+0000 7fef4dffb700 1 -- 172.21.15.2:0/213017938 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c009790 con 0x7fef500689f0 2023-09-13T04:48:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.180+0000 7fef56bff700 1 -- 172.21.15.2:0/213017938 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef5019e6e0 con 0x7fef500689f0 2023-09-13T04:48:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.180+0000 7fef4dffb700 1 -- 172.21.15.2:0/213017938 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef3c010430 con 0x7fef500689f0 2023-09-13T04:48:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.180+0000 7fef56bff700 1 -- 172.21.15.2:0/213017938 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef501a5f20 con 0x7fef500689f0 2023-09-13T04:48:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.180+0000 7fef4dffb700 1 -- 172.21.15.2:0/213017938 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c018680 con 0x7fef500689f0 2023-09-13T04:48:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.185+0000 7fef6a4dbb80 1 Processor -- start 2023-09-13T04:48:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.185+0000 7fef6a4dbb80 1 -- start start 2023-09-13T04:48:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.185+0000 7fef4dffb700 1 -- 172.21.15.2:0/213017938 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7fef3c00ca10 con 0x7fef500689f0 2023-09-13T04:48:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.185+0000 7fef6a4dbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55820443fbe0 0x55820443ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.185+0000 7fef6a4dbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558204440500 0x55820443b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.185+0000 7fef6a4dbb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820443b840 0x55820443bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.185+0000 7fef6a4dbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558204394720 con 0x55820443fbe0 2023-09-13T04:48:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.185+0000 7fef6a4dbb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558204392040 con 0x55820443b840 2023-09-13T04:48:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.185+0000 7fef6a4dbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55820438d930 con 0x558204440500 2023-09-13T04:48:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.186+0000 7fef5499b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558204440500 0x55820443b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.186+0000 7fef5519c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55820443fbe0 0x55820443ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.186+0000 7fef5499b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558204440500 0x55820443b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:48760/0 (socket says 172.21.15.2:48760) 2023-09-13T04:48:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.186+0000 7fef5499b700 1 -- 172.21.15.2:0/4217125599 learned_addr learned my addr 172.21.15.2:0/4217125599 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.186+0000 7fef4ffff700 1 --2- 172.21.15.2:0/4217125599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820443b840 0x55820443bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.186+0000 7fef5499b700 1 -- 172.21.15.2:0/4217125599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820443b840 msgr2=0x55820443bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.186+0000 7fef5499b700 1 --2- 172.21.15.2:0/4217125599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820443b840 0x55820443bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.186+0000 7fef5499b700 1 -- 172.21.15.2:0/4217125599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55820443fbe0 msgr2=0x55820443ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.186+0000 7fef5499b700 1 --2- 172.21.15.2:0/4217125599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55820443fbe0 0x55820443ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.186+0000 7fef5499b700 1 -- 172.21.15.2:0/4217125599 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55820420d5d0 con 0x558204440500 2023-09-13T04:48:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.186+0000 7fef4ffff700 1 --2- 172.21.15.2:0/4217125599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820443b840 0x55820443bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:48:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.187+0000 7fef5499b700 1 --2- 172.21.15.2:0/4217125599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558204440500 0x55820443b300 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fef44011650 tx=0x7fef44009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82e1f5b776103f57 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.187+0000 7fef3affd700 1 -- 172.21.15.2:0/4217125599 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4400e040 con 0x558204440500 2023-09-13T04:48:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.187+0000 7fef3affd700 1 -- 172.21.15.2:0/4217125599 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef4400f040 con 0x558204440500 2023-09-13T04:48:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.187+0000 7fef3affd700 1 -- 172.21.15.2:0/4217125599 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef44015960 con 0x558204440500 2023-09-13T04:48:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.188+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/4217125599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558204440500 msgr2=0x55820443b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.188+0000 7fef6a4dbb80 1 --2- 172.21.15.2:0/4217125599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558204440500 0x55820443b300 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fef44011650 tx=0x7fef44009920 comp rx=0 tx=0).stop 2023-09-13T04:48:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.188+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/4217125599 shutdown_connections 2023-09-13T04:48:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.188+0000 7fef6a4dbb80 1 --2- 172.21.15.2:0/4217125599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820443b840 0x55820443bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.188+0000 7fef6a4dbb80 1 --2- 172.21.15.2:0/4217125599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55820443fbe0 0x55820443ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.188+0000 7fef6a4dbb80 1 --2- 172.21.15.2:0/4217125599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558204440500 0x55820443b300 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.188+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/4217125599 >> 172.21.15.2:0/4217125599 conn(0x558204399c20 msgr2=0x558204435b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.188+0000 7fef4dffb700 1 --2- 172.21.15.2:0/213017938 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fef400a07b0 0x7fef400a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.188+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/4217125599 shutdown_connections 2023-09-13T04:48:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.188+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/4217125599 wait complete. 2023-09-13T04:48:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.188+0000 7fef4dffb700 1 -- 172.21.15.2:0/213017938 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7fef3c014070 con 0x7fef500689f0 2023-09-13T04:48:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.188+0000 7fef5519c700 1 --2- 172.21.15.2:0/213017938 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fef400a07b0 0x7fef400a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.189+0000 7fef6a4dbb80 1 -- start start 2023-09-13T04:48:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.189+0000 7fef6a4dbb80 1 Processor -- start 2023-09-13T04:48:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.189+0000 7fef5519c700 1 --2- 172.21.15.2:0/213017938 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fef400a07b0 0x7fef400a2c70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fef4800b580 tx=0x7fef48005fb0 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.190+0000 7fef6a4dbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55820443b840 0x5582044475b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.190+0000 7fef6a4dbb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820443fbe0 0x5582044447c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.190+0000 7fef6a4dbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558204440500 0x558204444f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.190+0000 7fef6a4dbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55820432bb50 con 0x558204440500 2023-09-13T04:48:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.190+0000 7fef6a4dbb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55820432b850 con 0x55820443fbe0 2023-09-13T04:48:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.190+0000 7fef6a4dbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55820420f8c0 con 0x55820443b840 2023-09-13T04:48:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.190+0000 7fef6a4dbb80 10 client.0 _ll_register_callbacks cb 0x558204329620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:48:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.190+0000 7fef5499b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55820443b840 0x5582044475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.190+0000 7fef5499b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55820443b840 0x5582044475b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:48768/0 (socket says 172.21.15.2:48768) 2023-09-13T04:48:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.190+0000 7fef5499b700 1 -- 172.21.15.2:0/1412311840 learned_addr learned my addr 172.21.15.2:0/1412311840 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.190+0000 7fef4ffff700 1 --2- 172.21.15.2:0/1412311840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820443fbe0 0x5582044447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.190+0000 7fef5519c700 1 --2- 172.21.15.2:0/1412311840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558204440500 0x558204444f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.191+0000 7fef5499b700 1 -- 172.21.15.2:0/1412311840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820443fbe0 msgr2=0x5582044447c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.191+0000 7fef5499b700 1 --2- 172.21.15.2:0/1412311840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820443fbe0 0x5582044447c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.191+0000 7fef5499b700 1 -- 172.21.15.2:0/1412311840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558204440500 msgr2=0x558204444f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.191+0000 7fef5499b700 1 --2- 172.21.15.2:0/1412311840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558204440500 0x558204444f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.191+0000 7fef5499b700 1 -- 172.21.15.2:0/1412311840 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef44011330 con 0x55820443b840 2023-09-13T04:48:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.191+0000 7fef4ffff700 1 --2- 172.21.15.2:0/1412311840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820443fbe0 0x5582044447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:48:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.191+0000 7fef5499b700 1 --2- 172.21.15.2:0/1412311840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55820443b840 0x5582044475b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fef44009040 tx=0x7fef44007730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8ef9d9512ea9d8d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.191+0000 7fef2ffff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:48:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.191+0000 7fef2ffff700 1 -- 172.21.15.2:0/1412311840 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4400e040 con 0x55820443b840 2023-09-13T04:48:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.191+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/1412311840 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558204400ef0 con 0x55820443b840 2023-09-13T04:48:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.191+0000 7fef2ffff700 1 -- 172.21.15.2:0/1412311840 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7fef440095e0 con 0x55820443b840 2023-09-13T04:48:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.192+0000 7fef2ffff700 1 -- 172.21.15.2:0/1412311840 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef44007e30 con 0x55820443b840 2023-09-13T04:48:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.192+0000 7fef6a4dbb80 10 client.26665 fetch_fsmap learned FSMap version 36 2023-09-13T04:48:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.192+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/1412311840 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5582043a5740 con 0x55820443b840 2023-09-13T04:48:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.192+0000 7fef2ffff700 1 -- 172.21.15.2:0/1412311840 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558204400ef0 con 0x55820443b840 2023-09-13T04:48:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.192+0000 7fef2ffff700 1 -- 172.21.15.2:0/1412311840 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fef4401e450 con 0x55820443b840 2023-09-13T04:48:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.192+0000 7fef6a4dbb80 10 client.26665 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:48:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.192+0000 7fef6a4dbb80 10 client.26665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:48:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.192+0000 7fef6a4dbb80 20 client.26665 populate_metadata read hostname 'smithi002' 2023-09-13T04:48:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.192+0000 7fef6a4dbb80 1 --2- 172.21.15.2:0/1412311840 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x558204454e70 0x558204457290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.192+0000 7fef6a4dbb80 4 client.26665 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:48:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.192+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/1412311840 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558204386910 con 0x558204454e70 2023-09-13T04:48:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.193+0000 7fef4ffff700 1 --2- 172.21.15.2:0/1412311840 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x558204454e70 0x558204457290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.193+0000 7fef4ffff700 1 --2- 172.21.15.2:0/1412311840 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x558204454e70 0x558204457290 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f37f05486280dc8 server_cookie=2cdac033e796dc57 in_seq=0 out_seq=0 2023-09-13T04:48:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.194+0000 7fef2ffff700 10 client.26665 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.194+0000 7fef2ffff700 1 -- 172.21.15.2:0/1412311840 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558204386910 con 0x558204454e70 2023-09-13T04:48:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.194+0000 7fef2ffff700 10 client.26665 handle_command_reply: tid=0 2023-09-13T04:48:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.194+0000 7fef6a4dbb80 1 client.26665 shutdown 2023-09-13T04:48:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.195+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/1412311840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55820443b840 msgr2=0x5582044475b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.195+0000 7fef6a4dbb80 1 --2- 172.21.15.2:0/1412311840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55820443b840 0x5582044475b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fef44009040 tx=0x7fef44007730 comp rx=0 tx=0).stop 2023-09-13T04:48:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.195+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/1412311840 shutdown_connections 2023-09-13T04:48:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.195+0000 7fef6a4dbb80 1 --2- 172.21.15.2:0/1412311840 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x558204454e70 0x558204457290 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.195+0000 7fef6a4dbb80 1 --2- 172.21.15.2:0/1412311840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558204440500 0x558204444f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.195+0000 7fef6a4dbb80 1 --2- 172.21.15.2:0/1412311840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55820443b840 0x5582044475b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.195+0000 7fef6a4dbb80 1 --2- 172.21.15.2:0/1412311840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820443fbe0 0x5582044447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.195+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/1412311840 >> 172.21.15.2:0/1412311840 conn(0x558204399c20 msgr2=0x558204430630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.195+0000 7fef2ffff700 0 client.26665 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.195+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/1412311840 shutdown_connections 2023-09-13T04:48:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.195+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/1412311840 wait complete. 2023-09-13T04:48:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.196+0000 7fef6a4dbb80 20 client.26665 trim_cache size 0 max 16384 2023-09-13T04:48:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.212+0000 7fef6a4dbb80 1 Processor -- start 2023-09-13T04:48:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.212+0000 7fef6a4dbb80 1 -- start start 2023-09-13T04:48:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.212+0000 7fef6a4dbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55820443b840 0x558204430730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.212+0000 7fef6a4dbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55820443fbe0 0x5582044392f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.212+0000 7fef6a4dbb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558204440500 0x5582044470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.212+0000 7fef6a4dbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55820443f790 con 0x55820443b840 2023-09-13T04:48:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.212+0000 7fef6a4dbb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558204430c70 con 0x558204440500 2023-09-13T04:48:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.212+0000 7fef6a4dbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558204436c80 con 0x55820443fbe0 2023-09-13T04:48:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.213+0000 7fef5499b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55820443b840 0x558204430730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.213+0000 7fef4ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55820443fbe0 0x5582044392f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.213+0000 7fef4ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55820443fbe0 0x5582044392f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:48780/0 (socket says 172.21.15.2:48780) 2023-09-13T04:48:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.213+0000 7fef5499b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55820443b840 0x558204430730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46540/0 (socket says 172.21.15.2:46540) 2023-09-13T04:48:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.213+0000 7fef4ffff700 1 -- 172.21.15.2:0/2988974204 learned_addr learned my addr 172.21.15.2:0/2988974204 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.213+0000 7fef5519c700 1 --2- 172.21.15.2:0/2988974204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558204440500 0x5582044470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.213+0000 7fef4ffff700 1 -- 172.21.15.2:0/2988974204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558204440500 msgr2=0x5582044470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.213+0000 7fef4ffff700 1 --2- 172.21.15.2:0/2988974204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558204440500 0x5582044470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.213+0000 7fef4ffff700 1 -- 172.21.15.2:0/2988974204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55820443b840 msgr2=0x558204430730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.213+0000 7fef4ffff700 1 --2- 172.21.15.2:0/2988974204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55820443b840 0x558204430730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.213+0000 7fef4ffff700 1 -- 172.21.15.2:0/2988974204 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558204436110 con 0x55820443fbe0 2023-09-13T04:48:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.213+0000 7fef5519c700 1 --2- 172.21.15.2:0/2988974204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558204440500 0x5582044470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:48:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.213+0000 7fef4ffff700 1 --2- 172.21.15.2:0/2988974204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55820443fbe0 0x5582044392f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fef3c00e970 tx=0x7fef3c0ce6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afbce727f6e70b04 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.214+0000 7fef3b7fe700 1 -- 172.21.15.2:0/2988974204 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c09d170 con 0x55820443fbe0 2023-09-13T04:48:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.214+0000 7fef3b7fe700 1 -- 172.21.15.2:0/2988974204 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef3c0a15b0 con 0x55820443fbe0 2023-09-13T04:48:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.214+0000 7fef3b7fe700 1 -- 172.21.15.2:0/2988974204 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c0997a0 con 0x55820443fbe0 2023-09-13T04:48:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.215+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/2988974204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55820443fbe0 msgr2=0x5582044392f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.215+0000 7fef6a4dbb80 1 --2- 172.21.15.2:0/2988974204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55820443fbe0 0x5582044392f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fef3c00e970 tx=0x7fef3c0ce6b0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.215+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/2988974204 shutdown_connections 2023-09-13T04:48:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.215+0000 7fef6a4dbb80 1 --2- 172.21.15.2:0/2988974204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558204440500 0x5582044470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.215+0000 7fef6a4dbb80 1 --2- 172.21.15.2:0/2988974204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55820443b840 0x558204430730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.215+0000 7fef6a4dbb80 1 --2- 172.21.15.2:0/2988974204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55820443fbe0 0x5582044392f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.215+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/2988974204 >> 172.21.15.2:0/2988974204 conn(0x558204399c20 msgr2=0x558204450700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.215+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/2988974204 shutdown_connections 2023-09-13T04:48:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.215+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/2988974204 wait complete. 2023-09-13T04:48:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.216+0000 7fef6a4dbb80 1 -- start start 2023-09-13T04:48:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.216+0000 7fef6a4dbb80 1 Processor -- start 2023-09-13T04:48:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.217+0000 7fef6a4dbb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820443b840 0x558204449f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.217+0000 7fef6a4dbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55820443fbe0 0x55820444a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.217+0000 7fef6a4dbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558204440500 0x55820444aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.217+0000 7fef6a4dbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582044a7d80 con 0x558204440500 2023-09-13T04:48:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.217+0000 7fef6a4dbb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582044a7ec0 con 0x55820443b840 2023-09-13T04:48:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.217+0000 7fef6a4dbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582044a8080 con 0x55820443fbe0 2023-09-13T04:48:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.217+0000 7fef6a4dbb80 10 client.0 _ll_register_callbacks cb 0x558204329620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:48:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.218+0000 7fef4ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55820443fbe0 0x55820444a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.218+0000 7fef4ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55820443fbe0 0x55820444a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:48788/0 (socket says 172.21.15.2:48788) 2023-09-13T04:48:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.218+0000 7fef4ffff700 1 -- 172.21.15.2:0/4058765044 learned_addr learned my addr 172.21.15.2:0/4058765044 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.218+0000 7fef5499b700 1 --2- 172.21.15.2:0/4058765044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820443b840 0x558204449f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.218+0000 7fef5519c700 1 --2- 172.21.15.2:0/4058765044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558204440500 0x55820444aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.218+0000 7fef4ffff700 1 -- 172.21.15.2:0/4058765044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820443b840 msgr2=0x558204449f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.218+0000 7fef4ffff700 1 --2- 172.21.15.2:0/4058765044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820443b840 0x558204449f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.218+0000 7fef4ffff700 1 -- 172.21.15.2:0/4058765044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558204440500 msgr2=0x55820444aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.218+0000 7fef4ffff700 1 --2- 172.21.15.2:0/4058765044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558204440500 0x55820444aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.218+0000 7fef4ffff700 1 -- 172.21.15.2:0/4058765044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef501a5f20 con 0x55820443fbe0 2023-09-13T04:48:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.218+0000 7fef5499b700 1 --2- 172.21.15.2:0/4058765044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820443b840 0x558204449f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:48:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.219+0000 7fef4ffff700 1 --2- 172.21.15.2:0/4058765044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55820443fbe0 0x55820444a4d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fef3c095a40 tx=0x7fef3c09d170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49f2ce2a9fb0aa3c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.219+0000 7fef2ffff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:48:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.219+0000 7fef2ffff700 1 -- 172.21.15.2:0/4058765044 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c098c30 con 0x55820443fbe0 2023-09-13T04:48:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.219+0000 7fef2ffff700 1 -- 172.21.15.2:0/4058765044 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7fef3c0997a0 con 0x55820443fbe0 2023-09-13T04:48:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.219+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/4058765044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef4400f040 con 0x55820443fbe0 2023-09-13T04:48:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.219+0000 7fef2ffff700 1 -- 172.21.15.2:0/4058765044 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef3c0c9c70 con 0x55820443fbe0 2023-09-13T04:48:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.219+0000 7fef6a4dbb80 10 client.26671 fetch_fsmap learned FSMap version 36 2023-09-13T04:48:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.219+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/4058765044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5582041b9d80 con 0x55820443fbe0 2023-09-13T04:48:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.219+0000 7fef2ffff700 1 -- 172.21.15.2:0/4058765044 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef4400f040 con 0x55820443fbe0 2023-09-13T04:48:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.220+0000 7fef2ffff700 1 -- 172.21.15.2:0/4058765044 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fef3c0985a0 con 0x55820443fbe0 2023-09-13T04:48:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.220+0000 7fef6a4dbb80 10 client.26671 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:48:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.220+0000 7fef6a4dbb80 10 client.26671 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:48:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.220+0000 7fef6a4dbb80 20 client.26671 populate_metadata read hostname 'smithi002' 2023-09-13T04:48:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.220+0000 7fef6a4dbb80 1 --2- 172.21.15.2:0/4058765044 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5582044a95f0 0x5582044aba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.220+0000 7fef6a4dbb80 4 client.26671 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:48:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.220+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/4058765044 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558204346ff0 con 0x5582044a95f0 2023-09-13T04:48:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.220+0000 7fef5499b700 1 --2- 172.21.15.2:0/4058765044 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5582044a95f0 0x5582044aba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.220+0000 7fef5499b700 1 --2- 172.21.15.2:0/4058765044 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5582044a95f0 0x5582044aba10 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5117fc02b6e57a00 server_cookie=2657f6649883512e in_seq=0 out_seq=0 2023-09-13T04:48:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.220+0000 7fef2ffff700 10 client.26671 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.252+0000 7fef2ffff700 1 -- 172.21.15.2:0/4058765044 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558204346ff0 con 0x5582044a95f0 2023-09-13T04:48:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.252+0000 7fef2ffff700 10 client.26671 handle_command_reply: tid=0 2023-09-13T04:48:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.252+0000 7fef6a4dbb80 1 client.26671 shutdown 2023-09-13T04:48:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.253+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/4058765044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55820443fbe0 msgr2=0x55820444a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:49.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.253+0000 7fef6a4dbb80 1 --2- 172.21.15.2:0/4058765044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55820443fbe0 0x55820444a4d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fef3c095a40 tx=0x7fef3c09d170 comp rx=0 tx=0).stop 2023-09-13T04:48:49.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.253+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/4058765044 shutdown_connections 2023-09-13T04:48:49.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.253+0000 7fef6a4dbb80 1 --2- 172.21.15.2:0/4058765044 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x5582044a95f0 0x5582044aba10 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.253+0000 7fef6a4dbb80 1 --2- 172.21.15.2:0/4058765044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558204440500 0x55820444aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.253+0000 7fef6a4dbb80 1 --2- 172.21.15.2:0/4058765044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820443b840 0x558204449f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.253+0000 7fef6a4dbb80 1 --2- 172.21.15.2:0/4058765044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55820443fbe0 0x55820444a4d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.253+0000 7fef2ffff700 0 client.26671 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:49.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.253+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/4058765044 >> 172.21.15.2:0/4058765044 conn(0x558204399c20 msgr2=0x55820443a860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:49.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.253+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/4058765044 shutdown_connections 2023-09-13T04:48:49.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.253+0000 7fef6a4dbb80 1 -- 172.21.15.2:0/4058765044 wait complete. 2023-09-13T04:48:49.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.255+0000 7fef6a4dbb80 20 client.26671 trim_cache size 0 max 16384 2023-09-13T04:48:49.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:48:49.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:48:49.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:48:49.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:48:49.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.256+0000 7fef56bff700 1 -- 172.21.15.2:0/213017938 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fef400a07b0 msgr2=0x7fef400a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:49.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.256+0000 7fef56bff700 1 --2- 172.21.15.2:0/213017938 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fef400a07b0 0x7fef400a2c70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fef4800b580 tx=0x7fef48005fb0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.256+0000 7fef56bff700 1 -- 172.21.15.2:0/213017938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef500689f0 msgr2=0x7fef50199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:49.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.256+0000 7fef56bff700 1 --2- 172.21.15.2:0/213017938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef500689f0 0x7fef50199420 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fef3c00e9a0 tx=0x7fef3c00ecb0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.256+0000 7fef56bff700 1 -- 172.21.15.2:0/213017938 shutdown_connections 2023-09-13T04:48:49.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.256+0000 7fef56bff700 1 --2- 172.21.15.2:0/213017938 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fef400a07b0 0x7fef400a2c70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.256+0000 7fef56bff700 1 --2- 172.21.15.2:0/213017938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef5010d920 0x7fef5019da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.256+0000 7fef56bff700 1 --2- 172.21.15.2:0/213017938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef500680d0 0x7fef50198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.256+0000 7fef56bff700 1 --2- 172.21.15.2:0/213017938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef500689f0 0x7fef50199420 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:49.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.256+0000 7fef56bff700 1 -- 172.21.15.2:0/213017938 >> 172.21.15.2:0/213017938 conn(0x7fef50076970 msgr2=0x7fef50107430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:49.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.256+0000 7fef56bff700 1 -- 172.21.15.2:0/213017938 shutdown_connections 2023-09-13T04:48:49.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:48:49.256+0000 7fef56bff700 1 -- 172.21.15.2:0/213017938 wait complete. 2023-09-13T04:48:50.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:49 smithi002 ceph-mon[97651]: pgmap v258: 129 pgs: 129 active+clean; 987 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 409 op/s 2023-09-13T04:48:50.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:49 smithi149 ceph-mon[133315]: pgmap v258: 129 pgs: 129 active+clean; 987 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 409 op/s 2023-09-13T04:48:50.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:49 smithi156 ceph-mon[133675]: pgmap v258: 129 pgs: 129 active+clean; 987 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 409 op/s 2023-09-13T04:48:52.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:51 smithi002 ceph-mon[97651]: pgmap v259: 129 pgs: 129 active+clean; 997 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 462 op/s 2023-09-13T04:48:52.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:51 smithi149 ceph-mon[133315]: pgmap v259: 129 pgs: 129 active+clean; 997 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 462 op/s 2023-09-13T04:48:52.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:51 smithi156 ceph-mon[133675]: pgmap v259: 129 pgs: 129 active+clean; 997 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 462 op/s 2023-09-13T04:48:54.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:53 smithi002 ceph-mon[97651]: pgmap v260: 129 pgs: 129 active+clean; 997 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 303 op/s 2023-09-13T04:48:54.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:53 smithi149 ceph-mon[133315]: pgmap v260: 129 pgs: 129 active+clean; 997 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 303 op/s 2023-09-13T04:48:54.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:53 smithi156 ceph-mon[133675]: pgmap v260: 129 pgs: 129 active+clean; 997 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 303 op/s 2023-09-13T04:48:55.173 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T04:48:55.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.466+0000 7f3bc63ff700 1 -- 172.21.15.2:0/4254924110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bc01081f0 msgr2=0x7f3bc0108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:55.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.466+0000 7f3bc63ff700 1 --2- 172.21.15.2:0/4254924110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bc01081f0 0x7f3bc0108670 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009990 tx=0x7f3bb0009ca0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.469+0000 7f3bc63ff700 1 -- 172.21.15.2:0/4254924110 shutdown_connections 2023-09-13T04:48:55.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.469+0000 7f3bc63ff700 1 --2- 172.21.15.2:0/4254924110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bc011a490 0x7f3bc011c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.469+0000 7f3bc63ff700 1 --2- 172.21.15.2:0/4254924110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bc010f440 0x7f3bc0107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.469+0000 7f3bc63ff700 1 --2- 172.21.15.2:0/4254924110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bc01081f0 0x7f3bc0108670 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.469+0000 7f3bc63ff700 1 -- 172.21.15.2:0/4254924110 >> 172.21.15.2:0/4254924110 conn(0x7f3bc006cfa0 msgr2=0x7f3bc006d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:55.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.469+0000 7f3bc63ff700 1 -- 172.21.15.2:0/4254924110 shutdown_connections 2023-09-13T04:48:55.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.470+0000 7f3bc63ff700 1 -- 172.21.15.2:0/4254924110 wait complete. 2023-09-13T04:48:55.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.471+0000 7f3bc63ff700 1 Processor -- start 2023-09-13T04:48:55.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.472+0000 7f3bc63ff700 1 -- start start 2023-09-13T04:48:55.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.472+0000 7f3bc63ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bc01081f0 0x7f3bc01aba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:55.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.472+0000 7f3bc63ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bc010f440 0x7f3bc01abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:55.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.472+0000 7f3bc63ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bc011a490 0x7f3bc01a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:55.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.472+0000 7f3bc63ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc01a6150 con 0x7f3bc01081f0 2023-09-13T04:48:55.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.473+0000 7f3bc63ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc01a62c0 con 0x7f3bc011a490 2023-09-13T04:48:55.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.473+0000 7f3bc63ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc01a6480 con 0x7f3bc010f440 2023-09-13T04:48:55.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.473+0000 7f3bbf7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bc010f440 0x7f3bc01abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:55.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.473+0000 7f3bbf7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bc010f440 0x7f3bc01abfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:38422/0 (socket says 172.21.15.2:38422) 2023-09-13T04:48:55.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.473+0000 7f3bbf7fe700 1 -- 172.21.15.2:0/1589796019 learned_addr learned my addr 172.21.15.2:0/1589796019 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:55.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.473+0000 7f3bbf7fe700 1 -- 172.21.15.2:0/1589796019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bc011a490 msgr2=0x7f3bc01a5ae0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-13T04:48:55.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.473+0000 7f3bbf7fe700 1 --2- 172.21.15.2:0/1589796019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bc011a490 0x7f3bc01a5ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.473+0000 7f3bbf7fe700 1 -- 172.21.15.2:0/1589796019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bc01081f0 msgr2=0x7f3bc01aba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-13T04:48:55.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.474+0000 7f3bbf7fe700 1 --2- 172.21.15.2:0/1589796019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bc01081f0 0x7f3bc01aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.474+0000 7f3bbf7fe700 1 -- 172.21.15.2:0/1589796019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb00095f0 con 0x7f3bc010f440 2023-09-13T04:48:55.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.474+0000 7f3bbf7fe700 1 --2- 172.21.15.2:0/1589796019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bc010f440 0x7f3bc01abfb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0003b70 tx=0x7f3bb00037c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:55.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.474+0000 7f3bbd7fa700 1 -- 172.21.15.2:0/1589796019 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb001d070 con 0x7f3bc010f440 2023-09-13T04:48:55.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.475+0000 7f3bc63ff700 1 -- 172.21.15.2:0/1589796019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc01a6700 con 0x7f3bc010f440 2023-09-13T04:48:55.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.475+0000 7f3bc63ff700 1 -- 172.21.15.2:0/1589796019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc010fe50 con 0x7f3bc010f440 2023-09-13T04:48:55.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.475+0000 7f3bbd7fa700 1 -- 172.21.15.2:0/1589796019 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bb0003e30 con 0x7f3bc010f440 2023-09-13T04:48:55.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.475+0000 7f3bbd7fa700 1 -- 172.21.15.2:0/1589796019 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb000fde0 con 0x7f3bc010f440 2023-09-13T04:48:55.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.482+0000 7f3bd9ce4b80 1 Processor -- start 2023-09-13T04:48:55.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.483+0000 7f3bd9ce4b80 1 -- start start 2023-09-13T04:48:55.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.483+0000 7f3bd9ce4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59be0cfe0 0x55f59be0d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:55.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.483+0000 7f3bd9ce4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f59be0d900 0x55f59be074c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:55.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.483+0000 7f3bd9ce4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f59be07a00 0x55f59be07e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:55.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.483+0000 7f3bd9ce4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59bd60780 con 0x55f59be07a00 2023-09-13T04:48:55.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.483+0000 7f3bd9ce4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59bd5e0a0 con 0x55f59be0d900 2023-09-13T04:48:55.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.483+0000 7f3bd9ce4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59bd59990 con 0x55f59be0cfe0 2023-09-13T04:48:55.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.484+0000 7f3bc499c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59be0cfe0 0x55f59be0d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:55.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.484+0000 7f3bc499c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59be0cfe0 0x55f59be0d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:38426/0 (socket says 172.21.15.2:38426) 2023-09-13T04:48:55.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.484+0000 7f3bc499c700 1 -- 172.21.15.2:0/3323895044 learned_addr learned my addr 172.21.15.2:0/3323895044 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:55.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.485+0000 7f3bbffff700 1 --2- 172.21.15.2:0/3323895044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f59be0d900 0x55f59be074c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:55.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.485+0000 7f3bc499c700 1 -- 172.21.15.2:0/3323895044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f59be0d900 msgr2=0x55f59be074c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:55.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.485+0000 7f3bc499c700 1 --2- 172.21.15.2:0/3323895044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f59be0d900 0x55f59be074c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.485+0000 7f3bc499c700 1 -- 172.21.15.2:0/3323895044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f59be07a00 msgr2=0x55f59be07e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:48:55.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.485+0000 7f3bc499c700 1 --2- 172.21.15.2:0/3323895044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f59be07a00 0x55f59be07e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.485+0000 7f3bc499c700 1 -- 172.21.15.2:0/3323895044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f59bbd9630 con 0x55f59be0cfe0 2023-09-13T04:48:55.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.485+0000 7f3bbffff700 1 --2- 172.21.15.2:0/3323895044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f59be0d900 0x55f59be074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T04:48:55.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.487+0000 7f3bc499c700 1 --2- 172.21.15.2:0/3323895044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59be0cfe0 0x55f59be0d3c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8001f30 tx=0x7f3bb800c0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ef5b4460e4745d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:55.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.488+0000 7f3ba67fc700 1 -- 172.21.15.2:0/3323895044 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8016070 con 0x55f59be0cfe0 2023-09-13T04:48:55.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.488+0000 7f3ba67fc700 1 -- 172.21.15.2:0/3323895044 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bb8003a00 con 0x55f59be0cfe0 2023-09-13T04:48:55.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.488+0000 7f3ba67fc700 1 -- 172.21.15.2:0/3323895044 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8004730 con 0x55f59be0cfe0 2023-09-13T04:48:55.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.489+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/3323895044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59be0cfe0 msgr2=0x55f59be0d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:55.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.489+0000 7f3bd9ce4b80 1 --2- 172.21.15.2:0/3323895044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59be0cfe0 0x55f59be0d3c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8001f30 tx=0x7f3bb800c0e0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.489+0000 7f3bbd7fa700 1 -- 172.21.15.2:0/1589796019 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f3bb000f460 con 0x7f3bc010f440 2023-09-13T04:48:55.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.490+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/3323895044 shutdown_connections 2023-09-13T04:48:55.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.490+0000 7f3bd9ce4b80 1 --2- 172.21.15.2:0/3323895044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f59be07a00 0x55f59be07e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.490+0000 7f3bd9ce4b80 1 --2- 172.21.15.2:0/3323895044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59be0cfe0 0x55f59be0d3c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.490+0000 7f3bd9ce4b80 1 --2- 172.21.15.2:0/3323895044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f59be0d900 0x55f59be074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.490+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/3323895044 >> 172.21.15.2:0/3323895044 conn(0x55f59bd65c80 msgr2=0x55f59be01f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:55.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.491+0000 7f3bbd7fa700 1 --2- 172.21.15.2:0/1589796019 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f3ba80a0870 0x7f3ba80a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:55.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.492+0000 7f3bbd7fa700 1 -- 172.21.15.2:0/1589796019 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f3bb00cd480 con 0x7f3bc010f440 2023-09-13T04:48:55.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.495+0000 7f3bbffff700 1 --2- 172.21.15.2:0/1589796019 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f3ba80a0870 0x7f3ba80a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:55.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.497+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/3323895044 shutdown_connections 2023-09-13T04:48:55.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.498+0000 7f3bbffff700 1 --2- 172.21.15.2:0/1589796019 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f3ba80a0870 0x7f3ba80a2d30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4005fd0 tx=0x7f3bb4005f00 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:55.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.499+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/3323895044 wait complete. 2023-09-13T04:48:55.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.499+0000 7f3bd9ce4b80 1 -- start start 2023-09-13T04:48:55.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.500+0000 7f3bd9ce4b80 1 Processor -- start 2023-09-13T04:48:55.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.503+0000 7f3bd9ce4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f59be07a00 0x55f59be13910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:55.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.503+0000 7f3bd9ce4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59be0cfe0 0x55f59be10c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:55.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.503+0000 7f3bd9ce4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f59be0d900 0x55f59be112f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:55.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.503+0000 7f3bd9ce4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59bcf7bb0 con 0x55f59be0d900 2023-09-13T04:48:55.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.503+0000 7f3bd9ce4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59bcf78b0 con 0x55f59be07a00 2023-09-13T04:48:55.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.503+0000 7f3bd9ce4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59bbdb920 con 0x55f59be0cfe0 2023-09-13T04:48:55.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.503+0000 7f3bd9ce4b80 10 client.0 _ll_register_callbacks cb 0x55f59bcf5680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:48:55.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.505+0000 7f3bc499c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f59be0d900 0x55f59be112f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:55.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.505+0000 7f3bc499c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f59be0d900 0x55f59be112f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42948/0 (socket says 172.21.15.2:42948) 2023-09-13T04:48:55.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.505+0000 7f3bc499c700 1 -- 172.21.15.2:0/756849700 learned_addr learned my addr 172.21.15.2:0/756849700 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:55.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.505+0000 7f3bbf7fe700 1 --2- 172.21.15.2:0/756849700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59be0cfe0 0x55f59be10c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:55.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.506+0000 7f3bbffff700 1 --2- 172.21.15.2:0/756849700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f59be07a00 0x55f59be13910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:55.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.506+0000 7f3bbffff700 1 -- 172.21.15.2:0/756849700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59be0cfe0 msgr2=0x55f59be10c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:55.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.506+0000 7f3bbffff700 1 --2- 172.21.15.2:0/756849700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59be0cfe0 0x55f59be10c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.506+0000 7f3bbffff700 1 -- 172.21.15.2:0/756849700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f59be0d900 msgr2=0x55f59be112f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:55.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.506+0000 7f3bbffff700 1 --2- 172.21.15.2:0/756849700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f59be0d900 0x55f59be112f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.506+0000 7f3bbffff700 1 -- 172.21.15.2:0/756849700 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb8001b60 con 0x55f59be07a00 2023-09-13T04:48:55.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.507+0000 7f3bbffff700 1 --2- 172.21.15.2:0/756849700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f59be07a00 0x55f59be13910 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4005d90 tx=0x7f3bb4005d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df477c621f54133 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:55.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.507+0000 7f3b9f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:48:55.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.507+0000 7f3b9f7fe700 1 -- 172.21.15.2:0/756849700 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb401e440 con 0x55f59be07a00 2023-09-13T04:48:55.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.507+0000 7f3b9f7fe700 1 -- 172.21.15.2:0/756849700 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f3bb401e5a0 con 0x55f59be07a00 2023-09-13T04:48:55.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.508+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/756849700 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f59bdfa670 con 0x55f59be07a00 2023-09-13T04:48:55.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.508+0000 7f3b9f7fe700 1 -- 172.21.15.2:0/756849700 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bb401e760 con 0x55f59be07a00 2023-09-13T04:48:55.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.508+0000 7f3bd9ce4b80 10 client.26657 fetch_fsmap learned FSMap version 36 2023-09-13T04:48:55.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.508+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/756849700 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55f59bd717a0 con 0x55f59be07a00 2023-09-13T04:48:55.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.508+0000 7f3b9f7fe700 1 -- 172.21.15.2:0/756849700 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f59bdfa670 con 0x55f59be07a00 2023-09-13T04:48:55.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.509+0000 7f3b9f7fe700 1 -- 172.21.15.2:0/756849700 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f3bb40109d0 con 0x55f59be07a00 2023-09-13T04:48:55.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.509+0000 7f3bd9ce4b80 10 client.26657 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:48:55.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.509+0000 7f3bd9ce4b80 10 client.26657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:48:55.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.509+0000 7f3bd9ce4b80 20 client.26657 populate_metadata read hostname 'smithi002' 2023-09-13T04:48:55.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.509+0000 7f3bd9ce4b80 1 --2- 172.21.15.2:0/756849700 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f59be212f0 0x55f59be23710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:55.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.509+0000 7f3bd9ce4b80 4 client.26657 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:48:55.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.509+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/756849700 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f59bd52970 con 0x55f59be212f0 2023-09-13T04:48:55.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.509+0000 7f3bbf7fe700 1 --2- 172.21.15.2:0/756849700 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f59be212f0 0x55f59be23710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:55.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.513+0000 7f3bbf7fe700 1 --2- 172.21.15.2:0/756849700 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f59be212f0 0x55f59be23710 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=436144153ff5b8c8 server_cookie=a9a38c5a3a1b5b36 in_seq=0 out_seq=0 2023-09-13T04:48:55.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.513+0000 7f3b9f7fe700 10 client.26657 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:55.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.514+0000 7f3b9f7fe700 1 -- 172.21.15.2:0/756849700 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f59bd52970 con 0x55f59be212f0 2023-09-13T04:48:55.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.514+0000 7f3b9f7fe700 10 client.26657 handle_command_reply: tid=0 2023-09-13T04:48:55.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.515+0000 7f3bd9ce4b80 1 client.26657 shutdown 2023-09-13T04:48:55.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.515+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/756849700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f59be07a00 msgr2=0x55f59be13910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:55.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.515+0000 7f3bd9ce4b80 1 --2- 172.21.15.2:0/756849700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f59be07a00 0x55f59be13910 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4005d90 tx=0x7f3bb4005d20 comp rx=0 tx=0).stop 2023-09-13T04:48:55.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.516+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/756849700 shutdown_connections 2023-09-13T04:48:55.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.516+0000 7f3bd9ce4b80 1 --2- 172.21.15.2:0/756849700 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f59be212f0 0x55f59be23710 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.516+0000 7f3bd9ce4b80 1 --2- 172.21.15.2:0/756849700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f59be0d900 0x55f59be112f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.516+0000 7f3bd9ce4b80 1 --2- 172.21.15.2:0/756849700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f59be07a00 0x55f59be13910 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.516+0000 7f3bd9ce4b80 1 --2- 172.21.15.2:0/756849700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59be0cfe0 0x55f59be10c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.516+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/756849700 >> 172.21.15.2:0/756849700 conn(0x55f59bd65c80 msgr2=0x55f59bdfc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:55.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.516+0000 7f3b9f7fe700 0 client.26657 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:55.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.516+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/756849700 shutdown_connections 2023-09-13T04:48:55.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.517+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/756849700 wait complete. 2023-09-13T04:48:55.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.517+0000 7f3bd9ce4b80 20 client.26657 trim_cache size 0 max 16384 2023-09-13T04:48:55.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.553+0000 7f3bd9ce4b80 1 Processor -- start 2023-09-13T04:48:55.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.554+0000 7f3bd9ce4b80 1 -- start start 2023-09-13T04:48:55.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.554+0000 7f3bd9ce4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59bd65c80 0x55f59be182a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:55.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.554+0000 7f3bd9ce4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f59be07a00 0x55f59bdfc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:55.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.554+0000 7f3bd9ce4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f59be0cfe0 0x55f59be134b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:55.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.554+0000 7f3bd9ce4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59be024f0 con 0x55f59be07a00 2023-09-13T04:48:55.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.554+0000 7f3bd9ce4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59be1e870 con 0x55f59be0cfe0 2023-09-13T04:48:55.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.554+0000 7f3bd9ce4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59be187e0 con 0x55f59bd65c80 2023-09-13T04:48:55.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.555+0000 7f3bc499c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f59be0cfe0 0x55f59be134b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:55.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.555+0000 7f3bc499c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f59be0cfe0 0x55f59be134b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:59358/0 (socket says 172.21.15.2:59358) 2023-09-13T04:48:55.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.555+0000 7f3bc499c700 1 -- 172.21.15.2:0/4006476537 learned_addr learned my addr 172.21.15.2:0/4006476537 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:55.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.555+0000 7f3bbffff700 1 --2- 172.21.15.2:0/4006476537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59bd65c80 0x55f59be182a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:55.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.556+0000 7f3bc499c700 1 -- 172.21.15.2:0/4006476537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59bd65c80 msgr2=0x55f59be182a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:55.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.556+0000 7f3bc499c700 1 --2- 172.21.15.2:0/4006476537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59bd65c80 0x55f59be182a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.556+0000 7f3bc499c700 1 -- 172.21.15.2:0/4006476537 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f59be07a00 msgr2=0x55f59bdfc8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T04:48:55.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.556+0000 7f3bc499c700 1 --2- 172.21.15.2:0/4006476537 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f59be07a00 0x55f59bdfc8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.556+0000 7f3bc499c700 1 -- 172.21.15.2:0/4006476537 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f59be28320 con 0x55f59be0cfe0 2023-09-13T04:48:55.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.556+0000 7f3bc499c700 1 --2- 172.21.15.2:0/4006476537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f59be0cfe0 0x55f59be134b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8004270 tx=0x7f3bb800c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6965ca1d0ddaf7bd server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:55.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.568+0000 7f3ba6ffd700 1 -- 172.21.15.2:0/4006476537 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8016040 con 0x55f59be0cfe0 2023-09-13T04:48:55.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.568+0000 7f3ba6ffd700 1 -- 172.21.15.2:0/4006476537 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bb800ce50 con 0x55f59be0cfe0 2023-09-13T04:48:55.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.568+0000 7f3ba6ffd700 1 -- 172.21.15.2:0/4006476537 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb80256c0 con 0x55f59be0cfe0 2023-09-13T04:48:55.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.571+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/4006476537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f59be0cfe0 msgr2=0x55f59be134b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:55.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.571+0000 7f3bd9ce4b80 1 --2- 172.21.15.2:0/4006476537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f59be0cfe0 0x55f59be134b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8004270 tx=0x7f3bb800c520 comp rx=0 tx=0).stop 2023-09-13T04:48:55.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.571+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/4006476537 shutdown_connections 2023-09-13T04:48:55.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.571+0000 7f3bd9ce4b80 1 --2- 172.21.15.2:0/4006476537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f59be0cfe0 0x55f59be134b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.571+0000 7f3bd9ce4b80 1 --2- 172.21.15.2:0/4006476537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59bd65c80 0x55f59be182a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.571+0000 7f3bd9ce4b80 1 --2- 172.21.15.2:0/4006476537 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f59be07a00 0x55f59bdfc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.571+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/4006476537 >> 172.21.15.2:0/4006476537 conn(0x55f59be08140 msgr2=0x55f59be0ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:55.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.573+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/4006476537 shutdown_connections 2023-09-13T04:48:55.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.573+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/4006476537 wait complete. 2023-09-13T04:48:55.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.574+0000 7f3bd9ce4b80 1 -- start start 2023-09-13T04:48:55.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.574+0000 7f3bd9ce4b80 1 Processor -- start 2023-09-13T04:48:55.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.575+0000 7f3bd9ce4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59bd65c80 0x55f59be0a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:55.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.575+0000 7f3bd9ce4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f59be07a00 0x55f59be08a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:55.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.575+0000 7f3bd9ce4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f59be0cfe0 0x55f59be08fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:55.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.575+0000 7f3bd9ce4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59be09520 con 0x55f59be0cfe0 2023-09-13T04:48:55.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.575+0000 7f3bd9ce4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59be09690 con 0x55f59be07a00 2023-09-13T04:48:55.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.575+0000 7f3bd9ce4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59be09850 con 0x55f59bd65c80 2023-09-13T04:48:55.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.575+0000 7f3bd9ce4b80 10 client.0 _ll_register_callbacks cb 0x55f59bcf5680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:48:55.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.576+0000 7f3bbffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59bd65c80 0x55f59be0a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:55.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.576+0000 7f3bbffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59bd65c80 0x55f59be0a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:38446/0 (socket says 172.21.15.2:38446) 2023-09-13T04:48:55.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.576+0000 7f3bbffff700 1 -- 172.21.15.2:0/2781136114 learned_addr learned my addr 172.21.15.2:0/2781136114 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:55.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.576+0000 7f3bbf7fe700 1 --2- 172.21.15.2:0/2781136114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f59be07a00 0x55f59be08a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:55.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.577+0000 7f3bbffff700 1 -- 172.21.15.2:0/2781136114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f59be07a00 msgr2=0x55f59be08a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:55.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.577+0000 7f3bbffff700 1 --2- 172.21.15.2:0/2781136114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f59be07a00 0x55f59be08a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.577+0000 7f3bbffff700 1 -- 172.21.15.2:0/2781136114 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f59be0cfe0 msgr2=0x55f59be08fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:55.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.577+0000 7f3bbffff700 1 --2- 172.21.15.2:0/2781136114 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f59be0cfe0 0x55f59be08fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.577+0000 7f3bbffff700 1 -- 172.21.15.2:0/2781136114 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f59bbd9630 con 0x55f59bd65c80 2023-09-13T04:48:55.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.577+0000 7f3bbffff700 1 --2- 172.21.15.2:0/2781136114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59bd65c80 0x55f59be0a210 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4010fd0 tx=0x7f3bb4011ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cb1d5964bb42017 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:55.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.578+0000 7f3b9f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:48:55.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.578+0000 7f3b9f7fe700 1 -- 172.21.15.2:0/2781136114 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb40105d0 con 0x55f59bd65c80 2023-09-13T04:48:55.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.578+0000 7f3b9f7fe700 1 -- 172.21.15.2:0/2781136114 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f3bb4010730 con 0x55f59bd65c80 2023-09-13T04:48:55.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.578+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/2781136114 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3bb8003a00 con 0x55f59bd65c80 2023-09-13T04:48:55.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.579+0000 7f3b9f7fe700 1 -- 172.21.15.2:0/2781136114 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bb40108f0 con 0x55f59bd65c80 2023-09-13T04:48:55.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.582+0000 7f3bd9ce4b80 10 client.26686 fetch_fsmap learned FSMap version 36 2023-09-13T04:48:55.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.582+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/2781136114 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55f59bb85de0 con 0x55f59bd65c80 2023-09-13T04:48:55.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.586+0000 7f3b9f7fe700 1 -- 172.21.15.2:0/2781136114 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bb8003a00 con 0x55f59bd65c80 2023-09-13T04:48:55.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.586+0000 7f3b9f7fe700 1 -- 172.21.15.2:0/2781136114 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f3bb4045470 con 0x55f59bd65c80 2023-09-13T04:48:55.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.586+0000 7f3bd9ce4b80 10 client.26686 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:48:55.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.586+0000 7f3bd9ce4b80 10 client.26686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:48:55.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.586+0000 7f3bd9ce4b80 20 client.26686 populate_metadata read hostname 'smithi002' 2023-09-13T04:48:55.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.586+0000 7f3bd9ce4b80 1 --2- 172.21.15.2:0/2781136114 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f59be0d900 0x55f59be78640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:55.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.586+0000 7f3bd9ce4b80 4 client.26686 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-09-13T04:48:55.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.586+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/2781136114 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f59bd13050 con 0x55f59be0d900 2023-09-13T04:48:55.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.587+0000 7f3bbf7fe700 1 --2- 172.21.15.2:0/2781136114 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f59be0d900 0x55f59be78640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:55.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.589+0000 7f3bbf7fe700 1 --2- 172.21.15.2:0/2781136114 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f59be0d900 0x55f59be78640 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b68cce97526e06bc server_cookie=455139d91a16116d in_seq=0 out_seq=0 2023-09-13T04:48:55.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.590+0000 7f3b9f7fe700 10 client.26686 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:55.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.590+0000 7f3b9f7fe700 1 -- 172.21.15.2:0/2781136114 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f59bd13050 con 0x55f59be0d900 2023-09-13T04:48:55.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.590+0000 7f3b9f7fe700 10 client.26686 handle_command_reply: tid=0 2023-09-13T04:48:55.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.591+0000 7f3bd9ce4b80 1 client.26686 shutdown 2023-09-13T04:48:55.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.593+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/2781136114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59bd65c80 msgr2=0x55f59be0a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:55.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.593+0000 7f3bd9ce4b80 1 --2- 172.21.15.2:0/2781136114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59bd65c80 0x55f59be0a210 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4010fd0 tx=0x7f3bb4011ec0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.594+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/2781136114 shutdown_connections 2023-09-13T04:48:55.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.594+0000 7f3bd9ce4b80 1 --2- 172.21.15.2:0/2781136114 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55f59be0d900 0x55f59be78640 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.594+0000 7f3bd9ce4b80 1 --2- 172.21.15.2:0/2781136114 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f59be0cfe0 0x55f59be08fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.594+0000 7f3bd9ce4b80 1 --2- 172.21.15.2:0/2781136114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59bd65c80 0x55f59be0a210 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.594+0000 7f3bd9ce4b80 1 --2- 172.21.15.2:0/2781136114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f59be07a00 0x55f59be08a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.594+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/2781136114 >> 172.21.15.2:0/2781136114 conn(0x55f59be08140 msgr2=0x55f59be0dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:55.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.594+0000 7f3b9f7fe700 0 client.26686 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:55.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.595+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/2781136114 shutdown_connections 2023-09-13T04:48:55.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.595+0000 7f3bd9ce4b80 1 -- 172.21.15.2:0/2781136114 wait complete. 2023-09-13T04:48:55.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.595+0000 7f3bd9ce4b80 20 client.26686 trim_cache size 0 max 16384 2023-09-13T04:48:55.600 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:48:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2023-09-13T04:48:55.601 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2023-09-13T04:48:55.601 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:48:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.603+0000 7f3bc63ff700 1 -- 172.21.15.2:0/1589796019 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f3ba80a0870 msgr2=0x7f3ba80a2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.603+0000 7f3bc63ff700 1 --2- 172.21.15.2:0/1589796019 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f3ba80a0870 0x7f3ba80a2d30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4005fd0 tx=0x7f3bb4005f00 comp rx=0 tx=0).stop 2023-09-13T04:48:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.603+0000 7f3bc63ff700 1 -- 172.21.15.2:0/1589796019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bc010f440 msgr2=0x7f3bc01abfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.603+0000 7f3bc63ff700 1 --2- 172.21.15.2:0/1589796019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bc010f440 0x7f3bc01abfb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0003b70 tx=0x7f3bb00037c0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.604+0000 7f3bc63ff700 1 -- 172.21.15.2:0/1589796019 shutdown_connections 2023-09-13T04:48:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.604+0000 7f3bc63ff700 1 --2- 172.21.15.2:0/1589796019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bc011a490 0x7f3bc01a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.604+0000 7f3bc63ff700 1 --2- 172.21.15.2:0/1589796019 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f3ba80a0870 0x7f3ba80a2d30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.604+0000 7f3bc63ff700 1 --2- 172.21.15.2:0/1589796019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bc01081f0 0x7f3bc01aba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.604+0000 7f3bc63ff700 1 --2- 172.21.15.2:0/1589796019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bc010f440 0x7f3bc01abfb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.604+0000 7f3bc63ff700 1 -- 172.21.15.2:0/1589796019 >> 172.21.15.2:0/1589796019 conn(0x7f3bc006cfa0 msgr2=0x7f3bc0118480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.604+0000 7f3bc63ff700 1 -- 172.21.15.2:0/1589796019 shutdown_connections 2023-09-13T04:48:55.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:55.605+0000 7f3bc63ff700 1 -- 172.21.15.2:0/1589796019 wait complete. 2023-09-13T04:48:55.648 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-13T04:48:55.648 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:48:56.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.052+0000 7f53825ff700 1 -- 172.21.15.2:0/1869091171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f537c11a350 msgr2=0x7f537c11c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:56.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.052+0000 7f53825ff700 1 --2- 172.21.15.2:0/1869091171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f537c11a350 0x7f537c11c740 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f537400b0a0 tx=0x7f537400b3b0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.052+0000 7f53825ff700 1 -- 172.21.15.2:0/1869091171 shutdown_connections 2023-09-13T04:48:56.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.052+0000 7f53825ff700 1 --2- 172.21.15.2:0/1869091171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f537c11a350 0x7f537c11c740 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.052+0000 7f53825ff700 1 --2- 172.21.15.2:0/1869091171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f537c06c460 0x7f537c06c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.052+0000 7f53825ff700 1 --2- 172.21.15.2:0/1869091171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f537c06cd80 0x7f537c119e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.052+0000 7f53825ff700 1 -- 172.21.15.2:0/1869091171 >> 172.21.15.2:0/1869091171 conn(0x7f537c06bb70 msgr2=0x7f537c109cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:56.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.060+0000 7f53825ff700 1 -- 172.21.15.2:0/1869091171 shutdown_connections 2023-09-13T04:48:56.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.060+0000 7f53825ff700 1 -- 172.21.15.2:0/1869091171 wait complete. 2023-09-13T04:48:56.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.062+0000 7f53825ff700 1 Processor -- start 2023-09-13T04:48:56.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.062+0000 7f53825ff700 1 -- start start 2023-09-13T04:48:56.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.062+0000 7f53825ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f537c06c460 0x7f537c1aba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.062+0000 7f53825ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f537c06cd80 0x7f537c1abf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.062+0000 7f53825ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f537c11a350 0x7f537c1a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.062+0000 7f53825ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c1a6100 con 0x7f537c11a350 2023-09-13T04:48:56.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.062+0000 7f53825ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c1a6270 con 0x7f537c06cd80 2023-09-13T04:48:56.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.062+0000 7f53825ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c1a6430 con 0x7f537c06c460 2023-09-13T04:48:56.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.063+0000 7f53737fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f537c06cd80 0x7f537c1abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.063+0000 7f53737fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f537c06cd80 0x7f537c1abf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:59384/0 (socket says 172.21.15.2:59384) 2023-09-13T04:48:56.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.063+0000 7f53737fe700 1 -- 172.21.15.2:0/1435476030 learned_addr learned my addr 172.21.15.2:0/1435476030 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:56.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.063+0000 7f5380b9c700 1 --2- 172.21.15.2:0/1435476030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f537c11a350 0x7f537c1a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.063+0000 7f53737fe700 1 -- 172.21.15.2:0/1435476030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f537c06c460 msgr2=0x7f537c1aba20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-13T04:48:56.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.063+0000 7f53737fe700 1 --2- 172.21.15.2:0/1435476030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f537c06c460 0x7f537c1aba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.063+0000 7f53737fe700 1 -- 172.21.15.2:0/1435476030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f537c11a350 msgr2=0x7f537c1a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:56.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.063+0000 7f53737fe700 1 --2- 172.21.15.2:0/1435476030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f537c11a350 0x7f537c1a5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.063+0000 7f53737fe700 1 -- 172.21.15.2:0/1435476030 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5374009d00 con 0x7f537c06cd80 2023-09-13T04:48:56.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.064+0000 7f53737fe700 1 --2- 172.21.15.2:0/1435476030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f537c06cd80 0x7f537c1abf60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f536c00b550 tx=0x7f536c00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:56.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.064+0000 7f5379ffb700 1 -- 172.21.15.2:0/1435476030 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c011830 con 0x7f537c06cd80 2023-09-13T04:48:56.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.064+0000 7f5379ffb700 1 -- 172.21.15.2:0/1435476030 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f536c011e50 con 0x7f537c06cd80 2023-09-13T04:48:56.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.064+0000 7f53825ff700 1 -- 172.21.15.2:0/1435476030 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f537c1a6710 con 0x7f537c06cd80 2023-09-13T04:48:56.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.065+0000 7f5379ffb700 1 -- 172.21.15.2:0/1435476030 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c00f2e0 con 0x7f537c06cd80 2023-09-13T04:48:56.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.065+0000 7f53825ff700 1 -- 172.21.15.2:0/1435476030 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f537c1b2a10 con 0x7f537c06cd80 2023-09-13T04:48:56.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.067+0000 7f53825ff700 1 -- 172.21.15.2:0/1435476030 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f537c067ad0 con 0x7f537c06cd80 2023-09-13T04:48:56.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.070+0000 7f5379ffb700 1 -- 172.21.15.2:0/1435476030 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f536c0105f0 con 0x7f537c06cd80 2023-09-13T04:48:56.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.072+0000 7f5379ffb700 1 --2- 172.21.15.2:0/1435476030 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f53680a0930 0x7f53680a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.073+0000 7f5379ffb700 1 -- 172.21.15.2:0/1435476030 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f536c0cbda0 con 0x7f537c06cd80 2023-09-13T04:48:56.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.076+0000 7f537bfff700 1 --2- 172.21.15.2:0/1435476030 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f53680a0930 0x7f53680a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.077+0000 7f5379ffb700 1 -- 172.21.15.2:0/1435476030 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f536c0952d0 con 0x7f537c06cd80 2023-09-13T04:48:56.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.085+0000 7f537bfff700 1 --2- 172.21.15.2:0/1435476030 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f53680a0930 0x7f53680a2df0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5364009730 tx=0x7f5364006cb0 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:56.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:56 smithi002 ceph-mon[97651]: pgmap v261: 129 pgs: 129 active+clean; 1009 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 401 op/s 2023-09-13T04:48:56.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.349+0000 7f53825ff700 1 -- 172.21.15.2:0/1435476030 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f537c11fba0 con 0x7f537c06cd80 2023-09-13T04:48:56.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.350+0000 7f5379ffb700 1 -- 172.21.15.2:0/1435476030 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+11316 (secure 0 0 0) 0x7f536c015090 con 0x7f537c06cd80 2023-09-13T04:48:56.351 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:48:56.351 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2469307307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2469307307},{"type":"v1","addr":"172.21.15.2:6839","nonce":2469307307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T04:37:59.110701+0000","modified":"2023-09-13T04:47:02.884864+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24554,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/846579077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":846579077},{"type":"v1","addr":"172.21.15.156:6835","nonce":846579077}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1896833690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1896833690},{"type":"v1","addr":"172.21.15.149:6837","nonce":1896833690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6839/1785833227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1785833227},{"type":"v1","addr":"172.21.15.156:6839","nonce":1785833227}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":136,"addr":"172.21.15.2:6841/482545391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":482545391},{"type":"v1","addr":"172.21.15.2:6841","nonce":482545391}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/389167841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":389167841},{"type":"v1","addr":"172.21.15.149:6839","nonce":389167841}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26149":{"gid":26149,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/2098926245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2098926245},{"type":"v1","addr":"172.21.15.156:6837","nonce":2098926245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T04:48:56.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.353+0000 7f53825ff700 1 -- 172.21.15.2:0/1435476030 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f53680a0930 msgr2=0x7f53680a2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:56.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.353+0000 7f53825ff700 1 --2- 172.21.15.2:0/1435476030 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f53680a0930 0x7f53680a2df0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5364009730 tx=0x7f5364006cb0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.353+0000 7f53825ff700 1 -- 172.21.15.2:0/1435476030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f537c06cd80 msgr2=0x7f537c1abf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:56.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.353+0000 7f53825ff700 1 --2- 172.21.15.2:0/1435476030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f537c06cd80 0x7f537c1abf60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f536c00b550 tx=0x7f536c00b910 comp rx=0 tx=0).stop 2023-09-13T04:48:56.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.353+0000 7f53825ff700 1 -- 172.21.15.2:0/1435476030 shutdown_connections 2023-09-13T04:48:56.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.353+0000 7f53825ff700 1 --2- 172.21.15.2:0/1435476030 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f53680a0930 0x7f53680a2df0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.353+0000 7f53825ff700 1 --2- 172.21.15.2:0/1435476030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f537c11a350 0x7f537c1a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.353+0000 7f53825ff700 1 --2- 172.21.15.2:0/1435476030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f537c06c460 0x7f537c1aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.353+0000 7f53825ff700 1 --2- 172.21.15.2:0/1435476030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f537c06cd80 0x7f537c1abf60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.353+0000 7f53825ff700 1 -- 172.21.15.2:0/1435476030 >> 172.21.15.2:0/1435476030 conn(0x7f537c06bb70 msgr2=0x7f537c11b650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:56.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.353+0000 7f53825ff700 1 -- 172.21.15.2:0/1435476030 shutdown_connections 2023-09-13T04:48:56.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.353+0000 7f53825ff700 1 -- 172.21.15.2:0/1435476030 wait complete. 2023-09-13T04:48:56.356 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 36 2023-09-13T04:48:56.372 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-13T04:48:56.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:56 smithi149 ceph-mon[133315]: pgmap v261: 129 pgs: 129 active+clean; 1009 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 401 op/s 2023-09-13T04:48:56.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:56 smithi156 ceph-mon[133675]: pgmap v261: 129 pgs: 129 active+clean; 1009 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 401 op/s 2023-09-13T04:48:56.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.646+0000 7f22ae9ff700 1 -- 172.21.15.2:0/3568304733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22a8105fd0 msgr2=0x7f22a810db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:56.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.646+0000 7f22ae9ff700 1 --2- 172.21.15.2:0/3568304733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22a8105fd0 0x7f22a810db50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f22a00089e0 tx=0x7f22a0008cf0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.646+0000 7f22ae9ff700 1 -- 172.21.15.2:0/3568304733 shutdown_connections 2023-09-13T04:48:56.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.646+0000 7f22ae9ff700 1 --2- 172.21.15.2:0/3568304733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22a8105fd0 0x7f22a810db50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.646+0000 7f22ae9ff700 1 --2- 172.21.15.2:0/3568304733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22a8100eb0 0x7f22a8101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.646+0000 7f22ae9ff700 1 --2- 172.21.15.2:0/3568304733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22a8101860 0x7f22a81058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.646+0000 7f22ae9ff700 1 -- 172.21.15.2:0/3568304733 >> 172.21.15.2:0/3568304733 conn(0x7f22a80fc750 msgr2=0x7f22a80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:56.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.646+0000 7f22ae9ff700 1 -- 172.21.15.2:0/3568304733 shutdown_connections 2023-09-13T04:48:56.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.646+0000 7f22ae9ff700 1 -- 172.21.15.2:0/3568304733 wait complete. 2023-09-13T04:48:56.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.647+0000 7f22ae9ff700 1 Processor -- start 2023-09-13T04:48:56.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.648+0000 7f22ae9ff700 1 -- start start 2023-09-13T04:48:56.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.648+0000 7f22ae9ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22a8100eb0 0x7f22a819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.648+0000 7f22ae9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22a8105fd0 0x7f22a819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.648+0000 7f22ae9ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22a8194920 0x7f22a8194dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.648+0000 7f22ae9ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a81953a0 con 0x7f22a8194920 2023-09-13T04:48:56.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.648+0000 7f22ae9ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a81954e0 con 0x7f22a8100eb0 2023-09-13T04:48:56.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.648+0000 7f22ae9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a81956a0 con 0x7f22a8105fd0 2023-09-13T04:48:56.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.648+0000 7f22a77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22a8105fd0 0x7f22a819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.648+0000 7f22a77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22a8105fd0 0x7f22a819adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:38474/0 (socket says 172.21.15.2:38474) 2023-09-13T04:48:56.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.648+0000 7f22a77fe700 1 -- 172.21.15.2:0/288723131 learned_addr learned my addr 172.21.15.2:0/288723131 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:56.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.649+0000 7f22a7fff700 1 --2- 172.21.15.2:0/288723131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22a8100eb0 0x7f22a819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.649+0000 7f22acf9c700 1 --2- 172.21.15.2:0/288723131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22a8194920 0x7f22a8194dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.649+0000 7f22a77fe700 1 -- 172.21.15.2:0/288723131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22a8100eb0 msgr2=0x7f22a819a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:56.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.649+0000 7f22a77fe700 1 --2- 172.21.15.2:0/288723131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22a8100eb0 0x7f22a819a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.649+0000 7f22a77fe700 1 -- 172.21.15.2:0/288723131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22a8194920 msgr2=0x7f22a8194dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:56.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.649+0000 7f22a77fe700 1 --2- 172.21.15.2:0/288723131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22a8194920 0x7f22a8194dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.649+0000 7f22a77fe700 1 -- 172.21.15.2:0/288723131 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f229c009520 con 0x7f22a8105fd0 2023-09-13T04:48:56.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.649+0000 7f22acf9c700 1 --2- 172.21.15.2:0/288723131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22a8194920 0x7f22a8194dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:48:56.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.649+0000 7f22a77fe700 1 --2- 172.21.15.2:0/288723131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22a8105fd0 0x7f22a819adf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f229c00ea30 tx=0x7f229c00ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:56.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.649+0000 7f22a57fa700 1 -- 172.21.15.2:0/288723131 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c009820 con 0x7f22a8105fd0 2023-09-13T04:48:56.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.649+0000 7f22a57fa700 1 -- 172.21.15.2:0/288723131 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f229c0044e0 con 0x7f22a8105fd0 2023-09-13T04:48:56.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.649+0000 7f22ae9ff700 1 -- 172.21.15.2:0/288723131 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a0008610 con 0x7f22a8105fd0 2023-09-13T04:48:56.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.650+0000 7f22a57fa700 1 -- 172.21.15.2:0/288723131 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c003dc0 con 0x7f22a8105fd0 2023-09-13T04:48:56.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.650+0000 7f22ae9ff700 1 -- 172.21.15.2:0/288723131 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a81a1af0 con 0x7f22a8105fd0 2023-09-13T04:48:56.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.653+0000 7f22a57fa700 1 -- 172.21.15.2:0/288723131 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f229c003f80 con 0x7f22a8105fd0 2023-09-13T04:48:56.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.655+0000 7f22a57fa700 1 --2- 172.21.15.2:0/288723131 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f22900a0830 0x7f22900a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.655+0000 7f22a57fa700 1 -- 172.21.15.2:0/288723131 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f229c014070 con 0x7f22a8105fd0 2023-09-13T04:48:56.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.655+0000 7f22a7fff700 1 --2- 172.21.15.2:0/288723131 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f22900a0830 0x7f22900a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.656+0000 7f22a7fff700 1 --2- 172.21.15.2:0/288723131 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f22900a0830 0x7f22900a2cf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2298004510 tx=0x7f2298013040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:56.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.656+0000 7f22c241bb80 1 Processor -- start 2023-09-13T04:48:56.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.656+0000 7f22c241bb80 1 -- start start 2023-09-13T04:48:56.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.657+0000 7f22c241bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55558bba9fe0 0x55558bbaa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.657+0000 7f22c241bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55558bbaa900 0x55558bba44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.657+0000 7f22c241bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba4a00 0x55558bba4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.657+0000 7f22c241bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558bafd780 con 0x55558bbaa900 2023-09-13T04:48:56.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.657+0000 7f22c241bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558bafb0a0 con 0x55558bba9fe0 2023-09-13T04:48:56.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.657+0000 7f22c241bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558baf6990 con 0x55558bba4a00 2023-09-13T04:48:56.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.657+0000 7f22a77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55558bbaa900 0x55558bba44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.657+0000 7f22a77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55558bbaa900 0x55558bba44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42998/0 (socket says 172.21.15.2:42998) 2023-09-13T04:48:56.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.657+0000 7f22a77fe700 1 -- 172.21.15.2:0/1499627866 learned_addr learned my addr 172.21.15.2:0/1499627866 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:56.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.657+0000 7f22acf9c700 1 --2- 172.21.15.2:0/1499627866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba4a00 0x55558bba4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.657+0000 7f22a7fff700 1 --2- 172.21.15.2:0/1499627866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55558bba9fe0 0x55558bbaa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.658+0000 7f22a77fe700 1 -- 172.21.15.2:0/1499627866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba4a00 msgr2=0x55558bba4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:56.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.658+0000 7f22a77fe700 1 --2- 172.21.15.2:0/1499627866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba4a00 0x55558bba4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.658+0000 7f22a77fe700 1 -- 172.21.15.2:0/1499627866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55558bba9fe0 msgr2=0x55558bbaa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:56.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.658+0000 7f22a77fe700 1 --2- 172.21.15.2:0/1499627866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55558bba9fe0 0x55558bbaa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.658+0000 7f22a77fe700 1 -- 172.21.15.2:0/1499627866 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55558b976630 con 0x55558bbaa900 2023-09-13T04:48:56.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.658+0000 7f22acf9c700 1 --2- 172.21.15.2:0/1499627866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba4a00 0x55558bba4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:48:56.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.658+0000 7f22a77fe700 1 --2- 172.21.15.2:0/1499627866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55558bbaa900 0x55558bba44c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f229c0a12a0 tx=0x7f229c099190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3aefb127b18bce server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:56.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.658+0000 7f228e7fc700 1 -- 172.21.15.2:0/1499627866 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c099760 con 0x55558bbaa900 2023-09-13T04:48:56.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.658+0000 7f228e7fc700 1 -- 172.21.15.2:0/1499627866 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f229c09d590 con 0x55558bbaa900 2023-09-13T04:48:56.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.658+0000 7f228e7fc700 1 -- 172.21.15.2:0/1499627866 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c004230 con 0x55558bbaa900 2023-09-13T04:48:56.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.659+0000 7f22c241bb80 1 -- 172.21.15.2:0/1499627866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55558bbaa900 msgr2=0x55558bba44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:56.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.659+0000 7f22c241bb80 1 --2- 172.21.15.2:0/1499627866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55558bbaa900 0x55558bba44c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f229c0a12a0 tx=0x7f229c099190 comp rx=0 tx=0).stop 2023-09-13T04:48:56.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.660+0000 7f22c241bb80 1 -- 172.21.15.2:0/1499627866 shutdown_connections 2023-09-13T04:48:56.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.660+0000 7f22c241bb80 1 --2- 172.21.15.2:0/1499627866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba4a00 0x55558bba4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.660+0000 7f22c241bb80 1 --2- 172.21.15.2:0/1499627866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55558bba9fe0 0x55558bbaa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.660+0000 7f22c241bb80 1 --2- 172.21.15.2:0/1499627866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55558bbaa900 0x55558bba44c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.660+0000 7f22c241bb80 1 -- 172.21.15.2:0/1499627866 >> 172.21.15.2:0/1499627866 conn(0x55558bb02c80 msgr2=0x55558bb9ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:56.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.660+0000 7f22c241bb80 1 -- 172.21.15.2:0/1499627866 shutdown_connections 2023-09-13T04:48:56.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.660+0000 7f22c241bb80 1 -- 172.21.15.2:0/1499627866 wait complete. 2023-09-13T04:48:56.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.660+0000 7f22c241bb80 1 -- start start 2023-09-13T04:48:56.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.661+0000 7f22c241bb80 1 Processor -- start 2023-09-13T04:48:56.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.661+0000 7f22c241bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55558bba4a00 0x55558bbadc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.661+0000 7f22c241bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba9fe0 0x55558bbae180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.661+0000 7f22c241bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55558bbaa900 0x55558bbae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.661+0000 7f22c241bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558ba94bb0 con 0x55558bba4a00 2023-09-13T04:48:56.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.661+0000 7f22c241bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558ba948b0 con 0x55558bbaa900 2023-09-13T04:48:56.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.661+0000 7f22c241bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558b978920 con 0x55558bba9fe0 2023-09-13T04:48:56.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.662+0000 7f22c241bb80 10 client.0 _ll_register_callbacks cb 0x55558ba92680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:48:56.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.662+0000 7f22a77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba9fe0 0x55558bbae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.662+0000 7f22a7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55558bba4a00 0x55558bbadc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.662+0000 7f22acf9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55558bbaa900 0x55558bbae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.662+0000 7f22a77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba9fe0 0x55558bbae180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:38498/0 (socket says 172.21.15.2:38498) 2023-09-13T04:48:56.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.662+0000 7f22a77fe700 1 -- 172.21.15.2:0/83131093 learned_addr learned my addr 172.21.15.2:0/83131093 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:56.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.662+0000 7f22a77fe700 1 -- 172.21.15.2:0/83131093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55558bbaa900 msgr2=0x55558bbae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:56.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.662+0000 7f22a77fe700 1 --2- 172.21.15.2:0/83131093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55558bbaa900 0x55558bbae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.662+0000 7f22a77fe700 1 -- 172.21.15.2:0/83131093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55558bba4a00 msgr2=0x55558bbadc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:56.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.662+0000 7f22a77fe700 1 --2- 172.21.15.2:0/83131093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55558bba4a00 0x55558bbadc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.662+0000 7f22a77fe700 1 -- 172.21.15.2:0/83131093 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a81a1af0 con 0x55558bba9fe0 2023-09-13T04:48:56.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.662+0000 7f22a77fe700 1 --2- 172.21.15.2:0/83131093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba9fe0 0x55558bbae180 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f229c0961d0 tx=0x7f229c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55cc0c2337330b5e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:56.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.662+0000 7f22877fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:48:56.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.663+0000 7f22877fe700 1 -- 172.21.15.2:0/83131093 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c0a17d0 con 0x55558bba9fe0 2023-09-13T04:48:56.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.663+0000 7f22877fe700 1 -- 172.21.15.2:0/83131093 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f229c099760 con 0x55558bba9fe0 2023-09-13T04:48:56.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.663+0000 7f22c241bb80 1 -- 172.21.15.2:0/83131093 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55558bb97670 con 0x55558bba9fe0 2023-09-13T04:48:56.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.663+0000 7f22877fe700 1 -- 172.21.15.2:0/83131093 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f229c097b00 con 0x55558bba9fe0 2023-09-13T04:48:56.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.663+0000 7f22c241bb80 10 client.26698 fetch_fsmap learned FSMap version 36 2023-09-13T04:48:56.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.663+0000 7f22c241bb80 1 -- 172.21.15.2:0/83131093 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55558bb0e7a0 con 0x55558bba9fe0 2023-09-13T04:48:56.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.663+0000 7f22877fe700 1 -- 172.21.15.2:0/83131093 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55558bb97670 con 0x55558bba9fe0 2023-09-13T04:48:56.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.663+0000 7f22877fe700 1 -- 172.21.15.2:0/83131093 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f229c0999c0 con 0x55558bba9fe0 2023-09-13T04:48:56.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.663+0000 7f22c241bb80 10 client.26698 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:48:56.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.663+0000 7f22c241bb80 10 client.26698 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-09-13T04:48:56.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.663+0000 7f22c241bb80 20 client.26698 populate_metadata read hostname 'smithi002' 2023-09-13T04:48:56.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.664+0000 7f22c241bb80 1 --2- 172.21.15.2:0/83131093 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x55558bbbde20 0x55558bbc0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.664+0000 7f22c241bb80 4 client.26698 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:48:56.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.664+0000 7f22c241bb80 1 -- 172.21.15.2:0/83131093 --> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55558baef970 con 0x55558bbbde20 2023-09-13T04:48:56.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.664+0000 7f22a7fff700 1 --2- 172.21.15.2:0/83131093 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x55558bbbde20 0x55558bbc0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.664+0000 7f22a7fff700 1 --2- 172.21.15.2:0/83131093 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x55558bbbde20 0x55558bbc0210 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46409adcddece7e5 server_cookie=de804468f418f612 in_seq=0 out_seq=0 2023-09-13T04:48:56.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.664+0000 7f22877fe700 10 client.26698 ms_handle_connect on v2:172.21.15.156:6838/1785833227 2023-09-13T04:48:56.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.665+0000 7f22877fe700 1 -- 172.21.15.2:0/83131093 <== mds.2 v2:172.21.15.156:6838/1785833227 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55558baef970 con 0x55558bbbde20 2023-09-13T04:48:56.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.665+0000 7f22877fe700 10 client.26698 handle_command_reply: tid=0 2023-09-13T04:48:56.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.665+0000 7f22c241bb80 1 client.26698 shutdown 2023-09-13T04:48:56.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.665+0000 7f22c241bb80 1 -- 172.21.15.2:0/83131093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba9fe0 msgr2=0x55558bbae180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:56.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.665+0000 7f22c241bb80 1 --2- 172.21.15.2:0/83131093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba9fe0 0x55558bbae180 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f229c0961d0 tx=0x7f229c000bc0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.665+0000 7f22c241bb80 1 -- 172.21.15.2:0/83131093 shutdown_connections 2023-09-13T04:48:56.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.665+0000 7f22c241bb80 1 --2- 172.21.15.2:0/83131093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55558bbaa900 0x55558bbae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.665+0000 7f22c241bb80 1 --2- 172.21.15.2:0/83131093 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x55558bbbde20 0x55558bbc0210 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.665+0000 7f22c241bb80 1 --2- 172.21.15.2:0/83131093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55558bba4a00 0x55558bbadc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.665+0000 7f22c241bb80 1 --2- 172.21.15.2:0/83131093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba9fe0 0x55558bbae180 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.666+0000 7f22c241bb80 1 -- 172.21.15.2:0/83131093 >> 172.21.15.2:0/83131093 conn(0x55558bb02c80 msgr2=0x55558bb997f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:56.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.666+0000 7f22877fe700 0 client.26698 ms_handle_reset on v2:172.21.15.156:6838/1785833227 2023-09-13T04:48:56.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.666+0000 7f22c241bb80 1 -- 172.21.15.2:0/83131093 shutdown_connections 2023-09-13T04:48:56.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.666+0000 7f22c241bb80 1 -- 172.21.15.2:0/83131093 wait complete. 2023-09-13T04:48:56.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.666+0000 7f22c241bb80 20 client.26698 trim_cache size 0 max 16384 2023-09-13T04:48:56.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.683+0000 7f22c241bb80 1 Processor -- start 2023-09-13T04:48:56.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.683+0000 7f22c241bb80 1 -- start start 2023-09-13T04:48:56.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.684+0000 7f22c241bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba4a00 0x55558bba4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.684+0000 7f22c241bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55558bba9fe0 0x55558bbb5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.684+0000 7f22c241bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55558bbaa900 0x55558bb998f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.684+0000 7f22c241bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558bb9f4f0 con 0x55558bbaa900 2023-09-13T04:48:56.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.684+0000 7f22c241bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558bb9fde0 con 0x55558bba9fe0 2023-09-13T04:48:56.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.684+0000 7f22c241bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558bb97400 con 0x55558bba4a00 2023-09-13T04:48:56.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.684+0000 7f22a7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba4a00 0x55558bba4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.684+0000 7f22a7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba4a00 0x55558bba4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:38514/0 (socket says 172.21.15.2:38514) 2023-09-13T04:48:56.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.684+0000 7f22a7fff700 1 -- 172.21.15.2:0/3939966846 learned_addr learned my addr 172.21.15.2:0/3939966846 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:56.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.684+0000 7f22a77fe700 1 --2- 172.21.15.2:0/3939966846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55558bba9fe0 0x55558bbb5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.684+0000 7f22acf9c700 1 --2- 172.21.15.2:0/3939966846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55558bbaa900 0x55558bb998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.684+0000 7f22a7fff700 1 -- 172.21.15.2:0/3939966846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55558bba9fe0 msgr2=0x55558bbb5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:56.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.684+0000 7f22a7fff700 1 --2- 172.21.15.2:0/3939966846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55558bba9fe0 0x55558bbb5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.684+0000 7f22a7fff700 1 -- 172.21.15.2:0/3939966846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55558bbaa900 msgr2=0x55558bb998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:56.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.684+0000 7f22a7fff700 1 --2- 172.21.15.2:0/3939966846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55558bbaa900 0x55558bb998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.684+0000 7f22a7fff700 1 -- 172.21.15.2:0/3939966846 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55558b922de0 con 0x55558bba4a00 2023-09-13T04:48:56.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.685+0000 7f22a7fff700 1 --2- 172.21.15.2:0/3939966846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba4a00 0x55558bba4de0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55558bbc0b20 tx=0x7f229801c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b145185c99732343 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:56.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.685+0000 7f228effd700 1 -- 172.21.15.2:0/3939966846 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229801fc20 con 0x55558bba4a00 2023-09-13T04:48:56.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.685+0000 7f228effd700 1 -- 172.21.15.2:0/3939966846 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2298017040 con 0x55558bba4a00 2023-09-13T04:48:56.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.685+0000 7f228effd700 1 -- 172.21.15.2:0/3939966846 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2298032810 con 0x55558bba4a00 2023-09-13T04:48:56.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.686+0000 7f22c241bb80 1 -- 172.21.15.2:0/3939966846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba4a00 msgr2=0x55558bba4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:56.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.686+0000 7f22c241bb80 1 --2- 172.21.15.2:0/3939966846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba4a00 0x55558bba4de0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55558bbc0b20 tx=0x7f229801c000 comp rx=0 tx=0).stop 2023-09-13T04:48:56.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.686+0000 7f22c241bb80 1 -- 172.21.15.2:0/3939966846 shutdown_connections 2023-09-13T04:48:56.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.686+0000 7f22c241bb80 1 --2- 172.21.15.2:0/3939966846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55558bbaa900 0x55558bb998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.686+0000 7f22c241bb80 1 --2- 172.21.15.2:0/3939966846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba4a00 0x55558bba4de0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.686+0000 7f22c241bb80 1 --2- 172.21.15.2:0/3939966846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55558bba9fe0 0x55558bbb5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.686+0000 7f22c241bb80 1 -- 172.21.15.2:0/3939966846 >> 172.21.15.2:0/3939966846 conn(0x55558bb02c80 msgr2=0x55558bba7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:56.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.686+0000 7f22c241bb80 1 -- 172.21.15.2:0/3939966846 shutdown_connections 2023-09-13T04:48:56.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.686+0000 7f22c241bb80 1 -- 172.21.15.2:0/3939966846 wait complete. 2023-09-13T04:48:56.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.687+0000 7f22c241bb80 1 -- start start 2023-09-13T04:48:56.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.687+0000 7f22c241bb80 1 Processor -- start 2023-09-13T04:48:56.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.688+0000 7f22c241bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55558bba4a00 0x55558bbb3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.688+0000 7f22c241bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba9fe0 0x55558bbb3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.688+0000 7f22c241bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55558bbaa900 0x55558bbb3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.688+0000 7f22c241bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558bc10d10 con 0x55558bba4a00 2023-09-13T04:48:56.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.688+0000 7f22c241bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558bc10e80 con 0x55558bbaa900 2023-09-13T04:48:56.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.688+0000 7f22c241bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558bc11040 con 0x55558bba9fe0 2023-09-13T04:48:56.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.688+0000 7f22c241bb80 10 client.0 _ll_register_callbacks cb 0x55558ba92680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:48:56.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.688+0000 7f22a7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55558bba4a00 0x55558bbb3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.688+0000 7f22a77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba9fe0 0x55558bbb3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.688+0000 7f22a77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba9fe0 0x55558bbb3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:38526/0 (socket says 172.21.15.2:38526) 2023-09-13T04:48:56.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.688+0000 7f22a77fe700 1 -- 172.21.15.2:0/187169474 learned_addr learned my addr 172.21.15.2:0/187169474 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:56.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.688+0000 7f22acf9c700 1 --2- 172.21.15.2:0/187169474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55558bbaa900 0x55558bbb3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.688+0000 7f22a77fe700 1 -- 172.21.15.2:0/187169474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55558bbaa900 msgr2=0x55558bbb3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:56.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.688+0000 7f22a77fe700 1 --2- 172.21.15.2:0/187169474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55558bbaa900 0x55558bbb3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.688+0000 7f22a77fe700 1 -- 172.21.15.2:0/187169474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55558bba4a00 msgr2=0x55558bbb3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:56.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.688+0000 7f22a77fe700 1 --2- 172.21.15.2:0/187169474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55558bba4a00 0x55558bbb3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.689+0000 7f22a77fe700 1 -- 172.21.15.2:0/187169474 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22980109e0 con 0x55558bba9fe0 2023-09-13T04:48:56.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.689+0000 7f22a7fff700 1 --2- 172.21.15.2:0/187169474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55558bba4a00 0x55558bbb3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:48:56.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.689+0000 7f22a77fe700 1 --2- 172.21.15.2:0/187169474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba9fe0 0x55558bbb3740 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f229c0a12a0 tx=0x7f229c097d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31fdbc434aafac97 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:56.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.689+0000 7f22877fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:48:56.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.689+0000 7f22877fe700 1 -- 172.21.15.2:0/187169474 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c0990c0 con 0x55558bba9fe0 2023-09-13T04:48:56.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.689+0000 7f22877fe700 1 -- 172.21.15.2:0/187169474 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f229c099760 con 0x55558bba9fe0 2023-09-13T04:48:56.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.689+0000 7f22c241bb80 1 -- 172.21.15.2:0/187169474 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f229c09d590 con 0x55558bba9fe0 2023-09-13T04:48:56.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.689+0000 7f22877fe700 1 -- 172.21.15.2:0/187169474 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f229c0a17d0 con 0x55558bba9fe0 2023-09-13T04:48:56.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.690+0000 7f22c241bb80 10 client.26704 fetch_fsmap learned FSMap version 36 2023-09-13T04:48:56.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.690+0000 7f22c241bb80 1 -- 172.21.15.2:0/187169474 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55558b90bc60 con 0x55558bba9fe0 2023-09-13T04:48:56.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.690+0000 7f22877fe700 1 -- 172.21.15.2:0/187169474 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f229c09d590 con 0x55558bba9fe0 2023-09-13T04:48:56.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.690+0000 7f22877fe700 1 -- 172.21.15.2:0/187169474 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f229c0cc300 con 0x55558bba9fe0 2023-09-13T04:48:56.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.690+0000 7f22c241bb80 10 client.26704 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:48:56.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.690+0000 7f22c241bb80 10 client.26704 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-09-13T04:48:56.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.690+0000 7f22c241bb80 20 client.26704 populate_metadata read hostname 'smithi002' 2023-09-13T04:48:56.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.690+0000 7f22c241bb80 1 --2- 172.21.15.2:0/187169474 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x55558bc12290 0x55558bc14680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.690+0000 7f22c241bb80 4 client.26704 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-09-13T04:48:56.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.690+0000 7f22c241bb80 1 -- 172.21.15.2:0/187169474 --> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55558bab0050 con 0x55558bc12290 2023-09-13T04:48:56.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.690+0000 7f22a7fff700 1 --2- 172.21.15.2:0/187169474 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x55558bc12290 0x55558bc14680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.691+0000 7f22a7fff700 1 --2- 172.21.15.2:0/187169474 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x55558bc12290 0x55558bc14680 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5bd0d3f266d3c060 server_cookie=96cc6d50b30a72a4 in_seq=0 out_seq=0 2023-09-13T04:48:56.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.691+0000 7f22877fe700 10 client.26704 ms_handle_connect on v2:172.21.15.156:6838/1785833227 2023-09-13T04:48:56.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.691+0000 7f22877fe700 1 -- 172.21.15.2:0/187169474 <== mds.2 v2:172.21.15.156:6838/1785833227 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55558bab0050 con 0x55558bc12290 2023-09-13T04:48:56.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.691+0000 7f22877fe700 10 client.26704 handle_command_reply: tid=0 2023-09-13T04:48:56.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.691+0000 7f22c241bb80 1 client.26704 shutdown 2023-09-13T04:48:56.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.691+0000 7f22c241bb80 1 -- 172.21.15.2:0/187169474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba9fe0 msgr2=0x55558bbb3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:56.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.691+0000 7f22c241bb80 1 --2- 172.21.15.2:0/187169474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba9fe0 0x55558bbb3740 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f229c0a12a0 tx=0x7f229c097d10 comp rx=0 tx=0).stop 2023-09-13T04:48:56.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.692+0000 7f22c241bb80 1 -- 172.21.15.2:0/187169474 shutdown_connections 2023-09-13T04:48:56.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.692+0000 7f22c241bb80 1 --2- 172.21.15.2:0/187169474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55558bbaa900 0x55558bbb3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.692+0000 7f22c241bb80 1 --2- 172.21.15.2:0/187169474 >> [v2:172.21.15.156:6838/1785833227,v1:172.21.15.156:6839/1785833227] conn(0x55558bc12290 0x55558bc14680 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.692+0000 7f22c241bb80 1 --2- 172.21.15.2:0/187169474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55558bba4a00 0x55558bbb3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.692+0000 7f22c241bb80 1 --2- 172.21.15.2:0/187169474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55558bba9fe0 0x55558bbb3740 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.692+0000 7f22877fe700 0 client.26704 ms_handle_reset on v2:172.21.15.156:6838/1785833227 2023-09-13T04:48:56.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.692+0000 7f22c241bb80 1 -- 172.21.15.2:0/187169474 >> 172.21.15.2:0/187169474 conn(0x55558bb02c80 msgr2=0x55558bba8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:56.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.692+0000 7f22c241bb80 1 -- 172.21.15.2:0/187169474 shutdown_connections 2023-09-13T04:48:56.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.692+0000 7f22c241bb80 1 -- 172.21.15.2:0/187169474 wait complete. 2023-09-13T04:48:56.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.692+0000 7f22c241bb80 20 client.26704 trim_cache size 0 max 16384 2023-09-13T04:48:56.707 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-09-13T04:48:56.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.693+0000 7f22ae9ff700 1 -- 172.21.15.2:0/288723131 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f22900a0830 msgr2=0x7f22900a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:56.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.693+0000 7f22ae9ff700 1 --2- 172.21.15.2:0/288723131 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f22900a0830 0x7f22900a2cf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2298004510 tx=0x7f2298013040 comp rx=0 tx=0).stop 2023-09-13T04:48:56.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.693+0000 7f22ae9ff700 1 -- 172.21.15.2:0/288723131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22a8105fd0 msgr2=0x7f22a819adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:56.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.693+0000 7f22ae9ff700 1 --2- 172.21.15.2:0/288723131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22a8105fd0 0x7f22a819adf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f229c00ea30 tx=0x7f229c00ed40 comp rx=0 tx=0).stop 2023-09-13T04:48:56.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.693+0000 7f22ae9ff700 1 -- 172.21.15.2:0/288723131 shutdown_connections 2023-09-13T04:48:56.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.693+0000 7f22ae9ff700 1 --2- 172.21.15.2:0/288723131 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f22900a0830 0x7f22900a2cf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.693+0000 7f22ae9ff700 1 --2- 172.21.15.2:0/288723131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22a8194920 0x7f22a8194dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.693+0000 7f22ae9ff700 1 --2- 172.21.15.2:0/288723131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22a8100eb0 0x7f22a819a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.693+0000 7f22ae9ff700 1 --2- 172.21.15.2:0/288723131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22a8105fd0 0x7f22a819adf0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.693+0000 7f22ae9ff700 1 -- 172.21.15.2:0/288723131 >> 172.21.15.2:0/288723131 conn(0x7f22a80fc750 msgr2=0x7f22a80fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:56.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.693+0000 7f22ae9ff700 1 -- 172.21.15.2:0/288723131 shutdown_connections 2023-09-13T04:48:56.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.693+0000 7f22ae9ff700 1 -- 172.21.15.2:0/288723131 wait complete. 2023-09-13T04:48:56.718 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-13T04:48:56.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.929+0000 7f55859ff700 1 -- 172.21.15.2:0/3496222070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5580100eb0 msgr2=0x7f5580101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:56.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.929+0000 7f55859ff700 1 --2- 172.21.15.2:0/3496222070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5580100eb0 0x7f5580101290 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f55700098f0 tx=0x7f5570009c00 comp rx=0 tx=0).stop 2023-09-13T04:48:56.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.930+0000 7f55859ff700 1 -- 172.21.15.2:0/3496222070 shutdown_connections 2023-09-13T04:48:56.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.930+0000 7f55859ff700 1 --2- 172.21.15.2:0/3496222070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5580105fd0 0x7f558010db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.930+0000 7f55859ff700 1 --2- 172.21.15.2:0/3496222070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5580100eb0 0x7f5580101290 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.930+0000 7f55859ff700 1 --2- 172.21.15.2:0/3496222070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5580101860 0x7f55801058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.930+0000 7f55859ff700 1 -- 172.21.15.2:0/3496222070 >> 172.21.15.2:0/3496222070 conn(0x7f55800fc730 msgr2=0x7f55800feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:56.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.930+0000 7f55859ff700 1 -- 172.21.15.2:0/3496222070 shutdown_connections 2023-09-13T04:48:56.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.930+0000 7f55859ff700 1 -- 172.21.15.2:0/3496222070 wait complete. 2023-09-13T04:48:56.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.931+0000 7f55859ff700 1 Processor -- start 2023-09-13T04:48:56.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.931+0000 7f55859ff700 1 -- start start 2023-09-13T04:48:56.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.931+0000 7f55859ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5580100eb0 0x7f5580072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.931+0000 7f55859ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5580105fd0 0x7f55800734f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.931+0000 7f55859ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5580073a30 0x7f55801a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.931+0000 7f55859ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55801a5690 con 0x7f5580073a30 2023-09-13T04:48:56.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.931+0000 7f55859ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55801a5800 con 0x7f5580105fd0 2023-09-13T04:48:56.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.931+0000 7f55859ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55801a59c0 con 0x7f5580100eb0 2023-09-13T04:48:56.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.931+0000 7f557effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5580100eb0 0x7f5580072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.932+0000 7f557effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5580100eb0 0x7f5580072fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:38540/0 (socket says 172.21.15.2:38540) 2023-09-13T04:48:56.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.932+0000 7f557effd700 1 -- 172.21.15.2:0/860969259 learned_addr learned my addr 172.21.15.2:0/860969259 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:56.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.932+0000 7f5577fff700 1 --2- 172.21.15.2:0/860969259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5580105fd0 0x7f55800734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.932+0000 7f557f7fe700 1 --2- 172.21.15.2:0/860969259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5580073a30 0x7f55801a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.932+0000 7f5577fff700 1 -- 172.21.15.2:0/860969259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5580100eb0 msgr2=0x7f5580072fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:56.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.932+0000 7f5577fff700 1 --2- 172.21.15.2:0/860969259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5580100eb0 0x7f5580072fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.932+0000 7f5577fff700 1 -- 172.21.15.2:0/860969259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5580073a30 msgr2=0x7f55801a5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.932+0000 7f5577fff700 1 --2- 172.21.15.2:0/860969259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5580073a30 0x7f55801a5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.932+0000 7f5577fff700 1 -- 172.21.15.2:0/860969259 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5568009520 con 0x7f5580105fd0 2023-09-13T04:48:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.932+0000 7f5577fff700 1 --2- 172.21.15.2:0/860969259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5580105fd0 0x7f55800734f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f556800e9c0 tx=0x7f556800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.932+0000 7f55849fd700 1 -- 172.21.15.2:0/860969259 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556800cb70 con 0x7f5580105fd0 2023-09-13T04:48:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.932+0000 7f55859ff700 1 -- 172.21.15.2:0/860969259 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5578009d00 con 0x7f5580105fd0 2023-09-13T04:48:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.932+0000 7f55849fd700 1 -- 172.21.15.2:0/860969259 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f556800ee60 con 0x7f5580105fd0 2023-09-13T04:48:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.932+0000 7f55849fd700 1 -- 172.21.15.2:0/860969259 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5568003c60 con 0x7f5580105fd0 2023-09-13T04:48:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.933+0000 7f55859ff700 1 -- 172.21.15.2:0/860969259 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5570009520 con 0x7f5580105fd0 2023-09-13T04:48:56.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.936+0000 7f55849fd700 1 -- 172.21.15.2:0/860969259 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f5568003e00 con 0x7f5580105fd0 2023-09-13T04:48:56.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.937+0000 7f5599293b80 1 Processor -- start 2023-09-13T04:48:56.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.937+0000 7f5599293b80 1 -- start start 2023-09-13T04:48:56.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.937+0000 7f5599293b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c293f2ffe0 0x55c293f303c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.937+0000 7f5599293b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f30900 0x55c293f2a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.937+0000 7f5599293b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c293f2aa00 0x55c293f2ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.937+0000 7f5599293b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c293e83780 con 0x55c293f2aa00 2023-09-13T04:48:56.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.937+0000 7f5599293b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c293e810a0 con 0x55c293f30900 2023-09-13T04:48:56.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.937+0000 7f5599293b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c293e7c990 con 0x55c293f2ffe0 2023-09-13T04:48:56.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.938+0000 7f55849fd700 1 --2- 172.21.15.2:0/860969259 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f556c0a07b0 0x7f556c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.938+0000 7f5577fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c293f2aa00 0x55c293f2ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.938+0000 7f557effd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f30900 0x55c293f2a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.938+0000 7f557effd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f30900 0x55c293f2a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:59480/0 (socket says 172.21.15.2:59480) 2023-09-13T04:48:56.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.938+0000 7f557effd700 1 -- 172.21.15.2:0/2055702623 learned_addr learned my addr 172.21.15.2:0/2055702623 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:56.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.938+0000 7f55849fd700 1 -- 172.21.15.2:0/860969259 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f5568014070 con 0x7f5580105fd0 2023-09-13T04:48:56.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.938+0000 7f557f7fe700 1 --2- 172.21.15.2:0/2055702623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c293f2ffe0 0x55c293f303c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.938+0000 7f557f7fe700 1 --2- 172.21.15.2:0/860969259 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f556c0a07b0 0x7f556c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.938+0000 7f5577fff700 1 -- 172.21.15.2:0/2055702623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c293f2ffe0 msgr2=0x55c293f303c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:56.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.938+0000 7f5577fff700 1 --2- 172.21.15.2:0/2055702623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c293f2ffe0 0x55c293f303c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.938+0000 7f5577fff700 1 -- 172.21.15.2:0/2055702623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f30900 msgr2=0x55c293f2a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:56.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.938+0000 7f5577fff700 1 --2- 172.21.15.2:0/2055702623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f30900 0x55c293f2a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.938+0000 7f5577fff700 1 -- 172.21.15.2:0/2055702623 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c293cfc630 con 0x55c293f2aa00 2023-09-13T04:48:56.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.939+0000 7f5577fff700 1 --2- 172.21.15.2:0/2055702623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c293f2aa00 0x55c293f2ae60 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f55680c9970 tx=0x7f556806b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8b1ab61ae2e233 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:56.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.939+0000 7f557f7fe700 1 --2- 172.21.15.2:0/860969259 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f556c0a07b0 0x7f556c0a2c70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5578009fd0 tx=0x7f557800b040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:56.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.939+0000 7f5575ffb700 1 -- 172.21.15.2:0/2055702623 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5568065410 con 0x55c293f2aa00 2023-09-13T04:48:56.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.939+0000 7f5575ffb700 1 -- 172.21.15.2:0/2055702623 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5568099dd0 con 0x55c293f2aa00 2023-09-13T04:48:56.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.939+0000 7f5575ffb700 1 -- 172.21.15.2:0/2055702623 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5568064f70 con 0x55c293f2aa00 2023-09-13T04:48:56.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.939+0000 7f5599293b80 1 -- 172.21.15.2:0/2055702623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c293f2aa00 msgr2=0x55c293f2ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:56.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.939+0000 7f5599293b80 1 --2- 172.21.15.2:0/2055702623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c293f2aa00 0x55c293f2ae60 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f55680c9970 tx=0x7f556806b740 comp rx=0 tx=0).stop 2023-09-13T04:48:56.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.940+0000 7f5599293b80 1 -- 172.21.15.2:0/2055702623 shutdown_connections 2023-09-13T04:48:56.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.940+0000 7f5599293b80 1 --2- 172.21.15.2:0/2055702623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c293f2aa00 0x55c293f2ae60 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.940+0000 7f5599293b80 1 --2- 172.21.15.2:0/2055702623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c293f2ffe0 0x55c293f303c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.940+0000 7f5599293b80 1 --2- 172.21.15.2:0/2055702623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f30900 0x55c293f2a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.940+0000 7f5599293b80 1 -- 172.21.15.2:0/2055702623 >> 172.21.15.2:0/2055702623 conn(0x55c293e88c80 msgr2=0x55c293f24f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:56.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.940+0000 7f5599293b80 1 -- 172.21.15.2:0/2055702623 shutdown_connections 2023-09-13T04:48:56.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.940+0000 7f5599293b80 1 -- 172.21.15.2:0/2055702623 wait complete. 2023-09-13T04:48:56.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.940+0000 7f5599293b80 1 -- start start 2023-09-13T04:48:56.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.940+0000 7f5599293b80 1 Processor -- start 2023-09-13T04:48:56.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.941+0000 7f5599293b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c293f2aa00 0x55c293f33be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.941+0000 7f5599293b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f2ffe0 0x55c293f34120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.941+0000 7f5599293b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c293f30900 0x55c293f34660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.941+0000 7f5599293b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c293e1abb0 con 0x55c293f2aa00 2023-09-13T04:48:56.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.941+0000 7f5599293b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c293e1a8b0 con 0x55c293f2ffe0 2023-09-13T04:48:56.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.941+0000 7f5599293b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c293cfe920 con 0x55c293f30900 2023-09-13T04:48:56.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.941+0000 7f5599293b80 10 client.0 _ll_register_callbacks cb 0x55c293e18680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:48:56.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.941+0000 7f557effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c293f2aa00 0x55c293f33be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.941+0000 7f557effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c293f2aa00 0x55c293f33be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43058/0 (socket says 172.21.15.2:43058) 2023-09-13T04:48:56.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.941+0000 7f557effd700 1 -- 172.21.15.2:0/3580476601 learned_addr learned my addr 172.21.15.2:0/3580476601 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:56.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.941+0000 7f557f7fe700 1 --2- 172.21.15.2:0/3580476601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c293f30900 0x55c293f34660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.941+0000 7f5577fff700 1 --2- 172.21.15.2:0/3580476601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f2ffe0 0x55c293f34120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:56.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.941+0000 7f557effd700 1 -- 172.21.15.2:0/3580476601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c293f30900 msgr2=0x55c293f34660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:56.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.941+0000 7f557effd700 1 --2- 172.21.15.2:0/3580476601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c293f30900 0x55c293f34660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.941+0000 7f557effd700 1 -- 172.21.15.2:0/3580476601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f2ffe0 msgr2=0x55c293f34120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:56.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.941+0000 7f557effd700 1 --2- 172.21.15.2:0/3580476601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f2ffe0 0x55c293f34120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:56.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.941+0000 7f557effd700 1 -- 172.21.15.2:0/3580476601 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5570009520 con 0x55c293f2aa00 2023-09-13T04:48:56.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.941+0000 7f557f7fe700 1 --2- 172.21.15.2:0/3580476601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c293f30900 0x55c293f34660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:48:56.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.941+0000 7f5577fff700 1 --2- 172.21.15.2:0/3580476601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f2ffe0 0x55c293f34120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:48:56.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.942+0000 7f557effd700 1 --2- 172.21.15.2:0/3580476601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c293f2aa00 0x55c293f33be0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f557000f690 tx=0x7f557000f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7c56d068618c6e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:56.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.942+0000 7f555effd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:48:56.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.942+0000 7f555effd700 1 -- 172.21.15.2:0/3580476601 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557000b910 con 0x55c293f2aa00 2023-09-13T04:48:56.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.942+0000 7f5599293b80 1 -- 172.21.15.2:0/3580476601 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c293f1d670 con 0x55c293f2aa00 2023-09-13T04:48:56.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.942+0000 7f555effd700 1 -- 172.21.15.2:0/3580476601 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f557000ba70 con 0x55c293f2aa00 2023-09-13T04:48:56.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.942+0000 7f555effd700 1 -- 172.21.15.2:0/3580476601 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5570005040 con 0x55c293f2aa00 2023-09-13T04:48:56.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.942+0000 7f5599293b80 10 client.16530 fetch_fsmap learned FSMap version 36 2023-09-13T04:48:56.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.942+0000 7f5599293b80 1 -- 172.21.15.2:0/3580476601 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55c293e947a0 con 0x55c293f2aa00 2023-09-13T04:48:56.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.943+0000 7f555effd700 1 -- 172.21.15.2:0/3580476601 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c293f1d670 con 0x55c293f2aa00 2023-09-13T04:48:56.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.943+0000 7f555effd700 1 -- 172.21.15.2:0/3580476601 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f5570015020 con 0x55c293f2aa00 2023-09-13T04:48:56.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.943+0000 7f5599293b80 10 client.16530 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:48:56.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.943+0000 7f5599293b80 10 client.16530 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-09-13T04:48:56.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.943+0000 7f5599293b80 20 client.16530 populate_metadata read hostname 'smithi002' 2023-09-13T04:48:56.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.943+0000 7f5599293b80 1 --2- 172.21.15.2:0/3580476601 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x55c293f442f0 0x55c293f46710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:56.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.943+0000 7f5599293b80 4 client.16530 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:48:56.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:56.943+0000 7f5599293b80 1 -- 172.21.15.2:0/3580476601 --> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c293e75970 con 0x55c293f442f0 2023-09-13T04:48:57.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.056+0000 7f5577fff700 1 --2- 172.21.15.2:0/3580476601 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x55c293f442f0 0x55c293f46710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.057+0000 7f5577fff700 1 --2- 172.21.15.2:0/3580476601 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x55c293f442f0 0x55c293f46710 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6cb51f341b5458c8 server_cookie=89698b8fbaef1e79 in_seq=0 out_seq=0 2023-09-13T04:48:57.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.057+0000 7f555effd700 10 client.16530 ms_handle_connect on v2:172.21.15.2:6840/482545391 2023-09-13T04:48:57.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.058+0000 7f555effd700 1 -- 172.21.15.2:0/3580476601 <== mds.1 v2:172.21.15.2:6840/482545391 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c293e75970 con 0x55c293f442f0 2023-09-13T04:48:57.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.058+0000 7f555effd700 10 client.16530 handle_command_reply: tid=0 2023-09-13T04:48:57.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.058+0000 7f5599293b80 1 client.16530 shutdown 2023-09-13T04:48:57.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.058+0000 7f5599293b80 1 -- 172.21.15.2:0/3580476601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c293f2aa00 msgr2=0x55c293f33be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:57.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.058+0000 7f5599293b80 1 --2- 172.21.15.2:0/3580476601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c293f2aa00 0x55c293f33be0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f557000f690 tx=0x7f557000f770 comp rx=0 tx=0).stop 2023-09-13T04:48:57.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.058+0000 7f5599293b80 1 -- 172.21.15.2:0/3580476601 shutdown_connections 2023-09-13T04:48:57.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.058+0000 7f5599293b80 1 --2- 172.21.15.2:0/3580476601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c293f30900 0x55c293f34660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.058+0000 7f5599293b80 1 --2- 172.21.15.2:0/3580476601 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x55c293f442f0 0x55c293f46710 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.058+0000 7f5599293b80 1 --2- 172.21.15.2:0/3580476601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c293f2aa00 0x55c293f33be0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.058+0000 7f5599293b80 1 --2- 172.21.15.2:0/3580476601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f2ffe0 0x55c293f34120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.058+0000 7f5599293b80 1 -- 172.21.15.2:0/3580476601 >> 172.21.15.2:0/3580476601 conn(0x55c293e88c80 msgr2=0x55c293f1f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:57.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.058+0000 7f555effd700 0 client.16530 ms_handle_reset on v2:172.21.15.2:6840/482545391 2023-09-13T04:48:57.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.059+0000 7f5599293b80 1 -- 172.21.15.2:0/3580476601 shutdown_connections 2023-09-13T04:48:57.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.059+0000 7f5599293b80 1 -- 172.21.15.2:0/3580476601 wait complete. 2023-09-13T04:48:57.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.059+0000 7f5599293b80 20 client.16530 trim_cache size 0 max 16384 2023-09-13T04:48:57.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.073+0000 7f5599293b80 1 Processor -- start 2023-09-13T04:48:57.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.073+0000 7f5599293b80 1 -- start start 2023-09-13T04:48:57.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.074+0000 7f5599293b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c293f2aa00 0x55c293f2a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.074+0000 7f5599293b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c293f2ffe0 0x55c293f2ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.074+0000 7f5599293b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f30900 0x55c293f3b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.074+0000 7f5599293b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c293f254f0 con 0x55c293f2ffe0 2023-09-13T04:48:57.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.074+0000 7f5599293b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c293f25db0 con 0x55c293f30900 2023-09-13T04:48:57.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.074+0000 7f5599293b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c293f3b750 con 0x55c293f2aa00 2023-09-13T04:48:57.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.074+0000 7f557f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f30900 0x55c293f3b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.074+0000 7f5577fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c293f2ffe0 0x55c293f2ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.074+0000 7f557effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c293f2aa00 0x55c293f2a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.074+0000 7f557f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f30900 0x55c293f3b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:59494/0 (socket says 172.21.15.2:59494) 2023-09-13T04:48:57.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.074+0000 7f557f7fe700 1 -- 172.21.15.2:0/2955587331 learned_addr learned my addr 172.21.15.2:0/2955587331 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:57.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.074+0000 7f557effd700 1 -- 172.21.15.2:0/2955587331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f30900 msgr2=0x55c293f3b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:57.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.074+0000 7f557effd700 1 --2- 172.21.15.2:0/2955587331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f30900 0x55c293f3b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.074+0000 7f557effd700 1 -- 172.21.15.2:0/2955587331 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c293f2ffe0 msgr2=0x55c293f2ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:57.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.074+0000 7f557effd700 1 --2- 172.21.15.2:0/2955587331 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c293f2ffe0 0x55c293f2ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.074+0000 7f557effd700 1 -- 172.21.15.2:0/2955587331 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c293ca8de0 con 0x55c293f2aa00 2023-09-13T04:48:57.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.075+0000 7f557effd700 1 --2- 172.21.15.2:0/2955587331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c293f2aa00 0x55c293f2a4c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5570003c70 tx=0x7f557000f890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f771d4c5c6cca330 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:57.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.075+0000 7f55767fc700 1 -- 172.21.15.2:0/2955587331 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55700058e0 con 0x55c293f2aa00 2023-09-13T04:48:57.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.075+0000 7f55767fc700 1 -- 172.21.15.2:0/2955587331 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f557001d070 con 0x55c293f2aa00 2023-09-13T04:48:57.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.075+0000 7f55767fc700 1 -- 172.21.15.2:0/2955587331 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55700052d0 con 0x55c293f2aa00 2023-09-13T04:48:57.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.076+0000 7f5599293b80 1 -- 172.21.15.2:0/2955587331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c293f2aa00 msgr2=0x55c293f2a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:57.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.076+0000 7f5599293b80 1 --2- 172.21.15.2:0/2955587331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c293f2aa00 0x55c293f2a4c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5570003c70 tx=0x7f557000f890 comp rx=0 tx=0).stop 2023-09-13T04:48:57.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.076+0000 7f5599293b80 1 -- 172.21.15.2:0/2955587331 shutdown_connections 2023-09-13T04:48:57.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.076+0000 7f5599293b80 1 --2- 172.21.15.2:0/2955587331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f30900 0x55c293f3b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.076+0000 7f5599293b80 1 --2- 172.21.15.2:0/2955587331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c293f2aa00 0x55c293f2a4c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.076+0000 7f5599293b80 1 --2- 172.21.15.2:0/2955587331 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c293f2ffe0 0x55c293f2ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.076+0000 7f5599293b80 1 -- 172.21.15.2:0/2955587331 >> 172.21.15.2:0/2955587331 conn(0x55c293e88c80 msgr2=0x55c293f2ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:57.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.076+0000 7f5599293b80 1 -- 172.21.15.2:0/2955587331 shutdown_connections 2023-09-13T04:48:57.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.076+0000 7f5599293b80 1 -- 172.21.15.2:0/2955587331 wait complete. 2023-09-13T04:48:57.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.076+0000 7f5599293b80 1 -- start start 2023-09-13T04:48:57.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.077+0000 7f5599293b80 1 Processor -- start 2023-09-13T04:48:57.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.077+0000 7f5599293b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f2aa00 0x55c293f371f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.077+0000 7f5599293b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c293f2ffe0 0x55c293f37730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.077+0000 7f5599293b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c293f30900 0x55c293f37c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.077+0000 7f5599293b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c293f97150 con 0x55c293f30900 2023-09-13T04:48:57.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.077+0000 7f5599293b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c293f97290 con 0x55c293f2aa00 2023-09-13T04:48:57.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.077+0000 7f5599293b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c293f973d0 con 0x55c293f2ffe0 2023-09-13T04:48:57.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.077+0000 7f5599293b80 10 client.0 _ll_register_callbacks cb 0x55c293e18680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:48:57.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.078+0000 7f557effd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f2aa00 0x55c293f371f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.078+0000 7f557effd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f2aa00 0x55c293f371f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:59504/0 (socket says 172.21.15.2:59504) 2023-09-13T04:48:57.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.078+0000 7f557effd700 1 -- 172.21.15.2:0/1736316238 learned_addr learned my addr 172.21.15.2:0/1736316238 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:57.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.078+0000 7f5577fff700 1 --2- 172.21.15.2:0/1736316238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c293f2ffe0 0x55c293f37730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.078+0000 7f557f7fe700 1 --2- 172.21.15.2:0/1736316238 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c293f30900 0x55c293f37c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.078+0000 7f557effd700 1 -- 172.21.15.2:0/1736316238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c293f2ffe0 msgr2=0x55c293f37730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:57.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.078+0000 7f557effd700 1 --2- 172.21.15.2:0/1736316238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c293f2ffe0 0x55c293f37730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.078+0000 7f557effd700 1 -- 172.21.15.2:0/1736316238 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c293f30900 msgr2=0x55c293f37c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:57.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.078+0000 7f557effd700 1 --2- 172.21.15.2:0/1736316238 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c293f30900 0x55c293f37c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.078+0000 7f557effd700 1 -- 172.21.15.2:0/1736316238 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c293e947a0 con 0x55c293f2aa00 2023-09-13T04:48:57.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.078+0000 7f557effd700 1 --2- 172.21.15.2:0/1736316238 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f2aa00 0x55c293f371f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f55700098c0 tx=0x7f55700058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2710955cc762707 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:57.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.078+0000 7f555effd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:48:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.079+0000 7f555effd700 1 -- 172.21.15.2:0/1736316238 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570003890 con 0x55c293f2aa00 2023-09-13T04:48:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.079+0000 7f5599293b80 1 -- 172.21.15.2:0/1736316238 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5568099dd0 con 0x55c293f2aa00 2023-09-13T04:48:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.079+0000 7f555effd700 1 -- 172.21.15.2:0/1736316238 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f5570000e40 con 0x55c293f2aa00 2023-09-13T04:48:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.079+0000 7f555effd700 1 -- 172.21.15.2:0/1736316238 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55700039f0 con 0x55c293f2aa00 2023-09-13T04:48:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.079+0000 7f5599293b80 10 client.26705 fetch_fsmap learned FSMap version 36 2023-09-13T04:48:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.079+0000 7f5599293b80 1 -- 172.21.15.2:0/1736316238 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55c293c91c60 con 0x55c293f2aa00 2023-09-13T04:48:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.080+0000 7f555effd700 1 -- 172.21.15.2:0/1736316238 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5568099dd0 con 0x55c293f2aa00 2023-09-13T04:48:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.080+0000 7f555effd700 1 -- 172.21.15.2:0/1736316238 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f5570009c70 con 0x55c293f2aa00 2023-09-13T04:48:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.080+0000 7f5599293b80 10 client.26705 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:48:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.080+0000 7f5599293b80 10 client.26705 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-09-13T04:48:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.080+0000 7f5599293b80 20 client.26705 populate_metadata read hostname 'smithi002' 2023-09-13T04:48:57.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.080+0000 7f5599293b80 1 --2- 172.21.15.2:0/1736316238 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x55c293f985c0 0x55c293f9a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.080+0000 7f5599293b80 4 client.26705 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-09-13T04:48:57.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.080+0000 7f5599293b80 1 -- 172.21.15.2:0/1736316238 --> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c293e36050 con 0x55c293f985c0 2023-09-13T04:48:57.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.080+0000 7f5577fff700 1 --2- 172.21.15.2:0/1736316238 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x55c293f985c0 0x55c293f9a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.081+0000 7f5577fff700 1 --2- 172.21.15.2:0/1736316238 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x55c293f985c0 0x55c293f9a9b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1cf26b07ead867a8 server_cookie=eecf380411940440 in_seq=0 out_seq=0 2023-09-13T04:48:57.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.081+0000 7f555effd700 10 client.26705 ms_handle_connect on v2:172.21.15.2:6840/482545391 2023-09-13T04:48:57.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.083+0000 7f555effd700 1 -- 172.21.15.2:0/1736316238 <== mds.1 v2:172.21.15.2:6840/482545391 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+10627 (crc 0 0 0) 0x55c293e36050 con 0x55c293f985c0 2023-09-13T04:48:57.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.083+0000 7f555effd700 10 client.26705 handle_command_reply: tid=0 2023-09-13T04:48:57.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.083+0000 7f5599293b80 1 client.26705 shutdown 2023-09-13T04:48:57.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.083+0000 7f5599293b80 1 -- 172.21.15.2:0/1736316238 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f2aa00 msgr2=0x55c293f371f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:57.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.083+0000 7f5599293b80 1 --2- 172.21.15.2:0/1736316238 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f2aa00 0x55c293f371f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f55700098c0 tx=0x7f55700058e0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.083+0000 7f5599293b80 1 -- 172.21.15.2:0/1736316238 shutdown_connections 2023-09-13T04:48:57.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.083+0000 7f5599293b80 1 --2- 172.21.15.2:0/1736316238 >> [v2:172.21.15.2:6840/482545391,v1:172.21.15.2:6841/482545391] conn(0x55c293f985c0 0x55c293f9a9b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.083+0000 7f5599293b80 1 --2- 172.21.15.2:0/1736316238 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c293f30900 0x55c293f37c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.083+0000 7f5599293b80 1 --2- 172.21.15.2:0/1736316238 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c293f2aa00 0x55c293f371f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.083+0000 7f555effd700 0 client.26705 ms_handle_reset on v2:172.21.15.2:6840/482545391 2023-09-13T04:48:57.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.083+0000 7f5599293b80 1 --2- 172.21.15.2:0/1736316238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c293f2ffe0 0x55c293f37730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.083+0000 7f5599293b80 1 -- 172.21.15.2:0/1736316238 >> 172.21.15.2:0/1736316238 conn(0x55c293e88c80 msgr2=0x55c293f34150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:57.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.083+0000 7f5599293b80 1 -- 172.21.15.2:0/1736316238 shutdown_connections 2023-09-13T04:48:57.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.083+0000 7f5599293b80 1 -- 172.21.15.2:0/1736316238 wait complete. 2023-09-13T04:48:57.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.083+0000 7f5599293b80 20 client.26705 trim_cache size 0 max 16384 2023-09-13T04:48:57.095 INFO:teuthology.orchestra.run.smithi002.stdout:[ 2023-09-13T04:48:57.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.096 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.096 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 38892227, 2023-09-13T04:48:57.096 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257041, 2023-09-13T04:48:57.096 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-52.xml" 2023-09-13T04:48:57.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.097 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.097 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 151342215, 2023-09-13T04:48:57.097 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511630004, 2023-09-13T04:48:57.097 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-44.xml" 2023-09-13T04:48:57.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.098 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.098 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 161001000, 2023-09-13T04:48:57.098 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511629987, 2023-09-13T04:48:57.098 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-52.xml" 2023-09-13T04:48:57.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.099 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.099 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 748306547, 2023-09-13T04:48:57.099 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511631051, 2023-09-13T04:48:57.100 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-15.xml" 2023-09-13T04:48:57.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.100 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.100 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 951973817, 2023-09-13T04:48:57.101 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511629996, 2023-09-13T04:48:57.101 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-48.xml" 2023-09-13T04:48:57.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.101 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.101 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 974112554, 2023-09-13T04:48:57.102 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257117, 2023-09-13T04:48:57.102 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-14.xml" 2023-09-13T04:48:57.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.102 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.103 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 1014578855, 2023-09-13T04:48:57.103 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511630655, 2023-09-13T04:48:57.103 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-33.xml" 2023-09-13T04:48:57.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.104 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.104 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 1163426907, 2023-09-13T04:48:57.104 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511631507, 2023-09-13T04:48:57.104 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-16.xml" 2023-09-13T04:48:57.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.105 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.105 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 1203397643, 2023-09-13T04:48:57.105 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511631072, 2023-09-13T04:48:57.105 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-7.xml" 2023-09-13T04:48:57.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.106 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.106 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 1277592931, 2023-09-13T04:48:57.106 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511630030, 2023-09-13T04:48:57.106 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-31.xml" 2023-09-13T04:48:57.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.107 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.107 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 1404451554, 2023-09-13T04:48:57.107 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257125, 2023-09-13T04:48:57.108 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-10.xml" 2023-09-13T04:48:57.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.108 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.108 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 1483557343, 2023-09-13T04:48:57.109 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511631074, 2023-09-13T04:48:57.109 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-6.xml" 2023-09-13T04:48:57.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.109 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.110 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 1545324818, 2023-09-13T04:48:57.110 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257171, 2023-09-13T04:48:57.110 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-12.jpg" 2023-09-13T04:48:57.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.111 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.111 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 1546233816, 2023-09-13T04:48:57.111 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511629992, 2023-09-13T04:48:57.111 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-50.xml" 2023-09-13T04:48:57.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.112 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.112 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 1655972945, 2023-09-13T04:48:57.112 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257147, 2023-09-13T04:48:57.112 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-22.jpg" 2023-09-13T04:48:57.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 1665254911, 2023-09-13T04:48:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257185, 2023-09-13T04:48:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-5.jpg" 2023-09-13T04:48:57.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.114 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.114 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 1731443853, 2023-09-13T04:48:57.114 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257123, 2023-09-13T04:48:57.115 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-11.xml" 2023-09-13T04:48:57.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.115 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.115 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 1801752941, 2023-09-13T04:48:57.115 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023256984, 2023-09-13T04:48:57.116 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-71.xml" 2023-09-13T04:48:57.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.116 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.116 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 1820958268, 2023-09-13T04:48:57.117 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257145, 2023-09-13T04:48:57.117 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-23.jpg" 2023-09-13T04:48:57.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.117 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.118 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 1833957116, 2023-09-13T04:48:57.118 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257165, 2023-09-13T04:48:57.118 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-15.jpg" 2023-09-13T04:48:57.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.119 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.119 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 1877034031, 2023-09-13T04:48:57.119 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257091, 2023-09-13T04:48:57.119 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-28.xml" 2023-09-13T04:48:57.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.120 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.120 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 1912082278, 2023-09-13T04:48:57.120 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257049, 2023-09-13T04:48:57.120 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-49.xml" 2023-09-13T04:48:57.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.121 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.121 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 1931285765, 2023-09-13T04:48:57.121 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257103, 2023-09-13T04:48:57.122 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-22.xml" 2023-09-13T04:48:57.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.122 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.122 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 2019923714, 2023-09-13T04:48:57.123 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511630022, 2023-09-13T04:48:57.123 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-35.xml" 2023-09-13T04:48:57.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.123 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.123 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 2060254871, 2023-09-13T04:48:57.124 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511631047, 2023-09-13T04:48:57.124 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-17.xml" 2023-09-13T04:48:57.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.124 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.125 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 2081770046, 2023-09-13T04:48:57.125 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257189, 2023-09-13T04:48:57.125 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-3.jpg" 2023-09-13T04:48:57.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.126 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.126 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 2152209473, 2023-09-13T04:48:57.126 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511631071, 2023-09-13T04:48:57.126 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-8.xml" 2023-09-13T04:48:57.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.127 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.127 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 2158798145, 2023-09-13T04:48:57.127 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511630127, 2023-09-13T04:48:57.127 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-79.jpg" 2023-09-13T04:48:57.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.128 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.128 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 2321888234, 2023-09-13T04:48:57.128 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257089, 2023-09-13T04:48:57.129 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-29.xml" 2023-09-13T04:48:57.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.129 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.129 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 2457930243, 2023-09-13T04:48:57.129 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257035, 2023-09-13T04:48:57.130 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-54.xml" 2023-09-13T04:48:57.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.130 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.130 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 2468678708, 2023-09-13T04:48:57.131 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511631068, 2023-09-13T04:48:57.131 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-9.xml" 2023-09-13T04:48:57.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.131 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.132 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 2472059171, 2023-09-13T04:48:57.132 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511631035, 2023-09-13T04:48:57.132 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-27.xml" 2023-09-13T04:48:57.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.132 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.133 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 2472931663, 2023-09-13T04:48:57.133 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257137, 2023-09-13T04:48:57.133 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-4.xml" 2023-09-13T04:48:57.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.134 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.134 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 2486018571, 2023-09-13T04:48:57.134 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257079, 2023-09-13T04:48:57.134 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-34.xml" 2023-09-13T04:48:57.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.135 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.135 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 2527519661, 2023-09-13T04:48:57.135 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257023, 2023-09-13T04:48:57.135 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-58.xml" 2023-09-13T04:48:57.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.136 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.136 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 2605691590, 2023-09-13T04:48:57.136 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257097, 2023-09-13T04:48:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-25.xml" 2023-09-13T04:48:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 2630414652, 2023-09-13T04:48:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511631042, 2023-09-13T04:48:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-22.xml" 2023-09-13T04:48:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 2663621206, 2023-09-13T04:48:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257051, 2023-09-13T04:48:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-48.xml" 2023-09-13T04:48:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 2800202397, 2023-09-13T04:48:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257055, 2023-09-13T04:48:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-46.xml" 2023-09-13T04:48:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 2889064089, 2023-09-13T04:48:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511631546, 2023-09-13T04:48:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-43.xml" 2023-09-13T04:48:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 2915809936, 2023-09-13T04:48:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511630015, 2023-09-13T04:48:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-38.xml" 2023-09-13T04:48:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 2920693249, 2023-09-13T04:48:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511631034, 2023-09-13T04:48:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-28.xml" 2023-09-13T04:48:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 2953327874, 2023-09-13T04:48:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257135, 2023-09-13T04:48:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-5.xml" 2023-09-13T04:48:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 2984841417, 2023-09-13T04:48:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257071, 2023-09-13T04:48:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-38.xml" 2023-09-13T04:48:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 3009921519, 2023-09-13T04:48:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511630005, 2023-09-13T04:48:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-43.xml" 2023-09-13T04:48:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 3192303016, 2023-09-13T04:48:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511631079, 2023-09-13T04:48:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-2.xml" 2023-09-13T04:48:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 3213502603, 2023-09-13T04:48:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257095, 2023-09-13T04:48:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-26.xml" 2023-09-13T04:48:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 3227661194, 2023-09-13T04:48:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511631060, 2023-09-13T04:48:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-10.xml" 2023-09-13T04:48:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 3246650744, 2023-09-13T04:48:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511630654, 2023-09-13T04:48:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-99.jpg" 2023-09-13T04:48:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 3468323463, 2023-09-13T04:48:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257026, 2023-09-13T04:48:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-57.xml" 2023-09-13T04:48:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 3537076758, 2023-09-13T04:48:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511629986, 2023-09-13T04:48:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-53.xml" 2023-09-13T04:48:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 3620397869, 2023-09-13T04:48:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511630002, 2023-09-13T04:48:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-45.xml" 2023-09-13T04:48:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 3709572885, 2023-09-13T04:48:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257173, 2023-09-13T04:48:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-11.jpg" 2023-09-13T04:48:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 3764334718, 2023-09-13T04:48:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257017, 2023-09-13T04:48:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-60.xml" 2023-09-13T04:48:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 3948640725, 2023-09-13T04:48:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257038, 2023-09-13T04:48:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-53.xml" 2023-09-13T04:48:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:48:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:48:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2023-09-13T04:48:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 4118117433, 2023-09-13T04:48:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 2199023257053, 2023-09-13T04:48:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-47.xml" 2023-09-13T04:48:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:48:57.160 INFO:teuthology.orchestra.run.smithi002.stdout:] 2023-09-13T04:48:57.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.084+0000 7f55859ff700 1 -- 172.21.15.2:0/860969259 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f556c0a07b0 msgr2=0x7f556c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:57.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.084+0000 7f55859ff700 1 --2- 172.21.15.2:0/860969259 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f556c0a07b0 0x7f556c0a2c70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5578009fd0 tx=0x7f557800b040 comp rx=0 tx=0).stop 2023-09-13T04:48:57.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.084+0000 7f55859ff700 1 -- 172.21.15.2:0/860969259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5580105fd0 msgr2=0x7f55800734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:57.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.084+0000 7f55859ff700 1 --2- 172.21.15.2:0/860969259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5580105fd0 0x7f55800734f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f556800e9c0 tx=0x7f556800ecd0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.084+0000 7f55859ff700 1 -- 172.21.15.2:0/860969259 shutdown_connections 2023-09-13T04:48:57.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.084+0000 7f55859ff700 1 --2- 172.21.15.2:0/860969259 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f556c0a07b0 0x7f556c0a2c70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.084+0000 7f55859ff700 1 --2- 172.21.15.2:0/860969259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5580073a30 0x7f55801a5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.084+0000 7f55859ff700 1 --2- 172.21.15.2:0/860969259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5580100eb0 0x7f5580072fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.084+0000 7f55859ff700 1 --2- 172.21.15.2:0/860969259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5580105fd0 0x7f55800734f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.084+0000 7f55859ff700 1 -- 172.21.15.2:0/860969259 >> 172.21.15.2:0/860969259 conn(0x7f55800fc730 msgr2=0x7f55800fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:57.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.085+0000 7f55859ff700 1 -- 172.21.15.2:0/860969259 shutdown_connections 2023-09-13T04:48:57.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.085+0000 7f55859ff700 1 -- 172.21.15.2:0/860969259 wait complete. 2023-09-13T04:48:57.164 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-13T04:48:57.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.385+0000 7fe3665ff700 1 -- 172.21.15.2:0/628518734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe360103100 msgr2=0x7fe3601034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:57.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.385+0000 7fe3665ff700 1 --2- 172.21.15.2:0/628518734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe360103100 0x7fe3601034e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe350009990 tx=0x7fe350009ca0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.385+0000 7fe3665ff700 1 -- 172.21.15.2:0/628518734 shutdown_connections 2023-09-13T04:48:57.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.385+0000 7fe3665ff700 1 --2- 172.21.15.2:0/628518734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe360108220 0x7fe36010fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.385+0000 7fe3665ff700 1 --2- 172.21.15.2:0/628518734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe360103100 0x7fe3601034e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.385+0000 7fe3665ff700 1 --2- 172.21.15.2:0/628518734 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe360103ab0 0x7fe360107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.385+0000 7fe3665ff700 1 -- 172.21.15.2:0/628518734 >> 172.21.15.2:0/628518734 conn(0x7fe3600fe940 msgr2=0x7fe360100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:57.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:57 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1435476030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:48:57.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.385+0000 7fe3665ff700 1 -- 172.21.15.2:0/628518734 shutdown_connections 2023-09-13T04:48:57.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.385+0000 7fe3665ff700 1 -- 172.21.15.2:0/628518734 wait complete. 2023-09-13T04:48:57.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.387+0000 7fe3665ff700 1 Processor -- start 2023-09-13T04:48:57.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.388+0000 7fe3665ff700 1 -- start start 2023-09-13T04:48:57.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.388+0000 7fe3665ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe360103100 0x7fe36010f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.388+0000 7fe3665ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe360108220 0x7fe36010fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.388+0000 7fe3665ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe36010c7f0 0x7fe36010cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.388+0000 7fe3665ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36010d450 con 0x7fe36010c7f0 2023-09-13T04:48:57.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.388+0000 7fe3665ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36010d5c0 con 0x7fe360108220 2023-09-13T04:48:57.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.388+0000 7fe3665ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3601a9d30 con 0x7fe360103100 2023-09-13T04:48:57.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.389+0000 7fe35ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe360103100 0x7fe36010f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.389+0000 7fe35ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe360103100 0x7fe36010f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:38602/0 (socket says 172.21.15.2:38602) 2023-09-13T04:48:57.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.389+0000 7fe35ffff700 1 -- 172.21.15.2:0/2047907339 learned_addr learned my addr 172.21.15.2:0/2047907339 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:57.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.389+0000 7fe364b9c700 1 --2- 172.21.15.2:0/2047907339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe36010c7f0 0x7fe36010cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.389+0000 7fe35f7fe700 1 --2- 172.21.15.2:0/2047907339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe360108220 0x7fe36010fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.390+0000 7fe35ffff700 1 -- 172.21.15.2:0/2047907339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe360108220 msgr2=0x7fe36010fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:57.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.390+0000 7fe35ffff700 1 --2- 172.21.15.2:0/2047907339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe360108220 0x7fe36010fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.390+0000 7fe35ffff700 1 -- 172.21.15.2:0/2047907339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe36010c7f0 msgr2=0x7fe36010cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:57.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.390+0000 7fe35ffff700 1 --2- 172.21.15.2:0/2047907339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe36010c7f0 0x7fe36010cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.390+0000 7fe35ffff700 1 -- 172.21.15.2:0/2047907339 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3500095f0 con 0x7fe360103100 2023-09-13T04:48:57.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.390+0000 7fe35ffff700 1 --2- 172.21.15.2:0/2047907339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe360103100 0x7fe36010f790 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe350005910 tx=0x7fe350004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:57.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.391+0000 7fe35d7fa700 1 -- 172.21.15.2:0/2047907339 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35001d070 con 0x7fe360103100 2023-09-13T04:48:57.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.391+0000 7fe35d7fa700 1 -- 172.21.15.2:0/2047907339 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe350022470 con 0x7fe360103100 2023-09-13T04:48:57.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.391+0000 7fe35d7fa700 1 -- 172.21.15.2:0/2047907339 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe350017650 con 0x7fe360103100 2023-09-13T04:48:57.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.391+0000 7fe3665ff700 1 -- 172.21.15.2:0/2047907339 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3601a9f80 con 0x7fe360103100 2023-09-13T04:48:57.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.391+0000 7fe3665ff700 1 -- 172.21.15.2:0/2047907339 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3601aa340 con 0x7fe360103100 2023-09-13T04:48:57.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.395+0000 7fe35d7fa700 1 -- 172.21.15.2:0/2047907339 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7fe3500225e0 con 0x7fe360103100 2023-09-13T04:48:57.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.396+0000 7fe35d7fa700 1 --2- 172.21.15.2:0/2047907339 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe3480a0830 0x7fe3480a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.396+0000 7fe379fa6b80 1 Processor -- start 2023-09-13T04:48:57.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.396+0000 7fe35d7fa700 1 -- 172.21.15.2:0/2047907339 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7fe3500cd720 con 0x7fe360103100 2023-09-13T04:48:57.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.397+0000 7fe379fa6b80 1 -- start start 2023-09-13T04:48:57.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.397+0000 7fe35f7fe700 1 --2- 172.21.15.2:0/2047907339 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe3480a0830 0x7fe3480a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.397+0000 7fe379fa6b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec339f0fe0 0x55ec339f13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.397+0000 7fe379fa6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec339f1900 0x55ec339eb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.397+0000 7fe379fa6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339eba00 0x55ec339ebe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.397+0000 7fe379fa6b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec33944780 con 0x55ec339f0fe0 2023-09-13T04:48:57.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.397+0000 7fe379fa6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec339420a0 con 0x55ec339f1900 2023-09-13T04:48:57.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.397+0000 7fe379fa6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3393d990 con 0x55ec339eba00 2023-09-13T04:48:57.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.397+0000 7fe35ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec339f0fe0 0x55ec339f13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.397+0000 7fe35ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec339f0fe0 0x55ec339f13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43126/0 (socket says 172.21.15.2:43126) 2023-09-13T04:48:57.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.397+0000 7fe35ffff700 1 -- 172.21.15.2:0/3628337026 learned_addr learned my addr 172.21.15.2:0/3628337026 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:57.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.398+0000 7fe364b9c700 1 --2- 172.21.15.2:0/3628337026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339eba00 0x55ec339ebe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.398+0000 7fe35f7fe700 1 --2- 172.21.15.2:0/2047907339 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe3480a0830 0x7fe3480a2cf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe354005ea0 tx=0x7fe354005e30 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:57.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.398+0000 7fe364b9c700 1 -- 172.21.15.2:0/3628337026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec339f1900 msgr2=0x55ec339eb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:57.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.398+0000 7fe364b9c700 1 --2- 172.21.15.2:0/3628337026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec339f1900 0x55ec339eb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.398+0000 7fe364b9c700 1 -- 172.21.15.2:0/3628337026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec339f0fe0 msgr2=0x55ec339f13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:57.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.398+0000 7fe364b9c700 1 --2- 172.21.15.2:0/3628337026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec339f0fe0 0x55ec339f13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.398+0000 7fe364b9c700 1 -- 172.21.15.2:0/3628337026 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec337bd630 con 0x55ec339eba00 2023-09-13T04:48:57.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.398+0000 7fe364b9c700 1 --2- 172.21.15.2:0/3628337026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339eba00 0x55ec339ebe60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe358011580 tx=0x7fe358009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58fa0d4aad169a8c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:57.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.399+0000 7fe3467fc700 1 -- 172.21.15.2:0/3628337026 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35800e040 con 0x55ec339eba00 2023-09-13T04:48:57.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.399+0000 7fe3467fc700 1 -- 172.21.15.2:0/3628337026 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe35800f040 con 0x55ec339eba00 2023-09-13T04:48:57.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.399+0000 7fe3467fc700 1 -- 172.21.15.2:0/3628337026 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe358004450 con 0x55ec339eba00 2023-09-13T04:48:57.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.399+0000 7fe379fa6b80 1 -- 172.21.15.2:0/3628337026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339eba00 msgr2=0x55ec339ebe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:57.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.399+0000 7fe379fa6b80 1 --2- 172.21.15.2:0/3628337026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339eba00 0x55ec339ebe60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe358011580 tx=0x7fe358009920 comp rx=0 tx=0).stop 2023-09-13T04:48:57.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.400+0000 7fe379fa6b80 1 -- 172.21.15.2:0/3628337026 shutdown_connections 2023-09-13T04:48:57.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.400+0000 7fe379fa6b80 1 --2- 172.21.15.2:0/3628337026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339eba00 0x55ec339ebe60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.400+0000 7fe379fa6b80 1 --2- 172.21.15.2:0/3628337026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec339f0fe0 0x55ec339f13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.400+0000 7fe379fa6b80 1 --2- 172.21.15.2:0/3628337026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec339f1900 0x55ec339eb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.400+0000 7fe379fa6b80 1 -- 172.21.15.2:0/3628337026 >> 172.21.15.2:0/3628337026 conn(0x55ec33949c80 msgr2=0x55ec339e5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:57.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.400+0000 7fe379fa6b80 1 -- 172.21.15.2:0/3628337026 shutdown_connections 2023-09-13T04:48:57.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.400+0000 7fe379fa6b80 1 -- 172.21.15.2:0/3628337026 wait complete. 2023-09-13T04:48:57.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.400+0000 7fe379fa6b80 1 -- start start 2023-09-13T04:48:57.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.400+0000 7fe379fa6b80 1 Processor -- start 2023-09-13T04:48:57.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.401+0000 7fe379fa6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec339eba00 0x55ec339f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.401+0000 7fe379fa6b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec339f0fe0 0x55ec339f5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.401+0000 7fe379fa6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339f1900 0x55ec339f5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.401+0000 7fe379fa6b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec338dbbb0 con 0x55ec339f0fe0 2023-09-13T04:48:57.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.401+0000 7fe379fa6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec338db8b0 con 0x55ec339eba00 2023-09-13T04:48:57.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.401+0000 7fe379fa6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec337bf920 con 0x55ec339f1900 2023-09-13T04:48:57.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.401+0000 7fe379fa6b80 10 client.0 _ll_register_callbacks cb 0x55ec338d9680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:48:57.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.401+0000 7fe35f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec339f0fe0 0x55ec339f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.401+0000 7fe364b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339f1900 0x55ec339f5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.401+0000 7fe35f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec339f0fe0 0x55ec339f5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43140/0 (socket says 172.21.15.2:43140) 2023-09-13T04:48:57.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.401+0000 7fe35f7fe700 1 -- 172.21.15.2:0/27778442 learned_addr learned my addr 172.21.15.2:0/27778442 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:57.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.401+0000 7fe35ffff700 1 --2- 172.21.15.2:0/27778442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec339eba00 0x55ec339f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.402+0000 7fe364b9c700 1 -- 172.21.15.2:0/27778442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec339eba00 msgr2=0x55ec339f4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:57.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.402+0000 7fe364b9c700 1 --2- 172.21.15.2:0/27778442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec339eba00 0x55ec339f4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.402+0000 7fe364b9c700 1 -- 172.21.15.2:0/27778442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec339f0fe0 msgr2=0x55ec339f5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:57.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.402+0000 7fe364b9c700 1 --2- 172.21.15.2:0/27778442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec339f0fe0 0x55ec339f5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.402+0000 7fe364b9c700 1 -- 172.21.15.2:0/27778442 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe358011230 con 0x55ec339f1900 2023-09-13T04:48:57.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.402+0000 7fe35ffff700 1 --2- 172.21.15.2:0/27778442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec339eba00 0x55ec339f4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:48:57.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.402+0000 7fe364b9c700 1 --2- 172.21.15.2:0/27778442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339f1900 0x55ec339f5680 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe35800b010 tx=0x7fe358003ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81f76cda2e3b55df server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:57.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.402+0000 7fe33f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-13T04:48:57.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.403+0000 7fe33f7fe700 1 -- 172.21.15.2:0/27778442 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35800e040 con 0x55ec339f1900 2023-09-13T04:48:57.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.403+0000 7fe379fa6b80 1 -- 172.21.15.2:0/27778442 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec339de670 con 0x55ec339f1900 2023-09-13T04:48:57.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.403+0000 7fe33f7fe700 1 -- 172.21.15.2:0/27778442 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7fe358004240 con 0x55ec339f1900 2023-09-13T04:48:57.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.403+0000 7fe33f7fe700 1 -- 172.21.15.2:0/27778442 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe358004400 con 0x55ec339f1900 2023-09-13T04:48:57.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.403+0000 7fe379fa6b80 10 client.26734 fetch_fsmap learned FSMap version 36 2023-09-13T04:48:57.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.403+0000 7fe379fa6b80 1 -- 172.21.15.2:0/27778442 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55ec339557a0 con 0x55ec339f1900 2023-09-13T04:48:57.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.403+0000 7fe33f7fe700 1 -- 172.21.15.2:0/27778442 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec339de670 con 0x55ec339f1900 2023-09-13T04:48:57.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.404+0000 7fe33f7fe700 1 -- 172.21.15.2:0/27778442 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fe35801b020 con 0x55ec339f1900 2023-09-13T04:48:57.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.404+0000 7fe379fa6b80 10 client.26734 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:48:57.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.404+0000 7fe379fa6b80 10 client.26734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:48:57.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.404+0000 7fe379fa6b80 20 client.26734 populate_metadata read hostname 'smithi002' 2023-09-13T04:48:57.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.404+0000 7fe379fa6b80 1 --2- 172.21.15.2:0/27778442 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55ec33a052a0 0x55ec33a076c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.404+0000 7fe379fa6b80 4 client.26734 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:48:57.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.404+0000 7fe379fa6b80 1 -- 172.21.15.2:0/27778442 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec33936970 con 0x55ec33a052a0 2023-09-13T04:48:57.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.404+0000 7fe35ffff700 1 --2- 172.21.15.2:0/27778442 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55ec33a052a0 0x55ec33a076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.405+0000 7fe35ffff700 1 --2- 172.21.15.2:0/27778442 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55ec33a052a0 0x55ec33a076c0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a50c30a0f272d98 server_cookie=677401684442526c in_seq=0 out_seq=0 2023-09-13T04:48:57.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.405+0000 7fe33f7fe700 10 client.26734 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:57.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.405+0000 7fe33f7fe700 1 -- 172.21.15.2:0/27778442 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ec33936970 con 0x55ec33a052a0 2023-09-13T04:48:57.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.405+0000 7fe33f7fe700 10 client.26734 handle_command_reply: tid=0 2023-09-13T04:48:57.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.406+0000 7fe379fa6b80 1 client.26734 shutdown 2023-09-13T04:48:57.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.406+0000 7fe379fa6b80 1 -- 172.21.15.2:0/27778442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339f1900 msgr2=0x55ec339f5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:57.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.406+0000 7fe379fa6b80 1 --2- 172.21.15.2:0/27778442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339f1900 0x55ec339f5680 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe35800b010 tx=0x7fe358003ec0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.406+0000 7fe379fa6b80 1 -- 172.21.15.2:0/27778442 shutdown_connections 2023-09-13T04:48:57.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.406+0000 7fe379fa6b80 1 --2- 172.21.15.2:0/27778442 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55ec33a052a0 0x55ec33a076c0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.406+0000 7fe379fa6b80 1 --2- 172.21.15.2:0/27778442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339f1900 0x55ec339f5680 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.406+0000 7fe379fa6b80 1 --2- 172.21.15.2:0/27778442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec339eba00 0x55ec339f4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.406+0000 7fe379fa6b80 1 --2- 172.21.15.2:0/27778442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec339f0fe0 0x55ec339f5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.407+0000 7fe379fa6b80 1 -- 172.21.15.2:0/27778442 >> 172.21.15.2:0/27778442 conn(0x55ec33949c80 msgr2=0x55ec339e07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:57.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.407+0000 7fe33f7fe700 0 client.26734 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:57.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.407+0000 7fe379fa6b80 1 -- 172.21.15.2:0/27778442 shutdown_connections 2023-09-13T04:48:57.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.407+0000 7fe379fa6b80 1 -- 172.21.15.2:0/27778442 wait complete. 2023-09-13T04:48:57.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.407+0000 7fe379fa6b80 20 client.26734 trim_cache size 0 max 16384 2023-09-13T04:48:57.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.424+0000 7fe379fa6b80 1 Processor -- start 2023-09-13T04:48:57.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.424+0000 7fe379fa6b80 1 -- start start 2023-09-13T04:48:57.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.424+0000 7fe379fa6b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec339eba00 0x55ec339eb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.424+0000 7fe379fa6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339f0fe0 0x55ec339f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.424+0000 7fe379fa6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec339f1900 0x55ec339fc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.424+0000 7fe379fa6b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec33a02900 con 0x55ec339eba00 2023-09-13T04:48:57.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.424+0000 7fe379fa6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec339fc780 con 0x55ec339f1900 2023-09-13T04:48:57.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.424+0000 7fe379fa6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec339f0bd0 con 0x55ec339f0fe0 2023-09-13T04:48:57.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.425+0000 7fe35f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339f0fe0 0x55ec339f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.425+0000 7fe35ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec339eba00 0x55ec339eb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.425+0000 7fe35f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339f0fe0 0x55ec339f3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:38630/0 (socket says 172.21.15.2:38630) 2023-09-13T04:48:57.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.425+0000 7fe35f7fe700 1 -- 172.21.15.2:0/2375708339 learned_addr learned my addr 172.21.15.2:0/2375708339 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:57.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.425+0000 7fe364b9c700 1 --2- 172.21.15.2:0/2375708339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec339f1900 0x55ec339fc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.425+0000 7fe35f7fe700 1 -- 172.21.15.2:0/2375708339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec339f1900 msgr2=0x55ec339fc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:57.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.425+0000 7fe35f7fe700 1 --2- 172.21.15.2:0/2375708339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec339f1900 0x55ec339fc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.425+0000 7fe35f7fe700 1 -- 172.21.15.2:0/2375708339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec339eba00 msgr2=0x55ec339eb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:57.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.425+0000 7fe35f7fe700 1 --2- 172.21.15.2:0/2375708339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec339eba00 0x55ec339eb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.425+0000 7fe35f7fe700 1 -- 172.21.15.2:0/2375708339 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec339e6130 con 0x55ec339f0fe0 2023-09-13T04:48:57.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.425+0000 7fe364b9c700 1 --2- 172.21.15.2:0/2375708339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec339f1900 0x55ec339fc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:48:57.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.425+0000 7fe35f7fe700 1 --2- 172.21.15.2:0/2375708339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339f0fe0 0x55ec339f3eb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe36006a250 tx=0x7fe3540047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5181256c257c6f6b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:57.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.426+0000 7fe346ffd700 1 -- 172.21.15.2:0/2375708339 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe354010d20 con 0x55ec339f0fe0 2023-09-13T04:48:57.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.426+0000 7fe346ffd700 1 -- 172.21.15.2:0/2375708339 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe354010e80 con 0x55ec339f0fe0 2023-09-13T04:48:57.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.426+0000 7fe346ffd700 1 -- 172.21.15.2:0/2375708339 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35401e760 con 0x55ec339f0fe0 2023-09-13T04:48:57.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.427+0000 7fe379fa6b80 1 -- 172.21.15.2:0/2375708339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339f0fe0 msgr2=0x55ec339f3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:57.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.427+0000 7fe379fa6b80 1 --2- 172.21.15.2:0/2375708339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339f0fe0 0x55ec339f3eb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe36006a250 tx=0x7fe3540047c0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.427+0000 7fe379fa6b80 1 -- 172.21.15.2:0/2375708339 shutdown_connections 2023-09-13T04:48:57.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.427+0000 7fe379fa6b80 1 --2- 172.21.15.2:0/2375708339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec339f1900 0x55ec339fc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.427+0000 7fe379fa6b80 1 --2- 172.21.15.2:0/2375708339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec339eba00 0x55ec339eb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.427+0000 7fe379fa6b80 1 --2- 172.21.15.2:0/2375708339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339f0fe0 0x55ec339f3eb0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.427+0000 7fe379fa6b80 1 -- 172.21.15.2:0/2375708339 >> 172.21.15.2:0/2375708339 conn(0x55ec33949c80 msgr2=0x55ec339eed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:57.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.427+0000 7fe379fa6b80 1 -- 172.21.15.2:0/2375708339 shutdown_connections 2023-09-13T04:48:57.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.427+0000 7fe379fa6b80 1 -- 172.21.15.2:0/2375708339 wait complete. 2023-09-13T04:48:57.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.428+0000 7fe379fa6b80 1 -- start start 2023-09-13T04:48:57.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.428+0000 7fe379fa6b80 1 Processor -- start 2023-09-13T04:48:57.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.428+0000 7fe379fa6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339eba00 0x55ec339edb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.428+0000 7fe379fa6b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec339f0fe0 0x55ec339ee090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.428+0000 7fe379fa6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec339f1900 0x55ec339eca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.428+0000 7fe379fa6b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec339ee770 con 0x55ec339f0fe0 2023-09-13T04:48:57.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.428+0000 7fe379fa6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec339ee8e0 con 0x55ec339f1900 2023-09-13T04:48:57.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.428+0000 7fe379fa6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec339ecfa0 con 0x55ec339eba00 2023-09-13T04:48:57.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.429+0000 7fe379fa6b80 10 client.0 _ll_register_callbacks cb 0x55ec338d9680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:48:57.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.429+0000 7fe35f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec339f0fe0 0x55ec339ee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.429+0000 7fe35f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec339f0fe0 0x55ec339ee090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43168/0 (socket says 172.21.15.2:43168) 2023-09-13T04:48:57.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.429+0000 7fe364b9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec339f1900 0x55ec339eca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.429+0000 7fe35f7fe700 1 -- 172.21.15.2:0/3478630763 learned_addr learned my addr 172.21.15.2:0/3478630763 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:48:57.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.429+0000 7fe35ffff700 1 --2- 172.21.15.2:0/3478630763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339eba00 0x55ec339edb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.429+0000 7fe364b9c700 1 -- 172.21.15.2:0/3478630763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339eba00 msgr2=0x55ec339edb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:57.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.429+0000 7fe364b9c700 1 --2- 172.21.15.2:0/3478630763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339eba00 0x55ec339edb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.429+0000 7fe364b9c700 1 -- 172.21.15.2:0/3478630763 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec339f0fe0 msgr2=0x55ec339ee090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:48:57.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.429+0000 7fe364b9c700 1 --2- 172.21.15.2:0/3478630763 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec339f0fe0 0x55ec339ee090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.429+0000 7fe364b9c700 1 -- 172.21.15.2:0/3478630763 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec339557a0 con 0x55ec339f1900 2023-09-13T04:48:57.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.429+0000 7fe35ffff700 1 --2- 172.21.15.2:0/3478630763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339eba00 0x55ec339edb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:48:57.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.430+0000 7fe364b9c700 1 --2- 172.21.15.2:0/3478630763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec339f1900 0x55ec339eca60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe3580061e0 tx=0x7fe3580077b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=847d15324e04523 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:48:57.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.430+0000 7fe33f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:48:57.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.430+0000 7fe33f7fe700 1 -- 172.21.15.2:0/3478630763 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35800e070 con 0x55ec339f1900 2023-09-13T04:48:57.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.430+0000 7fe379fa6b80 1 -- 172.21.15.2:0/3478630763 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe35800f040 con 0x55ec339f1900 2023-09-13T04:48:57.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.430+0000 7fe33f7fe700 1 -- 172.21.15.2:0/3478630763 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7fe358011e80 con 0x55ec339f1900 2023-09-13T04:48:57.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.430+0000 7fe33f7fe700 1 -- 172.21.15.2:0/3478630763 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3580079a0 con 0x55ec339f1900 2023-09-13T04:48:57.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.430+0000 7fe379fa6b80 10 client.26720 fetch_fsmap learned FSMap version 36 2023-09-13T04:48:57.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.430+0000 7fe379fa6b80 1 -- 172.21.15.2:0/3478630763 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7fe35400ea90 con 0x55ec339f1900 2023-09-13T04:48:57.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.431+0000 7fe33f7fe700 1 -- 172.21.15.2:0/3478630763 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe35800f040 con 0x55ec339f1900 2023-09-13T04:48:57.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.431+0000 7fe33f7fe700 1 -- 172.21.15.2:0/3478630763 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fe358011890 con 0x55ec339f1900 2023-09-13T04:48:57.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.431+0000 7fe379fa6b80 10 client.26720 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:48:57.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.431+0000 7fe379fa6b80 10 client.26720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T04:48:57.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.431+0000 7fe379fa6b80 20 client.26720 populate_metadata read hostname 'smithi002' 2023-09-13T04:48:57.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.431+0000 7fe379fa6b80 1 --2- 172.21.15.2:0/3478630763 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55ec33a59860 0x55ec33a5bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:48:57.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.431+0000 7fe379fa6b80 4 client.26720 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-09-13T04:48:57.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.431+0000 7fe379fa6b80 1 -- 172.21.15.2:0/3478630763 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ec338f7050 con 0x55ec33a59860 2023-09-13T04:48:57.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.431+0000 7fe35ffff700 1 --2- 172.21.15.2:0/3478630763 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55ec33a59860 0x55ec33a5bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:48:57.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.432+0000 7fe35ffff700 1 --2- 172.21.15.2:0/3478630763 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55ec33a59860 0x55ec33a5bc50 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9921b8da055965a2 server_cookie=6395b5e1655a646e in_seq=0 out_seq=0 2023-09-13T04:48:57.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.432+0000 7fe33f7fe700 10 client.26720 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:57.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.432+0000 7fe33f7fe700 1 -- 172.21.15.2:0/3478630763 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ec338f7050 con 0x55ec33a59860 2023-09-13T04:48:57.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.432+0000 7fe33f7fe700 10 client.26720 handle_command_reply: tid=0 2023-09-13T04:48:57.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.432+0000 7fe379fa6b80 1 client.26720 shutdown 2023-09-13T04:48:57.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.432+0000 7fe379fa6b80 1 -- 172.21.15.2:0/3478630763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec339f1900 msgr2=0x55ec339eca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:57.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.432+0000 7fe379fa6b80 1 --2- 172.21.15.2:0/3478630763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec339f1900 0x55ec339eca60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe3580061e0 tx=0x7fe3580077b0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.433+0000 7fe379fa6b80 1 -- 172.21.15.2:0/3478630763 shutdown_connections 2023-09-13T04:48:57.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.433+0000 7fe379fa6b80 1 --2- 172.21.15.2:0/3478630763 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55ec33a59860 0x55ec33a5bc50 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.433+0000 7fe379fa6b80 1 --2- 172.21.15.2:0/3478630763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec339f1900 0x55ec339eca60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.433+0000 7fe379fa6b80 1 --2- 172.21.15.2:0/3478630763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec339eba00 0x55ec339edb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.433+0000 7fe33f7fe700 0 client.26720 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:48:57.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.433+0000 7fe379fa6b80 1 --2- 172.21.15.2:0/3478630763 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec339f0fe0 0x55ec339ee090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.433+0000 7fe379fa6b80 1 -- 172.21.15.2:0/3478630763 >> 172.21.15.2:0/3478630763 conn(0x55ec33949c80 msgr2=0x55ec339efcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:57.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.433+0000 7fe379fa6b80 1 -- 172.21.15.2:0/3478630763 shutdown_connections 2023-09-13T04:48:57.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.433+0000 7fe379fa6b80 1 -- 172.21.15.2:0/3478630763 wait complete. 2023-09-13T04:48:57.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.433+0000 7fe379fa6b80 20 client.26720 trim_cache size 0 max 16384 2023-09-13T04:48:57.448 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-09-13T04:48:57.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.434+0000 7fe3665ff700 1 -- 172.21.15.2:0/2047907339 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe3480a0830 msgr2=0x7fe3480a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:57.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.434+0000 7fe3665ff700 1 --2- 172.21.15.2:0/2047907339 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe3480a0830 0x7fe3480a2cf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe354005ea0 tx=0x7fe354005e30 comp rx=0 tx=0).stop 2023-09-13T04:48:57.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.434+0000 7fe3665ff700 1 -- 172.21.15.2:0/2047907339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe360103100 msgr2=0x7fe36010f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:48:57.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.434+0000 7fe3665ff700 1 --2- 172.21.15.2:0/2047907339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe360103100 0x7fe36010f790 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe350005910 tx=0x7fe350004730 comp rx=0 tx=0).stop 2023-09-13T04:48:57.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.434+0000 7fe3665ff700 1 -- 172.21.15.2:0/2047907339 shutdown_connections 2023-09-13T04:48:57.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.434+0000 7fe3665ff700 1 --2- 172.21.15.2:0/2047907339 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fe3480a0830 0x7fe3480a2cf0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.434+0000 7fe3665ff700 1 --2- 172.21.15.2:0/2047907339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe36010c7f0 0x7fe36010cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.434+0000 7fe3665ff700 1 --2- 172.21.15.2:0/2047907339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe360103100 0x7fe36010f790 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.434+0000 7fe3665ff700 1 --2- 172.21.15.2:0/2047907339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe360108220 0x7fe36010fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:48:57.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.434+0000 7fe3665ff700 1 -- 172.21.15.2:0/2047907339 >> 172.21.15.2:0/2047907339 conn(0x7fe3600fe940 msgr2=0x7fe360106ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:48:57.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.434+0000 7fe3665ff700 1 -- 172.21.15.2:0/2047907339 shutdown_connections 2023-09-13T04:48:57.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:48:57.435+0000 7fe3665ff700 1 -- 172.21.15.2:0/2047907339 wait complete. 2023-09-13T04:48:57.452 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:57 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1435476030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:48:57.453 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:57 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1435476030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:48:57.456 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 38892227, 'ino': 2199023257041, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-52.xml'}, {'damage_type': 'backtrace', 'id': 151342215, 'ino': 1099511630004, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-44.xml'}, {'damage_type': 'backtrace', 'id': 161001000, 'ino': 1099511629987, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-52.xml'}, {'damage_type': 'backtrace', 'id': 748306547, 'ino': 1099511631051, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-15.xml'}, {'damage_type': 'backtrace', 'id': 951973817, 'ino': 1099511629996, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-48.xml'}, {'damage_type': 'backtrace', 'id': 974112554, 'ino': 2199023257117, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-14.xml'}, {'damage_type': 'backtrace', 'id': 1014578855, 'ino': 1099511630655, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-33.xml'}, {'damage_type': 'backtrace', 'id': 1163426907, 'ino': 1099511631507, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-16.xml'}, {'damage_type': 'backtrace', 'id': 1203397643, 'ino': 1099511631072, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-7.xml'}, {'damage_type': 'backtrace', 'id': 1277592931, 'ino': 1099511630030, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-31.xml'}, {'damage_type': 'backtrace', 'id': 1404451554, 'ino': 2199023257125, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-10.xml'}, {'damage_type': 'backtrace', 'id': 1483557343, 'ino': 1099511631074, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-6.xml'}, {'damage_type': 'backtrace', 'id': 1545324818, 'ino': 2199023257171, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-12.jpg'}, {'damage_type': 'backtrace', 'id': 1546233816, 'ino': 1099511629992, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-50.xml'}, {'damage_type': 'backtrace', 'id': 1655972945, 'ino': 2199023257147, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-22.jpg'}, {'damage_type': 'backtrace', 'id': 1665254911, 'ino': 2199023257185, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-5.jpg'}, {'damage_type': 'backtrace', 'id': 1731443853, 'ino': 2199023257123, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-11.xml'}, {'damage_type': 'backtrace', 'id': 1801752941, 'ino': 2199023256984, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-71.xml'}, {'damage_type': 'backtrace', 'id': 1820958268, 'ino': 2199023257145, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-23.jpg'}, {'damage_type': 'backtrace', 'id': 1833957116, 'ino': 2199023257165, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-15.jpg'}, {'damage_type': 'backtrace', 'id': 1877034031, 'ino': 2199023257091, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-28.xml'}, {'damage_type': 'backtrace', 'id': 1912082278, 'ino': 2199023257049, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-49.xml'}, {'damage_type': 'backtrace', 'id': 1931285765, 'ino': 2199023257103, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-22.xml'}, {'damage_type': 'backtrace', 'id': 2019923714, 'ino': 1099511630022, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-35.xml'}, {'damage_type': 'backtrace', 'id': 2060254871, 'ino': 1099511631047, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-17.xml'}, {'damage_type': 'backtrace', 'id': 2081770046, 'ino': 2199023257189, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-3.jpg'}, {'damage_type': 'backtrace', 'id': 2152209473, 'ino': 1099511631071, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-8.xml'}, {'damage_type': 'backtrace', 'id': 2158798145, 'ino': 1099511630127, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-79.jpg'}, {'damage_type': 'backtrace', 'id': 2321888234, 'ino': 2199023257089, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-29.xml'}, {'damage_type': 'backtrace', 'id': 2457930243, 'ino': 2199023257035, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-54.xml'}, {'damage_type': 'backtrace', 'id': 2468678708, 'ino': 1099511631068, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-9.xml'}, {'damage_type': 'backtrace', 'id': 2472059171, 'ino': 1099511631035, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-27.xml'}, {'damage_type': 'backtrace', 'id': 2472931663, 'ino': 2199023257137, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-4.xml'}, {'damage_type': 'backtrace', 'id': 2486018571, 'ino': 2199023257079, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-34.xml'}, {'damage_type': 'backtrace', 'id': 2527519661, 'ino': 2199023257023, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-58.xml'}, {'damage_type': 'backtrace', 'id': 2605691590, 'ino': 2199023257097, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-25.xml'}, {'damage_type': 'backtrace', 'id': 2630414652, 'ino': 1099511631042, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-22.xml'}, {'damage_type': 'backtrace', 'id': 2663621206, 'ino': 2199023257051, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-48.xml'}, {'damage_type': 'backtrace', 'id': 2800202397, 'ino': 2199023257055, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-46.xml'}, {'damage_type': 'backtrace', 'id': 2889064089, 'ino': 1099511631546, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-43.xml'}, {'damage_type': 'backtrace', 'id': 2915809936, 'ino': 1099511630015, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-38.xml'}, {'damage_type': 'backtrace', 'id': 2920693249, 'ino': 1099511631034, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-28.xml'}, {'damage_type': 'backtrace', 'id': 2953327874, 'ino': 2199023257135, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-5.xml'}, {'damage_type': 'backtrace', 'id': 2984841417, 'ino': 2199023257071, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-38.xml'}, {'damage_type': 'backtrace', 'id': 3009921519, 'ino': 1099511630005, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-43.xml'}, {'damage_type': 'backtrace', 'id': 3192303016, 'ino': 1099511631079, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-2.xml'}, {'damage_type': 'backtrace', 'id': 3213502603, 'ino': 2199023257095, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-26.xml'}, {'damage_type': 'backtrace', 'id': 3227661194, 'ino': 1099511631060, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-10.xml'}, {'damage_type': 'backtrace', 'id': 3246650744, 'ino': 1099511630654, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-99.jpg'}, {'damage_type': 'backtrace', 'id': 3468323463, 'ino': 2199023257026, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-57.xml'}, {'damage_type': 'backtrace', 'id': 3537076758, 'ino': 1099511629986, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-53.xml'}, {'damage_type': 'backtrace', 'id': 3620397869, 'ino': 1099511630002, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/comment-45.xml'}, {'damage_type': 'backtrace', 'id': 3709572885, 'ino': 2199023257173, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/picture-11.jpg'}, {'damage_type': 'backtrace', 'id': 3764334718, 'ino': 2199023257017, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-60.xml'}, {'damage_type': 'backtrace', 'id': 3948640725, 'ino': 2199023257038, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-53.xml'}, {'damage_type': 'backtrace', 'id': 4118117433, 'ino': 2199023257053, 'path': '/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13/article-47.xml'}] 2023-09-13T04:48:57.456 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2023-09-13T04:48:58.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:58 smithi149 ceph-mon[133315]: pgmap v262: 129 pgs: 129 active+clean; 1013 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 337 op/s 2023-09-13T04:48:58.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:58 smithi156 ceph-mon[133675]: pgmap v262: 129 pgs: 129 active+clean; 1013 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 337 op/s 2023-09-13T04:48:58.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:58 smithi002 ceph-mon[97651]: pgmap v262: 129 pgs: 129 active+clean; 1013 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 337 op/s 2023-09-13T04:48:59.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:48:59 smithi149 ceph-mon[133315]: pgmap v263: 129 pgs: 129 active+clean; 1013 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 349 op/s 2023-09-13T04:48:59.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:48:59 smithi156 ceph-mon[133675]: pgmap v263: 129 pgs: 129 active+clean; 1013 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 349 op/s 2023-09-13T04:48:59.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:48:59 smithi002 ceph-mon[97651]: pgmap v263: 129 pgs: 129 active+clean; 1013 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 349 op/s 2023-09-13T04:49:02.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:01 smithi149 ceph-mon[133315]: pgmap v264: 129 pgs: 129 active+clean; 1021 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 427 op/s 2023-09-13T04:49:02.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:01 smithi156 ceph-mon[133675]: pgmap v264: 129 pgs: 129 active+clean; 1021 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 427 op/s 2023-09-13T04:49:02.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:01 smithi002 ceph-mon[97651]: pgmap v264: 129 pgs: 129 active+clean; 1021 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 427 op/s 2023-09-13T04:49:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.412+0000 7f895ddff700 1 -- 172.21.15.2:0/1947395545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8958101880 msgr2=0x7f89581058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:49:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.412+0000 7f895ddff700 1 --2- 172.21.15.2:0/1947395545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8958101880 0x7f89581058c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f894c009990 tx=0x7f894c009ca0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.412+0000 7f895ddff700 1 -- 172.21.15.2:0/1947395545 shutdown_connections 2023-09-13T04:49:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.412+0000 7f895ddff700 1 --2- 172.21.15.2:0/1947395545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8958105ff0 0x7f895810db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.412+0000 7f895ddff700 1 --2- 172.21.15.2:0/1947395545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8958100ed0 0x7f89581012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.412+0000 7f895ddff700 1 --2- 172.21.15.2:0/1947395545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8958101880 0x7f89581058c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.412+0000 7f895ddff700 1 -- 172.21.15.2:0/1947395545 >> 172.21.15.2:0/1947395545 conn(0x7f89580fc750 msgr2=0x7f89580feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:49:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.412+0000 7f895ddff700 1 -- 172.21.15.2:0/1947395545 shutdown_connections 2023-09-13T04:49:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.412+0000 7f895ddff700 1 -- 172.21.15.2:0/1947395545 wait complete. 2023-09-13T04:49:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.413+0000 7f895ddff700 1 Processor -- start 2023-09-13T04:49:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.414+0000 7f895ddff700 1 -- start start 2023-09-13T04:49:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.414+0000 7f895ddff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8958100ed0 0x7f8958198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:49:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.414+0000 7f895ddff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8958105ff0 0x7f8958199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:49:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.414+0000 7f895ddff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f895819db80 0x7f895819e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:49:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.414+0000 7f895ddff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895819e600 con 0x7f8958105ff0 2023-09-13T04:49:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.414+0000 7f895ddff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895819e770 con 0x7f8958100ed0 2023-09-13T04:49:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.414+0000 7f895ddff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895819e930 con 0x7f895819db80 2023-09-13T04:49:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.414+0000 7f8956ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8958105ff0 0x7f8958199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:49:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.414+0000 7f89577fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8958100ed0 0x7f8958198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:49:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.415+0000 7f8956ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8958105ff0 0x7f8958199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43188/0 (socket says 172.21.15.2:43188) 2023-09-13T04:49:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.415+0000 7f8956ffd700 1 -- 172.21.15.2:0/3527076143 learned_addr learned my addr 172.21.15.2:0/3527076143 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:49:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.415+0000 7f8957fff700 1 --2- 172.21.15.2:0/3527076143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f895819db80 0x7f895819e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:49:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.415+0000 7f8956ffd700 1 -- 172.21.15.2:0/3527076143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f895819db80 msgr2=0x7f895819e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:49:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.415+0000 7f8956ffd700 1 --2- 172.21.15.2:0/3527076143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f895819db80 0x7f895819e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.415+0000 7f8956ffd700 1 -- 172.21.15.2:0/3527076143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8958100ed0 msgr2=0x7f8958198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:49:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.415+0000 7f8956ffd700 1 --2- 172.21.15.2:0/3527076143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8958100ed0 0x7f8958198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.415+0000 7f8956ffd700 1 -- 172.21.15.2:0/3527076143 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8950009c40 con 0x7f8958105ff0 2023-09-13T04:49:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.415+0000 7f89577fe700 1 --2- 172.21.15.2:0/3527076143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8958100ed0 0x7f8958198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:49:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.415+0000 7f8956ffd700 1 --2- 172.21.15.2:0/3527076143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8958105ff0 0x7f8958199510 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f894c0059d0 tx=0x7f894c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:49:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.415+0000 7f8954ff9700 1 -- 172.21.15.2:0/3527076143 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c01d070 con 0x7f8958105ff0 2023-09-13T04:49:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.415+0000 7f895ddff700 1 -- 172.21.15.2:0/3527076143 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f894c0095f0 con 0x7f8958105ff0 2023-09-13T04:49:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.416+0000 7f8954ff9700 1 -- 172.21.15.2:0/3527076143 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f894c022470 con 0x7f8958105ff0 2023-09-13T04:49:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.416+0000 7f8954ff9700 1 -- 172.21.15.2:0/3527076143 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c017650 con 0x7f8958105ff0 2023-09-13T04:49:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.416+0000 7f895ddff700 1 -- 172.21.15.2:0/3527076143 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89581a6120 con 0x7f8958105ff0 2023-09-13T04:49:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.419+0000 7f8954ff9700 1 -- 172.21.15.2:0/3527076143 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f894c0225e0 con 0x7f8958105ff0 2023-09-13T04:49:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.420+0000 7f897169fb80 1 Processor -- start 2023-09-13T04:49:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.420+0000 7f8954ff9700 1 --2- 172.21.15.2:0/3527076143 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f89400a0830 0x7f89400a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:49:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.420+0000 7f8954ff9700 1 -- 172.21.15.2:0/3527076143 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f894c0cd620 con 0x7f8958105ff0 2023-09-13T04:49:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.420+0000 7f897169fb80 1 -- start start 2023-09-13T04:49:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.421+0000 7f89577fe700 1 --2- 172.21.15.2:0/3527076143 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f89400a0830 0x7f89400a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:49:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.421+0000 7f897169fb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa58be0 0x55ebcfa58fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:49:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.421+0000 7f897169fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ebcfa59500 0x55ebcfa54300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:49:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.421+0000 7f897169fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebcfa54840 0x55ebcfa54c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:49:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.421+0000 7f897169fb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcf9ad720 con 0x55ebcfa58be0 2023-09-13T04:49:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.421+0000 7f897169fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcf9ab040 con 0x55ebcfa59500 2023-09-13T04:49:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.421+0000 7f897169fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcf9a6930 con 0x55ebcfa54840 2023-09-13T04:49:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.421+0000 7f89577fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa58be0 0x55ebcfa58fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:49:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.421+0000 7f8957fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebcfa54840 0x55ebcfa54c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:49:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.421+0000 7f89577fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa58be0 0x55ebcfa58fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43202/0 (socket says 172.21.15.2:43202) 2023-09-13T04:49:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.421+0000 7f89577fe700 1 -- 172.21.15.2:0/3323046566 learned_addr learned my addr 172.21.15.2:0/3323046566 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:49:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.421+0000 7f8956ffd700 1 --2- 172.21.15.2:0/3323046566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ebcfa59500 0x55ebcfa54300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:49:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.421+0000 7f8957fff700 1 -- 172.21.15.2:0/3323046566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ebcfa59500 msgr2=0x55ebcfa54300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:49:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.421+0000 7f8957fff700 1 --2- 172.21.15.2:0/3323046566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ebcfa59500 0x55ebcfa54300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.421+0000 7f8957fff700 1 -- 172.21.15.2:0/3323046566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa58be0 msgr2=0x55ebcfa58fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:49:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.421+0000 7f8957fff700 1 --2- 172.21.15.2:0/3323046566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa58be0 0x55ebcfa58fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.421+0000 7f8957fff700 1 -- 172.21.15.2:0/3323046566 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebcf8265d0 con 0x55ebcfa54840 2023-09-13T04:49:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.422+0000 7f89577fe700 1 --2- 172.21.15.2:0/3527076143 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f89400a0830 0x7f89400a2cf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f89480139a0 tx=0x7f8948013d60 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:49:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.422+0000 7f8957fff700 1 --2- 172.21.15.2:0/3323046566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebcfa54840 0x55ebcfa54c80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f895000e650 tx=0x7f8950009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f86f2aeeb337ea4 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:49:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.422+0000 7f893dffb700 1 -- 172.21.15.2:0/3323046566 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895000e040 con 0x55ebcfa54840 2023-09-13T04:49:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.422+0000 7f893dffb700 1 -- 172.21.15.2:0/3323046566 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f895000f040 con 0x55ebcfa54840 2023-09-13T04:49:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.422+0000 7f893dffb700 1 -- 172.21.15.2:0/3323046566 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950007500 con 0x55ebcfa54840 2023-09-13T04:49:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.423+0000 7f897169fb80 1 -- 172.21.15.2:0/3323046566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebcfa54840 msgr2=0x55ebcfa54c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:49:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.423+0000 7f897169fb80 1 --2- 172.21.15.2:0/3323046566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebcfa54840 0x55ebcfa54c80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f895000e650 tx=0x7f8950009920 comp rx=0 tx=0).stop 2023-09-13T04:49:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.423+0000 7f897169fb80 1 -- 172.21.15.2:0/3323046566 shutdown_connections 2023-09-13T04:49:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.423+0000 7f897169fb80 1 --2- 172.21.15.2:0/3323046566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebcfa54840 0x55ebcfa54c80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.423+0000 7f897169fb80 1 --2- 172.21.15.2:0/3323046566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa58be0 0x55ebcfa58fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.423+0000 7f897169fb80 1 --2- 172.21.15.2:0/3323046566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ebcfa59500 0x55ebcfa54300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.423+0000 7f897169fb80 1 -- 172.21.15.2:0/3323046566 >> 172.21.15.2:0/3323046566 conn(0x55ebcf9b2c20 msgr2=0x55ebcfa4eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:49:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.423+0000 7f897169fb80 1 -- 172.21.15.2:0/3323046566 shutdown_connections 2023-09-13T04:49:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.423+0000 7f897169fb80 1 -- 172.21.15.2:0/3323046566 wait complete. 2023-09-13T04:49:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.423+0000 7f897169fb80 1 -- start start 2023-09-13T04:49:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.424+0000 7f897169fb80 1 Processor -- start 2023-09-13T04:49:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.424+0000 7f897169fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ebcfa54840 0x55ebcfa605e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:49:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.424+0000 7f897169fb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa58be0 0x55ebcfa5d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:49:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.424+0000 7f897169fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebcfa59500 0x55ebcfa5dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:49:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.424+0000 7f897169fb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcf944b50 con 0x55ebcfa58be0 2023-09-13T04:49:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.424+0000 7f897169fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcf944850 con 0x55ebcfa54840 2023-09-13T04:49:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.424+0000 7f897169fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcf8288c0 con 0x55ebcfa59500 2023-09-13T04:49:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.424+0000 7f897169fb80 10 client.0 _ll_register_callbacks cb 0x55ebcf942620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:49:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.425+0000 7f8956ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa58be0 0x55ebcfa5d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:49:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.425+0000 7f8956ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa58be0 0x55ebcfa5d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43206/0 (socket says 172.21.15.2:43206) 2023-09-13T04:49:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.425+0000 7f8956ffd700 1 -- 172.21.15.2:0/1798407859 learned_addr learned my addr 172.21.15.2:0/1798407859 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:49:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.425+0000 7f89577fe700 1 --2- 172.21.15.2:0/1798407859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ebcfa54840 0x55ebcfa605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:49:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.425+0000 7f8957fff700 1 --2- 172.21.15.2:0/1798407859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebcfa59500 0x55ebcfa5dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:49:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.425+0000 7f8956ffd700 1 -- 172.21.15.2:0/1798407859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebcfa59500 msgr2=0x55ebcfa5dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:49:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.425+0000 7f8956ffd700 1 --2- 172.21.15.2:0/1798407859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebcfa59500 0x55ebcfa5dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.425+0000 7f8956ffd700 1 -- 172.21.15.2:0/1798407859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ebcfa54840 msgr2=0x55ebcfa605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:49:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.425+0000 7f8956ffd700 1 --2- 172.21.15.2:0/1798407859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ebcfa54840 0x55ebcfa605e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.425+0000 7f8956ffd700 1 -- 172.21.15.2:0/1798407859 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895000e330 con 0x55ebcfa58be0 2023-09-13T04:49:02.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.425+0000 7f8957fff700 1 --2- 172.21.15.2:0/1798407859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebcfa59500 0x55ebcfa5dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:49:02.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.425+0000 7f89577fe700 1 --2- 172.21.15.2:0/1798407859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ebcfa54840 0x55ebcfa605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:49:02.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.425+0000 7f8956ffd700 1 --2- 172.21.15.2:0/1798407859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa58be0 0x55ebcfa5d7f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f894c00f740 tx=0x7f894c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa71caf49feda99 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:49:02.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.425+0000 7f8936ffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-09-13T04:49:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.425+0000 7f8936ffd700 1 -- 172.21.15.2:0/1798407859 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c0cd3e0 con 0x55ebcfa58be0 2023-09-13T04:49:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.425+0000 7f8936ffd700 1 -- 172.21.15.2:0/1798407859 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f894c09afb0 con 0x55ebcfa58be0 2023-09-13T04:49:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.425+0000 7f897169fb80 1 -- 172.21.15.2:0/1798407859 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ebcfa19ef0 con 0x55ebcfa58be0 2023-09-13T04:49:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.426+0000 7f8936ffd700 1 -- 172.21.15.2:0/1798407859 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f894c09a8b0 con 0x55ebcfa58be0 2023-09-13T04:49:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.426+0000 7f897169fb80 10 client.16563 fetch_fsmap learned FSMap version 36 2023-09-13T04:49:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.426+0000 7f897169fb80 1 -- 172.21.15.2:0/1798407859 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55ebcf9be740 con 0x55ebcfa58be0 2023-09-13T04:49:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.426+0000 7f8936ffd700 1 -- 172.21.15.2:0/1798407859 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ebcfa19ef0 con 0x55ebcfa58be0 2023-09-13T04:49:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.426+0000 7f8936ffd700 1 -- 172.21.15.2:0/1798407859 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f894c0a2400 con 0x55ebcfa58be0 2023-09-13T04:49:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.426+0000 7f897169fb80 10 client.16563 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:49:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.426+0000 7f897169fb80 10 client.16563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:49:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.426+0000 7f897169fb80 20 client.16563 populate_metadata read hostname 'smithi002' 2023-09-13T04:49:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.426+0000 7f897169fb80 1 --2- 172.21.15.2:0/1798407859 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55ebcfa6db20 0x55ebcfa6ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:49:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.426+0000 7f897169fb80 4 client.16563 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-13T04:49:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.426+0000 7f897169fb80 1 -- 172.21.15.2:0/1798407859 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ebcf99f910 con 0x55ebcfa6db20 2023-09-13T04:49:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.427+0000 7f89577fe700 1 --2- 172.21.15.2:0/1798407859 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55ebcfa6db20 0x55ebcfa6ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:49:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.427+0000 7f89577fe700 1 --2- 172.21.15.2:0/1798407859 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55ebcfa6db20 0x55ebcfa6ff40 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dda28df63fa5a096 server_cookie=caa5b4b65f63009c in_seq=0 out_seq=0 2023-09-13T04:49:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.427+0000 7f8936ffd700 10 client.16563 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:49:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.428+0000 7f8936ffd700 1 -- 172.21.15.2:0/1798407859 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ebcf99f910 con 0x55ebcfa6db20 2023-09-13T04:49:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.428+0000 7f8936ffd700 10 client.16563 handle_command_reply: tid=0 2023-09-13T04:49:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.428+0000 7f897169fb80 1 client.16563 shutdown 2023-09-13T04:49:02.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.428+0000 7f897169fb80 1 -- 172.21.15.2:0/1798407859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa58be0 msgr2=0x55ebcfa5d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:49:02.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.428+0000 7f897169fb80 1 --2- 172.21.15.2:0/1798407859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa58be0 0x55ebcfa5d7f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f894c00f740 tx=0x7f894c005fd0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.428+0000 7f897169fb80 1 -- 172.21.15.2:0/1798407859 shutdown_connections 2023-09-13T04:49:02.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.428+0000 7f897169fb80 1 --2- 172.21.15.2:0/1798407859 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55ebcfa6db20 0x55ebcfa6ff40 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.428+0000 7f897169fb80 1 --2- 172.21.15.2:0/1798407859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebcfa59500 0x55ebcfa5dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.428+0000 7f897169fb80 1 --2- 172.21.15.2:0/1798407859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ebcfa54840 0x55ebcfa605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.428+0000 7f897169fb80 1 --2- 172.21.15.2:0/1798407859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa58be0 0x55ebcfa5d7f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.428+0000 7f897169fb80 1 -- 172.21.15.2:0/1798407859 >> 172.21.15.2:0/1798407859 conn(0x55ebcf9b2c20 msgr2=0x55ebcfa49630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:49:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.428+0000 7f8936ffd700 0 client.16563 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:49:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.428+0000 7f897169fb80 1 -- 172.21.15.2:0/1798407859 shutdown_connections 2023-09-13T04:49:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.428+0000 7f897169fb80 1 -- 172.21.15.2:0/1798407859 wait complete. 2023-09-13T04:49:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.428+0000 7f897169fb80 20 client.16563 trim_cache size 0 max 16384 2023-09-13T04:49:02.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.444+0000 7f897169fb80 1 Processor -- start 2023-09-13T04:49:02.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.444+0000 7f897169fb80 1 -- start start 2023-09-13T04:49:02.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.444+0000 7f897169fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebcfa54840 0x55ebcfa49730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:49:02.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.444+0000 7f897169fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ebcfa58be0 0x55ebcfa522f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:49:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.444+0000 7f897169fb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa59500 0x55ebcfa60110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:49:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.444+0000 7f897169fb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcfa47340 con 0x55ebcfa59500 2023-09-13T04:49:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.444+0000 7f897169fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcfa5ecb0 con 0x55ebcfa58be0 2023-09-13T04:49:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.444+0000 7f897169fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcfa58750 con 0x55ebcfa54840 2023-09-13T04:49:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.444+0000 7f8957fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa59500 0x55ebcfa60110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:49:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.444+0000 7f8957fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa59500 0x55ebcfa60110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43212/0 (socket says 172.21.15.2:43212) 2023-09-13T04:49:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.444+0000 7f8957fff700 1 -- 172.21.15.2:0/2012894390 learned_addr learned my addr 172.21.15.2:0/2012894390 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:49:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.444+0000 7f89577fe700 1 --2- 172.21.15.2:0/2012894390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebcfa54840 0x55ebcfa49730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:49:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.445+0000 7f8956ffd700 1 --2- 172.21.15.2:0/2012894390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ebcfa58be0 0x55ebcfa522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:49:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.445+0000 7f8957fff700 1 -- 172.21.15.2:0/2012894390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebcfa54840 msgr2=0x55ebcfa49730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:49:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.445+0000 7f8957fff700 1 --2- 172.21.15.2:0/2012894390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebcfa54840 0x55ebcfa49730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.445+0000 7f8957fff700 1 -- 172.21.15.2:0/2012894390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ebcfa58be0 msgr2=0x55ebcfa522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:49:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.445+0000 7f8957fff700 1 --2- 172.21.15.2:0/2012894390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ebcfa58be0 0x55ebcfa522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.445+0000 7f8957fff700 1 -- 172.21.15.2:0/2012894390 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebcf7d2d80 con 0x55ebcfa59500 2023-09-13T04:49:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.445+0000 7f8956ffd700 1 --2- 172.21.15.2:0/2012894390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ebcfa58be0 0x55ebcfa522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:49:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.445+0000 7f8957fff700 1 --2- 172.21.15.2:0/2012894390 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa59500 0x55ebcfa60110 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f895000e650 tx=0x7f8950009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422e1b0d392cc33d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:49:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.445+0000 7f893e7fc700 1 -- 172.21.15.2:0/2012894390 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895000e040 con 0x55ebcfa59500 2023-09-13T04:49:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.445+0000 7f893e7fc700 1 -- 172.21.15.2:0/2012894390 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89500079c0 con 0x55ebcfa59500 2023-09-13T04:49:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.445+0000 7f893e7fc700 1 -- 172.21.15.2:0/2012894390 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950005560 con 0x55ebcfa59500 2023-09-13T04:49:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.446+0000 7f897169fb80 1 -- 172.21.15.2:0/2012894390 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa59500 msgr2=0x55ebcfa60110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:49:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.446+0000 7f897169fb80 1 --2- 172.21.15.2:0/2012894390 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa59500 0x55ebcfa60110 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f895000e650 tx=0x7f8950009650 comp rx=0 tx=0).stop 2023-09-13T04:49:02.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.446+0000 7f897169fb80 1 -- 172.21.15.2:0/2012894390 shutdown_connections 2023-09-13T04:49:02.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.446+0000 7f897169fb80 1 --2- 172.21.15.2:0/2012894390 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa59500 0x55ebcfa60110 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.446+0000 7f897169fb80 1 --2- 172.21.15.2:0/2012894390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebcfa54840 0x55ebcfa49730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.446+0000 7f897169fb80 1 --2- 172.21.15.2:0/2012894390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ebcfa58be0 0x55ebcfa522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.446+0000 7f897169fb80 1 -- 172.21.15.2:0/2012894390 >> 172.21.15.2:0/2012894390 conn(0x55ebcf9b2c20 msgr2=0x55ebcfa69670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:49:02.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.446+0000 7f897169fb80 1 -- 172.21.15.2:0/2012894390 shutdown_connections 2023-09-13T04:49:02.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.446+0000 7f897169fb80 1 -- 172.21.15.2:0/2012894390 wait complete. 2023-09-13T04:49:02.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.446+0000 7f897169fb80 1 -- start start 2023-09-13T04:49:02.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.447+0000 7f897169fb80 1 Processor -- start 2023-09-13T04:49:02.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.447+0000 7f897169fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ebcfa54840 0x55ebcfa62b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:49:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.447+0000 7f897169fb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa58be0 0x55ebcfa63040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:49:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.448+0000 7f897169fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebcfa59500 0x55ebcfac0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:49:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.448+0000 7f897169fb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcfa63700 con 0x55ebcfa58be0 2023-09-13T04:49:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.448+0000 7f897169fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcfac0e90 con 0x55ebcfa54840 2023-09-13T04:49:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.448+0000 7f897169fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcfac0fd0 con 0x55ebcfa59500 2023-09-13T04:49:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.448+0000 7f897169fb80 10 client.0 _ll_register_callbacks cb 0x55ebcf942620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T04:49:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.448+0000 7f89577fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ebcfa54840 0x55ebcfa62b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:49:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.448+0000 7f89577fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ebcfa54840 0x55ebcfa62b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:59608/0 (socket says 172.21.15.2:59608) 2023-09-13T04:49:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.448+0000 7f89577fe700 1 -- 172.21.15.2:0/4088835105 learned_addr learned my addr 172.21.15.2:0/4088835105 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:49:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.448+0000 7f8956ffd700 1 --2- 172.21.15.2:0/4088835105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa58be0 0x55ebcfa63040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:49:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.448+0000 7f8957fff700 1 --2- 172.21.15.2:0/4088835105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebcfa59500 0x55ebcfac0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:49:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.448+0000 7f89577fe700 1 -- 172.21.15.2:0/4088835105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebcfa59500 msgr2=0x55ebcfac0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:49:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.448+0000 7f89577fe700 1 --2- 172.21.15.2:0/4088835105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebcfa59500 0x55ebcfac0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.448+0000 7f89577fe700 1 -- 172.21.15.2:0/4088835105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa58be0 msgr2=0x55ebcfa63040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T04:49:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.448+0000 7f89577fe700 1 --2- 172.21.15.2:0/4088835105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa58be0 0x55ebcfa63040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.448+0000 7f89577fe700 1 -- 172.21.15.2:0/4088835105 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ebcf8265d0 con 0x55ebcfa54840 2023-09-13T04:49:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.449+0000 7f89577fe700 1 --2- 172.21.15.2:0/4088835105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ebcfa54840 0x55ebcfa62b00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55ebcfa70850 tx=0x7f89480047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=126a9a575fcbb856 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:49:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.449+0000 7f8936ffd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-09-13T04:49:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.449+0000 7f8936ffd700 1 -- 172.21.15.2:0/4088835105 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8948018070 con 0x55ebcfa54840 2023-09-13T04:49:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.449+0000 7f897169fb80 1 -- 172.21.15.2:0/4088835105 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f895000f040 con 0x55ebcfa54840 2023-09-13T04:49:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.449+0000 7f8936ffd700 1 -- 172.21.15.2:0/4088835105 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f894800fc60 con 0x55ebcfa54840 2023-09-13T04:49:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.449+0000 7f8936ffd700 1 -- 172.21.15.2:0/4088835105 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f894800fe20 con 0x55ebcfa54840 2023-09-13T04:49:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.449+0000 7f897169fb80 10 client.26738 fetch_fsmap learned FSMap version 36 2023-09-13T04:49:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.449+0000 7f897169fb80 1 -- 172.21.15.2:0/4088835105 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55ebcf7bbc00 con 0x55ebcfa54840 2023-09-13T04:49:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.450+0000 7f8936ffd700 1 -- 172.21.15.2:0/4088835105 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f895000f040 con 0x55ebcfa54840 2023-09-13T04:49:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.450+0000 7f8936ffd700 1 -- 172.21.15.2:0/4088835105 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f89480043c0 con 0x55ebcfa54840 2023-09-13T04:49:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.450+0000 7f897169fb80 10 client.26738 fetch_fsmap finished waiting for FSMap version 36 2023-09-13T04:49:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.450+0000 7f897169fb80 10 client.26738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T04:49:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.450+0000 7f897169fb80 20 client.26738 populate_metadata read hostname 'smithi002' 2023-09-13T04:49:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.450+0000 7f897169fb80 1 --2- 172.21.15.2:0/4088835105 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55ebcfac27c0 0x55ebcfac4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:49:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.450+0000 7f897169fb80 4 client.26738 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-13T04:49:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.450+0000 7f897169fb80 1 -- 172.21.15.2:0/4088835105 --> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ebcf95fff0 con 0x55ebcfac27c0 2023-09-13T04:49:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.451+0000 7f8956ffd700 1 --2- 172.21.15.2:0/4088835105 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55ebcfac27c0 0x55ebcfac4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:49:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.451+0000 7f8936ffd700 10 client.26738 ms_handle_connect on v2:172.21.15.149:6838/389167841 2023-09-13T04:49:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.451+0000 7f8956ffd700 1 --2- 172.21.15.2:0/4088835105 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55ebcfac27c0 0x55ebcfac4be0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdaa3d8c70b88672 server_cookie=ae320dabf779b797 in_seq=0 out_seq=0 2023-09-13T04:49:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.888+0000 7f8936ffd700 1 -- 172.21.15.2:0/4088835105 <== mds.0 v2:172.21.15.149:6838/389167841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ebcf95fff0 con 0x55ebcfac27c0 2023-09-13T04:49:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.888+0000 7f8936ffd700 10 client.26738 handle_command_reply: tid=0 2023-09-13T04:49:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.889+0000 7f897169fb80 1 client.26738 shutdown 2023-09-13T04:49:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.889+0000 7f897169fb80 1 -- 172.21.15.2:0/4088835105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ebcfa54840 msgr2=0x55ebcfa62b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:49:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.889+0000 7f897169fb80 1 --2- 172.21.15.2:0/4088835105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ebcfa54840 0x55ebcfa62b00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55ebcfa70850 tx=0x7f89480047c0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.889+0000 7f897169fb80 1 -- 172.21.15.2:0/4088835105 shutdown_connections 2023-09-13T04:49:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.889+0000 7f897169fb80 1 --2- 172.21.15.2:0/4088835105 >> [v2:172.21.15.149:6838/389167841,v1:172.21.15.149:6839/389167841] conn(0x55ebcfac27c0 0x55ebcfac4be0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.889+0000 7f897169fb80 1 --2- 172.21.15.2:0/4088835105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebcfa59500 0x55ebcfac0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.889+0000 7f897169fb80 1 --2- 172.21.15.2:0/4088835105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ebcfa54840 0x55ebcfa62b00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.889+0000 7f897169fb80 1 --2- 172.21.15.2:0/4088835105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ebcfa58be0 0x55ebcfa63040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.889+0000 7f897169fb80 1 -- 172.21.15.2:0/4088835105 >> 172.21.15.2:0/4088835105 conn(0x55ebcf9b2c20 msgr2=0x55ebcfa57740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:49:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.889+0000 7f8936ffd700 0 client.26738 ms_handle_reset on v2:172.21.15.149:6838/389167841 2023-09-13T04:49:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.889+0000 7f897169fb80 1 -- 172.21.15.2:0/4088835105 shutdown_connections 2023-09-13T04:49:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.889+0000 7f897169fb80 1 -- 172.21.15.2:0/4088835105 wait complete. 2023-09-13T04:49:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.889+0000 7f897169fb80 20 client.26738 trim_cache size 0 max 16384 2023-09-13T04:49:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2023-09-13T04:49:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2023-09-13T04:49:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2023-09-13T04:49:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2023-09-13T04:49:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.890+0000 7f895ddff700 1 -- 172.21.15.2:0/3527076143 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f89400a0830 msgr2=0x7f89400a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:49:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.890+0000 7f895ddff700 1 --2- 172.21.15.2:0/3527076143 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f89400a0830 0x7f89400a2cf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f89480139a0 tx=0x7f8948013d60 comp rx=0 tx=0).stop 2023-09-13T04:49:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.890+0000 7f895ddff700 1 -- 172.21.15.2:0/3527076143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8958105ff0 msgr2=0x7f8958199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:49:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.890+0000 7f895ddff700 1 --2- 172.21.15.2:0/3527076143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8958105ff0 0x7f8958199510 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f894c0059d0 tx=0x7f894c005740 comp rx=0 tx=0).stop 2023-09-13T04:49:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.890+0000 7f895ddff700 1 -- 172.21.15.2:0/3527076143 shutdown_connections 2023-09-13T04:49:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.890+0000 7f895ddff700 1 --2- 172.21.15.2:0/3527076143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f895819db80 0x7f895819e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.890+0000 7f895ddff700 1 --2- 172.21.15.2:0/3527076143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8958100ed0 0x7f8958198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.890+0000 7f895ddff700 1 --2- 172.21.15.2:0/3527076143 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f89400a0830 0x7f89400a2cf0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.890+0000 7f895ddff700 1 --2- 172.21.15.2:0/3527076143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8958105ff0 0x7f8958199510 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:49:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.890+0000 7f895ddff700 1 -- 172.21.15.2:0/3527076143 >> 172.21.15.2:0/3527076143 conn(0x7f89580fc750 msgr2=0x7f89580fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:49:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.890+0000 7f895ddff700 1 -- 172.21.15.2:0/3527076143 shutdown_connections 2023-09-13T04:49:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2023-09-13T04:49:02.890+0000 7f895ddff700 1 -- 172.21.15.2:0/3527076143 wait complete. 2023-09-13T04:49:02.983 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2023-09-13T04:49:02.983 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-09-13T04:49:03.877 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-mon[133675]: pgmap v265: 129 pgs: 129 active+clean; 1021 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 361 op/s 2023-09-13T04:49:03.879 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: *** Caught signal (Segmentation fault) ** 2023-09-13T04:49:03.880 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: in thread 7fcb4ae33700 thread_name:md_log_replay 2023-09-13T04:49:03.880 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: ceph version 18.0.0-6083-g4a754ac9 (4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5) reef (dev) 2023-09-13T04:49:03.880 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fcb5b9e2cf0] 2023-09-13T04:49:03.880 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 2: /usr/bin/ceph-mds(+0x4f3412) [0x560d11c73412] 2023-09-13T04:49:03.880 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x560d11c75663] 2023-09-13T04:49:03.881 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 4: (MDLog::_replay_thread()+0x647) [0x560d11cd0d07] 2023-09-13T04:49:03.881 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 5: (MDLog::ReplayThread::entry()+0x11) [0x560d11cda241] 2023-09-13T04:49:03.881 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fcb5b9d81ca] 2023-09-13T04:49:03.881 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 7: clone() 2023-09-13T04:49:03.881 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 2023-09-13T04:49:03.542+0000 7fcb4ae33700 -1 *** Caught signal (Segmentation fault) ** 2023-09-13T04:49:03.882 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: in thread 7fcb4ae33700 thread_name:md_log_replay 2023-09-13T04:49:03.882 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 2023-09-13T04:49:03.882 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: ceph version 18.0.0-6083-g4a754ac9 (4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5) reef (dev) 2023-09-13T04:49:03.882 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fcb5b9e2cf0] 2023-09-13T04:49:03.882 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 2: /usr/bin/ceph-mds(+0x4f3412) [0x560d11c73412] 2023-09-13T04:49:03.883 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x560d11c75663] 2023-09-13T04:49:03.883 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 4: (MDLog::_replay_thread()+0x647) [0x560d11cd0d07] 2023-09-13T04:49:03.883 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 5: (MDLog::ReplayThread::entry()+0x11) [0x560d11cda241] 2023-09-13T04:49:03.883 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fcb5b9d81ca] 2023-09-13T04:49:03.883 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 7: clone() 2023-09-13T04:49:03.884 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-13T04:49:03.884 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 2023-09-13T04:49:03.884 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 0> 2023-09-13T04:49:03.542+0000 7fcb4ae33700 -1 *** Caught signal (Segmentation fault) ** 2023-09-13T04:49:03.884 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: in thread 7fcb4ae33700 thread_name:md_log_replay 2023-09-13T04:49:03.884 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 2023-09-13T04:49:03.884 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: ceph version 18.0.0-6083-g4a754ac9 (4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5) reef (dev) 2023-09-13T04:49:03.885 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fcb5b9e2cf0] 2023-09-13T04:49:03.885 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 2: /usr/bin/ceph-mds(+0x4f3412) [0x560d11c73412] 2023-09-13T04:49:03.885 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x560d11c75663] 2023-09-13T04:49:03.885 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 4: (MDLog::_replay_thread()+0x647) [0x560d11cd0d07] 2023-09-13T04:49:03.886 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 5: (MDLog::ReplayThread::entry()+0x11) [0x560d11cda241] 2023-09-13T04:49:03.886 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fcb5b9d81ca] 2023-09-13T04:49:03.886 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 7: clone() 2023-09-13T04:49:03.886 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-13T04:49:03.887 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 2023-09-13T04:49:03.887 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: -7> 2023-09-13T04:49:03.542+0000 7fcb4ae33700 -1 *** Caught signal (Segmentation fault) ** 2023-09-13T04:49:03.887 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: in thread 7fcb4ae33700 thread_name:md_log_replay 2023-09-13T04:49:03.887 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 2023-09-13T04:49:03.887 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: ceph version 18.0.0-6083-g4a754ac9 (4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5) reef (dev) 2023-09-13T04:49:03.887 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fcb5b9e2cf0] 2023-09-13T04:49:03.888 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 2: /usr/bin/ceph-mds(+0x4f3412) [0x560d11c73412] 2023-09-13T04:49:03.888 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x560d11c75663] 2023-09-13T04:49:03.888 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 4: (MDLog::_replay_thread()+0x647) [0x560d11cd0d07] 2023-09-13T04:49:03.888 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 5: (MDLog::ReplayThread::entry()+0x11) [0x560d11cda241] 2023-09-13T04:49:03.888 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fcb5b9d81ca] 2023-09-13T04:49:03.889 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 7: clone() 2023-09-13T04:49:03.889 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-13T04:49:03.889 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[175903]: 2023-09-13T04:49:04.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:03 smithi149 ceph-mon[133315]: pgmap v265: 129 pgs: 129 active+clean; 1021 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 361 op/s 2023-09-13T04:49:04.188 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:04 smithi156 systemd[1]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.c.service: Main process exited, code=exited, status=139/n/a 2023-09-13T04:49:04.196 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:03 smithi002 ceph-mon[97651]: pgmap v265: 129 pgs: 129 active+clean; 1021 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 361 op/s 2023-09-13T04:49:04.197 DEBUG:teuthology.parallel:result is None 2023-09-13T04:49:04.198 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2023-09-13T04:49:04.389 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2023-09-13T04:49:04.389 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-09-13T04:49:04.399 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi002.front.sepia.ceph.com 2023-09-13T04:49:04.418 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-09-13T04:49:04.514 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-09-13T04:49:04.514 INFO:teuthology.task.exec:Executing custom commands... 2023-09-13T04:49:04.515 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2023-09-13T04:49:04.515 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-09-13T04:49:04.661 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-09-13T04:49:04.690 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:04 smithi156 systemd[1]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.c.service: Failed with result 'exit-code'. 2023-09-13T04:49:05.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:04 smithi002 ceph-mon[97651]: osdmap e99: 12 total, 12 up, 12 in 2023-09-13T04:49:05.139 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:04 smithi149 ceph-mon[133315]: osdmap e99: 12 total, 12 up, 12 in 2023-09-13T04:49:05.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:04 smithi156 ceph-mon[133675]: osdmap e99: 12 total, 12 up, 12 in 2023-09-13T04:49:06.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:05 smithi156 ceph-mon[133675]: pgmap v267: 129 pgs: 16 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 69 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 331 op/s 2023-09-13T04:49:06.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:05 smithi002 ceph-mon[97651]: pgmap v267: 129 pgs: 16 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 69 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 331 op/s 2023-09-13T04:49:06.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:05 smithi149 ceph-mon[133315]: pgmap v267: 129 pgs: 16 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 69 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 331 op/s 2023-09-13T04:49:08.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:07 smithi156 ceph-mon[133675]: pgmap v268: 129 pgs: 18 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 60 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 321 op/s 2023-09-13T04:49:08.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:07 smithi002 ceph-mon[97651]: pgmap v268: 129 pgs: 18 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 60 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 321 op/s 2023-09-13T04:49:08.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:07 smithi149 ceph-mon[133315]: pgmap v268: 129 pgs: 18 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 60 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 321 op/s 2023-09-13T04:49:10.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:09 smithi156 ceph-mon[133675]: pgmap v269: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 298 op/s 2023-09-13T04:49:10.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:09 smithi002 ceph-mon[97651]: pgmap v269: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 298 op/s 2023-09-13T04:49:10.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:09 smithi149 ceph-mon[133315]: pgmap v269: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 298 op/s 2023-09-13T04:49:12.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:11 smithi002 ceph-mon[97651]: pgmap v270: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 180 op/s 2023-09-13T04:49:12.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:11 smithi149 ceph-mon[133315]: pgmap v270: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 180 op/s 2023-09-13T04:49:12.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:11 smithi156 ceph-mon[133675]: pgmap v270: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 180 op/s 2023-09-13T04:49:14.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:13 smithi002 ceph-mon[97651]: pgmap v271: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1020 KiB/s wr, 134 op/s 2023-09-13T04:49:14.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:13 smithi149 ceph-mon[133315]: pgmap v271: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1020 KiB/s wr, 134 op/s 2023-09-13T04:49:14.432 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:13 smithi156 ceph-mon[133675]: pgmap v271: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1020 KiB/s wr, 134 op/s 2023-09-13T04:49:14.432 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:14 smithi156 systemd[1]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.c.service: Service RestartSec=10s expired, scheduling restart. 2023-09-13T04:49:14.432 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:14 smithi156 systemd[1]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.c.service: Scheduled restart job, restart counter is at 2. 2023-09-13T04:49:14.432 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:14 smithi156 systemd[1]: Stopped Ceph mds.c for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:49:14.689 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:14 smithi156 systemd[1]: Starting Ceph mds.c for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:49:15.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:14 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:49:15.257 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:14 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:49:15.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:14 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:49:15.686 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:15 smithi156 podman[177361]: 2023-09-13T04:49:15.687 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:15 smithi156 bash[177361]: 9d1a23c6dd90dc2235904a17b96fcaddd67cb2f185bb91655d8ff69f019bdaa2 2023-09-13T04:49:15.687 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:15 smithi156 ceph-mds[177411]: -- 172.21.15.156:0/4056722450 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5588313111e0 con 0x5588312e6800 2023-09-13T04:49:15.687 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:15 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[177386]: starting mds.c at 2023-09-13T04:49:15.687 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:49:15 smithi156 systemd[1]: Started Ceph mds.c for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:49:16.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:15 smithi002 ceph-mon[97651]: pgmap v272: 129 pgs: 15 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 66 active+clean; 1.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 962 KiB/s wr, 112 op/s 2023-09-13T04:49:16.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:15 smithi002 ceph-mon[97651]: daemon mds.c restarted 2023-09-13T04:49:16.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:15 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:49:16.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:15 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:49:16.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:15 smithi002 ceph-mon[97651]: osdmap e100: 12 total, 12 up, 12 in 2023-09-13T04:49:16.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:15 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:49:16.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:15 smithi149 ceph-mon[133315]: pgmap v272: 129 pgs: 15 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 66 active+clean; 1.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 962 KiB/s wr, 112 op/s 2023-09-13T04:49:16.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:15 smithi149 ceph-mon[133315]: daemon mds.c restarted 2023-09-13T04:49:16.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:15 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:49:16.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:15 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:49:16.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:15 smithi149 ceph-mon[133315]: osdmap e100: 12 total, 12 up, 12 in 2023-09-13T04:49:16.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:15 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:49:16.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:15 smithi156 ceph-mon[133675]: pgmap v272: 129 pgs: 15 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 66 active+clean; 1.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 962 KiB/s wr, 112 op/s 2023-09-13T04:49:16.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:15 smithi156 ceph-mon[133675]: daemon mds.c restarted 2023-09-13T04:49:16.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:15 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:49:16.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:15 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:49:16.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:15 smithi156 ceph-mon[133675]: osdmap e100: 12 total, 12 up, 12 in 2023-09-13T04:49:16.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:15 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:49:17.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:16 smithi002 ceph-mon[97651]: mds.? [v2:172.21.15.156:6836/3447644694,v1:172.21.15.156:6837/3447644694] up:boot 2023-09-13T04:49:17.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:16 smithi002 ceph-mon[97651]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2023-09-13T04:49:17.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:16 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-13T04:49:17.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:16 smithi002 ceph-mon[97651]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:49:17.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:16 smithi149 ceph-mon[133315]: mds.? [v2:172.21.15.156:6836/3447644694,v1:172.21.15.156:6837/3447644694] up:boot 2023-09-13T04:49:17.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:16 smithi149 ceph-mon[133315]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2023-09-13T04:49:17.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:16 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-13T04:49:17.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:16 smithi149 ceph-mon[133315]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:49:17.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:16 smithi156 ceph-mon[133675]: mds.? [v2:172.21.15.156:6836/3447644694,v1:172.21.15.156:6837/3447644694] up:boot 2023-09-13T04:49:17.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:16 smithi156 ceph-mon[133675]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2023-09-13T04:49:17.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:16 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-13T04:49:17.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:16 smithi156 ceph-mon[133675]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:49:18.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:18 smithi002 ceph-mon[97651]: pgmap v274: 129 pgs: 14 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 74 active+clean; 1.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 159 KiB/s wr, 18 op/s 2023-09-13T04:49:18.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:18 smithi149 ceph-mon[133315]: pgmap v274: 129 pgs: 14 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 74 active+clean; 1.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 159 KiB/s wr, 18 op/s 2023-09-13T04:49:18.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:18 smithi156 ceph-mon[133675]: pgmap v274: 129 pgs: 14 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 74 active+clean; 1.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 159 KiB/s wr, 18 op/s 2023-09-13T04:49:20.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:20 smithi149 ceph-mon[133315]: pgmap v275: 129 pgs: 11 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 83 active+clean; 1.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 6.9 KiB/s wr, 11 op/s 2023-09-13T04:49:20.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:20 smithi156 ceph-mon[133675]: pgmap v275: 129 pgs: 11 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 83 active+clean; 1.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 6.9 KiB/s wr, 11 op/s 2023-09-13T04:49:20.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:20 smithi002 ceph-mon[97651]: pgmap v275: 129 pgs: 11 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 83 active+clean; 1.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 6.9 KiB/s wr, 11 op/s 2023-09-13T04:49:22.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:22 smithi149 ceph-mon[133315]: pgmap v276: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 1023 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 3.8 KiB/s wr, 6 op/s 2023-09-13T04:49:22.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:22 smithi156 ceph-mon[133675]: pgmap v276: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 1023 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 3.8 KiB/s wr, 6 op/s 2023-09-13T04:49:22.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:22 smithi002 ceph-mon[97651]: pgmap v276: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 1023 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 3.8 KiB/s wr, 6 op/s 2023-09-13T04:49:24.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:24 smithi149 ceph-mon[133315]: pgmap v277: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2023-09-13T04:49:24.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:24 smithi156 ceph-mon[133675]: pgmap v277: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2023-09-13T04:49:24.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:24 smithi002 ceph-mon[97651]: pgmap v277: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2023-09-13T04:49:26.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:25 smithi149 ceph-mon[133315]: pgmap v278: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 511 B/s wr, 6 op/s 2023-09-13T04:49:26.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:25 smithi149 ceph-mon[133315]: osdmap e101: 12 total, 12 up, 12 in 2023-09-13T04:49:26.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:25 smithi156 ceph-mon[133675]: pgmap v278: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 511 B/s wr, 6 op/s 2023-09-13T04:49:26.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:25 smithi156 ceph-mon[133675]: osdmap e101: 12 total, 12 up, 12 in 2023-09-13T04:49:26.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:25 smithi002 ceph-mon[97651]: pgmap v278: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 511 B/s wr, 6 op/s 2023-09-13T04:49:26.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:25 smithi002 ceph-mon[97651]: osdmap e101: 12 total, 12 up, 12 in 2023-09-13T04:49:28.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:27 smithi149 ceph-mon[133315]: pgmap v280: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 511 B/s wr, 6 op/s 2023-09-13T04:49:28.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:27 smithi156 ceph-mon[133675]: pgmap v280: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 511 B/s wr, 6 op/s 2023-09-13T04:49:28.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:27 smithi002 ceph-mon[97651]: pgmap v280: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 511 B/s wr, 6 op/s 2023-09-13T04:49:30.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:29 smithi149 ceph-mon[133315]: pgmap v281: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 409 B/s wr, 6 op/s 2023-09-13T04:49:30.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:29 smithi156 ceph-mon[133675]: pgmap v281: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 409 B/s wr, 6 op/s 2023-09-13T04:49:30.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:29 smithi002 ceph-mon[97651]: pgmap v281: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 409 B/s wr, 6 op/s 2023-09-13T04:49:31.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:30 smithi149 ceph-mon[133315]: osdmap e102: 12 total, 12 up, 12 in 2023-09-13T04:49:31.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:30 smithi156 ceph-mon[133675]: osdmap e102: 12 total, 12 up, 12 in 2023-09-13T04:49:31.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:30 smithi002 ceph-mon[97651]: osdmap e102: 12 total, 12 up, 12 in 2023-09-13T04:49:32.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:31 smithi156 ceph-mon[133675]: pgmap v283: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2023-09-13T04:49:32.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:31 smithi002 ceph-mon[97651]: pgmap v283: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2023-09-13T04:49:32.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:31 smithi149 ceph-mon[133315]: pgmap v283: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2023-09-13T04:49:34.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:33 smithi156 ceph-mon[133675]: pgmap v284: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:49:34.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:33 smithi002 ceph-mon[97651]: pgmap v284: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:49:34.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:33 smithi149 ceph-mon[133315]: pgmap v284: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-09-13T04:49:36.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:35 smithi156 ceph-mon[133675]: pgmap v285: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2023-09-13T04:49:36.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:35 smithi002 ceph-mon[97651]: pgmap v285: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2023-09-13T04:49:36.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:35 smithi149 ceph-mon[133315]: pgmap v285: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2023-09-13T04:49:38.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:37 smithi156 ceph-mon[133675]: pgmap v286: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-09-13T04:49:38.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:37 smithi002 ceph-mon[97651]: pgmap v286: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-09-13T04:49:38.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:37 smithi149 ceph-mon[133315]: pgmap v286: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-09-13T04:49:40.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:39 smithi156 ceph-mon[133675]: pgmap v287: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-09-13T04:49:40.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:39 smithi002 ceph-mon[97651]: pgmap v287: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-09-13T04:49:40.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:39 smithi149 ceph-mon[133315]: pgmap v287: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-09-13T04:49:42.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:41 smithi156 ceph-mon[133675]: pgmap v288: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-09-13T04:49:42.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:41 smithi002 ceph-mon[97651]: pgmap v288: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-09-13T04:49:42.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:41 smithi149 ceph-mon[133315]: pgmap v288: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-09-13T04:49:44.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:43 smithi156 ceph-mon[133675]: pgmap v289: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:49:44.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:43 smithi002 ceph-mon[97651]: pgmap v289: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:49:44.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:43 smithi149 ceph-mon[133315]: pgmap v289: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:49:46.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:45 smithi156 ceph-mon[133675]: pgmap v290: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:49:46.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:45 smithi002 ceph-mon[97651]: pgmap v290: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:49:46.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:45 smithi149 ceph-mon[133315]: pgmap v290: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:49:48.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:47 smithi156 ceph-mon[133675]: pgmap v291: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:49:48.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:47 smithi002 ceph-mon[97651]: pgmap v291: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:49:48.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:47 smithi149 ceph-mon[133315]: pgmap v291: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:49:50.143 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:49 smithi156 ceph-mon[133675]: pgmap v292: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-13T04:49:50.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:49 smithi002 ceph-mon[97651]: pgmap v292: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-13T04:49:50.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:49 smithi149 ceph-mon[133315]: pgmap v292: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-13T04:49:52.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:51 smithi156 ceph-mon[133675]: pgmap v293: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:49:52.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:51 smithi002 ceph-mon[97651]: pgmap v293: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:49:52.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:51 smithi149 ceph-mon[133315]: pgmap v293: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:49:54.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:53 smithi156 ceph-mon[133675]: pgmap v294: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:49:54.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:53 smithi002 ceph-mon[97651]: pgmap v294: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:49:54.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:53 smithi149 ceph-mon[133315]: pgmap v294: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:49:56.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:55 smithi156 ceph-mon[133675]: pgmap v295: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:49:56.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:55 smithi002 ceph-mon[97651]: pgmap v295: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:49:56.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:55 smithi149 ceph-mon[133315]: pgmap v295: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:49:58.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:57 smithi156 ceph-mon[133675]: pgmap v296: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:49:58.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:57 smithi002 ceph-mon[97651]: pgmap v296: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:49:58.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:57 smithi149 ceph-mon[133315]: pgmap v296: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:50:00.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:49:59 smithi002 ceph-mon[97651]: pgmap v297: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-13T04:50:00.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mon-a[97627]: 2023-09-13T04:49:59.999+0000 7fbd28d3d700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-09-13T04:50:00.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mon-a[97627]: 2023-09-13T04:49:59.999+0000 7fbd28d3d700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-09-13T04:50:00.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mon-a[97627]: 2023-09-13T04:49:59.999+0000 7fbd28d3d700 -1 log_channel(cluster) log [ERR] : mds.a(mds.1): Metadata damage detected 2023-09-13T04:50:00.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:49:59 smithi149 ceph-mon[133315]: pgmap v297: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-13T04:50:00.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:49:59 smithi156 ceph-mon[133675]: pgmap v297: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-13T04:50:01.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:00 smithi002 ceph-mon[97651]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-09-13T04:50:01.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:00 smithi002 ceph-mon[97651]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-09-13T04:50:01.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:00 smithi002 ceph-mon[97651]: mds.a(mds.1): Metadata damage detected 2023-09-13T04:50:01.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:00 smithi149 ceph-mon[133315]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-09-13T04:50:01.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:00 smithi149 ceph-mon[133315]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-09-13T04:50:01.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:00 smithi149 ceph-mon[133315]: mds.a(mds.1): Metadata damage detected 2023-09-13T04:50:01.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:00 smithi156 ceph-mon[133675]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-09-13T04:50:01.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:00 smithi156 ceph-mon[133675]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-09-13T04:50:01.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:00 smithi156 ceph-mon[133675]: mds.a(mds.1): Metadata damage detected 2023-09-13T04:50:02.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:01 smithi002 ceph-mon[97651]: pgmap v298: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:50:02.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:01 smithi149 ceph-mon[133315]: pgmap v298: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:50:02.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:01 smithi156 ceph-mon[133675]: pgmap v298: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:50:04.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:03 smithi002 ceph-mon[97651]: pgmap v299: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 KiB/s wr, 17 op/s 2023-09-13T04:50:04.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:03 smithi149 ceph-mon[133315]: pgmap v299: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 KiB/s wr, 17 op/s 2023-09-13T04:50:04.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:03 smithi156 ceph-mon[133675]: pgmap v299: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 KiB/s wr, 17 op/s 2023-09-13T04:50:06.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:05 smithi002 ceph-mon[97651]: pgmap v300: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-09-13T04:50:06.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:05 smithi149 ceph-mon[133315]: pgmap v300: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-09-13T04:50:06.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:05 smithi156 ceph-mon[133675]: pgmap v300: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-09-13T04:50:08.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:07 smithi002 ceph-mon[97651]: pgmap v301: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-09-13T04:50:08.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:07 smithi149 ceph-mon[133315]: pgmap v301: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-09-13T04:50:08.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:07 smithi156 ceph-mon[133675]: pgmap v301: 129 pgs: 129 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-09-13T04:50:09.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:09 smithi002 ceph-mon[97651]: osdmap e103: 12 total, 12 up, 12 in 2023-09-13T04:50:09.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:09 smithi149 ceph-mon[133315]: osdmap e103: 12 total, 12 up, 12 in 2023-09-13T04:50:09.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:09 smithi156 ceph-mon[133675]: osdmap e103: 12 total, 12 up, 12 in 2023-09-13T04:50:10.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:10 smithi149 ceph-mon[133315]: pgmap v303: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-09-13T04:50:10.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:10 smithi156 ceph-mon[133675]: pgmap v303: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-09-13T04:50:10.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:10 smithi002 ceph-mon[97651]: pgmap v303: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-09-13T04:50:12.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:12 smithi149 ceph-mon[133315]: pgmap v304: 129 pgs: 20 active+clean+snaptrim, 59 active+clean+snaptrim_wait, 50 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-09-13T04:50:12.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:12 smithi156 ceph-mon[133675]: pgmap v304: 129 pgs: 20 active+clean+snaptrim, 59 active+clean+snaptrim_wait, 50 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-09-13T04:50:12.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:12 smithi002 ceph-mon[97651]: pgmap v304: 129 pgs: 20 active+clean+snaptrim, 59 active+clean+snaptrim_wait, 50 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-09-13T04:50:14.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:14 smithi149 ceph-mon[133315]: pgmap v305: 129 pgs: 22 active+clean+snaptrim, 66 active+clean+snaptrim_wait, 41 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 28 KiB/s wr, 9 op/s 2023-09-13T04:50:14.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:14 smithi156 ceph-mon[133675]: pgmap v305: 129 pgs: 22 active+clean+snaptrim, 66 active+clean+snaptrim_wait, 41 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 28 KiB/s wr, 9 op/s 2023-09-13T04:50:14.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:14 smithi002 ceph-mon[97651]: pgmap v305: 129 pgs: 22 active+clean+snaptrim, 66 active+clean+snaptrim_wait, 41 active+clean; 1023 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 28 KiB/s wr, 9 op/s 2023-09-13T04:50:16.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:15 smithi156 ceph-mon[133675]: pgmap v306: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1020 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-09-13T04:50:16.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:15 smithi002 ceph-mon[97651]: pgmap v306: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1020 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-09-13T04:50:16.532 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:15 smithi149 ceph-mon[133315]: pgmap v306: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1020 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-09-13T04:50:16.932 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:16 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:50:17.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:16 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:50:17.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:16 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:50:18.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:17 smithi002 ceph-mon[97651]: pgmap v307: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1020 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-09-13T04:50:18.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:17 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:50:18.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:17 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:50:18.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:17 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:50:18.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:17 smithi149 ceph-mon[133315]: pgmap v307: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1020 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-09-13T04:50:18.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:17 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:50:18.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:17 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:50:18.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:17 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:50:18.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:17 smithi156 ceph-mon[133675]: pgmap v307: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1020 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-09-13T04:50:18.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:17 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:50:18.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:17 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:50:18.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:17 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:50:20.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:19 smithi002 ceph-mon[97651]: pgmap v308: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1020 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 193 B/s wr, 6 op/s 2023-09-13T04:50:20.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:19 smithi149 ceph-mon[133315]: pgmap v308: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1020 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 193 B/s wr, 6 op/s 2023-09-13T04:50:20.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:19 smithi156 ceph-mon[133675]: pgmap v308: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1020 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 193 B/s wr, 6 op/s 2023-09-13T04:50:22.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:21 smithi002 ceph-mon[97651]: pgmap v309: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1018 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2023-09-13T04:50:22.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:21 smithi149 ceph-mon[133315]: pgmap v309: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1018 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2023-09-13T04:50:22.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:21 smithi156 ceph-mon[133675]: pgmap v309: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1018 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2023-09-13T04:50:24.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:24 smithi149 ceph-mon[133315]: pgmap v310: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1018 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2023-09-13T04:50:24.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:24 smithi156 ceph-mon[133675]: pgmap v310: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1018 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2023-09-13T04:50:24.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:24 smithi002 ceph-mon[97651]: pgmap v310: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1018 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2023-09-13T04:50:26.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:26 smithi149 ceph-mon[133315]: pgmap v311: 129 pgs: 11 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 89 active+clean; 1016 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 7 op/s 2023-09-13T04:50:26.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:26 smithi156 ceph-mon[133675]: pgmap v311: 129 pgs: 11 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 89 active+clean; 1016 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 7 op/s 2023-09-13T04:50:26.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:26 smithi002 ceph-mon[97651]: pgmap v311: 129 pgs: 11 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 89 active+clean; 1016 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 7 op/s 2023-09-13T04:50:28.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:28 smithi149 ceph-mon[133315]: pgmap v312: 129 pgs: 11 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 89 active+clean; 1016 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:50:28.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:28 smithi156 ceph-mon[133675]: pgmap v312: 129 pgs: 11 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 89 active+clean; 1016 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:50:28.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:28 smithi002 ceph-mon[97651]: pgmap v312: 129 pgs: 11 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 89 active+clean; 1016 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:50:30.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:30 smithi149 ceph-mon[133315]: pgmap v313: 129 pgs: 11 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 89 active+clean; 1016 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-13T04:50:30.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:30 smithi156 ceph-mon[133675]: pgmap v313: 129 pgs: 11 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 89 active+clean; 1016 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-13T04:50:30.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:30 smithi002 ceph-mon[97651]: pgmap v313: 129 pgs: 11 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 89 active+clean; 1016 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-13T04:50:30.828 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-09-13T04:50:31.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.060+0000 7f81fa1ff700 1 -- 172.21.15.2:0/79559820 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81f4106d60 msgr2=0x7f81f4107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:31.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.060+0000 7f81fa1ff700 1 --2- 172.21.15.2:0/79559820 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81f4106d60 0x7f81f4107140 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009990 tx=0x7f81e4009ca0 comp rx=0 tx=0).stop 2023-09-13T04:50:31.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.060+0000 7f81fa1ff700 1 -- 172.21.15.2:0/79559820 shutdown_connections 2023-09-13T04:50:31.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.060+0000 7f81fa1ff700 1 --2- 172.21.15.2:0/79559820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81f4068a90 0x7f81f410db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:31.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.060+0000 7f81fa1ff700 1 --2- 172.21.15.2:0/79559820 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81f4106d60 0x7f81f4107140 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:31.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.060+0000 7f81fa1ff700 1 --2- 172.21.15.2:0/79559820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81f40680d0 0x7f81f4068550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:31.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.060+0000 7f81fa1ff700 1 -- 172.21.15.2:0/79559820 >> 172.21.15.2:0/79559820 conn(0x7f81f4076970 msgr2=0x7f81f4076d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:50:31.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.061+0000 7f81fa1ff700 1 -- 172.21.15.2:0/79559820 shutdown_connections 2023-09-13T04:50:31.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.061+0000 7f81fa1ff700 1 -- 172.21.15.2:0/79559820 wait complete. 2023-09-13T04:50:31.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.062+0000 7f81fa1ff700 1 Processor -- start 2023-09-13T04:50:31.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.062+0000 7f81fa1ff700 1 -- start start 2023-09-13T04:50:31.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.062+0000 7f81fa1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81f40680d0 0x7f81f4196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:31.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.062+0000 7f81fa1ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81f4068a90 0x7f81f41972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:31.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.062+0000 7f81fa1ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81f4106d60 0x7f81f419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:31.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.062+0000 7f81fa1ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f419bf30 con 0x7f81f4106d60 2023-09-13T04:50:31.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.062+0000 7f81fa1ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f419c0a0 con 0x7f81f4068a90 2023-09-13T04:50:31.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.062+0000 7f81fa1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f419c260 con 0x7f81f40680d0 2023-09-13T04:50:31.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.062+0000 7f81f3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81f4106d60 0x7f81f419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:31.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.063+0000 7f81ebfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81f4068a90 0x7f81f41972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:31.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.063+0000 7f81f3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81f4106d60 0x7f81f419b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41098/0 (socket says 172.21.15.2:41098) 2023-09-13T04:50:31.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.063+0000 7f81f3fff700 1 -- 172.21.15.2:0/3440289562 learned_addr learned my addr 172.21.15.2:0/3440289562 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:50:31.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.063+0000 7f81f37fe700 1 --2- 172.21.15.2:0/3440289562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81f40680d0 0x7f81f4196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:31.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.063+0000 7f81ebfff700 1 -- 172.21.15.2:0/3440289562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81f40680d0 msgr2=0x7f81f4196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:31.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.063+0000 7f81ebfff700 1 --2- 172.21.15.2:0/3440289562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81f40680d0 0x7f81f4196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:31.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.063+0000 7f81ebfff700 1 -- 172.21.15.2:0/3440289562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81f4106d60 msgr2=0x7f81f419b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:31.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.063+0000 7f81ebfff700 1 --2- 172.21.15.2:0/3440289562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81f4106d60 0x7f81f419b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:31.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.063+0000 7f81ebfff700 1 -- 172.21.15.2:0/3440289562 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e40095f0 con 0x7f81f4068a90 2023-09-13T04:50:31.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.063+0000 7f81f3fff700 1 --2- 172.21.15.2:0/3440289562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81f4106d60 0x7f81f419b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:50:31.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.063+0000 7f81ebfff700 1 --2- 172.21.15.2:0/3440289562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81f4068a90 0x7f81f41972d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00e9f0 tx=0x7f81dc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:50:31.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.063+0000 7f81f17fa700 1 -- 172.21.15.2:0/3440289562 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc00c9c0 con 0x7f81f4068a90 2023-09-13T04:50:31.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.063+0000 7f81f17fa700 1 -- 172.21.15.2:0/3440289562 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81dc010430 con 0x7f81f4068a90 2023-09-13T04:50:31.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.063+0000 7f81fa1ff700 1 -- 172.21.15.2:0/3440289562 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f419c540 con 0x7f81f4068a90 2023-09-13T04:50:31.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.063+0000 7f81f17fa700 1 -- 172.21.15.2:0/3440289562 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc018680 con 0x7f81f4068a90 2023-09-13T04:50:31.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.064+0000 7f81fa1ff700 1 -- 172.21.15.2:0/3440289562 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f41a3d80 con 0x7f81f4068a90 2023-09-13T04:50:31.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.065+0000 7f81fa1ff700 1 -- 172.21.15.2:0/3440289562 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f4077490 con 0x7f81f4068a90 2023-09-13T04:50:31.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.066+0000 7f81f17fa700 1 -- 172.21.15.2:0/3440289562 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f81dc010860 con 0x7f81f4068a90 2023-09-13T04:50:31.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.068+0000 7f81f17fa700 1 --2- 172.21.15.2:0/3440289562 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f81e00a07b0 0x7f81e00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:31.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.068+0000 7f81f17fa700 1 -- 172.21.15.2:0/3440289562 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9483+0+0 (secure 0 0 0) 0x7f81dc014070 con 0x7f81f4068a90 2023-09-13T04:50:31.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.068+0000 7f81f37fe700 1 --2- 172.21.15.2:0/3440289562 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f81e00a07b0 0x7f81e00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:31.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.069+0000 7f81f37fe700 1 --2- 172.21.15.2:0/3440289562 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f81e00a07b0 0x7f81e00a2c70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f81e400b2e0 tx=0x7f81e4005fd0 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:50:31.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.072+0000 7f81f17fa700 1 -- 172.21.15.2:0/3440289562 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f81dc051e10 con 0x7f81f4068a90 2023-09-13T04:50:31.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.313+0000 7f81fa1ff700 1 -- 172.21.15.2:0/3440289562 --> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f81f4110be0 con 0x7f81e00a07b0 2023-09-13T04:50:31.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.322+0000 7f81f17fa700 1 -- 172.21.15.2:0/3440289562 <== mgr.15492 v2:172.21.15.2:6800/3640961355 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+335 (secure 0 0 0) 0x7f81f4110be0 con 0x7f81e00a07b0 2023-09-13T04:50:31.322 INFO:teuthology.orchestra.run.smithi002.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-09-13T00:00:00", "created": "2023-09-13T04:40:28", "first": "2023-09-13T04:41:00", "last": "2023-09-13T04:50:00", "last_pruned": "2023-09-13T04:50:00", "created_count": 10, "pruned_count": 4, "active": true} 2023-09-13T04:50:31.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.324+0000 7f81fa1ff700 1 -- 172.21.15.2:0/3440289562 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f81e00a07b0 msgr2=0x7f81e00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:31.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.324+0000 7f81fa1ff700 1 --2- 172.21.15.2:0/3440289562 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f81e00a07b0 0x7f81e00a2c70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f81e400b2e0 tx=0x7f81e4005fd0 comp rx=0 tx=0).stop 2023-09-13T04:50:31.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.324+0000 7f81fa1ff700 1 -- 172.21.15.2:0/3440289562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81f4068a90 msgr2=0x7f81f41972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:31.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.324+0000 7f81fa1ff700 1 --2- 172.21.15.2:0/3440289562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81f4068a90 0x7f81f41972d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00e9f0 tx=0x7f81dc00ed00 comp rx=0 tx=0).stop 2023-09-13T04:50:31.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.324+0000 7f81fa1ff700 1 -- 172.21.15.2:0/3440289562 shutdown_connections 2023-09-13T04:50:31.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.324+0000 7f81fa1ff700 1 --2- 172.21.15.2:0/3440289562 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f81e00a07b0 0x7f81e00a2c70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:31.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.324+0000 7f81fa1ff700 1 --2- 172.21.15.2:0/3440289562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81f4106d60 0x7f81f419b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:31.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.324+0000 7f81fa1ff700 1 --2- 172.21.15.2:0/3440289562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81f40680d0 0x7f81f4196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:31.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.324+0000 7f81fa1ff700 1 --2- 172.21.15.2:0/3440289562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81f4068a90 0x7f81f41972d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:31.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.324+0000 7f81fa1ff700 1 -- 172.21.15.2:0/3440289562 >> 172.21.15.2:0/3440289562 conn(0x7f81f4076970 msgr2=0x7f81f40fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:50:31.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.324+0000 7f81fa1ff700 1 -- 172.21.15.2:0/3440289562 shutdown_connections 2023-09-13T04:50:31.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.324+0000 7f81fa1ff700 1 -- 172.21.15.2:0/3440289562 wait complete. 2023-09-13T04:50:31.342 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-09-13T04:50:31.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.546+0000 7f516bbff700 1 -- 172.21.15.2:0/601973145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5164105ff0 msgr2=0x7f516410db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:31.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.546+0000 7f516bbff700 1 --2- 172.21.15.2:0/601973145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5164105ff0 0x7f516410db70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f515c00b0a0 tx=0x7f515c00b3b0 comp rx=0 tx=0).stop 2023-09-13T04:50:31.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.547+0000 7f516bbff700 1 -- 172.21.15.2:0/601973145 shutdown_connections 2023-09-13T04:50:31.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.547+0000 7f516bbff700 1 --2- 172.21.15.2:0/601973145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5164105ff0 0x7f516410db70 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:31.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.547+0000 7f516bbff700 1 --2- 172.21.15.2:0/601973145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5164100ed0 0x7f51641012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:31.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.547+0000 7f516bbff700 1 --2- 172.21.15.2:0/601973145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5164101880 0x7f51641058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:31.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.547+0000 7f516bbff700 1 -- 172.21.15.2:0/601973145 >> 172.21.15.2:0/601973145 conn(0x7f51640fc750 msgr2=0x7f51640feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:50:31.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.547+0000 7f516bbff700 1 -- 172.21.15.2:0/601973145 shutdown_connections 2023-09-13T04:50:31.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.547+0000 7f516bbff700 1 -- 172.21.15.2:0/601973145 wait complete. 2023-09-13T04:50:31.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.548+0000 7f516bbff700 1 Processor -- start 2023-09-13T04:50:31.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.548+0000 7f516bbff700 1 -- start start 2023-09-13T04:50:31.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.549+0000 7f516bbff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5164100ed0 0x7f5164199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:31.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.549+0000 7f516bbff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5164105ff0 0x7f5164199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:31.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.549+0000 7f516bbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f516419dcb0 0x7f516419e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:31.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.549+0000 7f516bbff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516419e730 con 0x7f5164100ed0 2023-09-13T04:50:31.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.549+0000 7f516bbff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516419e8a0 con 0x7f5164105ff0 2023-09-13T04:50:31.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.549+0000 7f516bbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516419ea60 con 0x7f516419dcb0 2023-09-13T04:50:31.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.549+0000 7f516999b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5164100ed0 0x7f5164199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:31.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.549+0000 7f516999b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5164100ed0 0x7f5164199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41114/0 (socket says 172.21.15.2:41114) 2023-09-13T04:50:31.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.549+0000 7f516999b700 1 -- 172.21.15.2:0/3440285879 learned_addr learned my addr 172.21.15.2:0/3440285879 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:50:31.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.549+0000 7f516919a700 1 --2- 172.21.15.2:0/3440285879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5164105ff0 0x7f5164199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:31.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.549+0000 7f516a19c700 1 --2- 172.21.15.2:0/3440285879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f516419dcb0 0x7f516419e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:31.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.550+0000 7f516999b700 1 -- 172.21.15.2:0/3440285879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f516419dcb0 msgr2=0x7f516419e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:31.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.550+0000 7f516999b700 1 --2- 172.21.15.2:0/3440285879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f516419dcb0 0x7f516419e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:31.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.550+0000 7f516999b700 1 -- 172.21.15.2:0/3440285879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5164105ff0 msgr2=0x7f5164199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:31.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.550+0000 7f516999b700 1 --2- 172.21.15.2:0/3440285879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5164105ff0 0x7f5164199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:31.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.550+0000 7f516999b700 1 -- 172.21.15.2:0/3440285879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5154009520 con 0x7f5164100ed0 2023-09-13T04:50:31.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.550+0000 7f516999b700 1 --2- 172.21.15.2:0/3440285879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5164100ed0 0x7f5164199100 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f516000e9a0 tx=0x7f516000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:50:31.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.550+0000 7f515affd700 1 -- 172.21.15.2:0/3440285879 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160009790 con 0x7f5164100ed0 2023-09-13T04:50:31.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.550+0000 7f515affd700 1 -- 172.21.15.2:0/3440285879 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5160010430 con 0x7f5164100ed0 2023-09-13T04:50:31.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.550+0000 7f515affd700 1 -- 172.21.15.2:0/3440285879 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160018680 con 0x7f5164100ed0 2023-09-13T04:50:31.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.550+0000 7f516bbff700 1 -- 172.21.15.2:0/3440285879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f515c009d00 con 0x7f5164100ed0 2023-09-13T04:50:31.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.551+0000 7f516bbff700 1 -- 172.21.15.2:0/3440285879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51641a6260 con 0x7f5164100ed0 2023-09-13T04:50:31.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.552+0000 7f516bbff700 1 -- 172.21.15.2:0/3440285879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51641003a0 con 0x7f5164100ed0 2023-09-13T04:50:31.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.553+0000 7f515affd700 1 -- 172.21.15.2:0/3440285879 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f51600105a0 con 0x7f5164100ed0 2023-09-13T04:50:31.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.555+0000 7f515affd700 1 --2- 172.21.15.2:0/3440285879 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f51500a0830 0x7f51500a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:31.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.555+0000 7f515affd700 1 -- 172.21.15.2:0/3440285879 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9483+0+0 (secure 0 0 0) 0x7f5160014070 con 0x7f5164100ed0 2023-09-13T04:50:31.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.555+0000 7f516919a700 1 --2- 172.21.15.2:0/3440285879 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f51500a0830 0x7f51500a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:31.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.556+0000 7f516919a700 1 --2- 172.21.15.2:0/3440285879 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f51500a0830 0x7f51500a2cf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f516419a920 tx=0x7f5154009260 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:50:31.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.559+0000 7f515affd700 1 -- 172.21.15.2:0/3440285879 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5160051e10 con 0x7f5164100ed0 2023-09-13T04:50:31.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.801+0000 7f516bbff700 1 -- 172.21.15.2:0/3440285879 --> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f5164101990 con 0x7f51500a0830 2023-09-13T04:50:31.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.809+0000 7f515affd700 1 -- 172.21.15.2:0/3440285879 <== mgr.15492 v2:172.21.15.2:6800/3640961355 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f5164101990 con 0x7f51500a0830 2023-09-13T04:50:31.810 INFO:teuthology.orchestra.run.smithi002.stdout:/ 1M 6M3h 2023-09-13T04:50:31.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.812+0000 7f516bbff700 1 -- 172.21.15.2:0/3440285879 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f51500a0830 msgr2=0x7f51500a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:31.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.812+0000 7f516bbff700 1 --2- 172.21.15.2:0/3440285879 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f51500a0830 0x7f51500a2cf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f516419a920 tx=0x7f5154009260 comp rx=0 tx=0).stop 2023-09-13T04:50:31.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.812+0000 7f516bbff700 1 -- 172.21.15.2:0/3440285879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5164100ed0 msgr2=0x7f5164199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:31.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.812+0000 7f516bbff700 1 --2- 172.21.15.2:0/3440285879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5164100ed0 0x7f5164199100 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f516000e9a0 tx=0x7f516000ecb0 comp rx=0 tx=0).stop 2023-09-13T04:50:31.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.812+0000 7f516bbff700 1 -- 172.21.15.2:0/3440285879 shutdown_connections 2023-09-13T04:50:31.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.812+0000 7f516bbff700 1 --2- 172.21.15.2:0/3440285879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f516419dcb0 0x7f516419e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:31.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.812+0000 7f516bbff700 1 --2- 172.21.15.2:0/3440285879 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f51500a0830 0x7f51500a2cf0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:31.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.812+0000 7f516bbff700 1 --2- 172.21.15.2:0/3440285879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5164100ed0 0x7f5164199100 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:31.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.812+0000 7f516bbff700 1 --2- 172.21.15.2:0/3440285879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5164105ff0 0x7f5164199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:31.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.812+0000 7f516bbff700 1 -- 172.21.15.2:0/3440285879 >> 172.21.15.2:0/3440285879 conn(0x7f51640fc750 msgr2=0x7f51640fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:50:31.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.812+0000 7f516bbff700 1 -- 172.21.15.2:0/3440285879 shutdown_connections 2023-09-13T04:50:31.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:31.812+0000 7f516bbff700 1 -- 172.21.15.2:0/3440285879 wait complete. 2023-09-13T04:50:31.830 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-09-13T04:50:31.842 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-09-13T04:50:31.842 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-09-13T04:50:31.843 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-09-13T04:50:31.853 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T04:50:32.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.057+0000 7fa0a89ff700 1 -- 172.21.15.2:0/2962310966 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0a4101df0 msgr2=0x7fa0a41021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:32.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.057+0000 7fa0a89ff700 1 --2- 172.21.15.2:0/2962310966 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0a4101df0 0x7fa0a41021d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa094009990 tx=0x7fa094009ca0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.057+0000 7fa0a89ff700 1 -- 172.21.15.2:0/2962310966 shutdown_connections 2023-09-13T04:50:32.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.057+0000 7fa0a89ff700 1 --2- 172.21.15.2:0/2962310966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0a410b210 0x7fa0a410d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.057+0000 7fa0a89ff700 1 --2- 172.21.15.2:0/2962310966 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0a4101df0 0x7fa0a41021d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.057+0000 7fa0a89ff700 1 --2- 172.21.15.2:0/2962310966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0a4102710 0x7fa0a410abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.057+0000 7fa0a89ff700 1 -- 172.21.15.2:0/2962310966 >> 172.21.15.2:0/2962310966 conn(0x7fa0a40fb5f0 msgr2=0x7fa0a40fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:50:32.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.057+0000 7fa0a89ff700 1 -- 172.21.15.2:0/2962310966 shutdown_connections 2023-09-13T04:50:32.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.057+0000 7fa0a89ff700 1 -- 172.21.15.2:0/2962310966 wait complete. 2023-09-13T04:50:32.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.059+0000 7fa0a89ff700 1 Processor -- start 2023-09-13T04:50:32.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.059+0000 7fa0a89ff700 1 -- start start 2023-09-13T04:50:32.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.060+0000 7fa0a89ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0a4101df0 0x7fa0a40ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:32.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.060+0000 7fa0a89ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0a4102710 0x7fa0a40fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:32.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.060+0000 7fa0a89ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0a410b210 0x7fa0a41a5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:32.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.060+0000 7fa0a89ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a41007b0 con 0x7fa0a4101df0 2023-09-13T04:50:32.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.060+0000 7fa0a89ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a41a5560 con 0x7fa0a410b210 2023-09-13T04:50:32.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.060+0000 7fa0a89ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a41a56a0 con 0x7fa0a4102710 2023-09-13T04:50:32.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.060+0000 7fa0a259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0a4101df0 0x7fa0a40ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:32.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.060+0000 7fa0a1d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0a4102710 0x7fa0a40fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:32.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.060+0000 7fa0a259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0a4101df0 0x7fa0a40ffa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41140/0 (socket says 172.21.15.2:41140) 2023-09-13T04:50:32.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.060+0000 7fa0a259c700 1 -- 172.21.15.2:0/243294080 learned_addr learned my addr 172.21.15.2:0/243294080 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:50:32.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.060+0000 7fa0a2d9d700 1 --2- 172.21.15.2:0/243294080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0a410b210 0x7fa0a41a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:32.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.060+0000 7fa0a259c700 1 -- 172.21.15.2:0/243294080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0a4102710 msgr2=0x7fa0a40fff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:32.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.060+0000 7fa0a259c700 1 --2- 172.21.15.2:0/243294080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0a4102710 0x7fa0a40fff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.060+0000 7fa0a259c700 1 -- 172.21.15.2:0/243294080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0a410b210 msgr2=0x7fa0a41a5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:32.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.060+0000 7fa0a259c700 1 --2- 172.21.15.2:0/243294080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0a410b210 0x7fa0a41a5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.060+0000 7fa0a259c700 1 -- 172.21.15.2:0/243294080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa098009520 con 0x7fa0a4101df0 2023-09-13T04:50:32.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.061+0000 7fa0a259c700 1 --2- 172.21.15.2:0/243294080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0a4101df0 0x7fa0a40ffa40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa094005910 tx=0x7fa094004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:50:32.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.061+0000 7fa0937fe700 1 -- 172.21.15.2:0/243294080 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09401d070 con 0x7fa0a4101df0 2023-09-13T04:50:32.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.061+0000 7fa0a89ff700 1 -- 172.21.15.2:0/243294080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0940095f0 con 0x7fa0a4101df0 2023-09-13T04:50:32.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.061+0000 7fa0937fe700 1 -- 172.21.15.2:0/243294080 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa094022470 con 0x7fa0a4101df0 2023-09-13T04:50:32.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.061+0000 7fa0937fe700 1 -- 172.21.15.2:0/243294080 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094017650 con 0x7fa0a4101df0 2023-09-13T04:50:32.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.061+0000 7fa0a89ff700 1 -- 172.21.15.2:0/243294080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a41a5be0 con 0x7fa0a4101df0 2023-09-13T04:50:32.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.062+0000 7fa0a89ff700 1 -- 172.21.15.2:0/243294080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a4067ad0 con 0x7fa0a4101df0 2023-09-13T04:50:32.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.064+0000 7fa0937fe700 1 -- 172.21.15.2:0/243294080 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7fa0940225e0 con 0x7fa0a4101df0 2023-09-13T04:50:32.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.066+0000 7fa0937fe700 1 --2- 172.21.15.2:0/243294080 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fa08c0a07f0 0x7fa08c0a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:32.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.067+0000 7fa0937fe700 1 -- 172.21.15.2:0/243294080 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9483+0+0 (secure 0 0 0) 0x7fa0940cd680 con 0x7fa0a4101df0 2023-09-13T04:50:32.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.067+0000 7fa0a1d9b700 1 --2- 172.21.15.2:0/243294080 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fa08c0a07f0 0x7fa08c0a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:32.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.067+0000 7fa0a1d9b700 1 --2- 172.21.15.2:0/243294080 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fa08c0a07f0 0x7fa08c0a2cb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4101230 tx=0x7fa098009260 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:50:32.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.071+0000 7fa0937fe700 1 -- 172.21.15.2:0/243294080 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa09405ce10 con 0x7fa0a4101df0 2023-09-13T04:50:32.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:32 smithi002 ceph-mon[97651]: pgmap v314: 129 pgs: 6 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 103 active+clean; 1015 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:50:32.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.284+0000 7fa0a89ff700 1 -- 172.21.15.2:0/243294080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0a4110c70 con 0x7fa0a4101df0 2023-09-13T04:50:32.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.286+0000 7fa0937fe700 1 -- 172.21.15.2:0/243294080 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+25195 (secure 0 0 0) 0x7fa094027090 con 0x7fa0a4101df0 2023-09-13T04:50:32.287 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:50:32.287 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":103,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:50:07.953818+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T04:37:57.094802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T04:37:58.564487+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-13T04:38:02.096895+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1461251021":"2023-09-14T04:40:27.774561+0000","172.21.15.2:6800/1461251021":"2023-09-14T04:40:27.774561+0000","172.21.15.2:0/2528007246":"2023-09-14T04:40:27.774561+0000","172.21.15.2:0/2955382590":"2023-09-14T04:40:27.774561+0000","172.21.15.2:0/227933483":"2023-09-14T04:40:27.774561+0000","172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3568881483":"2023-09-14T04:40:27.774561+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:50:32.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.288+0000 7fa0a89ff700 1 -- 172.21.15.2:0/243294080 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fa08c0a07f0 msgr2=0x7fa08c0a2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:32.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.288+0000 7fa0a89ff700 1 --2- 172.21.15.2:0/243294080 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fa08c0a07f0 0x7fa08c0a2cb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4101230 tx=0x7fa098009260 comp rx=0 tx=0).stop 2023-09-13T04:50:32.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.288+0000 7fa0a89ff700 1 -- 172.21.15.2:0/243294080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0a4101df0 msgr2=0x7fa0a40ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:32.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.288+0000 7fa0a89ff700 1 --2- 172.21.15.2:0/243294080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0a4101df0 0x7fa0a40ffa40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa094005910 tx=0x7fa094004710 comp rx=0 tx=0).stop 2023-09-13T04:50:32.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.288+0000 7fa0a89ff700 1 -- 172.21.15.2:0/243294080 shutdown_connections 2023-09-13T04:50:32.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.288+0000 7fa0a89ff700 1 --2- 172.21.15.2:0/243294080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0a410b210 0x7fa0a41a5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.288+0000 7fa0a89ff700 1 --2- 172.21.15.2:0/243294080 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fa08c0a07f0 0x7fa08c0a2cb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.288+0000 7fa0a89ff700 1 --2- 172.21.15.2:0/243294080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0a4101df0 0x7fa0a40ffa40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.288+0000 7fa0a89ff700 1 --2- 172.21.15.2:0/243294080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0a4102710 0x7fa0a40fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.288+0000 7fa0a89ff700 1 -- 172.21.15.2:0/243294080 >> 172.21.15.2:0/243294080 conn(0x7fa0a40fb5f0 msgr2=0x7fa0a410d410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:50:32.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.289+0000 7fa0a89ff700 1 -- 172.21.15.2:0/243294080 shutdown_connections 2023-09-13T04:50:32.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.289+0000 7fa0a89ff700 1 -- 172.21.15.2:0/243294080 wait complete. 2023-09-13T04:50:32.307 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T04:33:42.025714+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-13T04:37:57.094802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-13T04:37:58.564487+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 103, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-13T04:38:02.096895+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 103, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-13T04:50:32.308 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-13T04:50:32.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:32 smithi149 ceph-mon[133315]: pgmap v314: 129 pgs: 6 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 103 active+clean; 1015 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:50:32.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:32 smithi156 ceph-mon[133675]: pgmap v314: 129 pgs: 6 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 103 active+clean; 1015 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:50:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.515+0000 7f21bbfff700 1 -- 172.21.15.2:0/1964478986 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21b4073a00 msgr2=0x7f21b410cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.515+0000 7f21bbfff700 1 --2- 172.21.15.2:0/1964478986 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21b4073a00 0x7f21b410cf40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f21a4009990 tx=0x7f21a4009ca0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.515+0000 7f21bbfff700 1 -- 172.21.15.2:0/1964478986 shutdown_connections 2023-09-13T04:50:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.515+0000 7f21bbfff700 1 --2- 172.21.15.2:0/1964478986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21b410d480 0x7f21b410f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.515+0000 7f21bbfff700 1 --2- 172.21.15.2:0/1964478986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21b40730e0 0x7f21b40734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.515+0000 7f21bbfff700 1 --2- 172.21.15.2:0/1964478986 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21b4073a00 0x7f21b410cf40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.515+0000 7f21bbfff700 1 -- 172.21.15.2:0/1964478986 >> 172.21.15.2:0/1964478986 conn(0x7f21b40fc750 msgr2=0x7f21b40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:50:32.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.515+0000 7f21bbfff700 1 -- 172.21.15.2:0/1964478986 shutdown_connections 2023-09-13T04:50:32.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.515+0000 7f21bbfff700 1 -- 172.21.15.2:0/1964478986 wait complete. 2023-09-13T04:50:32.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.516+0000 7f21bbfff700 1 Processor -- start 2023-09-13T04:50:32.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.517+0000 7f21bbfff700 1 -- start start 2023-09-13T04:50:32.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.517+0000 7f21bbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21b40730e0 0x7f21b4198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:32.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.517+0000 7f21bbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21b4073a00 0x7f21b4199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:32.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.517+0000 7f21bbfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21b410d480 0x7f21b419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:32.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.517+0000 7f21bbfff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b419dfb0 con 0x7f21b40730e0 2023-09-13T04:50:32.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.517+0000 7f21bbfff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b419e120 con 0x7f21b410d480 2023-09-13T04:50:32.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.517+0000 7f21bbfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b419e2e0 con 0x7f21b4073a00 2023-09-13T04:50:32.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.517+0000 7f21b9d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21b40730e0 0x7f21b4198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:32.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.517+0000 7f21b959a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21b4073a00 0x7f21b4199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:32.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.517+0000 7f21b959a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21b4073a00 0x7f21b4199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:42046/0 (socket says 172.21.15.2:42046) 2023-09-13T04:50:32.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.517+0000 7f21b959a700 1 -- 172.21.15.2:0/872515843 learned_addr learned my addr 172.21.15.2:0/872515843 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:50:32.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.517+0000 7f21ba59c700 1 --2- 172.21.15.2:0/872515843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21b410d480 0x7f21b419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:32.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.518+0000 7f21b959a700 1 -- 172.21.15.2:0/872515843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21b410d480 msgr2=0x7f21b419d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:32.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.518+0000 7f21b959a700 1 --2- 172.21.15.2:0/872515843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21b410d480 0x7f21b419d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.518+0000 7f21b959a700 1 -- 172.21.15.2:0/872515843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21b40730e0 msgr2=0x7f21b4198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:32.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.518+0000 7f21b959a700 1 --2- 172.21.15.2:0/872515843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21b40730e0 0x7f21b4198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.518+0000 7f21b959a700 1 -- 172.21.15.2:0/872515843 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b0009520 con 0x7f21b4073a00 2023-09-13T04:50:32.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.518+0000 7f21b959a700 1 --2- 172.21.15.2:0/872515843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21b4073a00 0x7f21b4199510 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f21a4005910 tx=0x7f21a4004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:50:32.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.518+0000 7f21aaffd700 1 -- 172.21.15.2:0/872515843 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a401d070 con 0x7f21b4073a00 2023-09-13T04:50:32.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.518+0000 7f21bbfff700 1 -- 172.21.15.2:0/872515843 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a40095f0 con 0x7f21b4073a00 2023-09-13T04:50:32.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.518+0000 7f21aaffd700 1 -- 172.21.15.2:0/872515843 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21a4022470 con 0x7f21b4073a00 2023-09-13T04:50:32.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.518+0000 7f21aaffd700 1 -- 172.21.15.2:0/872515843 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4017650 con 0x7f21b4073a00 2023-09-13T04:50:32.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.519+0000 7f21bbfff700 1 -- 172.21.15.2:0/872515843 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b41a5c20 con 0x7f21b4073a00 2023-09-13T04:50:32.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.520+0000 7f21bbfff700 1 -- 172.21.15.2:0/872515843 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b4067a60 con 0x7f21b4073a00 2023-09-13T04:50:32.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.522+0000 7f21aaffd700 1 -- 172.21.15.2:0/872515843 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f21a40225e0 con 0x7f21b4073a00 2023-09-13T04:50:32.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.523+0000 7f21aaffd700 1 --2- 172.21.15.2:0/872515843 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f21a00a07b0 0x7f21a00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:32.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.523+0000 7f21aaffd700 1 -- 172.21.15.2:0/872515843 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9483+0+0 (secure 0 0 0) 0x7f21a40cdaa0 con 0x7f21b4073a00 2023-09-13T04:50:32.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.524+0000 7f21b9d9b700 1 --2- 172.21.15.2:0/872515843 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f21a00a07b0 0x7f21a00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:32.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.524+0000 7f21b9d9b700 1 --2- 172.21.15.2:0/872515843 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f21a00a07b0 0x7f21a00a2c70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009df0 tx=0x7f21b0009260 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:50:32.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.526+0000 7f21aaffd700 1 -- 172.21.15.2:0/872515843 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f21a4096f00 con 0x7f21b4073a00 2023-09-13T04:50:32.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.739+0000 7f21bbfff700 1 -- 172.21.15.2:0/872515843 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f21b4112e90 con 0x7f21b4073a00 2023-09-13T04:50:32.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.739+0000 7f21aaffd700 1 -- 172.21.15.2:0/872515843 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7f21a409a520 con 0x7f21b4073a00 2023-09-13T04:50:32.740 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-09-13T04:50:32.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.742+0000 7f21bbfff700 1 -- 172.21.15.2:0/872515843 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f21a00a07b0 msgr2=0x7f21a00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:32.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.742+0000 7f21bbfff700 1 --2- 172.21.15.2:0/872515843 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f21a00a07b0 0x7f21a00a2c70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009df0 tx=0x7f21b0009260 comp rx=0 tx=0).stop 2023-09-13T04:50:32.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.742+0000 7f21bbfff700 1 -- 172.21.15.2:0/872515843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21b4073a00 msgr2=0x7f21b4199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:32.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.742+0000 7f21bbfff700 1 --2- 172.21.15.2:0/872515843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21b4073a00 0x7f21b4199510 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f21a4005910 tx=0x7f21a4004710 comp rx=0 tx=0).stop 2023-09-13T04:50:32.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.742+0000 7f21bbfff700 1 -- 172.21.15.2:0/872515843 shutdown_connections 2023-09-13T04:50:32.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.742+0000 7f21bbfff700 1 --2- 172.21.15.2:0/872515843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21b410d480 0x7f21b419d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.742+0000 7f21bbfff700 1 --2- 172.21.15.2:0/872515843 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f21a00a07b0 0x7f21a00a2c70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.742+0000 7f21bbfff700 1 --2- 172.21.15.2:0/872515843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21b40730e0 0x7f21b4198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.742+0000 7f21bbfff700 1 --2- 172.21.15.2:0/872515843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21b4073a00 0x7f21b4199510 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.742+0000 7f21bbfff700 1 -- 172.21.15.2:0/872515843 >> 172.21.15.2:0/872515843 conn(0x7f21b40fc750 msgr2=0x7f21b410f680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:50:32.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.742+0000 7f21bbfff700 1 -- 172.21.15.2:0/872515843 shutdown_connections 2023-09-13T04:50:32.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.742+0000 7f21bbfff700 1 -- 172.21.15.2:0/872515843 wait complete. 2023-09-13T04:50:32.761 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-13T04:50:32.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.969+0000 7fcfb59ff700 1 -- 172.21.15.2:0/2047448978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfb0100eb0 msgr2=0x7fcfb0101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:32.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.969+0000 7fcfb59ff700 1 --2- 172.21.15.2:0/2047448978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfb0100eb0 0x7fcfb0101290 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009990 tx=0x7fcfa0009ca0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.969+0000 7fcfb59ff700 1 -- 172.21.15.2:0/2047448978 shutdown_connections 2023-09-13T04:50:32.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.969+0000 7fcfb59ff700 1 --2- 172.21.15.2:0/2047448978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfb0105fd0 0x7fcfb010db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.969+0000 7fcfb59ff700 1 --2- 172.21.15.2:0/2047448978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfb0100eb0 0x7fcfb0101290 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.969+0000 7fcfb59ff700 1 --2- 172.21.15.2:0/2047448978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfb0101860 0x7fcfb01058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.969+0000 7fcfb59ff700 1 -- 172.21.15.2:0/2047448978 >> 172.21.15.2:0/2047448978 conn(0x7fcfb00fc750 msgr2=0x7fcfb00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:50:32.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.969+0000 7fcfb59ff700 1 -- 172.21.15.2:0/2047448978 shutdown_connections 2023-09-13T04:50:32.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.969+0000 7fcfb59ff700 1 -- 172.21.15.2:0/2047448978 wait complete. 2023-09-13T04:50:32.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.970+0000 7fcfb59ff700 1 Processor -- start 2023-09-13T04:50:32.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.970+0000 7fcfb59ff700 1 -- start start 2023-09-13T04:50:32.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.970+0000 7fcfb59ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfb0100eb0 0x7fcfb0072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:32.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.970+0000 7fcfb59ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfb0105fd0 0x7fcfb00734f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:32.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.970+0000 7fcfb59ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfb0073a30 0x7fcfb01a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:32.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.970+0000 7fcfb59ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb01a5630 con 0x7fcfb0105fd0 2023-09-13T04:50:32.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.970+0000 7fcfb59ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb01a57a0 con 0x7fcfb0073a30 2023-09-13T04:50:32.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.970+0000 7fcfb59ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb01a5960 con 0x7fcfb0100eb0 2023-09-13T04:50:32.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.971+0000 7fcfae7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfb0105fd0 0x7fcfb00734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:32.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.971+0000 7fcfaeffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfb0100eb0 0x7fcfb0072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:32.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.971+0000 7fcfae7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfb0105fd0 0x7fcfb00734f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41174/0 (socket says 172.21.15.2:41174) 2023-09-13T04:50:32.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.971+0000 7fcfae7fc700 1 -- 172.21.15.2:0/2455708286 learned_addr learned my addr 172.21.15.2:0/2455708286 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:50:32.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.971+0000 7fcfaf7fe700 1 --2- 172.21.15.2:0/2455708286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfb0073a30 0x7fcfb01a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:32.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.971+0000 7fcfae7fc700 1 -- 172.21.15.2:0/2455708286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfb0100eb0 msgr2=0x7fcfb0072fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:32.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.971+0000 7fcfae7fc700 1 --2- 172.21.15.2:0/2455708286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfb0100eb0 0x7fcfb0072fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.971+0000 7fcfae7fc700 1 -- 172.21.15.2:0/2455708286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfb0073a30 msgr2=0x7fcfb01a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:32.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.971+0000 7fcfae7fc700 1 --2- 172.21.15.2:0/2455708286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfb0073a30 0x7fcfb01a4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:32.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.971+0000 7fcfae7fc700 1 -- 172.21.15.2:0/2455708286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa00095f0 con 0x7fcfb0105fd0 2023-09-13T04:50:32.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.971+0000 7fcfae7fc700 1 --2- 172.21.15.2:0/2455708286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfb0105fd0 0x7fcfb00734f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fcfa400e970 tx=0x7fcfa400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:50:32.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.972+0000 7fcfb49fd700 1 -- 172.21.15.2:0/2455708286 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4009790 con 0x7fcfb0105fd0 2023-09-13T04:50:32.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.972+0000 7fcfb49fd700 1 -- 172.21.15.2:0/2455708286 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfa400c7e0 con 0x7fcfb0105fd0 2023-09-13T04:50:32.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.972+0000 7fcfb59ff700 1 -- 172.21.15.2:0/2455708286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb01a5c40 con 0x7fcfb0105fd0 2023-09-13T04:50:32.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.972+0000 7fcfb49fd700 1 -- 172.21.15.2:0/2455708286 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa40054e0 con 0x7fcfb0105fd0 2023-09-13T04:50:32.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.972+0000 7fcfb59ff700 1 -- 172.21.15.2:0/2455708286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb01a6150 con 0x7fcfb0105fd0 2023-09-13T04:50:32.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.973+0000 7fcfb59ff700 1 -- 172.21.15.2:0/2455708286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb0067a60 con 0x7fcfb0105fd0 2023-09-13T04:50:32.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.975+0000 7fcfb49fd700 1 -- 172.21.15.2:0/2455708286 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7fcfa400c950 con 0x7fcfb0105fd0 2023-09-13T04:50:32.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.976+0000 7fcfb49fd700 1 --2- 172.21.15.2:0/2455708286 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcf980a07b0 0x7fcf980a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:32.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.976+0000 7fcfb49fd700 1 -- 172.21.15.2:0/2455708286 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9483+0+0 (secure 0 0 0) 0x7fcfa4014070 con 0x7fcfb0105fd0 2023-09-13T04:50:32.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.976+0000 7fcfaeffd700 1 --2- 172.21.15.2:0/2455708286 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcf980a07b0 0x7fcf980a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:32.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.977+0000 7fcfaeffd700 1 --2- 172.21.15.2:0/2455708286 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcf980a07b0 0x7fcf980a2c70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000b2e0 tx=0x7fcfa0005960 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:50:32.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:32.980+0000 7fcfb49fd700 1 -- 172.21.15.2:0/2455708286 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcfa40959a0 con 0x7fcfb0105fd0 2023-09-13T04:50:33.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.195+0000 7fcfb59ff700 1 -- 172.21.15.2:0/2455708286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcfb0101ab0 con 0x7fcfb0105fd0 2023-09-13T04:50:33.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.195+0000 7fcfb49fd700 1 -- 172.21.15.2:0/2455708286 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcfb0101ab0 con 0x7fcfb0105fd0 2023-09-13T04:50:33.196 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-09-13T04:50:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.197+0000 7fcfb59ff700 1 -- 172.21.15.2:0/2455708286 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcf980a07b0 msgr2=0x7fcf980a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.197+0000 7fcfb59ff700 1 --2- 172.21.15.2:0/2455708286 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcf980a07b0 0x7fcf980a2c70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000b2e0 tx=0x7fcfa0005960 comp rx=0 tx=0).stop 2023-09-13T04:50:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.197+0000 7fcfb59ff700 1 -- 172.21.15.2:0/2455708286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfb0105fd0 msgr2=0x7fcfb00734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.197+0000 7fcfb59ff700 1 --2- 172.21.15.2:0/2455708286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfb0105fd0 0x7fcfb00734f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fcfa400e970 tx=0x7fcfa400ec80 comp rx=0 tx=0).stop 2023-09-13T04:50:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.197+0000 7fcfb59ff700 1 -- 172.21.15.2:0/2455708286 shutdown_connections 2023-09-13T04:50:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.197+0000 7fcfb59ff700 1 --2- 172.21.15.2:0/2455708286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfb0073a30 0x7fcfb01a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.197+0000 7fcfb59ff700 1 --2- 172.21.15.2:0/2455708286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfb0100eb0 0x7fcfb0072fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.197+0000 7fcfb59ff700 1 --2- 172.21.15.2:0/2455708286 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcf980a07b0 0x7fcf980a2c70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.197+0000 7fcfb59ff700 1 --2- 172.21.15.2:0/2455708286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfb0105fd0 0x7fcfb00734f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.197+0000 7fcfb59ff700 1 -- 172.21.15.2:0/2455708286 >> 172.21.15.2:0/2455708286 conn(0x7fcfb00fc750 msgr2=0x7fcfb00fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:50:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.198+0000 7fcfb59ff700 1 -- 172.21.15.2:0/2455708286 shutdown_connections 2023-09-13T04:50:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.198+0000 7fcfb59ff700 1 -- 172.21.15.2:0/2455708286 wait complete. 2023-09-13T04:50:33.216 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-13T04:50:33.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:33 smithi149 ceph-mon[133315]: from='client.26750 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:50:33.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:33 smithi149 ceph-mon[133315]: from='client.16581 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:50:33.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:33 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/243294080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:50:33.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:33 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/872515843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:50:33.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.424+0000 7f24c25ff700 1 -- 172.21.15.2:0/3256642080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24bc100eb0 msgr2=0x7f24bc101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:33.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.424+0000 7f24c25ff700 1 --2- 172.21.15.2:0/3256642080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24bc100eb0 0x7f24bc101290 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f24ac009990 tx=0x7f24ac009ca0 comp rx=0 tx=0).stop 2023-09-13T04:50:33.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.424+0000 7f24c25ff700 1 -- 172.21.15.2:0/3256642080 shutdown_connections 2023-09-13T04:50:33.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.424+0000 7f24c25ff700 1 --2- 172.21.15.2:0/3256642080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24bc105fd0 0x7f24bc10db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:33.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.424+0000 7f24c25ff700 1 --2- 172.21.15.2:0/3256642080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24bc100eb0 0x7f24bc101290 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:33.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.424+0000 7f24c25ff700 1 --2- 172.21.15.2:0/3256642080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24bc101860 0x7f24bc1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:33.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.424+0000 7f24c25ff700 1 -- 172.21.15.2:0/3256642080 >> 172.21.15.2:0/3256642080 conn(0x7f24bc0fc750 msgr2=0x7f24bc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:50:33.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.424+0000 7f24c25ff700 1 -- 172.21.15.2:0/3256642080 shutdown_connections 2023-09-13T04:50:33.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.424+0000 7f24c25ff700 1 -- 172.21.15.2:0/3256642080 wait complete. 2023-09-13T04:50:33.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.425+0000 7f24c25ff700 1 Processor -- start 2023-09-13T04:50:33.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.425+0000 7f24c25ff700 1 -- start start 2023-09-13T04:50:33.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.425+0000 7f24c25ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24bc100eb0 0x7f24bc072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:33.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.425+0000 7f24c25ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24bc105fd0 0x7f24bc0734f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:33.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.425+0000 7f24c25ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24bc073a30 0x7f24bc1a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:33.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.426+0000 7f24c25ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc1a5570 con 0x7f24bc105fd0 2023-09-13T04:50:33.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.426+0000 7f24c25ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc1a56e0 con 0x7f24bc073a30 2023-09-13T04:50:33.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.426+0000 7f24c25ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc1a58a0 con 0x7f24bc100eb0 2023-09-13T04:50:33.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.426+0000 7f24bb7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24bc105fd0 0x7f24bc0734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:33.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.426+0000 7f24bb7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24bc105fd0 0x7f24bc0734f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41184/0 (socket says 172.21.15.2:41184) 2023-09-13T04:50:33.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.426+0000 7f24bb7fe700 1 -- 172.21.15.2:0/1714171587 learned_addr learned my addr 172.21.15.2:0/1714171587 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:50:33.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.426+0000 7f24bbfff700 1 --2- 172.21.15.2:0/1714171587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24bc100eb0 0x7f24bc072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:33.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.426+0000 7f24c0b9c700 1 --2- 172.21.15.2:0/1714171587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24bc073a30 0x7f24bc1a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:33.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.426+0000 7f24bb7fe700 1 -- 172.21.15.2:0/1714171587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24bc100eb0 msgr2=0x7f24bc072fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:33.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.426+0000 7f24bb7fe700 1 --2- 172.21.15.2:0/1714171587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24bc100eb0 0x7f24bc072fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:33.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.426+0000 7f24bb7fe700 1 -- 172.21.15.2:0/1714171587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24bc073a30 msgr2=0x7f24bc1a4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:33.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.426+0000 7f24bb7fe700 1 --2- 172.21.15.2:0/1714171587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24bc073a30 0x7f24bc1a4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:33.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.426+0000 7f24bb7fe700 1 -- 172.21.15.2:0/1714171587 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ac0095f0 con 0x7f24bc105fd0 2023-09-13T04:50:33.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.427+0000 7f24bb7fe700 1 --2- 172.21.15.2:0/1714171587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24bc105fd0 0x7f24bc0734f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f24b000e970 tx=0x7f24b000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:50:33.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.427+0000 7f24b97fa700 1 -- 172.21.15.2:0/1714171587 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b0009790 con 0x7f24bc105fd0 2023-09-13T04:50:33.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.427+0000 7f24b97fa700 1 -- 172.21.15.2:0/1714171587 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24b000c7e0 con 0x7f24bc105fd0 2023-09-13T04:50:33.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.427+0000 7f24c25ff700 1 -- 172.21.15.2:0/1714171587 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24bc1a5b80 con 0x7f24bc105fd0 2023-09-13T04:50:33.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.427+0000 7f24b97fa700 1 -- 172.21.15.2:0/1714171587 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b00054e0 con 0x7f24bc105fd0 2023-09-13T04:50:33.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.427+0000 7f24c25ff700 1 -- 172.21.15.2:0/1714171587 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24bc1a6090 con 0x7f24bc105fd0 2023-09-13T04:50:33.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.428+0000 7f24c25ff700 1 -- 172.21.15.2:0/1714171587 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24bc067a60 con 0x7f24bc105fd0 2023-09-13T04:50:33.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.430+0000 7f24b97fa700 1 -- 172.21.15.2:0/1714171587 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f24b000c950 con 0x7f24bc105fd0 2023-09-13T04:50:33.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.432+0000 7f24b97fa700 1 --2- 172.21.15.2:0/1714171587 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f24a40a07b0 0x7f24a40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:33.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.432+0000 7f24b97fa700 1 -- 172.21.15.2:0/1714171587 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9483+0+0 (secure 0 0 0) 0x7f24b0014070 con 0x7f24bc105fd0 2023-09-13T04:50:33.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.433+0000 7f24bbfff700 1 --2- 172.21.15.2:0/1714171587 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f24a40a07b0 0x7f24a40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:33.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.433+0000 7f24bbfff700 1 --2- 172.21.15.2:0/1714171587 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f24a40a07b0 0x7f24a40a2c70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f24ac00b2e0 tx=0x7f24ac005960 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:50:33.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:33 smithi156 ceph-mon[133675]: from='client.26750 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:50:33.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:33 smithi156 ceph-mon[133675]: from='client.16581 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:50:33.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:33 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/243294080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:50:33.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:33 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/872515843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:50:33.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.436+0000 7f24b97fa700 1 -- 172.21.15.2:0/1714171587 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f24b00959a0 con 0x7f24bc105fd0 2023-09-13T04:50:33.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:33 smithi002 ceph-mon[97651]: from='client.26750 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:50:33.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:33 smithi002 ceph-mon[97651]: from='client.16581 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T04:50:33.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:33 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/243294080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:50:33.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:33 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/872515843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:50:33.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.649+0000 7f24c25ff700 1 -- 172.21.15.2:0/1714171587 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f24bc1019e0 con 0x7f24bc105fd0 2023-09-13T04:50:33.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.649+0000 7f24b97fa700 1 -- 172.21.15.2:0/1714171587 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) v1 ==== 100+0+11 (secure 0 0 0) 0x7f24b0082e60 con 0x7f24bc105fd0 2023-09-13T04:50:33.650 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-09-13T04:50:33.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.652+0000 7f24c25ff700 1 -- 172.21.15.2:0/1714171587 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f24a40a07b0 msgr2=0x7f24a40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:33.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.652+0000 7f24c25ff700 1 --2- 172.21.15.2:0/1714171587 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f24a40a07b0 0x7f24a40a2c70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f24ac00b2e0 tx=0x7f24ac005960 comp rx=0 tx=0).stop 2023-09-13T04:50:33.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.652+0000 7f24c25ff700 1 -- 172.21.15.2:0/1714171587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24bc105fd0 msgr2=0x7f24bc0734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:33.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.652+0000 7f24c25ff700 1 --2- 172.21.15.2:0/1714171587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24bc105fd0 0x7f24bc0734f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f24b000e970 tx=0x7f24b000ec80 comp rx=0 tx=0).stop 2023-09-13T04:50:33.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.652+0000 7f24c25ff700 1 -- 172.21.15.2:0/1714171587 shutdown_connections 2023-09-13T04:50:33.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.652+0000 7f24c25ff700 1 --2- 172.21.15.2:0/1714171587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24bc073a30 0x7f24bc1a4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:33.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.652+0000 7f24c25ff700 1 --2- 172.21.15.2:0/1714171587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24bc100eb0 0x7f24bc072fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:33.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.652+0000 7f24c25ff700 1 --2- 172.21.15.2:0/1714171587 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f24a40a07b0 0x7f24a40a2c70 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:33.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.652+0000 7f24c25ff700 1 --2- 172.21.15.2:0/1714171587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24bc105fd0 0x7f24bc0734f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:33.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.652+0000 7f24c25ff700 1 -- 172.21.15.2:0/1714171587 >> 172.21.15.2:0/1714171587 conn(0x7f24bc0fc750 msgr2=0x7f24bc104880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:50:33.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.652+0000 7f24c25ff700 1 -- 172.21.15.2:0/1714171587 shutdown_connections 2023-09-13T04:50:33.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.652+0000 7f24c25ff700 1 -- 172.21.15.2:0/1714171587 wait complete. 2023-09-13T04:50:33.671 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-13T04:50:33.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.879+0000 7f58a6bff700 1 -- 172.21.15.2:0/2990990860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58a0073a00 msgr2=0x7f58a010cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:33.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.879+0000 7f58a6bff700 1 --2- 172.21.15.2:0/2990990860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58a0073a00 0x7f58a010cf40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f588c0098f0 tx=0x7f588c009c00 comp rx=0 tx=0).stop 2023-09-13T04:50:33.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.879+0000 7f58a6bff700 1 -- 172.21.15.2:0/2990990860 shutdown_connections 2023-09-13T04:50:33.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.879+0000 7f58a6bff700 1 --2- 172.21.15.2:0/2990990860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58a010d480 0x7f58a010f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:33.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.879+0000 7f58a6bff700 1 --2- 172.21.15.2:0/2990990860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58a00730e0 0x7f58a00734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:33.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.879+0000 7f58a6bff700 1 --2- 172.21.15.2:0/2990990860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58a0073a00 0x7f58a010cf40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:33.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.879+0000 7f58a6bff700 1 -- 172.21.15.2:0/2990990860 >> 172.21.15.2:0/2990990860 conn(0x7f58a00fc750 msgr2=0x7f58a00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:50:33.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.879+0000 7f58a6bff700 1 -- 172.21.15.2:0/2990990860 shutdown_connections 2023-09-13T04:50:33.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.879+0000 7f58a6bff700 1 -- 172.21.15.2:0/2990990860 wait complete. 2023-09-13T04:50:33.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.880+0000 7f58a6bff700 1 Processor -- start 2023-09-13T04:50:33.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.880+0000 7f58a6bff700 1 -- start start 2023-09-13T04:50:33.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.881+0000 7f58a6bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58a00730e0 0x7f58a01990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:33.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.881+0000 7f58a6bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58a0073a00 0x7f58a0199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:33.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.881+0000 7f58a6bff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58a010d480 0x7f58a019da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:33.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.881+0000 7f58a6bff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a019e0d0 con 0x7f58a010d480 2023-09-13T04:50:33.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.881+0000 7f58a6bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a019e240 con 0x7f58a00730e0 2023-09-13T04:50:33.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.881+0000 7f58a6bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a019e400 con 0x7f58a0073a00 2023-09-13T04:50:33.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.881+0000 7f58a519c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58a010d480 0x7f58a019da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:33.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.881+0000 7f589ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58a0073a00 0x7f58a0199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:33.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.881+0000 7f58a499b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58a00730e0 0x7f58a01990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:33.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.881+0000 7f58a519c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58a010d480 0x7f58a019da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41194/0 (socket says 172.21.15.2:41194) 2023-09-13T04:50:33.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.881+0000 7f58a519c700 1 -- 172.21.15.2:0/3071499118 learned_addr learned my addr 172.21.15.2:0/3071499118 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:50:33.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.882+0000 7f58a519c700 1 -- 172.21.15.2:0/3071499118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58a0073a00 msgr2=0x7f58a0199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:33.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.882+0000 7f58a519c700 1 --2- 172.21.15.2:0/3071499118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58a0073a00 0x7f58a0199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:33.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.882+0000 7f58a519c700 1 -- 172.21.15.2:0/3071499118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58a00730e0 msgr2=0x7f58a01990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:33.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.882+0000 7f58a519c700 1 --2- 172.21.15.2:0/3071499118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58a00730e0 0x7f58a01990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:33.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.882+0000 7f58a519c700 1 -- 172.21.15.2:0/3071499118 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588c009520 con 0x7f58a010d480 2023-09-13T04:50:33.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.882+0000 7f58a499b700 1 --2- 172.21.15.2:0/3071499118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58a00730e0 0x7f58a01990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:50:33.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.882+0000 7f58a519c700 1 --2- 172.21.15.2:0/3071499118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58a010d480 0x7f58a019da90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5898007ee0 tx=0x7f589800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:50:33.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.882+0000 7f589dffb700 1 -- 172.21.15.2:0/3071499118 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589800ddd0 con 0x7f58a010d480 2023-09-13T04:50:33.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.882+0000 7f58a6bff700 1 -- 172.21.15.2:0/3071499118 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a019e6e0 con 0x7f58a010d480 2023-09-13T04:50:33.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.882+0000 7f589dffb700 1 -- 172.21.15.2:0/3071499118 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f589800f040 con 0x7f58a010d480 2023-09-13T04:50:33.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.882+0000 7f589dffb700 1 -- 172.21.15.2:0/3071499118 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5898004800 con 0x7f58a010d480 2023-09-13T04:50:33.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.882+0000 7f58a6bff700 1 -- 172.21.15.2:0/3071499118 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a01a5f20 con 0x7f58a010d480 2023-09-13T04:50:33.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.883+0000 7f58a6bff700 1 -- 172.21.15.2:0/3071499118 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a019e8d0 con 0x7f58a010d480 2023-09-13T04:50:33.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.885+0000 7f589dffb700 1 -- 172.21.15.2:0/3071499118 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f5898003a00 con 0x7f58a010d480 2023-09-13T04:50:33.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.886+0000 7f589dffb700 1 --2- 172.21.15.2:0/3071499118 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f58900a07b0 0x7f58900a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:33.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.887+0000 7f589dffb700 1 -- 172.21.15.2:0/3071499118 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9483+0+0 (secure 0 0 0) 0x7f58980cbbb0 con 0x7f58a010d480 2023-09-13T04:50:33.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.887+0000 7f58a499b700 1 --2- 172.21.15.2:0/3071499118 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f58900a07b0 0x7f58900a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:33.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.887+0000 7f58a499b700 1 --2- 172.21.15.2:0/3071499118 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f58900a07b0 0x7f58900a2c70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5894005fd0 tx=0x7f5894009310 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:50:33.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:33.890+0000 7f589dffb700 1 -- 172.21.15.2:0/3071499118 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f589804ee10 con 0x7f58a010d480 2023-09-13T04:50:34.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.102+0000 7f58a6bff700 1 -- 172.21.15.2:0/3071499118 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f58a0112df0 con 0x7f58a010d480 2023-09-13T04:50:34.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.103+0000 7f589dffb700 1 -- 172.21.15.2:0/3071499118 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v103) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5898094f80 con 0x7f58a010d480 2023-09-13T04:50:34.103 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-09-13T04:50:34.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.105+0000 7f58a6bff700 1 -- 172.21.15.2:0/3071499118 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f58900a07b0 msgr2=0x7f58900a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:34.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.105+0000 7f58a6bff700 1 --2- 172.21.15.2:0/3071499118 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f58900a07b0 0x7f58900a2c70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5894005fd0 tx=0x7f5894009310 comp rx=0 tx=0).stop 2023-09-13T04:50:34.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.105+0000 7f58a6bff700 1 -- 172.21.15.2:0/3071499118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58a010d480 msgr2=0x7f58a019da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:34.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.105+0000 7f58a6bff700 1 --2- 172.21.15.2:0/3071499118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58a010d480 0x7f58a019da90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5898007ee0 tx=0x7f589800d3b0 comp rx=0 tx=0).stop 2023-09-13T04:50:34.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.105+0000 7f58a6bff700 1 -- 172.21.15.2:0/3071499118 shutdown_connections 2023-09-13T04:50:34.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.105+0000 7f58a6bff700 1 --2- 172.21.15.2:0/3071499118 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f58900a07b0 0x7f58900a2c70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:34.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.105+0000 7f58a6bff700 1 --2- 172.21.15.2:0/3071499118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58a010d480 0x7f58a019da90 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:34.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.105+0000 7f58a6bff700 1 --2- 172.21.15.2:0/3071499118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58a00730e0 0x7f58a01990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:34.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.105+0000 7f58a6bff700 1 --2- 172.21.15.2:0/3071499118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58a0073a00 0x7f58a0199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:34.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.105+0000 7f58a6bff700 1 -- 172.21.15.2:0/3071499118 >> 172.21.15.2:0/3071499118 conn(0x7f58a00fc750 msgr2=0x7f58a010f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:50:34.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.105+0000 7f58a6bff700 1 -- 172.21.15.2:0/3071499118 shutdown_connections 2023-09-13T04:50:34.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.105+0000 7f58a6bff700 1 -- 172.21.15.2:0/3071499118 wait complete. 2023-09-13T04:50:34.125 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-09-13T04:50:34.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.329+0000 7f58755ff700 1 -- 172.21.15.2:0/2122341811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5870101880 msgr2=0x7f58701058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:34.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.329+0000 7f58755ff700 1 --2- 172.21.15.2:0/2122341811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5870101880 0x7f58701058c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5858009990 tx=0x7f5858009ca0 comp rx=0 tx=0).stop 2023-09-13T04:50:34.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.329+0000 7f58755ff700 1 -- 172.21.15.2:0/2122341811 shutdown_connections 2023-09-13T04:50:34.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.329+0000 7f58755ff700 1 --2- 172.21.15.2:0/2122341811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5870105ff0 0x7f587010db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:34.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.329+0000 7f58755ff700 1 --2- 172.21.15.2:0/2122341811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5870100ed0 0x7f58701012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:34.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.329+0000 7f58755ff700 1 --2- 172.21.15.2:0/2122341811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5870101880 0x7f58701058c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:34.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.329+0000 7f58755ff700 1 -- 172.21.15.2:0/2122341811 >> 172.21.15.2:0/2122341811 conn(0x7f58700fc790 msgr2=0x7f58700febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:50:34.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.329+0000 7f58755ff700 1 -- 172.21.15.2:0/2122341811 shutdown_connections 2023-09-13T04:50:34.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.329+0000 7f58755ff700 1 -- 172.21.15.2:0/2122341811 wait complete. 2023-09-13T04:50:34.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.330+0000 7f58755ff700 1 Processor -- start 2023-09-13T04:50:34.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.330+0000 7f58755ff700 1 -- start start 2023-09-13T04:50:34.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.330+0000 7f58755ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5870100ed0 0x7f5870196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:34.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.330+0000 7f58755ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5870105ff0 0x7f58701972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:34.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.330+0000 7f58755ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f587019b750 0x7f587019bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:34.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.330+0000 7f58755ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587019c1d0 con 0x7f5870105ff0 2023-09-13T04:50:34.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.330+0000 7f58755ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587019c340 con 0x7f587019b750 2023-09-13T04:50:34.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.330+0000 7f58755ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587019c500 con 0x7f5870100ed0 2023-09-13T04:50:34.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.331+0000 7f586effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5870100ed0 0x7f5870196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:34.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.331+0000 7f586effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5870100ed0 0x7f5870196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:58734/0 (socket says 172.21.15.2:58734) 2023-09-13T04:50:34.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.331+0000 7f5867fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5870105ff0 0x7f58701972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:34.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.331+0000 7f586effd700 1 -- 172.21.15.2:0/853484461 learned_addr learned my addr 172.21.15.2:0/853484461 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:50:34.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.331+0000 7f586f7fe700 1 --2- 172.21.15.2:0/853484461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f587019b750 0x7f587019bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:34.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.331+0000 7f586effd700 1 -- 172.21.15.2:0/853484461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f587019b750 msgr2=0x7f587019bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:34.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.331+0000 7f586effd700 1 --2- 172.21.15.2:0/853484461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f587019b750 0x7f587019bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:34.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.331+0000 7f586effd700 1 -- 172.21.15.2:0/853484461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5870105ff0 msgr2=0x7f58701972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:34.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.331+0000 7f586effd700 1 --2- 172.21.15.2:0/853484461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5870105ff0 0x7f58701972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:34.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.331+0000 7f586effd700 1 -- 172.21.15.2:0/853484461 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58580095f0 con 0x7f5870100ed0 2023-09-13T04:50:34.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.331+0000 7f5867fff700 1 --2- 172.21.15.2:0/853484461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5870105ff0 0x7f58701972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T04:50:34.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.331+0000 7f586f7fe700 1 --2- 172.21.15.2:0/853484461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f587019b750 0x7f587019bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T04:50:34.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.331+0000 7f586effd700 1 --2- 172.21.15.2:0/853484461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5870100ed0 0x7f5870196db0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f586000e9a0 tx=0x7f586000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:50:34.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.332+0000 7f586cff9700 1 -- 172.21.15.2:0/853484461 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5860009790 con 0x7f5870100ed0 2023-09-13T04:50:34.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.332+0000 7f58755ff700 1 -- 172.21.15.2:0/853484461 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58701a3a60 con 0x7f5870100ed0 2023-09-13T04:50:34.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.332+0000 7f586cff9700 1 -- 172.21.15.2:0/853484461 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f586000cbd0 con 0x7f5870100ed0 2023-09-13T04:50:34.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.332+0000 7f586cff9700 1 -- 172.21.15.2:0/853484461 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5860018680 con 0x7f5870100ed0 2023-09-13T04:50:34.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.332+0000 7f58755ff700 1 -- 172.21.15.2:0/853484461 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58701a3f90 con 0x7f5870100ed0 2023-09-13T04:50:34.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.333+0000 7f58755ff700 1 -- 172.21.15.2:0/853484461 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5870067ad0 con 0x7f5870100ed0 2023-09-13T04:50:34.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.335+0000 7f586cff9700 1 -- 172.21.15.2:0/853484461 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f5860010430 con 0x7f5870100ed0 2023-09-13T04:50:34.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.336+0000 7f586cff9700 1 --2- 172.21.15.2:0/853484461 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f585c0a07f0 0x7f585c0a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:50:34.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.336+0000 7f586cff9700 1 -- 172.21.15.2:0/853484461 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9483+0+0 (secure 0 0 0) 0x7f5860014070 con 0x7f5870100ed0 2023-09-13T04:50:34.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.336+0000 7f5867fff700 1 --2- 172.21.15.2:0/853484461 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f585c0a07f0 0x7f585c0a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:50:34.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.337+0000 7f5867fff700 1 --2- 172.21.15.2:0/853484461 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f585c0a07f0 0x7f585c0a2cb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f58701983c0 tx=0x7f5858005fd0 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:50:34.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.338+0000 7f586cff9700 1 -- 172.21.15.2:0/853484461 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5860051e10 con 0x7f5870100ed0 2023-09-13T04:50:34.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:34 smithi149 ceph-mon[133315]: pgmap v315: 129 pgs: 4 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 112 active+clean; 1015 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:50:34.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:34 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2455708286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:50:34.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:34 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1714171587' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:50:34.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:34 smithi156 ceph-mon[133675]: pgmap v315: 129 pgs: 4 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 112 active+clean; 1015 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:50:34.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:34 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2455708286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:50:34.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:34 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1714171587' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:50:34.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:34 smithi002 ceph-mon[97651]: pgmap v315: 129 pgs: 4 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 112 active+clean; 1015 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:50:34.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:34 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2455708286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:50:34.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:34 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1714171587' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:50:34.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.614+0000 7f58755ff700 1 -- 172.21.15.2:0/853484461 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f5870101a00 con 0x7f5870100ed0 2023-09-13T04:50:34.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.624+0000 7f586cff9700 1 -- 172.21.15.2:0/853484461 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) v1 ==== 81+0+730414 (secure 0 0 0) 0x7f5860095ad0 con 0x7f5870100ed0 2023-09-13T04:50:34.633 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:50:34.634 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:50:34.634 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 25, 2023-09-13T04:50:34.634 INFO:teuthology.orchestra.run.smithi002.stdout: "active_gid": 15492, 2023-09-13T04:50:34.635 INFO:teuthology.orchestra.run.smithi002.stdout: "active_name": "x", 2023-09-13T04:50:34.635 INFO:teuthology.orchestra.run.smithi002.stdout: "active_addrs": { 2023-09-13T04:50:34.635 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2023-09-13T04:50:34.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:34.635 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2023-09-13T04:50:34.636 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2023-09-13T04:50:34.636 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3640961355 2023-09-13T04:50:34.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:34.636 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2023-09-13T04:50:34.637 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2023-09-13T04:50:34.637 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3640961355 2023-09-13T04:50:34.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:34.637 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2023-09-13T04:50:34.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.638 INFO:teuthology.orchestra.run.smithi002.stdout: "active_addr": "172.21.15.2:6801/3640961355", 2023-09-13T04:50:34.638 INFO:teuthology.orchestra.run.smithi002.stdout: "active_change": "2023-09-13T04:40:27.774847+0000", 2023-09-13T04:50:34.638 INFO:teuthology.orchestra.run.smithi002.stdout: "active_mgr_features": 4540138322906710015, 2023-09-13T04:50:34.638 INFO:teuthology.orchestra.run.smithi002.stdout: "available": true, 2023-09-13T04:50:34.638 INFO:teuthology.orchestra.run.smithi002.stdout: "standbys": [ 2023-09-13T04:50:34.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:34.639 INFO:teuthology.orchestra.run.smithi002.stdout: "gid": 25517, 2023-09-13T04:50:34.639 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "y", 2023-09-13T04:50:34.639 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_features": 4540138322906710015, 2023-09-13T04:50:34.639 INFO:teuthology.orchestra.run.smithi002.stdout: "available_modules": [ 2023-09-13T04:50:34.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:34.640 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "alerts", 2023-09-13T04:50:34.640 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:34.640 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:34.640 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:34.641 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-09-13T04:50:34.641 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-09-13T04:50:34.641 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:34.641 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.641 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.642 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-09-13T04:50:34.642 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.642 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.642 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.642 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How frequently to reexamine health status", 2023-09-13T04:50:34.642 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.643 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.643 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.643 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:34.643 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:34.644 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.644 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.644 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.644 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:34.644 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.645 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.645 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:34.645 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:34.645 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:34.645 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:34.646 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:34.646 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:34.646 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:34.646 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:34.646 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.647 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.647 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.647 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.647 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:34.648 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:34.648 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.648 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.648 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.648 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:34.649 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.649 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.649 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.649 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.649 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.650 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.650 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.650 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:34.650 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:34.651 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.651 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.651 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.651 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:34.651 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.651 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.652 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:34.652 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:34.652 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:34.652 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:34.652 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:34.653 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:34.653 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:34.653 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:34.653 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.653 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.653 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.654 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.654 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:34.654 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:34.654 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.655 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.655 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.655 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:34.655 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.655 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.656 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.656 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.656 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.656 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.656 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.657 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_destination": { 2023-09-13T04:50:34.657 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_destination", 2023-09-13T04:50:34.657 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.657 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.658 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.658 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:34.658 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.658 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.658 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.658 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email address to send alerts to", 2023-09-13T04:50:34.659 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.659 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.659 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.659 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_from_name": { 2023-09-13T04:50:34.660 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_from_name", 2023-09-13T04:50:34.660 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.660 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.660 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.660 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "Ceph", 2023-09-13T04:50:34.661 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.661 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.661 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.661 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email From: name", 2023-09-13T04:50:34.661 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.662 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.662 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.662 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_host": { 2023-09-13T04:50:34.662 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_host", 2023-09-13T04:50:34.663 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.663 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.663 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.663 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:34.663 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.663 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.664 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.664 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP server", 2023-09-13T04:50:34.664 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.664 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.664 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.665 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_password": { 2023-09-13T04:50:34.665 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_password", 2023-09-13T04:50:34.665 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.665 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.666 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.666 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:34.666 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.666 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.666 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.667 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Password to authenticate with", 2023-09-13T04:50:34.667 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.667 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.667 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.668 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_port": { 2023-09-13T04:50:34.668 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_port", 2023-09-13T04:50:34.668 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:34.668 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.668 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.668 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "465", 2023-09-13T04:50:34.669 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.669 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.669 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.669 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP port", 2023-09-13T04:50:34.669 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.670 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.670 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.670 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_sender": { 2023-09-13T04:50:34.670 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_sender", 2023-09-13T04:50:34.671 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.671 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.671 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.671 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:34.671 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.672 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.672 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.672 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP envelope sender", 2023-09-13T04:50:34.672 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.672 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.672 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.673 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_ssl": { 2023-09-13T04:50:34.673 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_ssl", 2023-09-13T04:50:34.673 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.673 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.674 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.674 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.674 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.674 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.674 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.675 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use SSL to connect to SMTP server", 2023-09-13T04:50:34.675 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.675 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.675 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.676 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_user": { 2023-09-13T04:50:34.676 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_user", 2023-09-13T04:50:34.676 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.676 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.676 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.677 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:34.677 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.677 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.677 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.677 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "User to authenticate as", 2023-09-13T04:50:34.678 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.678 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.678 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.678 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:34.678 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:34.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:34.679 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "balancer", 2023-09-13T04:50:34.679 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:34.679 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:34.680 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:34.680 INFO:teuthology.orchestra.run.smithi002.stdout: "active": { 2023-09-13T04:50:34.680 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "active", 2023-09-13T04:50:34.680 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.680 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.680 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.681 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.681 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.681 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.681 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.681 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically balance PGs across cluster", 2023-09-13T04:50:34.682 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.682 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.682 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.682 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_time": { 2023-09-13T04:50:34.683 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_time", 2023-09-13T04:50:34.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.683 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.683 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.683 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0000", 2023-09-13T04:50:34.684 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.684 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.684 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.684 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "beginning time of day to automatically balance", 2023-09-13T04:50:34.684 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-13T04:50:34.684 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.685 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.685 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_weekday": { 2023-09-13T04:50:34.685 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_weekday", 2023-09-13T04:50:34.685 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-09-13T04:50:34.686 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.686 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.686 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-09-13T04:50:34.686 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2023-09-13T04:50:34.686 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2023-09-13T04:50:34.687 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.687 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-09-13T04:50:34.687 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-13T04:50:34.687 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.687 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.688 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_max_iterations": { 2023-09-13T04:50:34.688 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_max_iterations", 2023-09-13T04:50:34.688 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-09-13T04:50:34.688 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.688 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.689 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "25", 2023-09-13T04:50:34.689 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2023-09-13T04:50:34.689 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "250", 2023-09-13T04:50:34.689 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.689 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-09-13T04:50:34.690 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.690 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.690 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.690 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_metrics": { 2023-09-13T04:50:34.691 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_metrics", 2023-09-13T04:50:34.691 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.691 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.691 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.691 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "pgs,objects,bytes", 2023-09-13T04:50:34.692 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.692 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.692 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.692 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-09-13T04:50:34.692 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-09-13T04:50:34.693 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.693 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.693 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_step": { 2023-09-13T04:50:34.693 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_step", 2023-09-13T04:50:34.693 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-09-13T04:50:34.694 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.694 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.694 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.5", 2023-09-13T04:50:34.694 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0.001", 2023-09-13T04:50:34.694 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "0.999", 2023-09-13T04:50:34.695 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.695 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "aggressiveness of optimization", 2023-09-13T04:50:34.695 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-09-13T04:50:34.695 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.695 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.696 INFO:teuthology.orchestra.run.smithi002.stdout: "end_time": { 2023-09-13T04:50:34.696 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_time", 2023-09-13T04:50:34.696 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.696 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.696 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.697 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2359", 2023-09-13T04:50:34.697 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.697 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.697 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.697 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ending time of day to automatically balance", 2023-09-13T04:50:34.698 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-13T04:50:34.698 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.698 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.698 INFO:teuthology.orchestra.run.smithi002.stdout: "end_weekday": { 2023-09-13T04:50:34.699 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_weekday", 2023-09-13T04:50:34.699 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-09-13T04:50:34.699 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.699 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.699 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-09-13T04:50:34.699 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2023-09-13T04:50:34.700 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2023-09-13T04:50:34.700 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.700 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-09-13T04:50:34.700 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-13T04:50:34.700 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.701 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.701 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:34.701 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:34.701 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.702 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.702 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.702 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:34.702 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.702 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.703 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:34.703 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:34.703 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:34.703 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:34.703 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:34.704 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:34.704 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.707 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.707 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:34.707 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.707 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.708 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.708 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.708 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.708 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.708 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.709 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:34.709 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:34.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.709 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.709 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.710 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:34.710 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.710 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.710 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:34.710 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:34.711 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:34.711 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:34.711 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:34.711 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:34.711 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:34.712 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:34.712 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.712 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.712 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.712 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.713 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:34.713 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:34.713 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.713 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.713 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.714 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:34.714 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.714 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.714 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.714 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.715 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.715 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.715 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.715 INFO:teuthology.orchestra.run.smithi002.stdout: "min_score": { 2023-09-13T04:50:34.716 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_score", 2023-09-13T04:50:34.716 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-09-13T04:50:34.716 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.716 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.716 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-09-13T04:50:34.717 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.717 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.717 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.717 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-09-13T04:50:34.717 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.717 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.718 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.718 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2023-09-13T04:50:34.718 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2023-09-13T04:50:34.718 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.719 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.719 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.719 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "upmap", 2023-09-13T04:50:34.719 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.719 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.720 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:34.720 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-compat", 2023-09-13T04:50:34.720 INFO:teuthology.orchestra.run.smithi002.stdout: "none", 2023-09-13T04:50:34.720 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap" 2023-09-13T04:50:34.720 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:34.721 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Balancer mode", 2023-09-13T04:50:34.721 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.721 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.721 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.721 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_ids": { 2023-09-13T04:50:34.722 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_ids", 2023-09-13T04:50:34.722 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.722 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.722 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.725 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:34.725 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.726 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.726 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.726 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-09-13T04:50:34.726 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.726 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.727 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.727 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-09-13T04:50:34.727 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-09-13T04:50:34.727 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:34.728 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.728 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.728 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-09-13T04:50:34.728 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.728 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.729 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.729 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-09-13T04:50:34.729 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.729 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.729 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.730 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_deviation": { 2023-09-13T04:50:34.730 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_deviation", 2023-09-13T04:50:34.730 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:34.730 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.731 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.731 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2023-09-13T04:50:34.731 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2023-09-13T04:50:34.731 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.731 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.731 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "deviation below which no optimization is attempted", 2023-09-13T04:50:34.732 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-09-13T04:50:34.732 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.732 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.732 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_optimizations": { 2023-09-13T04:50:34.733 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_optimizations", 2023-09-13T04:50:34.733 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-09-13T04:50:34.733 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.733 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.733 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-09-13T04:50:34.734 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.734 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.734 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.734 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-09-13T04:50:34.734 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.735 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.735 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.735 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:34.735 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:34.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:34.736 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm", 2023-09-13T04:50:34.736 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:34.736 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:34.736 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:34.736 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_down_multiplier": { 2023-09-13T04:50:34.737 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_down_multiplier", 2023-09-13T04:50:34.737 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-09-13T04:50:34.737 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.737 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.737 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2023-09-13T04:50:34.738 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.738 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.738 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.738 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-09-13T04:50:34.738 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.739 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.739 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.739 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_refresh_rate": { 2023-09-13T04:50:34.740 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_refresh_rate", 2023-09-13T04:50:34.740 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:34.740 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.740 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.740 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "20", 2023-09-13T04:50:34.741 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.741 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.741 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.741 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-09-13T04:50:34.741 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.741 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.742 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.742 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_starting_port": { 2023-09-13T04:50:34.742 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_starting_port", 2023-09-13T04:50:34.742 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:34.743 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.743 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.743 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4721", 2023-09-13T04:50:34.743 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.743 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.743 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.744 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-09-13T04:50:34.744 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.744 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.744 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.745 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_ptrace": { 2023-09-13T04:50:34.745 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_ptrace", 2023-09-13T04:50:34.745 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.745 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.746 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.746 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:34.746 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.746 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.746 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.747 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-09-13T04:50:34.747 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-09-13T04:50:34.747 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.747 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.747 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_interval": { 2023-09-13T04:50:34.748 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_interval", 2023-09-13T04:50:34.748 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:34.748 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.748 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.748 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-09-13T04:50:34.749 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.749 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.749 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.749 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to autotune daemon memory", 2023-09-13T04:50:34.749 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.750 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.750 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.750 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_memory_target_ratio": { 2023-09-13T04:50:34.750 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_memory_target_ratio", 2023-09-13T04:50:34.751 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-09-13T04:50:34.751 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.751 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.751 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.7", 2023-09-13T04:50:34.751 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.752 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.752 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.752 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-09-13T04:50:34.752 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.752 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.753 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.753 INFO:teuthology.orchestra.run.smithi002.stdout: "cgroups_split": { 2023-09-13T04:50:34.753 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cgroups_split", 2023-09-13T04:50:34.754 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.754 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.754 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.754 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.754 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.755 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.755 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.755 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-09-13T04:50:34.755 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.755 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.756 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.756 INFO:teuthology.orchestra.run.smithi002.stdout: "config_checks_enabled": { 2023-09-13T04:50:34.756 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_checks_enabled", 2023-09-13T04:50:34.756 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.757 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.757 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.757 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:34.757 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.757 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.758 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.758 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-09-13T04:50:34.758 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.758 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.758 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.759 INFO:teuthology.orchestra.run.smithi002.stdout: "config_dashboard": { 2023-09-13T04:50:34.759 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_dashboard", 2023-09-13T04:50:34.759 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.759 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.760 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.760 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.760 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.760 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.760 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.761 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-09-13T04:50:34.761 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.761 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.761 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.762 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_alertmanager": { 2023-09-13T04:50:34.762 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_alertmanager", 2023-09-13T04:50:34.762 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.762 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.762 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.763 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-09-13T04:50:34.763 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.763 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.763 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.763 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2023-09-13T04:50:34.764 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.764 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.764 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.764 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_base": { 2023-09-13T04:50:34.765 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_base", 2023-09-13T04:50:34.765 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.765 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.765 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.765 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph", 2023-09-13T04:50:34.766 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.766 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.766 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.766 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Container image name, without the tag", 2023-09-13T04:50:34.766 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.767 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.767 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.767 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_elasticsearch": { 2023-09-13T04:50:34.767 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_elasticsearch", 2023-09-13T04:50:34.768 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.768 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.768 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.768 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-09-13T04:50:34.768 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.769 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.769 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.769 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "elasticsearch container image", 2023-09-13T04:50:34.769 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.769 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.770 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.770 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_grafana": { 2023-09-13T04:50:34.770 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_grafana", 2023-09-13T04:50:34.770 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.771 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.771 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.771 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-09-13T04:50:34.771 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.771 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.771 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.772 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2023-09-13T04:50:34.772 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.772 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.772 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.773 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_haproxy": { 2023-09-13T04:50:34.773 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_haproxy", 2023-09-13T04:50:34.773 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.773 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.773 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.774 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-09-13T04:50:34.774 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.774 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.774 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.774 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "HAproxy container image", 2023-09-13T04:50:34.775 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.775 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.775 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.775 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_agent": { 2023-09-13T04:50:34.775 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_agent", 2023-09-13T04:50:34.776 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.776 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.776 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.776 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-09-13T04:50:34.777 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.777 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.777 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.777 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger agent container image", 2023-09-13T04:50:34.777 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.778 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.778 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.778 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_collector": { 2023-09-13T04:50:34.778 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_collector", 2023-09-13T04:50:34.779 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.779 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.779 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.779 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-09-13T04:50:34.779 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.780 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.780 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.780 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger collector container image", 2023-09-13T04:50:34.780 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.780 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.781 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.781 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_query": { 2023-09-13T04:50:34.781 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_query", 2023-09-13T04:50:34.781 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.781 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.782 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.782 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-09-13T04:50:34.782 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.782 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.782 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.783 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger query container image", 2023-09-13T04:50:34.783 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.783 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.783 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.784 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_keepalived": { 2023-09-13T04:50:34.784 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_keepalived", 2023-09-13T04:50:34.784 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.784 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.784 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.785 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-09-13T04:50:34.785 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.785 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.785 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.785 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Keepalived container image", 2023-09-13T04:50:34.786 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.786 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.786 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.786 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_loki": { 2023-09-13T04:50:34.787 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_loki", 2023-09-13T04:50:34.787 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.787 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.787 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.787 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-09-13T04:50:34.788 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.788 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.788 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.788 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Loki container image", 2023-09-13T04:50:34.788 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.789 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.789 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.789 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_node_exporter": { 2023-09-13T04:50:34.789 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_node_exporter", 2023-09-13T04:50:34.790 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.790 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.790 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.790 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-09-13T04:50:34.790 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.791 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.791 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.791 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2023-09-13T04:50:34.791 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.791 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.792 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.792 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_nvmeof": { 2023-09-13T04:50:34.792 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_nvmeof", 2023-09-13T04:50:34.792 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.793 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.793 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.793 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-09-13T04:50:34.793 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.793 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.794 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.794 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Nvme-of container image", 2023-09-13T04:50:34.794 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.794 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.794 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.795 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_prometheus": { 2023-09-13T04:50:34.795 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_prometheus", 2023-09-13T04:50:34.795 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.795 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.795 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.796 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-09-13T04:50:34.796 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.796 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.796 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.796 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2023-09-13T04:50:34.797 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.797 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.797 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.797 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_promtail": { 2023-09-13T04:50:34.797 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_promtail", 2023-09-13T04:50:34.798 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.798 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.798 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.798 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-09-13T04:50:34.798 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.799 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.799 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.799 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Promtail container image", 2023-09-13T04:50:34.799 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.799 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.800 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.800 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_snmp_gateway": { 2023-09-13T04:50:34.800 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_snmp_gateway", 2023-09-13T04:50:34.800 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.801 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.801 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.801 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-09-13T04:50:34.801 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.801 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.801 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.802 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SNMP Gateway container image", 2023-09-13T04:50:34.802 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.802 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.802 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.803 INFO:teuthology.orchestra.run.smithi002.stdout: "container_init": { 2023-09-13T04:50:34.803 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_init", 2023-09-13T04:50:34.803 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.803 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.803 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.804 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.804 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.804 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.804 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.804 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Run podman/docker with `--init`", 2023-09-13T04:50:34.805 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.805 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.805 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.805 INFO:teuthology.orchestra.run.smithi002.stdout: "daemon_cache_timeout": { 2023-09-13T04:50:34.806 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "daemon_cache_timeout", 2023-09-13T04:50:34.806 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:34.806 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.806 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.806 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-09-13T04:50:34.807 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.807 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.807 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.807 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-09-13T04:50:34.807 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.808 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.808 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.808 INFO:teuthology.orchestra.run.smithi002.stdout: "default_cephadm_command_timeout": { 2023-09-13T04:50:34.808 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_cephadm_command_timeout", 2023-09-13T04:50:34.809 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:34.809 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.809 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.809 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "900", 2023-09-13T04:50:34.809 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.810 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.810 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.810 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-09-13T04:50:34.810 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.811 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.811 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.811 INFO:teuthology.orchestra.run.smithi002.stdout: "default_registry": { 2023-09-13T04:50:34.811 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_registry", 2023-09-13T04:50:34.812 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.812 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.812 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.812 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io", 2023-09-13T04:50:34.812 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.812 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.813 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.813 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-09-13T04:50:34.813 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.813 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.813 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.814 INFO:teuthology.orchestra.run.smithi002.stdout: "device_cache_timeout": { 2023-09-13T04:50:34.814 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_cache_timeout", 2023-09-13T04:50:34.814 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:34.814 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.815 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.815 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1800", 2023-09-13T04:50:34.815 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.815 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.815 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.816 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache device inventory", 2023-09-13T04:50:34.816 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.816 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.816 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.817 INFO:teuthology.orchestra.run.smithi002.stdout: "device_enhanced_scan": { 2023-09-13T04:50:34.817 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_enhanced_scan", 2023-09-13T04:50:34.817 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.817 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.817 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.818 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:34.818 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.818 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.818 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.818 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use libstoragemgmt during device scans", 2023-09-13T04:50:34.819 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.819 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.819 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.819 INFO:teuthology.orchestra.run.smithi002.stdout: "facts_cache_timeout": { 2023-09-13T04:50:34.819 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "facts_cache_timeout", 2023-09-13T04:50:34.820 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:34.820 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.820 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.820 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-09-13T04:50:34.820 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.821 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.821 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.821 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache host facts data", 2023-09-13T04:50:34.821 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.821 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.822 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.822 INFO:teuthology.orchestra.run.smithi002.stdout: "host_check_interval": { 2023-09-13T04:50:34.822 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "host_check_interval", 2023-09-13T04:50:34.822 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:34.823 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.823 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.823 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-09-13T04:50:34.823 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.823 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.823 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.824 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to perform a host check", 2023-09-13T04:50:34.824 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.824 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.824 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.825 INFO:teuthology.orchestra.run.smithi002.stdout: "inventory_list_all": { 2023-09-13T04:50:34.825 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "inventory_list_all", 2023-09-13T04:50:34.825 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.825 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.826 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.826 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:34.826 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.826 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.826 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.827 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-09-13T04:50:34.827 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.827 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.827 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.828 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:34.828 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:34.828 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.828 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.828 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.829 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:34.829 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.829 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.829 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:34.829 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:34.829 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:34.830 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:34.830 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:34.830 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:34.830 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:34.831 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:34.831 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.831 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.831 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.831 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.832 INFO:teuthology.orchestra.run.smithi002.stdout: "log_refresh_metadata": { 2023-09-13T04:50:34.832 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_refresh_metadata", 2023-09-13T04:50:34.832 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.832 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.832 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.833 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:34.833 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.833 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.833 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.833 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-09-13T04:50:34.834 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.834 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.834 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.835 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:34.835 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:34.835 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.835 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.835 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.835 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.836 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.836 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.836 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.836 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-09-13T04:50:34.836 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.837 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.837 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.837 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:34.837 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:34.838 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.838 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.838 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.838 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:34.838 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.839 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.839 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:34.839 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:34.839 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:34.839 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:34.840 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:34.840 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:34.840 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:34.840 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:34.840 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.840 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.841 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.841 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.841 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:34.841 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:34.842 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.842 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.842 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.842 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:34.842 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.843 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.843 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.843 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.843 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.843 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.844 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.844 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf": { 2023-09-13T04:50:34.844 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-09-13T04:50:34.844 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.844 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.845 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.845 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:34.845 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.845 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.846 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.846 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-09-13T04:50:34.846 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.846 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.846 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.847 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-09-13T04:50:34.847 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-09-13T04:50:34.847 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.847 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.848 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.848 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "*", 2023-09-13T04:50:34.848 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.848 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.848 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.848 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-09-13T04:50:34.849 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.849 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.849 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.849 INFO:teuthology.orchestra.run.smithi002.stdout: "max_count_per_host": { 2023-09-13T04:50:34.850 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_count_per_host", 2023-09-13T04:50:34.850 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:34.850 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.850 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.850 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-09-13T04:50:34.851 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.851 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.851 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.851 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of daemons per service per host", 2023-09-13T04:50:34.851 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.851 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.852 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.852 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd_draining_count": { 2023-09-13T04:50:34.852 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_osd_draining_count", 2023-09-13T04:50:34.852 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:34.853 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.853 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.853 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-09-13T04:50:34.853 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.853 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.854 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.854 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-09-13T04:50:34.854 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.854 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.854 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.855 INFO:teuthology.orchestra.run.smithi002.stdout: "migration_current": { 2023-09-13T04:50:34.855 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "migration_current", 2023-09-13T04:50:34.855 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:34.855 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.855 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.856 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:34.856 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.856 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.856 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.856 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "internal - do not modify", 2023-09-13T04:50:34.857 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.857 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.857 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.857 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2023-09-13T04:50:34.858 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2023-09-13T04:50:34.858 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.858 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.858 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.858 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "root", 2023-09-13T04:50:34.859 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.859 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.859 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:34.859 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm-package", 2023-09-13T04:50:34.859 INFO:teuthology.orchestra.run.smithi002.stdout: "root" 2023-09-13T04:50:34.859 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:34.860 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "mode for remote execution of cephadm", 2023-09-13T04:50:34.860 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.860 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.860 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.861 INFO:teuthology.orchestra.run.smithi002.stdout: "prometheus_alerts_path": { 2023-09-13T04:50:34.861 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus_alerts_path", 2023-09-13T04:50:34.861 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.861 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.861 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.862 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-09-13T04:50:34.862 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.862 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.862 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.862 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-09-13T04:50:34.863 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.863 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.863 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.863 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_insecure": { 2023-09-13T04:50:34.863 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_insecure", 2023-09-13T04:50:34.864 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.864 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.864 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.864 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:34.864 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.865 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.865 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.865 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-09-13T04:50:34.865 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.865 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.866 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.866 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_password": { 2023-09-13T04:50:34.866 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_password", 2023-09-13T04:50:34.866 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.867 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.867 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.867 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:34.867 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.867 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.868 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.868 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-09-13T04:50:34.868 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.868 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.868 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.869 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_url": { 2023-09-13T04:50:34.869 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_url", 2023-09-13T04:50:34.869 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.869 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.869 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.870 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:34.870 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.870 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.870 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.870 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-09-13T04:50:34.871 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.871 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.871 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.871 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_username": { 2023-09-13T04:50:34.872 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_username", 2023-09-13T04:50:34.872 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.872 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.872 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.872 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:34.872 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.873 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.873 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.873 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-09-13T04:50:34.873 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.873 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.874 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.874 INFO:teuthology.orchestra.run.smithi002.stdout: "secure_monitoring_stack": { 2023-09-13T04:50:34.874 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "secure_monitoring_stack", 2023-09-13T04:50:34.874 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.875 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.875 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.875 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:34.875 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.875 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.876 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.876 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-09-13T04:50:34.876 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.876 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.876 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.877 INFO:teuthology.orchestra.run.smithi002.stdout: "service_discovery_port": { 2023-09-13T04:50:34.877 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "service_discovery_port", 2023-09-13T04:50:34.877 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:34.877 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.877 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.878 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8765", 2023-09-13T04:50:34.878 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.878 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.878 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.878 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "cephadm service discovery port", 2023-09-13T04:50:34.879 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.879 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.879 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.879 INFO:teuthology.orchestra.run.smithi002.stdout: "ssh_config_file": { 2023-09-13T04:50:34.880 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssh_config_file", 2023-09-13T04:50:34.880 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.880 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.880 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.880 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:34.881 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.881 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.881 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.881 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-09-13T04:50:34.881 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.882 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.882 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.882 INFO:teuthology.orchestra.run.smithi002.stdout: "use_agent": { 2023-09-13T04:50:34.882 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_agent", 2023-09-13T04:50:34.883 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.883 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.883 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.883 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:34.883 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.884 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.884 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.884 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-09-13T04:50:34.884 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.884 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.885 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.885 INFO:teuthology.orchestra.run.smithi002.stdout: "use_repo_digest": { 2023-09-13T04:50:34.885 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_repo_digest", 2023-09-13T04:50:34.885 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.885 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.886 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.886 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.886 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.886 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.887 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.887 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-09-13T04:50:34.887 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.887 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.887 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.888 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_failed_host_check": { 2023-09-13T04:50:34.888 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_failed_host_check", 2023-09-13T04:50:34.888 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.888 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.889 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.889 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.889 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.889 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.889 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.890 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if the host check fails", 2023-09-13T04:50:34.890 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.890 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.890 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.891 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_daemons": { 2023-09-13T04:50:34.891 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_daemons", 2023-09-13T04:50:34.891 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.891 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.891 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.892 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.892 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.892 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.892 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.892 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-09-13T04:50:34.893 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.893 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.893 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.893 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_hosts": { 2023-09-13T04:50:34.893 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_hosts", 2023-09-13T04:50:34.894 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.894 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.894 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.894 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.894 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.895 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.895 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.895 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-09-13T04:50:34.895 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.896 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.896 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.896 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:34.896 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:34.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:34.897 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crash", 2023-09-13T04:50:34.897 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:34.897 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:34.897 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:34.898 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:34.898 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:34.898 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.898 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.898 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.899 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:34.899 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.899 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.899 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:34.899 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:34.900 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:34.900 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:34.900 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:34.900 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:34.900 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:34.901 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:34.901 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.901 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.901 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.901 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.902 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:34.902 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:34.902 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.902 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.903 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.903 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:34.903 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.903 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.903 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.903 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.904 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.904 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.904 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.904 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:34.905 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:34.905 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.905 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.905 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.905 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:34.906 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.906 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.906 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:34.906 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:34.906 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:34.907 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:34.907 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:34.907 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:34.907 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:34.907 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:34.908 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.908 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.908 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.908 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.908 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:34.909 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:34.909 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.909 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.909 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.909 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:34.910 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.910 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.910 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.910 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.910 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.911 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.911 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.911 INFO:teuthology.orchestra.run.smithi002.stdout: "retain_interval": { 2023-09-13T04:50:34.911 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retain_interval", 2023-09-13T04:50:34.912 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:34.912 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.912 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.912 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "31536000", 2023-09-13T04:50:34.912 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.913 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.913 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.913 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain crashes before pruning them", 2023-09-13T04:50:34.913 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.913 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.914 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.914 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_recent_interval": { 2023-09-13T04:50:34.914 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_recent_interval", 2023-09-13T04:50:34.915 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:34.915 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.915 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:34.915 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1209600", 2023-09-13T04:50:34.915 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.916 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.916 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.916 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "time interval in which to warn about recent crashes", 2023-09-13T04:50:34.916 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.916 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.916 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.917 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:34.917 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:34.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:34.917 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dashboard", 2023-09-13T04:50:34.918 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:34.918 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:34.918 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:34.918 INFO:teuthology.orchestra.run.smithi002.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-09-13T04:50:34.918 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-09-13T04:50:34.919 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:34.919 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.919 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.919 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-09-13T04:50:34.919 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.920 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.920 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.920 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.920 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.920 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.921 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.921 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_HOST": { 2023-09-13T04:50:34.921 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_HOST", 2023-09-13T04:50:34.921 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.922 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.922 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.922 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:34.922 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.922 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.923 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.923 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.923 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.923 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.923 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.924 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-09-13T04:50:34.924 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-09-13T04:50:34.924 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.924 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.925 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.925 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.925 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.925 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.925 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.925 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.926 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.926 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.926 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.926 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_ENABLED": { 2023-09-13T04:50:34.927 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_ENABLED", 2023-09-13T04:50:34.927 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.927 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.927 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.927 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:34.928 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.928 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.928 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.928 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.928 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.929 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.929 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.929 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-09-13T04:50:34.929 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-09-13T04:50:34.930 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.930 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.930 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.930 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.930 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.931 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.931 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.931 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.931 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.931 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.932 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.932 INFO:teuthology.orchestra.run.smithi002.stdout: "ENABLE_BROWSABLE_API": { 2023-09-13T04:50:34.932 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ENABLE_BROWSABLE_API", 2023-09-13T04:50:34.932 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.933 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.933 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.933 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.933 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.933 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.933 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.934 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.934 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.934 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.934 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.935 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-09-13T04:50:34.935 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-09-13T04:50:34.935 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.935 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.936 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.936 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.936 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.936 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.936 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.936 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.937 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.937 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.937 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.937 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-09-13T04:50:34.938 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-09-13T04:50:34.938 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.938 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.938 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.939 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.939 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.939 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.939 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.939 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.939 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.940 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.940 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.940 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-09-13T04:50:34.940 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-09-13T04:50:34.941 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.941 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.941 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.941 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.942 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.942 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.942 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.942 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.942 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.943 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.943 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.943 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-09-13T04:50:34.943 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-09-13T04:50:34.944 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.944 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.944 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.944 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.944 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.945 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.945 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.945 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.945 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.945 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.946 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.946 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_NFS": { 2023-09-13T04:50:34.946 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-09-13T04:50:34.946 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.947 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.947 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.947 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.947 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.947 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.948 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.948 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.948 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.948 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.948 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.949 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RBD": { 2023-09-13T04:50:34.949 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-09-13T04:50:34.949 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.949 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.950 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.950 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.950 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.950 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.950 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.951 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.951 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.951 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.951 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.952 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RGW": { 2023-09-13T04:50:34.952 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-09-13T04:50:34.952 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.952 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.952 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.953 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.953 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.953 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.953 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.953 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.954 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.954 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.954 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.955 INFO:teuthology.orchestra.run.smithi002.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-09-13T04:50:34.955 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-09-13T04:50:34.955 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.955 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.955 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.955 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:34.956 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.956 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.956 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.956 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.956 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.957 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.957 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.957 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_PASSWORD": { 2023-09-13T04:50:34.958 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_PASSWORD", 2023-09-13T04:50:34.958 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.958 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.958 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.958 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-09-13T04:50:34.959 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.959 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.959 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.959 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.959 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.959 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.960 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.960 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-09-13T04:50:34.960 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-09-13T04:50:34.960 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.961 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.961 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.961 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.961 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.961 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.962 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.962 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.962 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.962 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.962 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.963 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_URL": { 2023-09-13T04:50:34.963 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_URL", 2023-09-13T04:50:34.963 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.963 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.963 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.964 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:34.964 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.964 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.964 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.965 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.965 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.965 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.965 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.966 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_USERNAME": { 2023-09-13T04:50:34.966 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_USERNAME", 2023-09-13T04:50:34.966 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.966 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.966 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.967 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-09-13T04:50:34.967 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.967 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.967 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.967 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.968 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.968 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.968 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.968 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-09-13T04:50:34.968 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-09-13T04:50:34.969 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.969 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.969 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.969 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:34.969 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.970 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.970 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.970 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.970 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.971 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.971 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.971 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-09-13T04:50:34.971 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-09-13T04:50:34.971 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.972 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.972 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.972 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:34.972 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.972 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.973 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.973 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.973 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.973 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.973 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.974 INFO:teuthology.orchestra.run.smithi002.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-09-13T04:50:34.974 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-09-13T04:50:34.974 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.974 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.975 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.975 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.975 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.975 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.975 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.976 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.976 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.976 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.976 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.977 INFO:teuthology.orchestra.run.smithi002.stdout: "ISSUE_TRACKER_API_KEY": { 2023-09-13T04:50:34.977 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-09-13T04:50:34.977 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.977 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.977 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.978 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:34.978 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.978 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.978 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.978 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.979 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.979 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.979 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.979 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_HOST": { 2023-09-13T04:50:34.980 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_HOST", 2023-09-13T04:50:34.980 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:34.980 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.980 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.980 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:34.981 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.981 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.981 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.981 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.981 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.981 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.982 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.982 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-09-13T04:50:34.982 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-09-13T04:50:34.982 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.983 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.983 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.983 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.983 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.983 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.984 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.984 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.984 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.984 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.984 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.985 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-09-13T04:50:34.985 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-09-13T04:50:34.985 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.986 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.986 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.986 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:34.986 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.986 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.987 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.987 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.987 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.987 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.987 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.988 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-09-13T04:50:34.988 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-09-13T04:50:34.988 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.988 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.989 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.989 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:34.989 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.989 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.989 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.990 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.990 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.990 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.990 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.991 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-09-13T04:50:34.991 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-09-13T04:50:34.991 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.991 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.991 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.991 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.992 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.992 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.992 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.992 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.992 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.993 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.993 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.993 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-09-13T04:50:34.993 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-09-13T04:50:34.994 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.994 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.994 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.994 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:34.994 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.995 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.995 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.995 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.995 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.995 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.996 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.996 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-09-13T04:50:34.996 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-09-13T04:50:34.996 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.997 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.997 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:34.997 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:34.997 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:34.997 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:34.998 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:34.998 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:34.998 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:34.998 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:34.998 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:34.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:34.999 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-09-13T04:50:34.999 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-09-13T04:50:34.999 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:34.999 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:34.999 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.000 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.000 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.000 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.000 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.000 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.001 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.001 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.001 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.001 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-09-13T04:50:35.002 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-09-13T04:50:35.002 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.002 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.002 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.002 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.003 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.003 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.003 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.003 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.003 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.004 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.004 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.004 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_ENABLED": { 2023-09-13T04:50:35.004 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_ENABLED", 2023-09-13T04:50:35.004 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.005 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.005 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.005 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:35.005 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.005 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.006 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.006 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.006 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.006 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.007 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.007 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-09-13T04:50:35.007 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-09-13T04:50:35.007 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.008 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.008 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.008 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-09-13T04:50:35.008 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.008 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.009 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.009 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.009 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.009 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.009 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.010 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-09-13T04:50:35.010 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-09-13T04:50:35.010 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.010 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.011 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.011 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-09-13T04:50:35.011 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.011 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.011 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.012 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.012 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.012 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.012 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.013 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-09-13T04:50:35.013 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-09-13T04:50:35.013 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.013 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.013 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.014 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8", 2023-09-13T04:50:35.014 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.014 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.014 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.015 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.015 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.015 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.015 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.015 INFO:teuthology.orchestra.run.smithi002.stdout: "REST_REQUESTS_TIMEOUT": { 2023-09-13T04:50:35.016 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-09-13T04:50:35.016 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.016 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.016 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.016 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "45", 2023-09-13T04:50:35.017 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.017 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.017 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.017 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.017 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.017 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.018 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.018 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ACCESS_KEY": { 2023-09-13T04:50:35.018 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ACCESS_KEY", 2023-09-13T04:50:35.018 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.019 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.019 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.019 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.019 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.019 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.020 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.020 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.020 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.020 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.020 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.021 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-09-13T04:50:35.021 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-09-13T04:50:35.021 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.021 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.022 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.022 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-09-13T04:50:35.022 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.022 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.022 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.023 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.023 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.023 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.023 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.024 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SECRET_KEY": { 2023-09-13T04:50:35.024 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SECRET_KEY", 2023-09-13T04:50:35.024 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.024 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.025 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.025 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.025 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.025 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.025 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.025 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.026 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.026 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.026 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.026 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SSL_VERIFY": { 2023-09-13T04:50:35.027 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SSL_VERIFY", 2023-09-13T04:50:35.027 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.027 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.027 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.027 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:35.028 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.028 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.028 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.028 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.028 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.029 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.029 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.029 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-09-13T04:50:35.029 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-09-13T04:50:35.030 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.030 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.030 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.030 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-09-13T04:50:35.030 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.031 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.031 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.031 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.031 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.031 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.032 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.032 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-09-13T04:50:35.032 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-09-13T04:50:35.032 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.032 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.033 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.033 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-09-13T04:50:35.033 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.033 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.033 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.034 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.034 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.034 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.034 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.035 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-09-13T04:50:35.035 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-09-13T04:50:35.035 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.035 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.035 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.036 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2023-09-13T04:50:35.036 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.036 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.036 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.036 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.036 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.037 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.037 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.037 INFO:teuthology.orchestra.run.smithi002.stdout: "cross_origin_url": { 2023-09-13T04:50:35.037 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cross_origin_url", 2023-09-13T04:50:35.038 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.038 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.038 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.038 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.039 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.039 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.039 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.039 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.039 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.039 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.040 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.040 INFO:teuthology.orchestra.run.smithi002.stdout: "crt_file": { 2023-09-13T04:50:35.040 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crt_file", 2023-09-13T04:50:35.040 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.041 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.041 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.041 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.041 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.041 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.042 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.042 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.042 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.042 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.042 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.043 INFO:teuthology.orchestra.run.smithi002.stdout: "debug": { 2023-09-13T04:50:35.043 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "debug", 2023-09-13T04:50:35.043 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.043 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.044 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.044 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.044 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.044 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.044 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.045 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable/disable debug options", 2023-09-13T04:50:35.045 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.045 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.045 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.045 INFO:teuthology.orchestra.run.smithi002.stdout: "jwt_token_ttl": { 2023-09-13T04:50:35.046 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "jwt_token_ttl", 2023-09-13T04:50:35.046 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.046 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.046 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.046 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "28800", 2023-09-13T04:50:35.047 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.047 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.047 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.047 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.047 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.048 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.048 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.048 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2023-09-13T04:50:35.048 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2023-09-13T04:50:35.049 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.049 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.049 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.049 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.049 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.050 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.050 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.050 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.050 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.051 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.051 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.051 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.051 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.052 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.052 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.052 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.052 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.052 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.053 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.053 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.053 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.053 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.053 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.054 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.054 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.054 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.054 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.054 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.055 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.055 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.055 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.055 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.055 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.056 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.056 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.056 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.056 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.056 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.057 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.057 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.057 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.057 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.057 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.058 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.058 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.058 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.058 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.059 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.059 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.059 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.059 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.059 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.060 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.060 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.060 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.060 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.060 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.061 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.061 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.061 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.061 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.061 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.062 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.062 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.062 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.062 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.063 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.063 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.063 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.063 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.063 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.064 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.064 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.064 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.064 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.064 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.065 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.065 INFO:teuthology.orchestra.run.smithi002.stdout: "motd": { 2023-09-13T04:50:35.065 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "motd", 2023-09-13T04:50:35.065 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.066 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.066 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.066 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.066 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.066 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.067 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.067 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "The message of the day", 2023-09-13T04:50:35.067 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.067 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.067 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.068 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect_resolve_ip_addr": { 2023-09-13T04:50:35.068 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "redirect_resolve_ip_addr", 2023-09-13T04:50:35.068 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.068 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.069 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.069 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.069 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.069 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.069 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.070 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.070 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.070 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.070 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.071 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2023-09-13T04:50:35.071 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2023-09-13T04:50:35.071 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.071 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.072 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.072 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2023-09-13T04:50:35.072 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.072 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.072 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.072 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.073 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.073 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.073 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.073 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2023-09-13T04:50:35.074 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2023-09-13T04:50:35.074 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.074 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.074 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.074 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8080", 2023-09-13T04:50:35.075 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.075 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.075 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.075 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.075 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.076 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.076 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.076 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2023-09-13T04:50:35.076 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2023-09-13T04:50:35.077 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.077 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.077 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.077 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:35.077 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.078 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.078 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.078 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.078 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.078 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.079 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.079 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl_server_port": { 2023-09-13T04:50:35.079 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl_server_port", 2023-09-13T04:50:35.079 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.080 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.080 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.080 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8443", 2023-09-13T04:50:35.080 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.080 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.081 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.081 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.081 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.081 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.081 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.082 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2023-09-13T04:50:35.082 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2023-09-13T04:50:35.082 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.082 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.083 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.083 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "redirect", 2023-09-13T04:50:35.083 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.083 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.083 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.084 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.084 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect" 2023-09-13T04:50:35.084 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.084 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.084 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.085 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.085 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.085 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2023-09-13T04:50:35.085 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2023-09-13T04:50:35.085 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.086 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.086 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.086 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2023-09-13T04:50:35.086 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2023-09-13T04:50:35.086 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2023-09-13T04:50:35.087 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.087 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.087 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.087 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.087 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.088 INFO:teuthology.orchestra.run.smithi002.stdout: "url_prefix": { 2023-09-13T04:50:35.088 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url_prefix", 2023-09-13T04:50:35.088 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.088 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.089 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.089 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.089 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.089 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.089 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.090 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.090 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.090 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.090 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.090 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.091 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.091 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "devicehealth", 2023-09-13T04:50:35.091 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.091 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.092 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.092 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_monitoring": { 2023-09-13T04:50:35.092 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_monitoring", 2023-09-13T04:50:35.092 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.092 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.093 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.093 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:35.093 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.093 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.093 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "monitor device health metrics", 2023-09-13T04:50:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.095 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.095 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.095 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.095 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.095 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.096 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.096 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.096 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.096 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.096 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.097 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.097 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.097 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.097 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.097 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.098 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.098 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.098 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.098 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.099 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.099 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.099 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.099 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.100 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.100 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.100 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.100 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.100 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.101 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.101 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.101 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.101 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.101 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.102 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.102 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.102 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.102 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "mark_out_threshold": { 2023-09-13T04:50:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mark_out_threshold", 2023-09-13T04:50:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2419200", 2023-09-13T04:50:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-09-13T04:50:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": { 2023-09-13T04:50:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_name", 2023-09-13T04:50:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "device_health_metrics", 2023-09-13T04:50:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name of pool in which to store device health metrics", 2023-09-13T04:50:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "retention_period": { 2023-09-13T04:50:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retention_period", 2023-09-13T04:50:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15552000", 2023-09-13T04:50:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain device health metrics", 2023-09-13T04:50:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_frequency": { 2023-09-13T04:50:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_frequency", 2023-09-13T04:50:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2023-09-13T04:50:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to scrape device health metrics", 2023-09-13T04:50:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "self_heal": { 2023-09-13T04:50:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "self_heal", 2023-09-13T04:50:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-09-13T04:50:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-09-13T04:50:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-09-13T04:50:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-09-13T04:50:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and check device health", 2023-09-13T04:50:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_threshold": { 2023-09-13T04:50:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_threshold", 2023-09-13T04:50:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7257600", 2023-09-13T04:50:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-09-13T04:50:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "diskprediction_local", 2023-09-13T04:50:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "predict_interval": { 2023-09-13T04:50:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predict_interval", 2023-09-13T04:50:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2023-09-13T04:50:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: "predictor_model": { 2023-09-13T04:50:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predictor_model", 2023-09-13T04:50:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "prophetstor", 2023-09-13T04:50:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-09-13T04:50:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-09-13T04:50:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-09-13T04:50:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "influx", 2023-09-13T04:50:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": false, 2023-09-13T04:50:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "influxdb python module not found", 2023-09-13T04:50:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: "batch_size": { 2023-09-13T04:50:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "batch_size", 2023-09-13T04:50:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5000", 2023-09-13T04:50:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-09-13T04:50:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: "database": { 2023-09-13T04:50:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "database", 2023-09-13T04:50:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "ceph", 2023-09-13T04:50:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-09-13T04:50:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": { 2023-09-13T04:50:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hostname", 2023-09-13T04:50:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server hostname", 2023-09-13T04:50:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-09-13T04:50:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-09-13T04:50:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "30", 2023-09-13T04:50:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "5", 2023-09-13T04:50:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-09-13T04:50:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.174 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.174 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.174 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.174 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.174 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.175 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.175 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.175 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.175 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.176 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.176 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.176 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.176 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.176 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.177 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.177 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.177 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.177 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.177 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.178 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.178 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.178 INFO:teuthology.orchestra.run.smithi002.stdout: "password": { 2023-09-13T04:50:35.178 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "password", 2023-09-13T04:50:35.178 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.179 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.179 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.179 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.179 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.179 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.180 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.180 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "password of InfluxDB server user", 2023-09-13T04:50:35.180 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.180 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.180 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.181 INFO:teuthology.orchestra.run.smithi002.stdout: "port": { 2023-09-13T04:50:35.181 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "port", 2023-09-13T04:50:35.181 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.181 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.182 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.182 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8086", 2023-09-13T04:50:35.182 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.182 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.182 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.183 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server port", 2023-09-13T04:50:35.183 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.183 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.183 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.184 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2023-09-13T04:50:35.184 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2023-09-13T04:50:35.184 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.184 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.184 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.185 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "false", 2023-09-13T04:50:35.185 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.185 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.185 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.185 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-09-13T04:50:35.186 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.186 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.186 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.186 INFO:teuthology.orchestra.run.smithi002.stdout: "threads": { 2023-09-13T04:50:35.186 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threads", 2023-09-13T04:50:35.187 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.187 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.187 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.187 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2023-09-13T04:50:35.187 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2023-09-13T04:50:35.188 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "32", 2023-09-13T04:50:35.188 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.188 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-09-13T04:50:35.188 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.188 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.189 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.189 INFO:teuthology.orchestra.run.smithi002.stdout: "username": { 2023-09-13T04:50:35.190 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "username", 2023-09-13T04:50:35.190 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.193 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.193 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.193 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.193 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.193 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.194 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.194 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "username of InfluxDB server user", 2023-09-13T04:50:35.194 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.194 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.194 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.195 INFO:teuthology.orchestra.run.smithi002.stdout: "verify_ssl": { 2023-09-13T04:50:35.195 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "verify_ssl", 2023-09-13T04:50:35.195 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.195 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.196 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.196 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "true", 2023-09-13T04:50:35.196 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.196 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.196 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.197 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-09-13T04:50:35.197 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.197 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.197 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.197 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.198 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.198 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "insights", 2023-09-13T04:50:35.198 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.199 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.199 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.199 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.199 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.199 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.200 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.200 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.200 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.200 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.200 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.201 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.201 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.201 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.201 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.201 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.202 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.202 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.202 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.202 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.202 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.203 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.203 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.203 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.203 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.204 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.204 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.204 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.204 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.204 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.204 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.205 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.205 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.205 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.205 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.205 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.206 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.206 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.206 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.206 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.207 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.207 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.207 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.207 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.207 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.208 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.208 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.208 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.208 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.208 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.209 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.209 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.209 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.209 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.209 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.209 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.210 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.210 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.210 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.210 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.211 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.211 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.211 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.211 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.211 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.212 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.212 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.212 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.212 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.213 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.213 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.213 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "iostat", 2023-09-13T04:50:35.214 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.214 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.214 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.214 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.214 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.215 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.215 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.215 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.215 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.215 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.215 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.216 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.216 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.216 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.216 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.217 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.217 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.217 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.217 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.217 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.217 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.218 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.218 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.218 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.219 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.219 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.219 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.219 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.219 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.219 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.220 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.220 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.220 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.220 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.221 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.221 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.221 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.221 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.221 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.222 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.222 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.222 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.222 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.222 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.223 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.223 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.223 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.223 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.223 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.223 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.224 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.224 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.224 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.224 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.225 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.225 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.225 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.225 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.225 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.226 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.226 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.226 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.226 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.226 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.227 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.227 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.227 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.227 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.227 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.228 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.228 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.228 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "k8sevents", 2023-09-13T04:50:35.229 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.229 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.229 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.229 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_event_retention_days": { 2023-09-13T04:50:35.229 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ceph_event_retention_days", 2023-09-13T04:50:35.230 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.230 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.230 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.230 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7", 2023-09-13T04:50:35.230 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.231 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.231 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.231 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Days to hold ceph event information within local cache", 2023-09-13T04:50:35.231 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.231 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.232 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.232 INFO:teuthology.orchestra.run.smithi002.stdout: "config_check_secs": { 2023-09-13T04:50:35.232 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_check_secs", 2023-09-13T04:50:35.232 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.233 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.233 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.233 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-09-13T04:50:35.233 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "10", 2023-09-13T04:50:35.233 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.234 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.234 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-09-13T04:50:35.234 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.234 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.234 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.235 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.235 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.235 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.235 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.235 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.236 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.236 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.236 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.236 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.236 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.237 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.237 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.237 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.237 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.237 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.238 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.238 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.238 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.238 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.238 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.239 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.239 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.239 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.239 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.240 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.240 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.240 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.240 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.240 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.241 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.241 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.241 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.241 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.242 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.242 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.242 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.242 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.242 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.243 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.243 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.243 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.243 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.243 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.244 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.244 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.244 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.244 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.244 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.245 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.245 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.245 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.245 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.245 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.246 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.246 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.246 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.246 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.247 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.247 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.247 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.247 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.247 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.248 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.248 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.248 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.248 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.248 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.249 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.249 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "localpool", 2023-09-13T04:50:35.249 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.250 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.250 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.250 INFO:teuthology.orchestra.run.smithi002.stdout: "failure_domain": { 2023-09-13T04:50:35.250 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "failure_domain", 2023-09-13T04:50:35.250 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.251 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.251 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.251 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "host", 2023-09-13T04:50:35.251 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.252 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.252 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.252 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "failure domain for any created local pool", 2023-09-13T04:50:35.252 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-09-13T04:50:35.252 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.253 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.253 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.253 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.253 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.254 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.254 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.254 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.254 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.254 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.255 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.255 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.255 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.255 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.255 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.256 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.256 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.256 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.256 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.256 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.257 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.257 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.257 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.257 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.258 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.258 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.258 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.258 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.258 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.259 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.259 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.259 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.259 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.259 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.260 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.260 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.260 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.260 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.261 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.261 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.261 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.261 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.261 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.262 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.262 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.262 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.262 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.262 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.263 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.263 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.263 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.263 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.263 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.264 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.264 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.264 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.264 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.265 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.265 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.265 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.265 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.265 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.266 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.266 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.266 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.266 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.266 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.267 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.267 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": { 2023-09-13T04:50:35.267 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_size", 2023-09-13T04:50:35.267 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.268 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.268 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.268 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.268 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.268 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.269 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.269 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default min_size for any created local pool", 2023-09-13T04:50:35.269 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-09-13T04:50:35.269 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.269 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.270 INFO:teuthology.orchestra.run.smithi002.stdout: "num_rep": { 2023-09-13T04:50:35.270 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "num_rep", 2023-09-13T04:50:35.270 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.271 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.271 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.271 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3", 2023-09-13T04:50:35.271 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.272 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.272 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.272 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default replica count for any created local pool", 2023-09-13T04:50:35.272 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.272 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.273 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.273 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": { 2023-09-13T04:50:35.273 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_num", 2023-09-13T04:50:35.273 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.274 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.274 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.274 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "128", 2023-09-13T04:50:35.274 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.274 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.275 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.275 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default pg_num for any created local pool", 2023-09-13T04:50:35.275 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.275 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.275 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.276 INFO:teuthology.orchestra.run.smithi002.stdout: "prefix": { 2023-09-13T04:50:35.276 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prefix", 2023-09-13T04:50:35.276 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.276 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.277 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.277 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.277 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.277 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.277 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.278 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name prefix for any created local pool", 2023-09-13T04:50:35.278 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.278 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.278 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.279 INFO:teuthology.orchestra.run.smithi002.stdout: "subtree": { 2023-09-13T04:50:35.279 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "subtree", 2023-09-13T04:50:35.279 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.279 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.279 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.280 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "rack", 2023-09-13T04:50:35.280 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.280 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.280 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.281 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "CRUSH level for which to create a local pool", 2023-09-13T04:50:35.281 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-09-13T04:50:35.281 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.281 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.281 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.282 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.282 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mds_autoscaler", 2023-09-13T04:50:35.282 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.283 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.283 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.283 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.283 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.283 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.283 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.284 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.284 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.284 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.284 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.284 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.285 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.285 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.285 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.285 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.285 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.286 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.286 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.286 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.286 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.286 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.287 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.287 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.287 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.287 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.288 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.288 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.288 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.288 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.288 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.289 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.289 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.289 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.289 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.289 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.290 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.290 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.290 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.290 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.291 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.291 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.291 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.291 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.291 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.292 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.292 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.292 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.292 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.292 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.293 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.293 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.293 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.293 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.293 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.293 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.294 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.294 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.294 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.294 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.295 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.295 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.295 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.295 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.295 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.296 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.296 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.296 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.296 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.296 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.297 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.297 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirroring", 2023-09-13T04:50:35.297 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.298 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.298 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.298 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.298 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.299 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.299 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.299 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.299 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.299 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.300 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.300 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.300 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.300 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.300 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.301 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.301 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.301 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.301 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.301 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.302 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.302 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.302 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.302 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.302 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.303 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.303 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.303 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.303 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.303 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.304 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.304 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.304 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.304 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.304 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.305 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.305 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.305 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.305 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.306 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.306 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.306 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.306 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.306 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.307 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.307 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.307 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.307 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.307 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.308 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.308 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.308 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.308 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.308 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.309 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.309 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.309 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.309 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.310 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.310 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.310 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.310 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.310 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.311 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.311 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.311 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.311 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.311 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.312 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.312 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.312 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.312 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "nfs", 2023-09-13T04:50:35.313 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.313 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.313 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.313 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.313 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.314 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.314 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.314 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.314 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.314 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.315 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.315 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.315 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.315 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.315 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.316 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.316 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.316 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.316 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.316 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.317 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.317 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.317 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.317 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.318 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.318 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.318 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.318 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.319 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.319 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.319 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.319 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.319 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.320 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.320 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.320 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.321 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.321 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.321 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.321 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.321 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.322 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.322 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.322 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.322 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.322 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.323 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.323 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.323 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.323 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.323 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.324 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.324 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.324 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.324 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.324 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.325 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.325 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.325 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.325 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.326 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.326 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.326 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.326 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.326 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.327 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.327 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.327 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.327 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.327 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.328 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.328 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2023-09-13T04:50:35.328 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.329 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.329 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.329 INFO:teuthology.orchestra.run.smithi002.stdout: "fail_fs": { 2023-09-13T04:50:35.329 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "fail_fs", 2023-09-13T04:50:35.329 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.330 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.330 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.330 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.330 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.330 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.331 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.331 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-09-13T04:50:35.331 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.331 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.331 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.332 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.332 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.332 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.332 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.333 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.333 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.333 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.333 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.333 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.333 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.334 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.334 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.334 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.334 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.335 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.335 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.335 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.335 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.335 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.336 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.336 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.336 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.336 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.337 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.337 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.337 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.337 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.337 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.338 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.338 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.338 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.338 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.338 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.339 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.339 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.339 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.339 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.340 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.340 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.340 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.340 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.340 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.341 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.341 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.341 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.341 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.341 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.342 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.342 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.342 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.342 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.342 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.343 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.343 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.343 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.343 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.344 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.344 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.344 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.344 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.344 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.345 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.345 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.345 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.345 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.345 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.346 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator": { 2023-09-13T04:50:35.346 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2023-09-13T04:50:35.346 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.346 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.347 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.347 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.347 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.347 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.347 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.348 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm", 2023-09-13T04:50:35.348 INFO:teuthology.orchestra.run.smithi002.stdout: "rook", 2023-09-13T04:50:35.348 INFO:teuthology.orchestra.run.smithi002.stdout: "test_orchestrator" 2023-09-13T04:50:35.348 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.348 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Orchestrator backend", 2023-09-13T04:50:35.349 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.349 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.349 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.349 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.349 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.350 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_perf_query", 2023-09-13T04:50:35.350 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.350 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.351 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.351 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.351 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.351 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.351 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.352 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.352 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.352 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.352 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.352 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.353 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.353 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.353 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.353 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.353 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.354 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.354 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.354 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.354 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.354 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.355 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.355 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.355 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.355 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.356 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.356 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.356 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.356 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.357 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.357 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.357 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.357 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.357 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.358 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.358 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.358 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.358 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.359 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.359 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.359 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.359 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.359 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.360 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.360 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.360 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.360 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.360 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.361 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.361 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.361 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.361 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.361 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.362 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.362 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.362 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.362 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.363 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.363 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.363 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.363 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.363 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.364 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.364 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.364 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.364 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.364 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.364 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.365 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.365 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.365 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_support", 2023-09-13T04:50:35.366 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.366 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.366 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.366 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.366 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.367 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.367 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.367 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.367 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.367 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.368 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.368 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.368 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.368 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.368 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.369 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.369 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.369 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.369 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.369 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.370 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.370 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.370 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.370 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.371 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.371 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.371 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.371 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.371 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.371 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.372 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.372 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.372 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.372 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.372 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.373 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.373 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.373 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.373 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.374 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.374 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.374 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.374 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.374 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.375 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.375 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.375 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.375 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.375 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.376 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.376 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.376 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.376 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.376 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.376 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.377 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.377 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.377 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.377 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.378 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.378 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.378 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.378 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.379 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.379 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.379 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.379 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.379 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.379 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.380 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.380 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.380 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_autoscaler", 2023-09-13T04:50:35.381 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.381 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.381 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.381 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.381 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.382 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.382 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.382 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.382 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.382 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.383 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.383 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.383 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.383 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.383 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.384 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.384 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.384 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.384 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.384 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.384 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.385 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.385 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.385 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.385 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.386 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.386 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.386 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.386 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.386 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.387 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.387 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.387 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.387 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.387 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.388 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.388 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.388 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.388 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.389 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.389 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.389 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.389 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.389 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.390 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.390 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.390 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.390 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.390 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.390 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.391 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.391 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.391 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.391 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.392 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.392 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.392 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.392 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.393 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.393 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.393 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.393 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.393 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.394 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.394 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.394 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.394 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.394 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.395 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.395 INFO:teuthology.orchestra.run.smithi002.stdout: "noautoscale": { 2023-09-13T04:50:35.395 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "noautoscale", 2023-09-13T04:50:35.397 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.397 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.398 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.398 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.398 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.398 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.398 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.399 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "global autoscale flag", 2023-09-13T04:50:35.399 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-09-13T04:50:35.399 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.399 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.400 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-09-13T04:50:35.400 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-09-13T04:50:35.400 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:35.400 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.400 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.400 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-09-13T04:50:35.401 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.401 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.401 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.401 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.401 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.402 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.402 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.402 INFO:teuthology.orchestra.run.smithi002.stdout: "threshold": { 2023-09-13T04:50:35.402 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threshold", 2023-09-13T04:50:35.403 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-09-13T04:50:35.403 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.403 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.403 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2023-09-13T04:50:35.403 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1.0", 2023-09-13T04:50:35.404 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.404 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.404 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "scaling threshold", 2023-09-13T04:50:35.404 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-09-13T04:50:35.404 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.404 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.405 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.405 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.405 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "progress", 2023-09-13T04:50:35.406 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.406 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.406 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.406 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_pg_recovery_event": { 2023-09-13T04:50:35.406 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_pg_recovery_event", 2023-09-13T04:50:35.407 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.407 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.407 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.407 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.407 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.408 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.408 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.408 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow the module to show pg recovery progress", 2023-09-13T04:50:35.408 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.408 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.409 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.409 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2023-09-13T04:50:35.409 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2023-09-13T04:50:35.409 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.409 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.410 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.410 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:35.410 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.410 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.411 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.411 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.411 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.411 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.411 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.412 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.412 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.412 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.412 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.412 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.413 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.413 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.413 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.413 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.413 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.414 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.414 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.414 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.414 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.414 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.415 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.415 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.415 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.415 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.415 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.416 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.416 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.416 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.416 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.416 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.417 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.417 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.417 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.417 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.417 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.418 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.418 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.418 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.418 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.418 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.419 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.419 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.419 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.419 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.419 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.420 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.420 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.420 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.420 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.420 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.421 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.421 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.421 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.421 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.421 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.421 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.422 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.422 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.422 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.423 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.423 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.423 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.423 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.423 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.424 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.424 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.424 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.424 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.424 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.424 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.425 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.425 INFO:teuthology.orchestra.run.smithi002.stdout: "max_completed_events": { 2023-09-13T04:50:35.425 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_completed_events", 2023-09-13T04:50:35.425 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.426 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.426 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.426 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "50", 2023-09-13T04:50:35.426 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.426 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.427 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.427 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "number of past completed events to remember", 2023-09-13T04:50:35.427 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.427 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.427 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.428 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-09-13T04:50:35.428 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-09-13T04:50:35.428 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:35.428 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.429 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.429 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2023-09-13T04:50:35.429 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.429 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.429 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.429 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long the module is going to sleep", 2023-09-13T04:50:35.430 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.430 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.430 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.430 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.430 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.431 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus", 2023-09-13T04:50:35.431 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.431 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.432 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.432 INFO:teuthology.orchestra.run.smithi002.stdout: "cache": { 2023-09-13T04:50:35.432 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cache", 2023-09-13T04:50:35.432 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.432 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.433 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.433 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:35.433 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.433 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.433 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.433 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.434 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.434 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.434 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.434 INFO:teuthology.orchestra.run.smithi002.stdout: "exclude_perf_counters": { 2023-09-13T04:50:35.435 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "exclude_perf_counters", 2023-09-13T04:50:35.435 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.435 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.435 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.435 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:35.436 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.436 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.436 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.436 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-09-13T04:50:35.436 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-09-13T04:50:35.437 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.437 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.437 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.437 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.437 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.438 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.438 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.438 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.438 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.438 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.439 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.439 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.439 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.439 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.439 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.440 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.440 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.440 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.440 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.440 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.441 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.441 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.441 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.441 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.441 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.442 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.442 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.442 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.442 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.443 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.443 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.443 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.443 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.443 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.444 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.444 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.444 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.444 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.444 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.445 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.445 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.445 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.445 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.445 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.446 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.446 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.446 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.446 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.446 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.447 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.447 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.447 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.447 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.447 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.447 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.448 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.448 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.448 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.448 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.449 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.449 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.449 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.449 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.449 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.450 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.450 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.450 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.450 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.450 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools": { 2023-09-13T04:50:35.451 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools", 2023-09-13T04:50:35.451 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.451 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.451 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.451 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.452 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.452 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.452 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.452 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.452 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.453 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.453 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.453 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools_refresh_interval": { 2023-09-13T04:50:35.453 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-09-13T04:50:35.453 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.454 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.454 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.454 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "300", 2023-09-13T04:50:35.454 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.454 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.455 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.455 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.455 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.455 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.455 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.456 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_interval": { 2023-09-13T04:50:35.456 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_interval", 2023-09-13T04:50:35.456 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-09-13T04:50:35.456 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.457 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.457 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15.0", 2023-09-13T04:50:35.457 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.457 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.457 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.458 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.458 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.458 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.458 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.458 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2023-09-13T04:50:35.459 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2023-09-13T04:50:35.459 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.459 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.459 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.459 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2023-09-13T04:50:35.460 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.460 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.460 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.460 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-09-13T04:50:35.460 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.461 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.461 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.461 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2023-09-13T04:50:35.461 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2023-09-13T04:50:35.461 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.462 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.462 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.462 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "9283", 2023-09-13T04:50:35.462 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.462 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.463 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.463 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-09-13T04:50:35.463 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.463 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.463 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.464 INFO:teuthology.orchestra.run.smithi002.stdout: "stale_cache_strategy": { 2023-09-13T04:50:35.464 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stale_cache_strategy", 2023-09-13T04:50:35.464 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.464 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.464 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.465 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "log", 2023-09-13T04:50:35.465 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.465 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.465 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.465 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.466 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.466 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.466 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.466 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2023-09-13T04:50:35.467 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2023-09-13T04:50:35.467 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.467 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.467 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.467 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "default", 2023-09-13T04:50:35.467 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.468 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.468 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.468 INFO:teuthology.orchestra.run.smithi002.stdout: "default", 2023-09-13T04:50:35.468 INFO:teuthology.orchestra.run.smithi002.stdout: "error" 2023-09-13T04:50:35.468 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.469 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.469 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.469 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.469 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.469 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2023-09-13T04:50:35.470 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2023-09-13T04:50:35.470 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.470 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.470 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.470 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2023-09-13T04:50:35.471 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2023-09-13T04:50:35.471 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2023-09-13T04:50:35.471 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.471 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.471 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.472 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.472 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.473 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_support", 2023-09-13T04:50:35.473 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.473 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.473 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.474 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.474 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.474 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.474 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.474 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.474 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.475 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.475 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.475 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.475 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.475 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.476 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.476 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.476 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.476 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.476 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.477 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.477 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.477 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.477 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.478 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.478 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.478 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.478 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.479 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.479 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.479 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.479 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.479 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.479 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.480 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.480 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.480 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.480 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.481 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.481 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.481 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.481 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.481 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.482 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.482 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.482 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.482 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.482 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.483 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.483 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.483 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.483 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.483 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.483 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.484 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.484 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.484 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.484 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.485 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.485 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.485 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.485 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.485 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.486 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.486 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.486 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.486 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.486 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.487 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.487 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.487 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_snap_create": { 2023-09-13T04:50:35.487 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_snap_create", 2023-09-13T04:50:35.487 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.488 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.488 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.488 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-09-13T04:50:35.488 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.488 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.489 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.489 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.489 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.489 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.489 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.490 INFO:teuthology.orchestra.run.smithi002.stdout: "mirror_snapshot_schedule": { 2023-09-13T04:50:35.490 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirror_snapshot_schedule", 2023-09-13T04:50:35.490 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.490 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.491 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.491 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.491 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.491 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.491 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.491 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.492 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.492 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.492 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.492 INFO:teuthology.orchestra.run.smithi002.stdout: "trash_purge_schedule": { 2023-09-13T04:50:35.493 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "trash_purge_schedule", 2023-09-13T04:50:35.493 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.493 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.493 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.493 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.493 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.494 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.494 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.494 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.494 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.494 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.495 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.496 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "restful", 2023-09-13T04:50:35.496 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.496 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.496 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.496 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_auth": { 2023-09-13T04:50:35.496 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_auth", 2023-09-13T04:50:35.497 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.497 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.497 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.497 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:35.497 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.498 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.498 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.498 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.498 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.498 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.499 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.499 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2023-09-13T04:50:35.499 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2023-09-13T04:50:35.499 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.500 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.500 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.500 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.500 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.500 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.500 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.501 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.501 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.501 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.501 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.502 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.502 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.502 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.502 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.502 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.503 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.503 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.503 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.503 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.503 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.503 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.504 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.504 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.504 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.504 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.504 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.505 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.505 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.505 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.505 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.506 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.506 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.506 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.506 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.506 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.507 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.507 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.507 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.507 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.507 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.507 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.508 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.508 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.508 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.508 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.509 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.509 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.509 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.509 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.509 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.509 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.510 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.510 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.510 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.510 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.510 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.511 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.511 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.511 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.511 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.511 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.512 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.512 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.512 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.512 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.513 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.513 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.513 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.513 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.513 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.513 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.514 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.514 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.514 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.514 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.515 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.515 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2023-09-13T04:50:35.515 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2023-09-13T04:50:35.516 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.516 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.516 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.516 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.516 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.517 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.517 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.517 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.517 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.517 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.518 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.518 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2023-09-13T04:50:35.518 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2023-09-13T04:50:35.518 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.519 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.519 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.519 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.519 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.519 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.519 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.520 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.520 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.520 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.520 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.520 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.521 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.521 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rgw", 2023-09-13T04:50:35.521 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.522 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.522 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.522 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.522 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.522 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.523 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.523 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.523 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.523 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.523 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.523 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.524 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.524 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.524 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.524 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.524 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.525 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.525 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.525 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.525 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.525 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.526 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.526 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.526 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.526 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.527 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.527 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.527 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.527 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.527 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.528 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.528 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.528 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.528 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.528 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.529 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.529 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.529 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.529 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.529 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.530 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.530 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.530 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.530 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.530 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.531 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.531 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.531 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.531 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.531 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.532 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.532 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.532 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.532 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.532 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.533 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.533 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.533 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.533 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.533 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.534 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.534 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.534 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.534 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.534 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.535 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.535 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.535 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.535 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.535 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.536 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rook", 2023-09-13T04:50:35.536 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.536 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.536 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.537 INFO:teuthology.orchestra.run.smithi002.stdout: "drive_group_interval": { 2023-09-13T04:50:35.537 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "drive_group_interval", 2023-09-13T04:50:35.537 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-09-13T04:50:35.537 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.537 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.538 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "300.0", 2023-09-13T04:50:35.538 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.538 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.538 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.538 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-09-13T04:50:35.539 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.539 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.539 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.539 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.539 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.540 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.540 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.540 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.540 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.541 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.541 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.541 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.541 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.541 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.541 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.542 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.542 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.542 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.542 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.543 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.543 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.543 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.543 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.543 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.544 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.544 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.544 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.544 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.544 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.545 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.545 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.545 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.545 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.545 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.546 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.546 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.546 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.546 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.547 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.547 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.547 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.547 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.547 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.548 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.548 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.548 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.548 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.548 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.549 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.549 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.549 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.549 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.549 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.549 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.550 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.550 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.550 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.550 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.551 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.551 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.551 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.551 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.551 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.552 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.552 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.552 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.552 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.552 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.553 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.553 INFO:teuthology.orchestra.run.smithi002.stdout: "storage_class": { 2023-09-13T04:50:35.553 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "storage_class", 2023-09-13T04:50:35.553 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.554 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.554 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.554 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "local", 2023-09-13T04:50:35.554 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.554 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-09-13T04:50:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.556 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.556 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.556 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "selftest", 2023-09-13T04:50:35.556 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.558 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.558 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.558 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.558 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.558 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.568 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.568 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.568 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.568 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: "roption1": { 2023-09-13T04:50:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption1", 2023-09-13T04:50:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.573 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.573 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.573 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.573 INFO:teuthology.orchestra.run.smithi002.stdout: "roption2": { 2023-09-13T04:50:35.574 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption2", 2023-09-13T04:50:35.574 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.574 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.574 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.574 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "xyz", 2023-09-13T04:50:35.575 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.575 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.575 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.575 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.575 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.575 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption1": { 2023-09-13T04:50:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption1", 2023-09-13T04:50:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.577 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.577 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.577 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.577 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.577 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.578 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.578 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.578 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.578 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.578 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.579 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption2": { 2023-09-13T04:50:35.579 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption2", 2023-09-13T04:50:35.579 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.579 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.580 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.580 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.580 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.580 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.580 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.581 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.581 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.581 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.581 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.582 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption3": { 2023-09-13T04:50:35.582 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption3", 2023-09-13T04:50:35.582 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-09-13T04:50:35.582 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.582 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.583 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.583 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.583 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.583 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.583 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.584 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.584 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.584 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.584 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption4": { 2023-09-13T04:50:35.585 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption4", 2023-09-13T04:50:35.585 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.585 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.585 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.585 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.585 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.586 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.586 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.586 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.586 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.586 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.587 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.587 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption5": { 2023-09-13T04:50:35.587 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption5", 2023-09-13T04:50:35.587 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.588 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.588 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.588 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.588 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.588 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.589 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.589 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.589 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.589 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.589 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.590 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption6": { 2023-09-13T04:50:35.590 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption6", 2023-09-13T04:50:35.590 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.590 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.591 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.591 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:35.591 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.591 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.591 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.592 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.592 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.592 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.592 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.592 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption7": { 2023-09-13T04:50:35.593 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption7", 2023-09-13T04:50:35.593 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.593 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.593 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.593 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.594 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2023-09-13T04:50:35.594 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "42", 2023-09-13T04:50:35.594 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.594 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.594 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.595 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.595 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.595 INFO:teuthology.orchestra.run.smithi002.stdout: "testkey": { 2023-09-13T04:50:35.595 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testkey", 2023-09-13T04:50:35.595 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.596 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.596 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.596 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.596 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.596 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.597 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.597 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.597 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.597 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.597 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.598 INFO:teuthology.orchestra.run.smithi002.stdout: "testlkey": { 2023-09-13T04:50:35.598 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testlkey", 2023-09-13T04:50:35.598 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.598 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.599 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.599 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.599 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.599 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.599 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.599 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.600 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.600 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.600 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.600 INFO:teuthology.orchestra.run.smithi002.stdout: "testnewline": { 2023-09-13T04:50:35.601 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testnewline", 2023-09-13T04:50:35.601 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.601 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.601 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.601 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.601 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.602 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.602 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.602 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.602 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.602 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.603 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.603 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.603 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.604 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snap_schedule", 2023-09-13T04:50:35.604 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.604 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.605 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.605 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_m_granularity": { 2023-09-13T04:50:35.605 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_m_granularity", 2023-09-13T04:50:35.605 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.605 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.606 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.606 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.606 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.606 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.606 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.607 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow minute scheduled snapshots", 2023-09-13T04:50:35.607 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.607 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.607 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.608 INFO:teuthology.orchestra.run.smithi002.stdout: "dump_on_update": { 2023-09-13T04:50:35.608 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dump_on_update", 2023-09-13T04:50:35.608 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.608 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.608 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.608 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.609 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.609 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.609 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.609 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "dump database to debug log on update", 2023-09-13T04:50:35.609 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.610 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.610 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.610 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.610 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.611 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.611 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.611 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.611 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.611 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.612 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.612 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.612 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.612 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.612 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.613 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.613 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.613 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.613 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.613 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.613 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.614 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.614 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.614 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.614 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.615 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.615 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.615 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.615 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.615 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.616 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.616 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.616 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.616 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.616 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.617 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.617 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.617 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.617 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.617 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.618 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.618 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.618 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.618 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.618 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.619 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.619 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.619 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.619 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.619 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.620 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.620 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.620 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.620 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.620 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.620 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.621 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.621 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.621 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.621 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.622 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.622 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.622 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.622 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.622 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.622 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.623 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.623 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.623 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.623 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.623 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.624 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stats", 2023-09-13T04:50:35.624 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.624 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.625 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.625 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.625 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.625 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.625 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.626 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.626 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.626 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.626 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.626 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.626 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.627 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.627 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.627 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.627 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.627 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.628 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.628 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.628 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.628 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.629 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.629 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.629 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.629 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.630 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.630 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.630 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.630 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.630 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.631 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.631 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.631 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.631 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.631 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.632 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.632 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.632 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.632 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.633 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.633 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.633 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.633 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.633 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.633 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.634 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.634 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.634 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.634 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.634 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.635 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.635 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.635 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.635 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.635 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.636 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.636 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.636 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.636 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.637 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.637 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.637 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.637 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.637 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.638 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.638 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.638 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.638 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.638 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.639 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.639 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "status", 2023-09-13T04:50:35.639 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.639 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.640 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.640 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.640 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.640 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.640 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.641 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.641 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.641 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.641 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.641 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.642 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.642 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.642 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.642 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.642 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.643 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.643 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.643 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.643 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.643 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.643 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.644 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.644 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.644 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.644 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.645 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.645 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.645 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.645 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.645 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.646 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.646 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.646 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.646 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.647 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.647 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.647 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.647 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.647 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.648 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.648 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.648 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.648 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.648 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.648 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.649 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.649 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.649 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.649 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.649 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.650 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.650 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.650 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.650 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.651 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.651 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.651 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.651 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.651 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.651 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.652 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.652 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.652 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.652 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.652 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.653 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.653 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.653 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.653 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.654 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telegraf", 2023-09-13T04:50:35.654 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.654 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.654 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.655 INFO:teuthology.orchestra.run.smithi002.stdout: "address": { 2023-09-13T04:50:35.655 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "address", 2023-09-13T04:50:35.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.655 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.655 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.655 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-09-13T04:50:35.656 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.656 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.656 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.656 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.656 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.657 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.657 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.657 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-09-13T04:50:35.657 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-09-13T04:50:35.658 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:35.658 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.658 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.658 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15", 2023-09-13T04:50:35.658 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.658 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.659 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.659 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.659 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.659 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.659 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.660 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.660 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.660 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.660 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.661 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.661 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.661 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.661 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.661 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.662 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.662 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.662 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.662 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.662 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.663 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.663 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.663 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.663 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.663 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.664 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.664 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.664 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.664 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.664 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.665 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.665 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.665 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.665 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.665 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.666 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.666 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.666 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.666 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.667 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.667 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.667 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.667 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.667 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.668 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.668 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.668 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.668 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.668 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.669 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.669 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.669 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.669 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.669 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.669 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.670 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.670 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.670 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.670 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.671 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.671 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.671 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.671 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.671 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.672 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.672 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.672 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.672 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.672 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.672 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.673 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.673 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.673 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.673 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.674 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telemetry", 2023-09-13T04:50:35.674 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.674 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.674 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.675 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_basic": { 2023-09-13T04:50:35.675 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_basic", 2023-09-13T04:50:35.675 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.675 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.675 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.675 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:35.676 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.676 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.676 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.676 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share basic cluster information (size, version)", 2023-09-13T04:50:35.676 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.677 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.677 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.677 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_crash": { 2023-09-13T04:50:35.677 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_crash", 2023-09-13T04:50:35.678 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.678 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.678 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.678 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:35.678 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.679 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.679 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.679 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-09-13T04:50:35.679 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.679 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.679 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.680 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_device": { 2023-09-13T04:50:35.680 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_device", 2023-09-13T04:50:35.680 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.680 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.681 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.681 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:35.681 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.681 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.681 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.682 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-09-13T04:50:35.682 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.682 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.682 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.683 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_ident": { 2023-09-13T04:50:35.683 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_ident", 2023-09-13T04:50:35.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.683 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.683 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.684 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.684 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.684 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.684 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.684 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-09-13T04:50:35.684 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.685 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.685 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.685 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_perf": { 2023-09-13T04:50:35.685 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_perf", 2023-09-13T04:50:35.686 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.686 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.686 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.686 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.686 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.687 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.687 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.687 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share various performance metrics of a cluster", 2023-09-13T04:50:35.687 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.687 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.688 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.688 INFO:teuthology.orchestra.run.smithi002.stdout: "contact": { 2023-09-13T04:50:35.688 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "contact", 2023-09-13T04:50:35.688 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.689 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.689 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.689 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.689 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.689 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.689 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.690 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.690 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.690 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.690 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.691 INFO:teuthology.orchestra.run.smithi002.stdout: "description": { 2023-09-13T04:50:35.691 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "description", 2023-09-13T04:50:35.691 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.691 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.691 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.692 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.692 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.692 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.692 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.692 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.692 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.693 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.693 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.693 INFO:teuthology.orchestra.run.smithi002.stdout: "device_url": { 2023-09-13T04:50:35.693 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_url", 2023-09-13T04:50:35.694 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.694 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.694 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.694 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-09-13T04:50:35.694 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.695 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.695 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.695 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.695 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.695 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.695 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.696 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2023-09-13T04:50:35.696 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2023-09-13T04:50:35.696 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.696 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.697 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.697 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.697 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.697 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.697 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.698 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.698 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.698 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.698 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.698 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-09-13T04:50:35.699 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-09-13T04:50:35.699 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.699 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.699 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.699 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "24", 2023-09-13T04:50:35.700 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "8", 2023-09-13T04:50:35.700 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.700 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.700 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.700 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.701 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.701 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.701 INFO:teuthology.orchestra.run.smithi002.stdout: "last_opt_revision": { 2023-09-13T04:50:35.701 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "last_opt_revision", 2023-09-13T04:50:35.702 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.702 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.702 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.702 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1", 2023-09-13T04:50:35.702 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.703 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.703 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.703 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.703 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.703 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.704 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.704 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard": { 2023-09-13T04:50:35.704 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard", 2023-09-13T04:50:35.704 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.704 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.705 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.705 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.705 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.705 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.705 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.706 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.706 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.706 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.706 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.707 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard_description": { 2023-09-13T04:50:35.707 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard_description", 2023-09-13T04:50:35.707 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.707 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.707 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.707 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.708 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.708 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.708 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.708 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.708 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.709 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.709 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.709 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.709 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.710 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.710 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.710 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.710 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.710 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.711 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.711 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.711 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.711 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.711 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.712 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.712 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.712 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.712 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.712 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.713 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.713 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.713 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.713 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.713 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.714 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.714 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.714 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.714 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.714 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.715 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.715 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.715 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.715 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.715 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.716 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.716 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.716 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.716 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.716 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.717 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.717 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.717 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.717 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.717 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.717 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.718 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.718 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.718 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.718 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.718 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.719 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.719 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.719 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.719 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.720 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.720 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.720 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.720 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.720 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.720 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.721 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.721 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.721 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.721 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.721 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.722 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.722 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.722 INFO:teuthology.orchestra.run.smithi002.stdout: "organization": { 2023-09-13T04:50:35.722 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "organization", 2023-09-13T04:50:35.723 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.723 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.723 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.723 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.723 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.723 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.724 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.724 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.724 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.724 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.724 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.725 INFO:teuthology.orchestra.run.smithi002.stdout: "proxy": { 2023-09-13T04:50:35.725 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "proxy", 2023-09-13T04:50:35.725 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.725 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.725 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.726 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.726 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.726 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.726 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.726 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.727 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.727 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.727 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.727 INFO:teuthology.orchestra.run.smithi002.stdout: "url": { 2023-09-13T04:50:35.728 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url", 2023-09-13T04:50:35.728 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.728 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.728 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.728 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-09-13T04:50:35.728 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.729 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.729 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.729 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.729 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.729 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.730 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.730 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.730 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.731 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "test_orchestrator", 2023-09-13T04:50:35.731 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.731 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.731 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.731 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.732 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.732 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.732 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.732 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.732 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.732 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.733 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.733 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.733 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.733 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.733 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.734 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.734 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.734 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.734 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.734 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.735 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.735 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.735 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.735 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.736 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.736 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.736 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.736 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.736 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.736 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.737 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.737 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.737 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.737 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.737 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.738 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.738 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.738 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.738 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.739 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.739 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.739 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.739 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.739 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.739 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.740 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.740 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.740 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.740 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.740 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.741 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.741 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.741 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.741 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.741 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.742 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.742 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.742 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.742 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.743 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.743 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.743 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.743 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.743 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.744 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.744 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.744 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.744 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.744 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.744 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.745 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.745 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "volumes", 2023-09-13T04:50:35.745 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.746 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.746 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.746 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.746 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.746 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.746 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.747 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.747 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.747 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.747 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.747 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.747 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.748 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.748 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.748 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.748 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.748 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.749 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.749 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.749 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.749 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.749 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.750 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.750 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.750 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.750 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.750 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.751 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.751 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.751 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.751 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.751 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.751 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.752 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.752 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.752 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.752 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.753 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.753 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.753 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.753 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.753 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.753 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.754 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.754 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.754 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.754 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.754 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.755 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.755 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.755 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.755 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.755 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.755 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.756 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.756 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.756 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.756 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.756 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.757 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.757 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.757 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.757 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.757 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.758 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.758 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.758 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.758 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.759 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_clones": { 2023-09-13T04:50:35.759 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_clones", 2023-09-13T04:50:35.759 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.759 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.759 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.759 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4", 2023-09-13T04:50:35.760 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.760 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.760 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.760 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Number of asynchronous cloner threads", 2023-09-13T04:50:35.760 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.761 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.761 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.761 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_delay": { 2023-09-13T04:50:35.761 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_delay", 2023-09-13T04:50:35.762 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.762 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.762 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.762 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-09-13T04:50:35.762 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.763 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.763 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.763 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-09-13T04:50:35.763 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.763 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.764 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.764 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.764 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.764 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix", 2023-09-13T04:50:35.765 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.765 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.765 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.765 INFO:teuthology.orchestra.run.smithi002.stdout: "discovery_interval": { 2023-09-13T04:50:35.765 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "discovery_interval", 2023-09-13T04:50:35.766 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-09-13T04:50:35.766 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.766 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.766 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "100", 2023-09-13T04:50:35.766 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.767 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.767 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.767 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.767 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.767 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.767 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.768 INFO:teuthology.orchestra.run.smithi002.stdout: "identifier": { 2023-09-13T04:50:35.768 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "identifier", 2023-09-13T04:50:35.768 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.768 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.769 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.769 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.769 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.769 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.769 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.770 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.770 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.770 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.770 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.771 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-09-13T04:50:35.771 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-09-13T04:50:35.771 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:35.771 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.771 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.771 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-09-13T04:50:35.772 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.772 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.772 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.772 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.772 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.773 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.773 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.773 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.773 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.773 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.774 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.774 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.774 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.774 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.775 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.775 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.775 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.775 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.775 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.775 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.776 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.776 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.776 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.776 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.776 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.777 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.777 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.777 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.777 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.778 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.778 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.778 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.778 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.778 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.779 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.779 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.779 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.779 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.779 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.779 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.780 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.780 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.780 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.780 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.781 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.781 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.781 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.781 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.781 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.782 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.782 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.782 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.782 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.782 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.783 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.783 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.783 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.783 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.783 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.784 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.784 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.784 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.784 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.784 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.785 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.785 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.785 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.785 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.785 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.785 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.786 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.786 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.786 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.786 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_host": { 2023-09-13T04:50:35.787 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_host", 2023-09-13T04:50:35.787 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.787 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.787 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.787 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.788 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.788 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.788 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.788 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.788 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.788 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.789 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.789 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_port": { 2023-09-13T04:50:35.789 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_port", 2023-09-13T04:50:35.789 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.790 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.790 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.790 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10051", 2023-09-13T04:50:35.790 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.790 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.791 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.791 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.791 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.791 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.791 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.792 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_sender": { 2023-09-13T04:50:35.792 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_sender", 2023-09-13T04:50:35.792 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.792 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.792 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.793 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-09-13T04:50:35.793 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.793 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.793 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.793 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.794 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.794 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.794 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.794 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.794 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.795 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.795 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2023-09-13T04:50:35.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.795 INFO:teuthology.orchestra.run.smithi002.stdout: "gid": 25525, 2023-09-13T04:50:35.796 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "z", 2023-09-13T04:50:35.796 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_features": 4540138322906710015, 2023-09-13T04:50:35.796 INFO:teuthology.orchestra.run.smithi002.stdout: "available_modules": [ 2023-09-13T04:50:35.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.796 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "alerts", 2023-09-13T04:50:35.797 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.797 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.797 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.797 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-09-13T04:50:35.797 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-09-13T04:50:35.797 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:35.798 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.798 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.798 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-09-13T04:50:35.798 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.798 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.799 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.799 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How frequently to reexamine health status", 2023-09-13T04:50:35.799 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.799 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.799 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.800 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.800 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.800 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.800 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.800 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.801 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.801 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.801 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.801 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.801 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.802 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.802 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.802 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.802 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.802 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.803 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.803 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.803 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.803 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.803 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.804 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.804 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.804 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.804 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.804 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.804 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.805 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.805 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.805 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.805 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.805 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.806 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.806 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.806 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.809 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.809 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.809 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.809 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.810 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.810 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.810 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.810 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.810 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.811 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.811 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.811 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.811 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.811 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.812 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.812 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.812 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.812 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.812 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.813 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.813 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.813 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.813 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.814 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.814 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.814 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.814 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.814 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.814 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.815 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.815 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.815 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.815 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_destination": { 2023-09-13T04:50:35.816 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_destination", 2023-09-13T04:50:35.816 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.816 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.816 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.816 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.817 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.817 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.817 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.817 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email address to send alerts to", 2023-09-13T04:50:35.817 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.817 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.818 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.818 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_from_name": { 2023-09-13T04:50:35.818 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_from_name", 2023-09-13T04:50:35.818 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.819 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.819 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.819 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "Ceph", 2023-09-13T04:50:35.819 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.819 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.820 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.820 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email From: name", 2023-09-13T04:50:35.820 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.820 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.820 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.821 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_host": { 2023-09-13T04:50:35.821 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_host", 2023-09-13T04:50:35.821 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.821 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.821 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.822 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.822 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.822 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.822 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.822 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP server", 2023-09-13T04:50:35.823 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.823 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.823 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.823 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_password": { 2023-09-13T04:50:35.823 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_password", 2023-09-13T04:50:35.824 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.824 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.824 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.824 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.824 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.825 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.825 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.825 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Password to authenticate with", 2023-09-13T04:50:35.825 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.825 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.826 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.826 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_port": { 2023-09-13T04:50:35.826 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_port", 2023-09-13T04:50:35.826 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.826 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.827 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.827 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "465", 2023-09-13T04:50:35.827 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.827 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.827 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.828 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP port", 2023-09-13T04:50:35.828 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.828 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.828 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.829 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_sender": { 2023-09-13T04:50:35.829 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_sender", 2023-09-13T04:50:35.829 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.829 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.829 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.829 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.830 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.830 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.830 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.830 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP envelope sender", 2023-09-13T04:50:35.830 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.831 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.831 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.831 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_ssl": { 2023-09-13T04:50:35.831 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_ssl", 2023-09-13T04:50:35.831 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.832 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.832 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.832 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:35.832 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.832 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.833 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.833 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use SSL to connect to SMTP server", 2023-09-13T04:50:35.833 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.833 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.833 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.834 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_user": { 2023-09-13T04:50:35.834 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_user", 2023-09-13T04:50:35.834 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.834 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.834 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.835 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.835 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.835 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.835 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.835 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "User to authenticate as", 2023-09-13T04:50:35.836 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.836 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.836 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.836 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.836 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.837 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "balancer", 2023-09-13T04:50:35.837 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.837 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.837 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.838 INFO:teuthology.orchestra.run.smithi002.stdout: "active": { 2023-09-13T04:50:35.838 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "active", 2023-09-13T04:50:35.838 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.838 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.838 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.838 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:35.839 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.839 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.839 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.839 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically balance PGs across cluster", 2023-09-13T04:50:35.839 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.840 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.840 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.840 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_time": { 2023-09-13T04:50:35.840 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_time", 2023-09-13T04:50:35.841 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.841 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.841 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.841 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0000", 2023-09-13T04:50:35.841 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.841 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.842 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.842 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "beginning time of day to automatically balance", 2023-09-13T04:50:35.842 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-13T04:50:35.842 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.842 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.843 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_weekday": { 2023-09-13T04:50:35.843 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_weekday", 2023-09-13T04:50:35.843 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-09-13T04:50:35.843 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.844 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.844 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-09-13T04:50:35.844 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2023-09-13T04:50:35.844 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2023-09-13T04:50:35.844 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.844 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-09-13T04:50:35.845 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-13T04:50:35.845 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.845 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.845 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_max_iterations": { 2023-09-13T04:50:35.846 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_max_iterations", 2023-09-13T04:50:35.846 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-09-13T04:50:35.846 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.846 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.846 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "25", 2023-09-13T04:50:35.847 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2023-09-13T04:50:35.847 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "250", 2023-09-13T04:50:35.847 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.847 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-09-13T04:50:35.847 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.848 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.848 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.848 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_metrics": { 2023-09-13T04:50:35.848 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_metrics", 2023-09-13T04:50:35.849 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.849 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.849 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.849 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "pgs,objects,bytes", 2023-09-13T04:50:35.849 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.850 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.850 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.850 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-09-13T04:50:35.850 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-09-13T04:50:35.850 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.851 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.851 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_step": { 2023-09-13T04:50:35.851 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_step", 2023-09-13T04:50:35.851 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-09-13T04:50:35.852 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.852 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.852 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.5", 2023-09-13T04:50:35.852 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0.001", 2023-09-13T04:50:35.852 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "0.999", 2023-09-13T04:50:35.852 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.853 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "aggressiveness of optimization", 2023-09-13T04:50:35.853 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-09-13T04:50:35.853 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.853 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.854 INFO:teuthology.orchestra.run.smithi002.stdout: "end_time": { 2023-09-13T04:50:35.854 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_time", 2023-09-13T04:50:35.854 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.854 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.854 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.855 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2359", 2023-09-13T04:50:35.855 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.855 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.855 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.855 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ending time of day to automatically balance", 2023-09-13T04:50:35.856 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-13T04:50:35.856 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.856 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.856 INFO:teuthology.orchestra.run.smithi002.stdout: "end_weekday": { 2023-09-13T04:50:35.856 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_weekday", 2023-09-13T04:50:35.857 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-09-13T04:50:35.857 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.857 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.857 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-09-13T04:50:35.857 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2023-09-13T04:50:35.858 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2023-09-13T04:50:35.858 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.858 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-09-13T04:50:35.858 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-13T04:50:35.858 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.859 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.859 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.859 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.859 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.860 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.860 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.860 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.860 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.860 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.860 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.861 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.861 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.861 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.861 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.861 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.862 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.862 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.862 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.862 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.862 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.863 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.863 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.863 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.863 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.863 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.864 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.864 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.864 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.864 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.864 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.865 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.865 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.865 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.865 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.866 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.866 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.866 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.866 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.866 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.866 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.867 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.867 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.867 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.867 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.867 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.868 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.868 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.868 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.868 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.868 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.868 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.869 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.869 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.869 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.869 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.870 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.870 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.870 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.870 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.870 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.871 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.871 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.871 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.871 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.871 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.872 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.872 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.872 INFO:teuthology.orchestra.run.smithi002.stdout: "min_score": { 2023-09-13T04:50:35.872 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_score", 2023-09-13T04:50:35.873 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-09-13T04:50:35.873 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.873 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.873 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-09-13T04:50:35.873 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.873 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.874 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.874 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-09-13T04:50:35.874 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.874 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.874 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.875 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2023-09-13T04:50:35.875 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2023-09-13T04:50:35.875 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.875 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.876 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.876 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "upmap", 2023-09-13T04:50:35.876 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.876 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.876 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.876 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-compat", 2023-09-13T04:50:35.877 INFO:teuthology.orchestra.run.smithi002.stdout: "none", 2023-09-13T04:50:35.877 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap" 2023-09-13T04:50:35.877 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.877 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Balancer mode", 2023-09-13T04:50:35.877 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.878 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.878 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.878 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_ids": { 2023-09-13T04:50:35.878 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_ids", 2023-09-13T04:50:35.879 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.879 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.879 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.879 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.879 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.879 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.880 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.880 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-09-13T04:50:35.880 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.880 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.881 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.881 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-09-13T04:50:35.881 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-09-13T04:50:35.881 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:35.882 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.882 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.882 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-09-13T04:50:35.882 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.883 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.883 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.883 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-09-13T04:50:35.883 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.883 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.884 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.884 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_deviation": { 2023-09-13T04:50:35.884 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_deviation", 2023-09-13T04:50:35.884 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.885 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.885 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.885 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2023-09-13T04:50:35.885 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2023-09-13T04:50:35.885 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.885 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.886 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "deviation below which no optimization is attempted", 2023-09-13T04:50:35.886 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-09-13T04:50:35.886 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.886 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.887 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_optimizations": { 2023-09-13T04:50:35.887 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_optimizations", 2023-09-13T04:50:35.887 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-09-13T04:50:35.887 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.887 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.888 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-09-13T04:50:35.888 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.888 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.888 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.888 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-09-13T04:50:35.889 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.889 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.889 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.889 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.889 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:35.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:35.890 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm", 2023-09-13T04:50:35.890 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:35.890 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:35.890 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:35.891 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_down_multiplier": { 2023-09-13T04:50:35.891 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_down_multiplier", 2023-09-13T04:50:35.891 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-09-13T04:50:35.891 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.891 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.892 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2023-09-13T04:50:35.892 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.892 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.892 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.892 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-09-13T04:50:35.892 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.893 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.893 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.893 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_refresh_rate": { 2023-09-13T04:50:35.893 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_refresh_rate", 2023-09-13T04:50:35.894 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:35.894 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.894 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.894 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "20", 2023-09-13T04:50:35.894 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.895 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.895 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.895 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-09-13T04:50:35.895 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.895 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.896 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.896 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_starting_port": { 2023-09-13T04:50:35.896 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_starting_port", 2023-09-13T04:50:35.896 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:35.897 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.897 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.897 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4721", 2023-09-13T04:50:35.897 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.897 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.897 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.898 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-09-13T04:50:35.898 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.898 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.898 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.899 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_ptrace": { 2023-09-13T04:50:35.899 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_ptrace", 2023-09-13T04:50:35.899 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.899 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.899 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.900 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.900 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.900 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.900 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.900 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-09-13T04:50:35.901 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-09-13T04:50:35.901 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.901 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.901 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_interval": { 2023-09-13T04:50:35.902 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_interval", 2023-09-13T04:50:35.902 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:35.902 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.902 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.902 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-09-13T04:50:35.903 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.903 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.903 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.903 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to autotune daemon memory", 2023-09-13T04:50:35.903 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.903 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.904 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.904 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_memory_target_ratio": { 2023-09-13T04:50:35.904 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_memory_target_ratio", 2023-09-13T04:50:35.904 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-09-13T04:50:35.905 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.905 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.905 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.7", 2023-09-13T04:50:35.905 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.905 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.905 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.906 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-09-13T04:50:35.906 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.906 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.906 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.907 INFO:teuthology.orchestra.run.smithi002.stdout: "cgroups_split": { 2023-09-13T04:50:35.907 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cgroups_split", 2023-09-13T04:50:35.907 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.907 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.907 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.908 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:35.908 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.908 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.908 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.908 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-09-13T04:50:35.909 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.909 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.909 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.909 INFO:teuthology.orchestra.run.smithi002.stdout: "config_checks_enabled": { 2023-09-13T04:50:35.909 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_checks_enabled", 2023-09-13T04:50:35.910 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.910 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.910 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.910 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.910 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.911 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.911 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.911 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-09-13T04:50:35.911 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.911 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.912 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.912 INFO:teuthology.orchestra.run.smithi002.stdout: "config_dashboard": { 2023-09-13T04:50:35.912 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_dashboard", 2023-09-13T04:50:35.912 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.912 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.913 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.913 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:35.913 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.913 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.913 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.914 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-09-13T04:50:35.914 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.914 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.914 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.915 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_alertmanager": { 2023-09-13T04:50:35.915 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_alertmanager", 2023-09-13T04:50:35.915 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.915 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.915 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.916 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-09-13T04:50:35.916 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.916 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.916 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.916 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2023-09-13T04:50:35.917 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.917 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.917 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.917 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_base": { 2023-09-13T04:50:35.918 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_base", 2023-09-13T04:50:35.918 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.918 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.918 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.918 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph", 2023-09-13T04:50:35.919 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.919 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.919 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.919 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Container image name, without the tag", 2023-09-13T04:50:35.919 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.920 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.920 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.920 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_elasticsearch": { 2023-09-13T04:50:35.920 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_elasticsearch", 2023-09-13T04:50:35.921 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.921 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.921 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.921 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-09-13T04:50:35.921 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.921 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.922 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.922 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "elasticsearch container image", 2023-09-13T04:50:35.922 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.922 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.922 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.923 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_grafana": { 2023-09-13T04:50:35.923 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_grafana", 2023-09-13T04:50:35.923 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.923 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.924 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.924 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-09-13T04:50:35.924 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.924 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.924 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.925 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2023-09-13T04:50:35.925 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.925 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.925 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.925 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_haproxy": { 2023-09-13T04:50:35.926 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_haproxy", 2023-09-13T04:50:35.926 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.926 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.926 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.927 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-09-13T04:50:35.927 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.927 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.927 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.927 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "HAproxy container image", 2023-09-13T04:50:35.927 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.928 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.928 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.928 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_agent": { 2023-09-13T04:50:35.928 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_agent", 2023-09-13T04:50:35.929 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.929 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.929 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.929 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-09-13T04:50:35.929 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.930 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.930 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.930 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger agent container image", 2023-09-13T04:50:35.930 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.930 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.930 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.931 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_collector": { 2023-09-13T04:50:35.931 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_collector", 2023-09-13T04:50:35.931 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.931 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.932 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.932 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-09-13T04:50:35.932 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.932 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.932 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.933 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger collector container image", 2023-09-13T04:50:35.933 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.933 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.933 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.933 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_query": { 2023-09-13T04:50:35.934 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_query", 2023-09-13T04:50:35.934 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.934 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.934 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.934 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-09-13T04:50:35.935 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.935 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.935 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.935 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger query container image", 2023-09-13T04:50:35.935 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.936 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.936 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.936 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_keepalived": { 2023-09-13T04:50:35.936 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_keepalived", 2023-09-13T04:50:35.937 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.937 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.937 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.937 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-09-13T04:50:35.937 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.937 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.938 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.938 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Keepalived container image", 2023-09-13T04:50:35.938 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.938 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.938 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.939 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_loki": { 2023-09-13T04:50:35.939 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_loki", 2023-09-13T04:50:35.939 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.939 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.940 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.940 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-09-13T04:50:35.940 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.940 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.940 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.940 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Loki container image", 2023-09-13T04:50:35.941 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.941 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.941 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.941 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_node_exporter": { 2023-09-13T04:50:35.942 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_node_exporter", 2023-09-13T04:50:35.942 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.942 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.942 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.942 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-09-13T04:50:35.943 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.943 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.943 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.943 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2023-09-13T04:50:35.943 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.944 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.944 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.944 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_nvmeof": { 2023-09-13T04:50:35.944 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_nvmeof", 2023-09-13T04:50:35.944 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.945 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.945 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.945 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-09-13T04:50:35.945 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.946 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.946 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.946 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Nvme-of container image", 2023-09-13T04:50:35.946 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.946 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.947 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.947 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_prometheus": { 2023-09-13T04:50:35.947 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_prometheus", 2023-09-13T04:50:35.947 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.948 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.948 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.948 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-09-13T04:50:35.948 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.948 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.949 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.949 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2023-09-13T04:50:35.949 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.949 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.949 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.950 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_promtail": { 2023-09-13T04:50:35.950 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_promtail", 2023-09-13T04:50:35.950 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.950 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.951 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.951 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-09-13T04:50:35.951 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.951 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.951 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.952 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Promtail container image", 2023-09-13T04:50:35.952 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.952 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.952 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.953 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_snmp_gateway": { 2023-09-13T04:50:35.953 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_snmp_gateway", 2023-09-13T04:50:35.953 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.953 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.953 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.954 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-09-13T04:50:35.954 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.954 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.954 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.954 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SNMP Gateway container image", 2023-09-13T04:50:35.955 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.955 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.955 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.955 INFO:teuthology.orchestra.run.smithi002.stdout: "container_init": { 2023-09-13T04:50:35.956 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_init", 2023-09-13T04:50:35.956 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.956 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.956 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.956 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:35.957 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.957 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.957 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.957 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Run podman/docker with `--init`", 2023-09-13T04:50:35.957 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.957 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.958 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.958 INFO:teuthology.orchestra.run.smithi002.stdout: "daemon_cache_timeout": { 2023-09-13T04:50:35.958 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "daemon_cache_timeout", 2023-09-13T04:50:35.959 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:35.959 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.959 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.959 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-09-13T04:50:35.959 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.959 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.960 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.960 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-09-13T04:50:35.960 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.960 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.960 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.961 INFO:teuthology.orchestra.run.smithi002.stdout: "default_cephadm_command_timeout": { 2023-09-13T04:50:35.961 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_cephadm_command_timeout", 2023-09-13T04:50:35.961 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:35.961 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.962 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.962 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "900", 2023-09-13T04:50:35.962 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.962 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.962 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.963 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-09-13T04:50:35.963 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.963 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.963 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.963 INFO:teuthology.orchestra.run.smithi002.stdout: "default_registry": { 2023-09-13T04:50:35.964 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_registry", 2023-09-13T04:50:35.964 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.964 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.964 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.964 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io", 2023-09-13T04:50:35.965 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.965 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.965 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.965 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-09-13T04:50:35.965 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.966 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.966 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.966 INFO:teuthology.orchestra.run.smithi002.stdout: "device_cache_timeout": { 2023-09-13T04:50:35.966 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_cache_timeout", 2023-09-13T04:50:35.966 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:35.967 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.967 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.967 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1800", 2023-09-13T04:50:35.967 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.967 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.968 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.968 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache device inventory", 2023-09-13T04:50:35.968 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.968 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.968 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.969 INFO:teuthology.orchestra.run.smithi002.stdout: "device_enhanced_scan": { 2023-09-13T04:50:35.969 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_enhanced_scan", 2023-09-13T04:50:35.969 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.969 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.969 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.970 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.970 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.970 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.970 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.970 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use libstoragemgmt during device scans", 2023-09-13T04:50:35.971 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.971 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.971 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.971 INFO:teuthology.orchestra.run.smithi002.stdout: "facts_cache_timeout": { 2023-09-13T04:50:35.972 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "facts_cache_timeout", 2023-09-13T04:50:35.972 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:35.972 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.972 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.972 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-09-13T04:50:35.973 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.973 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.973 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.973 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache host facts data", 2023-09-13T04:50:35.973 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.974 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.974 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.974 INFO:teuthology.orchestra.run.smithi002.stdout: "host_check_interval": { 2023-09-13T04:50:35.974 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "host_check_interval", 2023-09-13T04:50:35.975 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:35.975 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.975 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.975 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-09-13T04:50:35.975 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.976 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.976 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.976 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to perform a host check", 2023-09-13T04:50:35.976 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.976 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.976 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.977 INFO:teuthology.orchestra.run.smithi002.stdout: "inventory_list_all": { 2023-09-13T04:50:35.977 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "inventory_list_all", 2023-09-13T04:50:35.977 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.977 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.978 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.978 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.978 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.978 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.978 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.979 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-09-13T04:50:35.979 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.979 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.979 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.980 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:35.980 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:35.980 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.980 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.981 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.981 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:35.981 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.981 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.981 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.982 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.982 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.982 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.982 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.982 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.983 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.983 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.983 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.983 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.983 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.983 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.984 INFO:teuthology.orchestra.run.smithi002.stdout: "log_refresh_metadata": { 2023-09-13T04:50:35.984 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_refresh_metadata", 2023-09-13T04:50:35.984 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.984 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.985 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.985 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.985 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.985 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.985 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.986 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-09-13T04:50:35.986 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.986 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.986 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.987 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:35.987 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:35.987 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.987 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.987 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.987 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:35.988 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.988 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.988 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.988 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-09-13T04:50:35.988 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.989 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.989 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.989 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:35.989 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:35.990 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.990 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.990 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.990 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:35.990 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.991 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.991 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:35.991 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:35.991 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:35.991 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:35.991 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:35.992 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:35.992 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:35.992 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:35.992 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.992 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.993 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.993 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.993 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:35.993 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:35.993 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.994 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.994 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:35.994 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.994 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.994 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.995 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.995 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:35.995 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.995 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.995 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.996 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf": { 2023-09-13T04:50:35.996 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-09-13T04:50:35.996 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:35.996 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.997 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.997 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:35.997 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:35.997 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:35.997 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:35.997 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-09-13T04:50:35.998 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:35.998 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:35.998 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:35.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:35.998 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-09-13T04:50:35.999 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-09-13T04:50:35.999 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:35.999 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:35.999 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:35.999 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "*", 2023-09-13T04:50:36.000 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.000 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.000 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.000 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-09-13T04:50:36.000 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.001 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.001 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.001 INFO:teuthology.orchestra.run.smithi002.stdout: "max_count_per_host": { 2023-09-13T04:50:36.001 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_count_per_host", 2023-09-13T04:50:36.001 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.002 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.002 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.002 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-09-13T04:50:36.002 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.002 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.003 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.003 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of daemons per service per host", 2023-09-13T04:50:36.003 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.003 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.003 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.004 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd_draining_count": { 2023-09-13T04:50:36.004 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_osd_draining_count", 2023-09-13T04:50:36.004 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.004 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.004 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.005 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-09-13T04:50:36.005 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.005 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.005 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.005 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-09-13T04:50:36.006 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.006 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.006 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.006 INFO:teuthology.orchestra.run.smithi002.stdout: "migration_current": { 2023-09-13T04:50:36.007 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "migration_current", 2023-09-13T04:50:36.007 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.007 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.007 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.007 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.007 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.008 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.008 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.008 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "internal - do not modify", 2023-09-13T04:50:36.008 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.008 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.009 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.009 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2023-09-13T04:50:36.009 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2023-09-13T04:50:36.009 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.010 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.010 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.010 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "root", 2023-09-13T04:50:36.010 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.010 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.011 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.011 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm-package", 2023-09-13T04:50:36.011 INFO:teuthology.orchestra.run.smithi002.stdout: "root" 2023-09-13T04:50:36.011 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.011 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "mode for remote execution of cephadm", 2023-09-13T04:50:36.012 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.012 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.012 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.012 INFO:teuthology.orchestra.run.smithi002.stdout: "prometheus_alerts_path": { 2023-09-13T04:50:36.013 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus_alerts_path", 2023-09-13T04:50:36.013 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.013 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.013 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.013 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-09-13T04:50:36.014 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.014 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.014 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.014 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-09-13T04:50:36.014 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.015 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.015 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.015 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_insecure": { 2023-09-13T04:50:36.015 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_insecure", 2023-09-13T04:50:36.016 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.016 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.016 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.016 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.016 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.016 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.017 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.017 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-09-13T04:50:36.017 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.017 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.017 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.018 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_password": { 2023-09-13T04:50:36.018 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_password", 2023-09-13T04:50:36.018 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.018 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.019 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.019 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.019 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.019 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.019 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.020 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-09-13T04:50:36.020 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.020 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.020 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.020 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_url": { 2023-09-13T04:50:36.021 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_url", 2023-09-13T04:50:36.021 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.021 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.021 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.021 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.022 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.022 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.022 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.022 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-09-13T04:50:36.022 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.023 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.023 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.023 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_username": { 2023-09-13T04:50:36.023 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_username", 2023-09-13T04:50:36.023 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.024 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.024 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.024 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.024 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.024 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.025 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.025 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-09-13T04:50:36.025 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.025 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.025 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.026 INFO:teuthology.orchestra.run.smithi002.stdout: "secure_monitoring_stack": { 2023-09-13T04:50:36.026 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "secure_monitoring_stack", 2023-09-13T04:50:36.026 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.026 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.027 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.027 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.027 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.027 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.027 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.028 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-09-13T04:50:36.028 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.028 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.028 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.029 INFO:teuthology.orchestra.run.smithi002.stdout: "service_discovery_port": { 2023-09-13T04:50:36.029 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "service_discovery_port", 2023-09-13T04:50:36.029 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.029 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.029 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.030 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8765", 2023-09-13T04:50:36.030 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.030 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.030 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.030 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "cephadm service discovery port", 2023-09-13T04:50:36.031 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.031 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.031 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.031 INFO:teuthology.orchestra.run.smithi002.stdout: "ssh_config_file": { 2023-09-13T04:50:36.032 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssh_config_file", 2023-09-13T04:50:36.032 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.032 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.032 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.032 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.032 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.033 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.033 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.033 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-09-13T04:50:36.033 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.033 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.034 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.034 INFO:teuthology.orchestra.run.smithi002.stdout: "use_agent": { 2023-09-13T04:50:36.034 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_agent", 2023-09-13T04:50:36.034 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.035 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.035 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.035 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.035 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.035 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.036 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.036 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-09-13T04:50:36.036 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.036 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.036 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.037 INFO:teuthology.orchestra.run.smithi002.stdout: "use_repo_digest": { 2023-09-13T04:50:36.037 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_repo_digest", 2023-09-13T04:50:36.037 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.037 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.038 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.038 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.038 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.038 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.038 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.039 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-09-13T04:50:36.039 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.039 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.039 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.039 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_failed_host_check": { 2023-09-13T04:50:36.040 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_failed_host_check", 2023-09-13T04:50:36.040 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.040 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.040 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.040 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.041 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.041 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.041 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.041 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if the host check fails", 2023-09-13T04:50:36.041 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.041 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.042 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.042 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_daemons": { 2023-09-13T04:50:36.042 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_daemons", 2023-09-13T04:50:36.042 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.043 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.043 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.043 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.043 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.043 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.044 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.044 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-09-13T04:50:36.044 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.044 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.044 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.045 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_hosts": { 2023-09-13T04:50:36.045 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_hosts", 2023-09-13T04:50:36.045 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.045 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.045 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.046 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.046 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.046 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.046 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.047 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-09-13T04:50:36.047 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.047 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.047 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.047 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.047 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.048 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crash", 2023-09-13T04:50:36.048 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.048 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.049 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.049 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.049 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.049 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.049 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.050 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.050 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.050 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.050 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.050 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.051 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.051 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.051 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.051 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.051 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.052 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.052 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.052 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.052 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.052 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.052 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.053 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.053 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.053 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.053 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.054 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.054 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.054 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.054 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.054 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.055 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.055 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.055 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.055 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.055 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.056 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.056 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.056 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.056 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.056 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.057 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.057 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.057 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.057 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.057 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.058 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.058 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.058 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.058 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.058 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.058 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.059 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.059 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.059 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.059 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.060 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.060 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.060 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.060 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.060 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.060 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.061 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.061 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.061 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.061 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.061 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.062 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.062 INFO:teuthology.orchestra.run.smithi002.stdout: "retain_interval": { 2023-09-13T04:50:36.062 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retain_interval", 2023-09-13T04:50:36.062 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:36.063 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.063 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.063 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "31536000", 2023-09-13T04:50:36.063 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.063 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.064 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.064 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain crashes before pruning them", 2023-09-13T04:50:36.064 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.064 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.064 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.065 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_recent_interval": { 2023-09-13T04:50:36.065 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_recent_interval", 2023-09-13T04:50:36.065 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:36.065 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.065 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.066 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1209600", 2023-09-13T04:50:36.066 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.066 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.066 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.066 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "time interval in which to warn about recent crashes", 2023-09-13T04:50:36.066 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.067 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.067 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.067 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.067 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.068 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dashboard", 2023-09-13T04:50:36.068 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.068 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.068 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.069 INFO:teuthology.orchestra.run.smithi002.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-09-13T04:50:36.069 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-09-13T04:50:36.069 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.069 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.069 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.069 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-09-13T04:50:36.070 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.070 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.070 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.070 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.070 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.071 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.071 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.071 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_HOST": { 2023-09-13T04:50:36.071 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_HOST", 2023-09-13T04:50:36.072 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.072 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.072 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.072 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.072 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.072 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.073 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.073 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.073 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.073 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.073 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.074 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-09-13T04:50:36.074 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-09-13T04:50:36.074 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.074 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.075 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.075 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.075 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.075 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.075 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.076 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.076 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.076 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.076 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.076 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_ENABLED": { 2023-09-13T04:50:36.077 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_ENABLED", 2023-09-13T04:50:36.077 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.077 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.077 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.077 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.078 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.078 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.078 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.078 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.078 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.079 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.079 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.079 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-09-13T04:50:36.079 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-09-13T04:50:36.079 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.080 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.080 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.080 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.080 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.080 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.081 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.081 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.081 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.081 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.081 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.082 INFO:teuthology.orchestra.run.smithi002.stdout: "ENABLE_BROWSABLE_API": { 2023-09-13T04:50:36.082 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ENABLE_BROWSABLE_API", 2023-09-13T04:50:36.082 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.082 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.083 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.083 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.083 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.083 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.083 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.084 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.084 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.084 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.084 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.085 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-09-13T04:50:36.085 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-09-13T04:50:36.085 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.085 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.085 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.086 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.086 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.086 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.086 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.086 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.087 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.087 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.087 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.087 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-09-13T04:50:36.087 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-09-13T04:50:36.088 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.088 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.088 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.088 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.088 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.089 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.089 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.089 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.089 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.089 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.090 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.090 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-09-13T04:50:36.090 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-09-13T04:50:36.090 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.091 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.091 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.091 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.091 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.091 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.092 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.092 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.092 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.092 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.092 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.093 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-09-13T04:50:36.093 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-09-13T04:50:36.093 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.093 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.093 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.094 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.094 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.094 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.094 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.094 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.095 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.095 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.095 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.095 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_NFS": { 2023-09-13T04:50:36.096 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-09-13T04:50:36.096 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.096 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.096 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.096 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.096 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.097 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.097 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.097 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.097 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.097 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.098 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.098 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RBD": { 2023-09-13T04:50:36.098 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-09-13T04:50:36.098 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.099 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.099 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.099 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.099 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.099 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.100 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.100 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.100 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.100 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.100 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.101 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RGW": { 2023-09-13T04:50:36.101 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-09-13T04:50:36.101 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.101 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.101 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.102 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.102 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.102 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.102 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.102 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.103 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.103 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.103 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.103 INFO:teuthology.orchestra.run.smithi002.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-09-13T04:50:36.104 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-09-13T04:50:36.104 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.104 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.104 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.104 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.104 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.105 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.105 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.105 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.105 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.105 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.106 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.106 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_PASSWORD": { 2023-09-13T04:50:36.106 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_PASSWORD", 2023-09-13T04:50:36.106 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.107 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.107 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.107 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-09-13T04:50:36.107 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.107 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.107 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.108 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.108 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.108 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.108 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.109 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-09-13T04:50:36.109 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-09-13T04:50:36.109 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.109 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.109 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.110 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.110 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.110 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.110 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.110 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.111 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.111 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.111 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.111 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_URL": { 2023-09-13T04:50:36.111 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_URL", 2023-09-13T04:50:36.112 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.112 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.112 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.112 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.112 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.113 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.113 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.113 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.113 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.113 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.114 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.114 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_USERNAME": { 2023-09-13T04:50:36.114 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_USERNAME", 2023-09-13T04:50:36.114 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.115 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.115 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.115 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-09-13T04:50:36.115 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.115 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.115 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.116 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.116 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.116 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.116 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.117 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-09-13T04:50:36.117 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-09-13T04:50:36.117 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.117 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.117 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.118 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.118 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.118 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.118 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.118 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.119 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.119 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.119 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.119 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-09-13T04:50:36.120 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-09-13T04:50:36.120 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.120 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.120 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.120 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.120 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.121 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.121 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.121 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.121 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.121 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.122 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.122 INFO:teuthology.orchestra.run.smithi002.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-09-13T04:50:36.122 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-09-13T04:50:36.122 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.123 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.123 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.123 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.123 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.123 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.123 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.124 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.124 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.124 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.124 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.125 INFO:teuthology.orchestra.run.smithi002.stdout: "ISSUE_TRACKER_API_KEY": { 2023-09-13T04:50:36.125 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-09-13T04:50:36.125 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.125 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.125 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.126 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.126 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.126 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.126 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.126 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.127 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.127 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.127 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.127 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_HOST": { 2023-09-13T04:50:36.128 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_HOST", 2023-09-13T04:50:36.128 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.128 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.128 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.128 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.129 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.129 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.129 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.129 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.129 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.129 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.130 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.130 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-09-13T04:50:36.130 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-09-13T04:50:36.130 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.131 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.131 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.131 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.131 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.131 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.132 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.132 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.132 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.132 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.132 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.133 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-09-13T04:50:36.133 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-09-13T04:50:36.133 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.133 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.134 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.134 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.134 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.134 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.134 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.135 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.135 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.135 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.135 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.135 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-09-13T04:50:36.136 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-09-13T04:50:36.136 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.136 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.136 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.136 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.137 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.137 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.137 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.137 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.137 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.138 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.138 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.138 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-09-13T04:50:36.138 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-09-13T04:50:36.138 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.139 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.139 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.139 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.139 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.139 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.140 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.140 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.140 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.140 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.140 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.141 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-09-13T04:50:36.141 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-09-13T04:50:36.141 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.141 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.142 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.142 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.142 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.142 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.142 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.143 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.143 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.143 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.143 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.143 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-09-13T04:50:36.144 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-09-13T04:50:36.144 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.144 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.144 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.144 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.145 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.145 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.145 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.145 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.145 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.145 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.146 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.146 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-09-13T04:50:36.146 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-09-13T04:50:36.147 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.147 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.147 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.147 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.147 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.147 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.148 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.148 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.148 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.148 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.148 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.149 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-09-13T04:50:36.149 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-09-13T04:50:36.149 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.149 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.149 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.150 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.150 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.150 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.150 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.150 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.151 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.151 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.151 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.151 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_ENABLED": { 2023-09-13T04:50:36.152 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_ENABLED", 2023-09-13T04:50:36.152 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.152 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.152 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.152 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.153 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.153 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.153 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.153 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.153 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.153 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.154 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.154 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-09-13T04:50:36.154 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-09-13T04:50:36.154 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.155 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.155 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.155 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-09-13T04:50:36.155 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.155 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.156 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.156 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.156 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.156 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.156 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.157 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-09-13T04:50:36.157 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-09-13T04:50:36.157 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.157 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.157 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.158 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-09-13T04:50:36.158 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.158 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.158 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.158 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.159 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.159 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.159 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.159 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-09-13T04:50:36.159 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-09-13T04:50:36.160 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.160 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.160 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.160 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8", 2023-09-13T04:50:36.160 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.161 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.161 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.161 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.161 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.161 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.162 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.162 INFO:teuthology.orchestra.run.smithi002.stdout: "REST_REQUESTS_TIMEOUT": { 2023-09-13T04:50:36.162 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-09-13T04:50:36.162 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.162 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.163 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.163 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "45", 2023-09-13T04:50:36.163 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.163 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.163 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.164 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.164 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.164 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.164 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.164 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ACCESS_KEY": { 2023-09-13T04:50:36.165 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ACCESS_KEY", 2023-09-13T04:50:36.165 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.165 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.165 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.165 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.166 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.166 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.166 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.166 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.166 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.167 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.167 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.167 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-09-13T04:50:36.167 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-09-13T04:50:36.167 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.168 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.168 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.168 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-09-13T04:50:36.168 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.168 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.169 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.169 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.169 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.169 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.169 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.170 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SECRET_KEY": { 2023-09-13T04:50:36.170 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SECRET_KEY", 2023-09-13T04:50:36.170 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.170 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.171 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.171 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.171 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.171 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.171 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.171 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.172 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.172 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.172 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.172 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SSL_VERIFY": { 2023-09-13T04:50:36.173 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SSL_VERIFY", 2023-09-13T04:50:36.173 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.173 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.173 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.173 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.174 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.174 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.174 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.174 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.174 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.175 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.175 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.175 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-09-13T04:50:36.175 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-09-13T04:50:36.176 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.176 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.176 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.176 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-09-13T04:50:36.176 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.176 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.177 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.177 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.177 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.177 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.177 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.178 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-09-13T04:50:36.178 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-09-13T04:50:36.178 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.178 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.179 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.179 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-09-13T04:50:36.179 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.179 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.179 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.180 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.180 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.180 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.180 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.181 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-09-13T04:50:36.181 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-09-13T04:50:36.181 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.181 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.181 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.181 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2023-09-13T04:50:36.182 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.182 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.182 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.182 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.182 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.183 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.183 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.183 INFO:teuthology.orchestra.run.smithi002.stdout: "cross_origin_url": { 2023-09-13T04:50:36.183 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cross_origin_url", 2023-09-13T04:50:36.184 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.184 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.184 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.184 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.184 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.185 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.185 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.185 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.185 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.185 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.186 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.186 INFO:teuthology.orchestra.run.smithi002.stdout: "crt_file": { 2023-09-13T04:50:36.186 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crt_file", 2023-09-13T04:50:36.186 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.187 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.187 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.187 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.187 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.187 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.188 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.188 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.188 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.188 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.188 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.189 INFO:teuthology.orchestra.run.smithi002.stdout: "debug": { 2023-09-13T04:50:36.189 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "debug", 2023-09-13T04:50:36.189 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.189 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.190 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.190 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.190 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.190 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.190 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.191 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable/disable debug options", 2023-09-13T04:50:36.191 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.191 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.191 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.191 INFO:teuthology.orchestra.run.smithi002.stdout: "jwt_token_ttl": { 2023-09-13T04:50:36.192 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "jwt_token_ttl", 2023-09-13T04:50:36.192 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.192 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.192 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.192 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "28800", 2023-09-13T04:50:36.193 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.193 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.193 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.193 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.193 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.194 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.194 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.194 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2023-09-13T04:50:36.194 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2023-09-13T04:50:36.195 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.195 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.195 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.195 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.195 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.196 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.196 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.196 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.196 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.196 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.196 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.197 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.197 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.197 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.197 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.198 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.198 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.198 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.198 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.198 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.199 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.199 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.199 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.199 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.199 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.199 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.200 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.200 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.200 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.200 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.200 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.201 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.201 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.201 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.201 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.201 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.202 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.202 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.202 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.202 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.202 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.203 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.203 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.203 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.203 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.204 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.204 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.204 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.204 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.204 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.204 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.205 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.205 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.205 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.205 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.205 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.206 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.206 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.206 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.206 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.206 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.207 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.207 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.207 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.207 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.208 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.208 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.208 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.208 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.208 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.209 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.209 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.209 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.209 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.209 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.209 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.210 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.210 INFO:teuthology.orchestra.run.smithi002.stdout: "motd": { 2023-09-13T04:50:36.210 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "motd", 2023-09-13T04:50:36.210 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.211 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.211 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.211 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.211 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.211 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.212 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.212 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "The message of the day", 2023-09-13T04:50:36.212 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.212 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.212 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.213 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect_resolve_ip_addr": { 2023-09-13T04:50:36.213 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "redirect_resolve_ip_addr", 2023-09-13T04:50:36.213 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.213 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.214 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.214 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.214 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.214 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.214 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.215 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.215 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.215 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.215 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.216 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2023-09-13T04:50:36.216 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2023-09-13T04:50:36.216 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.216 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.216 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.216 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2023-09-13T04:50:36.217 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.217 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.217 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.217 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.217 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.218 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.218 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.218 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2023-09-13T04:50:36.218 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2023-09-13T04:50:36.219 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.219 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.219 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.219 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8080", 2023-09-13T04:50:36.219 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.221 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.221 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.221 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.221 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.221 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.222 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.222 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2023-09-13T04:50:36.222 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2023-09-13T04:50:36.222 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.223 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.223 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.223 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.223 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.223 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.224 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.224 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.224 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.224 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.224 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.225 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl_server_port": { 2023-09-13T04:50:36.225 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl_server_port", 2023-09-13T04:50:36.225 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.225 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.225 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.226 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8443", 2023-09-13T04:50:36.226 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.226 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.226 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.226 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.227 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.227 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.227 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.227 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2023-09-13T04:50:36.228 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2023-09-13T04:50:36.228 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.228 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.228 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.228 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "redirect", 2023-09-13T04:50:36.229 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.229 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.229 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.229 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.229 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect" 2023-09-13T04:50:36.229 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.230 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.230 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.230 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.230 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.231 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2023-09-13T04:50:36.231 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2023-09-13T04:50:36.231 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.231 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.231 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.232 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2023-09-13T04:50:36.232 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2023-09-13T04:50:36.232 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2023-09-13T04:50:36.232 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.232 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.233 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.233 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.233 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.233 INFO:teuthology.orchestra.run.smithi002.stdout: "url_prefix": { 2023-09-13T04:50:36.233 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url_prefix", 2023-09-13T04:50:36.234 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.234 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.234 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.234 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.235 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.235 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.235 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.235 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.235 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.235 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.236 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.236 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.236 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.237 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "devicehealth", 2023-09-13T04:50:36.237 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.237 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.237 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.237 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_monitoring": { 2023-09-13T04:50:36.237 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_monitoring", 2023-09-13T04:50:36.238 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.238 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.238 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.238 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.238 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.239 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.239 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.239 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "monitor device health metrics", 2023-09-13T04:50:36.239 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.239 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.240 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.240 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.240 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.240 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.241 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.241 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.241 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.241 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.241 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.242 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.242 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.242 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.242 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.242 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.243 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.243 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.243 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.243 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.243 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.243 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.244 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.244 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.244 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.244 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.245 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.245 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.245 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.245 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.245 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.245 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.246 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.246 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.246 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.246 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.247 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.247 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.247 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.247 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.247 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.247 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.248 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.248 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.248 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.248 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.248 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.249 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.249 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.249 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.249 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.249 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.250 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.250 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.250 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.250 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.251 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.251 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.251 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.251 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.251 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.252 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.252 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.252 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.252 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.252 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.253 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.253 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.253 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.253 INFO:teuthology.orchestra.run.smithi002.stdout: "mark_out_threshold": { 2023-09-13T04:50:36.253 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mark_out_threshold", 2023-09-13T04:50:36.254 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:36.254 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.254 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.254 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2419200", 2023-09-13T04:50:36.254 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.255 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.255 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.255 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-09-13T04:50:36.255 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.255 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.255 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.256 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": { 2023-09-13T04:50:36.256 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_name", 2023-09-13T04:50:36.256 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.256 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.257 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.257 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "device_health_metrics", 2023-09-13T04:50:36.257 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.257 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.257 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.257 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name of pool in which to store device health metrics", 2023-09-13T04:50:36.258 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.258 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.258 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.258 INFO:teuthology.orchestra.run.smithi002.stdout: "retention_period": { 2023-09-13T04:50:36.259 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retention_period", 2023-09-13T04:50:36.259 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:36.259 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.259 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.259 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15552000", 2023-09-13T04:50:36.260 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.260 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.260 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.260 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain device health metrics", 2023-09-13T04:50:36.260 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.260 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.261 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.261 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_frequency": { 2023-09-13T04:50:36.261 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_frequency", 2023-09-13T04:50:36.262 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:36.262 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.262 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.262 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2023-09-13T04:50:36.262 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.262 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.263 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.263 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to scrape device health metrics", 2023-09-13T04:50:36.263 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.263 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.263 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.264 INFO:teuthology.orchestra.run.smithi002.stdout: "self_heal": { 2023-09-13T04:50:36.264 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "self_heal", 2023-09-13T04:50:36.264 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.264 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.265 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.265 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.265 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.265 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.265 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.265 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-09-13T04:50:36.266 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.266 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.266 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.266 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-09-13T04:50:36.267 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-09-13T04:50:36.267 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:36.267 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.267 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.267 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-09-13T04:50:36.268 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.268 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.268 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.268 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and check device health", 2023-09-13T04:50:36.268 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.269 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.269 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.269 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_threshold": { 2023-09-13T04:50:36.269 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_threshold", 2023-09-13T04:50:36.270 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:36.270 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.270 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.270 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7257600", 2023-09-13T04:50:36.270 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.271 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.271 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.271 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-09-13T04:50:36.271 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.271 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.271 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.272 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.272 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.272 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "diskprediction_local", 2023-09-13T04:50:36.273 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.273 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.273 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.273 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.273 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.274 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.274 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.274 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.274 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.274 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.274 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.275 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.275 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.275 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.275 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.275 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.276 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.276 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.276 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.276 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.276 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.277 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.277 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.277 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.277 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.277 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.278 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.278 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.278 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.278 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.278 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.279 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.279 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.279 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.279 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.279 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.280 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.280 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.280 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.280 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.280 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.281 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.281 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.281 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.281 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.281 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.282 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.282 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.282 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.282 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.282 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: "predict_interval": { 2023-09-13T04:50:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predict_interval", 2023-09-13T04:50:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2023-09-13T04:50:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: "predictor_model": { 2023-09-13T04:50:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predictor_model", 2023-09-13T04:50:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "prophetstor", 2023-09-13T04:50:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-09-13T04:50:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-09-13T04:50:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-09-13T04:50:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "influx", 2023-09-13T04:50:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": false, 2023-09-13T04:50:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "influxdb python module not found", 2023-09-13T04:50:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "batch_size": { 2023-09-13T04:50:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "batch_size", 2023-09-13T04:50:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5000", 2023-09-13T04:50:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-09-13T04:50:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "database": { 2023-09-13T04:50:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "database", 2023-09-13T04:50:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "ceph", 2023-09-13T04:50:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-09-13T04:50:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": { 2023-09-13T04:50:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hostname", 2023-09-13T04:50:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server hostname", 2023-09-13T04:50:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-09-13T04:50:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-09-13T04:50:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "30", 2023-09-13T04:50:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "5", 2023-09-13T04:50:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-09-13T04:50:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: "password": { 2023-09-13T04:50:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "password", 2023-09-13T04:50:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "password of InfluxDB server user", 2023-09-13T04:50:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "port": { 2023-09-13T04:50:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "port", 2023-09-13T04:50:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8086", 2023-09-13T04:50:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server port", 2023-09-13T04:50:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2023-09-13T04:50:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2023-09-13T04:50:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "false", 2023-09-13T04:50:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-09-13T04:50:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: "threads": { 2023-09-13T04:50:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threads", 2023-09-13T04:50:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2023-09-13T04:50:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2023-09-13T04:50:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "32", 2023-09-13T04:50:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-09-13T04:50:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: "username": { 2023-09-13T04:50:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "username", 2023-09-13T04:50:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "username of InfluxDB server user", 2023-09-13T04:50:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: "verify_ssl": { 2023-09-13T04:50:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "verify_ssl", 2023-09-13T04:50:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "true", 2023-09-13T04:50:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-09-13T04:50:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "insights", 2023-09-13T04:50:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.345 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.345 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.345 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.345 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.345 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.346 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.346 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.346 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.346 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.346 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.347 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.347 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.347 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.347 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.347 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.348 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.348 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.348 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.348 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.348 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.349 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.349 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.349 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.349 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.349 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.350 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.350 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.350 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.350 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.350 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.351 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "iostat", 2023-09-13T04:50:36.351 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.351 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.351 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.352 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.352 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.352 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.352 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.352 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "k8sevents", 2023-09-13T04:50:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_event_retention_days": { 2023-09-13T04:50:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ceph_event_retention_days", 2023-09-13T04:50:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7", 2023-09-13T04:50:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Days to hold ceph event information within local cache", 2023-09-13T04:50:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: "config_check_secs": { 2023-09-13T04:50:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_check_secs", 2023-09-13T04:50:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-09-13T04:50:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "10", 2023-09-13T04:50:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-09-13T04:50:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "localpool", 2023-09-13T04:50:36.386 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.386 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.386 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.386 INFO:teuthology.orchestra.run.smithi002.stdout: "failure_domain": { 2023-09-13T04:50:36.386 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "failure_domain", 2023-09-13T04:50:36.387 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.387 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.387 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.387 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "host", 2023-09-13T04:50:36.388 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.388 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.388 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.388 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "failure domain for any created local pool", 2023-09-13T04:50:36.388 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-09-13T04:50:36.389 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.389 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.389 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.389 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.390 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.390 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.390 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.390 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.390 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.391 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.391 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.391 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.391 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.391 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.391 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.392 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.392 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.392 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.392 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.392 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.393 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.393 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.393 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.393 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.394 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.394 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.394 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.394 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.394 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.394 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.395 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.395 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.395 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.395 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.395 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.396 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.396 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.396 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.396 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.396 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.397 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.397 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.397 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.397 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.397 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.398 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.398 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.398 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.398 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.398 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.399 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.399 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.399 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.399 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.399 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.400 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.400 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.400 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.400 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.400 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.401 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.401 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.401 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.401 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.401 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.402 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.402 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.402 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.402 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": { 2023-09-13T04:50:36.403 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_size", 2023-09-13T04:50:36.403 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.403 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.403 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.403 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.404 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.404 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.404 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.404 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default min_size for any created local pool", 2023-09-13T04:50:36.404 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-09-13T04:50:36.405 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.405 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.405 INFO:teuthology.orchestra.run.smithi002.stdout: "num_rep": { 2023-09-13T04:50:36.405 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "num_rep", 2023-09-13T04:50:36.406 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.406 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.406 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.406 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3", 2023-09-13T04:50:36.406 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.407 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.407 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.407 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default replica count for any created local pool", 2023-09-13T04:50:36.407 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.407 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.407 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.408 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": { 2023-09-13T04:50:36.408 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_num", 2023-09-13T04:50:36.408 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.408 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.409 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.409 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "128", 2023-09-13T04:50:36.409 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.409 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.409 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.410 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default pg_num for any created local pool", 2023-09-13T04:50:36.410 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.410 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.410 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.411 INFO:teuthology.orchestra.run.smithi002.stdout: "prefix": { 2023-09-13T04:50:36.411 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prefix", 2023-09-13T04:50:36.411 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.411 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.411 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.411 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.412 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.412 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.412 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.412 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name prefix for any created local pool", 2023-09-13T04:50:36.412 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.413 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.413 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.413 INFO:teuthology.orchestra.run.smithi002.stdout: "subtree": { 2023-09-13T04:50:36.413 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "subtree", 2023-09-13T04:50:36.414 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.414 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.414 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.414 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "rack", 2023-09-13T04:50:36.414 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.415 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.415 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.415 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "CRUSH level for which to create a local pool", 2023-09-13T04:50:36.415 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-09-13T04:50:36.415 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.416 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.416 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.416 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.416 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mds_autoscaler", 2023-09-13T04:50:36.417 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.417 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.417 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.417 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.417 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.418 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.418 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.418 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.418 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.418 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.419 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.419 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.419 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.419 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.419 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.420 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.420 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.420 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.420 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.420 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.420 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.421 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.421 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.421 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.421 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.422 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.422 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.422 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.422 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.422 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.423 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.423 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.423 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.423 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.423 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.423 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.424 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.424 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.424 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.424 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.425 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.425 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.425 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.425 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.425 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.426 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.426 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.426 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.426 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.426 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.426 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.427 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.427 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.427 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.427 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.427 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.428 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.428 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.428 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.428 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.429 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.429 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.429 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.429 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.429 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.429 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.430 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.430 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.430 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.430 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.430 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.431 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirroring", 2023-09-13T04:50:36.431 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.431 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.432 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.432 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.432 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.432 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.432 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.433 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.433 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.433 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.433 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.433 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.433 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.434 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.434 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.434 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.434 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.434 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.435 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.435 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.435 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.435 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.435 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.436 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.436 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.436 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.436 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.436 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.437 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.437 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.437 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.437 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.437 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.438 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.438 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.438 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.438 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.438 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.439 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.439 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.439 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.439 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.439 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.440 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.440 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.440 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.440 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.440 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.441 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.441 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.441 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.441 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.442 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.442 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.442 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.442 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.443 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.443 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.443 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.443 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.443 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.444 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.444 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.444 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.444 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.444 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.445 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.445 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.445 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.445 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.445 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.446 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "nfs", 2023-09-13T04:50:36.446 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.446 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.446 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.447 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.447 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.447 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.447 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.447 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.448 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.448 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.448 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.448 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.448 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.449 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.449 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.449 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.449 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.449 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.449 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.450 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.450 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.450 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.450 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.451 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.451 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.451 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.451 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.451 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.451 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.452 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.452 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.452 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.452 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.452 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.453 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.453 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.453 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.453 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.454 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.454 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.454 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.454 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.454 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.454 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.455 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.455 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.455 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.455 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.455 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.456 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.456 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.456 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.456 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.456 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.457 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.457 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.457 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.457 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.457 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.458 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.458 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.458 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.458 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.458 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.459 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.459 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.459 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.459 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.459 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.460 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.460 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.460 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2023-09-13T04:50:36.461 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.461 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.461 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.461 INFO:teuthology.orchestra.run.smithi002.stdout: "fail_fs": { 2023-09-13T04:50:36.461 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "fail_fs", 2023-09-13T04:50:36.461 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.462 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.462 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.462 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.462 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.462 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.463 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.463 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-09-13T04:50:36.463 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.463 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.463 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.464 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.464 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.464 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.464 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.464 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.465 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.465 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.465 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.465 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.465 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.466 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.466 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.466 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.466 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.466 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.466 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.467 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.467 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.467 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.467 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.468 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.468 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.468 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.468 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.468 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.469 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.469 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.469 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.469 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.469 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.470 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.470 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.470 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.470 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.471 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.471 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.471 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.471 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.471 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.471 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.472 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.472 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.472 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.472 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.472 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.473 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.473 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.473 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.473 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.473 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.474 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.474 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.474 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.474 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.475 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.475 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.475 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.475 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.475 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.475 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.476 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.476 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.476 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.476 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.476 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.477 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.477 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator": { 2023-09-13T04:50:36.477 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2023-09-13T04:50:36.477 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.477 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.478 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.478 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.478 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.478 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.478 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.479 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm", 2023-09-13T04:50:36.479 INFO:teuthology.orchestra.run.smithi002.stdout: "rook", 2023-09-13T04:50:36.479 INFO:teuthology.orchestra.run.smithi002.stdout: "test_orchestrator" 2023-09-13T04:50:36.479 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.479 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Orchestrator backend", 2023-09-13T04:50:36.480 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.480 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.480 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.480 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.480 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.481 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_perf_query", 2023-09-13T04:50:36.481 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.481 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.481 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.482 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.482 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.482 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.482 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.482 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.483 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.483 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.483 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.483 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.483 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.484 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.484 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.484 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.484 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.484 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.485 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.485 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.485 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.485 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.485 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.486 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.486 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.486 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.486 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.487 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.487 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.487 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.487 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.487 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.488 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.488 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.488 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.488 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.488 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.489 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.489 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.489 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.489 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.489 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.490 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.490 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.490 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.490 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.490 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.491 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.491 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.491 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.491 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.491 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.492 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.492 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.492 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.492 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.493 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.493 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.493 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.493 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.493 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.494 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.494 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.494 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.494 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.494 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.495 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.495 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.495 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.496 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_support", 2023-09-13T04:50:36.496 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.496 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.496 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.497 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.497 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.497 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.497 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.497 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.498 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.498 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.498 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.498 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.498 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.498 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.499 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.499 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.499 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.499 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.499 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.500 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.500 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.500 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.500 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.501 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.501 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.501 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.501 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.501 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.501 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.502 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.502 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.502 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.502 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.502 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.503 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.503 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.503 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.503 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.504 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.504 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.504 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.504 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.504 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.505 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.505 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.505 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.505 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.505 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.506 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.506 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.506 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.506 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.506 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.507 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.507 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.507 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.507 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.508 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.508 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.508 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.508 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.508 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.508 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.509 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.509 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.509 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.509 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.509 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.509 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.510 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.510 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.510 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_autoscaler", 2023-09-13T04:50:36.511 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.511 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.511 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.511 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.511 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.512 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.512 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.512 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.512 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.512 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.513 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.513 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.513 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.513 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.513 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.513 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.514 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.514 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.514 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.514 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.514 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.515 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.515 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.515 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.515 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.515 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.516 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.516 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.516 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.516 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.516 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.517 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.517 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.517 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.517 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.517 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.518 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.518 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.518 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.518 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.518 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.519 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.519 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.519 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.519 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.519 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.520 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.520 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.520 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.520 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.520 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.521 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.521 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.521 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.521 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.521 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.522 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.522 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.522 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.522 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.522 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.523 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.523 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.523 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.523 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.523 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.524 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.524 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.524 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.524 INFO:teuthology.orchestra.run.smithi002.stdout: "noautoscale": { 2023-09-13T04:50:36.524 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "noautoscale", 2023-09-13T04:50:36.525 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.525 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.525 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.525 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.525 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.526 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.526 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.526 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "global autoscale flag", 2023-09-13T04:50:36.526 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-09-13T04:50:36.526 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.527 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.527 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-09-13T04:50:36.527 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-09-13T04:50:36.527 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:36.528 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.528 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.528 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-09-13T04:50:36.528 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.528 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.529 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.529 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.529 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.529 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.529 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.530 INFO:teuthology.orchestra.run.smithi002.stdout: "threshold": { 2023-09-13T04:50:36.530 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threshold", 2023-09-13T04:50:36.530 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-09-13T04:50:36.530 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.531 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.531 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2023-09-13T04:50:36.531 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1.0", 2023-09-13T04:50:36.531 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.531 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.531 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "scaling threshold", 2023-09-13T04:50:36.532 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-09-13T04:50:36.532 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.532 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.532 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.532 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.533 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "progress", 2023-09-13T04:50:36.533 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.533 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.534 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.534 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_pg_recovery_event": { 2023-09-13T04:50:36.534 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_pg_recovery_event", 2023-09-13T04:50:36.534 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.534 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.535 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.535 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.535 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.535 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.535 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.536 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow the module to show pg recovery progress", 2023-09-13T04:50:36.536 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.536 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.536 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.536 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2023-09-13T04:50:36.537 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2023-09-13T04:50:36.537 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.537 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.537 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.537 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.538 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.538 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.538 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.538 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.538 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.539 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.539 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.539 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.539 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.540 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.540 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.540 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.540 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.540 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.541 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.541 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.541 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.541 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.541 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.541 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.542 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.542 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.542 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.542 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.542 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.543 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.543 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.543 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.543 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.544 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.544 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.544 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.544 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.544 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.545 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.545 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.545 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.545 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.545 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.545 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.546 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.546 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.546 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.546 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.547 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.547 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.547 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.547 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.547 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.548 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.548 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.548 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.548 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.548 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.548 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.549 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.549 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.549 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.549 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.549 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.550 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.550 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.550 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.550 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.551 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.551 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.551 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.551 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.551 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.552 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.552 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.552 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.552 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.553 INFO:teuthology.orchestra.run.smithi002.stdout: "max_completed_events": { 2023-09-13T04:50:36.553 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_completed_events", 2023-09-13T04:50:36.553 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.553 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.553 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.554 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "50", 2023-09-13T04:50:36.554 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.554 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.554 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.554 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "number of past completed events to remember", 2023-09-13T04:50:36.555 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.555 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.555 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.555 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-09-13T04:50:36.556 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-09-13T04:50:36.556 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:36.556 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.556 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.556 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2023-09-13T04:50:36.556 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.557 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.557 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.557 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long the module is going to sleep", 2023-09-13T04:50:36.557 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.557 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.558 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.558 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.558 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.559 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus", 2023-09-13T04:50:36.559 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.559 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.559 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.559 INFO:teuthology.orchestra.run.smithi002.stdout: "cache": { 2023-09-13T04:50:36.560 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cache", 2023-09-13T04:50:36.560 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.560 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.560 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.560 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.560 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.561 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.561 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.561 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.561 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.561 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.562 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.562 INFO:teuthology.orchestra.run.smithi002.stdout: "exclude_perf_counters": { 2023-09-13T04:50:36.562 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "exclude_perf_counters", 2023-09-13T04:50:36.562 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.563 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.563 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.563 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.563 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.563 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.564 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.564 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-09-13T04:50:36.564 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-09-13T04:50:36.564 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.564 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.565 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.565 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.565 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.565 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.565 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.566 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.566 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.566 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.566 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.566 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.567 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.567 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.567 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.567 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.567 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.567 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.568 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.568 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.568 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.568 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.569 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.569 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.569 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.569 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.569 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.570 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.570 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.570 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.570 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.570 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.571 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.571 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.571 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.571 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.572 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.572 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.572 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.572 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.572 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.572 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.573 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.573 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.573 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.573 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.573 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.574 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.574 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.574 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.574 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.574 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.575 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.575 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.575 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.575 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.576 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.576 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.576 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.576 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.576 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.577 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.577 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.577 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.577 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.577 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.577 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.578 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.578 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools": { 2023-09-13T04:50:36.578 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools", 2023-09-13T04:50:36.578 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.579 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.579 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.579 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.579 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.579 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.580 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.580 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.580 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.580 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.580 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.581 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools_refresh_interval": { 2023-09-13T04:50:36.581 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-09-13T04:50:36.581 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.581 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.582 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.582 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "300", 2023-09-13T04:50:36.582 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.582 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.582 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.583 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.583 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.583 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.583 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.584 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_interval": { 2023-09-13T04:50:36.584 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_interval", 2023-09-13T04:50:36.584 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-09-13T04:50:36.584 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.584 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.585 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15.0", 2023-09-13T04:50:36.585 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.585 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.585 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.585 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.586 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.586 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.586 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.586 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2023-09-13T04:50:36.587 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2023-09-13T04:50:36.587 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.587 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.587 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.587 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2023-09-13T04:50:36.587 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.588 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.588 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.588 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-09-13T04:50:36.588 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.589 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.589 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.589 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2023-09-13T04:50:36.589 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2023-09-13T04:50:36.589 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.590 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.590 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.590 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "9283", 2023-09-13T04:50:36.590 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.590 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.591 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.591 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-09-13T04:50:36.591 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.591 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.591 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.592 INFO:teuthology.orchestra.run.smithi002.stdout: "stale_cache_strategy": { 2023-09-13T04:50:36.592 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stale_cache_strategy", 2023-09-13T04:50:36.592 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.592 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.593 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.593 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "log", 2023-09-13T04:50:36.593 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.593 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.593 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.593 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.594 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.594 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.594 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.594 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2023-09-13T04:50:36.595 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2023-09-13T04:50:36.595 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.595 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.595 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.595 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "default", 2023-09-13T04:50:36.596 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.596 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.596 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.596 INFO:teuthology.orchestra.run.smithi002.stdout: "default", 2023-09-13T04:50:36.596 INFO:teuthology.orchestra.run.smithi002.stdout: "error" 2023-09-13T04:50:36.596 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.597 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.597 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.597 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.597 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.598 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2023-09-13T04:50:36.598 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2023-09-13T04:50:36.598 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.598 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.598 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.599 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2023-09-13T04:50:36.599 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2023-09-13T04:50:36.599 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2023-09-13T04:50:36.599 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.599 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.600 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.600 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.600 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.600 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.600 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.601 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_support", 2023-09-13T04:50:36.601 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.601 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.601 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.602 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.602 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.602 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.602 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.602 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.603 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.603 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.603 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.603 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.603 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.604 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.604 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.604 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.604 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.604 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.605 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.605 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.605 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.605 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.605 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.606 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.606 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.606 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.606 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.607 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.607 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.607 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.607 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.607 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.608 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.608 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.608 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.608 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.609 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.609 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.609 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.609 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.609 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.609 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.610 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.610 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.610 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.610 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.610 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.611 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.611 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.611 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.611 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.611 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.612 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.612 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.612 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.612 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.613 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.613 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.613 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.613 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.613 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.614 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.614 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.614 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.614 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.614 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.615 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.615 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.615 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.615 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_snap_create": { 2023-09-13T04:50:36.615 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_snap_create", 2023-09-13T04:50:36.616 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.616 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.616 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.616 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-09-13T04:50:36.616 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.617 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.617 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.617 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.617 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.617 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.618 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.618 INFO:teuthology.orchestra.run.smithi002.stdout: "mirror_snapshot_schedule": { 2023-09-13T04:50:36.618 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirror_snapshot_schedule", 2023-09-13T04:50:36.618 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.619 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.619 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.619 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.619 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.619 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.620 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.620 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.620 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.620 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.620 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.621 INFO:teuthology.orchestra.run.smithi002.stdout: "trash_purge_schedule": { 2023-09-13T04:50:36.621 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "trash_purge_schedule", 2023-09-13T04:50:36.621 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.621 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.621 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.622 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.622 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.622 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.622 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.622 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.623 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.623 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.623 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.623 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.623 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.624 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "restful", 2023-09-13T04:50:36.624 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.624 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.624 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.625 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_auth": { 2023-09-13T04:50:36.625 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_auth", 2023-09-13T04:50:36.625 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.625 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.626 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.626 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.626 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.626 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.626 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.626 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.627 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.627 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.627 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.629 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2023-09-13T04:50:36.629 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2023-09-13T04:50:36.630 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.630 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.630 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.630 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.630 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.631 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.631 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.631 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.631 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.631 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.632 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.632 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.632 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.632 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.633 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.633 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.633 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.633 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.633 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.634 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.634 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.634 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.634 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.635 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.635 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.635 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.635 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.635 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.636 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.636 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.636 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.636 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.637 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.637 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.637 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.637 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.637 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.638 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.638 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.638 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.638 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.638 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.639 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.639 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.639 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.639 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.640 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.640 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.640 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.640 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.640 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.641 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.641 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.641 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.641 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.641 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.642 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.642 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.642 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.642 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.642 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.643 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.643 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.643 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.643 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.643 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.644 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.644 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.644 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.644 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.644 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.645 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.645 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.645 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.645 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.645 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.646 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.646 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2023-09-13T04:50:36.646 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2023-09-13T04:50:36.646 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.647 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.647 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.647 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.647 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.647 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.648 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.648 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.648 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.648 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.648 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.649 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2023-09-13T04:50:36.649 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2023-09-13T04:50:36.649 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.649 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.650 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.650 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.650 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.650 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.650 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.651 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.651 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.651 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.651 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.651 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.652 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.652 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rgw", 2023-09-13T04:50:36.652 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.653 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.653 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.653 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.653 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.653 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.654 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.654 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.654 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.654 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.654 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.655 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.655 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.655 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.655 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.655 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.656 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.656 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.656 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.656 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.656 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.657 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.657 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.657 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.657 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.658 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.658 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.658 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.658 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.658 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.659 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.659 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.659 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.659 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.659 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.660 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.660 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.660 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.660 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.660 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.661 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.661 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.661 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.661 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.661 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.662 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.662 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.662 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.662 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.662 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.663 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.663 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.663 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.663 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.663 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.664 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.664 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.664 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.664 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.665 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.665 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.665 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.665 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.665 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.666 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.666 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.666 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.666 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.666 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.667 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.667 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.667 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rook", 2023-09-13T04:50:36.668 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.668 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.668 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.668 INFO:teuthology.orchestra.run.smithi002.stdout: "drive_group_interval": { 2023-09-13T04:50:36.668 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "drive_group_interval", 2023-09-13T04:50:36.669 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-09-13T04:50:36.669 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.669 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.669 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "300.0", 2023-09-13T04:50:36.669 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.670 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.670 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.670 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-09-13T04:50:36.670 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.670 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.671 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.671 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.671 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.671 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.672 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.672 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.672 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.672 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.672 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.673 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.673 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.673 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.673 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.673 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.673 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.674 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.674 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.674 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.674 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.674 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.675 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.675 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.675 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.675 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.676 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.676 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.676 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.676 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.676 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.677 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.677 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.677 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.677 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.677 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.678 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.678 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.678 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.678 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.679 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.679 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.679 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.679 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.679 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.679 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.680 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.680 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.680 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.680 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.680 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.681 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.681 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.681 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.681 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.681 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.682 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.682 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.682 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.682 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.683 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.683 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.683 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.683 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.683 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.684 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.684 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.684 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.684 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.685 INFO:teuthology.orchestra.run.smithi002.stdout: "storage_class": { 2023-09-13T04:50:36.685 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "storage_class", 2023-09-13T04:50:36.685 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.685 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.685 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.686 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "local", 2023-09-13T04:50:36.686 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.686 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.686 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.686 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-09-13T04:50:36.687 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.687 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.687 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.688 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.688 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "selftest", 2023-09-13T04:50:36.688 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.689 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.689 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.689 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.689 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.689 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.690 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.690 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.690 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.690 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.690 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.691 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.691 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.691 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.691 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.692 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.692 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.692 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.692 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.692 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.693 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.693 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.693 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.693 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.694 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.694 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.694 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.694 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.694 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.695 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.695 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.695 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.695 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.696 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.696 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.696 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.696 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.697 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.697 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.697 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.697 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.697 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.698 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.698 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.698 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.698 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.698 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.699 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.699 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.699 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.699 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.699 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.700 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.700 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.700 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.700 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.700 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.701 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.701 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.701 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.701 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.701 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.702 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.702 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.702 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.702 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.702 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.703 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.703 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.703 INFO:teuthology.orchestra.run.smithi002.stdout: "roption1": { 2023-09-13T04:50:36.703 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption1", 2023-09-13T04:50:36.704 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.704 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.704 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.704 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.704 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.705 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.705 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.705 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.705 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.705 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.706 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.706 INFO:teuthology.orchestra.run.smithi002.stdout: "roption2": { 2023-09-13T04:50:36.706 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption2", 2023-09-13T04:50:36.707 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.707 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.707 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.707 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "xyz", 2023-09-13T04:50:36.707 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.708 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.708 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.708 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.708 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.708 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.708 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.709 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption1": { 2023-09-13T04:50:36.709 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption1", 2023-09-13T04:50:36.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.709 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.710 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.710 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.710 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.710 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.710 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.711 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.711 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.711 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.711 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.712 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption2": { 2023-09-13T04:50:36.712 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption2", 2023-09-13T04:50:36.712 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.712 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.712 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.713 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.713 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.713 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.713 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.713 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.714 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.714 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.714 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.714 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption3": { 2023-09-13T04:50:36.715 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption3", 2023-09-13T04:50:36.715 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-09-13T04:50:36.715 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.715 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.715 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.716 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.716 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.716 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.716 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.716 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.717 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.717 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.717 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption4": { 2023-09-13T04:50:36.717 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption4", 2023-09-13T04:50:36.718 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.718 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.718 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.718 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.718 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.719 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.719 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.719 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.719 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.719 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.720 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.720 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption5": { 2023-09-13T04:50:36.720 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption5", 2023-09-13T04:50:36.720 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.721 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.721 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.721 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.721 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.721 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.722 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.722 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.722 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.722 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.722 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.723 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption6": { 2023-09-13T04:50:36.723 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption6", 2023-09-13T04:50:36.723 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.723 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.723 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.724 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.724 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.724 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.724 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.724 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.725 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.725 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.725 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.725 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption7": { 2023-09-13T04:50:36.726 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption7", 2023-09-13T04:50:36.726 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.726 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.726 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.726 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.727 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2023-09-13T04:50:36.727 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "42", 2023-09-13T04:50:36.727 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.727 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.727 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.728 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.728 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.728 INFO:teuthology.orchestra.run.smithi002.stdout: "testkey": { 2023-09-13T04:50:36.728 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testkey", 2023-09-13T04:50:36.729 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.729 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.729 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.729 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.729 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.730 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.730 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.730 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.730 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.730 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.731 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.731 INFO:teuthology.orchestra.run.smithi002.stdout: "testlkey": { 2023-09-13T04:50:36.731 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testlkey", 2023-09-13T04:50:36.731 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.731 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.732 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.732 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.732 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.732 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.732 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.733 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.733 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.733 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.733 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.734 INFO:teuthology.orchestra.run.smithi002.stdout: "testnewline": { 2023-09-13T04:50:36.734 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testnewline", 2023-09-13T04:50:36.734 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.734 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.734 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.734 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.735 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.735 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.735 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.735 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.736 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.736 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.736 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.736 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.736 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.737 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snap_schedule", 2023-09-13T04:50:36.737 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.737 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.738 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.738 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_m_granularity": { 2023-09-13T04:50:36.738 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_m_granularity", 2023-09-13T04:50:36.738 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.739 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.739 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.739 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.739 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.739 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.740 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.740 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow minute scheduled snapshots", 2023-09-13T04:50:36.740 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.740 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.740 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.741 INFO:teuthology.orchestra.run.smithi002.stdout: "dump_on_update": { 2023-09-13T04:50:36.741 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dump_on_update", 2023-09-13T04:50:36.741 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.741 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.742 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.742 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.742 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.742 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.743 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.743 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "dump database to debug log on update", 2023-09-13T04:50:36.743 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.743 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.743 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.744 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.744 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.744 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.744 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.745 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.745 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.745 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.745 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.745 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.746 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.746 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.746 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.746 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.747 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.747 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.747 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.747 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.747 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.748 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.748 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.748 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.748 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.749 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.749 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.749 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.749 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.749 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.749 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.750 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.750 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.750 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.750 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.750 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.751 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.751 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.751 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.752 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.752 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.752 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.752 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.752 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.753 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.753 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.753 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.753 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.753 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.754 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.754 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.754 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.754 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.754 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.755 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.755 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.755 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.755 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.756 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.756 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.756 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.756 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.756 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.757 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.757 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.757 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.757 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.757 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.758 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.758 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.758 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.759 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stats", 2023-09-13T04:50:36.759 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.759 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.759 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.759 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.760 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.760 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.760 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.760 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.760 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.761 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.761 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.761 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.761 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.761 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.762 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.762 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.762 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.762 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.762 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.763 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.763 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.763 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.763 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.764 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.764 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.764 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.764 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.764 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.765 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.765 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.765 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.765 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.765 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.766 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.766 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.766 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.766 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.767 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.767 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.767 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.767 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.767 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.768 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.768 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.768 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.768 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.768 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.769 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.769 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.769 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.769 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.769 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.770 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.770 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.770 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.770 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.771 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.771 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.771 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.771 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.772 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.772 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.772 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.772 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.772 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.773 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.773 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.773 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.773 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.773 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.774 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.774 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "status", 2023-09-13T04:50:36.774 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.775 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.775 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.775 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.775 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.775 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.776 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.776 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.776 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.776 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.777 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.777 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.777 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.777 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.777 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.778 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.778 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.778 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.778 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.779 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.779 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.779 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.779 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.780 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.780 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.780 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.780 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.780 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.781 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.781 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.781 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.781 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.781 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.782 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.782 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.782 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.782 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.783 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.783 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.783 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.783 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.784 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.784 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.784 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.784 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.784 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.784 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.785 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.785 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.785 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.785 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.786 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.786 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.786 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.786 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.786 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.787 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.787 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.787 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.787 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.788 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.788 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.788 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.788 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.788 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.789 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.789 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.789 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.789 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.790 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.790 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.790 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telegraf", 2023-09-13T04:50:36.791 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.791 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.791 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.791 INFO:teuthology.orchestra.run.smithi002.stdout: "address": { 2023-09-13T04:50:36.792 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "address", 2023-09-13T04:50:36.792 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.792 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.792 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.792 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-09-13T04:50:36.793 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.793 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.793 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.793 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.793 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.794 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.794 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.794 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-09-13T04:50:36.795 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-09-13T04:50:36.795 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:36.795 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.795 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.795 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15", 2023-09-13T04:50:36.796 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.796 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.796 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.796 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.796 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.797 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.797 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.797 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.797 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.798 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.798 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.798 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.798 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.798 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.799 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.799 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.799 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.799 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.799 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.799 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.800 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.800 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.800 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.800 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.800 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.801 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.801 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.801 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.801 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.802 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.802 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.802 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.802 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.802 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.803 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.803 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.803 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.803 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.803 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.804 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.804 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.804 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.804 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.804 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.805 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.805 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.805 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.805 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.806 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.806 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.806 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.806 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.806 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.807 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.807 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.807 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.807 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.807 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.808 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.808 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.808 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.808 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.809 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.809 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.809 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.809 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.809 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.810 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.810 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.810 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.810 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.810 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.811 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.811 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.811 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.812 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telemetry", 2023-09-13T04:50:36.812 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.812 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.812 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.812 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_basic": { 2023-09-13T04:50:36.813 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_basic", 2023-09-13T04:50:36.813 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.813 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.813 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.813 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.813 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.814 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.814 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.814 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share basic cluster information (size, version)", 2023-09-13T04:50:36.814 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.814 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.815 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.815 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_crash": { 2023-09-13T04:50:36.815 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_crash", 2023-09-13T04:50:36.815 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.816 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.816 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.816 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.816 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.816 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.817 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.817 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-09-13T04:50:36.817 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.817 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.817 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.818 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_device": { 2023-09-13T04:50:36.818 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_device", 2023-09-13T04:50:36.818 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.818 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.819 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.819 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.819 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.819 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.819 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.820 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-09-13T04:50:36.820 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.820 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.820 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.821 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_ident": { 2023-09-13T04:50:36.821 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_ident", 2023-09-13T04:50:36.821 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.821 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.822 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.822 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.822 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.822 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.823 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.823 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-09-13T04:50:36.823 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.823 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.823 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.824 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_perf": { 2023-09-13T04:50:36.824 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_perf", 2023-09-13T04:50:36.824 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.824 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.825 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.825 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.825 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.825 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.825 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.826 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share various performance metrics of a cluster", 2023-09-13T04:50:36.826 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.826 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.826 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.827 INFO:teuthology.orchestra.run.smithi002.stdout: "contact": { 2023-09-13T04:50:36.827 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "contact", 2023-09-13T04:50:36.827 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.827 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.828 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.828 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.828 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.828 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.828 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.829 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.829 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.829 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.829 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.830 INFO:teuthology.orchestra.run.smithi002.stdout: "description": { 2023-09-13T04:50:36.830 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "description", 2023-09-13T04:50:36.830 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.830 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.830 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.831 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.831 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.831 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.831 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.831 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.832 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.832 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.832 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.832 INFO:teuthology.orchestra.run.smithi002.stdout: "device_url": { 2023-09-13T04:50:36.833 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_url", 2023-09-13T04:50:36.833 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.833 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.833 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.833 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-09-13T04:50:36.834 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.834 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.834 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.834 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.834 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.835 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.835 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.837 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2023-09-13T04:50:36.837 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2023-09-13T04:50:36.837 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.837 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.837 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.838 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.838 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.838 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.838 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.839 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.839 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.839 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.839 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.840 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-09-13T04:50:36.840 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-09-13T04:50:36.840 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.840 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.841 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.841 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "24", 2023-09-13T04:50:36.841 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "8", 2023-09-13T04:50:36.841 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.841 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.842 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.842 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.842 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.842 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.843 INFO:teuthology.orchestra.run.smithi002.stdout: "last_opt_revision": { 2023-09-13T04:50:36.843 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "last_opt_revision", 2023-09-13T04:50:36.843 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.843 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.844 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.844 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1", 2023-09-13T04:50:36.844 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.844 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.844 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.845 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.845 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.845 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.845 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.846 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard": { 2023-09-13T04:50:36.846 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard", 2023-09-13T04:50:36.846 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.846 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.846 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.847 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.847 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.847 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.847 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.847 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.848 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.848 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.848 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.848 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard_description": { 2023-09-13T04:50:36.849 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard_description", 2023-09-13T04:50:36.849 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.849 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.849 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.849 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.850 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.850 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.850 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.850 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.851 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.851 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.851 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.851 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.852 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.852 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.852 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.852 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.852 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.853 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.853 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.853 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.853 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.853 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.854 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.854 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.854 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.854 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.854 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.854 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.855 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.855 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.855 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.855 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.856 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.856 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.856 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.856 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.856 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.857 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.857 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.857 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.857 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.857 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.857 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.858 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.858 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.858 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.858 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.859 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.859 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.859 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.859 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.859 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.860 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.860 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.860 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.860 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.860 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.861 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.861 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.861 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.861 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.861 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.862 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.862 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.862 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.862 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.863 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.863 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.863 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.863 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.863 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.864 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.864 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.864 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.864 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.864 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.865 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.865 INFO:teuthology.orchestra.run.smithi002.stdout: "organization": { 2023-09-13T04:50:36.865 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "organization", 2023-09-13T04:50:36.865 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.865 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.866 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.866 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.866 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.866 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.866 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.867 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.867 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.867 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.867 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: "proxy": { 2023-09-13T04:50:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "proxy", 2023-09-13T04:50:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: "url": { 2023-09-13T04:50:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url", 2023-09-13T04:50:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-09-13T04:50:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "test_orchestrator", 2023-09-13T04:50:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "volumes", 2023-09-13T04:50:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_clones": { 2023-09-13T04:50:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_clones", 2023-09-13T04:50:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4", 2023-09-13T04:50:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Number of asynchronous cloner threads", 2023-09-13T04:50:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_delay": { 2023-09-13T04:50:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_delay", 2023-09-13T04:50:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-09-13T04:50:36.908 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.908 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.908 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.908 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-09-13T04:50:36.908 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.909 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.909 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.909 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.909 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.910 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix", 2023-09-13T04:50:36.910 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.910 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.910 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.911 INFO:teuthology.orchestra.run.smithi002.stdout: "discovery_interval": { 2023-09-13T04:50:36.911 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "discovery_interval", 2023-09-13T04:50:36.911 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-09-13T04:50:36.911 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.911 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.911 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "100", 2023-09-13T04:50:36.912 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.912 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.912 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.912 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.912 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.913 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.913 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.913 INFO:teuthology.orchestra.run.smithi002.stdout: "identifier": { 2023-09-13T04:50:36.913 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "identifier", 2023-09-13T04:50:36.914 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.914 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.914 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.914 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.914 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.915 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.915 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.915 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.915 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.915 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.915 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.916 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-09-13T04:50:36.916 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-09-13T04:50:36.916 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:36.917 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.917 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.917 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-09-13T04:50:36.917 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.917 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.917 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.918 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.918 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.918 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.918 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.919 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.919 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.919 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.919 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.919 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.920 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.920 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.920 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.920 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.920 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.921 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.921 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.921 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.921 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.921 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.922 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.922 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.922 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.922 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.923 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.923 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.923 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.923 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.923 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.924 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.924 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.924 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.924 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.924 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.925 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.925 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.925 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.925 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.926 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.926 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.926 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.926 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.926 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.927 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.927 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.927 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.927 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.928 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.928 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.928 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.928 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.928 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.928 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.929 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.929 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.929 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.929 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.930 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.930 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.930 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.930 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.931 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.931 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.931 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.931 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.931 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.931 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.932 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.932 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.932 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.932 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.933 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_host": { 2023-09-13T04:50:36.933 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_host", 2023-09-13T04:50:36.933 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.933 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.933 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.934 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.934 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.934 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.934 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.935 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.935 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.935 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.935 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.936 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_port": { 2023-09-13T04:50:36.936 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_port", 2023-09-13T04:50:36.936 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.936 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.936 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.936 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10051", 2023-09-13T04:50:36.937 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.937 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.937 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.937 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.937 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.938 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.938 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.938 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_sender": { 2023-09-13T04:50:36.939 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_sender", 2023-09-13T04:50:36.939 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.939 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.939 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:36.939 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-09-13T04:50:36.940 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.940 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.940 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.940 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.940 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.940 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.941 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.941 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.941 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.941 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.942 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2023-09-13T04:50:36.942 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.942 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.942 INFO:teuthology.orchestra.run.smithi002.stdout: "modules": [ 2023-09-13T04:50:36.942 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm", 2023-09-13T04:50:36.943 INFO:teuthology.orchestra.run.smithi002.stdout: "dashboard", 2023-09-13T04:50:36.943 INFO:teuthology.orchestra.run.smithi002.stdout: "iostat", 2023-09-13T04:50:36.943 INFO:teuthology.orchestra.run.smithi002.stdout: "nfs", 2023-09-13T04:50:36.943 INFO:teuthology.orchestra.run.smithi002.stdout: "restful", 2023-09-13T04:50:36.943 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_schedule" 2023-09-13T04:50:36.944 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.944 INFO:teuthology.orchestra.run.smithi002.stdout: "available_modules": [ 2023-09-13T04:50:36.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.944 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "alerts", 2023-09-13T04:50:36.944 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.945 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.945 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.945 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-09-13T04:50:36.945 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-09-13T04:50:36.945 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:36.946 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.946 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.946 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-09-13T04:50:36.946 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.946 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.946 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.947 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How frequently to reexamine health status", 2023-09-13T04:50:36.947 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.947 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.947 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.948 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:36.948 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:36.948 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.948 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.948 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.949 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.949 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.949 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.949 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.949 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.950 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.950 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.950 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.950 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.950 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.951 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.951 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.951 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.951 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.951 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.952 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:36.952 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:36.952 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.952 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.952 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.953 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.953 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.953 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.953 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.953 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.954 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.954 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.954 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.954 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:36.955 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:36.955 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.955 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.955 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.955 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:36.956 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.956 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.956 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:36.956 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:36.956 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:36.957 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:36.957 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:36.957 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:36.957 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:36.957 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:36.957 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.958 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.958 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.958 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.958 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:36.959 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:36.959 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.959 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.959 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.959 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:36.960 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.960 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.960 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.960 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:36.960 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.961 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.961 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.961 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_destination": { 2023-09-13T04:50:36.961 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_destination", 2023-09-13T04:50:36.962 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.962 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.962 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.962 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.962 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.963 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.963 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.963 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email address to send alerts to", 2023-09-13T04:50:36.963 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.963 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.963 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.964 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_from_name": { 2023-09-13T04:50:36.964 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_from_name", 2023-09-13T04:50:36.964 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.964 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.965 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.965 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "Ceph", 2023-09-13T04:50:36.965 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.965 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.965 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.966 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email From: name", 2023-09-13T04:50:36.966 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.966 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.966 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.967 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_host": { 2023-09-13T04:50:36.967 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_host", 2023-09-13T04:50:36.967 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.967 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.967 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.968 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.968 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.968 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.968 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.968 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP server", 2023-09-13T04:50:36.969 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.969 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.969 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.969 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_password": { 2023-09-13T04:50:36.969 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_password", 2023-09-13T04:50:36.970 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.970 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.970 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.970 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.970 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.971 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.971 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.971 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Password to authenticate with", 2023-09-13T04:50:36.971 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.972 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.972 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.972 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_port": { 2023-09-13T04:50:36.972 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_port", 2023-09-13T04:50:36.973 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:36.973 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.973 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.973 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "465", 2023-09-13T04:50:36.973 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.974 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.974 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.974 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP port", 2023-09-13T04:50:36.974 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.974 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_sender": { 2023-09-13T04:50:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_sender", 2023-09-13T04:50:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP envelope sender", 2023-09-13T04:50:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.978 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_ssl": { 2023-09-13T04:50:36.978 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_ssl", 2023-09-13T04:50:36.978 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.978 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.979 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.979 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.979 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.979 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.979 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.980 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use SSL to connect to SMTP server", 2023-09-13T04:50:36.980 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.980 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.980 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.981 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_user": { 2023-09-13T04:50:36.981 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_user", 2023-09-13T04:50:36.981 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.981 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.981 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.982 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:36.982 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.982 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.982 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.982 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "User to authenticate as", 2023-09-13T04:50:36.983 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.983 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.983 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.983 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.983 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "balancer", 2023-09-13T04:50:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: "active": { 2023-09-13T04:50:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "active", 2023-09-13T04:50:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically balance PGs across cluster", 2023-09-13T04:50:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_time": { 2023-09-13T04:50:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_time", 2023-09-13T04:50:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0000", 2023-09-13T04:50:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "beginning time of day to automatically balance", 2023-09-13T04:50:36.990 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-13T04:50:36.990 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.990 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.990 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_weekday": { 2023-09-13T04:50:36.991 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_weekday", 2023-09-13T04:50:36.991 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-09-13T04:50:36.991 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.991 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.991 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-09-13T04:50:36.992 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2023-09-13T04:50:36.992 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2023-09-13T04:50:36.992 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.992 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-09-13T04:50:36.992 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-13T04:50:36.993 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.993 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.993 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_max_iterations": { 2023-09-13T04:50:36.993 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_max_iterations", 2023-09-13T04:50:36.994 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-09-13T04:50:36.994 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.994 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.994 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "25", 2023-09-13T04:50:36.994 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2023-09-13T04:50:36.995 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "250", 2023-09-13T04:50:36.995 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.995 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-09-13T04:50:36.995 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:36.995 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.996 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.996 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_metrics": { 2023-09-13T04:50:36.996 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_metrics", 2023-09-13T04:50:36.996 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:36.997 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:36.997 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:36.997 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "pgs,objects,bytes", 2023-09-13T04:50:36.997 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:36.997 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:36.998 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:36.998 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-09-13T04:50:36.998 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-09-13T04:50:36.998 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:36.999 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:36.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:36.999 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_step": { 2023-09-13T04:50:36.999 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_step", 2023-09-13T04:50:36.999 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-09-13T04:50:37.000 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.000 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.000 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.5", 2023-09-13T04:50:37.000 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0.001", 2023-09-13T04:50:37.000 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "0.999", 2023-09-13T04:50:37.001 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.001 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "aggressiveness of optimization", 2023-09-13T04:50:37.001 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-09-13T04:50:37.001 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.001 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.002 INFO:teuthology.orchestra.run.smithi002.stdout: "end_time": { 2023-09-13T04:50:37.002 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_time", 2023-09-13T04:50:37.002 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.002 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.003 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.003 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2359", 2023-09-13T04:50:37.003 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.003 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.003 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.004 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ending time of day to automatically balance", 2023-09-13T04:50:37.004 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-13T04:50:37.004 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.004 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.005 INFO:teuthology.orchestra.run.smithi002.stdout: "end_weekday": { 2023-09-13T04:50:37.005 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_weekday", 2023-09-13T04:50:37.005 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-09-13T04:50:37.005 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.005 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.006 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-09-13T04:50:37.006 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2023-09-13T04:50:37.006 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2023-09-13T04:50:37.006 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.006 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-09-13T04:50:37.007 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-13T04:50:37.007 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.007 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.007 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.008 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.008 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.008 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.008 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.008 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.009 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.009 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.009 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.009 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.009 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.010 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.010 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.010 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.010 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.010 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.011 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.011 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.011 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.011 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.012 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.012 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.012 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.012 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.012 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.013 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.013 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.013 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.013 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.013 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.014 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.014 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.014 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.014 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.015 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.015 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.015 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.015 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.015 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.016 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.016 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.016 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.016 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.016 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.017 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.017 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.017 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.017 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.017 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.018 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.018 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.018 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.018 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.019 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.019 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.019 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.019 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.019 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.020 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.020 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.020 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.020 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.020 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.020 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.021 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.021 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.021 INFO:teuthology.orchestra.run.smithi002.stdout: "min_score": { 2023-09-13T04:50:37.021 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_score", 2023-09-13T04:50:37.022 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-09-13T04:50:37.022 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.022 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.022 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-09-13T04:50:37.022 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.023 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.023 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.023 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-09-13T04:50:37.023 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.023 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.024 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.024 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2023-09-13T04:50:37.024 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2023-09-13T04:50:37.024 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.025 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.025 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.025 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "upmap", 2023-09-13T04:50:37.025 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.025 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.025 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.026 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-compat", 2023-09-13T04:50:37.026 INFO:teuthology.orchestra.run.smithi002.stdout: "none", 2023-09-13T04:50:37.026 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap" 2023-09-13T04:50:37.026 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.027 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Balancer mode", 2023-09-13T04:50:37.027 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.027 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.027 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.028 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_ids": { 2023-09-13T04:50:37.028 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_ids", 2023-09-13T04:50:37.028 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.028 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.029 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.029 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.029 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.029 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.029 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.030 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-09-13T04:50:37.030 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.030 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.030 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.031 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-09-13T04:50:37.031 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-09-13T04:50:37.031 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:37.031 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.031 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.032 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-09-13T04:50:37.032 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.032 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.032 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.033 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-09-13T04:50:37.033 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.033 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.033 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.034 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_deviation": { 2023-09-13T04:50:37.034 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_deviation", 2023-09-13T04:50:37.034 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.034 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.034 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.035 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2023-09-13T04:50:37.035 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2023-09-13T04:50:37.035 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.035 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.035 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "deviation below which no optimization is attempted", 2023-09-13T04:50:37.036 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-09-13T04:50:37.036 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.036 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.036 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_optimizations": { 2023-09-13T04:50:37.037 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_optimizations", 2023-09-13T04:50:37.037 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-09-13T04:50:37.037 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.037 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.038 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-09-13T04:50:37.038 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.038 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.038 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.038 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-09-13T04:50:37.039 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.039 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.039 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.039 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.039 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.040 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm", 2023-09-13T04:50:37.040 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.040 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.041 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.041 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_down_multiplier": { 2023-09-13T04:50:37.041 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_down_multiplier", 2023-09-13T04:50:37.041 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-09-13T04:50:37.041 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.042 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.042 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2023-09-13T04:50:37.042 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.042 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.042 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.043 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-09-13T04:50:37.045 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.045 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.046 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.046 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_refresh_rate": { 2023-09-13T04:50:37.046 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_refresh_rate", 2023-09-13T04:50:37.046 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:37.047 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.047 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.047 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "20", 2023-09-13T04:50:37.047 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.047 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.048 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.048 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-09-13T04:50:37.048 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.048 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.048 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.049 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_starting_port": { 2023-09-13T04:50:37.049 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_starting_port", 2023-09-13T04:50:37.049 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.049 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.050 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.050 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4721", 2023-09-13T04:50:37.050 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.050 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.050 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.051 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-09-13T04:50:37.051 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.051 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.051 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.052 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_ptrace": { 2023-09-13T04:50:37.052 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_ptrace", 2023-09-13T04:50:37.052 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.052 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.052 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.053 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.053 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.053 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.053 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.053 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-09-13T04:50:37.054 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-09-13T04:50:37.054 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.054 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.054 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_interval": { 2023-09-13T04:50:37.055 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_interval", 2023-09-13T04:50:37.055 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:37.055 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.055 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.055 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-09-13T04:50:37.056 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.056 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.056 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.056 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to autotune daemon memory", 2023-09-13T04:50:37.056 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.057 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.057 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.057 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_memory_target_ratio": { 2023-09-13T04:50:37.057 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_memory_target_ratio", 2023-09-13T04:50:37.058 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-09-13T04:50:37.058 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.058 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.058 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.7", 2023-09-13T04:50:37.058 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.059 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.059 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.059 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-09-13T04:50:37.059 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.059 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.060 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.060 INFO:teuthology.orchestra.run.smithi002.stdout: "cgroups_split": { 2023-09-13T04:50:37.060 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cgroups_split", 2023-09-13T04:50:37.060 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.061 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.061 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.061 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.061 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.061 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.061 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.062 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-09-13T04:50:37.062 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.062 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.062 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.063 INFO:teuthology.orchestra.run.smithi002.stdout: "config_checks_enabled": { 2023-09-13T04:50:37.063 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_checks_enabled", 2023-09-13T04:50:37.063 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.063 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.063 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.064 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.064 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.064 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.064 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.064 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-09-13T04:50:37.065 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.065 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.065 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.065 INFO:teuthology.orchestra.run.smithi002.stdout: "config_dashboard": { 2023-09-13T04:50:37.066 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_dashboard", 2023-09-13T04:50:37.066 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.066 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.066 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.066 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.067 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.067 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.067 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.067 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-09-13T04:50:37.067 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.068 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.068 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.068 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_alertmanager": { 2023-09-13T04:50:37.068 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_alertmanager", 2023-09-13T04:50:37.069 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.069 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.069 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.069 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-09-13T04:50:37.069 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.069 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.070 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.070 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2023-09-13T04:50:37.070 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.070 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.070 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.071 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_base": { 2023-09-13T04:50:37.071 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_base", 2023-09-13T04:50:37.071 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.071 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.072 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.072 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph", 2023-09-13T04:50:37.072 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.072 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.072 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.073 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Container image name, without the tag", 2023-09-13T04:50:37.073 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.073 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.073 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.074 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_elasticsearch": { 2023-09-13T04:50:37.074 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_elasticsearch", 2023-09-13T04:50:37.074 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.074 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.075 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.075 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-09-13T04:50:37.075 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.075 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.075 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.075 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "elasticsearch container image", 2023-09-13T04:50:37.076 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.076 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.076 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.076 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_grafana": { 2023-09-13T04:50:37.077 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_grafana", 2023-09-13T04:50:37.077 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.077 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.077 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.077 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-09-13T04:50:37.078 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.078 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.078 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.078 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2023-09-13T04:50:37.078 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.079 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.079 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.079 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_haproxy": { 2023-09-13T04:50:37.079 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_haproxy", 2023-09-13T04:50:37.080 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.080 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.080 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.080 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-09-13T04:50:37.080 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.081 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.081 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.081 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "HAproxy container image", 2023-09-13T04:50:37.081 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.081 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.082 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.082 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_agent": { 2023-09-13T04:50:37.082 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_agent", 2023-09-13T04:50:37.082 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.083 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.083 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.083 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-09-13T04:50:37.083 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.083 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.084 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.084 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger agent container image", 2023-09-13T04:50:37.084 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.084 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.084 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.085 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_collector": { 2023-09-13T04:50:37.085 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_collector", 2023-09-13T04:50:37.085 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.085 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.085 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.086 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-09-13T04:50:37.086 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.086 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.086 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.087 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger collector container image", 2023-09-13T04:50:37.087 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.087 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.087 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.088 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_query": { 2023-09-13T04:50:37.088 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_query", 2023-09-13T04:50:37.088 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.088 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.088 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.089 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-09-13T04:50:37.089 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.089 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.089 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.090 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger query container image", 2023-09-13T04:50:37.090 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.090 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.090 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.091 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_keepalived": { 2023-09-13T04:50:37.091 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_keepalived", 2023-09-13T04:50:37.091 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.091 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.092 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.092 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-09-13T04:50:37.092 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.092 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.092 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.093 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Keepalived container image", 2023-09-13T04:50:37.093 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.093 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.093 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.094 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_loki": { 2023-09-13T04:50:37.094 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_loki", 2023-09-13T04:50:37.094 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.094 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.095 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.095 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-09-13T04:50:37.095 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.095 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.095 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.096 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Loki container image", 2023-09-13T04:50:37.096 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.096 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.096 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.097 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_node_exporter": { 2023-09-13T04:50:37.097 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_node_exporter", 2023-09-13T04:50:37.097 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.098 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.098 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.098 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-09-13T04:50:37.098 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.098 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.099 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.099 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2023-09-13T04:50:37.099 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.099 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.099 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.100 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_nvmeof": { 2023-09-13T04:50:37.100 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_nvmeof", 2023-09-13T04:50:37.100 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.100 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.101 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.101 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-09-13T04:50:37.101 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.101 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.101 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.102 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Nvme-of container image", 2023-09-13T04:50:37.102 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.102 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.102 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.103 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_prometheus": { 2023-09-13T04:50:37.103 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_prometheus", 2023-09-13T04:50:37.103 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.103 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.104 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.104 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-09-13T04:50:37.104 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.104 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.104 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.105 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2023-09-13T04:50:37.105 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.105 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.105 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.106 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_promtail": { 2023-09-13T04:50:37.106 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_promtail", 2023-09-13T04:50:37.106 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.106 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.106 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.107 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-09-13T04:50:37.107 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.107 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.107 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.107 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Promtail container image", 2023-09-13T04:50:37.107 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.108 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.108 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.108 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_snmp_gateway": { 2023-09-13T04:50:37.108 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_snmp_gateway", 2023-09-13T04:50:37.109 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.109 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.109 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.109 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-09-13T04:50:37.109 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.110 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.110 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.110 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SNMP Gateway container image", 2023-09-13T04:50:37.110 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.110 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.111 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.111 INFO:teuthology.orchestra.run.smithi002.stdout: "container_init": { 2023-09-13T04:50:37.111 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_init", 2023-09-13T04:50:37.111 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.112 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.112 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.112 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.112 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.112 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.113 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.113 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Run podman/docker with `--init`", 2023-09-13T04:50:37.113 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.113 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.113 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.114 INFO:teuthology.orchestra.run.smithi002.stdout: "daemon_cache_timeout": { 2023-09-13T04:50:37.114 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "daemon_cache_timeout", 2023-09-13T04:50:37.114 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:37.114 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.115 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.115 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-09-13T04:50:37.115 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.115 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.115 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.115 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-09-13T04:50:37.116 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.116 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.116 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.116 INFO:teuthology.orchestra.run.smithi002.stdout: "default_cephadm_command_timeout": { 2023-09-13T04:50:37.117 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_cephadm_command_timeout", 2023-09-13T04:50:37.117 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:37.117 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.117 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.117 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "900", 2023-09-13T04:50:37.118 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.118 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.118 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.118 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-09-13T04:50:37.119 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.119 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.119 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.119 INFO:teuthology.orchestra.run.smithi002.stdout: "default_registry": { 2023-09-13T04:50:37.120 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_registry", 2023-09-13T04:50:37.120 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.120 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.120 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.120 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io", 2023-09-13T04:50:37.121 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.121 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.121 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.121 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-09-13T04:50:37.121 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.122 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.122 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.122 INFO:teuthology.orchestra.run.smithi002.stdout: "device_cache_timeout": { 2023-09-13T04:50:37.122 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_cache_timeout", 2023-09-13T04:50:37.123 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:37.123 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.123 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.123 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1800", 2023-09-13T04:50:37.123 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.124 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.124 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.124 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache device inventory", 2023-09-13T04:50:37.124 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.124 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.125 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.125 INFO:teuthology.orchestra.run.smithi002.stdout: "device_enhanced_scan": { 2023-09-13T04:50:37.125 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_enhanced_scan", 2023-09-13T04:50:37.125 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.125 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.126 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.126 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.126 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.126 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.126 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.127 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use libstoragemgmt during device scans", 2023-09-13T04:50:37.127 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.127 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.127 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.128 INFO:teuthology.orchestra.run.smithi002.stdout: "facts_cache_timeout": { 2023-09-13T04:50:37.128 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "facts_cache_timeout", 2023-09-13T04:50:37.128 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:37.128 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.128 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.128 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-09-13T04:50:37.129 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.129 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.129 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.129 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache host facts data", 2023-09-13T04:50:37.129 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.130 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.130 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.130 INFO:teuthology.orchestra.run.smithi002.stdout: "host_check_interval": { 2023-09-13T04:50:37.130 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "host_check_interval", 2023-09-13T04:50:37.131 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:37.131 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.131 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.131 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-09-13T04:50:37.131 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.132 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.132 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.132 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to perform a host check", 2023-09-13T04:50:37.132 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.132 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.133 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.133 INFO:teuthology.orchestra.run.smithi002.stdout: "inventory_list_all": { 2023-09-13T04:50:37.133 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "inventory_list_all", 2023-09-13T04:50:37.133 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.134 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.134 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.134 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.134 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.134 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.135 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.135 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-09-13T04:50:37.135 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.135 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.135 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.136 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.136 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.136 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.136 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.137 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.137 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.137 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.137 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.137 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.138 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.138 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.138 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.138 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.138 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.139 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.139 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.139 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.139 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.139 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.140 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.140 INFO:teuthology.orchestra.run.smithi002.stdout: "log_refresh_metadata": { 2023-09-13T04:50:37.140 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_refresh_metadata", 2023-09-13T04:50:37.140 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.141 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.141 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.141 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.141 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.141 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.141 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.142 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-09-13T04:50:37.142 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.142 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.142 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.143 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.143 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.143 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.143 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.143 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.144 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.144 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.144 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.144 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.144 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-09-13T04:50:37.145 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.145 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.145 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.145 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.146 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.146 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.146 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.146 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.146 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.146 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.147 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.147 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.147 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.147 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.147 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.148 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.148 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.148 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.148 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.148 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.149 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.149 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.149 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.149 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.150 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.150 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.150 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.150 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.150 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.151 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.151 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.151 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.151 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.151 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.151 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.152 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.152 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf": { 2023-09-13T04:50:37.152 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-09-13T04:50:37.152 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.153 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.153 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.153 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.153 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.153 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.154 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.154 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-09-13T04:50:37.154 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.154 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.154 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.155 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-09-13T04:50:37.155 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-09-13T04:50:37.155 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.156 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.156 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.156 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "*", 2023-09-13T04:50:37.156 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.156 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.156 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.157 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-09-13T04:50:37.157 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.157 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.157 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.158 INFO:teuthology.orchestra.run.smithi002.stdout: "max_count_per_host": { 2023-09-13T04:50:37.158 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_count_per_host", 2023-09-13T04:50:37.158 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.158 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.158 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.159 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-09-13T04:50:37.159 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.159 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.159 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.159 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of daemons per service per host", 2023-09-13T04:50:37.160 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.160 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.160 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.160 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd_draining_count": { 2023-09-13T04:50:37.161 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_osd_draining_count", 2023-09-13T04:50:37.161 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.161 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.161 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.161 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-09-13T04:50:37.162 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.162 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.162 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.162 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-09-13T04:50:37.162 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.163 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.163 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.163 INFO:teuthology.orchestra.run.smithi002.stdout: "migration_current": { 2023-09-13T04:50:37.163 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "migration_current", 2023-09-13T04:50:37.164 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.164 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.164 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.164 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.164 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.165 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.165 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.165 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "internal - do not modify", 2023-09-13T04:50:37.165 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.165 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.165 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.166 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2023-09-13T04:50:37.166 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2023-09-13T04:50:37.166 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.167 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.167 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.167 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "root", 2023-09-13T04:50:37.167 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.167 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.167 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.168 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm-package", 2023-09-13T04:50:37.168 INFO:teuthology.orchestra.run.smithi002.stdout: "root" 2023-09-13T04:50:37.168 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.168 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "mode for remote execution of cephadm", 2023-09-13T04:50:37.168 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.169 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.169 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.169 INFO:teuthology.orchestra.run.smithi002.stdout: "prometheus_alerts_path": { 2023-09-13T04:50:37.169 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus_alerts_path", 2023-09-13T04:50:37.170 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.170 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.170 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.170 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-09-13T04:50:37.170 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.171 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.171 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.171 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-09-13T04:50:37.171 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.171 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.172 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.172 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_insecure": { 2023-09-13T04:50:37.172 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_insecure", 2023-09-13T04:50:37.172 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.173 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.173 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.173 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.173 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.173 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.174 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.174 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-09-13T04:50:37.174 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.174 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.174 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.175 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_password": { 2023-09-13T04:50:37.175 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_password", 2023-09-13T04:50:37.175 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.175 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.176 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.176 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.176 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.176 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.176 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.177 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-09-13T04:50:37.177 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.177 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.177 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.178 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_url": { 2023-09-13T04:50:37.178 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_url", 2023-09-13T04:50:37.178 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.178 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.178 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.179 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.179 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.179 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.179 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.179 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-09-13T04:50:37.180 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.180 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.180 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.180 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_username": { 2023-09-13T04:50:37.181 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_username", 2023-09-13T04:50:37.181 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.181 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.181 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.181 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.182 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.182 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.182 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.182 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-09-13T04:50:37.182 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.183 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.183 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.183 INFO:teuthology.orchestra.run.smithi002.stdout: "secure_monitoring_stack": { 2023-09-13T04:50:37.183 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "secure_monitoring_stack", 2023-09-13T04:50:37.183 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.184 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.184 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.184 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.184 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.184 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.185 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.185 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-09-13T04:50:37.185 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.185 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.185 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.186 INFO:teuthology.orchestra.run.smithi002.stdout: "service_discovery_port": { 2023-09-13T04:50:37.186 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "service_discovery_port", 2023-09-13T04:50:37.186 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.186 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.187 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.187 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8765", 2023-09-13T04:50:37.187 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.187 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.187 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.188 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "cephadm service discovery port", 2023-09-13T04:50:37.188 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.188 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.188 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.189 INFO:teuthology.orchestra.run.smithi002.stdout: "ssh_config_file": { 2023-09-13T04:50:37.189 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssh_config_file", 2023-09-13T04:50:37.189 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.189 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.189 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.190 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.190 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.190 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.190 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.190 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-09-13T04:50:37.191 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.191 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.191 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.191 INFO:teuthology.orchestra.run.smithi002.stdout: "use_agent": { 2023-09-13T04:50:37.192 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_agent", 2023-09-13T04:50:37.192 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.192 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.192 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.192 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.193 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.193 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.193 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.193 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-09-13T04:50:37.194 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.194 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.194 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.194 INFO:teuthology.orchestra.run.smithi002.stdout: "use_repo_digest": { 2023-09-13T04:50:37.195 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_repo_digest", 2023-09-13T04:50:37.195 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.195 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.195 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.195 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.196 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.196 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.196 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.196 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-09-13T04:50:37.196 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.197 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.197 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.197 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_failed_host_check": { 2023-09-13T04:50:37.197 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_failed_host_check", 2023-09-13T04:50:37.197 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.198 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.198 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.198 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.198 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.198 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.199 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.199 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if the host check fails", 2023-09-13T04:50:37.199 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.199 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.199 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.200 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_daemons": { 2023-09-13T04:50:37.200 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_daemons", 2023-09-13T04:50:37.200 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.200 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.201 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.201 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.201 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.201 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.201 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.202 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-09-13T04:50:37.202 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.202 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.202 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.202 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_hosts": { 2023-09-13T04:50:37.203 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_hosts", 2023-09-13T04:50:37.203 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.203 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.203 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.203 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.204 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.204 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.204 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.204 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-09-13T04:50:37.204 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.205 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.205 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.205 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.205 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.206 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crash", 2023-09-13T04:50:37.206 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.206 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.206 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.207 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.207 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.207 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.207 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.207 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.207 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.208 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.208 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.208 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.208 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.208 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.209 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.209 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.209 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.209 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.209 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.210 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.210 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.210 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.210 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.211 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.211 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.211 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.211 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.211 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.212 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.212 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.212 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.212 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.212 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.213 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.213 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.213 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.213 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.214 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.214 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.214 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.214 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.214 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.215 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.215 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.215 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.215 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.215 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.216 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.216 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.216 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.216 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.216 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.217 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.217 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.217 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.217 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.218 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.218 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.218 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.218 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.218 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.219 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.219 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.219 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.219 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.219 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.220 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.220 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.220 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.220 INFO:teuthology.orchestra.run.smithi002.stdout: "retain_interval": { 2023-09-13T04:50:37.220 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retain_interval", 2023-09-13T04:50:37.221 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:37.221 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.221 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.221 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "31536000", 2023-09-13T04:50:37.222 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.222 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.222 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.222 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain crashes before pruning them", 2023-09-13T04:50:37.222 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.223 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.223 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.223 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_recent_interval": { 2023-09-13T04:50:37.223 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_recent_interval", 2023-09-13T04:50:37.224 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:37.224 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.224 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.224 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1209600", 2023-09-13T04:50:37.224 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.225 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.225 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.225 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "time interval in which to warn about recent crashes", 2023-09-13T04:50:37.225 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.225 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.226 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.226 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.226 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.227 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dashboard", 2023-09-13T04:50:37.227 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.227 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.227 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.227 INFO:teuthology.orchestra.run.smithi002.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-09-13T04:50:37.228 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-09-13T04:50:37.228 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.228 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.228 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.228 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-09-13T04:50:37.229 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.229 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.229 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.229 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.229 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.230 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.230 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.230 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_HOST": { 2023-09-13T04:50:37.230 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_HOST", 2023-09-13T04:50:37.231 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.231 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.231 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.231 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.231 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.232 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.232 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.232 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.232 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.232 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.233 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.233 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-09-13T04:50:37.233 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-09-13T04:50:37.233 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.234 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.234 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.234 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.234 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.234 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.235 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.235 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.235 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.235 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.235 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.236 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_ENABLED": { 2023-09-13T04:50:37.236 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_ENABLED", 2023-09-13T04:50:37.236 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.236 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.237 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.237 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.237 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.237 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.237 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.238 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.238 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.238 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.238 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.239 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-09-13T04:50:37.239 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-09-13T04:50:37.239 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.239 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.239 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.240 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.240 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.240 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.240 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.240 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.241 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.241 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.241 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.241 INFO:teuthology.orchestra.run.smithi002.stdout: "ENABLE_BROWSABLE_API": { 2023-09-13T04:50:37.241 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ENABLE_BROWSABLE_API", 2023-09-13T04:50:37.242 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.242 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.242 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.242 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.242 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.243 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.243 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.243 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.243 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.243 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.244 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.244 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-09-13T04:50:37.244 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-09-13T04:50:37.244 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.245 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.245 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.245 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.245 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.245 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.246 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.246 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.246 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.246 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.246 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.247 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-09-13T04:50:37.247 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-09-13T04:50:37.247 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.249 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.249 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.249 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.249 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.249 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.249 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.250 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.250 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.250 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.250 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.251 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-09-13T04:50:37.251 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-09-13T04:50:37.251 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.251 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.251 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.252 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.252 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.252 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.252 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.252 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.253 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.253 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.253 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.253 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-09-13T04:50:37.254 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-09-13T04:50:37.254 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.254 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.254 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.254 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.255 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.255 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.255 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.255 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.255 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.255 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.256 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.256 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_NFS": { 2023-09-13T04:50:37.256 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-09-13T04:50:37.256 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.257 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.257 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.257 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.257 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.257 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.258 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.258 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.258 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.258 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.258 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.259 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RBD": { 2023-09-13T04:50:37.259 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-09-13T04:50:37.259 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.259 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.260 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.260 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.260 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.260 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.260 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.261 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.261 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.261 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.261 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.261 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RGW": { 2023-09-13T04:50:37.262 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-09-13T04:50:37.262 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.262 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.262 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.262 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.263 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.263 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.263 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.263 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.263 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.264 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.264 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.264 INFO:teuthology.orchestra.run.smithi002.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-09-13T04:50:37.264 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-09-13T04:50:37.265 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.265 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.265 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.265 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.265 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.265 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.266 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.266 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.266 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.266 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.267 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.267 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_PASSWORD": { 2023-09-13T04:50:37.267 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_PASSWORD", 2023-09-13T04:50:37.267 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.268 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.268 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.268 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-09-13T04:50:37.268 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.268 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.269 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.269 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.269 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.269 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.269 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.270 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-09-13T04:50:37.270 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-09-13T04:50:37.270 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.270 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.271 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.271 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.271 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.271 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.271 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.272 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.272 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.272 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.272 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.273 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_URL": { 2023-09-13T04:50:37.273 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_URL", 2023-09-13T04:50:37.273 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.273 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.273 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.274 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.274 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.274 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.274 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.274 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.275 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.275 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.275 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.275 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_USERNAME": { 2023-09-13T04:50:37.276 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_USERNAME", 2023-09-13T04:50:37.276 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.276 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.276 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.276 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-09-13T04:50:37.277 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.277 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.277 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.277 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.277 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.278 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.278 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.278 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-09-13T04:50:37.278 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-09-13T04:50:37.279 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.279 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.279 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.279 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.279 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.280 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.280 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.280 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.280 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.281 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.281 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.281 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-09-13T04:50:37.281 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-09-13T04:50:37.281 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.282 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.282 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.282 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.282 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.282 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.283 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.283 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.283 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.283 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.283 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.284 INFO:teuthology.orchestra.run.smithi002.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-09-13T04:50:37.284 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-09-13T04:50:37.284 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.284 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.285 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.285 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.285 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.285 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.285 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.286 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.286 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.286 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.286 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.287 INFO:teuthology.orchestra.run.smithi002.stdout: "ISSUE_TRACKER_API_KEY": { 2023-09-13T04:50:37.287 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-09-13T04:50:37.287 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.287 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.287 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.287 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.288 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.288 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.288 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.288 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.288 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.289 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.289 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.289 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_HOST": { 2023-09-13T04:50:37.289 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_HOST", 2023-09-13T04:50:37.290 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.290 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.290 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.290 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.290 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.291 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.291 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.291 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.291 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.291 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.292 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.292 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-09-13T04:50:37.292 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-09-13T04:50:37.292 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.292 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.293 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.293 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.293 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.293 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.293 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.294 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.294 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.294 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.294 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.295 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-09-13T04:50:37.295 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-09-13T04:50:37.295 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.295 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.295 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.296 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.296 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.296 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.296 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.296 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.297 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.297 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.297 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.297 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-09-13T04:50:37.298 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-09-13T04:50:37.298 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.298 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.298 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.298 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.299 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.299 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.299 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.299 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.299 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.300 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.300 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.300 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-09-13T04:50:37.300 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-09-13T04:50:37.300 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.301 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.301 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.301 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.301 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.301 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.302 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.302 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.302 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.302 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.302 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.303 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-09-13T04:50:37.303 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-09-13T04:50:37.303 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.303 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.304 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.304 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.304 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.304 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.304 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.305 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.305 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.305 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.305 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.306 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-09-13T04:50:37.306 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-09-13T04:50:37.306 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.306 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.306 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.307 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.307 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.307 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.307 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.307 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.308 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.308 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.308 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.308 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-09-13T04:50:37.309 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-09-13T04:50:37.309 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.309 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.309 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.309 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.309 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.310 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.310 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.310 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.310 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.310 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.311 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.311 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-09-13T04:50:37.311 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-09-13T04:50:37.311 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.312 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.312 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.312 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.312 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.313 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.313 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.313 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.313 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.313 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.313 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.314 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_ENABLED": { 2023-09-13T04:50:37.314 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_ENABLED", 2023-09-13T04:50:37.314 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.314 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.315 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.315 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.315 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.315 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.315 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.316 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.316 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.316 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.316 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.317 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-09-13T04:50:37.317 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-09-13T04:50:37.317 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.317 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.317 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.318 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-09-13T04:50:37.318 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.318 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.318 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.318 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.319 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.319 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.319 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.319 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-09-13T04:50:37.320 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-09-13T04:50:37.320 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.320 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.320 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.320 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-09-13T04:50:37.320 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.321 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.321 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.321 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.321 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.321 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.322 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.322 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-09-13T04:50:37.322 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-09-13T04:50:37.323 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.323 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.323 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.323 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8", 2023-09-13T04:50:37.323 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.324 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.324 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.324 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.324 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.324 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.324 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.325 INFO:teuthology.orchestra.run.smithi002.stdout: "REST_REQUESTS_TIMEOUT": { 2023-09-13T04:50:37.325 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-09-13T04:50:37.325 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.325 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.326 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.326 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "45", 2023-09-13T04:50:37.326 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.326 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.326 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.327 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.327 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.327 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.327 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.328 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ACCESS_KEY": { 2023-09-13T04:50:37.328 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ACCESS_KEY", 2023-09-13T04:50:37.328 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.328 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.329 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.329 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.329 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.329 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.329 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.330 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.330 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.330 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.330 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.331 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-09-13T04:50:37.331 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-09-13T04:50:37.331 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.331 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.331 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.331 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-09-13T04:50:37.332 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.332 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.332 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.332 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.332 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.333 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.333 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.333 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SECRET_KEY": { 2023-09-13T04:50:37.333 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SECRET_KEY", 2023-09-13T04:50:37.334 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.334 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.334 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.334 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.335 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.335 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.335 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.335 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.335 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.335 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.336 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.336 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SSL_VERIFY": { 2023-09-13T04:50:37.336 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SSL_VERIFY", 2023-09-13T04:50:37.336 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.337 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.337 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.337 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.337 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.338 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.338 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.338 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.338 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.338 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.339 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.339 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-09-13T04:50:37.339 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-09-13T04:50:37.339 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.339 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.340 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.340 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-09-13T04:50:37.340 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.340 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.340 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.341 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.341 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.341 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.341 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.342 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-09-13T04:50:37.342 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-09-13T04:50:37.342 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.342 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.342 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.343 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-09-13T04:50:37.343 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.343 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.343 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.343 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.344 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.344 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.344 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.344 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-09-13T04:50:37.345 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-09-13T04:50:37.345 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.345 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.345 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.345 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2023-09-13T04:50:37.346 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.346 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.346 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.346 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.346 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.347 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.347 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.347 INFO:teuthology.orchestra.run.smithi002.stdout: "cross_origin_url": { 2023-09-13T04:50:37.347 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cross_origin_url", 2023-09-13T04:50:37.348 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.348 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.348 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.348 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.348 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.349 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.349 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.349 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.349 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.349 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.349 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.350 INFO:teuthology.orchestra.run.smithi002.stdout: "crt_file": { 2023-09-13T04:50:37.350 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crt_file", 2023-09-13T04:50:37.350 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.350 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.351 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.351 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.351 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.351 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.351 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.352 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.352 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.352 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.352 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.353 INFO:teuthology.orchestra.run.smithi002.stdout: "debug": { 2023-09-13T04:50:37.353 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "debug", 2023-09-13T04:50:37.353 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.353 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.353 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.354 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.354 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.354 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.354 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.354 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable/disable debug options", 2023-09-13T04:50:37.355 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.355 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.355 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.355 INFO:teuthology.orchestra.run.smithi002.stdout: "jwt_token_ttl": { 2023-09-13T04:50:37.356 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "jwt_token_ttl", 2023-09-13T04:50:37.356 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.356 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.356 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.356 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "28800", 2023-09-13T04:50:37.356 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.357 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.357 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.357 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.357 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.357 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.358 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.358 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2023-09-13T04:50:37.358 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2023-09-13T04:50:37.358 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.359 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.359 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.359 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.359 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.359 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.360 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.360 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.360 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.360 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.360 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.361 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.361 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.361 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.361 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.362 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.362 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.362 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.362 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.362 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.363 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.363 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.363 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.363 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.363 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.364 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.364 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.364 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.364 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.364 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.365 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.365 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.365 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.365 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.366 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.366 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.366 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.366 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.366 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.367 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.367 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.367 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.367 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.367 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.368 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.368 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.368 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.368 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.368 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.369 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.369 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.369 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.369 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.369 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.371 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.371 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.371 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.371 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.371 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.372 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.372 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.372 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.372 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.373 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.373 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.373 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.373 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.373 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.374 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.374 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.374 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.374 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.374 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.375 INFO:teuthology.orchestra.run.smithi002.stdout: "motd": { 2023-09-13T04:50:37.375 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "motd", 2023-09-13T04:50:37.375 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.375 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.376 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.376 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.376 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.376 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.376 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.377 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "The message of the day", 2023-09-13T04:50:37.377 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.377 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.377 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.378 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect_resolve_ip_addr": { 2023-09-13T04:50:37.378 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "redirect_resolve_ip_addr", 2023-09-13T04:50:37.378 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.378 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.378 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.379 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.379 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.379 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.379 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.379 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.380 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.380 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.380 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.380 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2023-09-13T04:50:37.381 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2023-09-13T04:50:37.381 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.381 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.381 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.381 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2023-09-13T04:50:37.382 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.382 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.382 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.382 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.382 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.383 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.383 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.383 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2023-09-13T04:50:37.383 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2023-09-13T04:50:37.384 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.384 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.384 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.384 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8080", 2023-09-13T04:50:37.384 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.385 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.385 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.385 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.385 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.385 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.386 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.386 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2023-09-13T04:50:37.386 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2023-09-13T04:50:37.386 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.387 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.387 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.387 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.387 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.387 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.388 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.388 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.388 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.388 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.388 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.389 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl_server_port": { 2023-09-13T04:50:37.389 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl_server_port", 2023-09-13T04:50:37.389 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.389 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.389 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.390 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8443", 2023-09-13T04:50:37.390 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.390 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.390 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.391 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.391 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.391 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.391 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.392 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2023-09-13T04:50:37.392 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2023-09-13T04:50:37.392 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.392 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.392 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.392 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "redirect", 2023-09-13T04:50:37.393 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.393 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.393 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.393 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.393 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect" 2023-09-13T04:50:37.394 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.394 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.394 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.394 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.394 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.395 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2023-09-13T04:50:37.395 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2023-09-13T04:50:37.395 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.395 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.396 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.396 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2023-09-13T04:50:37.396 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2023-09-13T04:50:37.396 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2023-09-13T04:50:37.396 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.397 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.397 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.397 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.397 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.398 INFO:teuthology.orchestra.run.smithi002.stdout: "url_prefix": { 2023-09-13T04:50:37.398 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url_prefix", 2023-09-13T04:50:37.398 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.398 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.398 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.399 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.399 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.399 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.399 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.399 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.400 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.400 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.400 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.400 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.400 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.401 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "devicehealth", 2023-09-13T04:50:37.401 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.401 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.402 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.402 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_monitoring": { 2023-09-13T04:50:37.402 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_monitoring", 2023-09-13T04:50:37.402 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.402 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.403 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.403 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.403 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.403 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.403 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.404 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "monitor device health metrics", 2023-09-13T04:50:37.404 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.404 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.404 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.404 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.405 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.405 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.405 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.405 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.405 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.406 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.406 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.406 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.406 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.406 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.407 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.407 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.407 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.407 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.407 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.408 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.408 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.408 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.408 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.409 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.409 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.409 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.409 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.409 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.410 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.410 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.410 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.410 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.410 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.411 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.411 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.411 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.411 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.411 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.412 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.412 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.412 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.412 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.412 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.413 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.413 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.413 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.413 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.413 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.414 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.414 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.414 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.414 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.414 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.415 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.415 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.415 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.415 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.416 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.416 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.416 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.416 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.416 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.417 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.417 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.417 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.417 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.417 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.417 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.418 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.418 INFO:teuthology.orchestra.run.smithi002.stdout: "mark_out_threshold": { 2023-09-13T04:50:37.418 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mark_out_threshold", 2023-09-13T04:50:37.418 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:37.419 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.419 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.419 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2419200", 2023-09-13T04:50:37.419 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.419 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.420 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.420 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-09-13T04:50:37.420 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.420 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.420 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.421 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": { 2023-09-13T04:50:37.421 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_name", 2023-09-13T04:50:37.421 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.421 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.422 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.422 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "device_health_metrics", 2023-09-13T04:50:37.422 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.422 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.423 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.423 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name of pool in which to store device health metrics", 2023-09-13T04:50:37.423 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.423 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.423 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.424 INFO:teuthology.orchestra.run.smithi002.stdout: "retention_period": { 2023-09-13T04:50:37.424 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retention_period", 2023-09-13T04:50:37.424 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:37.424 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.425 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.425 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15552000", 2023-09-13T04:50:37.425 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.425 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.425 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.426 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain device health metrics", 2023-09-13T04:50:37.426 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.426 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.426 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.427 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_frequency": { 2023-09-13T04:50:37.427 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_frequency", 2023-09-13T04:50:37.427 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:37.427 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.428 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.428 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2023-09-13T04:50:37.428 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.428 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.428 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.429 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to scrape device health metrics", 2023-09-13T04:50:37.429 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.429 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.429 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.430 INFO:teuthology.orchestra.run.smithi002.stdout: "self_heal": { 2023-09-13T04:50:37.430 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "self_heal", 2023-09-13T04:50:37.430 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.430 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.430 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.431 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.431 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.431 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.431 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.432 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-09-13T04:50:37.432 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.432 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.432 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.433 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-09-13T04:50:37.433 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-09-13T04:50:37.433 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:37.433 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.433 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.434 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-09-13T04:50:37.434 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.434 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.434 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.434 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and check device health", 2023-09-13T04:50:37.435 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.435 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.435 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.435 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_threshold": { 2023-09-13T04:50:37.436 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_threshold", 2023-09-13T04:50:37.436 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:37.436 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.436 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.436 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7257600", 2023-09-13T04:50:37.437 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.437 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.437 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.437 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-09-13T04:50:37.437 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.437 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.438 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.438 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.438 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.439 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "diskprediction_local", 2023-09-13T04:50:37.439 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.439 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.439 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.439 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.440 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.440 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.440 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.440 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.440 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.441 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.441 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.441 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.441 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.441 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.442 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.442 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.442 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.442 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.442 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.442 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.443 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.443 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.443 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.444 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.444 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.444 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.444 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.444 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.445 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.445 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.445 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.445 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.445 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.446 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.446 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.446 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.446 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.447 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.447 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.447 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.447 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.447 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.448 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.448 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.448 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.448 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.448 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.448 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.449 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.449 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.449 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.449 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.449 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.450 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.450 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.450 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.451 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.451 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.451 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.451 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.451 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.452 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.452 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.452 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.452 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.453 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.453 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.453 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.453 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.454 INFO:teuthology.orchestra.run.smithi002.stdout: "predict_interval": { 2023-09-13T04:50:37.454 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predict_interval", 2023-09-13T04:50:37.454 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.454 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.454 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.455 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2023-09-13T04:50:37.455 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.457 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.457 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.457 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.457 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.458 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.458 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.458 INFO:teuthology.orchestra.run.smithi002.stdout: "predictor_model": { 2023-09-13T04:50:37.458 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predictor_model", 2023-09-13T04:50:37.459 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.459 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.459 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.459 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "prophetstor", 2023-09-13T04:50:37.459 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.459 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.460 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.460 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.460 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.460 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.460 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.461 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-09-13T04:50:37.461 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-09-13T04:50:37.461 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.461 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.462 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.462 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-09-13T04:50:37.462 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.462 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.462 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.463 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.463 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.463 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.463 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.463 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "influx", 2023-09-13T04:50:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": false, 2023-09-13T04:50:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "influxdb python module not found", 2023-09-13T04:50:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "batch_size": { 2023-09-13T04:50:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "batch_size", 2023-09-13T04:50:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5000", 2023-09-13T04:50:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-09-13T04:50:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: "database": { 2023-09-13T04:50:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "database", 2023-09-13T04:50:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "ceph", 2023-09-13T04:50:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-09-13T04:50:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": { 2023-09-13T04:50:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hostname", 2023-09-13T04:50:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server hostname", 2023-09-13T04:50:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-09-13T04:50:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-09-13T04:50:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "30", 2023-09-13T04:50:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "5", 2023-09-13T04:50:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-09-13T04:50:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.488 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.488 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.488 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.488 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.488 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.489 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.489 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.489 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.489 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.489 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.489 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.490 INFO:teuthology.orchestra.run.smithi002.stdout: "password": { 2023-09-13T04:50:37.490 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "password", 2023-09-13T04:50:37.490 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.490 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.491 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.491 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.491 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.491 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.491 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.492 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "password of InfluxDB server user", 2023-09-13T04:50:37.492 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.492 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.492 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.493 INFO:teuthology.orchestra.run.smithi002.stdout: "port": { 2023-09-13T04:50:37.493 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "port", 2023-09-13T04:50:37.493 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.493 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.493 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.493 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8086", 2023-09-13T04:50:37.494 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.494 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.494 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.494 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server port", 2023-09-13T04:50:37.494 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.495 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.495 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.495 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2023-09-13T04:50:37.495 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2023-09-13T04:50:37.496 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.496 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.496 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.496 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "false", 2023-09-13T04:50:37.496 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.497 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.497 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.497 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-09-13T04:50:37.497 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.498 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.498 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.498 INFO:teuthology.orchestra.run.smithi002.stdout: "threads": { 2023-09-13T04:50:37.498 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threads", 2023-09-13T04:50:37.499 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.499 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.499 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.499 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2023-09-13T04:50:37.499 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2023-09-13T04:50:37.500 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "32", 2023-09-13T04:50:37.500 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.500 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-09-13T04:50:37.500 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.500 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.501 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.501 INFO:teuthology.orchestra.run.smithi002.stdout: "username": { 2023-09-13T04:50:37.501 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "username", 2023-09-13T04:50:37.501 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.502 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.502 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.502 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.502 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.503 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.503 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.503 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "username of InfluxDB server user", 2023-09-13T04:50:37.503 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.503 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.503 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.504 INFO:teuthology.orchestra.run.smithi002.stdout: "verify_ssl": { 2023-09-13T04:50:37.504 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "verify_ssl", 2023-09-13T04:50:37.504 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.505 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.505 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.505 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "true", 2023-09-13T04:50:37.505 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.505 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.506 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.506 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-09-13T04:50:37.506 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.506 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.506 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.507 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.507 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.507 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "insights", 2023-09-13T04:50:37.508 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.508 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.508 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.508 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.508 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.509 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.509 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.509 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.509 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.509 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.510 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.510 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.510 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.510 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.511 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.511 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.511 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.511 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.511 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.512 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.512 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.512 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.512 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.513 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.513 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.513 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.514 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.514 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.514 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.514 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.514 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.515 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.515 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.515 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.515 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.516 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.516 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.516 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.516 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.517 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.517 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.517 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.517 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.517 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.518 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.518 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.518 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.518 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.518 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.519 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.519 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.519 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.519 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.519 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.520 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.520 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.520 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.520 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.521 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.521 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.521 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.521 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.521 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.522 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.522 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.522 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.522 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.523 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.523 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.523 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.523 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.524 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "iostat", 2023-09-13T04:50:37.524 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.524 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.525 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.525 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.525 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.525 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.525 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.526 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.526 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.526 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.526 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.526 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.527 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.527 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.527 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.527 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.527 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.528 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.528 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.528 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.528 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.529 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.529 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.529 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.529 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.530 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.530 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.530 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.530 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.530 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.531 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.531 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.531 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.531 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.532 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.532 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.532 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.532 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.533 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.533 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.533 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.533 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.533 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.534 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.534 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.534 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.534 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.534 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.535 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.535 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.535 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.535 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.535 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.536 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.536 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.536 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.537 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.537 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.537 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.537 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.537 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.538 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.538 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.538 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.538 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.538 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.539 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.539 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.539 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.539 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.540 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.540 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "k8sevents", 2023-09-13T04:50:37.540 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.541 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.541 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.541 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_event_retention_days": { 2023-09-13T04:50:37.541 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ceph_event_retention_days", 2023-09-13T04:50:37.541 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.542 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.542 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.542 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7", 2023-09-13T04:50:37.542 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.543 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.543 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.543 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Days to hold ceph event information within local cache", 2023-09-13T04:50:37.543 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.543 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.544 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.544 INFO:teuthology.orchestra.run.smithi002.stdout: "config_check_secs": { 2023-09-13T04:50:37.544 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_check_secs", 2023-09-13T04:50:37.544 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.545 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.545 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.545 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-09-13T04:50:37.545 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "10", 2023-09-13T04:50:37.545 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.546 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.546 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-09-13T04:50:37.546 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.547 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.547 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.547 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.547 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.548 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.548 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.548 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.548 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.548 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.548 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.549 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.549 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.549 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.549 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.549 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.550 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.550 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.550 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.550 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.550 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.551 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.551 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.551 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.551 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.552 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.552 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.552 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.552 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.552 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.553 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.553 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.553 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.553 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.553 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.554 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.554 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.554 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.554 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.555 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.555 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.555 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.555 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.555 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.555 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.556 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.556 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.556 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.556 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.556 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.557 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.557 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.557 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.557 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.557 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.558 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.558 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.558 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.558 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.559 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.559 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.559 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.559 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.559 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.560 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.560 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.560 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.560 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.560 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.560 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.561 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.561 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "localpool", 2023-09-13T04:50:37.561 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.562 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.562 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.562 INFO:teuthology.orchestra.run.smithi002.stdout: "failure_domain": { 2023-09-13T04:50:37.562 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "failure_domain", 2023-09-13T04:50:37.562 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.563 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.563 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.563 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "host", 2023-09-13T04:50:37.563 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.563 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.564 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.564 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "failure domain for any created local pool", 2023-09-13T04:50:37.564 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-09-13T04:50:37.564 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.564 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.565 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.565 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.565 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.565 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.566 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.566 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.566 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.566 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.566 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.567 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.567 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.567 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.567 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.567 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.568 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.568 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.568 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.568 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.568 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.569 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.569 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.569 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.569 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.570 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.570 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.570 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.570 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.570 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.571 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.571 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.571 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.571 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.571 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.572 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.572 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.572 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.572 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.573 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.573 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.573 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.573 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.573 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.573 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.574 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.574 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.574 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.574 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.574 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.575 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.575 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.575 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.575 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.575 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.576 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.576 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.576 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.576 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.577 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.577 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.577 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.577 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.577 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.578 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.578 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.578 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.578 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.579 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": { 2023-09-13T04:50:37.579 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_size", 2023-09-13T04:50:37.579 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.579 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.579 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.579 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.580 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.580 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.580 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.580 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default min_size for any created local pool", 2023-09-13T04:50:37.580 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-09-13T04:50:37.581 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.581 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.581 INFO:teuthology.orchestra.run.smithi002.stdout: "num_rep": { 2023-09-13T04:50:37.581 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "num_rep", 2023-09-13T04:50:37.582 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.582 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.582 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.582 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3", 2023-09-13T04:50:37.582 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.583 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.583 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.583 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default replica count for any created local pool", 2023-09-13T04:50:37.583 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.583 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.584 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.584 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": { 2023-09-13T04:50:37.584 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_num", 2023-09-13T04:50:37.584 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.585 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.585 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.585 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "128", 2023-09-13T04:50:37.585 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.585 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.586 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.586 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default pg_num for any created local pool", 2023-09-13T04:50:37.586 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.586 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.586 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.587 INFO:teuthology.orchestra.run.smithi002.stdout: "prefix": { 2023-09-13T04:50:37.587 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prefix", 2023-09-13T04:50:37.587 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.587 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.587 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.588 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.588 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.588 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.588 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.588 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name prefix for any created local pool", 2023-09-13T04:50:37.589 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.589 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.589 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.589 INFO:teuthology.orchestra.run.smithi002.stdout: "subtree": { 2023-09-13T04:50:37.590 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "subtree", 2023-09-13T04:50:37.590 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.590 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.590 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.590 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "rack", 2023-09-13T04:50:37.591 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.591 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.591 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.591 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "CRUSH level for which to create a local pool", 2023-09-13T04:50:37.591 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-09-13T04:50:37.592 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.592 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.592 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.592 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.593 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mds_autoscaler", 2023-09-13T04:50:37.593 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.593 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.593 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.594 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.594 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.594 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.594 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.594 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.594 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.595 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.595 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.595 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.595 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.595 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.596 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.596 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.596 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.596 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.596 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.597 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.597 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.597 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.597 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.598 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.598 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.598 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.598 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.598 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.599 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.599 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.599 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.599 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.599 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.600 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.600 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.600 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.600 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.601 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.601 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.601 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.601 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.601 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.601 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.602 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.602 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.602 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.602 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.602 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.603 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.603 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.603 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.603 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.603 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.604 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.604 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.604 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.604 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.605 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.605 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.605 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.605 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.605 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.606 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.606 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.606 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.606 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.606 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.607 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.607 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.607 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.607 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.608 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirroring", 2023-09-13T04:50:37.608 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.608 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.608 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.609 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.609 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.609 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.609 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.609 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.610 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.610 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.610 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.610 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.611 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.611 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.611 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.611 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.611 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.611 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.612 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.612 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.612 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.612 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.612 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.613 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.613 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.613 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.613 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.614 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.614 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.614 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.614 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.614 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.615 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.615 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.615 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.615 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.616 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.616 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.616 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.616 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.616 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.617 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.617 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.617 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.617 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.617 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.617 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.618 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.618 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.618 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.618 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.618 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.619 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.619 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.619 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.619 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.620 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.620 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.620 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.620 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.620 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.621 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.621 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.621 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.621 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.621 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.622 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.622 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.622 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.622 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.622 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.623 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "nfs", 2023-09-13T04:50:37.623 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.623 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.624 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.624 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.624 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.624 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.624 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.625 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.625 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.625 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.625 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.625 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.625 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.626 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.626 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.626 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.626 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.627 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.627 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.627 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.627 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.627 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.627 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.628 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.628 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.628 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.628 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.629 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.629 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.629 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.629 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.629 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.630 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.630 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.630 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.630 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.631 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.631 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.631 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.631 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.631 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.631 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.632 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.632 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.632 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.632 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.632 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.633 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.633 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.633 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.633 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.633 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.634 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.634 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.634 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.634 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.635 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.635 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.635 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.635 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.635 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.636 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.636 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.636 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.636 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.636 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.636 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.637 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.637 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.638 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2023-09-13T04:50:37.638 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.638 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.638 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.639 INFO:teuthology.orchestra.run.smithi002.stdout: "fail_fs": { 2023-09-13T04:50:37.639 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "fail_fs", 2023-09-13T04:50:37.639 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.639 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.639 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.640 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.640 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.640 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.640 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.640 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-09-13T04:50:37.641 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.641 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.641 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.641 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.642 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.642 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.642 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.642 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.642 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.643 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.643 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.643 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.643 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.643 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.643 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.644 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.644 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.644 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.644 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.644 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.645 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.645 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.645 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.645 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.646 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.646 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.646 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.646 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.646 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.647 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.647 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.647 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.647 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.647 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.648 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.648 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.648 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.648 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.649 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.649 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.649 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.649 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.649 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.650 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.650 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.650 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.650 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.650 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.651 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.651 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.651 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.651 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.651 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.652 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.652 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.652 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.652 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.652 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.653 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.653 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.653 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.653 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.653 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.654 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.654 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.654 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.654 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.654 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.655 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.655 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator": { 2023-09-13T04:50:37.655 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2023-09-13T04:50:37.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.656 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.656 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.656 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.656 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.656 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.657 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.657 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm", 2023-09-13T04:50:37.657 INFO:teuthology.orchestra.run.smithi002.stdout: "rook", 2023-09-13T04:50:37.657 INFO:teuthology.orchestra.run.smithi002.stdout: "test_orchestrator" 2023-09-13T04:50:37.657 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.658 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Orchestrator backend", 2023-09-13T04:50:37.658 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.658 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.658 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.658 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.659 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_perf_query", 2023-09-13T04:50:37.659 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.660 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.660 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.660 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.660 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.660 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.661 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.661 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.661 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.661 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.661 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.662 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.662 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.662 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.662 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.662 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.663 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.663 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.663 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.663 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.664 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.665 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.665 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.665 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.665 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.666 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.666 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.666 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.666 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.666 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.667 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.667 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.667 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.667 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.667 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.668 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.668 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.668 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.668 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.669 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.669 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.669 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.669 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.669 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.670 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.670 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.670 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.670 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.670 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.671 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.671 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.671 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.672 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.672 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.672 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.672 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.672 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.673 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.673 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.673 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.673 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.673 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.674 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.674 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.674 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.674 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.674 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.675 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.675 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.675 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.675 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.676 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_support", 2023-09-13T04:50:37.676 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.676 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.676 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.676 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.677 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.677 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.677 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.677 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.677 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.678 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.678 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.678 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.678 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.678 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.679 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.679 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.679 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.679 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.679 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.679 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.680 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.680 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.680 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.680 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.681 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.681 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.681 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.681 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.681 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.682 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.682 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.682 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.682 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.682 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.683 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.683 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.683 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.683 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.684 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.684 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.684 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.684 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.684 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.684 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.685 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.685 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.685 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.685 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.685 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.686 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.686 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.686 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.686 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.686 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.687 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.687 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.687 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.687 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.688 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.688 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.688 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.688 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.688 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.689 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.689 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.689 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.689 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.689 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.689 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.690 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.690 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_autoscaler", 2023-09-13T04:50:37.690 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.691 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.691 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.691 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.691 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.691 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.692 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.692 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.692 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.692 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.692 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.692 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.693 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.693 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.693 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.693 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.693 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.694 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.694 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.694 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.694 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.694 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.695 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.695 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.695 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.695 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.696 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.696 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.696 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.696 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.696 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.696 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.697 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.697 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.697 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.697 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.698 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.698 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.698 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.698 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.698 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.699 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.699 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.699 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.699 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.699 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.699 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.700 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.700 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.700 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.700 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.700 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.700 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.701 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.701 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.701 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.701 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.702 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.702 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.702 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.702 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.702 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.703 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.703 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.703 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.703 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.703 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.704 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.704 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.704 INFO:teuthology.orchestra.run.smithi002.stdout: "noautoscale": { 2023-09-13T04:50:37.704 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "noautoscale", 2023-09-13T04:50:37.705 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.705 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.705 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.705 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.705 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.705 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.706 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.706 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "global autoscale flag", 2023-09-13T04:50:37.706 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-09-13T04:50:37.706 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.707 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.707 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-09-13T04:50:37.707 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-09-13T04:50:37.707 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:37.707 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.708 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.708 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-09-13T04:50:37.708 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.708 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.708 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.709 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.709 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.709 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.709 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.710 INFO:teuthology.orchestra.run.smithi002.stdout: "threshold": { 2023-09-13T04:50:37.710 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threshold", 2023-09-13T04:50:37.710 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-09-13T04:50:37.710 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.710 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.711 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2023-09-13T04:50:37.711 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1.0", 2023-09-13T04:50:37.711 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.711 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.711 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "scaling threshold", 2023-09-13T04:50:37.712 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-09-13T04:50:37.712 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.712 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.712 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.712 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.713 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "progress", 2023-09-13T04:50:37.713 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.713 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.713 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.714 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_pg_recovery_event": { 2023-09-13T04:50:37.714 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_pg_recovery_event", 2023-09-13T04:50:37.714 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.714 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.714 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.715 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.715 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.715 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.715 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.715 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow the module to show pg recovery progress", 2023-09-13T04:50:37.716 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.716 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.716 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.716 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2023-09-13T04:50:37.716 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2023-09-13T04:50:37.717 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.717 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.717 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.717 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.717 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.718 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.718 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.718 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.718 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.718 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.719 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.719 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.719 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.719 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.720 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.720 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.720 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.720 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.720 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.720 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.721 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.721 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.721 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.721 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.721 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.722 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.722 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.722 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.722 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.722 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.723 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.723 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.723 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.723 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.724 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.724 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.724 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.724 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.724 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.725 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.725 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.725 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.725 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.725 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.726 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.726 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.726 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.726 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.727 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.727 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.727 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.727 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.727 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.727 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.728 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.728 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.728 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.728 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.728 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.729 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.729 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.729 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.729 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.729 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.730 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.730 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.730 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.730 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.731 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.731 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.731 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.731 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.731 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.731 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.732 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.732 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.732 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.732 INFO:teuthology.orchestra.run.smithi002.stdout: "max_completed_events": { 2023-09-13T04:50:37.733 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_completed_events", 2023-09-13T04:50:37.733 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.733 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.733 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.733 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "50", 2023-09-13T04:50:37.733 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.734 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.734 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.734 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "number of past completed events to remember", 2023-09-13T04:50:37.734 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.734 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.735 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.735 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-09-13T04:50:37.735 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-09-13T04:50:37.735 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:37.735 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.736 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.736 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2023-09-13T04:50:37.736 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.736 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.736 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.737 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long the module is going to sleep", 2023-09-13T04:50:37.737 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.737 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.737 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.737 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.738 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.738 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus", 2023-09-13T04:50:37.738 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.739 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.739 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.739 INFO:teuthology.orchestra.run.smithi002.stdout: "cache": { 2023-09-13T04:50:37.739 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cache", 2023-09-13T04:50:37.739 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.740 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.740 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.740 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.740 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.740 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.741 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.741 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.741 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.741 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.741 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.742 INFO:teuthology.orchestra.run.smithi002.stdout: "exclude_perf_counters": { 2023-09-13T04:50:37.742 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "exclude_perf_counters", 2023-09-13T04:50:37.742 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.742 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.743 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.743 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.743 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.743 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.743 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.744 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-09-13T04:50:37.744 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-09-13T04:50:37.744 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.744 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.744 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.745 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.745 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.745 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.745 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.745 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.746 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.746 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.746 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.746 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.747 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.747 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.747 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.747 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.747 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.747 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.748 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.748 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.748 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.748 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.748 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.749 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.749 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.749 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.749 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.749 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.750 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.750 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.750 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.750 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.750 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.751 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.751 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.751 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.751 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.751 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.752 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.752 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.752 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.752 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.752 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.753 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.753 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.753 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.753 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.753 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.753 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.754 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.754 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.754 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.754 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.754 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.755 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.755 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.755 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.755 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.755 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.756 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.756 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.756 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.756 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.756 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.757 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.757 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.757 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.757 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.757 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools": { 2023-09-13T04:50:37.758 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools", 2023-09-13T04:50:37.758 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.758 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.758 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.758 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.759 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.759 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.759 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.759 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.759 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.760 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.760 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.760 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools_refresh_interval": { 2023-09-13T04:50:37.760 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-09-13T04:50:37.760 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.761 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.761 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.761 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "300", 2023-09-13T04:50:37.761 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.761 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.762 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.762 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.762 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.762 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.762 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.763 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_interval": { 2023-09-13T04:50:37.763 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_interval", 2023-09-13T04:50:37.763 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-09-13T04:50:37.763 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.764 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.764 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15.0", 2023-09-13T04:50:37.764 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.764 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.764 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.764 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.765 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.765 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.765 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.765 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2023-09-13T04:50:37.765 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2023-09-13T04:50:37.766 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.766 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.766 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.766 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2023-09-13T04:50:37.766 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.767 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.767 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.767 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-09-13T04:50:37.767 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.767 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.768 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.768 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2023-09-13T04:50:37.768 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2023-09-13T04:50:37.768 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.769 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.769 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.769 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "9283", 2023-09-13T04:50:37.769 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.769 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.769 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.770 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-09-13T04:50:37.770 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.770 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.770 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.771 INFO:teuthology.orchestra.run.smithi002.stdout: "stale_cache_strategy": { 2023-09-13T04:50:37.771 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stale_cache_strategy", 2023-09-13T04:50:37.771 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.771 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.771 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.772 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "log", 2023-09-13T04:50:37.772 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.772 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.772 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.772 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.773 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.773 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.773 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.773 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2023-09-13T04:50:37.774 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2023-09-13T04:50:37.774 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.774 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.774 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.774 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "default", 2023-09-13T04:50:37.775 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.775 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.775 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.775 INFO:teuthology.orchestra.run.smithi002.stdout: "default", 2023-09-13T04:50:37.775 INFO:teuthology.orchestra.run.smithi002.stdout: "error" 2023-09-13T04:50:37.776 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.776 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.776 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.776 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.776 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.777 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2023-09-13T04:50:37.777 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2023-09-13T04:50:37.777 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.777 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.777 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.778 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2023-09-13T04:50:37.778 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2023-09-13T04:50:37.778 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2023-09-13T04:50:37.778 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.778 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.779 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.779 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.779 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.779 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.779 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.780 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_support", 2023-09-13T04:50:37.780 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.780 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.780 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.781 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.781 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.781 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.781 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.781 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.782 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.782 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.782 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.782 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.782 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.783 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.783 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.783 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.783 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.783 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.784 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.784 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.784 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.784 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.784 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.785 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.785 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.785 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.785 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.785 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.786 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.786 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.786 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.786 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.786 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.786 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.787 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.787 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.787 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.787 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.788 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.788 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.788 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.788 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.788 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.788 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.789 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.789 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.789 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.789 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.789 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.790 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.790 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.790 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.790 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.790 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.791 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.791 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.791 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.791 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.792 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.792 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.792 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.792 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.792 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.793 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.793 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.793 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.793 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.793 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.794 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.794 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_snap_create": { 2023-09-13T04:50:37.794 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_snap_create", 2023-09-13T04:50:37.794 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.795 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.795 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.795 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-09-13T04:50:37.795 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.795 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.795 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.796 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.796 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.796 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.796 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.797 INFO:teuthology.orchestra.run.smithi002.stdout: "mirror_snapshot_schedule": { 2023-09-13T04:50:37.797 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirror_snapshot_schedule", 2023-09-13T04:50:37.797 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.797 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.797 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.798 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.798 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.798 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.798 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.798 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.799 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.799 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.799 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.799 INFO:teuthology.orchestra.run.smithi002.stdout: "trash_purge_schedule": { 2023-09-13T04:50:37.800 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "trash_purge_schedule", 2023-09-13T04:50:37.800 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.800 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.800 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.800 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.800 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.801 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.801 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.801 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.801 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.801 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.802 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.802 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.802 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.802 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "restful", 2023-09-13T04:50:37.803 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.803 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.803 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.803 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_auth": { 2023-09-13T04:50:37.803 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_auth", 2023-09-13T04:50:37.803 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.804 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.804 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.804 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.804 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.804 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.805 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.805 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.805 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.805 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.805 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.806 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2023-09-13T04:50:37.806 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2023-09-13T04:50:37.806 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.806 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.806 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.807 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.807 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.807 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.807 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.807 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.808 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.808 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.808 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.808 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.809 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.809 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.809 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.809 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.809 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.810 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.810 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.810 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.810 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.810 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.811 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.811 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.811 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.811 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.811 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.811 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.812 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.812 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.812 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.812 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.813 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.813 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.813 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.813 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.813 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.813 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.814 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.814 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.814 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.814 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.814 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.815 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.815 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.815 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.815 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.816 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.816 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.816 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.816 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.816 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.817 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.817 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.817 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.817 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.817 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.817 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.818 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.818 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.818 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.818 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.819 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.819 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.819 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.819 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.819 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.820 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.820 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.820 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.820 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.820 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.821 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.821 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.821 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.821 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.821 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.822 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2023-09-13T04:50:37.822 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2023-09-13T04:50:37.822 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.822 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.822 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.823 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.823 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.823 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.823 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.823 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.824 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.824 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.824 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.824 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2023-09-13T04:50:37.825 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2023-09-13T04:50:37.825 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.825 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.825 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.825 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.825 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.826 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.826 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.826 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.826 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.826 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.827 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.827 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.827 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.828 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rgw", 2023-09-13T04:50:37.828 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.828 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.828 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.828 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.828 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.829 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.829 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.829 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.829 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.829 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.830 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.830 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.830 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.830 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.830 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.831 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.831 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.831 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.831 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.831 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.832 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.832 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.832 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.832 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.833 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.833 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.833 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.833 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.833 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.833 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.834 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.834 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.834 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.834 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.834 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.835 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.835 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.835 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.835 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.835 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.836 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.836 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.836 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.836 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.836 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.836 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.837 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.837 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.837 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.837 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.837 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.838 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.838 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.838 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.838 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.838 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.839 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.839 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.839 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.839 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.840 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.840 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.840 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.840 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.840 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.840 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.841 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.841 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.841 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.841 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.841 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.842 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rook", 2023-09-13T04:50:37.842 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.842 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.843 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.843 INFO:teuthology.orchestra.run.smithi002.stdout: "drive_group_interval": { 2023-09-13T04:50:37.843 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "drive_group_interval", 2023-09-13T04:50:37.843 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-09-13T04:50:37.843 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.844 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.844 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "300.0", 2023-09-13T04:50:37.844 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.844 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.844 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.845 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-09-13T04:50:37.845 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.845 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.845 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.845 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.846 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.846 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.846 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.846 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.846 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.847 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.847 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.847 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.847 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.847 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.848 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.848 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.848 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.848 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.848 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.849 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.849 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.849 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.849 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.849 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.850 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.850 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.850 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.850 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.850 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.851 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.851 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.851 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.851 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.851 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.852 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.852 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.852 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.852 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.853 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.853 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.853 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.853 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.853 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.853 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.854 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.854 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.854 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.854 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.854 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.855 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.855 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.855 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.855 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.855 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.856 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.856 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.856 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.856 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.856 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.857 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.857 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.857 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.857 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.857 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.858 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.858 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.858 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.858 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.858 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.859 INFO:teuthology.orchestra.run.smithi002.stdout: "storage_class": { 2023-09-13T04:50:37.859 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "storage_class", 2023-09-13T04:50:37.859 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.859 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.859 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.860 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "local", 2023-09-13T04:50:37.860 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.860 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.860 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.860 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-09-13T04:50:37.860 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.861 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.861 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.861 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.861 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.862 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "selftest", 2023-09-13T04:50:37.862 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.862 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.862 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.863 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.863 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.863 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.863 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.865 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.865 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.865 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.865 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.866 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.866 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.866 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.866 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.866 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.867 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.867 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.867 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.867 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.867 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.868 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.868 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.868 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.868 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.868 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.869 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.869 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.869 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.869 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.869 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.870 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.870 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.870 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.870 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.870 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.871 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.871 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.871 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.871 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.872 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.872 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.872 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.872 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.872 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.872 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.873 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.873 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.873 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.873 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.873 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.874 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.874 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.874 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.874 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.874 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.875 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.875 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.875 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.875 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.876 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.876 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.876 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.876 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.876 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.876 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.877 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.877 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.877 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.877 INFO:teuthology.orchestra.run.smithi002.stdout: "roption1": { 2023-09-13T04:50:37.878 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption1", 2023-09-13T04:50:37.878 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.878 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.878 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.878 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.879 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.879 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.879 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.879 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.879 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.879 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.880 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.880 INFO:teuthology.orchestra.run.smithi002.stdout: "roption2": { 2023-09-13T04:50:37.880 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption2", 2023-09-13T04:50:37.880 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.881 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.881 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.881 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "xyz", 2023-09-13T04:50:37.881 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.881 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.882 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.882 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.882 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.882 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.882 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.883 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption1": { 2023-09-13T04:50:37.883 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption1", 2023-09-13T04:50:37.883 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.883 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.884 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.884 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.884 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.884 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.884 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.884 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.885 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.885 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.885 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.885 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption2": { 2023-09-13T04:50:37.886 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption2", 2023-09-13T04:50:37.886 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.886 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.886 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.886 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.887 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.887 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.887 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.887 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.887 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.888 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.888 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.888 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption3": { 2023-09-13T04:50:37.888 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption3", 2023-09-13T04:50:37.889 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-09-13T04:50:37.889 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.889 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.889 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.889 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.890 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.890 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.890 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.890 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.890 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.891 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.891 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption4": { 2023-09-13T04:50:37.891 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption4", 2023-09-13T04:50:37.891 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.892 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.892 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.892 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.892 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.892 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.892 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.893 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.893 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.893 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.893 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.894 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption5": { 2023-09-13T04:50:37.894 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption5", 2023-09-13T04:50:37.894 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.894 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.894 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.895 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.895 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.895 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.895 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.895 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.896 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.896 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.896 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.896 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption6": { 2023-09-13T04:50:37.896 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption6", 2023-09-13T04:50:37.897 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.897 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.897 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.897 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.897 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.898 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.898 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.898 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.898 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.898 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.899 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.899 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption7": { 2023-09-13T04:50:37.899 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption7", 2023-09-13T04:50:37.899 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:37.899 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.900 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.900 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.900 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2023-09-13T04:50:37.900 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "42", 2023-09-13T04:50:37.900 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.901 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.901 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.901 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.901 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.901 INFO:teuthology.orchestra.run.smithi002.stdout: "testkey": { 2023-09-13T04:50:37.902 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testkey", 2023-09-13T04:50:37.902 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.902 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.902 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.902 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.903 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.903 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.903 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.903 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.903 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.904 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.904 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.904 INFO:teuthology.orchestra.run.smithi002.stdout: "testlkey": { 2023-09-13T04:50:37.904 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testlkey", 2023-09-13T04:50:37.905 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.905 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.905 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.905 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.905 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.905 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.906 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.906 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.906 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.906 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.906 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.907 INFO:teuthology.orchestra.run.smithi002.stdout: "testnewline": { 2023-09-13T04:50:37.907 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testnewline", 2023-09-13T04:50:37.907 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.907 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.908 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.908 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.908 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.908 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.908 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.909 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.909 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.909 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.909 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.909 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.909 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.910 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snap_schedule", 2023-09-13T04:50:37.910 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.911 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.911 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.911 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_m_granularity": { 2023-09-13T04:50:37.911 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_m_granularity", 2023-09-13T04:50:37.911 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.912 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.912 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.912 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.912 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.912 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.912 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.913 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow minute scheduled snapshots", 2023-09-13T04:50:37.913 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.913 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.913 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.914 INFO:teuthology.orchestra.run.smithi002.stdout: "dump_on_update": { 2023-09-13T04:50:37.914 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dump_on_update", 2023-09-13T04:50:37.914 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.914 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.914 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.915 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.915 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.915 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.915 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.915 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "dump database to debug log on update", 2023-09-13T04:50:37.916 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.916 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.916 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.916 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.916 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.917 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.917 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.917 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.917 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.917 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.918 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.918 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.918 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.918 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.918 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.919 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.919 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.919 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.919 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.919 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.920 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.920 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.920 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.920 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.920 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.921 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.921 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.921 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.921 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.921 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.922 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.922 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.922 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.922 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.922 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.923 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.923 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.923 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.923 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.924 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.924 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.924 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.924 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.924 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.925 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.925 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.925 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.925 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.925 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.925 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.926 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.926 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.926 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.926 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.926 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.927 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.927 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.927 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.927 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.928 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.928 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.928 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.928 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.928 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.929 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.929 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.929 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.929 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.929 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.930 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.930 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.930 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stats", 2023-09-13T04:50:37.931 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.931 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.931 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.931 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.931 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.932 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.932 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.932 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.932 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.932 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.932 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.933 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.933 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.933 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.933 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.933 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.934 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.934 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.934 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.934 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.934 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.935 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.935 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.935 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.935 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.936 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.936 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.936 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.936 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.936 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.936 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.937 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.937 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.937 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.937 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.937 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.938 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.938 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.938 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.938 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.938 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.939 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.939 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.939 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.939 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.939 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.940 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.940 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.940 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.940 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.940 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.941 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.941 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.941 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.941 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.941 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.942 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.942 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.942 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.942 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.943 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.943 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.943 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.943 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.943 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.944 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.944 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.944 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.944 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.944 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.944 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.945 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "status", 2023-09-13T04:50:37.945 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.945 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.946 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.946 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.946 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.946 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.946 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.947 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.947 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.947 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.947 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.947 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.948 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.948 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.948 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.948 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.948 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.949 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.949 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.949 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.949 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.949 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.950 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.950 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.950 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.950 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.951 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.951 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.951 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.951 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.951 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.951 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.952 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.952 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.952 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.952 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.953 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.953 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.953 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.953 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.953 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.954 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.954 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.954 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.954 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.954 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.955 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.955 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.955 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.955 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.955 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.955 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.956 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.956 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.956 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.956 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.956 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.957 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.957 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.957 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.957 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.957 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.957 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.958 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.958 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.958 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.958 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.958 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.959 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.959 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.959 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.960 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telegraf", 2023-09-13T04:50:37.960 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.960 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.960 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.960 INFO:teuthology.orchestra.run.smithi002.stdout: "address": { 2023-09-13T04:50:37.960 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "address", 2023-09-13T04:50:37.961 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.961 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.961 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.961 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-09-13T04:50:37.961 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.962 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.962 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.962 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.962 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.962 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.963 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.963 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-09-13T04:50:37.963 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-09-13T04:50:37.963 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:37.963 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.964 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.964 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15", 2023-09-13T04:50:37.964 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.964 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.964 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.964 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.965 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.965 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.965 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.965 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:37.966 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:37.966 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.966 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.966 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.966 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.967 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.967 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.967 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.967 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.967 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.968 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.968 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.968 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.968 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.968 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.968 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.969 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.969 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.969 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.969 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:37.970 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:37.970 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.970 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.970 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.970 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.971 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.971 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.971 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.971 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.971 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.972 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.972 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.972 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:37.972 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:37.972 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.973 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.973 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.973 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:37.973 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.973 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.974 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:37.974 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:37.974 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:37.974 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:37.974 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:37.975 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:37.975 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:37.975 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:37.975 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.975 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.976 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.976 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.976 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:37.976 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:37.976 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.977 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.977 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:37.977 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.977 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.977 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.978 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.978 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.978 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.978 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.978 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:37.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:37.979 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telemetry", 2023-09-13T04:50:37.980 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:37.980 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:37.980 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:37.980 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_basic": { 2023-09-13T04:50:37.980 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_basic", 2023-09-13T04:50:37.981 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.981 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.981 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.981 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.981 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.982 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.982 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.982 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share basic cluster information (size, version)", 2023-09-13T04:50:37.982 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.982 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.983 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.983 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_crash": { 2023-09-13T04:50:37.983 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_crash", 2023-09-13T04:50:37.983 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.983 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.984 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.984 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.984 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.984 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.984 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.985 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-09-13T04:50:37.985 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.985 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.985 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.986 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_device": { 2023-09-13T04:50:37.986 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_device", 2023-09-13T04:50:37.986 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.986 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.986 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.986 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-09-13T04:50:37.987 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.987 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.987 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.987 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-09-13T04:50:37.987 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.988 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.988 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.988 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_ident": { 2023-09-13T04:50:37.988 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_ident", 2023-09-13T04:50:37.989 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.989 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.989 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.989 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.989 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.989 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.990 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.990 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-09-13T04:50:37.990 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.990 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.990 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.991 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_perf": { 2023-09-13T04:50:37.991 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_perf", 2023-09-13T04:50:37.991 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:37.991 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.992 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.992 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:37.992 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.992 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.992 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.993 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share various performance metrics of a cluster", 2023-09-13T04:50:37.993 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.993 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.993 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.994 INFO:teuthology.orchestra.run.smithi002.stdout: "contact": { 2023-09-13T04:50:37.994 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "contact", 2023-09-13T04:50:37.994 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.994 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.994 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.995 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.995 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.995 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.995 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.995 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.996 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.996 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.996 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.996 INFO:teuthology.orchestra.run.smithi002.stdout: "description": { 2023-09-13T04:50:37.996 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "description", 2023-09-13T04:50:37.997 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:37.997 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:37.997 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:37.997 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:37.997 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:37.998 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:37.998 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:37.998 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:37.998 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:37.998 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:37.999 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:37.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:37.999 INFO:teuthology.orchestra.run.smithi002.stdout: "device_url": { 2023-09-13T04:50:37.999 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_url", 2023-09-13T04:50:37.999 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:38.000 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.000 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:38.000 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-09-13T04:50:38.000 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.000 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.001 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.001 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.001 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.001 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.001 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.002 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2023-09-13T04:50:38.002 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2023-09-13T04:50:38.002 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:38.002 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.003 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:38.003 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:38.003 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.003 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.003 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.004 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.004 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.004 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.004 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.005 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-09-13T04:50:38.005 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-09-13T04:50:38.005 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:38.005 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.005 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:38.006 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "24", 2023-09-13T04:50:38.006 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "8", 2023-09-13T04:50:38.006 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.006 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.006 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.007 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.007 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.007 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.007 INFO:teuthology.orchestra.run.smithi002.stdout: "last_opt_revision": { 2023-09-13T04:50:38.008 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "last_opt_revision", 2023-09-13T04:50:38.008 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:38.008 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.008 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:38.009 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1", 2023-09-13T04:50:38.009 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.009 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.009 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.009 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.010 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.010 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.010 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.010 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard": { 2023-09-13T04:50:38.011 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard", 2023-09-13T04:50:38.011 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:38.011 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.011 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:38.011 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:38.012 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.012 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.012 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.012 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.012 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.013 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.013 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.013 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard_description": { 2023-09-13T04:50:38.013 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard_description", 2023-09-13T04:50:38.014 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:38.014 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.014 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:38.014 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:38.014 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.014 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.015 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.015 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.015 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.015 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.015 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.016 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:38.016 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:38.016 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:38.016 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.016 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:38.017 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:38.017 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.017 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.017 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:38.017 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:38.017 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:38.018 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:38.018 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:38.018 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:38.018 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:38.018 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:38.019 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.019 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.019 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.019 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.019 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:38.020 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:38.020 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:38.020 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.020 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:38.020 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:38.020 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.021 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.021 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.021 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.021 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.021 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.022 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.022 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:38.022 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:38.022 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:38.023 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.023 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:38.023 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:38.023 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.023 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.024 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:38.024 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:38.024 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:38.024 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:38.024 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:38.025 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:38.025 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:38.025 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:38.025 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.025 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.026 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.026 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.026 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:38.026 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:38.027 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:38.027 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.027 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:38.027 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:38.027 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.028 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.028 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.028 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.028 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.028 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.029 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.029 INFO:teuthology.orchestra.run.smithi002.stdout: "organization": { 2023-09-13T04:50:38.029 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "organization", 2023-09-13T04:50:38.029 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:38.029 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.030 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:38.030 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:38.030 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.030 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.030 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.031 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.031 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.031 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.031 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.031 INFO:teuthology.orchestra.run.smithi002.stdout: "proxy": { 2023-09-13T04:50:38.032 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "proxy", 2023-09-13T04:50:38.032 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:38.032 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.032 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:38.032 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:38.033 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.033 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.033 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.033 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.033 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.034 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.034 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.034 INFO:teuthology.orchestra.run.smithi002.stdout: "url": { 2023-09-13T04:50:38.034 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url", 2023-09-13T04:50:38.034 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:38.035 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.035 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:38.035 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-09-13T04:50:38.035 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.035 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.036 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.036 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.036 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.036 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.036 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.037 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:38.037 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:38.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:38.037 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "test_orchestrator", 2023-09-13T04:50:38.037 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:38.038 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:38.038 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:38.038 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:38.038 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:38.038 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:38.039 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.039 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:38.039 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:38.039 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.039 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.039 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:38.040 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:38.040 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:38.040 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:38.040 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:38.040 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:38.040 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:38.041 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:38.041 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.041 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.041 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.041 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.042 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:38.042 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:38.042 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:38.042 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.042 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:38.043 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:38.043 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.043 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.043 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.043 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.044 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.044 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.044 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.044 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:38.045 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:38.045 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:38.045 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.045 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:38.045 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:38.045 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.046 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.046 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:38.046 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:38.046 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:38.046 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:38.047 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:38.047 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:38.047 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:38.047 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:38.047 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.047 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.048 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.048 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.048 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:38.048 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:38.048 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:38.049 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.049 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:38.049 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:38.049 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.049 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.049 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.050 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.050 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.050 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.050 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.050 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:38.050 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:38.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:38.051 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "volumes", 2023-09-13T04:50:38.051 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:38.051 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:38.054 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:38.054 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:38.054 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:38.054 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:38.054 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:38.055 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.055 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.055 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.055 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.056 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:38.056 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:38.056 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:38.056 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.056 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:38.057 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:38.057 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.057 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.057 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.057 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.058 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.058 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.058 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.058 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:38.059 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:38.059 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:38.059 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.059 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:38.059 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:38.060 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.060 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.060 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:38.060 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:38.060 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:38.060 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:38.061 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:38.061 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:38.061 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:38.061 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:38.062 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.062 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.062 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.062 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.062 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:38.063 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:38.063 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:38.063 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.063 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:38.063 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:38.064 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.064 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.064 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.064 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.064 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.065 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.065 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.065 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_clones": { 2023-09-13T04:50:38.065 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_clones", 2023-09-13T04:50:38.066 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:38.066 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.066 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:38.066 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4", 2023-09-13T04:50:38.066 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.067 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.067 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.067 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Number of asynchronous cloner threads", 2023-09-13T04:50:38.067 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.067 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.068 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.068 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_delay": { 2023-09-13T04:50:38.068 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_delay", 2023-09-13T04:50:38.068 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:38.068 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.069 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:38.069 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-09-13T04:50:38.069 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.069 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.069 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.070 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-09-13T04:50:38.070 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.070 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.070 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.070 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:38.071 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:38.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:38.071 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix", 2023-09-13T04:50:38.071 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-09-13T04:50:38.072 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-09-13T04:50:38.072 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-09-13T04:50:38.072 INFO:teuthology.orchestra.run.smithi002.stdout: "discovery_interval": { 2023-09-13T04:50:38.072 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "discovery_interval", 2023-09-13T04:50:38.072 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-09-13T04:50:38.073 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.073 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:38.073 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "100", 2023-09-13T04:50:38.073 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.073 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.073 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.074 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.074 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.074 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.074 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.075 INFO:teuthology.orchestra.run.smithi002.stdout: "identifier": { 2023-09-13T04:50:38.075 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "identifier", 2023-09-13T04:50:38.075 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:38.075 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.075 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:38.076 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:38.076 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.076 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.076 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.076 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.077 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.077 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.077 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.077 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-09-13T04:50:38.077 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-09-13T04:50:38.078 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-09-13T04:50:38.078 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.078 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:38.078 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-09-13T04:50:38.078 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.079 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.079 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.079 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.079 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.079 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.080 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.080 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-09-13T04:50:38.080 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-09-13T04:50:38.080 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:38.080 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.081 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:38.081 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:38.081 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.081 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.081 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:38.082 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:38.082 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:38.082 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:38.082 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:38.082 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:38.083 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:38.083 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:38.083 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.083 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.083 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.083 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.084 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-09-13T04:50:38.084 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-09-13T04:50:38.084 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:38.084 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.085 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:38.085 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:38.085 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.085 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.085 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.086 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.086 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.086 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.086 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.087 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-09-13T04:50:38.087 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-09-13T04:50:38.087 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:38.087 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.087 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:38.087 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-09-13T04:50:38.088 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.088 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.088 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-09-13T04:50:38.088 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-09-13T04:50:38.088 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-09-13T04:50:38.089 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-09-13T04:50:38.089 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-09-13T04:50:38.089 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-09-13T04:50:38.089 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-09-13T04:50:38.089 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:38.089 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.090 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.090 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.090 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.090 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-09-13T04:50:38.091 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-09-13T04:50:38.091 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-09-13T04:50:38.091 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.091 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-09-13T04:50:38.091 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-09-13T04:50:38.092 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.092 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.092 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.092 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.092 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.093 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.093 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.093 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_host": { 2023-09-13T04:50:38.093 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_host", 2023-09-13T04:50:38.093 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:38.094 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.094 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:38.094 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-09-13T04:50:38.094 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.094 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.095 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.095 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.095 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.095 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.095 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.096 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_port": { 2023-09-13T04:50:38.096 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_port", 2023-09-13T04:50:38.096 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-09-13T04:50:38.096 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.096 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:38.097 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10051", 2023-09-13T04:50:38.097 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.097 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.097 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.097 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.098 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.098 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.098 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.098 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_sender": { 2023-09-13T04:50:38.099 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_sender", 2023-09-13T04:50:38.099 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-09-13T04:50:38.099 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-09-13T04:50:38.099 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-09-13T04:50:38.099 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-09-13T04:50:38.100 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-09-13T04:50:38.100 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-09-13T04:50:38.100 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-09-13T04:50:38.100 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-09-13T04:50:38.100 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-09-13T04:50:38.100 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-09-13T04:50:38.101 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-09-13T04:50:38.101 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:38.101 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:38.101 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:38.101 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:38.102 INFO:teuthology.orchestra.run.smithi002.stdout: "services": { 2023-09-13T04:50:38.102 INFO:teuthology.orchestra.run.smithi002.stdout: "dashboard": "https://172.21.15.2:8443/" 2023-09-13T04:50:38.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.102 INFO:teuthology.orchestra.run.smithi002.stdout: "always_on_modules": { 2023-09-13T04:50:38.102 INFO:teuthology.orchestra.run.smithi002.stdout: "octopus": [ 2023-09-13T04:50:38.103 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2023-09-13T04:50:38.103 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2023-09-13T04:50:38.103 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2023-09-13T04:50:38.103 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2023-09-13T04:50:38.103 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2023-09-13T04:50:38.104 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2023-09-13T04:50:38.104 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2023-09-13T04:50:38.104 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2023-09-13T04:50:38.104 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2023-09-13T04:50:38.104 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2023-09-13T04:50:38.105 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:38.105 INFO:teuthology.orchestra.run.smithi002.stdout: "pacific": [ 2023-09-13T04:50:38.105 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2023-09-13T04:50:38.105 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2023-09-13T04:50:38.105 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2023-09-13T04:50:38.105 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2023-09-13T04:50:38.106 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2023-09-13T04:50:38.106 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2023-09-13T04:50:38.106 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2023-09-13T04:50:38.106 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2023-09-13T04:50:38.107 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2023-09-13T04:50:38.107 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2023-09-13T04:50:38.107 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:38.107 INFO:teuthology.orchestra.run.smithi002.stdout: "quincy": [ 2023-09-13T04:50:38.107 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2023-09-13T04:50:38.107 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2023-09-13T04:50:38.108 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2023-09-13T04:50:38.108 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2023-09-13T04:50:38.108 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2023-09-13T04:50:38.108 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2023-09-13T04:50:38.108 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2023-09-13T04:50:38.109 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2023-09-13T04:50:38.109 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2023-09-13T04:50:38.109 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2023-09-13T04:50:38.109 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-09-13T04:50:38.109 INFO:teuthology.orchestra.run.smithi002.stdout: "reef": [ 2023-09-13T04:50:38.109 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2023-09-13T04:50:38.110 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2023-09-13T04:50:38.110 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2023-09-13T04:50:38.110 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2023-09-13T04:50:38.110 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2023-09-13T04:50:38.110 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2023-09-13T04:50:38.111 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2023-09-13T04:50:38.111 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2023-09-13T04:50:38.111 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2023-09-13T04:50:38.111 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2023-09-13T04:50:38.111 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2023-09-13T04:50:38.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.112 INFO:teuthology.orchestra.run.smithi002.stdout: "last_failure_osd_epoch": 93, 2023-09-13T04:50:38.112 INFO:teuthology.orchestra.run.smithi002.stdout: "active_clients": [ 2023-09-13T04:50:38.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:38.112 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "devicehealth", 2023-09-13T04:50:38.112 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2023-09-13T04:50:38.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:38.113 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2023-09-13T04:50:38.113 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2023-09-13T04:50:38.113 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 576028891 2023-09-13T04:50:38.113 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:38.114 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2023-09-13T04:50:38.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:38.114 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "libcephsqlite", 2023-09-13T04:50:38.114 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2023-09-13T04:50:38.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:38.115 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2023-09-13T04:50:38.115 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2023-09-13T04:50:38.115 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2060828802 2023-09-13T04:50:38.115 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:38.115 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2023-09-13T04:50:38.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:38.116 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_support", 2023-09-13T04:50:38.116 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2023-09-13T04:50:38.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:38.117 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2023-09-13T04:50:38.117 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2023-09-13T04:50:38.117 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2393797677 2023-09-13T04:50:38.117 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:38.117 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2023-09-13T04:50:38.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:38.118 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snap_schedule", 2023-09-13T04:50:38.118 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2023-09-13T04:50:38.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:38.118 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2023-09-13T04:50:38.119 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2023-09-13T04:50:38.119 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 326577432 2023-09-13T04:50:38.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:38.119 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2023-09-13T04:50:38.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:38.120 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snap_schedule", 2023-09-13T04:50:38.120 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2023-09-13T04:50:38.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:38.120 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2023-09-13T04:50:38.120 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2023-09-13T04:50:38.121 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 871842601 2023-09-13T04:50:38.121 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:38.121 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2023-09-13T04:50:38.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:38.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:38.122 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "volumes", 2023-09-13T04:50:38.122 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2023-09-13T04:50:38.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-09-13T04:50:38.122 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2023-09-13T04:50:38.122 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2023-09-13T04:50:38.123 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1746343540 2023-09-13T04:50:38.123 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:38.123 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2023-09-13T04:50:38.123 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:38.123 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2023-09-13T04:50:38.124 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:50:38.126 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:35 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3071499118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:50:38.126 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:35 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/853484461' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-09-13T04:50:38.126 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:36 smithi156 ceph-mon[133675]: pgmap v316: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2023-09-13T04:50:38.126 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:35 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3071499118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:50:38.127 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:35 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/853484461' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-09-13T04:50:38.127 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:36 smithi149 ceph-mon[133315]: pgmap v316: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2023-09-13T04:50:38.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.631+0000 7f58755ff700 1 -- 172.21.15.2:0/853484461 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f585c0a07f0 msgr2=0x7f585c0a2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:38.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.631+0000 7f58755ff700 1 --2- 172.21.15.2:0/853484461 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f585c0a07f0 0x7f585c0a2cb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f58701983c0 tx=0x7f5858005fd0 comp rx=0 tx=0).stop 2023-09-13T04:50:38.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.631+0000 7f58755ff700 1 -- 172.21.15.2:0/853484461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5870100ed0 msgr2=0x7f5870196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:50:38.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.631+0000 7f58755ff700 1 --2- 172.21.15.2:0/853484461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5870100ed0 0x7f5870196db0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f586000e9a0 tx=0x7f586000ecb0 comp rx=0 tx=0).stop 2023-09-13T04:50:38.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.631+0000 7f58755ff700 1 -- 172.21.15.2:0/853484461 shutdown_connections 2023-09-13T04:50:38.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.631+0000 7f58755ff700 1 --2- 172.21.15.2:0/853484461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f587019b750 0x7f587019bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:38.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.631+0000 7f58755ff700 1 --2- 172.21.15.2:0/853484461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5870100ed0 0x7f5870196db0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:38.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.631+0000 7f58755ff700 1 --2- 172.21.15.2:0/853484461 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f585c0a07f0 0x7f585c0a2cb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:38.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.631+0000 7f58755ff700 1 --2- 172.21.15.2:0/853484461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5870105ff0 0x7f58701972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:50:38.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.631+0000 7f58755ff700 1 -- 172.21.15.2:0/853484461 >> 172.21.15.2:0/853484461 conn(0x7f58700fc790 msgr2=0x7f58700fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:50:38.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.631+0000 7f58755ff700 1 -- 172.21.15.2:0/853484461 shutdown_connections 2023-09-13T04:50:38.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:50:34.631+0000 7f58755ff700 1 -- 172.21.15.2:0/853484461 wait complete. 2023-09-13T04:50:38.169 DEBUG:tasks.check_counter:Getting stats from a 2023-09-13T04:50:38.169 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph daemon mds.a perf dump 2023-09-13T04:50:38.170 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:35 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3071499118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:50:38.170 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:35 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/853484461' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-09-13T04:50:38.170 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:36 smithi002 ceph-mon[97651]: pgmap v316: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2023-09-13T04:50:38.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:38 smithi149 ceph-mon[133315]: pgmap v317: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 7 op/s 2023-09-13T04:50:38.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:38 smithi156 ceph-mon[133675]: pgmap v317: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 7 op/s 2023-09-13T04:50:38.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:38 smithi002 ceph-mon[97651]: pgmap v317: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 7 op/s 2023-09-13T04:50:38.914 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:50:40.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:40 smithi149 ceph-mon[133315]: pgmap v318: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2023-09-13T04:50:40.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:40 smithi149 ceph-mon[133315]: osdmap e104: 12 total, 12 up, 12 in 2023-09-13T04:50:40.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:40 smithi156 ceph-mon[133675]: pgmap v318: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2023-09-13T04:50:40.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:40 smithi156 ceph-mon[133675]: osdmap e104: 12 total, 12 up, 12 in 2023-09-13T04:50:40.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:40 smithi002 ceph-mon[97651]: pgmap v318: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2023-09-13T04:50:40.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:40 smithi002 ceph-mon[97651]: osdmap e104: 12 total, 12 up, 12 in 2023-09-13T04:50:40.529 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:50:40.530 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-0": { 2023-09-13T04:50:40.530 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 150859, 2023-09-13T04:50:40.530 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 161965, 2023-09-13T04:50:40.530 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 43594719, 2023-09-13T04:50:40.531 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 204282465, 2023-09-13T04:50:40.531 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 11, 2023-09-13T04:50:40.531 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 5, 2023-09-13T04:50:40.531 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 13.179953583, 2023-09-13T04:50:40.532 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 5.188570994, 2023-09-13T04:50:40.532 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 8.697588699, 2023-09-13T04:50:40.532 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 1.727365387, 2023-09-13T04:50:40.532 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:50:40.532 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 161963, 2023-09-13T04:50:40.533 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 145.602099037, 2023-09-13T04:50:40.533 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000898983 2023-09-13T04:50:40.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.533 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:50:40.533 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 256486, 2023-09-13T04:50:40.534 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.113215574, 2023-09-13T04:50:40.534 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000441 2023-09-13T04:50:40.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.534 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-13T04:50:40.534 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 0 2023-09-13T04:50:40.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.535 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-1": { 2023-09-13T04:50:40.537 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 467109, 2023-09-13T04:50:40.537 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 573182, 2023-09-13T04:50:40.537 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 114448650, 2023-09-13T04:50:40.538 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 286508082, 2023-09-13T04:50:40.538 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 20, 2023-09-13T04:50:40.538 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 4, 2023-09-13T04:50:40.538 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 40.715843997, 2023-09-13T04:50:40.538 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 24.282686731, 2023-09-13T04:50:40.539 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 30.107816606, 2023-09-13T04:50:40.539 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.885892314, 2023-09-13T04:50:40.539 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:50:40.539 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 573181, 2023-09-13T04:50:40.539 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 709.356632959, 2023-09-13T04:50:40.540 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.001237578 2023-09-13T04:50:40.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.540 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:50:40.540 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 688530, 2023-09-13T04:50:40.540 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.520772527, 2023-09-13T04:50:40.541 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000756 2023-09-13T04:50:40.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.541 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 1776, 2023-09-13T04:50:40.541 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 129424 2023-09-13T04:50:40.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.542 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-2": { 2023-09-13T04:50:40.542 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 15572, 2023-09-13T04:50:40.542 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 15667, 2023-09-13T04:50:40.542 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 7893405, 2023-09-13T04:50:40.542 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 89435773, 2023-09-13T04:50:40.543 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 12, 2023-09-13T04:50:40.543 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 7, 2023-09-13T04:50:40.543 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 2.972524635, 2023-09-13T04:50:40.543 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.766303709, 2023-09-13T04:50:40.543 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.463804155, 2023-09-13T04:50:40.544 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 1.620233697, 2023-09-13T04:50:40.544 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:50:40.544 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 15665, 2023-09-13T04:50:40.544 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 10.281542510, 2023-09-13T04:50:40.544 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000656338 2023-09-13T04:50:40.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.545 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:50:40.545 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 4, 2023-09-13T04:50:40.545 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000458, 2023-09-13T04:50:40.545 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000114 2023-09-13T04:50:40.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.546 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 1523360, 2023-09-13T04:50:40.546 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 458064 2023-09-13T04:50:40.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.547 INFO:teuthology.orchestra.run.smithi002.stdout: "cct": { 2023-09-13T04:50:40.547 INFO:teuthology.orchestra.run.smithi002.stdout: "total_workers": 1, 2023-09-13T04:50:40.547 INFO:teuthology.orchestra.run.smithi002.stdout: "unhealthy_workers": 0 2023-09-13T04:50:40.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.547 INFO:teuthology.orchestra.run.smithi002.stdout: "finisher-MDSRank": { 2023-09-13T04:50:40.548 INFO:teuthology.orchestra.run.smithi002.stdout: "queue_len": 0, 2023-09-13T04:50:40.548 INFO:teuthology.orchestra.run.smithi002.stdout: "complete_latency": { 2023-09-13T04:50:40.548 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 7559, 2023-09-13T04:50:40.548 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 132.575447335, 2023-09-13T04:50:40.549 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.017538754 2023-09-13T04:50:40.549 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:40.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.549 INFO:teuthology.orchestra.run.smithi002.stdout: "finisher-PurgeQueue": { 2023-09-13T04:50:40.549 INFO:teuthology.orchestra.run.smithi002.stdout: "queue_len": 0, 2023-09-13T04:50:40.550 INFO:teuthology.orchestra.run.smithi002.stdout: "complete_latency": { 2023-09-13T04:50:40.550 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 24022, 2023-09-13T04:50:40.550 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 15.853978456, 2023-09-13T04:50:40.551 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000659977 2023-09-13T04:50:40.551 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:40.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.551 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2023-09-13T04:50:40.551 INFO:teuthology.orchestra.run.smithi002.stdout: "request": 114705, 2023-09-13T04:50:40.552 INFO:teuthology.orchestra.run.smithi002.stdout: "reply": 104951, 2023-09-13T04:50:40.552 INFO:teuthology.orchestra.run.smithi002.stdout: "reply_latency": { 2023-09-13T04:50:40.552 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 104951, 2023-09-13T04:50:40.552 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 6195.271399175, 2023-09-13T04:50:40.552 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.059030132 2023-09-13T04:50:40.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.553 INFO:teuthology.orchestra.run.smithi002.stdout: "slow_reply": 0, 2023-09-13T04:50:40.553 INFO:teuthology.orchestra.run.smithi002.stdout: "forward": 9747, 2023-09-13T04:50:40.553 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_fetch_complete": 65, 2023-09-13T04:50:40.553 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_fetch_keys": 0, 2023-09-13T04:50:40.554 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_commit": 604, 2023-09-13T04:50:40.554 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_split": 66, 2023-09-13T04:50:40.554 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_merge": 1, 2023-09-13T04:50:40.554 INFO:teuthology.orchestra.run.smithi002.stdout: "inodes": 32340, 2023-09-13T04:50:40.554 INFO:teuthology.orchestra.run.smithi002.stdout: "inodes_top": 1109, 2023-09-13T04:50:40.555 INFO:teuthology.orchestra.run.smithi002.stdout: "inodes_bottom": 15554, 2023-09-13T04:50:40.555 INFO:teuthology.orchestra.run.smithi002.stdout: "inodes_pin_tail": 15677, 2023-09-13T04:50:40.555 INFO:teuthology.orchestra.run.smithi002.stdout: "inodes_pinned": 30754, 2023-09-13T04:50:40.555 INFO:teuthology.orchestra.run.smithi002.stdout: "inodes_expired": 39427, 2023-09-13T04:50:40.555 INFO:teuthology.orchestra.run.smithi002.stdout: "inodes_with_caps": 0, 2023-09-13T04:50:40.556 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": 0, 2023-09-13T04:50:40.556 INFO:teuthology.orchestra.run.smithi002.stdout: "subtrees": 3, 2023-09-13T04:50:40.556 INFO:teuthology.orchestra.run.smithi002.stdout: "traverse": 190656, 2023-09-13T04:50:40.556 INFO:teuthology.orchestra.run.smithi002.stdout: "traverse_hit": 158559, 2023-09-13T04:50:40.556 INFO:teuthology.orchestra.run.smithi002.stdout: "traverse_forward": 9578, 2023-09-13T04:50:40.557 INFO:teuthology.orchestra.run.smithi002.stdout: "traverse_discover": 656, 2023-09-13T04:50:40.557 INFO:teuthology.orchestra.run.smithi002.stdout: "traverse_dir_fetch": 11, 2023-09-13T04:50:40.557 INFO:teuthology.orchestra.run.smithi002.stdout: "traverse_remote_ino": 0, 2023-09-13T04:50:40.557 INFO:teuthology.orchestra.run.smithi002.stdout: "traverse_lock": 875, 2023-09-13T04:50:40.557 INFO:teuthology.orchestra.run.smithi002.stdout: "load_cent": 0, 2023-09-13T04:50:40.558 INFO:teuthology.orchestra.run.smithi002.stdout: "q": 0, 2023-09-13T04:50:40.558 INFO:teuthology.orchestra.run.smithi002.stdout: "exported": 158, 2023-09-13T04:50:40.558 INFO:teuthology.orchestra.run.smithi002.stdout: "exported_inodes": 30664, 2023-09-13T04:50:40.558 INFO:teuthology.orchestra.run.smithi002.stdout: "imported": 243, 2023-09-13T04:50:40.558 INFO:teuthology.orchestra.run.smithi002.stdout: "imported_inodes": 25567, 2023-09-13T04:50:40.559 INFO:teuthology.orchestra.run.smithi002.stdout: "openino_dir_fetch": 0, 2023-09-13T04:50:40.559 INFO:teuthology.orchestra.run.smithi002.stdout: "openino_backtrace_fetch": 3764, 2023-09-13T04:50:40.559 INFO:teuthology.orchestra.run.smithi002.stdout: "openino_peer_discover": 0, 2023-09-13T04:50:40.559 INFO:teuthology.orchestra.run.smithi002.stdout: "root_rfiles": 0, 2023-09-13T04:50:40.559 INFO:teuthology.orchestra.run.smithi002.stdout: "root_rbytes": 0, 2023-09-13T04:50:40.560 INFO:teuthology.orchestra.run.smithi002.stdout: "root_rsnaps": 6, 2023-09-13T04:50:40.560 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_backtrace_fetch": 3764, 2023-09-13T04:50:40.560 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_set_tag": 0, 2023-09-13T04:50:40.560 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_backtrace_repaired": 0, 2023-09-13T04:50:40.560 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_inotable_repaired": 0, 2023-09-13T04:50:40.561 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_dir_inodes": 39, 2023-09-13T04:50:40.561 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_dir_base_inodes": 0, 2023-09-13T04:50:40.561 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_dirfrag_rstats": 39, 2023-09-13T04:50:40.562 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_file_inodes": 3725, 2023-09-13T04:50:40.562 INFO:teuthology.orchestra.run.smithi002.stdout: "handle_inode_file_caps": 95, 2023-09-13T04:50:40.562 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_cap_op_revoke": 19418, 2023-09-13T04:50:40.563 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_cap_op_grant": 37839, 2023-09-13T04:50:40.563 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_cap_op_trunc": 1, 2023-09-13T04:50:40.564 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_cap_op_flushsnap_ack": 24, 2023-09-13T04:50:40.564 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-13T04:50:40.564 INFO:teuthology.orchestra.run.smithi002.stdout: "handle_client_caps": 46566, 2023-09-13T04:50:40.565 INFO:teuthology.orchestra.run.smithi002.stdout: "handle_client_caps_dirty": 19652, 2023-09-13T04:50:40.565 INFO:teuthology.orchestra.run.smithi002.stdout: "handle_client_cap_release": 1045, 2023-09-13T04:50:40.565 INFO:teuthology.orchestra.run.smithi002.stdout: "process_request_cap_release": 38457 2023-09-13T04:50:40.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.566 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache": { 2023-09-13T04:50:40.566 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_update": 1080, 2023-09-13T04:50:40.567 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_update_receipt": 1032, 2023-09-13T04:50:40.567 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_try_discover": 549, 2023-09-13T04:50:40.567 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_send_discover": 1495, 2023-09-13T04:50:40.567 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_handle_discover": 5603, 2023-09-13T04:50:40.568 INFO:teuthology.orchestra.run.smithi002.stdout: "num_strays": 82, 2023-09-13T04:50:40.568 INFO:teuthology.orchestra.run.smithi002.stdout: "num_strays_delayed": 0, 2023-09-13T04:50:40.568 INFO:teuthology.orchestra.run.smithi002.stdout: "num_strays_enqueuing": 0, 2023-09-13T04:50:40.569 INFO:teuthology.orchestra.run.smithi002.stdout: "strays_created": 15460, 2023-09-13T04:50:40.569 INFO:teuthology.orchestra.run.smithi002.stdout: "strays_enqueued": 15378, 2023-09-13T04:50:40.569 INFO:teuthology.orchestra.run.smithi002.stdout: "strays_reintegrated": 0, 2023-09-13T04:50:40.569 INFO:teuthology.orchestra.run.smithi002.stdout: "strays_migrated": 0, 2023-09-13T04:50:40.570 INFO:teuthology.orchestra.run.smithi002.stdout: "num_recovering_processing": 0, 2023-09-13T04:50:40.570 INFO:teuthology.orchestra.run.smithi002.stdout: "num_recovering_enqueued": 0, 2023-09-13T04:50:40.570 INFO:teuthology.orchestra.run.smithi002.stdout: "num_recovering_prioritized": 0, 2023-09-13T04:50:40.571 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_started": 0, 2023-09-13T04:50:40.571 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_completed": 0, 2023-09-13T04:50:40.571 INFO:teuthology.orchestra.run.smithi002.stdout: "ireq_enqueue_scrub": 0, 2023-09-13T04:50:40.571 INFO:teuthology.orchestra.run.smithi002.stdout: "ireq_exportdir": 158, 2023-09-13T04:50:40.572 INFO:teuthology.orchestra.run.smithi002.stdout: "ireq_flush": 0, 2023-09-13T04:50:40.572 INFO:teuthology.orchestra.run.smithi002.stdout: "ireq_fragmentdir": 67, 2023-09-13T04:50:40.572 INFO:teuthology.orchestra.run.smithi002.stdout: "ireq_fragstats": 0, 2023-09-13T04:50:40.572 INFO:teuthology.orchestra.run.smithi002.stdout: "ireq_inodestats": 0 2023-09-13T04:50:40.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.573 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log": { 2023-09-13T04:50:40.573 INFO:teuthology.orchestra.run.smithi002.stdout: "evlrg": 5, 2023-09-13T04:50:40.573 INFO:teuthology.orchestra.run.smithi002.stdout: "evadd": 98286, 2023-09-13T04:50:40.573 INFO:teuthology.orchestra.run.smithi002.stdout: "evex": 8402, 2023-09-13T04:50:40.573 INFO:teuthology.orchestra.run.smithi002.stdout: "evtrm": 1, 2023-09-13T04:50:40.574 INFO:teuthology.orchestra.run.smithi002.stdout: "ev": 98285, 2023-09-13T04:50:40.574 INFO:teuthology.orchestra.run.smithi002.stdout: "evexg": 0, 2023-09-13T04:50:40.574 INFO:teuthology.orchestra.run.smithi002.stdout: "evexd": 8401, 2023-09-13T04:50:40.574 INFO:teuthology.orchestra.run.smithi002.stdout: "segadd": 140, 2023-09-13T04:50:40.574 INFO:teuthology.orchestra.run.smithi002.stdout: "segex": 12, 2023-09-13T04:50:40.575 INFO:teuthology.orchestra.run.smithi002.stdout: "segtrm": 1, 2023-09-13T04:50:40.575 INFO:teuthology.orchestra.run.smithi002.stdout: "seg": 139, 2023-09-13T04:50:40.575 INFO:teuthology.orchestra.run.smithi002.stdout: "segmjr": 8, 2023-09-13T04:50:40.576 INFO:teuthology.orchestra.run.smithi002.stdout: "segexg": 0, 2023-09-13T04:50:40.576 INFO:teuthology.orchestra.run.smithi002.stdout: "segexd": 11, 2023-09-13T04:50:40.576 INFO:teuthology.orchestra.run.smithi002.stdout: "expos": 4194352, 2023-09-13T04:50:40.577 INFO:teuthology.orchestra.run.smithi002.stdout: "wrpos": 341284086, 2023-09-13T04:50:40.577 INFO:teuthology.orchestra.run.smithi002.stdout: "rdpos": 0, 2023-09-13T04:50:40.577 INFO:teuthology.orchestra.run.smithi002.stdout: "jlat": { 2023-09-13T04:50:40.578 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 4163, 2023-09-13T04:50:40.578 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 768.431358657, 2023-09-13T04:50:40.578 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.184585961 2023-09-13T04:50:40.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.579 INFO:teuthology.orchestra.run.smithi002.stdout: "replayed": 0 2023-09-13T04:50:40.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.580 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_mem": { 2023-09-13T04:50:40.580 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 15910, 2023-09-13T04:50:40.581 INFO:teuthology.orchestra.run.smithi002.stdout: "ino+": 50959, 2023-09-13T04:50:40.581 INFO:teuthology.orchestra.run.smithi002.stdout: "ino-": 35049, 2023-09-13T04:50:40.581 INFO:teuthology.orchestra.run.smithi002.stdout: "dir": 592, 2023-09-13T04:50:40.582 INFO:teuthology.orchestra.run.smithi002.stdout: "dir+": 1954, 2023-09-13T04:50:40.582 INFO:teuthology.orchestra.run.smithi002.stdout: "dir-": 1362, 2023-09-13T04:50:40.582 INFO:teuthology.orchestra.run.smithi002.stdout: "dn": 32340, 2023-09-13T04:50:40.583 INFO:teuthology.orchestra.run.smithi002.stdout: "dn+": 90556, 2023-09-13T04:50:40.583 INFO:teuthology.orchestra.run.smithi002.stdout: "dn-": 58216, 2023-09-13T04:50:40.583 INFO:teuthology.orchestra.run.smithi002.stdout: "cap": 0, 2023-09-13T04:50:40.583 INFO:teuthology.orchestra.run.smithi002.stdout: "cap+": 33419, 2023-09-13T04:50:40.584 INFO:teuthology.orchestra.run.smithi002.stdout: "cap-": 33419, 2023-09-13T04:50:40.584 INFO:teuthology.orchestra.run.smithi002.stdout: "rss": 173856, 2023-09-13T04:50:40.584 INFO:teuthology.orchestra.run.smithi002.stdout: "heap": 198960 2023-09-13T04:50:40.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.584 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_server": { 2023-09-13T04:50:40.585 INFO:teuthology.orchestra.run.smithi002.stdout: "dispatch_client_request": 142968, 2023-09-13T04:50:40.585 INFO:teuthology.orchestra.run.smithi002.stdout: "dispatch_server_request": 3093, 2023-09-13T04:50:40.585 INFO:teuthology.orchestra.run.smithi002.stdout: "handle_client_request": 114705, 2023-09-13T04:50:40.585 INFO:teuthology.orchestra.run.smithi002.stdout: "handle_client_session": 298, 2023-09-13T04:50:40.585 INFO:teuthology.orchestra.run.smithi002.stdout: "handle_peer_request": 5422, 2023-09-13T04:50:40.586 INFO:teuthology.orchestra.run.smithi002.stdout: "req_create_latency": { 2023-09-13T04:50:40.586 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 19273, 2023-09-13T04:50:40.586 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 344.535308737, 2023-09-13T04:50:40.586 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.017876579 2023-09-13T04:50:40.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.587 INFO:teuthology.orchestra.run.smithi002.stdout: "req_getattr_latency": { 2023-09-13T04:50:40.587 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 2037, 2023-09-13T04:50:40.587 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 440.722873699, 2023-09-13T04:50:40.587 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.216358799 2023-09-13T04:50:40.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.588 INFO:teuthology.orchestra.run.smithi002.stdout: "req_getfilelock_latency": { 2023-09-13T04:50:40.588 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.588 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.588 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.589 INFO:teuthology.orchestra.run.smithi002.stdout: "req_link_latency": { 2023-09-13T04:50:40.589 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.589 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.589 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.590 INFO:teuthology.orchestra.run.smithi002.stdout: "req_lookup_latency": { 2023-09-13T04:50:40.590 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 47575, 2023-09-13T04:50:40.590 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 4292.966935685, 2023-09-13T04:50:40.591 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.090235773 2023-09-13T04:50:40.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.591 INFO:teuthology.orchestra.run.smithi002.stdout: "req_lookuphash_latency": { 2023-09-13T04:50:40.591 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.592 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.592 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.593 INFO:teuthology.orchestra.run.smithi002.stdout: "req_lookupino_latency": { 2023-09-13T04:50:40.593 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.593 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.594 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.594 INFO:teuthology.orchestra.run.smithi002.stdout: "req_lookupname_latency": { 2023-09-13T04:50:40.594 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.594 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.595 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.595 INFO:teuthology.orchestra.run.smithi002.stdout: "req_lookupparent_latency": { 2023-09-13T04:50:40.595 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.595 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.596 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.596 INFO:teuthology.orchestra.run.smithi002.stdout: "req_lookupsnap_latency": { 2023-09-13T04:50:40.596 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.596 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.597 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.597 INFO:teuthology.orchestra.run.smithi002.stdout: "req_lssnap_latency": { 2023-09-13T04:50:40.597 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.597 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.598 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.598 INFO:teuthology.orchestra.run.smithi002.stdout: "req_mkdir_latency": { 2023-09-13T04:50:40.598 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 118, 2023-09-13T04:50:40.599 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 9.125415851, 2023-09-13T04:50:40.599 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.077334032 2023-09-13T04:50:40.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.599 INFO:teuthology.orchestra.run.smithi002.stdout: "req_mknod_latency": { 2023-09-13T04:50:40.599 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.600 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.600 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.601 INFO:teuthology.orchestra.run.smithi002.stdout: "req_mksnap_latency": { 2023-09-13T04:50:40.601 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.601 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.602 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.603 INFO:teuthology.orchestra.run.smithi002.stdout: "req_open_latency": { 2023-09-13T04:50:40.603 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 3854, 2023-09-13T04:50:40.603 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 71.503389303, 2023-09-13T04:50:40.604 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.018553033 2023-09-13T04:50:40.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.604 INFO:teuthology.orchestra.run.smithi002.stdout: "req_readdir_latency": { 2023-09-13T04:50:40.605 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 546, 2023-09-13T04:50:40.605 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 2.125121529, 2023-09-13T04:50:40.605 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.003892163 2023-09-13T04:50:40.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.606 INFO:teuthology.orchestra.run.smithi002.stdout: "req_rename_latency": { 2023-09-13T04:50:40.607 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 19318, 2023-09-13T04:50:40.607 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 994.938017219, 2023-09-13T04:50:40.607 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.051503158 2023-09-13T04:50:40.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.608 INFO:teuthology.orchestra.run.smithi002.stdout: "req_renamesnap_latency": { 2023-09-13T04:50:40.608 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.609 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.609 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.610 INFO:teuthology.orchestra.run.smithi002.stdout: "req_snapdiff_latency": { 2023-09-13T04:50:40.610 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.610 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.611 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.611 INFO:teuthology.orchestra.run.smithi002.stdout: "req_rmdir_latency": { 2023-09-13T04:50:40.612 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 83, 2023-09-13T04:50:40.612 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 15.382854960, 2023-09-13T04:50:40.612 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.185335601 2023-09-13T04:50:40.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.613 INFO:teuthology.orchestra.run.smithi002.stdout: "req_rmsnap_latency": { 2023-09-13T04:50:40.613 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.614 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.614 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.615 INFO:teuthology.orchestra.run.smithi002.stdout: "req_rmxattr_latency": { 2023-09-13T04:50:40.615 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.616 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.616 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.617 INFO:teuthology.orchestra.run.smithi002.stdout: "req_setattr_latency": { 2023-09-13T04:50:40.617 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 1, 2023-09-13T04:50:40.618 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.088909247, 2023-09-13T04:50:40.618 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.088909247 2023-09-13T04:50:40.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.619 INFO:teuthology.orchestra.run.smithi002.stdout: "req_setdirlayout_latency": { 2023-09-13T04:50:40.619 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.619 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.619 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.620 INFO:teuthology.orchestra.run.smithi002.stdout: "req_setfilelock_latency": { 2023-09-13T04:50:40.620 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.620 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.620 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.621 INFO:teuthology.orchestra.run.smithi002.stdout: "req_setlayout_latency": { 2023-09-13T04:50:40.621 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.621 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.621 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.622 INFO:teuthology.orchestra.run.smithi002.stdout: "req_setxattr_latency": { 2023-09-13T04:50:40.622 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.623 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.623 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.623 INFO:teuthology.orchestra.run.smithi002.stdout: "req_symlink_latency": { 2023-09-13T04:50:40.624 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.624 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.624 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.625 INFO:teuthology.orchestra.run.smithi002.stdout: "req_unlink_latency": { 2023-09-13T04:50:40.625 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 12146, 2023-09-13T04:50:40.626 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 23.882572945, 2023-09-13T04:50:40.626 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.001966291 2023-09-13T04:50:40.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.627 INFO:teuthology.orchestra.run.smithi002.stdout: "cap_revoke_eviction": 0, 2023-09-13T04:50:40.627 INFO:teuthology.orchestra.run.smithi002.stdout: "cap_acquisition_throttle": 0, 2023-09-13T04:50:40.628 INFO:teuthology.orchestra.run.smithi002.stdout: "req_getvxattr_latency": { 2023-09-13T04:50:40.628 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.628 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.629 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.629 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:40.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.630 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_sessions": { 2023-09-13T04:50:40.630 INFO:teuthology.orchestra.run.smithi002.stdout: "session_count": 1, 2023-09-13T04:50:40.630 INFO:teuthology.orchestra.run.smithi002.stdout: "session_add": 2, 2023-09-13T04:50:40.631 INFO:teuthology.orchestra.run.smithi002.stdout: "session_remove": 1, 2023-09-13T04:50:40.631 INFO:teuthology.orchestra.run.smithi002.stdout: "sessions_open": 1, 2023-09-13T04:50:40.632 INFO:teuthology.orchestra.run.smithi002.stdout: "sessions_stale": 0, 2023-09-13T04:50:40.632 INFO:teuthology.orchestra.run.smithi002.stdout: "total_load": 16710, 2023-09-13T04:50:40.632 INFO:teuthology.orchestra.run.smithi002.stdout: "average_load": 16710, 2023-09-13T04:50:40.633 INFO:teuthology.orchestra.run.smithi002.stdout: "avg_session_uptime": 536, 2023-09-13T04:50:40.633 INFO:teuthology.orchestra.run.smithi002.stdout: "mdthresh_evicted": 0 2023-09-13T04:50:40.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.634 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool": { 2023-09-13T04:50:40.634 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_bytes": 0, 2023-09-13T04:50:40.635 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_items": 0, 2023-09-13T04:50:40.635 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_bytes": 0, 2023-09-13T04:50:40.635 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_items": 0, 2023-09-13T04:50:40.636 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_bytes": 0, 2023-09-13T04:50:40.636 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_items": 0, 2023-09-13T04:50:40.637 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-13T04:50:40.637 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_items": 0, 2023-09-13T04:50:40.638 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-13T04:50:40.638 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_items": 0, 2023-09-13T04:50:40.639 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_bytes": 0, 2023-09-13T04:50:40.639 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_items": 0, 2023-09-13T04:50:40.640 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-13T04:50:40.640 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_items": 0, 2023-09-13T04:50:40.641 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_bytes": 0, 2023-09-13T04:50:40.641 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_items": 0, 2023-09-13T04:50:40.641 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_bytes": 0, 2023-09-13T04:50:40.641 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_items": 0, 2023-09-13T04:50:40.642 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-13T04:50:40.642 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_items": 0, 2023-09-13T04:50:40.642 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-13T04:50:40.642 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_items": 0, 2023-09-13T04:50:40.643 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_bytes": 0, 2023-09-13T04:50:40.643 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_items": 0, 2023-09-13T04:50:40.643 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_bytes": 0, 2023-09-13T04:50:40.643 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_items": 0, 2023-09-13T04:50:40.643 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-13T04:50:40.644 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_items": 0, 2023-09-13T04:50:40.644 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_bytes": 0, 2023-09-13T04:50:40.644 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_items": 0, 2023-09-13T04:50:40.644 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_bytes": 0, 2023-09-13T04:50:40.644 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_items": 0, 2023-09-13T04:50:40.645 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_bytes": 0, 2023-09-13T04:50:40.645 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_items": 0, 2023-09-13T04:50:40.645 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_bytes": 0, 2023-09-13T04:50:40.645 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_items": 0, 2023-09-13T04:50:40.646 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_bytes": 1502245, 2023-09-13T04:50:40.646 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_items": 16316, 2023-09-13T04:50:40.646 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_bytes": 88, 2023-09-13T04:50:40.646 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_items": 1, 2023-09-13T04:50:40.646 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bytes": 0, 2023-09-13T04:50:40.647 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_items": 0, 2023-09-13T04:50:40.647 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_bytes": 0, 2023-09-13T04:50:40.647 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_items": 0, 2023-09-13T04:50:40.647 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_bytes": 0, 2023-09-13T04:50:40.647 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_items": 0, 2023-09-13T04:50:40.648 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_bytes": 9656, 2023-09-13T04:50:40.648 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_items": 186, 2023-09-13T04:50:40.648 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_bytes": 0, 2023-09-13T04:50:40.648 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_items": 0, 2023-09-13T04:50:40.648 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_bytes": 0, 2023-09-13T04:50:40.648 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_items": 0, 2023-09-13T04:50:40.649 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_bytes": 61029288, 2023-09-13T04:50:40.649 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_items": 676405, 2023-09-13T04:50:40.649 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_bytes": 0, 2023-09-13T04:50:40.649 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_items": 0, 2023-09-13T04:50:40.649 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_bytes": 0, 2023-09-13T04:50:40.650 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_items": 0 2023-09-13T04:50:40.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.650 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter": { 2023-09-13T04:50:40.650 INFO:teuthology.orchestra.run.smithi002.stdout: "op_active": 0, 2023-09-13T04:50:40.651 INFO:teuthology.orchestra.run.smithi002.stdout: "op_laggy": 0, 2023-09-13T04:50:40.651 INFO:teuthology.orchestra.run.smithi002.stdout: "op_send": 41754, 2023-09-13T04:50:40.651 INFO:teuthology.orchestra.run.smithi002.stdout: "op_send_bytes": 352272602, 2023-09-13T04:50:40.651 INFO:teuthology.orchestra.run.smithi002.stdout: "op_resend": 27, 2023-09-13T04:50:40.651 INFO:teuthology.orchestra.run.smithi002.stdout: "op_reply": 41754, 2023-09-13T04:50:40.652 INFO:teuthology.orchestra.run.smithi002.stdout: "op_latency": { 2023-09-13T04:50:40.652 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 41754, 2023-09-13T04:50:40.652 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 1141.221421458, 2023-09-13T04:50:40.652 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.027332026 2023-09-13T04:50:40.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.653 INFO:teuthology.orchestra.run.smithi002.stdout: "op_inflight": 0, 2023-09-13T04:50:40.653 INFO:teuthology.orchestra.run.smithi002.stdout: "oplen_avg": { 2023-09-13T04:50:40.653 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 41754, 2023-09-13T04:50:40.653 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 42685 2023-09-13T04:50:40.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.654 INFO:teuthology.orchestra.run.smithi002.stdout: "op": 41754, 2023-09-13T04:50:40.654 INFO:teuthology.orchestra.run.smithi002.stdout: "op_r": 4795, 2023-09-13T04:50:40.654 INFO:teuthology.orchestra.run.smithi002.stdout: "op_w": 36959, 2023-09-13T04:50:40.654 INFO:teuthology.orchestra.run.smithi002.stdout: "op_rmw": 0, 2023-09-13T04:50:40.654 INFO:teuthology.orchestra.run.smithi002.stdout: "op_pg": 0, 2023-09-13T04:50:40.655 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_stat": 0, 2023-09-13T04:50:40.655 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_create": 20, 2023-09-13T04:50:40.655 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_read": 746, 2023-09-13T04:50:40.655 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_write": 4912, 2023-09-13T04:50:40.655 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_writefull": 406, 2023-09-13T04:50:40.656 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_writesame": 0, 2023-09-13T04:50:40.656 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_append": 0, 2023-09-13T04:50:40.656 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_zero": 0, 2023-09-13T04:50:40.657 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_truncate": 66, 2023-09-13T04:50:40.657 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_delete": 30863, 2023-09-13T04:50:40.658 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_mapext": 0, 2023-09-13T04:50:40.658 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_sparse_read": 0, 2023-09-13T04:50:40.659 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_clonerange": 0, 2023-09-13T04:50:40.659 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_getxattr": 3766, 2023-09-13T04:50:40.659 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_setxattr": 40, 2023-09-13T04:50:40.659 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_cmpxattr": 0, 2023-09-13T04:50:40.660 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_rmxattr": 0, 2023-09-13T04:50:40.660 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_resetxattrs": 0, 2023-09-13T04:50:40.661 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_call": 0, 2023-09-13T04:50:40.661 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_watch": 0, 2023-09-13T04:50:40.661 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_notify": 0, 2023-09-13T04:50:40.661 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_src_cmpxattr": 0, 2023-09-13T04:50:40.662 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_pgls": 0, 2023-09-13T04:50:40.662 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_pgls_filter": 0, 2023-09-13T04:50:40.662 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_other": 1, 2023-09-13T04:50:40.663 INFO:teuthology.orchestra.run.smithi002.stdout: "linger_active": 0, 2023-09-13T04:50:40.663 INFO:teuthology.orchestra.run.smithi002.stdout: "linger_send": 0, 2023-09-13T04:50:40.663 INFO:teuthology.orchestra.run.smithi002.stdout: "linger_resend": 0, 2023-09-13T04:50:40.664 INFO:teuthology.orchestra.run.smithi002.stdout: "linger_ping": 0, 2023-09-13T04:50:40.664 INFO:teuthology.orchestra.run.smithi002.stdout: "poolop_active": 0, 2023-09-13T04:50:40.664 INFO:teuthology.orchestra.run.smithi002.stdout: "poolop_send": 0, 2023-09-13T04:50:40.664 INFO:teuthology.orchestra.run.smithi002.stdout: "poolop_resend": 0, 2023-09-13T04:50:40.664 INFO:teuthology.orchestra.run.smithi002.stdout: "poolstat_active": 0, 2023-09-13T04:50:40.665 INFO:teuthology.orchestra.run.smithi002.stdout: "poolstat_send": 0, 2023-09-13T04:50:40.665 INFO:teuthology.orchestra.run.smithi002.stdout: "poolstat_resend": 0, 2023-09-13T04:50:40.665 INFO:teuthology.orchestra.run.smithi002.stdout: "statfs_active": 0, 2023-09-13T04:50:40.665 INFO:teuthology.orchestra.run.smithi002.stdout: "statfs_send": 0, 2023-09-13T04:50:40.665 INFO:teuthology.orchestra.run.smithi002.stdout: "statfs_resend": 0, 2023-09-13T04:50:40.666 INFO:teuthology.orchestra.run.smithi002.stdout: "command_active": 0, 2023-09-13T04:50:40.666 INFO:teuthology.orchestra.run.smithi002.stdout: "command_send": 0, 2023-09-13T04:50:40.666 INFO:teuthology.orchestra.run.smithi002.stdout: "command_resend": 0, 2023-09-13T04:50:40.666 INFO:teuthology.orchestra.run.smithi002.stdout: "map_epoch": 104, 2023-09-13T04:50:40.666 INFO:teuthology.orchestra.run.smithi002.stdout: "map_full": 0, 2023-09-13T04:50:40.667 INFO:teuthology.orchestra.run.smithi002.stdout: "map_inc": 15, 2023-09-13T04:50:40.667 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_sessions": 12, 2023-09-13T04:50:40.668 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_session_open": 12, 2023-09-13T04:50:40.668 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_session_close": 0, 2023-09-13T04:50:40.669 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_laggy": 0, 2023-09-13T04:50:40.669 INFO:teuthology.orchestra.run.smithi002.stdout: "omap_wr": 1367, 2023-09-13T04:50:40.669 INFO:teuthology.orchestra.run.smithi002.stdout: "omap_rd": 350, 2023-09-13T04:50:40.670 INFO:teuthology.orchestra.run.smithi002.stdout: "omap_del": 148 2023-09-13T04:50:40.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.670 INFO:teuthology.orchestra.run.smithi002.stdout: "oft": { 2023-09-13T04:50:40.671 INFO:teuthology.orchestra.run.smithi002.stdout: "omap_total_objs": 1, 2023-09-13T04:50:40.671 INFO:teuthology.orchestra.run.smithi002.stdout: "omap_total_kv_pairs": 0, 2023-09-13T04:50:40.672 INFO:teuthology.orchestra.run.smithi002.stdout: "omap_total_updates": 0, 2023-09-13T04:50:40.672 INFO:teuthology.orchestra.run.smithi002.stdout: "omap_total_removes": 0 2023-09-13T04:50:40.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.673 INFO:teuthology.orchestra.run.smithi002.stdout: "osd-slow-ops": { 2023-09-13T04:50:40.674 INFO:teuthology.orchestra.run.smithi002.stdout: "slow_ops_count": 108 2023-09-13T04:50:40.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.675 INFO:teuthology.orchestra.run.smithi002.stdout: "purge_queue": { 2023-09-13T04:50:40.676 INFO:teuthology.orchestra.run.smithi002.stdout: "pq_executing_ops": 0, 2023-09-13T04:50:40.676 INFO:teuthology.orchestra.run.smithi002.stdout: "pq_executing_ops_high_water": 23, 2023-09-13T04:50:40.677 INFO:teuthology.orchestra.run.smithi002.stdout: "pq_executing": 0, 2023-09-13T04:50:40.677 INFO:teuthology.orchestra.run.smithi002.stdout: "pq_executing_high_water": 8, 2023-09-13T04:50:40.678 INFO:teuthology.orchestra.run.smithi002.stdout: "pq_executed": 15378, 2023-09-13T04:50:40.678 INFO:teuthology.orchestra.run.smithi002.stdout: "pq_item_in_journal": 0 2023-09-13T04:50:40.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.679 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-13T04:50:40.680 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2023-09-13T04:50:40.680 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2023-09-13T04:50:40.680 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2023-09-13T04:50:40.681 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 633536, 2023-09-13T04:50:40.681 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 111709132, 2023-09-13T04:50:40.681 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2023-09-13T04:50:40.681 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 633536, 2023-09-13T04:50:40.682 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2023-09-13T04:50:40.682 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2023-09-13T04:50:40.682 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 633536, 2023-09-13T04:50:40.683 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 111709132, 2023-09-13T04:50:40.683 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2023-09-13T04:50:40.683 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.683 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.684 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.684 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:40.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.685 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-objecter_bytes": { 2023-09-13T04:50:40.685 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2023-09-13T04:50:40.685 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2023-09-13T04:50:40.686 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2023-09-13T04:50:40.686 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 0, 2023-09-13T04:50:40.686 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 0, 2023-09-13T04:50:40.687 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2023-09-13T04:50:40.687 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 0, 2023-09-13T04:50:40.687 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 41754, 2023-09-13T04:50:40.687 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 354294647, 2023-09-13T04:50:40.687 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 41754, 2023-09-13T04:50:40.688 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 354294647, 2023-09-13T04:50:40.688 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2023-09-13T04:50:40.688 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.688 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.688 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.689 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:40.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.689 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-objecter_ops": { 2023-09-13T04:50:40.690 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2023-09-13T04:50:40.690 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 1024, 2023-09-13T04:50:40.690 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2023-09-13T04:50:40.690 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 0, 2023-09-13T04:50:40.691 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 0, 2023-09-13T04:50:40.691 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2023-09-13T04:50:40.691 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 0, 2023-09-13T04:50:40.692 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 41754, 2023-09-13T04:50:40.692 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 41754, 2023-09-13T04:50:40.692 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 41754, 2023-09-13T04:50:40.693 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 41754, 2023-09-13T04:50:40.693 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2023-09-13T04:50:40.693 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.694 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.694 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.694 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:40.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.694 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-write_buf_throttle": { 2023-09-13T04:50:40.695 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2023-09-13T04:50:40.695 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 3758096384, 2023-09-13T04:50:40.695 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2023-09-13T04:50:40.695 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 15378, 2023-09-13T04:50:40.695 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 2030402, 2023-09-13T04:50:40.696 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2023-09-13T04:50:40.696 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 15378, 2023-09-13T04:50:40.696 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2023-09-13T04:50:40.696 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2023-09-13T04:50:40.696 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 749, 2023-09-13T04:50:40.697 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 2030402, 2023-09-13T04:50:40.697 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2023-09-13T04:50:40.697 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.697 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.698 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.698 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:40.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:40.698 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-write_buf_throttle-0x5648bc5126e0": { 2023-09-13T04:50:40.698 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2023-09-13T04:50:40.699 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 3758096384, 2023-09-13T04:50:40.699 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2023-09-13T04:50:40.699 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 98286, 2023-09-13T04:50:40.700 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 337089782, 2023-09-13T04:50:40.700 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2023-09-13T04:50:40.700 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 98286, 2023-09-13T04:50:40.700 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2023-09-13T04:50:40.701 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2023-09-13T04:50:40.701 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 4163, 2023-09-13T04:50:40.701 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 337089782, 2023-09-13T04:50:40.701 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2023-09-13T04:50:40.701 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:40.701 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:40.702 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:40.702 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:40.702 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:40.702 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:50:41.120 INFO:tasks.check_counter:Daemon mds.a mds.exported=158 2023-09-13T04:50:41.121 INFO:tasks.check_counter:Daemon mds.a mds.imported=243 2023-09-13T04:50:41.121 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=1080 2023-09-13T04:50:41.122 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=1032 2023-09-13T04:50:41.122 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2023-09-13T04:50:41.122 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2023-09-13T04:50:41.123 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2023-09-13T04:50:41.123 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=66 2023-09-13T04:50:41.123 DEBUG:tasks.check_counter:Getting stats from d 2023-09-13T04:50:41.124 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph daemon mds.d perf dump 2023-09-13T04:50:41.857 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:50:42.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:42 smithi149 ceph-mon[133315]: pgmap v320: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2023-09-13T04:50:42.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:42 smithi156 ceph-mon[133675]: pgmap v320: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2023-09-13T04:50:42.444 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:42 smithi002 ceph-mon[97651]: pgmap v320: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2023-09-13T04:50:43.426 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:50:43.427 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-0": { 2023-09-13T04:50:43.427 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 0, 2023-09-13T04:50:43.427 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 2, 2023-09-13T04:50:43.427 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 0, 2023-09-13T04:50:43.428 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 0, 2023-09-13T04:50:43.428 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2023-09-13T04:50:43.428 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 0, 2023-09-13T04:50:43.428 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.003377739, 2023-09-13T04:50:43.428 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.000000000, 2023-09-13T04:50:43.429 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.000526923, 2023-09-13T04:50:43.429 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-13T04:50:43.429 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:50:43.429 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:43.430 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:43.430 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:43.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:43.430 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:50:43.430 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:43.431 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:43.431 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:43.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:43.431 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-13T04:50:43.431 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 0 2023-09-13T04:50:43.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:43.432 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-1": { 2023-09-13T04:50:43.432 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 2, 2023-09-13T04:50:43.432 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 162, 2023-09-13T04:50:43.432 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 320, 2023-09-13T04:50:43.433 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 181072, 2023-09-13T04:50:43.433 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 5, 2023-09-13T04:50:43.433 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 3, 2023-09-13T04:50:43.433 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.019236004, 2023-09-13T04:50:43.434 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.013490121, 2023-09-13T04:50:43.434 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.002462458, 2023-09-13T04:50:43.434 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-13T04:50:43.434 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:50:43.434 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 159, 2023-09-13T04:50:43.435 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.045162344, 2023-09-13T04:50:43.435 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000284039 2023-09-13T04:50:43.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:43.435 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:50:43.435 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:43.436 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:43.436 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:43.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:43.436 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-13T04:50:43.437 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 181072 2023-09-13T04:50:43.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:43.437 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-2": { 2023-09-13T04:50:43.437 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 226, 2023-09-13T04:50:43.437 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 214, 2023-09-13T04:50:43.438 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 1420208, 2023-09-13T04:50:43.438 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 127728, 2023-09-13T04:50:43.438 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 2, 2023-09-13T04:50:43.438 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 1, 2023-09-13T04:50:43.438 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.060022285, 2023-09-13T04:50:43.439 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.013681707, 2023-09-13T04:50:43.439 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.060409713, 2023-09-13T04:50:43.439 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.006124664, 2023-09-13T04:50:43.439 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:50:43.440 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 214, 2023-09-13T04:50:43.440 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.064003648, 2023-09-13T04:50:43.440 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000299082 2023-09-13T04:50:43.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:43.440 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:50:43.441 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:43.441 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:43.441 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:43.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:43.442 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 1420208, 2023-09-13T04:50:43.442 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 127728 2023-09-13T04:50:43.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:43.442 INFO:teuthology.orchestra.run.smithi002.stdout: "cct": { 2023-09-13T04:50:43.443 INFO:teuthology.orchestra.run.smithi002.stdout: "total_workers": 0, 2023-09-13T04:50:43.443 INFO:teuthology.orchestra.run.smithi002.stdout: "unhealthy_workers": 0 2023-09-13T04:50:43.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:43.443 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool": { 2023-09-13T04:50:43.443 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_bytes": 0, 2023-09-13T04:50:43.444 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_items": 0, 2023-09-13T04:50:43.444 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_bytes": 0, 2023-09-13T04:50:43.444 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_items": 0, 2023-09-13T04:50:43.444 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_bytes": 0, 2023-09-13T04:50:43.444 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_items": 0, 2023-09-13T04:50:43.445 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-13T04:50:43.445 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_items": 0, 2023-09-13T04:50:43.445 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-13T04:50:43.445 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_items": 0, 2023-09-13T04:50:43.445 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_bytes": 0, 2023-09-13T04:50:43.446 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_items": 0, 2023-09-13T04:50:43.446 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-13T04:50:43.446 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_items": 0, 2023-09-13T04:50:43.446 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_bytes": 0, 2023-09-13T04:50:43.446 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_items": 0, 2023-09-13T04:50:43.447 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_bytes": 0, 2023-09-13T04:50:43.447 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_items": 0, 2023-09-13T04:50:43.447 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-13T04:50:43.447 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_items": 0, 2023-09-13T04:50:43.447 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-13T04:50:43.447 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_items": 0, 2023-09-13T04:50:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_bytes": 0, 2023-09-13T04:50:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_items": 0, 2023-09-13T04:50:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_bytes": 0, 2023-09-13T04:50:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_items": 0, 2023-09-13T04:50:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-13T04:50:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_items": 0, 2023-09-13T04:50:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_bytes": 0, 2023-09-13T04:50:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_items": 0, 2023-09-13T04:50:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_bytes": 0, 2023-09-13T04:50:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_items": 0, 2023-09-13T04:50:43.450 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_bytes": 0, 2023-09-13T04:50:43.450 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_items": 0, 2023-09-13T04:50:43.450 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_bytes": 0, 2023-09-13T04:50:43.450 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_items": 0, 2023-09-13T04:50:43.450 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_bytes": 148977, 2023-09-13T04:50:43.450 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_items": 30, 2023-09-13T04:50:43.451 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_bytes": 0, 2023-09-13T04:50:43.451 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_items": 0, 2023-09-13T04:50:43.451 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bytes": 0, 2023-09-13T04:50:43.451 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_items": 0, 2023-09-13T04:50:43.451 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_bytes": 0, 2023-09-13T04:50:43.452 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_items": 0, 2023-09-13T04:50:43.452 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_bytes": 0, 2023-09-13T04:50:43.452 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_items": 0, 2023-09-13T04:50:43.452 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_bytes": 0, 2023-09-13T04:50:43.452 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_items": 0, 2023-09-13T04:50:43.453 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_bytes": 0, 2023-09-13T04:50:43.453 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_items": 0, 2023-09-13T04:50:43.453 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_bytes": 0, 2023-09-13T04:50:43.453 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_items": 0, 2023-09-13T04:50:43.453 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_bytes": 928, 2023-09-13T04:50:43.453 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_items": 2, 2023-09-13T04:50:43.454 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_bytes": 0, 2023-09-13T04:50:43.454 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_items": 0, 2023-09-13T04:50:43.454 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_bytes": 0, 2023-09-13T04:50:43.454 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_items": 0 2023-09-13T04:50:43.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:43.455 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-13T04:50:43.455 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2023-09-13T04:50:43.455 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2023-09-13T04:50:43.455 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2023-09-13T04:50:43.455 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 224, 2023-09-13T04:50:43.456 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 1385925, 2023-09-13T04:50:43.456 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2023-09-13T04:50:43.456 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 224, 2023-09-13T04:50:43.456 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2023-09-13T04:50:43.456 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2023-09-13T04:50:43.457 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 224, 2023-09-13T04:50:43.457 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 1385925, 2023-09-13T04:50:43.457 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2023-09-13T04:50:43.457 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:43.457 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:43.457 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:43.458 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:43.458 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:43.458 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:50:43.953 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-09-13T04:50:43.953 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-09-13T04:50:43.953 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2023-09-13T04:50:43.954 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2023-09-13T04:50:43.954 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-09-13T04:50:43.954 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-09-13T04:50:43.954 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-09-13T04:50:43.955 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2023-09-13T04:50:43.955 DEBUG:tasks.check_counter:Getting stats from g 2023-09-13T04:50:43.955 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph daemon mds.g perf dump 2023-09-13T04:50:44.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:44 smithi002 ceph-mon[97651]: pgmap v321: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2023-09-13T04:50:44.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:44 smithi149 ceph-mon[133315]: pgmap v321: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2023-09-13T04:50:44.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:44 smithi156 ceph-mon[133675]: pgmap v321: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2023-09-13T04:50:44.696 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:50:46.299 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:50:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-0": { 2023-09-13T04:50:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 0, 2023-09-13T04:50:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 2, 2023-09-13T04:50:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 0, 2023-09-13T04:50:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 0, 2023-09-13T04:50:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2023-09-13T04:50:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 0, 2023-09-13T04:50:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.003697651, 2023-09-13T04:50:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.000000000, 2023-09-13T04:50:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.000761724, 2023-09-13T04:50:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-13T04:50:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:50:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:50:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-13T04:50:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 0 2023-09-13T04:50:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-1": { 2023-09-13T04:50:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 6, 2023-09-13T04:50:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 162, 2023-09-13T04:50:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 3312, 2023-09-13T04:50:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 181168, 2023-09-13T04:50:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 5, 2023-09-13T04:50:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 3, 2023-09-13T04:50:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.021044552, 2023-09-13T04:50:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.014416191, 2023-09-13T04:50:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.003571284, 2023-09-13T04:50:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-13T04:50:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:50:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 160, 2023-09-13T04:50:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.027650258, 2023-09-13T04:50:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000172814 2023-09-13T04:50:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:50:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-09-13T04:50:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 181168 2023-09-13T04:50:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-2": { 2023-09-13T04:50:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 219, 2023-09-13T04:50:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 209, 2023-09-13T04:50:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 1416400, 2023-09-13T04:50:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 125408, 2023-09-13T04:50:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 2, 2023-09-13T04:50:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 1, 2023-09-13T04:50:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.049895226, 2023-09-13T04:50:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.017574328, 2023-09-13T04:50:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.036241701, 2023-09-13T04:50:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.005544777, 2023-09-13T04:50:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:50:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 208, 2023-09-13T04:50:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.060012448, 2023-09-13T04:50:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000288521 2023-09-13T04:50:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:50:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 1416400, 2023-09-13T04:50:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 125408 2023-09-13T04:50:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "cct": { 2023-09-13T04:50:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "total_workers": 0, 2023-09-13T04:50:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "unhealthy_workers": 0 2023-09-13T04:50:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool": { 2023-09-13T04:50:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_bytes": 0, 2023-09-13T04:50:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_items": 0, 2023-09-13T04:50:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_bytes": 0, 2023-09-13T04:50:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_items": 0, 2023-09-13T04:50:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_bytes": 0, 2023-09-13T04:50:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_items": 0, 2023-09-13T04:50:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-13T04:50:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_items": 0, 2023-09-13T04:50:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-13T04:50:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_items": 0, 2023-09-13T04:50:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_bytes": 0, 2023-09-13T04:50:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_items": 0, 2023-09-13T04:50:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-13T04:50:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_items": 0, 2023-09-13T04:50:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_bytes": 0, 2023-09-13T04:50:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_items": 0, 2023-09-13T04:50:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_bytes": 0, 2023-09-13T04:50:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_items": 0, 2023-09-13T04:50:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-13T04:50:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_items": 0, 2023-09-13T04:50:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-13T04:50:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_items": 0, 2023-09-13T04:50:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_bytes": 0, 2023-09-13T04:50:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_items": 0, 2023-09-13T04:50:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_bytes": 0, 2023-09-13T04:50:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_items": 0, 2023-09-13T04:50:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-13T04:50:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_items": 0, 2023-09-13T04:50:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_bytes": 0, 2023-09-13T04:50:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_items": 0, 2023-09-13T04:50:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_bytes": 0, 2023-09-13T04:50:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_items": 0, 2023-09-13T04:50:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_bytes": 0, 2023-09-13T04:50:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_items": 0, 2023-09-13T04:50:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_bytes": 0, 2023-09-13T04:50:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_items": 0, 2023-09-13T04:50:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_bytes": 148977, 2023-09-13T04:50:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_items": 30, 2023-09-13T04:50:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_bytes": 0, 2023-09-13T04:50:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_items": 0, 2023-09-13T04:50:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bytes": 0, 2023-09-13T04:50:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_items": 0, 2023-09-13T04:50:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_bytes": 0, 2023-09-13T04:50:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_items": 0, 2023-09-13T04:50:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_bytes": 0, 2023-09-13T04:50:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_items": 0, 2023-09-13T04:50:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_bytes": 0, 2023-09-13T04:50:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_items": 0, 2023-09-13T04:50:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_bytes": 0, 2023-09-13T04:50:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_items": 0, 2023-09-13T04:50:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_bytes": 0, 2023-09-13T04:50:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_items": 0, 2023-09-13T04:50:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_bytes": 928, 2023-09-13T04:50:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_items": 2, 2023-09-13T04:50:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_bytes": 0, 2023-09-13T04:50:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_items": 0, 2023-09-13T04:50:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_bytes": 0, 2023-09-13T04:50:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_items": 0 2023-09-13T04:50:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-13T04:50:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2023-09-13T04:50:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2023-09-13T04:50:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2023-09-13T04:50:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 221, 2023-09-13T04:50:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 1385535, 2023-09-13T04:50:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2023-09-13T04:50:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 221, 2023-09-13T04:50:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2023-09-13T04:50:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2023-09-13T04:50:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 221, 2023-09-13T04:50:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 1385535, 2023-09-13T04:50:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2023-09-13T04:50:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:46.333 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:50:46.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:46 smithi002 ceph-mon[97651]: pgmap v322: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-09-13T04:50:46.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:46 smithi149 ceph-mon[133315]: pgmap v322: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-09-13T04:50:46.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:46 smithi156 ceph-mon[133675]: pgmap v322: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-09-13T04:50:46.892 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-09-13T04:50:46.892 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-09-13T04:50:46.893 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2023-09-13T04:50:46.893 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2023-09-13T04:50:46.893 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2023-09-13T04:50:46.893 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2023-09-13T04:50:46.894 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2023-09-13T04:50:46.894 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2023-09-13T04:50:46.894 DEBUG:tasks.check_counter:Getting stats from j 2023-09-13T04:50:46.895 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph daemon mds.j perf dump 2023-09-13T04:50:47.636 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-13T04:50:48.202 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:48 smithi002 ceph-mon[97651]: pgmap v323: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-09-13T04:50:48.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:48 smithi149 ceph-mon[133315]: pgmap v323: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-09-13T04:50:48.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:48 smithi156 ceph-mon[133675]: pgmap v323: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-09-13T04:50:49.175 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-09-13T04:50:49.176 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-0": { 2023-09-13T04:50:49.176 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 4, 2023-09-13T04:50:49.176 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 4, 2023-09-13T04:50:49.176 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 2992, 2023-09-13T04:50:49.177 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 448, 2023-09-13T04:50:49.177 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2023-09-13T04:50:49.178 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 0, 2023-09-13T04:50:49.178 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.004115238, 2023-09-13T04:50:49.178 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.000087425, 2023-09-13T04:50:49.179 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.001374996, 2023-09-13T04:50:49.179 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-13T04:50:49.179 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:50:49.180 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 3, 2023-09-13T04:50:49.180 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.001394891, 2023-09-13T04:50:49.181 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000464963 2023-09-13T04:50:49.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:49.181 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:50:49.183 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:49.183 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:49.183 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:49.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:49.183 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-09-13T04:50:49.184 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 448 2023-09-13T04:50:49.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:49.184 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-1": { 2023-09-13T04:50:49.184 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 224, 2023-09-13T04:50:49.185 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 214, 2023-09-13T04:50:49.185 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 1417760, 2023-09-13T04:50:49.185 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 128528, 2023-09-13T04:50:49.185 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 2, 2023-09-13T04:50:49.185 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 1, 2023-09-13T04:50:49.186 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.052956031, 2023-09-13T04:50:49.186 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.019254232, 2023-09-13T04:50:49.186 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.038319635, 2023-09-13T04:50:49.186 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.005574102, 2023-09-13T04:50:49.186 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:50:49.187 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 213, 2023-09-13T04:50:49.187 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.088987211, 2023-09-13T04:50:49.187 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000417780 2023-09-13T04:50:49.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:49.187 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:50:49.188 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:49.188 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:49.188 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:49.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:49.189 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 1417760, 2023-09-13T04:50:49.189 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 128528 2023-09-13T04:50:49.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:49.189 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-2": { 2023-09-13T04:50:49.189 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 2, 2023-09-13T04:50:49.190 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 164, 2023-09-13T04:50:49.190 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 320, 2023-09-13T04:50:49.190 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 181424, 2023-09-13T04:50:49.190 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 5, 2023-09-13T04:50:49.190 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 3, 2023-09-13T04:50:49.191 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.019049911, 2023-09-13T04:50:49.191 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.013023356, 2023-09-13T04:50:49.191 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.002767912, 2023-09-13T04:50:49.191 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-13T04:50:49.192 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:50:49.192 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 161, 2023-09-13T04:50:49.192 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.044482635, 2023-09-13T04:50:49.192 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000276289 2023-09-13T04:50:49.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:49.193 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:50:49.193 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:49.193 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:49.193 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:49.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:49.193 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-13T04:50:49.194 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 181424 2023-09-13T04:50:49.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:49.194 INFO:teuthology.orchestra.run.smithi002.stdout: "cct": { 2023-09-13T04:50:49.194 INFO:teuthology.orchestra.run.smithi002.stdout: "total_workers": 0, 2023-09-13T04:50:49.194 INFO:teuthology.orchestra.run.smithi002.stdout: "unhealthy_workers": 0 2023-09-13T04:50:49.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:49.195 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool": { 2023-09-13T04:50:49.195 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_bytes": 0, 2023-09-13T04:50:49.196 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_items": 0, 2023-09-13T04:50:49.196 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_bytes": 0, 2023-09-13T04:50:49.196 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_items": 0, 2023-09-13T04:50:49.196 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_bytes": 0, 2023-09-13T04:50:49.197 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_items": 0, 2023-09-13T04:50:49.197 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-13T04:50:49.197 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_items": 0, 2023-09-13T04:50:49.197 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-13T04:50:49.198 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_items": 0, 2023-09-13T04:50:49.198 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_bytes": 0, 2023-09-13T04:50:49.198 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_items": 0, 2023-09-13T04:50:49.198 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-13T04:50:49.198 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_items": 0, 2023-09-13T04:50:49.199 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_bytes": 0, 2023-09-13T04:50:49.199 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_items": 0, 2023-09-13T04:50:49.199 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_bytes": 0, 2023-09-13T04:50:49.199 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_items": 0, 2023-09-13T04:50:49.199 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-13T04:50:49.200 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_items": 0, 2023-09-13T04:50:49.200 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-13T04:50:49.200 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_items": 0, 2023-09-13T04:50:49.200 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_bytes": 0, 2023-09-13T04:50:49.200 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_items": 0, 2023-09-13T04:50:49.201 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_bytes": 0, 2023-09-13T04:50:49.201 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_items": 0, 2023-09-13T04:50:49.201 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-13T04:50:49.201 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_items": 0, 2023-09-13T04:50:49.201 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_bytes": 0, 2023-09-13T04:50:49.202 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_items": 0, 2023-09-13T04:50:49.202 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_bytes": 0, 2023-09-13T04:50:49.202 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_items": 0, 2023-09-13T04:50:49.202 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_bytes": 0, 2023-09-13T04:50:49.202 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_items": 0, 2023-09-13T04:50:49.202 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_bytes": 0, 2023-09-13T04:50:49.203 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_items": 0, 2023-09-13T04:50:49.203 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_bytes": 150008, 2023-09-13T04:50:49.203 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_items": 31, 2023-09-13T04:50:49.203 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_bytes": 0, 2023-09-13T04:50:49.203 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_items": 0, 2023-09-13T04:50:49.204 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bytes": 0, 2023-09-13T04:50:49.204 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_items": 0, 2023-09-13T04:50:49.204 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_bytes": 0, 2023-09-13T04:50:49.204 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_items": 0, 2023-09-13T04:50:49.204 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_bytes": 0, 2023-09-13T04:50:49.205 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_items": 0, 2023-09-13T04:50:49.205 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_bytes": 0, 2023-09-13T04:50:49.205 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_items": 0, 2023-09-13T04:50:49.205 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_bytes": 0, 2023-09-13T04:50:49.205 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_items": 0, 2023-09-13T04:50:49.206 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_bytes": 0, 2023-09-13T04:50:49.206 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_items": 0, 2023-09-13T04:50:49.206 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_bytes": 928, 2023-09-13T04:50:49.206 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_items": 2, 2023-09-13T04:50:49.206 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_bytes": 0, 2023-09-13T04:50:49.207 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_items": 0, 2023-09-13T04:50:49.207 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_bytes": 0, 2023-09-13T04:50:49.207 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_items": 0 2023-09-13T04:50:49.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-09-13T04:50:49.207 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-13T04:50:49.208 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2023-09-13T04:50:49.208 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2023-09-13T04:50:49.208 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2023-09-13T04:50:49.208 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 226, 2023-09-13T04:50:49.208 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 1386185, 2023-09-13T04:50:49.209 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2023-09-13T04:50:49.209 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 226, 2023-09-13T04:50:49.209 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2023-09-13T04:50:49.209 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2023-09-13T04:50:49.209 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 226, 2023-09-13T04:50:49.209 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 1386185, 2023-09-13T04:50:49.210 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2023-09-13T04:50:49.210 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-09-13T04:50:49.210 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-09-13T04:50:49.210 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-09-13T04:50:49.210 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:49.211 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-09-13T04:50:49.211 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-09-13T04:50:49.212 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:49 smithi002 ceph-mon[97651]: pgmap v324: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-09-13T04:50:49.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:49 smithi149 ceph-mon[133315]: pgmap v324: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-09-13T04:50:49.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:49 smithi156 ceph-mon[133675]: pgmap v324: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-09-13T04:50:49.737 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-09-13T04:50:49.737 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-09-13T04:50:49.737 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-09-13T04:50:49.738 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-09-13T04:50:49.738 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-09-13T04:50:49.738 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-09-13T04:50:49.738 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-09-13T04:50:49.739 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2023-09-13T04:50:49.739 DEBUG:tasks.check_counter:Getting stats from b 2023-09-13T04:50:49.739 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph daemon mds.b perf dump 2023-09-13T04:50:52.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:51 smithi156 ceph-mon[133675]: pgmap v325: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-09-13T04:50:52.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:51 smithi002 ceph-mon[97651]: pgmap v325: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-09-13T04:50:52.400 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:51 smithi149 ceph-mon[133315]: pgmap v325: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-09-13T04:50:53.288 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.b/config 2023-09-13T04:50:54.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:53 smithi156 ceph-mon[133675]: pgmap v326: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:50:54.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:53 smithi002 ceph-mon[97651]: pgmap v326: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:50:54.277 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:53 smithi149 ceph-mon[133315]: pgmap v326: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:50:55.080 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2023-09-13T04:50:55.080 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2023-09-13T04:50:55.080 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 187983, 2023-09-13T04:50:55.081 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 234845, 2023-09-13T04:50:55.081 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 63165910, 2023-09-13T04:50:55.081 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 202477187, 2023-09-13T04:50:55.081 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 16, 2023-09-13T04:50:55.081 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 5, 2023-09-13T04:50:55.082 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 6.950695357, 2023-09-13T04:50:55.082 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 3.106571804, 2023-09-13T04:50:55.082 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 5.610562363, 2023-09-13T04:50:55.082 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.760931945, 2023-09-13T04:50:55.083 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:50:55.083 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 234841, 2023-09-13T04:50:55.083 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 60.749917057, 2023-09-13T04:50:55.083 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000258685 2023-09-13T04:50:55.083 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.084 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:50:55.084 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 86148, 2023-09-13T04:50:55.084 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.033007086, 2023-09-13T04:50:55.084 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000383 2023-09-13T04:50:55.084 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.085 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-13T04:50:55.085 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 0 2023-09-13T04:50:55.085 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.085 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2023-09-13T04:50:55.085 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 2639289, 2023-09-13T04:50:55.086 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 2640550, 2023-09-13T04:50:55.086 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 434086136, 2023-09-13T04:50:55.086 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 487092086, 2023-09-13T04:50:55.086 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 113, 2023-09-13T04:50:55.086 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 8, 2023-09-13T04:50:55.087 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 51.209206478, 2023-09-13T04:50:55.087 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 21.160410938, 2023-09-13T04:50:55.087 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 38.456665797, 2023-09-13T04:50:55.087 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.539086809, 2023-09-13T04:50:55.087 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:50:55.088 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2640552, 2023-09-13T04:50:55.088 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 75.160880558, 2023-09-13T04:50:55.088 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000028464 2023-09-13T04:50:55.088 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.088 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:50:55.089 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 3983434, 2023-09-13T04:50:55.089 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.920849033, 2023-09-13T04:50:55.089 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000231 2023-09-13T04:50:55.089 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.089 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 1537248, 2023-09-13T04:50:55.090 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 150544 2023-09-13T04:50:55.090 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.090 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2023-09-13T04:50:55.090 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 303357, 2023-09-13T04:50:55.090 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 302296, 2023-09-13T04:50:55.091 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 66483191, 2023-09-13T04:50:55.091 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 91150219, 2023-09-13T04:50:55.091 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 33, 2023-09-13T04:50:55.091 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 6, 2023-09-13T04:50:55.091 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 9.580142375, 2023-09-13T04:50:55.092 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 3.703057632, 2023-09-13T04:50:55.092 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 6.910379673, 2023-09-13T04:50:55.092 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.583319204, 2023-09-13T04:50:55.092 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:50:55.092 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 302293, 2023-09-13T04:50:55.093 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 44.025091974, 2023-09-13T04:50:55.093 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000145637 2023-09-13T04:50:55.093 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.093 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:50:55.093 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 480350, 2023-09-13T04:50:55.094 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.058578799, 2023-09-13T04:50:55.094 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000121 2023-09-13T04:50:55.094 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.094 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 312112, 2023-09-13T04:50:55.095 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 536896 2023-09-13T04:50:55.095 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.095 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2023-09-13T04:50:55.095 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2023-09-13T04:50:55.095 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2023-09-13T04:50:55.096 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.096 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2023-09-13T04:50:55.096 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2023-09-13T04:50:55.096 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2023-09-13T04:50:55.096 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 8834, 2023-09-13T04:50:55.097 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 36.694398769, 2023-09-13T04:50:55.097 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.004153769 2023-09-13T04:50:55.097 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-09-13T04:50:55.097 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.097 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2023-09-13T04:50:55.097 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2023-09-13T04:50:55.098 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2023-09-13T04:50:55.098 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 20297, 2023-09-13T04:50:55.098 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 3.481233459, 2023-09-13T04:50:55.098 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000171514 2023-09-13T04:50:55.098 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-09-13T04:50:55.099 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.099 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2023-09-13T04:50:55.099 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 79123, 2023-09-13T04:50:55.099 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 73938, 2023-09-13T04:50:55.099 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2023-09-13T04:50:55.100 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 73938, 2023-09-13T04:50:55.100 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 1522.060528895, 2023-09-13T04:50:55.100 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.020585632 2023-09-13T04:50:55.100 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.100 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2023-09-13T04:50:55.101 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 5185, 2023-09-13T04:50:55.101 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 78, 2023-09-13T04:50:55.101 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2023-09-13T04:50:55.101 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 4295, 2023-09-13T04:50:55.101 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 40, 2023-09-13T04:50:55.102 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2023-09-13T04:50:55.102 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 19175, 2023-09-13T04:50:55.102 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 13282, 2023-09-13T04:50:55.102 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 5836, 2023-09-13T04:50:55.102 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 57, 2023-09-13T04:50:55.103 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 200, 2023-09-13T04:50:55.105 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 15561, 2023-09-13T04:50:55.105 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 1, 2023-09-13T04:50:55.105 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 2, 2023-09-13T04:50:55.105 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 4, 2023-09-13T04:50:55.105 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 136827, 2023-09-13T04:50:55.106 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 114487, 2023-09-13T04:50:55.106 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 5102, 2023-09-13T04:50:55.106 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 395, 2023-09-13T04:50:55.106 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 1, 2023-09-13T04:50:55.106 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2023-09-13T04:50:55.107 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 985, 2023-09-13T04:50:55.107 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2023-09-13T04:50:55.107 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2023-09-13T04:50:55.107 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 200, 2023-09-13T04:50:55.107 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 11348, 2023-09-13T04:50:55.108 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 136, 2023-09-13T04:50:55.108 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 16208, 2023-09-13T04:50:55.108 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2023-09-13T04:50:55.108 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 3029, 2023-09-13T04:50:55.108 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2023-09-13T04:50:55.109 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 0, 2023-09-13T04:50:55.109 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 0, 2023-09-13T04:50:55.109 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 6, 2023-09-13T04:50:55.109 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 3029, 2023-09-13T04:50:55.109 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2023-09-13T04:50:55.109 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2023-09-13T04:50:55.110 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2023-09-13T04:50:55.110 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 30, 2023-09-13T04:50:55.110 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 5, 2023-09-13T04:50:55.110 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 25, 2023-09-13T04:50:55.110 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 2999, 2023-09-13T04:50:55.111 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 199, 2023-09-13T04:50:55.111 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 7073, 2023-09-13T04:50:55.111 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 20102, 2023-09-13T04:50:55.111 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 6, 2023-09-13T04:50:55.111 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-13T04:50:55.112 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-13T04:50:55.112 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 19159, 2023-09-13T04:50:55.112 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 8339, 2023-09-13T04:50:55.112 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 407, 2023-09-13T04:50:55.112 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 29119 2023-09-13T04:50:55.113 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.113 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2023-09-13T04:50:55.113 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 1281, 2023-09-13T04:50:55.113 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 3250, 2023-09-13T04:50:55.113 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 2135, 2023-09-13T04:50:55.114 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 2655, 2023-09-13T04:50:55.114 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 1633, 2023-09-13T04:50:55.114 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 52, 2023-09-13T04:50:55.114 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2023-09-13T04:50:55.114 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2023-09-13T04:50:55.115 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 11531, 2023-09-13T04:50:55.115 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 11479, 2023-09-13T04:50:55.115 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2023-09-13T04:50:55.115 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2023-09-13T04:50:55.115 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2023-09-13T04:50:55.115 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2023-09-13T04:50:55.116 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2023-09-13T04:50:55.116 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2023-09-13T04:50:55.116 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2023-09-13T04:50:55.116 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 5, 2023-09-13T04:50:55.116 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 206, 2023-09-13T04:50:55.117 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2023-09-13T04:50:55.117 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 40, 2023-09-13T04:50:55.117 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2023-09-13T04:50:55.117 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2023-09-13T04:50:55.117 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.118 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2023-09-13T04:50:55.118 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 2, 2023-09-13T04:50:55.118 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 59193, 2023-09-13T04:50:55.118 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 59087, 2023-09-13T04:50:55.118 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 59087, 2023-09-13T04:50:55.119 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 106, 2023-09-13T04:50:55.119 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2023-09-13T04:50:55.119 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 11, 2023-09-13T04:50:55.119 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 106, 2023-09-13T04:50:55.119 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 105, 2023-09-13T04:50:55.120 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 105, 2023-09-13T04:50:55.120 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 1, 2023-09-13T04:50:55.120 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 1, 2023-09-13T04:50:55.120 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2023-09-13T04:50:55.120 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 1, 2023-09-13T04:50:55.121 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 172870329, 2023-09-13T04:50:55.121 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 173275287, 2023-09-13T04:50:55.121 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 0, 2023-09-13T04:50:55.121 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2023-09-13T04:50:55.121 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2168, 2023-09-13T04:50:55.122 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 80.177313433, 2023-09-13T04:50:55.122 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.036982155 2023-09-13T04:50:55.122 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.122 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 0 2023-09-13T04:50:55.122 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.123 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2023-09-13T04:50:55.123 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 11082, 2023-09-13T04:50:55.123 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 31232, 2023-09-13T04:50:55.123 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 20150, 2023-09-13T04:50:55.123 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 383, 2023-09-13T04:50:55.124 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 1533, 2023-09-13T04:50:55.124 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 1150, 2023-09-13T04:50:55.124 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 19175, 2023-09-13T04:50:55.124 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 52275, 2023-09-13T04:50:55.124 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 33100, 2023-09-13T04:50:55.125 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 2, 2023-09-13T04:50:55.125 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 17227, 2023-09-13T04:50:55.125 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 17225, 2023-09-13T04:50:55.125 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 160216, 2023-09-13T04:50:55.125 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 207152 2023-09-13T04:50:55.125 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.126 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2023-09-13T04:50:55.126 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 86308, 2023-09-13T04:50:55.126 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 1279, 2023-09-13T04:50:55.126 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 79123, 2023-09-13T04:50:55.126 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 124, 2023-09-13T04:50:55.127 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 2167, 2023-09-13T04:50:55.127 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2023-09-13T04:50:55.127 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 7419, 2023-09-13T04:50:55.127 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 43.433320396, 2023-09-13T04:50:55.127 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.005854336 2023-09-13T04:50:55.128 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.128 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2023-09-13T04:50:55.128 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1378, 2023-09-13T04:50:55.128 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 92.721318644, 2023-09-13T04:50:55.128 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.067286878 2023-09-13T04:50:55.129 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.129 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2023-09-13T04:50:55.129 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:55.129 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:55.129 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:55.129 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.130 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2023-09-13T04:50:55.130 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:55.130 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:55.130 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:55.130 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.131 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2023-09-13T04:50:55.131 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 44173, 2023-09-13T04:50:55.131 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 1194.088192293, 2023-09-13T04:50:55.131 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.027032082 2023-09-13T04:50:55.131 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.132 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2023-09-13T04:50:55.132 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:55.132 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:55.132 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:55.132 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.133 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2023-09-13T04:50:55.133 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:55.133 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:55.133 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:55.133 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.134 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2023-09-13T04:50:55.134 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:55.134 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:55.134 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:55.134 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.135 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2023-09-13T04:50:55.135 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:55.135 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:55.135 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:55.135 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.135 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2023-09-13T04:50:55.136 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:55.136 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:55.136 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:55.136 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.136 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2023-09-13T04:50:55.137 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 10, 2023-09-13T04:50:55.137 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.001880210, 2023-09-13T04:50:55.137 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000188021 2023-09-13T04:50:55.137 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.137 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2023-09-13T04:50:55.138 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 30, 2023-09-13T04:50:55.138 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.238441786, 2023-09-13T04:50:55.138 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.007948059 2023-09-13T04:50:55.138 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.138 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2023-09-13T04:50:55.139 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:55.139 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:55.139 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:55.139 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.139 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2023-09-13T04:50:55.140 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 10, 2023-09-13T04:50:55.140 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 5.818823731, 2023-09-13T04:50:55.140 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.581882373 2023-09-13T04:50:55.140 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.140 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2023-09-13T04:50:55.140 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4194, 2023-09-13T04:50:55.141 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 17.171630824, 2023-09-13T04:50:55.141 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.004094332 2023-09-13T04:50:55.141 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.141 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2023-09-13T04:50:55.141 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 474, 2023-09-13T04:50:55.142 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 2.068405137, 2023-09-13T04:50:55.142 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.004363723 2023-09-13T04:50:55.142 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.142 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2023-09-13T04:50:55.142 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 7009, 2023-09-13T04:50:55.143 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 141.431051029, 2023-09-13T04:50:55.143 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.020178492 2023-09-13T04:50:55.143 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.143 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2023-09-13T04:50:55.143 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:55.144 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:55.144 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:55.144 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.144 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2023-09-13T04:50:55.144 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:55.145 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:55.145 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:55.145 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.145 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2023-09-13T04:50:55.145 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 56, 2023-09-13T04:50:55.146 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 9.444151328, 2023-09-13T04:50:55.146 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.168645559 2023-09-13T04:50:55.146 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.146 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2023-09-13T04:50:55.147 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4, 2023-09-13T04:50:55.147 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.165637837, 2023-09-13T04:50:55.147 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.041409459 2023-09-13T04:50:55.147 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.147 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2023-09-13T04:50:55.148 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:55.148 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:55.148 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:55.148 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.148 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2023-09-13T04:50:55.149 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 7, 2023-09-13T04:50:55.149 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.040010809, 2023-09-13T04:50:55.149 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.005715829 2023-09-13T04:50:55.149 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.149 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2023-09-13T04:50:55.150 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:55.150 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:55.150 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:55.150 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.150 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2023-09-13T04:50:55.151 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:55.151 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:55.151 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:55.151 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.151 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2023-09-13T04:50:55.152 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:55.152 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:55.152 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:55.152 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.152 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2023-09-13T04:50:55.152 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1, 2023-09-13T04:50:55.153 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.002536585, 2023-09-13T04:50:55.153 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002536585 2023-09-13T04:50:55.153 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.153 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2023-09-13T04:50:55.153 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 3, 2023-09-13T04:50:55.154 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.002202348, 2023-09-13T04:50:55.154 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000734116 2023-09-13T04:50:55.154 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.154 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2023-09-13T04:50:55.154 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 9170, 2023-09-13T04:50:55.155 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 15.432925938, 2023-09-13T04:50:55.155 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001682979 2023-09-13T04:50:55.155 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.155 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2023-09-13T04:50:55.155 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2023-09-13T04:50:55.156 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2023-09-13T04:50:55.156 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:55.156 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:55.156 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:55.156 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-09-13T04:50:55.156 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.157 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2023-09-13T04:50:55.157 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 2, 2023-09-13T04:50:55.157 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 3, 2023-09-13T04:50:55.157 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 1, 2023-09-13T04:50:55.157 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 2, 2023-09-13T04:50:55.158 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2023-09-13T04:50:55.158 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 5643, 2023-09-13T04:50:55.158 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 2821, 2023-09-13T04:50:55.158 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 619, 2023-09-13T04:50:55.158 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2023-09-13T04:50:55.159 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.159 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2023-09-13T04:50:55.159 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2023-09-13T04:50:55.159 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2023-09-13T04:50:55.159 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2023-09-13T04:50:55.160 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2023-09-13T04:50:55.160 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2023-09-13T04:50:55.160 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2023-09-13T04:50:55.160 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-13T04:50:55.160 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2023-09-13T04:50:55.161 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-13T04:50:55.161 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2023-09-13T04:50:55.161 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2023-09-13T04:50:55.161 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2023-09-13T04:50:55.161 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-13T04:50:55.161 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2023-09-13T04:50:55.162 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2023-09-13T04:50:55.162 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2023-09-13T04:50:55.162 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2023-09-13T04:50:55.162 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2023-09-13T04:50:55.162 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-13T04:50:55.163 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2023-09-13T04:50:55.163 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-13T04:50:55.163 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2023-09-13T04:50:55.163 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2023-09-13T04:50:55.163 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2023-09-13T04:50:55.164 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2023-09-13T04:50:55.164 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2023-09-13T04:50:55.164 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-13T04:50:55.164 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2023-09-13T04:50:55.164 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2023-09-13T04:50:55.165 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2023-09-13T04:50:55.165 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2023-09-13T04:50:55.165 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2023-09-13T04:50:55.165 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2023-09-13T04:50:55.165 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2023-09-13T04:50:55.165 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2023-09-13T04:50:55.166 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2023-09-13T04:50:55.166 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 1104198, 2023-09-13T04:50:55.166 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 10626, 2023-09-13T04:50:55.166 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2023-09-13T04:50:55.166 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2023-09-13T04:50:55.167 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2023-09-13T04:50:55.167 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2023-09-13T04:50:55.167 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2023-09-13T04:50:55.167 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2023-09-13T04:50:55.167 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2023-09-13T04:50:55.168 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2023-09-13T04:50:55.168 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 9416, 2023-09-13T04:50:55.168 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 180, 2023-09-13T04:50:55.168 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2023-09-13T04:50:55.168 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2023-09-13T04:50:55.169 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2023-09-13T04:50:55.169 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2023-09-13T04:50:55.169 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 39881907, 2023-09-13T04:50:55.169 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 391253, 2023-09-13T04:50:55.169 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2023-09-13T04:50:55.170 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2023-09-13T04:50:55.170 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2023-09-13T04:50:55.170 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2023-09-13T04:50:55.170 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.170 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2023-09-13T04:50:55.171 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2023-09-13T04:50:55.171 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2023-09-13T04:50:55.171 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 62807, 2023-09-13T04:50:55.171 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 209253894, 2023-09-13T04:50:55.171 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 37, 2023-09-13T04:50:55.172 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 62807, 2023-09-13T04:50:55.172 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2023-09-13T04:50:55.172 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 62807, 2023-09-13T04:50:55.172 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 27835.066633040, 2023-09-13T04:50:55.172 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.443184145 2023-09-13T04:50:55.173 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.173 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2023-09-13T04:50:55.173 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2023-09-13T04:50:55.173 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 62807, 2023-09-13T04:50:55.173 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 115216 2023-09-13T04:50:55.174 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.174 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 62807, 2023-09-13T04:50:55.174 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 3790, 2023-09-13T04:50:55.174 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 59017, 2023-09-13T04:50:55.175 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2023-09-13T04:50:55.175 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2023-09-13T04:50:55.175 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 3911, 2023-09-13T04:50:55.175 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 28523, 2023-09-13T04:50:55.176 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 421, 2023-09-13T04:50:55.176 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 2586, 2023-09-13T04:50:55.176 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 370, 2023-09-13T04:50:55.176 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2023-09-13T04:50:55.177 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2023-09-13T04:50:55.177 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2023-09-13T04:50:55.177 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 40, 2023-09-13T04:50:55.177 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 23043, 2023-09-13T04:50:55.177 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2023-09-13T04:50:55.177 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2023-09-13T04:50:55.178 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2023-09-13T04:50:55.178 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 3036, 2023-09-13T04:50:55.178 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 43526, 2023-09-13T04:50:55.178 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2023-09-13T04:50:55.178 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2023-09-13T04:50:55.179 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2023-09-13T04:50:55.179 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2023-09-13T04:50:55.179 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2023-09-13T04:50:55.179 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2023-09-13T04:50:55.179 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2023-09-13T04:50:55.180 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2023-09-13T04:50:55.180 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2023-09-13T04:50:55.180 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 6, 2023-09-13T04:50:55.180 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2023-09-13T04:50:55.180 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2023-09-13T04:50:55.181 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2023-09-13T04:50:55.181 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2023-09-13T04:50:55.181 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2023-09-13T04:50:55.181 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2023-09-13T04:50:55.181 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2023-09-13T04:50:55.181 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2023-09-13T04:50:55.182 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2023-09-13T04:50:55.182 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2023-09-13T04:50:55.182 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2023-09-13T04:50:55.182 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2023-09-13T04:50:55.182 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2023-09-13T04:50:55.183 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2023-09-13T04:50:55.183 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2023-09-13T04:50:55.183 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2023-09-13T04:50:55.183 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 104, 2023-09-13T04:50:55.183 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2023-09-13T04:50:55.184 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 21, 2023-09-13T04:50:55.184 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 12, 2023-09-13T04:50:55.184 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 12, 2023-09-13T04:50:55.184 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2023-09-13T04:50:55.184 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2023-09-13T04:50:55.185 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 7626, 2023-09-13T04:50:55.185 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 413, 2023-09-13T04:50:55.185 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 1715 2023-09-13T04:50:55.185 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.185 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2023-09-13T04:50:55.186 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 1, 2023-09-13T04:50:55.186 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2023-09-13T04:50:55.186 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2023-09-13T04:50:55.186 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2023-09-13T04:50:55.186 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.187 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2023-09-13T04:50:55.187 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 31 2023-09-13T04:50:55.187 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.187 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2023-09-13T04:50:55.187 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2023-09-13T04:50:55.187 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 17, 2023-09-13T04:50:55.188 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2023-09-13T04:50:55.188 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 10, 2023-09-13T04:50:55.188 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 11479, 2023-09-13T04:50:55.188 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2023-09-13T04:50:55.188 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.189 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-13T04:50:55.189 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-09-13T04:50:55.189 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2023-09-13T04:50:55.189 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-09-13T04:50:55.189 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 3130625, 2023-09-13T04:50:55.190 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 284530890, 2023-09-13T04:50:55.190 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-09-13T04:50:55.190 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 3130625, 2023-09-13T04:50:55.190 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2023-09-13T04:50:55.190 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2023-09-13T04:50:55.191 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 3130625, 2023-09-13T04:50:55.191 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 284530890, 2023-09-13T04:50:55.191 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-09-13T04:50:55.191 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:55.191 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:55.192 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:55.192 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-09-13T04:50:55.192 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.192 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2023-09-13T04:50:55.192 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-09-13T04:50:55.193 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2023-09-13T04:50:55.193 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-09-13T04:50:55.193 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2023-09-13T04:50:55.193 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2023-09-13T04:50:55.193 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-09-13T04:50:55.194 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2023-09-13T04:50:55.194 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 62807, 2023-09-13T04:50:55.194 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 210788725, 2023-09-13T04:50:55.194 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 62807, 2023-09-13T04:50:55.194 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 210788725, 2023-09-13T04:50:55.195 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-09-13T04:50:55.195 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:55.195 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:55.195 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:55.195 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-09-13T04:50:55.195 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.196 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2023-09-13T04:50:55.196 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-09-13T04:50:55.196 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2023-09-13T04:50:55.196 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-09-13T04:50:55.196 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2023-09-13T04:50:55.197 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2023-09-13T04:50:55.197 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-09-13T04:50:55.197 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2023-09-13T04:50:55.197 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 62807, 2023-09-13T04:50:55.197 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 62807, 2023-09-13T04:50:55.198 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 62807, 2023-09-13T04:50:55.198 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 62807, 2023-09-13T04:50:55.198 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-09-13T04:50:55.198 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:55.198 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:55.199 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:55.199 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-09-13T04:50:55.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.199 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2023-09-13T04:50:55.199 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-09-13T04:50:55.200 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2023-09-13T04:50:55.200 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-09-13T04:50:55.200 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 11479, 2023-09-13T04:50:55.200 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 1544667, 2023-09-13T04:50:55.200 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-09-13T04:50:55.200 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 11479, 2023-09-13T04:50:55.201 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2023-09-13T04:50:55.201 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2023-09-13T04:50:55.201 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 418, 2023-09-13T04:50:55.201 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 1544667, 2023-09-13T04:50:55.201 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-09-13T04:50:55.202 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:55.202 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:55.202 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:55.202 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-09-13T04:50:55.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:55.203 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x560c3af646e0": { 2023-09-13T04:50:55.203 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-09-13T04:50:55.203 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2023-09-13T04:50:55.203 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-09-13T04:50:55.203 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 59193, 2023-09-13T04:50:55.204 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 169080983, 2023-09-13T04:50:55.204 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-09-13T04:50:55.204 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 59193, 2023-09-13T04:50:55.204 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2023-09-13T04:50:55.204 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2023-09-13T04:50:55.205 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 2168, 2023-09-13T04:50:55.205 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 169080983, 2023-09-13T04:50:55.205 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-09-13T04:50:55.205 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:55.205 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:55.206 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:55.206 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-09-13T04:50:55.206 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-09-13T04:50:55.206 INFO:teuthology.orchestra.run.smithi149.stdout:} 2023-09-13T04:50:55.482 INFO:tasks.check_counter:Daemon mds.b mds.exported=200 2023-09-13T04:50:55.482 INFO:tasks.check_counter:Daemon mds.b mds.imported=136 2023-09-13T04:50:55.483 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=1281 2023-09-13T04:50:55.483 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=3250 2023-09-13T04:50:55.484 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2023-09-13T04:50:55.484 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2023-09-13T04:50:55.484 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2023-09-13T04:50:55.485 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=40 2023-09-13T04:50:55.485 DEBUG:tasks.check_counter:Getting stats from e 2023-09-13T04:50:55.486 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph daemon mds.e perf dump 2023-09-13T04:50:56.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:55 smithi156 ceph-mon[133675]: pgmap v327: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:50:56.203 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.b/config 2023-09-13T04:50:56.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:55 smithi002 ceph-mon[97651]: pgmap v327: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:50:56.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:55 smithi149 ceph-mon[133315]: pgmap v327: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:50:57.772 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2023-09-13T04:50:57.772 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2023-09-13T04:50:57.772 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 0, 2023-09-13T04:50:57.772 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 2, 2023-09-13T04:50:57.773 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 0, 2023-09-13T04:50:57.773 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 0, 2023-09-13T04:50:57.773 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 4, 2023-09-13T04:50:57.773 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 0, 2023-09-13T04:50:57.773 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.001430149, 2023-09-13T04:50:57.774 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.000000000, 2023-09-13T04:50:57.774 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.000400797, 2023-09-13T04:50:57.774 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-13T04:50:57.774 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:50:57.774 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:57.775 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:57.775 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:57.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:57.775 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:50:57.775 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:57.775 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:57.776 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:57.776 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:57.776 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-13T04:50:57.776 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 0 2023-09-13T04:50:57.777 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:57.777 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2023-09-13T04:50:57.777 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 2, 2023-09-13T04:50:57.777 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 164, 2023-09-13T04:50:57.777 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 320, 2023-09-13T04:50:57.777 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 181424, 2023-09-13T04:50:57.778 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 5, 2023-09-13T04:50:57.778 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 3, 2023-09-13T04:50:57.778 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.009528010, 2023-09-13T04:50:57.778 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.005961826, 2023-09-13T04:50:57.778 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.001568852, 2023-09-13T04:50:57.779 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-13T04:50:57.779 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:50:57.779 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 161, 2023-09-13T04:50:57.779 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.012202670, 2023-09-13T04:50:57.779 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000075792 2023-09-13T04:50:57.779 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:57.780 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:50:57.780 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:57.780 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:57.780 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:57.780 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:57.781 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-13T04:50:57.781 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 181424 2023-09-13T04:50:57.781 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:57.781 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2023-09-13T04:50:57.781 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 230, 2023-09-13T04:50:57.781 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 218, 2023-09-13T04:50:57.782 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 1421360, 2023-09-13T04:50:57.782 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 129744, 2023-09-13T04:50:57.782 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 2, 2023-09-13T04:50:57.782 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 1, 2023-09-13T04:50:57.782 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.041267042, 2023-09-13T04:50:57.783 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.011575903, 2023-09-13T04:50:57.783 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.032311956, 2023-09-13T04:50:57.783 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.005866666, 2023-09-13T04:50:57.783 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:50:57.783 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 218, 2023-09-13T04:50:57.783 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.015392480, 2023-09-13T04:50:57.784 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000070607 2023-09-13T04:50:57.784 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:57.784 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:50:57.784 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:57.784 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:57.784 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:57.785 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:57.785 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 1421360, 2023-09-13T04:50:57.785 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 129744 2023-09-13T04:50:57.785 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:57.785 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2023-09-13T04:50:57.785 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 0, 2023-09-13T04:50:57.786 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2023-09-13T04:50:57.786 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:57.786 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2023-09-13T04:50:57.786 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2023-09-13T04:50:57.786 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2023-09-13T04:50:57.786 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2023-09-13T04:50:57.787 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2023-09-13T04:50:57.787 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2023-09-13T04:50:57.787 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2023-09-13T04:50:57.787 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-13T04:50:57.787 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2023-09-13T04:50:57.788 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-13T04:50:57.788 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2023-09-13T04:50:57.788 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2023-09-13T04:50:57.788 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2023-09-13T04:50:57.788 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-13T04:50:57.788 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2023-09-13T04:50:57.789 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2023-09-13T04:50:57.789 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2023-09-13T04:50:57.789 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2023-09-13T04:50:57.789 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2023-09-13T04:50:57.789 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-13T04:50:57.789 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2023-09-13T04:50:57.790 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-13T04:50:57.790 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2023-09-13T04:50:57.790 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2023-09-13T04:50:57.790 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2023-09-13T04:50:57.790 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2023-09-13T04:50:57.790 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2023-09-13T04:50:57.791 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-13T04:50:57.791 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2023-09-13T04:50:57.791 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2023-09-13T04:50:57.791 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2023-09-13T04:50:57.791 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2023-09-13T04:50:57.791 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2023-09-13T04:50:57.792 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2023-09-13T04:50:57.792 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2023-09-13T04:50:57.792 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2023-09-13T04:50:57.792 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2023-09-13T04:50:57.792 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 149057, 2023-09-13T04:50:57.793 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 32, 2023-09-13T04:50:57.793 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2023-09-13T04:50:57.793 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2023-09-13T04:50:57.793 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2023-09-13T04:50:57.793 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2023-09-13T04:50:57.793 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2023-09-13T04:50:57.794 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2023-09-13T04:50:57.794 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2023-09-13T04:50:57.794 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2023-09-13T04:50:57.794 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 0, 2023-09-13T04:50:57.794 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 0, 2023-09-13T04:50:57.794 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2023-09-13T04:50:57.795 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2023-09-13T04:50:57.795 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2023-09-13T04:50:57.795 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2023-09-13T04:50:57.795 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 928, 2023-09-13T04:50:57.795 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 2, 2023-09-13T04:50:57.795 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2023-09-13T04:50:57.796 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2023-09-13T04:50:57.796 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2023-09-13T04:50:57.796 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2023-09-13T04:50:57.796 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:50:57.796 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-13T04:50:57.796 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-09-13T04:50:57.797 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2023-09-13T04:50:57.797 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-09-13T04:50:57.797 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 228, 2023-09-13T04:50:57.797 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 1386509, 2023-09-13T04:50:57.797 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-09-13T04:50:57.797 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 228, 2023-09-13T04:50:57.798 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2023-09-13T04:50:57.798 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2023-09-13T04:50:57.798 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 228, 2023-09-13T04:50:57.798 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 1386509, 2023-09-13T04:50:57.798 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-09-13T04:50:57.799 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:50:57.799 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:50:57.799 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:50:57.799 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-09-13T04:50:57.799 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-09-13T04:50:57.799 INFO:teuthology.orchestra.run.smithi149.stdout:} 2023-09-13T04:50:58.136 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:57 smithi149 ceph-mon[133315]: pgmap v328: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:50:58.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:57 smithi156 ceph-mon[133675]: pgmap v328: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:50:58.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:57 smithi002 ceph-mon[97651]: pgmap v328: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:50:58.267 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2023-09-13T04:50:58.267 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2023-09-13T04:50:58.267 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2023-09-13T04:50:58.268 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2023-09-13T04:50:58.268 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2023-09-13T04:50:58.268 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2023-09-13T04:50:58.268 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2023-09-13T04:50:58.268 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.e 2023-09-13T04:50:58.269 DEBUG:tasks.check_counter:Getting stats from h 2023-09-13T04:50:58.269 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph daemon mds.h perf dump 2023-09-13T04:50:58.837 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.b/config 2023-09-13T04:51:00.036 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2023-09-13T04:51:00.037 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2023-09-13T04:51:00.037 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 1354, 2023-09-13T04:51:00.037 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 1329, 2023-09-13T04:51:00.037 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 69012173, 2023-09-13T04:51:00.037 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 472513, 2023-09-13T04:51:00.038 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 7, 2023-09-13T04:51:00.038 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 3, 2023-09-13T04:51:00.038 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.168432335, 2023-09-13T04:51:00.038 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.046223152, 2023-09-13T04:51:00.038 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.082156459, 2023-09-13T04:51:00.039 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.038742662, 2023-09-13T04:51:00.039 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:51:00.039 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1328, 2023-09-13T04:51:00.039 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.078598166, 2023-09-13T04:51:00.039 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000059185 2023-09-13T04:51:00.040 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.040 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:51:00.040 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.040 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.040 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.041 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.041 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 1507120, 2023-09-13T04:51:00.041 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 131968 2023-09-13T04:51:00.041 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.042 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2023-09-13T04:51:00.042 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 416, 2023-09-13T04:51:00.042 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 578, 2023-09-13T04:51:00.042 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 96591423, 2023-09-13T04:51:00.042 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 565824, 2023-09-13T04:51:00.043 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 9, 2023-09-13T04:51:00.043 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 7, 2023-09-13T04:51:00.043 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.083717665, 2023-09-13T04:51:00.043 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.013279951, 2023-09-13T04:51:00.043 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.055326089, 2023-09-13T04:51:00.044 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.010612946, 2023-09-13T04:51:00.044 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:51:00.044 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 575, 2023-09-13T04:51:00.044 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.035303451, 2023-09-13T04:51:00.044 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000061397 2023-09-13T04:51:00.044 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.045 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:51:00.045 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.045 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.045 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.045 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.046 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-13T04:51:00.046 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 437008 2023-09-13T04:51:00.046 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.046 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2023-09-13T04:51:00.046 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 1220, 2023-09-13T04:51:00.047 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 1219, 2023-09-13T04:51:00.047 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 173867439, 2023-09-13T04:51:00.047 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 378320, 2023-09-13T04:51:00.047 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 6, 2023-09-13T04:51:00.047 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 5, 2023-09-13T04:51:00.048 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.172340740, 2023-09-13T04:51:00.048 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.020146411, 2023-09-13T04:51:00.048 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.113473632, 2023-09-13T04:51:00.048 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.031335220, 2023-09-13T04:51:00.048 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:51:00.049 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1218, 2023-09-13T04:51:00.049 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.323510295, 2023-09-13T04:51:00.049 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000265607 2023-09-13T04:51:00.049 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.049 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:51:00.050 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.050 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.050 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.050 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.050 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-09-13T04:51:00.051 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 448 2023-09-13T04:51:00.051 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.051 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2023-09-13T04:51:00.051 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2023-09-13T04:51:00.051 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2023-09-13T04:51:00.052 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.052 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2023-09-13T04:51:00.052 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2023-09-13T04:51:00.052 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2023-09-13T04:51:00.052 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4497, 2023-09-13T04:51:00.053 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.114209490, 2023-09-13T04:51:00.053 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000025396 2023-09-13T04:51:00.053 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-09-13T04:51:00.053 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.053 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2023-09-13T04:51:00.054 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2023-09-13T04:51:00.054 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2023-09-13T04:51:00.054 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.054 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.054 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.055 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-09-13T04:51:00.055 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.055 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2023-09-13T04:51:00.055 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 0, 2023-09-13T04:51:00.055 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 0, 2023-09-13T04:51:00.055 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2023-09-13T04:51:00.056 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.056 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.056 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.056 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.056 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2023-09-13T04:51:00.057 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 0, 2023-09-13T04:51:00.057 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2023-09-13T04:51:00.057 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2023-09-13T04:51:00.057 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 0, 2023-09-13T04:51:00.057 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2023-09-13T04:51:00.058 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2023-09-13T04:51:00.058 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 51374, 2023-09-13T04:51:00.058 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 2772, 2023-09-13T04:51:00.058 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 37163, 2023-09-13T04:51:00.058 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 11439, 2023-09-13T04:51:00.059 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 47413, 2023-09-13T04:51:00.059 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 343, 2023-09-13T04:51:00.059 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 0, 2023-09-13T04:51:00.059 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 0, 2023-09-13T04:51:00.060 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 1, 2023-09-13T04:51:00.060 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 0, 2023-09-13T04:51:00.060 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 0, 2023-09-13T04:51:00.060 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 0, 2023-09-13T04:51:00.060 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 0, 2023-09-13T04:51:00.061 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2023-09-13T04:51:00.061 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2023-09-13T04:51:00.061 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2023-09-13T04:51:00.061 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2023-09-13T04:51:00.061 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2023-09-13T04:51:00.062 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 0, 2023-09-13T04:51:00.062 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 0, 2023-09-13T04:51:00.062 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 0, 2023-09-13T04:51:00.062 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 0, 2023-09-13T04:51:00.062 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2023-09-13T04:51:00.063 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 0, 2023-09-13T04:51:00.063 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2023-09-13T04:51:00.063 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 0, 2023-09-13T04:51:00.063 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 0, 2023-09-13T04:51:00.063 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 0, 2023-09-13T04:51:00.063 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 0, 2023-09-13T04:51:00.064 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2023-09-13T04:51:00.064 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2023-09-13T04:51:00.064 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2023-09-13T04:51:00.064 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 0, 2023-09-13T04:51:00.064 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2023-09-13T04:51:00.065 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 0, 2023-09-13T04:51:00.065 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 0, 2023-09-13T04:51:00.065 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2023-09-13T04:51:00.065 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 0, 2023-09-13T04:51:00.065 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 0, 2023-09-13T04:51:00.066 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2023-09-13T04:51:00.066 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-13T04:51:00.066 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-13T04:51:00.066 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 0, 2023-09-13T04:51:00.066 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 0, 2023-09-13T04:51:00.067 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 0, 2023-09-13T04:51:00.067 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 0 2023-09-13T04:51:00.067 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.067 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2023-09-13T04:51:00.067 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 0, 2023-09-13T04:51:00.067 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 0, 2023-09-13T04:51:00.068 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 0, 2023-09-13T04:51:00.068 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 0, 2023-09-13T04:51:00.068 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 0, 2023-09-13T04:51:00.068 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2023-09-13T04:51:00.068 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2023-09-13T04:51:00.069 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2023-09-13T04:51:00.069 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 0, 2023-09-13T04:51:00.069 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 0, 2023-09-13T04:51:00.069 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2023-09-13T04:51:00.069 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2023-09-13T04:51:00.070 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2023-09-13T04:51:00.070 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2023-09-13T04:51:00.070 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2023-09-13T04:51:00.070 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2023-09-13T04:51:00.070 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2023-09-13T04:51:00.071 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2023-09-13T04:51:00.071 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 0, 2023-09-13T04:51:00.071 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2023-09-13T04:51:00.071 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2023-09-13T04:51:00.071 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2023-09-13T04:51:00.072 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2023-09-13T04:51:00.072 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.072 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2023-09-13T04:51:00.072 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2023-09-13T04:51:00.072 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 0, 2023-09-13T04:51:00.072 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2023-09-13T04:51:00.073 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2023-09-13T04:51:00.073 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 98286, 2023-09-13T04:51:00.073 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2023-09-13T04:51:00.073 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2023-09-13T04:51:00.073 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 0, 2023-09-13T04:51:00.073 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2023-09-13T04:51:00.074 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2023-09-13T04:51:00.074 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 139, 2023-09-13T04:51:00.074 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 9, 2023-09-13T04:51:00.074 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2023-09-13T04:51:00.074 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2023-09-13T04:51:00.075 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 4194352, 2023-09-13T04:51:00.075 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 341284086, 2023-09-13T04:51:00.075 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 341272967, 2023-09-13T04:51:00.075 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2023-09-13T04:51:00.075 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.075 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.076 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.076 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.076 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 98286 2023-09-13T04:51:00.076 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.076 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2023-09-13T04:51:00.077 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 16351, 2023-09-13T04:51:00.077 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 51009, 2023-09-13T04:51:00.077 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 34658, 2023-09-13T04:51:00.077 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 583, 2023-09-13T04:51:00.077 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 1557, 2023-09-13T04:51:00.078 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 974, 2023-09-13T04:51:00.078 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 51374, 2023-09-13T04:51:00.078 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 87507, 2023-09-13T04:51:00.078 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 36133, 2023-09-13T04:51:00.078 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 0, 2023-09-13T04:51:00.079 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 0, 2023-09-13T04:51:00.079 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 0, 2023-09-13T04:51:00.079 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 142368, 2023-09-13T04:51:00.079 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 207152 2023-09-13T04:51:00.079 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.080 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2023-09-13T04:51:00.080 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 0, 2023-09-13T04:51:00.080 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 0, 2023-09-13T04:51:00.080 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 0, 2023-09-13T04:51:00.080 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 0, 2023-09-13T04:51:00.081 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 0, 2023-09-13T04:51:00.081 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2023-09-13T04:51:00.081 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.081 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.081 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.081 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.082 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2023-09-13T04:51:00.082 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.082 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.082 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.082 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.083 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2023-09-13T04:51:00.083 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.083 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.083 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.083 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.084 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2023-09-13T04:51:00.084 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.084 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.084 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.084 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.085 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2023-09-13T04:51:00.085 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.085 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.085 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.085 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.086 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2023-09-13T04:51:00.086 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.086 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.086 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.086 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.087 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2023-09-13T04:51:00.087 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.087 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.087 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.087 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.088 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2023-09-13T04:51:00.088 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.088 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.088 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.088 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.088 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2023-09-13T04:51:00.089 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.089 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.089 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.089 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.090 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2023-09-13T04:51:00.090 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.090 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.090 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.090 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.091 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2023-09-13T04:51:00.091 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.091 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.091 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.091 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.092 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2023-09-13T04:51:00.092 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.092 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.092 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.092 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.093 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2023-09-13T04:51:00.093 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.093 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.093 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.093 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.094 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2023-09-13T04:51:00.094 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.094 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.094 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.094 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.095 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2023-09-13T04:51:00.095 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.095 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.095 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.095 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.096 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2023-09-13T04:51:00.096 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.096 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.096 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.096 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.097 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2023-09-13T04:51:00.097 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.097 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.097 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.097 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.097 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2023-09-13T04:51:00.098 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.098 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.098 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.098 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.099 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2023-09-13T04:51:00.099 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.099 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.099 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.099 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.099 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2023-09-13T04:51:00.100 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.100 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.100 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.100 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.101 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2023-09-13T04:51:00.101 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.101 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.101 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.101 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.101 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2023-09-13T04:51:00.102 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.102 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.102 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.102 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.102 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2023-09-13T04:51:00.103 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.103 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.103 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.103 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.103 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2023-09-13T04:51:00.104 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.104 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.104 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.104 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.104 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2023-09-13T04:51:00.104 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.105 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.105 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.105 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.105 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2023-09-13T04:51:00.105 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.106 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.106 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.106 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.106 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2023-09-13T04:51:00.106 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.107 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.107 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.107 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.107 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2023-09-13T04:51:00.107 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.107 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.108 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.108 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.108 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2023-09-13T04:51:00.108 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.108 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.108 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.109 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.109 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2023-09-13T04:51:00.109 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2023-09-13T04:51:00.109 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2023-09-13T04:51:00.109 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.110 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.110 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.110 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-09-13T04:51:00.110 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.110 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2023-09-13T04:51:00.111 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 1, 2023-09-13T04:51:00.111 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 2, 2023-09-13T04:51:00.111 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 1, 2023-09-13T04:51:00.111 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 2, 2023-09-13T04:51:00.111 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2023-09-13T04:51:00.112 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 0, 2023-09-13T04:51:00.112 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 0, 2023-09-13T04:51:00.112 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 4323, 2023-09-13T04:51:00.112 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2023-09-13T04:51:00.112 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.112 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2023-09-13T04:51:00.113 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2023-09-13T04:51:00.113 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2023-09-13T04:51:00.113 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2023-09-13T04:51:00.113 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2023-09-13T04:51:00.113 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2023-09-13T04:51:00.113 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2023-09-13T04:51:00.114 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-13T04:51:00.114 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2023-09-13T04:51:00.114 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-13T04:51:00.114 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2023-09-13T04:51:00.114 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2023-09-13T04:51:00.115 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2023-09-13T04:51:00.115 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-13T04:51:00.115 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2023-09-13T04:51:00.115 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2023-09-13T04:51:00.115 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2023-09-13T04:51:00.115 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2023-09-13T04:51:00.116 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2023-09-13T04:51:00.116 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-13T04:51:00.116 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2023-09-13T04:51:00.116 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-13T04:51:00.116 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2023-09-13T04:51:00.116 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2023-09-13T04:51:00.117 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2023-09-13T04:51:00.117 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2023-09-13T04:51:00.117 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2023-09-13T04:51:00.117 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-13T04:51:00.117 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2023-09-13T04:51:00.118 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2023-09-13T04:51:00.118 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2023-09-13T04:51:00.118 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2023-09-13T04:51:00.118 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2023-09-13T04:51:00.118 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2023-09-13T04:51:00.119 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2023-09-13T04:51:00.119 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2023-09-13T04:51:00.119 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2023-09-13T04:51:00.119 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 805292, 2023-09-13T04:51:00.119 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 16722, 2023-09-13T04:51:00.119 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2023-09-13T04:51:00.120 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2023-09-13T04:51:00.120 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2023-09-13T04:51:00.120 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2023-09-13T04:51:00.120 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2023-09-13T04:51:00.120 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2023-09-13T04:51:00.121 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2023-09-13T04:51:00.121 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2023-09-13T04:51:00.121 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 9416, 2023-09-13T04:51:00.121 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 180, 2023-09-13T04:51:00.121 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2023-09-13T04:51:00.122 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2023-09-13T04:51:00.122 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2023-09-13T04:51:00.122 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2023-09-13T04:51:00.122 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 74600623, 2023-09-13T04:51:00.122 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 705834, 2023-09-13T04:51:00.123 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2023-09-13T04:51:00.123 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2023-09-13T04:51:00.123 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2023-09-13T04:51:00.123 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2023-09-13T04:51:00.123 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.124 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2023-09-13T04:51:00.124 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2023-09-13T04:51:00.124 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2023-09-13T04:51:00.124 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 2724, 2023-09-13T04:51:00.124 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 21, 2023-09-13T04:51:00.125 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 3, 2023-09-13T04:51:00.125 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 2724, 2023-09-13T04:51:00.125 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2023-09-13T04:51:00.125 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2724, 2023-09-13T04:51:00.125 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 26.019476316, 2023-09-13T04:51:00.126 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.009551936 2023-09-13T04:51:00.126 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.126 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2023-09-13T04:51:00.126 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2023-09-13T04:51:00.126 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2724, 2023-09-13T04:51:00.127 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 2725 2023-09-13T04:51:00.127 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.127 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 2724, 2023-09-13T04:51:00.127 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 2724, 2023-09-13T04:51:00.127 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 0, 2023-09-13T04:51:00.128 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2023-09-13T04:51:00.128 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2023-09-13T04:51:00.128 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 1577, 2023-09-13T04:51:00.128 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 0, 2023-09-13T04:51:00.128 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 1145, 2023-09-13T04:51:00.129 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 0, 2023-09-13T04:51:00.129 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 0, 2023-09-13T04:51:00.129 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2023-09-13T04:51:00.129 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2023-09-13T04:51:00.129 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2023-09-13T04:51:00.129 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2023-09-13T04:51:00.130 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 0, 2023-09-13T04:51:00.130 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2023-09-13T04:51:00.130 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2023-09-13T04:51:00.130 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2023-09-13T04:51:00.130 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 1, 2023-09-13T04:51:00.131 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 0, 2023-09-13T04:51:00.131 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2023-09-13T04:51:00.131 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2023-09-13T04:51:00.131 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2023-09-13T04:51:00.131 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2023-09-13T04:51:00.132 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2023-09-13T04:51:00.132 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2023-09-13T04:51:00.132 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2023-09-13T04:51:00.132 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2023-09-13T04:51:00.132 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2023-09-13T04:51:00.133 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2023-09-13T04:51:00.133 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2023-09-13T04:51:00.133 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2023-09-13T04:51:00.133 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2023-09-13T04:51:00.133 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2023-09-13T04:51:00.134 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2023-09-13T04:51:00.134 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2023-09-13T04:51:00.134 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2023-09-13T04:51:00.134 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2023-09-13T04:51:00.134 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2023-09-13T04:51:00.135 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2023-09-13T04:51:00.135 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2023-09-13T04:51:00.135 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2023-09-13T04:51:00.135 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2023-09-13T04:51:00.135 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2023-09-13T04:51:00.135 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2023-09-13T04:51:00.136 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2023-09-13T04:51:00.136 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 104, 2023-09-13T04:51:00.136 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2023-09-13T04:51:00.136 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 14, 2023-09-13T04:51:00.136 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 11, 2023-09-13T04:51:00.136 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 11, 2023-09-13T04:51:00.137 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2023-09-13T04:51:00.137 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2023-09-13T04:51:00.137 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 0, 2023-09-13T04:51:00.137 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 2, 2023-09-13T04:51:00.137 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 0 2023-09-13T04:51:00.138 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.138 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2023-09-13T04:51:00.138 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 0, 2023-09-13T04:51:00.138 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2023-09-13T04:51:00.138 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2023-09-13T04:51:00.139 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2023-09-13T04:51:00.139 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.139 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2023-09-13T04:51:00.139 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2023-09-13T04:51:00.139 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.139 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2023-09-13T04:51:00.140 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2023-09-13T04:51:00.140 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 0, 2023-09-13T04:51:00.140 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2023-09-13T04:51:00.140 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 0, 2023-09-13T04:51:00.140 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 0, 2023-09-13T04:51:00.141 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2023-09-13T04:51:00.141 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.141 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-13T04:51:00.141 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-09-13T04:51:00.141 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2023-09-13T04:51:00.142 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-09-13T04:51:00.142 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 2986, 2023-09-13T04:51:00.142 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 339186142, 2023-09-13T04:51:00.142 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-09-13T04:51:00.142 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 2986, 2023-09-13T04:51:00.143 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2023-09-13T04:51:00.143 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2023-09-13T04:51:00.143 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 2986, 2023-09-13T04:51:00.143 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 339186142, 2023-09-13T04:51:00.143 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-09-13T04:51:00.143 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.144 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.144 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.144 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-09-13T04:51:00.144 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.144 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2023-09-13T04:51:00.145 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-09-13T04:51:00.145 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2023-09-13T04:51:00.145 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-09-13T04:51:00.145 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2023-09-13T04:51:00.145 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2023-09-13T04:51:00.146 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-09-13T04:51:00.146 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2023-09-13T04:51:00.146 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 2724, 2023-09-13T04:51:00.146 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 337204962, 2023-09-13T04:51:00.146 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 2724, 2023-09-13T04:51:00.147 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 337204962, 2023-09-13T04:51:00.147 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-09-13T04:51:00.147 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.147 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.147 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.148 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-09-13T04:51:00.148 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.148 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2023-09-13T04:51:00.148 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-09-13T04:51:00.148 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2023-09-13T04:51:00.149 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-09-13T04:51:00.149 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2023-09-13T04:51:00.149 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2023-09-13T04:51:00.149 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-09-13T04:51:00.149 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2023-09-13T04:51:00.150 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 2724, 2023-09-13T04:51:00.150 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 2724, 2023-09-13T04:51:00.150 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 2724, 2023-09-13T04:51:00.150 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 2724, 2023-09-13T04:51:00.150 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-09-13T04:51:00.151 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.151 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.151 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.151 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-09-13T04:51:00.151 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.152 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2023-09-13T04:51:00.152 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-09-13T04:51:00.152 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2023-09-13T04:51:00.152 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-09-13T04:51:00.152 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2023-09-13T04:51:00.152 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2023-09-13T04:51:00.153 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-09-13T04:51:00.153 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2023-09-13T04:51:00.153 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2023-09-13T04:51:00.153 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2023-09-13T04:51:00.153 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2023-09-13T04:51:00.154 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2023-09-13T04:51:00.154 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-09-13T04:51:00.154 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.154 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.154 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.155 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-09-13T04:51:00.155 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:00.155 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x55b027340960": { 2023-09-13T04:51:00.155 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-09-13T04:51:00.155 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2023-09-13T04:51:00.156 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-09-13T04:51:00.156 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2023-09-13T04:51:00.156 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2023-09-13T04:51:00.156 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-09-13T04:51:00.156 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2023-09-13T04:51:00.157 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2023-09-13T04:51:00.157 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2023-09-13T04:51:00.157 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2023-09-13T04:51:00.157 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2023-09-13T04:51:00.157 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-09-13T04:51:00.157 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:00.158 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:00.158 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:00.158 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-09-13T04:51:00.158 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-09-13T04:51:00.158 INFO:teuthology.orchestra.run.smithi149.stdout:} 2023-09-13T04:51:00.159 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:50:59 smithi149 ceph-mon[133315]: pgmap v329: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-13T04:51:00.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:50:59 smithi156 ceph-mon[133675]: pgmap v329: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-13T04:51:00.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:50:59 smithi002 ceph-mon[97651]: pgmap v329: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-13T04:51:00.381 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2023-09-13T04:51:00.382 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2023-09-13T04:51:00.382 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2023-09-13T04:51:00.382 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2023-09-13T04:51:00.382 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2023-09-13T04:51:00.383 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-09-13T04:51:00.383 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-09-13T04:51:00.383 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2023-09-13T04:51:00.383 DEBUG:tasks.check_counter:Getting stats from k 2023-09-13T04:51:00.384 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph daemon mds.k perf dump 2023-09-13T04:51:00.945 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.b/config 2023-09-13T04:51:02.165 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2023-09-13T04:51:02.166 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2023-09-13T04:51:02.166 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 4, 2023-09-13T04:51:02.166 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 4, 2023-09-13T04:51:02.167 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 2992, 2023-09-13T04:51:02.167 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 448, 2023-09-13T04:51:02.167 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 4, 2023-09-13T04:51:02.167 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 0, 2023-09-13T04:51:02.167 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.001523468, 2023-09-13T04:51:02.168 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.000041015, 2023-09-13T04:51:02.168 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.000617062, 2023-09-13T04:51:02.168 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-13T04:51:02.168 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:51:02.168 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 3, 2023-09-13T04:51:02.169 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000754008, 2023-09-13T04:51:02.169 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000251336 2023-09-13T04:51:02.169 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:02.169 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:51:02.169 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:02.170 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:02.170 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:02.170 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:02.170 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-09-13T04:51:02.170 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 448 2023-09-13T04:51:02.171 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:02.171 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2023-09-13T04:51:02.171 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 2, 2023-09-13T04:51:02.171 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 167, 2023-09-13T04:51:02.171 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 320, 2023-09-13T04:51:02.172 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 181952, 2023-09-13T04:51:02.172 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 5, 2023-09-13T04:51:02.172 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 3, 2023-09-13T04:51:02.172 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.008938051, 2023-09-13T04:51:02.172 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.005403700, 2023-09-13T04:51:02.173 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.001554918, 2023-09-13T04:51:02.173 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-13T04:51:02.173 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:51:02.173 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 164, 2023-09-13T04:51:02.173 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.017150551, 2023-09-13T04:51:02.174 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000104576 2023-09-13T04:51:02.174 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:02.174 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:51:02.174 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:02.174 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:02.175 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:02.175 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:02.175 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-13T04:51:02.175 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 181952 2023-09-13T04:51:02.176 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:02.176 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2023-09-13T04:51:02.176 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 229, 2023-09-13T04:51:02.176 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 219, 2023-09-13T04:51:02.176 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 1419184, 2023-09-13T04:51:02.176 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 131168, 2023-09-13T04:51:02.177 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 2, 2023-09-13T04:51:02.177 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 1, 2023-09-13T04:51:02.177 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.042084326, 2023-09-13T04:51:02.177 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.011877072, 2023-09-13T04:51:02.177 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.033238022, 2023-09-13T04:51:02.178 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.006024138, 2023-09-13T04:51:02.178 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:51:02.178 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 218, 2023-09-13T04:51:02.178 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.013750567, 2023-09-13T04:51:02.178 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000063075 2023-09-13T04:51:02.179 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:02.179 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:51:02.179 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:02.179 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:02.179 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:02.179 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:02.180 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 1419184, 2023-09-13T04:51:02.180 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 131168 2023-09-13T04:51:02.180 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:02.180 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2023-09-13T04:51:02.180 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 0, 2023-09-13T04:51:02.181 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2023-09-13T04:51:02.181 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:02.181 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2023-09-13T04:51:02.181 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2023-09-13T04:51:02.181 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2023-09-13T04:51:02.182 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2023-09-13T04:51:02.182 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2023-09-13T04:51:02.182 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2023-09-13T04:51:02.182 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2023-09-13T04:51:02.182 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-13T04:51:02.183 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2023-09-13T04:51:02.183 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-13T04:51:02.183 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2023-09-13T04:51:02.183 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2023-09-13T04:51:02.183 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2023-09-13T04:51:02.184 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-13T04:51:02.184 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2023-09-13T04:51:02.184 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2023-09-13T04:51:02.184 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2023-09-13T04:51:02.184 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2023-09-13T04:51:02.184 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2023-09-13T04:51:02.185 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-13T04:51:02.185 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2023-09-13T04:51:02.185 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-13T04:51:02.185 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2023-09-13T04:51:02.185 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2023-09-13T04:51:02.186 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2023-09-13T04:51:02.186 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2023-09-13T04:51:02.186 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2023-09-13T04:51:02.186 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-13T04:51:02.186 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2023-09-13T04:51:02.187 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2023-09-13T04:51:02.187 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2023-09-13T04:51:02.187 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2023-09-13T04:51:02.187 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2023-09-13T04:51:02.187 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2023-09-13T04:51:02.187 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2023-09-13T04:51:02.188 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2023-09-13T04:51:02.188 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2023-09-13T04:51:02.188 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 148714, 2023-09-13T04:51:02.188 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 30, 2023-09-13T04:51:02.188 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2023-09-13T04:51:02.189 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2023-09-13T04:51:02.189 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2023-09-13T04:51:02.189 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2023-09-13T04:51:02.189 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2023-09-13T04:51:02.189 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2023-09-13T04:51:02.189 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2023-09-13T04:51:02.190 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2023-09-13T04:51:02.190 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 0, 2023-09-13T04:51:02.190 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 0, 2023-09-13T04:51:02.190 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2023-09-13T04:51:02.190 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2023-09-13T04:51:02.191 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2023-09-13T04:51:02.191 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2023-09-13T04:51:02.191 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 928, 2023-09-13T04:51:02.191 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 2, 2023-09-13T04:51:02.191 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2023-09-13T04:51:02.192 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2023-09-13T04:51:02.192 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2023-09-13T04:51:02.192 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2023-09-13T04:51:02.192 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-09-13T04:51:02.192 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-13T04:51:02.192 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-09-13T04:51:02.193 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2023-09-13T04:51:02.193 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-09-13T04:51:02.193 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 231, 2023-09-13T04:51:02.193 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 1386899, 2023-09-13T04:51:02.193 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-09-13T04:51:02.193 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 231, 2023-09-13T04:51:02.194 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2023-09-13T04:51:02.194 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2023-09-13T04:51:02.194 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 231, 2023-09-13T04:51:02.194 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 1386899, 2023-09-13T04:51:02.194 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-09-13T04:51:02.195 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-09-13T04:51:02.195 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-09-13T04:51:02.195 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-09-13T04:51:02.195 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-09-13T04:51:02.195 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-09-13T04:51:02.195 INFO:teuthology.orchestra.run.smithi149.stdout:} 2023-09-13T04:51:02.196 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:01 smithi156 ceph-mon[133675]: pgmap v330: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:51:02.196 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:01 smithi149 ceph-mon[133315]: pgmap v330: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:51:02.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:01 smithi002 ceph-mon[97651]: pgmap v330: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-13T04:51:02.592 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-09-13T04:51:02.592 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-09-13T04:51:02.592 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2023-09-13T04:51:02.593 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2023-09-13T04:51:02.593 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2023-09-13T04:51:02.593 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2023-09-13T04:51:02.593 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2023-09-13T04:51:02.594 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2023-09-13T04:51:02.594 DEBUG:tasks.check_counter:Getting stats from c 2023-09-13T04:51:02.594 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph daemon mds.c perf dump 2023-09-13T04:51:03.175 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-13T04:51:04.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:03 smithi156 ceph-mon[133675]: pgmap v331: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:51:04.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:03 smithi002 ceph-mon[97651]: pgmap v331: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:51:04.291 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-09-13T04:51:04.292 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2023-09-13T04:51:04.292 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 112, 2023-09-13T04:51:04.292 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 114, 2023-09-13T04:51:04.292 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 40846, 2023-09-13T04:51:04.292 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 34842, 2023-09-13T04:51:04.293 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 6, 2023-09-13T04:51:04.293 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 2, 2023-09-13T04:51:04.293 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.014805978, 2023-09-13T04:51:04.293 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.003638450, 2023-09-13T04:51:04.293 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.004201373, 2023-09-13T04:51:04.294 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.004526319, 2023-09-13T04:51:04.294 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:51:04.294 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 112, 2023-09-13T04:51:04.294 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.009211255, 2023-09-13T04:51:04.294 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000082243 2023-09-13T04:51:04.295 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.295 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:51:04.295 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.295 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.295 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.295 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.296 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-13T04:51:04.296 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 0 2023-09-13T04:51:04.296 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.296 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2023-09-13T04:51:04.296 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 126, 2023-09-13T04:51:04.297 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 148, 2023-09-13T04:51:04.297 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 451664, 2023-09-13T04:51:04.297 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 162577, 2023-09-13T04:51:04.297 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 5, 2023-09-13T04:51:04.297 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 4, 2023-09-13T04:51:04.298 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.013231266, 2023-09-13T04:51:04.298 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.005101061, 2023-09-13T04:51:04.298 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.003980707, 2023-09-13T04:51:04.298 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.002949841, 2023-09-13T04:51:04.298 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:51:04.299 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 147, 2023-09-13T04:51:04.299 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.028908518, 2023-09-13T04:51:04.299 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000196656 2023-09-13T04:51:04.299 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.299 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:51:04.300 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.300 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.300 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.300 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.300 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-09-13T04:51:04.301 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 124880 2023-09-13T04:51:04.301 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.301 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2023-09-13T04:51:04.301 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 156, 2023-09-13T04:51:04.301 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 155, 2023-09-13T04:51:04.301 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 193757, 2023-09-13T04:51:04.302 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 55123, 2023-09-13T04:51:04.302 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 3, 2023-09-13T04:51:04.302 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 2, 2023-09-13T04:51:04.302 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.016769330, 2023-09-13T04:51:04.302 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.005622775, 2023-09-13T04:51:04.303 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.006150318, 2023-09-13T04:51:04.303 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.004698659, 2023-09-13T04:51:04.303 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:51:04.303 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 154, 2023-09-13T04:51:04.303 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.013760952, 2023-09-13T04:51:04.304 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000089356 2023-09-13T04:51:04.304 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.304 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:51:04.304 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.304 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.304 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.305 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.305 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 166080, 2023-09-13T04:51:04.305 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 20912 2023-09-13T04:51:04.305 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.305 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2023-09-13T04:51:04.305 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2023-09-13T04:51:04.306 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2023-09-13T04:51:04.306 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.306 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2023-09-13T04:51:04.306 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-09-13T04:51:04.306 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-09-13T04:51:04.307 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 561, 2023-09-13T04:51:04.307 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.016371949, 2023-09-13T04:51:04.307 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000029183 2023-09-13T04:51:04.307 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:04.307 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.307 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2023-09-13T04:51:04.308 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-09-13T04:51:04.308 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-09-13T04:51:04.308 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.308 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.308 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.309 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:04.309 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.309 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2023-09-13T04:51:04.309 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 0, 2023-09-13T04:51:04.309 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 0, 2023-09-13T04:51:04.310 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2023-09-13T04:51:04.310 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.310 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.310 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.310 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.311 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2023-09-13T04:51:04.311 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 0, 2023-09-13T04:51:04.311 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2023-09-13T04:51:04.311 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2023-09-13T04:51:04.311 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 0, 2023-09-13T04:51:04.311 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 0, 2023-09-13T04:51:04.312 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2023-09-13T04:51:04.312 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 170, 2023-09-13T04:51:04.312 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 0, 2023-09-13T04:51:04.312 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 0, 2023-09-13T04:51:04.312 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 170, 2023-09-13T04:51:04.313 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 170, 2023-09-13T04:51:04.313 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 0, 2023-09-13T04:51:04.313 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 0, 2023-09-13T04:51:04.313 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 0, 2023-09-13T04:51:04.313 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 2, 2023-09-13T04:51:04.313 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 0, 2023-09-13T04:51:04.314 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 0, 2023-09-13T04:51:04.314 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 0, 2023-09-13T04:51:04.314 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 0, 2023-09-13T04:51:04.314 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2023-09-13T04:51:04.314 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2023-09-13T04:51:04.315 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 0, 2023-09-13T04:51:04.315 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2023-09-13T04:51:04.315 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2023-09-13T04:51:04.315 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 0, 2023-09-13T04:51:04.315 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 0, 2023-09-13T04:51:04.316 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 0, 2023-09-13T04:51:04.316 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 0, 2023-09-13T04:51:04.316 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2023-09-13T04:51:04.316 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 0, 2023-09-13T04:51:04.316 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2023-09-13T04:51:04.316 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 0, 2023-09-13T04:51:04.317 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 0, 2023-09-13T04:51:04.317 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 6, 2023-09-13T04:51:04.317 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 0, 2023-09-13T04:51:04.317 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2023-09-13T04:51:04.317 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2023-09-13T04:51:04.318 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2023-09-13T04:51:04.318 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 0, 2023-09-13T04:51:04.318 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2023-09-13T04:51:04.318 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 0, 2023-09-13T04:51:04.318 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 0, 2023-09-13T04:51:04.319 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 0, 2023-09-13T04:51:04.319 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 0, 2023-09-13T04:51:04.319 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 0, 2023-09-13T04:51:04.319 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2023-09-13T04:51:04.319 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-13T04:51:04.320 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-13T04:51:04.320 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 0, 2023-09-13T04:51:04.320 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 0, 2023-09-13T04:51:04.320 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 0, 2023-09-13T04:51:04.320 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 0 2023-09-13T04:51:04.321 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.321 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2023-09-13T04:51:04.321 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 0, 2023-09-13T04:51:04.321 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 0, 2023-09-13T04:51:04.321 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 0, 2023-09-13T04:51:04.322 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 0, 2023-09-13T04:51:04.322 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 0, 2023-09-13T04:51:04.322 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 0, 2023-09-13T04:51:04.322 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2023-09-13T04:51:04.322 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2023-09-13T04:51:04.323 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 0, 2023-09-13T04:51:04.323 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 0, 2023-09-13T04:51:04.323 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2023-09-13T04:51:04.323 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2023-09-13T04:51:04.323 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2023-09-13T04:51:04.324 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2023-09-13T04:51:04.324 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2023-09-13T04:51:04.324 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2023-09-13T04:51:04.324 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2023-09-13T04:51:04.324 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2023-09-13T04:51:04.325 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 0, 2023-09-13T04:51:04.325 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2023-09-13T04:51:04.325 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 0, 2023-09-13T04:51:04.325 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2023-09-13T04:51:04.325 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2023-09-13T04:51:04.326 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.326 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2023-09-13T04:51:04.326 INFO:teuthology.orchestra.run.smithi156.stdout: "evlrg": 0, 2023-09-13T04:51:04.326 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 0, 2023-09-13T04:51:04.326 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2023-09-13T04:51:04.327 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2023-09-13T04:51:04.327 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 113, 2023-09-13T04:51:04.327 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2023-09-13T04:51:04.327 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2023-09-13T04:51:04.327 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 0, 2023-09-13T04:51:04.328 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2023-09-13T04:51:04.328 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2023-09-13T04:51:04.328 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 1, 2023-09-13T04:51:04.328 INFO:teuthology.orchestra.run.smithi156.stdout: "segmjr": 1, 2023-09-13T04:51:04.328 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2023-09-13T04:51:04.329 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2023-09-13T04:51:04.329 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 172870329, 2023-09-13T04:51:04.329 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 173283223, 2023-09-13T04:51:04.329 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 173283139, 2023-09-13T04:51:04.329 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2023-09-13T04:51:04.329 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.330 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.330 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.330 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.330 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 113 2023-09-13T04:51:04.330 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.331 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2023-09-13T04:51:04.331 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 77, 2023-09-13T04:51:04.331 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 103, 2023-09-13T04:51:04.331 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 26, 2023-09-13T04:51:04.331 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 8, 2023-09-13T04:51:04.332 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 8, 2023-09-13T04:51:04.332 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 0, 2023-09-13T04:51:04.332 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 170, 2023-09-13T04:51:04.332 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 170, 2023-09-13T04:51:04.332 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 0, 2023-09-13T04:51:04.332 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 0, 2023-09-13T04:51:04.333 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 0, 2023-09-13T04:51:04.333 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 0, 2023-09-13T04:51:04.333 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 45388, 2023-09-13T04:51:04.333 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 207152 2023-09-13T04:51:04.333 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.334 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2023-09-13T04:51:04.334 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 0, 2023-09-13T04:51:04.334 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 0, 2023-09-13T04:51:04.334 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 0, 2023-09-13T04:51:04.334 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 0, 2023-09-13T04:51:04.334 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 0, 2023-09-13T04:51:04.335 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2023-09-13T04:51:04.335 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.335 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.335 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.335 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.336 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2023-09-13T04:51:04.336 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.336 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.336 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.336 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.336 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2023-09-13T04:51:04.337 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.337 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.337 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.337 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.337 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2023-09-13T04:51:04.338 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.338 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.338 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.338 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.338 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2023-09-13T04:51:04.339 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.339 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.339 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.339 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.339 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2023-09-13T04:51:04.339 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.340 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.340 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.340 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.340 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2023-09-13T04:51:04.340 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.341 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.341 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.341 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.341 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2023-09-13T04:51:04.341 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.341 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.342 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.342 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.342 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2023-09-13T04:51:04.342 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.342 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.343 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.343 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.343 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2023-09-13T04:51:04.343 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.343 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.344 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.344 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.344 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2023-09-13T04:51:04.344 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.344 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.344 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.345 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.345 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2023-09-13T04:51:04.345 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.345 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.345 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.346 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.346 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2023-09-13T04:51:04.346 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.346 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.346 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.347 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.347 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2023-09-13T04:51:04.347 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.347 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.347 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.348 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.348 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2023-09-13T04:51:04.348 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.348 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.348 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.348 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.349 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2023-09-13T04:51:04.349 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.349 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.349 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.349 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.350 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2023-09-13T04:51:04.350 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.350 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.350 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.350 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.351 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2023-09-13T04:51:04.351 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.351 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.351 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.351 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.352 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2023-09-13T04:51:04.352 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.352 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.352 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.352 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.352 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2023-09-13T04:51:04.353 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.353 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.353 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.353 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.353 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2023-09-13T04:51:04.354 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.354 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.354 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.354 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.354 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2023-09-13T04:51:04.355 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.355 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.355 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.355 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.355 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2023-09-13T04:51:04.356 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.356 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.356 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.356 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.356 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2023-09-13T04:51:04.356 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.357 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.357 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.357 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.357 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2023-09-13T04:51:04.357 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.357 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.358 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.358 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.358 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2023-09-13T04:51:04.358 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.358 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.359 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.359 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.359 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2023-09-13T04:51:04.359 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.359 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.360 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.360 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.360 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2023-09-13T04:51:04.360 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.360 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.360 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.361 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.361 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2023-09-13T04:51:04.361 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.361 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.361 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.362 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.362 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2023-09-13T04:51:04.362 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2023-09-13T04:51:04.362 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2023-09-13T04:51:04.362 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.363 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.363 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.363 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:04.363 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.363 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2023-09-13T04:51:04.363 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 2, 2023-09-13T04:51:04.364 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 2, 2023-09-13T04:51:04.364 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 0, 2023-09-13T04:51:04.364 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 0, 2023-09-13T04:51:04.364 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2023-09-13T04:51:04.364 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 0, 2023-09-13T04:51:04.365 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 0, 2023-09-13T04:51:04.365 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 2478, 2023-09-13T04:51:04.365 INFO:teuthology.orchestra.run.smithi156.stdout: "mdthresh_evicted": 0 2023-09-13T04:51:04.365 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.365 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2023-09-13T04:51:04.365 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2023-09-13T04:51:04.366 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2023-09-13T04:51:04.366 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2023-09-13T04:51:04.366 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2023-09-13T04:51:04.366 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2023-09-13T04:51:04.366 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2023-09-13T04:51:04.367 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-13T04:51:04.367 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2023-09-13T04:51:04.367 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-13T04:51:04.367 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2023-09-13T04:51:04.367 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2023-09-13T04:51:04.368 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2023-09-13T04:51:04.368 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-13T04:51:04.368 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2023-09-13T04:51:04.368 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2023-09-13T04:51:04.368 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2023-09-13T04:51:04.368 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2023-09-13T04:51:04.369 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2023-09-13T04:51:04.369 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-13T04:51:04.369 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2023-09-13T04:51:04.369 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-13T04:51:04.369 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2023-09-13T04:51:04.369 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2023-09-13T04:51:04.370 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2023-09-13T04:51:04.370 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2023-09-13T04:51:04.370 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2023-09-13T04:51:04.370 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-13T04:51:04.370 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2023-09-13T04:51:04.371 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2023-09-13T04:51:04.371 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2023-09-13T04:51:04.371 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2023-09-13T04:51:04.371 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2023-09-13T04:51:04.371 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2023-09-13T04:51:04.372 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2023-09-13T04:51:04.372 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2023-09-13T04:51:04.372 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2023-09-13T04:51:04.372 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 191749, 2023-09-13T04:51:04.372 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 143, 2023-09-13T04:51:04.373 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2023-09-13T04:51:04.373 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2023-09-13T04:51:04.373 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2023-09-13T04:51:04.373 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2023-09-13T04:51:04.373 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2023-09-13T04:51:04.373 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2023-09-13T04:51:04.374 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2023-09-13T04:51:04.374 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2023-09-13T04:51:04.374 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 9416, 2023-09-13T04:51:04.374 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 180, 2023-09-13T04:51:04.374 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2023-09-13T04:51:04.375 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2023-09-13T04:51:04.375 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2023-09-13T04:51:04.375 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2023-09-13T04:51:04.375 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 332738, 2023-09-13T04:51:04.375 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 3131, 2023-09-13T04:51:04.375 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2023-09-13T04:51:04.376 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2023-09-13T04:51:04.376 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2023-09-13T04:51:04.376 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2023-09-13T04:51:04.376 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.376 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2023-09-13T04:51:04.377 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2023-09-13T04:51:04.377 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2023-09-13T04:51:04.377 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 343, 2023-09-13T04:51:04.377 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 26, 2023-09-13T04:51:04.377 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 4, 2023-09-13T04:51:04.378 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 343, 2023-09-13T04:51:04.378 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2023-09-13T04:51:04.378 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 343, 2023-09-13T04:51:04.378 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.260995907, 2023-09-13T04:51:04.378 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000760921 2023-09-13T04:51:04.379 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.379 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2023-09-13T04:51:04.379 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2023-09-13T04:51:04.379 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 343, 2023-09-13T04:51:04.379 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 344 2023-09-13T04:51:04.379 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.380 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 343, 2023-09-13T04:51:04.380 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 343, 2023-09-13T04:51:04.380 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 0, 2023-09-13T04:51:04.380 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2023-09-13T04:51:04.380 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2023-09-13T04:51:04.380 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 218, 2023-09-13T04:51:04.381 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 0, 2023-09-13T04:51:04.381 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 122, 2023-09-13T04:51:04.381 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 0, 2023-09-13T04:51:04.381 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 0, 2023-09-13T04:51:04.381 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2023-09-13T04:51:04.382 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2023-09-13T04:51:04.382 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2023-09-13T04:51:04.382 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 0, 2023-09-13T04:51:04.382 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 0, 2023-09-13T04:51:04.382 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2023-09-13T04:51:04.383 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2023-09-13T04:51:04.383 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2023-09-13T04:51:04.383 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 2, 2023-09-13T04:51:04.383 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 0, 2023-09-13T04:51:04.383 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2023-09-13T04:51:04.383 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2023-09-13T04:51:04.384 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2023-09-13T04:51:04.384 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2023-09-13T04:51:04.384 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2023-09-13T04:51:04.384 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2023-09-13T04:51:04.384 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2023-09-13T04:51:04.385 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2023-09-13T04:51:04.385 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2023-09-13T04:51:04.385 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2023-09-13T04:51:04.385 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2023-09-13T04:51:04.385 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2023-09-13T04:51:04.385 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2023-09-13T04:51:04.386 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2023-09-13T04:51:04.386 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2023-09-13T04:51:04.386 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2023-09-13T04:51:04.386 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2023-09-13T04:51:04.386 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2023-09-13T04:51:04.387 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2023-09-13T04:51:04.387 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2023-09-13T04:51:04.387 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2023-09-13T04:51:04.387 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2023-09-13T04:51:04.387 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2023-09-13T04:51:04.387 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2023-09-13T04:51:04.388 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2023-09-13T04:51:04.388 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2023-09-13T04:51:04.388 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 104, 2023-09-13T04:51:04.388 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2023-09-13T04:51:04.388 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 4, 2023-09-13T04:51:04.389 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 5, 2023-09-13T04:51:04.389 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 5, 2023-09-13T04:51:04.389 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2023-09-13T04:51:04.389 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2023-09-13T04:51:04.389 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 0, 2023-09-13T04:51:04.389 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 2, 2023-09-13T04:51:04.390 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 0 2023-09-13T04:51:04.390 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.390 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2023-09-13T04:51:04.390 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 0, 2023-09-13T04:51:04.390 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2023-09-13T04:51:04.391 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2023-09-13T04:51:04.391 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2023-09-13T04:51:04.391 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.391 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2023-09-13T04:51:04.391 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2023-09-13T04:51:04.392 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.392 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2023-09-13T04:51:04.392 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2023-09-13T04:51:04.392 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 0, 2023-09-13T04:51:04.392 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2023-09-13T04:51:04.393 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 0, 2023-09-13T04:51:04.393 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 0, 2023-09-13T04:51:04.393 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2023-09-13T04:51:04.393 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.393 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-13T04:51:04.394 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-13T04:51:04.394 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-09-13T04:51:04.394 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-13T04:51:04.394 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 390, 2023-09-13T04:51:04.394 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 645847, 2023-09-13T04:51:04.395 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-13T04:51:04.395 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 390, 2023-09-13T04:51:04.395 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-09-13T04:51:04.395 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-09-13T04:51:04.395 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 390, 2023-09-13T04:51:04.395 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 645847, 2023-09-13T04:51:04.396 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-13T04:51:04.396 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.396 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.396 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.396 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:04.397 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.397 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2023-09-13T04:51:04.397 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-13T04:51:04.397 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-09-13T04:51:04.397 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-13T04:51:04.398 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-09-13T04:51:04.398 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-09-13T04:51:04.398 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-13T04:51:04.398 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-09-13T04:51:04.398 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 343, 2023-09-13T04:51:04.399 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 412904, 2023-09-13T04:51:04.399 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 343, 2023-09-13T04:51:04.399 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 412904, 2023-09-13T04:51:04.399 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-13T04:51:04.399 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.399 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.400 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.400 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:04.400 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.400 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2023-09-13T04:51:04.400 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-13T04:51:04.401 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2023-09-13T04:51:04.401 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-13T04:51:04.401 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-09-13T04:51:04.401 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-09-13T04:51:04.401 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-13T04:51:04.402 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-09-13T04:51:04.402 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 343, 2023-09-13T04:51:04.402 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 343, 2023-09-13T04:51:04.402 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 343, 2023-09-13T04:51:04.402 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 343, 2023-09-13T04:51:04.403 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-13T04:51:04.403 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.403 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.403 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.403 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:04.404 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.404 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2023-09-13T04:51:04.404 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-13T04:51:04.404 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-09-13T04:51:04.404 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-13T04:51:04.405 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-09-13T04:51:04.405 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-09-13T04:51:04.405 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-13T04:51:04.405 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-09-13T04:51:04.405 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-09-13T04:51:04.405 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-09-13T04:51:04.406 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2023-09-13T04:51:04.406 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2023-09-13T04:51:04.406 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-13T04:51:04.406 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.406 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.407 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.407 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:04.407 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:04.407 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x558832238960": { 2023-09-13T04:51:04.407 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-13T04:51:04.408 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-09-13T04:51:04.408 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-13T04:51:04.408 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-09-13T04:51:04.408 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-09-13T04:51:04.408 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-13T04:51:04.409 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-09-13T04:51:04.409 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-09-13T04:51:04.409 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-09-13T04:51:04.409 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2023-09-13T04:51:04.409 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2023-09-13T04:51:04.409 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-13T04:51:04.410 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:04.410 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:04.410 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:04.410 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:04.410 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:04.411 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-09-13T04:51:04.411 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:03 smithi149 ceph-mon[133315]: pgmap v331: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-13T04:51:04.718 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-09-13T04:51:04.718 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-09-13T04:51:04.719 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2023-09-13T04:51:04.719 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2023-09-13T04:51:04.719 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2023-09-13T04:51:04.719 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2023-09-13T04:51:04.720 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2023-09-13T04:51:04.720 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2023-09-13T04:51:04.720 DEBUG:tasks.check_counter:Getting stats from f 2023-09-13T04:51:04.721 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph daemon mds.f perf dump 2023-09-13T04:51:05.274 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-13T04:51:06.103 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:05 smithi156 ceph-mon[133675]: pgmap v332: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-09-13T04:51:06.255 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:05 smithi002 ceph-mon[97651]: pgmap v332: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-09-13T04:51:06.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:05 smithi149 ceph-mon[133315]: pgmap v332: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-09-13T04:51:06.637 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-09-13T04:51:06.637 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2023-09-13T04:51:06.638 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 618, 2023-09-13T04:51:06.638 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 591, 2023-09-13T04:51:06.638 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 43544716, 2023-09-13T04:51:06.638 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 243765, 2023-09-13T04:51:06.638 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 6, 2023-09-13T04:51:06.639 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 2, 2023-09-13T04:51:06.639 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.087552974, 2023-09-13T04:51:06.639 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.019078935, 2023-09-13T04:51:06.639 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.058965285, 2023-09-13T04:51:06.639 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.015040628, 2023-09-13T04:51:06.640 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:51:06.640 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 591, 2023-09-13T04:51:06.640 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.030603252, 2023-09-13T04:51:06.640 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000051782 2023-09-13T04:51:06.640 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.641 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:51:06.641 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.641 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.641 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.641 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.642 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1510928, 2023-09-13T04:51:06.642 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 134288 2023-09-13T04:51:06.642 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.642 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2023-09-13T04:51:06.642 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 347, 2023-09-13T04:51:06.643 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 511, 2023-09-13T04:51:06.643 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 46246005, 2023-09-13T04:51:06.643 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 546245, 2023-09-13T04:51:06.643 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 8, 2023-09-13T04:51:06.643 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 6, 2023-09-13T04:51:06.643 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.055198019, 2023-09-13T04:51:06.644 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.012603908, 2023-09-13T04:51:06.644 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.029938580, 2023-09-13T04:51:06.644 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.008661050, 2023-09-13T04:51:06.644 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:51:06.644 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 508, 2023-09-13T04:51:06.645 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.075506098, 2023-09-13T04:51:06.645 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000148634 2023-09-13T04:51:06.645 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.645 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:51:06.645 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.645 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.646 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.646 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.646 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-13T04:51:06.646 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 438944 2023-09-13T04:51:06.646 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.647 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2023-09-13T04:51:06.647 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 1850, 2023-09-13T04:51:06.647 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 1852, 2023-09-13T04:51:06.647 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 65950138, 2023-09-13T04:51:06.647 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 575408, 2023-09-13T04:51:06.648 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 6, 2023-09-13T04:51:06.648 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 5, 2023-09-13T04:51:06.648 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.185563735, 2023-09-13T04:51:06.648 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.040036557, 2023-09-13T04:51:06.648 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.073434189, 2023-09-13T04:51:06.648 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.058704947, 2023-09-13T04:51:06.649 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:51:06.649 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1850, 2023-09-13T04:51:06.649 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.087683887, 2023-09-13T04:51:06.649 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000047396 2023-09-13T04:51:06.649 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.650 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:51:06.650 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.650 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.650 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.650 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.651 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-13T04:51:06.651 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 0 2023-09-13T04:51:06.651 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.651 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2023-09-13T04:51:06.651 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2023-09-13T04:51:06.652 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2023-09-13T04:51:06.652 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.652 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2023-09-13T04:51:06.652 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-09-13T04:51:06.652 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-09-13T04:51:06.653 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 4254, 2023-09-13T04:51:06.653 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.124644324, 2023-09-13T04:51:06.653 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000029300 2023-09-13T04:51:06.653 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:06.653 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.653 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2023-09-13T04:51:06.654 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-09-13T04:51:06.654 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-09-13T04:51:06.654 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.654 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.654 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.655 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:06.655 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.655 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2023-09-13T04:51:06.655 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 0, 2023-09-13T04:51:06.655 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 0, 2023-09-13T04:51:06.655 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2023-09-13T04:51:06.656 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.656 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.656 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.656 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.656 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2023-09-13T04:51:06.657 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 0, 2023-09-13T04:51:06.657 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2023-09-13T04:51:06.657 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2023-09-13T04:51:06.657 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 0, 2023-09-13T04:51:06.657 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 0, 2023-09-13T04:51:06.657 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2023-09-13T04:51:06.658 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 21661, 2023-09-13T04:51:06.658 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 522, 2023-09-13T04:51:06.658 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 9270, 2023-09-13T04:51:06.658 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 11869, 2023-09-13T04:51:06.658 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 20915, 2023-09-13T04:51:06.659 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 1100, 2023-09-13T04:51:06.659 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 0, 2023-09-13T04:51:06.659 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 0, 2023-09-13T04:51:06.659 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 1, 2023-09-13T04:51:06.659 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 0, 2023-09-13T04:51:06.659 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 0, 2023-09-13T04:51:06.660 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 0, 2023-09-13T04:51:06.660 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 0, 2023-09-13T04:51:06.660 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2023-09-13T04:51:06.660 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2023-09-13T04:51:06.660 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 0, 2023-09-13T04:51:06.661 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2023-09-13T04:51:06.661 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2023-09-13T04:51:06.661 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 0, 2023-09-13T04:51:06.661 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 0, 2023-09-13T04:51:06.661 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 0, 2023-09-13T04:51:06.662 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 0, 2023-09-13T04:51:06.662 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2023-09-13T04:51:06.662 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 0, 2023-09-13T04:51:06.662 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2023-09-13T04:51:06.662 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 0, 2023-09-13T04:51:06.663 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 0, 2023-09-13T04:51:06.663 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 0, 2023-09-13T04:51:06.663 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 0, 2023-09-13T04:51:06.663 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2023-09-13T04:51:06.663 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2023-09-13T04:51:06.663 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2023-09-13T04:51:06.664 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 0, 2023-09-13T04:51:06.664 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2023-09-13T04:51:06.664 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 0, 2023-09-13T04:51:06.664 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 0, 2023-09-13T04:51:06.664 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 0, 2023-09-13T04:51:06.664 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 0, 2023-09-13T04:51:06.665 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 0, 2023-09-13T04:51:06.665 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2023-09-13T04:51:06.665 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-13T04:51:06.665 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-13T04:51:06.665 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 0, 2023-09-13T04:51:06.666 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 0, 2023-09-13T04:51:06.666 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 0, 2023-09-13T04:51:06.666 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 0 2023-09-13T04:51:06.666 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.666 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2023-09-13T04:51:06.666 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 0, 2023-09-13T04:51:06.667 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 0, 2023-09-13T04:51:06.667 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 0, 2023-09-13T04:51:06.667 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 0, 2023-09-13T04:51:06.667 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 0, 2023-09-13T04:51:06.667 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 0, 2023-09-13T04:51:06.668 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2023-09-13T04:51:06.668 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2023-09-13T04:51:06.668 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 0, 2023-09-13T04:51:06.668 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 0, 2023-09-13T04:51:06.668 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2023-09-13T04:51:06.668 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2023-09-13T04:51:06.669 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2023-09-13T04:51:06.669 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2023-09-13T04:51:06.669 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2023-09-13T04:51:06.669 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2023-09-13T04:51:06.669 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2023-09-13T04:51:06.669 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2023-09-13T04:51:06.670 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 0, 2023-09-13T04:51:06.670 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2023-09-13T04:51:06.670 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 0, 2023-09-13T04:51:06.670 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2023-09-13T04:51:06.670 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2023-09-13T04:51:06.671 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.671 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2023-09-13T04:51:06.671 INFO:teuthology.orchestra.run.smithi156.stdout: "evlrg": 0, 2023-09-13T04:51:06.671 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 0, 2023-09-13T04:51:06.671 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2023-09-13T04:51:06.672 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2023-09-13T04:51:06.672 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 48505, 2023-09-13T04:51:06.672 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2023-09-13T04:51:06.672 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2023-09-13T04:51:06.672 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 0, 2023-09-13T04:51:06.672 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2023-09-13T04:51:06.673 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2023-09-13T04:51:06.673 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 72, 2023-09-13T04:51:06.673 INFO:teuthology.orchestra.run.smithi156.stdout: "segmjr": 5, 2023-09-13T04:51:06.673 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2023-09-13T04:51:06.673 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2023-09-13T04:51:06.673 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 4194304, 2023-09-13T04:51:06.674 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 157661155, 2023-09-13T04:51:06.674 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 157658237, 2023-09-13T04:51:06.674 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2023-09-13T04:51:06.674 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.674 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.674 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.675 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.675 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 48505 2023-09-13T04:51:06.675 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.675 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2023-09-13T04:51:06.675 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 6553, 2023-09-13T04:51:06.675 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 26813, 2023-09-13T04:51:06.676 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 20260, 2023-09-13T04:51:06.676 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 243, 2023-09-13T04:51:06.676 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 880, 2023-09-13T04:51:06.676 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 637, 2023-09-13T04:51:06.676 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 21661, 2023-09-13T04:51:06.676 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 44158, 2023-09-13T04:51:06.677 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 22497, 2023-09-13T04:51:06.677 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 0, 2023-09-13T04:51:06.677 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 0, 2023-09-13T04:51:06.677 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 0, 2023-09-13T04:51:06.677 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 106820, 2023-09-13T04:51:06.678 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 207152 2023-09-13T04:51:06.678 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.678 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2023-09-13T04:51:06.678 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 0, 2023-09-13T04:51:06.678 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 0, 2023-09-13T04:51:06.678 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 0, 2023-09-13T04:51:06.679 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 0, 2023-09-13T04:51:06.679 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 0, 2023-09-13T04:51:06.679 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2023-09-13T04:51:06.679 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.679 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.679 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.680 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.680 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2023-09-13T04:51:06.680 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.680 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.680 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.680 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.681 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2023-09-13T04:51:06.681 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.681 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.681 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.681 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.681 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2023-09-13T04:51:06.682 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.682 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.682 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.682 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.682 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2023-09-13T04:51:06.682 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.683 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.683 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.683 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.683 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2023-09-13T04:51:06.683 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.683 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.684 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.684 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.684 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2023-09-13T04:51:06.684 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.684 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.685 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.685 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.685 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2023-09-13T04:51:06.685 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.685 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.685 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.686 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.686 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2023-09-13T04:51:06.686 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.686 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.686 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.687 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.687 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2023-09-13T04:51:06.687 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.687 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.687 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.688 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.688 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2023-09-13T04:51:06.688 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.688 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.688 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.688 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.689 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2023-09-13T04:51:06.689 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.689 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.689 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.689 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.689 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2023-09-13T04:51:06.690 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.690 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.690 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.690 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.690 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2023-09-13T04:51:06.691 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.691 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.691 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.691 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.691 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2023-09-13T04:51:06.692 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.692 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.692 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.692 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.692 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2023-09-13T04:51:06.692 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.693 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.693 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.693 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.693 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2023-09-13T04:51:06.693 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.694 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.694 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.694 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.694 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2023-09-13T04:51:06.694 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.694 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.695 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.695 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.695 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2023-09-13T04:51:06.695 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.695 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.696 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.696 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.696 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2023-09-13T04:51:06.696 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.696 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.696 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.697 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.697 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2023-09-13T04:51:06.697 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.697 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.697 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.697 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.698 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2023-09-13T04:51:06.698 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.698 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.698 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.698 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.698 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2023-09-13T04:51:06.699 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.699 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.699 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.699 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.699 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2023-09-13T04:51:06.699 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.700 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.700 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.700 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.700 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2023-09-13T04:51:06.700 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.701 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.701 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.701 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.701 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2023-09-13T04:51:06.701 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.701 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.702 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.702 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.702 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2023-09-13T04:51:06.702 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.703 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.703 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.703 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.703 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2023-09-13T04:51:06.703 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.703 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.704 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.704 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.704 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2023-09-13T04:51:06.704 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.704 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.705 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.705 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.705 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2023-09-13T04:51:06.705 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2023-09-13T04:51:06.705 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2023-09-13T04:51:06.705 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.706 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.706 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.706 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:06.706 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.706 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2023-09-13T04:51:06.706 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 1, 2023-09-13T04:51:06.707 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 3, 2023-09-13T04:51:06.707 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 2, 2023-09-13T04:51:06.707 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 3, 2023-09-13T04:51:06.707 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2023-09-13T04:51:06.707 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 0, 2023-09-13T04:51:06.707 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 0, 2023-09-13T04:51:06.708 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 6175, 2023-09-13T04:51:06.708 INFO:teuthology.orchestra.run.smithi156.stdout: "mdthresh_evicted": 0 2023-09-13T04:51:06.708 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.708 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2023-09-13T04:51:06.708 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2023-09-13T04:51:06.708 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2023-09-13T04:51:06.709 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2023-09-13T04:51:06.709 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2023-09-13T04:51:06.709 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2023-09-13T04:51:06.709 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2023-09-13T04:51:06.709 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-13T04:51:06.710 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2023-09-13T04:51:06.710 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-13T04:51:06.710 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2023-09-13T04:51:06.710 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2023-09-13T04:51:06.710 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2023-09-13T04:51:06.710 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-13T04:51:06.711 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2023-09-13T04:51:06.711 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2023-09-13T04:51:06.711 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2023-09-13T04:51:06.711 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2023-09-13T04:51:06.711 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2023-09-13T04:51:06.712 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-13T04:51:06.712 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2023-09-13T04:51:06.712 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-13T04:51:06.712 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2023-09-13T04:51:06.712 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2023-09-13T04:51:06.712 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2023-09-13T04:51:06.713 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2023-09-13T04:51:06.713 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2023-09-13T04:51:06.713 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-13T04:51:06.713 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2023-09-13T04:51:06.713 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2023-09-13T04:51:06.713 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2023-09-13T04:51:06.713 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2023-09-13T04:51:06.714 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2023-09-13T04:51:06.714 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2023-09-13T04:51:06.714 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2023-09-13T04:51:06.714 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2023-09-13T04:51:06.714 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2023-09-13T04:51:06.715 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 450095, 2023-09-13T04:51:06.715 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 6742, 2023-09-13T04:51:06.715 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2023-09-13T04:51:06.715 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2023-09-13T04:51:06.715 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2023-09-13T04:51:06.715 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2023-09-13T04:51:06.716 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2023-09-13T04:51:06.716 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2023-09-13T04:51:06.716 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2023-09-13T04:51:06.716 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2023-09-13T04:51:06.716 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 9416, 2023-09-13T04:51:06.716 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 180, 2023-09-13T04:51:06.717 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2023-09-13T04:51:06.717 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2023-09-13T04:51:06.717 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2023-09-13T04:51:06.717 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2023-09-13T04:51:06.717 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 30739980, 2023-09-13T04:51:06.717 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 300084, 2023-09-13T04:51:06.718 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2023-09-13T04:51:06.718 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2023-09-13T04:51:06.718 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2023-09-13T04:51:06.718 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2023-09-13T04:51:06.718 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.718 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2023-09-13T04:51:06.719 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2023-09-13T04:51:06.719 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2023-09-13T04:51:06.719 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 2547, 2023-09-13T04:51:06.719 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 21, 2023-09-13T04:51:06.719 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 3, 2023-09-13T04:51:06.719 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 2547, 2023-09-13T04:51:06.720 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2023-09-13T04:51:06.720 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2547, 2023-09-13T04:51:06.720 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 24.198610220, 2023-09-13T04:51:06.720 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.009500828 2023-09-13T04:51:06.720 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.720 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2023-09-13T04:51:06.721 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2023-09-13T04:51:06.721 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2547, 2023-09-13T04:51:06.721 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 2548 2023-09-13T04:51:06.721 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.721 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 2547, 2023-09-13T04:51:06.722 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 2547, 2023-09-13T04:51:06.722 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 0, 2023-09-13T04:51:06.722 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2023-09-13T04:51:06.722 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2023-09-13T04:51:06.722 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 1502, 2023-09-13T04:51:06.723 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 0, 2023-09-13T04:51:06.723 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 1043, 2023-09-13T04:51:06.723 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 0, 2023-09-13T04:51:06.723 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 0, 2023-09-13T04:51:06.723 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2023-09-13T04:51:06.723 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2023-09-13T04:51:06.724 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2023-09-13T04:51:06.724 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 0, 2023-09-13T04:51:06.724 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 0, 2023-09-13T04:51:06.724 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2023-09-13T04:51:06.724 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2023-09-13T04:51:06.725 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2023-09-13T04:51:06.725 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 1, 2023-09-13T04:51:06.725 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 0, 2023-09-13T04:51:06.725 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2023-09-13T04:51:06.725 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2023-09-13T04:51:06.726 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2023-09-13T04:51:06.726 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2023-09-13T04:51:06.726 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2023-09-13T04:51:06.726 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2023-09-13T04:51:06.726 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2023-09-13T04:51:06.726 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2023-09-13T04:51:06.727 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2023-09-13T04:51:06.727 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2023-09-13T04:51:06.727 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2023-09-13T04:51:06.727 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2023-09-13T04:51:06.727 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2023-09-13T04:51:06.727 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2023-09-13T04:51:06.728 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2023-09-13T04:51:06.728 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2023-09-13T04:51:06.728 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2023-09-13T04:51:06.728 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2023-09-13T04:51:06.728 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2023-09-13T04:51:06.729 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2023-09-13T04:51:06.729 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2023-09-13T04:51:06.729 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2023-09-13T04:51:06.729 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2023-09-13T04:51:06.729 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2023-09-13T04:51:06.729 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2023-09-13T04:51:06.730 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2023-09-13T04:51:06.730 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 104, 2023-09-13T04:51:06.730 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2023-09-13T04:51:06.730 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 14, 2023-09-13T04:51:06.730 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 9, 2023-09-13T04:51:06.730 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 9, 2023-09-13T04:51:06.731 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2023-09-13T04:51:06.731 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2023-09-13T04:51:06.731 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 0, 2023-09-13T04:51:06.731 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 2, 2023-09-13T04:51:06.731 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 0 2023-09-13T04:51:06.732 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.732 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2023-09-13T04:51:06.732 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 0, 2023-09-13T04:51:06.732 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2023-09-13T04:51:06.732 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2023-09-13T04:51:06.732 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2023-09-13T04:51:06.733 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.733 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2023-09-13T04:51:06.733 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2023-09-13T04:51:06.733 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.733 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2023-09-13T04:51:06.733 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2023-09-13T04:51:06.734 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 0, 2023-09-13T04:51:06.734 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2023-09-13T04:51:06.734 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 0, 2023-09-13T04:51:06.734 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 0, 2023-09-13T04:51:06.734 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2023-09-13T04:51:06.735 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.735 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-13T04:51:06.735 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-13T04:51:06.735 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-09-13T04:51:06.735 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-13T04:51:06.736 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 2811, 2023-09-13T04:51:06.736 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 155471560, 2023-09-13T04:51:06.736 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-13T04:51:06.736 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 2811, 2023-09-13T04:51:06.736 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-09-13T04:51:06.736 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-09-13T04:51:06.737 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 2811, 2023-09-13T04:51:06.737 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 155471560, 2023-09-13T04:51:06.737 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-13T04:51:06.737 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.737 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.737 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.738 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:06.738 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.738 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2023-09-13T04:51:06.738 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-13T04:51:06.738 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-09-13T04:51:06.739 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-13T04:51:06.739 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-09-13T04:51:06.739 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-09-13T04:51:06.739 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-13T04:51:06.739 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-09-13T04:51:06.739 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 2547, 2023-09-13T04:51:06.740 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 153519813, 2023-09-13T04:51:06.740 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 2547, 2023-09-13T04:51:06.740 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 153519813, 2023-09-13T04:51:06.740 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-13T04:51:06.740 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.741 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.741 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.741 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:06.741 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.741 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2023-09-13T04:51:06.741 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-13T04:51:06.742 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2023-09-13T04:51:06.742 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-13T04:51:06.742 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-09-13T04:51:06.742 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-09-13T04:51:06.742 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-13T04:51:06.743 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-09-13T04:51:06.743 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 2547, 2023-09-13T04:51:06.743 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 2547, 2023-09-13T04:51:06.743 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 2547, 2023-09-13T04:51:06.743 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 2547, 2023-09-13T04:51:06.743 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-13T04:51:06.744 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.744 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.744 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.744 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:06.744 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.745 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2023-09-13T04:51:06.745 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-13T04:51:06.745 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-09-13T04:51:06.745 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-13T04:51:06.745 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-09-13T04:51:06.745 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-09-13T04:51:06.746 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-13T04:51:06.746 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-09-13T04:51:06.746 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-09-13T04:51:06.746 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-09-13T04:51:06.746 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2023-09-13T04:51:06.746 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2023-09-13T04:51:06.747 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-13T04:51:06.747 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.747 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.747 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.747 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:06.747 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:06.748 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x55b22dff2960": { 2023-09-13T04:51:06.748 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-13T04:51:06.748 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-09-13T04:51:06.748 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-13T04:51:06.748 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-09-13T04:51:06.748 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-09-13T04:51:06.749 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-13T04:51:06.749 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-09-13T04:51:06.749 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-09-13T04:51:06.749 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-09-13T04:51:06.749 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2023-09-13T04:51:06.749 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2023-09-13T04:51:06.750 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-13T04:51:06.750 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:06.750 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:06.750 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:06.750 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:06.751 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:06.751 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-09-13T04:51:07.026 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-09-13T04:51:07.026 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-09-13T04:51:07.026 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2023-09-13T04:51:07.026 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2023-09-13T04:51:07.027 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2023-09-13T04:51:07.027 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2023-09-13T04:51:07.027 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2023-09-13T04:51:07.027 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2023-09-13T04:51:07.028 DEBUG:tasks.check_counter:Getting stats from i 2023-09-13T04:51:07.028 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph daemon mds.i perf dump 2023-09-13T04:51:07.582 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-13T04:51:08.087 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:07 smithi156 ceph-mon[133675]: pgmap v333: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-09-13T04:51:08.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:07 smithi002 ceph-mon[97651]: pgmap v333: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-09-13T04:51:08.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:07 smithi149 ceph-mon[133315]: pgmap v333: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-09-13T04:51:08.951 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:08 smithi156 ceph-mon[133675]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:51:08.951 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:08 smithi156 ceph-mon[133675]: osdmap e105: 12 total, 12 up, 12 in 2023-09-13T04:51:09.025 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-09-13T04:51:09.025 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2023-09-13T04:51:09.025 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 2627276, 2023-09-13T04:51:09.025 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 2626119, 2023-09-13T04:51:09.026 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 422615188, 2023-09-13T04:51:09.026 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 471089668, 2023-09-13T04:51:09.026 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 13, 2023-09-13T04:51:09.026 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 5, 2023-09-13T04:51:09.027 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 49.130752738, 2023-09-13T04:51:09.027 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 20.310649704, 2023-09-13T04:51:09.027 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 37.913123766, 2023-09-13T04:51:09.027 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.206594836, 2023-09-13T04:51:09.027 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:51:09.028 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2626117, 2023-09-13T04:51:09.028 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 47.034956481, 2023-09-13T04:51:09.028 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000017910 2023-09-13T04:51:09.028 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.028 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:51:09.028 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 4069731, 2023-09-13T04:51:09.029 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.976743374, 2023-09-13T04:51:09.029 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000240 2023-09-13T04:51:09.029 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.029 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 160, 2023-09-13T04:51:09.029 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 315088 2023-09-13T04:51:09.030 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.030 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2023-09-13T04:51:09.030 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 150157, 2023-09-13T04:51:09.030 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 195089, 2023-09-13T04:51:09.030 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 50621902, 2023-09-13T04:51:09.031 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 143250722, 2023-09-13T04:51:09.031 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 19, 2023-09-13T04:51:09.031 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 4, 2023-09-13T04:51:09.031 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 5.117534432, 2023-09-13T04:51:09.031 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 2.478516553, 2023-09-13T04:51:09.032 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 3.951086973, 2023-09-13T04:51:09.032 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.212590053, 2023-09-13T04:51:09.032 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:51:09.032 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 195088, 2023-09-13T04:51:09.032 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 14.181498214, 2023-09-13T04:51:09.033 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000072692 2023-09-13T04:51:09.033 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.033 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:51:09.033 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 68831, 2023-09-13T04:51:09.033 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.025876259, 2023-09-13T04:51:09.033 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000375 2023-09-13T04:51:09.034 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.034 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1527728, 2023-09-13T04:51:09.034 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 143008 2023-09-13T04:51:09.034 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.034 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2023-09-13T04:51:09.035 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 151373, 2023-09-13T04:51:09.035 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 140303, 2023-09-13T04:51:09.035 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 54979591, 2023-09-13T04:51:09.035 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 87995666, 2023-09-13T04:51:09.035 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 12, 2023-09-13T04:51:09.036 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 6, 2023-09-13T04:51:09.036 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 5.201668043, 2023-09-13T04:51:09.036 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 2.015262687, 2023-09-13T04:51:09.036 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 4.137269946, 2023-09-13T04:51:09.036 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.242794372, 2023-09-13T04:51:09.036 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:51:09.037 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 140301, 2023-09-13T04:51:09.037 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 4.428388113, 2023-09-13T04:51:09.037 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000031563 2023-09-13T04:51:09.037 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.037 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:51:09.037 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 246644, 2023-09-13T04:51:09.038 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.029340976, 2023-09-13T04:51:09.038 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000118 2023-09-13T04:51:09.038 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.038 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 912, 2023-09-13T04:51:09.039 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 126816 2023-09-13T04:51:09.039 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.039 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2023-09-13T04:51:09.039 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2023-09-13T04:51:09.039 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2023-09-13T04:51:09.039 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.040 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2023-09-13T04:51:09.040 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-09-13T04:51:09.040 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-09-13T04:51:09.040 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 6424, 2023-09-13T04:51:09.040 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 27.439259077, 2023-09-13T04:51:09.040 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.004271366 2023-09-13T04:51:09.041 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:09.041 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.041 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2023-09-13T04:51:09.041 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-09-13T04:51:09.041 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-09-13T04:51:09.042 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 12622, 2023-09-13T04:51:09.042 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 2.837911965, 2023-09-13T04:51:09.042 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000224838 2023-09-13T04:51:09.042 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:09.042 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.043 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2023-09-13T04:51:09.043 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 73436, 2023-09-13T04:51:09.043 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 69184, 2023-09-13T04:51:09.043 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2023-09-13T04:51:09.043 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 69184, 2023-09-13T04:51:09.044 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 1808.035059351, 2023-09-13T04:51:09.044 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.026133716 2023-09-13T04:51:09.044 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.044 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2023-09-13T04:51:09.044 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 4248, 2023-09-13T04:51:09.044 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 22, 2023-09-13T04:51:09.045 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2023-09-13T04:51:09.045 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 344, 2023-09-13T04:51:09.045 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 37, 2023-09-13T04:51:09.045 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2023-09-13T04:51:09.045 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 13215, 2023-09-13T04:51:09.046 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 65, 2023-09-13T04:51:09.046 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 29, 2023-09-13T04:51:09.046 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 13121, 2023-09-13T04:51:09.046 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 13121, 2023-09-13T04:51:09.046 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 25619, 2023-09-13T04:51:09.047 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 0, 2023-09-13T04:51:09.047 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 0, 2023-09-13T04:51:09.047 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 3, 2023-09-13T04:51:09.047 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 136830, 2023-09-13T04:51:09.047 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 107655, 2023-09-13T04:51:09.047 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 4068, 2023-09-13T04:51:09.048 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 758, 2023-09-13T04:51:09.048 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 8, 2023-09-13T04:51:09.048 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2023-09-13T04:51:09.048 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 1088, 2023-09-13T04:51:09.048 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2023-09-13T04:51:09.048 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2023-09-13T04:51:09.049 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 141, 2023-09-13T04:51:09.049 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 19020, 2023-09-13T04:51:09.049 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 120, 2023-09-13T04:51:09.049 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 19257, 2023-09-13T04:51:09.049 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2023-09-13T04:51:09.050 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 2189, 2023-09-13T04:51:09.050 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2023-09-13T04:51:09.050 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 0, 2023-09-13T04:51:09.050 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 0, 2023-09-13T04:51:09.050 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 6, 2023-09-13T04:51:09.051 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 2189, 2023-09-13T04:51:09.051 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2023-09-13T04:51:09.051 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2023-09-13T04:51:09.051 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2023-09-13T04:51:09.053 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 13, 2023-09-13T04:51:09.053 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2023-09-13T04:51:09.053 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 13, 2023-09-13T04:51:09.054 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 2176, 2023-09-13T04:51:09.054 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 128, 2023-09-13T04:51:09.054 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 7022, 2023-09-13T04:51:09.054 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 14529, 2023-09-13T04:51:09.055 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2023-09-13T04:51:09.055 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 2, 2023-09-13T04:51:09.055 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-13T04:51:09.055 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 20894, 2023-09-13T04:51:09.055 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 7358, 2023-09-13T04:51:09.055 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 373, 2023-09-13T04:51:09.056 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 17970 2023-09-13T04:51:09.056 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.056 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2023-09-13T04:51:09.056 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 790, 2023-09-13T04:51:09.056 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 4672, 2023-09-13T04:51:09.056 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 3119, 2023-09-13T04:51:09.057 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 4343, 2023-09-13T04:51:09.057 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 1575, 2023-09-13T04:51:09.057 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 36, 2023-09-13T04:51:09.057 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2023-09-13T04:51:09.057 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2023-09-13T04:51:09.058 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 6771, 2023-09-13T04:51:09.058 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 6735, 2023-09-13T04:51:09.058 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2023-09-13T04:51:09.058 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2023-09-13T04:51:09.058 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2023-09-13T04:51:09.058 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2023-09-13T04:51:09.059 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2023-09-13T04:51:09.059 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2023-09-13T04:51:09.059 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2023-09-13T04:51:09.059 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2023-09-13T04:51:09.059 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 141, 2023-09-13T04:51:09.060 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2023-09-13T04:51:09.060 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 37, 2023-09-13T04:51:09.060 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2023-09-13T04:51:09.060 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2023-09-13T04:51:09.060 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.060 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2023-09-13T04:51:09.061 INFO:teuthology.orchestra.run.smithi156.stdout: "evlrg": 6, 2023-09-13T04:51:09.061 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 48505, 2023-09-13T04:51:09.061 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2023-09-13T04:51:09.061 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2023-09-13T04:51:09.061 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 48505, 2023-09-13T04:51:09.061 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2023-09-13T04:51:09.062 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2023-09-13T04:51:09.062 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 72, 2023-09-13T04:51:09.062 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2023-09-13T04:51:09.062 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2023-09-13T04:51:09.062 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 72, 2023-09-13T04:51:09.063 INFO:teuthology.orchestra.run.smithi156.stdout: "segmjr": 5, 2023-09-13T04:51:09.063 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2023-09-13T04:51:09.063 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2023-09-13T04:51:09.063 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 4194304, 2023-09-13T04:51:09.063 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 157661155, 2023-09-13T04:51:09.064 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 0, 2023-09-13T04:51:09.064 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2023-09-13T04:51:09.064 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2174, 2023-09-13T04:51:09.064 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 142.681246642, 2023-09-13T04:51:09.064 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.065630748 2023-09-13T04:51:09.064 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.065 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 0 2023-09-13T04:51:09.065 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.065 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2023-09-13T04:51:09.065 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 5950, 2023-09-13T04:51:09.065 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 26692, 2023-09-13T04:51:09.065 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 20742, 2023-09-13T04:51:09.066 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 263, 2023-09-13T04:51:09.066 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 2128, 2023-09-13T04:51:09.066 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 1865, 2023-09-13T04:51:09.066 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 13215, 2023-09-13T04:51:09.066 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 46320, 2023-09-13T04:51:09.067 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 33105, 2023-09-13T04:51:09.067 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 0, 2023-09-13T04:51:09.067 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 18618, 2023-09-13T04:51:09.067 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 18618, 2023-09-13T04:51:09.067 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 159816, 2023-09-13T04:51:09.067 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 198960 2023-09-13T04:51:09.068 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.068 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2023-09-13T04:51:09.068 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 83233, 2023-09-13T04:51:09.068 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 3499, 2023-09-13T04:51:09.068 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 73436, 2023-09-13T04:51:09.068 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 308, 2023-09-13T04:51:09.069 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 4996, 2023-09-13T04:51:09.069 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2023-09-13T04:51:09.069 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 6892, 2023-09-13T04:51:09.069 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 23.971160845, 2023-09-13T04:51:09.069 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.003478113 2023-09-13T04:51:09.069 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.070 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2023-09-13T04:51:09.070 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 84, 2023-09-13T04:51:09.070 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 18.496770862, 2023-09-13T04:51:09.070 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.220199653 2023-09-13T04:51:09.070 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.071 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2023-09-13T04:51:09.071 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.071 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.071 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.071 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.072 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2023-09-13T04:51:09.072 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.072 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.072 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.072 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.073 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2023-09-13T04:51:09.073 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 47503, 2023-09-13T04:51:09.073 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 1473.756880220, 2023-09-13T04:51:09.073 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.031024501 2023-09-13T04:51:09.073 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.073 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2023-09-13T04:51:09.074 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.074 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.074 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.074 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.074 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2023-09-13T04:51:09.074 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.075 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.075 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.075 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.075 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2023-09-13T04:51:09.075 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.076 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.076 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.076 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.076 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2023-09-13T04:51:09.076 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.076 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.077 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.077 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.077 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2023-09-13T04:51:09.077 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.077 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.078 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.078 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.078 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2023-09-13T04:51:09.078 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.078 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.079 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.079 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.079 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2023-09-13T04:51:09.079 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 27, 2023-09-13T04:51:09.079 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 2.692716149, 2023-09-13T04:51:09.079 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.099730227 2023-09-13T04:51:09.080 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.080 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2023-09-13T04:51:09.080 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.080 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.080 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.080 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.081 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2023-09-13T04:51:09.081 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.081 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.081 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.081 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.082 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2023-09-13T04:51:09.082 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2402, 2023-09-13T04:51:09.082 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 3.633255113, 2023-09-13T04:51:09.082 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001512595 2023-09-13T04:51:09.082 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.083 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2023-09-13T04:51:09.083 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 216, 2023-09-13T04:51:09.083 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.521017529, 2023-09-13T04:51:09.083 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.002412118 2023-09-13T04:51:09.083 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.083 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2023-09-13T04:51:09.084 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 6867, 2023-09-13T04:51:09.084 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 262.152440830, 2023-09-13T04:51:09.084 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.038175686 2023-09-13T04:51:09.084 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.084 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2023-09-13T04:51:09.084 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.085 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.085 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.085 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.085 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2023-09-13T04:51:09.085 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.086 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.086 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.086 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.086 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2023-09-13T04:51:09.086 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 36, 2023-09-13T04:51:09.086 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 13.199675928, 2023-09-13T04:51:09.087 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.366657664 2023-09-13T04:51:09.087 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.087 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2023-09-13T04:51:09.087 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.087 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.087 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.088 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.088 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2023-09-13T04:51:09.088 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.088 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.088 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.089 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.089 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2023-09-13T04:51:09.089 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.089 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.089 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.089 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.090 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2023-09-13T04:51:09.090 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.090 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.090 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.090 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.091 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2023-09-13T04:51:09.091 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.091 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.091 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.091 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.092 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2023-09-13T04:51:09.092 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.092 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.092 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.092 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.092 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2023-09-13T04:51:09.093 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.093 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.093 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.093 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.093 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2023-09-13T04:51:09.093 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.094 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.094 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.094 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.094 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2023-09-13T04:51:09.094 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 5157, 2023-09-13T04:51:09.095 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 9.611141875, 2023-09-13T04:51:09.095 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001863707 2023-09-13T04:51:09.095 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.095 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2023-09-13T04:51:09.095 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2023-09-13T04:51:09.095 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2023-09-13T04:51:09.096 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.096 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.096 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.096 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:09.096 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.097 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2023-09-13T04:51:09.097 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 1, 2023-09-13T04:51:09.097 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 3, 2023-09-13T04:51:09.097 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 2, 2023-09-13T04:51:09.097 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 1, 2023-09-13T04:51:09.097 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2023-09-13T04:51:09.098 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 12205, 2023-09-13T04:51:09.098 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 12205, 2023-09-13T04:51:09.098 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 564, 2023-09-13T04:51:09.098 INFO:teuthology.orchestra.run.smithi156.stdout: "mdthresh_evicted": 0 2023-09-13T04:51:09.098 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.098 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2023-09-13T04:51:09.099 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2023-09-13T04:51:09.099 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2023-09-13T04:51:09.099 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2023-09-13T04:51:09.099 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2023-09-13T04:51:09.099 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2023-09-13T04:51:09.100 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2023-09-13T04:51:09.100 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-13T04:51:09.100 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2023-09-13T04:51:09.100 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-13T04:51:09.100 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2023-09-13T04:51:09.100 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2023-09-13T04:51:09.101 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2023-09-13T04:51:09.101 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-13T04:51:09.101 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2023-09-13T04:51:09.101 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2023-09-13T04:51:09.101 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2023-09-13T04:51:09.102 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2023-09-13T04:51:09.102 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2023-09-13T04:51:09.102 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-13T04:51:09.102 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2023-09-13T04:51:09.102 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-13T04:51:09.103 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2023-09-13T04:51:09.103 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2023-09-13T04:51:09.103 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2023-09-13T04:51:09.103 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2023-09-13T04:51:09.103 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2023-09-13T04:51:09.104 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-13T04:51:09.104 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2023-09-13T04:51:09.104 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2023-09-13T04:51:09.104 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2023-09-13T04:51:09.104 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2023-09-13T04:51:09.105 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2023-09-13T04:51:09.105 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2023-09-13T04:51:09.105 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2023-09-13T04:51:09.105 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2023-09-13T04:51:09.105 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2023-09-13T04:51:09.106 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 793054, 2023-09-13T04:51:09.106 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 5965, 2023-09-13T04:51:09.106 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2023-09-13T04:51:09.106 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2023-09-13T04:51:09.106 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2023-09-13T04:51:09.107 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2023-09-13T04:51:09.107 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2023-09-13T04:51:09.107 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2023-09-13T04:51:09.107 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2023-09-13T04:51:09.107 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2023-09-13T04:51:09.108 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 9416, 2023-09-13T04:51:09.108 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 180, 2023-09-13T04:51:09.108 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2023-09-13T04:51:09.108 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2023-09-13T04:51:09.108 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2023-09-13T04:51:09.108 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2023-09-13T04:51:09.109 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 23776683, 2023-09-13T04:51:09.109 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 270842, 2023-09-13T04:51:09.109 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2023-09-13T04:51:09.109 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2023-09-13T04:51:09.109 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2023-09-13T04:51:09.110 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2023-09-13T04:51:09.110 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.110 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2023-09-13T04:51:09.110 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2023-09-13T04:51:09.110 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2023-09-13T04:51:09.111 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 19700, 2023-09-13T04:51:09.111 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 161952101, 2023-09-13T04:51:09.111 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 29, 2023-09-13T04:51:09.111 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 19700, 2023-09-13T04:51:09.111 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2023-09-13T04:51:09.112 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 19700, 2023-09-13T04:51:09.112 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 453.272687209, 2023-09-13T04:51:09.112 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.023008765 2023-09-13T04:51:09.112 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.112 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2023-09-13T04:51:09.112 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2023-09-13T04:51:09.113 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 19700, 2023-09-13T04:51:09.113 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 20261 2023-09-13T04:51:09.113 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.113 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 19700, 2023-09-13T04:51:09.113 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 2747, 2023-09-13T04:51:09.114 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 16953, 2023-09-13T04:51:09.114 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2023-09-13T04:51:09.114 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2023-09-13T04:51:09.114 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 0, 2023-09-13T04:51:09.114 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 10, 2023-09-13T04:51:09.115 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 498, 2023-09-13T04:51:09.115 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 2673, 2023-09-13T04:51:09.115 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 291, 2023-09-13T04:51:09.115 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2023-09-13T04:51:09.115 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2023-09-13T04:51:09.115 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2023-09-13T04:51:09.116 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 37, 2023-09-13T04:51:09.116 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 13518, 2023-09-13T04:51:09.116 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2023-09-13T04:51:09.116 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2023-09-13T04:51:09.116 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2023-09-13T04:51:09.116 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 2189, 2023-09-13T04:51:09.117 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 20, 2023-09-13T04:51:09.117 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2023-09-13T04:51:09.117 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2023-09-13T04:51:09.117 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2023-09-13T04:51:09.117 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2023-09-13T04:51:09.118 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2023-09-13T04:51:09.118 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2023-09-13T04:51:09.118 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2023-09-13T04:51:09.118 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2023-09-13T04:51:09.118 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2023-09-13T04:51:09.119 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2023-09-13T04:51:09.119 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2023-09-13T04:51:09.119 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2023-09-13T04:51:09.119 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2023-09-13T04:51:09.119 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2023-09-13T04:51:09.119 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2023-09-13T04:51:09.120 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2023-09-13T04:51:09.120 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2023-09-13T04:51:09.120 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2023-09-13T04:51:09.120 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2023-09-13T04:51:09.120 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2023-09-13T04:51:09.121 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2023-09-13T04:51:09.121 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2023-09-13T04:51:09.121 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2023-09-13T04:51:09.121 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2023-09-13T04:51:09.121 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2023-09-13T04:51:09.122 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2023-09-13T04:51:09.122 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 105, 2023-09-13T04:51:09.122 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2023-09-13T04:51:09.122 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 15, 2023-09-13T04:51:09.122 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 12, 2023-09-13T04:51:09.123 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 12, 2023-09-13T04:51:09.123 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2023-09-13T04:51:09.123 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2023-09-13T04:51:09.123 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 832, 2023-09-13T04:51:09.123 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 82, 2023-09-13T04:51:09.123 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 111 2023-09-13T04:51:09.124 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.124 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2023-09-13T04:51:09.124 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 1, 2023-09-13T04:51:09.124 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2023-09-13T04:51:09.124 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2023-09-13T04:51:09.125 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2023-09-13T04:51:09.125 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.125 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2023-09-13T04:51:09.125 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 23 2023-09-13T04:51:09.125 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.125 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2023-09-13T04:51:09.126 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2023-09-13T04:51:09.126 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 16, 2023-09-13T04:51:09.126 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2023-09-13T04:51:09.126 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 8, 2023-09-13T04:51:09.126 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 6735, 2023-09-13T04:51:09.127 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2023-09-13T04:51:09.127 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.127 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-13T04:51:09.127 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-13T04:51:09.127 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-09-13T04:51:09.128 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-13T04:51:09.128 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 2928802, 2023-09-13T04:51:09.128 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 266903470, 2023-09-13T04:51:09.128 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-13T04:51:09.128 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 2928802, 2023-09-13T04:51:09.129 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-09-13T04:51:09.129 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-09-13T04:51:09.129 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 2928802, 2023-09-13T04:51:09.129 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 266903470, 2023-09-13T04:51:09.129 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-13T04:51:09.129 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.130 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.130 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.130 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:09.130 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.130 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2023-09-13T04:51:09.131 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-13T04:51:09.131 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-09-13T04:51:09.131 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-13T04:51:09.131 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-09-13T04:51:09.131 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-09-13T04:51:09.132 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-13T04:51:09.132 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-09-13T04:51:09.132 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 19700, 2023-09-13T04:51:09.132 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 162826800, 2023-09-13T04:51:09.132 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 19700, 2023-09-13T04:51:09.132 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 162826800, 2023-09-13T04:51:09.133 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-13T04:51:09.133 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.133 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.133 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.133 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:09.133 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.134 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2023-09-13T04:51:09.134 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-13T04:51:09.134 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2023-09-13T04:51:09.134 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-13T04:51:09.134 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-09-13T04:51:09.135 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-09-13T04:51:09.135 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-13T04:51:09.135 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-09-13T04:51:09.135 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 19700, 2023-09-13T04:51:09.135 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 19700, 2023-09-13T04:51:09.135 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 19700, 2023-09-13T04:51:09.136 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 19700, 2023-09-13T04:51:09.136 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-13T04:51:09.136 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.136 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.136 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.136 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:09.137 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.137 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2023-09-13T04:51:09.137 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-13T04:51:09.137 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-09-13T04:51:09.137 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-13T04:51:09.138 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 6735, 2023-09-13T04:51:09.138 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 876339, 2023-09-13T04:51:09.138 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-13T04:51:09.138 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 6735, 2023-09-13T04:51:09.138 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-09-13T04:51:09.139 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-09-13T04:51:09.139 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 499, 2023-09-13T04:51:09.139 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 876339, 2023-09-13T04:51:09.139 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-13T04:51:09.139 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.139 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.140 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.140 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:09.140 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:09.140 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x55fafead86e0": { 2023-09-13T04:51:09.140 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-13T04:51:09.141 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-09-13T04:51:09.141 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-13T04:51:09.141 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 48505, 2023-09-13T04:51:09.141 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 153466851, 2023-09-13T04:51:09.141 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-13T04:51:09.142 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 48505, 2023-09-13T04:51:09.142 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-09-13T04:51:09.142 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-09-13T04:51:09.142 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 2174, 2023-09-13T04:51:09.142 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 153466851, 2023-09-13T04:51:09.142 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-13T04:51:09.143 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:09.143 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:09.143 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:09.143 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:09.143 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:09.143 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-09-13T04:51:09.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:08 smithi002 ceph-mon[97651]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:51:09.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:08 smithi002 ceph-mon[97651]: osdmap e105: 12 total, 12 up, 12 in 2023-09-13T04:51:09.416 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:08 smithi149 ceph-mon[133315]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:51:09.416 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:08 smithi149 ceph-mon[133315]: osdmap e105: 12 total, 12 up, 12 in 2023-09-13T04:51:09.536 INFO:tasks.check_counter:Daemon mds.i mds.exported=141 2023-09-13T04:51:09.536 INFO:tasks.check_counter:Daemon mds.i mds.imported=120 2023-09-13T04:51:09.537 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=790 2023-09-13T04:51:09.537 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=4672 2023-09-13T04:51:09.537 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2023-09-13T04:51:09.537 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2023-09-13T04:51:09.538 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2023-09-13T04:51:09.538 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=37 2023-09-13T04:51:09.538 DEBUG:tasks.check_counter:Getting stats from l 2023-09-13T04:51:09.539 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4a754ac981aa956c0a6fd67e5140c9a1c7d53ab5 shell --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da -- ceph daemon mds.l perf dump 2023-09-13T04:51:10.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:09 smithi002 ceph-mon[97651]: pgmap v335: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-09-13T04:51:10.315 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-13T04:51:10.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:09 smithi149 ceph-mon[133315]: pgmap v335: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-09-13T04:51:10.441 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:09 smithi156 ceph-mon[133675]: pgmap v335: 129 pgs: 129 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-09-13T04:51:11.749 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-09-13T04:51:11.750 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2023-09-13T04:51:11.750 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 236, 2023-09-13T04:51:11.750 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 224, 2023-09-13T04:51:11.751 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 1422992, 2023-09-13T04:51:11.751 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 133488, 2023-09-13T04:51:11.751 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 4, 2023-09-13T04:51:11.751 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 0, 2023-09-13T04:51:11.751 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.041431903, 2023-09-13T04:51:11.752 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.012067428, 2023-09-13T04:51:11.752 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.030693990, 2023-09-13T04:51:11.752 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.005637834, 2023-09-13T04:51:11.752 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:51:11.752 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 224, 2023-09-13T04:51:11.753 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.013836089, 2023-09-13T04:51:11.753 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000061768 2023-09-13T04:51:11.753 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:11.753 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:51:11.753 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:11.754 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:11.754 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:11.754 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:11.754 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1422992, 2023-09-13T04:51:11.754 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 133488 2023-09-13T04:51:11.755 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:11.755 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2023-09-13T04:51:11.755 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 2, 2023-09-13T04:51:11.755 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 169, 2023-09-13T04:51:11.755 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 320, 2023-09-13T04:51:11.756 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 182304, 2023-09-13T04:51:11.756 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 5, 2023-09-13T04:51:11.756 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 3, 2023-09-13T04:51:11.756 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.009543540, 2023-09-13T04:51:11.756 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.005883831, 2023-09-13T04:51:11.757 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.001640967, 2023-09-13T04:51:11.757 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-13T04:51:11.757 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:51:11.757 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 166, 2023-09-13T04:51:11.757 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.021952908, 2023-09-13T04:51:11.757 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000132246 2023-09-13T04:51:11.758 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:11.758 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:51:11.758 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:11.758 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:11.758 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:11.759 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:11.759 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-13T04:51:11.759 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 182304 2023-09-13T04:51:11.759 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:11.759 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2023-09-13T04:51:11.760 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 0, 2023-09-13T04:51:11.760 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 2, 2023-09-13T04:51:11.760 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 0, 2023-09-13T04:51:11.760 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 0, 2023-09-13T04:51:11.760 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 2, 2023-09-13T04:51:11.761 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 1, 2023-09-13T04:51:11.761 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.000274090, 2023-09-13T04:51:11.761 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.000000000, 2023-09-13T04:51:11.761 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.000209989, 2023-09-13T04:51:11.761 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-13T04:51:11.762 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T04:51:11.762 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:11.762 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:11.762 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:11.763 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:11.763 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-09-13T04:51:11.763 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:11.763 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:11.763 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:11.763 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:11.764 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-13T04:51:11.764 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 0 2023-09-13T04:51:11.764 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:11.764 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2023-09-13T04:51:11.764 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 0, 2023-09-13T04:51:11.765 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2023-09-13T04:51:11.765 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:11.765 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2023-09-13T04:51:11.765 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2023-09-13T04:51:11.765 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2023-09-13T04:51:11.765 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2023-09-13T04:51:11.766 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2023-09-13T04:51:11.766 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2023-09-13T04:51:11.766 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2023-09-13T04:51:11.766 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-13T04:51:11.766 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2023-09-13T04:51:11.767 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-13T04:51:11.767 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2023-09-13T04:51:11.767 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2023-09-13T04:51:11.767 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2023-09-13T04:51:11.767 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-13T04:51:11.768 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2023-09-13T04:51:11.768 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2023-09-13T04:51:11.768 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2023-09-13T04:51:11.768 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2023-09-13T04:51:11.768 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2023-09-13T04:51:11.768 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-13T04:51:11.769 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2023-09-13T04:51:11.769 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-13T04:51:11.769 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2023-09-13T04:51:11.769 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2023-09-13T04:51:11.769 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2023-09-13T04:51:11.770 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2023-09-13T04:51:11.770 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2023-09-13T04:51:11.770 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-13T04:51:11.770 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2023-09-13T04:51:11.770 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2023-09-13T04:51:11.771 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2023-09-13T04:51:11.771 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2023-09-13T04:51:11.771 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2023-09-13T04:51:11.771 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2023-09-13T04:51:11.771 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2023-09-13T04:51:11.772 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2023-09-13T04:51:11.772 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2023-09-13T04:51:11.772 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 148714, 2023-09-13T04:51:11.772 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 30, 2023-09-13T04:51:11.772 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2023-09-13T04:51:11.773 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2023-09-13T04:51:11.773 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2023-09-13T04:51:11.773 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2023-09-13T04:51:11.773 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2023-09-13T04:51:11.773 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2023-09-13T04:51:11.773 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2023-09-13T04:51:11.774 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2023-09-13T04:51:11.774 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 0, 2023-09-13T04:51:11.774 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 0, 2023-09-13T04:51:11.774 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2023-09-13T04:51:11.774 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2023-09-13T04:51:11.775 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2023-09-13T04:51:11.775 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2023-09-13T04:51:11.775 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 928, 2023-09-13T04:51:11.775 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 2, 2023-09-13T04:51:11.775 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2023-09-13T04:51:11.776 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2023-09-13T04:51:11.776 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2023-09-13T04:51:11.776 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2023-09-13T04:51:11.776 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-13T04:51:11.776 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-13T04:51:11.776 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-13T04:51:11.777 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-09-13T04:51:11.777 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-13T04:51:11.777 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 234, 2023-09-13T04:51:11.777 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 1387289, 2023-09-13T04:51:11.777 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-13T04:51:11.777 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 234, 2023-09-13T04:51:11.778 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-09-13T04:51:11.778 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-09-13T04:51:11.778 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 234, 2023-09-13T04:51:11.778 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 1387289, 2023-09-13T04:51:11.778 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-13T04:51:11.779 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-13T04:51:11.779 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-13T04:51:11.779 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-13T04:51:11.779 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:11.779 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-13T04:51:11.780 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-09-13T04:51:12.099 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:11 smithi156 ceph-mon[133675]: pgmap v336: 129 pgs: 20 active+clean+snaptrim, 59 active+clean+snaptrim_wait, 50 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-09-13T04:51:12.162 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-09-13T04:51:12.162 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-09-13T04:51:12.162 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2023-09-13T04:51:12.162 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2023-09-13T04:51:12.163 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2023-09-13T04:51:12.163 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2023-09-13T04:51:12.163 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2023-09-13T04:51:12.163 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2023-09-13T04:51:12.164 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-09-13T04:51:12.174 INFO:tasks.kclient:Unmounting kernel clients... 2023-09-13T04:51:12.175 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-09-13T04:51:12.175 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-13T04:51:12.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:11 smithi002 ceph-mon[97651]: pgmap v336: 129 pgs: 20 active+clean+snaptrim, 59 active+clean+snaptrim_wait, 50 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-09-13T04:51:12.290 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-09-13T04:51:12.290 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-13T04:51:12.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:12 smithi149 ceph-mon[133315]: pgmap v336: 129 pgs: 20 active+clean+snaptrim, 59 active+clean+snaptrim_wait, 50 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-09-13T04:51:12.409 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T04:51:12.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.822+0000 7f82675ff700 1 -- 172.21.15.2:0/912725485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82600ffbe0 msgr2=0x7f82600fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:12.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.822+0000 7f82675ff700 1 --2- 172.21.15.2:0/912725485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82600ffbe0 0x7f82600fffc0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8254009f90 tx=0x7f825400b330 comp rx=0 tx=0).stop 2023-09-13T04:51:12.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.828+0000 7f82675ff700 1 -- 172.21.15.2:0/912725485 shutdown_connections 2023-09-13T04:51:12.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.828+0000 7f82675ff700 1 --2- 172.21.15.2:0/912725485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8260104fe0 0x7f8260105490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:12.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.828+0000 7f82675ff700 1 --2- 172.21.15.2:0/912725485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82600ffbe0 0x7f82600fffc0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:12.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.828+0000 7f82675ff700 1 --2- 172.21.15.2:0/912725485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8260100500 0x7f82601049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:12.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.828+0000 7f82675ff700 1 -- 172.21.15.2:0/912725485 >> 172.21.15.2:0/912725485 conn(0x7f82600fb5f0 msgr2=0x7f82600fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:12.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.841+0000 7f82675ff700 1 -- 172.21.15.2:0/912725485 shutdown_connections 2023-09-13T04:51:12.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.842+0000 7f82675ff700 1 -- 172.21.15.2:0/912725485 wait complete. 2023-09-13T04:51:12.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.843+0000 7f82675ff700 1 Processor -- start 2023-09-13T04:51:12.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.845+0000 7f82675ff700 1 -- start start 2023-09-13T04:51:12.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.845+0000 7f82675ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82600ffbe0 0x7f8260113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:12.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.845+0000 7f82675ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8260100500 0x7f8260113fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:12.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.845+0000 7f82675ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8260104fe0 0x7f826010ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:12.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.846+0000 7f82675ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826010d0e0 con 0x7f8260104fe0 2023-09-13T04:51:12.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.846+0000 7f82675ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826010d250 con 0x7f8260100500 2023-09-13T04:51:12.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.846+0000 7f82675ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826010d410 con 0x7f82600ffbe0 2023-09-13T04:51:12.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.846+0000 7f8265b9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8260104fe0 0x7f826010ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:12.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.846+0000 7f8265b9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8260104fe0 0x7f826010ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43404/0 (socket says 172.21.15.2:43404) 2023-09-13T04:51:12.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.846+0000 7f8265b9c700 1 -- 172.21.15.2:0/2987184888 learned_addr learned my addr 172.21.15.2:0/2987184888 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:51:12.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.847+0000 7f8265b9c700 1 -- 172.21.15.2:0/2987184888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82600ffbe0 msgr2=0x7f8260113a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-13T04:51:12.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.847+0000 7f8265b9c700 1 --2- 172.21.15.2:0/2987184888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82600ffbe0 0x7f8260113a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:12.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.847+0000 7f8265b9c700 1 -- 172.21.15.2:0/2987184888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8260100500 msgr2=0x7f8260113fa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-13T04:51:12.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.847+0000 7f8265b9c700 1 --2- 172.21.15.2:0/2987184888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8260100500 0x7f8260113fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:12.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.847+0000 7f8265b9c700 1 -- 172.21.15.2:0/2987184888 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8254009c40 con 0x7f8260104fe0 2023-09-13T04:51:12.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.852+0000 7f8265b9c700 1 --2- 172.21.15.2:0/2987184888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8260104fe0 0x7f826010ca00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f825800e970 tx=0x7f825800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:12.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.853+0000 7f825e7fc700 1 -- 172.21.15.2:0/2987184888 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825800cbd0 con 0x7f8260104fe0 2023-09-13T04:51:12.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.853+0000 7f825e7fc700 1 -- 172.21.15.2:0/2987184888 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f825800ee50 con 0x7f8260104fe0 2023-09-13T04:51:12.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.853+0000 7f825e7fc700 1 -- 172.21.15.2:0/2987184888 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258005610 con 0x7f8260104fe0 2023-09-13T04:51:12.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.856+0000 7f82675ff700 1 -- 172.21.15.2:0/2987184888 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826010d6f0 con 0x7f8260104fe0 2023-09-13T04:51:12.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.857+0000 7f82675ff700 1 -- 172.21.15.2:0/2987184888 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8260111760 con 0x7f8260104fe0 2023-09-13T04:51:12.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.866+0000 7f825e7fc700 1 -- 172.21.15.2:0/2987184888 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f8258004750 con 0x7f8260104fe0 2023-09-13T04:51:12.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.866+0000 7f82675ff700 1 -- 172.21.15.2:0/2987184888 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826010d880 con 0x7f8260104fe0 2023-09-13T04:51:12.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.868+0000 7f825e7fc700 1 --2- 172.21.15.2:0/2987184888 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f82500a09c0 0x7f82500a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:12.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.868+0000 7f825e7fc700 1 -- 172.21.15.2:0/2987184888 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9483+0+0 (secure 0 0 0) 0x7f8258014070 con 0x7f8260104fe0 2023-09-13T04:51:12.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.869+0000 7f826539b700 1 --2- 172.21.15.2:0/2987184888 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f82500a09c0 0x7f82500a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:12.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.880+0000 7f825e7fc700 1 -- 172.21.15.2:0/2987184888 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8258095970 con 0x7f8260104fe0 2023-09-13T04:51:12.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:12.882+0000 7f826539b700 1 --2- 172.21.15.2:0/2987184888 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f82500a09c0 0x7f82500a2e80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8254009550 tx=0x7f8254007be0 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:13.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.300+0000 7f82675ff700 1 -- 172.21.15.2:0/2987184888 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8260067ad0 con 0x7f8260104fe0 2023-09-13T04:51:13.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.304+0000 7f825e7fc700 1 -- 172.21.15.2:0/2987184888 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+25195 (secure 0 0 0) 0x7f8258098f90 con 0x7f8260104fe0 2023-09-13T04:51:13.305 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:51:13.306 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":105,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:51:07.935860+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T04:37:57.094802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T04:37:58.564487+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-13T04:38:02.096895+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1461251021":"2023-09-14T04:40:27.774561+0000","172.21.15.2:6800/1461251021":"2023-09-14T04:40:27.774561+0000","172.21.15.2:0/2528007246":"2023-09-14T04:40:27.774561+0000","172.21.15.2:0/2955382590":"2023-09-14T04:40:27.774561+0000","172.21.15.2:0/227933483":"2023-09-14T04:40:27.774561+0000","172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3568881483":"2023-09-14T04:40:27.774561+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:51:13.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.313+0000 7f82675ff700 1 -- 172.21.15.2:0/2987184888 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f82500a09c0 msgr2=0x7f82500a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:13.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.313+0000 7f82675ff700 1 --2- 172.21.15.2:0/2987184888 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f82500a09c0 0x7f82500a2e80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8254009550 tx=0x7f8254007be0 comp rx=0 tx=0).stop 2023-09-13T04:51:13.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.313+0000 7f82675ff700 1 -- 172.21.15.2:0/2987184888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8260104fe0 msgr2=0x7f826010ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:13.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.313+0000 7f82675ff700 1 --2- 172.21.15.2:0/2987184888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8260104fe0 0x7f826010ca00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f825800e970 tx=0x7f825800ed30 comp rx=0 tx=0).stop 2023-09-13T04:51:13.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.324+0000 7f82675ff700 1 -- 172.21.15.2:0/2987184888 shutdown_connections 2023-09-13T04:51:13.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.324+0000 7f82675ff700 1 --2- 172.21.15.2:0/2987184888 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f82500a09c0 0x7f82500a2e80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:13.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.324+0000 7f82675ff700 1 --2- 172.21.15.2:0/2987184888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8260104fe0 0x7f826010ca00 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:13.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.324+0000 7f82675ff700 1 --2- 172.21.15.2:0/2987184888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82600ffbe0 0x7f8260113a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:13.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.324+0000 7f82675ff700 1 --2- 172.21.15.2:0/2987184888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8260100500 0x7f8260113fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:13.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.324+0000 7f82675ff700 1 -- 172.21.15.2:0/2987184888 >> 172.21.15.2:0/2987184888 conn(0x7f82600fb5f0 msgr2=0x7f8260102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:13.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.335+0000 7f82675ff700 1 -- 172.21.15.2:0/2987184888 shutdown_connections 2023-09-13T04:51:13.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.341+0000 7f82675ff700 1 -- 172.21.15.2:0/2987184888 wait complete. 2023-09-13T04:51:13.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:13 smithi149 ceph-mon[133315]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:51:13.394 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T04:33:42.025714+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-13T04:37:57.094802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-13T04:37:58.564487+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-13T04:38:02.096895+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-13T04:51:13.395 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-13T04:51:13.396 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:13 smithi002 ceph-mon[97651]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:51:13.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:13 smithi156 ceph-mon[133675]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:51:13.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.846+0000 7fc8001ff700 1 -- 172.21.15.2:0/712231528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7f810d690 msgr2=0x7fc7f810da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:13.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.846+0000 7fc8001ff700 1 --2- 172.21.15.2:0/712231528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7f810d690 0x7fc7f810da70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00af90 tx=0x7fc7ec008850 comp rx=0 tx=0).stop 2023-09-13T04:51:13.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.857+0000 7fc8001ff700 1 -- 172.21.15.2:0/712231528 shutdown_connections 2023-09-13T04:51:13.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.857+0000 7fc8001ff700 1 --2- 172.21.15.2:0/712231528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7f8108670 0x7fc7f8114550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:13.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.857+0000 7fc8001ff700 1 --2- 172.21.15.2:0/712231528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7f810d690 0x7fc7f810da70 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:13.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.857+0000 7fc8001ff700 1 --2- 172.21.15.2:0/712231528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc7f8107cb0 0x7fc7f8108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:13.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.857+0000 7fc8001ff700 1 -- 172.21.15.2:0/712231528 >> 172.21.15.2:0/712231528 conn(0x7fc7f806d200 msgr2=0x7fc7f806d610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:13.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.861+0000 7fc8001ff700 1 -- 172.21.15.2:0/712231528 shutdown_connections 2023-09-13T04:51:13.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.861+0000 7fc8001ff700 1 -- 172.21.15.2:0/712231528 wait complete. 2023-09-13T04:51:13.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.862+0000 7fc8001ff700 1 Processor -- start 2023-09-13T04:51:13.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.866+0000 7fc8001ff700 1 -- start start 2023-09-13T04:51:13.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.867+0000 7fc8001ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7f8107cb0 0x7fc7f811dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:13.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.867+0000 7fc8001ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc7f8108670 0x7fc7f811e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:13.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.867+0000 7fc8001ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7f810d690 0x7fc7f8116ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:13.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.867+0000 7fc8001ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f81173b0 con 0x7fc7f8108670 2023-09-13T04:51:13.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.867+0000 7fc8001ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f8117520 con 0x7fc7f8107cb0 2023-09-13T04:51:13.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.867+0000 7fc8001ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f81176e0 con 0x7fc7f810d690 2023-09-13T04:51:13.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.867+0000 7fc7fd79a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc7f8108670 0x7fc7f811e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:13.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.868+0000 7fc7fd79a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc7f8108670 0x7fc7f811e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43406/0 (socket says 172.21.15.2:43406) 2023-09-13T04:51:13.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.868+0000 7fc7fd79a700 1 -- 172.21.15.2:0/1890305920 learned_addr learned my addr 172.21.15.2:0/1890305920 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:51:13.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.868+0000 7fc7fe79c700 1 --2- 172.21.15.2:0/1890305920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7f810d690 0x7fc7f8116ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:13.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.869+0000 7fc7fd79a700 1 -- 172.21.15.2:0/1890305920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7f810d690 msgr2=0x7fc7f8116ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:13.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.869+0000 7fc7fd79a700 1 --2- 172.21.15.2:0/1890305920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7f810d690 0x7fc7f8116ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:13.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.869+0000 7fc7fd79a700 1 -- 172.21.15.2:0/1890305920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7f8107cb0 msgr2=0x7fc7f811dd00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-13T04:51:13.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.869+0000 7fc7fd79a700 1 --2- 172.21.15.2:0/1890305920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7f8107cb0 0x7fc7f811dd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:13.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.869+0000 7fc7fd79a700 1 -- 172.21.15.2:0/1890305920 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ec0081d0 con 0x7fc7f8108670 2023-09-13T04:51:13.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.869+0000 7fc7fd79a700 1 --2- 172.21.15.2:0/1890305920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc7f8108670 0x7fc7f811e240 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000ba50 tx=0x7fc7f000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:13.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.871+0000 7fc7eaffd700 1 -- 172.21.15.2:0/1890305920 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f000cab0 con 0x7fc7f8108670 2023-09-13T04:51:13.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.871+0000 7fc7eaffd700 1 -- 172.21.15.2:0/1890305920 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7f000cc10 con 0x7fc7f8108670 2023-09-13T04:51:13.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.871+0000 7fc7eaffd700 1 -- 172.21.15.2:0/1890305920 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f00075b0 con 0x7fc7f8108670 2023-09-13T04:51:13.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.871+0000 7fc8001ff700 1 -- 172.21.15.2:0/1890305920 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f81179c0 con 0x7fc7f8108670 2023-09-13T04:51:13.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.871+0000 7fc8001ff700 1 -- 172.21.15.2:0/1890305920 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f811ba00 con 0x7fc7f8108670 2023-09-13T04:51:13.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.874+0000 7fc8001ff700 1 -- 172.21.15.2:0/1890305920 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f8067a60 con 0x7fc7f8108670 2023-09-13T04:51:13.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.882+0000 7fc7eaffd700 1 -- 172.21.15.2:0/1890305920 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7fc7f00077d0 con 0x7fc7f8108670 2023-09-13T04:51:13.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.884+0000 7fc7eaffd700 1 --2- 172.21.15.2:0/1890305920 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc7e40a09c0 0x7fc7e40a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:13.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.885+0000 7fc7eaffd700 1 -- 172.21.15.2:0/1890305920 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9483+0+0 (secure 0 0 0) 0x7fc7f00cba60 con 0x7fc7f8108670 2023-09-13T04:51:13.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.887+0000 7fc7fdf9b700 1 --2- 172.21.15.2:0/1890305920 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc7e40a09c0 0x7fc7e40a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:13.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.893+0000 7fc7eaffd700 1 -- 172.21.15.2:0/1890305920 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc7f0095170 con 0x7fc7f8108670 2023-09-13T04:51:13.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:13.910+0000 7fc7fdf9b700 1 --2- 172.21.15.2:0/1890305920 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc7e40a09c0 0x7fc7e40a2e80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec006010 tx=0x7fc7ec00d040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:14.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.306+0000 7fc8001ff700 1 -- 172.21.15.2:0/1890305920 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc7f811fb80 con 0x7fc7f8108670 2023-09-13T04:51:14.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.311+0000 7fc7eaffd700 1 -- 172.21.15.2:0/1890305920 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc7f0098790 con 0x7fc7f8108670 2023-09-13T04:51:14.312 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-09-13T04:51:14.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.318+0000 7fc8001ff700 1 -- 172.21.15.2:0/1890305920 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc7e40a09c0 msgr2=0x7fc7e40a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:14.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.318+0000 7fc8001ff700 1 --2- 172.21.15.2:0/1890305920 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc7e40a09c0 0x7fc7e40a2e80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec006010 tx=0x7fc7ec00d040 comp rx=0 tx=0).stop 2023-09-13T04:51:14.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.318+0000 7fc8001ff700 1 -- 172.21.15.2:0/1890305920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc7f8108670 msgr2=0x7fc7f811e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:14.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.318+0000 7fc8001ff700 1 --2- 172.21.15.2:0/1890305920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc7f8108670 0x7fc7f811e240 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000ba50 tx=0x7fc7f000be10 comp rx=0 tx=0).stop 2023-09-13T04:51:14.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.321+0000 7fc8001ff700 1 -- 172.21.15.2:0/1890305920 shutdown_connections 2023-09-13T04:51:14.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.321+0000 7fc8001ff700 1 --2- 172.21.15.2:0/1890305920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7f810d690 0x7fc7f8116ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:14.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.321+0000 7fc8001ff700 1 --2- 172.21.15.2:0/1890305920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7f8107cb0 0x7fc7f811dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:14.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.321+0000 7fc8001ff700 1 --2- 172.21.15.2:0/1890305920 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc7e40a09c0 0x7fc7e40a2e80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:14.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.321+0000 7fc8001ff700 1 --2- 172.21.15.2:0/1890305920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc7f8108670 0x7fc7f811e240 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:14.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.321+0000 7fc8001ff700 1 -- 172.21.15.2:0/1890305920 >> 172.21.15.2:0/1890305920 conn(0x7fc7f806d200 msgr2=0x7fc7f8071460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:14.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.328+0000 7fc8001ff700 1 -- 172.21.15.2:0/1890305920 shutdown_connections 2023-09-13T04:51:14.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.332+0000 7fc8001ff700 1 -- 172.21.15.2:0/1890305920 wait complete. 2023-09-13T04:51:14.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:14 smithi149 ceph-mon[133315]: pgmap v337: 129 pgs: 22 active+clean+snaptrim, 66 active+clean+snaptrim_wait, 41 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-09-13T04:51:14.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:14 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2987184888' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:51:14.395 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:14 smithi002 ceph-mon[97651]: pgmap v337: 129 pgs: 22 active+clean+snaptrim, 66 active+clean+snaptrim_wait, 41 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-09-13T04:51:14.395 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:14 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2987184888' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:51:14.396 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-13T04:51:14.441 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:14 smithi156 ceph-mon[133675]: pgmap v337: 129 pgs: 22 active+clean+snaptrim, 66 active+clean+snaptrim_wait, 41 active+clean; 1014 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-09-13T04:51:14.441 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:14 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2987184888' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:51:14.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.838+0000 7fb1927ff700 1 -- 172.21.15.2:0/3813664321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb18c1081f0 msgr2=0x7fb18c108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:14.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.838+0000 7fb1927ff700 1 --2- 172.21.15.2:0/3813664321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb18c1081f0 0x7fb18c108670 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb184009f90 tx=0x7fb18400b330 comp rx=0 tx=0).stop 2023-09-13T04:51:14.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.839+0000 7fb1927ff700 1 -- 172.21.15.2:0/3813664321 shutdown_connections 2023-09-13T04:51:14.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.839+0000 7fb1927ff700 1 --2- 172.21.15.2:0/3813664321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb18c11a490 0x7fb18c11c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:14.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.839+0000 7fb1927ff700 1 --2- 172.21.15.2:0/3813664321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb18c10f440 0x7fb18c107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:14.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.839+0000 7fb1927ff700 1 --2- 172.21.15.2:0/3813664321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb18c1081f0 0x7fb18c108670 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:14.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.839+0000 7fb1927ff700 1 -- 172.21.15.2:0/3813664321 >> 172.21.15.2:0/3813664321 conn(0x7fb18c06cfa0 msgr2=0x7fb18c06d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:14.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.847+0000 7fb1927ff700 1 -- 172.21.15.2:0/3813664321 shutdown_connections 2023-09-13T04:51:14.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.852+0000 7fb1927ff700 1 -- 172.21.15.2:0/3813664321 wait complete. 2023-09-13T04:51:14.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.853+0000 7fb1927ff700 1 Processor -- start 2023-09-13T04:51:14.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.854+0000 7fb1927ff700 1 -- start start 2023-09-13T04:51:14.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.854+0000 7fb1927ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb18c1081f0 0x7fb18c116470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:14.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.854+0000 7fb1927ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb18c10f440 0x7fb18c1169b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:14.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.854+0000 7fb1927ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb18c11a490 0x7fb18c111660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:14.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.854+0000 7fb1927ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c111ca0 con 0x7fb18c10f440 2023-09-13T04:51:14.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.854+0000 7fb1927ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c111e10 con 0x7fb18c11a490 2023-09-13T04:51:14.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.854+0000 7fb1927ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c111fd0 con 0x7fb18c1081f0 2023-09-13T04:51:14.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.857+0000 7fb18b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb18c10f440 0x7fb18c1169b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:14.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.857+0000 7fb18b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb18c10f440 0x7fb18c1169b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59800/0 (socket says 172.21.15.2:59800) 2023-09-13T04:51:14.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.857+0000 7fb18b7fe700 1 -- 172.21.15.2:0/4178238155 learned_addr learned my addr 172.21.15.2:0/4178238155 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:51:14.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.860+0000 7fb18b7fe700 1 -- 172.21.15.2:0/4178238155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb18c1081f0 msgr2=0x7fb18c116470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-13T04:51:14.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.860+0000 7fb18b7fe700 1 --2- 172.21.15.2:0/4178238155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb18c1081f0 0x7fb18c116470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:14.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.860+0000 7fb18b7fe700 1 -- 172.21.15.2:0/4178238155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb18c11a490 msgr2=0x7fb18c111660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:14.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.860+0000 7fb18b7fe700 1 --2- 172.21.15.2:0/4178238155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb18c11a490 0x7fb18c111660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:14.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.860+0000 7fb18b7fe700 1 -- 172.21.15.2:0/4178238155 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb184009c40 con 0x7fb18c10f440 2023-09-13T04:51:14.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.861+0000 7fb18b7fe700 1 --2- 172.21.15.2:0/4178238155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb18c10f440 0x7fb18c1169b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb184007730 tx=0x7fb184007810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:14.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.865+0000 7fb1897fa700 1 -- 172.21.15.2:0/4178238155 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18400e040 con 0x7fb18c10f440 2023-09-13T04:51:14.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.865+0000 7fb1897fa700 1 -- 172.21.15.2:0/4178238155 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb184004e90 con 0x7fb18c10f440 2023-09-13T04:51:14.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.865+0000 7fb1897fa700 1 -- 172.21.15.2:0/4178238155 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18401b790 con 0x7fb18c10f440 2023-09-13T04:51:14.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.866+0000 7fb1927ff700 1 -- 172.21.15.2:0/4178238155 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb18c112250 con 0x7fb18c10f440 2023-09-13T04:51:14.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.866+0000 7fb1927ff700 1 -- 172.21.15.2:0/4178238155 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb18c1153b0 con 0x7fb18c10f440 2023-09-13T04:51:14.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.873+0000 7fb1927ff700 1 -- 172.21.15.2:0/4178238155 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb18c067a60 con 0x7fb18c10f440 2023-09-13T04:51:14.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.875+0000 7fb1897fa700 1 -- 172.21.15.2:0/4178238155 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7fb18401b8f0 con 0x7fb18c10f440 2023-09-13T04:51:14.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.877+0000 7fb1897fa700 1 --2- 172.21.15.2:0/4178238155 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fb1740a09c0 0x7fb1740a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:14.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.878+0000 7fb1897fa700 1 -- 172.21.15.2:0/4178238155 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9483+0+0 (secure 0 0 0) 0x7fb1840ccfb0 con 0x7fb18c10f440 2023-09-13T04:51:14.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.884+0000 7fb1897fa700 1 -- 172.21.15.2:0/4178238155 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb184096410 con 0x7fb18c10f440 2023-09-13T04:51:14.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.892+0000 7fb18bfff700 1 --2- 172.21.15.2:0/4178238155 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fb1740a09c0 0x7fb1740a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:14.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:14.895+0000 7fb18bfff700 1 --2- 172.21.15.2:0/4178238155 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fb1740a09c0 0x7fb1740a2e80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb180005fd0 tx=0x7fb180009310 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:15.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.302+0000 7fb1927ff700 1 -- 172.21.15.2:0/4178238155 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb18c0008d0 con 0x7fb18c10f440 2023-09-13T04:51:15.314 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-09-13T04:51:15.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.307+0000 7fb1897fa700 1 -- 172.21.15.2:0/4178238155 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb184099a30 con 0x7fb18c10f440 2023-09-13T04:51:15.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.322+0000 7fb1927ff700 1 -- 172.21.15.2:0/4178238155 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fb1740a09c0 msgr2=0x7fb1740a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:15.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.322+0000 7fb1927ff700 1 --2- 172.21.15.2:0/4178238155 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fb1740a09c0 0x7fb1740a2e80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb180005fd0 tx=0x7fb180009310 comp rx=0 tx=0).stop 2023-09-13T04:51:15.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.322+0000 7fb1927ff700 1 -- 172.21.15.2:0/4178238155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb18c10f440 msgr2=0x7fb18c1169b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:15.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.322+0000 7fb1927ff700 1 --2- 172.21.15.2:0/4178238155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb18c10f440 0x7fb18c1169b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb184007730 tx=0x7fb184007810 comp rx=0 tx=0).stop 2023-09-13T04:51:15.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.323+0000 7fb1927ff700 1 -- 172.21.15.2:0/4178238155 shutdown_connections 2023-09-13T04:51:15.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.323+0000 7fb1927ff700 1 --2- 172.21.15.2:0/4178238155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb18c11a490 0x7fb18c111660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:15.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.323+0000 7fb1927ff700 1 --2- 172.21.15.2:0/4178238155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb18c1081f0 0x7fb18c116470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:15.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.323+0000 7fb1927ff700 1 --2- 172.21.15.2:0/4178238155 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fb1740a09c0 0x7fb1740a2e80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:15.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.323+0000 7fb1927ff700 1 --2- 172.21.15.2:0/4178238155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb18c10f440 0x7fb18c1169b0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:15.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.323+0000 7fb1927ff700 1 -- 172.21.15.2:0/4178238155 >> 172.21.15.2:0/4178238155 conn(0x7fb18c06cfa0 msgr2=0x7fb18c06f670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:15.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.329+0000 7fb1927ff700 1 -- 172.21.15.2:0/4178238155 shutdown_connections 2023-09-13T04:51:15.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.329+0000 7fb1927ff700 1 -- 172.21.15.2:0/4178238155 wait complete. 2023-09-13T04:51:15.364 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-13T04:51:15.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:15 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1890305920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:51:15.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:15 smithi149 ceph-mon[133315]: pgmap v338: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1013 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-09-13T04:51:15.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:15 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1890305920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:51:15.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:15 smithi156 ceph-mon[133675]: pgmap v338: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1013 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-09-13T04:51:15.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:15 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1890305920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T04:51:15.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:15 smithi002 ceph-mon[97651]: pgmap v338: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1013 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-09-13T04:51:15.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.811+0000 7fc3715ff700 1 -- 172.21.15.2:0/128990899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc36c073050 msgr2=0x7fc36c073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:15.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.811+0000 7fc3715ff700 1 --2- 172.21.15.2:0/128990899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc36c073050 0x7fc36c073430 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc354009990 tx=0x7fc354009ca0 comp rx=0 tx=0).stop 2023-09-13T04:51:15.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.815+0000 7fc3715ff700 1 -- 172.21.15.2:0/128990899 shutdown_connections 2023-09-13T04:51:15.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.815+0000 7fc3715ff700 1 --2- 172.21.15.2:0/128990899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc36c111c50 0x7fc36c114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:15.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.815+0000 7fc3715ff700 1 --2- 172.21.15.2:0/128990899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc36c073050 0x7fc36c073430 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:15.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.815+0000 7fc3715ff700 1 --2- 172.21.15.2:0/128990899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc36c073970 0x7fc36c111710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:15.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.815+0000 7fc3715ff700 1 -- 172.21.15.2:0/128990899 >> 172.21.15.2:0/128990899 conn(0x7fc36c0fc770 msgr2=0x7fc36c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:15.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.817+0000 7fc3715ff700 1 -- 172.21.15.2:0/128990899 shutdown_connections 2023-09-13T04:51:15.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.819+0000 7fc3715ff700 1 -- 172.21.15.2:0/128990899 wait complete. 2023-09-13T04:51:15.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.820+0000 7fc3715ff700 1 Processor -- start 2023-09-13T04:51:15.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.821+0000 7fc3715ff700 1 -- start start 2023-09-13T04:51:15.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.821+0000 7fc3715ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc36c073050 0x7fc36c06ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:15.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.821+0000 7fc3715ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc36c073970 0x7fc36c0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:15.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.821+0000 7fc3715ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc36c111c50 0x7fc36c06b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:15.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.821+0000 7fc3715ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c06b7f0 con 0x7fc36c073970 2023-09-13T04:51:15.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.821+0000 7fc3715ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c06b960 con 0x7fc36c073050 2023-09-13T04:51:15.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.821+0000 7fc3715ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c06bb20 con 0x7fc36c111c50 2023-09-13T04:51:15.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.822+0000 7fc3627fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc36c073970 0x7fc36c0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:15.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.822+0000 7fc3627fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc36c073970 0x7fc36c0704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59822/0 (socket says 172.21.15.2:59822) 2023-09-13T04:51:15.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.822+0000 7fc3627fc700 1 -- 172.21.15.2:0/4292119934 learned_addr learned my addr 172.21.15.2:0/4292119934 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:51:15.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.823+0000 7fc3627fc700 1 -- 172.21.15.2:0/4292119934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc36c111c50 msgr2=0x7fc36c06b110 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-13T04:51:15.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.823+0000 7fc3627fc700 1 --2- 172.21.15.2:0/4292119934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc36c111c50 0x7fc36c06b110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:15.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.823+0000 7fc3627fc700 1 -- 172.21.15.2:0/4292119934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc36c073050 msgr2=0x7fc36c06ff90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-13T04:51:15.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.823+0000 7fc3627fc700 1 --2- 172.21.15.2:0/4292119934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc36c073050 0x7fc36c06ff90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:15.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.823+0000 7fc3627fc700 1 -- 172.21.15.2:0/4292119934 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3540095f0 con 0x7fc36c073970 2023-09-13T04:51:15.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.823+0000 7fc3627fc700 1 --2- 172.21.15.2:0/4292119934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc36c073970 0x7fc36c0704d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00b530 tx=0x7fc35c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:15.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.825+0000 7fc368ff9700 1 -- 172.21.15.2:0/4292119934 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c011830 con 0x7fc36c073970 2023-09-13T04:51:15.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.825+0000 7fc368ff9700 1 -- 172.21.15.2:0/4292119934 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc35c011e50 con 0x7fc36c073970 2023-09-13T04:51:15.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.825+0000 7fc368ff9700 1 -- 172.21.15.2:0/4292119934 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c00f2e0 con 0x7fc36c073970 2023-09-13T04:51:15.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.825+0000 7fc3715ff700 1 -- 172.21.15.2:0/4292119934 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36c06be00 con 0x7fc36c073970 2023-09-13T04:51:15.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.825+0000 7fc3715ff700 1 -- 172.21.15.2:0/4292119934 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36c06eed0 con 0x7fc36c073970 2023-09-13T04:51:15.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.827+0000 7fc3715ff700 1 -- 172.21.15.2:0/4292119934 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc36c067eb0 con 0x7fc36c073970 2023-09-13T04:51:15.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.840+0000 7fc368ff9700 1 -- 172.21.15.2:0/4292119934 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7fc35c011990 con 0x7fc36c073970 2023-09-13T04:51:15.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.843+0000 7fc368ff9700 1 --2- 172.21.15.2:0/4292119934 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc3580a09c0 0x7fc3580a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:15.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.843+0000 7fc368ff9700 1 -- 172.21.15.2:0/4292119934 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9483+0+0 (secure 0 0 0) 0x7fc35c0cbcf0 con 0x7fc36c073970 2023-09-13T04:51:15.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.848+0000 7fc36affd700 1 --2- 172.21.15.2:0/4292119934 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc3580a09c0 0x7fc3580a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:15.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.853+0000 7fc368ff9700 1 -- 172.21.15.2:0/4292119934 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc35c0951d0 con 0x7fc36c073970 2023-09-13T04:51:15.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:15.860+0000 7fc36affd700 1 --2- 172.21.15.2:0/4292119934 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc3580a09c0 0x7fc3580a2e80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc354005cb0 tx=0x7fc354005bc0 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:16.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.264+0000 7fc3715ff700 1 -- 172.21.15.2:0/4292119934 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc36c06fa70 con 0x7fc36c073970 2023-09-13T04:51:16.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.264+0000 7fc368ff9700 1 -- 172.21.15.2:0/4292119934 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc35c015020 con 0x7fc36c073970 2023-09-13T04:51:16.271 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-09-13T04:51:16.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.273+0000 7fc3715ff700 1 -- 172.21.15.2:0/4292119934 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc3580a09c0 msgr2=0x7fc3580a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:16.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.273+0000 7fc3715ff700 1 --2- 172.21.15.2:0/4292119934 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc3580a09c0 0x7fc3580a2e80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc354005cb0 tx=0x7fc354005bc0 comp rx=0 tx=0).stop 2023-09-13T04:51:16.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.273+0000 7fc3715ff700 1 -- 172.21.15.2:0/4292119934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc36c073970 msgr2=0x7fc36c0704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:16.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.273+0000 7fc3715ff700 1 --2- 172.21.15.2:0/4292119934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc36c073970 0x7fc36c0704d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00b530 tx=0x7fc35c00b8f0 comp rx=0 tx=0).stop 2023-09-13T04:51:16.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.279+0000 7fc3715ff700 1 -- 172.21.15.2:0/4292119934 shutdown_connections 2023-09-13T04:51:16.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.279+0000 7fc3715ff700 1 --2- 172.21.15.2:0/4292119934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc36c111c50 0x7fc36c06b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:16.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.279+0000 7fc3715ff700 1 --2- 172.21.15.2:0/4292119934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc36c073050 0x7fc36c06ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:16.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.279+0000 7fc3715ff700 1 --2- 172.21.15.2:0/4292119934 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fc3580a09c0 0x7fc3580a2e80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:16.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.279+0000 7fc3715ff700 1 --2- 172.21.15.2:0/4292119934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc36c073970 0x7fc36c0704d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:16.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.279+0000 7fc3715ff700 1 -- 172.21.15.2:0/4292119934 >> 172.21.15.2:0/4292119934 conn(0x7fc36c0fc770 msgr2=0x7fc36c0fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:16.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.286+0000 7fc3715ff700 1 -- 172.21.15.2:0/4292119934 shutdown_connections 2023-09-13T04:51:16.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.289+0000 7fc3715ff700 1 -- 172.21.15.2:0/4292119934 wait complete. 2023-09-13T04:51:16.334 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-13T04:51:16.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:16 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/4178238155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:51:16.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:16 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/4178238155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:51:16.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:16 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/4178238155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T04:51:16.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.803+0000 7fdc38bff700 1 -- 172.21.15.2:0/2783375374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc34113e00 msgr2=0x7fdc341161f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:16.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.803+0000 7fdc38bff700 1 --2- 172.21.15.2:0/2783375374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc34113e00 0x7fdc341161f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fdc240098f0 tx=0x7fdc24009c00 comp rx=0 tx=0).stop 2023-09-13T04:51:16.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.804+0000 7fdc38bff700 1 -- 172.21.15.2:0/2783375374 shutdown_connections 2023-09-13T04:51:16.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.804+0000 7fdc38bff700 1 --2- 172.21.15.2:0/2783375374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc34113e00 0x7fdc341161f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:16.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.804+0000 7fdc38bff700 1 --2- 172.21.15.2:0/2783375374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc34107cb0 0x7fdc34108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:16.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.804+0000 7fdc38bff700 1 --2- 172.21.15.2:0/2783375374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc341085d0 0x7fdc341138c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:16.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.804+0000 7fdc38bff700 1 -- 172.21.15.2:0/2783375374 >> 172.21.15.2:0/2783375374 conn(0x7fdc3406cfa0 msgr2=0x7fdc3406d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.810+0000 7fdc38bff700 1 -- 172.21.15.2:0/2783375374 shutdown_connections 2023-09-13T04:51:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.810+0000 7fdc38bff700 1 -- 172.21.15.2:0/2783375374 wait complete. 2023-09-13T04:51:16.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.812+0000 7fdc38bff700 1 Processor -- start 2023-09-13T04:51:16.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.812+0000 7fdc38bff700 1 -- start start 2023-09-13T04:51:16.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.812+0000 7fdc38bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc34107cb0 0x7fdc34113470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:16.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.812+0000 7fdc38bff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc341085d0 0x7fdc34110430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:16.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.812+0000 7fdc38bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc34113e00 0x7fdc34110970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.812+0000 7fdc38bff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc34110eb0 con 0x7fdc341085d0 2023-09-13T04:51:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.812+0000 7fdc38bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc34111020 con 0x7fdc34113e00 2023-09-13T04:51:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.812+0000 7fdc38bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc341111e0 con 0x7fdc34107cb0 2023-09-13T04:51:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.813+0000 7fdc3259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc34107cb0 0x7fdc34113470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.813+0000 7fdc3259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc34107cb0 0x7fdc34113470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:43392/0 (socket says 172.21.15.2:43392) 2023-09-13T04:51:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.813+0000 7fdc3259c700 1 -- 172.21.15.2:0/1990526823 learned_addr learned my addr 172.21.15.2:0/1990526823 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:51:16.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.813+0000 7fdc31d9b700 1 --2- 172.21.15.2:0/1990526823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc341085d0 0x7fdc34110430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:16.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.814+0000 7fdc31d9b700 1 -- 172.21.15.2:0/1990526823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc34107cb0 msgr2=0x7fdc34113470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:16.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.814+0000 7fdc31d9b700 1 --2- 172.21.15.2:0/1990526823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc34107cb0 0x7fdc34113470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:16.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.814+0000 7fdc31d9b700 1 -- 172.21.15.2:0/1990526823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc34113e00 msgr2=0x7fdc34110970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:16.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.814+0000 7fdc31d9b700 1 --2- 172.21.15.2:0/1990526823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc34113e00 0x7fdc34110970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:16.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.814+0000 7fdc31d9b700 1 -- 172.21.15.2:0/1990526823 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc24009520 con 0x7fdc341085d0 2023-09-13T04:51:16.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.818+0000 7fdc31d9b700 1 --2- 172.21.15.2:0/1990526823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc341085d0 0x7fdc34110430 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800e9c0 tx=0x7fdc2800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:16.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.822+0000 7fdc237fe700 1 -- 172.21.15.2:0/1990526823 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2800c9c0 con 0x7fdc341085d0 2023-09-13T04:51:16.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.823+0000 7fdc237fe700 1 -- 172.21.15.2:0/1990526823 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc28010430 con 0x7fdc341085d0 2023-09-13T04:51:16.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.823+0000 7fdc237fe700 1 -- 172.21.15.2:0/1990526823 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc28018680 con 0x7fdc341085d0 2023-09-13T04:51:16.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.823+0000 7fdc38bff700 1 -- 172.21.15.2:0/1990526823 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc341b2670 con 0x7fdc341085d0 2023-09-13T04:51:16.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.823+0000 7fdc38bff700 1 -- 172.21.15.2:0/1990526823 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc341b2b00 con 0x7fdc341085d0 2023-09-13T04:51:16.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.831+0000 7fdc38bff700 1 -- 172.21.15.2:0/1990526823 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc34067a60 con 0x7fdc341085d0 2023-09-13T04:51:16.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.841+0000 7fdc237fe700 1 -- 172.21.15.2:0/1990526823 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7fdc28010860 con 0x7fdc341085d0 2023-09-13T04:51:16.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.843+0000 7fdc237fe700 1 --2- 172.21.15.2:0/1990526823 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fdc1c0a09c0 0x7fdc1c0a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:16.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.843+0000 7fdc237fe700 1 -- 172.21.15.2:0/1990526823 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9483+0+0 (secure 0 0 0) 0x7fdc28014070 con 0x7fdc341085d0 2023-09-13T04:51:16.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.844+0000 7fdc3259c700 1 --2- 172.21.15.2:0/1990526823 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fdc1c0a09c0 0x7fdc1c0a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:16.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.845+0000 7fdc3259c700 1 --2- 172.21.15.2:0/1990526823 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fdc1c0a09c0 0x7fdc1c0a2e80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c009fd0 tx=0x7fdc2c00b040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:16.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:16.851+0000 7fdc237fe700 1 -- 172.21.15.2:0/1990526823 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fdc280958a0 con 0x7fdc341085d0 2023-09-13T04:51:17.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.269+0000 7fdc38bff700 1 -- 172.21.15.2:0/1990526823 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdc34111960 con 0x7fdc341085d0 2023-09-13T04:51:17.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.281+0000 7fdc237fe700 1 -- 172.21.15.2:0/1990526823 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v105) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdc28095030 con 0x7fdc341085d0 2023-09-13T04:51:17.283 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-09-13T04:51:17.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.289+0000 7fdc38bff700 1 -- 172.21.15.2:0/1990526823 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fdc1c0a09c0 msgr2=0x7fdc1c0a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:17.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.289+0000 7fdc38bff700 1 --2- 172.21.15.2:0/1990526823 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fdc1c0a09c0 0x7fdc1c0a2e80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c009fd0 tx=0x7fdc2c00b040 comp rx=0 tx=0).stop 2023-09-13T04:51:17.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.289+0000 7fdc38bff700 1 -- 172.21.15.2:0/1990526823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc341085d0 msgr2=0x7fdc34110430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:17.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.289+0000 7fdc38bff700 1 --2- 172.21.15.2:0/1990526823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc341085d0 0x7fdc34110430 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800e9c0 tx=0x7fdc2800ecd0 comp rx=0 tx=0).stop 2023-09-13T04:51:17.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.290+0000 7fdc38bff700 1 -- 172.21.15.2:0/1990526823 shutdown_connections 2023-09-13T04:51:17.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.290+0000 7fdc38bff700 1 --2- 172.21.15.2:0/1990526823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc34113e00 0x7fdc34110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:17.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.290+0000 7fdc38bff700 1 --2- 172.21.15.2:0/1990526823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc34107cb0 0x7fdc34113470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:17.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.290+0000 7fdc38bff700 1 --2- 172.21.15.2:0/1990526823 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fdc1c0a09c0 0x7fdc1c0a2e80 secure :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c009fd0 tx=0x7fdc2c00b040 comp rx=0 tx=0).stop 2023-09-13T04:51:17.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.290+0000 7fdc38bff700 1 --2- 172.21.15.2:0/1990526823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc341085d0 0x7fdc34110430 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:17.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.290+0000 7fdc38bff700 1 -- 172.21.15.2:0/1990526823 >> 172.21.15.2:0/1990526823 conn(0x7fdc3406cfa0 msgr2=0x7fdc3406f670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:17.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.290+0000 7fdc38bff700 1 -- 172.21.15.2:0/1990526823 shutdown_connections 2023-09-13T04:51:17.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.290+0000 7fdc38bff700 1 -- 172.21.15.2:0/1990526823 wait complete. 2023-09-13T04:51:17.329 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-13T04:51:17.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:17 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/4292119934' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:51:17.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:17 smithi156 ceph-mon[133675]: pgmap v339: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1013 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-09-13T04:51:17.481 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:17 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/4292119934' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:51:17.482 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:17 smithi002 ceph-mon[97651]: pgmap v339: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1013 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-09-13T04:51:17.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:17 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/4292119934' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T04:51:17.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:17 smithi149 ceph-mon[133315]: pgmap v339: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1013 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-09-13T04:51:17.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.739+0000 7f9eb37ff700 1 -- 172.21.15.2:0/2403166825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9eac1081f0 msgr2=0x7f9eac108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:17.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.740+0000 7f9eb37ff700 1 --2- 172.21.15.2:0/2403166825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9eac1081f0 0x7f9eac108670 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9ea401c0d0 tx=0x7f9ea401c3e0 comp rx=0 tx=0).stop 2023-09-13T04:51:17.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.749+0000 7f9eb37ff700 1 -- 172.21.15.2:0/2403166825 shutdown_connections 2023-09-13T04:51:17.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.749+0000 7f9eb37ff700 1 --2- 172.21.15.2:0/2403166825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9eac11a490 0x7f9eac11c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:17.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.749+0000 7f9eb37ff700 1 --2- 172.21.15.2:0/2403166825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eac10f440 0x7f9eac107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:17.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.749+0000 7f9eb37ff700 1 --2- 172.21.15.2:0/2403166825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9eac1081f0 0x7f9eac108670 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:17.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.749+0000 7f9eb37ff700 1 -- 172.21.15.2:0/2403166825 >> 172.21.15.2:0/2403166825 conn(0x7f9eac06cfa0 msgr2=0x7f9eac06d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:17.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.753+0000 7f9eb37ff700 1 -- 172.21.15.2:0/2403166825 shutdown_connections 2023-09-13T04:51:17.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.753+0000 7f9eb37ff700 1 -- 172.21.15.2:0/2403166825 wait complete. 2023-09-13T04:51:17.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.755+0000 7f9eb37ff700 1 Processor -- start 2023-09-13T04:51:17.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.755+0000 7f9eb37ff700 1 -- start start 2023-09-13T04:51:17.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.755+0000 7f9eb37ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eac1081f0 0x7f9eac1157e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:17.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.756+0000 7f9eb37ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9eac10f440 0x7f9eac115d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:17.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.756+0000 7f9eb37ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9eac11a490 0x7f9eac1a91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:17.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.756+0000 7f9eb37ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac1a97a0 con 0x7f9eac11a490 2023-09-13T04:51:17.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.756+0000 7f9eb37ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac1a98e0 con 0x7f9eac10f440 2023-09-13T04:51:17.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.756+0000 7f9eb37ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac1a9aa0 con 0x7f9eac1081f0 2023-09-13T04:51:17.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.756+0000 7f9eb159b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eac1081f0 0x7f9eac1157e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:17.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.756+0000 7f9eb159b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eac1081f0 0x7f9eac1157e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:43406/0 (socket says 172.21.15.2:43406) 2023-09-13T04:51:17.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.756+0000 7f9eb159b700 1 -- 172.21.15.2:0/1676930234 learned_addr learned my addr 172.21.15.2:0/1676930234 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:51:17.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.757+0000 7f9eb159b700 1 -- 172.21.15.2:0/1676930234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9eac10f440 msgr2=0x7f9eac115d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:17.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.757+0000 7f9eb159b700 1 --2- 172.21.15.2:0/1676930234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9eac10f440 0x7f9eac115d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:17.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.757+0000 7f9eb159b700 1 -- 172.21.15.2:0/1676930234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9eac11a490 msgr2=0x7f9eac1a91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:17.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.757+0000 7f9eb159b700 1 --2- 172.21.15.2:0/1676930234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9eac11a490 0x7f9eac1a91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:17.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.757+0000 7f9eb159b700 1 -- 172.21.15.2:0/1676930234 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea401ad10 con 0x7f9eac1081f0 2023-09-13T04:51:17.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.758+0000 7f9eb159b700 1 --2- 172.21.15.2:0/1676930234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eac1081f0 0x7f9eac1157e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00b580 tx=0x7f9e9c00b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:17.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.759+0000 7f9ea27fc700 1 -- 172.21.15.2:0/1676930234 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c011830 con 0x7f9eac1081f0 2023-09-13T04:51:17.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.759+0000 7f9ea27fc700 1 -- 172.21.15.2:0/1676930234 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e9c011e50 con 0x7f9eac1081f0 2023-09-13T04:51:17.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.759+0000 7f9ea27fc700 1 -- 172.21.15.2:0/1676930234 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00f2e0 con 0x7f9eac1081f0 2023-09-13T04:51:17.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.759+0000 7f9eb37ff700 1 -- 172.21.15.2:0/1676930234 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eac1a9d80 con 0x7f9eac1081f0 2023-09-13T04:51:17.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.759+0000 7f9eb37ff700 1 -- 172.21.15.2:0/1676930234 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eac1aa1e0 con 0x7f9eac1081f0 2023-09-13T04:51:17.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.771+0000 7f9eb37ff700 1 -- 172.21.15.2:0/1676930234 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eac067a60 con 0x7f9eac1081f0 2023-09-13T04:51:17.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.774+0000 7f9ea27fc700 1 -- 172.21.15.2:0/1676930234 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f9e9c0105f0 con 0x7f9eac1081f0 2023-09-13T04:51:17.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.777+0000 7f9ea27fc700 1 --2- 172.21.15.2:0/1676930234 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f9e980a0940 0x7f9e980a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:17.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.777+0000 7f9ea27fc700 1 -- 172.21.15.2:0/1676930234 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9483+0+0 (secure 0 0 0) 0x7f9e9c011a30 con 0x7f9eac1081f0 2023-09-13T04:51:17.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.784+0000 7f9eb0d9a700 1 --2- 172.21.15.2:0/1676930234 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f9e980a0940 0x7f9e980a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:17.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.784+0000 7f9ea27fc700 1 -- 172.21.15.2:0/1676930234 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9e9c095920 con 0x7f9eac1081f0 2023-09-13T04:51:17.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:17.788+0000 7f9eb0d9a700 1 --2- 172.21.15.2:0/1676930234 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f9e980a0940 0x7f9e980a2e00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9eac06d6a0 tx=0x7f9ea401c730 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:18.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:18.227+0000 7f9eb37ff700 1 -- 172.21.15.2:0/1676930234 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9eac1125b0 con 0x7f9eac1081f0 2023-09-13T04:51:18.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:18.231+0000 7f9ea27fc700 1 -- 172.21.15.2:0/1676930234 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v105) v1 ==== 81+0+928 (secure 0 0 0) 0x7f9e9c015070 con 0x7f9eac1081f0 2023-09-13T04:51:18.236 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/1461251021 2023-09-14T04:40:27.774561+0000 2023-09-13T04:51:18.236 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/1461251021 2023-09-14T04:40:27.774561+0000 2023-09-13T04:51:18.236 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2528007246 2023-09-14T04:40:27.774561+0000 2023-09-13T04:51:18.236 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2955382590 2023-09-14T04:40:27.774561+0000 2023-09-13T04:51:18.237 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/227933483 2023-09-14T04:40:27.774561+0000 2023-09-13T04:51:18.237 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/1393377368 2023-09-14T04:31:23.417691+0000 2023-09-13T04:51:18.237 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/2772434518 2023-09-14T04:30:54.963309+0000 2023-09-13T04:51:18.237 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3810974822 2023-09-14T04:31:23.417691+0000 2023-09-13T04:51:18.237 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/2772434518 2023-09-14T04:30:54.963309+0000 2023-09-13T04:51:18.238 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1907805835 2023-09-14T04:30:54.963309+0000 2023-09-13T04:51:18.238 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2811866029 2023-09-14T04:30:54.963309+0000 2023-09-13T04:51:18.238 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3568881483 2023-09-14T04:40:27.774561+0000 2023-09-13T04:51:18.238 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2538614772 2023-09-14T04:31:23.417691+0000 2023-09-13T04:51:18.238 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/1393377368 2023-09-14T04:31:23.417691+0000 2023-09-13T04:51:18.239 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3722465120 2023-09-14T04:31:23.417691+0000 2023-09-13T04:51:18.239 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/516951274 2023-09-14T04:30:54.963309+0000 2023-09-13T04:51:18.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:18.242+0000 7f9eb37ff700 1 -- 172.21.15.2:0/1676930234 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f9e980a0940 msgr2=0x7f9e980a2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:18.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:18.242+0000 7f9eb37ff700 1 --2- 172.21.15.2:0/1676930234 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f9e980a0940 0x7f9e980a2e00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9eac06d6a0 tx=0x7f9ea401c730 comp rx=0 tx=0).stop 2023-09-13T04:51:18.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:18.242+0000 7f9eb37ff700 1 -- 172.21.15.2:0/1676930234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eac1081f0 msgr2=0x7f9eac1157e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:18.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:18.242+0000 7f9eb37ff700 1 --2- 172.21.15.2:0/1676930234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eac1081f0 0x7f9eac1157e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00b580 tx=0x7f9e9c00b940 comp rx=0 tx=0).stop 2023-09-13T04:51:18.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:18.244+0000 7f9eb37ff700 1 -- 172.21.15.2:0/1676930234 shutdown_connections 2023-09-13T04:51:18.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:18.244+0000 7f9eb37ff700 1 --2- 172.21.15.2:0/1676930234 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f9e980a0940 0x7f9e980a2e00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:18.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:18.244+0000 7f9eb37ff700 1 --2- 172.21.15.2:0/1676930234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9eac11a490 0x7f9eac1a91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:18.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:18.244+0000 7f9eb37ff700 1 --2- 172.21.15.2:0/1676930234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eac1081f0 0x7f9eac1157e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:18.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:18.244+0000 7f9eb37ff700 1 --2- 172.21.15.2:0/1676930234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9eac10f440 0x7f9eac115d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:18.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:18.244+0000 7f9eb37ff700 1 -- 172.21.15.2:0/1676930234 >> 172.21.15.2:0/1676930234 conn(0x7f9eac06cfa0 msgr2=0x7f9eac071680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:18.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:18.248+0000 7f9eb37ff700 1 -- 172.21.15.2:0/1676930234 shutdown_connections 2023-09-13T04:51:18.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:18.248+0000 7f9eb37ff700 1 -- 172.21.15.2:0/1676930234 wait complete. 2023-09-13T04:51:18.256 INFO:teuthology.orchestra.run.smithi002.stderr:listed 16 entries 2023-09-13T04:51:18.286 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-09-13T04:51:18.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:51:18.286 DEBUG:teuthology.orchestra.run.smithi002:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-09-13T04:51:18.432 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:18 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1990526823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:51:18.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:18 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1990526823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:51:18.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:18 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1990526823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-13T04:51:19.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:19 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1676930234' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-13T04:51:19.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:19 smithi002 ceph-mon[97651]: pgmap v340: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1013 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 192 B/s wr, 5 op/s 2023-09-13T04:51:19.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:19 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:51:19.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:19 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1676930234' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-13T04:51:19.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:19 smithi149 ceph-mon[133315]: pgmap v340: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1013 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 192 B/s wr, 5 op/s 2023-09-13T04:51:19.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:19 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:51:19.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:19 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1676930234' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-13T04:51:19.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:19 smithi156 ceph-mon[133675]: pgmap v340: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1013 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 192 B/s wr, 5 op/s 2023-09-13T04:51:19.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:19 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T04:51:20.176 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi002.front.sepia.ceph.com 2023-09-13T04:51:20.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:51:20.177 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-13T04:51:20.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:51:20.379 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2023-09-13T04:51:20.570 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-13T04:51:20.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:51:20.571 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-13T04:51:20.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:51:20.803 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link delete ceph-brx 2023-09-13T04:51:21.069 INFO:tasks.kclient:Unmounting kernel clients... 2023-09-13T04:51:21.070 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-09-13T04:51:21.070 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-13T04:51:21.195 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348692k,nr_inodes=4087173,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/9448a226694c6fe7a193249f6dd32aab9105638574e1246c5b3cd22f59f71649/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/6HVRRVFVJJUYPWFZCFSSQP7O5M:/var/lib/containers/storage/overlay/l/PZEAVFTWFR6NBJI2J2M36UW7QW,upperdir=/var/lib/containers/storage/overlay/9448a226694c6fe7a193249f6dd32aab9105638574e1246c5b3cd22f59f71649/diff,workdir=/var/lib/containers/storage/overlay/9448a226694c6fe7a193249f6dd32aab9105638574e1246c5b3cd22f59f71649/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/58feb3c7e9ad2a0512f9a35b7ad9df270c61c59f8e78af605514375a95ee7e6a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/6HVRRVFVJJUYPWFZCFSSQP7O5M:/var/lib/containers/storage/overlay/l/PZEAVFTWFR6NBJI2J2M36UW7QW,upperdir=/var/lib/containers/storage/overlay/58feb3c7e9ad2a0512f9a35b7ad9df270c61c59f8e78af605514375a95ee7e6a/diff,workdir=/var/lib/containers/storage/overlay/58feb3c7e9ad2a0512f9a35b7ad9df270c61c59f8e78af605514375a95ee7e6a/work,redirect_dir=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/e83a788fc6fb020faa0be35123baf0da922bd5f0e780c2767388ddb6c99f4dff/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/6HVRRVFVJJUYPWFZCFSSQP7O5M:/var/lib/containers/storage/overlay/l/PZEAVFTWFR6NBJI2J2M36UW7QW,upperdir=/var/lib/containers/storage/overlay/e83a788fc6fb020faa0be35123baf0da922bd5f0e780c2767388ddb6c99f4dff/diff,workdir=/var/lib/containers/storage/overlay/e83a788fc6fb020faa0be35123baf0da922bd5f0e780c2767388ddb6c99f4dff/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/3a00de84dd0e0706da447387696d07cf88d2120c94e07f43396ba3affe4382ff/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/6HVRRVFVJJUYPWFZCFSSQP7O5M:/var/lib/containers/storage/overlay/l/PZEAVFTWFR6NBJI2J2M36UW7QW,upperdir=/var/lib/containers/storage/overlay/3a00de84dd0e0706da447387696d07cf88d2120c94e07f43396ba3affe4382ff/diff,workdir=/var/lib/containers/storage/overlay/3a00de84dd0e0706da447387696d07cf88d2120c94e07f43396ba3affe4382ff/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f2f304f45f9c1a673a53decf5a776041f31dab34a5363c43d112fc45f4aed334/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/6HVRRVFVJJUYPWFZCFSSQP7O5M:/var/lib/containers/storage/overlay/l/PZEAVFTWFR6NBJI2J2M36UW7QW,upperdir=/var/lib/containers/storage/overlay/f2f304f45f9c1a673a53decf5a776041f31dab34a5363c43d112fc45f4aed334/diff,workdir=/var/lib/containers/storage/overlay/f2f304f45f9c1a673a53decf5a776041f31dab34a5363c43d112fc45f4aed334/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9fd66c4afe6ab40f577eb0d9d8167c1694cf834437fbd7109564537e464060a5/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/6HVRRVFVJJUYPWFZCFSSQP7O5M:/var/lib/containers/storage/overlay/l/PZEAVFTWFR6NBJI2J2M36UW7QW,upperdir=/var/lib/containers/storage/overlay/9fd66c4afe6ab40f577eb0d9d8167c1694cf834437fbd7109564537e464060a5/diff,workdir=/var/lib/containers/storage/overlay/9fd66c4afe6ab40f577eb0d9d8167c1694cf834437fbd7109564537e464060a5/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f325f7624707328c01c9c829501b27a2e08eaa497f0715ac225706c1202fde12/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/6HVRRVFVJJUYPWFZCFSSQP7O5M:/var/lib/containers/storage/overlay/l/PZEAVFTWFR6NBJI2J2M36UW7QW,upperdir=/var/lib/containers/storage/overlay/f325f7624707328c01c9c829501b27a2e08eaa497f0715ac225706c1202fde12/diff,workdir=/var/lib/containers/storage/overlay/f325f7624707328c01c9c829501b27a2e08eaa497f0715ac225706c1202fde12/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ac7a04b9d720dfab2fbe51856f099d131d5819a18dfebd9bca8a59fafb292be9/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/6HVRRVFVJJUYPWFZCFSSQP7O5M:/var/lib/containers/storage/overlay/l/PZEAVFTWFR6NBJI2J2M36UW7QW,upperdir=/var/lib/containers/storage/overlay/ac7a04b9d720dfab2fbe51856f099d131d5819a18dfebd9bca8a59fafb292be9/diff,workdir=/var/lib/containers/storage/overlay/ac7a04b9d720dfab2fbe51856f099d131d5819a18dfebd9bca8a59fafb292be9/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e30cc29750d8f91ad8e23b5943642edf1ea85c91aa34ba77ca818f81ba801f01/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/6HVRRVFVJJUYPWFZCFSSQP7O5M:/var/lib/containers/storage/overlay/l/PZEAVFTWFR6NBJI2J2M36UW7QW,upperdir=/var/lib/containers/storage/overlay/e30cc29750d8f91ad8e23b5943642edf1ea85c91aa34ba77ca818f81ba801f01/diff,workdir=/var/lib/containers/storage/overlay/e30cc29750d8f91ad8e23b5943642edf1ea85c91aa34ba77ca818f81ba801f01/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/95a8d67b7d0e016a611b9ff813c1091ad0e4a6e14e0c6d3cd1d80659a610324d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/6HVRRVFVJJUYPWFZCFSSQP7O5M:/var/lib/containers/storage/overlay/l/PZEAVFTWFR6NBJI2J2M36UW7QW,upperdir=/var/lib/containers/storage/overlay/95a8d67b7d0e016a611b9ff813c1091ad0e4a6e14e0c6d3cd1d80659a610324d/diff,workdir=/var/lib/containers/storage/overlay/95a8d67b7d0e016a611b9ff813c1091ad0e4a6e14e0c6d3cd1d80659a610324d/work,redirect_dir=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-09-13T04:51:21.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:51:21.196 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2023-09-13T04:51:21.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T04:51:21.309 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link delete ceph-brx 2023-09-13T04:51:21.460 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:21 smithi002 ceph-mon[97651]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:51:21.460 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:21 smithi002 ceph-mon[97651]: pgmap v341: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1011 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2023-09-13T04:51:21.496 INFO:teuthology.orchestra.run.smithi002.stderr:Cannot find device "ceph-brx" 2023-09-13T04:51:21.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T04:51:21.508 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-09-13T04:51:21.519 INFO:tasks.cephadm:Teardown begin 2023-09-13T04:51:21.520 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-13T04:51:21.520 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:51:21.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:21 smithi149 ceph-mon[133315]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:51:21.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:21 smithi149 ceph-mon[133315]: pgmap v341: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1011 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2023-09-13T04:51:21.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:21 smithi156 ceph-mon[133675]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-13T04:51:21.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:21 smithi156 ceph-mon[133675]: pgmap v341: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1011 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2023-09-13T04:51:21.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.950+0000 7f59509ff700 1 -- 172.21.15.2:0/4029697471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f594c11a490 msgr2=0x7f594c11c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:21.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.950+0000 7f59509ff700 1 --2- 172.21.15.2:0/4029697471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f594c11a490 0x7f594c11c880 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f594400b0a0 tx=0x7f594400b3b0 comp rx=0 tx=0).stop 2023-09-13T04:51:21.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.951+0000 7f59509ff700 1 -- 172.21.15.2:0/4029697471 shutdown_connections 2023-09-13T04:51:21.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.951+0000 7f59509ff700 1 --2- 172.21.15.2:0/4029697471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f594c11a490 0x7f594c11c880 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:21.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.951+0000 7f59509ff700 1 --2- 172.21.15.2:0/4029697471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f594c10f440 0x7f594c107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:21.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.951+0000 7f59509ff700 1 --2- 172.21.15.2:0/4029697471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f594c1081f0 0x7f594c108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:21.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.951+0000 7f59509ff700 1 -- 172.21.15.2:0/4029697471 >> 172.21.15.2:0/4029697471 conn(0x7f594c06bd20 msgr2=0x7f594c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:21.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.958+0000 7f5949d9b700 1 --2- 172.21.15.2:0/4029697471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f594c1081f0 0x7f594c108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2023-09-13T04:51:21.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.958+0000 7f594a59c700 1 --2- 172.21.15.2:0/4029697471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f594c10f440 0x7f594c107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2023-09-13T04:51:21.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.958+0000 7f59509ff700 1 -- 172.21.15.2:0/4029697471 shutdown_connections 2023-09-13T04:51:21.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.960+0000 7f59509ff700 1 -- 172.21.15.2:0/4029697471 wait complete. 2023-09-13T04:51:21.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.961+0000 7f59509ff700 1 Processor -- start 2023-09-13T04:51:21.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.961+0000 7f59509ff700 1 -- start start 2023-09-13T04:51:21.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.962+0000 7f59509ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f594c11a490 0x7f594c1a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:21.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.962+0000 7f59509ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f594c1a6440 0x7f594c1aa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:21.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.962+0000 7f59509ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f594c1aade0 0x7f594c1ab290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:21.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.962+0000 7f59509ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594c1a6b30 con 0x7f594c1aade0 2023-09-13T04:51:21.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.962+0000 7f59509ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594c1b2b50 con 0x7f594c1a6440 2023-09-13T04:51:21.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.962+0000 7f59509ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594c1b2c90 con 0x7f594c11a490 2023-09-13T04:51:21.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.963+0000 7f594ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f594c1aade0 0x7f594c1ab290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:21.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.963+0000 7f594ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f594c1aade0 0x7f594c1ab290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59892/0 (socket says 172.21.15.2:59892) 2023-09-13T04:51:21.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.963+0000 7f594ad9d700 1 -- 172.21.15.2:0/1361994929 learned_addr learned my addr 172.21.15.2:0/1361994929 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:51:21.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.963+0000 7f5949d9b700 1 --2- 172.21.15.2:0/1361994929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f594c1a6440 0x7f594c1aa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:21.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.963+0000 7f594a59c700 1 --2- 172.21.15.2:0/1361994929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f594c11a490 0x7f594c1a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:21.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.965+0000 7f5949d9b700 1 -- 172.21.15.2:0/1361994929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f594c11a490 msgr2=0x7f594c1a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:21.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.965+0000 7f5949d9b700 1 --2- 172.21.15.2:0/1361994929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f594c11a490 0x7f594c1a5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:21.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.965+0000 7f5949d9b700 1 -- 172.21.15.2:0/1361994929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f594c1aade0 msgr2=0x7f594c1ab290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:21.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.965+0000 7f5949d9b700 1 --2- 172.21.15.2:0/1361994929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f594c1aade0 0x7f594c1ab290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:21.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.965+0000 7f5949d9b700 1 -- 172.21.15.2:0/1361994929 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5944009d00 con 0x7f594c1a6440 2023-09-13T04:51:21.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.973+0000 7f5949d9b700 1 --2- 172.21.15.2:0/1361994929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f594c1a6440 0x7f594c1aa8a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f594c06cb40 tx=0x7f593c02ea50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:21.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.974+0000 7f593b7fe700 1 -- 172.21.15.2:0/1361994929 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c030890 con 0x7f594c1a6440 2023-09-13T04:51:21.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.974+0000 7f593b7fe700 1 -- 172.21.15.2:0/1361994929 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f593c031460 con 0x7f594c1a6440 2023-09-13T04:51:21.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.974+0000 7f593b7fe700 1 -- 172.21.15.2:0/1361994929 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c03a640 con 0x7f594c1a6440 2023-09-13T04:51:21.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.976+0000 7f59509ff700 1 -- 172.21.15.2:0/1361994929 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f594c072f40 con 0x7f594c1a6440 2023-09-13T04:51:21.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.976+0000 7f59509ff700 1 -- 172.21.15.2:0/1361994929 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f594c073400 con 0x7f594c1a6440 2023-09-13T04:51:21.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.978+0000 7f59509ff700 1 -- 172.21.15.2:0/1361994929 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f594c067ad0 con 0x7f594c1a6440 2023-09-13T04:51:21.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.989+0000 7f593b7fe700 1 -- 172.21.15.2:0/1361994929 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f593c0315d0 con 0x7f594c1a6440 2023-09-13T04:51:21.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.992+0000 7f593b7fe700 1 --2- 172.21.15.2:0/1361994929 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f59340a0a10 0x7f59340a2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:21.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.992+0000 7f593b7fe700 1 -- 172.21.15.2:0/1361994929 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9483+0+0 (secure 0 0 0) 0x7f593c0ee720 con 0x7f594c1a6440 2023-09-13T04:51:21.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:21.992+0000 7f593b7fe700 1 -- 172.21.15.2:0/1361994929 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f593c0eeb80 con 0x7f594c1a6440 2023-09-13T04:51:22.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.007+0000 7f594a59c700 1 --2- 172.21.15.2:0/1361994929 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f59340a0a10 0x7f59340a2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:22.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.009+0000 7f594a59c700 1 --2- 172.21.15.2:0/1361994929 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f59340a0a10 0x7f59340a2ed0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f594c06cbc0 tx=0x7f594000a250 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:22.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.300+0000 7f59509ff700 1 -- 172.21.15.2:0/1361994929 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f594c1a7250 con 0x7f594c1a6440 2023-09-13T04:51:22.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.301+0000 7f593b7fe700 1 -- 172.21.15.2:0/1361994929 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+11310 (secure 0 0 0) 0x7f593c0b7bf0 con 0x7f594c1a6440 2023-09-13T04:51:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:51:22.302 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2469307307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2469307307},{"type":"v1","addr":"172.21.15.2:6839","nonce":2469307307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T04:37:59.110701+0000","modified":"2023-09-13T04:51:19.325240+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24554,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/846579077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":846579077},{"type":"v1","addr":"172.21.15.156:6835","nonce":846579077}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1896833690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1896833690},{"type":"v1","addr":"172.21.15.149:6837","nonce":1896833690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6839/1785833227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1785833227},{"type":"v1","addr":"172.21.15.156:6839","nonce":1785833227}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":136,"addr":"172.21.15.2:6841/482545391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":482545391},{"type":"v1","addr":"172.21.15.2:6841","nonce":482545391}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/389167841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":389167841},{"type":"v1","addr":"172.21.15.149:6839","nonce":389167841}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26764":{"gid":26764,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3447644694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3447644694},{"type":"v1","addr":"172.21.15.156:6837","nonce":3447644694}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T04:51:22.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.303+0000 7f59509ff700 1 -- 172.21.15.2:0/1361994929 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f59340a0a10 msgr2=0x7f59340a2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:22.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.303+0000 7f59509ff700 1 --2- 172.21.15.2:0/1361994929 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f59340a0a10 0x7f59340a2ed0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f594c06cbc0 tx=0x7f594000a250 comp rx=0 tx=0).stop 2023-09-13T04:51:22.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.303+0000 7f59509ff700 1 -- 172.21.15.2:0/1361994929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f594c1a6440 msgr2=0x7f594c1aa8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:22.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.303+0000 7f59509ff700 1 --2- 172.21.15.2:0/1361994929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f594c1a6440 0x7f594c1aa8a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f594c06cb40 tx=0x7f593c02ea50 comp rx=0 tx=0).stop 2023-09-13T04:51:22.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.303+0000 7f59509ff700 1 -- 172.21.15.2:0/1361994929 shutdown_connections 2023-09-13T04:51:22.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.303+0000 7f59509ff700 1 --2- 172.21.15.2:0/1361994929 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f59340a0a10 0x7f59340a2ed0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:22.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.303+0000 7f59509ff700 1 --2- 172.21.15.2:0/1361994929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f594c1aade0 0x7f594c1ab290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:22.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.303+0000 7f59509ff700 1 --2- 172.21.15.2:0/1361994929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f594c11a490 0x7f594c1a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:22.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.303+0000 7f59509ff700 1 --2- 172.21.15.2:0/1361994929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f594c1a6440 0x7f594c1aa8a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:22.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.303+0000 7f59509ff700 1 -- 172.21.15.2:0/1361994929 >> 172.21.15.2:0/1361994929 conn(0x7f594c06bd20 msgr2=0x7f594c118f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:22.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.303+0000 7f59509ff700 1 -- 172.21.15.2:0/1361994929 shutdown_connections 2023-09-13T04:51:22.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.304+0000 7f59509ff700 1 -- 172.21.15.2:0/1361994929 wait complete. 2023-09-13T04:51:22.307 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 41 2023-09-13T04:51:22.324 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T04:51:22.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.736+0000 7fd9079ff700 1 -- 172.21.15.2:0/3412495949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd90010f440 msgr2=0x7fd900107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:22.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.736+0000 7fd9079ff700 1 --2- 172.21.15.2:0/3412495949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd90010f440 0x7fd900107cb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd8f001a9a0 tx=0x7fd8f001acb0 comp rx=0 tx=0).stop 2023-09-13T04:51:22.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.740+0000 7fd9079ff700 1 -- 172.21.15.2:0/3412495949 shutdown_connections 2023-09-13T04:51:22.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.740+0000 7fd9079ff700 1 --2- 172.21.15.2:0/3412495949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd90011a490 0x7fd90011c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:22.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.740+0000 7fd9079ff700 1 --2- 172.21.15.2:0/3412495949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd90010f440 0x7fd900107cb0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:22.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.740+0000 7fd9079ff700 1 --2- 172.21.15.2:0/3412495949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9001081f0 0x7fd900108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:22.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.740+0000 7fd9079ff700 1 -- 172.21.15.2:0/3412495949 >> 172.21.15.2:0/3412495949 conn(0x7fd90006bd20 msgr2=0x7fd90006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:22.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.740+0000 7fd9079ff700 1 -- 172.21.15.2:0/3412495949 shutdown_connections 2023-09-13T04:51:22.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.740+0000 7fd9079ff700 1 -- 172.21.15.2:0/3412495949 wait complete. 2023-09-13T04:51:22.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.742+0000 7fd9079ff700 1 Processor -- start 2023-09-13T04:51:22.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.743+0000 7fd9079ff700 1 -- start start 2023-09-13T04:51:22.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.743+0000 7fd9079ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9001081f0 0x7fd900115830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:22.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.743+0000 7fd9079ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd90010f440 0x7fd900115d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:22.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.743+0000 7fd9079ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd90011a490 0x7fd9001a31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:22.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.743+0000 7fd9079ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9001a37a0 con 0x7fd90010f440 2023-09-13T04:51:22.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.743+0000 7fd9079ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9001a3910 con 0x7fd9001081f0 2023-09-13T04:51:22.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.743+0000 7fd9079ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9001a3ad0 con 0x7fd90011a490 2023-09-13T04:51:22.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.744+0000 7fd905f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd90011a490 0x7fd9001a31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:22.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.744+0000 7fd905f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd90011a490 0x7fd9001a31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:43426/0 (socket says 172.21.15.2:43426) 2023-09-13T04:51:22.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.744+0000 7fd905f9c700 1 -- 172.21.15.2:0/2122890885 learned_addr learned my addr 172.21.15.2:0/2122890885 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:51:22.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.744+0000 7fd90579b700 1 --2- 172.21.15.2:0/2122890885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9001081f0 0x7fd900115830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:22.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.744+0000 7fd905f9c700 1 -- 172.21.15.2:0/2122890885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9001081f0 msgr2=0x7fd900115830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:22.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.744+0000 7fd905f9c700 1 --2- 172.21.15.2:0/2122890885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9001081f0 0x7fd900115830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:22.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.744+0000 7fd905f9c700 1 -- 172.21.15.2:0/2122890885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd90010f440 msgr2=0x7fd900115d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-13T04:51:22.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.744+0000 7fd905f9c700 1 --2- 172.21.15.2:0/2122890885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd90010f440 0x7fd900115d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:22.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.744+0000 7fd905f9c700 1 -- 172.21.15.2:0/2122890885 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f001a600 con 0x7fd90011a490 2023-09-13T04:51:22.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.745+0000 7fd905f9c700 1 --2- 172.21.15.2:0/2122890885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd90011a490 0x7fd9001a31a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc00b450 tx=0x7fd8fc00b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:22.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.745+0000 7fd8f67fc700 1 -- 172.21.15.2:0/2122890885 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc011830 con 0x7fd90011a490 2023-09-13T04:51:22.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.745+0000 7fd8f67fc700 1 -- 172.21.15.2:0/2122890885 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8fc011e50 con 0x7fd90011a490 2023-09-13T04:51:22.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.745+0000 7fd8f67fc700 1 -- 172.21.15.2:0/2122890885 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc00f2e0 con 0x7fd90011a490 2023-09-13T04:51:22.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.745+0000 7fd9079ff700 1 -- 172.21.15.2:0/2122890885 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9001a3db0 con 0x7fd90011a490 2023-09-13T04:51:22.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.745+0000 7fd9079ff700 1 -- 172.21.15.2:0/2122890885 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9001aa430 con 0x7fd90011a490 2023-09-13T04:51:22.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.747+0000 7fd9079ff700 1 -- 172.21.15.2:0/2122890885 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd900067ad0 con 0x7fd90011a490 2023-09-13T04:51:22.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.752+0000 7fd8f67fc700 1 -- 172.21.15.2:0/2122890885 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7fd8fc0444d0 con 0x7fd90011a490 2023-09-13T04:51:22.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.754+0000 7fd8f67fc700 1 --2- 172.21.15.2:0/2122890885 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fd8ec0a0a10 0x7fd8ec0a2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:22.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.755+0000 7fd8f67fc700 1 -- 172.21.15.2:0/2122890885 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9483+0+0 (secure 0 0 0) 0x7fd8fc0cbed0 con 0x7fd90011a490 2023-09-13T04:51:22.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.760+0000 7fd8f67fc700 1 -- 172.21.15.2:0/2122890885 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd8fc0963b0 con 0x7fd90011a490 2023-09-13T04:51:22.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.765+0000 7fd90579b700 1 --2- 172.21.15.2:0/2122890885 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fd8ec0a0a10 0x7fd8ec0a2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:22.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:22.767+0000 7fd90579b700 1 --2- 172.21.15.2:0/2122890885 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fd8ec0a0a10 0x7fd8ec0a2ed0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd8f001af50 tx=0x7fd8f0005670 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:23.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.275+0000 7fd9079ff700 1 -- 172.21.15.2:0/2122890885 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9001124b0 con 0x7fd90011a490 2023-09-13T04:51:23.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.283+0000 7fd8f67fc700 1 -- 172.21.15.2:0/2122890885 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+11310 (secure 0 0 0) 0x7fd8fc015020 con 0x7fd90011a490 2023-09-13T04:51:23.288 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:51:23.289 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2478119463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2478119463},{"type":"v1","addr":"172.21.15.2:6835","nonce":2478119463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2469307307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2469307307},{"type":"v1","addr":"172.21.15.2:6839","nonce":2469307307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3368344346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3368344346},{"type":"v1","addr":"172.21.15.149:6833","nonce":3368344346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/4209382763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4209382763},{"type":"v1","addr":"172.21.15.156:6833","nonce":4209382763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/258142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":258142559},{"type":"v1","addr":"172.21.15.2:6837","nonce":258142559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/725547291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":725547291},{"type":"v1","addr":"172.21.15.149:6835","nonce":725547291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T04:37:59.110701+0000","modified":"2023-09-13T04:51:19.325240+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24554,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/846579077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":846579077},{"type":"v1","addr":"172.21.15.156:6835","nonce":846579077}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1896833690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1896833690},{"type":"v1","addr":"172.21.15.149:6837","nonce":1896833690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6839/1785833227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1785833227},{"type":"v1","addr":"172.21.15.156:6839","nonce":1785833227}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":136,"addr":"172.21.15.2:6841/482545391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":482545391},{"type":"v1","addr":"172.21.15.2:6841","nonce":482545391}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/389167841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":389167841},{"type":"v1","addr":"172.21.15.149:6839","nonce":389167841}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26764":{"gid":26764,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3447644694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3447644694},{"type":"v1","addr":"172.21.15.156:6837","nonce":3447644694}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T04:51:23.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.296+0000 7fd9079ff700 1 -- 172.21.15.2:0/2122890885 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fd8ec0a0a10 msgr2=0x7fd8ec0a2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:23.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.296+0000 7fd9079ff700 1 --2- 172.21.15.2:0/2122890885 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fd8ec0a0a10 0x7fd8ec0a2ed0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd8f001af50 tx=0x7fd8f0005670 comp rx=0 tx=0).stop 2023-09-13T04:51:23.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.296+0000 7fd9079ff700 1 -- 172.21.15.2:0/2122890885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd90011a490 msgr2=0x7fd9001a31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:23.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.296+0000 7fd9079ff700 1 --2- 172.21.15.2:0/2122890885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd90011a490 0x7fd9001a31a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc00b450 tx=0x7fd8fc00b810 comp rx=0 tx=0).stop 2023-09-13T04:51:23.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.298+0000 7fd9079ff700 1 -- 172.21.15.2:0/2122890885 shutdown_connections 2023-09-13T04:51:23.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.298+0000 7fd9079ff700 1 --2- 172.21.15.2:0/2122890885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd90011a490 0x7fd9001a31a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:23.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.298+0000 7fd9079ff700 1 --2- 172.21.15.2:0/2122890885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9001081f0 0x7fd900115830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:23.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.298+0000 7fd9079ff700 1 --2- 172.21.15.2:0/2122890885 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fd8ec0a0a10 0x7fd8ec0a2ed0 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd8f001af50 tx=0x7fd8f0005670 comp rx=0 tx=0).stop 2023-09-13T04:51:23.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.298+0000 7fd9079ff700 1 --2- 172.21.15.2:0/2122890885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd90010f440 0x7fd900115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:23.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.298+0000 7fd9079ff700 1 -- 172.21.15.2:0/2122890885 >> 172.21.15.2:0/2122890885 conn(0x7fd90006bd20 msgr2=0x7fd90011a870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:23.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.306+0000 7fd9079ff700 1 -- 172.21.15.2:0/2122890885 shutdown_connections 2023-09-13T04:51:23.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.307+0000 7fd9079ff700 1 -- 172.21.15.2:0/2122890885 wait complete. 2023-09-13T04:51:23.321 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 41 2023-09-13T04:51:23.342 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:23 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:51:23.342 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:23 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:51:23.343 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:23 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1361994929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:51:23.343 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:23 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:23.343 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:23 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:23.343 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:23 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:23.344 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:23 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:23.345 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T04:51:23.391 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:23 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:51:23.391 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:23 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:51:23.391 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:23 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1361994929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:51:23.391 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:23 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:23.392 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:23 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:23.392 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:23 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:23.392 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:23 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:23.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:23 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:51:23.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:23 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:51:23.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:23 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1361994929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:51:23.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:23 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:23.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:23 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:23.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:23 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:23.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:23 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:23.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.809+0000 7f67825ff700 1 -- 172.21.15.2:0/2400703949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f677c10c650 msgr2=0x7f677c1141d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:23.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.809+0000 7f67825ff700 1 --2- 172.21.15.2:0/2400703949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f677c10c650 0x7f677c1141d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f677400b0d0 tx=0x7f677400b3e0 comp rx=0 tx=0).stop 2023-09-13T04:51:23.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.809+0000 7f67825ff700 1 -- 172.21.15.2:0/2400703949 shutdown_connections 2023-09-13T04:51:23.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.809+0000 7f67825ff700 1 --2- 172.21.15.2:0/2400703949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f677c10c650 0x7f677c1141d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:23.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.809+0000 7f67825ff700 1 --2- 172.21.15.2:0/2400703949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f677c107640 0x7f677c107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:23.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.809+0000 7f67825ff700 1 --2- 172.21.15.2:0/2400703949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f677c107ff0 0x7f677c10bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:23.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.809+0000 7f67825ff700 1 -- 172.21.15.2:0/2400703949 >> 172.21.15.2:0/2400703949 conn(0x7f677c076b50 msgr2=0x7f677c105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:23.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.813+0000 7f67825ff700 1 -- 172.21.15.2:0/2400703949 shutdown_connections 2023-09-13T04:51:23.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.814+0000 7f67825ff700 1 -- 172.21.15.2:0/2400703949 wait complete. 2023-09-13T04:51:23.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.815+0000 7f67825ff700 1 Processor -- start 2023-09-13T04:51:23.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.815+0000 7f67825ff700 1 -- start start 2023-09-13T04:51:23.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.816+0000 7f67825ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f677c107640 0x7f677c07b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:23.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.816+0000 7f67825ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f677c10c650 0x7f677c07b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:23.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.816+0000 7f67825ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f677c07fa20 0x7f677c07fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:23.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.816+0000 7f67825ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c0804a0 con 0x7f677c10c650 2023-09-13T04:51:23.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.816+0000 7f67825ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c0805e0 con 0x7f677c07fa20 2023-09-13T04:51:23.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.816+0000 7f67825ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c0807a0 con 0x7f677c107640 2023-09-13T04:51:23.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.816+0000 7f677bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f677c107640 0x7f677c07b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:23.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.816+0000 7f677bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f677c107640 0x7f677c07b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:43446/0 (socket says 172.21.15.2:43446) 2023-09-13T04:51:23.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.816+0000 7f677bfff700 1 -- 172.21.15.2:0/789893824 learned_addr learned my addr 172.21.15.2:0/789893824 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:51:23.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.817+0000 7f6780b9c700 1 --2- 172.21.15.2:0/789893824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f677c07fa20 0x7f677c07fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:23.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.818+0000 7f677b7fe700 1 --2- 172.21.15.2:0/789893824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f677c10c650 0x7f677c07b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:23.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.819+0000 7f677b7fe700 1 -- 172.21.15.2:0/789893824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f677c107640 msgr2=0x7f677c07b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:23.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.819+0000 7f677b7fe700 1 --2- 172.21.15.2:0/789893824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f677c107640 0x7f677c07b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:23.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.819+0000 7f677b7fe700 1 -- 172.21.15.2:0/789893824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f677c07fa20 msgr2=0x7f677c07fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:23.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.819+0000 7f677b7fe700 1 --2- 172.21.15.2:0/789893824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f677c07fa20 0x7f677c07fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:23.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.819+0000 7f677b7fe700 1 -- 172.21.15.2:0/789893824 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6774009d00 con 0x7f677c10c650 2023-09-13T04:51:23.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.820+0000 7f677b7fe700 1 --2- 172.21.15.2:0/789893824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f677c10c650 0x7f677c07b5c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f677c06a240 tx=0x7f676c00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:23.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.822+0000 7f67797fa700 1 -- 172.21.15.2:0/789893824 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c033640 con 0x7f677c10c650 2023-09-13T04:51:23.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.822+0000 7f67797fa700 1 -- 172.21.15.2:0/789893824 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f676c033c60 con 0x7f677c10c650 2023-09-13T04:51:23.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.822+0000 7f67797fa700 1 -- 172.21.15.2:0/789893824 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c032c30 con 0x7f677c10c650 2023-09-13T04:51:23.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.822+0000 7f67825ff700 1 -- 172.21.15.2:0/789893824 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f677c06a6d0 con 0x7f677c10c650 2023-09-13T04:51:23.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.822+0000 7f67825ff700 1 -- 172.21.15.2:0/789893824 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f677c06ab90 con 0x7f677c10c650 2023-09-13T04:51:23.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.824+0000 7f67825ff700 1 -- 172.21.15.2:0/789893824 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6768005320 con 0x7f677c10c650 2023-09-13T04:51:23.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.829+0000 7f67797fa700 1 -- 172.21.15.2:0/789893824 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f676c032d90 con 0x7f677c10c650 2023-09-13T04:51:23.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.831+0000 7f67797fa700 1 --2- 172.21.15.2:0/789893824 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f67640a0a10 0x7f67640a2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:23.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.831+0000 7f67797fa700 1 -- 172.21.15.2:0/789893824 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9483+0+0 (secure 0 0 0) 0x7f676c0ed9a0 con 0x7f677c10c650 2023-09-13T04:51:23.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.839+0000 7f677bfff700 1 --2- 172.21.15.2:0/789893824 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f67640a0a10 0x7f67640a2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:23.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.842+0000 7f67797fa700 1 -- 172.21.15.2:0/789893824 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f676c0b6e80 con 0x7f677c10c650 2023-09-13T04:51:23.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:23.854+0000 7f677bfff700 1 --2- 172.21.15.2:0/789893824 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f67640a0a10 0x7f67640a2ed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6770009ad0 tx=0x7f6770009290 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:24.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.272+0000 7f67825ff700 1 -- 172.21.15.2:0/789893824 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6768005190 con 0x7f677c10c650 2023-09-13T04:51:24.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.276+0000 7f67797fa700 1 -- 172.21.15.2:0/789893824 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+25195 (secure 0 0 0) 0x7f676c0ba4a0 con 0x7f677c10c650 2023-09-13T04:51:24.277 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-09-13T04:51:24.278 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":105,"fsid":"22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da","created":"2023-09-13T04:30:22.164674+0000","modified":"2023-09-13T04:51:07.935860+0000","last_up_change":"2023-09-13T04:37:09.564118+0000","last_in_change":"2023-09-13T04:36:52.914559+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T04:33:42.025714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T04:37:57.094802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T04:37:58.564487+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-13T04:38:02.096895+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c9a443bc-d0b1-41ea-8409-e0cccb0116ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6803","nonce":1800239506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6805","nonce":1800239506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6809","nonce":1800239506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1800239506},{"type":"v1","addr":"172.21.15.2:6807","nonce":1800239506}]},"public_addr":"172.21.15.2:6803/1800239506","cluster_addr":"172.21.15.2:6805/1800239506","heartbeat_back_addr":"172.21.15.2:6809/1800239506","heartbeat_front_addr":"172.21.15.2:6807/1800239506","state":["exists","up"]},{"osd":1,"uuid":"620d0f1a-56e1-4edd-afa4-100d96afd043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6801","nonce":1737673954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6803","nonce":1737673954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6807","nonce":1737673954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1737673954},{"type":"v1","addr":"172.21.15.149:6805","nonce":1737673954}]},"public_addr":"172.21.15.149:6801/1737673954","cluster_addr":"172.21.15.149:6803/1737673954","heartbeat_back_addr":"172.21.15.149:6807/1737673954","heartbeat_front_addr":"172.21.15.149:6805/1737673954","state":["exists","up"]},{"osd":2,"uuid":"9ee5587a-dd0b-42b0-ae9a-a717ae6afad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6801","nonce":3090983787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6803","nonce":3090983787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6807","nonce":3090983787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3090983787},{"type":"v1","addr":"172.21.15.156:6805","nonce":3090983787}]},"public_addr":"172.21.15.156:6801/3090983787","cluster_addr":"172.21.15.156:6803/3090983787","heartbeat_back_addr":"172.21.15.156:6807/3090983787","heartbeat_front_addr":"172.21.15.156:6805/3090983787","state":["exists","up"]},{"osd":3,"uuid":"93397693-f57f-4e1a-a1f8-10e3651e5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6811","nonce":3764424070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6813","nonce":3764424070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6817","nonce":3764424070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3764424070},{"type":"v1","addr":"172.21.15.2:6815","nonce":3764424070}]},"public_addr":"172.21.15.2:6811/3764424070","cluster_addr":"172.21.15.2:6813/3764424070","heartbeat_back_addr":"172.21.15.2:6817/3764424070","heartbeat_front_addr":"172.21.15.2:6815/3764424070","state":["exists","up"]},{"osd":4,"uuid":"09ed1cc4-cce9-454b-bced-dbe54d622f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6809","nonce":3095161596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6811","nonce":3095161596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6815","nonce":3095161596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3095161596},{"type":"v1","addr":"172.21.15.149:6813","nonce":3095161596}]},"public_addr":"172.21.15.149:6809/3095161596","cluster_addr":"172.21.15.149:6811/3095161596","heartbeat_back_addr":"172.21.15.149:6815/3095161596","heartbeat_front_addr":"172.21.15.149:6813/3095161596","state":["exists","up"]},{"osd":5,"uuid":"2ede570f-f574-4d3f-934d-245afd175128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6809","nonce":1709817635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6811","nonce":1709817635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6815","nonce":1709817635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1709817635},{"type":"v1","addr":"172.21.15.156:6813","nonce":1709817635}]},"public_addr":"172.21.15.156:6809/1709817635","cluster_addr":"172.21.15.156:6811/1709817635","heartbeat_back_addr":"172.21.15.156:6815/1709817635","heartbeat_front_addr":"172.21.15.156:6813/1709817635","state":["exists","up"]},{"osd":6,"uuid":"46e6e453-ced8-44b7-8ec7-e38497f8d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6819","nonce":3548993414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6821","nonce":3548993414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6825","nonce":3548993414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":3548993414},{"type":"v1","addr":"172.21.15.2:6823","nonce":3548993414}]},"public_addr":"172.21.15.2:6819/3548993414","cluster_addr":"172.21.15.2:6821/3548993414","heartbeat_back_addr":"172.21.15.2:6825/3548993414","heartbeat_front_addr":"172.21.15.2:6823/3548993414","state":["exists","up"]},{"osd":7,"uuid":"8d82c256-1078-4815-83d9-6fcdb98746ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6817","nonce":3932319846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6819","nonce":3932319846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6823","nonce":3932319846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3932319846},{"type":"v1","addr":"172.21.15.149:6821","nonce":3932319846}]},"public_addr":"172.21.15.149:6817/3932319846","cluster_addr":"172.21.15.149:6819/3932319846","heartbeat_back_addr":"172.21.15.149:6823/3932319846","heartbeat_front_addr":"172.21.15.149:6821/3932319846","state":["exists","up"]},{"osd":8,"uuid":"fbb7a301-2a19-4239-a27d-d07410dd84c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6817","nonce":1139967383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6819","nonce":1139967383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6823","nonce":1139967383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1139967383},{"type":"v1","addr":"172.21.15.156:6821","nonce":1139967383}]},"public_addr":"172.21.15.156:6817/1139967383","cluster_addr":"172.21.15.156:6819/1139967383","heartbeat_back_addr":"172.21.15.156:6823/1139967383","heartbeat_front_addr":"172.21.15.156:6821/1139967383","state":["exists","up"]},{"osd":9,"uuid":"eb50721f-5036-47d4-89f8-0fa09538266d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6827","nonce":3097876180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6829","nonce":3097876180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6833","nonce":3097876180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3097876180},{"type":"v1","addr":"172.21.15.2:6831","nonce":3097876180}]},"public_addr":"172.21.15.2:6827/3097876180","cluster_addr":"172.21.15.2:6829/3097876180","heartbeat_back_addr":"172.21.15.2:6833/3097876180","heartbeat_front_addr":"172.21.15.2:6831/3097876180","state":["exists","up"]},{"osd":10,"uuid":"23ee78a4-63ef-48b5-9383-046730f45bf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6825","nonce":3663849505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6827","nonce":3663849505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6831","nonce":3663849505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3663849505},{"type":"v1","addr":"172.21.15.149:6829","nonce":3663849505}]},"public_addr":"172.21.15.149:6825/3663849505","cluster_addr":"172.21.15.149:6827/3663849505","heartbeat_back_addr":"172.21.15.149:6831/3663849505","heartbeat_front_addr":"172.21.15.149:6829/3663849505","state":["exists","up"]},{"osd":11,"uuid":"41b43893-e5cc-422c-9efd-0f25edbf5d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6825","nonce":215467467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6827","nonce":215467467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6831","nonce":215467467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":215467467},{"type":"v1","addr":"172.21.15.156:6829","nonce":215467467}]},"public_addr":"172.21.15.156:6825/215467467","cluster_addr":"172.21.15.156:6827/215467467","heartbeat_back_addr":"172.21.15.156:6831/215467467","heartbeat_front_addr":"172.21.15.156:6829/215467467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:32:56.695478+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:18.668453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:33:39.329563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:05.135889+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:27.494300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:34:48.809265+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:14.843452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:37.667796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:35:58.675192+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:25.115146+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:36:47.431996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T04:37:08.449261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/1461251021":"2023-09-14T04:40:27.774561+0000","172.21.15.2:6800/1461251021":"2023-09-14T04:40:27.774561+0000","172.21.15.2:0/2528007246":"2023-09-14T04:40:27.774561+0000","172.21.15.2:0/2955382590":"2023-09-14T04:40:27.774561+0000","172.21.15.2:0/227933483":"2023-09-14T04:40:27.774561+0000","172.21.15.2:6801/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6801/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3810974822":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/2772434518":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/1907805835":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/2811866029":"2023-09-14T04:30:54.963309+0000","172.21.15.2:0/3568881483":"2023-09-14T04:40:27.774561+0000","172.21.15.2:0/2538614772":"2023-09-14T04:31:23.417691+0000","172.21.15.2:6800/1393377368":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/3722465120":"2023-09-14T04:31:23.417691+0000","172.21.15.2:0/516951274":"2023-09-14T04:30:54.963309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T04:51:24.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.296+0000 7f67825ff700 1 -- 172.21.15.2:0/789893824 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f67640a0a10 msgr2=0x7f67640a2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:24.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.296+0000 7f67825ff700 1 --2- 172.21.15.2:0/789893824 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f67640a0a10 0x7f67640a2ed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6770009ad0 tx=0x7f6770009290 comp rx=0 tx=0).stop 2023-09-13T04:51:24.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.296+0000 7f67825ff700 1 -- 172.21.15.2:0/789893824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f677c10c650 msgr2=0x7f677c07b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:24.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.296+0000 7f67825ff700 1 --2- 172.21.15.2:0/789893824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f677c10c650 0x7f677c07b5c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f677c06a240 tx=0x7f676c00b730 comp rx=0 tx=0).stop 2023-09-13T04:51:24.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.297+0000 7f67825ff700 1 -- 172.21.15.2:0/789893824 shutdown_connections 2023-09-13T04:51:24.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.297+0000 7f67825ff700 1 --2- 172.21.15.2:0/789893824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f677c07fa20 0x7f677c07fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:24.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.297+0000 7f67825ff700 1 --2- 172.21.15.2:0/789893824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f677c107640 0x7f677c07b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:24.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.297+0000 7f67825ff700 1 --2- 172.21.15.2:0/789893824 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f67640a0a10 0x7f67640a2ed0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:24.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.297+0000 7f67825ff700 1 --2- 172.21.15.2:0/789893824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f677c10c650 0x7f677c07b5c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:24.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.297+0000 7f67825ff700 1 -- 172.21.15.2:0/789893824 >> 172.21.15.2:0/789893824 conn(0x7f677c076b50 msgr2=0x7f677c076f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:24.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.297+0000 7f67825ff700 1 -- 172.21.15.2:0/789893824 shutdown_connections 2023-09-13T04:51:24.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.298+0000 7f67825ff700 1 -- 172.21.15.2:0/789893824 wait complete. 2023-09-13T04:51:24.353 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-13T04:51:24.354 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:24 smithi002 ceph-mon[97651]: pgmap v342: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1011 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2023-09-13T04:51:24.354 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:24 smithi002 ceph-mon[97651]: Adjusting osd_memory_target on smithi156 to 196.2M 2023-09-13T04:51:24.355 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:24 smithi002 ceph-mon[97651]: Unable to set osd_memory_target on smithi156 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-09-13T04:51:24.355 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:24 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/2122890885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:51:24.355 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:24 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:51:24.355 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:24 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:51:24.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:24 smithi149 ceph-mon[133315]: pgmap v342: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1011 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2023-09-13T04:51:24.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:24 smithi149 ceph-mon[133315]: Adjusting osd_memory_target on smithi156 to 196.2M 2023-09-13T04:51:24.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:24 smithi149 ceph-mon[133315]: Unable to set osd_memory_target on smithi156 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-09-13T04:51:24.389 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:24 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/2122890885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:51:24.390 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:24 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:51:24.390 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:24 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:51:24.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:24 smithi156 ceph-mon[133675]: pgmap v342: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1011 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2023-09-13T04:51:24.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:24 smithi156 ceph-mon[133675]: Adjusting osd_memory_target on smithi156 to 196.2M 2023-09-13T04:51:24.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:24 smithi156 ceph-mon[133675]: Unable to set osd_memory_target on smithi156 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-09-13T04:51:24.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:24 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/2122890885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T04:51:24.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:24 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:51:24.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:24 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:51:24.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.804+0000 7f67673ff700 1 -- 172.21.15.2:0/2315708755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f676011a450 msgr2=0x7f676011c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:24.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.804+0000 7f67673ff700 1 --2- 172.21.15.2:0/2315708755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f676011a450 0x7f676011c840 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6754009990 tx=0x7f6754009ca0 comp rx=0 tx=0).stop 2023-09-13T04:51:24.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.809+0000 7f67673ff700 1 -- 172.21.15.2:0/2315708755 shutdown_connections 2023-09-13T04:51:24.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.809+0000 7f67673ff700 1 --2- 172.21.15.2:0/2315708755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f676011a450 0x7f676011c840 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:24.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.809+0000 7f67673ff700 1 --2- 172.21.15.2:0/2315708755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f676006a310 0x7f676006a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:24.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.809+0000 7f67673ff700 1 --2- 172.21.15.2:0/2315708755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f676006ac30 0x7f6760119f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:24.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.809+0000 7f67673ff700 1 -- 172.21.15.2:0/2315708755 >> 172.21.15.2:0/2315708755 conn(0x7f6760072210 msgr2=0x7f6760072620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:24.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.814+0000 7f67673ff700 1 -- 172.21.15.2:0/2315708755 shutdown_connections 2023-09-13T04:51:24.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.815+0000 7f67673ff700 1 -- 172.21.15.2:0/2315708755 wait complete. 2023-09-13T04:51:24.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.817+0000 7f67673ff700 1 Processor -- start 2023-09-13T04:51:24.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.817+0000 7f67673ff700 1 -- start start 2023-09-13T04:51:24.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.818+0000 7f67673ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f676006a310 0x7f67601a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:24.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.818+0000 7f67673ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f676006ac30 0x7f67601a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:24.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.818+0000 7f67673ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f676011a450 0x7f67601aa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:24.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.818+0000 7f67673ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67601aae30 con 0x7f676006a310 2023-09-13T04:51:24.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.818+0000 7f67673ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67601aafa0 con 0x7f676006ac30 2023-09-13T04:51:24.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.818+0000 7f67673ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67601ab160 con 0x7f676011a450 2023-09-13T04:51:24.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.818+0000 7f676499a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f676006ac30 0x7f67601a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:24.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.818+0000 7f676519b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f676006a310 0x7f67601a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:24.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.818+0000 7f676519b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f676006a310 0x7f67601a5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47328/0 (socket says 172.21.15.2:47328) 2023-09-13T04:51:24.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.819+0000 7f676519b700 1 -- 172.21.15.2:0/3575170067 learned_addr learned my addr 172.21.15.2:0/3575170067 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:51:24.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.822+0000 7f676519b700 1 -- 172.21.15.2:0/3575170067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f676011a450 msgr2=0x7f67601aa7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-13T04:51:24.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.822+0000 7f676519b700 1 --2- 172.21.15.2:0/3575170067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f676011a450 0x7f67601aa7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:24.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.822+0000 7f676519b700 1 -- 172.21.15.2:0/3575170067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f676006ac30 msgr2=0x7f67601a6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:24.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.822+0000 7f676519b700 1 --2- 172.21.15.2:0/3575170067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f676006ac30 0x7f67601a6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:24.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.822+0000 7f676519b700 1 -- 172.21.15.2:0/3575170067 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67540095f0 con 0x7f676006a310 2023-09-13T04:51:24.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.822+0000 7f676519b700 1 --2- 172.21.15.2:0/3575170067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f676006a310 0x7f67601a5e50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f675000b840 tx=0x7f675000bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:24.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.830+0000 7f675e7fc700 1 -- 172.21.15.2:0/3575170067 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675000d890 con 0x7f676006a310 2023-09-13T04:51:24.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.830+0000 7f675e7fc700 1 -- 172.21.15.2:0/3575170067 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f675000e430 con 0x7f676006a310 2023-09-13T04:51:24.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.830+0000 7f675e7fc700 1 -- 172.21.15.2:0/3575170067 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6750017660 con 0x7f676006a310 2023-09-13T04:51:24.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.830+0000 7f67673ff700 1 -- 172.21.15.2:0/3575170067 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67601ab440 con 0x7f676006a310 2023-09-13T04:51:24.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.830+0000 7f67673ff700 1 -- 172.21.15.2:0/3575170067 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67600790f0 con 0x7f676006a310 2023-09-13T04:51:24.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.835+0000 7f67673ff700 1 -- 172.21.15.2:0/3575170067 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f674c005320 con 0x7f676006a310 2023-09-13T04:51:24.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.842+0000 7f675e7fc700 1 -- 172.21.15.2:0/3575170067 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f675000d9f0 con 0x7f676006a310 2023-09-13T04:51:24.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.844+0000 7f675e7fc700 1 --2- 172.21.15.2:0/3575170067 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f67480a09c0 0x7f67480a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:24.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.844+0000 7f675e7fc700 1 -- 172.21.15.2:0/3575170067 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9483+0+0 (secure 0 0 0) 0x7f67500cc1f0 con 0x7f676006a310 2023-09-13T04:51:24.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.852+0000 7f675e7fc700 1 -- 172.21.15.2:0/3575170067 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6750095640 con 0x7f676006a310 2023-09-13T04:51:24.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.854+0000 7f676499a700 1 --2- 172.21.15.2:0/3575170067 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f67480a09c0 0x7f67480a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:24.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:24.862+0000 7f676499a700 1 --2- 172.21.15.2:0/3575170067 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f67480a09c0 0x7f67480a2e80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f67601a7460 tx=0x7f6758008040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:25.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.373+0000 7f67673ff700 1 -- 172.21.15.2:0/3575170067 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f674c005190 con 0x7f676006a310 2023-09-13T04:51:25.380 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:25 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/789893824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:51:25.380 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:25 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:25.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:25 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/789893824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:51:25.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:25 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:25.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.418+0000 7f675e7fc700 1 -- 172.21.15.2:0/3575170067 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6750098c60 con 0x7f676006a310 2023-09-13T04:51:25.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:25 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/789893824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T04:51:25.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:25 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:25.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.439+0000 7f67673ff700 1 -- 172.21.15.2:0/3575170067 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f67480a09c0 msgr2=0x7f67480a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:25.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.439+0000 7f67673ff700 1 --2- 172.21.15.2:0/3575170067 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f67480a09c0 0x7f67480a2e80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f67601a7460 tx=0x7f6758008040 comp rx=0 tx=0).stop 2023-09-13T04:51:25.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.439+0000 7f67673ff700 1 -- 172.21.15.2:0/3575170067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f676006a310 msgr2=0x7f67601a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:25.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.439+0000 7f67673ff700 1 --2- 172.21.15.2:0/3575170067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f676006a310 0x7f67601a5e50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f675000b840 tx=0x7f675000bc00 comp rx=0 tx=0).stop 2023-09-13T04:51:25.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.444+0000 7f67673ff700 1 -- 172.21.15.2:0/3575170067 shutdown_connections 2023-09-13T04:51:25.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.444+0000 7f67673ff700 1 --2- 172.21.15.2:0/3575170067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f676011a450 0x7f67601aa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:25.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.444+0000 7f67673ff700 1 --2- 172.21.15.2:0/3575170067 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f67480a09c0 0x7f67480a2e80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:25.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.444+0000 7f67673ff700 1 --2- 172.21.15.2:0/3575170067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f676006a310 0x7f67601a5e50 secure :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f675000b840 tx=0x7f675000bc00 comp rx=0 tx=0).stop 2023-09-13T04:51:25.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.444+0000 7f67673ff700 1 --2- 172.21.15.2:0/3575170067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f676006ac30 0x7f67601a6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:25.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.444+0000 7f67673ff700 1 -- 172.21.15.2:0/3575170067 >> 172.21.15.2:0/3575170067 conn(0x7f6760072210 msgr2=0x7f676011a890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:25.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.453+0000 7f67673ff700 1 -- 172.21.15.2:0/3575170067 shutdown_connections 2023-09-13T04:51:25.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.463+0000 7f67673ff700 1 -- 172.21.15.2:0/3575170067 wait complete. 2023-09-13T04:51:25.475 INFO:teuthology.orchestra.run.smithi002.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-13T04:51:25.504 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-13T04:51:25.665 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:25 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mon-a[97627]: 2023-09-13T04:51:25.374+0000 7fbd26538700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-13T04:51:25.666 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:51:25 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: ignoring --setuser ceph since I am not root 2023-09-13T04:51:25.666 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:51:25 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: ignoring --setgroup ceph since I am not root 2023-09-13T04:51:25.666 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:51:25 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: starting mds.a at 2023-09-13T04:51:25.887 INFO:journalctl@ceph.mds.b.smithi149.stdout:Sep 13 04:51:25 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-b[169954]: ignoring --setuser ceph since I am not root 2023-09-13T04:51:25.887 INFO:journalctl@ceph.mds.b.smithi149.stdout:Sep 13 04:51:25 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-b[169954]: ignoring --setgroup ceph since I am not root 2023-09-13T04:51:25.887 INFO:journalctl@ceph.mds.b.smithi149.stdout:Sep 13 04:51:25 smithi149 ceph-mds[169978]: -- 172.21.15.149:0/1470017352 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557d62b0f1e0 con 0x557d62ae5000 2023-09-13T04:51:25.887 INFO:journalctl@ceph.mds.b.smithi149.stdout:Sep 13 04:51:25 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-b[169954]: starting mds.b at 2023-09-13T04:51:25.888 INFO:journalctl@ceph.mds.h.smithi149.stdout:Sep 13 04:51:25 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-h[169606]: ignoring --setuser ceph since I am not root 2023-09-13T04:51:25.888 INFO:journalctl@ceph.mds.h.smithi149.stdout:Sep 13 04:51:25 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-h[169606]: ignoring --setgroup ceph since I am not root 2023-09-13T04:51:25.888 INFO:journalctl@ceph.mds.h.smithi149.stdout:Sep 13 04:51:25 smithi149 ceph-mds[169630]: -- 172.21.15.149:0/13979524 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5629435bf1e0 con 0x562943594c00 2023-09-13T04:51:25.888 INFO:journalctl@ceph.mds.h.smithi149.stdout:Sep 13 04:51:25 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-h[169606]: starting mds.h at 2023-09-13T04:51:25.936 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 13 04:51:25 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-i[172038]: ignoring --setuser ceph since I am not root 2023-09-13T04:51:25.937 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 13 04:51:25 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-i[172038]: ignoring --setgroup ceph since I am not root 2023-09-13T04:51:25.937 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 13 04:51:25 smithi156 ceph-mds[172063]: -- 172.21.15.156:0/969785957 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5584a900b1e0 con 0x5584a8fe1000 2023-09-13T04:51:25.937 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 13 04:51:25 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-i[172038]: starting mds.i at 2023-09-13T04:51:25.937 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 13 04:51:25 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-f[171361]: ignoring --setuser ceph since I am not root 2023-09-13T04:51:25.938 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 13 04:51:25 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-f[171361]: ignoring --setgroup ceph since I am not root 2023-09-13T04:51:25.938 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 13 04:51:25 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-f[171361]: starting mds.f at 2023-09-13T04:51:25.938 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:51:25 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[177386]: ignoring --setuser ceph since I am not root 2023-09-13T04:51:25.939 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:51:25 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[177386]: ignoring --setgroup ceph since I am not root 2023-09-13T04:51:25.939 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:51:25 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[177386]: starting mds.c at 2023-09-13T04:51:25.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.936+0000 7f05717ff700 1 -- 172.21.15.2:0/1252242888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f056c1081f0 msgr2=0x7f056c108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:25.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.936+0000 7f05717ff700 1 --2- 172.21.15.2:0/1252242888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f056c1081f0 0x7f056c108670 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f055c0098f0 tx=0x7f055c009c00 comp rx=0 tx=0).stop 2023-09-13T04:51:25.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.941+0000 7f05717ff700 1 -- 172.21.15.2:0/1252242888 shutdown_connections 2023-09-13T04:51:25.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.941+0000 7f05717ff700 1 --2- 172.21.15.2:0/1252242888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f056c11a490 0x7f056c11c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:25.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.941+0000 7f05717ff700 1 --2- 172.21.15.2:0/1252242888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f056c10f440 0x7f056c107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:25.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.941+0000 7f05717ff700 1 --2- 172.21.15.2:0/1252242888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f056c1081f0 0x7f056c108670 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:25.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.941+0000 7f05717ff700 1 -- 172.21.15.2:0/1252242888 >> 172.21.15.2:0/1252242888 conn(0x7f056c06cfa0 msgr2=0x7f056c06d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:25.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.945+0000 7f05717ff700 1 -- 172.21.15.2:0/1252242888 shutdown_connections 2023-09-13T04:51:25.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.947+0000 7f05717ff700 1 -- 172.21.15.2:0/1252242888 wait complete. 2023-09-13T04:51:25.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.948+0000 7f05717ff700 1 Processor -- start 2023-09-13T04:51:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.949+0000 7f05717ff700 1 -- start start 2023-09-13T04:51:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.949+0000 7f05717ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f056c1081f0 0x7f056c119720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.949+0000 7f05717ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f056c10f440 0x7f056c119c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.949+0000 7f05717ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f056c11a490 0x7f056c113700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:25.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.949+0000 7f05717ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c113e10 con 0x7f056c10f440 2023-09-13T04:51:25.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.949+0000 7f05717ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c113f80 con 0x7f056c11a490 2023-09-13T04:51:25.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.949+0000 7f05717ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c114140 con 0x7f056c1081f0 2023-09-13T04:51:25.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.951+0000 7f056affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f056c1081f0 0x7f056c119720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:25.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.951+0000 7f056b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f056c11a490 0x7f056c113700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:25.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.951+0000 7f056affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f056c1081f0 0x7f056c119720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.2:53106/0 (socket says 172.21.15.2:53106) 2023-09-13T04:51:25.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.951+0000 7f056affd700 1 -- 172.21.15.2:0/1294575240 learned_addr learned my addr 172.21.15.2:0/1294575240 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:51:25.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.951+0000 7f056a7fc700 1 --2- 172.21.15.2:0/1294575240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f056c10f440 0x7f056c119c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:25.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.951+0000 7f056affd700 1 -- 172.21.15.2:0/1294575240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f056c11a490 msgr2=0x7f056c113700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:25.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.951+0000 7f056affd700 1 --2- 172.21.15.2:0/1294575240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f056c11a490 0x7f056c113700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:25.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.951+0000 7f056affd700 1 -- 172.21.15.2:0/1294575240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f056c10f440 msgr2=0x7f056c119c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:25.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.951+0000 7f056affd700 1 --2- 172.21.15.2:0/1294575240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f056c10f440 0x7f056c119c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:25.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.951+0000 7f056affd700 1 -- 172.21.15.2:0/1294575240 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0564009d00 con 0x7f056c1081f0 2023-09-13T04:51:25.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.952+0000 7f056affd700 1 --2- 172.21.15.2:0/1294575240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f056c1081f0 0x7f056c119720 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f056000b530 tx=0x7f056000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:25.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.953+0000 7f0553fff700 1 -- 172.21.15.2:0/1294575240 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0560011830 con 0x7f056c1081f0 2023-09-13T04:51:25.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.953+0000 7f0553fff700 1 -- 172.21.15.2:0/1294575240 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0560011e50 con 0x7f056c1081f0 2023-09-13T04:51:25.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.953+0000 7f0553fff700 1 -- 172.21.15.2:0/1294575240 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056000f2e0 con 0x7f056c1081f0 2023-09-13T04:51:25.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.954+0000 7f05717ff700 1 -- 172.21.15.2:0/1294575240 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f055c009520 con 0x7f056c1081f0 2023-09-13T04:51:25.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.954+0000 7f05717ff700 1 -- 172.21.15.2:0/1294575240 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056c114540 con 0x7f056c1081f0 2023-09-13T04:51:25.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.955+0000 7f05717ff700 1 -- 172.21.15.2:0/1294575240 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056c067a60 con 0x7f056c1081f0 2023-09-13T04:51:25.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.959+0000 7f0553fff700 1 -- 172.21.15.2:0/1294575240 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f05600105a0 con 0x7f056c1081f0 2023-09-13T04:51:25.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.961+0000 7f0553fff700 1 --2- 172.21.15.2:0/1294575240 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f05540a0940 0x7f05540a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:25.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.961+0000 7f0553fff700 1 -- 172.21.15.2:0/1294575240 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9653+0+0 (secure 0 0 0) 0x7f05600cc160 con 0x7f056c1081f0 2023-09-13T04:51:25.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.963+0000 7f056a7fc700 1 --2- 172.21.15.2:0/1294575240 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f05540a0940 0x7f05540a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:25.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.964+0000 7f056a7fc700 1 --2- 172.21.15.2:0/1294575240 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f05540a0940 0x7f05540a2e00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f056c06d650 tx=0x7f055c019040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:25.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:25.966+0000 7f0553fff700 1 -- 172.21.15.2:0/1294575240 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0560095670 con 0x7f056c1081f0 2023-09-13T04:51:26.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:26 smithi149 ceph-mon[133315]: pgmap v343: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1010 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 8 op/s 2023-09-13T04:51:26.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:26 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3575170067' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-13T04:51:26.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:26 smithi149 ceph-mon[133315]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-09-13T04:51:26.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:26 smithi149 ceph-mon[133315]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-13T04:51:26.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:26 smithi149 ceph-mon[133315]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-09-13T04:51:26.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:26 smithi149 ceph-mon[133315]: osdmap e106: 12 total, 12 up, 12 in 2023-09-13T04:51:26.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:26 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3575170067' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-09-13T04:51:26.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:26 smithi149 ceph-mon[133315]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-09-13T04:51:26.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:26 smithi156 ceph-mon[133675]: pgmap v343: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1010 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 8 op/s 2023-09-13T04:51:26.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:26 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3575170067' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-13T04:51:26.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:26 smithi156 ceph-mon[133675]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-09-13T04:51:26.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:26 smithi156 ceph-mon[133675]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-13T04:51:26.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:26 smithi156 ceph-mon[133675]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-09-13T04:51:26.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:26 smithi156 ceph-mon[133675]: osdmap e106: 12 total, 12 up, 12 in 2023-09-13T04:51:26.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:26 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3575170067' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-09-13T04:51:26.438 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:26 smithi156 ceph-mon[133675]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-09-13T04:51:26.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:26.485+0000 7f05717ff700 1 -- 172.21.15.2:0/1294575240 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f056c1142d0 con 0x7f056c1081f0 2023-09-13T04:51:26.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:26 smithi002 ceph-mon[97651]: pgmap v343: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1010 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 8 op/s 2023-09-13T04:51:26.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:26 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3575170067' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-13T04:51:26.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:26 smithi002 ceph-mon[97651]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-09-13T04:51:26.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:26 smithi002 ceph-mon[97651]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-13T04:51:26.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:26 smithi002 ceph-mon[97651]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-09-13T04:51:26.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:26 smithi002 ceph-mon[97651]: osdmap e106: 12 total, 12 up, 12 in 2023-09-13T04:51:26.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:26 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3575170067' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-09-13T04:51:26.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:26 smithi002 ceph-mon[97651]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-09-13T04:51:27.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:27.428+0000 7f0553fff700 1 -- 172.21.15.2:0/1294575240 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0560098c90 con 0x7f056c1081f0 2023-09-13T04:51:27.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:27.438+0000 7f05717ff700 1 -- 172.21.15.2:0/1294575240 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f05540a0940 msgr2=0x7f05540a2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:27.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:27.438+0000 7f05717ff700 1 --2- 172.21.15.2:0/1294575240 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f05540a0940 0x7f05540a2e00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f056c06d650 tx=0x7f055c019040 comp rx=0 tx=0).stop 2023-09-13T04:51:27.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:27.438+0000 7f05717ff700 1 -- 172.21.15.2:0/1294575240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f056c1081f0 msgr2=0x7f056c119720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:27.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:27.438+0000 7f05717ff700 1 --2- 172.21.15.2:0/1294575240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f056c1081f0 0x7f056c119720 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f056000b530 tx=0x7f056000b840 comp rx=0 tx=0).stop 2023-09-13T04:51:27.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:27.447+0000 7f05717ff700 1 -- 172.21.15.2:0/1294575240 shutdown_connections 2023-09-13T04:51:27.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:27.447+0000 7f05717ff700 1 --2- 172.21.15.2:0/1294575240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f056c11a490 0x7f056c113700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:27.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:27.447+0000 7f05717ff700 1 --2- 172.21.15.2:0/1294575240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f056c1081f0 0x7f056c119720 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:27.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:27.447+0000 7f05717ff700 1 --2- 172.21.15.2:0/1294575240 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f05540a0940 0x7f05540a2e00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:27.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:27.447+0000 7f05717ff700 1 --2- 172.21.15.2:0/1294575240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f056c10f440 0x7f056c119c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:27.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:27.447+0000 7f05717ff700 1 -- 172.21.15.2:0/1294575240 >> 172.21.15.2:0/1294575240 conn(0x7f056c06cfa0 msgr2=0x7f056c11ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:27.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:27.460+0000 7f05717ff700 1 -- 172.21.15.2:0/1294575240 shutdown_connections 2023-09-13T04:51:27.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:27.462+0000 7f05717ff700 1 -- 172.21.15.2:0/1294575240 wait complete. 2023-09-13T04:51:27.531 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-13T04:51:27.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:27 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:51:27.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:27 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:51:27.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:27 smithi149 ceph-mon[133315]: mds.? [v2:172.21.15.156:6834/3499265949,v1:172.21.15.156:6836/3499265949] up:boot 2023-09-13T04:51:27.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:27 smithi149 ceph-mon[133315]: mds.? [v2:172.21.15.156:6835/4288627999,v1:172.21.15.156:6837/4288627999] up:boot 2023-09-13T04:51:27.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:27 smithi149 ceph-mon[133315]: mds.? [v2:172.21.15.156:6838/4124734396,v1:172.21.15.156:6839/4124734396] up:boot 2023-09-13T04:51:27.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:27 smithi149 ceph-mon[133315]: mds.? [v2:172.21.15.149:6838/3970804395,v1:172.21.15.149:6839/3970804395] up:boot 2023-09-13T04:51:27.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:27 smithi149 ceph-mon[133315]: mds.? [v2:172.21.15.149:6836/2865164850,v1:172.21.15.149:6837/2865164850] up:boot 2023-09-13T04:51:27.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:27 smithi149 ceph-mon[133315]: mds.? [v2:172.21.15.2:6840/3399650098,v1:172.21.15.2:6841/3399650098] up:boot 2023-09-13T04:51:27.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:27 smithi149 ceph-mon[133315]: fsmap cephfs:0/3 12 up:standby, 3 failed 2023-09-13T04:51:27.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:27 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-13T04:51:27.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:27 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-13T04:51:27.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:27 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-13T04:51:27.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:27 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-13T04:51:27.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:27 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-13T04:51:27.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:27 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-13T04:51:27.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:27 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1294575240' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-13T04:51:27.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:27 smithi149 ceph-mon[133315]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-13T04:51:27.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:27 smithi149 ceph-mon[133315]: pgmap v345: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1010 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 920 B/s wr, 7 op/s 2023-09-13T04:51:27.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:27 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:51:27.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:27 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:51:27.690 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:27 smithi156 ceph-mon[133675]: mds.? [v2:172.21.15.156:6834/3499265949,v1:172.21.15.156:6836/3499265949] up:boot 2023-09-13T04:51:27.690 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:27 smithi156 ceph-mon[133675]: mds.? [v2:172.21.15.156:6835/4288627999,v1:172.21.15.156:6837/4288627999] up:boot 2023-09-13T04:51:27.690 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:27 smithi156 ceph-mon[133675]: mds.? [v2:172.21.15.156:6838/4124734396,v1:172.21.15.156:6839/4124734396] up:boot 2023-09-13T04:51:27.690 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:27 smithi156 ceph-mon[133675]: mds.? [v2:172.21.15.149:6838/3970804395,v1:172.21.15.149:6839/3970804395] up:boot 2023-09-13T04:51:27.691 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:27 smithi156 ceph-mon[133675]: mds.? [v2:172.21.15.149:6836/2865164850,v1:172.21.15.149:6837/2865164850] up:boot 2023-09-13T04:51:27.691 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:27 smithi156 ceph-mon[133675]: mds.? [v2:172.21.15.2:6840/3399650098,v1:172.21.15.2:6841/3399650098] up:boot 2023-09-13T04:51:27.691 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:27 smithi156 ceph-mon[133675]: fsmap cephfs:0/3 12 up:standby, 3 failed 2023-09-13T04:51:27.691 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:27 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-13T04:51:27.691 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:27 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-13T04:51:27.692 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:27 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-13T04:51:27.692 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:27 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-13T04:51:27.692 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:27 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-13T04:51:27.692 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:27 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-13T04:51:27.692 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:27 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1294575240' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-13T04:51:27.693 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:27 smithi156 ceph-mon[133675]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-13T04:51:27.693 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:27 smithi156 ceph-mon[133675]: pgmap v345: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1010 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 920 B/s wr, 7 op/s 2023-09-13T04:51:27.730 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:27 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:51:27.731 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:27 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:51:27.731 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:27 smithi002 ceph-mon[97651]: mds.? [v2:172.21.15.156:6834/3499265949,v1:172.21.15.156:6836/3499265949] up:boot 2023-09-13T04:51:27.731 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:27 smithi002 ceph-mon[97651]: mds.? [v2:172.21.15.156:6835/4288627999,v1:172.21.15.156:6837/4288627999] up:boot 2023-09-13T04:51:27.733 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:27 smithi002 ceph-mon[97651]: mds.? [v2:172.21.15.156:6838/4124734396,v1:172.21.15.156:6839/4124734396] up:boot 2023-09-13T04:51:27.733 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:27 smithi002 ceph-mon[97651]: mds.? [v2:172.21.15.149:6838/3970804395,v1:172.21.15.149:6839/3970804395] up:boot 2023-09-13T04:51:27.733 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:27 smithi002 ceph-mon[97651]: mds.? [v2:172.21.15.149:6836/2865164850,v1:172.21.15.149:6837/2865164850] up:boot 2023-09-13T04:51:27.734 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:27 smithi002 ceph-mon[97651]: mds.? [v2:172.21.15.2:6840/3399650098,v1:172.21.15.2:6841/3399650098] up:boot 2023-09-13T04:51:27.734 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:27 smithi002 ceph-mon[97651]: fsmap cephfs:0/3 12 up:standby, 3 failed 2023-09-13T04:51:27.734 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:27 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-13T04:51:27.734 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:27 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-13T04:51:27.734 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:27 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-13T04:51:27.735 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:27 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-13T04:51:27.735 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:27 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-13T04:51:27.735 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:27 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-13T04:51:27.735 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:27 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1294575240' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-13T04:51:27.735 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:27 smithi002 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-13T04:51:27.736 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:27 smithi002 ceph-mon[97651]: pgmap v345: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1010 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 920 B/s wr, 7 op/s 2023-09-13T04:51:28.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:27.996+0000 7fadd67ff700 1 -- 172.21.15.2:0/119838514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadd010f440 msgr2=0x7fadd0107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:28.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:27.996+0000 7fadd67ff700 1 --2- 172.21.15.2:0/119838514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadd010f440 0x7fadd0107cb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fadc001a9a0 tx=0x7fadc001acb0 comp rx=0 tx=0).stop 2023-09-13T04:51:28.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.002+0000 7fadd67ff700 1 -- 172.21.15.2:0/119838514 shutdown_connections 2023-09-13T04:51:28.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.002+0000 7fadd67ff700 1 --2- 172.21.15.2:0/119838514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadd011a490 0x7fadd011c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:28.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.002+0000 7fadd67ff700 1 --2- 172.21.15.2:0/119838514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadd010f440 0x7fadd0107cb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:28.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.002+0000 7fadd67ff700 1 --2- 172.21.15.2:0/119838514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadd01081f0 0x7fadd0108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:28.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.002+0000 7fadd67ff700 1 -- 172.21.15.2:0/119838514 >> 172.21.15.2:0/119838514 conn(0x7fadd006cfa0 msgr2=0x7fadd006d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:28.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.007+0000 7fadd67ff700 1 -- 172.21.15.2:0/119838514 shutdown_connections 2023-09-13T04:51:28.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.008+0000 7fadd67ff700 1 -- 172.21.15.2:0/119838514 wait complete. 2023-09-13T04:51:28.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.010+0000 7fadd67ff700 1 Processor -- start 2023-09-13T04:51:28.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.011+0000 7fadd67ff700 1 -- start start 2023-09-13T04:51:28.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.011+0000 7fadd67ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadd01081f0 0x7fadd01157b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:28.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.012+0000 7fadd67ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadd010f440 0x7fadd0115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:28.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.012+0000 7fadd67ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadd011a490 0x7fadd01a91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:28.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.012+0000 7fadd67ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd01a9740 con 0x7fadd010f440 2023-09-13T04:51:28.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.012+0000 7fadd67ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd01a98b0 con 0x7fadd01081f0 2023-09-13T04:51:28.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.012+0000 7fadd67ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd01a9a70 con 0x7fadd011a490 2023-09-13T04:51:28.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.012+0000 7fadcffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadd01081f0 0x7fadd01157b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:28.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.012+0000 7fadcffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadd01081f0 0x7fadd01157b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:40506/0 (socket says 172.21.15.2:40506) 2023-09-13T04:51:28.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.012+0000 7fadcffff700 1 -- 172.21.15.2:0/3235382398 learned_addr learned my addr 172.21.15.2:0/3235382398 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:51:28.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.012+0000 7fadd4d9c700 1 --2- 172.21.15.2:0/3235382398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadd011a490 0x7fadd01a91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:28.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.013+0000 7fadd4d9c700 1 -- 172.21.15.2:0/3235382398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadd01081f0 msgr2=0x7fadd01157b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:28.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.013+0000 7fadd4d9c700 1 --2- 172.21.15.2:0/3235382398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadd01081f0 0x7fadd01157b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:28.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.013+0000 7fadd4d9c700 1 -- 172.21.15.2:0/3235382398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadd010f440 msgr2=0x7fadd0115cf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-13T04:51:28.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.013+0000 7fadd4d9c700 1 --2- 172.21.15.2:0/3235382398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadd010f440 0x7fadd0115cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:28.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.013+0000 7fadd4d9c700 1 -- 172.21.15.2:0/3235382398 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc001a600 con 0x7fadd011a490 2023-09-13T04:51:28.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.014+0000 7fadd4d9c700 1 --2- 172.21.15.2:0/3235382398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadd011a490 0x7fadd01a91d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fadc800bdd0 tx=0x7fadc8009e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:28.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.018+0000 7fadcd7fa700 1 -- 172.21.15.2:0/3235382398 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc800cb30 con 0x7fadd011a490 2023-09-13T04:51:28.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.018+0000 7fadcd7fa700 1 -- 172.21.15.2:0/3235382398 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadc800cc90 con 0x7fadd011a490 2023-09-13T04:51:28.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.018+0000 7fadcd7fa700 1 -- 172.21.15.2:0/3235382398 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc80125e0 con 0x7fadd011a490 2023-09-13T04:51:28.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.019+0000 7fadd67ff700 1 -- 172.21.15.2:0/3235382398 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd01a9d50 con 0x7fadd011a490 2023-09-13T04:51:28.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.019+0000 7fadd67ff700 1 -- 172.21.15.2:0/3235382398 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd01aa1e0 con 0x7fadd011a490 2023-09-13T04:51:28.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.023+0000 7fadd67ff700 1 -- 172.21.15.2:0/3235382398 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd0067a60 con 0x7fadd011a490 2023-09-13T04:51:28.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.035+0000 7fadcd7fa700 1 -- 172.21.15.2:0/3235382398 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7fadc8014440 con 0x7fadd011a490 2023-09-13T04:51:28.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.037+0000 7fadcd7fa700 1 --2- 172.21.15.2:0/3235382398 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fadb80a09c0 0x7fadb80a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:28.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.037+0000 7fadcd7fa700 1 -- 172.21.15.2:0/3235382398 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9653+0+0 (secure 0 0 0) 0x7fadc80cbf80 con 0x7fadd011a490 2023-09-13T04:51:28.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.038+0000 7fadcd7fa700 1 -- 172.21.15.2:0/3235382398 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fadc80cc3e0 con 0x7fadd011a490 2023-09-13T04:51:28.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.043+0000 7fadcffff700 1 --2- 172.21.15.2:0/3235382398 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fadb80a09c0 0x7fadb80a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:28.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.046+0000 7fadcffff700 1 --2- 172.21.15.2:0/3235382398 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fadb80a09c0 0x7fadb80a2e80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fadc0003800 tx=0x7fadc002a040 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:28.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:28.457+0000 7fadd67ff700 1 -- 172.21.15.2:0/3235382398 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fadd0116b00 con 0x7fadd011a490 2023-09-13T04:51:28.637 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:28 smithi149 ceph-mon[133315]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-09-13T04:51:28.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:28 smithi149 ceph-mon[133315]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-09-13T04:51:28.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:28 smithi149 ceph-mon[133315]: Cluster is now healthy 2023-09-13T04:51:28.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:28 smithi149 ceph-mon[133315]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-09-13T04:51:28.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:28 smithi149 ceph-mon[133315]: fsmap 12 up:standby 2023-09-13T04:51:28.638 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:28.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:28.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:28.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:28.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:28 smithi149 ceph-mon[133315]: Adjusting osd_memory_target on smithi002 to 222.0M 2023-09-13T04:51:28.639 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:28 smithi149 ceph-mon[133315]: Unable to set osd_memory_target on smithi002 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-09-13T04:51:28.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:51:28.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:51:28.640 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:28 smithi149 ceph-mon[133315]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:51:28.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:28 smithi156 ceph-mon[133675]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-09-13T04:51:28.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:28 smithi156 ceph-mon[133675]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-09-13T04:51:28.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:28 smithi156 ceph-mon[133675]: Cluster is now healthy 2023-09-13T04:51:28.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:28 smithi156 ceph-mon[133675]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-09-13T04:51:28.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:28 smithi156 ceph-mon[133675]: fsmap 12 up:standby 2023-09-13T04:51:28.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:28.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:28.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:28.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:28.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:28 smithi156 ceph-mon[133675]: Adjusting osd_memory_target on smithi002 to 222.0M 2023-09-13T04:51:28.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:28 smithi156 ceph-mon[133675]: Unable to set osd_memory_target on smithi002 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-09-13T04:51:28.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:51:28.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:51:28.689 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:28 smithi156 ceph-mon[133675]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:51:28.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:28 smithi002 ceph-mon[97651]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-09-13T04:51:28.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:28 smithi002 ceph-mon[97651]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-09-13T04:51:28.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:28 smithi002 ceph-mon[97651]: Cluster is now healthy 2023-09-13T04:51:28.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:28 smithi002 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-09-13T04:51:28.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:28 smithi002 ceph-mon[97651]: fsmap 12 up:standby 2023-09-13T04:51:28.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:28.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:28.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:28.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-13T04:51:28.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:28 smithi002 ceph-mon[97651]: Adjusting osd_memory_target on smithi002 to 222.0M 2023-09-13T04:51:28.760 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:28 smithi002 ceph-mon[97651]: Unable to set osd_memory_target on smithi002 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-09-13T04:51:28.760 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T04:51:28.760 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T04:51:28.760 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:28 smithi002 ceph-mon[97651]: from='mgr.15492 172.21.15.2:0/3575232966' entity='mgr.x' 2023-09-13T04:51:29.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.388+0000 7fadcd7fa700 1 -- 172.21.15.2:0/3235382398 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v107) v1 ==== 181+0+0 (secure 0 0 0) 0x7fadc80190d0 con 0x7fadd011a490 2023-09-13T04:51:29.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.394+0000 7fadd67ff700 1 -- 172.21.15.2:0/3235382398 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fadb80a09c0 msgr2=0x7fadb80a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:29.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.394+0000 7fadd67ff700 1 --2- 172.21.15.2:0/3235382398 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fadb80a09c0 0x7fadb80a2e80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fadc0003800 tx=0x7fadc002a040 comp rx=0 tx=0).stop 2023-09-13T04:51:29.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.394+0000 7fadd67ff700 1 -- 172.21.15.2:0/3235382398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadd011a490 msgr2=0x7fadd01a91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:29.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.394+0000 7fadd67ff700 1 --2- 172.21.15.2:0/3235382398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadd011a490 0x7fadd01a91d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fadc800bdd0 tx=0x7fadc8009e20 comp rx=0 tx=0).stop 2023-09-13T04:51:29.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.395+0000 7fadd67ff700 1 -- 172.21.15.2:0/3235382398 shutdown_connections 2023-09-13T04:51:29.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.395+0000 7fadd67ff700 1 --2- 172.21.15.2:0/3235382398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadd011a490 0x7fadd01a91d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:29.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.395+0000 7fadd67ff700 1 --2- 172.21.15.2:0/3235382398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadd01081f0 0x7fadd01157b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:29.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.395+0000 7fadd67ff700 1 --2- 172.21.15.2:0/3235382398 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fadb80a09c0 0x7fadb80a2e80 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fadc0003800 tx=0x7fadc002a040 comp rx=0 tx=0).stop 2023-09-13T04:51:29.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.395+0000 7fadd67ff700 1 --2- 172.21.15.2:0/3235382398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadd010f440 0x7fadd0115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:29.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.395+0000 7fadd67ff700 1 -- 172.21.15.2:0/3235382398 >> 172.21.15.2:0/3235382398 conn(0x7fadd006cfa0 msgr2=0x7fadd011a870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:29.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.399+0000 7fadd67ff700 1 -- 172.21.15.2:0/3235382398 shutdown_connections 2023-09-13T04:51:29.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.401+0000 7fadd67ff700 1 -- 172.21.15.2:0/3235382398 wait complete. 2023-09-13T04:51:29.414 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_metadata' removed 2023-09-13T04:51:29.455 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-13T04:51:29.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:29 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3235382398' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-13T04:51:29.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:29 smithi156 ceph-mon[133675]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-13T04:51:29.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:29 smithi156 ceph-mon[133675]: pgmap v346: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1010 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 920 B/s wr, 7 op/s 2023-09-13T04:51:29.761 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:29 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3235382398' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-13T04:51:29.761 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:29 smithi002 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-13T04:51:29.761 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:29 smithi002 ceph-mon[97651]: pgmap v346: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1010 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 920 B/s wr, 7 op/s 2023-09-13T04:51:29.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:29 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3235382398' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-13T04:51:29.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:29 smithi149 ceph-mon[133315]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-13T04:51:29.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:29 smithi149 ceph-mon[133315]: pgmap v346: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1010 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 920 B/s wr, 7 op/s 2023-09-13T04:51:29.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.882+0000 7fcfd71ff700 1 -- 172.21.15.2:0/4075538448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd01081f0 msgr2=0x7fcfd0108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:29.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.882+0000 7fcfd71ff700 1 --2- 172.21.15.2:0/4075538448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd01081f0 0x7fcfd0108670 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009990 tx=0x7fcfc0009ca0 comp rx=0 tx=0).stop 2023-09-13T04:51:29.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.884+0000 7fcfd71ff700 1 -- 172.21.15.2:0/4075538448 shutdown_connections 2023-09-13T04:51:29.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.884+0000 7fcfd71ff700 1 --2- 172.21.15.2:0/4075538448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfd011a490 0x7fcfd011c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:29.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.884+0000 7fcfd71ff700 1 --2- 172.21.15.2:0/4075538448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfd010f440 0x7fcfd0107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:29.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.884+0000 7fcfd71ff700 1 --2- 172.21.15.2:0/4075538448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd01081f0 0x7fcfd0108670 secure :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009990 tx=0x7fcfc0009ca0 comp rx=0 tx=0).stop 2023-09-13T04:51:29.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.884+0000 7fcfd71ff700 1 -- 172.21.15.2:0/4075538448 >> 172.21.15.2:0/4075538448 conn(0x7fcfd006cfa0 msgr2=0x7fcfd006d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:29.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.887+0000 7fcfd71ff700 1 -- 172.21.15.2:0/4075538448 shutdown_connections 2023-09-13T04:51:29.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.887+0000 7fcfd71ff700 1 -- 172.21.15.2:0/4075538448 wait complete. 2023-09-13T04:51:29.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.888+0000 7fcfd71ff700 1 Processor -- start 2023-09-13T04:51:29.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.892+0000 7fcfd71ff700 1 -- start start 2023-09-13T04:51:29.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.892+0000 7fcfd71ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd010f440 0x7fcfd0118890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:29.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.892+0000 7fcfd71ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfd011a490 0x7fcfd0118e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:29.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.892+0000 7fcfd71ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfd01118f0 0x7fcfd0111d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:29.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.892+0000 7fcfd71ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd0112540 con 0x7fcfd010f440 2023-09-13T04:51:29.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.892+0000 7fcfd71ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd01126b0 con 0x7fcfd011a490 2023-09-13T04:51:29.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.892+0000 7fcfd71ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd0116420 con 0x7fcfd01118f0 2023-09-13T04:51:29.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.893+0000 7fcfcffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfd011a490 0x7fcfd0118e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:29.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.893+0000 7fcfcffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfd011a490 0x7fcfd0118e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:40518/0 (socket says 172.21.15.2:40518) 2023-09-13T04:51:29.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.893+0000 7fcfcffff700 1 -- 172.21.15.2:0/1387063526 learned_addr learned my addr 172.21.15.2:0/1387063526 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:51:29.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.893+0000 7fcfcffff700 1 -- 172.21.15.2:0/1387063526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfd01118f0 msgr2=0x7fcfd0111d70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-13T04:51:29.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.893+0000 7fcfcffff700 1 --2- 172.21.15.2:0/1387063526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfd01118f0 0x7fcfd0111d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:29.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.893+0000 7fcfcffff700 1 -- 172.21.15.2:0/1387063526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd010f440 msgr2=0x7fcfd0118890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:29.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.893+0000 7fcfcffff700 1 --2- 172.21.15.2:0/1387063526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd010f440 0x7fcfd0118890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:29.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.893+0000 7fcfcffff700 1 -- 172.21.15.2:0/1387063526 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc00095f0 con 0x7fcfd011a490 2023-09-13T04:51:29.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.894+0000 7fcfcffff700 1 --2- 172.21.15.2:0/1387063526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfd011a490 0x7fcfd0118e10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0000c00 tx=0x7fcfc00037c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:29.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.895+0000 7fcfcdffb700 1 -- 172.21.15.2:0/1387063526 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc001d070 con 0x7fcfd011a490 2023-09-13T04:51:29.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.895+0000 7fcfcdffb700 1 -- 172.21.15.2:0/1387063526 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfc0003e70 con 0x7fcfd011a490 2023-09-13T04:51:29.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.895+0000 7fcfcdffb700 1 -- 172.21.15.2:0/1387063526 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc000fe90 con 0x7fcfd011a490 2023-09-13T04:51:29.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.895+0000 7fcfd71ff700 1 -- 172.21.15.2:0/1387063526 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd01166a0 con 0x7fcfd011a490 2023-09-13T04:51:29.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.895+0000 7fcfd71ff700 1 -- 172.21.15.2:0/1387063526 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd0116b00 con 0x7fcfd011a490 2023-09-13T04:51:29.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.897+0000 7fcfd71ff700 1 -- 172.21.15.2:0/1387063526 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd0067a60 con 0x7fcfd011a490 2023-09-13T04:51:29.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.908+0000 7fcfcdffb700 1 -- 172.21.15.2:0/1387063526 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7fcfc0022420 con 0x7fcfd011a490 2023-09-13T04:51:29.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.911+0000 7fcfcdffb700 1 --2- 172.21.15.2:0/1387063526 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcfb80a0980 0x7fcfb80a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:29.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.911+0000 7fcfcdffb700 1 -- 172.21.15.2:0/1387063526 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9157+0+0 (secure 0 0 0) 0x7fcfc00ce1b0 con 0x7fcfd011a490 2023-09-13T04:51:29.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.912+0000 7fcfcdffb700 1 -- 172.21.15.2:0/1387063526 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcfc00ce590 con 0x7fcfd011a490 2023-09-13T04:51:29.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.916+0000 7fcfd4f9b700 1 --2- 172.21.15.2:0/1387063526 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcfb80a0980 0x7fcfb80a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:29.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:29.917+0000 7fcfd4f9b700 1 --2- 172.21.15.2:0/1387063526 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcfb80a0980 0x7fcfb80a2e40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800ab90 tx=0x7fcfc8009230 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:30.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:30.330+0000 7fcfd71ff700 1 -- 172.21.15.2:0/1387063526 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcfd0112ec0 con 0x7fcfd011a490 2023-09-13T04:51:30.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:30 smithi156 ceph-mon[133675]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-09-13T04:51:30.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:30 smithi156 ceph-mon[133675]: osdmap e107: 12 total, 12 up, 12 in 2023-09-13T04:51:30.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:30 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/1387063526' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-13T04:51:30.688 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:30 smithi156 ceph-mon[133675]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-13T04:51:30.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:30 smithi002 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-09-13T04:51:30.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:30 smithi002 ceph-mon[97651]: osdmap e107: 12 total, 12 up, 12 in 2023-09-13T04:51:30.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:30 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/1387063526' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-13T04:51:30.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:30 smithi002 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-13T04:51:30.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:30 smithi149 ceph-mon[133315]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-09-13T04:51:30.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:30 smithi149 ceph-mon[133315]: osdmap e107: 12 total, 12 up, 12 in 2023-09-13T04:51:30.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:30 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/1387063526' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-13T04:51:30.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:30 smithi149 ceph-mon[133315]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-13T04:51:30.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:30.954+0000 7fcfcdffb700 1 -- 172.21.15.2:0/1387063526 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v108) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcfc00977d0 con 0x7fcfd011a490 2023-09-13T04:51:30.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:30.961+0000 7fcfd71ff700 1 -- 172.21.15.2:0/1387063526 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcfb80a0980 msgr2=0x7fcfb80a2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:30.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:30.961+0000 7fcfd71ff700 1 --2- 172.21.15.2:0/1387063526 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcfb80a0980 0x7fcfb80a2e40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800ab90 tx=0x7fcfc8009230 comp rx=0 tx=0).stop 2023-09-13T04:51:30.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:30.961+0000 7fcfd71ff700 1 -- 172.21.15.2:0/1387063526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfd011a490 msgr2=0x7fcfd0118e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:30.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:30.961+0000 7fcfd71ff700 1 --2- 172.21.15.2:0/1387063526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfd011a490 0x7fcfd0118e10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0000c00 tx=0x7fcfc00037c0 comp rx=0 tx=0).stop 2023-09-13T04:51:30.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:30.965+0000 7fcfd71ff700 1 -- 172.21.15.2:0/1387063526 shutdown_connections 2023-09-13T04:51:30.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:30.965+0000 7fcfd71ff700 1 --2- 172.21.15.2:0/1387063526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfd01118f0 0x7fcfd0111d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:30.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:30.965+0000 7fcfd71ff700 1 --2- 172.21.15.2:0/1387063526 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7fcfb80a0980 0x7fcfb80a2e40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:30.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:30.965+0000 7fcfd71ff700 1 --2- 172.21.15.2:0/1387063526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd010f440 0x7fcfd0118890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:30.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:30.965+0000 7fcfd71ff700 1 --2- 172.21.15.2:0/1387063526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfd011a490 0x7fcfd0118e10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:30.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:30.965+0000 7fcfd71ff700 1 -- 172.21.15.2:0/1387063526 >> 172.21.15.2:0/1387063526 conn(0x7fcfd006cfa0 msgr2=0x7fcfd006f670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:30.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:30.966+0000 7fcfd71ff700 1 -- 172.21.15.2:0/1387063526 shutdown_connections 2023-09-13T04:51:30.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:30.966+0000 7fcfd71ff700 1 -- 172.21.15.2:0/1387063526 wait complete. 2023-09-13T04:51:30.972 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data' removed 2023-09-13T04:51:31.005 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-09-13T04:51:31.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.416+0000 7f5ff01ff700 1 -- 172.21.15.2:0/2117547098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fe80ffbf0 msgr2=0x7f5fe80fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:31.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.416+0000 7f5ff01ff700 1 --2- 172.21.15.2:0/2117547098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fe80ffbf0 0x7f5fe80fffd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000b0a0 tx=0x7f5fe000b3b0 comp rx=0 tx=0).stop 2023-09-13T04:51:31.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.417+0000 7f5ff01ff700 1 -- 172.21.15.2:0/2117547098 shutdown_connections 2023-09-13T04:51:31.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.417+0000 7f5ff01ff700 1 --2- 172.21.15.2:0/2117547098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5fe8104ff0 0x7f5fe81054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:31.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.417+0000 7f5ff01ff700 1 --2- 172.21.15.2:0/2117547098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fe80ffbf0 0x7f5fe80fffd0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:31.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.417+0000 7f5ff01ff700 1 --2- 172.21.15.2:0/2117547098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fe8100510 0x7f5fe81049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:31.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.417+0000 7f5ff01ff700 1 -- 172.21.15.2:0/2117547098 >> 172.21.15.2:0/2117547098 conn(0x7f5fe80fb620 msgr2=0x7f5fe80fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:31.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.420+0000 7f5ff01ff700 1 -- 172.21.15.2:0/2117547098 shutdown_connections 2023-09-13T04:51:31.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.423+0000 7f5ff01ff700 1 -- 172.21.15.2:0/2117547098 wait complete. 2023-09-13T04:51:31.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.424+0000 7f5ff01ff700 1 Processor -- start 2023-09-13T04:51:31.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.427+0000 7f5ff01ff700 1 -- start start 2023-09-13T04:51:31.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.428+0000 7f5ff01ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fe80ffbf0 0x7f5fe819d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:31.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.428+0000 7f5ff01ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5fe8100510 0x7f5fe819dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:31.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.428+0000 7f5ff01ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fe8104ff0 0x7f5fe81a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:31.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.428+0000 7f5ff01ff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe81a2560 con 0x7f5fe8100510 2023-09-13T04:51:31.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.428+0000 7f5ff01ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe81a26d0 con 0x7f5fe80ffbf0 2023-09-13T04:51:31.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.428+0000 7f5ff01ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe81a2890 con 0x7f5fe8104ff0 2023-09-13T04:51:31.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.429+0000 7f5fedf9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fe80ffbf0 0x7f5fe819d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:31.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.429+0000 7f5fedf9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fe80ffbf0 0x7f5fe819d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:40544/0 (socket says 172.21.15.2:40544) 2023-09-13T04:51:31.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.429+0000 7f5fedf9b700 1 -- 172.21.15.2:0/3680391126 learned_addr learned my addr 172.21.15.2:0/3680391126 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-09-13T04:51:31.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.429+0000 7f5fed79a700 1 --2- 172.21.15.2:0/3680391126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5fe8100510 0x7f5fe819dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:31.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.430+0000 7f5fed79a700 1 -- 172.21.15.2:0/3680391126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fe8104ff0 msgr2=0x7f5fe81a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:31.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.430+0000 7f5fed79a700 1 --2- 172.21.15.2:0/3680391126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fe8104ff0 0x7f5fe81a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:31.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.430+0000 7f5fed79a700 1 -- 172.21.15.2:0/3680391126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fe80ffbf0 msgr2=0x7f5fe819d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:31.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.430+0000 7f5fed79a700 1 --2- 172.21.15.2:0/3680391126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fe80ffbf0 0x7f5fe819d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:31.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.430+0000 7f5fed79a700 1 -- 172.21.15.2:0/3680391126 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe0009d00 con 0x7f5fe8100510 2023-09-13T04:51:31.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.430+0000 7f5fed79a700 1 --2- 172.21.15.2:0/3680391126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5fe8100510 0x7f5fe819dac0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400e9f0 tx=0x7f5fe400edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:31.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.431+0000 7f5fdeffd700 1 -- 172.21.15.2:0/3680391126 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe400cbd0 con 0x7f5fe8100510 2023-09-13T04:51:31.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.431+0000 7f5fdeffd700 1 -- 172.21.15.2:0/3680391126 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fe40095c0 con 0x7f5fe8100510 2023-09-13T04:51:31.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.431+0000 7f5fdeffd700 1 -- 172.21.15.2:0/3680391126 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4018680 con 0x7f5fe8100510 2023-09-13T04:51:31.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.431+0000 7f5ff01ff700 1 -- 172.21.15.2:0/3680391126 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe81a2b70 con 0x7f5fe8100510 2023-09-13T04:51:31.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.431+0000 7f5ff01ff700 1 -- 172.21.15.2:0/3680391126 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe8106180 con 0x7f5fe8100510 2023-09-13T04:51:31.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.433+0000 7f5ff01ff700 1 -- 172.21.15.2:0/3680391126 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe80688a0 con 0x7f5fe8100510 2023-09-13T04:51:31.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.438+0000 7f5fdeffd700 1 -- 172.21.15.2:0/3680391126 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 135389+0+0 (secure 0 0 0) 0x7f5fe4010430 con 0x7f5fe8100510 2023-09-13T04:51:31.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.440+0000 7f5fdeffd700 1 --2- 172.21.15.2:0/3680391126 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f5fd40a09c0 0x7f5fd40a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T04:51:31.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.440+0000 7f5fdeffd700 1 -- 172.21.15.2:0/3680391126 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8598+0+0 (secure 0 0 0) 0x7f5fe4014070 con 0x7f5fe8100510 2023-09-13T04:51:31.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.442+0000 7f5fedf9b700 1 --2- 172.21.15.2:0/3680391126 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f5fd40a09c0 0x7f5fd40a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T04:51:31.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.447+0000 7f5fedf9b700 1 --2- 172.21.15.2:0/3680391126 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f5fd40a09c0 0x7f5fd40a2e80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009fd0 tx=0x7f5fe000b720 comp rx=0 tx=0).ready entity=mgr.15492 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T04:51:31.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.451+0000 7f5fdeffd700 1 -- 172.21.15.2:0/3680391126 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5fe4051e10 con 0x7f5fe8100510 2023-09-13T04:51:31.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:31.859+0000 7f5ff01ff700 1 -- 172.21.15.2:0/3680391126 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5fe8067eb0 con 0x7f5fe8100510 2023-09-13T04:51:32.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:31 smithi002 ceph-mon[97651]: pgmap v348: 97 pgs: 15 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 33 active+clean; 535 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:51:32.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:31 smithi002 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-09-13T04:51:32.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:31 smithi002 ceph-mon[97651]: osdmap e108: 12 total, 12 up, 12 in 2023-09-13T04:51:32.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:31 smithi002 ceph-mon[97651]: from='client.? 172.21.15.2:0/3680391126' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-13T04:51:32.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:31 smithi149 ceph-mon[133315]: pgmap v348: 97 pgs: 15 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 33 active+clean; 535 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:51:32.386 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:31 smithi149 ceph-mon[133315]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-09-13T04:51:32.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:31 smithi149 ceph-mon[133315]: osdmap e108: 12 total, 12 up, 12 in 2023-09-13T04:51:32.387 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:31 smithi149 ceph-mon[133315]: from='client.? 172.21.15.2:0/3680391126' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-13T04:51:32.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:32.428+0000 7f5fdeffd700 1 -- 172.21.15.2:0/3680391126 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v109) v1 ==== 178+0+0 (secure 0 0 0) 0x7f5fe40993e0 con 0x7f5fe8100510 2023-09-13T04:51:32.436 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:31 smithi156 ceph-mon[133675]: pgmap v348: 97 pgs: 15 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 33 active+clean; 535 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T04:51:32.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:31 smithi156 ceph-mon[133675]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-09-13T04:51:32.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:31 smithi156 ceph-mon[133675]: osdmap e108: 12 total, 12 up, 12 in 2023-09-13T04:51:32.437 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:31 smithi156 ceph-mon[133675]: from='client.? 172.21.15.2:0/3680391126' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-13T04:51:32.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:32.437+0000 7f5ff01ff700 1 -- 172.21.15.2:0/3680391126 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f5fd40a09c0 msgr2=0x7f5fd40a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:32.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:32.437+0000 7f5ff01ff700 1 --2- 172.21.15.2:0/3680391126 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f5fd40a09c0 0x7f5fd40a2e80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009fd0 tx=0x7f5fe000b720 comp rx=0 tx=0).stop 2023-09-13T04:51:32.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:32.437+0000 7f5ff01ff700 1 -- 172.21.15.2:0/3680391126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5fe8100510 msgr2=0x7f5fe819dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T04:51:32.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:32.437+0000 7f5ff01ff700 1 --2- 172.21.15.2:0/3680391126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5fe8100510 0x7f5fe819dac0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400e9f0 tx=0x7f5fe400edb0 comp rx=0 tx=0).stop 2023-09-13T04:51:32.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:32.442+0000 7f5ff01ff700 1 -- 172.21.15.2:0/3680391126 shutdown_connections 2023-09-13T04:51:32.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:32.442+0000 7f5ff01ff700 1 --2- 172.21.15.2:0/3680391126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fe8104ff0 0x7f5fe81a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:32.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:32.442+0000 7f5ff01ff700 1 --2- 172.21.15.2:0/3680391126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fe80ffbf0 0x7f5fe819d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:32.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:32.442+0000 7f5ff01ff700 1 --2- 172.21.15.2:0/3680391126 >> [v2:172.21.15.2:6800/3640961355,v1:172.21.15.2:6801/3640961355] conn(0x7f5fd40a09c0 0x7f5fd40a2e80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:32.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:32.442+0000 7f5ff01ff700 1 --2- 172.21.15.2:0/3680391126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5fe8100510 0x7f5fe819dac0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T04:51:32.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:32.442+0000 7f5ff01ff700 1 -- 172.21.15.2:0/3680391126 >> 172.21.15.2:0/3680391126 conn(0x7f5fe80fb620 msgr2=0x7f5fe8102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T04:51:32.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:32.447+0000 7f5ff01ff700 1 -- 172.21.15.2:0/3680391126 shutdown_connections 2023-09-13T04:51:32.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-09-13T04:51:32.452+0000 7f5ff01ff700 1 -- 172.21.15.2:0/3680391126 wait complete. 2023-09-13T04:51:32.470 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data_ec' removed 2023-09-13T04:51:32.501 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-13T04:51:32.572 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-13T04:51:32.635 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-13T04:51:32.667 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-09-13T04:51:32.667 DEBUG:teuthology.orchestra.run.smithi002:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-09-13T04:51:32.711 DEBUG:teuthology.orchestra.run.smithi149:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-09-13T04:51:32.728 DEBUG:teuthology.orchestra.run.smithi156:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-09-13T04:51:32.746 INFO:tasks.cephadm:Stopping all daemons... 2023-09-13T04:51:32.746 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-09-13T04:51:32.747 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mon.a 2023-09-13T04:51:33.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:32 smithi002 systemd[1]: Stopping Ceph mon.a for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:51:33.561 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:33 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mon-a[97627]: 2023-09-13T04:51:33.303+0000 7fbd2bd43700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:51:33.562 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:33 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mon-a[97627]: 2023-09-13T04:51:33.303+0000 7fbd2bd43700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-09-13T04:51:34.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 13 04:51:33 smithi002 bash[179126]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mon-a 2023-09-13T04:51:34.076 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mon.a.service' 2023-09-13T04:51:34.190 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:51:34.191 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-09-13T04:51:34.191 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-09-13T04:51:34.192 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mon.b 2023-09-13T04:51:34.629 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:34 smithi149 systemd[1]: Stopping Ceph mon.b for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:51:34.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:34 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mon-b[133291]: 2023-09-13T04:51:34.626+0000 7f78890ee700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:51:34.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 13 04:51:34 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mon-b[133291]: 2023-09-13T04:51:34.626+0000 7f78890ee700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-09-13T04:51:35.346 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mon.b.service' 2023-09-13T04:51:35.396 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:51:35.397 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-09-13T04:51:35.398 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-09-13T04:51:35.398 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mon.c 2023-09-13T04:51:35.687 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:35 smithi156 systemd[1]: Stopping Ceph mon.c for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:51:36.102 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:35 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mon-c[133651]: 2023-09-13T04:51:35.753+0000 7ffbd6822700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:51:36.103 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 13 04:51:35 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mon-c[133651]: 2023-09-13T04:51:35.753+0000 7ffbd6822700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-09-13T04:51:36.303 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mon.c.service' 2023-09-13T04:51:36.356 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:51:36.357 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-09-13T04:51:36.357 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-09-13T04:51:36.357 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mgr.x 2023-09-13T04:51:36.755 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:51:36 smithi002 systemd[1]: Stopping Ceph mgr.x for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:51:37.186 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:51:36 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x[97894]: 2023-09-13T04:51:36.918+0000 7f10a59d3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:51:37.505 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 13 04:51:37 smithi002 bash[179273]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-x 2023-09-13T04:51:37.735 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mgr.x.service' 2023-09-13T04:51:37.825 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:51:37.825 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-09-13T04:51:37.826 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-09-13T04:51:37.826 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mgr.y 2023-09-13T04:51:38.136 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:51:37 smithi149 systemd[1]: Stopping Ceph mgr.y for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:51:40.636 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:51:40 smithi149 bash[176360]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-y 2023-09-13T04:51:42.170 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Sep 13 04:51:42 smithi149 systemd[1]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-09-13T04:51:42.174 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mgr.y.service' 2023-09-13T04:51:42.212 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:51:42.212 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-09-13T04:51:42.213 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-09-13T04:51:42.213 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mgr.z 2023-09-13T04:51:42.643 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:51:42 smithi156 systemd[1]: Stopping Ceph mgr.z for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:51:42.920 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:51:42 smithi156 bash[179320]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mgr-z 2023-09-13T04:51:42.920 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 13 04:51:42 smithi156 systemd[1]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-09-13T04:51:43.120 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mgr.z.service' 2023-09-13T04:51:43.157 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:51:43.158 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-09-13T04:51:43.158 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-09-13T04:51:43.158 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.0 2023-09-13T04:51:43.505 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 13 04:51:43 smithi002 systemd[1]: Stopping Ceph osd.0 for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:51:44.005 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 13 04:51:43 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-0[115301]: 2023-09-13T04:51:43.714+0000 7f6550d1c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:51:44.006 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 13 04:51:43 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-0[115301]: 2023-09-13T04:51:43.714+0000 7f6550d1c700 -1 osd.0 109 *** Got signal Terminated *** 2023-09-13T04:51:44.006 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 13 04:51:43 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-0[115301]: 2023-09-13T04:51:43.715+0000 7f6550d1c700 -1 osd.0 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-13T04:51:49.792 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 13 04:51:49 smithi002 bash[179418]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-0 2023-09-13T04:51:51.554 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.0.service' 2023-09-13T04:51:51.653 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:51:51.653 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-09-13T04:51:51.654 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-09-13T04:51:51.654 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.1 2023-09-13T04:51:52.007 INFO:journalctl@ceph.osd.1.smithi149.stdout:Sep 13 04:51:51 smithi149 systemd[1]: Stopping Ceph osd.1 for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:51:52.386 INFO:journalctl@ceph.osd.1.smithi149.stdout:Sep 13 04:51:52 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-1[138391]: 2023-09-13T04:51:52.006+0000 7f2cbfd3e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:51:52.386 INFO:journalctl@ceph.osd.1.smithi149.stdout:Sep 13 04:51:52 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-1[138391]: 2023-09-13T04:51:52.006+0000 7f2cbfd3e700 -1 osd.1 109 *** Got signal Terminated *** 2023-09-13T04:51:52.387 INFO:journalctl@ceph.osd.1.smithi149.stdout:Sep 13 04:51:52 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-1[138391]: 2023-09-13T04:51:52.006+0000 7f2cbfd3e700 -1 osd.1 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-13T04:51:58.136 INFO:journalctl@ceph.osd.1.smithi149.stdout:Sep 13 04:51:57 smithi149 bash[176514]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-1 2023-09-13T04:51:59.572 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.1.service' 2023-09-13T04:51:59.612 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:51:59.612 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-09-13T04:51:59.613 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-09-13T04:51:59.613 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.2 2023-09-13T04:51:59.936 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 13 04:51:59 smithi156 systemd[1]: Stopping Ceph osd.2 for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:52:00.686 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 13 04:52:00 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-2[139457]: 2023-09-13T04:52:00.259+0000 7f1a9bac4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:52:00.687 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 13 04:52:00 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-2[139457]: 2023-09-13T04:52:00.259+0000 7f1a9bac4700 -1 osd.2 109 *** Got signal Terminated *** 2023-09-13T04:52:00.687 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 13 04:52:00 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-2[139457]: 2023-09-13T04:52:00.259+0000 7f1a9bac4700 -1 osd.2 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-13T04:52:06.436 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 13 04:52:06 smithi156 bash[179473]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-2 2023-09-13T04:52:07.450 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.2.service' 2023-09-13T04:52:07.490 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:52:07.491 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-09-13T04:52:07.491 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-09-13T04:52:07.491 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.3 2023-09-13T04:52:08.006 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 13 04:52:07 smithi002 systemd[1]: Stopping Ceph osd.3 for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:52:08.505 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 13 04:52:08 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-3[123164]: 2023-09-13T04:52:08.016+0000 7f109f3d8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:52:08.506 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 13 04:52:08 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-3[123164]: 2023-09-13T04:52:08.016+0000 7f109f3d8700 -1 osd.3 109 *** Got signal Terminated *** 2023-09-13T04:52:08.506 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 13 04:52:08 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-3[123164]: 2023-09-13T04:52:08.016+0000 7f109f3d8700 -1 osd.3 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-13T04:52:10.756 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:10 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:10.325+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:11.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:11 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:11.044+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:11.436 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:11 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:11.106+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:42.827382+0000 front 2023-09-13T04:51:42.827484+0000 (oldest deadline 2023-09-13T04:52:10.427833+0000) 2023-09-13T04:52:11.756 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 13 04:52:11 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-3[123164]: 2023-09-13T04:52:11.420+0000 7f109b9ce700 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.378812+0000 front 2023-09-13T04:51:46.378887+0000 (oldest deadline 2023-09-13T04:52:11.079121+0000) 2023-09-13T04:52:11.756 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:11 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:11.345+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:12.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:12 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:12.017+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:12.436 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:12 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:12.119+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:42.827382+0000 front 2023-09-13T04:51:42.827484+0000 (oldest deadline 2023-09-13T04:52:10.427833+0000) 2023-09-13T04:52:12.755 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 13 04:52:12 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-3[123164]: 2023-09-13T04:52:12.378+0000 7f109b9ce700 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.378812+0000 front 2023-09-13T04:51:46.378887+0000 (oldest deadline 2023-09-13T04:52:11.079121+0000) 2023-09-13T04:52:12.756 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:12 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:12.307+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:13.255 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:12 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:12.801+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.604850+0000 front 2023-09-13T04:51:47.605132+0000 (oldest deadline 2023-09-13T04:52:12.305210+0000) 2023-09-13T04:52:13.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:12 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:12.985+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:13.437 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:13 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:13.143+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:42.827382+0000 front 2023-09-13T04:51:42.827484+0000 (oldest deadline 2023-09-13T04:52:10.427833+0000) 2023-09-13T04:52:13.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:13 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:13.256+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:13.732 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:13 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:13.357+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:14.005 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 13 04:52:13 smithi002 bash[179703]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-3 2023-09-13T04:52:14.005 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:13 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:13.810+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.604850+0000 front 2023-09-13T04:51:47.605132+0000 (oldest deadline 2023-09-13T04:52:12.305210+0000) 2023-09-13T04:52:14.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:13 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:13.978+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:14.436 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:14 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:14.108+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:42.827382+0000 front 2023-09-13T04:51:42.827484+0000 (oldest deadline 2023-09-13T04:52:10.427833+0000) 2023-09-13T04:52:14.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:14 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:14.248+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:14.755 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:14 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:14.401+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:15.255 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:14 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:14.773+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.604850+0000 front 2023-09-13T04:51:47.605132+0000 (oldest deadline 2023-09-13T04:52:12.305210+0000) 2023-09-13T04:52:15.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:14 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:14.933+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:15.436 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:15 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:15.142+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:42.827382+0000 front 2023-09-13T04:51:42.827484+0000 (oldest deadline 2023-09-13T04:52:10.427833+0000) 2023-09-13T04:52:15.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:15 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:15.273+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:15.722 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:15 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:15.359+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:15.746 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.3.service' 2023-09-13T04:52:15.842 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:52:15.843 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-09-13T04:52:15.843 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-09-13T04:52:15.844 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.4 2023-09-13T04:52:16.005 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:15 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:15.764+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.604850+0000 front 2023-09-13T04:51:47.605132+0000 (oldest deadline 2023-09-13T04:52:12.305210+0000) 2023-09-13T04:52:16.136 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:15 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:15.953+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:16.137 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 13 04:52:15 smithi149 systemd[1]: Stopping Ceph osd.4 for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:52:16.436 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:16 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:16.159+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:42.827382+0000 front 2023-09-13T04:51:42.827484+0000 (oldest deadline 2023-09-13T04:52:10.427833+0000) 2023-09-13T04:52:16.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:16 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:16.254+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:16.636 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 13 04:52:16 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-4[146911]: 2023-09-13T04:52:16.193+0000 7fe519151700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:52:16.636 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 13 04:52:16 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-4[146911]: 2023-09-13T04:52:16.193+0000 7fe519151700 -1 osd.4 109 *** Got signal Terminated *** 2023-09-13T04:52:16.637 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 13 04:52:16 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-4[146911]: 2023-09-13T04:52:16.193+0000 7fe519151700 -1 osd.4 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-13T04:52:16.755 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:16 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:16.356+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:17.255 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:16 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:16.788+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.604850+0000 front 2023-09-13T04:51:47.605132+0000 (oldest deadline 2023-09-13T04:52:12.305210+0000) 2023-09-13T04:52:17.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:16 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:16.949+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:17.436 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:17 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:17.117+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:42.827382+0000 front 2023-09-13T04:51:42.827484+0000 (oldest deadline 2023-09-13T04:52:10.427833+0000) 2023-09-13T04:52:17.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:17 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:17.227+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:17.755 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:17 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:17.381+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:18.256 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:17 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:17.762+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.604850+0000 front 2023-09-13T04:51:47.605132+0000 (oldest deadline 2023-09-13T04:52:12.305210+0000) 2023-09-13T04:52:18.377 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:17 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:17.905+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:18.436 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:18 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:18.139+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:42.827382+0000 front 2023-09-13T04:51:42.827484+0000 (oldest deadline 2023-09-13T04:52:10.427833+0000) 2023-09-13T04:52:18.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:18 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:18.253+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:18.636 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 13 04:52:18 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-4[146911]: 2023-09-13T04:52:18.376+0000 7fe512741700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:55.701388+0000 front 2023-09-13T04:51:55.701358+0000 (oldest deadline 2023-09-13T04:52:17.902516+0000) 2023-09-13T04:52:18.739 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:18 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:18.363+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:19.005 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:18 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:18.738+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.604850+0000 front 2023-09-13T04:51:47.605132+0000 (oldest deadline 2023-09-13T04:52:12.305210+0000) 2023-09-13T04:52:19.349 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:18 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:18.949+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:19.636 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 13 04:52:19 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-4[146911]: 2023-09-13T04:52:19.348+0000 7fe512741700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:55.701388+0000 front 2023-09-13T04:51:55.701358+0000 (oldest deadline 2023-09-13T04:52:17.902516+0000) 2023-09-13T04:52:19.686 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:19 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:19.188+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:42.827382+0000 front 2023-09-13T04:51:42.827484+0000 (oldest deadline 2023-09-13T04:52:10.427833+0000) 2023-09-13T04:52:19.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:19 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:19.220+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:19.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:19 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:19.220+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:19.751 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:19 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:19.361+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:19.751 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:19 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:19.361+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:20.005 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:19 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:19.750+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.604850+0000 front 2023-09-13T04:51:47.605132+0000 (oldest deadline 2023-09-13T04:52:12.305210+0000) 2023-09-13T04:52:20.377 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:19 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:19.919+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:20.378 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:19 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:19.919+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:20.436 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:20 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:20.176+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:42.827382+0000 front 2023-09-13T04:51:42.827484+0000 (oldest deadline 2023-09-13T04:52:10.427833+0000) 2023-09-13T04:52:20.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:20 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:20.210+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:20.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:20 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:20.210+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:20.636 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 13 04:52:20 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-4[146911]: 2023-09-13T04:52:20.376+0000 7fe512741700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:55.701388+0000 front 2023-09-13T04:51:55.701358+0000 (oldest deadline 2023-09-13T04:52:17.902516+0000) 2023-09-13T04:52:20.637 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:20 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:20.484+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:20.724 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:20 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:20.312+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:20.725 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:20 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:20.312+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:21.005 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:20 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:20.723+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.604850+0000 front 2023-09-13T04:51:47.605132+0000 (oldest deadline 2023-09-13T04:52:12.305210+0000) 2023-09-13T04:52:21.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:20 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:20.723+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:57.006689+0000 front 2023-09-13T04:51:57.006728+0000 (oldest deadline 2023-09-13T04:52:20.506905+0000) 2023-09-13T04:52:21.136 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:20 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:20.870+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:21.137 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:20 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:20.870+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:21.437 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:21 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:21.163+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:42.827382+0000 front 2023-09-13T04:51:42.827484+0000 (oldest deadline 2023-09-13T04:52:10.427833+0000) 2023-09-13T04:52:21.438 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:21 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:21.163+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.829469+0000 front 2023-09-13T04:51:56.829676+0000 (oldest deadline 2023-09-13T04:52:20.329537+0000) 2023-09-13T04:52:21.438 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:21 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:21.295+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:21.439 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:21 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:21.173+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:21.439 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:21 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:21.173+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:21.752 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:21 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:21.319+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:21.752 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:21 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:21.319+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:21.801 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:21 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:21.472+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:22.005 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:21 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:21.751+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.604850+0000 front 2023-09-13T04:51:47.605132+0000 (oldest deadline 2023-09-13T04:52:12.305210+0000) 2023-09-13T04:52:22.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:21 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:21.751+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:57.006689+0000 front 2023-09-13T04:51:57.006728+0000 (oldest deadline 2023-09-13T04:52:20.506905+0000) 2023-09-13T04:52:22.081 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 13 04:52:21 smithi149 bash[176795]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-4 2023-09-13T04:52:22.082 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:21 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:21.861+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:22.082 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:21 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:21.861+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:22.436 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:22 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:22.147+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:42.827382+0000 front 2023-09-13T04:51:42.827484+0000 (oldest deadline 2023-09-13T04:52:10.427833+0000) 2023-09-13T04:52:22.437 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:22 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:22.147+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.829469+0000 front 2023-09-13T04:51:56.829676+0000 (oldest deadline 2023-09-13T04:52:20.329537+0000) 2023-09-13T04:52:22.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:22 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:22.254+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:22.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:22 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:22.139+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:22.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:22 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:22.139+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:22.704 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:22 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:22.352+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:22.705 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:22 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:22.352+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:22.824 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:22 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:22.499+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:23.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:22 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:22.703+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.604850+0000 front 2023-09-13T04:51:47.605132+0000 (oldest deadline 2023-09-13T04:52:12.305210+0000) 2023-09-13T04:52:23.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:22 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:22.703+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:57.006689+0000 front 2023-09-13T04:51:57.006728+0000 (oldest deadline 2023-09-13T04:52:20.506905+0000) 2023-09-13T04:52:23.136 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:22 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:22.823+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:23.137 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:22 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:22.823+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:23.436 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:23 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:23.128+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:42.827382+0000 front 2023-09-13T04:51:42.827484+0000 (oldest deadline 2023-09-13T04:52:10.427833+0000) 2023-09-13T04:52:23.437 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:23 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:23.128+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.829469+0000 front 2023-09-13T04:51:56.829676+0000 (oldest deadline 2023-09-13T04:52:20.329537+0000) 2023-09-13T04:52:23.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:23 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:23.230+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:23.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:23 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:23.150+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:23.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:23 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:23.150+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:23.444 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.4.service' 2023-09-13T04:52:23.487 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:52:23.488 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-09-13T04:52:23.488 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-09-13T04:52:23.489 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.5 2023-09-13T04:52:23.686 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:23 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:23.318+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:23.687 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:23 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:23.318+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:23.839 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:23 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:23.479+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:23.854 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:23 smithi156 systemd[1]: Stopping Ceph osd.5 for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:52:23.855 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:23 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:23.853+0000 7f4e2f2d2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:52:24.005 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:23 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:23.685+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.604850+0000 front 2023-09-13T04:51:47.605132+0000 (oldest deadline 2023-09-13T04:52:12.305210+0000) 2023-09-13T04:52:24.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:23 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:23.685+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:57.006689+0000 front 2023-09-13T04:51:57.006728+0000 (oldest deadline 2023-09-13T04:52:20.506905+0000) 2023-09-13T04:52:24.136 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:23 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:23.839+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:24.137 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:23 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:23.839+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:24.145 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:23 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:23.853+0000 7f4e2f2d2700 -1 osd.5 109 *** Got signal Terminated *** 2023-09-13T04:52:24.145 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:23 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:23.853+0000 7f4e2f2d2700 -1 osd.5 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-13T04:52:24.436 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:24 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:24.144+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:42.827382+0000 front 2023-09-13T04:51:42.827484+0000 (oldest deadline 2023-09-13T04:52:10.427833+0000) 2023-09-13T04:52:24.437 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:24 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:24.144+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.829469+0000 front 2023-09-13T04:51:56.829676+0000 (oldest deadline 2023-09-13T04:52:20.329537+0000) 2023-09-13T04:52:24.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:24 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:24.214+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:24.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:24 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:24.163+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:24.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:24 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:24.163+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:24.675 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:24 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:24.290+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:24.676 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:24 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:24.290+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:24.877 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:24 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:24.434+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:25.005 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:24 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:24.674+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.604850+0000 front 2023-09-13T04:51:47.605132+0000 (oldest deadline 2023-09-13T04:52:12.305210+0000) 2023-09-13T04:52:25.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:24 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:24.674+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:57.006689+0000 front 2023-09-13T04:51:57.006728+0000 (oldest deadline 2023-09-13T04:52:20.506905+0000) 2023-09-13T04:52:25.136 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:24 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:24.877+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:25.137 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:24 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:24.877+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:25.436 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:25 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:25.228+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:25.437 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:25 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:25.183+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:42.827382+0000 front 2023-09-13T04:51:42.827484+0000 (oldest deadline 2023-09-13T04:52:10.427833+0000) 2023-09-13T04:52:25.437 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:25 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:25.183+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.829469+0000 front 2023-09-13T04:51:56.829676+0000 (oldest deadline 2023-09-13T04:52:20.329537+0000) 2023-09-13T04:52:25.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:25 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:25.144+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:25.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:25 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:25.144+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:25.505 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:25 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:25.244+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:25.505 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:25 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:25.244+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:25.886 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:25 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:25.432+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:26.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:25 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:25.635+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.604850+0000 front 2023-09-13T04:51:47.605132+0000 (oldest deadline 2023-09-13T04:52:12.305210+0000) 2023-09-13T04:52:26.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:25 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:25.635+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:57.006689+0000 front 2023-09-13T04:51:57.006728+0000 (oldest deadline 2023-09-13T04:52:20.506905+0000) 2023-09-13T04:52:26.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:25 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:25.920+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:26.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:25 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:25.920+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:26.436 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:26 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:26.250+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:26.437 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:26 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:26.167+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:42.827382+0000 front 2023-09-13T04:51:42.827484+0000 (oldest deadline 2023-09-13T04:52:10.427833+0000) 2023-09-13T04:52:26.437 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:26 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:26.167+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.829469+0000 front 2023-09-13T04:51:56.829676+0000 (oldest deadline 2023-09-13T04:52:20.329537+0000) 2023-09-13T04:52:26.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:26 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:26.172+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:26.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:26 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:26.172+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:26.505 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:26 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:26.211+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:26.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:26 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:26.211+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:26.886 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:26 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:26.446+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:27.005 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:26 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:26.629+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.604850+0000 front 2023-09-13T04:51:47.605132+0000 (oldest deadline 2023-09-13T04:52:12.305210+0000) 2023-09-13T04:52:27.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:26 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:26.629+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:57.006689+0000 front 2023-09-13T04:51:57.006728+0000 (oldest deadline 2023-09-13T04:52:20.506905+0000) 2023-09-13T04:52:27.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:26 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:26.927+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:27.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:26 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:26.927+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:27.436 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:27 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:27.289+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:27.437 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:27 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:27.126+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:42.827382+0000 front 2023-09-13T04:51:42.827484+0000 (oldest deadline 2023-09-13T04:52:10.427833+0000) 2023-09-13T04:52:27.437 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:27 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:27.126+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.829469+0000 front 2023-09-13T04:51:56.829676+0000 (oldest deadline 2023-09-13T04:52:20.329537+0000) 2023-09-13T04:52:27.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:27 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:27.212+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:27.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:27 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:27.212+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:27.582 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:27 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:27.255+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:27.583 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:27 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:27.255+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:27.583 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:27 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:27.255+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:27.886 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:27 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:27.420+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:28.005 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:27 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:27.581+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.604850+0000 front 2023-09-13T04:51:47.605132+0000 (oldest deadline 2023-09-13T04:52:12.305210+0000) 2023-09-13T04:52:28.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:27 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:27.581+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:57.006689+0000 front 2023-09-13T04:51:57.006728+0000 (oldest deadline 2023-09-13T04:52:20.506905+0000) 2023-09-13T04:52:28.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:27 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:27.973+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:28.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:27 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:27.973+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:28.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:27 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:27.973+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:28.436 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:28 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:28.093+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:42.827382+0000 front 2023-09-13T04:51:42.827484+0000 (oldest deadline 2023-09-13T04:52:10.427833+0000) 2023-09-13T04:52:28.437 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:28 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5[147365]: 2023-09-13T04:52:28.093+0000 7f4e2a0c5700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.829469+0000 front 2023-09-13T04:51:56.829676+0000 (oldest deadline 2023-09-13T04:52:20.329537+0000) 2023-09-13T04:52:28.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:28 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:28.278+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:28.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:28 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:28.278+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:28.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:28 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:28.223+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:28.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:28 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:28.223+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:28.505 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:28 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:28.233+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:28.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:28 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:28.233+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:28.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:28 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:28.233+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:28.886 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:28 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:28.428+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:28.887 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:28 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:28.428+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:01.893556+0000 front 2023-09-13T04:52:01.893478+0000 (oldest deadline 2023-09-13T04:52:27.793569+0000) 2023-09-13T04:52:29.005 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:28 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:28.594+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.604850+0000 front 2023-09-13T04:51:47.605132+0000 (oldest deadline 2023-09-13T04:52:12.305210+0000) 2023-09-13T04:52:29.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:28 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:28.594+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:57.006689+0000 front 2023-09-13T04:51:57.006728+0000 (oldest deadline 2023-09-13T04:52:20.506905+0000) 2023-09-13T04:52:29.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:28 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:28.965+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:29.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:28 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:28.965+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:29.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:28 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:28.965+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:29.629 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:29 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:29.257+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:29.629 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:29 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:29.257+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:29.630 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:29 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:29.257+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:29.630 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:29 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:29.251+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:29.630 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:29 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:29.251+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:29.631 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:29 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:29.196+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:29.631 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:29 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:29.196+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:29.886 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:29 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:29.435+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:29.886 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:29 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:29.435+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:01.893556+0000 front 2023-09-13T04:52:01.893478+0000 (oldest deadline 2023-09-13T04:52:27.793569+0000) 2023-09-13T04:52:29.936 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 13 04:52:29 smithi156 bash[179739]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-5 2023-09-13T04:52:30.005 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:29 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:29.625+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.604850+0000 front 2023-09-13T04:51:47.605132+0000 (oldest deadline 2023-09-13T04:52:12.305210+0000) 2023-09-13T04:52:30.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:29 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:29.625+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:57.006689+0000 front 2023-09-13T04:51:57.006728+0000 (oldest deadline 2023-09-13T04:52:20.506905+0000) 2023-09-13T04:52:30.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:29 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:29.625+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:05.208103+0000 front 2023-09-13T04:52:05.208488+0000 (oldest deadline 2023-09-13T04:52:29.308357+0000) 2023-09-13T04:52:30.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:30 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:30.010+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:30.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:30 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:30.010+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:30.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:30 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:30.010+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:30.505 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:30 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:30.253+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:30.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:30 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:30.253+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:30.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:30 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:30.253+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:30.610 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:30 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:30.274+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:30.610 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:30 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:30.274+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:30.611 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:30 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:30.219+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:30.611 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:30 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:30.219+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:30.886 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:30 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:30.475+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:30.886 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:30 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:30.475+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:01.893556+0000 front 2023-09-13T04:52:01.893478+0000 (oldest deadline 2023-09-13T04:52:27.793569+0000) 2023-09-13T04:52:31.005 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:30 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:30.644+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.604850+0000 front 2023-09-13T04:51:47.605132+0000 (oldest deadline 2023-09-13T04:52:12.305210+0000) 2023-09-13T04:52:31.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:30 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:30.644+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:57.006689+0000 front 2023-09-13T04:51:57.006728+0000 (oldest deadline 2023-09-13T04:52:20.506905+0000) 2023-09-13T04:52:31.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:30 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:30.644+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:05.208103+0000 front 2023-09-13T04:52:05.208488+0000 (oldest deadline 2023-09-13T04:52:29.308357+0000) 2023-09-13T04:52:31.007 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.5.service' 2023-09-13T04:52:31.048 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:52:31.048 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-09-13T04:52:31.049 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-09-13T04:52:31.049 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.6 2023-09-13T04:52:31.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:30 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:30.994+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:31.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:30 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:30.994+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:31.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:30 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:30.994+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:31.436 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:31 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:31.227+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:31.436 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:31 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:31.227+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:31.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:31 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:31.170+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:31.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:31 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:31.170+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:31.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:31 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:31.170+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:31.505 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:31 smithi002 systemd[1]: Stopping Ceph osd.6 for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:52:31.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:31 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:31.214+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:31.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:31 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:31.214+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:31.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:31 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:31.214+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:31.889 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:31 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:31.457+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:31.889 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:31 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:31.457+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:01.893556+0000 front 2023-09-13T04:52:01.893478+0000 (oldest deadline 2023-09-13T04:52:27.793569+0000) 2023-09-13T04:52:32.005 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:31 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:31.553+0000 7efccf4ab700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:52:32.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:31 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:31.553+0000 7efccf4ab700 -1 osd.6 109 *** Got signal Terminated *** 2023-09-13T04:52:32.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:31 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:31.553+0000 7efccf4ab700 -1 osd.6 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-13T04:52:32.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:31 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:31.610+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.604850+0000 front 2023-09-13T04:51:47.605132+0000 (oldest deadline 2023-09-13T04:52:12.305210+0000) 2023-09-13T04:52:32.007 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:31 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:31.610+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:57.006689+0000 front 2023-09-13T04:51:57.006728+0000 (oldest deadline 2023-09-13T04:52:20.506905+0000) 2023-09-13T04:52:32.007 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:31 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:31.610+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:05.208103+0000 front 2023-09-13T04:52:05.208488+0000 (oldest deadline 2023-09-13T04:52:29.308357+0000) 2023-09-13T04:52:32.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:31 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:31.957+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:32.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:31 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:31.957+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:32.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:31 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:31.957+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:32.436 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:32 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:32.143+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:32.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:32 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:32.143+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:32.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:32 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:32.143+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:32.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:32 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:32.230+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:32.438 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:32 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:32.230+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:32.505 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:32 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:32.223+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:32.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:32 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:32.223+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:32.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:32 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:32.223+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:32.886 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:32 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:32.417+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:32.886 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:32 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:32.417+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:01.893556+0000 front 2023-09-13T04:52:01.893478+0000 (oldest deadline 2023-09-13T04:52:27.793569+0000) 2023-09-13T04:52:33.005 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:32 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:32.631+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.604850+0000 front 2023-09-13T04:51:47.605132+0000 (oldest deadline 2023-09-13T04:52:12.305210+0000) 2023-09-13T04:52:33.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:32 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:32.631+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:57.006689+0000 front 2023-09-13T04:51:57.006728+0000 (oldest deadline 2023-09-13T04:52:20.506905+0000) 2023-09-13T04:52:33.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:32 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:32.631+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:05.208103+0000 front 2023-09-13T04:52:05.208488+0000 (oldest deadline 2023-09-13T04:52:29.308357+0000) 2023-09-13T04:52:33.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:32 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:32.918+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:33.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:32 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:32.918+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:33.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:32 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:32.918+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:33.436 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:33 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:33.273+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:33.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:33 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:33.273+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:33.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:33 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:33.173+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:33.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:33 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:33.173+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:33.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:33 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:33.173+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:33.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:33 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:33.241+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:33.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:33 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:33.241+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:33.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:33 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:33.241+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:33.886 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:33 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:33.419+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:33.887 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:33 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:33.419+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:01.893556+0000 front 2023-09-13T04:52:01.893478+0000 (oldest deadline 2023-09-13T04:52:27.793569+0000) 2023-09-13T04:52:34.005 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:33 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:33.659+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.604850+0000 front 2023-09-13T04:51:47.605132+0000 (oldest deadline 2023-09-13T04:52:12.305210+0000) 2023-09-13T04:52:34.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:33 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:33.659+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:57.006689+0000 front 2023-09-13T04:51:57.006728+0000 (oldest deadline 2023-09-13T04:52:20.506905+0000) 2023-09-13T04:52:34.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:33 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:33.659+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:05.208103+0000 front 2023-09-13T04:52:05.208488+0000 (oldest deadline 2023-09-13T04:52:29.308357+0000) 2023-09-13T04:52:34.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:33 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:33.891+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:34.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:33 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:33.891+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:34.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:33 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:33.891+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:34.436 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:34 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:34.281+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:34.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:34 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:34.281+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:34.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:34 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:34.156+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:34.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:34 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:34.156+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:34.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:34 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:34.156+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:34.696 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:34 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:34.286+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:34.697 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:34 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:34.286+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:34.697 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:34 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:34.286+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:34.886 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:34 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:34.433+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:34.887 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:34 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:34.433+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:01.893556+0000 front 2023-09-13T04:52:01.893478+0000 (oldest deadline 2023-09-13T04:52:27.793569+0000) 2023-09-13T04:52:34.887 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:34 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:34.433+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.793957+0000 front 2023-09-13T04:52:07.794079+0000 (oldest deadline 2023-09-13T04:52:33.694128+0000) 2023-09-13T04:52:35.005 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:34 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:34.695+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.604850+0000 front 2023-09-13T04:51:47.605132+0000 (oldest deadline 2023-09-13T04:52:12.305210+0000) 2023-09-13T04:52:35.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:34 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:34.695+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:57.006689+0000 front 2023-09-13T04:51:57.006728+0000 (oldest deadline 2023-09-13T04:52:20.506905+0000) 2023-09-13T04:52:35.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:34 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:34.695+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:05.208103+0000 front 2023-09-13T04:52:05.208488+0000 (oldest deadline 2023-09-13T04:52:29.308357+0000) 2023-09-13T04:52:35.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:34 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:34.902+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:35.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:34 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:34.902+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:35.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:34 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:34.902+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:35.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:35 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:35.278+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:35.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:35 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:35.278+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:35.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:35 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:35.149+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:35.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:35 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:35.149+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:35.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:35 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:35.149+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:35.709 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:35 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:35.285+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:35.709 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:35 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:35.285+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:35.709 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:35 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:35.285+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:35.886 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:35 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:35.418+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:35.886 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:35 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:35.418+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:01.893556+0000 front 2023-09-13T04:52:01.893478+0000 (oldest deadline 2023-09-13T04:52:27.793569+0000) 2023-09-13T04:52:35.887 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:35 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:35.418+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.793957+0000 front 2023-09-13T04:52:07.794079+0000 (oldest deadline 2023-09-13T04:52:33.694128+0000) 2023-09-13T04:52:36.005 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:35 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:35.708+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.604850+0000 front 2023-09-13T04:51:47.605132+0000 (oldest deadline 2023-09-13T04:52:12.305210+0000) 2023-09-13T04:52:36.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:35 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:35.708+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:57.006689+0000 front 2023-09-13T04:51:57.006728+0000 (oldest deadline 2023-09-13T04:52:20.506905+0000) 2023-09-13T04:52:36.006 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:35 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6[131691]: 2023-09-13T04:52:35.708+0000 7efcc829a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:05.208103+0000 front 2023-09-13T04:52:05.208488+0000 (oldest deadline 2023-09-13T04:52:29.308357+0000) 2023-09-13T04:52:36.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:35 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:35.926+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:36.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:35 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:35.926+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:36.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:35 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:35.926+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:36.505 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:36 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:36.239+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:36.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:36 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:36.239+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:36.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:36 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:36.239+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:36.686 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:36 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:36.242+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:36.687 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:36 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:36.242+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:36.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:36 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:36.189+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:36.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:36 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:36.189+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:36.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:36 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:36.189+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:36.886 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:36 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:36.434+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:36.887 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:36 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:36.434+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:01.893556+0000 front 2023-09-13T04:52:01.893478+0000 (oldest deadline 2023-09-13T04:52:27.793569+0000) 2023-09-13T04:52:36.887 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:36 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:36.434+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.793957+0000 front 2023-09-13T04:52:07.794079+0000 (oldest deadline 2023-09-13T04:52:33.694128+0000) 2023-09-13T04:52:37.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:36 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:36.899+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:37.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:36 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:36.899+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:37.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:36 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:36.899+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:37.436 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:37 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:37.224+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:37.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:37 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:37.224+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:37.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:37 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:37.224+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:12.821347+0000 front 2023-09-13T04:52:12.821274+0000 (oldest deadline 2023-09-13T04:52:36.921110+0000) 2023-09-13T04:52:37.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:37 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:37.184+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:37.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:37 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:37.184+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:37.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:37 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:37.184+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:37.458 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:37 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:37.284+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:37.458 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:37 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:37.284+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:37.459 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:37 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:37.284+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:37.459 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:37 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:37.284+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:11.616844+0000 front 2023-09-13T04:52:11.616866+0000 (oldest deadline 2023-09-13T04:52:36.317047+0000) 2023-09-13T04:52:37.755 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:37 smithi002 bash[179969]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-6 2023-09-13T04:52:37.858 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:37 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:37.397+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:37.858 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:37 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:37.397+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:01.893556+0000 front 2023-09-13T04:52:01.893478+0000 (oldest deadline 2023-09-13T04:52:27.793569+0000) 2023-09-13T04:52:37.859 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:37 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:37.397+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.793957+0000 front 2023-09-13T04:52:07.794079+0000 (oldest deadline 2023-09-13T04:52:33.694128+0000) 2023-09-13T04:52:38.136 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:37 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:37.858+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:38.137 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:37 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:37.858+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:38.137 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:37 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:37.858+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:38.436 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:38 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:38.237+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:38.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:38 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:38.237+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:38.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:38 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:38.237+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:12.821347+0000 front 2023-09-13T04:52:12.821274+0000 (oldest deadline 2023-09-13T04:52:36.921110+0000) 2023-09-13T04:52:38.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:38 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:38.178+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:38.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:38 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:38.178+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:38.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:38 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:38.178+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:38.755 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:38 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:38.271+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:38.756 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:38 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:38.271+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:38.756 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:38 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:38.271+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:38.756 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:38 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:38.271+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:11.616844+0000 front 2023-09-13T04:52:11.616866+0000 (oldest deadline 2023-09-13T04:52:36.317047+0000) 2023-09-13T04:52:38.841 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:38 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:38.392+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:38.841 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:38 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:38.392+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:01.893556+0000 front 2023-09-13T04:52:01.893478+0000 (oldest deadline 2023-09-13T04:52:27.793569+0000) 2023-09-13T04:52:38.842 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:38 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:38.392+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.793957+0000 front 2023-09-13T04:52:07.794079+0000 (oldest deadline 2023-09-13T04:52:33.694128+0000) 2023-09-13T04:52:39.136 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:38 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:38.841+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:39.137 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:38 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:38.841+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:39.137 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:38 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:38.841+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:39.502 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.6.service' 2023-09-13T04:52:39.565 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:39 smithi002 systemd[1]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.6.service: Succeeded. 2023-09-13T04:52:39.566 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 13 04:52:39 smithi002 systemd[1]: Stopped Ceph osd.6 for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:52:39.566 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:39 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:39.268+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:39.566 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:39 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:39.268+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:39.567 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:39 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:39.268+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:39.567 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:39 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:39.268+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:11.616844+0000 front 2023-09-13T04:52:11.616866+0000 (oldest deadline 2023-09-13T04:52:36.317047+0000) 2023-09-13T04:52:39.593 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:52:39.593 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-09-13T04:52:39.594 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-09-13T04:52:39.594 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.7 2023-09-13T04:52:39.626 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:39 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:39.345+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:39.627 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:39 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:39.345+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:01.893556+0000 front 2023-09-13T04:52:01.893478+0000 (oldest deadline 2023-09-13T04:52:27.793569+0000) 2023-09-13T04:52:39.627 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:39 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:39.345+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.793957+0000 front 2023-09-13T04:52:07.794079+0000 (oldest deadline 2023-09-13T04:52:33.694128+0000) 2023-09-13T04:52:39.687 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:39 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:39.253+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:39.687 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:39 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:39.253+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:39.687 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:39 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:39.253+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:12.821347+0000 front 2023-09-13T04:52:12.821274+0000 (oldest deadline 2023-09-13T04:52:36.921110+0000) 2023-09-13T04:52:39.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:39 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:39.186+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:39.688 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:39 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:39.186+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:39.688 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:39 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:39.186+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:39.887 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:39 smithi149 systemd[1]: Stopping Ceph osd.7 for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:52:39.887 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:39 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:39.866+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:39.888 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:39 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:39.866+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:39.888 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:39 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:39.866+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:39.888 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:39 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:39.866+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:52:40.386 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:39 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:39.949+0000 7f531dfa9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:52:40.387 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:39 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:39.949+0000 7f531dfa9700 -1 osd.7 109 *** Got signal Terminated *** 2023-09-13T04:52:40.387 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:39 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:39.949+0000 7f531dfa9700 -1 osd.7 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-13T04:52:40.436 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:40 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:40.247+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:40.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:40 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:40.247+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:40.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:40 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:40.247+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:12.821347+0000 front 2023-09-13T04:52:12.821274+0000 (oldest deadline 2023-09-13T04:52:36.921110+0000) 2023-09-13T04:52:40.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:40 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:40.169+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:40.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:40 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:40.169+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:40.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:40 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:40.169+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:40.755 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:40 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:40.258+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:40.756 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:40 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:40.258+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:40.756 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:40 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:40.258+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:40.756 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:40 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:40.258+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:11.616844+0000 front 2023-09-13T04:52:11.616866+0000 (oldest deadline 2023-09-13T04:52:36.317047+0000) 2023-09-13T04:52:40.886 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:40 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:40.389+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:40.886 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:40 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:40.389+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:01.893556+0000 front 2023-09-13T04:52:01.893478+0000 (oldest deadline 2023-09-13T04:52:27.793569+0000) 2023-09-13T04:52:40.887 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:40 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:40.389+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.793957+0000 front 2023-09-13T04:52:07.794079+0000 (oldest deadline 2023-09-13T04:52:33.694128+0000) 2023-09-13T04:52:41.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:40 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:40.894+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:41.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:40 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:40.895+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:41.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:40 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:40.895+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:41.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:40 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:40.895+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:52:41.436 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:41 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:41.198+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:41.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:41 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:41.198+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:41.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:41 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:41.198+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:12.821347+0000 front 2023-09-13T04:52:12.821274+0000 (oldest deadline 2023-09-13T04:52:36.921110+0000) 2023-09-13T04:52:41.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:41 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:41.153+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:41.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:41 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:41.153+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:41.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:41 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:41.153+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:41.505 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:41 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:41.231+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:41.505 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:41 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:41.231+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:41.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:41 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:41.231+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:41.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:41 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:41.231+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:11.616844+0000 front 2023-09-13T04:52:11.616866+0000 (oldest deadline 2023-09-13T04:52:36.317047+0000) 2023-09-13T04:52:41.886 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:41 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:41.413+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:41.886 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:41 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:41.413+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:01.893556+0000 front 2023-09-13T04:52:01.893478+0000 (oldest deadline 2023-09-13T04:52:27.793569+0000) 2023-09-13T04:52:41.887 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:41 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:41.413+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.793957+0000 front 2023-09-13T04:52:07.794079+0000 (oldest deadline 2023-09-13T04:52:33.694128+0000) 2023-09-13T04:52:42.379 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:41 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:41.929+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:42.381 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:41 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:41.929+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:42.381 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:41 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:41.929+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:42.381 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:41 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:41.929+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:52:42.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:42 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:42.180+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:42.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:42 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:42.180+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:42.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:42 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:42.180+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:12.821347+0000 front 2023-09-13T04:52:12.821274+0000 (oldest deadline 2023-09-13T04:52:36.921110+0000) 2023-09-13T04:52:42.438 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:42 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:42.180+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:19.122756+0000 front 2023-09-13T04:52:19.122834+0000 (oldest deadline 2023-09-13T04:52:42.022806+0000) 2023-09-13T04:52:42.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:42 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:42.125+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:42.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:42 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:42.125+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:42.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:42 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:42.125+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:42.636 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:42 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:42.378+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:42.637 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:42 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:42.378+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:01.893556+0000 front 2023-09-13T04:52:01.893478+0000 (oldest deadline 2023-09-13T04:52:27.793569+0000) 2023-09-13T04:52:42.637 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:42 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:42.378+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.793957+0000 front 2023-09-13T04:52:07.794079+0000 (oldest deadline 2023-09-13T04:52:33.694128+0000) 2023-09-13T04:52:42.755 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:42 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:42.269+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:42.755 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:42 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:42.269+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:42.756 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:42 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:42.269+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:42.756 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:42 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:42.269+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:11.616844+0000 front 2023-09-13T04:52:11.616866+0000 (oldest deadline 2023-09-13T04:52:36.317047+0000) 2023-09-13T04:52:43.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:42 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:42.951+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:43.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:42 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:42.951+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:43.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:42 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:42.951+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:43.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:42 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:42.951+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:52:43.436 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:43 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:43.083+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:43.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:43 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:43.083+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:43.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:43 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:43.083+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:43.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:43 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:43.189+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:43.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:43 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:43.189+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:43.438 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:43 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:43.189+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:12.821347+0000 front 2023-09-13T04:52:12.821274+0000 (oldest deadline 2023-09-13T04:52:36.921110+0000) 2023-09-13T04:52:43.438 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:43 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:43.189+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:19.122756+0000 front 2023-09-13T04:52:19.122834+0000 (oldest deadline 2023-09-13T04:52:42.022806+0000) 2023-09-13T04:52:43.755 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:43 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:43.265+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:43.756 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:43 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:43.265+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:43.756 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:43 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:43.265+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:43.756 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:43 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:43.265+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:11.616844+0000 front 2023-09-13T04:52:11.616866+0000 (oldest deadline 2023-09-13T04:52:36.317047+0000) 2023-09-13T04:52:43.886 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:43 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:43.427+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:43.887 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:43 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:43.427+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:01.893556+0000 front 2023-09-13T04:52:01.893478+0000 (oldest deadline 2023-09-13T04:52:27.793569+0000) 2023-09-13T04:52:43.887 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:43 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:43.427+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.793957+0000 front 2023-09-13T04:52:07.794079+0000 (oldest deadline 2023-09-13T04:52:33.694128+0000) 2023-09-13T04:52:43.887 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:43 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:43.427+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.095664+0000 front 2023-09-13T04:52:20.095624+0000 (oldest deadline 2023-09-13T04:52:42.995799+0000) 2023-09-13T04:52:44.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:43 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:43.974+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:44.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:43 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:43.974+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:44.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:43 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:43.974+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:44.388 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:43 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:43.974+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:52:44.436 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:44 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:44.210+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:44.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:44 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:44.210+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:44.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:44 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:44.210+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:12.821347+0000 front 2023-09-13T04:52:12.821274+0000 (oldest deadline 2023-09-13T04:52:36.921110+0000) 2023-09-13T04:52:44.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:44 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:44.210+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:19.122756+0000 front 2023-09-13T04:52:19.122834+0000 (oldest deadline 2023-09-13T04:52:42.022806+0000) 2023-09-13T04:52:44.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:44 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:44.087+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:44.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:44 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:44.087+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:44.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:44 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:44.087+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:44.755 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:44 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:44.264+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:44.756 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:44 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:44.264+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:44.756 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:44 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:44.264+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:44.756 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:44 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:44.264+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:11.616844+0000 front 2023-09-13T04:52:11.616866+0000 (oldest deadline 2023-09-13T04:52:36.317047+0000) 2023-09-13T04:52:44.886 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:44 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:44.441+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.892280+0000 front 2023-09-13T04:51:54.892151+0000 (oldest deadline 2023-09-13T04:52:20.192369+0000) 2023-09-13T04:52:44.887 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:44 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:44.441+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:01.893556+0000 front 2023-09-13T04:52:01.893478+0000 (oldest deadline 2023-09-13T04:52:27.793569+0000) 2023-09-13T04:52:44.887 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:44 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:44.441+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.793957+0000 front 2023-09-13T04:52:07.794079+0000 (oldest deadline 2023-09-13T04:52:33.694128+0000) 2023-09-13T04:52:44.887 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:44 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7[154857]: 2023-09-13T04:52:44.441+0000 7f531859b700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.095664+0000 front 2023-09-13T04:52:20.095624+0000 (oldest deadline 2023-09-13T04:52:42.995799+0000) 2023-09-13T04:52:45.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:44 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:44.983+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:45.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:44 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:44.983+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:45.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:44 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:44.983+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:45.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:44 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:44.983+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:52:45.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:44 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:44.983+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:21.191553+0000 front 2023-09-13T04:52:21.191599+0000 (oldest deadline 2023-09-13T04:52:44.691812+0000) 2023-09-13T04:52:45.436 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:45 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:45.057+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:45.436 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:45 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:45.057+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:45.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:45 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:45.057+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:45.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:45 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:45.252+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:45.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:45 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:45.252+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:45.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:45 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:45.252+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:12.821347+0000 front 2023-09-13T04:52:12.821274+0000 (oldest deadline 2023-09-13T04:52:36.921110+0000) 2023-09-13T04:52:45.438 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:45 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:45.252+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:19.122756+0000 front 2023-09-13T04:52:19.122834+0000 (oldest deadline 2023-09-13T04:52:42.022806+0000) 2023-09-13T04:52:45.505 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:45 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:45.236+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:45.505 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:45 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:45.236+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:45.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:45 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:45.236+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:45.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:45 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:45.236+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:11.616844+0000 front 2023-09-13T04:52:11.616866+0000 (oldest deadline 2023-09-13T04:52:36.317047+0000) 2023-09-13T04:52:45.837 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 13 04:52:45 smithi149 bash[177058]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-7 2023-09-13T04:52:46.369 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:45 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:45.958+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:46.370 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:45 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:45.958+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:46.370 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:45 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:45.958+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:46.370 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:45 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:45.958+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:52:46.370 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:45 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:45.958+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:21.191553+0000 front 2023-09-13T04:52:21.191599+0000 (oldest deadline 2023-09-13T04:52:44.691812+0000) 2023-09-13T04:52:46.436 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:46 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:46.290+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:46.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:46 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:46.290+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:46.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:46 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:46.290+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:12.821347+0000 front 2023-09-13T04:52:12.821274+0000 (oldest deadline 2023-09-13T04:52:36.921110+0000) 2023-09-13T04:52:46.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:46 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:46.290+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:19.122756+0000 front 2023-09-13T04:52:19.122834+0000 (oldest deadline 2023-09-13T04:52:42.022806+0000) 2023-09-13T04:52:46.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:46 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:46.064+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:46.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:46 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:46.064+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:46.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:46 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:46.064+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:46.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:46 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:46.064+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:52:46.505 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:46 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:46.218+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:46.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:46 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:46.218+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:46.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:46 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:46.218+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:46.507 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:46 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:46.218+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:11.616844+0000 front 2023-09-13T04:52:11.616866+0000 (oldest deadline 2023-09-13T04:52:36.317047+0000) 2023-09-13T04:52:46.507 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:46 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:46.218+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.418404+0000 front 2023-09-13T04:52:20.418438+0000 (oldest deadline 2023-09-13T04:52:45.718285+0000) 2023-09-13T04:52:47.099 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.7.service' 2023-09-13T04:52:47.180 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:52:47.180 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-09-13T04:52:47.181 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-09-13T04:52:47.181 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.8 2023-09-13T04:52:47.320 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:47 smithi156 systemd[1]: Stopping Ceph osd.8 for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:52:47.320 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:47 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:47.056+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:47.320 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:47 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:47.056+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:47.321 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:47 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:47.056+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:47.321 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:47 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:47.056+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:52:47.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:46 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:46.995+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:47.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:46 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:46.995+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:47.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:46 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:46.995+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:47.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:46 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:46.995+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:52:47.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:46 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:46.995+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:21.191553+0000 front 2023-09-13T04:52:21.191599+0000 (oldest deadline 2023-09-13T04:52:44.691812+0000) 2023-09-13T04:52:47.505 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:47 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:47.208+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:47.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:47 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:47.208+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:47.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:47 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:47.208+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:47.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:47 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:47.208+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:11.616844+0000 front 2023-09-13T04:52:11.616866+0000 (oldest deadline 2023-09-13T04:52:36.317047+0000) 2023-09-13T04:52:47.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:47 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:47.208+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.418404+0000 front 2023-09-13T04:52:20.418438+0000 (oldest deadline 2023-09-13T04:52:45.718285+0000) 2023-09-13T04:52:47.686 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:47 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:47.318+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:47.687 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:47 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:47.318+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:47.687 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:47 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:47.318+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:12.821347+0000 front 2023-09-13T04:52:12.821274+0000 (oldest deadline 2023-09-13T04:52:36.921110+0000) 2023-09-13T04:52:47.687 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:47 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:47.318+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:19.122756+0000 front 2023-09-13T04:52:19.122834+0000 (oldest deadline 2023-09-13T04:52:42.022806+0000) 2023-09-13T04:52:47.687 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:47 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:47.549+0000 7f79a0cbf700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:52:47.688 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:47 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:47.549+0000 7f79a0cbf700 -1 osd.8 109 *** Got signal Terminated *** 2023-09-13T04:52:47.688 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:47 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:47.549+0000 7f79a0cbf700 -1 osd.8 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-13T04:52:48.318 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:48 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:48.052+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:48.319 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:48 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:48.052+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:48.319 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:48 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:48.052+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:48.319 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:48 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:48.052+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:52:48.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:47 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:47.960+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:48.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:47 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:47.960+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:48.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:47 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:47.960+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:48.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:47 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:47.960+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:52:48.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:47 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:47.960+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:21.191553+0000 front 2023-09-13T04:52:21.191599+0000 (oldest deadline 2023-09-13T04:52:44.691812+0000) 2023-09-13T04:52:48.505 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:48 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:48.235+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:48.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:48 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:48.235+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:48.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:48 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:48.235+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:48.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:48 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:48.235+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:11.616844+0000 front 2023-09-13T04:52:11.616866+0000 (oldest deadline 2023-09-13T04:52:36.317047+0000) 2023-09-13T04:52:48.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:48 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:48.235+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.418404+0000 front 2023-09-13T04:52:20.418438+0000 (oldest deadline 2023-09-13T04:52:45.718285+0000) 2023-09-13T04:52:48.686 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:48 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:48.318+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:48.687 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:48 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:48.318+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:48.687 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:48 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:48.318+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:12.821347+0000 front 2023-09-13T04:52:12.821274+0000 (oldest deadline 2023-09-13T04:52:36.921110+0000) 2023-09-13T04:52:48.687 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:48 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:48.318+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:19.122756+0000 front 2023-09-13T04:52:19.122834+0000 (oldest deadline 2023-09-13T04:52:42.022806+0000) 2023-09-13T04:52:49.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:49 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:49.003+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:49.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:49 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:49.003+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:49.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:49 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:49.003+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:49.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:49 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:49.003+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:52:49.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:49 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:49.003+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:21.191553+0000 front 2023-09-13T04:52:21.191599+0000 (oldest deadline 2023-09-13T04:52:44.691812+0000) 2023-09-13T04:52:49.436 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:49 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:49.329+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:49.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:49 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:49.329+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:49.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:49 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:49.329+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:12.821347+0000 front 2023-09-13T04:52:12.821274+0000 (oldest deadline 2023-09-13T04:52:36.921110+0000) 2023-09-13T04:52:49.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:49 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:49.329+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:19.122756+0000 front 2023-09-13T04:52:19.122834+0000 (oldest deadline 2023-09-13T04:52:42.022806+0000) 2023-09-13T04:52:49.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:49 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:49.094+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:49.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:49 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:49.094+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:49.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:49 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:49.094+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:49.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:49 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:49.094+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:52:49.505 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:49 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:49.217+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:49.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:49 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:49.217+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:49.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:49 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:49.217+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:49.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:49 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:49.217+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:11.616844+0000 front 2023-09-13T04:52:11.616866+0000 (oldest deadline 2023-09-13T04:52:36.317047+0000) 2023-09-13T04:52:49.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:49 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:49.217+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.418404+0000 front 2023-09-13T04:52:20.418438+0000 (oldest deadline 2023-09-13T04:52:45.718285+0000) 2023-09-13T04:52:50.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:50 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:50.027+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:50.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:50 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:50.027+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:50.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:50 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:50.027+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:50.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:50 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:50.027+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:52:50.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:50 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:50.027+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:21.191553+0000 front 2023-09-13T04:52:21.191599+0000 (oldest deadline 2023-09-13T04:52:44.691812+0000) 2023-09-13T04:52:50.436 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:50 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:50.361+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:50.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:50 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:50.361+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:50.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:50 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:50.361+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:12.821347+0000 front 2023-09-13T04:52:12.821274+0000 (oldest deadline 2023-09-13T04:52:36.921110+0000) 2023-09-13T04:52:50.437 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:50 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:50.361+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:19.122756+0000 front 2023-09-13T04:52:19.122834+0000 (oldest deadline 2023-09-13T04:52:42.022806+0000) 2023-09-13T04:52:50.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:50 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:50.124+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:50.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:50 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:50.124+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:50.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:50 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:50.124+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:50.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:50 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:50.124+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:52:50.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:50 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:50.124+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:52:50.505 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:50 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:50.188+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:50.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:50 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:50.188+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:50.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:50 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:50.188+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:50.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:50 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:50.188+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:11.616844+0000 front 2023-09-13T04:52:11.616866+0000 (oldest deadline 2023-09-13T04:52:36.317047+0000) 2023-09-13T04:52:50.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:50 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:50.188+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.418404+0000 front 2023-09-13T04:52:20.418438+0000 (oldest deadline 2023-09-13T04:52:45.718285+0000) 2023-09-13T04:52:50.507 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:50 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:50.188+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.718852+0000 front 2023-09-13T04:52:25.718873+0000 (oldest deadline 2023-09-13T04:52:49.818931+0000) 2023-09-13T04:52:51.382 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:51 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:51.096+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:51.382 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:51 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:51.096+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:51.382 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:51 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:51.096+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:51.383 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:51 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:51.096+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:52:51.383 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:51 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:51.096+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:52:51.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:51 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:51.069+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:51.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:51 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:51.069+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:51.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:51 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:51.069+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:51.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:51 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:51.069+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:52:51.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:51 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:51.069+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:21.191553+0000 front 2023-09-13T04:52:21.191599+0000 (oldest deadline 2023-09-13T04:52:44.691812+0000) 2023-09-13T04:52:51.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:51 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:51.069+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.193149+0000 front 2023-09-13T04:52:25.193078+0000 (oldest deadline 2023-09-13T04:52:50.493119+0000) 2023-09-13T04:52:51.505 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:51 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:51.145+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:51.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:51 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:51.145+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:51.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:51 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:51.145+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:51.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:51 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:51.145+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:11.616844+0000 front 2023-09-13T04:52:11.616866+0000 (oldest deadline 2023-09-13T04:52:36.317047+0000) 2023-09-13T04:52:51.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:51 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:51.145+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.418404+0000 front 2023-09-13T04:52:20.418438+0000 (oldest deadline 2023-09-13T04:52:45.718285+0000) 2023-09-13T04:52:51.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:51 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:51.145+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.718852+0000 front 2023-09-13T04:52:25.718873+0000 (oldest deadline 2023-09-13T04:52:49.818931+0000) 2023-09-13T04:52:51.686 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:51 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:51.380+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:51.687 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:51 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:51.380+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:51.687 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:51 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:51.380+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:12.821347+0000 front 2023-09-13T04:52:12.821274+0000 (oldest deadline 2023-09-13T04:52:36.921110+0000) 2023-09-13T04:52:51.687 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:51 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:51.380+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:19.122756+0000 front 2023-09-13T04:52:19.122834+0000 (oldest deadline 2023-09-13T04:52:42.022806+0000) 2023-09-13T04:52:51.687 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:51 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:51.380+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:26.024529+0000 front 2023-09-13T04:52:26.024566+0000 (oldest deadline 2023-09-13T04:52:51.324616+0000) 2023-09-13T04:52:52.386 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:52 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:52.127+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:52.386 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:52 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:52.127+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:52.386 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:52 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:52.127+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:52.386 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:52 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:52.127+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:52:52.387 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:52 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:52.127+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:52:52.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:52 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:52.025+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:52.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:52 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:52.025+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:52.388 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:52 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:52.025+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:52.388 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:52 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:52.025+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:52:52.388 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:52 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:52.025+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:21.191553+0000 front 2023-09-13T04:52:21.191599+0000 (oldest deadline 2023-09-13T04:52:44.691812+0000) 2023-09-13T04:52:52.388 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:52 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:52.025+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.193149+0000 front 2023-09-13T04:52:25.193078+0000 (oldest deadline 2023-09-13T04:52:50.493119+0000) 2023-09-13T04:52:52.505 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:52 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:52.189+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:52.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:52 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:52.189+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:52.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:52 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:52.189+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:52.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:52 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:52.189+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:11.616844+0000 front 2023-09-13T04:52:11.616866+0000 (oldest deadline 2023-09-13T04:52:36.317047+0000) 2023-09-13T04:52:52.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:52 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:52.189+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.418404+0000 front 2023-09-13T04:52:20.418438+0000 (oldest deadline 2023-09-13T04:52:45.718285+0000) 2023-09-13T04:52:52.507 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:52 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:52.189+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.718852+0000 front 2023-09-13T04:52:25.718873+0000 (oldest deadline 2023-09-13T04:52:49.818931+0000) 2023-09-13T04:52:52.686 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:52 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:52.385+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:56.017496+0000 front 2023-09-13T04:51:56.017290+0000 (oldest deadline 2023-09-13T04:52:20.717258+0000) 2023-09-13T04:52:52.687 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:52 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:52.385+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:02.418334+0000 front 2023-09-13T04:52:02.418309+0000 (oldest deadline 2023-09-13T04:52:27.718694+0000) 2023-09-13T04:52:52.687 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:52 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:52.385+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:12.821347+0000 front 2023-09-13T04:52:12.821274+0000 (oldest deadline 2023-09-13T04:52:36.921110+0000) 2023-09-13T04:52:52.687 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:52 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:52.385+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:19.122756+0000 front 2023-09-13T04:52:19.122834+0000 (oldest deadline 2023-09-13T04:52:42.022806+0000) 2023-09-13T04:52:52.687 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:52 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8[156098]: 2023-09-13T04:52:52.385+0000 7f799a2af700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:26.024529+0000 front 2023-09-13T04:52:26.024566+0000 (oldest deadline 2023-09-13T04:52:51.324616+0000) 2023-09-13T04:52:53.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:53 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:53.081+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:53.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:53 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:53.081+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:53.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:53 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:53.081+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:53.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:53 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:53.081+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:52:53.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:53 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:53.081+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:52:53.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:53 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:53.012+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:53.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:53 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:53.013+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:53.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:53 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:53.013+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:53.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:53 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:53.013+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:52:53.388 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:53 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:53.013+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:21.191553+0000 front 2023-09-13T04:52:21.191599+0000 (oldest deadline 2023-09-13T04:52:44.691812+0000) 2023-09-13T04:52:53.388 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:53 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:53.013+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.193149+0000 front 2023-09-13T04:52:25.193078+0000 (oldest deadline 2023-09-13T04:52:50.493119+0000) 2023-09-13T04:52:53.505 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:53 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:53.185+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:53.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:53 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:53.185+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:53.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:53 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:53.185+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:53.507 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:53 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:53.185+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:11.616844+0000 front 2023-09-13T04:52:11.616866+0000 (oldest deadline 2023-09-13T04:52:36.317047+0000) 2023-09-13T04:52:53.507 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:53 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:53.185+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.418404+0000 front 2023-09-13T04:52:20.418438+0000 (oldest deadline 2023-09-13T04:52:45.718285+0000) 2023-09-13T04:52:53.507 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:53 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:53.185+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.718852+0000 front 2023-09-13T04:52:25.718873+0000 (oldest deadline 2023-09-13T04:52:49.818931+0000) 2023-09-13T04:52:53.686 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 13 04:52:53 smithi156 bash[180004]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-8 2023-09-13T04:52:54.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:54 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:54.056+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:54.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:54 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:54.056+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:54.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:54 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:54.056+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:54.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:54 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:54.056+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:52:54.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:54 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:54.056+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:21.191553+0000 front 2023-09-13T04:52:21.191599+0000 (oldest deadline 2023-09-13T04:52:44.691812+0000) 2023-09-13T04:52:54.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:54 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:54.056+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.193149+0000 front 2023-09-13T04:52:25.193078+0000 (oldest deadline 2023-09-13T04:52:50.493119+0000) 2023-09-13T04:52:54.436 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:54 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:54.097+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:54.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:54 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:54.097+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:54.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:54 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:54.097+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:54.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:54 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:54.097+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:52:54.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:54 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:54.097+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:52:54.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:54 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:54.174+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:54.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:54 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:54.174+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:54.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:54 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:54.174+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:54.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:54 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:54.174+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:11.616844+0000 front 2023-09-13T04:52:11.616866+0000 (oldest deadline 2023-09-13T04:52:36.317047+0000) 2023-09-13T04:52:54.507 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:54 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:54.174+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.418404+0000 front 2023-09-13T04:52:20.418438+0000 (oldest deadline 2023-09-13T04:52:45.718285+0000) 2023-09-13T04:52:54.507 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:54 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:54.174+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.718852+0000 front 2023-09-13T04:52:25.718873+0000 (oldest deadline 2023-09-13T04:52:49.818931+0000) 2023-09-13T04:52:54.820 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.8.service' 2023-09-13T04:52:54.861 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:52:54.862 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-09-13T04:52:54.862 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-09-13T04:52:54.862 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.9 2023-09-13T04:52:55.256 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:54 smithi002 systemd[1]: Stopping Ceph osd.9 for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:52:55.256 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:55 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:55.126+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:55.256 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:55 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:55.126+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:55.256 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:55 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:55.126+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:55.257 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:55 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:55.126+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:11.616844+0000 front 2023-09-13T04:52:11.616866+0000 (oldest deadline 2023-09-13T04:52:36.317047+0000) 2023-09-13T04:52:55.257 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:55 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:55.126+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.418404+0000 front 2023-09-13T04:52:20.418438+0000 (oldest deadline 2023-09-13T04:52:45.718285+0000) 2023-09-13T04:52:55.257 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:55 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:55.126+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.718852+0000 front 2023-09-13T04:52:25.718873+0000 (oldest deadline 2023-09-13T04:52:49.818931+0000) 2023-09-13T04:52:55.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:55 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:55.042+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:55.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:55 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:55.042+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:55.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:55 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:55.042+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:55.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:55 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:55.042+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:52:55.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:55 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:55.042+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:21.191553+0000 front 2023-09-13T04:52:21.191599+0000 (oldest deadline 2023-09-13T04:52:44.691812+0000) 2023-09-13T04:52:55.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:55 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:55.042+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.193149+0000 front 2023-09-13T04:52:25.193078+0000 (oldest deadline 2023-09-13T04:52:50.493119+0000) 2023-09-13T04:52:55.436 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:55 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:55.131+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:55.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:55 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:55.131+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:55.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:55 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:55.131+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:55.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:55 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:55.131+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:52:55.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:55 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:55.131+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:52:55.755 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:55 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:55.384+0000 7f6dd2763700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:52:55.756 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:55 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:55.384+0000 7f6dd2763700 -1 osd.9 109 *** Got signal Terminated *** 2023-09-13T04:52:55.756 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:55 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:55.384+0000 7f6dd2763700 -1 osd.9 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-13T04:52:56.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:56 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:56.019+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:56.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:56 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:56.019+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:56.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:56 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:56.019+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:56.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:56 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:56.019+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:52:56.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:56 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:56.019+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:21.191553+0000 front 2023-09-13T04:52:21.191599+0000 (oldest deadline 2023-09-13T04:52:44.691812+0000) 2023-09-13T04:52:56.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:56 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:56.019+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.193149+0000 front 2023-09-13T04:52:25.193078+0000 (oldest deadline 2023-09-13T04:52:50.493119+0000) 2023-09-13T04:52:56.436 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:56 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:56.100+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:56.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:56 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:56.100+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:56.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:56 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:56.100+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:56.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:56 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:56.100+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:52:56.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:56 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:56.100+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:52:56.505 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:56 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:56.080+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:56.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:56 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:56.080+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:56.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:56 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:56.080+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:56.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:56 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:56.080+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:11.616844+0000 front 2023-09-13T04:52:11.616866+0000 (oldest deadline 2023-09-13T04:52:36.317047+0000) 2023-09-13T04:52:56.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:56 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:56.080+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.418404+0000 front 2023-09-13T04:52:20.418438+0000 (oldest deadline 2023-09-13T04:52:45.718285+0000) 2023-09-13T04:52:56.507 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:56 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:56.080+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.718852+0000 front 2023-09-13T04:52:25.718873+0000 (oldest deadline 2023-09-13T04:52:49.818931+0000) 2023-09-13T04:52:57.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:57 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:57.001+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:57.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:57 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:57.001+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:57.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:57 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:57.001+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:57.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:57 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:57.001+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:52:57.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:57 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:57.001+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:21.191553+0000 front 2023-09-13T04:52:21.191599+0000 (oldest deadline 2023-09-13T04:52:44.691812+0000) 2023-09-13T04:52:57.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:57 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:57.001+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.193149+0000 front 2023-09-13T04:52:25.193078+0000 (oldest deadline 2023-09-13T04:52:50.493119+0000) 2023-09-13T04:52:57.436 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:57 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:57.148+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:57.436 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:57 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:57.148+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:57.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:57 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:57.148+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:57.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:57 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:57.148+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:52:57.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:57 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:57.148+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:52:57.505 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:57 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:57.088+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:57.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:57 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:57.088+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:57.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:57 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:57.088+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:57.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:57 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:57.088+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:11.616844+0000 front 2023-09-13T04:52:11.616866+0000 (oldest deadline 2023-09-13T04:52:36.317047+0000) 2023-09-13T04:52:57.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:57 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:57.088+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.418404+0000 front 2023-09-13T04:52:20.418438+0000 (oldest deadline 2023-09-13T04:52:45.718285+0000) 2023-09-13T04:52:57.507 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:57 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:57.088+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.718852+0000 front 2023-09-13T04:52:25.718873+0000 (oldest deadline 2023-09-13T04:52:49.818931+0000) 2023-09-13T04:52:58.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:58 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:58.042+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:58.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:58 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:58.042+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:58.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:58 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:58.042+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:58.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:58 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:58.042+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:52:58.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:58 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:58.042+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:21.191553+0000 front 2023-09-13T04:52:21.191599+0000 (oldest deadline 2023-09-13T04:52:44.691812+0000) 2023-09-13T04:52:58.388 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:58 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:58.042+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.193149+0000 front 2023-09-13T04:52:25.193078+0000 (oldest deadline 2023-09-13T04:52:50.493119+0000) 2023-09-13T04:52:58.436 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:58 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:58.102+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:58.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:58 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:58.102+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:58.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:58 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:58.102+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:58.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:58 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:58.102+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:52:58.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:58 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:58.102+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:52:58.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:58 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:58.102+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:33.768678+0000 front 2023-09-13T04:52:33.768757+0000 (oldest deadline 2023-09-13T04:52:57.868723+0000) 2023-09-13T04:52:58.505 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:58 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:58.114+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:58.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:58 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:58.114+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:58.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:58 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:58.114+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:58.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:58 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:58.114+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:11.616844+0000 front 2023-09-13T04:52:11.616866+0000 (oldest deadline 2023-09-13T04:52:36.317047+0000) 2023-09-13T04:52:58.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:58 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:58.114+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.418404+0000 front 2023-09-13T04:52:20.418438+0000 (oldest deadline 2023-09-13T04:52:45.718285+0000) 2023-09-13T04:52:58.507 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:58 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:58.114+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.718852+0000 front 2023-09-13T04:52:25.718873+0000 (oldest deadline 2023-09-13T04:52:49.818931+0000) 2023-09-13T04:52:59.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:59 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:59.080+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:52:59.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:59 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:59.080+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:52:59.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:59 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:59.080+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:52:59.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:59 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:59.080+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:52:59.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:59 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:59.080+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:21.191553+0000 front 2023-09-13T04:52:21.191599+0000 (oldest deadline 2023-09-13T04:52:44.691812+0000) 2023-09-13T04:52:59.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:52:59 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:52:59.080+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.193149+0000 front 2023-09-13T04:52:25.193078+0000 (oldest deadline 2023-09-13T04:52:50.493119+0000) 2023-09-13T04:52:59.436 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:59 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:59.152+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:52:59.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:59 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:59.152+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:52:59.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:59 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:59.152+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:52:59.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:59 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:59.152+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:52:59.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:59 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:59.152+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:52:59.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:52:59 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:52:59.152+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:33.768678+0000 front 2023-09-13T04:52:33.768757+0000 (oldest deadline 2023-09-13T04:52:57.868723+0000) 2023-09-13T04:52:59.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:59 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:59.092+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:52:59.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:59 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:59.092+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:52:59.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:59 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:59.092+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:52:59.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:59 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:59.092+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:11.616844+0000 front 2023-09-13T04:52:11.616866+0000 (oldest deadline 2023-09-13T04:52:36.317047+0000) 2023-09-13T04:52:59.507 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:59 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:59.092+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.418404+0000 front 2023-09-13T04:52:20.418438+0000 (oldest deadline 2023-09-13T04:52:45.718285+0000) 2023-09-13T04:52:59.507 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:52:59 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:52:59.092+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.718852+0000 front 2023-09-13T04:52:25.718873+0000 (oldest deadline 2023-09-13T04:52:49.818931+0000) 2023-09-13T04:53:00.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:00 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:00.054+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:53:00.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:00 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:00.054+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:53:00.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:00 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:00.054+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:53:00.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:00 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:00.054+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:53:00.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:00 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:00.054+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:21.191553+0000 front 2023-09-13T04:52:21.191599+0000 (oldest deadline 2023-09-13T04:52:44.691812+0000) 2023-09-13T04:53:00.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:00 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:00.054+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.193149+0000 front 2023-09-13T04:52:25.193078+0000 (oldest deadline 2023-09-13T04:52:50.493119+0000) 2023-09-13T04:53:00.388 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:00 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:00.054+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:34.196067+0000 front 2023-09-13T04:52:34.196195+0000 (oldest deadline 2023-09-13T04:52:59.996448+0000) 2023-09-13T04:53:00.436 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:00 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:00.112+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:53:00.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:00 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:00.112+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:53:00.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:00 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:00.112+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:53:00.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:00 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:00.112+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:53:00.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:00 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:00.112+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:53:00.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:00 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:00.112+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:33.768678+0000 front 2023-09-13T04:52:33.768757+0000 (oldest deadline 2023-09-13T04:52:57.868723+0000) 2023-09-13T04:53:00.505 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:53:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:53:00.082+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.810594+0000 front 2023-09-13T04:51:47.810633+0000 (oldest deadline 2023-09-13T04:52:10.110738+0000) 2023-09-13T04:53:00.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:53:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:53:00.082+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:53.612312+0000 front 2023-09-13T04:51:53.612154+0000 (oldest deadline 2023-09-13T04:52:18.912177+0000) 2023-09-13T04:53:00.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:53:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:53:00.082+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:03.514393+0000 front 2023-09-13T04:52:03.514564+0000 (oldest deadline 2023-09-13T04:52:26.414659+0000) 2023-09-13T04:53:00.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:53:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:53:00.082+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:11.616844+0000 front 2023-09-13T04:52:11.616866+0000 (oldest deadline 2023-09-13T04:52:36.317047+0000) 2023-09-13T04:53:00.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:53:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:53:00.082+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.418404+0000 front 2023-09-13T04:52:20.418438+0000 (oldest deadline 2023-09-13T04:52:45.718285+0000) 2023-09-13T04:53:00.507 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:53:00 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9[140297]: 2023-09-13T04:53:00.082+0000 7f6dccd55700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.718852+0000 front 2023-09-13T04:52:25.718873+0000 (oldest deadline 2023-09-13T04:52:49.818931+0000) 2023-09-13T04:53:01.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:01 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:01.013+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:53:01.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:01 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:01.013+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:53:01.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:01 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:01.013+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:53:01.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:01 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:01.013+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:53:01.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:01 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:01.013+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:21.191553+0000 front 2023-09-13T04:52:21.191599+0000 (oldest deadline 2023-09-13T04:52:44.691812+0000) 2023-09-13T04:53:01.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:01 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:01.013+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.193149+0000 front 2023-09-13T04:52:25.193078+0000 (oldest deadline 2023-09-13T04:52:50.493119+0000) 2023-09-13T04:53:01.388 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:01 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:01.013+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:34.196067+0000 front 2023-09-13T04:52:34.196195+0000 (oldest deadline 2023-09-13T04:52:59.996448+0000) 2023-09-13T04:53:01.436 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:01 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:01.156+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:53:01.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:01 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:01.156+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:53:01.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:01 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:01.156+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:53:01.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:01 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:01.156+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:53:01.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:01 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:01.156+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:53:01.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:01 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:01.156+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:33.768678+0000 front 2023-09-13T04:52:33.768757+0000 (oldest deadline 2023-09-13T04:52:57.868723+0000) 2023-09-13T04:53:01.506 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 13 04:53:01 smithi002 bash[180242]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-9 2023-09-13T04:53:02.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:02 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:02.027+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:53:02.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:02 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:02.027+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:53:02.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:02 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:02.027+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:53:02.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:02 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:02.027+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:53:02.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:02 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:02.027+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:21.191553+0000 front 2023-09-13T04:52:21.191599+0000 (oldest deadline 2023-09-13T04:52:44.691812+0000) 2023-09-13T04:53:02.388 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:02 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:02.027+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.193149+0000 front 2023-09-13T04:52:25.193078+0000 (oldest deadline 2023-09-13T04:52:50.493119+0000) 2023-09-13T04:53:02.388 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:02 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:02.027+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:34.196067+0000 front 2023-09-13T04:52:34.196195+0000 (oldest deadline 2023-09-13T04:52:59.996448+0000) 2023-09-13T04:53:02.436 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:02 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:02.129+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:53:02.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:02 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:02.129+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:53:02.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:02 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:02.129+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:53:02.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:02 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:02.129+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:53:02.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:02 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:02.129+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:53:02.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:02 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:02.129+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:33.768678+0000 front 2023-09-13T04:52:33.768757+0000 (oldest deadline 2023-09-13T04:52:57.868723+0000) 2023-09-13T04:53:03.157 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.9.service' 2023-09-13T04:53:03.248 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:53:03.248 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-09-13T04:53:03.248 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-09-13T04:53:03.249 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.10 2023-09-13T04:53:03.281 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:03 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:02.998+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:53:03.281 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:03 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:02.998+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:53:03.281 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:03 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:02.998+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:53:03.281 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:03 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:02.998+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:53:03.282 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:03 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:02.999+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:21.191553+0000 front 2023-09-13T04:52:21.191599+0000 (oldest deadline 2023-09-13T04:52:44.691812+0000) 2023-09-13T04:53:03.282 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:03 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:02.999+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.193149+0000 front 2023-09-13T04:52:25.193078+0000 (oldest deadline 2023-09-13T04:52:50.493119+0000) 2023-09-13T04:53:03.282 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:03 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:02.999+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:34.196067+0000 front 2023-09-13T04:52:34.196195+0000 (oldest deadline 2023-09-13T04:52:59.996448+0000) 2023-09-13T04:53:03.436 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:03.168+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:53:03.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:03.168+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:53:03.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:03.168+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:53:03.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:03.168+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:53:03.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:03.168+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:53:03.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:03 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:03.168+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:33.768678+0000 front 2023-09-13T04:52:33.768757+0000 (oldest deadline 2023-09-13T04:52:57.868723+0000) 2023-09-13T04:53:03.621 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:03 smithi149 systemd[1]: Stopping Ceph osd.10 for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:53:03.886 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:03 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:03.619+0000 7f586283e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:53:03.887 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:03 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:03.619+0000 7f586283e700 -1 osd.10 109 *** Got signal Terminated *** 2023-09-13T04:53:03.887 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:03 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:03.619+0000 7f586283e700 -1 osd.10 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-13T04:53:04.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:03 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:03.962+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:53:04.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:03 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:03.962+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:53:04.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:03 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:03.962+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:53:04.388 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:03 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:03.962+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:53:04.388 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:03 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:03.962+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:21.191553+0000 front 2023-09-13T04:52:21.191599+0000 (oldest deadline 2023-09-13T04:52:44.691812+0000) 2023-09-13T04:53:04.388 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:03 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:03.962+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.193149+0000 front 2023-09-13T04:52:25.193078+0000 (oldest deadline 2023-09-13T04:52:50.493119+0000) 2023-09-13T04:53:04.388 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:03 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:03.962+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:34.196067+0000 front 2023-09-13T04:52:34.196195+0000 (oldest deadline 2023-09-13T04:52:59.996448+0000) 2023-09-13T04:53:04.436 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:04 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:04.134+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:53:04.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:04 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:04.134+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:53:04.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:04 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:04.134+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:53:04.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:04 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:04.134+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:53:04.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:04 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:04.134+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:53:04.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:04 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:04.134+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:33.768678+0000 front 2023-09-13T04:52:33.768757+0000 (oldest deadline 2023-09-13T04:52:57.868723+0000) 2023-09-13T04:53:05.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:04 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:04.933+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:53:05.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:04 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:04.933+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:53:05.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:04 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:04.933+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:53:05.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:04 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:04.933+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:53:05.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:04 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:04.933+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:21.191553+0000 front 2023-09-13T04:52:21.191599+0000 (oldest deadline 2023-09-13T04:52:44.691812+0000) 2023-09-13T04:53:05.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:04 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:04.933+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.193149+0000 front 2023-09-13T04:52:25.193078+0000 (oldest deadline 2023-09-13T04:52:50.493119+0000) 2023-09-13T04:53:05.388 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:04 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:04.933+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:34.196067+0000 front 2023-09-13T04:52:34.196195+0000 (oldest deadline 2023-09-13T04:52:59.996448+0000) 2023-09-13T04:53:05.436 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:05 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:05.142+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:53:05.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:05 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:05.142+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:53:05.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:05 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:05.142+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:53:05.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:05 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:05.142+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:53:05.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:05 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:05.142+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:53:05.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:05 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:05.142+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:33.768678+0000 front 2023-09-13T04:52:33.768757+0000 (oldest deadline 2023-09-13T04:52:57.868723+0000) 2023-09-13T04:53:06.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:05 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:05.895+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:53:06.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:05 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:05.895+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:53:06.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:05 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:05.895+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:53:06.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:05 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:05.895+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:53:06.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:05 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:05.895+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:21.191553+0000 front 2023-09-13T04:52:21.191599+0000 (oldest deadline 2023-09-13T04:52:44.691812+0000) 2023-09-13T04:53:06.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:05 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:05.895+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.193149+0000 front 2023-09-13T04:52:25.193078+0000 (oldest deadline 2023-09-13T04:52:50.493119+0000) 2023-09-13T04:53:06.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:05 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:05.895+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:34.196067+0000 front 2023-09-13T04:52:34.196195+0000 (oldest deadline 2023-09-13T04:52:59.996448+0000) 2023-09-13T04:53:06.436 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:06 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:06.176+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:53:06.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:06 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:06.176+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:53:06.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:06 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:06.176+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:53:06.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:06 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:06.176+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:53:06.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:06 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:06.176+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:53:06.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:06 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:06.176+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:33.768678+0000 front 2023-09-13T04:52:33.768757+0000 (oldest deadline 2023-09-13T04:52:57.868723+0000) 2023-09-13T04:53:06.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:06 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:06.176+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2023-09-13T04:52:41.969808+0000 front 2023-09-13T04:52:41.969773+0000 (oldest deadline 2023-09-13T04:53:05.469606+0000) 2023-09-13T04:53:07.136 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:06 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:06.884+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:53:07.137 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:06 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:06.884+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:53:07.137 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:06 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:06.884+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:53:07.137 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:06 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:06.884+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:53:07.137 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:06 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:06.884+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:21.191553+0000 front 2023-09-13T04:52:21.191599+0000 (oldest deadline 2023-09-13T04:52:44.691812+0000) 2023-09-13T04:53:07.138 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:06 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:06.884+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.193149+0000 front 2023-09-13T04:52:25.193078+0000 (oldest deadline 2023-09-13T04:52:50.493119+0000) 2023-09-13T04:53:07.138 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:06 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:06.884+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:34.196067+0000 front 2023-09-13T04:52:34.196195+0000 (oldest deadline 2023-09-13T04:52:59.996448+0000) 2023-09-13T04:53:07.686 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:07 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:07.210+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:53:07.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:07 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:07.210+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:53:07.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:07 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:07.210+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:53:07.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:07 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:07.210+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:53:07.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:07 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:07.210+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:53:07.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:07 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:07.210+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:33.768678+0000 front 2023-09-13T04:52:33.768757+0000 (oldest deadline 2023-09-13T04:52:57.868723+0000) 2023-09-13T04:53:07.688 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:07 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:07.210+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2023-09-13T04:52:41.969808+0000 front 2023-09-13T04:52:41.969773+0000 (oldest deadline 2023-09-13T04:53:05.469606+0000) 2023-09-13T04:53:08.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:07 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:07.923+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:46.984618+0000 front 2023-09-13T04:51:46.984601+0000 (oldest deadline 2023-09-13T04:52:10.484535+0000) 2023-09-13T04:53:08.386 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:07 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:07.923+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:54.387089+0000 front 2023-09-13T04:51:54.387150+0000 (oldest deadline 2023-09-13T04:52:19.087184+0000) 2023-09-13T04:53:08.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:07 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:07.923+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.288878+0000 front 2023-09-13T04:52:04.288715+0000 (oldest deadline 2023-09-13T04:52:27.788931+0000) 2023-09-13T04:53:08.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:07 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:07.923+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-09-13T04:52:07.789678+0000 front 2023-09-13T04:52:07.789595+0000 (oldest deadline 2023-09-13T04:52:38.990121+0000) 2023-09-13T04:53:08.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:07 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:07.923+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:21.191553+0000 front 2023-09-13T04:52:21.191599+0000 (oldest deadline 2023-09-13T04:52:44.691812+0000) 2023-09-13T04:53:08.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:07 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:07.923+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:25.193149+0000 front 2023-09-13T04:52:25.193078+0000 (oldest deadline 2023-09-13T04:52:50.493119+0000) 2023-09-13T04:53:08.388 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:07 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10[162824]: 2023-09-13T04:53:07.923+0000 7f585c62f700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:34.196067+0000 front 2023-09-13T04:52:34.196195+0000 (oldest deadline 2023-09-13T04:52:59.996448+0000) 2023-09-13T04:53:08.686 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:08 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:08.246+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:53:08.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:08 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:08.246+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:53:08.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:08 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:08.246+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:53:08.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:08 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:08.246+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:53:08.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:08 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:08.246+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:53:08.688 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:08 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:08.246+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:33.768678+0000 front 2023-09-13T04:52:33.768757+0000 (oldest deadline 2023-09-13T04:52:57.868723+0000) 2023-09-13T04:53:08.688 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:08 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:08.246+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2023-09-13T04:52:41.969808+0000 front 2023-09-13T04:52:41.969773+0000 (oldest deadline 2023-09-13T04:53:05.469606+0000) 2023-09-13T04:53:09.686 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:09 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:09.201+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:53:09.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:09 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:09.201+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:53:09.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:09 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:09.201+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:53:09.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:09 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:09.201+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:53:09.688 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:09 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:09.201+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:53:09.688 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:09 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:09.201+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:33.768678+0000 front 2023-09-13T04:52:33.768757+0000 (oldest deadline 2023-09-13T04:52:57.868723+0000) 2023-09-13T04:53:09.688 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:09 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:09.201+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2023-09-13T04:52:41.969808+0000 front 2023-09-13T04:52:41.969773+0000 (oldest deadline 2023-09-13T04:53:05.469606+0000) 2023-09-13T04:53:09.886 INFO:journalctl@ceph.osd.10.smithi149.stdout:Sep 13 04:53:09 smithi149 bash[177327]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-10 2023-09-13T04:53:10.436 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:10 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:10.185+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:53:10.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:10 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:10.185+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:53:10.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:10 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:10.185+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:53:10.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:10 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:10.185+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:53:10.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:10 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:10.185+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:53:10.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:10 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:10.185+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:33.768678+0000 front 2023-09-13T04:52:33.768757+0000 (oldest deadline 2023-09-13T04:52:57.868723+0000) 2023-09-13T04:53:10.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:10 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:10.185+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2023-09-13T04:52:41.969808+0000 front 2023-09-13T04:52:41.969773+0000 (oldest deadline 2023-09-13T04:53:05.469606+0000) 2023-09-13T04:53:10.758 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.10.service' 2023-09-13T04:53:10.837 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:53:10.837 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-09-13T04:53:10.837 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-09-13T04:53:10.838 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.11 2023-09-13T04:53:11.143 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:10 smithi156 systemd[1]: Stopping Ceph osd.11 for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:53:11.144 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:11 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:11.142+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:53:11.436 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:11 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:11.142+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:53:11.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:11 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:11.142+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:53:11.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:11 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:11.142+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:53:11.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:11 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:11.142+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:53:11.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:11 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:11.142+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:33.768678+0000 front 2023-09-13T04:52:33.768757+0000 (oldest deadline 2023-09-13T04:52:57.868723+0000) 2023-09-13T04:53:11.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:11 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:11.142+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2023-09-13T04:52:41.969808+0000 front 2023-09-13T04:52:41.969773+0000 (oldest deadline 2023-09-13T04:53:05.469606+0000) 2023-09-13T04:53:11.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:11 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:11.196+0000 7fc5cab89700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:53:11.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:11 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:11.196+0000 7fc5cab89700 -1 osd.11 109 *** Got signal Terminated *** 2023-09-13T04:53:11.438 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:11 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:11.196+0000 7fc5cab89700 -1 osd.11 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-13T04:53:12.436 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:12 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:12.175+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:53:12.436 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:12 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:12.175+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:53:12.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:12 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:12.175+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:53:12.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:12 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:12.175+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:53:12.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:12 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:12.175+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:53:12.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:12 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:12.175+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:33.768678+0000 front 2023-09-13T04:52:33.768757+0000 (oldest deadline 2023-09-13T04:52:57.868723+0000) 2023-09-13T04:53:12.437 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:12 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:12.175+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2023-09-13T04:52:41.969808+0000 front 2023-09-13T04:52:41.969773+0000 (oldest deadline 2023-09-13T04:53:05.469606+0000) 2023-09-13T04:53:13.686 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:13 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:13.186+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:53:13.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:13 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:13.186+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:53:13.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:13 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:13.186+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:53:13.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:13 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:13.186+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:53:13.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:13 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:13.186+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:53:13.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:13 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:13.186+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:33.768678+0000 front 2023-09-13T04:52:33.768757+0000 (oldest deadline 2023-09-13T04:52:57.868723+0000) 2023-09-13T04:53:13.688 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:13 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:13.186+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2023-09-13T04:52:41.969808+0000 front 2023-09-13T04:52:41.969773+0000 (oldest deadline 2023-09-13T04:53:05.469606+0000) 2023-09-13T04:53:14.686 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:14 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:14.236+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:53:14.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:14 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:14.236+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:53:14.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:14 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:14.236+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:53:14.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:14 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:14.236+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:53:14.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:14 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:14.236+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:53:14.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:14 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:14.236+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:33.768678+0000 front 2023-09-13T04:52:33.768757+0000 (oldest deadline 2023-09-13T04:52:57.868723+0000) 2023-09-13T04:53:14.688 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:14 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:14.236+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2023-09-13T04:52:41.969808+0000 front 2023-09-13T04:52:41.969773+0000 (oldest deadline 2023-09-13T04:53:05.469606+0000) 2023-09-13T04:53:15.686 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:15 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:15.212+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-09-13T04:51:47.160202+0000 front 2023-09-13T04:51:47.160215+0000 (oldest deadline 2023-09-13T04:52:12.460228+0000) 2023-09-13T04:53:15.686 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:15 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:15.212+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2023-09-13T04:51:52.460845+0000 front 2023-09-13T04:51:52.460761+0000 (oldest deadline 2023-09-13T04:52:18.360898+0000) 2023-09-13T04:53:15.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:15 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:15.212+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-13T04:52:04.663406+0000 front 2023-09-13T04:52:04.663310+0000 (oldest deadline 2023-09-13T04:52:30.563456+0000) 2023-09-13T04:53:15.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:15 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:15.212+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2023-09-13T04:52:20.366619+0000 front 2023-09-13T04:52:20.366651+0000 (oldest deadline 2023-09-13T04:52:45.066554+0000) 2023-09-13T04:53:15.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:15 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:15.212+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-13T04:52:27.367688+0000 front 2023-09-13T04:52:27.367603+0000 (oldest deadline 2023-09-13T04:52:49.667685+0000) 2023-09-13T04:53:15.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:15 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:15.212+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-09-13T04:52:33.768678+0000 front 2023-09-13T04:52:33.768757+0000 (oldest deadline 2023-09-13T04:52:57.868723+0000) 2023-09-13T04:53:15.687 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:15 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11[164858]: 2023-09-13T04:53:15.212+0000 7fc5c4179700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2023-09-13T04:52:41.969808+0000 front 2023-09-13T04:52:41.969773+0000 (oldest deadline 2023-09-13T04:53:05.469606+0000) 2023-09-13T04:53:17.077 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 13 04:53:17 smithi156 bash[180275]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-osd-11 2023-09-13T04:53:18.539 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@osd.11.service' 2023-09-13T04:53:18.580 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:53:18.580 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-09-13T04:53:18.580 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-09-13T04:53:18.581 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.a 2023-09-13T04:53:19.005 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:53:18 smithi002 systemd[1]: Stopping Ceph mds.a for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:53:19.506 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:53:19 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:53:19.096+0000 7f8900b03700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:53:19.506 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:53:19 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a[148826]: 2023-09-13T04:53:19.096+0000 7f8900b03700 -1 mds.a *** got signal Terminated *** 2023-09-13T04:53:22.255 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 13 04:53:21 smithi002 bash[180515]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-a 2023-09-13T04:53:22.371 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.a.service' 2023-09-13T04:53:22.457 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:53:22.458 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-09-13T04:53:22.459 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-09-13T04:53:22.459 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.d 2023-09-13T04:53:22.755 INFO:journalctl@ceph.mds.d.smithi002.stdout:Sep 13 04:53:22 smithi002 systemd[1]: Stopping Ceph mds.d for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:53:23.505 INFO:journalctl@ceph.mds.d.smithi002.stdout:Sep 13 04:53:23 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-d[147821]: 2023-09-13T04:53:23.026+0000 7f8b806c6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:53:23.506 INFO:journalctl@ceph.mds.d.smithi002.stdout:Sep 13 04:53:23 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-d[147821]: 2023-09-13T04:53:23.026+0000 7f8b806c6700 -1 mds.d *** got signal Terminated *** 2023-09-13T04:53:25.873 INFO:journalctl@ceph.mds.d.smithi002.stdout:Sep 13 04:53:25 smithi002 bash[180657]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-d 2023-09-13T04:53:26.181 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.d.service' 2023-09-13T04:53:26.266 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:53:26.267 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-09-13T04:53:26.267 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-09-13T04:53:26.267 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.g 2023-09-13T04:53:26.505 INFO:journalctl@ceph.mds.g.smithi002.stdout:Sep 13 04:53:26 smithi002 systemd[1]: Stopping Ceph mds.g for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:53:27.255 INFO:journalctl@ceph.mds.g.smithi002.stdout:Sep 13 04:53:26 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-g[148436]: 2023-09-13T04:53:26.761+0000 7f2002dc0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:53:27.256 INFO:journalctl@ceph.mds.g.smithi002.stdout:Sep 13 04:53:26 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-g[148436]: 2023-09-13T04:53:26.761+0000 7f2002dc0700 -1 mds.g *** got signal Terminated *** 2023-09-13T04:53:31.255 INFO:journalctl@ceph.mds.g.smithi002.stdout:Sep 13 04:53:31 smithi002 bash[180799]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-g 2023-09-13T04:53:31.493 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.g.service' 2023-09-13T04:53:31.578 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:53:31.578 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-09-13T04:53:31.579 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-09-13T04:53:31.579 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.j 2023-09-13T04:53:31.755 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 13 04:53:31 smithi002 systemd[1]: Stopping Ceph mds.j for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:53:32.505 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 13 04:53:32 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-j[147436]: 2023-09-13T04:53:32.086+0000 7fa39bd1e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:53:32.506 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 13 04:53:32 smithi002 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-j[147436]: 2023-09-13T04:53:32.086+0000 7fa39bd1e700 -1 mds.j *** got signal Terminated *** 2023-09-13T04:53:34.883 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 13 04:53:34 smithi002 bash[180946]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-j 2023-09-13T04:53:35.129 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.j.service' 2023-09-13T04:53:35.190 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 13 04:53:35 smithi002 systemd[1]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.j.service: Succeeded. 2023-09-13T04:53:35.190 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 13 04:53:35 smithi002 systemd[1]: Stopped Ceph mds.j for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:53:35.212 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:53:35.213 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-09-13T04:53:35.213 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-09-13T04:53:35.214 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.b 2023-09-13T04:53:35.573 INFO:journalctl@ceph.mds.b.smithi149.stdout:Sep 13 04:53:35 smithi149 systemd[1]: Stopping Ceph mds.b for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:53:35.886 INFO:journalctl@ceph.mds.b.smithi149.stdout:Sep 13 04:53:35 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-b[169954]: 2023-09-13T04:53:35.573+0000 7f37d6b88700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:53:35.887 INFO:journalctl@ceph.mds.b.smithi149.stdout:Sep 13 04:53:35 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-b[169954]: 2023-09-13T04:53:35.573+0000 7f37d6b88700 -1 mds.b *** got signal Terminated *** 2023-09-13T04:53:37.833 INFO:journalctl@ceph.mds.b.smithi149.stdout:Sep 13 04:53:37 smithi149 bash[177590]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-b 2023-09-13T04:53:38.019 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.b.service' 2023-09-13T04:53:38.056 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:53:38.057 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-09-13T04:53:38.057 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-09-13T04:53:38.058 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.e 2023-09-13T04:53:38.386 INFO:journalctl@ceph.mds.e.smithi149.stdout:Sep 13 04:53:38 smithi149 systemd[1]: Stopping Ceph mds.e for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:53:38.886 INFO:journalctl@ceph.mds.e.smithi149.stdout:Sep 13 04:53:38 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-e[169266]: 2023-09-13T04:53:38.439+0000 7fa7830c2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:53:38.886 INFO:journalctl@ceph.mds.e.smithi149.stdout:Sep 13 04:53:38 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-e[169266]: 2023-09-13T04:53:38.439+0000 7fa7830c2700 -1 mds.e *** got signal Terminated *** 2023-09-13T04:53:40.180 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.e.service' 2023-09-13T04:53:40.205 INFO:journalctl@ceph.mds.e.smithi149.stdout:Sep 13 04:53:39 smithi149 bash[177729]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-e 2023-09-13T04:53:40.205 INFO:journalctl@ceph.mds.e.smithi149.stdout:Sep 13 04:53:40 smithi149 systemd[1]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.e.service: Succeeded. 2023-09-13T04:53:40.205 INFO:journalctl@ceph.mds.e.smithi149.stdout:Sep 13 04:53:40 smithi149 systemd[1]: Stopped Ceph mds.e for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da. 2023-09-13T04:53:40.213 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:53:40.214 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-09-13T04:53:40.214 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-09-13T04:53:40.214 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.h 2023-09-13T04:53:40.598 INFO:journalctl@ceph.mds.h.smithi149.stdout:Sep 13 04:53:40 smithi149 systemd[1]: Stopping Ceph mds.h for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:53:40.886 INFO:journalctl@ceph.mds.h.smithi149.stdout:Sep 13 04:53:40 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-h[169606]: 2023-09-13T04:53:40.598+0000 7f4bbb7b6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:53:40.887 INFO:journalctl@ceph.mds.h.smithi149.stdout:Sep 13 04:53:40 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-h[169606]: 2023-09-13T04:53:40.598+0000 7f4bbb7b6700 -1 mds.h *** got signal Terminated *** 2023-09-13T04:53:45.104 INFO:journalctl@ceph.mds.h.smithi149.stdout:Sep 13 04:53:44 smithi149 bash[177871]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-h 2023-09-13T04:53:45.110 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.h.service' 2023-09-13T04:53:45.143 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:53:45.143 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-09-13T04:53:45.144 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-09-13T04:53:45.144 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.k 2023-09-13T04:53:45.386 INFO:journalctl@ceph.mds.k.smithi149.stdout:Sep 13 04:53:45 smithi149 systemd[1]: Stopping Ceph mds.k for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:53:45.886 INFO:journalctl@ceph.mds.k.smithi149.stdout:Sep 13 04:53:45 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-k[168925]: 2023-09-13T04:53:45.522+0000 7f6d7283c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:53:45.886 INFO:journalctl@ceph.mds.k.smithi149.stdout:Sep 13 04:53:45 smithi149 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-k[168925]: 2023-09-13T04:53:45.522+0000 7f6d7283c700 -1 mds.k *** got signal Terminated *** 2023-09-13T04:53:47.279 INFO:journalctl@ceph.mds.k.smithi149.stdout:Sep 13 04:53:47 smithi149 bash[178015]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-k 2023-09-13T04:53:47.481 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.k.service' 2023-09-13T04:53:47.515 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:53:47.515 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-09-13T04:53:47.516 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-09-13T04:53:47.516 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.c 2023-09-13T04:53:47.900 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:53:47 smithi156 systemd[1]: Stopping Ceph mds.c for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:53:48.186 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:53:47 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[177386]: 2023-09-13T04:53:47.898+0000 7f8e99033700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:53:48.187 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:53:47 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c[177386]: 2023-09-13T04:53:47.898+0000 7f8e99033700 -1 mds.c *** got signal Terminated *** 2023-09-13T04:53:49.833 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 13 04:53:49 smithi156 bash[180536]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-c 2023-09-13T04:53:50.021 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.c.service' 2023-09-13T04:53:50.058 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:53:50.058 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-09-13T04:53:50.059 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-09-13T04:53:50.059 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.f 2023-09-13T04:53:50.436 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 13 04:53:50 smithi156 systemd[1]: Stopping Ceph mds.f for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:53:50.936 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 13 04:53:50 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-f[171361]: 2023-09-13T04:53:50.447+0000 7f091beb0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:53:50.936 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 13 04:53:50 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-f[171361]: 2023-09-13T04:53:50.447+0000 7f091beb0700 -1 mds.f *** got signal Terminated *** 2023-09-13T04:53:53.806 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 13 04:53:53 smithi156 bash[180673]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-f 2023-09-13T04:53:53.991 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.f.service' 2023-09-13T04:53:54.025 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:53:54.026 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-09-13T04:53:54.026 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-09-13T04:53:54.026 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.i 2023-09-13T04:53:54.415 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 13 04:53:54 smithi156 systemd[1]: Stopping Ceph mds.i for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:53:54.686 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 13 04:53:54 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-i[172038]: 2023-09-13T04:53:54.413+0000 7f8745bc4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:53:54.687 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 13 04:53:54 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-i[172038]: 2023-09-13T04:53:54.413+0000 7f8745bc4700 -1 mds.i *** got signal Terminated *** 2023-09-13T04:53:57.936 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 13 04:53:57 smithi156 bash[180818]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-i 2023-09-13T04:53:58.017 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.i.service' 2023-09-13T04:53:58.053 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:53:58.054 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-09-13T04:53:58.054 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-09-13T04:53:58.054 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.l 2023-09-13T04:53:58.437 INFO:journalctl@ceph.mds.l.smithi156.stdout:Sep 13 04:53:58 smithi156 systemd[1]: Stopping Ceph mds.l for 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da... 2023-09-13T04:53:58.936 INFO:journalctl@ceph.mds.l.smithi156.stdout:Sep 13 04:53:58 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-l[171020]: 2023-09-13T04:53:58.454+0000 7f6a42b3c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T04:53:58.937 INFO:journalctl@ceph.mds.l.smithi156.stdout:Sep 13 04:53:58 smithi156 ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-l[171020]: 2023-09-13T04:53:58.454+0000 7f6a42b3c700 -1 mds.l *** got signal Terminated *** 2023-09-13T04:54:01.097 INFO:journalctl@ceph.mds.l.smithi156.stdout:Sep 13 04:54:01 smithi156 bash[180960]: ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da-mds-l 2023-09-13T04:54:01.301 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da@mds.l.service' 2023-09-13T04:54:01.336 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T04:54:01.336 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-09-13T04:54:01.337 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da --force --keep-logs 2023-09-13T04:54:01.739 INFO:teuthology.orchestra.run.smithi002.stdout:Deleting cluster with fsid: 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:54:05.161 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da --force --keep-logs 2023-09-13T04:54:05.554 INFO:teuthology.orchestra.run.smithi149.stdout:Deleting cluster with fsid: 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:54:07.212 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da --force --keep-logs 2023-09-13T04:54:07.517 INFO:teuthology.orchestra.run.smithi156.stdout:Deleting cluster with fsid: 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:54:09.089 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-13T04:54:09.169 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-13T04:54:09.206 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-13T04:54:09.236 INFO:tasks.cephadm:Archiving crash dumps... 2023-09-13T04:54:09.238 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/crash to /home/teuthworker/archive/rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi/7394725/remote/smithi002/crash 2023-09-13T04:54:09.239 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/crash -- . 2023-09-13T04:54:09.310 INFO:teuthology.orchestra.run.smithi002.stderr:tar: /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/crash: Cannot open: No such file or directory 2023-09-13T04:54:09.311 INFO:teuthology.orchestra.run.smithi002.stderr:tar: Error is not recoverable: exiting now 2023-09-13T04:54:09.315 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/crash to /home/teuthworker/archive/rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi/7394725/remote/smithi149/crash 2023-09-13T04:54:09.316 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/crash -- . 2023-09-13T04:54:09.374 INFO:teuthology.orchestra.run.smithi149.stderr:tar: /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/crash: Cannot open: No such file or directory 2023-09-13T04:54:09.374 INFO:teuthology.orchestra.run.smithi149.stderr:tar: Error is not recoverable: exiting now 2023-09-13T04:54:09.376 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/crash to /home/teuthworker/archive/rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi/7394725/remote/smithi156/crash 2023-09-13T04:54:09.377 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/crash -- . 2023-09-13T04:54:09.410 INFO:teuthology.orchestra.run.smithi156.stderr:tar: /var/lib/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/crash: Cannot open: No such file or directory 2023-09-13T04:54:09.411 INFO:teuthology.orchestra.run.smithi156.stderr:tar: Error is not recoverable: exiting now 2023-09-13T04:54:09.413 INFO:tasks.cephadm:Checking cluster log for badness... 2023-09-13T04:54:09.413 DEBUG:teuthology.orchestra.run.smithi002:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-09-13T04:54:09.494 INFO:teuthology.orchestra.run.smithi002.stderr:grep: /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph.log: No such file or directory 2023-09-13T04:54:09.499 INFO:tasks.cephadm:Compressing logs... 2023-09-13T04:54:09.499 DEBUG:teuthology.orchestra.run.smithi002:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-13T04:54:09.504 DEBUG:teuthology.orchestra.run.smithi149:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-13T04:54:09.507 DEBUG:teuthology.orchestra.run.smithi156:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-13T04:54:09.535 INFO:teuthology.orchestra.run.smithi156.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-09-13T04:54:09.535 INFO:teuthology.orchestra.run.smithi156.stderr:: No such file or directory 2023-09-13T04:54:09.535 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.5.log 2023-09-13T04:54:09.535 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.2.log 2023-09-13T04:54:09.536 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.11.log 2023-09-13T04:54:09.536 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.5.log: /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mgr.z.log 2023-09-13T04:54:09.536 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.11.log: --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.8.log 2023-09-13T04:54:09.537 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.f.log 2023-09-13T04:54:09.537 INFO:teuthology.orchestra.run.smithi156.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-09-13T04:54:09.538 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.8.log: /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.c.log 2023-09-13T04:54:09.544 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.c.log: 91.9% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mgr.z.log.gz 2023-09-13T04:54:09.547 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.i.log 2023-09-13T04:54:09.547 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mon.c.log 2023-09-13T04:54:09.553 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-volume.log 2023-09-13T04:54:09.568 INFO:teuthology.orchestra.run.smithi002.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-09-13T04:54:09.569 INFO:teuthology.orchestra.run.smithi002.stderr:: No such file or directory 2023-09-13T04:54:09.569 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.0.log 2023-09-13T04:54:09.570 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.g.log 2023-09-13T04:54:09.571 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.a.log 2023-09-13T04:54:09.572 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.6.log 2023-09-13T04:54:09.574 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.3.log 2023-09-13T04:54:09.576 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.d.log 2023-09-13T04:54:09.576 INFO:teuthology.orchestra.run.smithi002.stderr: 94.3% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.g.log.gz 2023-09-13T04:54:09.577 INFO:teuthology.orchestra.run.smithi002.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-09-13T04:54:09.577 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mgr.x.log 2023-09-13T04:54:09.579 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.9.log 2023-09-13T04:54:09.580 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-volume.log: 91.0% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-volume.log.gz 2023-09-13T04:54:09.580 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.l.log 2023-09-13T04:54:09.581 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-volume.log 2023-09-13T04:54:09.581 INFO:teuthology.orchestra.run.smithi149.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-09-13T04:54:09.582 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-09-13T04:54:09.589 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.9.log: /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-volume.log: 94.0% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.d.log.gz 2023-09-13T04:54:09.590 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.j.log 2023-09-13T04:54:09.591 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.10.log 2023-09-13T04:54:09.591 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.h.log 2023-09-13T04:54:09.594 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.e.log 2023-09-13T04:54:09.595 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.4.log 2023-09-13T04:54:09.595 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.1.log 2023-09-13T04:54:09.595 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.k.log 2023-09-13T04:54:09.595 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-volume.log 2023-09-13T04:54:09.595 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.7.log 2023-09-13T04:54:09.596 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mon.b.log 2023-09-13T04:54:09.596 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.b.log 2023-09-13T04:54:09.596 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mgr.y.log 2023-09-13T04:54:09.597 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mon.a.log 2023-09-13T04:54:09.597 INFO:teuthology.orchestra.run.smithi002.stderr: 90.9% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-volume.log.gz 2023-09-13T04:54:09.598 INFO:teuthology.orchestra.run.smithi002.stderr: 94.3% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.j.log.gz 2023-09-13T04:54:09.598 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2023-09-13T04:54:09.598 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.b.log: /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mgr.y.log: 94.3% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.e.log.gz 2023-09-13T04:54:09.598 INFO:teuthology.orchestra.run.smithi149.stderr: 94.4% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.k.log.gz 2023-09-13T04:54:09.599 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.l.log: 94.4% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.l.log.gz 2023-09-13T04:54:09.600 INFO:teuthology.orchestra.run.smithi149.stderr: 91.8% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mgr.y.log.gz 2023-09-13T04:54:09.601 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.log: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2023-09-13T04:54:09.826 INFO:teuthology.orchestra.run.smithi149.stderr: 91.0% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-volume.log.gz 2023-09-13T04:54:09.827 INFO:teuthology.orchestra.run.smithi149.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-09-13T04:54:09.914 INFO:teuthology.orchestra.run.smithi002.stderr: 89.9% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mgr.x.log.gz 2023-09-13T04:54:10.035 INFO:teuthology.orchestra.run.smithi156.stderr: 91.8% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mon.c.log.gz 2023-09-13T04:54:10.583 INFO:teuthology.orchestra.run.smithi002.stderr: 90.5% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mon.a.log.gz 2023-09-13T04:54:12.126 INFO:teuthology.orchestra.run.smithi149.stderr: 91.9% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mon.b.log.gz 2023-09-13T04:54:14.055 INFO:teuthology.orchestra.run.smithi156.stderr: 94.8% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.f.log.gz 2023-09-13T04:54:15.033 INFO:teuthology.orchestra.run.smithi156.stderr: 94.5% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.c.log.gz 2023-09-13T04:54:30.582 INFO:teuthology.orchestra.run.smithi156.stderr: 91.8% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.8.log.gz 2023-09-13T04:54:32.489 INFO:teuthology.orchestra.run.smithi002.stderr: 91.5% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.0.log.gz 2023-09-13T04:54:35.991 INFO:teuthology.orchestra.run.smithi002.stderr: 91.4% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.3.log.gz 2023-09-13T04:54:36.121 INFO:teuthology.orchestra.run.smithi156.stderr: 91.6% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.2.log.gz 2023-09-13T04:54:36.442 INFO:teuthology.orchestra.run.smithi156.stderr: 91.6% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.11.log.gz 2023-09-13T04:54:37.350 INFO:teuthology.orchestra.run.smithi002.stderr: 91.5% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.6.log.gz 2023-09-13T04:54:39.532 INFO:teuthology.orchestra.run.smithi149.stderr: 94.8% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.h.log.gz 2023-09-13T04:54:42.494 INFO:teuthology.orchestra.run.smithi156.stderr: 91.5% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.5.log.gz 2023-09-13T04:54:44.259 INFO:teuthology.orchestra.run.smithi149.stderr: 91.8% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.10.log.gz 2023-09-13T04:54:48.484 INFO:teuthology.orchestra.run.smithi002.stderr: 91.5% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.9.log.gz 2023-09-13T04:54:53.546 INFO:teuthology.orchestra.run.smithi149.stderr: 91.6% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.1.log.gz 2023-09-13T04:54:56.844 INFO:teuthology.orchestra.run.smithi149.stderr: 91.5% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.7.log.gz 2023-09-13T04:55:11.273 INFO:teuthology.orchestra.run.smithi149.stderr: 91.6% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-osd.4.log.gz 2023-09-13T04:55:29.069 INFO:teuthology.orchestra.run.smithi156.stderr: 95.3% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.i.log.gz 2023-09-13T04:55:29.072 INFO:teuthology.orchestra.run.smithi156.stderr: 2023-09-13T04:55:29.072 INFO:teuthology.orchestra.run.smithi156.stderr:real 1m19.548s 2023-09-13T04:55:29.072 INFO:teuthology.orchestra.run.smithi156.stderr:user 3m10.255s 2023-09-13T04:55:29.072 INFO:teuthology.orchestra.run.smithi156.stderr:sys 0m6.670s 2023-09-13T04:55:59.822 INFO:teuthology.orchestra.run.smithi002.stderr: 93.0% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.a.log.gz 2023-09-13T04:55:59.825 INFO:teuthology.orchestra.run.smithi002.stderr: 2023-09-13T04:55:59.826 INFO:teuthology.orchestra.run.smithi002.stderr:real 1m50.285s 2023-09-13T04:55:59.826 INFO:teuthology.orchestra.run.smithi002.stderr:user 3m31.474s 2023-09-13T04:55:59.827 INFO:teuthology.orchestra.run.smithi002.stderr:sys 0m15.561s 2023-09-13T04:56:06.707 INFO:teuthology.orchestra.run.smithi149.stderr: 95.0% -- replaced with /var/log/ceph/22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da/ceph-mds.b.log.gz 2023-09-13T04:56:06.725 INFO:teuthology.orchestra.run.smithi149.stderr: 2023-09-13T04:56:06.726 INFO:teuthology.orchestra.run.smithi149.stderr:real 1m57.202s 2023-09-13T04:56:06.726 INFO:teuthology.orchestra.run.smithi149.stderr:user 2m46.243s 2023-09-13T04:56:06.726 INFO:teuthology.orchestra.run.smithi149.stderr:sys 0m7.110s 2023-09-13T04:56:06.727 INFO:tasks.cephadm:Archiving logs... 2023-09-13T04:56:06.728 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi/7394725/remote/smithi002/log 2023-09-13T04:56:06.729 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-13T04:56:38.276 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi/7394725/remote/smithi149/log 2023-09-13T04:56:38.294 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-13T04:57:04.424 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi/7394725/remote/smithi156/log 2023-09-13T04:57:04.425 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-13T04:57:29.823 INFO:tasks.cephadm:Removing cluster... 2023-09-13T04:57:29.824 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da --force 2023-09-13T04:57:30.227 INFO:teuthology.orchestra.run.smithi002.stdout:Deleting cluster with fsid: 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:57:31.690 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da --force 2023-09-13T04:57:32.482 INFO:teuthology.orchestra.run.smithi149.stdout:Deleting cluster with fsid: 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:57:33.164 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da --force 2023-09-13T04:57:33.466 INFO:teuthology.orchestra.run.smithi156.stdout:Deleting cluster with fsid: 22f7c1c6-51ee-11ee-9ab8-7b867c8bd7da 2023-09-13T04:57:34.187 INFO:tasks.cephadm:Removing cephadm ... 2023-09-13T04:57:34.187 DEBUG:teuthology.orchestra.run.smithi002:> rm -rf /home/ubuntu/cephtest/cephadm 2023-09-13T04:57:34.237 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf /home/ubuntu/cephtest/cephadm 2023-09-13T04:57:34.269 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf /home/ubuntu/cephtest/cephadm 2023-09-13T04:57:34.287 INFO:tasks.cephadm:Teardown complete 2023-09-13T04:57:34.287 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-09-13T04:57:34.298 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-09-13T04:57:34.299 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-13T04:57:34.304 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-13T04:57:34.311 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-13T04:57:34.450 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-09-13T04:57:34.462 INFO:teuthology.task.clock:Checking final clock skew... 2023-09-13T04:57:34.462 DEBUG:teuthology.orchestra.run.smithi002:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-13T04:57:34.467 DEBUG:teuthology.orchestra.run.smithi149:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-13T04:57:34.470 DEBUG:teuthology.orchestra.run.smithi156:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-13T04:57:34.485 INFO:teuthology.orchestra.run.smithi156.stderr:bash: ntpq: command not found 2023-09-13T04:57:34.486 INFO:teuthology.orchestra.run.smithi149.stderr:bash: ntpq: command not found 2023-09-13T04:57:34.490 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-13T04:57:34.490 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2023-09-13T04:57:34.490 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 48 +695us[ +695us] +/- 79ms 2023-09-13T04:57:34.490 INFO:teuthology.orchestra.run.smithi156.stdout:^* hv02.front.sepia.ceph.com 3 6 377 49 -1288us[-1246us] +/- 65ms 2023-09-13T04:57:34.490 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 +847us[ +847us] +/- 69ms 2023-09-13T04:57:34.491 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T04:57:34.506 INFO:teuthology.orchestra.run.smithi002.stderr:bash: ntpq: command not found 2023-09-13T04:57:34.507 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-13T04:57:34.507 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2023-09-13T04:57:34.508 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 46 +683us[ +683us] +/- 79ms 2023-09-13T04:57:34.508 INFO:teuthology.orchestra.run.smithi149.stdout:^* hv02.front.sepia.ceph.com 3 6 377 46 -1312us[-1304us] +/- 65ms 2023-09-13T04:57:34.508 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 48 +823us[ +831us] +/- 69ms 2023-09-13T04:57:34.508 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T04:57:34.513 INFO:teuthology.orchestra.run.smithi002.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-13T04:57:34.514 INFO:teuthology.orchestra.run.smithi002.stdout:=============================================================================== 2023-09-13T04:57:34.514 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 46 +688us[ +688us] +/- 79ms 2023-09-13T04:57:34.514 INFO:teuthology.orchestra.run.smithi002.stdout:^* hv02.front.sepia.ceph.com 3 6 377 47 -1311us[-1275us] +/- 65ms 2023-09-13T04:57:34.514 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 +826us[ +862us] +/- 69ms 2023-09-13T04:57:34.515 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T04:57:34.515 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-09-13T04:57:34.526 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-09-13T04:57:34.527 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-09-13T04:57:34.539 DEBUG:teuthology.orchestra.run.smithi002:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-13T04:57:34.738 DEBUG:teuthology.orchestra.run.smithi149:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-13T04:57:34.871 DEBUG:teuthology.orchestra.run.smithi156:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-13T04:57:34.967 DEBUG:teuthology.orchestra.run.smithi002:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-13T04:57:35.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T04:57:35.046 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-13T04:57:35.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T04:57:35.082 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-13T04:57:35.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T04:57:35.114 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-09-13T04:57:35.126 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-09-13T04:57:35.137 INFO:teuthology.task.internal:Duration was 2618.378073 seconds 2023-09-13T04:57:35.138 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-09-13T04:57:35.149 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-09-13T04:57:35.150 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-13T04:57:35.155 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-13T04:57:35.158 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-13T04:57:35.204 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-13T04:57:35.219 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-13T04:57:35.270 INFO:teuthology.orchestra.run.smithi002.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-13T04:57:35.446 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-09-13T04:57:35.447 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi002.front.sepia.ceph.com 2023-09-13T04:57:35.447 DEBUG:teuthology.orchestra.run.smithi002:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-13T04:57:35.506 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi149.front.sepia.ceph.com 2023-09-13T04:57:35.507 DEBUG:teuthology.orchestra.run.smithi149:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-13T04:57:35.543 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi156.front.sepia.ceph.com 2023-09-13T04:57:35.544 DEBUG:teuthology.orchestra.run.smithi156:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-13T04:57:35.565 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-09-13T04:57:35.566 DEBUG:teuthology.orchestra.run.smithi002:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-13T04:57:35.571 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-13T04:57:35.586 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-13T04:57:35.668 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-09-13T04:57:35.669 DEBUG:teuthology.orchestra.run.smithi002:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-13T04:57:35.674 DEBUG:teuthology.orchestra.run.smithi149:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-13T04:57:35.676 DEBUG:teuthology.orchestra.run.smithi156:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-13T04:57:35.841 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-09-13T04:57:35.854 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-09-13T04:57:35.854 DEBUG:teuthology.orchestra.run.smithi002:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-13T04:57:35.924 DEBUG:teuthology.orchestra.run.smithi149:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-13T04:57:35.961 DEBUG:teuthology.orchestra.run.smithi156:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-13T04:57:35.988 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-09-13T04:57:36.001 DEBUG:teuthology.orchestra.run.smithi002:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-13T04:57:36.005 DEBUG:teuthology.orchestra.run.smithi149:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-13T04:57:36.008 DEBUG:teuthology.orchestra.run.smithi156:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-13T04:57:36.041 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = core 2023-09-13T04:57:36.054 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = core 2023-09-13T04:57:36.069 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = core 2023-09-13T04:57:36.106 DEBUG:teuthology.orchestra.run.smithi002:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-13T04:57:36.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T04:57:36.149 DEBUG:teuthology.orchestra.run.smithi149:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-13T04:57:36.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T04:57:36.165 DEBUG:teuthology.orchestra.run.smithi156:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-13T04:57:36.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T04:57:36.182 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-09-13T04:57:36.195 INFO:teuthology.task.internal:Transferring archived files... 2023-09-13T04:57:36.196 DEBUG:teuthology.misc:Transferring archived files from smithi002:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi/7394725/remote/smithi002 2023-09-13T04:57:36.197 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-13T04:57:36.281 DEBUG:teuthology.misc:Transferring archived files from smithi149:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi/7394725/remote/smithi149 2023-09-13T04:57:36.281 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-13T04:57:36.322 DEBUG:teuthology.misc:Transferring archived files from smithi156:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi/7394725/remote/smithi156 2023-09-13T04:57:36.322 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-13T04:57:36.362 INFO:teuthology.task.internal:Removing archive directory... 2023-09-13T04:57:36.363 DEBUG:teuthology.orchestra.run.smithi002:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-13T04:57:36.367 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-13T04:57:36.370 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-13T04:57:36.413 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-09-13T04:57:36.427 INFO:teuthology.task.internal:Not uploading archives. 2023-09-13T04:57:36.428 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-09-13T04:57:36.439 INFO:teuthology.task.internal:Tidying up after the test... 2023-09-13T04:57:36.440 DEBUG:teuthology.orchestra.run.smithi002:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-13T04:57:36.451 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-13T04:57:36.454 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-13T04:57:36.468 INFO:teuthology.orchestra.run.smithi149.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 04:57 /home/ubuntu/cephtest 2023-09-13T04:57:36.470 INFO:teuthology.orchestra.run.smithi156.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 04:57 /home/ubuntu/cephtest 2023-09-13T04:57:36.490 INFO:teuthology.orchestra.run.smithi002.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 04:57 /home/ubuntu/cephtest 2023-09-13T04:57:36.493 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-09-13T04:57:36.506 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-09-13T04:57:36.549 INFO:teuthology.nuke:Checking targets against current locks 2023-09-13T04:57:36.575 DEBUG:teuthology.nuke:shortname: smithi002 2023-09-13T04:57:36.575 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-13T04:57:36.601 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi002.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi/7394725', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-13 04:05:19.552561', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-13T04:57:36.605 DEBUG:teuthology.nuke:shortname: smithi149 2023-09-13T04:57:36.606 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-13T04:57:36.633 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi/7394725', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-13 04:05:19.550477', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-13T04:57:36.634 DEBUG:teuthology.nuke:shortname: smithi156 2023-09-13T04:57:36.634 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-13T04:57:36.662 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi/7394725', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-13 04:05:19.554928', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-13T04:57:36.668 INFO:teuthology.orchestra.console.smithi002:Power off 2023-09-13T04:57:36.669 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-13T04:57:36.684 INFO:teuthology.orchestra.console.smithi149:Power off 2023-09-13T04:57:36.684 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-13T04:57:36.693 DEBUG:teuthology.orchestra.console.smithi002:power off output: Chassis Power Control: Down/Off 2023-09-13T04:57:36.693 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T04:57:36.705 DEBUG:teuthology.orchestra.console.smithi149:power off output: Chassis Power Control: Down/Off 2023-09-13T04:57:36.706 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T04:57:36.718 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on 2023-09-13T04:57:36.730 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2023-09-13T04:57:36.742 INFO:teuthology.orchestra.console.smithi156:Power off 2023-09-13T04:57:36.743 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-13T04:57:36.764 DEBUG:teuthology.orchestra.console.smithi156:power off output: Chassis Power Control: Down/Off 2023-09-13T04:57:36.765 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T04:57:36.788 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2023-09-13T04:57:40.719 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T04:57:40.731 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T04:57:40.789 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T04:57:40.830 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on 2023-09-13T04:57:40.840 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2023-09-13T04:57:40.900 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2023-09-13T04:57:44.834 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T04:57:44.842 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T04:57:44.901 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T04:57:44.945 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off 2023-09-13T04:57:44.951 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2023-09-13T04:57:45.011 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is off 2023-09-13T04:57:45.047 INFO:teuthology.orchestra.console.smithi002:Power off completed 2023-09-13T04:57:45.112 INFO:teuthology.orchestra.console.smithi156:Power off completed 2023-09-13T04:57:48.954 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T04:57:49.063 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off 2023-09-13T04:57:49.164 INFO:teuthology.orchestra.console.smithi149:Power off completed 2023-09-13T04:57:49.268 INFO:teuthology.run:Summary data: client.0-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/blogbench}} duration: 2618.3780727386475 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_rishabh@teuthology status: fail success: false 2023-09-13T04:57:49.269 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T04:57:49.355 INFO:teuthology.run:FAIL